2023-11-28T02:39:25.371 INFO:root:teuthology version: 0.0.1.dev205+g2442239 2023-11-28T02:39:25.371 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi --description fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703 --verbose -- /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/orig.config.yaml 2023-11-28T02:39:25.417 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T02:39:25.494 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703 branch: wip-yuri-testing-2023-11-27-1028-pacific description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: yweinste@redhat.com first_in_suite: false job_id: '7468703' kernel: client: sha1: distro kdb: true sha1: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.4' overrides: admin_socket: branch: wip-yuri-testing-2023-11-27-1028-pacific ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-11-27-1028-pacific sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 3153 sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-11-27-1028-pacific suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_5474366c84b0c469f7b5285fcfee86232293d8b4/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 targets: smithi073.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrYwp82qOkNzBtVWZdiiw7/4vzuuaQgmcQg21uwlcmpDvvwZwnShDLiOf7e7CBPwpQcWBLI4+9CevKpbl/G7vQ= smithi150.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOcsXlOEX6iheXxSe0SiQYyxcdk15GcmPxRXO6f5afn5qNlnTo44BVJqyXGhrJt5x77Pw+PUYTWTkbOBwmE5x6A= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 2442239d2653456406c25ae0c71b689c8f2657b6 timestamp: 2023-11-27_22:36:25 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-11-28T02:39:25.494 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_5474366c84b0c469f7b5285fcfee86232293d8b4/qa; will attempt to use it 2023-11-28T02:39:25.494 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_5474366c84b0c469f7b5285fcfee86232293d8b4/qa/tasks 2023-11-28T02:39:25.495 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-11-28T02:39:25.500 INFO:teuthology.task.internal:Checking packages... 2023-11-28T02:39:25.523 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '5474366c84b0c469f7b5285fcfee86232293d8b4' 2023-11-28T02:39:25.523 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-11-28T02:39:25.523 INFO:teuthology.packaging:ref: None 2023-11-28T02:39:25.524 INFO:teuthology.packaging:tag: None 2023-11-28T02:39:25.524 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-11-27-1028-pacific 2023-11-28T02:39:25.524 INFO:teuthology.packaging:sha1: 5474366c84b0c469f7b5285fcfee86232293d8b4 2023-11-28T02:39:25.524 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-11-27-1028-pacific 2023-11-28T02:39:25.665 INFO:teuthology.task.internal:Found packages for ceph version 16.2.14-270.g5474366c 2023-11-28T02:39:25.665 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-11-28T02:39:25.671 INFO:teuthology.task.internal:no buildpackages task found 2023-11-28T02:39:25.671 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-11-28T02:39:25.693 INFO:teuthology.task.internal:Saving configuration 2023-11-28T02:39:25.709 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-11-28T02:39:25.715 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T02:39:25.741 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 02:34:04.405562', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrYwp82qOkNzBtVWZdiiw7/4vzuuaQgmcQg21uwlcmpDvvwZwnShDLiOf7e7CBPwpQcWBLI4+9CevKpbl/G7vQ='} 2023-11-28T02:39:25.763 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 02:34:04.406976', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOcsXlOEX6iheXxSe0SiQYyxcdk15GcmPxRXO6f5afn5qNlnTo44BVJqyXGhrJt5x77Pw+PUYTWTkbOBwmE5x6A='} 2023-11-28T02:39:25.763 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-11-28T02:39:25.771 INFO:teuthology.task.internal:roles: ubuntu@smithi073.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-11-28T02:39:25.771 INFO:teuthology.task.internal:roles: ubuntu@smithi150.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-11-28T02:39:25.771 INFO:teuthology.run_tasks:Running task console_log... 2023-11-28T02:39:25.847 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f66ed0d3b80>, signals=[15]) 2023-11-28T02:39:25.848 INFO:teuthology.run_tasks:Running task internal.connect... 2023-11-28T02:39:25.854 INFO:teuthology.task.internal:Opening connections... 2023-11-28T02:39:25.854 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:39:25.856 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:39:25.925 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:39:25.926 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:39:25.993 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-11-28T02:39:26.000 DEBUG:teuthology.orchestra.run.smithi073:> uname -m 2023-11-28T02:39:26.018 INFO:teuthology.orchestra.run.smithi073.stdout:x86_64 2023-11-28T02:39:26.018 DEBUG:teuthology.orchestra.run.smithi073:> cat /etc/os-release 2023-11-28T02:39:26.075 INFO:teuthology.orchestra.run.smithi073.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T02:39:26.075 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION="8.4 (Ootpa)" 2023-11-28T02:39:26.075 INFO:teuthology.orchestra.run.smithi073.stdout:ID="rhel" 2023-11-28T02:39:26.075 INFO:teuthology.orchestra.run.smithi073.stdout:ID_LIKE="fedora" 2023-11-28T02:39:26.075 INFO:teuthology.orchestra.run.smithi073.stdout:VERSION_ID="8.4" 2023-11-28T02:39:26.076 INFO:teuthology.orchestra.run.smithi073.stdout:PLATFORM_ID="platform:el8" 2023-11-28T02:39:26.076 INFO:teuthology.orchestra.run.smithi073.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-11-28T02:39:26.076 INFO:teuthology.orchestra.run.smithi073.stdout:ANSI_COLOR="0;31" 2023-11-28T02:39:26.076 INFO:teuthology.orchestra.run.smithi073.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-11-28T02:39:26.077 INFO:teuthology.orchestra.run.smithi073.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T02:39:26.077 INFO:teuthology.orchestra.run.smithi073.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T02:39:26.078 INFO:teuthology.orchestra.run.smithi073.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T02:39:26.078 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:39:26.078 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T02:39:26.078 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-11-28T02:39:26.078 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T02:39:26.079 INFO:teuthology.orchestra.run.smithi073.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-11-28T02:39:26.080 INFO:teuthology.lock.ops:Updating smithi073.front.sepia.ceph.com on lock server 2023-11-28T02:39:26.102 DEBUG:teuthology.orchestra.run.smithi150:> uname -m 2023-11-28T02:39:26.119 INFO:teuthology.orchestra.run.smithi150.stdout:x86_64 2023-11-28T02:39:26.119 DEBUG:teuthology.orchestra.run.smithi150:> cat /etc/os-release 2023-11-28T02:39:26.175 INFO:teuthology.orchestra.run.smithi150.stdout:NAME="Red Hat Enterprise Linux" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION="8.4 (Ootpa)" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:ID="rhel" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:ID_LIKE="fedora" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:VERSION_ID="8.4" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:PLATFORM_ID="platform:el8" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.4 (Ootpa)" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:ANSI_COLOR="0;31" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8.4:GA" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:HOME_URL="https://www.redhat.com/" 2023-11-28T02:39:26.176 INFO:teuthology.orchestra.run.smithi150.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.4 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-11-28T02:39:26.177 INFO:teuthology.orchestra.run.smithi150.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.4" 2023-11-28T02:39:26.177 INFO:teuthology.lock.ops:Updating smithi150.front.sepia.ceph.com on lock server 2023-11-28T02:39:26.201 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-11-28T02:39:26.211 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-11-28T02:39:26.216 INFO:teuthology.task.internal:Checking for old test directory... 2023-11-28T02:39:26.217 DEBUG:teuthology.orchestra.run.smithi073:> test '!' -e /home/ubuntu/cephtest 2023-11-28T02:39:26.219 DEBUG:teuthology.orchestra.run.smithi150:> test '!' -e /home/ubuntu/cephtest 2023-11-28T02:39:26.235 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-11-28T02:39:26.242 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-11-28T02:39:26.242 DEBUG:teuthology.orchestra.run.smithi073:> test -z $(ls -A /var/lib/ceph) 2023-11-28T02:39:26.275 DEBUG:teuthology.orchestra.run.smithi150:> test -z $(ls -A /var/lib/ceph) 2023-11-28T02:39:26.290 INFO:teuthology.orchestra.run.smithi073.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-11-28T02:39:26.291 INFO:teuthology.orchestra.run.smithi150.stderr:ls: cannot access '/var/lib/ceph': No such file or directory 2023-11-28T02:39:26.293 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-11-28T02:39:26.352 INFO:teuthology.run_tasks:Running task kernel... 2023-11-28T02:39:26.367 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-11-28T02:39:26.367 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}, 'client.2': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}, 'client.3': {'sha1': 'distro'}} 2023-11-28T02:39:26.367 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}, 'client.1': {'sha1': 'distro'}}, timeout 300 2023-11-28T02:39:26.368 DEBUG:teuthology.orchestra.run.smithi073:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T02:39:26.368 DEBUG:teuthology.orchestra.run.smithi150:> test -f /run/.containerenv -o -f /.dockerenv 2023-11-28T02:39:26.383 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:39:26.384 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-11-28T02:39:26.385 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:39:26.385 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2023-11-28T02:39:26.440 INFO:teuthology.orchestra.run.smithi150.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-11-28T02:39:26.440 INFO:teuthology.task.kernel:Running kernel on smithi150: 4.18.0-305.25.1.el8_4.x86_64 2023-11-28T02:39:26.441 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum install -y kernel 2023-11-28T02:39:26.442 INFO:teuthology.orchestra.run.smithi073.stdout:4.18.0-305.25.1.el8_4.x86_64 2023-11-28T02:39:26.442 INFO:teuthology.task.kernel:Running kernel on smithi073: 4.18.0-305.25.1.el8_4.x86_64 2023-11-28T02:39:26.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum install -y kernel 2023-11-28T02:39:27.855 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:27.856 INFO:teuthology.orchestra.run.smithi073.stdout:Unable to read consumer identity 2023-11-28T02:39:27.954 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:27.954 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2023-11-28T02:39:29.377 INFO:teuthology.orchestra.run.smithi073.stdout:Copr repo for python3-asyncssh owned by ceph 16 kB/s | 3.5 kB 00:00 2023-11-28T02:39:29.597 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 11 kB/s | 3.5 kB 00:00 2023-11-28T02:39:31.550 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 8.3 MB/s | 16 MB 00:01 2023-11-28T02:39:31.604 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 7.3 MB/s | 16 MB 00:02 2023-11-28T02:39:34.883 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 4.1 MB/s | 24 kB 00:00 2023-11-28T02:39:34.968 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 4.7 MB/s | 24 kB 00:00 2023-11-28T02:39:35.625 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-11-28T02:39:35.625 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-11-28T02:39:35.626 INFO:teuthology.orchestra.run.smithi150.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-11-28T02:39:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-4.18.0-80.el8.x86_64 is already installed. 2023-11-28T02:39:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-4.18.0-425.3.1.el8.x86_64 is already installed. 2023-11-28T02:39:35.747 INFO:teuthology.orchestra.run.smithi073.stdout:Package kernel-4.18.0-305.25.1.el8_4.x86_64 is already installed. 2023-11-28T02:39:35.781 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2023-11-28T02:39:35.782 INFO:teuthology.orchestra.run.smithi150.stdout:Nothing to do. 2023-11-28T02:39:35.782 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2023-11-28T02:39:35.884 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2023-11-28T02:39:35.885 INFO:teuthology.orchestra.run.smithi073.stdout:Nothing to do. 2023-11-28T02:39:35.885 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2023-11-28T02:39:35.893 DEBUG:teuthology.orchestra.run.smithi150:> echo no | sudo yum reinstall kernel || true 2023-11-28T02:39:36.005 DEBUG:teuthology.orchestra.run.smithi073:> echo no | sudo yum reinstall kernel || true 2023-11-28T02:39:36.238 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:36.239 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2023-11-28T02:39:36.347 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:36.347 INFO:teuthology.orchestra.run.smithi073.stdout:Unable to read consumer identity 2023-11-28T02:39:36.528 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 02:39:34 AM UTC. 2023-11-28T02:39:36.639 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:02 ago on Tue 28 Nov 2023 02:39:34 AM UTC. 2023-11-28T02:39:36.680 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-11-28T02:39:36.680 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:36.680 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:36.710 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2023-11-28T02:39:36.761 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum reinstall -y kernel || true 2023-11-28T02:39:36.770 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-11-28T02:39:36.770 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:36.770 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:36.798 INFO:teuthology.orchestra.run.smithi073.stderr:Error: No packages marked for reinstall. 2023-11-28T02:39:36.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum reinstall -y kernel || true 2023-11-28T02:39:37.103 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:37.103 INFO:teuthology.orchestra.run.smithi150.stdout:Unable to read consumer identity 2023-11-28T02:39:37.191 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2023-11-28T02:39:37.192 INFO:teuthology.orchestra.run.smithi073.stdout:Unable to read consumer identity 2023-11-28T02:39:37.386 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:00:03 ago on Tue 28 Nov 2023 02:39:34 AM UTC. 2023-11-28T02:39:37.474 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:00:03 ago on Tue 28 Nov 2023 02:39:34 AM UTC. 2023-11-28T02:39:37.521 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-11-28T02:39:37.521 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:37.521 INFO:teuthology.orchestra.run.smithi150.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:37.534 INFO:teuthology.orchestra.run.smithi150.stderr:Error: No packages marked for reinstall. 2023-11-28T02:39:37.583 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T02:39:37.611 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-80.el8.x86_64 (from anaconda) not available. 2023-11-28T02:39:37.611 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-425.3.1.el8.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:37.611 INFO:teuthology.orchestra.run.smithi073.stdout:Installed package kernel-4.18.0-305.25.1.el8_4.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-11-28T02:39:37.626 INFO:teuthology.orchestra.run.smithi073.stderr:Error: No packages marked for reinstall. 2023-11-28T02:39:37.681 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T02:39:37.709 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.709 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.709 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.709 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-28T02:39:37.709 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-28T02:39:37.709 INFO:teuthology.task.kernel:Installing distro kernel on client.1... 2023-11-28T02:39:37.709 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi150.front.sepia.ceph.com, path=None, version=distro) 2023-11-28T02:39:37.710 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T02:39:37.806 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.806 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.806 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-305.25.1.el8_4.x86_64 Expected: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.806 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-11-28T02:39:37.806 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-11-28T02:39:37.806 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2023-11-28T02:39:37.806 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi073.front.sepia.ceph.com, path=None, version=distro) 2023-11-28T02:39:37.807 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q kernel | sort -rV | head -n 1 2023-11-28T02:39:37.834 INFO:teuthology.orchestra.run.smithi150.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.835 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.835 DEBUG:teuthology.orchestra.run.smithi150:> sudo rpm -qi grub2-tools 2023-11-28T02:39:37.861 INFO:teuthology.orchestra.run.smithi150.stdout:Name : grub2-tools 2023-11-28T02:39:37.861 INFO:teuthology.orchestra.run.smithi150.stdout:Epoch : 1 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Version : 2.02 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Release : 99.el8_4.1 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Architecture: x86_64 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Group : System Environment/Base 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Size : 9264631 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:License : GPLv3+ 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Relocations : (not relocatable) 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Packager : Red Hat, Inc. 2023-11-28T02:39:37.862 INFO:teuthology.orchestra.run.smithi150.stdout:Vendor : Red Hat, Inc. 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:Summary : Support tools for GRUB. 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:Description : 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:hardware devices. 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:39:37.863 INFO:teuthology.orchestra.run.smithi150.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T02:39:37.864 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T02:39:37.864 INFO:teuthology.task.kernel:Updating grub on smithi150 to boot 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.864 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T02:39:37.930 INFO:teuthology.orchestra.run.smithi073.stdout:kernel-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.930 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo rpm -qi grub2-tools 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Name : grub2-tools 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Epoch : 1 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Version : 2.02 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Release : 99.el8_4.1 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Architecture: x86_64 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Install Date: Wed 21 Dec 2022 01:02:32 PM UTC 2023-11-28T02:39:37.957 INFO:teuthology.orchestra.run.smithi073.stdout:Group : System Environment/Base 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Size : 9264631 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:License : GPLv3+ 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Signature : RSA/SHA256, Thu 09 Sep 2021 09:41:55 AM UTC, Key ID 199e2f91fd431d51 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Source RPM : grub2-2.02-99.el8_4.1.src.rpm 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Build Date : Tue 18 May 2021 07:46:34 AM UTC 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Build Host : x86-vm-07.build.eng.bos.redhat.com 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Relocations : (not relocatable) 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Packager : Red Hat, Inc. 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Vendor : Red Hat, Inc. 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:URL : http://www.gnu.org/software/grub/ 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Summary : Support tools for GRUB. 2023-11-28T02:39:37.958 INFO:teuthology.orchestra.run.smithi073.stdout:Description : 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout:customizable bootloader with modular architecture. It supports a rich 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout:variety of kernel formats, file systems, computer architectures and 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout:hardware devices. 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:39:37.959 INFO:teuthology.orchestra.run.smithi073.stdout:This subpackage provides tools for support of all platforms. 2023-11-28T02:39:37.960 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-11-28T02:39:37.960 INFO:teuthology.task.kernel:Updating grub on smithi073 to boot 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:37.960 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-11-28T02:39:39.236 INFO:teuthology.orchestra.run.smithi073.stderr:Generating grub configuration file ... 2023-11-28T02:39:39.332 INFO:teuthology.orchestra.run.smithi150.stderr:Generating grub configuration file ... 2023-11-28T02:39:41.442 INFO:teuthology.orchestra.run.smithi073.stderr:done 2023-11-28T02:39:41.444 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2023-11-28T02:39:41.457 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.LDc65BELyx 2023-11-28T02:39:41.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.LDc65BELyx 2023-11-28T02:39:41.527 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.LDc65BELyx 2023-11-28T02:39:41.617 INFO:teuthology.orchestra.run.smithi150.stderr:done 2023-11-28T02:39:41.620 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-28T02:39:41.633 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.THWepwYnVZ 2023-11-28T02:39:41.634 DEBUG:teuthology.orchestra.run.smithi150:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.THWepwYnVZ 2023-11-28T02:39:41.683 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.LDc65BELyx is 6KB 2023-11-28T02:39:41.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.THWepwYnVZ 2023-11-28T02:39:41.731 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.LDc65BELyx 2023-11-28T02:39:41.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T02:39:41.807 INFO:teuthology.orchestra.run.smithi073.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-11-28T02:39:41.808 INFO:teuthology.orchestra.run.smithi073.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-11-28T02:39:41.808 INFO:teuthology.orchestra.run.smithi073.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-11-28T02:39:41.808 INFO:teuthology.orchestra.run.smithi073.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-11-28T02:39:41.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:41.870 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.THWepwYnVZ is 6KB 2023-11-28T02:39:41.918 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.THWepwYnVZ 2023-11-28T02:39:41.932 DEBUG:teuthology.orchestra.run.smithi150:> sudo /bin/ls /boot/loader/entries || true 2023-11-28T02:39:41.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo shutdown -r now 2023-11-28T02:39:42.004 INFO:teuthology.orchestra.run.smithi150.stdout:298e228f3b5e401e8ac0f5ae159a2e06-0-rescue.conf 2023-11-28T02:39:42.004 INFO:teuthology.orchestra.run.smithi150.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-305.25.1.el8_4.x86_64.conf 2023-11-28T02:39:42.004 INFO:teuthology.orchestra.run.smithi150.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64.conf 2023-11-28T02:39:42.004 INFO:teuthology.orchestra.run.smithi150.stdout:298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-80.el8.x86_64.conf 2023-11-28T02:39:42.005 DEBUG:teuthology.orchestra.run.smithi150:> sudo grub2-set-default 298e228f3b5e401e8ac0f5ae159a2e06-4.18.0-425.3.1.el8.x86_64 2023-11-28T02:39:42.184 DEBUG:teuthology.orchestra.run.smithi150:> sudo shutdown -r now 2023-11-28T02:40:11.990 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-11-28T02:40:11.991 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-11-28T02:40:11.991 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:40:12.187 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-11-28T02:40:12.187 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:40:12.188 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:40:46.694 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-28T02:40:55.702 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:40:55.703 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:41:12.010 DEBUG:teuthology.orchestra.remote:timed out 2023-11-28T02:41:14.082 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.150 2023-11-28T02:41:21.018 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-11-28T02:41:21.019 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:41:21.510 DEBUG:teuthology.orchestra.run.smithi073:> true 2023-11-28T02:41:21.927 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-11-28T02:41:21.927 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "4.18.0-425.3.1.el8.x86_64"... 2023-11-28T02:41:21.927 DEBUG:teuthology.orchestra.run.smithi073:> uname -r 2023-11-28T02:41:21.941 INFO:teuthology.orchestra.run.smithi073.stdout:4.18.0-425.3.1.el8.x86_64 2023-11-28T02:41:21.941 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:41:21.942 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T02:41:21.942 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T02:41:22.942 DEBUG:teuthology.parallel:result is None 2023-11-28T02:41:26.083 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:41:26.083 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:41:26.644 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-28T02:41:26.899 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:41:26.899 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "4.18.0-425.3.1.el8.x86_64"... 2023-11-28T02:41:26.899 DEBUG:teuthology.orchestra.run.smithi150:> uname -r 2023-11-28T02:41:26.914 INFO:teuthology.orchestra.run.smithi150.stdout:4.18.0-425.3.1.el8.x86_64 2023-11-28T02:41:26.915 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-425.3.1.el8.x86_64 vs 4.18.0-425.3.1.el8.x86_64 2023-11-28T02:41:26.915 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-11-28T02:41:26.915 DEBUG:teuthology.task.kernel:Distro of this test job: rhel 2023-11-28T02:41:27.916 DEBUG:teuthology.parallel:result is None 2023-11-28T02:41:27.916 INFO:teuthology.run_tasks:Running task internal.base... 2023-11-28T02:41:28.001 INFO:teuthology.task.internal:Creating test directory... 2023-11-28T02:41:28.001 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T02:41:28.048 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-11-28T02:41:28.272 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-11-28T02:41:28.283 INFO:teuthology.run_tasks:Running task internal.archive... 2023-11-28T02:41:28.297 INFO:teuthology.task.internal:Creating archive directory... 2023-11-28T02:41:28.297 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T02:41:28.336 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-11-28T02:41:28.680 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-11-28T02:41:28.692 INFO:teuthology.task.internal:Enabling coredump saving... 2023-11-28T02:41:28.692 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T02:41:28.730 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-11-28T02:41:28.877 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T02:41:28.933 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T02:41:29.443 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T02:41:29.614 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-11-28T02:41:29.616 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-11-28T02:41:29.628 INFO:teuthology.task.internal:Configuring sudo... 2023-11-28T02:41:29.628 DEBUG:teuthology.orchestra.run.smithi073:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T02:41:29.630 DEBUG:teuthology.orchestra.run.smithi150:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-11-28T02:41:29.671 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-11-28T02:41:29.683 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-11-28T02:41:29.684 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T02:41:29.714 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-11-28T02:41:29.728 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T02:41:29.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T02:41:29.917 DEBUG:teuthology.orchestra.run.smithi073:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T02:41:29.958 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T02:41:29.997 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:41:29.997 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T02:41:30.061 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T02:41:30.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-11-28T02:41:30.146 DEBUG:teuthology.orchestra.run.smithi150:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T02:41:30.227 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-11-28T02:41:30.269 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:41:30.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-11-28T02:41:30.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo service rsyslog restart 2023-11-28T02:41:30.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo service rsyslog restart 2023-11-28T02:41:30.456 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T02:41:30.559 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T02:41:30.732 INFO:teuthology.run_tasks:Running task internal.timer... 2023-11-28T02:41:30.744 INFO:teuthology.task.internal:Starting timer... 2023-11-28T02:41:30.744 INFO:teuthology.run_tasks:Running task pcp... 2023-11-28T02:41:30.785 INFO:teuthology.run_tasks:Running task selinux... 2023-11-28T02:41:30.801 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-11-28T02:41:30.802 DEBUG:teuthology.orchestra.run.smithi073:> sudo service auditd rotate 2023-11-28T02:41:30.864 INFO:teuthology.orchestra.run.smithi073.stdout:Rotating logs: [ OK ] 2023-11-28T02:41:30.866 DEBUG:teuthology.orchestra.run.smithi150:> sudo service auditd rotate 2023-11-28T02:41:30.941 INFO:teuthology.orchestra.run.smithi150.stdout:Rotating logs: [ OK ] 2023-11-28T02:41:30.943 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-11-28T02:41:30.943 DEBUG:teuthology.orchestra.run.smithi073:> /usr/sbin/getenforce 2023-11-28T02:41:30.972 INFO:teuthology.orchestra.run.smithi073.stdout:Permissive 2023-11-28T02:41:30.973 DEBUG:teuthology.orchestra.run.smithi150:> /usr/sbin/getenforce 2023-11-28T02:41:30.991 INFO:teuthology.orchestra.run.smithi150.stdout:Permissive 2023-11-28T02:41:30.991 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi073.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi150.front.sepia.ceph.com': 'permissive'} 2023-11-28T02:41:30.991 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T02:41:31.037 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:41:31.037 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T02:41:31.065 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:41:31.065 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-11-28T02:41:31.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo /usr/sbin/setenforce permissive 2023-11-28T02:41:31.110 DEBUG:teuthology.orchestra.run.smithi150:> sudo /usr/sbin/setenforce permissive 2023-11-28T02:41:31.137 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-11-28T02:41:31.148 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2023-11-28T02:41:31.154 INFO:teuthology.repo_utils:Fetching main from origin 2023-11-28T02:41:31.294 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-11-28T02:41:31.316 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-11-28T02:41:31.317 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi073.front.sepia.ceph.com,smithi150.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-11-28T02:49:49.129 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi073.front.sepia.ceph.com'), Remote(name='ubuntu@smithi150.front.sepia.ceph.com')] 2023-11-28T02:49:49.130 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-11-28T02:49:49.131 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi073.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:49:49.201 DEBUG:teuthology.orchestra.run.smithi073:> true 2023-11-28T02:49:49.225 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi073.front.sepia.ceph.com' 2023-11-28T02:49:49.225 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:49:49.226 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi150.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-11-28T02:49:49.294 DEBUG:teuthology.orchestra.run.smithi150:> true 2023-11-28T02:49:49.357 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi150.front.sepia.ceph.com' 2023-11-28T02:49:49.357 INFO:teuthology.run_tasks:Running task clock... 2023-11-28T02:49:49.370 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-11-28T02:49:49.370 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T02:49:49.371 DEBUG:teuthology.orchestra.run.smithi073:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T02:49:49.373 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-11-28T02:49:49.373 DEBUG:teuthology.orchestra.run.smithi150:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T02:49:49.402 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T02:49:49.413 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T02:49:49.426 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-11-28T02:49:49.440 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-11-28T02:49:49.444 INFO:teuthology.orchestra.run.smithi073.stderr:sudo: ntpd: command not found 2023-11-28T02:49:49.455 INFO:teuthology.orchestra.run.smithi073.stdout:506 Cannot talk to daemon 2023-11-28T02:49:49.469 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T02:49:49.473 INFO:teuthology.orchestra.run.smithi150.stderr:sudo: ntpd: command not found 2023-11-28T02:49:49.482 INFO:teuthology.orchestra.run.smithi073.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T02:49:49.483 INFO:teuthology.orchestra.run.smithi150.stdout:506 Cannot talk to daemon 2023-11-28T02:49:49.498 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-11-28T02:49:49.511 INFO:teuthology.orchestra.run.smithi150.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-11-28T02:49:49.548 INFO:teuthology.orchestra.run.smithi073.stderr:bash: ntpq: command not found 2023-11-28T02:49:49.550 INFO:teuthology.orchestra.run.smithi073.stdout:210 Number of sources = 4 2023-11-28T02:49:49.550 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T02:49:49.550 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2023-11-28T02:49:49.551 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.551 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.551 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.551 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.585 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:210 Number of sources = 4 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.587 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T02:49:49.588 INFO:teuthology.run_tasks:Running task install... 2023-11-28T02:49:49.596 DEBUG:teuthology.task.install:project ceph 2023-11-28T02:49:49.597 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'}} 2023-11-28T02:49:49.597 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'} 2023-11-28T02:49:49.597 INFO:teuthology.task.install:Using flavor: default 2023-11-28T02:49:49.603 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-11-28T02:49:49.603 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-11-28T02:49:49.604 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4', 'tag': None, 'wait_for_package': False} 2023-11-28T02:49:49.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5474366c84b0c469f7b5285fcfee86232293d8b4 2023-11-28T02:49:49.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4', 'tag': None, 'wait_for_package': False} 2023-11-28T02:49:49.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5474366c84b0c469f7b5285fcfee86232293d8b4 2023-11-28T02:49:49.756 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/ 2023-11-28T02:49:49.756 INFO:teuthology.task.install.rpm:Package version is 16.2.14-270.g5474366c 2023-11-28T02:49:49.824 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/ 2023-11-28T02:49:49.825 INFO:teuthology.task.install.rpm:Package version is 16.2.14-270.g5474366c 2023-11-28T02:49:49.876 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T02:49:49.876 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:49:49.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T02:49:49.906 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-28T02:49:49.906 DEBUG:teuthology.orchestra.run.smithi073:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5474366c84b0c469f7b5285fcfee86232293d8b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T02:49:49.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T02:49:49.990 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-11-27-1028-pacific/5474366c84b0c469f7b5285fcfee86232293d8b4/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-11-28T02:49:49.991 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:49:49.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-11-28T02:49:50.017 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-11-28T02:49:50.017 DEBUG:teuthology.orchestra.run.smithi150:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5474366c84b0c469f7b5285fcfee86232293d8b4/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-11-28T02:49:50.046 DEBUG:teuthology.orchestra.run.smithi073:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T02:49:50.083 DEBUG:teuthology.orchestra.run.smithi150:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-11-28T02:49:50.109 INFO:teuthology.orchestra.run.smithi073.stdout:check_obsoletes = 1 2023-11-28T02:49:50.113 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum clean all 2023-11-28T02:49:50.158 DEBUG:teuthology.orchestra.run.smithi150:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-11-28T02:49:50.221 INFO:teuthology.orchestra.run.smithi150.stdout:check_obsoletes = 1 2023-11-28T02:49:50.223 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum clean all 2023-11-28T02:49:50.504 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2023-11-28T02:49:50.613 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-28T02:49:50.816 INFO:teuthology.orchestra.run.smithi073.stdout:44 files removed 2023-11-28T02:49:50.866 DEBUG:teuthology.orchestra.run.smithi073:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-28T02:49:50.891 INFO:teuthology.orchestra.run.smithi150.stdout:44 files removed 2023-11-28T02:49:50.948 DEBUG:teuthology.orchestra.run.smithi150:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-11-28T02:49:51.217 INFO:teuthology.orchestra.run.smithi073.stdout:Updating Subscription Management repositories. 2023-11-28T02:49:51.299 INFO:teuthology.orchestra.run.smithi150.stdout:Updating Subscription Management repositories. 2023-11-28T02:49:52.013 INFO:teuthology.orchestra.run.smithi073.stdout:ceph packages for x86_64 207 kB/s | 76 kB 00:00 2023-11-28T02:49:52.077 INFO:teuthology.orchestra.run.smithi150.stdout:ceph packages for x86_64 225 kB/s | 76 kB 00:00 2023-11-28T02:49:52.294 INFO:teuthology.orchestra.run.smithi073.stdout:ceph noarch packages 63 kB/s | 15 kB 00:00 2023-11-28T02:49:52.359 INFO:teuthology.orchestra.run.smithi150.stdout:ceph noarch packages 63 kB/s | 15 kB 00:00 2023-11-28T02:49:52.631 INFO:teuthology.orchestra.run.smithi073.stdout:ceph source packages 4.7 kB/s | 1.4 kB 00:00 2023-11-28T02:49:52.692 INFO:teuthology.orchestra.run.smithi150.stdout:ceph source packages 4.8 kB/s | 1.4 kB 00:00 2023-11-28T02:49:52.757 INFO:teuthology.orchestra.run.smithi073.stdout:Copr repo for python3-asyncssh owned by ceph 36 kB/s | 3.5 kB 00:00 2023-11-28T02:49:52.838 INFO:teuthology.orchestra.run.smithi150.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-11-28T02:49:53.256 INFO:teuthology.orchestra.run.smithi150.stdout:Extra Packages for Enterprise Linux 41 MB/s | 16 MB 00:00 2023-11-28T02:49:54.905 INFO:teuthology.orchestra.run.smithi073.stdout:Extra Packages for Enterprise Linux 7.5 MB/s | 16 MB 00:02 2023-11-28T02:49:56.594 INFO:teuthology.orchestra.run.smithi150.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-11-28T02:49:56.991 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 121 MB/s | 38 MB 00:00 2023-11-28T02:49:58.203 INFO:teuthology.orchestra.run.smithi073.stdout:lab-extras 4.2 MB/s | 24 kB 00:00 2023-11-28T02:49:58.566 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 118 MB/s | 33 MB 00:00 2023-11-28T02:50:02.493 INFO:teuthology.orchestra.run.smithi150.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 112 MB/s | 33 MB 00:00 2023-11-28T02:50:03.820 INFO:teuthology.orchestra.run.smithi073.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 118 MB/s | 38 MB 00:00 2023-11-28T02:50:10.430 INFO:teuthology.orchestra.run.smithi150.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T02:50:10.431 INFO:teuthology.orchestra.run.smithi150.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T02:50:10.441 INFO:teuthology.orchestra.run.smithi150.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-11-28T02:50:10.442 INFO:teuthology.orchestra.run.smithi150.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-11-28T02:50:10.443 INFO:teuthology.orchestra.run.smithi150.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-11-28T02:50:10.444 INFO:teuthology.orchestra.run.smithi150.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T02:50:10.445 INFO:teuthology.orchestra.run.smithi150.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T02:50:10.448 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-11-28T02:50:10.449 INFO:teuthology.orchestra.run.smithi150.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-11-28T02:50:10.943 INFO:teuthology.orchestra.run.smithi150.stdout:Dependencies resolved. 2023-11-28T02:50:10.954 INFO:teuthology.orchestra.run.smithi150.stdout:====================================================================================================================== 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: Package Arch Version Repository Size 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout:====================================================================================================================== 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout:Installing: 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph x86_64 2:16.2.14-270.g5474366c.el8 ceph 6.4 k 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base x86_64 2:16.2.14-270.g5474366c.el8 ceph 6.1 M 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse x86_64 2:16.2.14-270.g5474366c.el8 ceph 910 k 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-270.g5474366c.el8 ceph 161 k 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr x86_64 2:16.2.14-270.g5474366c.el8 ceph 1.6 M 2023-11-28T02:50:10.955 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 116 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 3.1 M 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 8.6 M 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 27 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw x86_64 2:16.2.14-270.g5474366c.el8 ceph 12 M 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test x86_64 2:16.2.14-270.g5474366c.el8 ceph 52 M 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 85 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror x86_64 2:16.2.14-270.g5474366c.el8 ceph 257 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 19 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T02:50:10.956 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel x86_64 2:16.2.14-270.g5474366c.el8 ceph 25 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 824 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel x86_64 2:16.2.14-270.g5474366c.el8 ceph 131 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs x86_64 2:16.2.14-270.g5474366c.el8 ceph 215 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados x86_64 2:16.2.14-270.g5474366c.el8 ceph 390 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd x86_64 2:16.2.14-270.g5474366c.el8 ceph 369 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw x86_64 2:16.2.14-270.g5474366c.el8 ceph 114 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse x86_64 2:16.2.14-270.g5474366c.el8 ceph 88 k 2023-11-28T02:50:10.957 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.7 M 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd x86_64 2:16.2.14-270.g5474366c.el8 ceph 175 k 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrading: 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: librados2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.7 M 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1 x86_64 2:16.2.14-270.g5474366c.el8 ceph 4.2 M 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout:Installing dependencies: 2023-11-28T02:50:10.958 INFO:teuthology.orchestra.run.smithi150.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common x86_64 2:16.2.14-270.g5474366c.el8 ceph 24 M 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 24 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds x86_64 2:16.2.14-270.g5474366c.el8 ceph 2.4 M 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 245 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon x86_64 2:16.2.14-270.g5474366c.el8 ceph 5.0 M 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd x86_64 2:16.2.14-270.g5474366c.el8 ceph 18 M 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 15 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux x86_64 2:16.2.14-270.g5474366c.el8 ceph 26 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T02:50:10.959 INFO:teuthology.orchestra.run.smithi150.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite x86_64 2:16.2.14-270.g5474366c.el8 ceph 174 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1 x86_64 2:16.2.14-270.g5474366c.el8 ceph 494 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.8 M 2023-11-28T02:50:10.960 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T02:50:10.961 INFO:teuthology.orchestra.run.smithi150.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse x86_64 2:16.2.14-270.g5474366c.el8 ceph 45 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common x86_64 2:16.2.14-270.g5474366c.el8 ceph 99 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T02:50:10.962 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T02:50:10.963 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T02:50:10.965 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T02:50:10.966 INFO:teuthology.orchestra.run.smithi150.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-11-28T02:50:10.970 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T02:50:10.971 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T02:50:10.972 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction Summary 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout:====================================================================================================================== 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout:Install 118 Packages 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout:Upgrade 2 Packages 2023-11-28T02:50:10.973 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:50:10.974 INFO:teuthology.orchestra.run.smithi150.stdout:Total download size: 201 M 2023-11-28T02:50:10.975 INFO:teuthology.orchestra.run.smithi150.stdout:Downloading Packages: 2023-11-28T02:50:11.099 INFO:teuthology.orchestra.run.smithi150.stdout:(1/120): ceph-16.2.14-270.g5474366c.el8.x86_64. 67 kB/s | 6.4 kB 00:00 2023-11-28T02:50:11.309 INFO:teuthology.orchestra.run.smithi150.stdout:(2/120): ceph-fuse-16.2.14-270.g5474366c.el8.x8 4.2 MB/s | 910 kB 00:00 2023-11-28T02:50:11.331 INFO:teuthology.orchestra.run.smithi150.stdout:(3/120): ceph-immutable-object-cache-16.2.14-27 7.1 MB/s | 161 kB 00:00 2023-11-28T02:50:11.592 INFO:teuthology.orchestra.run.smithi150.stdout:(4/120): ceph-mds-16.2.14-270.g5474366c.el8.x86 9.4 MB/s | 2.4 MB 00:00 2023-11-28T02:50:11.762 INFO:teuthology.orchestra.run.smithi150.stdout:(5/120): ceph-mgr-16.2.14-270.g5474366c.el8.x86 9.5 MB/s | 1.6 MB 00:00 2023-11-28T02:50:12.177 INFO:teuthology.orchestra.run.smithi073.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T02:50:12.177 INFO:teuthology.orchestra.run.smithi073.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-11-28T02:50:12.189 INFO:teuthology.orchestra.run.smithi073.stdout:Package NetworkManager-1:1.30.0-13.el8_4.x86_64 is already installed. 2023-11-28T02:50:12.189 INFO:teuthology.orchestra.run.smithi073.stdout:Package iproute-5.9.0-4.el8.x86_64 is already installed. 2023-11-28T02:50:12.190 INFO:teuthology.orchestra.run.smithi073.stdout:Package util-linux-2.32.1-27.el8.x86_64 is already installed. 2023-11-28T02:50:12.191 INFO:teuthology.orchestra.run.smithi073.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-11-28T02:50:12.192 INFO:teuthology.orchestra.run.smithi073.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-11-28T02:50:12.195 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsdump-3.1.8-2.el8.x86_64 is already installed. 2023-11-28T02:50:12.196 INFO:teuthology.orchestra.run.smithi073.stdout:Package xfsprogs-5.0.0-8.el8.x86_64 is already installed. 2023-11-28T02:50:12.271 INFO:teuthology.orchestra.run.smithi150.stdout:(6/120): ceph-mon-16.2.14-270.g5474366c.el8.x86 9.9 MB/s | 5.0 MB 00:00 2023-11-28T02:50:12.470 INFO:teuthology.orchestra.run.smithi150.stdout:(7/120): ceph-base-16.2.14-270.g5474366c.el8.x8 4.1 MB/s | 6.1 MB 00:01 2023-11-28T02:50:12.682 INFO:teuthology.orchestra.run.smithi073.stdout:Dependencies resolved. 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================================================== 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: Package Arch Version Repository Size 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================================================== 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout:Installing: 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: ceph x86_64 2:16.2.14-270.g5474366c.el8 ceph 6.4 k 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base x86_64 2:16.2.14-270.g5474366c.el8 ceph 6.1 M 2023-11-28T02:50:12.693 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse x86_64 2:16.2.14-270.g5474366c.el8 ceph 910 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache x86_64 2:16.2.14-270.g5474366c.el8 ceph 161 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr x86_64 2:16.2.14-270.g5474366c.el8 ceph 1.6 M 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 116 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 3.1 M 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 8.6 M 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 27 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw x86_64 2:16.2.14-270.g5474366c.el8 ceph 12 M 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test x86_64 2:16.2.14-270.g5474366c.el8 ceph 52 M 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 85 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror x86_64 2:16.2.14-270.g5474366c.el8 ceph 257 k 2023-11-28T02:50:12.694 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 19 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel x86_64 0.182-3.el8 rhel-8-for-x86_64-baseos-rpms 59 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel x86_64 2:16.2.14-270.g5474366c.el8 ceph 25 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 824 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel x86_64 2:16.2.14-270.g5474366c.el8 ceph 131 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel x86_64 2.32.1-27.el8 rhel-8-for-x86_64-baseos-rpms 97 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel x86_64 1:1.1.1g-15.el8_3 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs x86_64 2:16.2.14-270.g5474366c.el8 ceph 215 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados x86_64 2:16.2.14-270.g5474366c.el8 ceph 390 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd x86_64 2:16.2.14-270.g5474366c.el8 ceph 369 k 2023-11-28T02:50:12.695 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw x86_64 2:16.2.14-270.g5474366c.el8 ceph 114 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse x86_64 2:16.2.14-270.g5474366c.el8 ceph 88 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.7 M 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd x86_64 2:16.2.14-270.g5474366c.el8 ceph 175 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel x86_64 5.0.0-8.el8 rhel-8-for-x86_64-baseos-rpms 70 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrading: 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: librados2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.7 M 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1 x86_64 2:16.2.14-270.g5474366c.el8 ceph 4.2 M 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout:Installing dependencies: 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: automake noarch 1.16.1-7.el8 rhel-8-for-x86_64-appstream-rpms 713 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common x86_64 2:16.2.14-270.g5474366c.el8 ceph 24 M 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 24 k 2023-11-28T02:50:12.696 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds x86_64 2:16.2.14-270.g5474366c.el8 ceph 2.4 M 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 245 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon x86_64 2:16.2.14-270.g5474366c.el8 ceph 5.0 M 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd x86_64 2:16.2.14-270.g5474366c.el8 ceph 18 M 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts noarch 2:16.2.14-270.g5474366c.el8 ceph-noarch 15 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux x86_64 2:16.2.14-270.g5474366c.el8 ceph 26 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: fuse x86_64 2.9.7-12.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: jq x86_64 1.5-12.el8 rhel-8-for-x86_64-appstream-rpms 161 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: keyutils-libs-devel x86_64 1.5.10-6.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: krb5-devel x86_64 1.18.2-8.3.el8_4 rhel-8-for-x86_64-baseos-rpms 559 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: leveldb x86_64 1.22-1.el8 epel 181 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite x86_64 2:16.2.14-270.g5474366c.el8 ceph 174 k 2023-11-28T02:50:12.697 INFO:teuthology.orchestra.run.smithi073.stdout: libcom_err-devel x86_64 1.45.6-1.el8 rhel-8-for-x86_64-baseos-rpms 38 k 2023-11-28T02:50:12.698 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 639 k 2023-11-28T02:50:12.698 INFO:teuthology.orchestra.run.smithi073.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-11-28T02:50:12.698 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath x86_64 8.4.1-1.el8 rhel-8-for-x86_64-baseos-rpms 169 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq x86_64 0.9.0-3.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1 x86_64 2:16.2.14-270.g5474366c.el8 ceph 494 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka x86_64 0.11.4-1.el8 rhel-8-for-x86_64-appstream-rpms 353 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2 x86_64 2:16.2.14-270.g5474366c.el8 ceph 3.8 M 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: libsepol-devel x86_64 2.9-2.el8 rhel-8-for-x86_64-baseos-rpms 86 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-11-28T02:50:12.700 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: openblas x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.6 M 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-threads x86_64 0.3.12-1.el8 rhel-8-for-x86_64-appstream-rpms 4.7 M 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-apipkg noarch 1.5-6.el8 epel 21 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel noarch 2.5.1-5.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-11-28T02:50:12.701 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse x86_64 2:16.2.14-270.g5474366c.el8 ceph 45 k 2023-11-28T02:50:12.702 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common x86_64 2:16.2.14-270.g5474366c.el8 ceph 99 k 2023-11-28T02:50:12.702 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-11-28T02:50:12.702 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-11-28T02:50:12.702 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-11-28T02:50:12.705 INFO:teuthology.orchestra.run.smithi073.stdout: python3-execnet noarch 1.7.1-1.el8 epel 355 k 2023-11-28T02:50:12.705 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-11-28T02:50:12.705 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-11-28T02:50:12.705 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2 noarch 2.10.1-2.el8_0 rhel-8-for-x86_64-appstream-rpms 538 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako noarch 1.0.6-13.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-11-28T02:50:12.706 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py x86_64 1:1.14.3-9.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz noarch 2017.2-9.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-remoto noarch 1.2.1-1.el8 epel 49 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-11-28T02:50:12.707 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy x86_64 1.0.0-20.module+el8.1.0+3334+5cb623d7 rhel-8-for-x86_64-appstream-rpms 15 M 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-11-28T02:50:12.708 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: socat x86_64 1.7.3.3-2.el8 rhel-8-for-x86_64-appstream-rpms 302 k 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel x86_64 1.2.11-17.el8 rhel-8-for-x86_64-baseos-rpms 58 k 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction Summary 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout:====================================================================================================================== 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout:Install 118 Packages 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout:Upgrade 2 Packages 2023-11-28T02:50:12.709 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:50:12.711 INFO:teuthology.orchestra.run.smithi073.stdout:Total download size: 201 M 2023-11-28T02:50:12.711 INFO:teuthology.orchestra.run.smithi073.stdout:Downloading Packages: 2023-11-28T02:50:13.052 INFO:teuthology.orchestra.run.smithi073.stdout:(1/120): ceph-16.2.14-270.g5474366c.el8.x86_64. 32 kB/s | 6.4 kB 00:00 2023-11-28T02:50:13.136 INFO:teuthology.orchestra.run.smithi150.stdout:(8/120): ceph-common-16.2.14-270.g5474366c.el8. 11 MB/s | 24 MB 00:02 2023-11-28T02:50:13.168 INFO:teuthology.orchestra.run.smithi150.stdout:(9/120): ceph-selinux-16.2.14-270.g5474366c.el8 810 kB/s | 26 kB 00:00 2023-11-28T02:50:13.830 INFO:teuthology.orchestra.run.smithi073.stdout:(2/120): ceph-fuse-16.2.14-270.g5474366c.el8.x8 1.1 MB/s | 910 kB 00:00 2023-11-28T02:50:14.006 INFO:teuthology.orchestra.run.smithi073.stdout:(3/120): ceph-immutable-object-cache-16.2.14-27 919 kB/s | 161 kB 00:00 2023-11-28T02:50:14.301 INFO:teuthology.orchestra.run.smithi150.stdout:(10/120): ceph-osd-16.2.14-270.g5474366c.el8.x8 9.0 MB/s | 18 MB 00:02 2023-11-28T02:50:14.362 INFO:teuthology.orchestra.run.smithi150.stdout:(11/120): cephfs-mirror-16.2.14-270.g5474366c.e 4.1 MB/s | 257 kB 00:00 2023-11-28T02:50:14.414 INFO:teuthology.orchestra.run.smithi150.stdout:(12/120): libcephfs-devel-16.2.14-270.g5474366c 495 kB/s | 25 kB 00:00 2023-11-28T02:50:14.545 INFO:teuthology.orchestra.run.smithi150.stdout:(13/120): libcephfs2-16.2.14-270.g5474366c.el8. 6.2 MB/s | 824 kB 00:00 2023-11-28T02:50:14.604 INFO:teuthology.orchestra.run.smithi150.stdout:(14/120): libcephsqlite-16.2.14-270.g5474366c.e 2.9 MB/s | 174 kB 00:00 2023-11-28T02:50:14.663 INFO:teuthology.orchestra.run.smithi150.stdout:(15/120): librados-devel-16.2.14-270.g5474366c. 2.2 MB/s | 131 kB 00:00 2023-11-28T02:50:14.737 INFO:teuthology.orchestra.run.smithi150.stdout:(16/120): libradosstriper1-16.2.14-270.g5474366 6.5 MB/s | 494 kB 00:00 2023-11-28T02:50:14.802 INFO:teuthology.orchestra.run.smithi150.stdout:(17/120): ceph-radosgw-16.2.14-270.g5474366c.el 5.0 MB/s | 12 MB 00:02 2023-11-28T02:50:14.869 INFO:teuthology.orchestra.run.smithi150.stdout:(18/120): python3-ceph-argparse-16.2.14-270.g54 675 kB/s | 45 kB 00:00 2023-11-28T02:50:14.960 INFO:teuthology.orchestra.run.smithi150.stdout:(19/120): python3-ceph-common-16.2.14-270.g5474 1.1 MB/s | 99 kB 00:00 2023-11-28T02:50:15.099 INFO:teuthology.orchestra.run.smithi073.stdout:(4/120): ceph-mds-16.2.14-270.g5474366c.el8.x86 2.2 MB/s | 2.4 MB 00:01 2023-11-28T02:50:15.157 INFO:teuthology.orchestra.run.smithi150.stdout:(20/120): librgw2-16.2.14-270.g5474366c.el8.x86 9.0 MB/s | 3.8 MB 00:00 2023-11-28T02:50:15.210 INFO:teuthology.orchestra.run.smithi150.stdout:(21/120): python3-rados-16.2.14-270.g5474366c.e 7.3 MB/s | 390 kB 00:00 2023-11-28T02:50:15.273 INFO:teuthology.orchestra.run.smithi150.stdout:(22/120): python3-rbd-16.2.14-270.g5474366c.el8 5.8 MB/s | 369 kB 00:00 2023-11-28T02:50:15.327 INFO:teuthology.orchestra.run.smithi150.stdout:(23/120): python3-rgw-16.2.14-270.g5474366c.el8 2.1 MB/s | 114 kB 00:00 2023-11-28T02:50:15.362 INFO:teuthology.orchestra.run.smithi073.stdout:(5/120): ceph-base-16.2.14-270.g5474366c.el8.x8 2.4 MB/s | 6.1 MB 00:02 2023-11-28T02:50:15.371 INFO:teuthology.orchestra.run.smithi150.stdout:(24/120): rbd-fuse-16.2.14-270.g5474366c.el8.x8 1.9 MB/s | 88 kB 00:00 2023-11-28T02:50:15.394 INFO:teuthology.orchestra.run.smithi150.stdout:(25/120): python3-cephfs-16.2.14-270.g5474366c. 496 kB/s | 215 kB 00:00 2023-11-28T02:50:15.490 INFO:teuthology.orchestra.run.smithi150.stdout:(26/120): rbd-nbd-16.2.14-270.g5474366c.el8.x86 1.8 MB/s | 175 kB 00:00 2023-11-28T02:50:15.577 INFO:teuthology.orchestra.run.smithi150.stdout:(27/120): ceph-grafana-dashboards-16.2.14-270.g 280 kB/s | 24 kB 00:00 2023-11-28T02:50:15.601 INFO:teuthology.orchestra.run.smithi073.stdout:(6/120): ceph-mgr-16.2.14-270.g5474366c.el8.x86 3.2 MB/s | 1.6 MB 00:00 2023-11-28T02:50:15.662 INFO:teuthology.orchestra.run.smithi150.stdout:(28/120): ceph-mgr-cephadm-16.2.14-270.g5474366 1.4 MB/s | 116 kB 00:00 2023-11-28T02:50:15.859 INFO:teuthology.orchestra.run.smithi150.stdout:(29/120): rbd-mirror-16.2.14-270.g5474366c.el8. 7.5 MB/s | 3.7 MB 00:00 2023-11-28T02:50:16.781 INFO:teuthology.orchestra.run.smithi150.stdout:(30/120): ceph-mgr-dashboard-16.2.14-270.g54743 2.8 MB/s | 3.1 MB 00:01 2023-11-28T02:50:16.826 INFO:teuthology.orchestra.run.smithi073.stdout:(7/120): ceph-mon-16.2.14-270.g5474366c.el8.x86 3.4 MB/s | 5.0 MB 00:01 2023-11-28T02:50:16.934 INFO:teuthology.orchestra.run.smithi150.stdout:(31/120): ceph-mgr-diskprediction-local-16.2.14 8.0 MB/s | 8.6 MB 00:01 2023-11-28T02:50:16.945 INFO:teuthology.orchestra.run.smithi150.stdout:(32/120): ceph-mgr-modules-core-16.2.14-270.g54 1.5 MB/s | 245 kB 00:00 2023-11-28T02:50:16.978 INFO:teuthology.orchestra.run.smithi150.stdout:(33/120): ceph-mgr-rook-16.2.14-270.g5474366c.e 634 kB/s | 27 kB 00:00 2023-11-28T02:50:17.010 INFO:teuthology.orchestra.run.smithi150.stdout:(34/120): ceph-prometheus-alerts-16.2.14-270.g5 232 kB/s | 15 kB 00:00 2023-11-28T02:50:17.027 INFO:teuthology.orchestra.run.smithi150.stdout:(35/120): cephadm-16.2.14-270.g5474366c.el8.noa 1.7 MB/s | 85 kB 00:00 2023-11-28T02:50:17.082 INFO:teuthology.orchestra.run.smithi150.stdout:(36/120): cephfs-top-16.2.14-270.g5474366c.el8. 260 kB/s | 19 kB 00:00 2023-11-28T02:50:17.657 INFO:teuthology.orchestra.run.smithi150.stdout:(37/120): leveldb-1.22-1.el8.x86_64.rpm 315 kB/s | 181 kB 00:00 2023-11-28T02:50:17.701 INFO:teuthology.orchestra.run.smithi150.stdout:(38/120): gperftools-libs-2.7-9.el8.x86_64.rpm 454 kB/s | 306 kB 00:00 2023-11-28T02:50:17.739 INFO:teuthology.orchestra.run.smithi150.stdout:(39/120): liboath-2.6.2-3.el8.x86_64.rpm 725 kB/s | 59 kB 00:00 2023-11-28T02:50:17.782 INFO:teuthology.orchestra.run.smithi150.stdout:(40/120): libunwind-1.3.1-3.el8.x86_64.rpm 925 kB/s | 75 kB 00:00 2023-11-28T02:50:17.819 INFO:teuthology.orchestra.run.smithi150.stdout:(41/120): python3-apipkg-1.5-6.el8.noarch.rpm 262 kB/s | 21 kB 00:00 2023-11-28T02:50:17.864 INFO:teuthology.orchestra.run.smithi150.stdout:(42/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 535 kB/s | 44 kB 00:00 2023-11-28T02:50:17.903 INFO:teuthology.orchestra.run.smithi150.stdout:(43/120): python3-beautifulsoup4-4.6.3-2.el8.1. 2.2 MB/s | 185 kB 00:00 2023-11-28T02:50:17.946 INFO:teuthology.orchestra.run.smithi150.stdout:(44/120): python3-cachetools-3.1.1-4.el8.noarch 402 kB/s | 33 kB 00:00 2023-11-28T02:50:17.967 INFO:teuthology.orchestra.run.smithi073.stdout:(8/120): ceph-common-16.2.14-270.g5474366c.el8. 4.7 MB/s | 24 MB 00:05 2023-11-28T02:50:17.982 INFO:teuthology.orchestra.run.smithi150.stdout:(45/120): python3-certifi-2018.10.15-7.el8.noar 203 kB/s | 16 kB 00:00 2023-11-28T02:50:17.992 INFO:teuthology.orchestra.run.smithi073.stdout:(9/120): ceph-selinux-16.2.14-270.g5474366c.el8 1.0 MB/s | 26 kB 00:00 2023-11-28T02:50:18.052 INFO:teuthology.orchestra.run.smithi150.stdout:(46/120): python3-cheroot-8.5.2-1.el8.noarch.rp 1.6 MB/s | 173 kB 00:00 2023-11-28T02:50:18.151 INFO:teuthology.orchestra.run.smithi150.stdout:(47/120): python3-cherrypy-18.4.0-1.el8.noarch. 2.2 MB/s | 384 kB 00:00 2023-11-28T02:50:18.214 INFO:teuthology.orchestra.run.smithi150.stdout:(48/120): python3-execnet-1.7.1-1.el8.noarch.rp 2.2 MB/s | 355 kB 00:00 2023-11-28T02:50:18.232 INFO:teuthology.orchestra.run.smithi150.stdout:(49/120): python3-google-auth-1.1.1-10.el8.noar 1.0 MB/s | 81 kB 00:00 2023-11-28T02:50:18.295 INFO:teuthology.orchestra.run.smithi150.stdout:(50/120): python3-jaraco-6.2-6.el8.noarch.rpm 137 kB/s | 11 kB 00:00 2023-11-28T02:50:18.313 INFO:teuthology.orchestra.run.smithi150.stdout:(51/120): python3-jaraco-functools-2.0-4.el8.no 217 kB/s | 18 kB 00:00 2023-11-28T02:50:18.395 INFO:teuthology.orchestra.run.smithi150.stdout:(52/120): python3-logutils-0.3.5-11.el8.noarch. 596 kB/s | 49 kB 00:00 2023-11-28T02:50:18.465 INFO:teuthology.orchestra.run.smithi150.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 6.9 MB/s | 1.2 MB 00:00 2023-11-28T02:50:18.476 INFO:teuthology.orchestra.run.smithi150.stdout:(54/120): python3-more-itertools-7.2.0-3.el8.no 725 kB/s | 59 kB 00:00 2023-11-28T02:50:18.548 INFO:teuthology.orchestra.run.smithi150.stdout:(55/120): python3-pecan-1.3.2-9.el8.noarch.rpm 3.3 MB/s | 283 kB 00:00 2023-11-28T02:50:18.557 INFO:teuthology.orchestra.run.smithi150.stdout:(56/120): python3-portend-2.6-1.el8.noarch.rpm 193 kB/s | 16 kB 00:00 2023-11-28T02:50:19.013 INFO:teuthology.orchestra.run.smithi150.stdout:(57/120): ceph-test-16.2.14-270.g5474366c.el8.x 8.9 MB/s | 52 MB 00:05 2023-11-28T02:50:19.014 INFO:teuthology.orchestra.run.smithi150.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 74 kB/s | 34 kB 00:00 2023-11-28T02:50:19.015 INFO:teuthology.orchestra.run.smithi150.stdout:(59/120): python3-remoto-1.2.1-1.el8.noarch.rpm 105 kB/s | 49 kB 00:00 2023-11-28T02:50:19.082 INFO:teuthology.orchestra.run.smithi073.stdout:(10/120): ceph-radosgw-16.2.14-270.g5474366c.el 5.1 MB/s | 12 MB 00:02 2023-11-28T02:50:19.098 INFO:teuthology.orchestra.run.smithi150.stdout:(60/120): python3-simplegeneric-0.8.1-17.el8.no 225 kB/s | 19 kB 00:00 2023-11-28T02:50:19.099 INFO:teuthology.orchestra.run.smithi150.stdout:(61/120): python3-rsa-4.9-2.el8.noarch.rpm 758 kB/s | 64 kB 00:00 2023-11-28T02:50:19.147 INFO:teuthology.orchestra.run.smithi073.stdout:(11/120): cephfs-mirror-16.2.14-270.g5474366c.e 3.8 MB/s | 257 kB 00:00 2023-11-28T02:50:19.169 INFO:teuthology.orchestra.run.smithi073.stdout:(12/120): libcephfs-devel-16.2.14-270.g5474366c 1.2 MB/s | 25 kB 00:00 2023-11-28T02:50:19.182 INFO:teuthology.orchestra.run.smithi150.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 298 kB/s | 24 kB 00:00 2023-11-28T02:50:19.182 INFO:teuthology.orchestra.run.smithi150.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 364 kB/s | 29 kB 00:00 2023-11-28T02:50:19.261 INFO:teuthology.orchestra.run.smithi150.stdout:(64/120): python3-trustme-0.6.0-4.el8.noarch.rp 343 kB/s | 27 kB 00:00 2023-11-28T02:50:19.263 INFO:teuthology.orchestra.run.smithi150.stdout:(65/120): python3-waitress-1.4.3-1.el8.noarch.r 3.0 MB/s | 255 kB 00:00 2023-11-28T02:50:19.293 INFO:teuthology.orchestra.run.smithi073.stdout:(13/120): libcephfs2-16.2.14-270.g5474366c.el8. 6.5 MB/s | 824 kB 00:00 2023-11-28T02:50:19.323 INFO:teuthology.orchestra.run.smithi073.stdout:(14/120): libcephsqlite-16.2.14-270.g5474366c.e 5.6 MB/s | 174 kB 00:00 2023-11-28T02:50:19.342 INFO:teuthology.orchestra.run.smithi150.stdout:(66/120): python3-webob-1.8.5-1.el8.1.noarch.rp 3.0 MB/s | 251 kB 00:00 2023-11-28T02:50:19.344 INFO:teuthology.orchestra.run.smithi150.stdout:(67/120): python3-websocket-client-0.56.0-5.el8 758 kB/s | 61 kB 00:00 2023-11-28T02:50:19.352 INFO:teuthology.orchestra.run.smithi073.stdout:(15/120): librados-devel-16.2.14-270.g5474366c. 4.5 MB/s | 131 kB 00:00 2023-11-28T02:50:19.424 INFO:teuthology.orchestra.run.smithi150.stdout:(68/120): python3-webtest-2.0.33-1.el8.noarch.r 1.0 MB/s | 85 kB 00:00 2023-11-28T02:50:19.424 INFO:teuthology.orchestra.run.smithi150.stdout:(69/120): python3-zc-lockfile-2.0-2.el8.noarch. 286 kB/s | 23 kB 00:00 2023-11-28T02:50:19.428 INFO:teuthology.orchestra.run.smithi150.stdout:(70/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 17 MB/s | 70 kB 00:00 2023-11-28T02:50:19.437 INFO:teuthology.orchestra.run.smithi073.stdout:(16/120): libradosstriper1-16.2.14-270.g5474366 5.6 MB/s | 494 kB 00:00 2023-11-28T02:50:19.453 INFO:teuthology.orchestra.run.smithi150.stdout:(71/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 670 kB/s | 19 kB 00:00 2023-11-28T02:50:19.459 INFO:teuthology.orchestra.run.smithi150.stdout:(72/120): python3-jwt-1.6.1-2.el8.noarch.rpm 1.3 MB/s | 43 kB 00:00 2023-11-28T02:50:19.470 INFO:teuthology.orchestra.run.smithi150.stdout:(73/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.7 MB/s | 155 kB 00:00 2023-11-28T02:50:19.475 INFO:teuthology.orchestra.run.smithi150.stdout:(74/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 13 MB/s | 220 kB 00:00 2023-11-28T02:50:19.484 INFO:teuthology.orchestra.run.smithi150.stdout:(75/120): keyutils-libs-devel-1.5.10-6.el8.x86_ 3.5 MB/s | 48 kB 00:00 2023-11-28T02:50:19.491 INFO:teuthology.orchestra.run.smithi150.stdout:(76/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 15 MB/s | 229 kB 00:00 2023-11-28T02:50:19.499 INFO:teuthology.orchestra.run.smithi150.stdout:(77/120): python3-requests-oauthlib-1.0.0-1.el8 2.8 MB/s | 43 kB 00:00 2023-11-28T02:50:19.513 INFO:teuthology.orchestra.run.smithi150.stdout:(78/120): libverto-devel-0.3.0-5.el8.x86_64.rpm 1.3 MB/s | 18 kB 00:00 2023-11-28T02:50:19.536 INFO:teuthology.orchestra.run.smithi150.stdout:(79/120): openssl-devel-1.1.1g-15.el8_3.x86_64. 52 MB/s | 2.3 MB 00:00 2023-11-28T02:50:19.540 INFO:teuthology.orchestra.run.smithi150.stdout:(80/120): pcre2-devel-10.32-2.el8.x86_64.rpm 22 MB/s | 605 kB 00:00 2023-11-28T02:50:19.551 INFO:teuthology.orchestra.run.smithi150.stdout:(81/120): fuse-2.9.7-12.el8.x86_64.rpm 5.4 MB/s | 83 kB 00:00 2023-11-28T02:50:19.554 INFO:teuthology.orchestra.run.smithi150.stdout:(82/120): elfutils-libelf-devel-0.182-3.el8.x86 4.0 MB/s | 59 kB 00:00 2023-11-28T02:50:19.565 INFO:teuthology.orchestra.run.smithi150.stdout:(83/120): libquadmath-8.4.1-1.el8.x86_64.rpm 11 MB/s | 169 kB 00:00 2023-11-28T02:50:19.567 INFO:teuthology.orchestra.run.smithi150.stdout:(84/120): libcom_err-devel-1.45.6-1.el8.x86_64. 2.8 MB/s | 38 kB 00:00 2023-11-28T02:50:19.581 INFO:teuthology.orchestra.run.smithi150.stdout:(85/120): libsepol-devel-2.9-2.el8.x86_64.rpm 5.9 MB/s | 86 kB 00:00 2023-11-28T02:50:19.582 INFO:teuthology.orchestra.run.smithi150.stdout:(86/120): python3-routes-2.4.1-12.el8.noarch.rp 346 kB/s | 196 kB 00:00 2023-11-28T02:50:19.583 INFO:teuthology.orchestra.run.smithi150.stdout:(87/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 3.8 MB/s | 58 kB 00:00 2023-11-28T02:50:19.597 INFO:teuthology.orchestra.run.smithi150.stdout:(88/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 2.8 MB/s | 47 kB 00:00 2023-11-28T02:50:19.604 INFO:teuthology.orchestra.run.smithi150.stdout:(89/120): libgfortran-8.4.1-1.el8.x86_64.rpm 30 MB/s | 639 kB 00:00 2023-11-28T02:50:19.607 INFO:teuthology.orchestra.run.smithi073.stdout:(17/120): ceph-osd-16.2.14-270.g5474366c.el8.x8 4.6 MB/s | 18 MB 00:04 2023-11-28T02:50:19.611 INFO:teuthology.orchestra.run.smithi150.stdout:(90/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rpm 4.6 MB/s | 70 kB 00:00 2023-11-28T02:50:19.617 INFO:teuthology.orchestra.run.smithi150.stdout:(91/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.rp 16 MB/s | 559 kB 00:00 2023-11-28T02:50:19.620 INFO:teuthology.orchestra.run.smithi150.stdout:(92/120): libselinux-devel-2.9-5.el8.x86_64.rpm 12 MB/s | 200 kB 00:00 2023-11-28T02:50:19.627 INFO:teuthology.orchestra.run.smithi150.stdout:(93/120): libuuid-devel-2.32.1-27.el8.x86_64.rp 6.4 MB/s | 97 kB 00:00 2023-11-28T02:50:19.640 INFO:teuthology.orchestra.run.smithi150.stdout:(94/120): python3-jsonpatch-1.21-2.el8.noarch.r 2.0 MB/s | 27 kB 00:00 2023-11-28T02:50:19.641 INFO:teuthology.orchestra.run.smithi150.stdout:(95/120): python3-prettytable-0.7.2-14.el8.noar 2.1 MB/s | 44 kB 00:00 2023-11-28T02:50:19.649 INFO:teuthology.orchestra.run.smithi073.stdout:(18/120): python3-ceph-argparse-16.2.14-270.g54 1.1 MB/s | 45 kB 00:00 2023-11-28T02:50:19.656 INFO:teuthology.orchestra.run.smithi150.stdout:(96/120): jq-1.5-12.el8.x86_64.rpm 10 MB/s | 161 kB 00:00 2023-11-28T02:50:19.660 INFO:teuthology.orchestra.run.smithi150.stdout:(97/120): flex-2.6.1-9.el8.x86_64.rpm 17 MB/s | 320 kB 00:00 2023-11-28T02:50:19.675 INFO:teuthology.orchestra.run.smithi150.stdout:(98/120): python3-werkzeug-0.12.2-4.el8.noarch. 24 MB/s | 457 kB 00:00 2023-11-28T02:50:19.689 INFO:teuthology.orchestra.run.smithi150.stdout:(99/120): python3-markupsafe-0.23-19.el8.x86_64 2.9 MB/s | 39 kB 00:00 2023-11-28T02:50:19.693 INFO:teuthology.orchestra.run.smithi073.stdout:(19/120): python3-ceph-common-16.2.14-270.g5474 2.2 MB/s | 99 kB 00:00 2023-11-28T02:50:19.713 INFO:teuthology.orchestra.run.smithi150.stdout:(100/120): bison-3.0.4-10.el8.x86_64.rpm 28 MB/s | 688 kB 00:00 2023-11-28T02:50:19.729 INFO:teuthology.orchestra.run.smithi150.stdout:(101/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 16 MB/s | 259 kB 00:00 2023-11-28T02:50:19.741 INFO:teuthology.orchestra.run.smithi073.stdout:(20/120): python3-cephfs-16.2.14-270.g5474366c. 4.5 MB/s | 215 kB 00:00 2023-11-28T02:50:19.750 INFO:teuthology.orchestra.run.smithi150.stdout:(102/120): python3-babel-2.5.1-5.el8.noarch.rpm 53 MB/s | 4.8 MB 00:00 2023-11-28T02:50:19.753 INFO:teuthology.orchestra.run.smithi150.stdout:(103/120): python3-numpy-f2py-1.14.3-9.el8.x86_ 9.2 MB/s | 225 kB 00:00 2023-11-28T02:50:19.763 INFO:teuthology.orchestra.run.smithi150.stdout:(104/120): python3-pyasn1-0.3.7-6.el8.noarch.rp 9.3 MB/s | 126 kB 00:00 2023-11-28T02:50:19.779 INFO:teuthology.orchestra.run.smithi150.stdout:(105/120): oniguruma-6.8.2-2.el8.x86_64.rpm 12 MB/s | 187 kB 00:00 2023-11-28T02:50:19.799 INFO:teuthology.orchestra.run.smithi150.stdout:(106/120): python3-jinja2-2.10.1-2.el8_0.noarch 26 MB/s | 538 kB 00:00 2023-11-28T02:50:19.823 INFO:teuthology.orchestra.run.smithi150.stdout:(107/120): python3-numpy-1.14.3-9.el8.x86_64.rp 54 MB/s | 3.7 MB 00:00 2023-11-28T02:50:19.825 INFO:teuthology.orchestra.run.smithi150.stdout:(108/120): socat-1.7.3.3-2.el8.x86_64.rpm 11 MB/s | 302 kB 00:00 2023-11-28T02:50:19.826 INFO:teuthology.orchestra.run.smithi073.stdout:(21/120): python3-rados-16.2.14-270.g5474366c.e 4.5 MB/s | 390 kB 00:00 2023-11-28T02:50:19.909 INFO:teuthology.orchestra.run.smithi073.stdout:(22/120): python3-rbd-16.2.14-270.g5474366c.el8 4.4 MB/s | 369 kB 00:00 2023-11-28T02:50:19.939 INFO:teuthology.orchestra.run.smithi150.stdout:(109/120): python3-scipy-1.0.0-20.module+el8.1. 45 MB/s | 15 MB 00:00 2023-11-28T02:50:19.941 INFO:teuthology.orchestra.run.smithi150.stdout:(110/120): python3-pyasn1-modules-0.3.7-6.el8.n 951 kB/s | 110 kB 00:00 2023-11-28T02:50:19.942 INFO:teuthology.orchestra.run.smithi150.stdout:(111/120): python3-mako-1.0.6-13.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2023-11-28T02:50:19.950 INFO:teuthology.orchestra.run.smithi073.stdout:(23/120): librgw2-16.2.14-270.g5474366c.el8.x86 7.4 MB/s | 3.8 MB 00:00 2023-11-28T02:50:19.953 INFO:teuthology.orchestra.run.smithi073.stdout:(24/120): python3-rgw-16.2.14-270.g5474366c.el8 2.5 MB/s | 114 kB 00:00 2023-11-28T02:50:19.956 INFO:teuthology.orchestra.run.smithi150.stdout:(112/120): python3-jsonpointer-1.10-11.el8.noar 1.4 MB/s | 20 kB 00:00 2023-11-28T02:50:19.958 INFO:teuthology.orchestra.run.smithi150.stdout:(113/120): librdkafka-0.11.4-1.el8.x86_64.rpm 19 MB/s | 353 kB 00:00 2023-11-28T02:50:19.965 INFO:teuthology.orchestra.run.smithi150.stdout:(114/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-11-28T02:50:19.970 INFO:teuthology.orchestra.run.smithi150.stdout:(115/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.9 MB/s | 54 kB 00:00 2023-11-28T02:50:19.975 INFO:teuthology.orchestra.run.smithi073.stdout:(25/120): rbd-fuse-16.2.14-270.g5474366c.el8.x8 3.5 MB/s | 88 kB 00:00 2023-11-28T02:50:19.994 INFO:teuthology.orchestra.run.smithi150.stdout:(116/120): automake-1.16.1-7.el8.noarch.rpm 29 MB/s | 713 kB 00:00 2023-11-28T02:50:20.009 INFO:teuthology.orchestra.run.smithi073.stdout:(26/120): rbd-nbd-16.2.14-270.g5474366c.el8.x86 5.1 MB/s | 175 kB 00:00 2023-11-28T02:50:20.037 INFO:teuthology.orchestra.run.smithi073.stdout:(27/120): ceph-grafana-dashboards-16.2.14-270.g 858 kB/s | 24 kB 00:00 2023-11-28T02:50:20.038 INFO:teuthology.orchestra.run.smithi150.stdout:(117/120): openblas-threads-0.3.12-1.el8.x86_64 59 MB/s | 4.7 MB 00:00 2023-11-28T02:50:20.056 INFO:teuthology.orchestra.run.smithi150.stdout:(118/120): openblas-0.3.12-1.el8.x86_64.rpm 51 MB/s | 4.6 MB 00:00 2023-11-28T02:50:20.063 INFO:teuthology.orchestra.run.smithi073.stdout:(28/120): ceph-mgr-cephadm-16.2.14-270.g5474366 4.4 MB/s | 116 kB 00:00 2023-11-28T02:50:20.593 INFO:teuthology.orchestra.run.smithi150.stdout:(119/120): librados2-16.2.14-270.g5474366c.el8. 6.3 MB/s | 3.7 MB 00:00 2023-11-28T02:50:20.664 INFO:teuthology.orchestra.run.smithi073.stdout:(29/120): rbd-mirror-16.2.14-270.g5474366c.el8. 5.2 MB/s | 3.7 MB 00:00 2023-11-28T02:50:20.673 INFO:teuthology.orchestra.run.smithi073.stdout:(30/120): ceph-mgr-dashboard-16.2.14-270.g54743 5.1 MB/s | 3.1 MB 00:00 2023-11-28T02:50:20.712 INFO:teuthology.orchestra.run.smithi073.stdout:(31/120): ceph-mgr-modules-core-16.2.14-270.g54 6.2 MB/s | 245 kB 00:00 2023-11-28T02:50:20.749 INFO:teuthology.orchestra.run.smithi073.stdout:(32/120): ceph-mgr-rook-16.2.14-270.g5474366c.e 754 kB/s | 27 kB 00:00 2023-11-28T02:50:20.776 INFO:teuthology.orchestra.run.smithi073.stdout:(33/120): ceph-prometheus-alerts-16.2.14-270.g5 564 kB/s | 15 kB 00:00 2023-11-28T02:50:20.809 INFO:teuthology.orchestra.run.smithi073.stdout:(34/120): cephadm-16.2.14-270.g5474366c.el8.noa 2.6 MB/s | 85 kB 00:00 2023-11-28T02:50:20.842 INFO:teuthology.orchestra.run.smithi073.stdout:(35/120): cephfs-top-16.2.14-270.g5474366c.el8. 563 kB/s | 19 kB 00:00 2023-11-28T02:50:21.001 INFO:teuthology.orchestra.run.smithi073.stdout:(36/120): gperftools-libs-2.7-9.el8.x86_64.rpm 1.9 MB/s | 306 kB 00:00 2023-11-28T02:50:21.051 INFO:teuthology.orchestra.run.smithi073.stdout:(37/120): leveldb-1.22-1.el8.x86_64.rpm 3.5 MB/s | 181 kB 00:00 2023-11-28T02:50:21.069 INFO:teuthology.orchestra.run.smithi073.stdout:(38/120): liboath-2.6.2-3.el8.x86_64.rpm 3.4 MB/s | 59 kB 00:00 2023-11-28T02:50:21.087 INFO:teuthology.orchestra.run.smithi073.stdout:(39/120): libunwind-1.3.1-3.el8.x86_64.rpm 4.2 MB/s | 75 kB 00:00 2023-11-28T02:50:21.106 INFO:teuthology.orchestra.run.smithi073.stdout:(40/120): python3-apipkg-1.5-6.el8.noarch.rpm 1.2 MB/s | 21 kB 00:00 2023-11-28T02:50:21.121 INFO:teuthology.orchestra.run.smithi073.stdout:(41/120): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 2.5 MB/s | 44 kB 00:00 2023-11-28T02:50:21.151 INFO:teuthology.orchestra.run.smithi150.stdout:(120/120): librbd1-16.2.14-270.g5474366c.el8.x8 3.8 MB/s | 4.2 MB 00:01 2023-11-28T02:50:21.156 INFO:teuthology.orchestra.run.smithi150.stdout:-------------------------------------------------------------------------------- 2023-11-28T02:50:21.156 INFO:teuthology.orchestra.run.smithi150.stdout:Total 20 MB/s | 201 MB 00:10 2023-11-28T02:50:21.171 INFO:teuthology.orchestra.run.smithi073.stdout:(42/120): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-11-28T02:50:21.188 INFO:teuthology.orchestra.run.smithi073.stdout:(43/120): python3-cachetools-3.1.1-4.el8.noarch 1.9 MB/s | 33 kB 00:00 2023-11-28T02:50:21.204 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction check 2023-11-28T02:50:21.205 INFO:teuthology.orchestra.run.smithi073.stdout:(44/120): python3-certifi-2018.10.15-7.el8.noar 971 kB/s | 16 kB 00:00 2023-11-28T02:50:21.255 INFO:teuthology.orchestra.run.smithi073.stdout:(45/120): python3-cheroot-8.5.2-1.el8.noarch.rp 3.4 MB/s | 173 kB 00:00 2023-11-28T02:50:21.317 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction check succeeded. 2023-11-28T02:50:21.317 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction test 2023-11-28T02:50:21.356 INFO:teuthology.orchestra.run.smithi073.stdout:(46/120): python3-cherrypy-18.4.0-1.el8.noarch. 3.8 MB/s | 384 kB 00:00 2023-11-28T02:50:21.455 INFO:teuthology.orchestra.run.smithi073.stdout:(47/120): python3-execnet-1.7.1-1.el8.noarch.rp 3.5 MB/s | 355 kB 00:00 2023-11-28T02:50:21.473 INFO:teuthology.orchestra.run.smithi073.stdout:(48/120): python3-google-auth-1.1.1-10.el8.noar 4.5 MB/s | 81 kB 00:00 2023-11-28T02:50:21.490 INFO:teuthology.orchestra.run.smithi073.stdout:(49/120): python3-jaraco-6.2-6.el8.noarch.rpm 669 kB/s | 11 kB 00:00 2023-11-28T02:50:21.507 INFO:teuthology.orchestra.run.smithi073.stdout:(50/120): python3-jaraco-functools-2.0-4.el8.no 1.0 MB/s | 18 kB 00:00 2023-11-28T02:50:22.018 INFO:teuthology.orchestra.run.smithi073.stdout:(51/120): ceph-test-16.2.14-270.g5474366c.el8.x 13 MB/s | 52 MB 00:04 2023-11-28T02:50:22.048 INFO:teuthology.orchestra.run.smithi073.stdout:(52/120): ceph-mgr-diskprediction-local-16.2.14 6.2 MB/s | 8.6 MB 00:01 2023-11-28T02:50:22.055 INFO:teuthology.orchestra.run.smithi073.stdout:(53/120): python3-kubernetes-11.0.0-6.el8.noarc 2.1 MB/s | 1.2 MB 00:00 2023-11-28T02:50:22.122 INFO:teuthology.orchestra.run.smithi073.stdout:(54/120): python3-pecan-1.3.2-9.el8.noarch.rpm 4.2 MB/s | 283 kB 00:00 2023-11-28T02:50:22.139 INFO:teuthology.orchestra.run.smithi073.stdout:(55/120): python3-portend-2.6-1.el8.noarch.rpm 920 kB/s | 16 kB 00:00 2023-11-28T02:50:22.140 INFO:teuthology.orchestra.run.smithi073.stdout:(56/120): python3-logutils-0.3.5-11.el8.noarch. 401 kB/s | 49 kB 00:00 2023-11-28T02:50:22.157 INFO:teuthology.orchestra.run.smithi073.stdout:(57/120): python3-remoto-1.2.1-1.el8.noarch.rpm 2.7 MB/s | 49 kB 00:00 2023-11-28T02:50:22.157 INFO:teuthology.orchestra.run.smithi073.stdout:(58/120): python3-repoze-lru-0.7-6.el8.noarch.r 1.9 MB/s | 34 kB 00:00 2023-11-28T02:50:22.162 INFO:teuthology.orchestra.run.smithi073.stdout:(59/120): python3-more-itertools-7.2.0-3.el8.no 531 kB/s | 59 kB 00:00 2023-11-28T02:50:22.176 INFO:teuthology.orchestra.run.smithi073.stdout:(60/120): python3-rsa-4.9-2.el8.noarch.rpm 3.5 MB/s | 64 kB 00:00 2023-11-28T02:50:22.180 INFO:teuthology.orchestra.run.smithi073.stdout:(61/120): python3-simplegeneric-0.8.1-17.el8.no 1.1 MB/s | 19 kB 00:00 2023-11-28T02:50:22.193 INFO:teuthology.orchestra.run.smithi073.stdout:(62/120): python3-singledispatch-3.4.0.3-18.el8 1.4 MB/s | 24 kB 00:00 2023-11-28T02:50:22.197 INFO:teuthology.orchestra.run.smithi073.stdout:(63/120): python3-tempora-1.14.1-5.el8.noarch.r 1.7 MB/s | 29 kB 00:00 2023-11-28T02:50:22.207 INFO:teuthology.orchestra.run.smithi073.stdout:(64/120): python3-routes-2.4.1-12.el8.noarch.rp 3.8 MB/s | 196 kB 00:00 2023-11-28T02:50:22.209 INFO:teuthology.orchestra.run.smithi073.stdout:(65/120): python3-trustme-0.6.0-4.el8.noarch.rp 1.6 MB/s | 27 kB 00:00 2023-11-28T02:50:22.234 INFO:teuthology.orchestra.run.smithi073.stdout:(66/120): python3-websocket-client-0.56.0-5.el8 2.5 MB/s | 61 kB 00:00 2023-11-28T02:50:22.283 INFO:teuthology.orchestra.run.smithi073.stdout:(67/120): python3-webtest-2.0.33-1.el8.noarch.r 1.7 MB/s | 85 kB 00:00 2023-11-28T02:50:22.284 INFO:teuthology.orchestra.run.smithi073.stdout:(68/120): python3-waitress-1.4.3-1.el8.noarch.r 2.9 MB/s | 255 kB 00:00 2023-11-28T02:50:22.289 INFO:teuthology.orchestra.run.smithi073.stdout:(69/120): xmlstarlet-1.6.1-20.el8.x86_64.rpm 16 MB/s | 70 kB 00:00 2023-11-28T02:50:22.296 INFO:teuthology.orchestra.run.smithi073.stdout:(70/120): python3-webob-1.8.5-1.el8.1.noarch.rp 2.8 MB/s | 251 kB 00:00 2023-11-28T02:50:22.309 INFO:teuthology.orchestra.run.smithi073.stdout:(71/120): python3-zc-lockfile-2.0-2.el8.noarch. 886 kB/s | 23 kB 00:00 2023-11-28T02:50:22.326 INFO:teuthology.orchestra.run.smithi073.stdout:(72/120): python3-prettytable-0.7.2-14.el8.noar 1.4 MB/s | 44 kB 00:00 2023-11-28T02:50:22.347 INFO:teuthology.orchestra.run.smithi073.stdout:(73/120): python3-jsonpatch-1.21-2.el8.noarch.r 884 kB/s | 27 kB 00:00 2023-11-28T02:50:22.347 INFO:teuthology.orchestra.run.smithi073.stdout:(74/120): jq-1.5-12.el8.x86_64.rpm 8.2 MB/s | 161 kB 00:00 2023-11-28T02:50:22.362 INFO:teuthology.orchestra.run.smithi073.stdout:(75/120): flex-2.6.1-9.el8.x86_64.rpm 16 MB/s | 320 kB 00:00 2023-11-28T02:50:22.367 INFO:teuthology.orchestra.run.smithi073.stdout:(76/120): python3-werkzeug-0.12.2-4.el8.noarch. 20 MB/s | 457 kB 00:00 2023-11-28T02:50:22.384 INFO:teuthology.orchestra.run.smithi073.stdout:(77/120): python3-markupsafe-0.23-19.el8.x86_64 2.3 MB/s | 39 kB 00:00 2023-11-28T02:50:22.410 INFO:teuthology.orchestra.run.smithi073.stdout:(78/120): bison-3.0.4-10.el8.x86_64.rpm 27 MB/s | 688 kB 00:00 2023-11-28T02:50:22.446 INFO:teuthology.orchestra.run.smithi073.stdout:(79/120): python3-babel-2.5.1-5.el8.noarch.rpm 56 MB/s | 4.8 MB 00:00 2023-11-28T02:50:22.448 INFO:teuthology.orchestra.run.smithi073.stdout:(80/120): lttng-ust-2.8.1-11.el8.x86_64.rpm 6.6 MB/s | 259 kB 00:00 2023-11-28T02:50:22.463 INFO:teuthology.orchestra.run.smithi073.stdout:(81/120): python3-numpy-f2py-1.14.3-9.el8.x86_6 13 MB/s | 225 kB 00:00 2023-11-28T02:50:22.465 INFO:teuthology.orchestra.run.smithi073.stdout:(82/120): python3-pyasn1-0.3.7-6.el8.noarch.rpm 7.3 MB/s | 126 kB 00:00 2023-11-28T02:50:22.487 INFO:teuthology.orchestra.run.smithi073.stdout:(83/120): oniguruma-6.8.2-2.el8.x86_64.rpm 8.6 MB/s | 187 kB 00:00 2023-11-28T02:50:22.603 INFO:teuthology.orchestra.run.smithi073.stdout:(84/120): python3-scipy-1.0.0-20.module+el8.1.0 46 MB/s | 15 MB 00:00 2023-11-28T02:50:22.610 INFO:teuthology.orchestra.run.smithi073.stdout:(85/120): python3-jinja2-2.10.1-2.el8_0.noarch. 4.3 MB/s | 538 kB 00:00 2023-11-28T02:50:22.620 INFO:teuthology.orchestra.run.smithi073.stdout:(86/120): socat-1.7.3.3-2.el8.x86_64.rpm 17 MB/s | 302 kB 00:00 2023-11-28T02:50:22.625 INFO:teuthology.orchestra.run.smithi073.stdout:(87/120): python3-mako-1.0.6-13.el8.noarch.rpm 11 MB/s | 157 kB 00:00 2023-11-28T02:50:22.640 INFO:teuthology.orchestra.run.smithi073.stdout:(88/120): python3-numpy-1.14.3-9.el8.x86_64.rpm 21 MB/s | 3.7 MB 00:00 2023-11-28T02:50:22.642 INFO:teuthology.orchestra.run.smithi073.stdout:(89/120): python3-pyasn1-modules-0.3.7-6.el8.no 5.0 MB/s | 110 kB 00:00 2023-11-28T02:50:22.645 INFO:teuthology.orchestra.run.smithi073.stdout:(90/120): librdkafka-0.11.4-1.el8.x86_64.rpm 18 MB/s | 353 kB 00:00 2023-11-28T02:50:22.653 INFO:teuthology.orchestra.run.smithi073.stdout:(91/120): python3-jsonpointer-1.10-11.el8.noarc 1.6 MB/s | 20 kB 00:00 2023-11-28T02:50:22.659 INFO:teuthology.orchestra.run.smithi073.stdout:(92/120): python3-pytz-2017.2-9.el8.noarch.rpm 3.9 MB/s | 54 kB 00:00 2023-11-28T02:50:22.664 INFO:teuthology.orchestra.run.smithi073.stdout:(93/120): libtool-2.4.6-25.el8.x86_64.rpm 31 MB/s | 709 kB 00:00 2023-11-28T02:50:22.691 INFO:teuthology.orchestra.run.smithi073.stdout:(94/120): automake-1.16.1-7.el8.noarch.rpm 27 MB/s | 713 kB 00:00 2023-11-28T02:50:22.703 INFO:teuthology.orchestra.run.smithi073.stdout:(95/120): libaio-devel-0.3.112-1.el8.x86_64.rpm 1.5 MB/s | 19 kB 00:00 2023-11-28T02:50:22.717 INFO:teuthology.orchestra.run.smithi073.stdout:(96/120): python3-jwt-1.6.1-2.el8.noarch.rpm 3.1 MB/s | 43 kB 00:00 2023-11-28T02:50:22.733 INFO:teuthology.orchestra.run.smithi073.stdout:(97/120): openblas-threads-0.3.12-1.el8.x86_64. 59 MB/s | 4.7 MB 00:00 2023-11-28T02:50:22.735 INFO:teuthology.orchestra.run.smithi073.stdout:(98/120): python3-oauthlib-2.1.0-1.el8.noarch.r 8.0 MB/s | 155 kB 00:00 2023-11-28T02:50:22.751 INFO:teuthology.orchestra.run.smithi073.stdout:(99/120): openblas-0.3.12-1.el8.x86_64.rpm 50 MB/s | 4.6 MB 00:00 2023-11-28T02:50:22.752 INFO:teuthology.orchestra.run.smithi073.stdout:(100/120): keyutils-libs-devel-1.5.10-6.el8.x86 2.9 MB/s | 48 kB 00:00 2023-11-28T02:50:22.754 INFO:teuthology.orchestra.run.smithi073.stdout:(101/120): pcre2-utf32-10.32-2.el8.x86_64.rpm 10 MB/s | 220 kB 00:00 2023-11-28T02:50:22.765 INFO:teuthology.orchestra.run.smithi073.stdout:(102/120): python3-requests-oauthlib-1.0.0-1.el 3.2 MB/s | 43 kB 00:00 2023-11-28T02:50:22.769 INFO:teuthology.orchestra.run.smithi073.stdout:(103/120): pcre2-utf16-10.32-2.el8.x86_64.rpm 12 MB/s | 229 kB 00:00 2023-11-28T02:50:22.779 INFO:teuthology.orchestra.run.smithi073.stdout:(104/120): libverto-devel-0.3.0-5.el8.x86_64.rp 1.3 MB/s | 18 kB 00:00 2023-11-28T02:50:22.791 INFO:teuthology.orchestra.run.smithi073.stdout:(105/120): pcre2-devel-10.32-2.el8.x86_64.rpm 28 MB/s | 605 kB 00:00 2023-11-28T02:50:22.793 INFO:teuthology.orchestra.run.smithi073.stdout:(106/120): fuse-2.9.7-12.el8.x86_64.rpm 6.1 MB/s | 83 kB 00:00 2023-11-28T02:50:22.804 INFO:teuthology.orchestra.run.smithi073.stdout:(107/120): openssl-devel-1.1.1g-15.el8_3.x86_64 47 MB/s | 2.3 MB 00:00 2023-11-28T02:50:22.805 INFO:teuthology.orchestra.run.smithi073.stdout:(108/120): elfutils-libelf-devel-0.182-3.el8.x8 4.2 MB/s | 59 kB 00:00 2023-11-28T02:50:22.807 INFO:teuthology.orchestra.run.smithi073.stdout:(109/120): libquadmath-8.4.1-1.el8.x86_64.rpm 12 MB/s | 169 kB 00:00 2023-11-28T02:50:22.819 INFO:teuthology.orchestra.run.smithi073.stdout:(110/120): libsepol-devel-2.9-2.el8.x86_64.rpm 6.0 MB/s | 86 kB 00:00 2023-11-28T02:50:22.819 INFO:teuthology.orchestra.run.smithi073.stdout:(111/120): libcom_err-devel-1.45.6-1.el8.x86_64 2.4 MB/s | 38 kB 00:00 2023-11-28T02:50:22.828 INFO:teuthology.orchestra.run.smithi073.stdout:(112/120): zlib-devel-1.2.11-17.el8.x86_64.rpm 2.7 MB/s | 58 kB 00:00 2023-11-28T02:50:22.833 INFO:teuthology.orchestra.run.smithi073.stdout:(113/120): librabbitmq-0.9.0-3.el8.x86_64.rpm 3.2 MB/s | 47 kB 00:00 2023-11-28T02:50:22.840 INFO:teuthology.orchestra.run.smithi073.stdout:(114/120): krb5-devel-1.18.2-8.3.el8_4.x86_64.r 29 MB/s | 559 kB 00:00 2023-11-28T02:50:22.848 INFO:teuthology.orchestra.run.smithi073.stdout:(115/120): libgfortran-8.4.1-1.el8.x86_64.rpm 32 MB/s | 639 kB 00:00 2023-11-28T02:50:22.849 INFO:teuthology.orchestra.run.smithi073.stdout:(116/120): xfsprogs-devel-5.0.0-8.el8.x86_64.rp 4.4 MB/s | 70 kB 00:00 2023-11-28T02:50:22.854 INFO:teuthology.orchestra.run.smithi073.stdout:(117/120): libselinux-devel-2.9-5.el8.x86_64.rp 13 MB/s | 200 kB 00:00 2023-11-28T02:50:22.861 INFO:teuthology.orchestra.run.smithi073.stdout:(118/120): libuuid-devel-2.32.1-27.el8.x86_64.r 7.3 MB/s | 97 kB 00:00 2023-11-28T02:50:22.990 INFO:teuthology.orchestra.run.smithi150.stdout:Transaction test succeeded. 2023-11-28T02:50:22.994 INFO:teuthology.orchestra.run.smithi150.stdout:Running transaction 2023-11-28T02:50:23.274 INFO:teuthology.orchestra.run.smithi073.stdout:(119/120): librbd1-16.2.14-270.g5474366c.el8.x8 9.9 MB/s | 4.2 MB 00:00 2023-11-28T02:50:23.399 INFO:teuthology.orchestra.run.smithi073.stdout:(120/120): librados2-16.2.14-270.g5474366c.el8. 6.8 MB/s | 3.7 MB 00:00 2023-11-28T02:50:23.404 INFO:teuthology.orchestra.run.smithi073.stdout:-------------------------------------------------------------------------------- 2023-11-28T02:50:23.404 INFO:teuthology.orchestra.run.smithi073.stdout:Total 19 MB/s | 201 MB 00:10 2023-11-28T02:50:23.451 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction check 2023-11-28T02:50:23.562 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction check succeeded. 2023-11-28T02:50:23.563 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction test 2023-11-28T02:50:25.335 INFO:teuthology.orchestra.run.smithi073.stdout:Transaction test succeeded. 2023-11-28T02:50:25.338 INFO:teuthology.orchestra.run.smithi073.stdout:Running transaction 2023-11-28T02:50:27.994 INFO:teuthology.orchestra.run.smithi150.stdout: Preparing : 1/1 2023-11-28T02:50:28.236 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-11-28T02:50:28.510 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/122 2023-11-28T02:50:28.723 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 3/122 2023-11-28T02:50:28.855 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 3/122 2023-11-28T02:50:29.073 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-11-28T02:50:29.539 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 4/122 2023-11-28T02:50:29.981 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librados2-2:16.2.14-270.g5474366c.el8.x86_64 5/122 2023-11-28T02:50:30.425 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-2:16.2.14-270.g5474366c.el8.x86_64 5/122 2023-11-28T02:50:30.532 INFO:teuthology.orchestra.run.smithi073.stdout: Preparing : 1/1 2023-11-28T02:50:30.639 INFO:teuthology.orchestra.run.smithi150.stdout: Upgrading : librbd1-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:50:30.767 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:50:30.778 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : liboath-2.6.2-3.el8.x86_64 1/122 2023-11-28T02:50:30.959 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : leveldb-1.22-1.el8.x86_64 2/122 2023-11-28T02:50:31.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:50:31.104 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:50:31.178 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 3/122 2023-11-28T02:50:31.303 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 8/122 2023-11-28T02:50:31.353 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librdkafka-0.11.4-1.el8.x86_64 4/122 2023-11-28T02:50:31.515 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librdkafka-0.11.4-1.el8.x86_64 4/122 2023-11-28T02:50:31.703 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 5/122 2023-11-28T02:50:31.803 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librabbitmq-0.9.0-3.el8.x86_64 9/122 2023-11-28T02:50:31.998 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librgw2-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:50:32.094 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librgw2-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:50:32.123 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 5/122 2023-11-28T02:50:32.312 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librados2-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:50:32.351 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-11-28T02:50:32.538 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-11-28T02:50:32.727 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:50:32.748 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:50:32.832 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:50:32.903 INFO:teuthology.orchestra.run.smithi073.stdout: Upgrading : librbd1-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:50:33.006 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 14/122 2023-11-28T02:50:33.023 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:50:33.065 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 14/122 2023-11-28T02:50:33.245 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 8/122 2023-11-28T02:50:33.268 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 15/122 2023-11-28T02:50:33.328 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 8/122 2023-11-28T02:50:33.562 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 16/122 2023-11-28T02:50:33.804 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 17/122 2023-11-28T02:50:33.867 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 9/122 2023-11-28T02:50:33.954 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 18/122 2023-11-28T02:50:34.013 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librgw2-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:50:34.123 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 19/122 2023-11-28T02:50:34.267 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 19/122 2023-11-28T02:50:34.431 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 20/122 2023-11-28T02:50:34.628 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librgw2-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:50:34.901 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 11/122 2023-11-28T02:50:35.013 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 20/122 2023-11-28T02:50:35.057 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 12/122 2023-11-28T02:50:35.253 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:50:35.366 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:50:35.545 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 14/122 2023-11-28T02:50:35.556 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-11-28T02:50:35.597 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 14/122 2023-11-28T02:50:35.740 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : zlib-devel-1.2.11-17.el8.x86_64 15/122 2023-11-28T02:50:35.920 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libquadmath-8.4.1-1.el8.x86_64 16/122 2023-11-28T02:50:36.041 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libquadmath-8.4.1-1.el8.x86_64 16/122 2023-11-28T02:50:36.115 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 21/122 2023-11-28T02:50:36.236 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libgfortran-8.4.1-1.el8.x86_64 17/122 2023-11-28T02:50:36.306 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openblas-0.3.12-1.el8.x86_64 22/122 2023-11-28T02:50:36.782 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libgfortran-8.4.1-1.el8.x86_64 17/122 2023-11-28T02:50:36.969 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-11-28T02:50:37.044 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 22/122 2023-11-28T02:50:37.344 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 23/122 2023-11-28T02:50:37.507 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openblas-threads-0.3.12-1.el8.x86_64 18/122 2023-11-28T02:50:37.623 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 24/122 2023-11-28T02:50:37.695 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openblas-0.3.12-1.el8.x86_64 19/122 2023-11-28T02:50:37.870 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-11-28T02:50:38.062 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-common-2:16.2.14-270.g5474366c.el8. 26/122 2023-11-28T02:50:38.195 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-ceph-argparse-2:16.2.14-270.g5474366c.el 27/122 2023-11-28T02:50:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: openblas-0.3.12-1.el8.x86_64 19/122 2023-11-28T02:50:38.472 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_6 28/122 2023-11-28T02:50:38.673 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-1:1.14.3-9.el8.x86_64 20/122 2023-11-28T02:50:38.694 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 29/122 2023-11-28T02:50:38.900 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 21/122 2023-11-28T02:50:38.996 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 30/122 2023-11-28T02:50:39.064 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pytz-2017.2-9.el8.noarch 22/122 2023-11-28T02:50:39.239 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-11-28T02:50:39.332 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 23/122 2023-11-28T02:50:39.628 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 24/122 2023-11-28T02:50:40.327 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 25/122 2023-11-28T02:50:40.466 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-common-2:16.2.14-270.g5474366c.el8. 26/122 2023-11-28T02:50:40.616 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-ceph-argparse-2:16.2.14-270.g5474366c.el 27/122 2023-11-28T02:50:40.833 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_6 28/122 2023-11-28T02:50:41.314 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 31/122 2023-11-28T02:50:41.678 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 32/122 2023-11-28T02:50:41.850 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 29/122 2023-11-28T02:50:42.181 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 30/122 2023-11-28T02:50:42.484 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 31/122 2023-11-28T02:50:42.728 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-mako-1.0.6-13.el8.noarch 33/122 2023-11-28T02:50:42.732 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 32/122 2023-11-28T02:50:43.025 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 33/122 2023-11-28T02:50:43.085 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-babel-2.5.1-5.el8.noarch 34/122 2023-11-28T02:50:43.260 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-11-28T02:50:43.391 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jinja2-2.10.1-2.el8_0.noarch 35/122 2023-11-28T02:50:43.644 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-11-28T02:50:43.874 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-11-28T02:50:44.089 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 38/122 2023-11-28T02:50:44.309 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 39/122 2023-11-28T02:50:44.599 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : librados-devel-2:16.2.14-270.g5474366c.el8.x86_6 40/122 2023-11-28T02:50:44.837 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : automake-1.16.1-7.el8.noarch 41/122 2023-11-28T02:50:45.045 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 42/122 2023-11-28T02:50:45.235 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 43/122 2023-11-28T02:50:45.299 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 34/122 2023-11-28T02:50:45.450 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 44/122 2023-11-28T02:50:45.646 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 45/122 2023-11-28T02:50:45.655 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 35/122 2023-11-28T02:50:45.721 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 45/122 2023-11-28T02:50:45.873 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rsa-4.9-2.el8.noarch 36/122 2023-11-28T02:50:45.921 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : jq-1.5-12.el8.x86_64 46/122 2023-11-28T02:50:46.100 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/122 2023-11-28T02:50:46.114 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 47/122 2023-11-28T02:50:46.310 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 38/122 2023-11-28T02:50:46.480 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 39/122 2023-11-28T02:50:46.677 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : librados-devel-2:16.2.14-270.g5474366c.el8.x86_6 40/122 2023-11-28T02:50:46.681 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 48/122 2023-11-28T02:50:46.842 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libuuid-devel-2.32.1-27.el8.x86_64 41/122 2023-11-28T02:50:46.843 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 49/122 2023-11-28T02:50:47.006 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libsepol-devel-2.9-2.el8.x86_64 42/122 2023-11-28T02:50:47.048 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 50/122 2023-11-28T02:50:47.144 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcom_err-devel-1.45.6-1.el8.x86_64 43/122 2023-11-28T02:50:47.249 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : fuse-2.9.7-12.el8.x86_64 51/122 2023-11-28T02:50:47.320 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : fuse-2.9.7-12.el8.x86_64 44/122 2023-11-28T02:50:47.413 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 52/122 2023-11-28T02:50:47.455 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 45/122 2023-11-28T02:50:47.626 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 53/122 2023-11-28T02:50:47.657 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 46/122 2023-11-28T02:50:47.820 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 47/122 2023-11-28T02:50:47.825 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : keyutils-libs-devel-1.5.10-6.el8.x86_64 54/122 2023-11-28T02:50:48.147 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 48/122 2023-11-28T02:50:48.159 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 55/122 2023-11-28T02:50:48.415 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 49/122 2023-11-28T02:50:48.468 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 56/122 2023-11-28T02:50:48.675 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 50/122 2023-11-28T02:50:48.740 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 57/122 2023-11-28T02:50:49.001 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 51/122 2023-11-28T02:50:49.058 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : krb5-devel-1.18.2-8.3.el8_4.x86_64 58/122 2023-11-28T02:50:49.242 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-11-28T02:50:49.261 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : automake-1.16.1-7.el8.noarch 52/122 2023-11-28T02:50:49.450 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 53/122 2023-11-28T02:50:49.459 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-11-28T02:50:49.654 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 54/122 2023-11-28T02:50:49.758 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-11-28T02:50:49.831 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : socat-1.7.3.3-2.el8.x86_64 55/122 2023-11-28T02:50:49.992 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 56/122 2023-11-28T02:50:50.038 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-11-28T02:50:50.076 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 56/122 2023-11-28T02:50:50.257 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-11-28T02:50:50.276 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : jq-1.5-12.el8.x86_64 57/122 2023-11-28T02:50:50.434 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 58/122 2023-11-28T02:50:50.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-11-28T02:50:50.649 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-11-28T02:50:50.670 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 59/122 2023-11-28T02:50:50.832 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 60/122 2023-11-28T02:50:50.841 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-11-28T02:50:51.075 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-11-28T02:50:51.111 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 61/122 2023-11-28T02:50:51.291 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 68/122 2023-11-28T02:50:51.368 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 62/122 2023-11-28T02:50:51.461 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 69/122 2023-11-28T02:50:51.556 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 63/122 2023-11-28T02:50:51.716 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 70/122 2023-11-28T02:50:51.823 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 64/122 2023-11-28T02:50:51.993 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 71/122 2023-11-28T02:50:52.005 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 65/122 2023-11-28T02:50:52.188 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 66/122 2023-11-28T02:50:52.238 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 72/122 2023-11-28T02:50:52.432 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 67/122 2023-11-28T02:50:52.561 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-portend-2.6-1.el8.noarch 73/122 2023-11-28T02:50:52.656 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 68/122 2023-11-28T02:50:52.816 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 69/122 2023-11-28T02:50:52.845 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 74/122 2023-11-28T02:50:53.044 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 70/122 2023-11-28T02:50:53.080 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 75/122 2023-11-28T02:50:53.221 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 71/122 2023-11-28T02:50:53.362 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 76/122 2023-11-28T02:50:53.456 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 72/122 2023-11-28T02:50:53.772 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-portend-2.6-1.el8.noarch 73/122 2023-11-28T02:50:54.025 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 74/122 2023-11-28T02:50:54.224 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 75/122 2023-11-28T02:50:54.492 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 76/122 2023-11-28T02:50:54.551 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 77/122 2023-11-28T02:50:54.867 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 78/122 2023-11-28T02:50:55.141 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 79/122 2023-11-28T02:50:55.471 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 80/122 2023-11-28T02:50:55.646 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 77/122 2023-11-28T02:50:55.761 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 81/122 2023-11-28T02:50:55.965 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 78/122 2023-11-28T02:50:56.010 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/122 2023-11-28T02:50:56.219 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 79/122 2023-11-28T02:50:56.276 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 83/122 2023-11-28T02:50:56.542 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 80/122 2023-11-28T02:50:56.567 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 84/122 2023-11-28T02:50:56.804 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 85/122 2023-11-28T02:50:56.818 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 81/122 2023-11-28T02:50:56.998 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/122 2023-11-28T02:50:57.030 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 86/122 2023-11-28T02:50:57.262 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 87/122 2023-11-28T02:50:57.266 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-apipkg-1.5-6.el8.noarch 83/122 2023-11-28T02:50:57.548 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-execnet-1.7.1-1.el8.noarch 84/122 2023-11-28T02:50:57.771 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : python3-remoto-1.2.1-1.el8.noarch 85/122 2023-11-28T02:50:57.950 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 86/122 2023-11-28T02:50:58.175 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 87/122 2023-11-28T02:51:02.384 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:02.837 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:03.176 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:03.408 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:03.868 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 89/122 2023-11-28T02:51:03.956 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 89/122 2023-11-28T02:51:04.127 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 90/122 2023-11-28T02:51:04.127 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 88/122 2023-11-28T02:51:04.389 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 89/122 2023-11-28T02:51:04.473 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 89/122 2023-11-28T02:51:04.607 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 90/122 2023-11-28T02:51:22.210 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 90/122 2023-11-28T02:51:22.211 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /sys 2023-11-28T02:51:22.212 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /proc 2023-11-28T02:51:22.212 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /dev 2023-11-28T02:51:22.213 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /run 2023-11-28T02:51:22.214 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /mnt 2023-11-28T02:51:22.216 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /var/tmp 2023-11-28T02:51:22.217 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /home 2023-11-28T02:51:22.217 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /tmp 2023-11-28T02:51:22.217 INFO:teuthology.orchestra.run.smithi073.stdout:skipping the directory /dev 2023-11-28T02:51:22.218 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 90/122 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /sys 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /proc 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /run 2023-11-28T02:51:22.387 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /mnt 2023-11-28T02:51:22.388 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /var/tmp 2023-11-28T02:51:22.388 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /home 2023-11-28T02:51:22.388 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /tmp 2023-11-28T02:51:22.388 INFO:teuthology.orchestra.run.smithi150.stdout:skipping the directory /dev 2023-11-28T02:51:22.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:22.604 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 91/122 2023-11-28T02:51:22.817 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 91/122 2023-11-28T02:51:23.030 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 91/122 2023-11-28T02:51:23.213 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 92/122 2023-11-28T02:51:23.242 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 91/122 2023-11-28T02:51:23.416 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 92/122 2023-11-28T02:51:24.713 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 92/122 2023-11-28T02:51:24.904 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 93/122 2023-11-28T02:51:24.925 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 93/122 2023-11-28T02:51:24.929 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 92/122 2023-11-28T02:51:25.125 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 93/122 2023-11-28T02:51:25.146 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 93/122 2023-11-28T02:51:27.770 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephadm-2:16.2.14-270.g5474366c.el8.noarch 94/122 2023-11-28T02:51:27.919 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephadm-2:16.2.14-270.g5474366c.el8.noarch 94/122 2023-11-28T02:51:28.066 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephadm-2:16.2.14-270.g5474366c.el8.noarch 94/122 2023-11-28T02:51:28.094 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-270.g5474366c.e 95/122 2023-11-28T02:51:28.251 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephadm-2:16.2.14-270.g5474366c.el8.noarch 94/122 2023-11-28T02:51:28.297 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-270.g5474366c. 96/122 2023-11-28T02:51:28.466 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-prometheus-alerts-2:16.2.14-270.g5474366c.e 95/122 2023-11-28T02:51:28.495 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 97/122 2023-11-28T02:51:28.652 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-grafana-dashboards-2:16.2.14-270.g5474366c. 96/122 2023-11-28T02:51:28.849 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 97/122 2023-11-28T02:51:29.449 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 97/122 2023-11-28T02:51:29.692 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 98/122 2023-11-28T02:51:29.805 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 97/122 2023-11-28T02:51:30.081 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 98/122 2023-11-28T02:51:30.737 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 98/122 2023-11-28T02:51:30.944 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-270.g547 99/122 2023-11-28T02:51:31.040 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-270.g547 99/122 2023-11-28T02:51:31.126 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 98/122 2023-11-28T02:51:31.306 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-diskprediction-local-2:16.2.14-270.g547 99/122 2023-11-28T02:51:31.402 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:16.2.14-270.g547 99/122 2023-11-28T02:51:31.409 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el 100/122 2023-11-28T02:51:31.599 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 101/122 2023-11-28T02:51:31.626 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 101/122 2023-11-28T02:51:31.778 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 102/122 2023-11-28T02:51:31.788 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el 100/122 2023-11-28T02:51:31.789 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 102/122 2023-11-28T02:51:31.989 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 101/122 2023-11-28T02:51:32.015 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 101/122 2023-11-28T02:51:32.190 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 102/122 2023-11-28T02:51:32.201 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 102/122 2023-11-28T02:51:32.893 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-2:16.2.14-270.g5474366c.el8.x86_64 103/122 2023-11-28T02:51:33.066 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 104/122 2023-11-28T02:51:33.180 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 104/122 2023-11-28T02:51:33.294 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-2:16.2.14-270.g5474366c.el8.x86_64 103/122 2023-11-28T02:51:33.311 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-270.g54743 105/122 2023-11-28T02:51:33.353 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-270.g54743 105/122 2023-11-28T02:51:33.511 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 106/122 2023-11-28T02:51:33.513 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 104/122 2023-11-28T02:51:33.593 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 104/122 2023-11-28T02:51:33.773 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-immutable-object-cache-2:16.2.14-270.g54743 105/122 2023-11-28T02:51:33.815 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: ceph-immutable-object-cache-2:16.2.14-270.g54743 105/122 2023-11-28T02:51:33.855 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 106/122 2023-11-28T02:51:33.999 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 106/122 2023-11-28T02:51:34.003 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 107/122 2023-11-28T02:51:34.346 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 106/122 2023-11-28T02:51:34.540 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 107/122 2023-11-28T02:51:38.568 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 107/122 2023-11-28T02:51:38.885 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 108/122 2023-11-28T02:51:39.043 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libtool-2.4.6-25.el8.x86_64 109/122 2023-11-28T02:51:39.118 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 107/122 2023-11-28T02:51:39.749 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 109/122 2023-11-28T02:51:40.068 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 110/122 2023-11-28T02:51:40.084 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 108/122 2023-11-28T02:51:40.273 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 111/122 2023-11-28T02:51:40.456 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : openssl-devel-1:1.1.1g-15.el8_3.x86_64 109/122 2023-11-28T02:51:40.472 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 112/122 2023-11-28T02:51:40.589 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_ 113/122 2023-11-28T02:51:40.693 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 110/122 2023-11-28T02:51:40.740 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-11-28T02:51:40.885 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 115/122 2023-11-28T02:51:40.995 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : xfsprogs-devel-5.0.0-8.el8.x86_64 111/122 2023-11-28T02:51:41.092 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 116/122 2023-11-28T02:51:41.172 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libtool-2.4.6-25.el8.x86_64 112/122 2023-11-28T02:51:41.186 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 112/122 2023-11-28T02:51:41.236 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 117/122 2023-11-28T02:51:41.334 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_ 113/122 2023-11-28T02:51:41.457 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 118/122 2023-11-28T02:51:41.502 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : elfutils-libelf-devel-0.182-3.el8.x86_64 114/122 2023-11-28T02:51:41.626 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : bison-3.0.4-10.el8.x86_64 119/122 2023-11-28T02:51:41.639 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 115/122 2023-11-28T02:51:41.684 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 119/122 2023-11-28T02:51:41.847 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 116/122 2023-11-28T02:51:41.893 INFO:teuthology.orchestra.run.smithi073.stdout: Installing : flex-2.6.1-9.el8.x86_64 120/122 2023-11-28T02:51:41.902 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 120/122 2023-11-28T02:51:41.903 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-11-28T02:51:42.115 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 117/122 2023-11-28T02:51:42.121 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-11-28T02:51:42.121 INFO:teuthology.orchestra.run.smithi073.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:42.338 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : bison-3.0.4-10.el8.x86_64 118/122 2023-11-28T02:51:42.395 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 118/122 2023-11-28T02:51:42.597 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : flex-2.6.1-9.el8.x86_64 119/122 2023-11-28T02:51:42.606 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 119/122 2023-11-28T02:51:42.783 INFO:teuthology.orchestra.run.smithi150.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 120/122 2023-11-28T02:51:42.783 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-11-28T02:51:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 121/122 2023-11-28T02:51:43.008 INFO:teuthology.orchestra.run.smithi150.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-2:16.2.14-270.g5474366c.el8.x86_64 1/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 2/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 3/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 4/122 2023-11-28T02:51:43.054 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-270.g54743 5/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 8/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 9/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 11/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 12/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_ 14/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 15/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 16/122 2023-11-28T02:51:43.055 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados-devel-2:16.2.14-270.g5474366c.el8.x86_6 17/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 18/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librgw2-2:16.2.14-270.g5474366c.el8.x86_64 19/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-argparse-2:16.2.14-270.g5474366c.el 20/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-ceph-common-2:16.2.14-270.g5474366c.el8. 21/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_6 22/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 23/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 24/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 25/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 26/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 27/122 2023-11-28T02:51:43.056 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 28/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-270.g5474366c. 29/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 30/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 31/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-270.g547 32/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el 33/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 34/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-270.g5474366c.e 35/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephadm-2:16.2.14-270.g5474366c.el8.noarch 36/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 37/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/122 2023-11-28T02:51:43.057 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/122 2023-11-28T02:51:43.058 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/122 2023-11-28T02:51:43.059 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/122 2023-11-28T02:51:43.060 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 72/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 74/122 2023-11-28T02:51:43.061 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : jq-1.5-12.el8.x86_64 75/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : flex-2.6.1-9.el8.x86_64 76/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 78/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 79/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : bison-3.0.4-10.el8.x86_64 80/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 81/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 82/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 83/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 84/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 85/122 2023-11-28T02:51:43.062 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 86/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 87/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 88/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 89/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 90/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 91/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 92/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 93/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 94/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 95/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : automake-1.16.1-7.el8.noarch 96/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 97/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 98/122 2023-11-28T02:51:43.063 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 99/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 100/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 101/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 102/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 103/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 104/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 105/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 106/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 107/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 108/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 109/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 110/122 2023-11-28T02:51:43.064 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 111/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 112/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 113/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 114/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 115/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 116/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 117/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 118/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-2:16.2.14-270.g5474366c.el8.x86_64 119/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-11-28T02:51:43.065 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-2:16.2.14-270.g5474366c.el8.x86_64 121/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-2:16.2.14-270.g5474366c.el8.x86_64 1/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 2/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 3/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 4/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-immutable-object-cache-2:16.2.14-270.g54743 5/122 2023-11-28T02:51:43.897 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 6/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 7/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 8/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 9/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 10/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 11/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 12/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 13/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_ 14/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 15/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 16/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados-devel-2:16.2.14-270.g5474366c.el8.x86_6 17/122 2023-11-28T02:51:43.898 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libradosstriper1-2:16.2.14-270.g5474366c.el8.x86 18/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librgw2-2:16.2.14-270.g5474366c.el8.x86_64 19/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-argparse-2:16.2.14-270.g5474366c.el 20/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-ceph-common-2:16.2.14-270.g5474366c.el8. 21/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_6 22/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 23/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 24/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 25/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 26/122 2023-11-28T02:51:43.899 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 27/122 2023-11-28T02:51:43.908 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 28/122 2023-11-28T02:51:43.908 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-grafana-dashboards-2:16.2.14-270.g5474366c. 29/122 2023-11-28T02:51:43.908 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noa 30/122 2023-11-28T02:51:43.908 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.n 31/122 2023-11-28T02:51:43.908 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-diskprediction-local-2:16.2.14-270.g547 32/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el 33/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 34/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : ceph-prometheus-alerts-2:16.2.14-270.g5474366c.e 35/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephadm-2:16.2.14-270.g5474366c.el8.noarch 36/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 37/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 38/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : leveldb-1.22-1.el8.x86_64 39/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-apipkg-1.5-6.el8.noarch 42/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/122 2023-11-28T02:51:43.909 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-execnet-1.7.1-1.el8.noarch 49/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 50/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 51/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 52/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 53/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 54/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 55/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/122 2023-11-28T02:51:43.910 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-remoto-1.2.1-1.el8.noarch 58/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 59/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 60/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 61/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 62/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 63/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 64/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 65/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 66/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 67/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 68/122 2023-11-28T02:51:43.911 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 69/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 70/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 72/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 75/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : keyutils-libs-devel-1.5.10-6.el8.x86_64 76/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 77/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 78/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openssl-devel-1:1.1.1g-15.el8_3.x86_64 79/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 80/122 2023-11-28T02:51:43.912 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 81/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : fuse-2.9.7-12.el8.x86_64 82/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : elfutils-libelf-devel-0.182-3.el8.x86_64 83/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libquadmath-8.4.1-1.el8.x86_64 84/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libcom_err-devel-1.45.6-1.el8.x86_64 85/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libsepol-devel-2.9-2.el8.x86_64 86/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : zlib-devel-1.2.11-17.el8.x86_64 87/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librabbitmq-0.9.0-3.el8.x86_64 88/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : krb5-devel-1.18.2-8.3.el8_4.x86_64 89/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libgfortran-8.4.1-1.el8.x86_64 90/122 2023-11-28T02:51:43.913 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : xfsprogs-devel-5.0.0-8.el8.x86_64 91/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 92/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libuuid-devel-2.32.1-27.el8.x86_64 93/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb62 94/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 95/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 96/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : jq-1.5-12.el8.x86_64 97/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : flex-2.6.1-9.el8.x86_64 98/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 99/122 2023-11-28T02:51:43.914 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-babel-2.5.1-5.el8.noarch 100/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 101/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : bison-3.0.4-10.el8.x86_64 102/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 103/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-f2py-1:1.14.3-9.el8.x86_64 104/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 105/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-numpy-1:1.14.3-9.el8.x86_64 106/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 107/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jinja2-2.10.1-2.el8_0.noarch 108/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : socat-1.7.3.3-2.el8.x86_64 109/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-mako-1.0.6-13.el8.noarch 110/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 111/122 2023-11-28T02:51:43.915 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librdkafka-0.11.4-1.el8.x86_64 112/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 113/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 114/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : python3-pytz-2017.2-9.el8.noarch 115/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-threads-0.3.12-1.el8.x86_64 116/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : openblas-0.3.12-1.el8.x86_64 117/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : automake-1.16.1-7.el8.noarch 118/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-2:16.2.14-270.g5474366c.el8.x86_64 119/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 120/122 2023-11-28T02:51:43.916 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-2:16.2.14-270.g5474366c.el8.x86_64 121/122 2023-11-28T02:51:45.278 INFO:teuthology.orchestra.run.smithi073.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:45.278 INFO:teuthology.orchestra.run.smithi073.stdout:Last metadata expiration check: 0:01:42 ago on Tue 28 Nov 2023 02:50:03 AM UTC. 2023-11-28T02:51:46.142 INFO:teuthology.orchestra.run.smithi150.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 122/122 2023-11-28T02:51:46.142 INFO:teuthology.orchestra.run.smithi150.stdout:Last metadata expiration check: 0:01:44 ago on Tue 28 Nov 2023 02:50:02 AM UTC. 2023-11-28T02:51:47.438 INFO:teuthology.orchestra.run.smithi073.stdout:Installed products updated. 2023-11-28T02:51:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:47.577 INFO:teuthology.orchestra.run.smithi073.stdout:Upgraded: 2023-11-28T02:51:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: librados2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.577 INFO:teuthology.orchestra.run.smithi073.stdout: librbd1-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.577 INFO:teuthology.orchestra.run.smithi073.stdout:Installed: 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-grafana-dashboards-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-immutable-object-cache-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.578 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-diskprediction-local-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-prometheus-alerts-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: cephadm-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.579 INFO:teuthology.orchestra.run.smithi073.stdout: cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: fuse-2.9.7-12.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: jq-1.5-12.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: leveldb-1.22-1.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.580 INFO:teuthology.orchestra.run.smithi073.stdout: libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: librados-devel-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libradosstriper1-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: librgw2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-11-28T02:51:47.581 INFO:teuthology.orchestra.run.smithi073.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-0.3.12-1.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T02:51:47.582 INFO:teuthology.orchestra.run.smithi073.stdout: python3-apipkg-1.5-6.el8.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-babel-2.5.1-5.el8.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-argparse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T02:51:47.583 INFO:teuthology.orchestra.run.smithi073.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T02:51:47.584 INFO:teuthology.orchestra.run.smithi073.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T02:51:47.585 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T02:51:47.586 INFO:teuthology.orchestra.run.smithi073.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-11-28T02:51:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T02:51:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T02:51:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T02:51:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T02:51:47.587 INFO:teuthology.orchestra.run.smithi073.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: socat-1.7.3.3-2.el8.x86_64 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-11-28T02:51:47.588 INFO:teuthology.orchestra.run.smithi073.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T02:51:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-11-28T02:51:47.589 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:47.589 INFO:teuthology.orchestra.run.smithi073.stdout:Complete! 2023-11-28T02:51:47.874 DEBUG:teuthology.parallel:result is None 2023-11-28T02:51:48.234 INFO:teuthology.orchestra.run.smithi150.stdout:Installed products updated. 2023-11-28T02:51:48.399 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:48.399 INFO:teuthology.orchestra.run.smithi150.stdout:Upgraded: 2023-11-28T02:51:48.399 INFO:teuthology.orchestra.run.smithi150.stdout: librados2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.399 INFO:teuthology.orchestra.run.smithi150.stdout: librbd1-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.399 INFO:teuthology.orchestra.run.smithi150.stdout:Installed: 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: automake-1.16.1-7.el8.noarch 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: bison-3.0.4-10.el8.x86_64 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-base-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.400 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-fuse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-grafana-dashboards-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-immutable-object-cache-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mds-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-cephadm-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-dashboard-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-diskprediction-local-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.401 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-modules-core-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.402 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mgr-rook-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.402 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-mon-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.402 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-osd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.402 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-prometheus-alerts-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.402 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-radosgw-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-selinux-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: ceph-test-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: cephadm-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-mirror-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: cephfs-top-2:16.2.14-270.g5474366c.el8.noarch 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: elfutils-libelf-devel-0.182-3.el8.x86_64 2023-11-28T02:51:48.403 INFO:teuthology.orchestra.run.smithi150.stdout: flex-2.6.1-9.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: fuse-2.9.7-12.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: jq-1.5-12.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: keyutils-libs-devel-1.5.10-6.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: krb5-devel-1.18.2-8.3.el8_4.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: leveldb-1.22-1.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-11-28T02:51:48.404 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs-devel-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: libcephfs2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: libcephsqlite-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: libcom_err-devel-1.45.6-1.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: libgfortran-8.4.1-1.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: liboath-2.6.2-3.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: libquadmath-8.4.1-1.el8.x86_64 2023-11-28T02:51:48.405 INFO:teuthology.orchestra.run.smithi150.stdout: librabbitmq-0.9.0-3.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: librados-devel-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: libradosstriper1-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: librdkafka-0.11.4-1.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: librgw2-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: libsepol-devel-2.9-2.el8.x86_64 2023-11-28T02:51:48.406 INFO:teuthology.orchestra.run.smithi150.stdout: libtool-2.4.6-25.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: libunwind-1.3.1-3.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: libuuid-devel-2.32.1-27.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-0.3.12-1.el8.x86_64 2023-11-28T02:51:48.407 INFO:teuthology.orchestra.run.smithi150.stdout: openblas-threads-0.3.12-1.el8.x86_64 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: openssl-devel-1:1.1.1g-15.el8_3.x86_64 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-apipkg-1.5-6.el8.noarch 2023-11-28T02:51:48.408 INFO:teuthology.orchestra.run.smithi150.stdout: python3-babel-2.5.1-5.el8.noarch 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-argparse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-ceph-common-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.409 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cephfs-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-execnet-1.7.1-1.el8.noarch 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-11-28T02:51:48.410 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-6.2-6.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jinja2-2.10.1-2.el8_0.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-11-28T02:51:48.411 INFO:teuthology.orchestra.run.smithi150.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-mako-1.0.6-13.el8.noarch 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-1:1.14.3-9.el8.x86_64 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-numpy-f2py-1:1.14.3-9.el8.x86_64 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-11-28T02:51:48.412 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-portend-2.6-1.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-pytz-2017.2-9.el8.noarch 2023-11-28T02:51:48.413 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rados-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rbd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-remoto-1.2.1-1.el8.noarch 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rgw-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.414 INFO:teuthology.orchestra.run.smithi150.stdout: python3-routes-2.4.1-12.el8.noarch 2023-11-28T02:51:48.415 INFO:teuthology.orchestra.run.smithi150.stdout: python3-rsa-4.9-2.el8.noarch 2023-11-28T02:51:48.415 INFO:teuthology.orchestra.run.smithi150.stdout: python3-scipy-1.0.0-20.module+el8.1.0+3334+5cb623d7.x86_64 2023-11-28T02:51:48.415 INFO:teuthology.orchestra.run.smithi150.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-11-28T02:51:48.415 INFO:teuthology.orchestra.run.smithi150.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-11-28T02:51:48.415 INFO:teuthology.orchestra.run.smithi150.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-11-28T02:51:48.416 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-fuse-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-mirror-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: rbd-nbd-2:16.2.14-270.g5474366c.el8.x86_64 2023-11-28T02:51:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: socat-1.7.3.3-2.el8.x86_64 2023-11-28T02:51:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: xfsprogs-devel-5.0.0-8.el8.x86_64 2023-11-28T02:51:48.417 INFO:teuthology.orchestra.run.smithi150.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-11-28T02:51:48.418 INFO:teuthology.orchestra.run.smithi150.stdout: zlib-devel-1.2.11-17.el8.x86_64 2023-11-28T02:51:48.418 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:48.418 INFO:teuthology.orchestra.run.smithi150.stdout:Complete! 2023-11-28T02:51:48.655 DEBUG:teuthology.parallel:result is None 2023-11-28T02:51:48.656 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5474366c84b0c469f7b5285fcfee86232293d8b4 2023-11-28T02:51:48.885 DEBUG:teuthology.orchestra.run.smithi073:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T02:51:48.910 INFO:teuthology.orchestra.run.smithi073.stdout:16.2.14-270.g5474366c.el8 2023-11-28T02:51:48.911 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-270.g5474366c.el8 2023-11-28T02:51:48.911 INFO:teuthology.task.install:The correct ceph version 16.2.14-270.g5474366c is installed. 2023-11-28T02:51:48.911 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=5474366c84b0c469f7b5285fcfee86232293d8b4 2023-11-28T02:51:49.134 DEBUG:teuthology.orchestra.run.smithi150:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-11-28T02:51:49.155 INFO:teuthology.orchestra.run.smithi150.stdout:16.2.14-270.g5474366c.el8 2023-11-28T02:51:49.155 INFO:teuthology.packaging:The installed version of ceph is 16.2.14-270.g5474366c.el8 2023-11-28T02:51:49.155 INFO:teuthology.task.install:The correct ceph version 16.2.14-270.g5474366c is installed. 2023-11-28T02:51:49.156 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-11-28T02:51:49.156 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:49.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T02:51:49.197 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:49.197 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-11-28T02:51:49.236 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-11-28T02:51:49.236 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:49.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T02:51:49.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T02:51:49.328 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:49.328 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/daemon-helper 2023-11-28T02:51:49.353 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-11-28T02:51:49.418 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-11-28T02:51:49.418 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:49.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T02:51:49.443 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T02:51:49.507 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:49.507 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/adjust-ulimits 2023-11-28T02:51:49.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-11-28T02:51:49.598 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-11-28T02:51:49.598 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:49.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T02:51:49.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T02:51:49.687 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:49.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/usr/bin/stdin-killer 2023-11-28T02:51:49.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-11-28T02:51:49.775 INFO:teuthology.run_tasks:Running task ceph... 2023-11-28T02:51:49.843 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-11-28T02:51:49.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 777 /var/log/ceph 2023-11-28T02:51:49.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 777 /var/log/ceph 2023-11-28T02:51:49.870 INFO:tasks.ceph:Disabling ceph logrotate... 2023-11-28T02:51:49.870 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T02:51:49.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/logrotate.d/ceph 2023-11-28T02:51:49.937 INFO:tasks.ceph:Creating extra log directories... 2023-11-28T02:51:49.938 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T02:51:49.977 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-11-28T02:51:50.008 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-11-28T02:51:50.008 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-11-28T02:51:50.008 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-11-28T02:51:50.008 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:50.008 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T02:51:50.053 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T02:51:50.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T02:51:50.194 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:50.194 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-11-28T02:51:50.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-11-28T02:51:50.320 DEBUG:teuthology.orchestra.run.smithi150:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-11-28T02:51:50.351 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-11-28T02:51:50.351 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-11-28T02:51:50.351 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703', 'branch': 'wip-yuri-testing-2023-11-27-1028-pacific', 'description': 'fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7468703', 'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'rhel', 'os_version': '8.4', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-11-27-1028-pacific'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-11-27-1028-pacific', 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 3153, 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-11-27-1028-pacific', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_5474366c84b0c469f7b5285fcfee86232293d8b4/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4', 'targets': {'smithi073.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrYwp82qOkNzBtVWZdiiw7/4vzuuaQgmcQg21uwlcmpDvvwZwnShDLiOf7e7CBPwpQcWBLI4+9CevKpbl/G7vQ=', 'smithi150.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOcsXlOEX6iheXxSe0SiQYyxcdk15GcmPxRXO6f5afn5qNlnTo44BVJqyXGhrJt5x77Pw+PUYTWTkbOBwmE5x6A='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '5474366c84b0c469f7b5285fcfee86232293d8b4'}}, {'ceph': None}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '2442239d2653456406c25ae0c71b689c8f2657b6', 'timestamp': '2023-11-27_22:36:25', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.232157'} 2023-11-28T02:51:50.353 DEBUG:teuthology.orchestra.run.smithi073:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T02:51:50.357 DEBUG:teuthology.orchestra.run.smithi150:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-11-28T02:51:50.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T02:51:50.411 DEBUG:teuthology.orchestra.run.smithi150:> sudo install -d -m0777 -- /var/run/ceph 2023-11-28T02:51:50.465 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:50.465 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T02:51:50.488 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T02:51:50.489 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_1 2023-11-28T02:51:50.543 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T02:51:50.543 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:50.543 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 219117 Links: 1 2023-11-28T02:51:50.543 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:50.543 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:50.544 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2023-11-28 02:51:42.610338004 +0000 2023-11-28T02:51:50.544 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2023-11-28 02:49:19.092113459 +0000 2023-11-28T02:51:50.544 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2023-11-28 02:49:19.092113459 +0000 2023-11-28T02:51:50.544 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2023-11-28T02:51:50.544 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T02:51:50.606 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2023-11-28T02:51:50.606 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2023-11-28T02:51:50.606 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000267493 s, 1.9 MB/s 2023-11-28T02:51:50.607 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T02:51:50.663 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_2 2023-11-28T02:51:50.718 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T02:51:50.718 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:50.718 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 217898 Links: 1 2023-11-28T02:51:50.718 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:50.718 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:50.719 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2023-11-28 02:51:42.611337980 +0000 2023-11-28T02:51:50.719 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2023-11-28 02:49:19.611101265 +0000 2023-11-28T02:51:50.719 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2023-11-28 02:49:19.611101265 +0000 2023-11-28T02:51:50.719 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2023-11-28T02:51:50.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T02:51:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2023-11-28T02:51:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2023-11-28T02:51:50.781 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000235747 s, 2.2 MB/s 2023-11-28T02:51:50.782 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T02:51:50.838 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_3 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 222359 Links: 1 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2023-11-28 02:51:42.611337980 +0000 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2023-11-28 02:49:20.145088719 +0000 2023-11-28T02:51:50.893 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2023-11-28 02:49:20.145088719 +0000 2023-11-28T02:51:50.894 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2023-11-28T02:51:50.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T02:51:50.956 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2023-11-28T02:51:50.956 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2023-11-28T02:51:50.956 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000234435 s, 2.2 MB/s 2023-11-28T02:51:50.957 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T02:51:51.014 DEBUG:teuthology.orchestra.run.smithi073:> stat /dev/vg_nvme/lv_4 2023-11-28T02:51:51.068 INFO:teuthology.orchestra.run.smithi073.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Device: 6h/6d Inode: 218057 Links: 1 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Access: 2023-11-28 02:51:42.611337980 +0000 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Modify: 2023-11-28 02:49:20.669076408 +0000 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout:Change: 2023-11-28 02:49:20.669076408 +0000 2023-11-28T02:51:51.069 INFO:teuthology.orchestra.run.smithi073.stdout: Birth: - 2023-11-28T02:51:51.069 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T02:51:51.132 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records in 2023-11-28T02:51:51.132 INFO:teuthology.orchestra.run.smithi073.stderr:1+0 records out 2023-11-28T02:51:51.132 INFO:teuthology.orchestra.run.smithi073.stderr:512 bytes copied, 0.000196744 s, 2.6 MB/s 2023-11-28T02:51:51.133 DEBUG:teuthology.orchestra.run.smithi073:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T02:51:51.188 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:51.189 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:51.189 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/scratch_devs of=/dev/stdout 2023-11-28T02:51:51.205 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-11-28T02:51:51.205 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_1 2023-11-28T02:51:51.261 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-11-28T02:51:51.261 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:51.261 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 218795 Links: 1 2023-11-28T02:51:51.261 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-28 02:51:43.441691337 +0000 2023-11-28T02:51:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-28 02:49:19.111120067 +0000 2023-11-28T02:51:51.262 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-28 02:49:19.111120067 +0000 2023-11-28T02:51:51.262 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-28T02:51:51.262 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-11-28T02:51:51.325 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-28T02:51:51.326 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-28T02:51:51.326 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000267293 s, 1.9 MB/s 2023-11-28T02:51:51.327 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-11-28T02:51:51.384 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_2 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 223365 Links: 1 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-28 02:51:43.441691337 +0000 2023-11-28T02:51:51.439 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-28 02:49:19.635109531 +0000 2023-11-28T02:51:51.440 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-28 02:49:19.635109531 +0000 2023-11-28T02:51:51.440 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-28T02:51:51.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-11-28T02:51:51.502 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-28T02:51:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-28T02:51:51.503 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000244723 s, 2.1 MB/s 2023-11-28T02:51:51.504 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-11-28T02:51:51.559 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_3 2023-11-28T02:51:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-11-28T02:51:51.616 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:51.616 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 218882 Links: 1 2023-11-28T02:51:51.616 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:51.617 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:51.617 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-28 02:51:43.441691337 +0000 2023-11-28T02:51:51.617 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-28 02:49:20.137099438 +0000 2023-11-28T02:51:51.617 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-28 02:49:20.137099438 +0000 2023-11-28T02:51:51.617 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-28T02:51:51.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-11-28T02:51:51.679 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-28T02:51:51.679 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-28T02:51:51.680 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000235182 s, 2.2 MB/s 2023-11-28T02:51:51.680 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-11-28T02:51:51.737 DEBUG:teuthology.orchestra.run.smithi150:> stat /dev/vg_nvme/lv_4 2023-11-28T02:51:51.793 INFO:teuthology.orchestra.run.smithi150.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-11-28T02:51:51.793 INFO:teuthology.orchestra.run.smithi150.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Device: 6h/6d Inode: 223580 Links: 1 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Context: system_u:object_r:device_t:s0 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Access: 2023-11-28 02:51:43.441691337 +0000 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Modify: 2023-11-28 02:49:20.654089043 +0000 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout:Change: 2023-11-28 02:49:20.654089043 +0000 2023-11-28T02:51:51.794 INFO:teuthology.orchestra.run.smithi150.stdout: Birth: - 2023-11-28T02:51:51.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-11-28T02:51:51.857 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records in 2023-11-28T02:51:51.857 INFO:teuthology.orchestra.run.smithi150.stderr:1+0 records out 2023-11-28T02:51:51.857 INFO:teuthology.orchestra.run.smithi150.stderr:512 bytes copied, 0.000197717 s, 2.6 MB/s 2023-11-28T02:51:51.858 DEBUG:teuthology.orchestra.run.smithi150:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-11-28T02:51:51.914 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:51.914 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T02:51:51.914 INFO:tasks.ceph:Generating config... 2023-11-28T02:51:51.915 INFO:tasks.ceph:[client] client mount timeout = 600 2023-11-28T02:51:51.915 INFO:tasks.ceph:[client] debug client = 20 2023-11-28T02:51:51.916 INFO:tasks.ceph:[client] debug ms = 1 2023-11-28T02:51:51.916 INFO:tasks.ceph:[client] pid file = 2023-11-28T02:51:51.916 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-11-28T02:51:51.916 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-11-28T02:51:51.916 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] debug mds = 20 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] debug ms = 1 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] mds debug frag = True 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-11-28T02:51:51.916 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mgr] debug client = 20 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mgr] debug ms = 1 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mon] debug mon = 20 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mon] debug ms = 1 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mon] debug paxos = 20 2023-11-28T02:51:51.917 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-11-28T02:51:51.917 INFO:tasks.ceph:[osd] bdev async discard = True 2023-11-28T02:51:51.917 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-11-28T02:51:51.917 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] debug ms = 1 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] debug osd = 20 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-11-28T02:51:51.918 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-11-28T02:51:51.919 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-11-28T02:51:51.919 INFO:tasks.ceph:Setting up mon.a... 2023-11-28T02:51:51.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-11-28T02:51:51.987 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.keyring 2023-11-28T02:51:51.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-11-28T02:51:52.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T02:51:52.093 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.73'), ('mon.b', '172.21.15.150'), ('mon.c', '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]')] 2023-11-28T02:51:52.093 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.73,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-11-28T02:51:52.094 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:52.094 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T02:51:52.148 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.73 --add b 172.21.15.150 --addv c '[v2:172.21.15.150:3301,v1:172.21.15.150:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:52.228 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: generated fsid c55698a5-e315-44c7-976c-effa5ffffb99 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:epoch 0 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:fsid c55698a5-e315-44c7-976c-effa5ffffb99 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:last_changed 2023-11-28T02:51:52.228570+0000 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:created 2023-11-28T02:51:52.228570+0000 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:min_mon_release 0 (unknown) 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:election_strategy: 1 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:0: [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] mon.a 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:1: [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] mon.b 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:2: [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] mon.c 2023-11-28T02:51:52.229 INFO:teuthology.orchestra.run.smithi073.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-11-28T02:51:52.231 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-11-28T02:51:52.285 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c55698a5-e315-44c7-976c-effa5ffffb99... 2023-11-28T02:51:52.286 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T02:51:52.327 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-11-28T02:51:52.366 INFO:teuthology.orchestra.run.smithi073.stdout:[global] 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: chdir = "" 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: auth supported = cephx 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: filestore xattr use omap = true 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: mon clock drift allowed = 1.000 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: osd crush chooseleaf type = 0 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: auth debug = true 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.367 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on old message = true 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on bug = true 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: mon pg warn max object skew = 0 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default size = 2 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow primary affinity = true 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd allow pg remap = true 2023-11-28T02:51:52.368 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on legacy crush tunables = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on crush straw calc version zero = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on no sortbitwise = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on osd down out interval zero = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon warn on too few osds = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: mon_allow_pool_size_one = true 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: osd default data pool replay window = 5 2023-11-28T02:51:52.369 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: mon allow pool delete = true 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: mon cluster log file level = debug 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: debug asserts on shutdown = true 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: mon health detail to clog = false 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: mon host = "172.21.15.73,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: ms die on skipped message = False 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: fsid = c55698a5-e315-44c7-976c-effa5ffffb99 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout:[osd] 2023-11-28T02:51:52.370 INFO:teuthology.orchestra.run.smithi073.stdout: osd journal size = 100 2023-11-28T02:51:52.371 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.371 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub load threshold = 5.0 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: osd scrub max interval = 600 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: osd recover clone overlap = true 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: osd recovery max chunk = 1048576 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug shutdown = true 2023-11-28T02:51:52.372 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug op order = true 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug verify stray on activate = true 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: osd open classes on start = true 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug pg log writeout = true 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: osd deep scrub update digest min age = 30 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: osd map max advance = 10 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: journal zero on create = true 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.373 INFO:teuthology.orchestra.run.smithi073.stdout: filestore ondisk finisher threads = 3 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: filestore apply finisher threads = 3 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: bdev debug aio = true 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: osd debug misdirected ops = true 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: bdev async discard = True 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: bdev enable discard = True 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore allocator = bitmap 2023-11-28T02:51:52.374 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore block size = 96636764160 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: bluestore fsck on mount = True 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluefs = 1/20 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: debug bluestore = 1/20 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: debug osd = 20 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: debug rocksdb = 4/10 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T02:51:52.375 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd full ratio = 0.9 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd nearfull ratio = 0.8 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: osd failsafe full ratio = 0.95 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: osd objectstore = bluestore 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr] 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: debug mgr = 20 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2023-11-28T02:51:52.376 INFO:teuthology.orchestra.run.smithi073.stdout: mgr/telemetry/nag = false 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug finisher = 20 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout:[mon] 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug mon = 20 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug paxos = 20 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: debug auth = 20 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: mon data avail warn = 5 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: mon mgr mkfs grace = 240 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min pgs per osd = 4 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd reporter subtree level = osd 2023-11-28T02:51:52.377 INFO:teuthology.orchestra.run.smithi073.stdout: mon osd prime pg temp = true 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: mon reweight min bytes per osd = 10 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: mon op complaint time = 120 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout:[client] 2023-11-28T02:51:52.378 INFO:teuthology.orchestra.run.smithi073.stdout: rgw cache enabled = true 2023-11-28T02:51:52.379 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable ops log = true 2023-11-28T02:51:52.379 INFO:teuthology.orchestra.run.smithi073.stdout: rgw enable usage log = true 2023-11-28T02:51:52.379 INFO:teuthology.orchestra.run.smithi073.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T02:51:52.379 INFO:teuthology.orchestra.run.smithi073.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: client mount timeout = 600 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: debug client = 20 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: pid file = "" 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.a] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.b] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mon.c] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2023-11-28T02:51:52.380 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout:[mds] 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds = 20 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: debug mds balancer = 20 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: debug ms = 1 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug frag = True 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: mds debug scatterstat = True 2023-11-28T02:51:52.381 INFO:teuthology.orchestra.run.smithi073.stdout: mds op complaint time = 180 2023-11-28T02:51:52.382 INFO:teuthology.orchestra.run.smithi073.stdout: mds verify scatter = True 2023-11-28T02:51:52.382 INFO:teuthology.orchestra.run.smithi073.stdout: osd op complaint time = 180 2023-11-28T02:51:52.382 INFO:teuthology.orchestra.run.smithi073.stdout: rados mon op timeout = 900 2023-11-28T02:51:52.382 INFO:teuthology.orchestra.run.smithi073.stdout: rados osd op timeout = 900 2023-11-28T02:51:52.383 INFO:teuthology.orchestra.run.smithi150.stdout:[global] 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: chdir = "" 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: auth supported = cephx 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: filestore xattr use omap = true 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: mon clock drift allowed = 1.000 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: osd crush chooseleaf type = 0 2023-11-28T02:51:52.384 INFO:teuthology.orchestra.run.smithi150.stdout: auth debug = true 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on old message = true 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on bug = true 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: mon max pg per osd = 10000 # >= luminous 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: mon pg warn max object skew = 0 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.385 INFO:teuthology.orchestra.run.smithi150.stdout: # disable pg_autoscaler by default for new pools 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: osd_pool_default_pg_autoscale_mode = off 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default size = 2 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow primary affinity = true 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd allow pg remap = true 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on legacy crush tunables = false 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on crush straw calc version zero = false 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on no sortbitwise = false 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on osd down out interval zero = false 2023-11-28T02:51:52.386 INFO:teuthology.orchestra.run.smithi150.stdout: mon warn on too few osds = false 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_pool_no_redundancy = false 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: mon_allow_pool_size_one = true 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 ruleset-failure-domain=osd crush-failure-domain=osd 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: osd default data pool replay window = 5 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: mon allow pool delete = true 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: mon cluster log file level = debug 2023-11-28T02:51:52.387 INFO:teuthology.orchestra.run.smithi150.stdout: debug asserts on shutdown = true 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: mon health detail to clog = false 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: mon host = "172.21.15.73,172.21.15.150,[v2:172.21.15.150:3301,v1:172.21.15.150:6790]" 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: ms die on skipped message = False 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: fsid = c55698a5-e315-44c7-976c-effa5ffffb99 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout:[osd] 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd journal size = 100 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub load threshold = 5.0 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd scrub max interval = 600 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.388 INFO:teuthology.orchestra.run.smithi150.stdout: osd recover clone overlap = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd recovery max chunk = 1048576 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug shutdown = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug op order = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug verify stray on activate = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd open classes on start = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug pg log writeout = true 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd deep scrub update digest min age = 30 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.389 INFO:teuthology.orchestra.run.smithi150.stdout: osd map max advance = 10 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: journal zero on create = true 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: filestore ondisk finisher threads = 3 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: filestore apply finisher threads = 3 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: bdev debug aio = true 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: osd debug misdirected ops = true 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: bdev async discard = True 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: bdev enable discard = True 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore allocator = bitmap 2023-11-28T02:51:52.390 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore block size = 96636764160 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: bluestore fsck on mount = True 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluefs = 1/20 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug bluestore = 1/20 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug osd = 20 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: debug rocksdb = 4/10 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd backfillfull_ratio = 0.85 2023-11-28T02:51:52.391 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd full ratio = 0.9 2023-11-28T02:51:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd nearfull ratio = 0.8 2023-11-28T02:51:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd failsafe full ratio = 0.95 2023-11-28T02:51:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd objectstore = bluestore 2023-11-28T02:51:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-28T02:51:52.393 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr] 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug mgr = 20 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: mgr/telemetry/nag = false 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: debug finisher = 20 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.394 INFO:teuthology.orchestra.run.smithi150.stdout:[mon] 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: debug mon = 20 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: debug paxos = 20 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: debug auth = 20 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon data avail warn = 5 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon mgr mkfs grace = 240 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min pgs per osd = 4 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd reporter subtree level = osd 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon osd prime pg temp = true 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: mon reweight min bytes per osd = 10 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.395 INFO:teuthology.orchestra.run.smithi150.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: auth mon ticket ttl = 660 # 11m 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: auth service ticket ttl = 240 # 4m 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: # don't complain about insecure global_id in the test suite 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: mon op complaint time = 120 2023-11-28T02:51:52.396 INFO:teuthology.orchestra.run.smithi150.stdout: 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout:[client] 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: rgw cache enabled = true 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable ops log = true 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: rgw enable usage log = true 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: client mount timeout = 600 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: debug client = 20 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: pid file = "" 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.a] 2023-11-28T02:51:52.397 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.b] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mon.c] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout:[mds] 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds = 20 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: debug mds balancer = 20 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: debug ms = 1 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug frag = True 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: mds debug scatterstat = True 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: mds op complaint time = 180 2023-11-28T02:51:52.398 INFO:teuthology.orchestra.run.smithi150.stdout: mds verify scatter = True 2023-11-28T02:51:52.399 INFO:teuthology.orchestra.run.smithi150.stdout: osd op complaint time = 180 2023-11-28T02:51:52.399 INFO:teuthology.orchestra.run.smithi150.stdout: rados mon op timeout = 900 2023-11-28T02:51:52.399 INFO:teuthology.orchestra.run.smithi150.stdout: rados osd op timeout = 900 2023-11-28T02:51:52.399 INFO:tasks.ceph:Creating admin key on mon.a... 2023-11-28T02:51:52.399 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-11-28T02:51:52.473 INFO:tasks.ceph:Copying monmap to all nodes... 2023-11-28T02:51:52.473 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:52.473 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-11-28T02:51:52.488 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:52.488 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-11-28T02:51:52.542 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:51:52.542 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:52.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T02:51:52.543 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T02:51:52.616 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:51:52.616 DEBUG:teuthology.orchestra.run.smithi073:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:52.671 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:51:52.671 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:52.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd of=/etc/ceph/ceph.keyring 2023-11-28T02:51:52.671 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-11-28T02:51:52.705 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:51:52.705 DEBUG:teuthology.orchestra.run.smithi150:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:52.759 INFO:tasks.ceph:Setting up mon nodes... 2023-11-28T02:51:52.760 INFO:tasks.ceph:Setting up mgr nodes... 2023-11-28T02:51:52.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T02:51:52.806 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-11-28T02:51:52.808 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T02:51:52.901 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-11-28T02:51:52.904 INFO:tasks.ceph:Setting up mds nodes... 2023-11-28T02:51:52.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T02:51:52.950 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-11-28T02:51:52.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-11-28T02:51:52.977 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T02:51:53.064 INFO:teuthology.orchestra.run.smithi073.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-11-28T02:51:53.066 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-11-28T02:51:53.090 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T02:51:53.137 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-11-28T02:51:53.139 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-11-28T02:51:53.163 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T02:51:53.251 INFO:teuthology.orchestra.run.smithi150.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-11-28T02:51:53.253 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-11-28T02:51:53.277 INFO:tasks.ceph_client:Setting up client nodes... 2023-11-28T02:51:53.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-11-28T02:51:53.314 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-11-28T02:51:53.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-11-28T02:51:53.402 INFO:teuthology.orchestra.run.smithi073.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-11-28T02:51:53.415 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-11-28T02:51:53.452 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-11-28T02:51:53.465 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-11-28T02:51:53.542 INFO:teuthology.orchestra.run.smithi150.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-11-28T02:51:53.555 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-11-28T02:51:53.555 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi073.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi150.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-11-28T02:51:53.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-11-28T02:51:53.579 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:53.579 INFO:tasks.ceph:role: osd.0 2023-11-28T02:51:53.579 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:51:53.579 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:51:53.650 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:51:53.651 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:51:53.651 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:51:53.651 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:51:54.083 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2023-11-28T02:51:54.085 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2023-11-28T02:51:54.085 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-11-28T02:51:54.273 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-11-28T02:51:54.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-11-28T02:51:54.366 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:54.367 INFO:tasks.ceph:role: osd.1 2023-11-28T02:51:54.367 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:51:54.367 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:51:54.439 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:51:54.440 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:51:54.441 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:51:54.441 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:51:54.441 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:51:54.864 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2023-11-28T02:51:54.866 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2023-11-28T02:51:54.866 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-11-28T02:51:54.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-11-28T02:51:55.001 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-11-28T02:51:55.065 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:55.066 INFO:tasks.ceph:role: osd.2 2023-11-28T02:51:55.066 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:51:55.066 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T02:51:55.137 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:51:55.138 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:51:55.139 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:51:55.589 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2023-11-28T02:51:55.591 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2023-11-28T02:51:55.591 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-11-28T02:51:55.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-11-28T02:51:55.761 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-11-28T02:51:55.825 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-11-28T02:51:55.825 INFO:tasks.ceph:role: osd.3 2023-11-28T02:51:55.825 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:51:55.825 DEBUG:teuthology.orchestra.run.smithi073:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T02:51:55.896 INFO:teuthology.orchestra.run.smithi073.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout: = reflink=1 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:51:55.897 INFO:teuthology.orchestra.run.smithi073.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:51:56.334 INFO:teuthology.orchestra.run.smithi073.stdout:Discarding blocks...Done. 2023-11-28T02:51:56.337 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi073.front.sepia.ceph.com -o noatime 2023-11-28T02:51:56.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-11-28T02:51:56.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-11-28T02:51:56.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.713+0000 7f783e6ec380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-11-28T02:51:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.713+0000 7f783e6ec380 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-11-28T02:51:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.713+0000 7f783e6ec380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T02:51:56.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.713+0000 7f783e6ec380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T02:51:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.713+0000 7f783e6ec380 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-11-28T02:51:56.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:56.714+0000 7f783e6ec380 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-11-28T02:51:59.560 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-11-28T02:51:59.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:51:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.664+0000 7fd90d1fa380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-11-28T02:51:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.664+0000 7fd90d1fa380 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-11-28T02:51:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.664+0000 7fd90d1fa380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T02:51:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.664+0000 7fd90d1fa380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T02:51:59.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.664+0000 7fd90d1fa380 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-11-28T02:51:59.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:51:59.665+0000 7fd90d1fa380 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-11-28T02:52:02.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-11-28T02:52:02.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.737+0000 7f8bbc91c380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.737+0000 7f8bbc91c380 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.737+0000 7f8bbc91c380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.737+0000 7f8bbc91c380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.737+0000 7f8bbc91c380 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-11-28T02:52:02.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:02.738+0000 7f8bbc91c380 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-11-28T02:52:05.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-11-28T02:52:05.603 DEBUG:teuthology.orchestra.run.smithi073:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:05.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.686+0000 7f35b4519380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-11-28T02:52:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.686+0000 7f35b4519380 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-11-28T02:52:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.686+0000 7f35b4519380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T02:52:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.686+0000 7f35b4519380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T02:52:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.686+0000 7f35b4519380 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-11-28T02:52:05.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:05.687+0000 7f35b4519380 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-11-28T02:52:08.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-11-28T02:52:08.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-11-28T02:52:08.638 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T02:52:08.639 INFO:tasks.ceph:role: osd.4 2023-11-28T02:52:08.639 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:52:08.639 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-11-28T02:52:08.712 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:52:08.712 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:52:08.712 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:52:08.712 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:52:08.713 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:52:09.130 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-28T02:52:09.131 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-28T02:52:09.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-11-28T02:52:09.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-11-28T02:52:09.349 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-11-28T02:52:09.415 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T02:52:09.415 INFO:tasks.ceph:role: osd.5 2023-11-28T02:52:09.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:52:09.415 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-11-28T02:52:09.487 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:52:09.488 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:52:09.489 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:52:09.489 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:52:09.489 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:52:09.911 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-28T02:52:09.914 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-28T02:52:09.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-11-28T02:52:10.018 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-11-28T02:52:10.048 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-11-28T02:52:10.113 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T02:52:10.113 INFO:tasks.ceph:role: osd.6 2023-11-28T02:52:10.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:52:10.114 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-11-28T02:52:10.185 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:52:10.185 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:52:10.185 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:52:10.185 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-11-28T02:52:10.185 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:52:10.186 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:52:10.186 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:52:10.186 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:52:10.186 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:52:10.186 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:52:10.634 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-28T02:52:10.636 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-28T02:52:10.637 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-11-28T02:52:10.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-11-28T02:52:10.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-11-28T02:52:10.914 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-11-28T02:52:10.915 INFO:tasks.ceph:role: osd.7 2023-11-28T02:52:10.915 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:52:10.915 DEBUG:teuthology.orchestra.run.smithi150:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-11-28T02:52:10.989 INFO:teuthology.orchestra.run.smithi150.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 attr=2, projid32bit=1 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout: = reflink=1 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout: = sunit=0 swidth=0 blks 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-11-28T02:52:10.990 INFO:teuthology.orchestra.run.smithi150.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-11-28T02:52:11.417 INFO:teuthology.orchestra.run.smithi150.stdout:Discarding blocks...Done. 2023-11-28T02:52:11.419 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi150.front.sepia.ceph.com -o noatime 2023-11-28T02:52:11.419 DEBUG:teuthology.orchestra.run.smithi150:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-11-28T02:52:11.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-11-28T02:52:11.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:11.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.719+0000 7fb481007380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-11-28T02:52:11.722 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.719+0000 7fb481007380 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-11-28T02:52:11.722 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.719+0000 7fb481007380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T02:52:11.722 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.719+0000 7fb481007380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T02:52:11.723 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.719+0000 7fb481007380 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-11-28T02:52:11.723 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:11.720+0000 7fb481007380 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-11-28T02:52:14.571 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-11-28T02:52:14.600 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:14.686 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.685+0000 7f1301c3c380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-11-28T02:52:14.686 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.685+0000 7f1301c3c380 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-11-28T02:52:14.686 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.685+0000 7f1301c3c380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T02:52:14.686 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.685+0000 7f1301c3c380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T02:52:14.687 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.685+0000 7f1301c3c380 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-11-28T02:52:14.687 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:14.686+0000 7f1301c3c380 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-11-28T02:52:17.521 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-11-28T02:52:17.547 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:17.628 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-11-28T02:52:17.628 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-11-28T02:52:17.629 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T02:52:17.629 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T02:52:17.629 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-11-28T02:52:17.629 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:17.628+0000 7f0da1974380 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-11-28T02:52:20.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:52:20.358 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:52:20.477 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-11-28T02:52:20.503 DEBUG:teuthology.orchestra.run.smithi150:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:20.585 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.584+0000 7f39b71f2380 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-11-28T02:52:20.586 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.584+0000 7f39b71f2380 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-11-28T02:52:20.586 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.584+0000 7f39b71f2380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T02:52:20.586 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.584+0000 7f39b71f2380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T02:52:20.586 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.584+0000 7f39b71f2380 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-11-28T02:52:20.586 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:20.585+0000 7f39b71f2380 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-11-28T02:52:23.431 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-11-28T02:52:23.497 INFO:tasks.ceph:Reading keys from all nodes... 2023-11-28T02:52:23.497 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.497 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-11-28T02:52:23.525 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.525 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-11-28T02:52:23.591 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.592 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-11-28T02:52:23.658 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-11-28T02:52:23.723 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-11-28T02:52:23.789 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-11-28T02:52:23.854 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:23.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-11-28T02:52:23.918 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:23.918 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-11-28T02:52:23.943 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:23.943 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-11-28T02:52:24.008 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-11-28T02:52:24.072 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.072 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-11-28T02:52:24.135 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.136 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-11-28T02:52:24.199 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.200 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-11-28T02:52:24.263 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.264 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-11-28T02:52:24.328 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:24.328 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-11-28T02:52:24.343 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:24.344 DEBUG:teuthology.orchestra.run.smithi073:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-11-28T02:52:24.399 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.399 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-11-28T02:52:24.414 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:24.414 DEBUG:teuthology.orchestra.run.smithi150:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-11-28T02:52:24.468 INFO:tasks.ceph:Adding keys to all mons... 2023-11-28T02:52:24.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T02:52:24.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo tee -a /etc/ceph/ceph.keyring 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.y] 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg== 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.a] 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA== 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.c] 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw== 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.0] 2023-11-28T02:52:24.513 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.1] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.2] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.3] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[mgr.x] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.b] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ== 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout:[mds.d] 2023-11-28T02:52:24.514 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.4] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.5] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.6] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[osd.7] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[client.0] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ== 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout:[client.2] 2023-11-28T02:52:24.515 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw== 2023-11-28T02:52:24.516 INFO:teuthology.orchestra.run.smithi073.stdout:[client.1] 2023-11-28T02:52:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ== 2023-11-28T02:52:24.516 INFO:teuthology.orchestra.run.smithi073.stdout:[client.3] 2023-11-28T02:52:24.516 INFO:teuthology.orchestra.run.smithi073.stdout: key = AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA== 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.y] 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg== 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.a] 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA== 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.c] 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw== 2023-11-28T02:52:24.532 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.0] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.1] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.2] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.3] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[mgr.x] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.b] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ== 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout:[mds.d] 2023-11-28T02:52:24.533 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.4] 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.5] 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.6] 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[osd.7] 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[client.0] 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ== 2023-11-28T02:52:24.534 INFO:teuthology.orchestra.run.smithi150.stdout:[client.2] 2023-11-28T02:52:24.535 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw== 2023-11-28T02:52:24.535 INFO:teuthology.orchestra.run.smithi150.stdout:[client.1] 2023-11-28T02:52:24.535 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ== 2023-11-28T02:52:24.535 INFO:teuthology.orchestra.run.smithi150.stdout:[client.3] 2023-11-28T02:52:24.535 INFO:teuthology.orchestra.run.smithi150.stdout: key = AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA== 2023-11-28T02:52:24.536 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T02:52:24.560 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T02:52:24.622 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:24.624 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:24.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:24.675 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:24.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.725 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.819 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.867 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.869 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:24.915 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T02:52:24.917 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-11-28T02:52:24.964 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:24.966 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:25.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:25.017 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-11-28T02:52:25.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.069 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.163 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.165 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.211 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-11-28T02:52:25.259 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.314 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.365 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.414 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.416 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-11-28T02:52:25.464 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-11-28T02:52:25.465 DEBUG:teuthology.orchestra.run.smithi073:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-11-28T02:52:25.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T02:52:25.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-11-28T02:52:25.606 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-11-28T02:52:25.630 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T02:52:25.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-11-28T02:52:25.746 DEBUG:teuthology.orchestra.run.smithi150:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-11-28T02:52:25.810 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-11-28T02:52:25.901 DEBUG:teuthology.orchestra.run.smithi150:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-11-28T02:52:25.925 DEBUG:teuthology.orchestra.run.smithi073:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:25.927 DEBUG:teuthology.orchestra.run.smithi150:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-11-28T02:52:25.980 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-11-28T02:52:25.980 INFO:tasks.ceph.mon.a:Restarting daemon 2023-11-28T02:52:25.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-11-28T02:52:25.983 INFO:tasks.ceph.mon.a:Started 2023-11-28T02:52:25.983 INFO:tasks.ceph.mon.b:Restarting daemon 2023-11-28T02:52:25.983 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-11-28T02:52:26.022 INFO:tasks.ceph.mon.b:Started 2023-11-28T02:52:26.023 INFO:tasks.ceph.mon.c:Restarting daemon 2023-11-28T02:52:26.023 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-11-28T02:52:26.025 INFO:tasks.ceph.mon.c:Started 2023-11-28T02:52:26.025 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-11-28T02:52:26.025 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T02:52:26.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T02:52:26.027 INFO:tasks.ceph.mgr.y:Started 2023-11-28T02:52:26.027 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-11-28T02:52:26.027 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-11-28T02:52:26.029 INFO:tasks.ceph.mgr.x:Started 2023-11-28T02:52:26.029 INFO:tasks.ceph:Setting crush tunables to default 2023-11-28T02:52:26.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd crush tunables default 2023-11-28T02:52:26.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.143+0000 7f12e3687700 1 Processor -- start 2023-11-28T02:52:26.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.143+0000 7f12e3687700 1 -- start start 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e3687700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 0x7f12dc153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e3687700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e3687700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12dc155f20 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e3687700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc153990 con 0x7f12dc0c5ea0 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e3687700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12dc0c62d0 con 0x7f12dc0c51a0 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45272/0 (socket says 172.21.15.73:45272) 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 -- 172.21.15.73:0/532033654 learned_addr learned my addr 172.21.15.73:0/532033654 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1423700 1 --1- 172.21.15.73:0/532033654 >> v1:172.21.15.73:6789/0 conn(0x7f12dc0c5ac0 0x7f12dc156690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60624/0 (socket says 172.21.15.73:60624) 2023-11-28T02:52:26.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 msgr2=0x7f12dc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 msgr2=0x7f12dc066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e1c24700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e0c22700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 0x7f12dc153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e0c22700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 msgr2=0x7f12dc153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e0c22700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 msgr2=0x7f12dc153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e0c22700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 0x7f12dc153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-11-28T02:52:26.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.144+0000 7f12e0c22700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 0x7f12dc153390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T02:52:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693265383 0 0) 0x7f12dc155f20 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12c80033f0 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2721153241 0 0) 0x7f12c80033f0 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 msgr2=0x7f12dc153390 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.150:3300/0 conn(0x7f12dc0c51a0 0x7f12dc153390 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 msgr2=0x7f12dc066a70 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 --2- 172.21.15.73:0/532033654 >> v2:172.21.15.73:3300/0 conn(0x7f12dc0c5ea0 0x7f12dc066a70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc156200 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f12d40017f0 con 0x7f12dc0c5ac0 2023-11-28T02:52:26.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 >> v1:172.21.15.73:6789/0 conn(0x7f12dc0c5ac0 legacy=0x7f12dc156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 0x7f12c8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12c8004690 0x7f12c8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c8005050 0x7f12c800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc153990 con 0x7f12c80038e0 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0c62d0 con 0x7f12c8005050 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c800c040 con 0x7f12c8004690 2023-11-28T02:52:26.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 0x7f12c8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e0c22700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12c8004690 0x7f12c8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1c24700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c8005050 0x7f12c800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 -- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12c8004690 msgr2=0x7f12c8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12c8004690 0x7f12c8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 -- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c8005050 msgr2=0x7f12c800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c8005050 0x7f12c800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1c24700 1 -- 172.21.15.73:0/532033654 reap_dead start 2023-11-28T02:52:26.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 -- 172.21.15.73:0/532033654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c80033f0 con 0x7f12c80038e0 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.309+0000 7f12e1423700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 0x7f12c8003d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f12d400c5f0 tx=0x7f12dc153880).ready entity=mon.0 client_cookie=2b0dc00f7b422271 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d400cdb0 con 0x7f12c80038e0 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d4005a60 con 0x7f12c80038e0 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12d3fff700 1 -- 172.21.15.73:0/532033654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d4005de0 con 0x7f12c80038e0 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 -- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 msgr2=0x7f12c8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 0x7f12c8003d00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7f12d400c5f0 tx=0x7f12dc153880).stop 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 -- 172.21.15.73:0/532033654 shutdown_connections 2023-11-28T02:52:26.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 --2- 172.21.15.73:0/532033654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12c80038e0 0x7f12c8003d00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 -- 172.21.15.73:0/532033654 >> 172.21.15.73:0/532033654 conn(0x7f12dc062700 msgr2=0x7f12dc062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 -- 172.21.15.73:0/532033654 shutdown_connections 2023-11-28T02:52:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.310+0000 7f12e3687700 1 -- 172.21.15.73:0/532033654 wait complete. 2023-11-28T02:52:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 Processor -- start 2023-11-28T02:52:26.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 -- start start 2023-11-28T02:52:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80038e0 0x7f12dc14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12dc066ed0 0x7f12dc067350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:26.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc1663a0 con 0x7f12dc165a30 2023-11-28T02:52:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc0abac0 con 0x7f12c80038e0 2023-11-28T02:52:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.312+0000 7f12e3687700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12dc162bc0 con 0x7f12dc066ed0 2023-11-28T02:52:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45294/0 (socket says 172.21.15.73:45294) 2023-11-28T02:52:26.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 -- 172.21.15.73:0/411119174 learned_addr learned my addr 172.21.15.73:0/411119174 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e1c24700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12dc066ed0 0x7f12dc067350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e1423700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80038e0 0x7f12dc14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 -- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12dc066ed0 msgr2=0x7f12dc067350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12dc066ed0 0x7f12dc067350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 -- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80038e0 msgr2=0x7f12dc14dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80038e0 0x7f12dc14dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 -- 172.21.15.73:0/411119174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12dc067840 con 0x7f12dc165a30 2023-11-28T02:52:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e0c22700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f12cc0109b0 tx=0x7f12cc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:26.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc014680 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12cc0147e0 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12dc166880 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12cc014b60 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12dc162570 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f12cc00e6e0 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.313+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 754+0+0 (secure 0 0 0) 0x7f12cc01d770 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.314+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12dc068270 con 0x7f12dc165a30 2023-11-28T02:52:26.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.316+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f12cc03ce50 con 0x7f12dc165a30 2023-11-28T02:52:26.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.389+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f12dc06c860 con 0x7f12dc165a30 2023-11-28T02:52:26.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.391+0000 7f12d17fa700 1 -- 172.21.15.73:0/411119174 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f12cc02b070 con 0x7f12dc165a30 2023-11-28T02:52:26.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 msgr2=0x7f12dc165eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:26.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f12cc0109b0 tx=0x7f12cc008040).stop 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 shutdown_connections 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f12dc066ed0 0x7f12dc067350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f12c80038e0 0x7f12dc14dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 --2- 172.21.15.73:0/411119174 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f12dc165a30 0x7f12dc165eb0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 >> 172.21.15.73:0/411119174 conn(0x7f12dc062700 msgr2=0x7f12dc0c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 shutdown_connections 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:26.392+0000 7f12e3687700 1 -- 172.21.15.73:0/411119174 wait complete. 2023-11-28T02:52:26.394 INFO:teuthology.orchestra.run.smithi073.stderr:adjusted tunables profile to default 2023-11-28T02:52:26.404 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-11-28T02:52:26.405 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:26.405 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-11-28T02:52:26.428 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:26.429 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-11-28T02:52:26.493 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:26.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-11-28T02:52:26.557 DEBUG:teuthology.orchestra.run.smithi073:> set -ex 2023-11-28T02:52:26.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-11-28T02:52:26.621 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:26.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-11-28T02:52:26.651 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:26.651 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-11-28T02:52:26.716 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:26.716 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-11-28T02:52:26.780 DEBUG:teuthology.orchestra.run.smithi150:> set -ex 2023-11-28T02:52:26.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-11-28T02:52:26.827 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:26.826+0000 7efc5a8e3680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T02:52:26.830 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:26.829+0000 7f07e8d52680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T02:52:26.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 9c698d30-5b01-4161-8d18-49a843ed4bf5 0 2023-11-28T02:52:26.895 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:26.894+0000 7efc5a8e3680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T02:52:26.903 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:26.902+0000 7f07e8d52680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T02:52:27.000 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:26.999+0000 7fb111c1f700 1 Processor -- start 2023-11-28T02:52:27.000 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:26.999+0000 7fb111c1f700 1 -- start start 2023-11-28T02:52:27.002 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:26.999+0000 7fb111c1f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 0x7fb10c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.002 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb111c1f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb10c0c5ea0 0x7fb10c066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.002 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb111c1f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb10c155f20 con 0x7fb10c153390 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb111c1f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c153990 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb111c1f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb10c0c62d0 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb10c0c5ea0 0x7fb10c066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb10c0c5ea0 0x7fb10c066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36700/0 (socket says 172.21.15.150:36700) 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 -- 172.21.15.150:0/1648149852 learned_addr learned my addr 172.21.15.150:0/1648149852 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10b7fe700 1 --2- 172.21.15.150:0/1648149852 >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 0x7fb10c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 -- 172.21.15.150:0/1648149852 >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 msgr2=0x7fb10c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 --2- 172.21.15.150:0/1648149852 >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 0x7fb10c0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 -- 172.21.15.150:0/1648149852 >> v1:172.21.15.73:6789/0 conn(0x7fb10c153390 legacy=0x7fb10c156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 -- 172.21.15.150:0/1648149852 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb10c156200 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10b7fe700 1 --2- 172.21.15.150:0/1648149852 >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 0x7fb10c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10bfff700 1 --2- 172.21.15.150:0/1648149852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c066a70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb10c150550 tx=0x7fb104009840).ready entity=mon.1 client_cookie=190649aef38634e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10affd700 1 -- 172.21.15.150:0/1648149852 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400e070 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.003 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10affd700 1 -- 172.21.15.150:0/1648149852 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb10400ba50 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.000+0000 7fb10affd700 1 -- 172.21.15.150:0/1648149852 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb10400bdd0 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 -- 172.21.15.150:0/1648149852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 msgr2=0x7fb10c066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 --2- 172.21.15.150:0/1648149852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c066a70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fb10c150550 tx=0x7fb104009840).stop 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 -- 172.21.15.150:0/1648149852 shutdown_connections 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 --2- 172.21.15.150:0/1648149852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c066a70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 --2- 172.21.15.150:0/1648149852 >> v2:172.21.15.73:3300/0 conn(0x7fb10c0c5ac0 0x7fb10c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 -- 172.21.15.150:0/1648149852 >> 172.21.15.150:0/1648149852 conn(0x7fb10c062700 msgr2=0x7fb10c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 -- 172.21.15.150:0/1648149852 shutdown_connections 2023-11-28T02:52:27.004 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.001+0000 7fb111c1f700 1 -- 172.21.15.150:0/1648149852 wait complete. 2023-11-28T02:52:27.006 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.002+0000 7fb111c1f700 1 Processor -- start 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.002+0000 7fb111c1f700 1 -- start start 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c0960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb10c153390 0x7fb10c1621e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c0af060 con 0x7fb10c153390 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1626d0 con 0x7fb10c0c5ea0 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb111c1f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb10c1631b0 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54542/0 (socket says 172.21.15.150:54542) 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 -- 172.21.15.150:0/3606355972 learned_addr learned my addr 172.21.15.150:0/3606355972 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb103fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c0960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10bfff700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb10c153390 0x7fb10c1621e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.007 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 -- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 msgr2=0x7fb10c0960d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c0960d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 -- 172.21.15.150:0/3606355972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb10c153390 msgr2=0x7fb10c1621e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb10c153390 0x7fb10c1621e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 -- 172.21.15.150:0/3606355972 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb104009ae0 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.003+0000 7fb10b7fe700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb0fc00d860 tx=0x7fb0fc008040).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00f520 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0fc00f680 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb10c162910 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0fc00fa20 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb10c162ed0 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fb0fc013480 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.004+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 754+0+0 (secure 0 0 0) 0x7fb0fc01d7b0 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.008 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.005+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb10c06c860 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.009 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.007+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fb0fc022060 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.089 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.087+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9c698d30-5b01-4161-8d18-49a843ed4bf5", "id": 0} v 0) v1 -- 0x7fb10c0c4f80 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.092 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.091+0000 7fb1037fe700 1 -- 172.21.15.150:0/3606355972 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9c698d30-5b01-4161-8d18-49a843ed4bf5", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb0fc018070 con 0x7fb10c0c5ac0 2023-11-28T02:52:27.094 INFO:teuthology.orchestra.run.smithi150.stdout:0 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 msgr2=0x7fb10c0c0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7fb0fc00d860 tx=0x7fb0fc008040).stop 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 shutdown_connections 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb10c153390 0x7fb10c1621e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb10c0c5ac0 0x7fb10c0c0bf0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 --2- 172.21.15.150:0/3606355972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb10c0c5ea0 0x7fb10c0960d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 >> 172.21.15.150:0/3606355972 conn(0x7fb10c062700 msgr2=0x7fb10c0c6ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.095 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 shutdown_connections 2023-11-28T02:52:27.096 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.092+0000 7fb111c1f700 1 -- 172.21.15.150:0/3606355972 wait complete. 2023-11-28T02:52:27.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 84f00a60-8e08-4ed3-b366-c77822b49826 1 2023-11-28T02:52:27.260 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.258+0000 7fc46bf61700 1 Processor -- start 2023-11-28T02:52:27.260 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.258+0000 7fc46bf61700 1 -- start start 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc464150930 0x7fc4640c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 0x7fc4640bd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4640c2460 con 0x7fc464150930 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc464062910 con 0x7fc4640bcfa0 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc46bf61700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4640bd8e0 con 0x7fc464150d10 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.259+0000 7fc4694fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54552/0 (socket says 172.21.15.150:54552) 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 -- 172.21.15.150:0/543739518 learned_addr learned my addr 172.21.15.150:0/543739518 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc46a4fe700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 0x7fc4640bd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc469cfd700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.73:3300/0 conn(0x7fc464150930 0x7fc4640c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.263 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 -- 172.21.15.150:0/543739518 >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 msgr2=0x7fc4640bd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:27.264 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 0x7fc4640bd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.264 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 -- 172.21.15.150:0/543739518 >> v2:172.21.15.73:3300/0 conn(0x7fc464150930 msgr2=0x7fc4640c6620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:27.264 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.73:3300/0 conn(0x7fc464150930 0x7fc4640c6620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.264 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 -- 172.21.15.150:0/543739518 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4640c6c00 con 0x7fc464150d10 2023-11-28T02:52:27.265 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc46a4fe700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 0x7fc4640bd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:27.265 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc4694fc700 1 --2- 172.21.15.150:0/543739518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fc4640c6b10 tx=0x7fc460009060).ready entity=mon.2 client_cookie=436ee0513ad7962e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.265 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc468cfb700 1 -- 172.21.15.150:0/543739518 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460005880 con 0x7fc464150d10 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc468cfb700 1 -- 172.21.15.150:0/543739518 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4600059e0 con 0x7fc464150d10 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.260+0000 7fc468cfb700 1 -- 172.21.15.150:0/543739518 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc460005d60 con 0x7fc464150d10 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 -- 172.21.15.150:0/543739518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 msgr2=0x7fc4640a6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 --2- 172.21.15.150:0/543739518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7fc4640c6b10 tx=0x7fc460009060).stop 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 -- 172.21.15.150:0/543739518 shutdown_connections 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.73:3300/0 conn(0x7fc464150930 0x7fc4640c6620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 --2- 172.21.15.150:0/543739518 >> v2:172.21.15.150:3300/0 conn(0x7fc4640bcfa0 0x7fc4640bd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 --2- 172.21.15.150:0/543739518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc464150d10 0x7fc4640a6ad0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.268 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 -- 172.21.15.150:0/543739518 >> 172.21.15.150:0/543739518 conn(0x7fc4640c8230 msgr2=0x7fc4640c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 -- 172.21.15.150:0/543739518 shutdown_connections 2023-11-28T02:52:27.271 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.261+0000 7fc46bf61700 1 -- 172.21.15.150:0/543739518 wait complete. 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.263+0000 7fc46bf61700 1 Processor -- start 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.263+0000 7fc46bf61700 1 -- start start 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 0x7fc46414f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464150d10 0x7fc4640a77d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46414e310 con 0x7fc464150d10 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc46414f170 con 0x7fc464150930 2023-11-28T02:52:27.274 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46bf61700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc46414c4e0 con 0x7fc4640bcfa0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54560/0 (socket says 172.21.15.150:54560) 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 -- 172.21.15.150:0/3045009699 learned_addr learned my addr 172.21.15.150:0/3045009699 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc4694fc700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 0x7fc46414f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc46a4fe700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464150d10 0x7fc4640a77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 -- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 msgr2=0x7fc46414f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 0x7fc46414f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 -- 172.21.15.150:0/3045009699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464150d10 msgr2=0x7fc4640a77d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464150d10 0x7fc4640a77d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 -- 172.21.15.150:0/3045009699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45c009a20 con 0x7fc4640bcfa0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc4694fc700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 0x7fc46414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.264+0000 7fc469cfd700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc45400e8d0 tx=0x7fc454006f40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454009450 con 0x7fc4640bcfa0 2023-11-28T02:52:27.275 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc454012600 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc460009300 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc454012980 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc46414e840 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fc45401b400 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.265+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 869+0+0 (secure 0 0 0) 0x7fc45401a3b0 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.266+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc46414f450 con 0x7fc4640bcfa0 2023-11-28T02:52:27.276 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.268+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fc45403ae50 con 0x7fc4640bcfa0 2023-11-28T02:52:27.350 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.349+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "84f00a60-8e08-4ed3-b366-c77822b49826", "id": 1} v 0) v1 -- 0x7fc46406c860 con 0x7fc4640bcfa0 2023-11-28T02:52:27.358 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.356+0000 7fc459ffb700 1 -- 172.21.15.150:0/3045009699 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "84f00a60-8e08-4ed3-b366-c77822b49826", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc454029030 con 0x7fc4640bcfa0 2023-11-28T02:52:27.359 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 msgr2=0x7fc46414bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7fc45400e8d0 tx=0x7fc454006f40).stop 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 shutdown_connections 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc464150d10 0x7fc4640a77d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4640bcfa0 0x7fc46414bff0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 --2- 172.21.15.150:0/3045009699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc464150930 0x7fc46414f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 >> 172.21.15.150:0/3045009699 conn(0x7fc4640c8230 msgr2=0x7fc4640c3890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 shutdown_connections 2023-11-28T02:52:27.360 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.357+0000 7fc46bf61700 1 -- 172.21.15.150:0/3045009699 wait complete. 2023-11-28T02:52:27.370 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 45f934f5-ca99-4a3e-a482-b6adc132b3d3 2 2023-11-28T02:52:27.380 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:27.379+0000 7efc5a8e3680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T02:52:27.485 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.483+0000 7fd4e36bf700 1 Processor -- start 2023-11-28T02:52:27.485 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 -- start start 2023-11-28T02:52:27.487 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4dc158160 0x7fd4dc0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.487 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.487 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4dc067770 con 0x7fd4dc0a8280 2023-11-28T02:52:27.487 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4dc159180 con 0x7fd4dc158160 2023-11-28T02:52:27.487 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.484+0000 7fd4e36bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0be8d0 con 0x7fd4dc14e600 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54568/0 (socket says 172.21.15.150:54568) 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 -- 172.21.15.150:0/2104071188 learned_addr learned my addr 172.21.15.150:0/2104071188 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e0c5a700 1 --2- 172.21.15.150:0/2104071188 >> v2:172.21.15.73:3300/0 conn(0x7fd4dc158160 0x7fd4dc0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 -- 172.21.15.150:0/2104071188 >> v2:172.21.15.73:3300/0 conn(0x7fd4dc158160 msgr2=0x7fd4dc0a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 --2- 172.21.15.150:0/2104071188 >> v2:172.21.15.73:3300/0 conn(0x7fd4dc158160 0x7fd4dc0a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 -- 172.21.15.150:0/2104071188 >> v1:172.21.15.73:6789/0 conn(0x7fd4dc0a8280 legacy=0x7fd4dc155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4d3fff700 1 -- 172.21.15.150:0/2104071188 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2533270561 0 0) 0x7fd4dc067770 con 0x7fd4dc0a8280 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 -- 172.21.15.150:0/2104071188 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4dc158600 con 0x7fd4dc14e600 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4e1c5c700 1 --2- 172.21.15.150:0/2104071188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd4d4009f20 tx=0x7fd4d4009840).ready entity=mon.2 client_cookie=daf22e2237a06e8c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.488 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4d3fff700 1 -- 172.21.15.150:0/2104071188 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400e070 con 0x7fd4dc14e600 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4d3fff700 1 -- 172.21.15.150:0/2104071188 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4d400b9d0 con 0x7fd4dc14e600 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.485+0000 7fd4d3fff700 1 -- 172.21.15.150:0/2104071188 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4d400bd50 con 0x7fd4dc14e600 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 -- 172.21.15.150:0/2104071188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 msgr2=0x7fd4dc14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/2104071188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7fd4d4009f20 tx=0x7fd4d4009840).stop 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 -- 172.21.15.150:0/2104071188 shutdown_connections 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/2104071188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc14ea90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/2104071188 >> v2:172.21.15.73:3300/0 conn(0x7fd4dc158160 0x7fd4dc0a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 -- 172.21.15.150:0/2104071188 >> 172.21.15.150:0/2104071188 conn(0x7fd4dc062700 msgr2=0x7fd4dc062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 -- 172.21.15.150:0/2104071188 shutdown_connections 2023-11-28T02:52:27.489 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.486+0000 7fd4e36bf700 1 -- 172.21.15.150:0/2104071188 wait complete. 2023-11-28T02:52:27.491 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.487+0000 7fd4e36bf700 1 Processor -- start 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.487+0000 7fd4e36bf700 1 -- start start 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4dc0a8280 0x7fd4dc0c5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 0x7fd4dc14c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0a9ae0 con 0x7fd4dc0a8280 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc0ad230 con 0x7fd4dc158160 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e36bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4dc14cb40 con 0x7fd4dc14e600 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e1c5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 0x7fd4dc14c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54582/0 (socket says 172.21.15.150:54582) 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 -- 172.21.15.150:0/3571717061 learned_addr learned my addr 172.21.15.150:0/3571717061 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e145b700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4dc0a8280 0x7fd4dc0c5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.492 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 -- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 msgr2=0x7fd4dc14c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 0x7fd4dc14c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 -- 172.21.15.150:0/3571717061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4dc0a8280 msgr2=0x7fd4dc0c5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4dc0a8280 0x7fd4dc0c5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 -- 172.21.15.150:0/3571717061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d4009ae0 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e1c5c700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 0x7fd4dc14c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.488+0000 7fd4e0c5a700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fd4cc00e900 tx=0x7fd4cc009060).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.489+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc012600 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.489+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4cc012760 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.489+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4dc0a9d20 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.489+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4cc012ae0 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.489+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4dc0ad790 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.490+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fd4cc01b660 con 0x7fd4dc14e600 2023-11-28T02:52:27.493 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.490+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 984+0+0 (secure 0 0 0) 0x7fd4cc01a800 con 0x7fd4dc14e600 2023-11-28T02:52:27.494 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.490+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4dc06c860 con 0x7fd4dc14e600 2023-11-28T02:52:27.494 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.492+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fd4cc016020 con 0x7fd4dc14e600 2023-11-28T02:52:27.501 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:27.500+0000 7f07e8d52680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T02:52:27.573 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.572+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "45f934f5-ca99-4a3e-a482-b6adc132b3d3", "id": 2} v 0) v1 -- 0x7fd4dc14d270 con 0x7fd4dc14e600 2023-11-28T02:52:27.582 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.580+0000 7fd4d17fa700 1 -- 172.21.15.150:0/3571717061 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "45f934f5-ca99-4a3e-a482-b6adc132b3d3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd4cc01bba0 con 0x7fd4dc14e600 2023-11-28T02:52:27.582 INFO:teuthology.orchestra.run.smithi150.stdout:2 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 msgr2=0x7fd4dc0a95f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7fd4cc00e900 tx=0x7fd4cc009060).stop 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 shutdown_connections 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4dc158160 0x7fd4dc14c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4dc0a8280 0x7fd4dc0c5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.583 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 --2- 172.21.15.150:0/3571717061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4dc14e600 0x7fd4dc0a95f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.584 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 >> 172.21.15.150:0/3571717061 conn(0x7fd4dc062700 msgr2=0x7fd4dc14db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.584 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 shutdown_connections 2023-11-28T02:52:27.584 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.582+0000 7fd4e36bf700 1 -- 172.21.15.150:0/3571717061 wait complete. 2023-11-28T02:52:27.598 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new e0055829-e29c-4f7a-8a17-e21c21919c3a 3 2023-11-28T02:52:27.712 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.710+0000 7f5a0d4cf700 1 Processor -- start 2023-11-28T02:52:27.712 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.710+0000 7f5a0d4cf700 1 -- start start 2023-11-28T02:52:27.714 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a0d4cf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5a080c86f0 0x7f5a080a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.714 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a0d4cf700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080c7e30 con 0x7f5a081547f0 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a0d4cf700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a080bcf40 con 0x7f5a08062700 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a0d4cf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a080c6b20 con 0x7f5a080c86f0 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a077fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5a08062700 0x7f5a080a4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:59050/0 (socket says 172.21.15.150:59050) 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.711+0000 7f5a077fe700 1 -- 172.21.15.150:0/2097149704 learned_addr learned my addr 172.21.15.150:0/2097149704 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a067fc700 1 --2- 172.21.15.150:0/2097149704 >> v2:172.21.15.73:3300/0 conn(0x7f5a080c86f0 0x7f5a080a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1906098214 0 0) 0x7f5a080bcf40 con 0x7f5a08062700 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59f00033f0 con 0x7f5a08062700 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1118513279 0 0) 0x7f5a080c7e30 con 0x7f5a081547f0 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a080bcf40 con 0x7f5a081547f0 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.1 v1:172.21.15.150:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2779097758 0 0) 0x7f59f00033f0 con 0x7f5a08062700 2023-11-28T02:52:27.715 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a067fc700 1 -- 172.21.15.150:0/2097149704 >> v1:172.21.15.150:6789/0 conn(0x7f5a08062700 legacy=0x7f5a080a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a067fc700 1 -- 172.21.15.150:0/2097149704 >> v1:172.21.15.73:6789/0 conn(0x7f5a081547f0 legacy=0x7f5a080c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a067fc700 1 -- 172.21.15.150:0/2097149704 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a080a77d0 con 0x7f5a080c86f0 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a067fc700 1 --2- 172.21.15.150:0/2097149704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a080c86f0 0x7f5a080a72e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f59f80096e0 tx=0x7f59f8009060).ready entity=mon.0 client_cookie=862797fb0a39b4b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a000017f0 con 0x7f5a080c86f0 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59f80090d0 con 0x7f5a080c86f0 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.712+0000 7f5a05ffb700 1 -- 172.21.15.150:0/2097149704 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59f8011610 con 0x7f5a080c86f0 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/2097149704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a080c86f0 msgr2=0x7f5a080a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/2097149704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a080c86f0 0x7f5a080a72e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f59f80096e0 tx=0x7f59f8009060).stop 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/2097149704 shutdown_connections 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/2097149704 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a080c86f0 0x7f5a080a72e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/2097149704 >> 172.21.15.150:0/2097149704 conn(0x7f5a080642d0 msgr2=0x7f5a080c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/2097149704 shutdown_connections 2023-11-28T02:52:27.716 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.713+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/2097149704 wait complete. 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.714+0000 7f5a0d4cf700 1 Processor -- start 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.714+0000 7f5a0d4cf700 1 -- start start 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a080c86f0 0x7f5a080aaf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a081547f0 0x7f5a080a8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.719 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080a9a30 con 0x7f5a081547f0 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a08158490 con 0x7f5a08062700 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a0d4cf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a080a9470 con 0x7f5a080c86f0 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a067fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a080c86f0 0x7f5a080aaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36746/0 (socket says 172.21.15.150:36746) 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 -- 172.21.15.150:0/666532869 learned_addr learned my addr 172.21.15.150:0/666532869 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a077fe700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a081547f0 0x7f5a080a8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 -- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a080c86f0 msgr2=0x7f5a080aaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a080c86f0 0x7f5a080aaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 -- 172.21.15.150:0/666532869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a081547f0 msgr2=0x7f5a080a8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a081547f0 0x7f5a080a8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.720 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 -- 172.21.15.150:0/666532869 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59f80092a0 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f5a06ffd700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f59fc00c490 tx=0x7f59fc007c90).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.715+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00ce40 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59fc00f470 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a080a9750 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59fc00f810 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a08151d20 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f59fc01c440 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.716+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1099+0+0 (secure 0 0 0) 0x7f59fc00fe20 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.717+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a08157f80 con 0x7f5a08062700 2023-11-28T02:52:27.721 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.719+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f5a08157f80 con 0x7f5a08062700 2023-11-28T02:52:27.801 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.798+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "e0055829-e29c-4f7a-8a17-e21c21919c3a", "id": 3} v 0) v1 -- 0x7f5a0806c860 con 0x7f5a08062700 2023-11-28T02:52:27.808 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.807+0000 7f59eeffd700 1 -- 172.21.15.150:0/666532869 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e0055829-e29c-4f7a-8a17-e21c21919c3a", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f59fc017070 con 0x7f5a08062700 2023-11-28T02:52:27.809 INFO:teuthology.orchestra.run.smithi150.stdout:3 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 msgr2=0x7f5a08151830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f59fc00c490 tx=0x7f59fc007c90).stop 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 shutdown_connections 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a081547f0 0x7f5a080a8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a08062700 0x7f5a08151830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 --2- 172.21.15.150:0/666532869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a080c86f0 0x7f5a080aaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 >> 172.21.15.150:0/666532869 conn(0x7f5a080642d0 msgr2=0x7f5a080654b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.810 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 shutdown_connections 2023-11-28T02:52:27.811 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.808+0000 7f5a0d4cf700 1 -- 172.21.15.150:0/666532869 wait complete. 2023-11-28T02:52:27.821 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 1309ffa4-9d68-4414-8506-32d09d7b1818 4 2023-11-28T02:52:27.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.934+0000 7f6e71a83700 1 Processor -- start 2023-11-28T02:52:27.935 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.934+0000 7f6e71a83700 1 -- start start 2023-11-28T02:52:27.938 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e71a83700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6e6c158160 0x7f6e6c0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.938 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e71a83700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e6c14e600 0x7f6e6c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.938 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e71a83700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e6c067770 con 0x7f6e6c0a8280 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e71a83700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c159180 con 0x7f6e6c158160 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e71a83700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e6c0be8d0 con 0x7f6e6c14e600 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e6c14e600 0x7f6e6c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6e6c14e600 0x7f6e6c14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36756/0 (socket says 172.21.15.150:36756) 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 -- 172.21.15.150:0/1261327364 learned_addr learned my addr 172.21.15.150:0/1261327364 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6b7fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6e6c0a8280 0x7f6e6c155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:59066/0 (socket says 172.21.15.150:59066) 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6affd700 1 --2- 172.21.15.150:0/1261327364 >> v2:172.21.15.73:3300/0 conn(0x7f6e6c158160 0x7f6e6c0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 -- 172.21.15.150:0/1261327364 >> v2:172.21.15.73:3300/0 conn(0x7f6e6c158160 msgr2=0x7f6e6c0a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 --2- 172.21.15.150:0/1261327364 >> v2:172.21.15.73:3300/0 conn(0x7f6e6c158160 0x7f6e6c0a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.939 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 -- 172.21.15.150:0/1261327364 >> v1:172.21.15.150:6789/0 conn(0x7f6e6c0a8280 legacy=0x7f6e6c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6a7fc700 1 -- 172.21.15.150:0/1261327364 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1748297125 0 0) 0x7f6e6c067770 con 0x7f6e6c0a8280 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 -- 172.21.15.150:0/1261327364 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e6c158600 con 0x7f6e6c14e600 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6bfff700 1 --2- 172.21.15.150:0/1261327364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c14e600 0x7f6e6c14ea90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f6e6c067fe0 tx=0x7f6e64009840).ready entity=mon.1 client_cookie=d24e7b5d73d3e9bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6a7fc700 1 -- 172.21.15.150:0/1261327364 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400e070 con 0x7f6e6c14e600 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6a7fc700 1 -- 172.21.15.150:0/1261327364 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e6400ba20 con 0x7f6e6c14e600 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.935+0000 7f6e6a7fc700 1 -- 172.21.15.150:0/1261327364 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e6400bda0 con 0x7f6e6c14e600 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 -- 172.21.15.150:0/1261327364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c14e600 msgr2=0x7f6e6c14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 --2- 172.21.15.150:0/1261327364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c14e600 0x7f6e6c14ea90 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f6e6c067fe0 tx=0x7f6e64009840).stop 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 -- 172.21.15.150:0/1261327364 shutdown_connections 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 --2- 172.21.15.150:0/1261327364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c14e600 0x7f6e6c14ea90 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 --2- 172.21.15.150:0/1261327364 >> v2:172.21.15.73:3300/0 conn(0x7f6e6c158160 0x7f6e6c0a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.940 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 -- 172.21.15.150:0/1261327364 >> 172.21.15.150:0/1261327364 conn(0x7f6e6c062700 msgr2=0x7f6e6c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:27.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 -- 172.21.15.150:0/1261327364 shutdown_connections 2023-11-28T02:52:27.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.936+0000 7f6e71a83700 1 -- 172.21.15.150:0/1261327364 wait complete. 2023-11-28T02:52:27.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.937+0000 7f6e71a83700 1 Processor -- start 2023-11-28T02:52:27.941 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.937+0000 7f6e71a83700 1 -- start start 2023-11-28T02:52:27.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 0x7f6e6c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.943 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c158160 0x7f6e6c065080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c1562f0 con 0x7f6e6c158160 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0a8d30 con 0x7f6e6c0a8280 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e71a83700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e6c0a9010 con 0x7f6e6c14e600 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6affd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 0x7f6e6c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36770/0 (socket says 172.21.15.150:36770) 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 -- 172.21.15.150:0/4202568412 learned_addr learned my addr 172.21.15.150:0/4202568412 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6bfff700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c158160 0x7f6e6c065080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 -- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 msgr2=0x7f6e6c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 0x7f6e6c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.944 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 -- 172.21.15.150:0/4202568412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c158160 msgr2=0x7f6e6c065080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c158160 0x7f6e6c065080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6b7fe700 1 -- 172.21.15.150:0/4202568412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e64009ae0 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.938+0000 7f6e6affd700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 0x7f6e6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e6b7fe700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6e5c00c440 tx=0x7f6e5c007e30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0070d0 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e5c007230 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e5c0056e0 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e6c155f80 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e6c0a92f0 con 0x7f6e6c0a8280 2023-11-28T02:52:27.945 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f6e5c005d30 con 0x7f6e6c0a8280 2023-11-28T02:52:27.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.939+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1214+0+0 (secure 0 0 0) 0x7f6e5c013de0 con 0x7f6e6c0a8280 2023-11-28T02:52:27.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.940+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e6c06c860 con 0x7f6e6c0a8280 2023-11-28T02:52:27.946 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:27.942+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f6e5c018020 con 0x7f6e6c0a8280 2023-11-28T02:52:28.022 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.021+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "1309ffa4-9d68-4414-8506-32d09d7b1818", "id": 4} v 0) v1 -- 0x7f6e6c0c3c90 con 0x7f6e6c0a8280 2023-11-28T02:52:28.033 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.032+0000 7f6e537fe700 1 -- 172.21.15.150:0/4202568412 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1309ffa4-9d68-4414-8506-32d09d7b1818", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f6e5c013450 con 0x7f6e6c0a8280 2023-11-28T02:52:28.033 INFO:teuthology.orchestra.run.smithi150.stdout:4 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 msgr2=0x7f6e6c0ac0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 rx=0x7f6e5c00c440 tx=0x7f6e5c007e30).stop 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 shutdown_connections 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6e6c158160 0x7f6e6c065080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6e6c0a8280 0x7f6e6c0ac0f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 --2- 172.21.15.150:0/4202568412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6e6c14e600 0x7f6e6c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 >> 172.21.15.150:0/4202568412 conn(0x7f6e6c062700 msgr2=0x7f6e6c155500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 shutdown_connections 2023-11-28T02:52:28.035 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.033+0000 7f6e71a83700 1 -- 172.21.15.150:0/4202568412 wait complete. 2023-11-28T02:52:28.047 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 7b292980-e2bd-4fec-afa8-b0eb42ca4039 5 2023-11-28T02:52:28.107 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:28.106+0000 7efc5a8e3680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T02:52:28.159 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.158+0000 7fa7c0225700 1 Processor -- start 2023-11-28T02:52:28.160 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.158+0000 7fa7c0225700 1 -- start start 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7c0225700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa7b8098d10 0x7fa7b80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7c0225700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 0x7fa7b8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7c0225700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b80c72d0 con 0x7fa7b8064350 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7c0225700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b8154bc0 con 0x7fa7b8098d10 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7c0225700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b80c66c0 con 0x7fa7b80c6190 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7bd7c0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 0x7fa7b8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7bdfc1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa7b8098d10 0x7fa7b80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.162 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7be7c2700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa7b8064350 0x7fa7b8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:59082/0 (socket says 172.21.15.150:59082) 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.159+0000 7fa7be7c2700 1 -- 172.21.15.150:0/309567610 learned_addr learned my addr 172.21.15.150:0/309567610 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 msgr2=0x7fa7b8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bdfc1700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 0x7fa7b8063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/309567610 >> v1:172.21.15.150:6789/0 conn(0x7fa7b8064350 legacy=0x7fa7b8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/309567610 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b80c69a0 con 0x7fa7b8098d10 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bcfbf700 1 -- 172.21.15.150:0/309567610 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1986872433 0 0) 0x7fa7b80c72d0 con 0x7fa7b8064350 2023-11-28T02:52:28.163 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bd7c0700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 0x7fa7b8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bdfc1700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8098d10 0x7fa7b80c7760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa7a8009760 tx=0x7fa7a8009060).ready entity=mon.1 client_cookie=af1b6b9c2600afe6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bcfbf700 1 -- 172.21.15.150:0/309567610 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f510 con 0x7fa7b8098d10 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bcfbf700 1 -- 172.21.15.150:0/309567610 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a800f670 con 0x7fa7b8098d10 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7bcfbf700 1 -- 172.21.15.150:0/309567610 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f9f0 con 0x7fa7b8098d10 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7c0225700 1 -- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8098d10 msgr2=0x7fa7b80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.160+0000 7fa7c0225700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8098d10 0x7fa7b80c7760 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fa7a8009760 tx=0x7fa7a8009060).stop 2023-11-28T02:52:28.167 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 -- 172.21.15.150:0/309567610 shutdown_connections 2023-11-28T02:52:28.168 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b8098d10 0x7fa7b80c7760 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.168 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 --2- 172.21.15.150:0/309567610 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b80c6190 0x7fa7b8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.168 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 -- 172.21.15.150:0/309567610 >> 172.21.15.150:0/309567610 conn(0x7fa7b80cb310 msgr2=0x7fa7b80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.168 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 -- 172.21.15.150:0/309567610 shutdown_connections 2023-11-28T02:52:28.168 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.161+0000 7fa7c0225700 1 -- 172.21.15.150:0/309567610 wait complete. 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 Processor -- start 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 -- start start 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b8098d10 0x7fa7b8150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80c6190 0x7fa7b80b03e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8155fb0 con 0x7fa7b8098d10 2023-11-28T02:52:28.170 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.162+0000 7fa7c0225700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8156290 con 0x7fa7b80c6190 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7c0225700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b8156570 con 0x7fa7b8064350 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54618/0 (socket says 172.21.15.150:54618) 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7be7c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80c6190 0x7fa7b80b03e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/2398141981 learned_addr learned my addr 172.21.15.150:0/2398141981 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bd7c0700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b8098d10 0x7fa7b8150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80c6190 msgr2=0x7fa7b80b03e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80c6190 0x7fa7b80b03e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/2398141981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b8098d10 msgr2=0x7fa7b8150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b8098d10 0x7fa7b8150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 -- 172.21.15.150:0/2398141981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40092a0 con 0x7fa7b8064350 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7bdfc1700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa7b80c2360 tx=0x7fa7a8017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f7e0 con 0x7fa7b8064350 2023-11-28T02:52:28.171 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a800f940 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a801b660 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7a80092a0 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.163+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b8155cc0 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.164+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fa7a801bcb0 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.164+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1329+0+0 (secure 0 0 0) 0x7fa7a8023b90 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.164+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b806c860 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.166+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fa7a80235e0 con 0x7fa7b8064350 2023-11-28T02:52:28.172 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:28.171+0000 7efc5a8e3680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T02:52:28.265 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:28.263+0000 7f07e8d52680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T02:52:28.280 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.279+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7b292980-e2bd-4fec-afa8-b0eb42ca4039", "id": 5} v 0) v1 -- 0x7fa7b80c6c80 con 0x7fa7b8064350 2023-11-28T02:52:28.289 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.288+0000 7fa7adffb700 1 -- 172.21.15.150:0/2398141981 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7b292980-e2bd-4fec-afa8-b0eb42ca4039", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa7a8023780 con 0x7fa7b8064350 2023-11-28T02:52:28.290 INFO:teuthology.orchestra.run.smithi150.stdout:5 2023-11-28T02:52:28.290 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 msgr2=0x7fa7b81557d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.290 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7fa7b80c2360 tx=0x7fa7a8017000).stop 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 shutdown_connections 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b80c6190 0x7fa7b80b03e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b8064350 0x7fa7b81557d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 --2- 172.21.15.150:0/2398141981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b8098d10 0x7fa7b8150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 >> 172.21.15.150:0/2398141981 conn(0x7fa7b80cb310 msgr2=0x7fa7b81523f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 shutdown_connections 2023-11-28T02:52:28.291 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.289+0000 7fa7c0225700 1 -- 172.21.15.150:0/2398141981 wait complete. 2023-11-28T02:52:28.302 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new f96f63b5-0a1b-48b7-8f51-19e5684c1d6b 6 2023-11-28T02:52:28.338 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:28.337+0000 7f07e8d52680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T02:52:28.361 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:28.360+0000 7efc5a8e3680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T02:52:28.422 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.417+0000 7f06393c7700 1 Processor -- start 2023-11-28T02:52:28.422 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 -- start start 2023-11-28T02:52:28.422 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 0x7f06340c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.422 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f06340c5ea0 0x7f0634066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0634155f20 con 0x7f0634153390 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0634153990 con 0x7f06340c5ea0 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f06393c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0634156200 con 0x7f06340c5ac0 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.418+0000 7f0632ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 0x7f06340c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f06340c5ea0 0x7f0634066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f06340c5ea0 0x7f0634066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36802/0 (socket says 172.21.15.150:36802) 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f0632ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 0x7f06340c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:54620/0 (socket says 172.21.15.150:54620) 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06327fc700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0634153390 0x7f0634156690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:59092/0 (socket says 172.21.15.150:59092) 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 -- 172.21.15.150:0/398168879 learned_addr learned my addr 172.21.15.150:0/398168879 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 -- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 msgr2=0x7f06340c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 --2- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 0x7f06340c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.423 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 -- 172.21.15.150:0/398168879 >> v1:172.21.15.150:6789/0 conn(0x7f0634153390 legacy=0x7f0634156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 -- 172.21.15.150:0/398168879 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06340bff70 con 0x7f06340c5ea0 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f0631ffb700 1 -- 172.21.15.150:0/398168879 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 417921024 0 0) 0x7f0634155f20 con 0x7f0634153390 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f06337fe700 1 --2- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ea0 0x7f0634066a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f062c009f20 tx=0x7f062c009840).ready entity=mon.1 client_cookie=e4a882c4d3a416fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f0631ffb700 1 -- 172.21.15.150:0/398168879 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00e070 con 0x7f06340c5ea0 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f0631ffb700 1 -- 172.21.15.150:0/398168879 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062c00b9d0 con 0x7f06340c5ea0 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.419+0000 7f0631ffb700 1 -- 172.21.15.150:0/398168879 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062c00bd50 con 0x7f06340c5ea0 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 -- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ea0 msgr2=0x7f0634066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 --2- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ea0 0x7f0634066a70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 rx=0x7f062c009f20 tx=0x7f062c009840).stop 2023-11-28T02:52:28.424 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 -- 172.21.15.150:0/398168879 shutdown_connections 2023-11-28T02:52:28.425 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 --2- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ea0 0x7f0634066a70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.425 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 --2- 172.21.15.150:0/398168879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ac0 0x7f06340c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.425 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 -- 172.21.15.150:0/398168879 >> 172.21.15.150:0/398168879 conn(0x7f0634062700 msgr2=0x7f0634062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.425 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 -- 172.21.15.150:0/398168879 shutdown_connections 2023-11-28T02:52:28.425 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.420+0000 7f06393c7700 1 -- 172.21.15.150:0/398168879 wait complete. 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.422+0000 7f06393c7700 1 Processor -- start 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.422+0000 7f06393c7700 1 -- start start 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 0x7f0634152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 0x7f063414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063414d280 con 0x7f0634153390 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0634165610 con 0x7f06340c5ac0 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0634162540 con 0x7f06340c5ea0 2023-11-28T02:52:28.428 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36806/0 (socket says 172.21.15.150:36806) 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 -- 172.21.15.150:0/3277823892 learned_addr learned my addr 172.21.15.150:0/3277823892 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06327fc700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 0x7f0634152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06337fe700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 0x7f063414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 -- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 msgr2=0x7f0634152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 0x7f0634152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 -- 172.21.15.150:0/3277823892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 msgr2=0x7f063414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 0x7f063414c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 -- 172.21.15.150:0/3277823892 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c009ae0 con 0x7f06340c5ac0 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06337fe700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 0x7f063414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06327fc700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 0x7f0634152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f0632ffd700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f062800e930 tx=0x7f0628009060).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.429 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800c670 con 0x7f06340c5ac0 2023-11-28T02:52:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f062800c7d0 con 0x7f06340c5ac0 2023-11-28T02:52:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634162820 con 0x7f06340c5ac0 2023-11-28T02:52:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f062800cb50 con 0x7f06340c5ac0 2023-11-28T02:52:28.430 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.424+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063414cc90 con 0x7f06340c5ac0 2023-11-28T02:52:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.425+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f062801b660 con 0x7f06340c5ac0 2023-11-28T02:52:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.425+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1444+0+0 (secure 0 0 0) 0x7f0628012a10 con 0x7f06340c5ac0 2023-11-28T02:52:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.426+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f063406c860 con 0x7f06340c5ac0 2023-11-28T02:52:28.431 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.428+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f0628016020 con 0x7f06340c5ac0 2023-11-28T02:52:28.509 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.507+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "f96f63b5-0a1b-48b7-8f51-19e5684c1d6b", "id": 6} v 0) v1 -- 0x7f06340c4f80 con 0x7f06340c5ac0 2023-11-28T02:52:28.515 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.513+0000 7f061affd700 1 -- 172.21.15.150:0/3277823892 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f96f63b5-0a1b-48b7-8f51-19e5684c1d6b", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0628012ca0 con 0x7f06340c5ac0 2023-11-28T02:52:28.515 INFO:teuthology.orchestra.run.smithi150.stdout:6 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 msgr2=0x7f0634161ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f062800e930 tx=0x7f0628009060).stop 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 shutdown_connections 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0634153390 0x7f063414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06340c5ac0 0x7f0634161ed0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 --2- 172.21.15.150:0/3277823892 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06340c5ea0 0x7f0634152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.516 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 >> 172.21.15.150:0/3277823892 conn(0x7f0634062700 msgr2=0x7f06340c3880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.517 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 shutdown_connections 2023-11-28T02:52:28.517 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.514+0000 7f06393c7700 1 -- 172.21.15.150:0/3277823892 wait complete. 2023-11-28T02:52:28.527 DEBUG:teuthology.orchestra.run.smithi150:> sudo ceph --cluster ceph osd new 415767ad-b315-4c70-ad94-d94695868e80 7 2023-11-28T02:52:28.539 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:28.536+0000 7f07e8d52680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T02:52:28.653 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.652+0000 7f98650d6700 1 Processor -- start 2023-11-28T02:52:28.653 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.652+0000 7f98650d6700 1 -- start start 2023-11-28T02:52:28.655 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f98650d6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9860098d10 0x7f98600c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f98650d6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98600c6190 0x7f9860063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f98650d6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600c72d0 con 0x7f9860064350 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f98650d6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9860154bc0 con 0x7f98600c6190 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f98650d6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600c66c0 con 0x7f9860098d10 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9860098d10 0x7f98600c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985f59e700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9860064350 0x7f9860158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:59096/0 (socket says 172.21.15.150:59096) 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985f59e700 1 -- 172.21.15.150:0/1118015186 learned_addr learned my addr 172.21.15.150:0/1118015186 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985e59c700 1 --2- 172.21.15.150:0/1118015186 >> v2:172.21.15.73:3300/0 conn(0x7f98600c6190 0x7f9860063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 -- 172.21.15.150:0/1118015186 >> v2:172.21.15.73:3300/0 conn(0x7f98600c6190 msgr2=0x7f9860063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 --2- 172.21.15.150:0/1118015186 >> v2:172.21.15.73:3300/0 conn(0x7f98600c6190 0x7f9860063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.656 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 -- 172.21.15.150:0/1118015186 >> v1:172.21.15.150:6789/0 conn(0x7f9860064350 legacy=0x7f9860158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985dd9b700 1 -- 172.21.15.150:0/1118015186 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1100429991 0 0) 0x7f98600c72d0 con 0x7f9860064350 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 -- 172.21.15.150:0/1118015186 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600c69a0 con 0x7f9860098d10 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985ed9d700 1 --2- 172.21.15.150:0/1118015186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860098d10 0x7f98600c7760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9860083f80 tx=0x7f9850009060).ready entity=mon.1 client_cookie=75ad73934649a417 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985dd9b700 1 -- 172.21.15.150:0/1118015186 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000f510 con 0x7f9860098d10 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985dd9b700 1 -- 172.21.15.150:0/1118015186 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000f670 con 0x7f9860098d10 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.653+0000 7f985dd9b700 1 -- 172.21.15.150:0/1118015186 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000f9f0 con 0x7f9860098d10 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 -- 172.21.15.150:0/1118015186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860098d10 msgr2=0x7f98600c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 --2- 172.21.15.150:0/1118015186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860098d10 0x7f98600c7760 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 rx=0x7f9860083f80 tx=0x7f9850009060).stop 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 -- 172.21.15.150:0/1118015186 shutdown_connections 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 --2- 172.21.15.150:0/1118015186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860098d10 0x7f98600c7760 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 --2- 172.21.15.150:0/1118015186 >> v2:172.21.15.73:3300/0 conn(0x7f98600c6190 0x7f9860063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.657 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 -- 172.21.15.150:0/1118015186 >> 172.21.15.150:0/1118015186 conn(0x7f98600cb310 msgr2=0x7f98600cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.658 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 -- 172.21.15.150:0/1118015186 shutdown_connections 2023-11-28T02:52:28.658 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.654+0000 7f98650d6700 1 -- 172.21.15.150:0/1118015186 wait complete. 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.655+0000 7f98650d6700 1 Processor -- start 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.655+0000 7f98650d6700 1 -- start start 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 0x7f98600c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98600c6190 0x7f9860150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.660 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98600ac6e0 con 0x7f9860098d10 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9860155b10 con 0x7f9860064350 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f98650d6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600c4f10 con 0x7f98600c6190 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f985ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f985ed9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.150:36810/0 (socket says 172.21.15.150:36810) 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f985ed9d700 1 -- 172.21.15.150:0/698489160 learned_addr learned my addr 172.21.15.150:0/698489160 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f985f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98600c6190 0x7f9860150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.656+0000 7f985e59c700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 0x7f98600c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 -- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98600c6190 msgr2=0x7f9860150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98600c6190 0x7f9860150970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 -- 172.21.15.150:0/698489160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 msgr2=0x7f98600c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 0x7f98600c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.661 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 -- 172.21.15.150:0/698489160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98500092a0 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985e59c700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 0x7f98600c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f985ed9d700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f9850009dd0 tx=0x7f9850017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985000f7e0 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f985000f940 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9860154f00 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f985001b660 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98601554c0 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f985001bc30 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.657+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1559+0+0 (secure 0 0 0) 0x7f9850023d60 con 0x7f9860064350 2023-11-28T02:52:28.662 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.658+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98600641b0 con 0x7f9860064350 2023-11-28T02:52:28.663 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.660+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f98600641b0 con 0x7f9860064350 2023-11-28T02:52:28.742 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.741+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd new", "uuid": "415767ad-b315-4c70-ad94-d94695868e80", "id": 7} v 0) v1 -- 0x7f986006c860 con 0x7f9860064350 2023-11-28T02:52:28.748 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.746+0000 7f9846ffd700 1 -- 172.21.15.150:0/698489160 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "415767ad-b315-4c70-ad94-d94695868e80", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f98500236d0 con 0x7f9860064350 2023-11-28T02:52:28.748 INFO:teuthology.orchestra.run.smithi150.stdout:7 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 msgr2=0x7f98600c3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f9850009dd0 tx=0x7f9850017000).stop 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 shutdown_connections 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98600c6190 0x7f9860150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9860064350 0x7f98600c3a00 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 --2- 172.21.15.150:0/698489160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9860098d10 0x7f98600c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.749 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 >> 172.21.15.150:0/698489160 conn(0x7f98600cb310 msgr2=0x7f98600aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.750 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.747+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 shutdown_connections 2023-11-28T02:52:28.750 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:52:28.748+0000 7f98650d6700 1 -- 172.21.15.150:0/698489160 wait complete. 2023-11-28T02:52:28.765 INFO:tasks.ceph.osd.0:Restarting daemon 2023-11-28T02:52:28.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-11-28T02:52:28.767 INFO:tasks.ceph.osd.0:Started 2023-11-28T02:52:28.767 INFO:tasks.ceph.osd.1:Restarting daemon 2023-11-28T02:52:28.768 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-11-28T02:52:28.769 INFO:tasks.ceph.osd.1:Started 2023-11-28T02:52:28.769 INFO:tasks.ceph.osd.2:Restarting daemon 2023-11-28T02:52:28.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-11-28T02:52:28.771 INFO:tasks.ceph.osd.2:Started 2023-11-28T02:52:28.771 INFO:tasks.ceph.osd.3:Restarting daemon 2023-11-28T02:52:28.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-11-28T02:52:28.773 INFO:tasks.ceph.osd.3:Started 2023-11-28T02:52:28.773 INFO:tasks.ceph.osd.4:Restarting daemon 2023-11-28T02:52:28.774 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-11-28T02:52:28.776 INFO:tasks.ceph.osd.4:Started 2023-11-28T02:52:28.776 INFO:tasks.ceph.osd.5:Restarting daemon 2023-11-28T02:52:28.776 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-11-28T02:52:28.778 INFO:tasks.ceph.osd.5:Started 2023-11-28T02:52:28.778 INFO:tasks.ceph.osd.6:Restarting daemon 2023-11-28T02:52:28.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-11-28T02:52:28.780 INFO:tasks.ceph.osd.6:Started 2023-11-28T02:52:28.780 INFO:tasks.ceph.osd.7:Restarting daemon 2023-11-28T02:52:28.780 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-11-28T02:52:28.783 INFO:tasks.ceph.osd.7:Started 2023-11-28T02:52:28.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:28.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:28.817+0000 7efc5a8e3680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T02:52:28.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.928+0000 7f88d9dea700 1 Processor -- start 2023-11-28T02:52:28.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.928+0000 7f88d9dea700 1 -- start start 2023-11-28T02:52:28.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d9dea700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f88d4157660 0x7f88d4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d9dea700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 0x7f88d40c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d9dea700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88d40c12c0 con 0x7f88d40c0180 2023-11-28T02:52:28.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d9dea700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88d40c06b0 con 0x7f88d4157660 2023-11-28T02:52:28.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d9dea700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40c0990 con 0x7f88d40c1750 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f88d40c0180 0x7f88d41589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60656/0 (socket says 172.21.15.73:60656) 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d3fff700 1 -- 172.21.15.73:0/841245678 learned_addr learned my addr 172.21.15.73:0/841245678 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d95e9700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 0x7f88d40c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d8de8700 1 --2- 172.21.15.73:0/841245678 >> v2:172.21.15.150:3300/0 conn(0x7f88d4157660 0x7f88d4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d37fe700 1 -- 172.21.15.73:0/841245678 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662506047 0 0) 0x7f88d40c12c0 con 0x7f88d40c0180 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.929+0000 7f88d37fe700 1 -- 172.21.15.73:0/841245678 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88c00033f0 con 0x7f88d40c0180 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d8de8700 1 -- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 msgr2=0x7f88d40c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d8de8700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 0x7f88d40c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d8de8700 1 -- 172.21.15.73:0/841245678 >> v1:172.21.15.73:6789/0 conn(0x7f88d40c0180 legacy=0x7f88d41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d8de8700 1 -- 172.21.15.73:0/841245678 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88d40c0c70 con 0x7f88d4157660 2023-11-28T02:52:28.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d95e9700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 0x7f88d40c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d8de8700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d4157660 0x7f88d4157a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f88cc009760 tx=0x7f88cc009060).ready entity=mon.1 client_cookie=147524bfdbab3623 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d37fe700 1 -- 172.21.15.73:0/841245678 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00d4a0 con 0x7f88d4157660 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d37fe700 1 -- 172.21.15.73:0/841245678 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88cc00d600 con 0x7f88d4157660 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d37fe700 1 -- 172.21.15.73:0/841245678 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88cc00d980 con 0x7f88d4157660 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 -- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d4157660 msgr2=0x7f88d4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d4157660 0x7f88d4157a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f88cc009760 tx=0x7f88cc009060).stop 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 -- 172.21.15.73:0/841245678 shutdown_connections 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d40c1750 0x7f88d40c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 --2- 172.21.15.73:0/841245678 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d4157660 0x7f88d4157a40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.930+0000 7f88d9dea700 1 -- 172.21.15.73:0/841245678 >> 172.21.15.73:0/841245678 conn(0x7f88d40cb000 msgr2=0x7f88d40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.931+0000 7f88d9dea700 1 -- 172.21.15.73:0/841245678 shutdown_connections 2023-11-28T02:52:28.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.931+0000 7f88d9dea700 1 -- 172.21.15.73:0/841245678 wait complete. 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.932+0000 7f88d9dea700 1 Processor -- start 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.932+0000 7f88d9dea700 1 -- start start 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.932+0000 7f88d9dea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d40c0180 0x7f88d40a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.932+0000 7f88d9dea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d4157660 0x7f88d4154460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4153d70 con 0x7f88d40c1750 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88d4154fd0 con 0x7f88d40c0180 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88d40c32d0 con 0x7f88d4157660 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d95e9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d4157660 0x7f88d4154460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45330/0 (socket says 172.21.15.73:45330) 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 -- 172.21.15.73:0/651875554 learned_addr learned my addr 172.21.15.73:0/651875554 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d8de8700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d40c0180 0x7f88d40a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 -- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d4157660 msgr2=0x7f88d4154460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d4157660 0x7f88d4154460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 -- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d40c0180 msgr2=0x7f88d40a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d40c0180 0x7f88d40a94b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 -- 172.21.15.73:0/651875554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88bc0092a0 con 0x7f88d40c1750 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d3fff700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f88c400c7c0 tx=0x7f88c40015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4005d30 con 0x7f88d40c1750 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88c4005e90 con 0x7f88d40c1750 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88cc0092a0 con 0x7f88d40c1750 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88c4007340 con 0x7f88d40c1750 2023-11-28T02:52:28.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88d40c3890 con 0x7f88d40c1750 2023-11-28T02:52:28.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.933+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7f88c401d440 con 0x7f88d40c1750 2023-11-28T02:52:28.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.934+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7f88c4013b10 con 0x7f88d40c1750 2023-11-28T02:52:28.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.934+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88d406c7c0 con 0x7f88d40c1750 2023-11-28T02:52:28.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:28.936+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7f88c4013e30 con 0x7f88d40c1750 2023-11-28T02:52:28.942 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:28.941+0000 7efc5a8e3680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:52:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.004+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f88d40c3ed0 con 0x7f88d40c1750 2023-11-28T02:52:29.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.005+0000 7f88d0ff9700 1 -- 172.21.15.73:0/651875554 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7f88d40c3ed0 con 0x7f88d40c1750 2023-11-28T02:52:29.006 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:29.007 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":10,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:28.743519+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:29.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.005+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 msgr2=0x7f88d40c4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.005+0000 7f88d9dea700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f88c400c7c0 tx=0x7f88c40015b0).stop 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 shutdown_connections 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f88d4157660 0x7f88d4154460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f88d40c0180 0x7f88d40a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 --2- 172.21.15.73:0/651875554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f88d40c1750 0x7f88d40c4090 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 >> 172.21.15.73:0/651875554 conn(0x7f88d40cb000 msgr2=0x7f88d40aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 shutdown_connections 2023-11-28T02:52:29.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:29.006+0000 7f88d9dea700 1 -- 172.21.15.73:0/651875554 wait complete. 2023-11-28T02:52:29.019 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.018+0000 7f07e8d52680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T02:52:29.020 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-11-28T02:52:29.020 INFO:tasks.ceph:Waiting for OSDs to come up 2023-11-28T02:52:29.098 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:29.097+0000 7efc5a8e3680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T02:52:29.122 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.121+0000 7f07e8d52680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:52:29.176 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:29.175+0000 7efc5a8e3680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T02:52:29.245 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:29.243+0000 7efc5a8e3680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T02:52:29.278 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.276+0000 7f07e8d52680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T02:52:29.357 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.355+0000 7f07e8d52680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T02:52:29.370 INFO:tasks.ceph.osd.0.smithi073.stderr:2023-11-28T02:52:29.368+0000 7f20d55ef380 -1 Falling back to public interface 2023-11-28T02:52:29.373 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-28T02:52:29.372+0000 7f9a3a4e6380 -1 Falling back to public interface 2023-11-28T02:52:29.377 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-28T02:52:29.376+0000 7effa837c380 -1 Falling back to public interface 2023-11-28T02:52:29.378 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-28T02:52:29.376+0000 7f7583f79380 -1 Falling back to public interface 2023-11-28T02:52:29.378 INFO:tasks.ceph.osd.2.smithi073.stderr:2023-11-28T02:52:29.377+0000 7f592dd2a380 -1 Falling back to public interface 2023-11-28T02:52:29.379 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-28T02:52:29.378+0000 7effa2541700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6817/0: (98) Address already in use 2023-11-28T02:52:29.382 INFO:tasks.ceph.osd.3.smithi073.stderr:2023-11-28T02:52:29.381+0000 7f635cd62380 -1 Falling back to public interface 2023-11-28T02:52:29.386 INFO:tasks.ceph.osd.1.smithi073.stderr:2023-11-28T02:52:29.385+0000 7f9bf44e1380 -1 Falling back to public interface 2023-11-28T02:52:29.403 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-28T02:52:29.402+0000 7fa6f102a380 -1 Falling back to public interface 2023-11-28T02:52:29.432 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.431+0000 7f07e8d52680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T02:52:29.513 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:29.512+0000 7efc5a8e3680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T02:52:29.699 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.698+0000 7f07e8d52680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T02:52:29.718 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:29.717+0000 7efc5a8e3680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T02:52:29.828 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:29.897 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:29.895+0000 7f07e8d52680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T02:52:29.934 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.933+0000 7fc6611a2700 1 Processor -- start 2023-11-28T02:52:29.935 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.933+0000 7fc6611a2700 1 -- start start 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc6611a2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc65c0c50a0 0x7fc65c15f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc6611a2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c0c3570 con 0x7fc65c154720 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc6611a2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c065bc0 con 0x7fc65c154340 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc6611a2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c066460 con 0x7fc65c0c50a0 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc65c0c50a0 0x7fc65c15f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc65c0c50a0 0x7fc65c15f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45342/0 (socket says 172.21.15.73:45342) 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65a59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc65c154720 0x7fc65c0c4990 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60662/0 (socket says 172.21.15.73:60662) 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 -- 172.21.15.73:0/1404456813 learned_addr learned my addr 172.21.15.73:0/1404456813 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:29.937 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 -- 172.21.15.73:0/1404456813 >> v1:172.21.15.150:6789/0 conn(0x7fc65c154340 legacy=0x7fc65c062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 -- 172.21.15.73:0/1404456813 >> v1:172.21.15.73:6789/0 conn(0x7fc65c154720 legacy=0x7fc65c0c4990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc65b59e700 1 -- 172.21.15.73:0/1404456813 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0631a0 con 0x7fc65c0c50a0 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.934+0000 7fc659d9b700 1 -- 172.21.15.73:0/1404456813 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 526719529 0 0) 0x7fc644001110 con 0x7fc65c154720 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc65b59e700 1 --2- 172.21.15.73:0/1404456813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c0c50a0 0x7fc65c15f120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc64c006fd0 tx=0x7fc64c0090f0).ready entity=mon.0 client_cookie=16a5b2e956738ce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc659d9b700 1 -- 172.21.15.73:0/1404456813 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00b600 con 0x7fc65c0c50a0 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc659d9b700 1 -- 172.21.15.73:0/1404456813 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c00b760 con 0x7fc65c0c50a0 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc659d9b700 1 -- 172.21.15.73:0/1404456813 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c00bae0 con 0x7fc65c0c50a0 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 -- 172.21.15.73:0/1404456813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c0c50a0 msgr2=0x7fc65c15f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 --2- 172.21.15.73:0/1404456813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c0c50a0 0x7fc65c15f120 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc64c006fd0 tx=0x7fc64c0090f0).stop 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 -- 172.21.15.73:0/1404456813 shutdown_connections 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 --2- 172.21.15.73:0/1404456813 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c0c50a0 0x7fc65c15f120 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.938 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 -- 172.21.15.73:0/1404456813 >> 172.21.15.73:0/1404456813 conn(0x7fc65c068770 msgr2=0x7fc65c068b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:29.939 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 -- 172.21.15.73:0/1404456813 shutdown_connections 2023-11-28T02:52:29.939 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.935+0000 7fc6611a2700 1 -- 172.21.15.73:0/1404456813 wait complete. 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 Processor -- start 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 -- start start 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc65c0c50a0 0x7fc65c141920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc65c154720 0x7fc65c157a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0abcc0 con 0x7fc65c154340 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0ab0b0 con 0x7fc65c154720 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc6611a2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c0ab390 con 0x7fc65c0c50a0 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc65a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:29.941 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc65a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45344/0 (socket says 172.21.15.73:45344) 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc65a59c700 1 -- 172.21.15.73:0/825097555 learned_addr learned my addr 172.21.15.73:0/825097555 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.937+0000 7fc65ad9d700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc65c0c50a0 0x7fc65c141920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65b59e700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc65c154720 0x7fc65c157a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 -- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc65c0c50a0 msgr2=0x7fc65c141920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc65c0c50a0 0x7fc65c141920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 -- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc65c154720 msgr2=0x7fc65c157a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc65c154720 0x7fc65c157a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 -- 172.21.15.73:0/825097555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c009370 con 0x7fc65c154340 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc65a59c700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc64400c490 tx=0x7fc644007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440070d0 con 0x7fc65c154340 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc644007250 con 0x7fc65c154340 2023-11-28T02:52:29.942 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c0ab670 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6440056e0 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c14dbb0 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 1) v1 ==== 495+0+0 (secure 0 0 0) 0x7fc644013450 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.938+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1674+0+0 (secure 0 0 0) 0x7fc64401db40 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.939+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c06c7c0 con 0x7fc65c154340 2023-11-28T02:52:29.943 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:29.941+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73265 (secure 0 0 0) 0x7fc644018070 con 0x7fc65c154340 2023-11-28T02:52:30.009 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.008+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc65c0abe00 con 0x7fc65c154340 2023-11-28T02:52:30.009 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.008+0000 7fc652ffd700 1 -- 172.21.15.73:0/825097555 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7045 (secure 0 0 0) 0x7fc65c0c3570 con 0x7fc65c154340 2023-11-28T02:52:30.010 INFO:teuthology.misc.health.smithi073.stdout: 2023-11-28T02:52:30.010 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":10,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:28.743519+0000","last_up_change":"0.000000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 msgr2=0x7fc65c157540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc64400c490 tx=0x7fc644007db0).stop 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 shutdown_connections 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc65c154720 0x7fc65c157a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc65c0c50a0 0x7fc65c141920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 --2- 172.21.15.73:0/825097555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc65c154340 0x7fc65c157540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 >> 172.21.15.73:0/825097555 conn(0x7fc65c068770 msgr2=0x7fc65c0c7300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:30.011 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 shutdown_connections 2023-11-28T02:52:30.012 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:30.009+0000 7fc6611a2700 1 -- 172.21.15.73:0/825097555 wait complete. 2023-11-28T02:52:30.021 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-11-28T02:52:30.389 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.387+0000 7efc5a8e3680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T02:52:30.457 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.456+0000 7efc5a8e3680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T02:52:30.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.535+0000 7efc5a8e3680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T02:52:30.572 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:30.571+0000 7f07e8d52680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T02:52:30.642 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:30.641+0000 7f07e8d52680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T02:52:30.683 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.681+0000 7efc5a8e3680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T02:52:30.721 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:30.720+0000 7f07e8d52680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T02:52:30.754 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.753+0000 7efc5a8e3680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T02:52:30.869 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.868+0000 7efc5a8e3680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T02:52:30.872 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:30.871+0000 7f07e8d52680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T02:52:30.944 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:30.943+0000 7f07e8d52680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T02:52:30.976 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:30.975+0000 7efc5a8e3680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:52:31.062 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:31.061+0000 7f07e8d52680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T02:52:31.172 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:31.170+0000 7f07e8d52680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:52:31.296 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:31.295+0000 7efc5a8e3680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T02:52:31.368 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:31.366+0000 7efc5a8e3680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T02:52:31.498 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:31.497+0000 7f07e8d52680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T02:52:31.570 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:52:31.569+0000 7f07e8d52680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T02:52:32.394 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:32.393+0000 7efc10b06700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-11-28T02:52:32.396 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:52:32.395+0000 7efc10b06700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-11-28T02:52:32.849 INFO:tasks.ceph.osd.2.smithi073.stderr:2023-11-28T02:52:32.848+0000 7f592dd2a380 -1 osd.2 0 log_to_monitors {default=true} 2023-11-28T02:52:32.855 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-28T02:52:32.854+0000 7f9a3a4e6380 -1 osd.5 0 log_to_monitors {default=true} 2023-11-28T02:52:32.898 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-28T02:52:32.897+0000 7f7583f79380 -1 osd.7 0 log_to_monitors {default=true} 2023-11-28T02:52:32.906 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-28T02:52:32.903+0000 7fa6f102a380 -1 osd.6 0 log_to_monitors {default=true} 2023-11-28T02:52:32.912 INFO:tasks.ceph.osd.0.smithi073.stderr:2023-11-28T02:52:32.911+0000 7f20d55ef380 -1 osd.0 0 log_to_monitors {default=true} 2023-11-28T02:52:32.912 INFO:tasks.ceph.osd.1.smithi073.stderr:2023-11-28T02:52:32.911+0000 7f9bf44e1380 -1 osd.1 0 log_to_monitors {default=true} 2023-11-28T02:52:32.913 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-28T02:52:32.912+0000 7effa837c380 -1 osd.4 0 log_to_monitors {default=true} 2023-11-28T02:52:32.925 INFO:tasks.ceph.osd.3.smithi073.stderr:2023-11-28T02:52:32.924+0000 7f635cd62380 -1 osd.3 0 log_to_monitors {default=true} 2023-11-28T02:52:34.399 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-28T02:52:34.398+0000 7fa6e79e8700 -1 osd.6 0 waiting for initial osdmap 2023-11-28T02:52:34.400 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-28T02:52:34.398+0000 7eff9ed3a700 -1 osd.4 0 waiting for initial osdmap 2023-11-28T02:52:34.400 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-28T02:52:34.398+0000 7f757a937700 -1 osd.7 0 waiting for initial osdmap 2023-11-28T02:52:34.401 INFO:tasks.ceph.osd.1.smithi073.stderr:2023-11-28T02:52:34.398+0000 7f9beae9f700 -1 osd.1 0 waiting for initial osdmap 2023-11-28T02:52:34.401 INFO:tasks.ceph.osd.2.smithi073.stderr:2023-11-28T02:52:34.399+0000 7f59246e8700 -1 osd.2 0 waiting for initial osdmap 2023-11-28T02:52:34.402 INFO:tasks.ceph.osd.0.smithi073.stderr:2023-11-28T02:52:34.400+0000 7f20cbfad700 -1 osd.0 0 waiting for initial osdmap 2023-11-28T02:52:34.402 INFO:tasks.ceph.osd.3.smithi073.stderr:2023-11-28T02:52:34.401+0000 7f6353720700 -1 osd.3 0 waiting for initial osdmap 2023-11-28T02:52:34.405 INFO:tasks.ceph.osd.4.smithi150.stderr:2023-11-28T02:52:34.404+0000 7eff9b4b6700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.406 INFO:tasks.ceph.osd.6.smithi150.stderr:2023-11-28T02:52:34.404+0000 7fa6e4164700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.406 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-28T02:52:34.405+0000 7f9a30ea4700 -1 osd.5 0 waiting for initial osdmap 2023-11-28T02:52:34.407 INFO:tasks.ceph.osd.7.smithi150.stderr:2023-11-28T02:52:34.405+0000 7f75770b3700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.411 INFO:tasks.ceph.osd.5.smithi150.stderr:2023-11-28T02:52:34.410+0000 7f9a2d620700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.412 INFO:tasks.ceph.osd.2.smithi073.stderr:2023-11-28T02:52:34.411+0000 7f5920e64700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.413 INFO:tasks.ceph.osd.3.smithi073.stderr:2023-11-28T02:52:34.411+0000 7f634fe9c700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.414 INFO:tasks.ceph.osd.1.smithi073.stderr:2023-11-28T02:52:34.411+0000 7f9be761b700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:34.414 INFO:tasks.ceph.osd.0.smithi073.stderr:2023-11-28T02:52:34.411+0000 7f20c8729700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-11-28T02:52:36.828 DEBUG:teuthology.orchestra.run.smithi073:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:36.944 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.944+0000 7f76c3923700 1 Processor -- start 2023-11-28T02:52:36.944 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.944+0000 7f76c3923700 1 -- start start 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c3923700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 0x7f76bc153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c3923700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc158aa0 con 0x7f76bc154490 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c3923700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76bc158170 con 0x7f76bc0aa360 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c3923700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc158450 con 0x7f76bc0aa740 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c1ec0700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f76bc154490 0x7f76bc0c08d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36176/0 (socket says 172.21.15.73:36176) 2023-11-28T02:52:36.947 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c1ec0700 1 -- 172.21.15.73:0/1739971748 learned_addr learned my addr 172.21.15.73:0/1739971748 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c0ebe700 1 --2- 172.21.15.73:0/1739971748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 0x7f76bc153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841726103 0 0) 0x7f76bc158aa0 con 0x7f76bc154490 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76a80033f0 con 0x7f76bc154490 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137665273 0 0) 0x7f76bc158170 con 0x7f76bc0aa360 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76bc158aa0 con 0x7f76bc0aa360 2023-11-28T02:52:36.948 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c0ebe700 1 -- 172.21.15.73:0/1739971748 >> v1:172.21.15.150:6789/0 conn(0x7f76bc0aa360 legacy=0x7f76bc0c6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c0ebe700 1 -- 172.21.15.73:0/1739971748 >> v1:172.21.15.73:6789/0 conn(0x7f76bc154490 legacy=0x7f76bc0c08d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3263201648 0 0) 0x7f76a80033f0 con 0x7f76bc154490 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c0ebe700 1 -- 172.21.15.73:0/1739971748 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc158730 con 0x7f76bc0aa740 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.945+0000 7f76c0ebe700 1 --2- 172.21.15.73:0/1739971748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 0x7f76bc153fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f76ac009790 tx=0x7f76ac009060).ready entity=mon.2 client_cookie=6846c4515aebe59e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40017f0 con 0x7f76bc0aa740 2023-11-28T02:52:36.949 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac011400 con 0x7f76bc0aa740 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76b3fff700 1 -- 172.21.15.73:0/1739971748 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac011780 con 0x7f76bc0aa740 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 -- 172.21.15.73:0/1739971748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 msgr2=0x7f76bc153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 --2- 172.21.15.73:0/1739971748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 0x7f76bc153fa0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 rx=0x7f76ac009790 tx=0x7f76ac009060).stop 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 -- 172.21.15.73:0/1739971748 shutdown_connections 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 --2- 172.21.15.73:0/1739971748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa740 0x7f76bc153fa0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:36.950 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 -- 172.21.15.73:0/1739971748 >> 172.21.15.73:0/1739971748 conn(0x7f76bc0cb270 msgr2=0x7f76bc0cb680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:36.951 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 -- 172.21.15.73:0/1739971748 shutdown_connections 2023-11-28T02:52:36.951 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.946+0000 7f76c3923700 1 -- 172.21.15.73:0/1739971748 wait complete. 2023-11-28T02:52:36.953 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.947+0000 7f76c3923700 1 Processor -- start 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.947+0000 7f76c3923700 1 -- start start 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 0x7f76bc153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76bc0aa740 0x7f76bc157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc155b40 con 0x7f76bc154490 2023-11-28T02:52:36.954 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc154f30 con 0x7f76bc0aa740 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c3923700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76bc155210 con 0x7f76bc0aa360 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c1ec0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c1ec0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60612/0 (socket says 172.21.15.73:60612) 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.948+0000 7f76c1ec0700 1 -- 172.21.15.73:0/1555128692 learned_addr learned my addr 172.21.15.73:0/1555128692 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c0ebe700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76bc0aa740 0x7f76bc157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c16bf700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 0x7f76bc153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:36.955 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 -- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 msgr2=0x7f76bc153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:36.956 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 0x7f76bc153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:36.956 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 -- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76bc0aa740 msgr2=0x7f76bc157230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:36.957 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76bc0aa740 0x7f76bc157230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:36.957 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 -- 172.21.15.73:0/1555128692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76ac0092a0 con 0x7f76bc154490 2023-11-28T02:52:36.957 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c16bf700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 0x7f76bc153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c1ec0700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f76b400cb90 tx=0x7f76b4007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b4004c40 con 0x7f76bc154490 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b4004dc0 con 0x7f76bc154490 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76bc1554f0 con 0x7f76bc154490 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b40073c0 con 0x7f76bc154490 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.949+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76bc154b50 con 0x7f76bc154490 2023-11-28T02:52:36.958 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.950+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76bc1542a0 con 0x7f76bc154490 2023-11-28T02:52:36.959 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.950+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f76b4007770 con 0x7f76bc154490 2023-11-28T02:52:36.959 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.951+0000 7f76b17fa700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 0x7f76a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:36.959 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.951+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f76b4079a90 con 0x7f76bc154490 2023-11-28T02:52:36.959 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.951+0000 7f76c16bf700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 0x7f76a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:36.959 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.951+0000 7f76c16bf700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 0x7f76a0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f76b8002750 tx=0x7f76b8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:36.960 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:36.955+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76b404b230 con 0x7f76bc154490 2023-11-28T02:52:37.120 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.120+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f76bc06c7c0 con 0x7f76bc154490 2023-11-28T02:52:37.121 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.121+0000 7f76b17fa700 1 -- 172.21.15.73:0/1555128692 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) v1 ==== 74+0+12180 (secure 0 0 0) 0x7f76b404e850 con 0x7f76bc154490 2023-11-28T02:52:37.121 INFO:teuthology.misc.health.smithi073.stdout: 2023-11-28T02:52:37.122 INFO:teuthology.misc.health.smithi073.stdout:{"epoch":14,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:36.415279+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:37.123 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.122+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 msgr2=0x7f76a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.122+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 0x7f76a0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f76b8002750 tx=0x7f76b8009040).stop 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.122+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 msgr2=0x7f76bc0acb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.122+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f76b400cb90 tx=0x7f76b4007af0).stop 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 shutdown_connections 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a005f150 0x7f76a0061610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76bc154490 0x7f76bc0acb20 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76bc0aa360 0x7f76bc153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.124 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 --2- 172.21.15.73:0/1555128692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76bc0aa740 0x7f76bc157230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.125 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 >> 172.21.15.73:0/1555128692 conn(0x7f76bc0cb270 msgr2=0x7f76bc0c7ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:37.125 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 shutdown_connections 2023-11-28T02:52:37.125 INFO:teuthology.misc.health.smithi073.stderr:2023-11-28T02:52:37.123+0000 7f76c3923700 1 -- 172.21.15.73:0/1555128692 wait complete. 2023-11-28T02:52:37.135 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-11-28T02:52:37.135 INFO:tasks.ceph:Creating RBD pool 2023-11-28T02:52:37.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool create rbd 8 2023-11-28T02:52:37.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.247+0000 7f33663c5700 1 Processor -- start 2023-11-28T02:52:37.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.247+0000 7f33663c5700 1 -- start start 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f33663c5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33600bde50 0x7f33600c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f33663c5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3360155650 con 0x7f33600c6b30 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f33663c5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3360155f60 con 0x7f3360155ae0 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f33663c5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f336014bff0 con 0x7f33600bde50 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f3364962700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33600bde50 0x7f33600c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f335ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f33600c6b30 0x7f33601524b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36184/0 (socket says 172.21.15.73:36184) 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f3364962700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f33600bde50 0x7f33600c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60622/0 (socket says 172.21.15.73:60622) 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f335ffff700 1 -- 172.21.15.73:0/583754146 learned_addr learned my addr 172.21.15.73:0/583754146 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f3364962700 1 -- 172.21.15.73:0/583754146 >> v1:172.21.15.150:6789/0 conn(0x7f3360155ae0 legacy=0x7f33600bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f3364962700 1 -- 172.21.15.73:0/583754146 >> v1:172.21.15.73:6789/0 conn(0x7f33600c6b30 legacy=0x7f33601524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.248+0000 7f335effd700 1 -- 172.21.15.73:0/583754146 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2996260993 0 0) 0x7f3360155650 con 0x7f33600c6b30 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f3364962700 1 -- 172.21.15.73:0/583754146 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3360063b60 con 0x7f33600bde50 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f3364962700 1 --2- 172.21.15.73:0/583754146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600bde50 0x7f33600c2ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3358009e60 tx=0x7f3358009780).ready entity=mon.0 client_cookie=f2045f1a31f195e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f335effd700 1 -- 172.21.15.73:0/583754146 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f335800e070 con 0x7f33600bde50 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f335effd700 1 -- 172.21.15.73:0/583754146 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f335800bc30 con 0x7f33600bde50 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f335effd700 1 -- 172.21.15.73:0/583754146 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3358012410 con 0x7f33600bde50 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 -- 172.21.15.73:0/583754146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600bde50 msgr2=0x7f33600c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 --2- 172.21.15.73:0/583754146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600bde50 0x7f33600c2ce0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3358009e60 tx=0x7f3358009780).stop 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 -- 172.21.15.73:0/583754146 shutdown_connections 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 --2- 172.21.15.73:0/583754146 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600bde50 0x7f33600c2ce0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 -- 172.21.15.73:0/583754146 >> 172.21.15.73:0/583754146 conn(0x7f33600c8230 msgr2=0x7f33600c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:37.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 -- 172.21.15.73:0/583754146 shutdown_connections 2023-11-28T02:52:37.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.249+0000 7f33663c5700 1 -- 172.21.15.73:0/583754146 wait complete. 2023-11-28T02:52:37.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.250+0000 7f33663c5700 1 Processor -- start 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 -- start start 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33600bde50 0x7f33600a7570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3360155ae0 0x7f33600a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33600c14c0 con 0x7f33600c6b30 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33600bfe70 con 0x7f33600bde50 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f33663c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33601546c0 con 0x7f3360155ae0 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f335f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f335f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60632/0 (socket says 172.21.15.73:60632) 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.251+0000 7f335f7fe700 1 -- 172.21.15.73:0/2179616974 learned_addr learned my addr 172.21.15.73:0/2179616974 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:37.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335ffff700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33600bde50 0x7f33600a7570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f3364962700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3360155ae0 0x7f33600a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 -- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3360155ae0 msgr2=0x7f33600a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3360155ae0 0x7f33600a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 -- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33600bde50 msgr2=0x7f33600a7570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33600bde50 0x7f33600a7570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 -- 172.21.15.73:0/2179616974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3358009a20 con 0x7f33600c6b30 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f335f7fe700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f335400c3c0 tx=0x7f3354007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354007020 con 0x7f33600c6b30 2023-11-28T02:52:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3354007180 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33600c0b90 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3354007520 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.252+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33600c1150 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.253+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336006c860 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.253+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f33540068b0 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.253+0000 7f3347fff700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 0x7f334c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.254+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 4892+0+0 (secure 0 0 0) 0x7f3354079140 con 0x7f33600c6b30 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.254+0000 7f335ffff700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 0x7f334c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:37.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.254+0000 7f335ffff700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 0x7f334c061610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3350001360 tx=0x7f3350009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:37.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.258+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3354047af0 con 0x7f33600c6b30 2023-11-28T02:52:37.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:37.421+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f33600c1a70 con 0x7f33600c6b30 2023-11-28T02:52:38.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.430+0000 7f3347fff700 1 -- 172.21.15.73:0/2179616974 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v16) v1 ==== 109+0+0 (secure 0 0 0) 0x7f335404a8e0 con 0x7f33600c6b30 2023-11-28T02:52:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 msgr2=0x7f334c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 0x7f334c061610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3350001360 tx=0x7f3350009040).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 msgr2=0x7f33600bf980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f335400c3c0 tx=0x7f3354007e00).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 shutdown_connections 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f150 0x7f334c061610 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3360155ae0 0x7f33600a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33600bde50 0x7f33600a7570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 --2- 172.21.15.73:0/2179616974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33600c6b30 0x7f33600bf980 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 >> 172.21.15.73:0/2179616974 conn(0x7f33600c8230 msgr2=0x7f3360069ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 shutdown_connections 2023-11-28T02:52:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.433+0000 7f33663c5700 1 -- 172.21.15.73:0/2179616974 wait complete. 2023-11-28T02:52:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'rbd' created 2023-11-28T02:52:38.447 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-11-28T02:52:38.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.564+0000 7f98a4b0a700 1 Processor -- start 2023-11-28T02:52:38.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.564+0000 7f98a4b0a700 1 -- start start 2023-11-28T02:52:38.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f98a4b0a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98a00c4d80 0x7f98a0156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f98a4b0a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98a0156610 0x7f98a014fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f98a4b0a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98a0063ae0 con 0x7f98a00c71b0 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f98a4b0a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00c7590 con 0x7f98a00c4d80 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f98a4b0a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98a00c6610 con 0x7f98a0156610 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98a00c4d80 0x7f98a0156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f989e59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f98a00c71b0 0x7f98a00bc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36190/0 (socket says 172.21.15.73:36190) 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f989e59c700 1 -- 172.21.15.73:0/2105751934 learned_addr learned my addr 172.21.15.73:0/2105751934 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f989ed9d700 1 --2- 172.21.15.73:0/2105751934 >> v2:172.21.15.150:3300/0 conn(0x7f98a0156610 0x7f98a014fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 -- 172.21.15.73:0/2105751934 >> v2:172.21.15.150:3300/0 conn(0x7f98a0156610 msgr2=0x7f98a014fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 --2- 172.21.15.73:0/2105751934 >> v2:172.21.15.150:3300/0 conn(0x7f98a0156610 0x7f98a014fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 -- 172.21.15.73:0/2105751934 >> v1:172.21.15.73:6789/0 conn(0x7f98a00c71b0 legacy=0x7f98a00bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 -- 172.21.15.73:0/2105751934 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a0156a40 con 0x7f98a00c4d80 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f989dd9b700 1 -- 172.21.15.73:0/2105751934 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079604199 0 0) 0x7f98a0063ae0 con 0x7f98a00c71b0 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.565+0000 7f9895d9b700 1 --2- 172.21.15.73:0/2105751934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a0156120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f98a0068f70 tx=0x7f9888009060).ready entity=mon.0 client_cookie=2574739e6ad43109 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f989dd9b700 1 -- 172.21.15.73:0/2105751934 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800b580 con 0x7f98a00c4d80 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f989dd9b700 1 -- 172.21.15.73:0/2105751934 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988800b6e0 con 0x7f98a00c4d80 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f989dd9b700 1 -- 172.21.15.73:0/2105751934 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988800ba60 con 0x7f98a00c4d80 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 -- 172.21.15.73:0/2105751934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 msgr2=0x7f98a0156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/2105751934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a0156120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f98a0068f70 tx=0x7f9888009060).stop 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 -- 172.21.15.73:0/2105751934 shutdown_connections 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/2105751934 >> v2:172.21.15.150:3300/0 conn(0x7f98a0156610 0x7f98a014fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/2105751934 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a0156120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 -- 172.21.15.73:0/2105751934 >> 172.21.15.73:0/2105751934 conn(0x7f98a00c8170 msgr2=0x7f98a00c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:38.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 -- 172.21.15.73:0/2105751934 shutdown_connections 2023-11-28T02:52:38.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.566+0000 7f98a4b0a700 1 -- 172.21.15.73:0/2105751934 wait complete. 2023-11-28T02:52:38.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.567+0000 7f98a4b0a700 1 Processor -- start 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.567+0000 7f98a4b0a700 1 -- start start 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98a00c71b0 0x7f98a00aa9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 0x7f98a00aae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00ac070 con 0x7f98a00c4d80 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00ab460 con 0x7f98a0156610 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f98a4b0a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98a00ab740 con 0x7f98a00c71b0 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f989e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f989e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60644/0 (socket says 172.21.15.73:60644) 2023-11-28T02:52:38.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f989e59c700 1 -- 172.21.15.73:0/614327167 learned_addr learned my addr 172.21.15.73:0/614327167 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.568+0000 7f989ed9d700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 0x7f98a00aae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f9895d9b700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98a00c71b0 0x7f98a00aa9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 -- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98a00c71b0 msgr2=0x7f98a00aa9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98a00c71b0 0x7f98a00aa9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 -- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 msgr2=0x7f98a00aae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 0x7f98a00aae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 -- 172.21.15.73:0/614327167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98880092a0 con 0x7f98a00c4d80 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989ed9d700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 0x7f98a00aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f989e59c700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f989000c4e0 tx=0x7f9890007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900072e0 con 0x7f98a00c4d80 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9890007440 con 0x7f98a00c4d80 2023-11-28T02:52:38.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98900057a0 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98a00aba20 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.569+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98a00ac9f0 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.570+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98a00ac350 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.570+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9890005bd0 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.570+0000 7f9896ffd700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 0x7f9880061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.571+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 5258+0+0 (secure 0 0 0) 0x7f98900797d0 con 0x7f98a00c4d80 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.571+0000 7f9895d9b700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 0x7f9880061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:38.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.571+0000 7f9895d9b700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 0x7f9880061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f98a00c5210 tx=0x7f9888017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:38.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.575+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f989004ae00 con 0x7f98a00c4d80 2023-11-28T02:52:38.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:38.739+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98a006b710 con 0x7f98a00c4d80 2023-11-28T02:52:39.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.440+0000 7f9896ffd700 1 -- 172.21.15.73:0/614327167 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v17) v1 ==== 173+0+0 (secure 0 0 0) 0x7f989004e420 con 0x7f98a00c4d80 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 msgr2=0x7f9880061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 0x7f9880061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f98a00c5210 tx=0x7f9888017000).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 msgr2=0x7f98a00ac500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f989000c4e0 tx=0x7f9890007c70).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 shutdown_connections 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f988005f150 0x7f9880061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98a0156610 0x7f98a00aae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98a00c4d80 0x7f98a00ac500 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 --2- 172.21.15.73:0/614327167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98a00c71b0 0x7f98a00aa9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 >> 172.21.15.73:0/614327167 conn(0x7f98a00c8170 msgr2=0x7f98a0153ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 shutdown_connections 2023-11-28T02:52:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.443+0000 7f98a4b0a700 1 -- 172.21.15.73:0/614327167 wait complete. 2023-11-28T02:52:39.445 INFO:teuthology.orchestra.run.smithi073.stderr:enabled application 'rbd' on pool 'rbd' 2023-11-28T02:52:39.458 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-11-28T02:52:39.458 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T02:52:39.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T02:52:39.460 INFO:tasks.ceph.mds.a:Started 2023-11-28T02:52:39.460 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T02:52:39.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T02:52:39.462 INFO:tasks.ceph.mds.c:Started 2023-11-28T02:52:39.462 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T02:52:39.463 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T02:52:39.465 INFO:tasks.ceph.mds.b:Started 2023-11-28T02:52:39.466 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T02:52:39.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T02:52:39.468 INFO:tasks.ceph.mds.d:Started 2023-11-28T02:52:39.468 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-11-28T02:52:39.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:39.533 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T02:52:39.534 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T02:52:39.546 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T02:52:39.546 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T02:52:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.588+0000 7f9810b17700 1 Processor -- start 2023-11-28T02:52:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.588+0000 7f9810b17700 1 -- start start 2023-11-28T02:52:39.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.589+0000 7f9810b17700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 0x7f980c0697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.589+0000 7f9810b17700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c065760 con 0x7f980c069ce0 2023-11-28T02:52:39.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.589+0000 7f9810b17700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f980c0c3250 con 0x7f980c062e10 2023-11-28T02:52:39.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.589+0000 7f9810b17700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c06a160 con 0x7f980c1542d0 2023-11-28T02:52:39.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f9809d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f980c069ce0 0x7f980c062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36204/0 (socket says 172.21.15.73:36204) 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f9809d9b700 1 -- 172.21.15.73:0/254397410 learned_addr learned my addr 172.21.15.73:0/254397410 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980a59c700 1 --2- 172.21.15.73:0/254397410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 0x7f980c0697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3777057238 0 0) 0x7f980c065760 con 0x7f980c069ce0 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97f40033f0 con 0x7f980c069ce0 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4116321629 0 0) 0x7f980c0c3250 con 0x7f980c062e10 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980a59c700 1 -- 172.21.15.73:0/254397410 >> v1:172.21.15.150:6789/0 conn(0x7f980c062e10 legacy=0x7f980c0661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980a59c700 1 -- 172.21.15.73:0/254397410 >> v1:172.21.15.73:6789/0 conn(0x7f980c069ce0 legacy=0x7f980c062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980a59c700 1 -- 172.21.15.73:0/254397410 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f980c06a340 con 0x7f980c1542d0 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980a59c700 1 --2- 172.21.15.73:0/254397410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 0x7f980c0697f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f97fc009e90 tx=0x7f97fc009780).ready entity=mon.2 client_cookie=9702cb2614c5916d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00e070 con 0x7f980c1542d0 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97fc00bb60 con 0x7f980c1542d0 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.590+0000 7f980959a700 1 -- 172.21.15.73:0/254397410 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97fc00bee0 con 0x7f980c1542d0 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 -- 172.21.15.73:0/254397410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 msgr2=0x7f980c0697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 --2- 172.21.15.73:0/254397410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 0x7f980c0697f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f97fc009e90 tx=0x7f97fc009780).stop 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 -- 172.21.15.73:0/254397410 shutdown_connections 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 --2- 172.21.15.73:0/254397410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c1542d0 0x7f980c0697f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 -- 172.21.15.73:0/254397410 >> 172.21.15.73:0/254397410 conn(0x7f980c0c7fd0 msgr2=0x7f980c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 -- 172.21.15.73:0/254397410 shutdown_connections 2023-11-28T02:52:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.591+0000 7f9810b17700 1 -- 172.21.15.73:0/254397410 wait complete. 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.592+0000 7f9810b17700 1 Processor -- start 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.592+0000 7f9810b17700 1 -- start start 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c062e10 0x7f980c159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c1542d0 0x7f980c14e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c159500 con 0x7f980c069ce0 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f980c1588f0 con 0x7f980c1542d0 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f980c158bd0 con 0x7f980c062e10 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60692/0 (socket says 172.21.15.73:60692) 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 -- 172.21.15.73:0/2465706525 learned_addr learned my addr 172.21.15.73:0/2465706525 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f980a59c700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c062e10 0x7f980c159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f980ad9d700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c1542d0 0x7f980c14e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 -- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c062e10 msgr2=0x7f980c159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c062e10 0x7f980c159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 -- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c1542d0 msgr2=0x7f980c14e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c1542d0 0x7f980c14e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 -- 172.21.15.73:0/2465706525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97fc009a20 con 0x7f980c069ce0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9809d9b700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f980000c4c0 tx=0x7f9800007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f980000ce30 con 0x7f980c069ce0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98000072e0 con 0x7f980c069ce0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f980c158eb0 con 0x7f980c069ce0 2023-11-28T02:52:39.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98000057a0 con 0x7f980c069ce0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.593+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f980c14ee60 con 0x7f980c069ce0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.594+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980c159670 con 0x7f980c069ce0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.594+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9800005b50 con 0x7f980c069ce0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.595+0000 7f97fa7fc700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.595+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f98000794e0 con 0x7f980c069ce0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.595+0000 7f980a59c700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.595+0000 7f980a59c700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f97fc009de0 tx=0x7f97fc009780).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.599+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f980004ab00 con 0x7f980c069ce0 2023-11-28T02:52:39.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.761+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f980c06a620 con 0x7f980c069ce0 2023-11-28T02:52:39.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.761+0000 7f97fa7fc700 1 -- 172.21.15.73:0/2465706525 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+13722 (secure 0 0 0) 0x7f980004e120 con 0x7f980c069ce0 2023-11-28T02:52:39.763 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:39.763 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":17,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:39.428558+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:39.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.763+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 msgr2=0x7f97ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.763+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f97fc009de0 tx=0x7f97fc009780).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.763+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 msgr2=0x7f980c158330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.763+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f980000c4c0 tx=0x7f9800007c10).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 shutdown_connections 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980c1542d0 0x7f980c14e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980c062e10 0x7f980c159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 --2- 172.21.15.73:0/2465706525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f980c069ce0 0x7f980c158330 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 >> 172.21.15.73:0/2465706525 conn(0x7f980c0c7fd0 msgr2=0x7f980c1571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:39.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 shutdown_connections 2023-11-28T02:52:39.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.764+0000 7f9810b17700 1 -- 172.21.15.73:0/2465706525 wait complete. 2023-11-28T02:52:39.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T02:52:39.777 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:52:39.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.892+0000 7fdd92a25700 1 Processor -- start 2023-11-28T02:52:39.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.892+0000 7fdd92a25700 1 -- start start 2023-11-28T02:52:39.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92a25700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 0x7fdd8c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92a25700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd8c154950 0x7fdd8c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92a25700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd8c0c7bc0 con 0x7fdd8c158570 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92a25700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8c155920 con 0x7fdd8c154950 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92a25700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c154dd0 con 0x7fdd8c14e530 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 0x7fdd8c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91222700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd8c154950 0x7fdd8c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92224700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fdd8c158570 0x7fdd8c0aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39508/0 (socket says 172.21.15.73:39508) 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd92224700 1 -- 172.21.15.73:0/1654100979 learned_addr learned my addr 172.21.15.73:0/1654100979 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd90a21700 1 -- 172.21.15.73:0/1654100979 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 512712577 0 0) 0x7fdd8c0c7bc0 con 0x7fdd8c158570 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 -- 172.21.15.73:0/1654100979 >> v2:172.21.15.150:3300/0 conn(0x7fdd8c154950 msgr2=0x7fdd8c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 --2- 172.21.15.73:0/1654100979 >> v2:172.21.15.150:3300/0 conn(0x7fdd8c154950 0x7fdd8c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 -- 172.21.15.73:0/1654100979 >> v1:172.21.15.150:6789/0 conn(0x7fdd8c158570 legacy=0x7fdd8c0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 -- 172.21.15.73:0/1654100979 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd8c155080 con 0x7fdd8c14e530 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.893+0000 7fdd91a23700 1 --2- 172.21.15.73:0/1654100979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 0x7fdd8c0c66f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fdd84009e60 tx=0x7fdd84009780).ready entity=mon.2 client_cookie=ca6f88be973a62b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd90a21700 1 -- 172.21.15.73:0/1654100979 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8400e070 con 0x7fdd8c14e530 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd90a21700 1 -- 172.21.15.73:0/1654100979 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400bb60 con 0x7fdd8c14e530 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd90a21700 1 -- 172.21.15.73:0/1654100979 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84012410 con 0x7fdd8c14e530 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 -- 172.21.15.73:0/1654100979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 msgr2=0x7fdd8c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 --2- 172.21.15.73:0/1654100979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 0x7fdd8c0c66f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fdd84009e60 tx=0x7fdd84009780).stop 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 -- 172.21.15.73:0/1654100979 shutdown_connections 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 --2- 172.21.15.73:0/1654100979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c14e530 0x7fdd8c0c66f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 --2- 172.21.15.73:0/1654100979 >> v2:172.21.15.150:3300/0 conn(0x7fdd8c154950 0x7fdd8c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 -- 172.21.15.73:0/1654100979 >> 172.21.15.73:0/1654100979 conn(0x7fdd8c064990 msgr2=0x7fdd8c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 -- 172.21.15.73:0/1654100979 shutdown_connections 2023-11-28T02:52:39.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.894+0000 7fdd92a25700 1 -- 172.21.15.73:0/1654100979 wait complete. 2023-11-28T02:52:39.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.895+0000 7fdd92a25700 1 Processor -- start 2023-11-28T02:52:39.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.895+0000 7fdd92a25700 1 -- start start 2023-11-28T02:52:39.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 0x7fdd8c14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 0x7fdd8c0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c25f0 con 0x7fdd8c14e530 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c153850 con 0x7fdd8c154950 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92a25700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8c0c3410 con 0x7fdd8c158570 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60708/0 (socket says 172.21.15.73:60708) 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 -- 172.21.15.73:0/3023116804 learned_addr learned my addr 172.21.15.73:0/3023116804 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91222700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 0x7fdd8c14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92224700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 0x7fdd8c0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 -- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 msgr2=0x7fdd8c0b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 0x7fdd8c0b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 -- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 msgr2=0x7fdd8c14dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 0x7fdd8c14dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91a23700 1 -- 172.21.15.73:0/3023116804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd84009a20 con 0x7fdd8c14e530 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd92224700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 0x7fdd8c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.896+0000 7fdd91222700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 0x7fdd8c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd91a23700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fdd8400b6c0 tx=0x7fdd84009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd84009440 con 0x7fdd8c14e530 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd8400bea0 con 0x7fdd8c14e530 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd8c0c3d90 con 0x7fdd8c14e530 2023-11-28T02:52:39.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd8401d880 con 0x7fdd8c14e530 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.897+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd8c0c2ae0 con 0x7fdd8c14e530 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.898+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd8c0c36f0 con 0x7fdd8c14e530 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.898+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdd84003040 con 0x7fdd8c14e530 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.898+0000 7fdd89ffb700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 0x7fdd78061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.899+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fdd8407bb40 con 0x7fdd8c14e530 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.899+0000 7fdd91222700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 0x7fdd78061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.899+0000 7fdd91222700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 0x7fdd78061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd80006d80 tx=0x7fdd80008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:39.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:39.903+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd8404d200 con 0x7fdd8c14e530 2023-11-28T02:52:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.066+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdd8c062ee0 con 0x7fdd8c14e530 2023-11-28T02:52:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.067+0000 7fdd89ffb700 1 -- 172.21.15.73:0/3023116804 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v17) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdd84050820 con 0x7fdd8c14e530 2023-11-28T02:52:40.069 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:52:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 msgr2=0x7fdd78061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 0x7fdd78061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd80006d80 tx=0x7fdd80008040).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 msgr2=0x7fdd8c0c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fdd8400b6c0 tx=0x7fdd84009780).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 shutdown_connections 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd7805f150 0x7fdd78061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd8c158570 0x7fdd8c0b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd8c14e530 0x7fdd8c0c38a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 --2- 172.21.15.73:0/3023116804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd8c154950 0x7fdd8c14dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 >> 172.21.15.73:0/3023116804 conn(0x7fdd8c064990 msgr2=0x7fdd8c0c0350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 shutdown_connections 2023-11-28T02:52:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.069+0000 7fdd92a25700 1 -- 172.21.15.73:0/3023116804 wait complete. 2023-11-28T02:52:40.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:52:40.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.196+0000 7f6d29112700 1 Processor -- start 2023-11-28T02:52:40.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.196+0000 7f6d29112700 1 -- start start 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d29112700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d24098d10 0x7f6d240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d29112700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d24158320 0x7f6d241587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d29112700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d24158cc0 con 0x7f6d240c6190 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d29112700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d240c72d0 con 0x7f6d24158320 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d29112700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d24064570 con 0x7f6d24098d10 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6d24158320 0x7f6d241587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6d240c6190 0x7f6d24063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36218/0 (socket says 172.21.15.73:36218) 2023-11-28T02:52:40.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2259c700 1 -- 172.21.15.73:0/528611024 learned_addr learned my addr 172.21.15.73:0/528611024 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d22d9d700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d24098d10 0x7f6d240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d21d9b700 1 -- 172.21.15.73:0/528611024 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3224180049 0 0) 0x7f6d18001110 con 0x7f6d240c6190 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d21d9b700 1 -- 172.21.15.73:0/528611024 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d0c0033f0 con 0x7f6d240c6190 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 -- 172.21.15.73:0/528611024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d24098d10 msgr2=0x7f6d240c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d24098d10 0x7f6d240c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 -- 172.21.15.73:0/528611024 >> v1:172.21.15.73:6789/0 conn(0x7f6d240c6190 legacy=0x7f6d24063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 -- 172.21.15.73:0/528611024 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d24154bc0 con 0x7f6d24158320 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.197+0000 7f6d2359e700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d241587d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6d1c009e60 tx=0x7f6d1c009780).ready entity=mon.0 client_cookie=b0a953f636ebe982 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d21d9b700 1 -- 172.21.15.73:0/528611024 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00e070 con 0x7f6d24158320 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d21d9b700 1 -- 172.21.15.73:0/528611024 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c00bab0 con 0x7f6d24158320 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d21d9b700 1 -- 172.21.15.73:0/528611024 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00be30 con 0x7f6d24158320 2023-11-28T02:52:40.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 -- 172.21.15.73:0/528611024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 msgr2=0x7f6d241587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d241587d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f6d1c009e60 tx=0x7f6d1c009780).stop 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 -- 172.21.15.73:0/528611024 shutdown_connections 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d24098d10 0x7f6d240c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 --2- 172.21.15.73:0/528611024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d241587d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 -- 172.21.15.73:0/528611024 >> 172.21.15.73:0/528611024 conn(0x7f6d240cb310 msgr2=0x7f6d240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 -- 172.21.15.73:0/528611024 shutdown_connections 2023-11-28T02:52:40.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.198+0000 7f6d29112700 1 -- 172.21.15.73:0/528611024 wait complete. 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.199+0000 7f6d29112700 1 Processor -- start 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.199+0000 7f6d29112700 1 -- start start 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d24098d10 0x7f6d2414f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 0x7f6d240c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240c56c0 con 0x7f6d24158320 2023-11-28T02:52:40.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b05c0 con 0x7f6d24098d10 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d29112700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d240b07f0 con 0x7f6d240c6190 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60734/0 (socket says 172.21.15.73:60734) 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 -- 172.21.15.73:0/3900107736 learned_addr learned my addr 172.21.15.73:0/3900107736 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2259c700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 0x7f6d240c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d22d9d700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d24098d10 0x7f6d2414f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 -- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 msgr2=0x7f6d240c4ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 0x7f6d240c4ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 -- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d24098d10 msgr2=0x7f6d2414f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d24098d10 0x7f6d2414f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2359e700 1 -- 172.21.15.73:0/3900107736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c009a20 con 0x7f6d24158320 2023-11-28T02:52:40.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.200+0000 7f6d2259c700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 0x7f6d240c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d2359e700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6d240c2360 tx=0x7f6d240a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c01d630 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1c01d790 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c01db50 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d240b0a90 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.201+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d240b1050 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.202+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d24155d70 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.202+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6d1c003040 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.202+0000 7f6d0affd700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 0x7f6d10061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.202+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6d1c07bb90 con 0x7f6d24158320 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.202+0000 7f6d22d9d700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 0x7f6d10061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.203+0000 7f6d22d9d700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 0x7f6d10061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6d14009c30 tx=0x7f6d14009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.207+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6d24155d70 con 0x7f6d24158320 2023-11-28T02:52:40.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.368+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f6d24155d70 con 0x7f6d24158320 2023-11-28T02:52:40.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.368+0000 7f6d0affd700 1 -- 172.21.15.73:0/3900107736 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v17) v1 ==== 92+0+10 (secure 0 0 0) 0x7f6d24155d70 con 0x7f6d24158320 2023-11-28T02:52:40.369 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:52:40.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 msgr2=0x7f6d10061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 0x7f6d10061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f6d14009c30 tx=0x7f6d14009060).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 msgr2=0x7f6d240c51d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6d240c2360 tx=0x7f6d240a8040).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 shutdown_connections 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6d1005f150 0x7f6d10061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6d24158320 0x7f6d240c51d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6d24098d10 0x7f6d2414f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 --2- 172.21.15.73:0/3900107736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6d240c6190 0x7f6d240c4ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 >> 172.21.15.73:0/3900107736 conn(0x7f6d240cb310 msgr2=0x7f6d24063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 shutdown_connections 2023-11-28T02:52:40.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.370+0000 7f6d29112700 1 -- 172.21.15.73:0/3900107736 wait complete. 2023-11-28T02:52:40.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:52:40.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.496+0000 7f6cd3ef1700 1 Processor -- start 2023-11-28T02:52:40.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.496+0000 7f6cd3ef1700 1 -- start start 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd3ef1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ccc0caab0 0x7f6ccc0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd3ef1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ccc0ca180 0x7f6ccc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd3ef1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ccc0a9a90 con 0x7f6ccc0cae90 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd3ef1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc154ce0 con 0x7f6ccc0ca180 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd3ef1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ccc0a8e80 con 0x7f6ccc0caab0 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd248e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ccc0ca180 0x7f6ccc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd148c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6ccc0cae90 0x7f6ccc0c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36232/0 (socket says 172.21.15.73:36232) 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd148c700 1 -- 172.21.15.73:0/3112852802 learned_addr learned my addr 172.21.15.73:0/3112852802 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:40.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.497+0000 7f6cd1c8d700 1 --2- 172.21.15.73:0/3112852802 >> v2:172.21.15.150:3300/0 conn(0x7f6ccc0caab0 0x7f6ccc0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd248e700 1 -- 172.21.15.73:0/3112852802 >> v2:172.21.15.150:3300/0 conn(0x7f6ccc0caab0 msgr2=0x7f6ccc0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd248e700 1 --2- 172.21.15.73:0/3112852802 >> v2:172.21.15.150:3300/0 conn(0x7f6ccc0caab0 0x7f6ccc0aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd248e700 1 -- 172.21.15.73:0/3112852802 >> v1:172.21.15.73:6789/0 conn(0x7f6ccc0cae90 legacy=0x7f6ccc0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd0c8b700 1 -- 172.21.15.73:0/3112852802 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919965297 0 0) 0x7f6ccc0a9a90 con 0x7f6ccc0cae90 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd248e700 1 -- 172.21.15.73:0/3112852802 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ccc0a9160 con 0x7f6ccc0ca180 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd248e700 1 --2- 172.21.15.73:0/3112852802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc15e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6ccc0848c0 tx=0x7f6cc4009060).ready entity=mon.0 client_cookie=4f017cab993b23b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd0c8b700 1 -- 172.21.15.73:0/3112852802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400b580 con 0x7f6ccc0ca180 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd0c8b700 1 -- 172.21.15.73:0/3112852802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc400b6e0 con 0x7f6ccc0ca180 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd0c8b700 1 -- 172.21.15.73:0/3112852802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400ba60 con 0x7f6ccc0ca180 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/3112852802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0ca180 msgr2=0x7f6ccc15e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/3112852802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc15e3c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f6ccc0848c0 tx=0x7f6cc4009060).stop 2023-11-28T02:52:40.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/3112852802 shutdown_connections 2023-11-28T02:52:40.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/3112852802 >> v2:172.21.15.150:3300/0 conn(0x7f6ccc0caab0 0x7f6ccc0aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/3112852802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc15e3c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/3112852802 >> 172.21.15.73:0/3112852802 conn(0x7f6ccc094920 msgr2=0x7f6ccc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/3112852802 shutdown_connections 2023-11-28T02:52:40.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.498+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/3112852802 wait complete. 2023-11-28T02:52:40.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 Processor -- start 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 -- start start 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 0x7f6ccc0c3930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c34a0 con 0x7f6ccc0cae90 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c2890 con 0x7f6ccc0ca180 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd3ef1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ccc0c2b70 con 0x7f6ccc0caab0 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd248e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd248e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60748/0 (socket says 172.21.15.73:60748) 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.500+0000 7f6cd248e700 1 -- 172.21.15.73:0/1279184642 learned_addr learned my addr 172.21.15.73:0/1279184642 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd1c8d700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd148c700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 0x7f6ccc0c3930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 -- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 msgr2=0x7f6ccc0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 0x7f6ccc0c3930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 -- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ccc0ca180 msgr2=0x7f6ccc154060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc154060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 -- 172.21.15.73:0/1279184642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cc40092a0 con 0x7f6ccc0cae90 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd148c700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 0x7f6ccc0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd248e700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6ccc083c30 tx=0x7f6cc4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400ba60 con 0x7f6ccc0cae90 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cc401b410 con 0x7f6ccc0cae90 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ccc0c2e50 con 0x7f6ccc0cae90 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cc401b7b0 con 0x7f6ccc0cae90 2023-11-28T02:52:40.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.501+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ccc154550 con 0x7f6ccc0cae90 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.502+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ccc0c3610 con 0x7f6ccc0cae90 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.502+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6cc401bba0 con 0x7f6ccc0cae90 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.503+0000 7f6cc1ffb700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 0x7f6cb0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.503+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6cc407bb80 con 0x7f6ccc0cae90 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.503+0000 7f6cd1c8d700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 0x7f6cb0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.503+0000 7f6cd1c8d700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 0x7f6cb0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6cc80075e0 tx=0x7f6cc80097f0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.507+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6cc404d240 con 0x7f6ccc0cae90 2023-11-28T02:52:40.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.710+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6ccc06c860 con 0x7f6ccc0cae90 2023-11-28T02:52:40.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.710+0000 7f6cc1ffb700 1 -- 172.21.15.73:0/1279184642 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3524 (secure 0 0 0) 0x7f6cc4029070 con 0x7f6ccc0cae90 2023-11-28T02:52:40.712 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:40.712 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4281,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 msgr2=0x7f6cb0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 0x7f6cb0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6cc80075e0 tx=0x7f6cc80097f0).stop 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 msgr2=0x7f6ccc0c3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f6ccc083c30 tx=0x7f6cc4017000).stop 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 shutdown_connections 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6cb005f150 0x7f6cb0061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ccc0cae90 0x7f6ccc0c3e20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ccc0ca180 0x7f6ccc154060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 --2- 172.21.15.73:0/1279184642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ccc0caab0 0x7f6ccc0c3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 >> 172.21.15.73:0/1279184642 conn(0x7f6ccc094920 msgr2=0x7f6ccc0a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 shutdown_connections 2023-11-28T02:52:40.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.712+0000 7f6cd3ef1700 1 -- 172.21.15.73:0/1279184642 wait complete. 2023-11-28T02:52:40.715 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 3 2023-11-28T02:52:40.725 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:40.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.836+0000 7ff1b359e700 1 Processor -- start 2023-11-28T02:52:40.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.836+0000 7ff1b359e700 1 -- start start 2023-11-28T02:52:40.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b359e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff1b40c0180 0x7ff1b4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1b40c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b359e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1b40c12c0 con 0x7ff1b4157660 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b359e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1b40c06b0 con 0x7ff1b40c0180 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c0990 con 0x7ff1b40c1750 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1b4157660 0x7ff1b41589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36242/0 (socket says 172.21.15.73:36242) 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.837+0000 7ff1b259c700 1 -- 172.21.15.73:0/528788245 learned_addr learned my addr 172.21.15.73:0/528788245 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933096549 0 0) 0x7ff1b40c12c0 con 0x7ff1b4157660 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1a9d9b700 1 --2- 172.21.15.73:0/528788245 >> v2:172.21.15.150:3300/0 conn(0x7ff1b40c0180 0x7ff1b4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1a00033f0 con 0x7ff1b4157660 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b2d9d700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1b40c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2303043317 0 0) 0x7ff1a00033f0 con 0x7ff1b4157660 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 msgr2=0x7ff1b40c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1b40c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 >> v2:172.21.15.150:3300/0 conn(0x7ff1b40c0180 msgr2=0x7ff1b4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 --2- 172.21.15.73:0/528788245 >> v2:172.21.15.150:3300/0 conn(0x7ff1b40c0180 0x7ff1b4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40c0c70 con 0x7ff1b4157660 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff1a4001470 con 0x7ff1b4157660 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 >> v1:172.21.15.73:6789/0 conn(0x7ff1b4157660 legacy=0x7ff1b41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1a0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a0004b80 0x7ff1a0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c06b0 con 0x7ff1a00038e0 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40c0990 con 0x7ff1a0004b80 2023-11-28T02:52:40.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a000b8e0 con 0x7ff1b40c1750 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1a9d9b700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1a0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b2d9d700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a0004b80 0x7ff1a0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 -- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 msgr2=0x7ff1a0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b40c1750 0x7ff1a0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 -- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a0004b80 msgr2=0x7ff1a0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1a0004b80 0x7ff1a0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 -- 172.21.15.73:0/528788245 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a00033f0 con 0x7ff1a00038e0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b259c700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1a0003d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff1a400c210 tx=0x7ff1a4001e90).ready entity=mon.0 client_cookie=9041dd06e726d88a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.838+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4007a60 con 0x7ff1a00038e0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a400cae0 con 0x7ff1a00038e0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b1d9b700 1 -- 172.21.15.73:0/528788245 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a400ce60 con 0x7ff1a00038e0 2023-11-28T02:52:40.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 -- 172.21.15.73:0/528788245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 msgr2=0x7ff1a0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 --2- 172.21.15.73:0/528788245 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1a0003d00 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff1a400c210 tx=0x7ff1a4001e90).stop 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b2d9d700 1 -- 172.21.15.73:0/528788245 reap_dead start 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 -- 172.21.15.73:0/528788245 shutdown_connections 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 -- 172.21.15.73:0/528788245 >> 172.21.15.73:0/528788245 conn(0x7ff1b40cb000 msgr2=0x7ff1b40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 -- 172.21.15.73:0/528788245 shutdown_connections 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.839+0000 7ff1b359e700 1 -- 172.21.15.73:0/528788245 wait complete. 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.840+0000 7ff1b359e700 1 Processor -- start 2023-11-28T02:52:40.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.840+0000 7ff1b359e700 1 -- start start 2023-11-28T02:52:40.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1b40af820 0x7ff1b4150490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b4150980 0x7ff1b4139820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40aeea0 con 0x7ff1a00038e0 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b4150de0 con 0x7ff1b40af820 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1b40adfb0 con 0x7ff1b4150980 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60770/0 (socket says 172.21.15.73:60770) 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 -- 172.21.15.73:0/3035042900 learned_addr learned my addr 172.21.15.73:0/3035042900 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1a9d9b700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1b40af820 0x7ff1b4150490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b2d9d700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b4150980 0x7ff1b4139820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 -- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b4150980 msgr2=0x7ff1b4139820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b4150980 0x7ff1b4139820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 -- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1b40af820 msgr2=0x7ff1b4150490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1b40af820 0x7ff1b4150490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 -- 172.21.15.73:0/3035042900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1b40ae290 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.841+0000 7ff1b259c700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff1a4001ed0 tx=0x7ff1a4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.842+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4007640 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.842+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a40046b0 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.842+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1b40ae570 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.842+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4004a30 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.842+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1b40aeb30 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.843+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1b406c7c0 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.843+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff1a4020450 con 0x7ff1a00038e0 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.843+0000 7ff1aaffd700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 0x7ff194061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:40.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.843+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff1a407aea0 con 0x7ff1a00038e0 2023-11-28T02:52:40.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.844+0000 7ff1a9d9b700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 0x7ff194061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:40.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.844+0000 7ff1a9d9b700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 0x7ff194061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff19c006c40 tx=0x7ff19c008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:40.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:40.848+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1a404c460 con 0x7ff1a00038e0 2023-11-28T02:52:41.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.012+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1b40af180 con 0x7ff1a00038e0 2023-11-28T02:52:41.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.013+0000 7ff1aaffd700 1 -- 172.21.15.73:0/3035042900 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13709 (secure 0 0 0) 0x7ff1a401a030 con 0x7ff1a00038e0 2023-11-28T02:52:41.014 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:41.014 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":18,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:40.436617+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 msgr2=0x7ff194061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 0x7ff194061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7ff19c006c40 tx=0x7ff19c008040).stop 2023-11-28T02:52:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 msgr2=0x7ff1b40af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7ff1a4001ed0 tx=0x7ff1a4015000).stop 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 shutdown_connections 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff19405f150 0x7ff194061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1b4150980 0x7ff1b4139820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1a00038e0 0x7ff1b40af330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 --2- 172.21.15.73:0/3035042900 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1b40af820 0x7ff1b4150490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 >> 172.21.15.73:0/3035042900 conn(0x7ff1b40cb000 msgr2=0x7ff1b414b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 shutdown_connections 2023-11-28T02:52:41.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.015+0000 7ff1b359e700 1 -- 172.21.15.73:0/3035042900 wait complete. 2023-11-28T02:52:41.028 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T02:52:41.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:52:41.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.144+0000 7f2d08430700 1 Processor -- start 2023-11-28T02:52:41.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.144+0000 7f2d08430700 1 -- start start 2023-11-28T02:52:41.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d08430700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2d000a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d08430700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2d00158160 0x7f2d00155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d08430700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d00067770 con 0x7f2d0014e600 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d08430700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d00159180 con 0x7f2d00158160 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d08430700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000be8d0 con 0x7f2d000a8280 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d069cd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2d0014e600 0x7f2d0014eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36252/0 (socket says 172.21.15.73:36252) 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d069cd700 1 -- 172.21.15.73:0/435814325 learned_addr learned my addr 172.21.15.73:0/435814325 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d059cb700 1 --2- 172.21.15.73:0/435814325 >> v2:172.21.15.150:3300/0 conn(0x7f2d00158160 0x7f2d00155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d061cc700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2d000a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4233265523 0 0) 0x7f2d00067770 con 0x7f2d0014e600 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2cec0033f0 con 0x7f2d0014e600 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 714649217 0 0) 0x7f2cec0033f0 con 0x7f2d0014e600 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 msgr2=0x7f2d000a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2d000a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 >> v2:172.21.15.150:3300/0 conn(0x7f2d00158160 msgr2=0x7f2d00155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 --2- 172.21.15.73:0/435814325 >> v2:172.21.15.150:3300/0 conn(0x7f2d00158160 0x7f2d00155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d00158600 con 0x7f2d0014e600 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2cf80017f0 con 0x7f2d0014e600 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 >> v1:172.21.15.73:6789/0 conn(0x7f2d0014e600 legacy=0x7f2d0014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2cec0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec003e10 0x7f2cec004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cec004b80 0x7f2cec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d00159180 con 0x7f2cec004b80 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000be8d0 con 0x7f2cec003e10 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.145+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cec00b8e0 con 0x7f2d000a8280 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cec004b80 0x7f2cec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d061cc700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2cec0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d059cb700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec003e10 0x7f2cec004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 -- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 msgr2=0x7f2cec0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a8280 0x7f2cec0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 -- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec003e10 msgr2=0x7f2cec004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec003e10 0x7f2cec004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 -- 172.21.15.73:0/435814325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cec0033f0 con 0x7f2cec004b80 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d059cb700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec003e10 0x7f2cec004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d069cd700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cec004b80 0x7f2cec005000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2cf800c560 tx=0x7f2cf8001d90).ready entity=mon.0 client_cookie=a5df98ce5dd64ce1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf800ce50 con 0x7f2cec004b80 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf8005b20 con 0x7f2cec004b80 2023-11-28T02:52:41.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d051ca700 1 -- 172.21.15.73:0/435814325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf80070d0 con 0x7f2cec004b80 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d08430700 1 -- 172.21.15.73:0/435814325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cec004b80 msgr2=0x7f2cec005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.146+0000 7f2d08430700 1 --2- 172.21.15.73:0/435814325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cec004b80 0x7f2cec005000 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f2cf800c560 tx=0x7f2cf8001d90).stop 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.147+0000 7f2d069cd700 1 -- 172.21.15.73:0/435814325 reap_dead start 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.147+0000 7f2d08430700 1 -- 172.21.15.73:0/435814325 shutdown_connections 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.147+0000 7f2d08430700 1 -- 172.21.15.73:0/435814325 >> 172.21.15.73:0/435814325 conn(0x7f2d00062700 msgr2=0x7f2d00062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.147+0000 7f2d08430700 1 -- 172.21.15.73:0/435814325 shutdown_connections 2023-11-28T02:52:41.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.147+0000 7f2d08430700 1 -- 172.21.15.73:0/435814325 wait complete. 2023-11-28T02:52:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.148+0000 7f2d08430700 1 Processor -- start 2023-11-28T02:52:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.148+0000 7f2d08430700 1 -- start start 2023-11-28T02:52:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.148+0000 7f2d08430700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec004b80 0x7f2d000c4880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.148+0000 7f2d08430700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 0x7f2d000c06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000c2d90 con 0x7f2d0014dce0 2023-11-28T02:52:41.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000c4d70 con 0x7f2cec004b80 2023-11-28T02:52:41.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d000a6af0 con 0x7f2d000a6670 2023-11-28T02:52:41.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60774/0 (socket says 172.21.15.73:60774) 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 -- 172.21.15.73:0/4038045401 learned_addr learned my addr 172.21.15.73:0/4038045401 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d061cc700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec004b80 0x7f2d000c4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d069cd700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 0x7f2d000c06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 -- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 msgr2=0x7f2d000c06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 0x7f2d000c06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 -- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec004b80 msgr2=0x7f2d000c4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec004b80 0x7f2d000c4880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 -- 172.21.15.73:0/4038045401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d000c2180 con 0x7f2d0014dce0 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d069cd700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 0x7f2d000c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:41.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d059cb700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f2cf0011620 tx=0x7f2cf0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf00145f0 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf0014750 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d000c2460 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf0014ad0 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.149+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d000c2a20 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.150+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d0006c860 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.150+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2cf0015480 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.151+0000 7f2cf67fc700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 0x7f2ce4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.151+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f2cf007a550 con 0x7f2d0014dce0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.151+0000 7f2d061cc700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 0x7f2ce4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.151+0000 7f2d061cc700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 0x7f2ce4061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2cfc006d80 tx=0x7f2cfc009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.155+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2cf004bb10 con 0x7f2d0014dce0 2023-11-28T02:52:41.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.316+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f2d001588e0 con 0x7f2d0014dce0 2023-11-28T02:52:41.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.316+0000 7f2cf67fc700 1 -- 172.21.15.73:0/4038045401 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v18) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2cf004f130 con 0x7f2d0014dce0 2023-11-28T02:52:41.318 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:52:41.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 msgr2=0x7f2ce4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 0x7f2ce4061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2cfc006d80 tx=0x7f2cfc009060).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 msgr2=0x7f2d000a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f2cf0011620 tx=0x7f2cf0008040).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 shutdown_connections 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce405f150 0x7f2ce4061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d000a6670 0x7f2d000c06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cec004b80 0x7f2d000c4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 --2- 172.21.15.73:0/4038045401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d0014dce0 0x7f2d000a6180 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 >> 172.21.15.73:0/4038045401 conn(0x7f2d00062700 msgr2=0x7f2d00096ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 shutdown_connections 2023-11-28T02:52:41.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.318+0000 7f2d08430700 1 -- 172.21.15.73:0/4038045401 wait complete. 2023-11-28T02:52:41.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:52:41.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.447+0000 7f0ed242c700 1 Processor -- start 2023-11-28T02:52:41.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.447+0000 7f0ed242c700 1 -- start start 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ed242c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ecc158a30 0x7f0ecc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ed242c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc0c72d0 con 0x7f0ecc0c6190 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ed242c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ecc064570 con 0x7f0ecc098d10 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ed242c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ecc154bc0 con 0x7f0ecc158a30 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ecb7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0ecc0c6190 0x7f0ecc158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36256/0 (socket says 172.21.15.73:36256) 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.448+0000 7f0ecb7fe700 1 -- 172.21.15.73:0/283794123 learned_addr learned my addr 172.21.15.73:0/283794123 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 584768941 0 0) 0x7f0ecc0c72d0 con 0x7f0ecc0c6190 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ed09c9700 1 --2- 172.21.15.73:0/283794123 >> v2:172.21.15.150:3300/0 conn(0x7f0ecc158a30 0x7f0ecc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0eb40033f0 con 0x7f0ecc0c6190 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1383924464 0 0) 0x7f0eb40033f0 con 0x7f0ecc0c6190 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 >> v2:172.21.15.150:3300/0 conn(0x7f0ecc158a30 msgr2=0x7f0ecc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 --2- 172.21.15.73:0/283794123 >> v2:172.21.15.150:3300/0 conn(0x7f0ecc158a30 0x7f0ecc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 >> v1:172.21.15.150:6789/0 conn(0x7f0ecc098d10 legacy=0x7f0ecc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0c66c0 con 0x7f0ecc0c6190 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f0ec0001470 con 0x7f0ecc0c6190 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 >> v1:172.21.15.73:6789/0 conn(0x7f0ecc0c6190 legacy=0x7f0ecc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0eb4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb4004690 0x7f0eb4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb4005000 0x7f0eb400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc154bc0 con 0x7f0eb40038e0 2023-11-28T02:52:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb400beb0 con 0x7f0eb4004690 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0eb400c070 con 0x7f0eb4005000 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0eb4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecb7fe700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb4004690 0x7f0eb4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ed09c9700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb4005000 0x7f0eb400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 -- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb4005000 msgr2=0x7f0eb400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0eb4005000 0x7f0eb400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 -- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb4004690 msgr2=0x7f0eb4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0eb4004690 0x7f0eb4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 -- 172.21.15.73:0/283794123 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0eb40033f0 con 0x7f0eb40038e0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ed09c9700 1 -- 172.21.15.73:0/283794123 reap_dead start 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0eb4003d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0ebc001730 tx=0x7f0ebc007bb0).ready entity=mon.0 client_cookie=20e4502bd706141c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc007020 con 0x7f0eb40038e0 2023-11-28T02:52:41.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.449+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc007180 con 0x7f0eb40038e0 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ecaffd700 1 -- 172.21.15.73:0/283794123 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc007520 con 0x7f0eb40038e0 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 -- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 msgr2=0x7f0eb4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0eb4003d00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f0ebc001730 tx=0x7f0ebc007bb0).stop 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 -- 172.21.15.73:0/283794123 shutdown_connections 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 --2- 172.21.15.73:0/283794123 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0eb4003d00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 -- 172.21.15.73:0/283794123 >> 172.21.15.73:0/283794123 conn(0x7f0ecc0cb310 msgr2=0x7f0ecc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 -- 172.21.15.73:0/283794123 shutdown_connections 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.450+0000 7f0ed242c700 1 -- 172.21.15.73:0/283794123 wait complete. 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.451+0000 7f0ed242c700 1 Processor -- start 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.451+0000 7f0ed242c700 1 -- start start 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 0x7f0ecc14a750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 0x7f0ecc150580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc155980 con 0x7f0eb40038e0 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0a9cb0 con 0x7f0ecc14ac40 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed242c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ecc0a9f90 con 0x7f0ecc0a94a0 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60782/0 (socket says 172.21.15.73:60782) 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 -- 172.21.15.73:0/4179054707 learned_addr learned my addr 172.21.15.73:0/4179054707 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ed09c9700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 0x7f0ecc150580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecb7fe700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 0x7f0ecc14a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 -- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 msgr2=0x7f0ecc14a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 0x7f0ecc14a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 -- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 msgr2=0x7f0ecc150580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.452+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 0x7f0ecc150580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ecbfff700 1 -- 172.21.15.73:0/4179054707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ecc0aa270 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ed09c9700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 0x7f0ecc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ecb7fe700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 0x7f0ecc14a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ecbfff700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0eb400be80 tx=0x7f0ebc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc006ff0 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ebc013670 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ecc0aa550 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ebc013a30 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.453+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ecc150a70 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.454+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ecc155c60 con 0x7f0eb40038e0 2023-11-28T02:52:41.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.454+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0ebc01f430 con 0x7f0eb40038e0 2023-11-28T02:52:41.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.454+0000 7f0eb3fff700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 0x7f0eb8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.454+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0ebc07a8b0 con 0x7f0eb40038e0 2023-11-28T02:52:41.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.454+0000 7f0ecb7fe700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 0x7f0eb8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.455+0000 7f0ecb7fe700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 0x7f0eb8061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0ec00015a0 tx=0x7f0ec0009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.458+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0ebc04be70 con 0x7f0eb40038e0 2023-11-28T02:52:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.620+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ecc06c860 con 0x7f0eb40038e0 2023-11-28T02:52:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.620+0000 7f0eb3fff700 1 -- 172.21.15.73:0/4179054707 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v18) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0ebc007e60 con 0x7f0eb40038e0 2023-11-28T02:52:41.622 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 msgr2=0x7f0eb8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 0x7f0eb8061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f0ec00015a0 tx=0x7f0ec0009040).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 msgr2=0x7f0ecc14b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f0eb400be80 tx=0x7f0ebc015000).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 shutdown_connections 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0eb805f150 0x7f0eb8061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ecc14ac40 0x7f0ecc150580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0eb40038e0 0x7f0ecc14b3b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 --2- 172.21.15.73:0/4179054707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ecc0a94a0 0x7f0ecc14a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 >> 172.21.15.73:0/4179054707 conn(0x7f0ecc0cb310 msgr2=0x7f0ecc151970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 shutdown_connections 2023-11-28T02:52:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.622+0000 7f0ed242c700 1 -- 172.21.15.73:0/4179054707 wait complete. 2023-11-28T02:52:41.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T02:52:41.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.748+0000 7f4bcaf6c700 1 Processor -- start 2023-11-28T02:52:41.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.748+0000 7f4bcaf6c700 1 -- start start 2023-11-28T02:52:41.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bcaf6c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4bc41544e0 0x7f4bc41548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bcaf6c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40c69a0 con 0x7f4bc406a610 2023-11-28T02:52:41.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bcaf6c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bc40c13a0 con 0x7f4bc40c6e30 2023-11-28T02:52:41.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bcaf6c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bc40c1680 con 0x7f4bc41544e0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bc8d08700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4bc41544e0 0x7f4bc41548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bc3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4bc406a610 0x7f4bc40c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36268/0 (socket says 172.21.15.73:36268) 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.749+0000 7f4bc3fff700 1 -- 172.21.15.73:0/2637948166 learned_addr learned my addr 172.21.15.73:0/2637948166 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc8d08700 1 -- 172.21.15.73:0/2637948166 >> v1:172.21.15.150:6789/0 conn(0x7f4bc40c6e30 legacy=0x7f4bc40a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc8d08700 1 -- 172.21.15.73:0/2637948166 >> v1:172.21.15.73:6789/0 conn(0x7f4bc406a610 legacy=0x7f4bc40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc8d08700 1 -- 172.21.15.73:0/2637948166 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bc40c6630 con 0x7f4bc41544e0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc37fe700 1 -- 172.21.15.73:0/2637948166 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865005617 0 0) 0x7f4bc40c69a0 con 0x7f4bc406a610 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc8d08700 1 --2- 172.21.15.73:0/2637948166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc41544e0 0x7f4bc41548c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4bb4009eb0 tx=0x7f4bb4009780).ready entity=mon.0 client_cookie=85764c1b1d26253f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc37fe700 1 -- 172.21.15.73:0/2637948166 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb400e070 con 0x7f4bc41544e0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc37fe700 1 -- 172.21.15.73:0/2637948166 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb400bc30 con 0x7f4bc41544e0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bc37fe700 1 -- 172.21.15.73:0/2637948166 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4012410 con 0x7f4bc41544e0 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/2637948166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc41544e0 msgr2=0x7f4bc41548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/2637948166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc41544e0 0x7f4bc41548c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f4bb4009eb0 tx=0x7f4bb4009780).stop 2023-11-28T02:52:41.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/2637948166 shutdown_connections 2023-11-28T02:52:41.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/2637948166 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc41544e0 0x7f4bc41548c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/2637948166 >> 172.21.15.73:0/2637948166 conn(0x7f4bc4085020 msgr2=0x7f4bc4094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/2637948166 shutdown_connections 2023-11-28T02:52:41.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.750+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/2637948166 wait complete. 2023-11-28T02:52:41.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.751+0000 7f4bcaf6c700 1 Processor -- start 2023-11-28T02:52:41.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 -- start start 2023-11-28T02:52:41.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 0x7f4bc4152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc41544e0 0x7f4bc41534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc41539c0 con 0x7f4bc406a610 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4150ec0 con 0x7f4bc41544e0 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bcaf6c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bc4151720 con 0x7f4bc40c6e30 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bc8d08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bc8d08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60788/0 (socket says 172.21.15.73:60788) 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.752+0000 7f4bc8d08700 1 -- 172.21.15.73:0/3420014721 learned_addr learned my addr 172.21.15.73:0/3420014721 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc9509700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc41544e0 0x7f4bc41534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc3fff700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 0x7f4bc4152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 -- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 msgr2=0x7f4bc4152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 0x7f4bc4152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 -- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc41544e0 msgr2=0x7f4bc41534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc41544e0 0x7f4bc41534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 -- 172.21.15.73:0/3420014721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bb4009a20 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc3fff700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 0x7f4bc4152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc8d08700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4bb40097c0 tx=0x7f4bb4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb4021070 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bb401c660 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bc4155900 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bb401c9e0 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.753+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bc41510d0 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.754+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bc406c860 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.754+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4bb401cd50 con 0x7f4bc406a610 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.755+0000 7f4bc0ff9700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 0x7f4bb0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:41.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.755+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4bb407bc60 con 0x7f4bc406a610 2023-11-28T02:52:41.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.755+0000 7f4bc3fff700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 0x7f4bb0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:41.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.755+0000 7f4bc3fff700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 0x7f4bb0061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4bc40ad530 tx=0x7f4bb8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:41.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.759+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4bb404d280 con 0x7f4bc406a610 2023-11-28T02:52:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.923+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f4bc40990e0 con 0x7f4bc406a610 2023-11-28T02:52:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.923+0000 7f4bc0ff9700 1 -- 172.21.15.73:0/3420014721 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v18) v1 ==== 84+0+143 (secure 0 0 0) 0x7f4bb40508a0 con 0x7f4bc406a610 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:52:41.925 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T02:52:41.926 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T02:52:41.926 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:52:41.926 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 msgr2=0x7f4bb0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 0x7f4bb0061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4bc40ad530 tx=0x7f4bb8009040).stop 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 msgr2=0x7f4bc4150170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f4bb40097c0 tx=0x7f4bb4017000).stop 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 shutdown_connections 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4bb005f150 0x7f4bb0061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4bc41544e0 0x7f4bc41534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4bc406a610 0x7f4bc4150170 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 --2- 172.21.15.73:0/3420014721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4bc40c6e30 0x7f4bc4152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:41.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 >> 172.21.15.73:0/3420014721 conn(0x7f4bc4085020 msgr2=0x7f4bc40caad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:41.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 shutdown_connections 2023-11-28T02:52:41.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:41.925+0000 7f4bcaf6c700 1 -- 172.21.15.73:0/3420014721 wait complete. 2023-11-28T02:52:41.939 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T02:52:41.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T02:52:42.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.054+0000 7f514882a700 1 Processor -- start 2023-11-28T02:52:42.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.054+0000 7f514882a700 1 -- start start 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514882a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144154530 0x7f5144154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514882a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51440aa220 0x7f51440aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514882a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51440c6630 con 0x7f51440a7ff0 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514882a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51440c6390 con 0x7f5144154530 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514882a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51440aaba0 con 0x7f51440aa220 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144154530 0x7f5144154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5144154530 0x7f5144154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60804/0 (socket says 172.21.15.73:60804) 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 -- 172.21.15.73:0/3121389064 learned_addr learned my addr 172.21.15.73:0/3121389064 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:42.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f5142d9d700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51440aa220 0x7f51440aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 -- 172.21.15.73:0/3121389064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51440aa220 msgr2=0x7f51440aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51440aa220 0x7f51440aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 -- 172.21.15.73:0/3121389064 >> v1:172.21.15.150:6789/0 conn(0x7f51440a7ff0 legacy=0x7f51440c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.055+0000 7f514259c700 1 -- 172.21.15.73:0/3121389064 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51440a8470 con 0x7f5144154530 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514259c700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144154530 0x7f5144154910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f5134009f70 tx=0x7f5134009840).ready entity=mon.0 client_cookie=570efe77d68e5696 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514159a700 1 -- 172.21.15.73:0/3121389064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400e070 con 0x7f5144154530 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514159a700 1 -- 172.21.15.73:0/3121389064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f513400bad0 con 0x7f5144154530 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514159a700 1 -- 172.21.15.73:0/3121389064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400be50 con 0x7f5144154530 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 -- 172.21.15.73:0/3121389064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144154530 msgr2=0x7f5144154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144154530 0x7f5144154910 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7f5134009f70 tx=0x7f5134009840).stop 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 -- 172.21.15.73:0/3121389064 shutdown_connections 2023-11-28T02:52:42.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51440aa220 0x7f51440aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 --2- 172.21.15.73:0/3121389064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5144154530 0x7f5144154910 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 -- 172.21.15.73:0/3121389064 >> 172.21.15.73:0/3121389064 conn(0x7f5144094930 msgr2=0x7f514406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 -- 172.21.15.73:0/3121389064 shutdown_connections 2023-11-28T02:52:42.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.056+0000 7f514882a700 1 -- 172.21.15.73:0/3121389064 wait complete. 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.057+0000 7f514882a700 1 Processor -- start 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.057+0000 7f514882a700 1 -- start start 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 0x7f51440c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5144154530 0x7f51440c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51440ae230 con 0x7f51440a7ff0 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5144151000 con 0x7f51440aa220 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514882a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51441682c0 con 0x7f5144154530 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60818/0 (socket says 172.21.15.73:60818) 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f514259c700 1 -- 172.21.15.73:0/2973766033 learned_addr learned my addr 172.21.15.73:0/2973766033 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f5141d9b700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 0x7f51440c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.058+0000 7f5142d9d700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5144154530 0x7f51440c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 -- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5144154530 msgr2=0x7f51440c0e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5144154530 0x7f51440c0e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 -- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 msgr2=0x7f51440c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 0x7f51440c3e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 -- 172.21.15.73:0/2973766033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5134009ae0 con 0x7f51440a7ff0 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f5141d9b700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 0x7f51440c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514259c700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f513400b770 tx=0x7f5134009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513400be50 con 0x7f51440a7ff0 2023-11-28T02:52:42.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5134001010 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51441685a0 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f513401d6c0 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.059+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5144168b60 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.060+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f514406c860 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.060+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f513401dab0 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.060+0000 7f513a7fc700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 0x7f5124061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.061+0000 7f5141d9b700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 0x7f5124061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.061+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f513407be80 con 0x7f51440a7ff0 2023-11-28T02:52:42.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.061+0000 7f5141d9b700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 0x7f5124061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5144167d20 tx=0x7f512c009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:42.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.065+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f513404d4a0 con 0x7f51440a7ff0 2023-11-28T02:52:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:42.228+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f5144064230 con 0x7f51440a7ff0 2023-11-28T02:52:43.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.073+0000 7f513a7fc700 1 -- 172.21.15.73:0/2973766033 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v19) v1 ==== 152+0+0 (secure 0 0 0) 0x7f5134050ac0 con 0x7f51440a7ff0 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 msgr2=0x7f5124061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 0x7f5124061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f5144167d20 tx=0x7f512c009040).stop 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 msgr2=0x7f5144150b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f513400b770 tx=0x7f5134009840).stop 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 shutdown_connections 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f512405f150 0x7f5124061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5144154530 0x7f51440c0e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51440a7ff0 0x7f5144150b10 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 --2- 172.21.15.73:0/2973766033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51440aa220 0x7f51440c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 >> 172.21.15.73:0/2973766033 conn(0x7f5144094930 msgr2=0x7f51440c8dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:43.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 shutdown_connections 2023-11-28T02:52:43.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.076+0000 7f514882a700 1 -- 172.21.15.73:0/2973766033 wait complete. 2023-11-28T02:52:43.079 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T02:52:43.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T02:52:43.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.222+0000 7fdf8c55b700 1 Processor -- start 2023-11-28T02:52:43.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.222+0000 7fdf8c55b700 1 -- start start 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf8c55b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf84098d10 0x7fdf840c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf8c55b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf840c72d0 con 0x7fdf840c6190 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf8c55b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf84064570 con 0x7fdf84158320 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf8c55b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf84154bc0 con 0x7fdf84098d10 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf89af6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdf840c6190 0x7fdf84063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36274/0 (socket says 172.21.15.73:36274) 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf89af6700 1 -- 172.21.15.73:0/3757828516 learned_addr learned my addr 172.21.15.73:0/3757828516 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4212854276 0 0) 0x7fdf840c72d0 con 0x7fdf840c6190 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf700033f0 con 0x7fdf840c6190 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.223+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3757828516 >> v2:172.21.15.150:3300/0 conn(0x7fdf84098d10 0x7fdf840c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423770022 0 0) 0x7fdf700033f0 con 0x7fdf840c6190 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 >> v2:172.21.15.150:3300/0 conn(0x7fdf84098d10 msgr2=0x7fdf840c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:43.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 --2- 172.21.15.73:0/3757828516 >> v2:172.21.15.150:3300/0 conn(0x7fdf84098d10 0x7fdf840c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 >> v1:172.21.15.150:6789/0 conn(0x7fdf84158320 legacy=0x7fdf840a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf840c66c0 con 0x7fdf840c6190 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fdf74001470 con 0x7fdf840c6190 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 >> v1:172.21.15.73:6789/0 conn(0x7fdf840c6190 legacy=0x7fdf84063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf70003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf70004690 0x7fdf70004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf70005000 0x7fdf7000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf84154bc0 con 0x7fdf700038e0 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7000beb0 con 0x7fdf70005000 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf7000c070 con 0x7fdf70004690 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf70003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8aaf8700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf70005000 0x7fdf7000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf89af6700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf70004690 0x7fdf70004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf70004690 msgr2=0x7fdf70004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf70004690 0x7fdf70004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf70005000 msgr2=0x7fdf7000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf70005000 0x7fdf7000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3757828516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf700033f0 con 0x7fdf700038e0 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8aaf8700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf70005000 0x7fdf7000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8aaf8700 1 -- 172.21.15.73:0/3757828516 reap_dead start 2023-11-28T02:52:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf70003d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fdf80009a60 tx=0x7fdf80009060).ready entity=mon.0 client_cookie=ed2404a47acd4a6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80009ea0 con 0x7fdf700038e0 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf80009430 con 0x7fdf700038e0 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.224+0000 7fdf892f5700 1 -- 172.21.15.73:0/3757828516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8000e870 con 0x7fdf700038e0 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 msgr2=0x7fdf70003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf70003d00 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fdf80009a60 tx=0x7fdf80009060).stop 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3757828516 shutdown_connections 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3757828516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf70003d00 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3757828516 >> 172.21.15.73:0/3757828516 conn(0x7fdf840cb310 msgr2=0x7fdf840cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3757828516 shutdown_connections 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.225+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3757828516 wait complete. 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.226+0000 7fdf8c55b700 1 Processor -- start 2023-11-28T02:52:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.226+0000 7fdf8c55b700 1 -- start start 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf840a88a0 0x7fdf840a8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf84155e60 0x7fdf841562c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840a9e10 con 0x7fdf700038e0 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf840a9210 con 0x7fdf840a88a0 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8c55b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8414b810 con 0x7fdf84155e60 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60846/0 (socket says 172.21.15.73:60846) 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3309317403 learned_addr learned my addr 172.21.15.73:0/3309317403 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf89af6700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf840a88a0 0x7fdf840a8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8aaf8700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf84155e60 0x7fdf841562c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf84155e60 msgr2=0x7fdf841562c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf84155e60 0x7fdf841562c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf840a88a0 msgr2=0x7fdf840a8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf840a88a0 0x7fdf840a8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 -- 172.21.15.73:0/3309317403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf84154dc0 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf8a2f7700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fdf8000e4b0 tx=0x7fdf8001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.227+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf8001ba60 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.228+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf8001bbc0 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.228+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf841550a0 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.228+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf800243f0 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.228+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf84155660 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.228+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8406c860 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.229+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdf800247e0 con 0x7fdf700038e0 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.229+0000 7fdf7a7fc700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 0x7fdf680615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.229+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fdf8007b3c0 con 0x7fdf700038e0 2023-11-28T02:52:43.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.229+0000 7fdf89af6700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 0x7fdf680615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:43.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.230+0000 7fdf89af6700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 0x7fdf680615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdf74000fd0 tx=0x7fdf74007040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:43.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.234+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdf8004c840 con 0x7fdf700038e0 2023-11-28T02:52:43.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:43.398+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdf840c69a0 con 0x7fdf700038e0 2023-11-28T02:52:44.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.079+0000 7fdf7a7fc700 1 -- 172.21.15.73:0/3309317403 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v20) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdf800290d0 con 0x7fdf700038e0 2023-11-28T02:52:44.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 msgr2=0x7fdf680615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 0x7fdf680615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fdf74000fd0 tx=0x7fdf74007040).stop 2023-11-28T02:52:44.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 msgr2=0x7fdf840a83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fdf8000e4b0 tx=0x7fdf8001a000).stop 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 shutdown_connections 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf6805f120 0x7fdf680615e0 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0x7fdf74000fd0 tx=0x7fdf74007040).stop 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf84155e60 0x7fdf841562c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf700038e0 0x7fdf840a83b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 --2- 172.21.15.73:0/3309317403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf840a88a0 0x7fdf840a8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.084+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 >> 172.21.15.73:0/3309317403 conn(0x7fdf840cb310 msgr2=0x7fdf840ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.085+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 shutdown_connections 2023-11-28T02:52:44.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.088+0000 7fdf8c55b700 1 -- 172.21.15.73:0/3309317403 wait complete. 2023-11-28T02:52:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T02:52:44.111 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T02:52:44.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.225+0000 7fd641d37700 1 Processor -- start 2023-11-28T02:52:44.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.225+0000 7fd641d37700 1 -- start start 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd641d37700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd63c0c6190 0x7fd63c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd641d37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 0x7fd63c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd641d37700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd63c158cc0 con 0x7fd63c098d10 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd641d37700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd63c0c72d0 con 0x7fd63c0c6190 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd641d37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c064570 con 0x7fd63c158320 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd63affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd63c0c6190 0x7fd63c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd63affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd63c0c6190 0x7fd63c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42350/0 (socket says 172.21.15.73:42350) 2023-11-28T02:52:44.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.226+0000 7fd63affd700 1 -- 172.21.15.73:0/1592102212 learned_addr learned my addr 172.21.15.73:0/1592102212 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63bfff700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 0x7fd63c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63affd700 1 -- 172.21.15.73:0/1592102212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 msgr2=0x7fd63c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63affd700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 0x7fd63c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63affd700 1 -- 172.21.15.73:0/1592102212 >> v1:172.21.15.150:6789/0 conn(0x7fd63c098d10 legacy=0x7fd63c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63affd700 1 -- 172.21.15.73:0/1592102212 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd63c154bc0 con 0x7fd63c0c6190 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63bfff700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 0x7fd63c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63affd700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c0c6190 0x7fd63c0c7760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd630006fd0 tx=0x7fd6300090f0).ready entity=mon.0 client_cookie=cc576dc44e7468bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63a7fc700 1 -- 172.21.15.73:0/1592102212 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63000b600 con 0x7fd63c0c6190 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63a7fc700 1 -- 172.21.15.73:0/1592102212 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd63000b760 con 0x7fd63c0c6190 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd63a7fc700 1 -- 172.21.15.73:0/1592102212 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd63000bae0 con 0x7fd63c0c6190 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd641d37700 1 -- 172.21.15.73:0/1592102212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c0c6190 msgr2=0x7fd63c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd641d37700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c0c6190 0x7fd63c0c7760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fd630006fd0 tx=0x7fd6300090f0).stop 2023-11-28T02:52:44.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd641d37700 1 -- 172.21.15.73:0/1592102212 shutdown_connections 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd641d37700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c158320 0x7fd63c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.227+0000 7fd641d37700 1 --2- 172.21.15.73:0/1592102212 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c0c6190 0x7fd63c0c7760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.228+0000 7fd641d37700 1 -- 172.21.15.73:0/1592102212 >> 172.21.15.73:0/1592102212 conn(0x7fd63c0cb310 msgr2=0x7fd63c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.228+0000 7fd641d37700 1 -- 172.21.15.73:0/1592102212 shutdown_connections 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.228+0000 7fd641d37700 1 -- 172.21.15.73:0/1592102212 wait complete. 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.229+0000 7fd641d37700 1 Processor -- start 2023-11-28T02:52:44.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.229+0000 7fd641d37700 1 -- start start 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c0c6190 0x7fd63c155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c158320 0x7fd63c156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0aeea0 con 0x7fd63c098d10 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c0af180 con 0x7fd63c158320 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd63c156f40 con 0x7fd63c0c6190 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42360/0 (socket says 172.21.15.73:42360) 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 -- 172.21.15.73:0/1286061672 learned_addr learned my addr 172.21.15.73:0/1286061672 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63bfff700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c158320 0x7fd63c156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63affd700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c0c6190 0x7fd63c155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 -- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c0c6190 msgr2=0x7fd63c155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c0c6190 0x7fd63c155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 -- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c158320 msgr2=0x7fd63c156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c158320 0x7fd63c156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 -- 172.21.15.73:0/1286061672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd630009370 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd63b7fe700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fd62c00c840 tx=0x7fd62c0025e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c010510 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd62c0106b0 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd63c157220 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd62c010a70 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.230+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd63c0b1200 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.231+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd63c06c860 con 0x7fd63c098d10 2023-11-28T02:52:44.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.231+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd62c01b400 con 0x7fd63c098d10 2023-11-28T02:52:44.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.232+0000 7fd6237fe700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 0x7fd628061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.232+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fd62c079ff0 con 0x7fd63c098d10 2023-11-28T02:52:44.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.232+0000 7fd63affd700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 0x7fd628061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.232+0000 7fd63affd700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 0x7fd628061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd630006fd0 tx=0x7fd630017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.236+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd62c04b2c0 con 0x7fd63c098d10 2023-11-28T02:52:44.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.443+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fd63c0c66c0 con 0x7fd63c098d10 2023-11-28T02:52:44.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.447+0000 7fd6237fe700 1 -- 172.21.15.73:0/1286061672 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fd62c010e60 con 0x7fd63c098d10 2023-11-28T02:52:44.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 msgr2=0x7fd628061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 0x7fd628061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd630006fd0 tx=0x7fd630017000).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 msgr2=0x7fd63c0b0d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7fd62c00c840 tx=0x7fd62c0025e0).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 shutdown_connections 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd62805f150 0x7fd628061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd63c158320 0x7fd63c156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd63c098d10 0x7fd63c0b0d10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 --2- 172.21.15.73:0/1286061672 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd63c0c6190 0x7fd63c155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 >> 172.21.15.73:0/1286061672 conn(0x7fd63c0cb310 msgr2=0x7fd63c063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 shutdown_connections 2023-11-28T02:52:44.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.451+0000 7fd641d37700 1 -- 172.21.15.73:0/1286061672 wait complete. 2023-11-28T02:52:44.465 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 3 and data pool 4 2023-11-28T02:52:44.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:44.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.586+0000 7f92ef59e700 1 Processor -- start 2023-11-28T02:52:44.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.586+0000 7f92ef59e700 1 -- start start 2023-11-28T02:52:44.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ef59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92f014e980 0x7f92f0152220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ef59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f92f014ed60 0x7f92f00c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ef59e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f006bbc0 con 0x7f92f0155470 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ef59e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f00c1df0 con 0x7f92f014e980 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ef59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f0154fe0 con 0x7f92f014ed60 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f92f014e980 0x7f92f0152220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92eed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92f0155470 0x7f92f00c61c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33656/0 (socket says 172.21.15.73:33656) 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92eed9d700 1 -- 172.21.15.73:0/2719470424 learned_addr learned my addr 172.21.15.73:0/2719470424 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92edd9b700 1 --2- 172.21.15.73:0/2719470424 >> v2:172.21.15.150:3300/0 conn(0x7f92f014ed60 0x7f92f00c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 -- 172.21.15.73:0/2719470424 >> v2:172.21.15.150:3300/0 conn(0x7f92f014ed60 msgr2=0x7f92f00c45f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 --2- 172.21.15.73:0/2719470424 >> v2:172.21.15.150:3300/0 conn(0x7f92f014ed60 0x7f92f00c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 -- 172.21.15.73:0/2719470424 >> v1:172.21.15.73:6789/0 conn(0x7f92f0155470 legacy=0x7f92f00c61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ed59a700 1 -- 172.21.15.73:0/2719470424 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 208612738 0 0) 0x7f92f006bbc0 con 0x7f92f0155470 2023-11-28T02:52:44.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 -- 172.21.15.73:0/2719470424 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f006b640 con 0x7f92f014e980 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.587+0000 7f92ee59c700 1 --2- 172.21.15.73:0/2719470424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014e980 0x7f92f0152220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f92f0065ff0 tx=0x7f92e0009060).ready entity=mon.0 client_cookie=4f2ba165b634efa5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ed59a700 1 -- 172.21.15.73:0/2719470424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e00090d0 con 0x7f92f014e980 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ed59a700 1 -- 172.21.15.73:0/2719470424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e000b580 con 0x7f92f014e980 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ed59a700 1 -- 172.21.15.73:0/2719470424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e000b900 con 0x7f92f014e980 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 -- 172.21.15.73:0/2719470424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014e980 msgr2=0x7f92f0152220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 --2- 172.21.15.73:0/2719470424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014e980 0x7f92f0152220 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f92f0065ff0 tx=0x7f92e0009060).stop 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 -- 172.21.15.73:0/2719470424 shutdown_connections 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 --2- 172.21.15.73:0/2719470424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014e980 0x7f92f0152220 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 --2- 172.21.15.73:0/2719470424 >> v2:172.21.15.150:3300/0 conn(0x7f92f014ed60 0x7f92f00c45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 -- 172.21.15.73:0/2719470424 >> 172.21.15.73:0/2719470424 conn(0x7f92f00c7eb0 msgr2=0x7f92f00c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 -- 172.21.15.73:0/2719470424 shutdown_connections 2023-11-28T02:52:44.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.588+0000 7f92ef59e700 1 -- 172.21.15.73:0/2719470424 wait complete. 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.589+0000 7f92ef59e700 1 Processor -- start 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.589+0000 7f92ef59e700 1 -- start start 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92f014e980 0x7f92f013f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 0x7f92f01436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00aa820 con 0x7f92f014ed60 2023-11-28T02:52:44.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00aa480 con 0x7f92f0155470 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92ef59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f013fae0 con 0x7f92f014e980 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42380/0 (socket says 172.21.15.73:42380) 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 -- 172.21.15.73:0/843427837 learned_addr learned my addr 172.21.15.73:0/843427837 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 -- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92f014e980 msgr2=0x7f92f013f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92f014e980 0x7f92f013f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92eed9d700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 0x7f92f01436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 -- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 msgr2=0x7f92f01436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 0x7f92f01436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 -- 172.21.15.73:0/843427837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e00092a0 con 0x7f92f014ed60 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92eed9d700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 0x7f92f01436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:44.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.590+0000 7f92edd9b700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f92e400c880 tx=0x7f92e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.591+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4010580 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.591+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e40106e0 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.591+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f013fdc0 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.591+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4010a60 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.591+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f00acd50 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.592+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f0154980 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.592+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f92e401b460 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.592+0000 7f92de7fc700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 0x7f92d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.592+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f92e407a180 con 0x7f92f014ed60 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.592+0000 7f92ee59c700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 0x7f92d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.593+0000 7f92ee59c700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 0x7f92d0061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f92e000b340 tx=0x7f92e0017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.596+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92e404b3f0 con 0x7f92f014ed60 2023-11-28T02:52:44.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.759+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f92f006c7c0 con 0x7f92f014ed60 2023-11-28T02:52:44.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.759+0000 7f92de7fc700 1 -- 172.21.15.73:0/843427837 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f92e404ea10 con 0x7f92f014ed60 2023-11-28T02:52:44.761 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:44.761 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":21,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:44.444621+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 msgr2=0x7f92d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 0x7f92d0061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f92e000b340 tx=0x7f92e0017000).stop 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 msgr2=0x7f92f00ac860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f92e400c880 tx=0x7f92e4008040).stop 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 shutdown_connections 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92d005f150 0x7f92d0061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92f0155470 0x7f92f01436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92f014e980 0x7f92f013f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 --2- 172.21.15.73:0/843427837 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92f014ed60 0x7f92f00ac860 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 >> 172.21.15.73:0/843427837 conn(0x7f92f00c7eb0 msgr2=0x7f92f0143fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 shutdown_connections 2023-11-28T02:52:44.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.761+0000 7f92ef59e700 1 -- 172.21.15.73:0/843427837 wait complete. 2023-11-28T02:52:44.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:44.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.887+0000 7fef08588700 1 Processor -- start 2023-11-28T02:52:44.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.887+0000 7fef08588700 1 -- start start 2023-11-28T02:52:44.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef08588700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef000ca180 0x7fef0015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef08588700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef000a9a90 con 0x7fef000caab0 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef08588700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef00154ce0 con 0x7fef000cae90 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef08588700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef000aab80 con 0x7fef000ca180 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef06324700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fef000caab0 0x7fef000aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33658/0 (socket says 172.21.15.73:33658) 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef06324700 1 -- 172.21.15.73:0/4038403951 learned_addr learned my addr 172.21.15.73:0/4038403951 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef06b25700 1 --2- 172.21.15.73:0/4038403951 >> v2:172.21.15.150:3300/0 conn(0x7fef000ca180 0x7fef0015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3853945326 0 0) 0x7fef000a9a90 con 0x7fef000caab0 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.888+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feeec0033f0 con 0x7fef000caab0 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 921328902 0 0) 0x7feeec0033f0 con 0x7fef000caab0 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 >> v2:172.21.15.150:3300/0 conn(0x7fef000ca180 msgr2=0x7fef0015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 --2- 172.21.15.73:0/4038403951 >> v2:172.21.15.150:3300/0 conn(0x7fef000ca180 0x7fef0015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 >> v1:172.21.15.150:6789/0 conn(0x7fef000cae90 legacy=0x7fef000c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000a8e80 con 0x7fef000caab0 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7feefc0015f0 con 0x7fef000caab0 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06b25700 1 --2- 172.21.15.73:0/4038403951 >> v2:172.21.15.150:3300/0 conn(0x7fef000ca180 0x7fef0015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 >> v1:172.21.15.73:6789/0 conn(0x7fef000caab0 legacy=0x7fef000aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7feeec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feeec004690 0x7feeec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeec005000 0x7feeec00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000aab80 con 0x7feeec0038e0 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00bf60 con 0x7feeec005000 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feeec00c0f0 con 0x7feeec004690 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7feeec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06b25700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeec005000 0x7feeec00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05b23700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feeec004690 0x7feeec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 -- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feeec004690 msgr2=0x7feeec004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feeec004690 0x7feeec004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 -- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeec005000 msgr2=0x7feeec00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feeec005000 0x7feeec00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06b25700 1 -- 172.21.15.73:0/4038403951 reap_dead start 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 -- 172.21.15.73:0/4038403951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec0033f0 con 0x7feeec0038e0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef06324700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7feeec003d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7feefc00c160 tx=0x7feefc0077a0).ready entity=mon.0 client_cookie=6f958fe549dfc38b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc0079d0 con 0x7feeec0038e0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0070d0 con 0x7feeec0038e0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.889+0000 7fef05322700 1 -- 172.21.15.73:0/4038403951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc007490 con 0x7feeec0038e0 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 -- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 msgr2=0x7feeec003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7feeec003d00 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7feefc00c160 tx=0x7feefc0077a0).stop 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 -- 172.21.15.73:0/4038403951 shutdown_connections 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 --2- 172.21.15.73:0/4038403951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7feeec003d00 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 -- 172.21.15.73:0/4038403951 >> 172.21.15.73:0/4038403951 conn(0x7fef00094920 msgr2=0x7fef0006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 -- 172.21.15.73:0/4038403951 shutdown_connections 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.890+0000 7fef08588700 1 -- 172.21.15.73:0/4038403951 wait complete. 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.891+0000 7fef08588700 1 Processor -- start 2023-11-28T02:52:44.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.891+0000 7fef08588700 1 -- start start 2023-11-28T02:52:44.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef00155ad0 0x7fef000c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef000c9510 0x7fef000c9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef00155f50 con 0x7feeec0038e0 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c3390 con 0x7fef000c9510 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef000c3710 con 0x7fef00155ad0 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42398/0 (socket says 172.21.15.73:42398) 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 -- 172.21.15.73:0/606093366 learned_addr learned my addr 172.21.15.73:0/606093366 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef05b23700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef00155ad0 0x7fef000c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06b25700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef000c9510 0x7fef000c9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 -- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef00155ad0 msgr2=0x7fef000c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef00155ad0 0x7fef000c2310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 -- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef000c9510 msgr2=0x7fef000c9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef000c9510 0x7fef000c9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 -- 172.21.15.73:0/606093366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef000c9e80 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef06324700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7feeec005640 tx=0x7feefc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc005670 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feefc0057d0 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.892+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef000c2af0 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.893+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feefc005b70 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.893+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef000c30b0 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.893+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef00155ed0 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.894+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feefc013450 con 0x7feeec0038e0 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.894+0000 7feef67fc700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 0x7feee4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:44.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.894+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feefc07af80 con 0x7feeec0038e0 2023-11-28T02:52:44.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.894+0000 7fef05b23700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 0x7feee4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:44.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.894+0000 7fef05b23700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 0x7feee4061640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fef000c5310 tx=0x7feef0009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:44.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:44.898+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feefc0a9af0 con 0x7feeec0038e0 2023-11-28T02:52:45.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.061+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fef0006c860 con 0x7feeec0038e0 2023-11-28T02:52:45.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.061+0000 7feef67fc700 1 -- 172.21.15.73:0/606093366 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+16941 (secure 0 0 0) 0x7feefc04c1f0 con 0x7feeec0038e0 2023-11-28T02:52:45.063 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:45.063 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":21,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:44.444621+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 msgr2=0x7feee4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 0x7feee4061640 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fef000c5310 tx=0x7feef0009040).stop 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 msgr2=0x7fef00154060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7feeec005640 tx=0x7feefc015000).stop 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 shutdown_connections 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feee405f180 0x7feee4061640 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef000c9510 0x7fef000c9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feeec0038e0 0x7fef00154060 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 --2- 172.21.15.73:0/606093366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef00155ad0 0x7fef000c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.063+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 >> 172.21.15.73:0/606093366 conn(0x7fef00094920 msgr2=0x7fef000ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:45.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.064+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 shutdown_connections 2023-11-28T02:52:45.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.064+0000 7fef08588700 1 -- 172.21.15.73:0/606093366 wait complete. 2023-11-28T02:52:45.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T02:52:45.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.191+0000 7feec171f700 1 Processor -- start 2023-11-28T02:52:45.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.191+0000 7feec171f700 1 -- start start 2023-11-28T02:52:45.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feec171f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 0x7feebc06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feec171f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feebc151a20 0x7feebc0be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feec171f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0c3470 con 0x7feebc0be6a0 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feec171f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc0c48f0 con 0x7feebc151a20 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feec171f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc151ea0 con 0x7feebc069d10 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feeb27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feebc151a20 0x7feebc0be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feeb27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feebc151a20 0x7feebc0be1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42410/0 (socket says 172.21.15.73:42410) 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feeb27fc700 1 -- 172.21.15.73:0/1467997501 learned_addr learned my addr 172.21.15.73:0/1467997501 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.192+0000 7feebaffd700 1 --2- 172.21.15.73:0/1467997501 >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 0x7feebc06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeb27fc700 1 -- 172.21.15.73:0/1467997501 >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 msgr2=0x7feebc06a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeb27fc700 1 --2- 172.21.15.73:0/1467997501 >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 0x7feebc06a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeb27fc700 1 -- 172.21.15.73:0/1467997501 >> v1:172.21.15.150:6789/0 conn(0x7feebc0be6a0 legacy=0x7feebc0bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeb27fc700 1 -- 172.21.15.73:0/1467997501 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc152180 con 0x7feebc151a20 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feebaffd700 1 --2- 172.21.15.73:0/1467997501 >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 0x7feebc06a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:45.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeb27fc700 1 --2- 172.21.15.73:0/1467997501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc151a20 0x7feebc0be1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7feebc05f7f0 tx=0x7feea40090f0).ready entity=mon.0 client_cookie=1621b3e824ce116a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeba7fc700 1 -- 172.21.15.73:0/1467997501 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400b600 con 0x7feebc151a20 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeba7fc700 1 -- 172.21.15.73:0/1467997501 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400b760 con 0x7feebc151a20 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feeba7fc700 1 -- 172.21.15.73:0/1467997501 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400bae0 con 0x7feebc151a20 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 -- 172.21.15.73:0/1467997501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc151a20 msgr2=0x7feebc0be1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 --2- 172.21.15.73:0/1467997501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc151a20 0x7feebc0be1b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7feebc05f7f0 tx=0x7feea40090f0).stop 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 -- 172.21.15.73:0/1467997501 shutdown_connections 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 --2- 172.21.15.73:0/1467997501 >> v2:172.21.15.150:3300/0 conn(0x7feebc069d10 0x7feebc06a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 --2- 172.21.15.73:0/1467997501 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc151a20 0x7feebc0be1b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 -- 172.21.15.73:0/1467997501 >> 172.21.15.73:0/1467997501 conn(0x7feebc0642d0 msgr2=0x7feebc0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.193+0000 7feec171f700 1 -- 172.21.15.73:0/1467997501 shutdown_connections 2023-11-28T02:52:45.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.194+0000 7feec171f700 1 -- 172.21.15.73:0/1467997501 wait complete. 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 Processor -- start 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 -- start start 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc069d10 0x7feebc0a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc151a20 0x7feebc0aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0a7f90 con 0x7feebc0be6a0 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.195+0000 7feec171f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0aaaa0 con 0x7feebc151a20 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feec171f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0ab0c0 con 0x7feebc069d10 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42424/0 (socket says 172.21.15.73:42424) 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 -- 172.21.15.73:0/838654176 learned_addr learned my addr 172.21.15.73:0/838654176 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feebb7fe700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc151a20 0x7feebc0aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feebaffd700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc069d10 0x7feebc0a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 -- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc069d10 msgr2=0x7feebc0a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc069d10 0x7feebc0a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 -- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc151a20 msgr2=0x7feebc0aa290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc151a20 0x7feebc0aa290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 -- 172.21.15.73:0/838654176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea4009370 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb27fc700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7feea4011000 tx=0x7feea4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea400b8d0 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea400ba30 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feebc0ab3a0 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea401b620 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.196+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc0ab960 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.197+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc06c860 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.197+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feea401ba10 con 0x7feebc0be6a0 2023-11-28T02:52:45.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.198+0000 7feeb37fe700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 0x7fee9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.198+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feea407b6e0 con 0x7feebc0be6a0 2023-11-28T02:52:45.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.198+0000 7feebaffd700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 0x7fee9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.198+0000 7feebaffd700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 0x7fee9c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7feeac006d80 tx=0x7feeac008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.202+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feea404c9b0 con 0x7feebc0be6a0 2023-11-28T02:52:45.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.411+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7feebc06a620 con 0x7feebc0be6a0 2023-11-28T02:52:45.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.458+0000 7feeb37fe700 1 -- 172.21.15.73:0/838654176 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7feea404ffd0 con 0x7feebc0be6a0 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 msgr2=0x7fee9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 0x7fee9c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7feeac006d80 tx=0x7feeac008040).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 msgr2=0x7feebc0a7aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7feea4011000 tx=0x7feea4017000).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 shutdown_connections 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee9c05f150 0x7fee9c061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc151a20 0x7feebc0aa290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc069d10 0x7feebc0a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 --2- 172.21.15.73:0/838654176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0be6a0 0x7feebc0a7aa0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 >> 172.21.15.73:0/838654176 conn(0x7feebc0642d0 msgr2=0x7feebc0c68b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:45.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 shutdown_connections 2023-11-28T02:52:45.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.460+0000 7feec171f700 1 -- 172.21.15.73:0/838654176 wait complete. 2023-11-28T02:52:45.473 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T02:52:45.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T02:52:45.557 INFO:tasks.ceph.mds.d.smithi150.stderr:2023-11-28T02:52:45.556+0000 7fdbd0bc1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T02:52:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.590+0000 7f2bbe3e2700 1 Processor -- start 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.590+0000 7f2bbe3e2700 1 -- start start 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bbe3e2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2bb80a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bbe3e2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb8067770 con 0x7f2bb8158160 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bbe3e2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb80be8d0 con 0x7f2bb814e600 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bbe3e2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8159180 con 0x7f2bb80a8280 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb77fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2bb8158160 0x7f2bb8155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33668/0 (socket says 172.21.15.73:33668) 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb77fe700 1 -- 172.21.15.73:0/3577218139 learned_addr learned my addr 172.21.15.73:0/3577218139 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2bb80a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1655990856 0 0) 0x7f2bb8067770 con 0x7f2bb8158160 2023-11-28T02:52:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ba00033f0 con 0x7f2bb8158160 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 251013666 0 0) 0x7f2bb80be8d0 con 0x7f2bb814e600 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb8067770 con 0x7f2bb814e600 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3979308822 0 0) 0x7f2ba00033f0 con 0x7f2bb8158160 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 msgr2=0x7f2bb80a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2bb80a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 >> v1:172.21.15.150:6789/0 conn(0x7f2bb814e600 legacy=0x7f2bb814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb8158600 con 0x7f2bb8158160 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2ba8001470 con 0x7f2bb8158160 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 >> v1:172.21.15.73:6789/0 conn(0x7f2bb8158160 legacy=0x7f2bb8155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba0003aa0 0x7f2ba0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0004690 0x7f2ba0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2ba0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb8159180 con 0x7f2ba0003aa0 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba000b910 con 0x7f2ba0004690 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ba000bad0 con 0x7f2bb80a8280 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba0003aa0 0x7f2ba0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.591+0000 7f2bb77fe700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0004690 0x7f2ba0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbc97f700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2ba0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 -- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 msgr2=0x7f2ba0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb80a8280 0x7f2ba0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 -- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0004690 msgr2=0x7f2ba0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0004690 0x7f2ba0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 -- 172.21.15.73:0/3577218139 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00033f0 con 0x7f2ba0003aa0 2023-11-28T02:52:45.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb77fe700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0004690 0x7f2ba0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba0003aa0 0x7f2ba0003ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2bac009a80 tx=0x7f2bac009060).ready entity=mon.0 client_cookie=174403b4078e878c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac009430 con 0x7f2ba0003aa0 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bac00e640 con 0x7f2ba0003aa0 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bb6ffd700 1 -- 172.21.15.73:0/3577218139 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bac00e9c0 con 0x7f2ba0003aa0 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/3577218139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba0003aa0 msgr2=0x7f2ba0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/3577218139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ba0003aa0 0x7f2ba0003ea0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f2bac009a80 tx=0x7f2bac009060).stop 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbc97f700 1 -- 172.21.15.73:0/3577218139 reap_dead start 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/3577218139 shutdown_connections 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/3577218139 >> 172.21.15.73:0/3577218139 conn(0x7f2bb8062700 msgr2=0x7f2bb8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/3577218139 shutdown_connections 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.592+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/3577218139 wait complete. 2023-11-28T02:52:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.593+0000 7f2bbe3e2700 1 Processor -- start 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 -- start start 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0003aa0 0x7f2bb8096040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb814d000 0x7f2bb8157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c1dc0 con 0x7f2bb8096530 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c0d20 con 0x7f2ba0003aa0 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.594+0000 7f2bbe3e2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb80c1000 con 0x7f2bb814d000 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42432/0 (socket says 172.21.15.73:42432) 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 -- 172.21.15.73:0/4085668485 learned_addr learned my addr 172.21.15.73:0/4085668485 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0003aa0 0x7f2bb8096040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bbc97f700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb814d000 0x7f2bb8157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 -- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb814d000 msgr2=0x7f2bb8157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb814d000 0x7f2bb8157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 -- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0003aa0 msgr2=0x7f2bb8096040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0003aa0 0x7f2bb8096040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 -- 172.21.15.73:0/4085668485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb80c12e0 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bb77fe700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2ba80010e0 tx=0x7f2bb80bea60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8010520 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba8010680 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb80c15c0 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba8010a20 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.595+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb814d460 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.596+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb806c860 con 0x7f2bb8096530 2023-11-28T02:52:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.596+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2ba8014430 con 0x7f2bb8096530 2023-11-28T02:52:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.597+0000 7f2b9ffff700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 0x7f2ba4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.597+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2ba807a1e0 con 0x7f2bb8096530 2023-11-28T02:52:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.597+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 0x7f2ba4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.597+0000 7f2bb7fff700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 0x7f2ba4061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2bac006d50 tx=0x7f2bac01a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.601+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ba804b450 con 0x7f2bb8096530 2023-11-28T02:52:45.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:45.806+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f2bb80c3c90 con 0x7f2bb8096530 2023-11-28T02:52:46.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.459+0000 7f2b9ffff700 1 -- 172.21.15.73:0/4085668485 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2ba804ea70 con 0x7f2bb8096530 2023-11-28T02:52:46.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.461+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 msgr2=0x7f2ba4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.461+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 0x7f2ba4061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f2bac006d50 tx=0x7f2bac01a000).stop 2023-11-28T02:52:46.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.461+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 msgr2=0x7f2bb814cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.461+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7f2ba80010e0 tx=0x7f2bb80bea60).stop 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 shutdown_connections 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ba405f150 0x7f2ba4061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb814d000 0x7f2bb8157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ba0003aa0 0x7f2bb8096040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 --2- 172.21.15.73:0/4085668485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb8096530 0x7f2bb814cb10 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 >> 172.21.15.73:0/4085668485 conn(0x7f2bb8062700 msgr2=0x7f2bb814dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 shutdown_connections 2023-11-28T02:52:46.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.462+0000 7f2bbe3e2700 1 -- 172.21.15.73:0/4085668485 wait complete. 2023-11-28T02:52:46.475 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T02:52:46.476 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:52:46.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.587+0000 7f11ff6d0700 1 Processor -- start 2023-11-28T02:52:46.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.587+0000 7f11ff6d0700 1 -- start start 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11ff6d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 0x7f11f80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11ff6d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11ff6d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f80a9a90 con 0x7f11f80caab0 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11ff6d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f80c1490 con 0x7f11f80cae90 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11ff6d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f8154ce0 con 0x7f11f80ca180 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.588+0000 7f11fdc6d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:47020/0 (socket says 172.21.15.73:47020) 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fd46c700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f11f80caab0 0x7f11f80aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59894/0 (socket says 172.21.15.73:59894) 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 -- 172.21.15.73:0/1979010339 learned_addr learned my addr 172.21.15.73:0/1979010339 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:46.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fcc6b700 1 --2- 172.21.15.73:0/1979010339 >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 0x7f11f80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 -- 172.21.15.73:0/1979010339 >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 msgr2=0x7f11f80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 --2- 172.21.15.73:0/1979010339 >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 0x7f11f80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 -- 172.21.15.73:0/1979010339 >> v1:172.21.15.150:6789/0 conn(0x7f11f80caab0 legacy=0x7f11f80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11effff700 1 -- 172.21.15.73:0/1979010339 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3763451882 0 0) 0x7f11f80a9a90 con 0x7f11f80caab0 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 -- 172.21.15.73:0/1979010339 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f80aab80 con 0x7f11f80ca180 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fcc6b700 1 --2- 172.21.15.73:0/1979010339 >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 0x7f11f80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11fdc6d700 1 --2- 172.21.15.73:0/1979010339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f11f0006fd0 tx=0x7f11f00090f0).ready entity=mon.2 client_cookie=45b3808180a6b807 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11effff700 1 -- 172.21.15.73:0/1979010339 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000b580 con 0x7f11f80ca180 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11effff700 1 -- 172.21.15.73:0/1979010339 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000b6e0 con 0x7f11f80ca180 2023-11-28T02:52:46.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.589+0000 7f11effff700 1 -- 172.21.15.73:0/1979010339 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f000ba60 con 0x7f11f80ca180 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 -- 172.21.15.73:0/1979010339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 msgr2=0x7f11f80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/1979010339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f11f0006fd0 tx=0x7f11f00090f0).stop 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 -- 172.21.15.73:0/1979010339 shutdown_connections 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/1979010339 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f80c0fa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/1979010339 >> v2:172.21.15.150:3300/0 conn(0x7f11f80cae90 0x7f11f80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 -- 172.21.15.73:0/1979010339 >> 172.21.15.73:0/1979010339 conn(0x7f11f8094920 msgr2=0x7f11f806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 -- 172.21.15.73:0/1979010339 shutdown_connections 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.590+0000 7f11ff6d0700 1 -- 172.21.15.73:0/1979010339 wait complete. 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.591+0000 7f11ff6d0700 1 Processor -- start 2023-11-28T02:52:46.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.591+0000 7f11ff6d0700 1 -- start start 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f8147fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f80caab0 0x7f11f80c4180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80ae220 con 0x7f11f80cae90 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80c4b60 con 0x7f11f80caab0 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11ff6d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f80af040 con 0x7f11f80ca180 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42448/0 (socket says 172.21.15.73:42448) 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 -- 172.21.15.73:0/2778339190 learned_addr learned my addr 172.21.15.73:0/2778339190 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fd46c700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f8147fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fcc6b700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f80caab0 0x7f11f80c4180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 -- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 msgr2=0x7f11f8147fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f8147fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 -- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f80caab0 msgr2=0x7f11f80c4180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f80caab0 0x7f11f80c4180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 -- 172.21.15.73:0/2778339190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e80092a0 con 0x7f11f80cae90 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.592+0000 7f11fdc6d700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f11f8154e20 tx=0x7f11f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.593+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f0006d90 con 0x7f11f80cae90 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.593+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11f000b850 con 0x7f11f80cae90 2023-11-28T02:52:46.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.593+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f0009370 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.593+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11f001c650 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.593+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f80af4d0 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.594+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f81580f0 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.594+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f11f001cc10 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.594+0000 7f11ed7fa700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 0x7f11dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.594+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f11f007c0c0 con 0x7f11f80cae90 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.594+0000 7f11fd46c700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 0x7f11dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.595+0000 7f11fd46c700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 0x7f11dc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f11f4007fd0 tx=0x7f11f4007f20).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.598+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11f004d390 con 0x7f11f80cae90 2023-11-28T02:52:46.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.801+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11f806c860 con 0x7f11f80cae90 2023-11-28T02:52:46.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.801+0000 7f11ed7fa700 1 -- 172.21.15.73:0/2778339190 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f11f002a070 con 0x7f11f80cae90 2023-11-28T02:52:46.803 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:46.803 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 msgr2=0x7f11dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 0x7f11dc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f11f4007fd0 tx=0x7f11f4007f20).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 msgr2=0x7f11f80c4670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7f11f8154e20 tx=0x7f11f0017000).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 shutdown_connections 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11dc05f150 0x7f11dc061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f80cae90 0x7f11f80c4670 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f80ca180 0x7f11f8147fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 --2- 172.21.15.73:0/2778339190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f80caab0 0x7f11f80c4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 >> 172.21.15.73:0/2778339190 conn(0x7f11f8094920 msgr2=0x7f11f80ad770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:46.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 shutdown_connections 2023-11-28T02:52:46.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.803+0000 7f11ff6d0700 1 -- 172.21.15.73:0/2778339190 wait complete. 2023-11-28T02:52:46.806 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:52:46.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:46.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.928+0000 7fa996cdf700 1 Processor -- start 2023-11-28T02:52:46.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.928+0000 7fa996cdf700 1 -- start start 2023-11-28T02:52:46.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa996cdf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa9900c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa996cdf700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa990154d20 con 0x7fa9900c2d50 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa996cdf700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9901555c0 con 0x7fa990068090 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa996cdf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa990151670 con 0x7fa990067cb0 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa99527c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa9900c2d50 0x7fa9900a6480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33684/0 (socket says 172.21.15.73:33684) 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.929+0000 7fa99527c700 1 -- 172.21.15.73:0/54806683 learned_addr learned my addr 172.21.15.73:0/54806683 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2871598594 0 0) 0x7fa990154d20 con 0x7fa9900c2d50 2023-11-28T02:52:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa9900c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa9780033f0 con 0x7fa9900c2d50 2023-11-28T02:52:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3484948230 0 0) 0x7fa9780033f0 con 0x7fa9900c2d50 2023-11-28T02:52:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 msgr2=0x7fa9900c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa9900c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 >> v1:172.21.15.150:6789/0 conn(0x7fa990068090 legacy=0x7fa9900c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900c3390 con 0x7fa9900c2d50 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fa9880017f0 con 0x7fa9900c2d50 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 >> v1:172.21.15.73:6789/0 conn(0x7fa9900c2d50 legacy=0x7fa9900a6480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa978003910 0x7fa978003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa978004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978004b80 0x7fa978005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97800b8e0 con 0x7fa978003910 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa97800ba50 con 0x7fa978004b80 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa97800bc10 con 0x7fa990067cb0 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa978003910 0x7fa978003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98ffff700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa978004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa99527c700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978004b80 0x7fa978005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 -- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 msgr2=0x7fa978004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa978004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 -- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978004b80 msgr2=0x7fa978005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978004b80 0x7fa978005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 -- 172.21.15.73:0/54806683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9780033f0 con 0x7fa978003910 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa99527c700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978004b80 0x7fa978005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98ffff700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa990067cb0 0x7fa978004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa994a7b700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa978003910 0x7fa978003d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa980009a80 tx=0x7fa980009060).ready entity=mon.0 client_cookie=21d63b52684b524 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa980009430 con 0x7fa978003910 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa98000e640 con 0x7fa978003910 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.930+0000 7fa98f7fe700 1 -- 172.21.15.73:0/54806683 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa98000e9c0 con 0x7fa978003910 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 -- 172.21.15.73:0/54806683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa978003910 msgr2=0x7fa978003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 --2- 172.21.15.73:0/54806683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa978003910 0x7fa978003d30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fa980009a80 tx=0x7fa980009060).stop 2023-11-28T02:52:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa99527c700 1 -- 172.21.15.73:0/54806683 reap_dead start 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 -- 172.21.15.73:0/54806683 shutdown_connections 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 -- 172.21.15.73:0/54806683 >> 172.21.15.73:0/54806683 conn(0x7fa9900c7fd0 msgr2=0x7fa9900c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 -- 172.21.15.73:0/54806683 shutdown_connections 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.931+0000 7fa996cdf700 1 -- 172.21.15.73:0/54806683 wait complete. 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.932+0000 7fa996cdf700 1 Processor -- start 2023-11-28T02:52:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.932+0000 7fa996cdf700 1 -- start start 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978003910 0x7fa990147db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 0x7fa990066c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900c05f0 con 0x7fa9900a88f0 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900c0f70 con 0x7fa978003910 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa996cdf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9900bf9e0 con 0x7fa9900bf070 2023-11-28T02:52:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42470/0 (socket says 172.21.15.73:42470) 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 -- 172.21.15.73:0/3984538681 learned_addr learned my addr 172.21.15.73:0/3984538681 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa994a7b700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978003910 0x7fa990147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa99527c700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 0x7fa990066c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 -- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 msgr2=0x7fa990066c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 0x7fa990066c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 -- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978003910 msgr2=0x7fa990147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978003910 0x7fa990147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 -- 172.21.15.73:0/3984538681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9900bfcc0 con 0x7fa9900a88f0 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa99527c700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 0x7fa990066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.933+0000 7fa98ffff700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa984000f60 tx=0x7fa984009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.934+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984010450 con 0x7fa9900a88f0 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.934+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9840105b0 con 0x7fa9900a88f0 2023-11-28T02:52:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.934+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9900bffa0 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.934+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa984010930 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.934+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9900bf4d0 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.935+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9900c0760 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.935+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa984010d20 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.935+0000 7fa98cff9700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 0x7fa97c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.935+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa98407a210 con 0x7fa9900a88f0 2023-11-28T02:52:46.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.935+0000 7fa994a7b700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 0x7fa97c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:46.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.936+0000 7fa994a7b700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 0x7fa97c061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa980006d50 tx=0x7fa98001a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:46.939+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa98404b4e0 con 0x7fa9900a88f0 2023-11-28T02:52:47.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.102+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa9900bf750 con 0x7fa9900a88f0 2023-11-28T02:52:47.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.103+0000 7fa98cff9700 1 -- 172.21.15.73:0/3984538681 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fa98404eb00 con 0x7fa9900a88f0 2023-11-28T02:52:47.104 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:47.104 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 msgr2=0x7fa97c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 0x7fa97c061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7fa980006d50 tx=0x7fa98001a000).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 msgr2=0x7fa9900c0a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa984000f60 tx=0x7fa984009040).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 shutdown_connections 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa97c05f150 0x7fa97c061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9900bf070 0x7fa990066c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa978003910 0x7fa990147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 --2- 172.21.15.73:0/3984538681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9900a88f0 0x7fa9900c0a80 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 >> 172.21.15.73:0/3984538681 conn(0x7fa9900c7fd0 msgr2=0x7fa990153cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 shutdown_connections 2023-11-28T02:52:47.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.104+0000 7fa996cdf700 1 -- 172.21.15.73:0/3984538681 wait complete. 2023-11-28T02:52:47.118 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T02:52:47.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:52:47.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.229+0000 7f4e1b462700 1 Processor -- start 2023-11-28T02:52:47.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.229+0000 7f4e1b462700 1 -- start start 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e1b462700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e1b462700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e141524a0 0x7f4e14066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e1b462700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140be110 con 0x7f4e14067a40 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e1b462700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e14066fc0 con 0x7f4e141524a0 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e1b462700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e140672a0 con 0x7f4e141520c0 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e199ff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4e14067a40 0x7f4e14155eb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33694/0 (socket says 172.21.15.73:33694) 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e199ff700 1 -- 172.21.15.73:0/3741864258 learned_addr learned my addr 172.21.15.73:0/3741864258 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 --2- 172.21.15.73:0/3741864258 >> v2:172.21.15.73:3300/0 conn(0x7f4e141524a0 0x7f4e14066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e191fe700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e0bfff700 1 -- 172.21.15.73:0/3741864258 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3465329593 0 0) 0x7f4e140be110 con 0x7f4e14067a40 2023-11-28T02:52:47.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e0bfff700 1 -- 172.21.15.73:0/3741864258 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e000033f0 con 0x7f4e14067a40 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 -- 172.21.15.73:0/3741864258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 msgr2=0x7f4e14066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 -- 172.21.15.73:0/3741864258 >> v1:172.21.15.73:6789/0 conn(0x7f4e14067a40 legacy=0x7f4e14155eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 -- 172.21.15.73:0/3741864258 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140be380 con 0x7f4e141524a0 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e191fe700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e189fd700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e14066a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4e04009760 tx=0x7f4e04009060).ready entity=mon.0 client_cookie=bbe8ac3d61681037 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.230+0000 7f4e0bfff700 1 -- 172.21.15.73:0/3741864258 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400d4a0 con 0x7f4e141524a0 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e0bfff700 1 -- 172.21.15.73:0/3741864258 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0400d600 con 0x7f4e141524a0 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e0bfff700 1 -- 172.21.15.73:0/3741864258 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0400d980 con 0x7f4e141524a0 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 -- 172.21.15.73:0/3741864258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 msgr2=0x7f4e14066a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e14066a10 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f4e04009760 tx=0x7f4e04009060).stop 2023-11-28T02:52:47.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 -- 172.21.15.73:0/3741864258 shutdown_connections 2023-11-28T02:52:47.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 --2- 172.21.15.73:0/3741864258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e14066a10 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 -- 172.21.15.73:0/3741864258 >> 172.21.15.73:0/3741864258 conn(0x7f4e140631d0 msgr2=0x7f4e140c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 -- 172.21.15.73:0/3741864258 shutdown_connections 2023-11-28T02:52:47.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.231+0000 7f4e1b462700 1 -- 172.21.15.73:0/3741864258 wait complete. 2023-11-28T02:52:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.232+0000 7f4e1b462700 1 Processor -- start 2023-11-28T02:52:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.232+0000 7f4e1b462700 1 -- start start 2023-11-28T02:52:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e14067a40 0x7f4e140c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14155590 con 0x7f4e141524a0 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14154980 con 0x7f4e14067a40 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e1b462700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14154c60 con 0x7f4e141520c0 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e199ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e199ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42476/0 (socket says 172.21.15.73:42476) 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e199ff700 1 -- 172.21.15.73:0/2167569015 learned_addr learned my addr 172.21.15.73:0/2167569015 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e189fd700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.233+0000 7f4e191fe700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e14067a40 0x7f4e140c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 -- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 msgr2=0x7f4e14153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 -- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e14067a40 msgr2=0x7f4e140c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e14067a40 0x7f4e140c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 -- 172.21.15.73:0/2167569015 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e10009250 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e199ff700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f4e0c005ad0 tx=0x7f4e0c001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c005d30 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c005e90 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c007320 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e040092a0 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.234+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14155220 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.235+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4e0c007710 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.235+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e14155740 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.235+0000 7f4e097fa700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 0x7f4df8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.235+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4e0c07a1e0 con 0x7f4e141524a0 2023-11-28T02:52:47.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.235+0000 7f4e191fe700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 0x7f4df8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.236+0000 7f4e191fe700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 0x7f4df8061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4e10006fd0 tx=0x7f4e10008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.239+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e14155740 con 0x7f4e141524a0 2023-11-28T02:52:47.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.397+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4e14155740 con 0x7f4df805f150 2023-11-28T02:52:47.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.407+0000 7f4e097fa700 1 -- 172.21.15.73:0/2167569015 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336451 (secure 0 0 0) 0x7f4e14155740 con 0x7f4df805f150 2023-11-28T02:52:47.412 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:47.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 msgr2=0x7f4df8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 0x7f4df8061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f4e10006fd0 tx=0x7f4e10008040).stop 2023-11-28T02:52:47.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 msgr2=0x7f4e1414d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f4e0c005ad0 tx=0x7f4e0c001650).stop 2023-11-28T02:52:47.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 shutdown_connections 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4df805f150 0x7f4df8061610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e141524a0 0x7f4e1414d560 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e14067a40 0x7f4e140c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 --2- 172.21.15.73:0/2167569015 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e141520c0 0x7f4e14153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 >> 172.21.15.73:0/2167569015 conn(0x7f4e140631d0 msgr2=0x7f4e140be7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 shutdown_connections 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.410+0000 7f4e1b462700 1 -- 172.21.15.73:0/2167569015 wait complete. 2023-11-28T02:52:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:52:47.424 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T02:52:47.389594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378624,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596096,"statfs":{"total":773094113280,"available":770658402304,"internally_reserved":0,"allocated":1900544,"data_stored":687340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470198+0000","last_change":"2023-11-28T02:52:44.467253+0000","last_active":"2023-11-28T02:52:44.470198+0000","last_peered":"2023-11-28T02:52:44.470198+0000","last_clean":"2023-11-28T02:52:44.470198+0000","last_became_active":"2023-11-28T02:52:44.466977+0000","last_became_peered":"2023-11-28T02:52:44.466977+0000","last_unstale":"2023-11-28T02:52:44.470198+0000","last_undegraded":"2023-11-28T02:52:44.470198+0000","last_fullsized":"2023-11-28T02:52:44.470198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455029+0000","last_change":"2023-11-28T02:52:44.098094+0000","last_active":"2023-11-28T02:52:44.455029+0000","last_peered":"2023-11-28T02:52:44.455029+0000","last_clean":"2023-11-28T02:52:44.455029+0000","last_became_active":"2023-11-28T02:52:44.097826+0000","last_became_peered":"2023-11-28T02:52:44.097826+0000","last_unstale":"2023-11-28T02:52:44.455029+0000","last_undegraded":"2023-11-28T02:52:44.455029+0000","last_fullsized":"2023-11-28T02:52:44.455029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479071+0000","last_change":"2023-11-28T02:52:44.478096+0000","last_active":"2023-11-28T02:52:44.479071+0000","last_peered":"2023-11-28T02:52:44.479071+0000","last_clean":"2023-11-28T02:52:44.479071+0000","last_became_active":"2023-11-28T02:52:44.477817+0000","last_became_peered":"2023-11-28T02:52:44.477817+0000","last_unstale":"2023-11-28T02:52:44.479071+0000","last_undegraded":"2023-11-28T02:52:44.479071+0000","last_fullsized":"2023-11-28T02:52:44.479071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462484+0000","last_change":"2023-11-28T02:52:44.087580+0000","last_active":"2023-11-28T02:52:44.462484+0000","last_peered":"2023-11-28T02:52:44.462484+0000","last_clean":"2023-11-28T02:52:44.462484+0000","last_became_active":"2023-11-28T02:52:44.087310+0000","last_became_peered":"2023-11-28T02:52:44.087310+0000","last_unstale":"2023-11-28T02:52:44.462484+0000","last_undegraded":"2023-11-28T02:52:44.462484+0000","last_fullsized":"2023-11-28T02:52:44.462484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472269+0000","last_change":"2023-11-28T02:52:44.466469+0000","last_active":"2023-11-28T02:52:44.472269+0000","last_peered":"2023-11-28T02:52:44.472269+0000","last_clean":"2023-11-28T02:52:44.472269+0000","last_became_active":"2023-11-28T02:52:44.465822+0000","last_became_peered":"2023-11-28T02:52:44.465822+0000","last_unstale":"2023-11-28T02:52:44.472269+0000","last_undegraded":"2023-11-28T02:52:44.472269+0000","last_fullsized":"2023-11-28T02:52:44.472269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451606+0000","last_change":"2023-11-28T02:52:44.097696+0000","last_active":"2023-11-28T02:52:44.451606+0000","last_peered":"2023-11-28T02:52:44.451606+0000","last_clean":"2023-11-28T02:52:44.451606+0000","last_became_active":"2023-11-28T02:52:44.097423+0000","last_became_peered":"2023-11-28T02:52:44.097423+0000","last_unstale":"2023-11-28T02:52:44.451606+0000","last_undegraded":"2023-11-28T02:52:44.451606+0000","last_fullsized":"2023-11-28T02:52:44.451606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469722+0000","last_change":"2023-11-28T02:52:44.465794+0000","last_active":"2023-11-28T02:52:44.469722+0000","last_peered":"2023-11-28T02:52:44.469722+0000","last_clean":"2023-11-28T02:52:44.469722+0000","last_became_active":"2023-11-28T02:52:44.465545+0000","last_became_peered":"2023-11-28T02:52:44.465545+0000","last_unstale":"2023-11-28T02:52:44.469722+0000","last_undegraded":"2023-11-28T02:52:44.469722+0000","last_fullsized":"2023-11-28T02:52:44.469722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450984+0000","last_change":"2023-11-28T02:52:44.099493+0000","last_active":"2023-11-28T02:52:44.450984+0000","last_peered":"2023-11-28T02:52:44.450984+0000","last_clean":"2023-11-28T02:52:44.450984+0000","last_became_active":"2023-11-28T02:52:44.099248+0000","last_became_peered":"2023-11-28T02:52:44.099248+0000","last_unstale":"2023-11-28T02:52:44.450984+0000","last_undegraded":"2023-11-28T02:52:44.450984+0000","last_fullsized":"2023-11-28T02:52:44.450984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479353+0000","last_change":"2023-11-28T02:52:44.477723+0000","last_active":"2023-11-28T02:52:44.479353+0000","last_peered":"2023-11-28T02:52:44.479353+0000","last_clean":"2023-11-28T02:52:44.479353+0000","last_became_active":"2023-11-28T02:52:44.477430+0000","last_became_peered":"2023-11-28T02:52:44.477430+0000","last_unstale":"2023-11-28T02:52:44.479353+0000","last_undegraded":"2023-11-28T02:52:44.479353+0000","last_fullsized":"2023-11-28T02:52:44.479353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462583+0000","last_change":"2023-11-28T02:52:44.102758+0000","last_active":"2023-11-28T02:52:44.462583+0000","last_peered":"2023-11-28T02:52:44.462583+0000","last_clean":"2023-11-28T02:52:44.462583+0000","last_became_active":"2023-11-28T02:52:44.102441+0000","last_became_peered":"2023-11-28T02:52:44.102441+0000","last_unstale":"2023-11-28T02:52:44.462583+0000","last_undegraded":"2023-11-28T02:52:44.462583+0000","last_fullsized":"2023-11-28T02:52:44.462583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468903+0000","last_change":"2023-11-28T02:52:44.466352+0000","last_active":"2023-11-28T02:52:44.468903+0000","last_peered":"2023-11-28T02:52:44.468903+0000","last_clean":"2023-11-28T02:52:44.468903+0000","last_became_active":"2023-11-28T02:52:44.466023+0000","last_became_peered":"2023-11-28T02:52:44.466023+0000","last_unstale":"2023-11-28T02:52:44.468903+0000","last_undegraded":"2023-11-28T02:52:44.468903+0000","last_fullsized":"2023-11-28T02:52:44.468903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479820+0000","last_change":"2023-11-28T02:52:44.100407+0000","last_active":"2023-11-28T02:52:44.479820+0000","last_peered":"2023-11-28T02:52:44.479820+0000","last_clean":"2023-11-28T02:52:44.479820+0000","last_became_active":"2023-11-28T02:52:44.100184+0000","last_became_peered":"2023-11-28T02:52:44.100184+0000","last_unstale":"2023-11-28T02:52:44.479820+0000","last_undegraded":"2023-11-28T02:52:44.479820+0000","last_fullsized":"2023-11-28T02:52:44.479820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469727+0000","last_change":"2023-11-28T02:52:44.467942+0000","last_active":"2023-11-28T02:52:44.469727+0000","last_peered":"2023-11-28T02:52:44.469727+0000","last_clean":"2023-11-28T02:52:44.469727+0000","last_became_active":"2023-11-28T02:52:44.467659+0000","last_became_peered":"2023-11-28T02:52:44.467659+0000","last_unstale":"2023-11-28T02:52:44.469727+0000","last_undegraded":"2023-11-28T02:52:44.469727+0000","last_fullsized":"2023-11-28T02:52:44.469727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472685+0000","last_change":"2023-11-28T02:52:44.094177+0000","last_active":"2023-11-28T02:52:44.472685+0000","last_peered":"2023-11-28T02:52:44.472685+0000","last_clean":"2023-11-28T02:52:44.472685+0000","last_became_active":"2023-11-28T02:52:44.093901+0000","last_became_peered":"2023-11-28T02:52:44.093901+0000","last_unstale":"2023-11-28T02:52:44.472685+0000","last_undegraded":"2023-11-28T02:52:44.472685+0000","last_fullsized":"2023-11-28T02:52:44.472685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470448+0000","last_change":"2023-11-28T02:52:44.467833+0000","last_active":"2023-11-28T02:52:44.470448+0000","last_peered":"2023-11-28T02:52:44.470448+0000","last_clean":"2023-11-28T02:52:44.470448+0000","last_became_active":"2023-11-28T02:52:44.467517+0000","last_became_peered":"2023-11-28T02:52:44.467517+0000","last_unstale":"2023-11-28T02:52:44.470448+0000","last_undegraded":"2023-11-28T02:52:44.470448+0000","last_fullsized":"2023-11-28T02:52:44.470448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485164+0000","last_change":"2023-11-28T02:52:44.104338+0000","last_active":"2023-11-28T02:52:44.485164+0000","last_peered":"2023-11-28T02:52:44.485164+0000","last_clean":"2023-11-28T02:52:44.485164+0000","last_became_active":"2023-11-28T02:52:44.104073+0000","last_became_peered":"2023-11-28T02:52:44.104073+0000","last_unstale":"2023-11-28T02:52:44.485164+0000","last_undegraded":"2023-11-28T02:52:44.485164+0000","last_fullsized":"2023-11-28T02:52:44.485164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477889+0000","last_change":"2023-11-28T02:52:44.476733+0000","last_active":"2023-11-28T02:52:44.477889+0000","last_peered":"2023-11-28T02:52:44.477889+0000","last_clean":"2023-11-28T02:52:44.477889+0000","last_became_active":"2023-11-28T02:52:44.476400+0000","last_became_peered":"2023-11-28T02:52:44.476400+0000","last_unstale":"2023-11-28T02:52:44.477889+0000","last_undegraded":"2023-11-28T02:52:44.477889+0000","last_fullsized":"2023-11-28T02:52:44.477889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453202+0000","last_change":"2023-11-28T02:52:44.096921+0000","last_active":"2023-11-28T02:52:44.453202+0000","last_peered":"2023-11-28T02:52:44.453202+0000","last_clean":"2023-11-28T02:52:44.453202+0000","last_became_active":"2023-11-28T02:52:44.096687+0000","last_became_peered":"2023-11-28T02:52:44.096687+0000","last_unstale":"2023-11-28T02:52:44.453202+0000","last_undegraded":"2023-11-28T02:52:44.453202+0000","last_fullsized":"2023-11-28T02:52:44.453202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466146+0000","last_change":"2023-11-28T02:52:44.461227+0000","last_active":"2023-11-28T02:52:44.466146+0000","last_peered":"2023-11-28T02:52:44.466146+0000","last_clean":"2023-11-28T02:52:44.466146+0000","last_became_active":"2023-11-28T02:52:44.460957+0000","last_became_peered":"2023-11-28T02:52:44.460957+0000","last_unstale":"2023-11-28T02:52:44.466146+0000","last_undegraded":"2023-11-28T02:52:44.466146+0000","last_fullsized":"2023-11-28T02:52:44.466146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475829+0000","last_change":"2023-11-28T02:52:44.106278+0000","last_active":"2023-11-28T02:52:44.475829+0000","last_peered":"2023-11-28T02:52:44.475829+0000","last_clean":"2023-11-28T02:52:44.475829+0000","last_became_active":"2023-11-28T02:52:44.106026+0000","last_became_peered":"2023-11-28T02:52:44.106026+0000","last_unstale":"2023-11-28T02:52:44.475829+0000","last_undegraded":"2023-11-28T02:52:44.475829+0000","last_fullsized":"2023-11-28T02:52:44.475829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480394+0000","last_change":"2023-11-28T02:52:44.479895+0000","last_active":"2023-11-28T02:52:44.480394+0000","last_peered":"2023-11-28T02:52:44.480394+0000","last_clean":"2023-11-28T02:52:44.480394+0000","last_became_active":"2023-11-28T02:52:44.479685+0000","last_became_peered":"2023-11-28T02:52:44.479685+0000","last_unstale":"2023-11-28T02:52:44.480394+0000","last_undegraded":"2023-11-28T02:52:44.480394+0000","last_fullsized":"2023-11-28T02:52:44.480394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470565+0000","last_change":"2023-11-28T02:52:44.097431+0000","last_active":"2023-11-28T02:52:44.470565+0000","last_peered":"2023-11-28T02:52:44.470565+0000","last_clean":"2023-11-28T02:52:44.470565+0000","last_became_active":"2023-11-28T02:52:44.097043+0000","last_became_peered":"2023-11-28T02:52:44.097043+0000","last_unstale":"2023-11-28T02:52:44.470565+0000","last_undegraded":"2023-11-28T02:52:44.470565+0000","last_fullsized":"2023-11-28T02:52:44.470565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478662+0000","last_change":"2023-11-28T02:52:44.477784+0000","last_active":"2023-11-28T02:52:44.478662+0000","last_peered":"2023-11-28T02:52:44.478662+0000","last_clean":"2023-11-28T02:52:44.478662+0000","last_became_active":"2023-11-28T02:52:44.477515+0000","last_became_peered":"2023-11-28T02:52:44.477515+0000","last_unstale":"2023-11-28T02:52:44.478662+0000","last_undegraded":"2023-11-28T02:52:44.478662+0000","last_fullsized":"2023-11-28T02:52:44.478662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485148+0000","last_change":"2023-11-28T02:52:44.104964+0000","last_active":"2023-11-28T02:52:44.485148+0000","last_peered":"2023-11-28T02:52:44.485148+0000","last_clean":"2023-11-28T02:52:44.485148+0000","last_became_active":"2023-11-28T02:52:44.104688+0000","last_became_peered":"2023-11-28T02:52:44.104688+0000","last_unstale":"2023-11-28T02:52:44.485148+0000","last_undegraded":"2023-11-28T02:52:44.485148+0000","last_fullsized":"2023-11-28T02:52:44.485148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469058+0000","last_change":"2023-11-28T02:52:44.465011+0000","last_active":"2023-11-28T02:52:44.469058+0000","last_peered":"2023-11-28T02:52:44.469058+0000","last_clean":"2023-11-28T02:52:44.469058+0000","last_became_active":"2023-11-28T02:52:44.464725+0000","last_became_peered":"2023-11-28T02:52:44.464725+0000","last_unstale":"2023-11-28T02:52:44.469058+0000","last_undegraded":"2023-11-28T02:52:44.469058+0000","last_fullsized":"2023-11-28T02:52:44.469058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.482297+0000","last_change":"2023-11-28T02:52:44.104530+0000","last_active":"2023-11-28T02:52:44.482297+0000","last_peered":"2023-11-28T02:52:44.482297+0000","last_clean":"2023-11-28T02:52:44.482297+0000","last_became_active":"2023-11-28T02:52:44.104035+0000","last_became_peered":"2023-11-28T02:52:44.104035+0000","last_unstale":"2023-11-28T02:52:44.482297+0000","last_undegraded":"2023-11-28T02:52:44.482297+0000","last_fullsized":"2023-11-28T02:52:44.482297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474584+0000","last_change":"2023-11-28T02:52:44.469216+0000","last_active":"2023-11-28T02:52:44.474584+0000","last_peered":"2023-11-28T02:52:44.474584+0000","last_clean":"2023-11-28T02:52:44.474584+0000","last_became_active":"2023-11-28T02:52:44.467199+0000","last_became_peered":"2023-11-28T02:52:44.467199+0000","last_unstale":"2023-11-28T02:52:44.474584+0000","last_undegraded":"2023-11-28T02:52:44.474584+0000","last_fullsized":"2023-11-28T02:52:44.474584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474622+0000","last_change":"2023-11-28T02:52:44.099749+0000","last_active":"2023-11-28T02:52:44.474622+0000","last_peered":"2023-11-28T02:52:44.474622+0000","last_clean":"2023-11-28T02:52:44.474622+0000","last_became_active":"2023-11-28T02:52:44.099525+0000","last_became_peered":"2023-11-28T02:52:44.099525+0000","last_unstale":"2023-11-28T02:52:44.474622+0000","last_undegraded":"2023-11-28T02:52:44.474622+0000","last_fullsized":"2023-11-28T02:52:44.474622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470191+0000","last_change":"2023-11-28T02:52:44.467373+0000","last_active":"2023-11-28T02:52:44.470191+0000","last_peered":"2023-11-28T02:52:44.470191+0000","last_clean":"2023-11-28T02:52:44.470191+0000","last_became_active":"2023-11-28T02:52:44.464529+0000","last_became_peered":"2023-11-28T02:52:44.464529+0000","last_unstale":"2023-11-28T02:52:44.470191+0000","last_undegraded":"2023-11-28T02:52:44.470191+0000","last_fullsized":"2023-11-28T02:52:44.470191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457475+0000","last_change":"2023-11-28T02:52:44.099774+0000","last_active":"2023-11-28T02:52:44.457475+0000","last_peered":"2023-11-28T02:52:44.457475+0000","last_clean":"2023-11-28T02:52:44.457475+0000","last_became_active":"2023-11-28T02:52:44.099484+0000","last_became_peered":"2023-11-28T02:52:44.099484+0000","last_unstale":"2023-11-28T02:52:44.457475+0000","last_undegraded":"2023-11-28T02:52:44.457475+0000","last_fullsized":"2023-11-28T02:52:44.457475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469631+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.469631+0000","last_peered":"2023-11-28T02:52:44.469631+0000","last_clean":"2023-11-28T02:52:44.469631+0000","last_became_active":"2023-11-28T02:52:44.460712+0000","last_became_peered":"2023-11-28T02:52:44.460712+0000","last_unstale":"2023-11-28T02:52:44.469631+0000","last_undegraded":"2023-11-28T02:52:44.469631+0000","last_fullsized":"2023-11-28T02:52:44.469631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465456+0000","last_change":"2023-11-28T02:52:44.098187+0000","last_active":"2023-11-28T02:52:44.465456+0000","last_peered":"2023-11-28T02:52:44.465456+0000","last_clean":"2023-11-28T02:52:44.465456+0000","last_became_active":"2023-11-28T02:52:44.097937+0000","last_became_peered":"2023-11-28T02:52:44.097937+0000","last_unstale":"2023-11-28T02:52:44.465456+0000","last_undegraded":"2023-11-28T02:52:44.465456+0000","last_fullsized":"2023-11-28T02:52:44.465456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464006+0000","last_change":"2023-11-28T02:52:44.458708+0000","last_active":"2023-11-28T02:52:44.464006+0000","last_peered":"2023-11-28T02:52:44.464006+0000","last_clean":"2023-11-28T02:52:44.464006+0000","last_became_active":"2023-11-28T02:52:44.458372+0000","last_became_peered":"2023-11-28T02:52:44.458372+0000","last_unstale":"2023-11-28T02:52:44.464006+0000","last_undegraded":"2023-11-28T02:52:44.464006+0000","last_fullsized":"2023-11-28T02:52:44.464006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463865+0000","last_change":"2023-11-28T02:52:44.098081+0000","last_active":"2023-11-28T02:52:44.463865+0000","last_peered":"2023-11-28T02:52:44.463865+0000","last_clean":"2023-11-28T02:52:44.463865+0000","last_became_active":"2023-11-28T02:52:44.097763+0000","last_became_peered":"2023-11-28T02:52:44.097763+0000","last_unstale":"2023-11-28T02:52:44.463865+0000","last_undegraded":"2023-11-28T02:52:44.463865+0000","last_fullsized":"2023-11-28T02:52:44.463865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468716+0000","last_change":"2023-11-28T02:52:44.464973+0000","last_active":"2023-11-28T02:52:44.468716+0000","last_peered":"2023-11-28T02:52:44.468716+0000","last_clean":"2023-11-28T02:52:44.468716+0000","last_became_active":"2023-11-28T02:52:44.464692+0000","last_became_peered":"2023-11-28T02:52:44.464692+0000","last_unstale":"2023-11-28T02:52:44.468716+0000","last_undegraded":"2023-11-28T02:52:44.468716+0000","last_fullsized":"2023-11-28T02:52:44.468716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454758+0000","last_change":"2023-11-28T02:52:44.096316+0000","last_active":"2023-11-28T02:52:44.454758+0000","last_peered":"2023-11-28T02:52:44.454758+0000","last_clean":"2023-11-28T02:52:44.454758+0000","last_became_active":"2023-11-28T02:52:44.090973+0000","last_became_peered":"2023-11-28T02:52:44.090973+0000","last_unstale":"2023-11-28T02:52:44.454758+0000","last_undegraded":"2023-11-28T02:52:44.454758+0000","last_fullsized":"2023-11-28T02:52:44.454758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467774+0000","last_change":"2023-11-28T02:52:44.463807+0000","last_active":"2023-11-28T02:52:44.467774+0000","last_peered":"2023-11-28T02:52:44.467774+0000","last_clean":"2023-11-28T02:52:44.467774+0000","last_became_active":"2023-11-28T02:52:44.463532+0000","last_became_peered":"2023-11-28T02:52:44.463532+0000","last_unstale":"2023-11-28T02:52:44.467774+0000","last_undegraded":"2023-11-28T02:52:44.467774+0000","last_fullsized":"2023-11-28T02:52:44.467774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451394+0000","last_change":"2023-11-28T02:52:44.099397+0000","last_active":"2023-11-28T02:52:44.451394+0000","last_peered":"2023-11-28T02:52:44.451394+0000","last_clean":"2023-11-28T02:52:44.451394+0000","last_became_active":"2023-11-28T02:52:44.099225+0000","last_became_peered":"2023-11-28T02:52:44.099225+0000","last_unstale":"2023-11-28T02:52:44.451394+0000","last_undegraded":"2023-11-28T02:52:44.451394+0000","last_fullsized":"2023-11-28T02:52:44.451394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464258+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:52:44.464258+0000","last_peered":"2023-11-28T02:52:44.464258+0000","last_clean":"2023-11-28T02:52:44.464258+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:52:44.464258+0000","last_undegraded":"2023-11-28T02:52:44.464258+0000","last_fullsized":"2023-11-28T02:52:44.464258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461086+0000","last_change":"2023-11-28T02:52:44.099940+0000","last_active":"2023-11-28T02:52:44.461086+0000","last_peered":"2023-11-28T02:52:44.461086+0000","last_clean":"2023-11-28T02:52:44.461086+0000","last_became_active":"2023-11-28T02:52:44.099735+0000","last_became_peered":"2023-11-28T02:52:44.099735+0000","last_unstale":"2023-11-28T02:52:44.461086+0000","last_undegraded":"2023-11-28T02:52:44.461086+0000","last_fullsized":"2023-11-28T02:52:44.461086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473095+0000","last_change":"2023-11-28T02:52:44.472735+0000","last_active":"2023-11-28T02:52:44.473095+0000","last_peered":"2023-11-28T02:52:44.473095+0000","last_clean":"2023-11-28T02:52:44.473095+0000","last_became_active":"2023-11-28T02:52:44.472418+0000","last_became_peered":"2023-11-28T02:52:44.472418+0000","last_unstale":"2023-11-28T02:52:44.473095+0000","last_undegraded":"2023-11-28T02:52:44.473095+0000","last_fullsized":"2023-11-28T02:52:44.473095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459288+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:52:44.459288+0000","last_peered":"2023-11-28T02:52:44.459288+0000","last_clean":"2023-11-28T02:52:44.459288+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:52:44.459288+0000","last_undegraded":"2023-11-28T02:52:44.459288+0000","last_fullsized":"2023-11-28T02:52:44.459288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455813+0000","last_change":"2023-11-28T02:52:44.100602+0000","last_active":"2023-11-28T02:52:44.455813+0000","last_peered":"2023-11-28T02:52:44.455813+0000","last_clean":"2023-11-28T02:52:44.455813+0000","last_became_active":"2023-11-28T02:52:44.100337+0000","last_became_peered":"2023-11-28T02:52:44.100337+0000","last_unstale":"2023-11-28T02:52:44.455813+0000","last_undegraded":"2023-11-28T02:52:44.455813+0000","last_fullsized":"2023-11-28T02:52:44.455813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468871+0000","last_change":"2023-11-28T02:52:44.465676+0000","last_active":"2023-11-28T02:52:44.468871+0000","last_peered":"2023-11-28T02:52:44.468871+0000","last_clean":"2023-11-28T02:52:44.468871+0000","last_became_active":"2023-11-28T02:52:44.465401+0000","last_became_peered":"2023-11-28T02:52:44.465401+0000","last_unstale":"2023-11-28T02:52:44.468871+0000","last_undegraded":"2023-11-28T02:52:44.468871+0000","last_fullsized":"2023-11-28T02:52:44.468871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471562+0000","last_change":"2023-11-28T02:52:44.470158+0000","last_active":"2023-11-28T02:52:44.471562+0000","last_peered":"2023-11-28T02:52:44.471562+0000","last_clean":"2023-11-28T02:52:44.471562+0000","last_became_active":"2023-11-28T02:52:44.469795+0000","last_became_peered":"2023-11-28T02:52:44.469795+0000","last_unstale":"2023-11-28T02:52:44.471562+0000","last_undegraded":"2023-11-28T02:52:44.471562+0000","last_fullsized":"2023-11-28T02:52:44.471562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448512+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:52:44.448512+0000","last_peered":"2023-11-28T02:52:44.448512+0000","last_clean":"2023-11-28T02:52:44.448512+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:52:44.448512+0000","last_undegraded":"2023-11-28T02:52:44.448512+0000","last_fullsized":"2023-11-28T02:52:44.448512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462396+0000","last_change":"2023-11-28T02:52:44.091912+0000","last_active":"2023-11-28T02:52:44.462396+0000","last_peered":"2023-11-28T02:52:44.462396+0000","last_clean":"2023-11-28T02:52:44.462396+0000","last_became_active":"2023-11-28T02:52:44.091630+0000","last_became_peered":"2023-11-28T02:52:44.091630+0000","last_unstale":"2023-11-28T02:52:44.462396+0000","last_undegraded":"2023-11-28T02:52:44.462396+0000","last_fullsized":"2023-11-28T02:52:44.462396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472298+0000","last_change":"2023-11-28T02:52:44.468106+0000","last_active":"2023-11-28T02:52:44.472298+0000","last_peered":"2023-11-28T02:52:44.472298+0000","last_clean":"2023-11-28T02:52:44.472298+0000","last_became_active":"2023-11-28T02:52:44.467833+0000","last_became_peered":"2023-11-28T02:52:44.467833+0000","last_unstale":"2023-11-28T02:52:44.472298+0000","last_undegraded":"2023-11-28T02:52:44.472298+0000","last_fullsized":"2023-11-28T02:52:44.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461405+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:52:44.461405+0000","last_peered":"2023-11-28T02:52:44.461405+0000","last_clean":"2023-11-28T02:52:44.461405+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:52:44.461405+0000","last_undegraded":"2023-11-28T02:52:44.461405+0000","last_fullsized":"2023-11-28T02:52:44.461405+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476368+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:52:44.476368+0000","last_peered":"2023-11-28T02:52:44.476368+0000","last_clean":"2023-11-28T02:52:44.476368+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:52:44.476368+0000","last_undegraded":"2023-11-28T02:52:44.476368+0000","last_fullsized":"2023-11-28T02:52:44.476368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453788+0000","last_change":"2023-11-28T02:52:44.095277+0000","last_active":"2023-11-28T02:52:44.453788+0000","last_peered":"2023-11-28T02:52:44.453788+0000","last_clean":"2023-11-28T02:52:44.453788+0000","last_became_active":"2023-11-28T02:52:44.095021+0000","last_became_peered":"2023-11-28T02:52:44.095021+0000","last_unstale":"2023-11-28T02:52:44.453788+0000","last_undegraded":"2023-11-28T02:52:44.453788+0000","last_fullsized":"2023-11-28T02:52:44.453788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469245+0000","last_change":"2023-11-28T02:52:44.464858+0000","last_active":"2023-11-28T02:52:44.469245+0000","last_peered":"2023-11-28T02:52:44.469245+0000","last_clean":"2023-11-28T02:52:44.469245+0000","last_became_active":"2023-11-28T02:52:44.464352+0000","last_became_peered":"2023-11-28T02:52:44.464352+0000","last_unstale":"2023-11-28T02:52:44.469245+0000","last_undegraded":"2023-11-28T02:52:44.469245+0000","last_fullsized":"2023-11-28T02:52:44.469245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476767+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:52:44.476767+0000","last_peered":"2023-11-28T02:52:44.476767+0000","last_clean":"2023-11-28T02:52:44.476767+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:52:44.476767+0000","last_undegraded":"2023-11-28T02:52:44.476767+0000","last_fullsized":"2023-11-28T02:52:44.476767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450488+0000","last_change":"2023-11-28T02:52:44.105713+0000","last_active":"2023-11-28T02:52:44.450488+0000","last_peered":"2023-11-28T02:52:44.450488+0000","last_clean":"2023-11-28T02:52:44.450488+0000","last_became_active":"2023-11-28T02:52:44.105352+0000","last_became_peered":"2023-11-28T02:52:44.105352+0000","last_unstale":"2023-11-28T02:52:44.450488+0000","last_undegraded":"2023-11-28T02:52:44.450488+0000","last_fullsized":"2023-11-28T02:52:44.450488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474530+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:52:44.474530+0000","last_peered":"2023-11-28T02:52:44.474530+0000","last_clean":"2023-11-28T02:52:44.474530+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:52:44.474530+0000","last_undegraded":"2023-11-28T02:52:44.474530+0000","last_fullsized":"2023-11-28T02:52:44.474530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464373+0000","last_change":"2023-11-28T02:52:44.096162+0000","last_active":"2023-11-28T02:52:44.464373+0000","last_peered":"2023-11-28T02:52:44.464373+0000","last_clean":"2023-11-28T02:52:44.464373+0000","last_became_active":"2023-11-28T02:52:44.095900+0000","last_became_peered":"2023-11-28T02:52:44.095900+0000","last_unstale":"2023-11-28T02:52:44.464373+0000","last_undegraded":"2023-11-28T02:52:44.464373+0000","last_fullsized":"2023-11-28T02:52:44.464373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470164+0000","last_change":"2023-11-28T02:52:44.464449+0000","last_active":"2023-11-28T02:52:44.470164+0000","last_peered":"2023-11-28T02:52:44.470164+0000","last_clean":"2023-11-28T02:52:44.470164+0000","last_became_active":"2023-11-28T02:52:44.464196+0000","last_became_peered":"2023-11-28T02:52:44.464196+0000","last_unstale":"2023-11-28T02:52:44.470164+0000","last_undegraded":"2023-11-28T02:52:44.470164+0000","last_fullsized":"2023-11-28T02:52:44.470164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478810+0000","last_change":"2023-11-28T02:52:44.094471+0000","last_active":"2023-11-28T02:52:44.478810+0000","last_peered":"2023-11-28T02:52:44.478810+0000","last_clean":"2023-11-28T02:52:44.478810+0000","last_became_active":"2023-11-28T02:52:44.094193+0000","last_became_peered":"2023-11-28T02:52:44.094193+0000","last_unstale":"2023-11-28T02:52:44.478810+0000","last_undegraded":"2023-11-28T02:52:44.478810+0000","last_fullsized":"2023-11-28T02:52:44.478810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464206+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:52:44.464206+0000","last_peered":"2023-11-28T02:52:44.464206+0000","last_clean":"2023-11-28T02:52:44.464206+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:52:44.464206+0000","last_undegraded":"2023-11-28T02:52:44.464206+0000","last_fullsized":"2023-11-28T02:52:44.464206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479397+0000","last_change":"2023-11-28T02:52:44.478383+0000","last_active":"2023-11-28T02:52:44.479397+0000","last_peered":"2023-11-28T02:52:44.479397+0000","last_clean":"2023-11-28T02:52:44.479397+0000","last_became_active":"2023-11-28T02:52:44.478122+0000","last_became_peered":"2023-11-28T02:52:44.478122+0000","last_unstale":"2023-11-28T02:52:44.479397+0000","last_undegraded":"2023-11-28T02:52:44.479397+0000","last_fullsized":"2023-11-28T02:52:44.479397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463683+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:52:44.463683+0000","last_peered":"2023-11-28T02:52:44.463683+0000","last_clean":"2023-11-28T02:52:44.463683+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:52:44.463683+0000","last_undegraded":"2023-11-28T02:52:44.463683+0000","last_fullsized":"2023-11-28T02:52:44.463683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451382+0000","last_change":"2023-11-28T02:52:44.099058+0000","last_active":"2023-11-28T02:52:44.451382+0000","last_peered":"2023-11-28T02:52:44.451382+0000","last_clean":"2023-11-28T02:52:44.451382+0000","last_became_active":"2023-11-28T02:52:44.098489+0000","last_became_peered":"2023-11-28T02:52:44.098489+0000","last_unstale":"2023-11-28T02:52:44.451382+0000","last_undegraded":"2023-11-28T02:52:44.451382+0000","last_fullsized":"2023-11-28T02:52:44.451382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475427+0000","last_change":"2023-11-28T02:52:44.473356+0000","last_active":"2023-11-28T02:52:44.475427+0000","last_peered":"2023-11-28T02:52:44.475427+0000","last_clean":"2023-11-28T02:52:44.475427+0000","last_became_active":"2023-11-28T02:52:44.473032+0000","last_became_peered":"2023-11-28T02:52:44.473032+0000","last_unstale":"2023-11-28T02:52:44.475427+0000","last_undegraded":"2023-11-28T02:52:44.475427+0000","last_fullsized":"2023-11-28T02:52:44.475427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471249+0000","last_change":"2023-11-28T02:52:44.469050+0000","last_active":"2023-11-28T02:52:44.471249+0000","last_peered":"2023-11-28T02:52:44.471249+0000","last_clean":"2023-11-28T02:52:44.471249+0000","last_became_active":"2023-11-28T02:52:44.468695+0000","last_became_peered":"2023-11-28T02:52:44.468695+0000","last_unstale":"2023-11-28T02:52:44.471249+0000","last_undegraded":"2023-11-28T02:52:44.471249+0000","last_fullsized":"2023-11-28T02:52:44.471249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452775+0000","last_change":"2023-11-28T02:52:44.105717+0000","last_active":"2023-11-28T02:52:44.452775+0000","last_peered":"2023-11-28T02:52:44.452775+0000","last_clean":"2023-11-28T02:52:44.452775+0000","last_became_active":"2023-11-28T02:52:44.105369+0000","last_became_peered":"2023-11-28T02:52:44.105369+0000","last_unstale":"2023-11-28T02:52:44.452775+0000","last_undegraded":"2023-11-28T02:52:44.452775+0000","last_fullsized":"2023-11-28T02:52:44.452775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465888+0000","last_change":"2023-11-28T02:52:44.463547+0000","last_active":"2023-11-28T02:52:44.465888+0000","last_peered":"2023-11-28T02:52:44.465888+0000","last_clean":"2023-11-28T02:52:44.465888+0000","last_became_active":"2023-11-28T02:52:44.463254+0000","last_became_peered":"2023-11-28T02:52:44.463254+0000","last_unstale":"2023-11-28T02:52:44.465888+0000","last_undegraded":"2023-11-28T02:52:44.465888+0000","last_fullsized":"2023-11-28T02:52:44.465888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457589+0000","last_change":"2023-11-28T02:52:44.102948+0000","last_active":"2023-11-28T02:52:44.457589+0000","last_peered":"2023-11-28T02:52:44.457589+0000","last_clean":"2023-11-28T02:52:44.457589+0000","last_became_active":"2023-11-28T02:52:44.102609+0000","last_became_peered":"2023-11-28T02:52:44.102609+0000","last_unstale":"2023-11-28T02:52:44.457589+0000","last_undegraded":"2023-11-28T02:52:44.457589+0000","last_fullsized":"2023-11-28T02:52:44.457589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465782+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.465782+0000","last_peered":"2023-11-28T02:52:44.465782+0000","last_clean":"2023-11-28T02:52:44.465782+0000","last_became_active":"2023-11-28T02:52:44.460662+0000","last_became_peered":"2023-11-28T02:52:44.460662+0000","last_unstale":"2023-11-28T02:52:44.465782+0000","last_undegraded":"2023-11-28T02:52:44.465782+0000","last_fullsized":"2023-11-28T02:52:44.465782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469497+0000","last_change":"2023-11-28T02:52:44.104319+0000","last_active":"2023-11-28T02:52:44.469497+0000","last_peered":"2023-11-28T02:52:44.469497+0000","last_clean":"2023-11-28T02:52:44.469497+0000","last_became_active":"2023-11-28T02:52:44.104055+0000","last_became_peered":"2023-11-28T02:52:44.104055+0000","last_unstale":"2023-11-28T02:52:44.469497+0000","last_undegraded":"2023-11-28T02:52:44.469497+0000","last_fullsized":"2023-11-28T02:52:44.469497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466536+0000","last_change":"2023-11-28T02:52:44.460800+0000","last_active":"2023-11-28T02:52:44.466536+0000","last_peered":"2023-11-28T02:52:44.466536+0000","last_clean":"2023-11-28T02:52:44.466536+0000","last_became_active":"2023-11-28T02:52:44.460518+0000","last_became_peered":"2023-11-28T02:52:44.460518+0000","last_unstale":"2023-11-28T02:52:44.466536+0000","last_undegraded":"2023-11-28T02:52:44.466536+0000","last_fullsized":"2023-11-28T02:52:44.466536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466150+0000","last_change":"2023-11-28T02:52:44.093970+0000","last_active":"2023-11-28T02:52:44.466150+0000","last_peered":"2023-11-28T02:52:44.466150+0000","last_clean":"2023-11-28T02:52:44.466150+0000","last_became_active":"2023-11-28T02:52:44.093701+0000","last_became_peered":"2023-11-28T02:52:44.093701+0000","last_unstale":"2023-11-28T02:52:44.466150+0000","last_undegraded":"2023-11-28T02:52:44.466150+0000","last_fullsized":"2023-11-28T02:52:44.466150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475064+0000","last_change":"2023-11-28T02:52:44.469053+0000","last_active":"2023-11-28T02:52:44.475064+0000","last_peered":"2023-11-28T02:52:44.475064+0000","last_clean":"2023-11-28T02:52:44.475064+0000","last_became_active":"2023-11-28T02:52:44.468580+0000","last_became_peered":"2023-11-28T02:52:44.468580+0000","last_unstale":"2023-11-28T02:52:44.475064+0000","last_undegraded":"2023-11-28T02:52:44.475064+0000","last_fullsized":"2023-11-28T02:52:44.475064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464009+0000","last_change":"2023-11-28T02:52:44.099449+0000","last_active":"2023-11-28T02:52:44.464009+0000","last_peered":"2023-11-28T02:52:44.464009+0000","last_clean":"2023-11-28T02:52:44.464009+0000","last_became_active":"2023-11-28T02:52:44.099097+0000","last_became_peered":"2023-11-28T02:52:44.099097+0000","last_unstale":"2023-11-28T02:52:44.464009+0000","last_undegraded":"2023-11-28T02:52:44.464009+0000","last_fullsized":"2023-11-28T02:52:44.464009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470464+0000","last_change":"2023-11-28T02:52:44.467418+0000","last_active":"2023-11-28T02:52:44.470464+0000","last_peered":"2023-11-28T02:52:44.470464+0000","last_clean":"2023-11-28T02:52:44.470464+0000","last_became_active":"2023-11-28T02:52:44.465656+0000","last_became_peered":"2023-11-28T02:52:44.465656+0000","last_unstale":"2023-11-28T02:52:44.470464+0000","last_undegraded":"2023-11-28T02:52:44.470464+0000","last_fullsized":"2023-11-28T02:52:44.470464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462541+0000","last_change":"2023-11-28T02:52:44.095567+0000","last_active":"2023-11-28T02:52:44.462541+0000","last_peered":"2023-11-28T02:52:44.462541+0000","last_clean":"2023-11-28T02:52:44.462541+0000","last_became_active":"2023-11-28T02:52:44.095290+0000","last_became_peered":"2023-11-28T02:52:44.095290+0000","last_unstale":"2023-11-28T02:52:44.462541+0000","last_undegraded":"2023-11-28T02:52:44.462541+0000","last_fullsized":"2023-11-28T02:52:44.462541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467572+0000","last_change":"2023-11-28T02:52:44.464051+0000","last_active":"2023-11-28T02:52:44.467572+0000","last_peered":"2023-11-28T02:52:44.467572+0000","last_clean":"2023-11-28T02:52:44.467572+0000","last_became_active":"2023-11-28T02:52:44.463832+0000","last_became_peered":"2023-11-28T02:52:44.463832+0000","last_unstale":"2023-11-28T02:52:44.467572+0000","last_undegraded":"2023-11-28T02:52:44.467572+0000","last_fullsized":"2023-11-28T02:52:44.467572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448090+0000","last_change":"2023-11-28T02:52:44.092817+0000","last_active":"2023-11-28T02:52:44.448090+0000","last_peered":"2023-11-28T02:52:44.448090+0000","last_clean":"2023-11-28T02:52:44.448090+0000","last_became_active":"2023-11-28T02:52:44.092559+0000","last_became_peered":"2023-11-28T02:52:44.092559+0000","last_unstale":"2023-11-28T02:52:44.448090+0000","last_undegraded":"2023-11-28T02:52:44.448090+0000","last_fullsized":"2023-11-28T02:52:44.448090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477920+0000","last_change":"2023-11-28T02:52:44.476928+0000","last_active":"2023-11-28T02:52:44.477920+0000","last_peered":"2023-11-28T02:52:44.477920+0000","last_clean":"2023-11-28T02:52:44.477920+0000","last_became_active":"2023-11-28T02:52:44.476404+0000","last_became_peered":"2023-11-28T02:52:44.476404+0000","last_unstale":"2023-11-28T02:52:44.477920+0000","last_undegraded":"2023-11-28T02:52:44.477920+0000","last_fullsized":"2023-11-28T02:52:44.477920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461903+0000","last_change":"2023-11-28T02:52:44.100532+0000","last_active":"2023-11-28T02:52:44.461903+0000","last_peered":"2023-11-28T02:52:44.461903+0000","last_clean":"2023-11-28T02:52:44.461903+0000","last_became_active":"2023-11-28T02:52:44.100241+0000","last_became_peered":"2023-11-28T02:52:44.100241+0000","last_unstale":"2023-11-28T02:52:44.461903+0000","last_undegraded":"2023-11-28T02:52:44.461903+0000","last_fullsized":"2023-11-28T02:52:44.461903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477206+0000","last_change":"2023-11-28T02:52:44.476740+0000","last_active":"2023-11-28T02:52:44.477206+0000","last_peered":"2023-11-28T02:52:44.477206+0000","last_clean":"2023-11-28T02:52:44.477206+0000","last_became_active":"2023-11-28T02:52:44.476473+0000","last_became_peered":"2023-11-28T02:52:44.476473+0000","last_unstale":"2023-11-28T02:52:44.477206+0000","last_undegraded":"2023-11-28T02:52:44.477206+0000","last_fullsized":"2023-11-28T02:52:44.477206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452872+0000","last_change":"2023-11-28T02:52:44.101558+0000","last_active":"2023-11-28T02:52:44.452872+0000","last_peered":"2023-11-28T02:52:44.452872+0000","last_clean":"2023-11-28T02:52:44.452872+0000","last_became_active":"2023-11-28T02:52:44.101242+0000","last_became_peered":"2023-11-28T02:52:44.101242+0000","last_unstale":"2023-11-28T02:52:44.452872+0000","last_undegraded":"2023-11-28T02:52:44.452872+0000","last_fullsized":"2023-11-28T02:52:44.452872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476230+0000","last_change":"2023-11-28T02:52:44.475794+0000","last_active":"2023-11-28T02:52:44.476230+0000","last_peered":"2023-11-28T02:52:44.476230+0000","last_clean":"2023-11-28T02:52:44.476230+0000","last_became_active":"2023-11-28T02:52:44.475520+0000","last_became_peered":"2023-11-28T02:52:44.475520+0000","last_unstale":"2023-11-28T02:52:44.476230+0000","last_undegraded":"2023-11-28T02:52:44.476230+0000","last_fullsized":"2023-11-28T02:52:44.476230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454590+0000","last_change":"2023-11-28T02:52:44.102703+0000","last_active":"2023-11-28T02:52:44.454590+0000","last_peered":"2023-11-28T02:52:44.454590+0000","last_clean":"2023-11-28T02:52:44.454590+0000","last_became_active":"2023-11-28T02:52:44.102344+0000","last_became_peered":"2023-11-28T02:52:44.102344+0000","last_unstale":"2023-11-28T02:52:44.454590+0000","last_undegraded":"2023-11-28T02:52:44.454590+0000","last_fullsized":"2023-11-28T02:52:44.454590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469219+0000","last_change":"2023-11-28T02:52:44.467020+0000","last_active":"2023-11-28T02:52:44.469219+0000","last_peered":"2023-11-28T02:52:44.469219+0000","last_clean":"2023-11-28T02:52:44.469219+0000","last_became_active":"2023-11-28T02:52:44.466737+0000","last_became_peered":"2023-11-28T02:52:44.466737+0000","last_unstale":"2023-11-28T02:52:44.469219+0000","last_undegraded":"2023-11-28T02:52:44.469219+0000","last_fullsized":"2023-11-28T02:52:44.469219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462456+0000","last_change":"2023-11-28T02:52:44.095605+0000","last_active":"2023-11-28T02:52:44.462456+0000","last_peered":"2023-11-28T02:52:44.462456+0000","last_clean":"2023-11-28T02:52:44.462456+0000","last_became_active":"2023-11-28T02:52:44.093886+0000","last_became_peered":"2023-11-28T02:52:44.093886+0000","last_unstale":"2023-11-28T02:52:44.462456+0000","last_undegraded":"2023-11-28T02:52:44.462456+0000","last_fullsized":"2023-11-28T02:52:44.462456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469773+0000","last_change":"2023-11-28T02:52:44.466446+0000","last_active":"2023-11-28T02:52:44.469773+0000","last_peered":"2023-11-28T02:52:44.469773+0000","last_clean":"2023-11-28T02:52:44.469773+0000","last_became_active":"2023-11-28T02:52:44.466194+0000","last_became_peered":"2023-11-28T02:52:44.466194+0000","last_unstale":"2023-11-28T02:52:44.469773+0000","last_undegraded":"2023-11-28T02:52:44.469773+0000","last_fullsized":"2023-11-28T02:52:44.469773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462065+0000","last_change":"2023-11-28T02:52:44.089689+0000","last_active":"2023-11-28T02:52:44.462065+0000","last_peered":"2023-11-28T02:52:44.462065+0000","last_clean":"2023-11-28T02:52:44.462065+0000","last_became_active":"2023-11-28T02:52:44.089421+0000","last_became_peered":"2023-11-28T02:52:44.089421+0000","last_unstale":"2023-11-28T02:52:44.462065+0000","last_undegraded":"2023-11-28T02:52:44.462065+0000","last_fullsized":"2023-11-28T02:52:44.462065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472331+0000","last_change":"2023-11-28T02:52:44.468436+0000","last_active":"2023-11-28T02:52:44.472331+0000","last_peered":"2023-11-28T02:52:44.472331+0000","last_clean":"2023-11-28T02:52:44.472331+0000","last_became_active":"2023-11-28T02:52:44.468102+0000","last_became_peered":"2023-11-28T02:52:44.468102+0000","last_unstale":"2023-11-28T02:52:44.472331+0000","last_undegraded":"2023-11-28T02:52:44.472331+0000","last_fullsized":"2023-11-28T02:52:44.472331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457616+0000","last_change":"2023-11-28T02:52:44.104653+0000","last_active":"2023-11-28T02:52:44.457616+0000","last_peered":"2023-11-28T02:52:44.457616+0000","last_clean":"2023-11-28T02:52:44.457616+0000","last_became_active":"2023-11-28T02:52:44.104404+0000","last_became_peered":"2023-11-28T02:52:44.104404+0000","last_unstale":"2023-11-28T02:52:44.457616+0000","last_undegraded":"2023-11-28T02:52:44.457616+0000","last_fullsized":"2023-11-28T02:52:44.457616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464296+0000","last_change":"2023-11-28T02:52:44.460305+0000","last_active":"2023-11-28T02:52:44.464296+0000","last_peered":"2023-11-28T02:52:44.464296+0000","last_clean":"2023-11-28T02:52:44.464296+0000","last_became_active":"2023-11-28T02:52:44.460040+0000","last_became_peered":"2023-11-28T02:52:44.460040+0000","last_unstale":"2023-11-28T02:52:44.464296+0000","last_undegraded":"2023-11-28T02:52:44.464296+0000","last_fullsized":"2023-11-28T02:52:44.464296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461498+0000","last_change":"2023-11-28T02:52:44.103715+0000","last_active":"2023-11-28T02:52:44.461498+0000","last_peered":"2023-11-28T02:52:44.461498+0000","last_clean":"2023-11-28T02:52:44.461498+0000","last_became_active":"2023-11-28T02:52:44.103451+0000","last_became_peered":"2023-11-28T02:52:44.103451+0000","last_unstale":"2023-11-28T02:52:44.461498+0000","last_undegraded":"2023-11-28T02:52:44.461498+0000","last_fullsized":"2023-11-28T02:52:44.461498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464165+0000","last_change":"2023-11-28T02:52:44.460282+0000","last_active":"2023-11-28T02:52:44.464165+0000","last_peered":"2023-11-28T02:52:44.464165+0000","last_clean":"2023-11-28T02:52:44.464165+0000","last_became_active":"2023-11-28T02:52:44.460011+0000","last_became_peered":"2023-11-28T02:52:44.460011+0000","last_unstale":"2023-11-28T02:52:44.464165+0000","last_undegraded":"2023-11-28T02:52:44.464165+0000","last_fullsized":"2023-11-28T02:52:44.464165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454784+0000","last_change":"2023-11-28T02:52:44.104288+0000","last_active":"2023-11-28T02:52:44.454784+0000","last_peered":"2023-11-28T02:52:44.454784+0000","last_clean":"2023-11-28T02:52:44.454784+0000","last_became_active":"2023-11-28T02:52:44.102746+0000","last_became_peered":"2023-11-28T02:52:44.102746+0000","last_unstale":"2023-11-28T02:52:44.454784+0000","last_undegraded":"2023-11-28T02:52:44.454784+0000","last_fullsized":"2023-11-28T02:52:44.454784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471224+0000","last_change":"2023-11-28T02:52:44.468068+0000","last_active":"2023-11-28T02:52:44.471224+0000","last_peered":"2023-11-28T02:52:44.471224+0000","last_clean":"2023-11-28T02:52:44.471224+0000","last_became_active":"2023-11-28T02:52:44.467788+0000","last_became_peered":"2023-11-28T02:52:44.467788+0000","last_unstale":"2023-11-28T02:52:44.471224+0000","last_undegraded":"2023-11-28T02:52:44.471224+0000","last_fullsized":"2023-11-28T02:52:44.471224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461485+0000","last_change":"2023-11-28T02:52:44.102690+0000","last_active":"2023-11-28T02:52:44.461485+0000","last_peered":"2023-11-28T02:52:44.461485+0000","last_clean":"2023-11-28T02:52:44.461485+0000","last_became_active":"2023-11-28T02:52:44.102408+0000","last_became_peered":"2023-11-28T02:52:44.102408+0000","last_unstale":"2023-11-28T02:52:44.461485+0000","last_undegraded":"2023-11-28T02:52:44.461485+0000","last_fullsized":"2023-11-28T02:52:44.461485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469649+0000","last_change":"2023-11-28T02:52:44.467827+0000","last_active":"2023-11-28T02:52:44.469649+0000","last_peered":"2023-11-28T02:52:44.469649+0000","last_clean":"2023-11-28T02:52:44.469649+0000","last_became_active":"2023-11-28T02:52:44.467547+0000","last_became_peered":"2023-11-28T02:52:44.467547+0000","last_unstale":"2023-11-28T02:52:44.469649+0000","last_undegraded":"2023-11-28T02:52:44.469649+0000","last_fullsized":"2023-11-28T02:52:44.469649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.460918+0000","last_change":"2023-11-28T02:52:44.097889+0000","last_active":"2023-11-28T02:52:44.460918+0000","last_peered":"2023-11-28T02:52:44.460918+0000","last_clean":"2023-11-28T02:52:44.460918+0000","last_became_active":"2023-11-28T02:52:44.097641+0000","last_became_peered":"2023-11-28T02:52:44.097641+0000","last_unstale":"2023-11-28T02:52:44.460918+0000","last_undegraded":"2023-11-28T02:52:44.460918+0000","last_fullsized":"2023-11-28T02:52:44.460918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479544+0000","last_change":"2023-11-28T02:52:44.477649+0000","last_active":"2023-11-28T02:52:44.479544+0000","last_peered":"2023-11-28T02:52:44.479544+0000","last_clean":"2023-11-28T02:52:44.479544+0000","last_became_active":"2023-11-28T02:52:44.477212+0000","last_became_peered":"2023-11-28T02:52:44.477212+0000","last_unstale":"2023-11-28T02:52:44.479544+0000","last_undegraded":"2023-11-28T02:52:44.479544+0000","last_fullsized":"2023-11-28T02:52:44.479544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461323+0000","last_change":"2023-11-28T02:52:44.098078+0000","last_active":"2023-11-28T02:52:44.461323+0000","last_peered":"2023-11-28T02:52:44.461323+0000","last_clean":"2023-11-28T02:52:44.461323+0000","last_became_active":"2023-11-28T02:52:44.097777+0000","last_became_peered":"2023-11-28T02:52:44.097777+0000","last_unstale":"2023-11-28T02:52:44.461323+0000","last_undegraded":"2023-11-28T02:52:44.461323+0000","last_fullsized":"2023-11-28T02:52:44.461323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470412+0000","last_change":"2023-11-28T02:52:44.467211+0000","last_active":"2023-11-28T02:52:44.470412+0000","last_peered":"2023-11-28T02:52:44.470412+0000","last_clean":"2023-11-28T02:52:44.470412+0000","last_became_active":"2023-11-28T02:52:44.466934+0000","last_became_peered":"2023-11-28T02:52:44.466934+0000","last_unstale":"2023-11-28T02:52:44.470412+0000","last_undegraded":"2023-11-28T02:52:44.470412+0000","last_fullsized":"2023-11-28T02:52:44.470412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479015+0000","last_change":"2023-11-28T02:52:44.097548+0000","last_active":"2023-11-28T02:52:44.479015+0000","last_peered":"2023-11-28T02:52:44.479015+0000","last_clean":"2023-11-28T02:52:44.479015+0000","last_became_active":"2023-11-28T02:52:44.097253+0000","last_became_peered":"2023-11-28T02:52:44.097253+0000","last_unstale":"2023-11-28T02:52:44.479015+0000","last_undegraded":"2023-11-28T02:52:44.479015+0000","last_fullsized":"2023-11-28T02:52:44.479015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469674+0000","last_change":"2023-11-28T02:52:44.463550+0000","last_active":"2023-11-28T02:52:44.469674+0000","last_peered":"2023-11-28T02:52:44.469674+0000","last_clean":"2023-11-28T02:52:44.469674+0000","last_became_active":"2023-11-28T02:52:44.463313+0000","last_became_peered":"2023-11-28T02:52:44.463313+0000","last_unstale":"2023-11-28T02:52:44.469674+0000","last_undegraded":"2023-11-28T02:52:44.469674+0000","last_fullsized":"2023-11-28T02:52:44.469674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448095+0000","last_change":"2023-11-28T02:52:44.093490+0000","last_active":"2023-11-28T02:52:44.448095+0000","last_peered":"2023-11-28T02:52:44.448095+0000","last_clean":"2023-11-28T02:52:44.448095+0000","last_became_active":"2023-11-28T02:52:44.093241+0000","last_became_peered":"2023-11-28T02:52:44.093241+0000","last_unstale":"2023-11-28T02:52:44.448095+0000","last_undegraded":"2023-11-28T02:52:44.448095+0000","last_fullsized":"2023-11-28T02:52:44.448095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485046+0000","last_change":"2023-11-28T02:52:44.480001+0000","last_active":"2023-11-28T02:52:44.485046+0000","last_peered":"2023-11-28T02:52:44.485046+0000","last_clean":"2023-11-28T02:52:44.485046+0000","last_became_active":"2023-11-28T02:52:44.479755+0000","last_became_peered":"2023-11-28T02:52:44.479755+0000","last_unstale":"2023-11-28T02:52:44.485046+0000","last_undegraded":"2023-11-28T02:52:44.485046+0000","last_fullsized":"2023-11-28T02:52:44.485046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463559+0000","last_change":"2023-11-28T02:52:44.102658+0000","last_active":"2023-11-28T02:52:44.463559+0000","last_peered":"2023-11-28T02:52:44.463559+0000","last_clean":"2023-11-28T02:52:44.463559+0000","last_became_active":"2023-11-28T02:52:44.102381+0000","last_became_peered":"2023-11-28T02:52:44.102381+0000","last_unstale":"2023-11-28T02:52:44.463559+0000","last_undegraded":"2023-11-28T02:52:44.463559+0000","last_fullsized":"2023-11-28T02:52:44.463559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464315+0000","last_change":"2023-11-28T02:52:44.460430+0000","last_active":"2023-11-28T02:52:44.464315+0000","last_peered":"2023-11-28T02:52:44.464315+0000","last_clean":"2023-11-28T02:52:44.464315+0000","last_became_active":"2023-11-28T02:52:44.460161+0000","last_became_peered":"2023-11-28T02:52:44.460161+0000","last_unstale":"2023-11-28T02:52:44.464315+0000","last_undegraded":"2023-11-28T02:52:44.464315+0000","last_fullsized":"2023-11-28T02:52:44.464315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472669+0000","last_change":"2023-11-28T02:52:44.099082+0000","last_active":"2023-11-28T02:52:44.472669+0000","last_peered":"2023-11-28T02:52:44.472669+0000","last_clean":"2023-11-28T02:52:44.472669+0000","last_became_active":"2023-11-28T02:52:44.098444+0000","last_became_peered":"2023-11-28T02:52:44.098444+0000","last_unstale":"2023-11-28T02:52:44.472669+0000","last_undegraded":"2023-11-28T02:52:44.472669+0000","last_fullsized":"2023-11-28T02:52:44.472669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472505+0000","last_change":"2023-11-28T02:52:44.467312+0000","last_active":"2023-11-28T02:52:44.472505+0000","last_peered":"2023-11-28T02:52:44.472505+0000","last_clean":"2023-11-28T02:52:44.472505+0000","last_became_active":"2023-11-28T02:52:44.467066+0000","last_became_peered":"2023-11-28T02:52:44.467066+0000","last_unstale":"2023-11-28T02:52:44.472505+0000","last_undegraded":"2023-11-28T02:52:44.472505+0000","last_fullsized":"2023-11-28T02:52:44.472505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462586+0000","last_change":"2023-11-28T02:52:44.089495+0000","last_active":"2023-11-28T02:52:44.462586+0000","last_peered":"2023-11-28T02:52:44.462586+0000","last_clean":"2023-11-28T02:52:44.462586+0000","last_became_active":"2023-11-28T02:52:44.089229+0000","last_became_peered":"2023-11-28T02:52:44.089229+0000","last_unstale":"2023-11-28T02:52:44.462586+0000","last_undegraded":"2023-11-28T02:52:44.462586+0000","last_fullsized":"2023-11-28T02:52:44.462586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469877+0000","last_change":"2023-11-28T02:52:44.465836+0000","last_active":"2023-11-28T02:52:44.469877+0000","last_peered":"2023-11-28T02:52:44.469877+0000","last_clean":"2023-11-28T02:52:44.469877+0000","last_became_active":"2023-11-28T02:52:44.465430+0000","last_became_peered":"2023-11-28T02:52:44.465430+0000","last_unstale":"2023-11-28T02:52:44.469877+0000","last_undegraded":"2023-11-28T02:52:44.469877+0000","last_fullsized":"2023-11-28T02:52:44.469877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453296+0000","last_change":"2023-11-28T02:52:44.100936+0000","last_active":"2023-11-28T02:52:44.453296+0000","last_peered":"2023-11-28T02:52:44.453296+0000","last_clean":"2023-11-28T02:52:44.453296+0000","last_became_active":"2023-11-28T02:52:44.100435+0000","last_became_peered":"2023-11-28T02:52:44.100435+0000","last_unstale":"2023-11-28T02:52:44.453296+0000","last_undegraded":"2023-11-28T02:52:44.453296+0000","last_fullsized":"2023-11-28T02:52:44.453296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468382+0000","last_change":"2023-11-28T02:52:44.464061+0000","last_active":"2023-11-28T02:52:44.468382+0000","last_peered":"2023-11-28T02:52:44.468382+0000","last_clean":"2023-11-28T02:52:44.468382+0000","last_became_active":"2023-11-28T02:52:44.463811+0000","last_became_peered":"2023-11-28T02:52:44.463811+0000","last_unstale":"2023-11-28T02:52:44.468382+0000","last_undegraded":"2023-11-28T02:52:44.468382+0000","last_fullsized":"2023-11-28T02:52:44.468382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466528+0000","last_change":"2023-11-28T02:52:44.095761+0000","last_active":"2023-11-28T02:52:44.466528+0000","last_peered":"2023-11-28T02:52:44.466528+0000","last_clean":"2023-11-28T02:52:44.466528+0000","last_became_active":"2023-11-28T02:52:44.095568+0000","last_became_peered":"2023-11-28T02:52:44.095568+0000","last_unstale":"2023-11-28T02:52:44.466528+0000","last_undegraded":"2023-11-28T02:52:44.466528+0000","last_fullsized":"2023-11-28T02:52:44.466528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462346+0000","last_change":"2023-11-28T02:52:44.460026+0000","last_active":"2023-11-28T02:52:44.462346+0000","last_peered":"2023-11-28T02:52:44.462346+0000","last_clean":"2023-11-28T02:52:44.462346+0000","last_became_active":"2023-11-28T02:52:44.459771+0000","last_became_peered":"2023-11-28T02:52:44.459771+0000","last_unstale":"2023-11-28T02:52:44.462346+0000","last_undegraded":"2023-11-28T02:52:44.462346+0000","last_fullsized":"2023-11-28T02:52:44.462346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479985+0000","last_change":"2023-11-28T02:52:44.101052+0000","last_active":"2023-11-28T02:52:44.479985+0000","last_peered":"2023-11-28T02:52:44.479985+0000","last_clean":"2023-11-28T02:52:44.479985+0000","last_became_active":"2023-11-28T02:52:44.099171+0000","last_became_peered":"2023-11-28T02:52:44.099171+0000","last_unstale":"2023-11-28T02:52:44.479985+0000","last_undegraded":"2023-11-28T02:52:44.479985+0000","last_fullsized":"2023-11-28T02:52:44.479985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471983+0000","last_change":"2023-11-28T02:52:44.464434+0000","last_active":"2023-11-28T02:52:44.471983+0000","last_peered":"2023-11-28T02:52:44.471983+0000","last_clean":"2023-11-28T02:52:44.471983+0000","last_became_active":"2023-11-28T02:52:44.463883+0000","last_became_peered":"2023-11-28T02:52:44.463883+0000","last_unstale":"2023-11-28T02:52:44.471983+0000","last_undegraded":"2023-11-28T02:52:44.471983+0000","last_fullsized":"2023-11-28T02:52:44.471983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462246+0000","last_change":"2023-11-28T02:52:44.099394+0000","last_active":"2023-11-28T02:52:44.462246+0000","last_peered":"2023-11-28T02:52:44.462246+0000","last_clean":"2023-11-28T02:52:44.462246+0000","last_became_active":"2023-11-28T02:52:44.099099+0000","last_became_peered":"2023-11-28T02:52:44.099099+0000","last_unstale":"2023-11-28T02:52:44.462246+0000","last_undegraded":"2023-11-28T02:52:44.462246+0000","last_fullsized":"2023-11-28T02:52:44.462246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480138+0000","last_change":"2023-11-28T02:52:44.478063+0000","last_active":"2023-11-28T02:52:44.480138+0000","last_peered":"2023-11-28T02:52:44.480138+0000","last_clean":"2023-11-28T02:52:44.480138+0000","last_became_active":"2023-11-28T02:52:44.477777+0000","last_became_peered":"2023-11-28T02:52:44.477777+0000","last_unstale":"2023-11-28T02:52:44.480138+0000","last_undegraded":"2023-11-28T02:52:44.480138+0000","last_fullsized":"2023-11-28T02:52:44.480138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461922+0000","last_change":"2023-11-28T02:52:44.099389+0000","last_active":"2023-11-28T02:52:44.461922+0000","last_peered":"2023-11-28T02:52:44.461922+0000","last_clean":"2023-11-28T02:52:44.461922+0000","last_became_active":"2023-11-28T02:52:44.099084+0000","last_became_peered":"2023-11-28T02:52:44.099084+0000","last_unstale":"2023-11-28T02:52:44.461922+0000","last_undegraded":"2023-11-28T02:52:44.461922+0000","last_fullsized":"2023-11-28T02:52:44.461922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459683+0000","last_change":"2023-11-28T02:52:44.454584+0000","last_active":"2023-11-28T02:52:44.459683+0000","last_peered":"2023-11-28T02:52:44.459683+0000","last_clean":"2023-11-28T02:52:44.459683+0000","last_became_active":"2023-11-28T02:52:44.454393+0000","last_became_peered":"2023-11-28T02:52:44.454393+0000","last_unstale":"2023-11-28T02:52:44.459683+0000","last_undegraded":"2023-11-28T02:52:44.459683+0000","last_fullsized":"2023-11-28T02:52:44.459683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478337+0000","last_change":"2023-11-28T02:52:44.098086+0000","last_active":"2023-11-28T02:52:44.478337+0000","last_peered":"2023-11-28T02:52:44.478337+0000","last_clean":"2023-11-28T02:52:44.478337+0000","last_became_active":"2023-11-28T02:52:44.097733+0000","last_became_peered":"2023-11-28T02:52:44.097733+0000","last_unstale":"2023-11-28T02:52:44.478337+0000","last_undegraded":"2023-11-28T02:52:44.478337+0000","last_fullsized":"2023-11-28T02:52:44.478337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463401+0000","last_change":"2023-11-28T02:52:44.460723+0000","last_active":"2023-11-28T02:52:44.463401+0000","last_peered":"2023-11-28T02:52:44.463401+0000","last_clean":"2023-11-28T02:52:44.463401+0000","last_became_active":"2023-11-28T02:52:44.460443+0000","last_became_peered":"2023-11-28T02:52:44.460443+0000","last_unstale":"2023-11-28T02:52:44.463401+0000","last_undegraded":"2023-11-28T02:52:44.463401+0000","last_fullsized":"2023-11-28T02:52:44.463401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451507+0000","last_change":"2023-11-28T02:52:44.094043+0000","last_active":"2023-11-28T02:52:44.451507+0000","last_peered":"2023-11-28T02:52:44.451507+0000","last_clean":"2023-11-28T02:52:44.451507+0000","last_became_active":"2023-11-28T02:52:44.093761+0000","last_became_peered":"2023-11-28T02:52:44.093761+0000","last_unstale":"2023-11-28T02:52:44.451507+0000","last_undegraded":"2023-11-28T02:52:44.451507+0000","last_fullsized":"2023-11-28T02:52:44.451507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473075+0000","last_change":"2023-11-28T02:52:44.469033+0000","last_active":"2023-11-28T02:52:44.473075+0000","last_peered":"2023-11-28T02:52:44.473075+0000","last_clean":"2023-11-28T02:52:44.473075+0000","last_became_active":"2023-11-28T02:52:44.468811+0000","last_became_peered":"2023-11-28T02:52:44.468811+0000","last_unstale":"2023-11-28T02:52:44.473075+0000","last_undegraded":"2023-11-28T02:52:44.473075+0000","last_fullsized":"2023-11-28T02:52:44.473075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475940+0000","last_change":"2023-11-28T02:52:44.097577+0000","last_active":"2023-11-28T02:52:44.475940+0000","last_peered":"2023-11-28T02:52:44.475940+0000","last_clean":"2023-11-28T02:52:44.475940+0000","last_became_active":"2023-11-28T02:52:44.097283+0000","last_became_peered":"2023-11-28T02:52:44.097283+0000","last_unstale":"2023-11-28T02:52:44.475940+0000","last_undegraded":"2023-11-28T02:52:44.475940+0000","last_fullsized":"2023-11-28T02:52:44.475940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464026+0000","last_change":"2023-11-28T02:52:44.459558+0000","last_active":"2023-11-28T02:52:44.464026+0000","last_peered":"2023-11-28T02:52:44.464026+0000","last_clean":"2023-11-28T02:52:44.464026+0000","last_became_active":"2023-11-28T02:52:44.459236+0000","last_became_peered":"2023-11-28T02:52:44.459236+0000","last_unstale":"2023-11-28T02:52:44.464026+0000","last_undegraded":"2023-11-28T02:52:44.464026+0000","last_fullsized":"2023-11-28T02:52:44.464026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469470+0000","last_change":"2023-11-28T02:52:44.098691+0000","last_active":"2023-11-28T02:52:44.469470+0000","last_peered":"2023-11-28T02:52:44.469470+0000","last_clean":"2023-11-28T02:52:44.469470+0000","last_became_active":"2023-11-28T02:52:44.098401+0000","last_became_peered":"2023-11-28T02:52:44.098401+0000","last_unstale":"2023-11-28T02:52:44.469470+0000","last_undegraded":"2023-11-28T02:52:44.469470+0000","last_fullsized":"2023-11-28T02:52:44.469470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469747+0000","last_change":"2023-11-28T02:52:44.464788+0000","last_active":"2023-11-28T02:52:44.469747+0000","last_peered":"2023-11-28T02:52:44.469747+0000","last_clean":"2023-11-28T02:52:44.469747+0000","last_became_active":"2023-11-28T02:52:44.464491+0000","last_became_peered":"2023-11-28T02:52:44.464491+0000","last_unstale":"2023-11-28T02:52:44.469747+0000","last_undegraded":"2023-11-28T02:52:44.469747+0000","last_fullsized":"2023-11-28T02:52:44.469747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477600+0000","last_change":"2023-11-28T02:52:44.095969+0000","last_active":"2023-11-28T02:52:44.477600+0000","last_peered":"2023-11-28T02:52:44.477600+0000","last_clean":"2023-11-28T02:52:44.477600+0000","last_became_active":"2023-11-28T02:52:44.095693+0000","last_became_peered":"2023-11-28T02:52:44.095693+0000","last_unstale":"2023-11-28T02:52:44.477600+0000","last_undegraded":"2023-11-28T02:52:44.477600+0000","last_fullsized":"2023-11-28T02:52:44.477600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470142+0000","last_change":"2023-11-28T02:52:44.462943+0000","last_active":"2023-11-28T02:52:44.470142+0000","last_peered":"2023-11-28T02:52:44.470142+0000","last_clean":"2023-11-28T02:52:44.470142+0000","last_became_active":"2023-11-28T02:52:44.462685+0000","last_became_peered":"2023-11-28T02:52:44.462685+0000","last_unstale":"2023-11-28T02:52:44.470142+0000","last_undegraded":"2023-11-28T02:52:44.470142+0000","last_fullsized":"2023-11-28T02:52:44.470142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469567+0000","last_change":"2023-11-28T02:52:44.100318+0000","last_active":"2023-11-28T02:52:44.469567+0000","last_peered":"2023-11-28T02:52:44.469567+0000","last_clean":"2023-11-28T02:52:44.469567+0000","last_became_active":"2023-11-28T02:52:44.100056+0000","last_became_peered":"2023-11-28T02:52:44.100056+0000","last_unstale":"2023-11-28T02:52:44.469567+0000","last_undegraded":"2023-11-28T02:52:44.469567+0000","last_fullsized":"2023-11-28T02:52:44.469567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463773+0000","last_change":"2023-11-28T02:52:44.460144+0000","last_active":"2023-11-28T02:52:44.463773+0000","last_peered":"2023-11-28T02:52:44.463773+0000","last_clean":"2023-11-28T02:52:44.463773+0000","last_became_active":"2023-11-28T02:52:44.459861+0000","last_became_peered":"2023-11-28T02:52:44.459861+0000","last_unstale":"2023-11-28T02:52:44.463773+0000","last_undegraded":"2023-11-28T02:52:44.463773+0000","last_fullsized":"2023-11-28T02:52:44.463773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450475+0000","last_change":"2023-11-28T02:52:44.101546+0000","last_active":"2023-11-28T02:52:44.450475+0000","last_peered":"2023-11-28T02:52:44.450475+0000","last_clean":"2023-11-28T02:52:44.450475+0000","last_became_active":"2023-11-28T02:52:44.101254+0000","last_became_peered":"2023-11-28T02:52:44.101254+0000","last_unstale":"2023-11-28T02:52:44.450475+0000","last_undegraded":"2023-11-28T02:52:44.450475+0000","last_fullsized":"2023-11-28T02:52:44.450475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479536+0000","last_change":"2023-11-28T02:52:44.477542+0000","last_active":"2023-11-28T02:52:44.479536+0000","last_peered":"2023-11-28T02:52:44.479536+0000","last_clean":"2023-11-28T02:52:44.479536+0000","last_became_active":"2023-11-28T02:52:44.477191+0000","last_became_peered":"2023-11-28T02:52:44.477191+0000","last_unstale":"2023-11-28T02:52:44.479536+0000","last_undegraded":"2023-11-28T02:52:44.479536+0000","last_fullsized":"2023-11-28T02:52:44.479536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464374+0000","last_change":"2023-11-28T02:52:44.097950+0000","last_active":"2023-11-28T02:52:44.464374+0000","last_peered":"2023-11-28T02:52:44.464374+0000","last_clean":"2023-11-28T02:52:44.464374+0000","last_became_active":"2023-11-28T02:52:44.097703+0000","last_became_peered":"2023-11-28T02:52:44.097703+0000","last_unstale":"2023-11-28T02:52:44.464374+0000","last_undegraded":"2023-11-28T02:52:44.464374+0000","last_fullsized":"2023-11-28T02:52:44.464374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472612+0000","last_change":"2023-11-28T02:52:44.466320+0000","last_active":"2023-11-28T02:52:44.472612+0000","last_peered":"2023-11-28T02:52:44.472612+0000","last_clean":"2023-11-28T02:52:44.472612+0000","last_became_active":"2023-11-28T02:52:44.466039+0000","last_became_peered":"2023-11-28T02:52:44.466039+0000","last_unstale":"2023-11-28T02:52:44.472612+0000","last_undegraded":"2023-11-28T02:52:44.472612+0000","last_fullsized":"2023-11-28T02:52:44.472612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485236+0000","last_change":"2023-11-28T02:52:44.093193+0000","last_active":"2023-11-28T02:52:44.485236+0000","last_peered":"2023-11-28T02:52:44.485236+0000","last_clean":"2023-11-28T02:52:44.485236+0000","last_became_active":"2023-11-28T02:52:44.092869+0000","last_became_peered":"2023-11-28T02:52:44.092869+0000","last_unstale":"2023-11-28T02:52:44.485236+0000","last_undegraded":"2023-11-28T02:52:44.485236+0000","last_fullsized":"2023-11-28T02:52:44.485236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297348,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074492,"statfs":{"total":96636764160,"available":96332279808,"internally_reserved":0,"allocated":258048,"data_stored":86755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":86787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297340,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074500,"statfs":{"total":96636764160,"available":96332288000,"internally_reserved":0,"allocated":249856,"data_stored":86933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:52:47.431 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:52:47.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.577+0000 7fa730c6b700 1 Processor -- start 2023-11-28T02:52:47.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.577+0000 7fa730c6b700 1 -- start start 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa730c6b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa72c069330 0x7fa72c06b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa730c6b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c062f60 con 0x7fa72c157680 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa730c6b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa72c069f80 con 0x7fa72c157a60 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa730c6b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa72c062700 con 0x7fa72c069330 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa72c157680 0x7fa72c068c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33708/0 (socket says 172.21.15.73:33708) 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72b7fe700 1 -- 172.21.15.73:0/1442055678 learned_addr learned my addr 172.21.15.73:0/1442055678 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72affd700 1 --2- 172.21.15.73:0/1442055678 >> v2:172.21.15.73:3300/0 conn(0x7fa72c069330 0x7fa72c06b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 75802237 0 0) 0x7fa72c062f60 con 0x7fa72c157680 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7140033f0 con 0x7fa72c157680 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72affd700 1 -- 172.21.15.73:0/1442055678 >> v1:172.21.15.150:6789/0 conn(0x7fa72c157a60 legacy=0x7fa72c0633f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405116124 0 0) 0x7fa72c069f80 con 0x7fa72c157a60 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72affd700 1 -- 172.21.15.73:0/1442055678 >> v1:172.21.15.73:6789/0 conn(0x7fa72c157680 legacy=0x7fa72c068c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.578+0000 7fa72affd700 1 -- 172.21.15.73:0/1442055678 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa72c062910 con 0x7fa72c069330 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa72affd700 1 --2- 172.21.15.73:0/1442055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c069330 0x7fa72c06b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa720009760 tx=0x7fa720009060).ready entity=mon.0 client_cookie=fd701d94a18fa22c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000f510 con 0x7fa72c069330 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72000f670 con 0x7fa72c069330 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa72a7fc700 1 -- 172.21.15.73:0/1442055678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000f9f0 con 0x7fa72c069330 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 -- 172.21.15.73:0/1442055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c069330 msgr2=0x7fa72c06b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 --2- 172.21.15.73:0/1442055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c069330 0x7fa72c06b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7fa720009760 tx=0x7fa720009060).stop 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 -- 172.21.15.73:0/1442055678 shutdown_connections 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 --2- 172.21.15.73:0/1442055678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c069330 0x7fa72c06b870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 -- 172.21.15.73:0/1442055678 >> 172.21.15.73:0/1442055678 conn(0x7fa72c0c7eb0 msgr2=0x7fa72c0c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 -- 172.21.15.73:0/1442055678 shutdown_connections 2023-11-28T02:52:47.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.579+0000 7fa730c6b700 1 -- 172.21.15.73:0/1442055678 wait complete. 2023-11-28T02:52:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.580+0000 7fa730c6b700 1 Processor -- start 2023-11-28T02:52:47.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.580+0000 7fa730c6b700 1 -- start start 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 0x7fa72c148620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 0x7fa72c167c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c154930 con 0x7fa72c157680 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0a5780 con 0x7fa72c069330 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa730c6b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72c0a5a60 con 0x7fa72c157a60 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa72affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa72affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42508/0 (socket says 172.21.15.73:42508) 2023-11-28T02:52:47.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa72affd700 1 -- 172.21.15.73:0/3623140886 learned_addr learned my addr 172.21.15.73:0/3623140886 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.581+0000 7fa72b7fe700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 0x7fa72c148620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72bfff700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 0x7fa72c167c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 -- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 msgr2=0x7fa72c167c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 0x7fa72c167c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 -- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 msgr2=0x7fa72c148620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 0x7fa72c148620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 -- 172.21.15.73:0/3623140886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7200092a0 con 0x7fa72c157680 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72bfff700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 0x7fa72c167c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:47.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72b7fe700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 0x7fa72c148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa72affd700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa72c068730 tx=0x7fa720017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72000f7e0 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72000f940 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72001b660 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72c0a5d40 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.582+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa72c164690 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.583+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa72c06c7c0 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.583+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa72001b9d0 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.583+0000 7fa7137fe700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 0x7fa718061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.583+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa72007bad0 con 0x7fa72c157680 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.583+0000 7fa72b7fe700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 0x7fa718061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.584+0000 7fa72b7fe700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 0x7fa718061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa71c005ad0 tx=0x7fa71c009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.588+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa72004cda0 con 0x7fa72c157680 2023-11-28T02:52:47.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.749+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa72c154a70 con 0x7fa71805f150 2023-11-28T02:52:47.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.758+0000 7fa7137fe700 1 -- 172.21.15.73:0/3623140886 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336451 (secure 0 0 0) 0x7fa72c154a70 con 0x7fa71805f150 2023-11-28T02:52:47.767 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:47.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 msgr2=0x7fa718061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 0x7fa718061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa71c005ad0 tx=0x7fa71c009040).stop 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 msgr2=0x7fa72c0a8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fa72c068730 tx=0x7fa720017000).stop 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 shutdown_connections 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa71805f150 0x7fa718061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa72c157a60 0x7fa72c167c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa72c069330 0x7fa72c148620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 --2- 172.21.15.73:0/3623140886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa72c157680 0x7fa72c0a8f90 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 >> 172.21.15.73:0/3623140886 conn(0x7fa72c0c7eb0 msgr2=0x7fa72c151130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 shutdown_connections 2023-11-28T02:52:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.761+0000 7fa730c6b700 1 -- 172.21.15.73:0/3623140886 wait complete. 2023-11-28T02:52:47.773 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:52:47.775 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T02:52:47.389594+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":207,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378624,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596096,"statfs":{"total":773094113280,"available":770658402304,"internally_reserved":0,"allocated":1900544,"data_stored":687340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470198+0000","last_change":"2023-11-28T02:52:44.467253+0000","last_active":"2023-11-28T02:52:44.470198+0000","last_peered":"2023-11-28T02:52:44.470198+0000","last_clean":"2023-11-28T02:52:44.470198+0000","last_became_active":"2023-11-28T02:52:44.466977+0000","last_became_peered":"2023-11-28T02:52:44.466977+0000","last_unstale":"2023-11-28T02:52:44.470198+0000","last_undegraded":"2023-11-28T02:52:44.470198+0000","last_fullsized":"2023-11-28T02:52:44.470198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455029+0000","last_change":"2023-11-28T02:52:44.098094+0000","last_active":"2023-11-28T02:52:44.455029+0000","last_peered":"2023-11-28T02:52:44.455029+0000","last_clean":"2023-11-28T02:52:44.455029+0000","last_became_active":"2023-11-28T02:52:44.097826+0000","last_became_peered":"2023-11-28T02:52:44.097826+0000","last_unstale":"2023-11-28T02:52:44.455029+0000","last_undegraded":"2023-11-28T02:52:44.455029+0000","last_fullsized":"2023-11-28T02:52:44.455029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479071+0000","last_change":"2023-11-28T02:52:44.478096+0000","last_active":"2023-11-28T02:52:44.479071+0000","last_peered":"2023-11-28T02:52:44.479071+0000","last_clean":"2023-11-28T02:52:44.479071+0000","last_became_active":"2023-11-28T02:52:44.477817+0000","last_became_peered":"2023-11-28T02:52:44.477817+0000","last_unstale":"2023-11-28T02:52:44.479071+0000","last_undegraded":"2023-11-28T02:52:44.479071+0000","last_fullsized":"2023-11-28T02:52:44.479071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462484+0000","last_change":"2023-11-28T02:52:44.087580+0000","last_active":"2023-11-28T02:52:44.462484+0000","last_peered":"2023-11-28T02:52:44.462484+0000","last_clean":"2023-11-28T02:52:44.462484+0000","last_became_active":"2023-11-28T02:52:44.087310+0000","last_became_peered":"2023-11-28T02:52:44.087310+0000","last_unstale":"2023-11-28T02:52:44.462484+0000","last_undegraded":"2023-11-28T02:52:44.462484+0000","last_fullsized":"2023-11-28T02:52:44.462484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472269+0000","last_change":"2023-11-28T02:52:44.466469+0000","last_active":"2023-11-28T02:52:44.472269+0000","last_peered":"2023-11-28T02:52:44.472269+0000","last_clean":"2023-11-28T02:52:44.472269+0000","last_became_active":"2023-11-28T02:52:44.465822+0000","last_became_peered":"2023-11-28T02:52:44.465822+0000","last_unstale":"2023-11-28T02:52:44.472269+0000","last_undegraded":"2023-11-28T02:52:44.472269+0000","last_fullsized":"2023-11-28T02:52:44.472269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451606+0000","last_change":"2023-11-28T02:52:44.097696+0000","last_active":"2023-11-28T02:52:44.451606+0000","last_peered":"2023-11-28T02:52:44.451606+0000","last_clean":"2023-11-28T02:52:44.451606+0000","last_became_active":"2023-11-28T02:52:44.097423+0000","last_became_peered":"2023-11-28T02:52:44.097423+0000","last_unstale":"2023-11-28T02:52:44.451606+0000","last_undegraded":"2023-11-28T02:52:44.451606+0000","last_fullsized":"2023-11-28T02:52:44.451606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469722+0000","last_change":"2023-11-28T02:52:44.465794+0000","last_active":"2023-11-28T02:52:44.469722+0000","last_peered":"2023-11-28T02:52:44.469722+0000","last_clean":"2023-11-28T02:52:44.469722+0000","last_became_active":"2023-11-28T02:52:44.465545+0000","last_became_peered":"2023-11-28T02:52:44.465545+0000","last_unstale":"2023-11-28T02:52:44.469722+0000","last_undegraded":"2023-11-28T02:52:44.469722+0000","last_fullsized":"2023-11-28T02:52:44.469722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450984+0000","last_change":"2023-11-28T02:52:44.099493+0000","last_active":"2023-11-28T02:52:44.450984+0000","last_peered":"2023-11-28T02:52:44.450984+0000","last_clean":"2023-11-28T02:52:44.450984+0000","last_became_active":"2023-11-28T02:52:44.099248+0000","last_became_peered":"2023-11-28T02:52:44.099248+0000","last_unstale":"2023-11-28T02:52:44.450984+0000","last_undegraded":"2023-11-28T02:52:44.450984+0000","last_fullsized":"2023-11-28T02:52:44.450984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479353+0000","last_change":"2023-11-28T02:52:44.477723+0000","last_active":"2023-11-28T02:52:44.479353+0000","last_peered":"2023-11-28T02:52:44.479353+0000","last_clean":"2023-11-28T02:52:44.479353+0000","last_became_active":"2023-11-28T02:52:44.477430+0000","last_became_peered":"2023-11-28T02:52:44.477430+0000","last_unstale":"2023-11-28T02:52:44.479353+0000","last_undegraded":"2023-11-28T02:52:44.479353+0000","last_fullsized":"2023-11-28T02:52:44.479353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462583+0000","last_change":"2023-11-28T02:52:44.102758+0000","last_active":"2023-11-28T02:52:44.462583+0000","last_peered":"2023-11-28T02:52:44.462583+0000","last_clean":"2023-11-28T02:52:44.462583+0000","last_became_active":"2023-11-28T02:52:44.102441+0000","last_became_peered":"2023-11-28T02:52:44.102441+0000","last_unstale":"2023-11-28T02:52:44.462583+0000","last_undegraded":"2023-11-28T02:52:44.462583+0000","last_fullsized":"2023-11-28T02:52:44.462583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468903+0000","last_change":"2023-11-28T02:52:44.466352+0000","last_active":"2023-11-28T02:52:44.468903+0000","last_peered":"2023-11-28T02:52:44.468903+0000","last_clean":"2023-11-28T02:52:44.468903+0000","last_became_active":"2023-11-28T02:52:44.466023+0000","last_became_peered":"2023-11-28T02:52:44.466023+0000","last_unstale":"2023-11-28T02:52:44.468903+0000","last_undegraded":"2023-11-28T02:52:44.468903+0000","last_fullsized":"2023-11-28T02:52:44.468903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479820+0000","last_change":"2023-11-28T02:52:44.100407+0000","last_active":"2023-11-28T02:52:44.479820+0000","last_peered":"2023-11-28T02:52:44.479820+0000","last_clean":"2023-11-28T02:52:44.479820+0000","last_became_active":"2023-11-28T02:52:44.100184+0000","last_became_peered":"2023-11-28T02:52:44.100184+0000","last_unstale":"2023-11-28T02:52:44.479820+0000","last_undegraded":"2023-11-28T02:52:44.479820+0000","last_fullsized":"2023-11-28T02:52:44.479820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469727+0000","last_change":"2023-11-28T02:52:44.467942+0000","last_active":"2023-11-28T02:52:44.469727+0000","last_peered":"2023-11-28T02:52:44.469727+0000","last_clean":"2023-11-28T02:52:44.469727+0000","last_became_active":"2023-11-28T02:52:44.467659+0000","last_became_peered":"2023-11-28T02:52:44.467659+0000","last_unstale":"2023-11-28T02:52:44.469727+0000","last_undegraded":"2023-11-28T02:52:44.469727+0000","last_fullsized":"2023-11-28T02:52:44.469727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472685+0000","last_change":"2023-11-28T02:52:44.094177+0000","last_active":"2023-11-28T02:52:44.472685+0000","last_peered":"2023-11-28T02:52:44.472685+0000","last_clean":"2023-11-28T02:52:44.472685+0000","last_became_active":"2023-11-28T02:52:44.093901+0000","last_became_peered":"2023-11-28T02:52:44.093901+0000","last_unstale":"2023-11-28T02:52:44.472685+0000","last_undegraded":"2023-11-28T02:52:44.472685+0000","last_fullsized":"2023-11-28T02:52:44.472685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470448+0000","last_change":"2023-11-28T02:52:44.467833+0000","last_active":"2023-11-28T02:52:44.470448+0000","last_peered":"2023-11-28T02:52:44.470448+0000","last_clean":"2023-11-28T02:52:44.470448+0000","last_became_active":"2023-11-28T02:52:44.467517+0000","last_became_peered":"2023-11-28T02:52:44.467517+0000","last_unstale":"2023-11-28T02:52:44.470448+0000","last_undegraded":"2023-11-28T02:52:44.470448+0000","last_fullsized":"2023-11-28T02:52:44.470448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485164+0000","last_change":"2023-11-28T02:52:44.104338+0000","last_active":"2023-11-28T02:52:44.485164+0000","last_peered":"2023-11-28T02:52:44.485164+0000","last_clean":"2023-11-28T02:52:44.485164+0000","last_became_active":"2023-11-28T02:52:44.104073+0000","last_became_peered":"2023-11-28T02:52:44.104073+0000","last_unstale":"2023-11-28T02:52:44.485164+0000","last_undegraded":"2023-11-28T02:52:44.485164+0000","last_fullsized":"2023-11-28T02:52:44.485164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477889+0000","last_change":"2023-11-28T02:52:44.476733+0000","last_active":"2023-11-28T02:52:44.477889+0000","last_peered":"2023-11-28T02:52:44.477889+0000","last_clean":"2023-11-28T02:52:44.477889+0000","last_became_active":"2023-11-28T02:52:44.476400+0000","last_became_peered":"2023-11-28T02:52:44.476400+0000","last_unstale":"2023-11-28T02:52:44.477889+0000","last_undegraded":"2023-11-28T02:52:44.477889+0000","last_fullsized":"2023-11-28T02:52:44.477889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453202+0000","last_change":"2023-11-28T02:52:44.096921+0000","last_active":"2023-11-28T02:52:44.453202+0000","last_peered":"2023-11-28T02:52:44.453202+0000","last_clean":"2023-11-28T02:52:44.453202+0000","last_became_active":"2023-11-28T02:52:44.096687+0000","last_became_peered":"2023-11-28T02:52:44.096687+0000","last_unstale":"2023-11-28T02:52:44.453202+0000","last_undegraded":"2023-11-28T02:52:44.453202+0000","last_fullsized":"2023-11-28T02:52:44.453202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466146+0000","last_change":"2023-11-28T02:52:44.461227+0000","last_active":"2023-11-28T02:52:44.466146+0000","last_peered":"2023-11-28T02:52:44.466146+0000","last_clean":"2023-11-28T02:52:44.466146+0000","last_became_active":"2023-11-28T02:52:44.460957+0000","last_became_peered":"2023-11-28T02:52:44.460957+0000","last_unstale":"2023-11-28T02:52:44.466146+0000","last_undegraded":"2023-11-28T02:52:44.466146+0000","last_fullsized":"2023-11-28T02:52:44.466146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475829+0000","last_change":"2023-11-28T02:52:44.106278+0000","last_active":"2023-11-28T02:52:44.475829+0000","last_peered":"2023-11-28T02:52:44.475829+0000","last_clean":"2023-11-28T02:52:44.475829+0000","last_became_active":"2023-11-28T02:52:44.106026+0000","last_became_peered":"2023-11-28T02:52:44.106026+0000","last_unstale":"2023-11-28T02:52:44.475829+0000","last_undegraded":"2023-11-28T02:52:44.475829+0000","last_fullsized":"2023-11-28T02:52:44.475829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480394+0000","last_change":"2023-11-28T02:52:44.479895+0000","last_active":"2023-11-28T02:52:44.480394+0000","last_peered":"2023-11-28T02:52:44.480394+0000","last_clean":"2023-11-28T02:52:44.480394+0000","last_became_active":"2023-11-28T02:52:44.479685+0000","last_became_peered":"2023-11-28T02:52:44.479685+0000","last_unstale":"2023-11-28T02:52:44.480394+0000","last_undegraded":"2023-11-28T02:52:44.480394+0000","last_fullsized":"2023-11-28T02:52:44.480394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470565+0000","last_change":"2023-11-28T02:52:44.097431+0000","last_active":"2023-11-28T02:52:44.470565+0000","last_peered":"2023-11-28T02:52:44.470565+0000","last_clean":"2023-11-28T02:52:44.470565+0000","last_became_active":"2023-11-28T02:52:44.097043+0000","last_became_peered":"2023-11-28T02:52:44.097043+0000","last_unstale":"2023-11-28T02:52:44.470565+0000","last_undegraded":"2023-11-28T02:52:44.470565+0000","last_fullsized":"2023-11-28T02:52:44.470565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478662+0000","last_change":"2023-11-28T02:52:44.477784+0000","last_active":"2023-11-28T02:52:44.478662+0000","last_peered":"2023-11-28T02:52:44.478662+0000","last_clean":"2023-11-28T02:52:44.478662+0000","last_became_active":"2023-11-28T02:52:44.477515+0000","last_became_peered":"2023-11-28T02:52:44.477515+0000","last_unstale":"2023-11-28T02:52:44.478662+0000","last_undegraded":"2023-11-28T02:52:44.478662+0000","last_fullsized":"2023-11-28T02:52:44.478662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485148+0000","last_change":"2023-11-28T02:52:44.104964+0000","last_active":"2023-11-28T02:52:44.485148+0000","last_peered":"2023-11-28T02:52:44.485148+0000","last_clean":"2023-11-28T02:52:44.485148+0000","last_became_active":"2023-11-28T02:52:44.104688+0000","last_became_peered":"2023-11-28T02:52:44.104688+0000","last_unstale":"2023-11-28T02:52:44.485148+0000","last_undegraded":"2023-11-28T02:52:44.485148+0000","last_fullsized":"2023-11-28T02:52:44.485148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469058+0000","last_change":"2023-11-28T02:52:44.465011+0000","last_active":"2023-11-28T02:52:44.469058+0000","last_peered":"2023-11-28T02:52:44.469058+0000","last_clean":"2023-11-28T02:52:44.469058+0000","last_became_active":"2023-11-28T02:52:44.464725+0000","last_became_peered":"2023-11-28T02:52:44.464725+0000","last_unstale":"2023-11-28T02:52:44.469058+0000","last_undegraded":"2023-11-28T02:52:44.469058+0000","last_fullsized":"2023-11-28T02:52:44.469058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.482297+0000","last_change":"2023-11-28T02:52:44.104530+0000","last_active":"2023-11-28T02:52:44.482297+0000","last_peered":"2023-11-28T02:52:44.482297+0000","last_clean":"2023-11-28T02:52:44.482297+0000","last_became_active":"2023-11-28T02:52:44.104035+0000","last_became_peered":"2023-11-28T02:52:44.104035+0000","last_unstale":"2023-11-28T02:52:44.482297+0000","last_undegraded":"2023-11-28T02:52:44.482297+0000","last_fullsized":"2023-11-28T02:52:44.482297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474584+0000","last_change":"2023-11-28T02:52:44.469216+0000","last_active":"2023-11-28T02:52:44.474584+0000","last_peered":"2023-11-28T02:52:44.474584+0000","last_clean":"2023-11-28T02:52:44.474584+0000","last_became_active":"2023-11-28T02:52:44.467199+0000","last_became_peered":"2023-11-28T02:52:44.467199+0000","last_unstale":"2023-11-28T02:52:44.474584+0000","last_undegraded":"2023-11-28T02:52:44.474584+0000","last_fullsized":"2023-11-28T02:52:44.474584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474622+0000","last_change":"2023-11-28T02:52:44.099749+0000","last_active":"2023-11-28T02:52:44.474622+0000","last_peered":"2023-11-28T02:52:44.474622+0000","last_clean":"2023-11-28T02:52:44.474622+0000","last_became_active":"2023-11-28T02:52:44.099525+0000","last_became_peered":"2023-11-28T02:52:44.099525+0000","last_unstale":"2023-11-28T02:52:44.474622+0000","last_undegraded":"2023-11-28T02:52:44.474622+0000","last_fullsized":"2023-11-28T02:52:44.474622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470191+0000","last_change":"2023-11-28T02:52:44.467373+0000","last_active":"2023-11-28T02:52:44.470191+0000","last_peered":"2023-11-28T02:52:44.470191+0000","last_clean":"2023-11-28T02:52:44.470191+0000","last_became_active":"2023-11-28T02:52:44.464529+0000","last_became_peered":"2023-11-28T02:52:44.464529+0000","last_unstale":"2023-11-28T02:52:44.470191+0000","last_undegraded":"2023-11-28T02:52:44.470191+0000","last_fullsized":"2023-11-28T02:52:44.470191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457475+0000","last_change":"2023-11-28T02:52:44.099774+0000","last_active":"2023-11-28T02:52:44.457475+0000","last_peered":"2023-11-28T02:52:44.457475+0000","last_clean":"2023-11-28T02:52:44.457475+0000","last_became_active":"2023-11-28T02:52:44.099484+0000","last_became_peered":"2023-11-28T02:52:44.099484+0000","last_unstale":"2023-11-28T02:52:44.457475+0000","last_undegraded":"2023-11-28T02:52:44.457475+0000","last_fullsized":"2023-11-28T02:52:44.457475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469631+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.469631+0000","last_peered":"2023-11-28T02:52:44.469631+0000","last_clean":"2023-11-28T02:52:44.469631+0000","last_became_active":"2023-11-28T02:52:44.460712+0000","last_became_peered":"2023-11-28T02:52:44.460712+0000","last_unstale":"2023-11-28T02:52:44.469631+0000","last_undegraded":"2023-11-28T02:52:44.469631+0000","last_fullsized":"2023-11-28T02:52:44.469631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465456+0000","last_change":"2023-11-28T02:52:44.098187+0000","last_active":"2023-11-28T02:52:44.465456+0000","last_peered":"2023-11-28T02:52:44.465456+0000","last_clean":"2023-11-28T02:52:44.465456+0000","last_became_active":"2023-11-28T02:52:44.097937+0000","last_became_peered":"2023-11-28T02:52:44.097937+0000","last_unstale":"2023-11-28T02:52:44.465456+0000","last_undegraded":"2023-11-28T02:52:44.465456+0000","last_fullsized":"2023-11-28T02:52:44.465456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464006+0000","last_change":"2023-11-28T02:52:44.458708+0000","last_active":"2023-11-28T02:52:44.464006+0000","last_peered":"2023-11-28T02:52:44.464006+0000","last_clean":"2023-11-28T02:52:44.464006+0000","last_became_active":"2023-11-28T02:52:44.458372+0000","last_became_peered":"2023-11-28T02:52:44.458372+0000","last_unstale":"2023-11-28T02:52:44.464006+0000","last_undegraded":"2023-11-28T02:52:44.464006+0000","last_fullsized":"2023-11-28T02:52:44.464006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463865+0000","last_change":"2023-11-28T02:52:44.098081+0000","last_active":"2023-11-28T02:52:44.463865+0000","last_peered":"2023-11-28T02:52:44.463865+0000","last_clean":"2023-11-28T02:52:44.463865+0000","last_became_active":"2023-11-28T02:52:44.097763+0000","last_became_peered":"2023-11-28T02:52:44.097763+0000","last_unstale":"2023-11-28T02:52:44.463865+0000","last_undegraded":"2023-11-28T02:52:44.463865+0000","last_fullsized":"2023-11-28T02:52:44.463865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468716+0000","last_change":"2023-11-28T02:52:44.464973+0000","last_active":"2023-11-28T02:52:44.468716+0000","last_peered":"2023-11-28T02:52:44.468716+0000","last_clean":"2023-11-28T02:52:44.468716+0000","last_became_active":"2023-11-28T02:52:44.464692+0000","last_became_peered":"2023-11-28T02:52:44.464692+0000","last_unstale":"2023-11-28T02:52:44.468716+0000","last_undegraded":"2023-11-28T02:52:44.468716+0000","last_fullsized":"2023-11-28T02:52:44.468716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454758+0000","last_change":"2023-11-28T02:52:44.096316+0000","last_active":"2023-11-28T02:52:44.454758+0000","last_peered":"2023-11-28T02:52:44.454758+0000","last_clean":"2023-11-28T02:52:44.454758+0000","last_became_active":"2023-11-28T02:52:44.090973+0000","last_became_peered":"2023-11-28T02:52:44.090973+0000","last_unstale":"2023-11-28T02:52:44.454758+0000","last_undegraded":"2023-11-28T02:52:44.454758+0000","last_fullsized":"2023-11-28T02:52:44.454758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467774+0000","last_change":"2023-11-28T02:52:44.463807+0000","last_active":"2023-11-28T02:52:44.467774+0000","last_peered":"2023-11-28T02:52:44.467774+0000","last_clean":"2023-11-28T02:52:44.467774+0000","last_became_active":"2023-11-28T02:52:44.463532+0000","last_became_peered":"2023-11-28T02:52:44.463532+0000","last_unstale":"2023-11-28T02:52:44.467774+0000","last_undegraded":"2023-11-28T02:52:44.467774+0000","last_fullsized":"2023-11-28T02:52:44.467774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451394+0000","last_change":"2023-11-28T02:52:44.099397+0000","last_active":"2023-11-28T02:52:44.451394+0000","last_peered":"2023-11-28T02:52:44.451394+0000","last_clean":"2023-11-28T02:52:44.451394+0000","last_became_active":"2023-11-28T02:52:44.099225+0000","last_became_peered":"2023-11-28T02:52:44.099225+0000","last_unstale":"2023-11-28T02:52:44.451394+0000","last_undegraded":"2023-11-28T02:52:44.451394+0000","last_fullsized":"2023-11-28T02:52:44.451394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464258+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:52:44.464258+0000","last_peered":"2023-11-28T02:52:44.464258+0000","last_clean":"2023-11-28T02:52:44.464258+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:52:44.464258+0000","last_undegraded":"2023-11-28T02:52:44.464258+0000","last_fullsized":"2023-11-28T02:52:44.464258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461086+0000","last_change":"2023-11-28T02:52:44.099940+0000","last_active":"2023-11-28T02:52:44.461086+0000","last_peered":"2023-11-28T02:52:44.461086+0000","last_clean":"2023-11-28T02:52:44.461086+0000","last_became_active":"2023-11-28T02:52:44.099735+0000","last_became_peered":"2023-11-28T02:52:44.099735+0000","last_unstale":"2023-11-28T02:52:44.461086+0000","last_undegraded":"2023-11-28T02:52:44.461086+0000","last_fullsized":"2023-11-28T02:52:44.461086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473095+0000","last_change":"2023-11-28T02:52:44.472735+0000","last_active":"2023-11-28T02:52:44.473095+0000","last_peered":"2023-11-28T02:52:44.473095+0000","last_clean":"2023-11-28T02:52:44.473095+0000","last_became_active":"2023-11-28T02:52:44.472418+0000","last_became_peered":"2023-11-28T02:52:44.472418+0000","last_unstale":"2023-11-28T02:52:44.473095+0000","last_undegraded":"2023-11-28T02:52:44.473095+0000","last_fullsized":"2023-11-28T02:52:44.473095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459288+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:52:44.459288+0000","last_peered":"2023-11-28T02:52:44.459288+0000","last_clean":"2023-11-28T02:52:44.459288+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:52:44.459288+0000","last_undegraded":"2023-11-28T02:52:44.459288+0000","last_fullsized":"2023-11-28T02:52:44.459288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455813+0000","last_change":"2023-11-28T02:52:44.100602+0000","last_active":"2023-11-28T02:52:44.455813+0000","last_peered":"2023-11-28T02:52:44.455813+0000","last_clean":"2023-11-28T02:52:44.455813+0000","last_became_active":"2023-11-28T02:52:44.100337+0000","last_became_peered":"2023-11-28T02:52:44.100337+0000","last_unstale":"2023-11-28T02:52:44.455813+0000","last_undegraded":"2023-11-28T02:52:44.455813+0000","last_fullsized":"2023-11-28T02:52:44.455813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468871+0000","last_change":"2023-11-28T02:52:44.465676+0000","last_active":"2023-11-28T02:52:44.468871+0000","last_peered":"2023-11-28T02:52:44.468871+0000","last_clean":"2023-11-28T02:52:44.468871+0000","last_became_active":"2023-11-28T02:52:44.465401+0000","last_became_peered":"2023-11-28T02:52:44.465401+0000","last_unstale":"2023-11-28T02:52:44.468871+0000","last_undegraded":"2023-11-28T02:52:44.468871+0000","last_fullsized":"2023-11-28T02:52:44.468871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471562+0000","last_change":"2023-11-28T02:52:44.470158+0000","last_active":"2023-11-28T02:52:44.471562+0000","last_peered":"2023-11-28T02:52:44.471562+0000","last_clean":"2023-11-28T02:52:44.471562+0000","last_became_active":"2023-11-28T02:52:44.469795+0000","last_became_peered":"2023-11-28T02:52:44.469795+0000","last_unstale":"2023-11-28T02:52:44.471562+0000","last_undegraded":"2023-11-28T02:52:44.471562+0000","last_fullsized":"2023-11-28T02:52:44.471562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448512+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:52:44.448512+0000","last_peered":"2023-11-28T02:52:44.448512+0000","last_clean":"2023-11-28T02:52:44.448512+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:52:44.448512+0000","last_undegraded":"2023-11-28T02:52:44.448512+0000","last_fullsized":"2023-11-28T02:52:44.448512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462396+0000","last_change":"2023-11-28T02:52:44.091912+0000","last_active":"2023-11-28T02:52:44.462396+0000","last_peered":"2023-11-28T02:52:44.462396+0000","last_clean":"2023-11-28T02:52:44.462396+0000","last_became_active":"2023-11-28T02:52:44.091630+0000","last_became_peered":"2023-11-28T02:52:44.091630+0000","last_unstale":"2023-11-28T02:52:44.462396+0000","last_undegraded":"2023-11-28T02:52:44.462396+0000","last_fullsized":"2023-11-28T02:52:44.462396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472298+0000","last_change":"2023-11-28T02:52:44.468106+0000","last_active":"2023-11-28T02:52:44.472298+0000","last_peered":"2023-11-28T02:52:44.472298+0000","last_clean":"2023-11-28T02:52:44.472298+0000","last_became_active":"2023-11-28T02:52:44.467833+0000","last_became_peered":"2023-11-28T02:52:44.467833+0000","last_unstale":"2023-11-28T02:52:44.472298+0000","last_undegraded":"2023-11-28T02:52:44.472298+0000","last_fullsized":"2023-11-28T02:52:44.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461405+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:52:44.461405+0000","last_peered":"2023-11-28T02:52:44.461405+0000","last_clean":"2023-11-28T02:52:44.461405+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:52:44.461405+0000","last_undegraded":"2023-11-28T02:52:44.461405+0000","last_fullsized":"2023-11-28T02:52:44.461405+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476368+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:52:44.476368+0000","last_peered":"2023-11-28T02:52:44.476368+0000","last_clean":"2023-11-28T02:52:44.476368+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:52:44.476368+0000","last_undegraded":"2023-11-28T02:52:44.476368+0000","last_fullsized":"2023-11-28T02:52:44.476368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453788+0000","last_change":"2023-11-28T02:52:44.095277+0000","last_active":"2023-11-28T02:52:44.453788+0000","last_peered":"2023-11-28T02:52:44.453788+0000","last_clean":"2023-11-28T02:52:44.453788+0000","last_became_active":"2023-11-28T02:52:44.095021+0000","last_became_peered":"2023-11-28T02:52:44.095021+0000","last_unstale":"2023-11-28T02:52:44.453788+0000","last_undegraded":"2023-11-28T02:52:44.453788+0000","last_fullsized":"2023-11-28T02:52:44.453788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469245+0000","last_change":"2023-11-28T02:52:44.464858+0000","last_active":"2023-11-28T02:52:44.469245+0000","last_peered":"2023-11-28T02:52:44.469245+0000","last_clean":"2023-11-28T02:52:44.469245+0000","last_became_active":"2023-11-28T02:52:44.464352+0000","last_became_peered":"2023-11-28T02:52:44.464352+0000","last_unstale":"2023-11-28T02:52:44.469245+0000","last_undegraded":"2023-11-28T02:52:44.469245+0000","last_fullsized":"2023-11-28T02:52:44.469245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476767+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:52:44.476767+0000","last_peered":"2023-11-28T02:52:44.476767+0000","last_clean":"2023-11-28T02:52:44.476767+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:52:44.476767+0000","last_undegraded":"2023-11-28T02:52:44.476767+0000","last_fullsized":"2023-11-28T02:52:44.476767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450488+0000","last_change":"2023-11-28T02:52:44.105713+0000","last_active":"2023-11-28T02:52:44.450488+0000","last_peered":"2023-11-28T02:52:44.450488+0000","last_clean":"2023-11-28T02:52:44.450488+0000","last_became_active":"2023-11-28T02:52:44.105352+0000","last_became_peered":"2023-11-28T02:52:44.105352+0000","last_unstale":"2023-11-28T02:52:44.450488+0000","last_undegraded":"2023-11-28T02:52:44.450488+0000","last_fullsized":"2023-11-28T02:52:44.450488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474530+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:52:44.474530+0000","last_peered":"2023-11-28T02:52:44.474530+0000","last_clean":"2023-11-28T02:52:44.474530+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:52:44.474530+0000","last_undegraded":"2023-11-28T02:52:44.474530+0000","last_fullsized":"2023-11-28T02:52:44.474530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464373+0000","last_change":"2023-11-28T02:52:44.096162+0000","last_active":"2023-11-28T02:52:44.464373+0000","last_peered":"2023-11-28T02:52:44.464373+0000","last_clean":"2023-11-28T02:52:44.464373+0000","last_became_active":"2023-11-28T02:52:44.095900+0000","last_became_peered":"2023-11-28T02:52:44.095900+0000","last_unstale":"2023-11-28T02:52:44.464373+0000","last_undegraded":"2023-11-28T02:52:44.464373+0000","last_fullsized":"2023-11-28T02:52:44.464373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470164+0000","last_change":"2023-11-28T02:52:44.464449+0000","last_active":"2023-11-28T02:52:44.470164+0000","last_peered":"2023-11-28T02:52:44.470164+0000","last_clean":"2023-11-28T02:52:44.470164+0000","last_became_active":"2023-11-28T02:52:44.464196+0000","last_became_peered":"2023-11-28T02:52:44.464196+0000","last_unstale":"2023-11-28T02:52:44.470164+0000","last_undegraded":"2023-11-28T02:52:44.470164+0000","last_fullsized":"2023-11-28T02:52:44.470164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478810+0000","last_change":"2023-11-28T02:52:44.094471+0000","last_active":"2023-11-28T02:52:44.478810+0000","last_peered":"2023-11-28T02:52:44.478810+0000","last_clean":"2023-11-28T02:52:44.478810+0000","last_became_active":"2023-11-28T02:52:44.094193+0000","last_became_peered":"2023-11-28T02:52:44.094193+0000","last_unstale":"2023-11-28T02:52:44.478810+0000","last_undegraded":"2023-11-28T02:52:44.478810+0000","last_fullsized":"2023-11-28T02:52:44.478810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464206+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:52:44.464206+0000","last_peered":"2023-11-28T02:52:44.464206+0000","last_clean":"2023-11-28T02:52:44.464206+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:52:44.464206+0000","last_undegraded":"2023-11-28T02:52:44.464206+0000","last_fullsized":"2023-11-28T02:52:44.464206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479397+0000","last_change":"2023-11-28T02:52:44.478383+0000","last_active":"2023-11-28T02:52:44.479397+0000","last_peered":"2023-11-28T02:52:44.479397+0000","last_clean":"2023-11-28T02:52:44.479397+0000","last_became_active":"2023-11-28T02:52:44.478122+0000","last_became_peered":"2023-11-28T02:52:44.478122+0000","last_unstale":"2023-11-28T02:52:44.479397+0000","last_undegraded":"2023-11-28T02:52:44.479397+0000","last_fullsized":"2023-11-28T02:52:44.479397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463683+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:52:44.463683+0000","last_peered":"2023-11-28T02:52:44.463683+0000","last_clean":"2023-11-28T02:52:44.463683+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:52:44.463683+0000","last_undegraded":"2023-11-28T02:52:44.463683+0000","last_fullsized":"2023-11-28T02:52:44.463683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451382+0000","last_change":"2023-11-28T02:52:44.099058+0000","last_active":"2023-11-28T02:52:44.451382+0000","last_peered":"2023-11-28T02:52:44.451382+0000","last_clean":"2023-11-28T02:52:44.451382+0000","last_became_active":"2023-11-28T02:52:44.098489+0000","last_became_peered":"2023-11-28T02:52:44.098489+0000","last_unstale":"2023-11-28T02:52:44.451382+0000","last_undegraded":"2023-11-28T02:52:44.451382+0000","last_fullsized":"2023-11-28T02:52:44.451382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475427+0000","last_change":"2023-11-28T02:52:44.473356+0000","last_active":"2023-11-28T02:52:44.475427+0000","last_peered":"2023-11-28T02:52:44.475427+0000","last_clean":"2023-11-28T02:52:44.475427+0000","last_became_active":"2023-11-28T02:52:44.473032+0000","last_became_peered":"2023-11-28T02:52:44.473032+0000","last_unstale":"2023-11-28T02:52:44.475427+0000","last_undegraded":"2023-11-28T02:52:44.475427+0000","last_fullsized":"2023-11-28T02:52:44.475427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471249+0000","last_change":"2023-11-28T02:52:44.469050+0000","last_active":"2023-11-28T02:52:44.471249+0000","last_peered":"2023-11-28T02:52:44.471249+0000","last_clean":"2023-11-28T02:52:44.471249+0000","last_became_active":"2023-11-28T02:52:44.468695+0000","last_became_peered":"2023-11-28T02:52:44.468695+0000","last_unstale":"2023-11-28T02:52:44.471249+0000","last_undegraded":"2023-11-28T02:52:44.471249+0000","last_fullsized":"2023-11-28T02:52:44.471249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452775+0000","last_change":"2023-11-28T02:52:44.105717+0000","last_active":"2023-11-28T02:52:44.452775+0000","last_peered":"2023-11-28T02:52:44.452775+0000","last_clean":"2023-11-28T02:52:44.452775+0000","last_became_active":"2023-11-28T02:52:44.105369+0000","last_became_peered":"2023-11-28T02:52:44.105369+0000","last_unstale":"2023-11-28T02:52:44.452775+0000","last_undegraded":"2023-11-28T02:52:44.452775+0000","last_fullsized":"2023-11-28T02:52:44.452775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465888+0000","last_change":"2023-11-28T02:52:44.463547+0000","last_active":"2023-11-28T02:52:44.465888+0000","last_peered":"2023-11-28T02:52:44.465888+0000","last_clean":"2023-11-28T02:52:44.465888+0000","last_became_active":"2023-11-28T02:52:44.463254+0000","last_became_peered":"2023-11-28T02:52:44.463254+0000","last_unstale":"2023-11-28T02:52:44.465888+0000","last_undegraded":"2023-11-28T02:52:44.465888+0000","last_fullsized":"2023-11-28T02:52:44.465888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457589+0000","last_change":"2023-11-28T02:52:44.102948+0000","last_active":"2023-11-28T02:52:44.457589+0000","last_peered":"2023-11-28T02:52:44.457589+0000","last_clean":"2023-11-28T02:52:44.457589+0000","last_became_active":"2023-11-28T02:52:44.102609+0000","last_became_peered":"2023-11-28T02:52:44.102609+0000","last_unstale":"2023-11-28T02:52:44.457589+0000","last_undegraded":"2023-11-28T02:52:44.457589+0000","last_fullsized":"2023-11-28T02:52:44.457589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465782+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.465782+0000","last_peered":"2023-11-28T02:52:44.465782+0000","last_clean":"2023-11-28T02:52:44.465782+0000","last_became_active":"2023-11-28T02:52:44.460662+0000","last_became_peered":"2023-11-28T02:52:44.460662+0000","last_unstale":"2023-11-28T02:52:44.465782+0000","last_undegraded":"2023-11-28T02:52:44.465782+0000","last_fullsized":"2023-11-28T02:52:44.465782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469497+0000","last_change":"2023-11-28T02:52:44.104319+0000","last_active":"2023-11-28T02:52:44.469497+0000","last_peered":"2023-11-28T02:52:44.469497+0000","last_clean":"2023-11-28T02:52:44.469497+0000","last_became_active":"2023-11-28T02:52:44.104055+0000","last_became_peered":"2023-11-28T02:52:44.104055+0000","last_unstale":"2023-11-28T02:52:44.469497+0000","last_undegraded":"2023-11-28T02:52:44.469497+0000","last_fullsized":"2023-11-28T02:52:44.469497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466536+0000","last_change":"2023-11-28T02:52:44.460800+0000","last_active":"2023-11-28T02:52:44.466536+0000","last_peered":"2023-11-28T02:52:44.466536+0000","last_clean":"2023-11-28T02:52:44.466536+0000","last_became_active":"2023-11-28T02:52:44.460518+0000","last_became_peered":"2023-11-28T02:52:44.460518+0000","last_unstale":"2023-11-28T02:52:44.466536+0000","last_undegraded":"2023-11-28T02:52:44.466536+0000","last_fullsized":"2023-11-28T02:52:44.466536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466150+0000","last_change":"2023-11-28T02:52:44.093970+0000","last_active":"2023-11-28T02:52:44.466150+0000","last_peered":"2023-11-28T02:52:44.466150+0000","last_clean":"2023-11-28T02:52:44.466150+0000","last_became_active":"2023-11-28T02:52:44.093701+0000","last_became_peered":"2023-11-28T02:52:44.093701+0000","last_unstale":"2023-11-28T02:52:44.466150+0000","last_undegraded":"2023-11-28T02:52:44.466150+0000","last_fullsized":"2023-11-28T02:52:44.466150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475064+0000","last_change":"2023-11-28T02:52:44.469053+0000","last_active":"2023-11-28T02:52:44.475064+0000","last_peered":"2023-11-28T02:52:44.475064+0000","last_clean":"2023-11-28T02:52:44.475064+0000","last_became_active":"2023-11-28T02:52:44.468580+0000","last_became_peered":"2023-11-28T02:52:44.468580+0000","last_unstale":"2023-11-28T02:52:44.475064+0000","last_undegraded":"2023-11-28T02:52:44.475064+0000","last_fullsized":"2023-11-28T02:52:44.475064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464009+0000","last_change":"2023-11-28T02:52:44.099449+0000","last_active":"2023-11-28T02:52:44.464009+0000","last_peered":"2023-11-28T02:52:44.464009+0000","last_clean":"2023-11-28T02:52:44.464009+0000","last_became_active":"2023-11-28T02:52:44.099097+0000","last_became_peered":"2023-11-28T02:52:44.099097+0000","last_unstale":"2023-11-28T02:52:44.464009+0000","last_undegraded":"2023-11-28T02:52:44.464009+0000","last_fullsized":"2023-11-28T02:52:44.464009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470464+0000","last_change":"2023-11-28T02:52:44.467418+0000","last_active":"2023-11-28T02:52:44.470464+0000","last_peered":"2023-11-28T02:52:44.470464+0000","last_clean":"2023-11-28T02:52:44.470464+0000","last_became_active":"2023-11-28T02:52:44.465656+0000","last_became_peered":"2023-11-28T02:52:44.465656+0000","last_unstale":"2023-11-28T02:52:44.470464+0000","last_undegraded":"2023-11-28T02:52:44.470464+0000","last_fullsized":"2023-11-28T02:52:44.470464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462541+0000","last_change":"2023-11-28T02:52:44.095567+0000","last_active":"2023-11-28T02:52:44.462541+0000","last_peered":"2023-11-28T02:52:44.462541+0000","last_clean":"2023-11-28T02:52:44.462541+0000","last_became_active":"2023-11-28T02:52:44.095290+0000","last_became_peered":"2023-11-28T02:52:44.095290+0000","last_unstale":"2023-11-28T02:52:44.462541+0000","last_undegraded":"2023-11-28T02:52:44.462541+0000","last_fullsized":"2023-11-28T02:52:44.462541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467572+0000","last_change":"2023-11-28T02:52:44.464051+0000","last_active":"2023-11-28T02:52:44.467572+0000","last_peered":"2023-11-28T02:52:44.467572+0000","last_clean":"2023-11-28T02:52:44.467572+0000","last_became_active":"2023-11-28T02:52:44.463832+0000","last_became_peered":"2023-11-28T02:52:44.463832+0000","last_unstale":"2023-11-28T02:52:44.467572+0000","last_undegraded":"2023-11-28T02:52:44.467572+0000","last_fullsized":"2023-11-28T02:52:44.467572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448090+0000","last_change":"2023-11-28T02:52:44.092817+0000","last_active":"2023-11-28T02:52:44.448090+0000","last_peered":"2023-11-28T02:52:44.448090+0000","last_clean":"2023-11-28T02:52:44.448090+0000","last_became_active":"2023-11-28T02:52:44.092559+0000","last_became_peered":"2023-11-28T02:52:44.092559+0000","last_unstale":"2023-11-28T02:52:44.448090+0000","last_undegraded":"2023-11-28T02:52:44.448090+0000","last_fullsized":"2023-11-28T02:52:44.448090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477920+0000","last_change":"2023-11-28T02:52:44.476928+0000","last_active":"2023-11-28T02:52:44.477920+0000","last_peered":"2023-11-28T02:52:44.477920+0000","last_clean":"2023-11-28T02:52:44.477920+0000","last_became_active":"2023-11-28T02:52:44.476404+0000","last_became_peered":"2023-11-28T02:52:44.476404+0000","last_unstale":"2023-11-28T02:52:44.477920+0000","last_undegraded":"2023-11-28T02:52:44.477920+0000","last_fullsized":"2023-11-28T02:52:44.477920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461903+0000","last_change":"2023-11-28T02:52:44.100532+0000","last_active":"2023-11-28T02:52:44.461903+0000","last_peered":"2023-11-28T02:52:44.461903+0000","last_clean":"2023-11-28T02:52:44.461903+0000","last_became_active":"2023-11-28T02:52:44.100241+0000","last_became_peered":"2023-11-28T02:52:44.100241+0000","last_unstale":"2023-11-28T02:52:44.461903+0000","last_undegraded":"2023-11-28T02:52:44.461903+0000","last_fullsized":"2023-11-28T02:52:44.461903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477206+0000","last_change":"2023-11-28T02:52:44.476740+0000","last_active":"2023-11-28T02:52:44.477206+0000","last_peered":"2023-11-28T02:52:44.477206+0000","last_clean":"2023-11-28T02:52:44.477206+0000","last_became_active":"2023-11-28T02:52:44.476473+0000","last_became_peered":"2023-11-28T02:52:44.476473+0000","last_unstale":"2023-11-28T02:52:44.477206+0000","last_undegraded":"2023-11-28T02:52:44.477206+0000","last_fullsized":"2023-11-28T02:52:44.477206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452872+0000","last_change":"2023-11-28T02:52:44.101558+0000","last_active":"2023-11-28T02:52:44.452872+0000","last_peered":"2023-11-28T02:52:44.452872+0000","last_clean":"2023-11-28T02:52:44.452872+0000","last_became_active":"2023-11-28T02:52:44.101242+0000","last_became_peered":"2023-11-28T02:52:44.101242+0000","last_unstale":"2023-11-28T02:52:44.452872+0000","last_undegraded":"2023-11-28T02:52:44.452872+0000","last_fullsized":"2023-11-28T02:52:44.452872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476230+0000","last_change":"2023-11-28T02:52:44.475794+0000","last_active":"2023-11-28T02:52:44.476230+0000","last_peered":"2023-11-28T02:52:44.476230+0000","last_clean":"2023-11-28T02:52:44.476230+0000","last_became_active":"2023-11-28T02:52:44.475520+0000","last_became_peered":"2023-11-28T02:52:44.475520+0000","last_unstale":"2023-11-28T02:52:44.476230+0000","last_undegraded":"2023-11-28T02:52:44.476230+0000","last_fullsized":"2023-11-28T02:52:44.476230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454590+0000","last_change":"2023-11-28T02:52:44.102703+0000","last_active":"2023-11-28T02:52:44.454590+0000","last_peered":"2023-11-28T02:52:44.454590+0000","last_clean":"2023-11-28T02:52:44.454590+0000","last_became_active":"2023-11-28T02:52:44.102344+0000","last_became_peered":"2023-11-28T02:52:44.102344+0000","last_unstale":"2023-11-28T02:52:44.454590+0000","last_undegraded":"2023-11-28T02:52:44.454590+0000","last_fullsized":"2023-11-28T02:52:44.454590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469219+0000","last_change":"2023-11-28T02:52:44.467020+0000","last_active":"2023-11-28T02:52:44.469219+0000","last_peered":"2023-11-28T02:52:44.469219+0000","last_clean":"2023-11-28T02:52:44.469219+0000","last_became_active":"2023-11-28T02:52:44.466737+0000","last_became_peered":"2023-11-28T02:52:44.466737+0000","last_unstale":"2023-11-28T02:52:44.469219+0000","last_undegraded":"2023-11-28T02:52:44.469219+0000","last_fullsized":"2023-11-28T02:52:44.469219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462456+0000","last_change":"2023-11-28T02:52:44.095605+0000","last_active":"2023-11-28T02:52:44.462456+0000","last_peered":"2023-11-28T02:52:44.462456+0000","last_clean":"2023-11-28T02:52:44.462456+0000","last_became_active":"2023-11-28T02:52:44.093886+0000","last_became_peered":"2023-11-28T02:52:44.093886+0000","last_unstale":"2023-11-28T02:52:44.462456+0000","last_undegraded":"2023-11-28T02:52:44.462456+0000","last_fullsized":"2023-11-28T02:52:44.462456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469773+0000","last_change":"2023-11-28T02:52:44.466446+0000","last_active":"2023-11-28T02:52:44.469773+0000","last_peered":"2023-11-28T02:52:44.469773+0000","last_clean":"2023-11-28T02:52:44.469773+0000","last_became_active":"2023-11-28T02:52:44.466194+0000","last_became_peered":"2023-11-28T02:52:44.466194+0000","last_unstale":"2023-11-28T02:52:44.469773+0000","last_undegraded":"2023-11-28T02:52:44.469773+0000","last_fullsized":"2023-11-28T02:52:44.469773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462065+0000","last_change":"2023-11-28T02:52:44.089689+0000","last_active":"2023-11-28T02:52:44.462065+0000","last_peered":"2023-11-28T02:52:44.462065+0000","last_clean":"2023-11-28T02:52:44.462065+0000","last_became_active":"2023-11-28T02:52:44.089421+0000","last_became_peered":"2023-11-28T02:52:44.089421+0000","last_unstale":"2023-11-28T02:52:44.462065+0000","last_undegraded":"2023-11-28T02:52:44.462065+0000","last_fullsized":"2023-11-28T02:52:44.462065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472331+0000","last_change":"2023-11-28T02:52:44.468436+0000","last_active":"2023-11-28T02:52:44.472331+0000","last_peered":"2023-11-28T02:52:44.472331+0000","last_clean":"2023-11-28T02:52:44.472331+0000","last_became_active":"2023-11-28T02:52:44.468102+0000","last_became_peered":"2023-11-28T02:52:44.468102+0000","last_unstale":"2023-11-28T02:52:44.472331+0000","last_undegraded":"2023-11-28T02:52:44.472331+0000","last_fullsized":"2023-11-28T02:52:44.472331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457616+0000","last_change":"2023-11-28T02:52:44.104653+0000","last_active":"2023-11-28T02:52:44.457616+0000","last_peered":"2023-11-28T02:52:44.457616+0000","last_clean":"2023-11-28T02:52:44.457616+0000","last_became_active":"2023-11-28T02:52:44.104404+0000","last_became_peered":"2023-11-28T02:52:44.104404+0000","last_unstale":"2023-11-28T02:52:44.457616+0000","last_undegraded":"2023-11-28T02:52:44.457616+0000","last_fullsized":"2023-11-28T02:52:44.457616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464296+0000","last_change":"2023-11-28T02:52:44.460305+0000","last_active":"2023-11-28T02:52:44.464296+0000","last_peered":"2023-11-28T02:52:44.464296+0000","last_clean":"2023-11-28T02:52:44.464296+0000","last_became_active":"2023-11-28T02:52:44.460040+0000","last_became_peered":"2023-11-28T02:52:44.460040+0000","last_unstale":"2023-11-28T02:52:44.464296+0000","last_undegraded":"2023-11-28T02:52:44.464296+0000","last_fullsized":"2023-11-28T02:52:44.464296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461498+0000","last_change":"2023-11-28T02:52:44.103715+0000","last_active":"2023-11-28T02:52:44.461498+0000","last_peered":"2023-11-28T02:52:44.461498+0000","last_clean":"2023-11-28T02:52:44.461498+0000","last_became_active":"2023-11-28T02:52:44.103451+0000","last_became_peered":"2023-11-28T02:52:44.103451+0000","last_unstale":"2023-11-28T02:52:44.461498+0000","last_undegraded":"2023-11-28T02:52:44.461498+0000","last_fullsized":"2023-11-28T02:52:44.461498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464165+0000","last_change":"2023-11-28T02:52:44.460282+0000","last_active":"2023-11-28T02:52:44.464165+0000","last_peered":"2023-11-28T02:52:44.464165+0000","last_clean":"2023-11-28T02:52:44.464165+0000","last_became_active":"2023-11-28T02:52:44.460011+0000","last_became_peered":"2023-11-28T02:52:44.460011+0000","last_unstale":"2023-11-28T02:52:44.464165+0000","last_undegraded":"2023-11-28T02:52:44.464165+0000","last_fullsized":"2023-11-28T02:52:44.464165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454784+0000","last_change":"2023-11-28T02:52:44.104288+0000","last_active":"2023-11-28T02:52:44.454784+0000","last_peered":"2023-11-28T02:52:44.454784+0000","last_clean":"2023-11-28T02:52:44.454784+0000","last_became_active":"2023-11-28T02:52:44.102746+0000","last_became_peered":"2023-11-28T02:52:44.102746+0000","last_unstale":"2023-11-28T02:52:44.454784+0000","last_undegraded":"2023-11-28T02:52:44.454784+0000","last_fullsized":"2023-11-28T02:52:44.454784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471224+0000","last_change":"2023-11-28T02:52:44.468068+0000","last_active":"2023-11-28T02:52:44.471224+0000","last_peered":"2023-11-28T02:52:44.471224+0000","last_clean":"2023-11-28T02:52:44.471224+0000","last_became_active":"2023-11-28T02:52:44.467788+0000","last_became_peered":"2023-11-28T02:52:44.467788+0000","last_unstale":"2023-11-28T02:52:44.471224+0000","last_undegraded":"2023-11-28T02:52:44.471224+0000","last_fullsized":"2023-11-28T02:52:44.471224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461485+0000","last_change":"2023-11-28T02:52:44.102690+0000","last_active":"2023-11-28T02:52:44.461485+0000","last_peered":"2023-11-28T02:52:44.461485+0000","last_clean":"2023-11-28T02:52:44.461485+0000","last_became_active":"2023-11-28T02:52:44.102408+0000","last_became_peered":"2023-11-28T02:52:44.102408+0000","last_unstale":"2023-11-28T02:52:44.461485+0000","last_undegraded":"2023-11-28T02:52:44.461485+0000","last_fullsized":"2023-11-28T02:52:44.461485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469649+0000","last_change":"2023-11-28T02:52:44.467827+0000","last_active":"2023-11-28T02:52:44.469649+0000","last_peered":"2023-11-28T02:52:44.469649+0000","last_clean":"2023-11-28T02:52:44.469649+0000","last_became_active":"2023-11-28T02:52:44.467547+0000","last_became_peered":"2023-11-28T02:52:44.467547+0000","last_unstale":"2023-11-28T02:52:44.469649+0000","last_undegraded":"2023-11-28T02:52:44.469649+0000","last_fullsized":"2023-11-28T02:52:44.469649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.460918+0000","last_change":"2023-11-28T02:52:44.097889+0000","last_active":"2023-11-28T02:52:44.460918+0000","last_peered":"2023-11-28T02:52:44.460918+0000","last_clean":"2023-11-28T02:52:44.460918+0000","last_became_active":"2023-11-28T02:52:44.097641+0000","last_became_peered":"2023-11-28T02:52:44.097641+0000","last_unstale":"2023-11-28T02:52:44.460918+0000","last_undegraded":"2023-11-28T02:52:44.460918+0000","last_fullsized":"2023-11-28T02:52:44.460918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479544+0000","last_change":"2023-11-28T02:52:44.477649+0000","last_active":"2023-11-28T02:52:44.479544+0000","last_peered":"2023-11-28T02:52:44.479544+0000","last_clean":"2023-11-28T02:52:44.479544+0000","last_became_active":"2023-11-28T02:52:44.477212+0000","last_became_peered":"2023-11-28T02:52:44.477212+0000","last_unstale":"2023-11-28T02:52:44.479544+0000","last_undegraded":"2023-11-28T02:52:44.479544+0000","last_fullsized":"2023-11-28T02:52:44.479544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461323+0000","last_change":"2023-11-28T02:52:44.098078+0000","last_active":"2023-11-28T02:52:44.461323+0000","last_peered":"2023-11-28T02:52:44.461323+0000","last_clean":"2023-11-28T02:52:44.461323+0000","last_became_active":"2023-11-28T02:52:44.097777+0000","last_became_peered":"2023-11-28T02:52:44.097777+0000","last_unstale":"2023-11-28T02:52:44.461323+0000","last_undegraded":"2023-11-28T02:52:44.461323+0000","last_fullsized":"2023-11-28T02:52:44.461323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470412+0000","last_change":"2023-11-28T02:52:44.467211+0000","last_active":"2023-11-28T02:52:44.470412+0000","last_peered":"2023-11-28T02:52:44.470412+0000","last_clean":"2023-11-28T02:52:44.470412+0000","last_became_active":"2023-11-28T02:52:44.466934+0000","last_became_peered":"2023-11-28T02:52:44.466934+0000","last_unstale":"2023-11-28T02:52:44.470412+0000","last_undegraded":"2023-11-28T02:52:44.470412+0000","last_fullsized":"2023-11-28T02:52:44.470412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479015+0000","last_change":"2023-11-28T02:52:44.097548+0000","last_active":"2023-11-28T02:52:44.479015+0000","last_peered":"2023-11-28T02:52:44.479015+0000","last_clean":"2023-11-28T02:52:44.479015+0000","last_became_active":"2023-11-28T02:52:44.097253+0000","last_became_peered":"2023-11-28T02:52:44.097253+0000","last_unstale":"2023-11-28T02:52:44.479015+0000","last_undegraded":"2023-11-28T02:52:44.479015+0000","last_fullsized":"2023-11-28T02:52:44.479015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469674+0000","last_change":"2023-11-28T02:52:44.463550+0000","last_active":"2023-11-28T02:52:44.469674+0000","last_peered":"2023-11-28T02:52:44.469674+0000","last_clean":"2023-11-28T02:52:44.469674+0000","last_became_active":"2023-11-28T02:52:44.463313+0000","last_became_peered":"2023-11-28T02:52:44.463313+0000","last_unstale":"2023-11-28T02:52:44.469674+0000","last_undegraded":"2023-11-28T02:52:44.469674+0000","last_fullsized":"2023-11-28T02:52:44.469674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448095+0000","last_change":"2023-11-28T02:52:44.093490+0000","last_active":"2023-11-28T02:52:44.448095+0000","last_peered":"2023-11-28T02:52:44.448095+0000","last_clean":"2023-11-28T02:52:44.448095+0000","last_became_active":"2023-11-28T02:52:44.093241+0000","last_became_peered":"2023-11-28T02:52:44.093241+0000","last_unstale":"2023-11-28T02:52:44.448095+0000","last_undegraded":"2023-11-28T02:52:44.448095+0000","last_fullsized":"2023-11-28T02:52:44.448095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485046+0000","last_change":"2023-11-28T02:52:44.480001+0000","last_active":"2023-11-28T02:52:44.485046+0000","last_peered":"2023-11-28T02:52:44.485046+0000","last_clean":"2023-11-28T02:52:44.485046+0000","last_became_active":"2023-11-28T02:52:44.479755+0000","last_became_peered":"2023-11-28T02:52:44.479755+0000","last_unstale":"2023-11-28T02:52:44.485046+0000","last_undegraded":"2023-11-28T02:52:44.485046+0000","last_fullsized":"2023-11-28T02:52:44.485046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463559+0000","last_change":"2023-11-28T02:52:44.102658+0000","last_active":"2023-11-28T02:52:44.463559+0000","last_peered":"2023-11-28T02:52:44.463559+0000","last_clean":"2023-11-28T02:52:44.463559+0000","last_became_active":"2023-11-28T02:52:44.102381+0000","last_became_peered":"2023-11-28T02:52:44.102381+0000","last_unstale":"2023-11-28T02:52:44.463559+0000","last_undegraded":"2023-11-28T02:52:44.463559+0000","last_fullsized":"2023-11-28T02:52:44.463559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464315+0000","last_change":"2023-11-28T02:52:44.460430+0000","last_active":"2023-11-28T02:52:44.464315+0000","last_peered":"2023-11-28T02:52:44.464315+0000","last_clean":"2023-11-28T02:52:44.464315+0000","last_became_active":"2023-11-28T02:52:44.460161+0000","last_became_peered":"2023-11-28T02:52:44.460161+0000","last_unstale":"2023-11-28T02:52:44.464315+0000","last_undegraded":"2023-11-28T02:52:44.464315+0000","last_fullsized":"2023-11-28T02:52:44.464315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472669+0000","last_change":"2023-11-28T02:52:44.099082+0000","last_active":"2023-11-28T02:52:44.472669+0000","last_peered":"2023-11-28T02:52:44.472669+0000","last_clean":"2023-11-28T02:52:44.472669+0000","last_became_active":"2023-11-28T02:52:44.098444+0000","last_became_peered":"2023-11-28T02:52:44.098444+0000","last_unstale":"2023-11-28T02:52:44.472669+0000","last_undegraded":"2023-11-28T02:52:44.472669+0000","last_fullsized":"2023-11-28T02:52:44.472669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472505+0000","last_change":"2023-11-28T02:52:44.467312+0000","last_active":"2023-11-28T02:52:44.472505+0000","last_peered":"2023-11-28T02:52:44.472505+0000","last_clean":"2023-11-28T02:52:44.472505+0000","last_became_active":"2023-11-28T02:52:44.467066+0000","last_became_peered":"2023-11-28T02:52:44.467066+0000","last_unstale":"2023-11-28T02:52:44.472505+0000","last_undegraded":"2023-11-28T02:52:44.472505+0000","last_fullsized":"2023-11-28T02:52:44.472505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462586+0000","last_change":"2023-11-28T02:52:44.089495+0000","last_active":"2023-11-28T02:52:44.462586+0000","last_peered":"2023-11-28T02:52:44.462586+0000","last_clean":"2023-11-28T02:52:44.462586+0000","last_became_active":"2023-11-28T02:52:44.089229+0000","last_became_peered":"2023-11-28T02:52:44.089229+0000","last_unstale":"2023-11-28T02:52:44.462586+0000","last_undegraded":"2023-11-28T02:52:44.462586+0000","last_fullsized":"2023-11-28T02:52:44.462586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469877+0000","last_change":"2023-11-28T02:52:44.465836+0000","last_active":"2023-11-28T02:52:44.469877+0000","last_peered":"2023-11-28T02:52:44.469877+0000","last_clean":"2023-11-28T02:52:44.469877+0000","last_became_active":"2023-11-28T02:52:44.465430+0000","last_became_peered":"2023-11-28T02:52:44.465430+0000","last_unstale":"2023-11-28T02:52:44.469877+0000","last_undegraded":"2023-11-28T02:52:44.469877+0000","last_fullsized":"2023-11-28T02:52:44.469877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453296+0000","last_change":"2023-11-28T02:52:44.100936+0000","last_active":"2023-11-28T02:52:44.453296+0000","last_peered":"2023-11-28T02:52:44.453296+0000","last_clean":"2023-11-28T02:52:44.453296+0000","last_became_active":"2023-11-28T02:52:44.100435+0000","last_became_peered":"2023-11-28T02:52:44.100435+0000","last_unstale":"2023-11-28T02:52:44.453296+0000","last_undegraded":"2023-11-28T02:52:44.453296+0000","last_fullsized":"2023-11-28T02:52:44.453296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468382+0000","last_change":"2023-11-28T02:52:44.464061+0000","last_active":"2023-11-28T02:52:44.468382+0000","last_peered":"2023-11-28T02:52:44.468382+0000","last_clean":"2023-11-28T02:52:44.468382+0000","last_became_active":"2023-11-28T02:52:44.463811+0000","last_became_peered":"2023-11-28T02:52:44.463811+0000","last_unstale":"2023-11-28T02:52:44.468382+0000","last_undegraded":"2023-11-28T02:52:44.468382+0000","last_fullsized":"2023-11-28T02:52:44.468382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466528+0000","last_change":"2023-11-28T02:52:44.095761+0000","last_active":"2023-11-28T02:52:44.466528+0000","last_peered":"2023-11-28T02:52:44.466528+0000","last_clean":"2023-11-28T02:52:44.466528+0000","last_became_active":"2023-11-28T02:52:44.095568+0000","last_became_peered":"2023-11-28T02:52:44.095568+0000","last_unstale":"2023-11-28T02:52:44.466528+0000","last_undegraded":"2023-11-28T02:52:44.466528+0000","last_fullsized":"2023-11-28T02:52:44.466528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462346+0000","last_change":"2023-11-28T02:52:44.460026+0000","last_active":"2023-11-28T02:52:44.462346+0000","last_peered":"2023-11-28T02:52:44.462346+0000","last_clean":"2023-11-28T02:52:44.462346+0000","last_became_active":"2023-11-28T02:52:44.459771+0000","last_became_peered":"2023-11-28T02:52:44.459771+0000","last_unstale":"2023-11-28T02:52:44.462346+0000","last_undegraded":"2023-11-28T02:52:44.462346+0000","last_fullsized":"2023-11-28T02:52:44.462346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479985+0000","last_change":"2023-11-28T02:52:44.101052+0000","last_active":"2023-11-28T02:52:44.479985+0000","last_peered":"2023-11-28T02:52:44.479985+0000","last_clean":"2023-11-28T02:52:44.479985+0000","last_became_active":"2023-11-28T02:52:44.099171+0000","last_became_peered":"2023-11-28T02:52:44.099171+0000","last_unstale":"2023-11-28T02:52:44.479985+0000","last_undegraded":"2023-11-28T02:52:44.479985+0000","last_fullsized":"2023-11-28T02:52:44.479985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471983+0000","last_change":"2023-11-28T02:52:44.464434+0000","last_active":"2023-11-28T02:52:44.471983+0000","last_peered":"2023-11-28T02:52:44.471983+0000","last_clean":"2023-11-28T02:52:44.471983+0000","last_became_active":"2023-11-28T02:52:44.463883+0000","last_became_peered":"2023-11-28T02:52:44.463883+0000","last_unstale":"2023-11-28T02:52:44.471983+0000","last_undegraded":"2023-11-28T02:52:44.471983+0000","last_fullsized":"2023-11-28T02:52:44.471983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462246+0000","last_change":"2023-11-28T02:52:44.099394+0000","last_active":"2023-11-28T02:52:44.462246+0000","last_peered":"2023-11-28T02:52:44.462246+0000","last_clean":"2023-11-28T02:52:44.462246+0000","last_became_active":"2023-11-28T02:52:44.099099+0000","last_became_peered":"2023-11-28T02:52:44.099099+0000","last_unstale":"2023-11-28T02:52:44.462246+0000","last_undegraded":"2023-11-28T02:52:44.462246+0000","last_fullsized":"2023-11-28T02:52:44.462246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480138+0000","last_change":"2023-11-28T02:52:44.478063+0000","last_active":"2023-11-28T02:52:44.480138+0000","last_peered":"2023-11-28T02:52:44.480138+0000","last_clean":"2023-11-28T02:52:44.480138+0000","last_became_active":"2023-11-28T02:52:44.477777+0000","last_became_peered":"2023-11-28T02:52:44.477777+0000","last_unstale":"2023-11-28T02:52:44.480138+0000","last_undegraded":"2023-11-28T02:52:44.480138+0000","last_fullsized":"2023-11-28T02:52:44.480138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461922+0000","last_change":"2023-11-28T02:52:44.099389+0000","last_active":"2023-11-28T02:52:44.461922+0000","last_peered":"2023-11-28T02:52:44.461922+0000","last_clean":"2023-11-28T02:52:44.461922+0000","last_became_active":"2023-11-28T02:52:44.099084+0000","last_became_peered":"2023-11-28T02:52:44.099084+0000","last_unstale":"2023-11-28T02:52:44.461922+0000","last_undegraded":"2023-11-28T02:52:44.461922+0000","last_fullsized":"2023-11-28T02:52:44.461922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459683+0000","last_change":"2023-11-28T02:52:44.454584+0000","last_active":"2023-11-28T02:52:44.459683+0000","last_peered":"2023-11-28T02:52:44.459683+0000","last_clean":"2023-11-28T02:52:44.459683+0000","last_became_active":"2023-11-28T02:52:44.454393+0000","last_became_peered":"2023-11-28T02:52:44.454393+0000","last_unstale":"2023-11-28T02:52:44.459683+0000","last_undegraded":"2023-11-28T02:52:44.459683+0000","last_fullsized":"2023-11-28T02:52:44.459683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478337+0000","last_change":"2023-11-28T02:52:44.098086+0000","last_active":"2023-11-28T02:52:44.478337+0000","last_peered":"2023-11-28T02:52:44.478337+0000","last_clean":"2023-11-28T02:52:44.478337+0000","last_became_active":"2023-11-28T02:52:44.097733+0000","last_became_peered":"2023-11-28T02:52:44.097733+0000","last_unstale":"2023-11-28T02:52:44.478337+0000","last_undegraded":"2023-11-28T02:52:44.478337+0000","last_fullsized":"2023-11-28T02:52:44.478337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463401+0000","last_change":"2023-11-28T02:52:44.460723+0000","last_active":"2023-11-28T02:52:44.463401+0000","last_peered":"2023-11-28T02:52:44.463401+0000","last_clean":"2023-11-28T02:52:44.463401+0000","last_became_active":"2023-11-28T02:52:44.460443+0000","last_became_peered":"2023-11-28T02:52:44.460443+0000","last_unstale":"2023-11-28T02:52:44.463401+0000","last_undegraded":"2023-11-28T02:52:44.463401+0000","last_fullsized":"2023-11-28T02:52:44.463401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451507+0000","last_change":"2023-11-28T02:52:44.094043+0000","last_active":"2023-11-28T02:52:44.451507+0000","last_peered":"2023-11-28T02:52:44.451507+0000","last_clean":"2023-11-28T02:52:44.451507+0000","last_became_active":"2023-11-28T02:52:44.093761+0000","last_became_peered":"2023-11-28T02:52:44.093761+0000","last_unstale":"2023-11-28T02:52:44.451507+0000","last_undegraded":"2023-11-28T02:52:44.451507+0000","last_fullsized":"2023-11-28T02:52:44.451507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473075+0000","last_change":"2023-11-28T02:52:44.469033+0000","last_active":"2023-11-28T02:52:44.473075+0000","last_peered":"2023-11-28T02:52:44.473075+0000","last_clean":"2023-11-28T02:52:44.473075+0000","last_became_active":"2023-11-28T02:52:44.468811+0000","last_became_peered":"2023-11-28T02:52:44.468811+0000","last_unstale":"2023-11-28T02:52:44.473075+0000","last_undegraded":"2023-11-28T02:52:44.473075+0000","last_fullsized":"2023-11-28T02:52:44.473075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475940+0000","last_change":"2023-11-28T02:52:44.097577+0000","last_active":"2023-11-28T02:52:44.475940+0000","last_peered":"2023-11-28T02:52:44.475940+0000","last_clean":"2023-11-28T02:52:44.475940+0000","last_became_active":"2023-11-28T02:52:44.097283+0000","last_became_peered":"2023-11-28T02:52:44.097283+0000","last_unstale":"2023-11-28T02:52:44.475940+0000","last_undegraded":"2023-11-28T02:52:44.475940+0000","last_fullsized":"2023-11-28T02:52:44.475940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464026+0000","last_change":"2023-11-28T02:52:44.459558+0000","last_active":"2023-11-28T02:52:44.464026+0000","last_peered":"2023-11-28T02:52:44.464026+0000","last_clean":"2023-11-28T02:52:44.464026+0000","last_became_active":"2023-11-28T02:52:44.459236+0000","last_became_peered":"2023-11-28T02:52:44.459236+0000","last_unstale":"2023-11-28T02:52:44.464026+0000","last_undegraded":"2023-11-28T02:52:44.464026+0000","last_fullsized":"2023-11-28T02:52:44.464026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469470+0000","last_change":"2023-11-28T02:52:44.098691+0000","last_active":"2023-11-28T02:52:44.469470+0000","last_peered":"2023-11-28T02:52:44.469470+0000","last_clean":"2023-11-28T02:52:44.469470+0000","last_became_active":"2023-11-28T02:52:44.098401+0000","last_became_peered":"2023-11-28T02:52:44.098401+0000","last_unstale":"2023-11-28T02:52:44.469470+0000","last_undegraded":"2023-11-28T02:52:44.469470+0000","last_fullsized":"2023-11-28T02:52:44.469470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469747+0000","last_change":"2023-11-28T02:52:44.464788+0000","last_active":"2023-11-28T02:52:44.469747+0000","last_peered":"2023-11-28T02:52:44.469747+0000","last_clean":"2023-11-28T02:52:44.469747+0000","last_became_active":"2023-11-28T02:52:44.464491+0000","last_became_peered":"2023-11-28T02:52:44.464491+0000","last_unstale":"2023-11-28T02:52:44.469747+0000","last_undegraded":"2023-11-28T02:52:44.469747+0000","last_fullsized":"2023-11-28T02:52:44.469747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477600+0000","last_change":"2023-11-28T02:52:44.095969+0000","last_active":"2023-11-28T02:52:44.477600+0000","last_peered":"2023-11-28T02:52:44.477600+0000","last_clean":"2023-11-28T02:52:44.477600+0000","last_became_active":"2023-11-28T02:52:44.095693+0000","last_became_peered":"2023-11-28T02:52:44.095693+0000","last_unstale":"2023-11-28T02:52:44.477600+0000","last_undegraded":"2023-11-28T02:52:44.477600+0000","last_fullsized":"2023-11-28T02:52:44.477600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470142+0000","last_change":"2023-11-28T02:52:44.462943+0000","last_active":"2023-11-28T02:52:44.470142+0000","last_peered":"2023-11-28T02:52:44.470142+0000","last_clean":"2023-11-28T02:52:44.470142+0000","last_became_active":"2023-11-28T02:52:44.462685+0000","last_became_peered":"2023-11-28T02:52:44.462685+0000","last_unstale":"2023-11-28T02:52:44.470142+0000","last_undegraded":"2023-11-28T02:52:44.470142+0000","last_fullsized":"2023-11-28T02:52:44.470142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469567+0000","last_change":"2023-11-28T02:52:44.100318+0000","last_active":"2023-11-28T02:52:44.469567+0000","last_peered":"2023-11-28T02:52:44.469567+0000","last_clean":"2023-11-28T02:52:44.469567+0000","last_became_active":"2023-11-28T02:52:44.100056+0000","last_became_peered":"2023-11-28T02:52:44.100056+0000","last_unstale":"2023-11-28T02:52:44.469567+0000","last_undegraded":"2023-11-28T02:52:44.469567+0000","last_fullsized":"2023-11-28T02:52:44.469567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463773+0000","last_change":"2023-11-28T02:52:44.460144+0000","last_active":"2023-11-28T02:52:44.463773+0000","last_peered":"2023-11-28T02:52:44.463773+0000","last_clean":"2023-11-28T02:52:44.463773+0000","last_became_active":"2023-11-28T02:52:44.459861+0000","last_became_peered":"2023-11-28T02:52:44.459861+0000","last_unstale":"2023-11-28T02:52:44.463773+0000","last_undegraded":"2023-11-28T02:52:44.463773+0000","last_fullsized":"2023-11-28T02:52:44.463773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450475+0000","last_change":"2023-11-28T02:52:44.101546+0000","last_active":"2023-11-28T02:52:44.450475+0000","last_peered":"2023-11-28T02:52:44.450475+0000","last_clean":"2023-11-28T02:52:44.450475+0000","last_became_active":"2023-11-28T02:52:44.101254+0000","last_became_peered":"2023-11-28T02:52:44.101254+0000","last_unstale":"2023-11-28T02:52:44.450475+0000","last_undegraded":"2023-11-28T02:52:44.450475+0000","last_fullsized":"2023-11-28T02:52:44.450475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479536+0000","last_change":"2023-11-28T02:52:44.477542+0000","last_active":"2023-11-28T02:52:44.479536+0000","last_peered":"2023-11-28T02:52:44.479536+0000","last_clean":"2023-11-28T02:52:44.479536+0000","last_became_active":"2023-11-28T02:52:44.477191+0000","last_became_peered":"2023-11-28T02:52:44.477191+0000","last_unstale":"2023-11-28T02:52:44.479536+0000","last_undegraded":"2023-11-28T02:52:44.479536+0000","last_fullsized":"2023-11-28T02:52:44.479536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464374+0000","last_change":"2023-11-28T02:52:44.097950+0000","last_active":"2023-11-28T02:52:44.464374+0000","last_peered":"2023-11-28T02:52:44.464374+0000","last_clean":"2023-11-28T02:52:44.464374+0000","last_became_active":"2023-11-28T02:52:44.097703+0000","last_became_peered":"2023-11-28T02:52:44.097703+0000","last_unstale":"2023-11-28T02:52:44.464374+0000","last_undegraded":"2023-11-28T02:52:44.464374+0000","last_fullsized":"2023-11-28T02:52:44.464374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472612+0000","last_change":"2023-11-28T02:52:44.466320+0000","last_active":"2023-11-28T02:52:44.472612+0000","last_peered":"2023-11-28T02:52:44.472612+0000","last_clean":"2023-11-28T02:52:44.472612+0000","last_became_active":"2023-11-28T02:52:44.466039+0000","last_became_peered":"2023-11-28T02:52:44.466039+0000","last_unstale":"2023-11-28T02:52:44.472612+0000","last_undegraded":"2023-11-28T02:52:44.472612+0000","last_fullsized":"2023-11-28T02:52:44.472612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485236+0000","last_change":"2023-11-28T02:52:44.093193+0000","last_active":"2023-11-28T02:52:44.485236+0000","last_peered":"2023-11-28T02:52:44.485236+0000","last_clean":"2023-11-28T02:52:44.485236+0000","last_became_active":"2023-11-28T02:52:44.092869+0000","last_became_peered":"2023-11-28T02:52:44.092869+0000","last_unstale":"2023-11-28T02:52:44.485236+0000","last_undegraded":"2023-11-28T02:52:44.485236+0000","last_fullsized":"2023-11-28T02:52:44.485236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297348,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074492,"statfs":{"total":96636764160,"available":96332279808,"internally_reserved":0,"allocated":258048,"data_stored":86755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.747}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":86787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35399999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34999999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.441}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44900000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.432}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26400000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]}]},{"osd":4,"up_from":13,"seq":55834574851,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297340,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074500,"statfs":{"total":96636764160,"available":96332288000,"internally_reserved":0,"allocated":249856,"data_stored":86933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68799999999999994}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:52:47.784 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T02:52:47.785 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-11-28T02:52:47.785 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-11-28T02:52:47.785 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T02:52:47.786 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-11-28T02:52:47.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 Processor -- start 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 -- start start 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2bc1510c0 0x7fc2bc0a9790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc2bc063e50 0x7fc2bc0c2100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2bc0ad4a0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0ac4d0 con 0x7fc2bc1510c0 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.928+0000 7fc2c2db4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2bc0ac6e0 con 0x7fc2bc063e50 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2bc1510c0 0x7fc2bc0a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bbfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc2bc0a9c80 0x7fc2bc069070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33720/0 (socket says 172.21.15.73:33720) 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2bc1510c0 0x7fc2bc0a9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42524/0 (socket says 172.21.15.73:42524) 2023-11-28T02:52:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bbfff700 1 -- 172.21.15.73:0/3692387419 learned_addr learned my addr 172.21.15.73:0/3692387419 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c1351700 1 --2- 172.21.15.73:0/3692387419 >> v2:172.21.15.150:3300/0 conn(0x7fc2bc063e50 0x7fc2bc0c2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 -- 172.21.15.73:0/3692387419 >> v2:172.21.15.150:3300/0 conn(0x7fc2bc063e50 msgr2=0x7fc2bc0c2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/3692387419 >> v2:172.21.15.150:3300/0 conn(0x7fc2bc063e50 0x7fc2bc0c2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bb7fe700 1 -- 172.21.15.73:0/3692387419 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1204633328 0 0) 0x7fc2bc0ad4a0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 -- 172.21.15.73:0/3692387419 >> v1:172.21.15.73:6789/0 conn(0x7fc2bc0a9c80 legacy=0x7fc2bc069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 -- 172.21.15.73:0/3692387419 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2bc0ac9c0 con 0x7fc2bc1510c0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/3692387419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc0a9790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc2bc069000 tx=0x7fc2ac009780).ready entity=mon.0 client_cookie=e3a6395fdd6c2fc0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bb7fe700 1 -- 172.21.15.73:0/3692387419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac00e070 con 0x7fc2bc1510c0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bb7fe700 1 -- 172.21.15.73:0/3692387419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac00bb60 con 0x7fc2bc1510c0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.929+0000 7fc2bb7fe700 1 -- 172.21.15.73:0/3692387419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac012410 con 0x7fc2bc1510c0 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 -- 172.21.15.73:0/3692387419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc1510c0 msgr2=0x7fc2bc0a9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/3692387419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc0a9790 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fc2bc069000 tx=0x7fc2ac009780).stop 2023-11-28T02:52:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 -- 172.21.15.73:0/3692387419 shutdown_connections 2023-11-28T02:52:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/3692387419 >> v2:172.21.15.150:3300/0 conn(0x7fc2bc063e50 0x7fc2bc0c2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/3692387419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc0a9790 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 -- 172.21.15.73:0/3692387419 >> 172.21.15.73:0/3692387419 conn(0x7fc2bc0c7760 msgr2=0x7fc2bc0c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 -- 172.21.15.73:0/3692387419 shutdown_connections 2023-11-28T02:52:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.930+0000 7fc2c2db4700 1 -- 172.21.15.73:0/3692387419 wait complete. 2023-11-28T02:52:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.931+0000 7fc2c2db4700 1 Processor -- start 2023-11-28T02:52:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.931+0000 7fc2c2db4700 1 -- start start 2023-11-28T02:52:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 0x7fc2bc14ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0c5f20 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc154d60 con 0x7fc2bc1510c0 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2bc0c55f0 con 0x7fc2bc063e50 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42538/0 (socket says 172.21.15.73:42538) 2023-11-28T02:52:47.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 -- 172.21.15.73:0/2342311127 learned_addr learned my addr 172.21.15.73:0/2342311127 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:47.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c1351700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 0x7fc2bc14ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 -- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 msgr2=0x7fc2bc14ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 0x7fc2bc14ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 -- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 msgr2=0x7fc2bc156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 -- 172.21.15.73:0/2342311127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2ac009a20 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 0x7fc2bc14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:47.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c1351700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:47.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2bbfff700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc2b000c4e0 tx=0x7fc2b0007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00072e0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b0007440 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b00057a0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2bc0c58d0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.932+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2bc168d40 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.933+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2bc06c860 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.934+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc2b0005bd0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.934+0000 7fc2b8ff9700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 0x7fc2a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:47.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.934+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc2b0079ba0 con 0x7fc2bc0a9c80 2023-11-28T02:52:47.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.934+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 0x7fc2a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:47.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.935+0000 7fc2c0b50700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 0x7fc2a8061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc2bc153670 tx=0x7fc2ac009780).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:47.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:47.938+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc2b004ae10 con 0x7fc2bc0a9c80 2023-11-28T02:52:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.145+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fc2bc0acca0 con 0x7fc2bc0a9c80 2023-11-28T02:52:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.148+0000 7fc2b8ff9700 1 -- 172.21.15.73:0/2342311127 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) v1 ==== 74+0+148949 (secure 0 0 0) 0x7fc2b004e430 con 0x7fc2bc0a9c80 2023-11-28T02:52:48.151 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 msgr2=0x7fc2a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 0x7fc2a8061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc2bc153670 tx=0x7fc2ac009780).stop 2023-11-28T02:52:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 msgr2=0x7fc2bc154360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7fc2b000c4e0 tx=0x7fc2b0007c70).stop 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 shutdown_connections 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc2a805f150 0x7fc2a8061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2bc1510c0 0x7fc2bc156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2bc063e50 0x7fc2bc14ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 --2- 172.21.15.73:0/2342311127 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2bc0a9c80 0x7fc2bc154360 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 >> 172.21.15.73:0/2342311127 conn(0x7fc2bc0c7760 msgr2=0x7fc2bc0bd6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 shutdown_connections 2023-11-28T02:52:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.150+0000 7fc2c2db4700 1 -- 172.21.15.73:0/2342311127 wait complete. 2023-11-28T02:52:48.164 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":4,"active_gid":4107,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":90017},{"type":"v1","addr":"172.21.15.73:6833","nonce":90017}]},"active_addr":"172.21.15.73:6833/90017","active_change":"2023-11-28T02:52:31.381023+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":4111,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1072502020}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":160657230}]}]} 2023-11-28T02:52:48.167 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-11-28T02:52:48.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-11-28T02:52:48.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.279+0000 7f72ad289700 1 Processor -- start 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.279+0000 7f72ad289700 1 -- start start 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72ad289700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72a81544e0 0x7f72a81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72ad289700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a80c6e30 0x7f72a80c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72ad289700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a80c69a0 con 0x7f72a806a610 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72ad289700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a80c13a0 con 0x7f72a81544e0 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72ad289700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80c1680 con 0x7f72a80c6e30 2023-11-28T02:52:48.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72a81544e0 0x7f72a81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f72a81544e0 0x7f72a81548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42540/0 (socket says 172.21.15.73:42540) 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 -- 172.21.15.73:0/1109400117 learned_addr learned my addr 172.21.15.73:0/1109400117 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 -- 172.21.15.73:0/1109400117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a80c6e30 msgr2=0x7f72a80c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a77fe700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a80c6e30 0x7f72a80c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a80c6e30 0x7f72a80c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 -- 172.21.15.73:0/1109400117 >> v1:172.21.15.150:6789/0 conn(0x7f72a806a610 legacy=0x7f72a80c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.280+0000 7f72a6ffd700 1 -- 172.21.15.73:0/1109400117 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a80c6630 con 0x7f72a81544e0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a81544e0 0x7f72a81548c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f72a0009810 tx=0x7f72a0009060).ready entity=mon.0 client_cookie=344b6ad8b57aa6ab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72a5ffb700 1 -- 172.21.15.73:0/1109400117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0005880 con 0x7f72a81544e0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72a5ffb700 1 -- 172.21.15.73:0/1109400117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72a00059e0 con 0x7f72a81544e0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72a5ffb700 1 -- 172.21.15.73:0/1109400117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72a0005d60 con 0x7f72a81544e0 2023-11-28T02:52:48.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 -- 172.21.15.73:0/1109400117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a81544e0 msgr2=0x7f72a81548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a81544e0 0x7f72a81548c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f72a0009810 tx=0x7f72a0009060).stop 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 -- 172.21.15.73:0/1109400117 shutdown_connections 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a80c6e30 0x7f72a80c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 --2- 172.21.15.73:0/1109400117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a81544e0 0x7f72a81548c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 -- 172.21.15.73:0/1109400117 >> 172.21.15.73:0/1109400117 conn(0x7f72a8085020 msgr2=0x7f72a8094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 -- 172.21.15.73:0/1109400117 shutdown_connections 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.281+0000 7f72ad289700 1 -- 172.21.15.73:0/1109400117 wait complete. 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.282+0000 7f72ad289700 1 Processor -- start 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.282+0000 7f72ad289700 1 -- start start 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 0x7f72a80c5210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a81544e0 0x7f72a8098c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80954a0 con 0x7f72a80c6e30 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a813ba40 con 0x7f72a81544e0 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72ad289700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a80987d0 con 0x7f72a806a610 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72a67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72a67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42548/0 (socket says 172.21.15.73:42548) 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72a67fc700 1 -- 172.21.15.73:0/3301399668 learned_addr learned my addr 172.21.15.73:0/3301399668 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72a77fe700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a81544e0 0x7f72a8098c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.283+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 0x7f72a80c5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 -- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 msgr2=0x7f72a80c5210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 0x7f72a80c5210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 -- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a81544e0 msgr2=0x7f72a8098c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a81544e0 0x7f72a8098c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 -- 172.21.15.73:0/3301399668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a0009300 con 0x7f72a80c6e30 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 0x7f72a80c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72a67fc700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f729c00b840 tx=0x7f729c0092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00d600 con 0x7f72a80c6e30 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f729c00d780 con 0x7f72a80c6e30 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72a8099150 con 0x7f72a80c6e30 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f729c00db40 con 0x7f72a80c6e30 2023-11-28T02:52:48.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.284+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a8097f20 con 0x7f72a80c6e30 2023-11-28T02:52:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.285+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a8098940 con 0x7f72a80c6e30 2023-11-28T02:52:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.285+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f729c019460 con 0x7f72a80c6e30 2023-11-28T02:52:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.285+0000 7f728effd700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 0x7f7294061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.285+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f729c07a310 con 0x7f72a80c6e30 2023-11-28T02:52:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.285+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 0x7f7294061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.286+0000 7f72a6ffd700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 0x7f7294061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f72a000b3c0 tx=0x7f72a0016000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.289+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f729c04b580 con 0x7f72a80c6e30 2023-11-28T02:52:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.452+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f72a806c860 con 0x7f72a80c6e30 2023-11-28T02:52:48.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.453+0000 7f728effd700 1 -- 172.21.15.73:0/3301399668 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f729c04eba0 con 0x7f72a80c6e30 2023-11-28T02:52:48.455 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:48.455 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:48.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 msgr2=0x7f7294061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 0x7f7294061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f72a000b3c0 tx=0x7f72a0016000).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 msgr2=0x7f72a8094fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f729c00b840 tx=0x7f729c0092d0).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 shutdown_connections 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f729405f150 0x7f7294061610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72a81544e0 0x7f72a8098c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f72a806a610 0x7f72a80c5210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 --2- 172.21.15.73:0/3301399668 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72a80c6e30 0x7f72a8094fb0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 >> 172.21.15.73:0/3301399668 conn(0x7f72a8085020 msgr2=0x7f72a80637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 shutdown_connections 2023-11-28T02:52:48.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.455+0000 7f72ad289700 1 -- 172.21.15.73:0/3301399668 wait complete. 2023-11-28T02:52:48.469 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-11-28T02:52:48.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.579+0000 7fc517feb700 1 Processor -- start 2023-11-28T02:52:48.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.580+0000 7fc517feb700 1 -- start start 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.580+0000 7fc517feb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5101544e0 0x7fc5101548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.580+0000 7fc517feb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc5100c6e30 0x7fc5100c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc517feb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5100c69a0 con 0x7fc51006a610 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc517feb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100c13a0 con 0x7fc5101544e0 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc517feb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5100c1680 con 0x7fc5100c6e30 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5101544e0 0x7fc5101548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515586700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc51006a610 0x7fc5100c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33730/0 (socket says 172.21.15.73:33730) 2023-11-28T02:52:48.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515586700 1 -- 172.21.15.73:0/1106607404 learned_addr learned my addr 172.21.15.73:0/1106607404 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc516588700 1 --2- 172.21.15.73:0/1106607404 >> v2:172.21.15.150:3300/0 conn(0x7fc5100c6e30 0x7fc5100c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 -- 172.21.15.73:0/1106607404 >> v2:172.21.15.150:3300/0 conn(0x7fc5100c6e30 msgr2=0x7fc5100c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:48.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 --2- 172.21.15.73:0/1106607404 >> v2:172.21.15.150:3300/0 conn(0x7fc5100c6e30 0x7fc5100c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 -- 172.21.15.73:0/1106607404 >> v1:172.21.15.73:6789/0 conn(0x7fc51006a610 legacy=0x7fc5100c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc514d85700 1 -- 172.21.15.73:0/1106607404 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2308694870 0 0) 0x7fc5100c69a0 con 0x7fc51006a610 2023-11-28T02:52:48.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 -- 172.21.15.73:0/1106607404 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5100c6630 con 0x7fc5101544e0 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc515d87700 1 --2- 172.21.15.73:0/1106607404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5101544e0 0x7fc5101548c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fc5100cb240 tx=0x7fc500009780).ready entity=mon.0 client_cookie=9aca6aaf531d0f8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc514d85700 1 -- 172.21.15.73:0/1106607404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50000e070 con 0x7fc5101544e0 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc514d85700 1 -- 172.21.15.73:0/1106607404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50000bc30 con 0x7fc5101544e0 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.581+0000 7fc514d85700 1 -- 172.21.15.73:0/1106607404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc500012410 con 0x7fc5101544e0 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 -- 172.21.15.73:0/1106607404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5101544e0 msgr2=0x7fc5101548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 --2- 172.21.15.73:0/1106607404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5101544e0 0x7fc5101548c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fc5100cb240 tx=0x7fc500009780).stop 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 -- 172.21.15.73:0/1106607404 shutdown_connections 2023-11-28T02:52:48.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 --2- 172.21.15.73:0/1106607404 >> v2:172.21.15.150:3300/0 conn(0x7fc5100c6e30 0x7fc5100c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 --2- 172.21.15.73:0/1106607404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5101544e0 0x7fc5101548c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 -- 172.21.15.73:0/1106607404 >> 172.21.15.73:0/1106607404 conn(0x7fc510085020 msgr2=0x7fc510094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 -- 172.21.15.73:0/1106607404 shutdown_connections 2023-11-28T02:52:48.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.582+0000 7fc517feb700 1 -- 172.21.15.73:0/1106607404 wait complete. 2023-11-28T02:52:48.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.583+0000 7fc517feb700 1 Processor -- start 2023-11-28T02:52:48.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.583+0000 7fc517feb700 1 -- start start 2023-11-28T02:52:48.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 0x7fc5100952e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5101544e0 0x7fc5100957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc510097890 con 0x7fc51006a610 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc510097b70 con 0x7fc5100c6e30 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc510094ae0 con 0x7fc5101544e0 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42558/0 (socket says 172.21.15.73:42558) 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 -- 172.21.15.73:0/3144677970 learned_addr learned my addr 172.21.15.73:0/3144677970 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515586700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 0x7fc5100952e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc516588700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5101544e0 0x7fc5100957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 -- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5101544e0 msgr2=0x7fc5100957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5101544e0 0x7fc5100957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 -- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 msgr2=0x7fc5100952e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 0x7fc5100952e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 -- 172.21.15.73:0/3144677970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc500009a20 con 0x7fc51006a610 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515586700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 0x7fc5100952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:48.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc515d87700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fc51005fb30 tx=0x7fc500009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50001d630 con 0x7fc51006a610 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc50001d790 con 0x7fc51006a610 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc50001db30 con 0x7fc51006a610 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.584+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc510094dc0 con 0x7fc51006a610 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.585+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5100af690 con 0x7fc51006a610 2023-11-28T02:52:48.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.586+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc510097cb0 con 0x7fc51006a610 2023-11-28T02:52:48.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.586+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc500026410 con 0x7fc51006a610 2023-11-28T02:52:48.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.586+0000 7fc50dffb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 0x7fc4fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.586+0000 7fc515586700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 0x7fc4fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.586+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc50007bfb0 con 0x7fc51006a610 2023-11-28T02:52:48.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.587+0000 7fc515586700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 0x7fc4fc061610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc510095df0 tx=0x7fc504009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.590+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc5000aaaf0 con 0x7fc51006a610 2023-11-28T02:52:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.754+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc510094940 con 0x7fc51006a610 2023-11-28T02:52:48.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.754+0000 7fc50dffb700 1 -- 172.21.15.73:0/3144677970 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fc510094940 con 0x7fc51006a610 2023-11-28T02:52:48.756 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:48.756 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:48.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 msgr2=0x7fc4fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 0x7fc4fc061610 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7fc510095df0 tx=0x7fc504009040).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 msgr2=0x7fc510153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fc51005fb30 tx=0x7fc500009780).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 shutdown_connections 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4fc05f150 0x7fc4fc061610 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5101544e0 0x7fc5100957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc51006a610 0x7fc510153370 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 --2- 172.21.15.73:0/3144677970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5100c6e30 0x7fc5100952e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 >> 172.21.15.73:0/3144677970 conn(0x7fc510085020 msgr2=0x7fc5100ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 shutdown_connections 2023-11-28T02:52:48.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.756+0000 7fc517feb700 1 -- 172.21.15.73:0/3144677970 wait complete. 2023-11-28T02:52:48.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-11-28T02:52:48.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.882+0000 7f8d5095f700 1 Processor -- start 2023-11-28T02:52:48.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.882+0000 7f8d5095f700 1 -- start start 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d5095f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c0c6190 0x7f8d4c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d5095f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d4c158320 0x7f8d4c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d5095f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d4c158cc0 con 0x7f8d4c098d10 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d5095f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d4c0c72d0 con 0x7f8d4c158320 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d5095f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c064570 con 0x7f8d4c0c6190 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d4ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d4c158320 0x7f8d4c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d4a59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8d4c098d10 0x7f8d4c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33738/0 (socket says 172.21.15.73:33738) 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d4a59c700 1 -- 172.21.15.73:0/790593615 learned_addr learned my addr 172.21.15.73:0/790593615 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.883+0000 7f8d49d9b700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c0c6190 0x7f8d4c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4ad9d700 1 -- 172.21.15.73:0/790593615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c0c6190 msgr2=0x7f8d4c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4ad9d700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c0c6190 0x7f8d4c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4ad9d700 1 -- 172.21.15.73:0/790593615 >> v1:172.21.15.73:6789/0 conn(0x7f8d4c098d10 legacy=0x7f8d4c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4ad9d700 1 -- 172.21.15.73:0/790593615 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d4c154bc0 con 0x7f8d4c158320 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4959a700 1 -- 172.21.15.73:0/790593615 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3360683043 0 0) 0x7f8d3c001490 con 0x7f8d4c098d10 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4ad9d700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c158320 0x7f8d4c1587d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8d44009760 tx=0x7f8d44009060).ready entity=mon.0 client_cookie=76d16f45047c3b11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4959a700 1 -- 172.21.15.73:0/790593615 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400b580 con 0x7f8d4c158320 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4959a700 1 -- 172.21.15.73:0/790593615 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d4400b6e0 con 0x7f8d4c158320 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d4959a700 1 -- 172.21.15.73:0/790593615 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d4400ba60 con 0x7f8d4c158320 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 -- 172.21.15.73:0/790593615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c158320 msgr2=0x7f8d4c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c158320 0x7f8d4c1587d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f8d44009760 tx=0x7f8d44009060).stop 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 -- 172.21.15.73:0/790593615 shutdown_connections 2023-11-28T02:52:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c158320 0x7f8d4c1587d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 --2- 172.21.15.73:0/790593615 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c0c6190 0x7f8d4c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 -- 172.21.15.73:0/790593615 >> 172.21.15.73:0/790593615 conn(0x7f8d4c0cb310 msgr2=0x7f8d4c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 -- 172.21.15.73:0/790593615 shutdown_connections 2023-11-28T02:52:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.884+0000 7f8d5095f700 1 -- 172.21.15.73:0/790593615 wait complete. 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.885+0000 7f8d5095f700 1 Processor -- start 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 -- start start 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d4c098d10 0x7f8d4c155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 0x7f8d4c14d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c158320 0x7f8d4c1532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c0aa0f0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c155c50 con 0x7f8d4c098d10 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d5095f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d4c1565f0 con 0x7f8d4c158320 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d49d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 0x7f8d4c14d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d49d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 0x7f8d4c14d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42562/0 (socket says 172.21.15.73:42562) 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d49d9b700 1 -- 172.21.15.73:0/3074779411 learned_addr learned my addr 172.21.15.73:0/3074779411 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.886+0000 7f8d4a59c700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d4c098d10 0x7f8d4c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d4ad9d700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c158320 0x7f8d4c1532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 -- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c158320 msgr2=0x7f8d4c1532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d4c158320 0x7f8d4c1532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 -- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d4c098d10 msgr2=0x7f8d4c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d4c098d10 0x7f8d4c155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d440092a0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d49d9b700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 0x7f8d4c14d320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8d3400d8c0 tx=0x7f8d34008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34011590 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d340116f0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d4c1568d0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d34011a70 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.887+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d4c157170 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.888+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8d4c156cc0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.888+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8d3401c460 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.888+0000 7f8d427fc700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8d2c05f150 0x7f8d2c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.888+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8d34079ed0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.888+0000 7f8d4a59c700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8d2c05f150 0x7f8d2c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d427fc700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7f8d2c0684f0 0x7f8d2c06a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8d2c06afd0 con 0x7f8d2c0684f0 2023-11-28T02:52:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8d3407c4f0 con 0x7f8d4c0c6190 2023-11-28T02:52:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d4ad9d700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7f8d2c0684f0 0x7f8d2c06a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d4a59c700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8d2c05f150 0x7f8d2c061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f8d3c007fd0 tx=0x7f8d3c007d90).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.889+0000 7f8d4ad9d700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7f8d2c0684f0 0x7f8d2c06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.890+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== osd.0 v2:172.21.15.73:6800/90247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f8d2c06afd0 con 0x7f8d2c0684f0 2023-11-28T02:52:48.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 --> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8d4c06c860 con 0x7f8d2c0684f0 2023-11-28T02:52:48.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d427fc700 1 -- 172.21.15.73:0/3074779411 <== osd.0 v2:172.21.15.73:6800/90247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8d4c06c860 con 0x7f8d2c0684f0 2023-11-28T02:52:48.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7f8d2c0684f0 msgr2=0x7f8d2c06a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7f8d2c0684f0 0x7f8d2c06a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8d2c05f150 msgr2=0x7f8d2c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8d2c05f150 0x7f8d2c061610 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f8d3c007fd0 tx=0x7f8d3c007d90).stop 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 msgr2=0x7f8d4c14d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.904+0000 7f8d5095f700 1 --2- 172.21.15.73:0/3074779411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d4c0c6190 0x7f8d4c14d320 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f8d3400d8c0 tx=0x7f8d34008040).stop 2023-11-28T02:52:48.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.905+0000 7f8d4ad9d700 1 -- 172.21.15.73:0/3074779411 reap_dead start 2023-11-28T02:52:48.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.905+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 shutdown_connections 2023-11-28T02:52:48.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.905+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 >> 172.21.15.73:0/3074779411 conn(0x7f8d4c0cb310 msgr2=0x7f8d4c0c0780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:48.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.905+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 shutdown_connections 2023-11-28T02:52:48.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:48.905+0000 7f8d5095f700 1 -- 172.21.15.73:0/3074779411 wait complete. 2023-11-28T02:52:48.917 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:48.918 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-11-28T02:52:49.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 Processor -- start 2023-11-28T02:52:49.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 -- start start 2023-11-28T02:52:49.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcd68066a30 0x7fcd68066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd680abf50 0x7fcd6813b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd680c75e0 con 0x7fcd680ac3d0 2023-11-28T02:52:49.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680c6d20 con 0x7fcd680abf50 2023-11-28T02:52:49.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.030+0000 7fcd6cbb6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd680c6f90 con 0x7fcd68066a30 2023-11-28T02:52:49.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd680abf50 0x7fcd6813b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcd680abf50 0x7fcd6813b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42578/0 (socket says 172.21.15.73:42578) 2023-11-28T02:52:49.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 -- 172.21.15.73:0/421308043 learned_addr learned my addr 172.21.15.73:0/421308043 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd677fe700 1 --2- 172.21.15.73:0/421308043 >> v2:172.21.15.150:3300/0 conn(0x7fcd68066a30 0x7fcd68066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 -- 172.21.15.73:0/421308043 >> v2:172.21.15.150:3300/0 conn(0x7fcd68066a30 msgr2=0x7fcd68066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 --2- 172.21.15.73:0/421308043 >> v2:172.21.15.150:3300/0 conn(0x7fcd68066a30 0x7fcd68066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 -- 172.21.15.73:0/421308043 >> v1:172.21.15.150:6789/0 conn(0x7fcd680ac3d0 legacy=0x7fcd6813b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 -- 172.21.15.73:0/421308043 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd680c7270 con 0x7fcd680abf50 2023-11-28T02:52:49.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd66ffd700 1 --2- 172.21.15.73:0/421308043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813b430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fcd680c6980 tx=0x7fcd500090f0).ready entity=mon.0 client_cookie=eb8e53f3ec1de44a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd667fc700 1 -- 172.21.15.73:0/421308043 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000b5a0 con 0x7fcd680abf50 2023-11-28T02:52:49.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd667fc700 1 -- 172.21.15.73:0/421308043 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b700 con 0x7fcd680abf50 2023-11-28T02:52:49.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.031+0000 7fcd667fc700 1 -- 172.21.15.73:0/421308043 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000ba80 con 0x7fcd680abf50 2023-11-28T02:52:49.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/421308043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 msgr2=0x7fcd6813b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/421308043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813b430 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7fcd680c6980 tx=0x7fcd500090f0).stop 2023-11-28T02:52:49.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/421308043 shutdown_connections 2023-11-28T02:52:49.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/421308043 >> v2:172.21.15.150:3300/0 conn(0x7fcd68066a30 0x7fcd68066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/421308043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813b430 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/421308043 >> 172.21.15.73:0/421308043 conn(0x7fcd680c8ed0 msgr2=0x7fcd680c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/421308043 shutdown_connections 2023-11-28T02:52:49.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.032+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/421308043 wait complete. 2023-11-28T02:52:49.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.033+0000 7fcd6cbb6700 1 Processor -- start 2023-11-28T02:52:49.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.033+0000 7fcd6cbb6700 1 -- start start 2023-11-28T02:52:49.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd68066a30 0x7fcd681562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd680ac3d0 0x7fcd6813a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd680c07d0 con 0x7fcd680abf50 2023-11-28T02:52:49.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6813ad10 con 0x7fcd68066a30 2023-11-28T02:52:49.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd681567e0 con 0x7fcd680ac3d0 2023-11-28T02:52:49.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42584/0 (socket says 172.21.15.73:42584) 2023-11-28T02:52:49.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 -- 172.21.15.73:0/2138677691 learned_addr learned my addr 172.21.15.73:0/2138677691 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd677fe700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd68066a30 0x7fcd681562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd67fff700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd680ac3d0 0x7fcd6813a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 -- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd680ac3d0 msgr2=0x7fcd6813a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcd680ac3d0 0x7fcd6813a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 -- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd68066a30 msgr2=0x7fcd681562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcd68066a30 0x7fcd681562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd50009370 con 0x7fcd680abf50 2023-11-28T02:52:49.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd66ffd700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813a330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fcd50009130 tx=0x7fcd50015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5000b870 con 0x7fcd680abf50 2023-11-28T02:52:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5000b9d0 con 0x7fcd680abf50 2023-11-28T02:52:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd680aaa00 con 0x7fcd680abf50 2023-11-28T02:52:49.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5001c670 con 0x7fcd680abf50 2023-11-28T02:52:49.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.034+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd680c03f0 con 0x7fcd680abf50 2023-11-28T02:52:49.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.035+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fcd680bff70 con 0x7fcd680abf50 2023-11-28T02:52:49.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcd5001c9e0 con 0x7fcd680abf50 2023-11-28T02:52:49.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcd4805f150 0x7fcd48061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcd5007b910 con 0x7fcd680abf50 2023-11-28T02:52:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd677fe700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcd4805f150 0x7fcd48061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7fcd480684f0 0x7fcd4806a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fcd4806afd0 con 0x7fcd480684f0 2023-11-28T02:52:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd50080220 con 0x7fcd680abf50 2023-11-28T02:52:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd67fff700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7fcd480684f0 0x7fcd4806a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd677fe700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcd4805f150 0x7fcd48061610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fcd60009fd0 tx=0x7fcd60009880).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.036+0000 7fcd67fff700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7fcd480684f0 0x7fcd4806a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.037+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== osd.1 v2:172.21.15.73:6824/90262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fcd4806afd0 con 0x7fcd480684f0 2023-11-28T02:52:49.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.051+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 --> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fcd680c0910 con 0x7fcd480684f0 2023-11-28T02:52:49.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.051+0000 7fcd5f7fe700 1 -- 172.21.15.73:0/2138677691 <== osd.1 v2:172.21.15.73:6824/90262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fcd680c0910 con 0x7fcd480684f0 2023-11-28T02:52:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7fcd480684f0 msgr2=0x7fcd4806a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7fcd480684f0 0x7fcd4806a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcd4805f150 msgr2=0x7fcd48061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcd4805f150 0x7fcd48061610 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7fcd60009fd0 tx=0x7fcd60009880).stop 2023-11-28T02:52:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 msgr2=0x7fcd6813a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 --2- 172.21.15.73:0/2138677691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcd680abf50 0x7fcd6813a330 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7fcd50009130 tx=0x7fcd50015000).stop 2023-11-28T02:52:49.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd67fff700 1 -- 172.21.15.73:0/2138677691 reap_dead start 2023-11-28T02:52:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 shutdown_connections 2023-11-28T02:52:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 >> 172.21.15.73:0/2138677691 conn(0x7fcd680c8ed0 msgr2=0x7fcd680ac7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 shutdown_connections 2023-11-28T02:52:49.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.052+0000 7fcd6cbb6700 1 -- 172.21.15.73:0/2138677691 wait complete. 2023-11-28T02:52:49.064 INFO:teuthology.orchestra.run.smithi073.stdout:55834574852 2023-11-28T02:52:49.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-11-28T02:52:49.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.177+0000 7f91bf24a700 1 Processor -- start 2023-11-28T02:52:49.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 -- start start 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91b8069d10 0x7f91b806a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91b80c3470 con 0x7f91b8151a20 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91b80c48f0 con 0x7f91b8069d10 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.178+0000 7f91bf24a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b80bdb50 con 0x7f91b80bd1f0 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91b8069d10 0x7f91b806a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f91b8151a20 0x7f91b80be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33752/0 (socket says 172.21.15.73:33752) 2023-11-28T02:52:49.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b7fff700 1 -- 172.21.15.73:0/2052122148 learned_addr learned my addr 172.21.15.73:0/2052122148 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bd7e7700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 -- 172.21.15.73:0/2052122148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 msgr2=0x7f91b80bd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80bd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 -- 172.21.15.73:0/2052122148 >> v1:172.21.15.73:6789/0 conn(0x7f91b8151a20 legacy=0x7f91b80be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b77fe700 1 -- 172.21.15.73:0/2052122148 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3354294367 0 0) 0x7f91b80c3470 con 0x7f91b8151a20 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 -- 172.21.15.73:0/2052122148 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b8151ea0 con 0x7f91b8069d10 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91bcfe6700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b806a0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f91a8009e60 tx=0x7f91a8009780).ready entity=mon.0 client_cookie=3e4657077edec1d8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b77fe700 1 -- 172.21.15.73:0/2052122148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a800e070 con 0x7f91b8069d10 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b77fe700 1 -- 172.21.15.73:0/2052122148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a800bc30 con 0x7f91b8069d10 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.179+0000 7f91b77fe700 1 -- 172.21.15.73:0/2052122148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a8012410 con 0x7f91b8069d10 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 -- 172.21.15.73:0/2052122148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 msgr2=0x7f91b806a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b806a0f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f91a8009e60 tx=0x7f91a8009780).stop 2023-11-28T02:52:49.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 -- 172.21.15.73:0/2052122148 shutdown_connections 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 --2- 172.21.15.73:0/2052122148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b806a0f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 -- 172.21.15.73:0/2052122148 >> 172.21.15.73:0/2052122148 conn(0x7f91b80642d0 msgr2=0x7f91b80c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 -- 172.21.15.73:0/2052122148 shutdown_connections 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.180+0000 7f91bf24a700 1 -- 172.21.15.73:0/2052122148 wait complete. 2023-11-28T02:52:49.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.181+0000 7f91bf24a700 1 Processor -- start 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.181+0000 7f91bf24a700 1 -- start start 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b814ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80a7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b8151a20 0x7f91b80a8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8065e70 con 0x7f91b8069d10 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8066c70 con 0x7f91b8151a20 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91b8066060 con 0x7f91b80bd1f0 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b814ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b814ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42594/0 (socket says 172.21.15.73:42594) 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 -- 172.21.15.73:0/1050170688 learned_addr learned my addr 172.21.15.73:0/1050170688 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bd7e7700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b8151a20 0x7f91b80a8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91b7fff700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80a7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 -- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 msgr2=0x7f91b80a7e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91b80bd1f0 0x7f91b80a7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 -- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b8151a20 msgr2=0x7f91b80a8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b8151a20 0x7f91b80a8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a8009a20 con 0x7f91b8069d10 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bd7e7700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91b8151a20 0x7f91b80a8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bcfe6700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b814ed40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f91a800b6c0 tx=0x7f91a8009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a801d630 con 0x7f91b8069d10 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91a801d790 con 0x7f91b8069d10 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91a801db10 con 0x7f91b8069d10 2023-11-28T02:52:49.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91b8066340 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.182+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91b8066900 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.183+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f91b806c860 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.183+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f91a8003040 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b4ff9700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a405f150 0x7f91a4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f91a807bf40 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b7fff700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a405f150 0x7f91a4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b4ff9700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7f91a40684f0 0x7f91a406a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f91a406afd0 con 0x7f91a40684f0 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f91a807e560 con 0x7f91b8069d10 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91bd7e7700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7f91a40684f0 0x7f91a406a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91b7fff700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a405f150 0x7f91a4061610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f91ac007fd0 tx=0x7f91ac009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.184+0000 7f91bd7e7700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7f91a40684f0 0x7f91a406a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.185+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== osd.2 v2:172.21.15.73:6808/90248 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f91a406afd0 con 0x7f91a40684f0 2023-11-28T02:52:49.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.199+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 --> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f91b8065290 con 0x7f91a40684f0 2023-11-28T02:52:49.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.199+0000 7f91b4ff9700 1 -- 172.21.15.73:0/1050170688 <== osd.2 v2:172.21.15.73:6808/90248 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f91b8065290 con 0x7f91a40684f0 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7f91a40684f0 msgr2=0x7f91a406a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7f91a40684f0 0x7f91a406a950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a405f150 msgr2=0x7f91a4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a405f150 0x7f91a4061610 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f91ac007fd0 tx=0x7f91ac009040).stop 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 msgr2=0x7f91b814ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 --2- 172.21.15.73:0/1050170688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91b8069d10 0x7f91b814ed40 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f91a800b6c0 tx=0x7f91a8009780).stop 2023-11-28T02:52:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bd7e7700 1 -- 172.21.15.73:0/1050170688 reap_dead start 2023-11-28T02:52:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 shutdown_connections 2023-11-28T02:52:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 >> 172.21.15.73:0/1050170688 conn(0x7f91b80642d0 msgr2=0x7f91b80c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 shutdown_connections 2023-11-28T02:52:49.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.200+0000 7f91bf24a700 1 -- 172.21.15.73:0/1050170688 wait complete. 2023-11-28T02:52:49.212 INFO:teuthology.orchestra.run.smithi073.stdout:55834574854 2023-11-28T02:52:49.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-11-28T02:52:49.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 Processor -- start 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 -- start start 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f49540c6190 0x7f49540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49540c72d0 con 0x7f4954158320 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4954064570 con 0x7f4954098d10 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.325+0000 7f495b071700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4954154bc0 con 0x7f49540c6190 2023-11-28T02:52:49.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f495960e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4954158320 0x7f49540a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33766/0 (socket says 172.21.15.73:33766) 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f495960e700 1 -- 172.21.15.73:0/1658943522 learned_addr learned my addr 172.21.15.73:0/1658943522 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f4953fff700 1 --2- 172.21.15.73:0/1658943522 >> v2:172.21.15.150:3300/0 conn(0x7f49540c6190 0x7f49540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4289489867 0 0) 0x7f49540c72d0 con 0x7f4954158320 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f493c0033f0 con 0x7f4954158320 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f4953fff700 1 -- 172.21.15.73:0/1658943522 >> v1:172.21.15.150:6789/0 conn(0x7f4954098d10 legacy=0x7f4954063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2249370766 0 0) 0x7f4954064570 con 0x7f4954098d10 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f4953fff700 1 -- 172.21.15.73:0/1658943522 >> v1:172.21.15.73:6789/0 conn(0x7f4954158320 legacy=0x7f49540a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f4953fff700 1 -- 172.21.15.73:0/1658943522 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49540c66c0 con 0x7f49540c6190 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f4953fff700 1 --2- 172.21.15.73:0/1658943522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49540c6190 0x7f49540c7760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f4948009760 tx=0x7f4948009060).ready entity=mon.1 client_cookie=db69ac5cffc2db6f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800f510 con 0x7f49540c6190 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494800f670 con 0x7f49540c6190 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.326+0000 7f49537fe700 1 -- 172.21.15.73:0/1658943522 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f494800f9f0 con 0x7f49540c6190 2023-11-28T02:52:49.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 -- 172.21.15.73:0/1658943522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49540c6190 msgr2=0x7f49540c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 --2- 172.21.15.73:0/1658943522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49540c6190 0x7f49540c7760 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f4948009760 tx=0x7f4948009060).stop 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 -- 172.21.15.73:0/1658943522 shutdown_connections 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 --2- 172.21.15.73:0/1658943522 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f49540c6190 0x7f49540c7760 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 -- 172.21.15.73:0/1658943522 >> 172.21.15.73:0/1658943522 conn(0x7f49540cb310 msgr2=0x7f49540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 -- 172.21.15.73:0/1658943522 shutdown_connections 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.327+0000 7f495b071700 1 -- 172.21.15.73:0/1658943522 wait complete. 2023-11-28T02:52:49.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.328+0000 7f495b071700 1 Processor -- start 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.328+0000 7f495b071700 1 -- start start 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4954098d10 0x7f4954148940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 0x7f4954156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4954158320 0x7f4954156740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954156d30 con 0x7f49540c6190 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4954157010 con 0x7f4954158320 2023-11-28T02:52:49.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49541572f0 con 0x7f4954098d10 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 0x7f4954156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 0x7f4954156250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42596/0 (socket says 172.21.15.73:42596) 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 -- 172.21.15.73:0/812424335 learned_addr learned my addr 172.21.15.73:0/812424335 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4958e0d700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4954098d10 0x7f4954148940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495960e700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4954158320 0x7f4954156740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 -- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4954098d10 msgr2=0x7f4954148940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4954098d10 0x7f4954148940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 -- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4954158320 msgr2=0x7f4954156740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4954158320 0x7f4954156740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49480092a0 con 0x7f49540c6190 2023-11-28T02:52:49.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4953fff700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 0x7f4954156250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f49540c2360 tx=0x7f4948017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49480115d0 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f494801e070 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4948011980 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.329+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49541575d0 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.330+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4954148e30 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.330+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4954157180 con 0x7f49540c6190 2023-11-28T02:52:49.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4948020030 con 0x7f49540c6190 2023-11-28T02:52:49.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494005f150 0x7f4940061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f494807b380 con 0x7f49540c6190 2023-11-28T02:52:49.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4958e0d700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494005f150 0x7f4940061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f49400684f0 0x7f494006a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f494006afd0 con 0x7f49400684f0 2023-11-28T02:52:49.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f494807d9a0 con 0x7f49540c6190 2023-11-28T02:52:49.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f495960e700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f49400684f0 0x7f494006a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.331+0000 7f4958e0d700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494005f150 0x7f4940061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4944001560 tx=0x7f4944009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.332+0000 7f495960e700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f49400684f0 0x7f494006a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.332+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== osd.3 v2:172.21.15.73:6816/90251 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f494006afd0 con 0x7f49400684f0 2023-11-28T02:52:49.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.346+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 --> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4954157af0 con 0x7f49400684f0 2023-11-28T02:52:49.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f4950ff9700 1 -- 172.21.15.73:0/812424335 <== osd.3 v2:172.21.15.73:6816/90251 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4954157af0 con 0x7f49400684f0 2023-11-28T02:52:49.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f49400684f0 msgr2=0x7f494006a950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f49400684f0 0x7f494006a950 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494005f150 msgr2=0x7f4940061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494005f150 0x7f4940061610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4944001560 tx=0x7f4944009040).stop 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 msgr2=0x7f4954156250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 --2- 172.21.15.73:0/812424335 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49540c6190 0x7f4954156250 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f49540c2360 tx=0x7f4948017000).stop 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495960e700 1 -- 172.21.15.73:0/812424335 reap_dead start 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 shutdown_connections 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 >> 172.21.15.73:0/812424335 conn(0x7f49540cb310 msgr2=0x7f49540ca630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 shutdown_connections 2023-11-28T02:52:49.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.347+0000 7f495b071700 1 -- 172.21.15.73:0/812424335 wait complete. 2023-11-28T02:52:49.377 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:49.377 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-11-28T02:52:49.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.491+0000 7fe78f16a700 1 Processor -- start 2023-11-28T02:52:49.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.491+0000 7fe78f16a700 1 -- start start 2023-11-28T02:52:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78f16a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78814e530 0x7fe7880c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78f16a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe788154950 0x7fe78814e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78f16a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7880c7bc0 con 0x7fe788158570 2023-11-28T02:52:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78f16a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe788155920 con 0x7fe788154950 2023-11-28T02:52:49.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78f16a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe788154dd0 con 0x7fe78814e530 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe788154950 0x7fe78814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78e969700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe788158570 0x7fe7880aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33768/0 (socket says 172.21.15.73:33768) 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78e969700 1 -- 172.21.15.73:0/3154911480 learned_addr learned my addr 172.21.15.73:0/3154911480 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78e168700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78814e530 0x7fe7880c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 -- 172.21.15.73:0/3154911480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78814e530 msgr2=0x7fe7880c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78814e530 0x7fe7880c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 -- 172.21.15.73:0/3154911480 >> v1:172.21.15.73:6789/0 conn(0x7fe788158570 legacy=0x7fe7880aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d166700 1 -- 172.21.15.73:0/3154911480 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3790723153 0 0) 0x7fe7880c7bc0 con 0x7fe788158570 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 -- 172.21.15.73:0/3154911480 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788155080 con 0x7fe788154950 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d967700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe788154950 0x7fe78814e910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe778009760 tx=0x7fe778009060).ready entity=mon.0 client_cookie=cc5d0117aa74b968 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d166700 1 -- 172.21.15.73:0/3154911480 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800f510 con 0x7fe788154950 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d166700 1 -- 172.21.15.73:0/3154911480 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe77800f670 con 0x7fe788154950 2023-11-28T02:52:49.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.492+0000 7fe78d166700 1 -- 172.21.15.73:0/3154911480 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe77800f9f0 con 0x7fe788154950 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 -- 172.21.15.73:0/3154911480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe788154950 msgr2=0x7fe78814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe788154950 0x7fe78814e910 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fe778009760 tx=0x7fe778009060).stop 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 -- 172.21.15.73:0/3154911480 shutdown_connections 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe78814e530 0x7fe7880c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 --2- 172.21.15.73:0/3154911480 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe788154950 0x7fe78814e910 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 -- 172.21.15.73:0/3154911480 >> 172.21.15.73:0/3154911480 conn(0x7fe788064990 msgr2=0x7fe788064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 -- 172.21.15.73:0/3154911480 shutdown_connections 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.493+0000 7fe78f16a700 1 -- 172.21.15.73:0/3154911480 wait complete. 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.494+0000 7fe78f16a700 1 Processor -- start 2023-11-28T02:52:49.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.494+0000 7fe78f16a700 1 -- start start 2023-11-28T02:52:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 0x7fe7881535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe788154950 0x7fe7880c3330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe788158570 0x7fe7880c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880c4f40 con 0x7fe78814e530 2023-11-28T02:52:49.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880c4330 con 0x7fe788154950 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78f16a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7880c4610 con 0x7fe788158570 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 0x7fe7881535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 0x7fe7881535e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42614/0 (socket says 172.21.15.73:42614) 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 -- 172.21.15.73:0/2164941188 learned_addr learned my addr 172.21.15.73:0/2164941188 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e969700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe788158570 0x7fe7880c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78d967700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe788154950 0x7fe7880c3330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 -- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe788158570 msgr2=0x7fe7880c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe788158570 0x7fe7880c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 -- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe788154950 msgr2=0x7fe7880c3330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe788154950 0x7fe7880c3330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7780092a0 con 0x7fe78814e530 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe78e168700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 0x7fe7881535e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe78400e930 tx=0x7fe784009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.495+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400eed0 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.496+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe78400c850 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.496+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7880c48f0 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.496+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe78400cbd0 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.496+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788153ad0 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.496+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe788062ee0 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe78401b400 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe76c05f150 0x7fe76c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe78407ae90 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe78d967700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe76c05f150 0x7fe76c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7fe76c068380 0x7fe76c06a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe76c06ae60 con 0x7fe76c068380 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.497+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe78407b180 con 0x7fe78814e530 2023-11-28T02:52:49.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.498+0000 7fe78d967700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe76c05f150 0x7fe76c061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe778009760 tx=0x7fe778017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.498+0000 7fe78e969700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7fe76c068380 0x7fe76c06a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.498+0000 7fe78e969700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7fe76c068380 0x7fe76c06a7e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.499+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== osd.4 v2:172.21.15.150:6808/90482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fe76c06ae60 con 0x7fe76c068380 2023-11-28T02:52:49.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.513+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 --> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe7880c5080 con 0x7fe76c068380 2023-11-28T02:52:49.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe77e7fc700 1 -- 172.21.15.73:0/2164941188 <== osd.4 v2:172.21.15.150:6808/90482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe780001d90 con 0x7fe76c068380 2023-11-28T02:52:49.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7fe76c068380 msgr2=0x7fe76c06a7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7fe76c068380 0x7fe76c06a7e0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe76c05f150 msgr2=0x7fe76c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe76c05f150 0x7fe76c061610 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7fe778009760 tx=0x7fe778017000).stop 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 msgr2=0x7fe7881535e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 --2- 172.21.15.73:0/2164941188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe78814e530 0x7fe7881535e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fe78400e930 tx=0x7fe784009060).stop 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78e969700 1 -- 172.21.15.73:0/2164941188 reap_dead start 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 shutdown_connections 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 >> 172.21.15.73:0/2164941188 conn(0x7fe788064990 msgr2=0x7fe7880c9530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 shutdown_connections 2023-11-28T02:52:49.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.514+0000 7fe78f16a700 1 -- 172.21.15.73:0/2164941188 wait complete. 2023-11-28T02:52:49.527 INFO:teuthology.orchestra.run.smithi073.stdout:55834574852 2023-11-28T02:52:49.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.640+0000 7f93acbdb700 1 Processor -- start 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.640+0000 7f93acbdb700 1 -- start start 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.641+0000 7f93acbdb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 0x7f93a80c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.641+0000 7f93acbdb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.641+0000 7f93acbdb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93a80630a0 con 0x7f93a80c8780 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.641+0000 7f93acbdb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93a80c7500 con 0x7f93a813be20 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.641+0000 7f93acbdb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a80bd650 con 0x7f93a8062700 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a659c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 0x7f93a80c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:47246/0 (socket says 172.21.15.73:47246) 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a5d9b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f93a80c8780 0x7f93a80bcf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59930/0 (socket says 172.21.15.73:59930) 2023-11-28T02:52:49.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a659c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 0x7f93a80c7990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:38114/0 (socket says 172.21.15.73:38114) 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 -- 172.21.15.73:0/501205683 learned_addr learned my addr 172.21.15.73:0/501205683 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 -- 172.21.15.73:0/501205683 >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 msgr2=0x7f93a80c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 --2- 172.21.15.73:0/501205683 >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 0x7f93a80c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 -- 172.21.15.73:0/501205683 >> v1:172.21.15.150:6789/0 conn(0x7f93a80c8780 legacy=0x7f93a80bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 -- 172.21.15.73:0/501205683 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a813c200 con 0x7f93a8062700 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a6d9d700 1 --2- 172.21.15.73:0/501205683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f93a0006fd0 tx=0x7f93a00090f0).ready entity=mon.2 client_cookie=bb5a6ae56cd92435 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a559a700 1 -- 172.21.15.73:0/501205683 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000b580 con 0x7f93a8062700 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a559a700 1 -- 172.21.15.73:0/501205683 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a000b6e0 con 0x7f93a8062700 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.642+0000 7f93a559a700 1 -- 172.21.15.73:0/501205683 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a000ba60 con 0x7f93a8062700 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 -- 172.21.15.73:0/501205683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 msgr2=0x7f93a8062bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 --2- 172.21.15.73:0/501205683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f93a0006fd0 tx=0x7f93a00090f0).stop 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 -- 172.21.15.73:0/501205683 shutdown_connections 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 --2- 172.21.15.73:0/501205683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a8062700 0x7f93a8062bb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 --2- 172.21.15.73:0/501205683 >> v2:172.21.15.150:3300/0 conn(0x7f93a813be20 0x7f93a80c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 -- 172.21.15.73:0/501205683 >> 172.21.15.73:0/501205683 conn(0x7f93a80642d0 msgr2=0x7f93a80c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 -- 172.21.15.73:0/501205683 shutdown_connections 2023-11-28T02:52:49.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.643+0000 7f93acbdb700 1 -- 172.21.15.73:0/501205683 wait complete. 2023-11-28T02:52:49.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.644+0000 7f93acbdb700 1 Processor -- start 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.644+0000 7f93acbdb700 1 -- start start 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 0x7f93a80c0cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93a80c8780 0x7f93a813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a813be20 0x7f93a80a8a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a8156e70 con 0x7f93a8062700 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93a813a8b0 con 0x7f93a80c8780 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93acbdb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93a81565d0 con 0x7f93a813be20 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 0x7f93a80c0cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 0x7f93a80c0cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42616/0 (socket says 172.21.15.73:42616) 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 -- 172.21.15.73:0/3889458133 learned_addr learned my addr 172.21.15.73:0/3889458133 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a5d9b700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93a80c8780 0x7f93a813a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a6d9d700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a813be20 0x7f93a80a8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 -- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a813be20 msgr2=0x7f93a80a8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93a813be20 0x7f93a80a8a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 -- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93a80c8780 msgr2=0x7f93a813a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93a80c8780 0x7f93a813a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.645+0000 7f93a659c700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a0009370 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f93a659c700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 0x7f93a80c0cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f9398009e70 tx=0x7f939800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f939800cbd0 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f939800f040 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93a81568b0 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9398013400 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93a80bf2c0 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.646+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f93a80bf120 con 0x7f93a8062700 2023-11-28T02:52:49.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.647+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f93980137b0 con 0x7f93a8062700 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.647+0000 7f939e7fc700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f938805f150 0x7f9388061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.647+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f939807ab10 con 0x7f93a8062700 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.647+0000 7f93a5d9b700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f938805f150 0x7f9388061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f939e7fc700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7f9388068470 0x7f938806a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f938806af50 con 0x7f9388068470 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f939807f220 con 0x7f93a8062700 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f93a5d9b700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f938805f150 0x7f9388061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f93a8066700 tx=0x7f9390009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f93a6d9d700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7f9388068470 0x7f938806a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.648+0000 7f93a6d9d700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7f9388068470 0x7f938806a8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.649+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== osd.5 v2:172.21.15.150:6800/90480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f938806af50 con 0x7f9388068470 2023-11-28T02:52:49.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.663+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 --> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f93a806c860 con 0x7f9388068470 2023-11-28T02:52:49.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f939e7fc700 1 -- 172.21.15.73:0/3889458133 <== osd.5 v2:172.21.15.150:6800/90480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f93a806c860 con 0x7f9388068470 2023-11-28T02:52:49.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7f9388068470 msgr2=0x7f938806a8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7f9388068470 0x7f938806a8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f938805f150 msgr2=0x7f9388061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f938805f150 0x7f9388061610 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f93a8066700 tx=0x7f9390009040).stop 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 msgr2=0x7f93a80c0cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 --2- 172.21.15.73:0/3889458133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93a8062700 0x7f93a80c0cd0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7f9398009e70 tx=0x7f939800b040).stop 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93a6d9d700 1 -- 172.21.15.73:0/3889458133 reap_dead start 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 shutdown_connections 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 >> 172.21.15.73:0/3889458133 conn(0x7f93a80642d0 msgr2=0x7f93a813ab40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 shutdown_connections 2023-11-28T02:52:49.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.664+0000 7f93acbdb700 1 -- 172.21.15.73:0/3889458133 wait complete. 2023-11-28T02:52:49.677 INFO:teuthology.orchestra.run.smithi073.stdout:55834574854 2023-11-28T02:52:49.677 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-11-28T02:52:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.790+0000 7eff5f59e700 1 Processor -- start 2023-11-28T02:52:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.790+0000 7eff5f59e700 1 -- start start 2023-11-28T02:52:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5f59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff6014e530 0x7eff600c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5f59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff600c7bc0 con 0x7eff60154950 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5f59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff60155920 con 0x7eff6014e530 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff60154e00 con 0x7eff60158570 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5ed9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff6014e530 0x7eff600c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff6014e530 0x7eff600c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:38124/0 (socket says 172.21.15.73:38124) 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 -- 172.21.15.73:0/245313298 learned_addr learned my addr 172.21.15.73:0/245313298 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5dd9b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7eff60154950 0x7eff6014e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59938/0 (socket says 172.21.15.73:59938) 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5ed9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:47258/0 (socket says 172.21.15.73:47258) 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 -- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 msgr2=0x7eff60158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 -- 172.21.15.73:0/245313298 >> v1:172.21.15.150:6789/0 conn(0x7eff60154950 legacy=0x7eff6014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5d59a700 1 -- 172.21.15.73:0/245313298 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 133657749 0 0) 0x7eff600c7bc0 con 0x7eff60154950 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5e59c700 1 -- 172.21.15.73:0/245313298 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff60155080 con 0x7eff6014e530 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.791+0000 7eff5ed9d700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5e59c700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6014e530 0x7eff600c66f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7eff50006fd0 tx=0x7eff500090f0).ready entity=mon.1 client_cookie=d42f23a3f4e0a38b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5d59a700 1 -- 172.21.15.73:0/245313298 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000b580 con 0x7eff6014e530 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5d59a700 1 -- 172.21.15.73:0/245313298 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000b6e0 con 0x7eff6014e530 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5d59a700 1 -- 172.21.15.73:0/245313298 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000ba60 con 0x7eff6014e530 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 -- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6014e530 msgr2=0x7eff600c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6014e530 0x7eff600c66f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7eff50006fd0 tx=0x7eff500090f0).stop 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 -- 172.21.15.73:0/245313298 shutdown_connections 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff60158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 --2- 172.21.15.73:0/245313298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff6014e530 0x7eff600c66f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 -- 172.21.15.73:0/245313298 >> 172.21.15.73:0/245313298 conn(0x7eff60064990 msgr2=0x7eff60064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 -- 172.21.15.73:0/245313298 shutdown_connections 2023-11-28T02:52:49.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.792+0000 7eff5f59e700 1 -- 172.21.15.73:0/245313298 wait complete. 2023-11-28T02:52:49.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.793+0000 7eff5f59e700 1 Processor -- start 2023-11-28T02:52:49.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.793+0000 7eff5f59e700 1 -- start start 2023-11-28T02:52:49.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 0x7eff600ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff60154950 0x7eff60165020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff600c41f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600afa00 con 0x7eff6014e530 2023-11-28T02:52:49.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff600af570 con 0x7eff60154950 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff600ae960 con 0x7eff60158570 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 0x7eff600ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 0x7eff600ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42628/0 (socket says 172.21.15.73:42628) 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5e59c700 1 -- 172.21.15.73:0/4285393148 learned_addr learned my addr 172.21.15.73:0/4285393148 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5dd9b700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff60154950 0x7eff60165020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.794+0000 7eff5ed9d700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff600c41f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 -- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 msgr2=0x7eff600c41f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff60158570 0x7eff600c41f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 -- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff60154950 msgr2=0x7eff60165020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff60154950 0x7eff60165020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff58009a20 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5e59c700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 0x7eff600ae030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7eff50009130 tx=0x7eff50017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5000b850 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff5000b9b0 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff50009370 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff5001c670 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff600aef20 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.795+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7eff60062ee0 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.796+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff5001ca20 con 0x7eff6014e530 2023-11-28T02:52:49.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.796+0000 7eff4e7fc700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff4005f150 0x7eff40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.796+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7eff5007beb0 con 0x7eff6014e530 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.796+0000 7eff5dd9b700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff4005f150 0x7eff40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.796+0000 7eff4e7fc700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7eff40068470 0x7eff4006a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.797+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7eff4006af50 con 0x7eff40068470 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.797+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7eff5007e4d0 con 0x7eff6014e530 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.797+0000 7eff5dd9b700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff4005f150 0x7eff40061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7eff600c5a90 tx=0x7eff54007bd0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.797+0000 7eff5ed9d700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7eff40068470 0x7eff4006a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.797+0000 7eff5ed9d700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7eff40068470 0x7eff4006a8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.798+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== osd.6 v2:172.21.15.150:6824/90525 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7eff4006af50 con 0x7eff40068470 2023-11-28T02:52:49.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.812+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 --> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7eff600af850 con 0x7eff40068470 2023-11-28T02:52:49.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.812+0000 7eff4e7fc700 1 -- 172.21.15.73:0/4285393148 <== osd.6 v2:172.21.15.150:6824/90525 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7eff600af850 con 0x7eff40068470 2023-11-28T02:52:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7eff40068470 msgr2=0x7eff4006a8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7eff40068470 0x7eff4006a8d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff4005f150 msgr2=0x7eff40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff4005f150 0x7eff40061610 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7eff600c5a90 tx=0x7eff54007bd0).stop 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 msgr2=0x7eff600ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 --2- 172.21.15.73:0/4285393148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff6014e530 0x7eff600ae030 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7eff50009130 tx=0x7eff50017000).stop 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5ed9d700 1 -- 172.21.15.73:0/4285393148 reap_dead start 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 shutdown_connections 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 >> 172.21.15.73:0/4285393148 conn(0x7eff60064990 msgr2=0x7eff60153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 shutdown_connections 2023-11-28T02:52:49.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.813+0000 7eff5f59e700 1 -- 172.21.15.73:0/4285393148 wait complete. 2023-11-28T02:52:49.826 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:49.827 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-11-28T02:52:49.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.938+0000 7f0a0b669700 1 Processor -- start 2023-11-28T02:52:49.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a0b669700 1 -- start start 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a0b669700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a04098d10 0x7f0a040c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a0b669700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a040c72d0 con 0x7f0a04158320 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a0b669700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a04064570 con 0x7f0a040c6190 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a0b669700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a04154bc0 con 0x7f0a04098d10 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.939+0000 7f0a09405700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0a04098d10 0x7f0a040c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09c06700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0a04158320 0x7f0a040a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33772/0 (socket says 172.21.15.73:33772) 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09c06700 1 -- 172.21.15.73:0/3429022592 learned_addr learned my addr 172.21.15.73:0/3429022592 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1461516970 0 0) 0x7f0a040c72d0 con 0x7f0a04158320 2023-11-28T02:52:49.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09f00033f0 con 0x7f0a04158320 2023-11-28T02:52:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1817214332 0 0) 0x7f09f00033f0 con 0x7f0a04158320 2023-11-28T02:52:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 >> v2:172.21.15.73:3300/0 conn(0x7f0a04098d10 msgr2=0x7f0a040c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 --2- 172.21.15.73:0/3429022592 >> v2:172.21.15.73:3300/0 conn(0x7f0a04098d10 0x7f0a040c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 >> v1:172.21.15.150:6789/0 conn(0x7f0a040c6190 legacy=0x7f0a04063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040c66c0 con 0x7f0a04158320 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f09fc0017f0 con 0x7f0a04158320 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 >> v1:172.21.15.73:6789/0 conn(0x7f0a04158320 legacy=0x7f0a040a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 0x7f09f0003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f0004690 0x7f09f0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f0005000 0x7f09f000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a04154bc0 con 0x7f09f00038e0 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000be00 con 0x7f09f0005000 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f000bfc0 con 0x7f09f0004690 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 0x7f09f0003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09c06700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f0005000 0x7f09f000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a08c04700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f0004690 0x7f09f0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 -- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f0004690 msgr2=0x7f09f0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09f0004690 0x7f09f0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 -- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f0005000 msgr2=0x7f09f000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f0005000 0x7f09f000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.940+0000 7f0a09405700 1 -- 172.21.15.73:0/3429022592 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f00033f0 con 0x7f09f00038e0 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a09c06700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f0005000 0x7f09f000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a09c06700 1 -- 172.21.15.73:0/3429022592 reap_dead start 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a09405700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 0x7f09f0003cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0a00009a80 tx=0x7f0a00009060).ready entity=mon.0 client_cookie=65596d698ed84d33 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a00009430 con 0x7f09f00038e0 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a0000e640 con 0x7f09f00038e0 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f09fbfff700 1 -- 172.21.15.73:0/3429022592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a0000e9c0 con 0x7f09f00038e0 2023-11-28T02:52:49.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 -- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 msgr2=0x7f09f0003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 0x7f09f0003cc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f0a00009a80 tx=0x7f0a00009060).stop 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 -- 172.21.15.73:0/3429022592 shutdown_connections 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 --2- 172.21.15.73:0/3429022592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09f00038e0 0x7f09f0003cc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 -- 172.21.15.73:0/3429022592 >> 172.21.15.73:0/3429022592 conn(0x7f0a040cb310 msgr2=0x7f0a040cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 -- 172.21.15.73:0/3429022592 shutdown_connections 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.941+0000 7f0a0b669700 1 -- 172.21.15.73:0/3429022592 wait complete. 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.942+0000 7f0a0b669700 1 Processor -- start 2023-11-28T02:52:49.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 -- start start 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f00038e0 0x7f0a040a82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040a99d0 0x7f0a040ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 0x7f0a040aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040a8930 con 0x7f0a0414a910 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040a8c10 con 0x7f09f00038e0 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.943+0000 7f0a0b669700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a040a8ef0 con 0x7f0a040a99d0 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 0x7f0a040aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 0x7f0a040aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42644/0 (socket says 172.21.15.73:42644) 2023-11-28T02:52:49.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 -- 172.21.15.73:0/499691372 learned_addr learned my addr 172.21.15.73:0/499691372 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09405700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f00038e0 0x7f0a040a82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a08c04700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040a99d0 0x7f0a040ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 -- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040a99d0 msgr2=0x7f0a040ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0a040a99d0 0x7f0a040ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 -- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f00038e0 msgr2=0x7f0a040a82a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f00038e0 0x7f0a040a82a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a040a91d0 con 0x7f0a0414a910 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09405700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09f00038e0 0x7f0a040a82a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a09c06700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 0x7f0a040aad70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f09fc000f80 tx=0x7f09fc007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00e580 con 0x7f0a0414a910 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09fc00e6e0 con 0x7f0a0414a910 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a040a94b0 con 0x7f0a0414a910 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09fc00eaa0 con 0x7f0a0414a910 2023-11-28T02:52:49.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.944+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a0414ad70 con 0x7f0a0414a910 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.945+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0a040a8d80 con 0x7f0a0414a910 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.945+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f09fc014430 con 0x7f0a0414a910 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f09f97fa700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f09e805f120 0x7f09e80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f09fc079f70 con 0x7f0a0414a910 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f0a09405700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f09e805f120 0x7f09e80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f09f97fa700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7f09e80684c0 0x7f09e806a920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f09e806afa0 con 0x7f09e80684c0 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f09fc07c590 con 0x7f0a0414a910 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f0a09405700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f09e805f120 0x7f09e80615e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0a00006d50 tx=0x7f0a0001a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f0a08c04700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7f09e80684c0 0x7f09e806a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.946+0000 7f0a08c04700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7f09e80684c0 0x7f09e806a920 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:49.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.947+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== osd.7 v2:172.21.15.150:6809/90481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f09e806afa0 con 0x7f09e80684c0 2023-11-28T02:52:49.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.961+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 --> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0a0406c860 con 0x7f09e80684c0 2023-11-28T02:52:49.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f09f97fa700 1 -- 172.21.15.73:0/499691372 <== osd.7 v2:172.21.15.150:6809/90481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0a0406c860 con 0x7f09e80684c0 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7f09e80684c0 msgr2=0x7f09e806a920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7f09e80684c0 0x7f09e806a920 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f09e805f120 msgr2=0x7f09e80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f09e805f120 0x7f09e80615e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f0a00006d50 tx=0x7f0a0001a000).stop 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 msgr2=0x7f0a040aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 --2- 172.21.15.73:0/499691372 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0a0414a910 0x7f0a040aad70 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f09fc000f80 tx=0x7f09fc007df0).stop 2023-11-28T02:52:49.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a09c06700 1 -- 172.21.15.73:0/499691372 reap_dead start 2023-11-28T02:52:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 shutdown_connections 2023-11-28T02:52:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 >> 172.21.15.73:0/499691372 conn(0x7f0a040cb310 msgr2=0x7f0a040ca800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 shutdown_connections 2023-11-28T02:52:49.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:49.962+0000 7f0a0b669700 1 -- 172.21.15.73:0/499691372 wait complete. 2023-11-28T02:52:49.975 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:49.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-11-28T02:52:50.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.087+0000 7fc5fe008700 1 Processor -- start 2023-11-28T02:52:50.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.087+0000 7fc5fe008700 1 -- start start 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc5f8098d10 0x7fc5f80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5f80c6190 0x7fc5f8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 0x7fc5f8158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f80c72d0 con 0x7fc5f80c6190 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5f8154bc0 con 0x7fc5f8098d10 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5fe008700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c66c0 con 0x7fc5f8064350 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5f80c6190 0x7fc5f8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5f80c6190 0x7fc5f8063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42650/0 (socket says 172.21.15.73:42650) 2023-11-28T02:52:50.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 -- 172.21.15.73:0/3858846528 learned_addr learned my addr 172.21.15.73:0/3858846528 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f77fe700 1 --2- 172.21.15.73:0/3858846528 >> v2:172.21.15.150:3300/0 conn(0x7fc5f8098d10 0x7fc5f80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f7fff700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 0x7fc5f8158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 -- 172.21.15.73:0/3858846528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 msgr2=0x7fc5f8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 0x7fc5f8158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 -- 172.21.15.73:0/3858846528 >> v2:172.21.15.150:3300/0 conn(0x7fc5f8098d10 msgr2=0x7fc5f80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/3858846528 >> v2:172.21.15.150:3300/0 conn(0x7fc5f8098d10 0x7fc5f80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f6ffd700 1 -- 172.21.15.73:0/3858846528 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5f80c69a0 con 0x7fc5f80c6190 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.088+0000 7fc5f7fff700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 0x7fc5f8158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f80c6190 0x7fc5f8063e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc5f8083f80 tx=0x7fc5ec0090f0).ready entity=mon.0 client_cookie=dfa8bbbe7edc3679 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5f67fc700 1 -- 172.21.15.73:0/3858846528 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00b600 con 0x7fc5f80c6190 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5f67fc700 1 -- 172.21.15.73:0/3858846528 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ec00b760 con 0x7fc5f80c6190 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5f67fc700 1 -- 172.21.15.73:0/3858846528 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ec00bae0 con 0x7fc5f80c6190 2023-11-28T02:52:50.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 -- 172.21.15.73:0/3858846528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f80c6190 msgr2=0x7fc5f8063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f80c6190 0x7fc5f8063e60 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fc5f8083f80 tx=0x7fc5ec0090f0).stop 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 -- 172.21.15.73:0/3858846528 shutdown_connections 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8064350 0x7fc5f8158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 --2- 172.21.15.73:0/3858846528 >> v2:172.21.15.150:3300/0 conn(0x7fc5f8098d10 0x7fc5f80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 --2- 172.21.15.73:0/3858846528 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f80c6190 0x7fc5f8063e60 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 -- 172.21.15.73:0/3858846528 >> 172.21.15.73:0/3858846528 conn(0x7fc5f80cb310 msgr2=0x7fc5f80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 -- 172.21.15.73:0/3858846528 shutdown_connections 2023-11-28T02:52:50.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.089+0000 7fc5fe008700 1 -- 172.21.15.73:0/3858846528 wait complete. 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.090+0000 7fc5fe008700 1 Processor -- start 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 -- start start 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8098d10 0x7fc5f80c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f80c6190 0x7fc5f814b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b1070 con 0x7fc5f8064350 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80c4d20 con 0x7fc5f80c6190 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5fe008700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5f80b0460 con 0x7fc5f8098d10 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5f77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5f77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42662/0 (socket says 172.21.15.73:42662) 2023-11-28T02:52:50.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5f77fe700 1 -- 172.21.15.73:0/1211192304 learned_addr learned my addr 172.21.15.73:0/1211192304 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5f7fff700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f80c6190 0x7fc5f814b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.091+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8098d10 0x7fc5f80c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 -- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8098d10 msgr2=0x7fc5f80c4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8098d10 0x7fc5f80c4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 -- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f80c6190 msgr2=0x7fc5f814b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f80c6190 0x7fc5f814b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 -- 172.21.15.73:0/1211192304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5ec009370 con 0x7fc5f8064350 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5f77fe700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc5e800e190 tx=0x7fc5e8009920).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e800ecd0 con 0x7fc5f8064350 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5e800ee30 con 0x7fc5f8064350 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5f80b0740 con 0x7fc5f8064350 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5e8014610 con 0x7fc5f8064350 2023-11-28T02:52:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.092+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5f80b0d00 con 0x7fc5f8064350 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.093+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5f806c860 con 0x7fc5f8064350 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.093+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc5e801d3b0 con 0x7fc5f8064350 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.094+0000 7fc5df7fe700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 0x7fc5e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.094+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc5e807a580 con 0x7fc5f8064350 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.094+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 0x7fc5e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.094+0000 7fc5f6ffd700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 0x7fc5e4061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc5f8083f80 tx=0x7fc5ec017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.098+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc5e804b7f0 con 0x7fc5f8064350 2023-11-28T02:52:50.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.261+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fc5f80b1350 con 0x7fc5f8064350 2023-11-28T02:52:50.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.261+0000 7fc5df7fe700 1 -- 172.21.15.73:0/1211192304 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fc5e804ee10 con 0x7fc5f8064350 2023-11-28T02:52:50.263 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 msgr2=0x7fc5e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 0x7fc5e4061610 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc5f8083f80 tx=0x7fc5ec017000).stop 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 msgr2=0x7fc5f80afdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7fc5e800e190 tx=0x7fc5e8009920).stop 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 shutdown_connections 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc5e405f150 0x7fc5e4061610 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc5f80c6190 0x7fc5f814b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5f8064350 0x7fc5f80afdf0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 --2- 172.21.15.73:0/1211192304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5f8098d10 0x7fc5f80c4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 >> 172.21.15.73:0/1211192304 conn(0x7fc5f80cb310 msgr2=0x7fc5f8063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 shutdown_connections 2023-11-28T02:52:50.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.263+0000 7fc5fe008700 1 -- 172.21.15.73:0/1211192304 wait complete. 2023-11-28T02:52:50.277 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.0 2023-11-28T02:52:50.278 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-11-28T02:52:50.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.391+0000 7f986ffff700 1 Processor -- start 2023-11-28T02:52:50.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.391+0000 7f986ffff700 1 -- start start 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9870066a30 0x7f9870066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986ffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f987013db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986ffff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98701586a0 con 0x7f98700c6a30 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986ffff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9870157de0 con 0x7f9870066a30 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986ffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9870064820 con 0x7f98700c6eb0 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9870066a30 0x7f9870066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9870066a30 0x7f9870066e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42668/0 (socket says 172.21.15.73:42668) 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986e7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f98700c6a30 0x7f987013d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33788/0 (socket says 172.21.15.73:33788) 2023-11-28T02:52:50.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 -- 172.21.15.73:0/3443577259 learned_addr learned my addr 172.21.15.73:0/3443577259 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986f7fe700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f987013db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 -- 172.21.15.73:0/3443577259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 msgr2=0x7f987013db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f987013db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 -- 172.21.15.73:0/3443577259 >> v1:172.21.15.73:6789/0 conn(0x7f98700c6a30 legacy=0x7f987013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 -- 172.21.15.73:0/3443577259 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9870158050 con 0x7f9870066a30 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986dffb700 1 -- 172.21.15.73:0/3443577259 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2971528717 0 0) 0x7f98701586a0 con 0x7f98700c6a30 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986effd700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f9870066e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9860009f20 tx=0x7f9860009840).ready entity=mon.0 client_cookie=cf9df14256202a2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.392+0000 7f986dffb700 1 -- 172.21.15.73:0/3443577259 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000e070 con 0x7f9870066a30 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986dffb700 1 -- 172.21.15.73:0/3443577259 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986000b9d0 con 0x7f9870066a30 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986dffb700 1 -- 172.21.15.73:0/3443577259 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000bd50 con 0x7f9870066a30 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 -- 172.21.15.73:0/3443577259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 msgr2=0x7f9870066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f9870066e10 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f9860009f20 tx=0x7f9860009840).stop 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 -- 172.21.15.73:0/3443577259 shutdown_connections 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f987013db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 --2- 172.21.15.73:0/3443577259 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f9870066e10 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 -- 172.21.15.73:0/3443577259 >> 172.21.15.73:0/3443577259 conn(0x7f98700c8ed0 msgr2=0x7f98700c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 -- 172.21.15.73:0/3443577259 shutdown_connections 2023-11-28T02:52:50.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.393+0000 7f986ffff700 1 -- 172.21.15.73:0/3443577259 wait complete. 2023-11-28T02:52:50.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.394+0000 7f986ffff700 1 Processor -- start 2023-11-28T02:52:50.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.394+0000 7f986ffff700 1 -- start start 2023-11-28T02:52:50.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98700c6a30 0x7f98700c3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f9870167f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700c3c50 con 0x7f9870066a30 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98700c4a50 con 0x7f98700c6a30 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986ffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9870168470 con 0x7f98700c6eb0 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42676/0 (socket says 172.21.15.73:42676) 2023-11-28T02:52:50.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 -- 172.21.15.73:0/526867530 learned_addr learned my addr 172.21.15.73:0/526867530 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986e7fc700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98700c6a30 0x7f98700c3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986f7fe700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f9870167f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 -- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 msgr2=0x7f9870167f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f9870167f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 -- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98700c6a30 msgr2=0x7f98700c3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98700c6a30 0x7f98700c3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 -- 172.21.15.73:0/526867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9860009ae0 con 0x7f9870066a30 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.395+0000 7f986effd700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f986000b610 tx=0x7f9860009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.396+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986000b640 con 0x7f9870066a30 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.396+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986000bb40 con 0x7f9870066a30 2023-11-28T02:52:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.396+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98700c3e40 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.396+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986001e6a0 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.396+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98700c4400 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.397+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f987006c7c0 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.397+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9860003040 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.397+0000 7f9866ffd700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 0x7f9850061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.397+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f986007b680 con 0x7f9870066a30 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.397+0000 7f986e7fc700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 0x7f9850061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.398+0000 7f986e7fc700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 0x7f9850061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9870067ef0 tx=0x7f9858007180).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.401+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f986004c950 con 0x7f9870066a30 2023-11-28T02:52:50.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:52:50.534 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:52:50.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.568+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f98700c4d30 con 0x7f9870066a30 2023-11-28T02:52:50.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.568+0000 7f9866ffd700 1 -- 172.21.15.73:0/526867530 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9860023070 con 0x7f9870066a30 2023-11-28T02:52:50.570 INFO:teuthology.orchestra.run.smithi073.stdout:55834574852 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 msgr2=0x7f9850061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 0x7f9850061610 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9870067ef0 tx=0x7f9858007180).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 msgr2=0x7f98700c1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f986000b610 tx=0x7f9860009840).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 shutdown_connections 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f985005f150 0x7f9850061610 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98700c6eb0 0x7f9870167f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9870066a30 0x7f98700c1f80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 --2- 172.21.15.73:0/526867530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98700c6a30 0x7f98700c3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 >> 172.21.15.73:0/526867530 conn(0x7f98700c8ed0 msgr2=0x7f9870064a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 shutdown_connections 2023-11-28T02:52:50.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.570+0000 7f986ffff700 1 -- 172.21.15.73:0/526867530 wait complete. 2023-11-28T02:52:50.584 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.1 2023-11-28T02:52:50.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-11-28T02:52:50.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.742+0000 7f800999a700 1 Processor -- start 2023-11-28T02:52:50.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.742+0000 7f800999a700 1 -- start start 2023-11-28T02:52:50.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f800999a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8004098d10 0x7f80040c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f800999a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040c6190 0x7f8004063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f800999a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80040c72d0 con 0x7f8004064350 2023-11-28T02:52:50.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f800999a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8004154bc0 con 0x7f8004098d10 2023-11-28T02:52:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f800999a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80040c66c0 con 0x7f80040c6190 2023-11-28T02:52:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8004098d10 0x7f80040c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8004098d10 0x7f80040c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42682/0 (socket says 172.21.15.73:42682) 2023-11-28T02:52:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 -- 172.21.15.73:0/2454765076 learned_addr learned my addr 172.21.15.73:0/2454765076 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f80027fc700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040c6190 0x7f8004063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 -- 172.21.15.73:0/2454765076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040c6190 msgr2=0x7f8004063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040c6190 0x7f8004063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.743+0000 7f8002ffd700 1 -- 172.21.15.73:0/2454765076 >> v1:172.21.15.150:6789/0 conn(0x7f8004064350 legacy=0x7f8004158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f8002ffd700 1 -- 172.21.15.73:0/2454765076 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80040c69a0 con 0x7f8004098d10 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f8002ffd700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8004098d10 0x7f80040c7760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f7ff4009f20 tx=0x7f7ff4009840).ready entity=mon.0 client_cookie=86fa29ee07c73ecb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f8001ffb700 1 -- 172.21.15.73:0/2454765076 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400e070 con 0x7f8004098d10 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f8001ffb700 1 -- 172.21.15.73:0/2454765076 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ff400b9d0 con 0x7f8004098d10 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f8001ffb700 1 -- 172.21.15.73:0/2454765076 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ff400bd50 con 0x7f8004098d10 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 -- 172.21.15.73:0/2454765076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8004098d10 msgr2=0x7f80040c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8004098d10 0x7f80040c7760 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f7ff4009f20 tx=0x7f7ff4009840).stop 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 -- 172.21.15.73:0/2454765076 shutdown_connections 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8004098d10 0x7f80040c7760 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 --2- 172.21.15.73:0/2454765076 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f80040c6190 0x7f8004063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 -- 172.21.15.73:0/2454765076 >> 172.21.15.73:0/2454765076 conn(0x7f80040cb310 msgr2=0x7f80040cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 -- 172.21.15.73:0/2454765076 shutdown_connections 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.744+0000 7f800999a700 1 -- 172.21.15.73:0/2454765076 wait complete. 2023-11-28T02:52:50.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 Processor -- start 2023-11-28T02:52:50.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 -- start start 2023-11-28T02:52:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004064350 0x7f800413eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004098d10 0x7f80040b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80040b0b20 con 0x7f80040c6190 2023-11-28T02:52:50.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8004150d90 con 0x7f8004064350 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f800999a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80040b0d20 con 0x7f8004098d10 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f80037fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f80037fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42690/0 (socket says 172.21.15.73:42690) 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f80037fe700 1 -- 172.21.15.73:0/1402058969 learned_addr learned my addr 172.21.15.73:0/1402058969 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.746+0000 7f8002ffd700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004064350 0x7f800413eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80027fc700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004098d10 0x7f80040b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 -- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004098d10 msgr2=0x7f80040b0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004098d10 0x7f80040b0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 -- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004064350 msgr2=0x7f800413eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004064350 0x7f800413eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 -- 172.21.15.73:0/1402058969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ff4009ae0 con 0x7f80040c6190 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f80037fe700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f7ffc00c440 tx=0x7f7ffc007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc0123f0 con 0x7f80040c6190 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ffc001e90 con 0x7f80040c6190 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc012760 con 0x7f80040c6190 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80040b1000 con 0x7f80040c6190 2023-11-28T02:52:50.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.747+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f800413f810 con 0x7f80040c6190 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.748+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f800406c860 con 0x7f80040c6190 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.748+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7ffc012b10 con 0x7f80040c6190 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.748+0000 7f7feaffd700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 0x7f7ff0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.749+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ffc079450 con 0x7f80040c6190 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.749+0000 7f8002ffd700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 0x7f7ff0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:50.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.749+0000 7f8002ffd700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 0x7f7ff0061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7ff400b9a0 tx=0x7f7ff4009840).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:50.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.753+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7ffc047af0 con 0x7f80040c6190 2023-11-28T02:52:50.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.915+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f80040c6c80 con 0x7f80040c6190 2023-11-28T02:52:50.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.915+0000 7f7feaffd700 1 -- 172.21.15.73:0/1402058969 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7ffc04a720 con 0x7f80040c6190 2023-11-28T02:52:50.917 INFO:teuthology.orchestra.run.smithi073.stdout:55834574854 2023-11-28T02:52:50.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.917+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 msgr2=0x7f7ff0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.917+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 0x7f7ff0061610 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f7ff400b9a0 tx=0x7f7ff4009840).stop 2023-11-28T02:52:50.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.917+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 msgr2=0x7f80040b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:50.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.917+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f7ffc00c440 tx=0x7f7ffc007d70).stop 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 shutdown_connections 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ff005f150 0x7f7ff0061610 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f80040c6190 0x7f80040b0630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8004064350 0x7f800413eee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 --2- 172.21.15.73:0/1402058969 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8004098d10 0x7f80040b0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 >> 172.21.15.73:0/1402058969 conn(0x7f80040cb310 msgr2=0x7f80040aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 shutdown_connections 2023-11-28T02:52:50.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:50.918+0000 7f800999a700 1 -- 172.21.15.73:0/1402058969 wait complete. 2023-11-28T02:52:50.931 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.2 2023-11-28T02:52:50.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-11-28T02:52:51.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.042+0000 7fef4d029700 1 Processor -- start 2023-11-28T02:52:51.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.042+0000 7fef4d029700 1 -- start start 2023-11-28T02:52:51.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef4d029700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef480c6190 0x7fef480c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef4d029700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef480c72d0 con 0x7fef48098d10 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef4d029700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef48064570 con 0x7fef48158320 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef4d029700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef48154bc0 con 0x7fef480c6190 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef46d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fef48098d10 0x7fef48063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33790/0 (socket says 172.21.15.73:33790) 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.043+0000 7fef46d9d700 1 -- 172.21.15.73:0/385365743 learned_addr learned my addr 172.21.15.73:0/385365743 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2688220146 0 0) 0x7fef480c72d0 con 0x7fef48098d10 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef4659c700 1 --2- 172.21.15.73:0/385365743 >> v2:172.21.15.150:3300/0 conn(0x7fef480c6190 0x7fef480c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef300033f0 con 0x7fef48098d10 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3958678045 0 0) 0x7fef300033f0 con 0x7fef48098d10 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 >> v2:172.21.15.150:3300/0 conn(0x7fef480c6190 msgr2=0x7fef480c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 --2- 172.21.15.73:0/385365743 >> v2:172.21.15.150:3300/0 conn(0x7fef480c6190 0x7fef480c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 >> v1:172.21.15.150:6789/0 conn(0x7fef48158320 legacy=0x7fef480a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef480c66c0 con 0x7fef48098d10 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fef380017f0 con 0x7fef48098d10 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 >> v1:172.21.15.73:6789/0 conn(0x7fef48098d10 legacy=0x7fef48063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 0x7fef30003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004690 0x7fef30004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef30005000 0x7fef3000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48154bc0 con 0x7fef300038e0 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef3000beb0 con 0x7fef30004690 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef3000c070 con 0x7fef30005000 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 0x7fef30003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef4659c700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004690 0x7fef30004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef4759e700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef30005000 0x7fef3000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 -- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef30005000 msgr2=0x7fef3000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef30005000 0x7fef3000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 -- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004690 msgr2=0x7fef30004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef30004690 0x7fef30004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 -- 172.21.15.73:0/385365743 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef300033f0 con 0x7fef300038e0 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef4759e700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef30005000 0x7fef3000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef4759e700 1 -- 172.21.15.73:0/385365743 reap_dead start 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.044+0000 7fef46d9d700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 0x7fef30003d00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fef3800c480 tx=0x7fef3800c790).ready entity=mon.0 client_cookie=6a9e5c6cdb14341e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38002030 con 0x7fef300038e0 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3800ce20 con 0x7fef300038e0 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef45d9b700 1 -- 172.21.15.73:0/385365743 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef38005d30 con 0x7fef300038e0 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 -- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 msgr2=0x7fef30003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 0x7fef30003d00 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7fef3800c480 tx=0x7fef3800c790).stop 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 -- 172.21.15.73:0/385365743 shutdown_connections 2023-11-28T02:52:51.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 --2- 172.21.15.73:0/385365743 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef300038e0 0x7fef30003d00 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 -- 172.21.15.73:0/385365743 >> 172.21.15.73:0/385365743 conn(0x7fef480cb310 msgr2=0x7fef480cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:51.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 -- 172.21.15.73:0/385365743 shutdown_connections 2023-11-28T02:52:51.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.045+0000 7fef4d029700 1 -- 172.21.15.73:0/385365743 wait complete. 2023-11-28T02:52:51.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.046+0000 7fef4d029700 1 Processor -- start 2023-11-28T02:52:51.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.046+0000 7fef4d029700 1 -- start start 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef300038e0 0x7fef480ca360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 0x7fef481682c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef48169290 con 0x7fef480ca850 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef480af320 con 0x7fef300038e0 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4d029700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef4814f450 con 0x7fef48151b40 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42720/0 (socket says 172.21.15.73:42720) 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 -- 172.21.15.73:0/4013242016 learned_addr learned my addr 172.21.15.73:0/4013242016 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef46d9d700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef300038e0 0x7fef480ca360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4759e700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 0x7fef481682c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 -- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 msgr2=0x7fef481682c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 0x7fef481682c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 -- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef300038e0 msgr2=0x7fef480ca360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef300038e0 0x7fef480ca360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4659c700 1 -- 172.21.15.73:0/4013242016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48151fc0 con 0x7fef480ca850 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.047+0000 7fef4759e700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 0x7fef481682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef4659c700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fef3c011a50 tx=0x7fef3c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00f940 con 0x7fef480ca850 2023-11-28T02:52:51.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef3c00faa0 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef48168fb0 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef3c00fe40 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.048+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef480aefb0 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.049+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef480aeb30 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.049+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fef3c014650 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.049+0000 7fef2effd700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 0x7fef34061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.049+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fef3c07ab40 con 0x7fef480ca850 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.049+0000 7fef46d9d700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 0x7fef34061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.050+0000 7fef46d9d700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 0x7fef34061610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fef3800c7d0 tx=0x7fef38015000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.054+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fef480aeb30 con 0x7fef480ca850 2023-11-28T02:52:51.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.216+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fef480aeb30 con 0x7fef480ca850 2023-11-28T02:52:51.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.216+0000 7fef2effd700 1 -- 172.21.15.73:0/4013242016 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fef480aeb30 con 0x7fef480ca850 2023-11-28T02:52:51.217 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:51.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 msgr2=0x7fef34061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 0x7fef34061610 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fef3800c7d0 tx=0x7fef38015000).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 msgr2=0x7fef48151650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7fef3c011a50 tx=0x7fef3c009060).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 shutdown_connections 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef3405f150 0x7fef34061610 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef48151b40 0x7fef481682c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef300038e0 0x7fef480ca360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 --2- 172.21.15.73:0/4013242016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef480ca850 0x7fef48151650 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 >> 172.21.15.73:0/4013242016 conn(0x7fef480cb310 msgr2=0x7fef480ad880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 shutdown_connections 2023-11-28T02:52:51.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.218+0000 7fef4d029700 1 -- 172.21.15.73:0/4013242016 wait complete. 2023-11-28T02:52:51.232 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.3 2023-11-28T02:52:51.232 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T02:52:51.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.344+0000 7f7ba358c700 1 Processor -- start 2023-11-28T02:52:51.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.344+0000 7f7ba358c700 1 -- start start 2023-11-28T02:52:51.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba358c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7b9c158a30 0x7f7b9c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba358c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c0c72d0 con 0x7f7b9c0c6190 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba358c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b9c064570 con 0x7f7b9c098d10 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba358c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b9c154bc0 con 0x7f7b9c158a30 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba0b27700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7b9c0c6190 0x7f7b9c158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33800/0 (socket says 172.21.15.73:33800) 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.345+0000 7f7ba0b27700 1 -- 172.21.15.73:0/1318051222 learned_addr learned my addr 172.21.15.73:0/1318051222 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/1318051222 >> v2:172.21.15.150:3300/0 conn(0x7f7b9c158a30 0x7f7b9c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760438181 0 0) 0x7f7b9c0c72d0 con 0x7f7b9c0c6190 2023-11-28T02:52:51.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7b880033f0 con 0x7f7b9c0c6190 2023-11-28T02:52:51.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3726133789 0 0) 0x7f7b880033f0 con 0x7f7b9c0c6190 2023-11-28T02:52:51.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 >> v2:172.21.15.150:3300/0 conn(0x7f7b9c158a30 msgr2=0x7f7b9c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 --2- 172.21.15.73:0/1318051222 >> v2:172.21.15.150:3300/0 conn(0x7f7b9c158a30 0x7f7b9c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 >> v1:172.21.15.150:6789/0 conn(0x7f7b9c098d10 legacy=0x7f7b9c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c0c66c0 con 0x7f7b9c0c6190 2023-11-28T02:52:51.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7b8c001470 con 0x7f7b9c0c6190 2023-11-28T02:52:51.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 >> v1:172.21.15.73:6789/0 conn(0x7f7b9c0c6190 legacy=0x7f7b9c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b880038e0 0x7f7b88003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88004690 0x7f7b88004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b8800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c154bc0 con 0x7f7b88005000 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8800beb0 con 0x7f7b880038e0 2023-11-28T02:52:51.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b8800c070 con 0x7f7b88004690 2023-11-28T02:52:51.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b8800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1328700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b880038e0 0x7f7b88003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba0b27700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88004690 0x7f7b88004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 -- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88004690 msgr2=0x7f7b88004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88004690 0x7f7b88004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 -- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b880038e0 msgr2=0x7f7b88003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b880038e0 0x7f7b88003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 -- 172.21.15.73:0/1318051222 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b880033f0 con 0x7f7b88005000 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1328700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b880038e0 0x7f7b88003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba0b27700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b88004690 0x7f7b88004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 -- 172.21.15.73:0/1318051222 reap_dead start 2023-11-28T02:52:51.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.346+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b8800b8e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7b9c062cb0 tx=0x7f7b94009060).ready entity=mon.0 client_cookie=98f685c0ddb6d53c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b94009430 con 0x7f7b88005000 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b94009e70 con 0x7f7b88005000 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7b93fff700 1 -- 172.21.15.73:0/1318051222 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9400e850 con 0x7f7b88005000 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 -- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 msgr2=0x7f7b8800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b8800b8e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f7b9c062cb0 tx=0x7f7b94009060).stop 2023-11-28T02:52:51.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 -- 172.21.15.73:0/1318051222 shutdown_connections 2023-11-28T02:52:51.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 --2- 172.21.15.73:0/1318051222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b8800b8e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 -- 172.21.15.73:0/1318051222 >> 172.21.15.73:0/1318051222 conn(0x7f7b9c0cb310 msgr2=0x7f7b9c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:51.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 -- 172.21.15.73:0/1318051222 shutdown_connections 2023-11-28T02:52:51.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.347+0000 7f7ba358c700 1 -- 172.21.15.73:0/1318051222 wait complete. 2023-11-28T02:52:51.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.348+0000 7f7ba358c700 1 Processor -- start 2023-11-28T02:52:51.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.348+0000 7f7ba358c700 1 -- start start 2023-11-28T02:52:51.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b9c0c2720 0x7f7b9c0c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b9c0c45a0 0x7f7b9c0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15ae80 con 0x7f7b88005000 2023-11-28T02:52:51.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15a380 con 0x7f7b9c0c45a0 2023-11-28T02:52:51.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba358c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b9c15a5e0 con 0x7f7b9c0c2720 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1328700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1328700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42736/0 (socket says 172.21.15.73:42736) 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1328700 1 -- 172.21.15.73:0/4262758021 learned_addr learned my addr 172.21.15.73:0/4262758021 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1b29700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b9c0c45a0 0x7f7b9c0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba0b27700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b9c0c2720 0x7f7b9c0c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1328700 1 -- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b9c0c2720 msgr2=0x7f7b9c0c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.349+0000 7f7ba1328700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b9c0c2720 0x7f7b9c0c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba1328700 1 -- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b9c0c45a0 msgr2=0x7f7b9c0c4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba1328700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b9c0c45a0 0x7f7b9c0c4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba1328700 1 -- 172.21.15.73:0/4262758021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b9c15a8c0 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba1328700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f7b98000f80 tx=0x7f7b98009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800a4e0 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b98010040 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b9800a850 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b9c15aba0 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.350+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b9c0ca1d0 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.351+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b9c0ab5e0 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.351+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7b98003070 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.351+0000 7f7b917fa700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 0x7f7b80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.352+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7b98079780 con 0x7f7b88005000 2023-11-28T02:52:51.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.352+0000 7f7ba0b27700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 0x7f7b80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:51.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.352+0000 7f7ba0b27700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 0x7f7b80061610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7b8c001ed0 tx=0x7f7b8c009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:51.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.356+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7b9804aa40 con 0x7f7b88005000 2023-11-28T02:52:51.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.524+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f7b9c0ab0d0 con 0x7f7b88005000 2023-11-28T02:52:51.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.524+0000 7f7b917fa700 1 -- 172.21.15.73:0/4262758021 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7b9c0ab0d0 con 0x7f7b88005000 2023-11-28T02:52:51.526 INFO:teuthology.orchestra.run.smithi073.stdout:55834574851 2023-11-28T02:52:51.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 msgr2=0x7f7b80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 0x7f7b80061610 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7f7b8c001ed0 tx=0x7f7b8c009040).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 msgr2=0x7f7b9c159e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f7b98000f80 tx=0x7f7b98009040).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 shutdown_connections 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7b8005f150 0x7f7b80061610 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7b9c0c45a0 0x7f7b9c0c4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7b88005000 0x7f7b9c159e90 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 --2- 172.21.15.73:0/4262758021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7b9c0c2720 0x7f7b9c0c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 >> 172.21.15.73:0/4262758021 conn(0x7f7b9c0cb310 msgr2=0x7f7b9c0aa190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 shutdown_connections 2023-11-28T02:52:51.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:51.526+0000 7f7ba358c700 1 -- 172.21.15.73:0/4262758021 wait complete. 2023-11-28T02:52:51.540 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574851 for osd.4 2023-11-28T02:52:52.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-11-28T02:52:52.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.655+0000 7f4856cde700 1 Processor -- start 2023-11-28T02:52:52.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.655+0000 7f4856cde700 1 -- start start 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4856cde700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f48500673b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4856cde700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48500c9ad0 con 0x7f4850096590 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4856cde700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4850067e70 con 0x7f48500678a0 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4856cde700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4850154f40 con 0x7f48501546f0 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f485527b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4850096590 0x7f48500beb80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33812/0 (socket says 172.21.15.73:33812) 2023-11-28T02:52:52.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f485527b700 1 -- 172.21.15.73:0/724917269 learned_addr learned my addr 172.21.15.73:0/724917269 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4854a7a700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f48500673b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 200290568 0 0) 0x7f48500c9ad0 con 0x7f4850096590 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f483c0033f0 con 0x7f4850096590 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2428226368 0 0) 0x7f483c0033f0 con 0x7f4850096590 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 msgr2=0x7f48500673b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f48500673b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 >> v1:172.21.15.150:6789/0 conn(0x7f48500678a0 legacy=0x7f4850095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4850155220 con 0x7f4850096590 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f4848001470 con 0x7f4850096590 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 >> v1:172.21.15.73:6789/0 conn(0x7f4850096590 legacy=0x7f48500beb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f483c003910 0x7f483c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c004690 0x7f483c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f483c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4850154f40 con 0x7f483c003910 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00b8e0 con 0x7f483c004690 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483c00ba50 con 0x7f48501546f0 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f4854a7a700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f483c003910 0x7f483c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f484ffff700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c004690 0x7f483c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.656+0000 7f485527b700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f483c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 -- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 msgr2=0x7f483c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:52.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48501546f0 0x7f483c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 -- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c004690 msgr2=0x7f483c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c004690 0x7f483c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 -- 172.21.15.73:0/724917269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f483c0033f0 con 0x7f483c003910 2023-11-28T02:52:52.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4854a7a700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f483c003910 0x7f483c003cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f484000e210 tx=0x7f484000b040).ready entity=mon.0 client_cookie=6f1386721972ed22 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4840009ac0 con 0x7f483c003910 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484000f040 con 0x7f483c003910 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f484f7fe700 1 -- 172.21.15.73:0/724917269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484000ec70 con 0x7f483c003910 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 -- 172.21.15.73:0/724917269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f483c003910 msgr2=0x7f483c003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 --2- 172.21.15.73:0/724917269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f483c003910 0x7f483c003cf0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f484000e210 tx=0x7f484000b040).stop 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f485527b700 1 -- 172.21.15.73:0/724917269 reap_dead start 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 -- 172.21.15.73:0/724917269 shutdown_connections 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 -- 172.21.15.73:0/724917269 >> 172.21.15.73:0/724917269 conn(0x7f48500631d0 msgr2=0x7f48500c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 -- 172.21.15.73:0/724917269 shutdown_connections 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.657+0000 7f4856cde700 1 -- 172.21.15.73:0/724917269 wait complete. 2023-11-28T02:52:52.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.658+0000 7f4856cde700 1 Processor -- start 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 -- start start 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c003910 0x7f48500a70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48500a75e0 0x7f4850167680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48500a6c60 con 0x7f4850167b70 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4850167ff0 con 0x7f483c003910 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f4856cde700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48500a6050 con 0x7f48500a75e0 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f485527b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f485527b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42746/0 (socket says 172.21.15.73:42746) 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.659+0000 7f485527b700 1 -- 172.21.15.73:0/2306089587 learned_addr learned my addr 172.21.15.73:0/2306089587 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:52.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f4854a7a700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c003910 0x7f48500a70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f484ffff700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48500a75e0 0x7f4850167680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 -- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48500a75e0 msgr2=0x7f4850167680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48500a75e0 0x7f4850167680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 -- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c003910 msgr2=0x7f48500a70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c003910 0x7f48500a70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 -- 172.21.15.73:0/2306089587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48500a6330 con 0x7f4850167b70 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f485527b700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f48500c9070 tx=0x7f4848000f80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848010520 con 0x7f4850167b70 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4848010680 con 0x7f4850167b70 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4848010a40 con 0x7f4850167b70 2023-11-28T02:52:52.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48500a6610 con 0x7f4850167b70 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.660+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4850165770 con 0x7f4850167b70 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.661+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48500a6ab0 con 0x7f4850167b70 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.661+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4848014430 con 0x7f4850167b70 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.661+0000 7f484cff9700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 0x7f4830061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.662+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f48480792d0 con 0x7f4850167b70 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.662+0000 7f4854a7a700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 0x7f4830061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.662+0000 7f4854a7a700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 0x7f4830061610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f484000afd0 tx=0x7f484001a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.665+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f484804a540 con 0x7f4850167b70 2023-11-28T02:52:52.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.829+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f485006c860 con 0x7f4850167b70 2023-11-28T02:52:52.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.829+0000 7f484cff9700 1 -- 172.21.15.73:0/2306089587 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4848046e60 con 0x7f4850167b70 2023-11-28T02:52:52.830 INFO:teuthology.orchestra.run.smithi073.stdout:55834574852 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 msgr2=0x7f4830061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 0x7f4830061610 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7f484000afd0 tx=0x7f484001a000).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 msgr2=0x7f4850165280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f48500c9070 tx=0x7f4848000f80).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 shutdown_connections 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f483005f150 0x7f4830061610 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4850167b70 0x7f4850165280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f483c003910 0x7f48500a70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 --2- 172.21.15.73:0/2306089587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48500a75e0 0x7f4850167680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 >> 172.21.15.73:0/2306089587 conn(0x7f48500631d0 msgr2=0x7f48500c7cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:52.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 shutdown_connections 2023-11-28T02:52:52.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.831+0000 7f4856cde700 1 -- 172.21.15.73:0/2306089587 wait complete. 2023-11-28T02:52:52.844 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574852 got 55834574852 for osd.4 2023-11-28T02:52:52.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-11-28T02:52:52.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.956+0000 7f4e24743700 1 Processor -- start 2023-11-28T02:52:52.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.956+0000 7f4e24743700 1 -- start start 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e1c098d10 0x7f4e1c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e1c0c6190 0x7f4e1c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c064350 0x7f4e1c158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c0c72d0 con 0x7f4e1c098d10 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e1c154bc0 con 0x7f4e1c0c6190 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e24743700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c0c66c0 con 0x7f4e1c064350 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e1c098d10 0x7f4e1c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e1c098d10 0x7f4e1c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42760/0 (socket says 172.21.15.73:42760) 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 -- 172.21.15.73:0/1567537356 learned_addr learned my addr 172.21.15.73:0/1567537356 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:52.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e21cde700 1 --2- 172.21.15.73:0/1567537356 >> v2:172.21.15.150:3300/0 conn(0x7f4e1c0c6190 0x7f4e1c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e22ce0700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c064350 0x7f4e1c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 -- 172.21.15.73:0/1567537356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c064350 msgr2=0x7f4e1c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c064350 0x7f4e1c158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 -- 172.21.15.73:0/1567537356 >> v2:172.21.15.150:3300/0 conn(0x7f4e1c0c6190 msgr2=0x7f4e1c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 --2- 172.21.15.73:0/1567537356 >> v2:172.21.15.150:3300/0 conn(0x7f4e1c0c6190 0x7f4e1c063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.957+0000 7f4e224df700 1 -- 172.21.15.73:0/1567537356 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e1c0c69a0 con 0x7f4e1c098d10 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e224df700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c098d10 0x7f4e1c0c7760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4e1c0c5b40 tx=0x7f4e0c009840).ready entity=mon.0 client_cookie=5a1bb45ab94e1655 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e214dd700 1 -- 172.21.15.73:0/1567537356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00e070 con 0x7f4e1c098d10 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e214dd700 1 -- 172.21.15.73:0/1567537356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e0c00ba50 con 0x7f4e1c098d10 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e214dd700 1 -- 172.21.15.73:0/1567537356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e0c00bdd0 con 0x7f4e1c098d10 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 -- 172.21.15.73:0/1567537356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c098d10 msgr2=0x7f4e1c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c098d10 0x7f4e1c0c7760 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 rx=0x7f4e1c0c5b40 tx=0x7f4e0c009840).stop 2023-11-28T02:52:52.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 -- 172.21.15.73:0/1567537356 shutdown_connections 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c098d10 0x7f4e1c0c7760 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 --2- 172.21.15.73:0/1567537356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c064350 0x7f4e1c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 --2- 172.21.15.73:0/1567537356 >> v2:172.21.15.150:3300/0 conn(0x7f4e1c0c6190 0x7f4e1c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 -- 172.21.15.73:0/1567537356 >> 172.21.15.73:0/1567537356 conn(0x7f4e1c0cb310 msgr2=0x7f4e1c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 -- 172.21.15.73:0/1567537356 shutdown_connections 2023-11-28T02:52:52.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.958+0000 7f4e24743700 1 -- 172.21.15.73:0/1567537356 wait complete. 2023-11-28T02:52:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.959+0000 7f4e24743700 1 Processor -- start 2023-11-28T02:52:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.959+0000 7f4e24743700 1 -- start start 2023-11-28T02:52:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e1c064350 0x7f4e1c157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c098d10 0x7f4e1c156d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c155d60 con 0x7f4e1c0c6190 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c153ce0 con 0x7f4e1c064350 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e24743700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e1c1568c0 con 0x7f4e1c098d10 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42772/0 (socket says 172.21.15.73:42772) 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 -- 172.21.15.73:0/210870688 learned_addr learned my addr 172.21.15.73:0/210870688 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e21cde700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c098d10 0x7f4e1c156d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e224df700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e1c064350 0x7f4e1c157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 -- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c098d10 msgr2=0x7f4e1c156d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c098d10 0x7f4e1c156d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 -- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e1c064350 msgr2=0x7f4e1c157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:52.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e1c064350 0x7f4e1c157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.960+0000 7f4e22ce0700 1 -- 172.21.15.73:0/210870688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c009ae0 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e22ce0700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4e1400c680 tx=0x7f4e14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14010580 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e140106e0 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e1c14c690 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e14010a60 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.961+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e1c156270 con 0x7f4e1c0c6190 2023-11-28T02:52:52.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.962+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e1c06c860 con 0x7f4e1c0c6190 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.962+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4e1401b460 con 0x7f4e1c0c6190 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.962+0000 7f4e1a7fc700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 0x7f4e080615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.962+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4e14079fc0 con 0x7f4e1c0c6190 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.962+0000 7f4e224df700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 0x7f4e080615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.963+0000 7f4e224df700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 0x7f4e080615e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4e1c0c5700 tx=0x7f4e0c009840).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:52.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:52.966+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e140a8af0 con 0x7f4e1c0c6190 2023-11-28T02:52:53.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.129+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f4e1c156a00 con 0x7f4e1c0c6190 2023-11-28T02:52:53.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.130+0000 7f4e1a7fc700 1 -- 172.21.15.73:0/210870688 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4e1404b230 con 0x7f4e1c0c6190 2023-11-28T02:52:53.131 INFO:teuthology.orchestra.run.smithi073.stdout:55834574854 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 msgr2=0x7f4e080615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 0x7f4e080615e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 rx=0x7f4e1c0c5700 tx=0x7f4e0c009840).stop 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 msgr2=0x7f4e1c14c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 rx=0x7f4e1400c680 tx=0x7f4e14008040).stop 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 shutdown_connections 2023-11-28T02:52:53.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e0805f120 0x7f4e080615e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e1c0c6190 0x7f4e1c14c1a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e1c064350 0x7f4e1c157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 --2- 172.21.15.73:0/210870688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e1c098d10 0x7f4e1c156d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 >> 172.21.15.73:0/210870688 conn(0x7f4e1c0cb310 msgr2=0x7f4e1c0c9180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 shutdown_connections 2023-11-28T02:52:53.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.132+0000 7f4e24743700 1 -- 172.21.15.73:0/210870688 wait complete. 2023-11-28T02:52:53.145 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.5 2023-11-28T02:52:53.145 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-11-28T02:52:53.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.257+0000 7ff454e4f700 1 Processor -- start 2023-11-28T02:52:53.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.257+0000 7ff454e4f700 1 -- start start 2023-11-28T02:52:53.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff454e4f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff450158160 0x7ff4500a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff454e4f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff450067770 con 0x7ff45014e600 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff454e4f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4500be8d0 con 0x7ff4500a8280 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff454e4f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff450159180 con 0x7ff450158160 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff44dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff450158160 0x7ff4500a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff44ed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff45014e600 0x7ff45014ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33828/0 (socket says 172.21.15.73:33828) 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff44ed9d700 1 -- 172.21.15.73:0/1981148708 learned_addr learned my addr 172.21.15.73:0/1981148708 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.258+0000 7ff44dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff450158160 0x7ff4500a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42786/0 (socket says 172.21.15.73:42786) 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44dd9b700 1 -- 172.21.15.73:0/1981148708 >> v1:172.21.15.150:6789/0 conn(0x7ff4500a8280 legacy=0x7ff450155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44dd9b700 1 -- 172.21.15.73:0/1981148708 >> v1:172.21.15.73:6789/0 conn(0x7ff45014e600 legacy=0x7ff45014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44d59a700 1 -- 172.21.15.73:0/1981148708 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136344662 0 0) 0x7ff450067770 con 0x7ff45014e600 2023-11-28T02:52:53.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44dd9b700 1 -- 172.21.15.73:0/1981148708 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff450158600 con 0x7ff450158160 2023-11-28T02:52:53.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44dd9b700 1 --2- 172.21.15.73:0/1981148708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff450158160 0x7ff4500a8660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff438009760 tx=0x7ff438009060).ready entity=mon.0 client_cookie=bd4dd64f0d2d70da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44d59a700 1 -- 172.21.15.73:0/1981148708 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800b580 con 0x7ff450158160 2023-11-28T02:52:53.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44d59a700 1 -- 172.21.15.73:0/1981148708 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff43800b6e0 con 0x7ff450158160 2023-11-28T02:52:53.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff44d59a700 1 -- 172.21.15.73:0/1981148708 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800ba60 con 0x7ff450158160 2023-11-28T02:52:53.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 -- 172.21.15.73:0/1981148708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff450158160 msgr2=0x7ff4500a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 --2- 172.21.15.73:0/1981148708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff450158160 0x7ff4500a8660 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 rx=0x7ff438009760 tx=0x7ff438009060).stop 2023-11-28T02:52:53.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 -- 172.21.15.73:0/1981148708 shutdown_connections 2023-11-28T02:52:53.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 --2- 172.21.15.73:0/1981148708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff450158160 0x7ff4500a8660 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 -- 172.21.15.73:0/1981148708 >> 172.21.15.73:0/1981148708 conn(0x7ff450062700 msgr2=0x7ff450062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 -- 172.21.15.73:0/1981148708 shutdown_connections 2023-11-28T02:52:53.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.259+0000 7ff454e4f700 1 -- 172.21.15.73:0/1981148708 wait complete. 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.260+0000 7ff454e4f700 1 Processor -- start 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.260+0000 7ff454e4f700 1 -- start start 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4500a8280 0x7ff45014ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450158160 0x7ff4500c7530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500a9d10 con 0x7ff45014e600 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500c01e0 con 0x7ff450158160 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff454e4f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4500a9100 con 0x7ff4500a8280 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44dd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42794/0 (socket says 172.21.15.73:42794) 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44dd9b700 1 -- 172.21.15.73:0/3297196033 learned_addr learned my addr 172.21.15.73:0/3297196033 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44ed9d700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450158160 0x7ff4500c7530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44e59c700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4500a8280 0x7ff45014ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44dd9b700 1 -- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4500a8280 msgr2=0x7ff45014ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.261+0000 7ff44dd9b700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4500a8280 0x7ff45014ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff44dd9b700 1 -- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450158160 msgr2=0x7ff4500c7530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff44dd9b700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450158160 0x7ff4500c7530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff44dd9b700 1 -- 172.21.15.73:0/3297196033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380092a0 con 0x7ff45014e600 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff44dd9b700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff438009eb0 tx=0x7ff438017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43800ba60 con 0x7ff45014e600 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff438000d10 con 0x7ff45014e600 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4500a93e0 con 0x7ff45014e600 2023-11-28T02:52:53.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff43801c630 con 0x7ff45014e600 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.262+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4500a99a0 con 0x7ff45014e600 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.263+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4500a9e80 con 0x7ff45014e600 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.263+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff43801c9e0 con 0x7ff45014e600 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.263+0000 7ff4467fc700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 0x7ff430061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.263+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff43807bf70 con 0x7ff45014e600 2023-11-28T02:52:53.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.263+0000 7ff44e59c700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 0x7ff430061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.264+0000 7ff44e59c700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 0x7ff430061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7ff440003dd0 tx=0x7ff440009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.267+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff43804d240 con 0x7ff45014e600 2023-11-28T02:52:53.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.432+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff45006c860 con 0x7ff45014e600 2023-11-28T02:52:53.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.433+0000 7ff4467fc700 1 -- 172.21.15.73:0/3297196033 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff438050860 con 0x7ff45014e600 2023-11-28T02:52:53.434 INFO:teuthology.orchestra.run.smithi073.stdout:55834574853 2023-11-28T02:52:53.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 msgr2=0x7ff430061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 0x7ff430061610 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7ff440003dd0 tx=0x7ff440009040).stop 2023-11-28T02:52:53.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 msgr2=0x7ff4500c7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7ff438009eb0 tx=0x7ff438017000).stop 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 shutdown_connections 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff43005f150 0x7ff430061610 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff450158160 0x7ff4500c7530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4500a8280 0x7ff45014ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 --2- 172.21.15.73:0/3297196033 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff45014e600 0x7ff4500c7040 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 >> 172.21.15.73:0/3297196033 conn(0x7ff450062700 msgr2=0x7ff45014ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 shutdown_connections 2023-11-28T02:52:53.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.435+0000 7ff454e4f700 1 -- 172.21.15.73:0/3297196033 wait complete. 2023-11-28T02:52:53.448 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574853 for osd.6 2023-11-28T02:52:53.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-11-28T02:52:53.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.561+0000 7f635cc86700 1 Processor -- start 2023-11-28T02:52:53.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.561+0000 7f635cc86700 1 -- start start 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f635cc86700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 0x7f63580c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f635cc86700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6358158320 0x7f63581587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f635cc86700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6358158cc0 con 0x7f6358098d10 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f635cc86700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63580c72d0 con 0x7f6358158320 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f635cc86700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6358064570 con 0x7f63580c6190 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f6356d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6358158320 0x7f63581587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f6356d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6358158320 0x7f63581587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44806/0 (socket says 172.21.15.73:44806) 2023-11-28T02:52:53.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f6356d9d700 1 -- 172.21.15.73:0/1756344358 learned_addr learned my addr 172.21.15.73:0/1756344358 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.562+0000 7f6355d9b700 1 --2- 172.21.15.73:0/1756344358 >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 0x7f63580c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6356d9d700 1 -- 172.21.15.73:0/1756344358 >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 msgr2=0x7f63580c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6356d9d700 1 --2- 172.21.15.73:0/1756344358 >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 0x7f63580c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6356d9d700 1 -- 172.21.15.73:0/1756344358 >> v1:172.21.15.150:6789/0 conn(0x7f6358098d10 legacy=0x7f6358063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6356d9d700 1 -- 172.21.15.73:0/1756344358 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6358154bc0 con 0x7f6358158320 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6355d9b700 1 --2- 172.21.15.73:0/1756344358 >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 0x7f63580c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f6356d9d700 1 --2- 172.21.15.73:0/1756344358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358158320 0x7f63581587d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f63580c5b40 tx=0x7f63500090f0).ready entity=mon.0 client_cookie=b5eabeb9b577178f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635559a700 1 -- 172.21.15.73:0/1756344358 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000b5a0 con 0x7f6358158320 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635559a700 1 -- 172.21.15.73:0/1756344358 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f635000b700 con 0x7f6358158320 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635559a700 1 -- 172.21.15.73:0/1756344358 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f635000ba80 con 0x7f6358158320 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 -- 172.21.15.73:0/1756344358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358158320 msgr2=0x7f63581587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 --2- 172.21.15.73:0/1756344358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358158320 0x7f63581587d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f63580c5b40 tx=0x7f63500090f0).stop 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 -- 172.21.15.73:0/1756344358 shutdown_connections 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 --2- 172.21.15.73:0/1756344358 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358158320 0x7f63581587d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 --2- 172.21.15.73:0/1756344358 >> v2:172.21.15.150:3300/0 conn(0x7f63580c6190 0x7f63580c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 -- 172.21.15.73:0/1756344358 >> 172.21.15.73:0/1756344358 conn(0x7f63580cb310 msgr2=0x7f63580cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 -- 172.21.15.73:0/1756344358 shutdown_connections 2023-11-28T02:52:53.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.563+0000 7f635cc86700 1 -- 172.21.15.73:0/1756344358 wait complete. 2023-11-28T02:52:53.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.564+0000 7f635cc86700 1 Processor -- start 2023-11-28T02:52:53.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 -- start start 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 0x7f63580b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 0x7f63580c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6358155d20 con 0x7f6358098d10 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63580b0fa0 con 0x7f6358158320 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635cc86700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63580b1280 con 0x7f63580c6190 2023-11-28T02:52:53.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44822/0 (socket says 172.21.15.73:44822) 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f635659c700 1 -- 172.21.15.73:0/4152254719 learned_addr learned my addr 172.21.15.73:0/4152254719 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f6355d9b700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 0x7f63580b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.565+0000 7f6356d9d700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 0x7f63580c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 -- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 msgr2=0x7f63580b0780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 0x7f63580b0780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 -- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 msgr2=0x7f63580c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 0x7f63580c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 -- 172.21.15.73:0/4152254719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6350009370 con 0x7f6358098d10 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f6355d9b700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 0x7f63580b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f6356d9d700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 0x7f63580c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635659c700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f634800c810 tx=0x7f63480025e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634800a580 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f634800a700 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f634800aac0 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6358155f70 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.566+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63580b0c70 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.567+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f635806c860 con 0x7f6358098d10 2023-11-28T02:52:53.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.567+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f634801b400 con 0x7f6358098d10 2023-11-28T02:52:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.567+0000 7f634e7fc700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 0x7f6338061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.568+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f634807a190 con 0x7f6358098d10 2023-11-28T02:52:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.568+0000 7f6355d9b700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 0x7f6338061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.568+0000 7f6355d9b700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 0x7f6338061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f635814fba0 tx=0x7f6340008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.572+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f634804b400 con 0x7f6358098d10 2023-11-28T02:52:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.738+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f63580b1440 con 0x7f6358098d10 2023-11-28T02:52:53.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.739+0000 7f634e7fc700 1 -- 172.21.15.73:0/4152254719 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f634804ea20 con 0x7f6358098d10 2023-11-28T02:52:53.740 INFO:teuthology.orchestra.run.smithi073.stdout:55834574854 2023-11-28T02:52:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 msgr2=0x7f6338061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 0x7f6338061610 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f635814fba0 tx=0x7f6340008040).stop 2023-11-28T02:52:53.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 msgr2=0x7f63580ae3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f634800c810 tx=0x7f63480025e0).stop 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 shutdown_connections 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f633805f150 0x7f6338061610 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6358158320 0x7f63580c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6358098d10 0x7f63580ae3a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 --2- 172.21.15.73:0/4152254719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63580c6190 0x7f63580b0780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 >> 172.21.15.73:0/4152254719 conn(0x7f63580cb310 msgr2=0x7f63580c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 shutdown_connections 2023-11-28T02:52:53.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.741+0000 7f635cc86700 1 -- 172.21.15.73:0/4152254719 wait complete. 2023-11-28T02:52:53.754 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-11-28T02:52:53.754 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T02:52:53.754 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:52:53.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.866+0000 7f5edec26700 1 Processor -- start 2023-11-28T02:52:53.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.866+0000 7f5edec26700 1 -- start start 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edec26700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80cae90 0x7f5ed80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edec26700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ed80ca180 0x7f5ed80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edec26700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ed80a9a90 con 0x7f5ed80caab0 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edec26700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ed80c1490 con 0x7f5ed80ca180 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edec26700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed8154ce0 con 0x7f5ed80cae90 2023-11-28T02:52:53.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edd1c3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ed80ca180 0x7f5ed80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edd1c3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5ed80ca180 0x7f5ed80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44832/0 (socket says 172.21.15.73:44832) 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5edd1c3700 1 -- 172.21.15.73:0/3514381859 learned_addr learned my addr 172.21.15.73:0/3514381859 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.867+0000 7f5ed7fff700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80cae90 0x7f5ed80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edd1c3700 1 -- 172.21.15.73:0/3514381859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80cae90 msgr2=0x7f5ed80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edd1c3700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80cae90 0x7f5ed80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edd1c3700 1 -- 172.21.15.73:0/3514381859 >> v1:172.21.15.150:6789/0 conn(0x7f5ed80caab0 legacy=0x7f5ed80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edd1c3700 1 -- 172.21.15.73:0/3514381859 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed80aab80 con 0x7f5ed80ca180 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edd1c3700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80ca180 0x7f5ed80c0fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5ed0006fd0 tx=0x7f5ed00090f0).ready entity=mon.0 client_cookie=29088a833b5bcd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5ed77fe700 1 -- 172.21.15.73:0/3514381859 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed000b600 con 0x7f5ed80ca180 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5ed77fe700 1 -- 172.21.15.73:0/3514381859 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ed000b760 con 0x7f5ed80ca180 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5ed77fe700 1 -- 172.21.15.73:0/3514381859 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ed000bae0 con 0x7f5ed80ca180 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 -- 172.21.15.73:0/3514381859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80ca180 msgr2=0x7f5ed80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80ca180 0x7f5ed80c0fa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f5ed0006fd0 tx=0x7f5ed00090f0).stop 2023-11-28T02:52:53.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 -- 172.21.15.73:0/3514381859 shutdown_connections 2023-11-28T02:52:53.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80ca180 0x7f5ed80c0fa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 --2- 172.21.15.73:0/3514381859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80cae90 0x7f5ed80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 -- 172.21.15.73:0/3514381859 >> 172.21.15.73:0/3514381859 conn(0x7f5ed8094920 msgr2=0x7f5ed806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:53.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 -- 172.21.15.73:0/3514381859 shutdown_connections 2023-11-28T02:52:53.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.868+0000 7f5edec26700 1 -- 172.21.15.73:0/3514381859 wait complete. 2023-11-28T02:52:53.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.869+0000 7f5edec26700 1 Processor -- start 2023-11-28T02:52:53.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 -- start start 2023-11-28T02:52:53.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 0x7f5ed8146d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 0x7f5ed80aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80c66a0 con 0x7f5ed80caab0 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80ac710 con 0x7f5ed80cae90 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5edec26700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ed80ac9f0 con 0x7f5ed80ca180 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5ed7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5ed7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44844/0 (socket says 172.21.15.73:44844) 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.870+0000 7f5ed7fff700 1 -- 172.21.15.73:0/1329848296 learned_addr learned my addr 172.21.15.73:0/1329848296 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edd1c3700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 0x7f5ed80aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edc9c2700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 0x7f5ed8146d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 -- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 msgr2=0x7f5ed8146d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 0x7f5ed8146d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 -- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 msgr2=0x7f5ed80aeb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:53.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 0x7f5ed80aeb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 -- 172.21.15.73:0/1329848296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ed0009370 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edc9c2700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 0x7f5ed8146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edd1c3700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 0x7f5ed80aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed7fff700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f5ecc00d890 tx=0x7f5ecc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc0125f0 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ecc012750 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ed80accd0 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ecc012ad0 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.871+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ed8153540 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.872+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ed80c6810 con 0x7f5ed80caab0 2023-11-28T02:52:53.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.872+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5ecc013480 con 0x7f5ed80caab0 2023-11-28T02:52:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.873+0000 7f5ed4ff9700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 0x7f5ec40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.873+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5ecc079f40 con 0x7f5ed80caab0 2023-11-28T02:52:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.873+0000 7f5edc9c2700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 0x7f5ec40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.873+0000 7f5edc9c2700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 0x7f5ec40615e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5ec8007fd0 tx=0x7f5ec8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:53.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:53.877+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5ecc04b1b0 con 0x7f5ed80caab0 2023-11-28T02:52:54.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.038+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5ed806c860 con 0x7f5ec405f120 2023-11-28T02:52:54.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.047+0000 7f5ed4ff9700 1 -- 172.21.15.73:0/1329848296 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336341 (secure 0 0 0) 0x7f5ed806c860 con 0x7f5ec405f120 2023-11-28T02:52:54.052 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 msgr2=0x7f5ec40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 0x7f5ec40615e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7f5ec8007fd0 tx=0x7f5ec8009040).stop 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 msgr2=0x7f5ed80abce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7f5ecc00d890 tx=0x7f5ecc008040).stop 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 shutdown_connections 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5ec405f120 0x7f5ec40615e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ed80cae90 0x7f5ed80aeb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ed80ca180 0x7f5ed8146d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 --2- 172.21.15.73:0/1329848296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ed80caab0 0x7f5ed80abce0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 >> 172.21.15.73:0/1329848296 conn(0x7f5ed8094920 msgr2=0x7f5ed80aa440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 shutdown_connections 2023-11-28T02:52:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.050+0000 7f5edec26700 1 -- 172.21.15.73:0/1329848296 wait complete. 2023-11-28T02:52:54.057 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:52:54.063 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T02:52:53.391060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378624,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596096,"statfs":{"total":773094113280,"available":770658402304,"internally_reserved":0,"allocated":1900544,"data_stored":687340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001465"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470198+0000","last_change":"2023-11-28T02:52:44.467253+0000","last_active":"2023-11-28T02:52:44.470198+0000","last_peered":"2023-11-28T02:52:44.470198+0000","last_clean":"2023-11-28T02:52:44.470198+0000","last_became_active":"2023-11-28T02:52:44.466977+0000","last_became_peered":"2023-11-28T02:52:44.466977+0000","last_unstale":"2023-11-28T02:52:44.470198+0000","last_undegraded":"2023-11-28T02:52:44.470198+0000","last_fullsized":"2023-11-28T02:52:44.470198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455029+0000","last_change":"2023-11-28T02:52:44.098094+0000","last_active":"2023-11-28T02:52:44.455029+0000","last_peered":"2023-11-28T02:52:44.455029+0000","last_clean":"2023-11-28T02:52:44.455029+0000","last_became_active":"2023-11-28T02:52:44.097826+0000","last_became_peered":"2023-11-28T02:52:44.097826+0000","last_unstale":"2023-11-28T02:52:44.455029+0000","last_undegraded":"2023-11-28T02:52:44.455029+0000","last_fullsized":"2023-11-28T02:52:44.455029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479071+0000","last_change":"2023-11-28T02:52:44.478096+0000","last_active":"2023-11-28T02:52:44.479071+0000","last_peered":"2023-11-28T02:52:44.479071+0000","last_clean":"2023-11-28T02:52:44.479071+0000","last_became_active":"2023-11-28T02:52:44.477817+0000","last_became_peered":"2023-11-28T02:52:44.477817+0000","last_unstale":"2023-11-28T02:52:44.479071+0000","last_undegraded":"2023-11-28T02:52:44.479071+0000","last_fullsized":"2023-11-28T02:52:44.479071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462484+0000","last_change":"2023-11-28T02:52:44.087580+0000","last_active":"2023-11-28T02:52:44.462484+0000","last_peered":"2023-11-28T02:52:44.462484+0000","last_clean":"2023-11-28T02:52:44.462484+0000","last_became_active":"2023-11-28T02:52:44.087310+0000","last_became_peered":"2023-11-28T02:52:44.087310+0000","last_unstale":"2023-11-28T02:52:44.462484+0000","last_undegraded":"2023-11-28T02:52:44.462484+0000","last_fullsized":"2023-11-28T02:52:44.462484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472269+0000","last_change":"2023-11-28T02:52:44.466469+0000","last_active":"2023-11-28T02:52:44.472269+0000","last_peered":"2023-11-28T02:52:44.472269+0000","last_clean":"2023-11-28T02:52:44.472269+0000","last_became_active":"2023-11-28T02:52:44.465822+0000","last_became_peered":"2023-11-28T02:52:44.465822+0000","last_unstale":"2023-11-28T02:52:44.472269+0000","last_undegraded":"2023-11-28T02:52:44.472269+0000","last_fullsized":"2023-11-28T02:52:44.472269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451606+0000","last_change":"2023-11-28T02:52:44.097696+0000","last_active":"2023-11-28T02:52:44.451606+0000","last_peered":"2023-11-28T02:52:44.451606+0000","last_clean":"2023-11-28T02:52:44.451606+0000","last_became_active":"2023-11-28T02:52:44.097423+0000","last_became_peered":"2023-11-28T02:52:44.097423+0000","last_unstale":"2023-11-28T02:52:44.451606+0000","last_undegraded":"2023-11-28T02:52:44.451606+0000","last_fullsized":"2023-11-28T02:52:44.451606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469722+0000","last_change":"2023-11-28T02:52:44.465794+0000","last_active":"2023-11-28T02:52:44.469722+0000","last_peered":"2023-11-28T02:52:44.469722+0000","last_clean":"2023-11-28T02:52:44.469722+0000","last_became_active":"2023-11-28T02:52:44.465545+0000","last_became_peered":"2023-11-28T02:52:44.465545+0000","last_unstale":"2023-11-28T02:52:44.469722+0000","last_undegraded":"2023-11-28T02:52:44.469722+0000","last_fullsized":"2023-11-28T02:52:44.469722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450984+0000","last_change":"2023-11-28T02:52:44.099493+0000","last_active":"2023-11-28T02:52:44.450984+0000","last_peered":"2023-11-28T02:52:44.450984+0000","last_clean":"2023-11-28T02:52:44.450984+0000","last_became_active":"2023-11-28T02:52:44.099248+0000","last_became_peered":"2023-11-28T02:52:44.099248+0000","last_unstale":"2023-11-28T02:52:44.450984+0000","last_undegraded":"2023-11-28T02:52:44.450984+0000","last_fullsized":"2023-11-28T02:52:44.450984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479353+0000","last_change":"2023-11-28T02:52:44.477723+0000","last_active":"2023-11-28T02:52:44.479353+0000","last_peered":"2023-11-28T02:52:44.479353+0000","last_clean":"2023-11-28T02:52:44.479353+0000","last_became_active":"2023-11-28T02:52:44.477430+0000","last_became_peered":"2023-11-28T02:52:44.477430+0000","last_unstale":"2023-11-28T02:52:44.479353+0000","last_undegraded":"2023-11-28T02:52:44.479353+0000","last_fullsized":"2023-11-28T02:52:44.479353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462583+0000","last_change":"2023-11-28T02:52:44.102758+0000","last_active":"2023-11-28T02:52:44.462583+0000","last_peered":"2023-11-28T02:52:44.462583+0000","last_clean":"2023-11-28T02:52:44.462583+0000","last_became_active":"2023-11-28T02:52:44.102441+0000","last_became_peered":"2023-11-28T02:52:44.102441+0000","last_unstale":"2023-11-28T02:52:44.462583+0000","last_undegraded":"2023-11-28T02:52:44.462583+0000","last_fullsized":"2023-11-28T02:52:44.462583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468903+0000","last_change":"2023-11-28T02:52:44.466352+0000","last_active":"2023-11-28T02:52:44.468903+0000","last_peered":"2023-11-28T02:52:44.468903+0000","last_clean":"2023-11-28T02:52:44.468903+0000","last_became_active":"2023-11-28T02:52:44.466023+0000","last_became_peered":"2023-11-28T02:52:44.466023+0000","last_unstale":"2023-11-28T02:52:44.468903+0000","last_undegraded":"2023-11-28T02:52:44.468903+0000","last_fullsized":"2023-11-28T02:52:44.468903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479820+0000","last_change":"2023-11-28T02:52:44.100407+0000","last_active":"2023-11-28T02:52:44.479820+0000","last_peered":"2023-11-28T02:52:44.479820+0000","last_clean":"2023-11-28T02:52:44.479820+0000","last_became_active":"2023-11-28T02:52:44.100184+0000","last_became_peered":"2023-11-28T02:52:44.100184+0000","last_unstale":"2023-11-28T02:52:44.479820+0000","last_undegraded":"2023-11-28T02:52:44.479820+0000","last_fullsized":"2023-11-28T02:52:44.479820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469727+0000","last_change":"2023-11-28T02:52:44.467942+0000","last_active":"2023-11-28T02:52:44.469727+0000","last_peered":"2023-11-28T02:52:44.469727+0000","last_clean":"2023-11-28T02:52:44.469727+0000","last_became_active":"2023-11-28T02:52:44.467659+0000","last_became_peered":"2023-11-28T02:52:44.467659+0000","last_unstale":"2023-11-28T02:52:44.469727+0000","last_undegraded":"2023-11-28T02:52:44.469727+0000","last_fullsized":"2023-11-28T02:52:44.469727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472685+0000","last_change":"2023-11-28T02:52:44.094177+0000","last_active":"2023-11-28T02:52:44.472685+0000","last_peered":"2023-11-28T02:52:44.472685+0000","last_clean":"2023-11-28T02:52:44.472685+0000","last_became_active":"2023-11-28T02:52:44.093901+0000","last_became_peered":"2023-11-28T02:52:44.093901+0000","last_unstale":"2023-11-28T02:52:44.472685+0000","last_undegraded":"2023-11-28T02:52:44.472685+0000","last_fullsized":"2023-11-28T02:52:44.472685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470448+0000","last_change":"2023-11-28T02:52:44.467833+0000","last_active":"2023-11-28T02:52:44.470448+0000","last_peered":"2023-11-28T02:52:44.470448+0000","last_clean":"2023-11-28T02:52:44.470448+0000","last_became_active":"2023-11-28T02:52:44.467517+0000","last_became_peered":"2023-11-28T02:52:44.467517+0000","last_unstale":"2023-11-28T02:52:44.470448+0000","last_undegraded":"2023-11-28T02:52:44.470448+0000","last_fullsized":"2023-11-28T02:52:44.470448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485164+0000","last_change":"2023-11-28T02:52:44.104338+0000","last_active":"2023-11-28T02:52:44.485164+0000","last_peered":"2023-11-28T02:52:44.485164+0000","last_clean":"2023-11-28T02:52:44.485164+0000","last_became_active":"2023-11-28T02:52:44.104073+0000","last_became_peered":"2023-11-28T02:52:44.104073+0000","last_unstale":"2023-11-28T02:52:44.485164+0000","last_undegraded":"2023-11-28T02:52:44.485164+0000","last_fullsized":"2023-11-28T02:52:44.485164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477889+0000","last_change":"2023-11-28T02:52:44.476733+0000","last_active":"2023-11-28T02:52:44.477889+0000","last_peered":"2023-11-28T02:52:44.477889+0000","last_clean":"2023-11-28T02:52:44.477889+0000","last_became_active":"2023-11-28T02:52:44.476400+0000","last_became_peered":"2023-11-28T02:52:44.476400+0000","last_unstale":"2023-11-28T02:52:44.477889+0000","last_undegraded":"2023-11-28T02:52:44.477889+0000","last_fullsized":"2023-11-28T02:52:44.477889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453202+0000","last_change":"2023-11-28T02:52:44.096921+0000","last_active":"2023-11-28T02:52:44.453202+0000","last_peered":"2023-11-28T02:52:44.453202+0000","last_clean":"2023-11-28T02:52:44.453202+0000","last_became_active":"2023-11-28T02:52:44.096687+0000","last_became_peered":"2023-11-28T02:52:44.096687+0000","last_unstale":"2023-11-28T02:52:44.453202+0000","last_undegraded":"2023-11-28T02:52:44.453202+0000","last_fullsized":"2023-11-28T02:52:44.453202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466146+0000","last_change":"2023-11-28T02:52:44.461227+0000","last_active":"2023-11-28T02:52:44.466146+0000","last_peered":"2023-11-28T02:52:44.466146+0000","last_clean":"2023-11-28T02:52:44.466146+0000","last_became_active":"2023-11-28T02:52:44.460957+0000","last_became_peered":"2023-11-28T02:52:44.460957+0000","last_unstale":"2023-11-28T02:52:44.466146+0000","last_undegraded":"2023-11-28T02:52:44.466146+0000","last_fullsized":"2023-11-28T02:52:44.466146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475829+0000","last_change":"2023-11-28T02:52:44.106278+0000","last_active":"2023-11-28T02:52:44.475829+0000","last_peered":"2023-11-28T02:52:44.475829+0000","last_clean":"2023-11-28T02:52:44.475829+0000","last_became_active":"2023-11-28T02:52:44.106026+0000","last_became_peered":"2023-11-28T02:52:44.106026+0000","last_unstale":"2023-11-28T02:52:44.475829+0000","last_undegraded":"2023-11-28T02:52:44.475829+0000","last_fullsized":"2023-11-28T02:52:44.475829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480394+0000","last_change":"2023-11-28T02:52:44.479895+0000","last_active":"2023-11-28T02:52:44.480394+0000","last_peered":"2023-11-28T02:52:44.480394+0000","last_clean":"2023-11-28T02:52:44.480394+0000","last_became_active":"2023-11-28T02:52:44.479685+0000","last_became_peered":"2023-11-28T02:52:44.479685+0000","last_unstale":"2023-11-28T02:52:44.480394+0000","last_undegraded":"2023-11-28T02:52:44.480394+0000","last_fullsized":"2023-11-28T02:52:44.480394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470565+0000","last_change":"2023-11-28T02:52:44.097431+0000","last_active":"2023-11-28T02:52:44.470565+0000","last_peered":"2023-11-28T02:52:44.470565+0000","last_clean":"2023-11-28T02:52:44.470565+0000","last_became_active":"2023-11-28T02:52:44.097043+0000","last_became_peered":"2023-11-28T02:52:44.097043+0000","last_unstale":"2023-11-28T02:52:44.470565+0000","last_undegraded":"2023-11-28T02:52:44.470565+0000","last_fullsized":"2023-11-28T02:52:44.470565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478662+0000","last_change":"2023-11-28T02:52:44.477784+0000","last_active":"2023-11-28T02:52:44.478662+0000","last_peered":"2023-11-28T02:52:44.478662+0000","last_clean":"2023-11-28T02:52:44.478662+0000","last_became_active":"2023-11-28T02:52:44.477515+0000","last_became_peered":"2023-11-28T02:52:44.477515+0000","last_unstale":"2023-11-28T02:52:44.478662+0000","last_undegraded":"2023-11-28T02:52:44.478662+0000","last_fullsized":"2023-11-28T02:52:44.478662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485148+0000","last_change":"2023-11-28T02:52:44.104964+0000","last_active":"2023-11-28T02:52:44.485148+0000","last_peered":"2023-11-28T02:52:44.485148+0000","last_clean":"2023-11-28T02:52:44.485148+0000","last_became_active":"2023-11-28T02:52:44.104688+0000","last_became_peered":"2023-11-28T02:52:44.104688+0000","last_unstale":"2023-11-28T02:52:44.485148+0000","last_undegraded":"2023-11-28T02:52:44.485148+0000","last_fullsized":"2023-11-28T02:52:44.485148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469058+0000","last_change":"2023-11-28T02:52:44.465011+0000","last_active":"2023-11-28T02:52:44.469058+0000","last_peered":"2023-11-28T02:52:44.469058+0000","last_clean":"2023-11-28T02:52:44.469058+0000","last_became_active":"2023-11-28T02:52:44.464725+0000","last_became_peered":"2023-11-28T02:52:44.464725+0000","last_unstale":"2023-11-28T02:52:44.469058+0000","last_undegraded":"2023-11-28T02:52:44.469058+0000","last_fullsized":"2023-11-28T02:52:44.469058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.482297+0000","last_change":"2023-11-28T02:52:44.104530+0000","last_active":"2023-11-28T02:52:44.482297+0000","last_peered":"2023-11-28T02:52:44.482297+0000","last_clean":"2023-11-28T02:52:44.482297+0000","last_became_active":"2023-11-28T02:52:44.104035+0000","last_became_peered":"2023-11-28T02:52:44.104035+0000","last_unstale":"2023-11-28T02:52:44.482297+0000","last_undegraded":"2023-11-28T02:52:44.482297+0000","last_fullsized":"2023-11-28T02:52:44.482297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474584+0000","last_change":"2023-11-28T02:52:44.469216+0000","last_active":"2023-11-28T02:52:44.474584+0000","last_peered":"2023-11-28T02:52:44.474584+0000","last_clean":"2023-11-28T02:52:44.474584+0000","last_became_active":"2023-11-28T02:52:44.467199+0000","last_became_peered":"2023-11-28T02:52:44.467199+0000","last_unstale":"2023-11-28T02:52:44.474584+0000","last_undegraded":"2023-11-28T02:52:44.474584+0000","last_fullsized":"2023-11-28T02:52:44.474584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474622+0000","last_change":"2023-11-28T02:52:44.099749+0000","last_active":"2023-11-28T02:52:44.474622+0000","last_peered":"2023-11-28T02:52:44.474622+0000","last_clean":"2023-11-28T02:52:44.474622+0000","last_became_active":"2023-11-28T02:52:44.099525+0000","last_became_peered":"2023-11-28T02:52:44.099525+0000","last_unstale":"2023-11-28T02:52:44.474622+0000","last_undegraded":"2023-11-28T02:52:44.474622+0000","last_fullsized":"2023-11-28T02:52:44.474622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470191+0000","last_change":"2023-11-28T02:52:44.467373+0000","last_active":"2023-11-28T02:52:44.470191+0000","last_peered":"2023-11-28T02:52:44.470191+0000","last_clean":"2023-11-28T02:52:44.470191+0000","last_became_active":"2023-11-28T02:52:44.464529+0000","last_became_peered":"2023-11-28T02:52:44.464529+0000","last_unstale":"2023-11-28T02:52:44.470191+0000","last_undegraded":"2023-11-28T02:52:44.470191+0000","last_fullsized":"2023-11-28T02:52:44.470191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457475+0000","last_change":"2023-11-28T02:52:44.099774+0000","last_active":"2023-11-28T02:52:44.457475+0000","last_peered":"2023-11-28T02:52:44.457475+0000","last_clean":"2023-11-28T02:52:44.457475+0000","last_became_active":"2023-11-28T02:52:44.099484+0000","last_became_peered":"2023-11-28T02:52:44.099484+0000","last_unstale":"2023-11-28T02:52:44.457475+0000","last_undegraded":"2023-11-28T02:52:44.457475+0000","last_fullsized":"2023-11-28T02:52:44.457475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469631+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.469631+0000","last_peered":"2023-11-28T02:52:44.469631+0000","last_clean":"2023-11-28T02:52:44.469631+0000","last_became_active":"2023-11-28T02:52:44.460712+0000","last_became_peered":"2023-11-28T02:52:44.460712+0000","last_unstale":"2023-11-28T02:52:44.469631+0000","last_undegraded":"2023-11-28T02:52:44.469631+0000","last_fullsized":"2023-11-28T02:52:44.469631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465456+0000","last_change":"2023-11-28T02:52:44.098187+0000","last_active":"2023-11-28T02:52:44.465456+0000","last_peered":"2023-11-28T02:52:44.465456+0000","last_clean":"2023-11-28T02:52:44.465456+0000","last_became_active":"2023-11-28T02:52:44.097937+0000","last_became_peered":"2023-11-28T02:52:44.097937+0000","last_unstale":"2023-11-28T02:52:44.465456+0000","last_undegraded":"2023-11-28T02:52:44.465456+0000","last_fullsized":"2023-11-28T02:52:44.465456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464006+0000","last_change":"2023-11-28T02:52:44.458708+0000","last_active":"2023-11-28T02:52:44.464006+0000","last_peered":"2023-11-28T02:52:44.464006+0000","last_clean":"2023-11-28T02:52:44.464006+0000","last_became_active":"2023-11-28T02:52:44.458372+0000","last_became_peered":"2023-11-28T02:52:44.458372+0000","last_unstale":"2023-11-28T02:52:44.464006+0000","last_undegraded":"2023-11-28T02:52:44.464006+0000","last_fullsized":"2023-11-28T02:52:44.464006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463865+0000","last_change":"2023-11-28T02:52:44.098081+0000","last_active":"2023-11-28T02:52:44.463865+0000","last_peered":"2023-11-28T02:52:44.463865+0000","last_clean":"2023-11-28T02:52:44.463865+0000","last_became_active":"2023-11-28T02:52:44.097763+0000","last_became_peered":"2023-11-28T02:52:44.097763+0000","last_unstale":"2023-11-28T02:52:44.463865+0000","last_undegraded":"2023-11-28T02:52:44.463865+0000","last_fullsized":"2023-11-28T02:52:44.463865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468716+0000","last_change":"2023-11-28T02:52:44.464973+0000","last_active":"2023-11-28T02:52:44.468716+0000","last_peered":"2023-11-28T02:52:44.468716+0000","last_clean":"2023-11-28T02:52:44.468716+0000","last_became_active":"2023-11-28T02:52:44.464692+0000","last_became_peered":"2023-11-28T02:52:44.464692+0000","last_unstale":"2023-11-28T02:52:44.468716+0000","last_undegraded":"2023-11-28T02:52:44.468716+0000","last_fullsized":"2023-11-28T02:52:44.468716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454758+0000","last_change":"2023-11-28T02:52:44.096316+0000","last_active":"2023-11-28T02:52:44.454758+0000","last_peered":"2023-11-28T02:52:44.454758+0000","last_clean":"2023-11-28T02:52:44.454758+0000","last_became_active":"2023-11-28T02:52:44.090973+0000","last_became_peered":"2023-11-28T02:52:44.090973+0000","last_unstale":"2023-11-28T02:52:44.454758+0000","last_undegraded":"2023-11-28T02:52:44.454758+0000","last_fullsized":"2023-11-28T02:52:44.454758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467774+0000","last_change":"2023-11-28T02:52:44.463807+0000","last_active":"2023-11-28T02:52:44.467774+0000","last_peered":"2023-11-28T02:52:44.467774+0000","last_clean":"2023-11-28T02:52:44.467774+0000","last_became_active":"2023-11-28T02:52:44.463532+0000","last_became_peered":"2023-11-28T02:52:44.463532+0000","last_unstale":"2023-11-28T02:52:44.467774+0000","last_undegraded":"2023-11-28T02:52:44.467774+0000","last_fullsized":"2023-11-28T02:52:44.467774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451394+0000","last_change":"2023-11-28T02:52:44.099397+0000","last_active":"2023-11-28T02:52:44.451394+0000","last_peered":"2023-11-28T02:52:44.451394+0000","last_clean":"2023-11-28T02:52:44.451394+0000","last_became_active":"2023-11-28T02:52:44.099225+0000","last_became_peered":"2023-11-28T02:52:44.099225+0000","last_unstale":"2023-11-28T02:52:44.451394+0000","last_undegraded":"2023-11-28T02:52:44.451394+0000","last_fullsized":"2023-11-28T02:52:44.451394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464258+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:52:44.464258+0000","last_peered":"2023-11-28T02:52:44.464258+0000","last_clean":"2023-11-28T02:52:44.464258+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:52:44.464258+0000","last_undegraded":"2023-11-28T02:52:44.464258+0000","last_fullsized":"2023-11-28T02:52:44.464258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461086+0000","last_change":"2023-11-28T02:52:44.099940+0000","last_active":"2023-11-28T02:52:44.461086+0000","last_peered":"2023-11-28T02:52:44.461086+0000","last_clean":"2023-11-28T02:52:44.461086+0000","last_became_active":"2023-11-28T02:52:44.099735+0000","last_became_peered":"2023-11-28T02:52:44.099735+0000","last_unstale":"2023-11-28T02:52:44.461086+0000","last_undegraded":"2023-11-28T02:52:44.461086+0000","last_fullsized":"2023-11-28T02:52:44.461086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473095+0000","last_change":"2023-11-28T02:52:44.472735+0000","last_active":"2023-11-28T02:52:44.473095+0000","last_peered":"2023-11-28T02:52:44.473095+0000","last_clean":"2023-11-28T02:52:44.473095+0000","last_became_active":"2023-11-28T02:52:44.472418+0000","last_became_peered":"2023-11-28T02:52:44.472418+0000","last_unstale":"2023-11-28T02:52:44.473095+0000","last_undegraded":"2023-11-28T02:52:44.473095+0000","last_fullsized":"2023-11-28T02:52:44.473095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459288+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:52:44.459288+0000","last_peered":"2023-11-28T02:52:44.459288+0000","last_clean":"2023-11-28T02:52:44.459288+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:52:44.459288+0000","last_undegraded":"2023-11-28T02:52:44.459288+0000","last_fullsized":"2023-11-28T02:52:44.459288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455813+0000","last_change":"2023-11-28T02:52:44.100602+0000","last_active":"2023-11-28T02:52:44.455813+0000","last_peered":"2023-11-28T02:52:44.455813+0000","last_clean":"2023-11-28T02:52:44.455813+0000","last_became_active":"2023-11-28T02:52:44.100337+0000","last_became_peered":"2023-11-28T02:52:44.100337+0000","last_unstale":"2023-11-28T02:52:44.455813+0000","last_undegraded":"2023-11-28T02:52:44.455813+0000","last_fullsized":"2023-11-28T02:52:44.455813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468871+0000","last_change":"2023-11-28T02:52:44.465676+0000","last_active":"2023-11-28T02:52:44.468871+0000","last_peered":"2023-11-28T02:52:44.468871+0000","last_clean":"2023-11-28T02:52:44.468871+0000","last_became_active":"2023-11-28T02:52:44.465401+0000","last_became_peered":"2023-11-28T02:52:44.465401+0000","last_unstale":"2023-11-28T02:52:44.468871+0000","last_undegraded":"2023-11-28T02:52:44.468871+0000","last_fullsized":"2023-11-28T02:52:44.468871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471562+0000","last_change":"2023-11-28T02:52:44.470158+0000","last_active":"2023-11-28T02:52:44.471562+0000","last_peered":"2023-11-28T02:52:44.471562+0000","last_clean":"2023-11-28T02:52:44.471562+0000","last_became_active":"2023-11-28T02:52:44.469795+0000","last_became_peered":"2023-11-28T02:52:44.469795+0000","last_unstale":"2023-11-28T02:52:44.471562+0000","last_undegraded":"2023-11-28T02:52:44.471562+0000","last_fullsized":"2023-11-28T02:52:44.471562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448512+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:52:44.448512+0000","last_peered":"2023-11-28T02:52:44.448512+0000","last_clean":"2023-11-28T02:52:44.448512+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:52:44.448512+0000","last_undegraded":"2023-11-28T02:52:44.448512+0000","last_fullsized":"2023-11-28T02:52:44.448512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462396+0000","last_change":"2023-11-28T02:52:44.091912+0000","last_active":"2023-11-28T02:52:44.462396+0000","last_peered":"2023-11-28T02:52:44.462396+0000","last_clean":"2023-11-28T02:52:44.462396+0000","last_became_active":"2023-11-28T02:52:44.091630+0000","last_became_peered":"2023-11-28T02:52:44.091630+0000","last_unstale":"2023-11-28T02:52:44.462396+0000","last_undegraded":"2023-11-28T02:52:44.462396+0000","last_fullsized":"2023-11-28T02:52:44.462396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472298+0000","last_change":"2023-11-28T02:52:44.468106+0000","last_active":"2023-11-28T02:52:44.472298+0000","last_peered":"2023-11-28T02:52:44.472298+0000","last_clean":"2023-11-28T02:52:44.472298+0000","last_became_active":"2023-11-28T02:52:44.467833+0000","last_became_peered":"2023-11-28T02:52:44.467833+0000","last_unstale":"2023-11-28T02:52:44.472298+0000","last_undegraded":"2023-11-28T02:52:44.472298+0000","last_fullsized":"2023-11-28T02:52:44.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461405+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:52:44.461405+0000","last_peered":"2023-11-28T02:52:44.461405+0000","last_clean":"2023-11-28T02:52:44.461405+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:52:44.461405+0000","last_undegraded":"2023-11-28T02:52:44.461405+0000","last_fullsized":"2023-11-28T02:52:44.461405+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476368+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:52:44.476368+0000","last_peered":"2023-11-28T02:52:44.476368+0000","last_clean":"2023-11-28T02:52:44.476368+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:52:44.476368+0000","last_undegraded":"2023-11-28T02:52:44.476368+0000","last_fullsized":"2023-11-28T02:52:44.476368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453788+0000","last_change":"2023-11-28T02:52:44.095277+0000","last_active":"2023-11-28T02:52:44.453788+0000","last_peered":"2023-11-28T02:52:44.453788+0000","last_clean":"2023-11-28T02:52:44.453788+0000","last_became_active":"2023-11-28T02:52:44.095021+0000","last_became_peered":"2023-11-28T02:52:44.095021+0000","last_unstale":"2023-11-28T02:52:44.453788+0000","last_undegraded":"2023-11-28T02:52:44.453788+0000","last_fullsized":"2023-11-28T02:52:44.453788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469245+0000","last_change":"2023-11-28T02:52:44.464858+0000","last_active":"2023-11-28T02:52:44.469245+0000","last_peered":"2023-11-28T02:52:44.469245+0000","last_clean":"2023-11-28T02:52:44.469245+0000","last_became_active":"2023-11-28T02:52:44.464352+0000","last_became_peered":"2023-11-28T02:52:44.464352+0000","last_unstale":"2023-11-28T02:52:44.469245+0000","last_undegraded":"2023-11-28T02:52:44.469245+0000","last_fullsized":"2023-11-28T02:52:44.469245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476767+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:52:44.476767+0000","last_peered":"2023-11-28T02:52:44.476767+0000","last_clean":"2023-11-28T02:52:44.476767+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:52:44.476767+0000","last_undegraded":"2023-11-28T02:52:44.476767+0000","last_fullsized":"2023-11-28T02:52:44.476767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450488+0000","last_change":"2023-11-28T02:52:44.105713+0000","last_active":"2023-11-28T02:52:44.450488+0000","last_peered":"2023-11-28T02:52:44.450488+0000","last_clean":"2023-11-28T02:52:44.450488+0000","last_became_active":"2023-11-28T02:52:44.105352+0000","last_became_peered":"2023-11-28T02:52:44.105352+0000","last_unstale":"2023-11-28T02:52:44.450488+0000","last_undegraded":"2023-11-28T02:52:44.450488+0000","last_fullsized":"2023-11-28T02:52:44.450488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474530+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:52:44.474530+0000","last_peered":"2023-11-28T02:52:44.474530+0000","last_clean":"2023-11-28T02:52:44.474530+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:52:44.474530+0000","last_undegraded":"2023-11-28T02:52:44.474530+0000","last_fullsized":"2023-11-28T02:52:44.474530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464373+0000","last_change":"2023-11-28T02:52:44.096162+0000","last_active":"2023-11-28T02:52:44.464373+0000","last_peered":"2023-11-28T02:52:44.464373+0000","last_clean":"2023-11-28T02:52:44.464373+0000","last_became_active":"2023-11-28T02:52:44.095900+0000","last_became_peered":"2023-11-28T02:52:44.095900+0000","last_unstale":"2023-11-28T02:52:44.464373+0000","last_undegraded":"2023-11-28T02:52:44.464373+0000","last_fullsized":"2023-11-28T02:52:44.464373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470164+0000","last_change":"2023-11-28T02:52:44.464449+0000","last_active":"2023-11-28T02:52:44.470164+0000","last_peered":"2023-11-28T02:52:44.470164+0000","last_clean":"2023-11-28T02:52:44.470164+0000","last_became_active":"2023-11-28T02:52:44.464196+0000","last_became_peered":"2023-11-28T02:52:44.464196+0000","last_unstale":"2023-11-28T02:52:44.470164+0000","last_undegraded":"2023-11-28T02:52:44.470164+0000","last_fullsized":"2023-11-28T02:52:44.470164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478810+0000","last_change":"2023-11-28T02:52:44.094471+0000","last_active":"2023-11-28T02:52:44.478810+0000","last_peered":"2023-11-28T02:52:44.478810+0000","last_clean":"2023-11-28T02:52:44.478810+0000","last_became_active":"2023-11-28T02:52:44.094193+0000","last_became_peered":"2023-11-28T02:52:44.094193+0000","last_unstale":"2023-11-28T02:52:44.478810+0000","last_undegraded":"2023-11-28T02:52:44.478810+0000","last_fullsized":"2023-11-28T02:52:44.478810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464206+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:52:44.464206+0000","last_peered":"2023-11-28T02:52:44.464206+0000","last_clean":"2023-11-28T02:52:44.464206+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:52:44.464206+0000","last_undegraded":"2023-11-28T02:52:44.464206+0000","last_fullsized":"2023-11-28T02:52:44.464206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479397+0000","last_change":"2023-11-28T02:52:44.478383+0000","last_active":"2023-11-28T02:52:44.479397+0000","last_peered":"2023-11-28T02:52:44.479397+0000","last_clean":"2023-11-28T02:52:44.479397+0000","last_became_active":"2023-11-28T02:52:44.478122+0000","last_became_peered":"2023-11-28T02:52:44.478122+0000","last_unstale":"2023-11-28T02:52:44.479397+0000","last_undegraded":"2023-11-28T02:52:44.479397+0000","last_fullsized":"2023-11-28T02:52:44.479397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463683+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:52:44.463683+0000","last_peered":"2023-11-28T02:52:44.463683+0000","last_clean":"2023-11-28T02:52:44.463683+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:52:44.463683+0000","last_undegraded":"2023-11-28T02:52:44.463683+0000","last_fullsized":"2023-11-28T02:52:44.463683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451382+0000","last_change":"2023-11-28T02:52:44.099058+0000","last_active":"2023-11-28T02:52:44.451382+0000","last_peered":"2023-11-28T02:52:44.451382+0000","last_clean":"2023-11-28T02:52:44.451382+0000","last_became_active":"2023-11-28T02:52:44.098489+0000","last_became_peered":"2023-11-28T02:52:44.098489+0000","last_unstale":"2023-11-28T02:52:44.451382+0000","last_undegraded":"2023-11-28T02:52:44.451382+0000","last_fullsized":"2023-11-28T02:52:44.451382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475427+0000","last_change":"2023-11-28T02:52:44.473356+0000","last_active":"2023-11-28T02:52:44.475427+0000","last_peered":"2023-11-28T02:52:44.475427+0000","last_clean":"2023-11-28T02:52:44.475427+0000","last_became_active":"2023-11-28T02:52:44.473032+0000","last_became_peered":"2023-11-28T02:52:44.473032+0000","last_unstale":"2023-11-28T02:52:44.475427+0000","last_undegraded":"2023-11-28T02:52:44.475427+0000","last_fullsized":"2023-11-28T02:52:44.475427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471249+0000","last_change":"2023-11-28T02:52:44.469050+0000","last_active":"2023-11-28T02:52:44.471249+0000","last_peered":"2023-11-28T02:52:44.471249+0000","last_clean":"2023-11-28T02:52:44.471249+0000","last_became_active":"2023-11-28T02:52:44.468695+0000","last_became_peered":"2023-11-28T02:52:44.468695+0000","last_unstale":"2023-11-28T02:52:44.471249+0000","last_undegraded":"2023-11-28T02:52:44.471249+0000","last_fullsized":"2023-11-28T02:52:44.471249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452775+0000","last_change":"2023-11-28T02:52:44.105717+0000","last_active":"2023-11-28T02:52:44.452775+0000","last_peered":"2023-11-28T02:52:44.452775+0000","last_clean":"2023-11-28T02:52:44.452775+0000","last_became_active":"2023-11-28T02:52:44.105369+0000","last_became_peered":"2023-11-28T02:52:44.105369+0000","last_unstale":"2023-11-28T02:52:44.452775+0000","last_undegraded":"2023-11-28T02:52:44.452775+0000","last_fullsized":"2023-11-28T02:52:44.452775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465888+0000","last_change":"2023-11-28T02:52:44.463547+0000","last_active":"2023-11-28T02:52:44.465888+0000","last_peered":"2023-11-28T02:52:44.465888+0000","last_clean":"2023-11-28T02:52:44.465888+0000","last_became_active":"2023-11-28T02:52:44.463254+0000","last_became_peered":"2023-11-28T02:52:44.463254+0000","last_unstale":"2023-11-28T02:52:44.465888+0000","last_undegraded":"2023-11-28T02:52:44.465888+0000","last_fullsized":"2023-11-28T02:52:44.465888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457589+0000","last_change":"2023-11-28T02:52:44.102948+0000","last_active":"2023-11-28T02:52:44.457589+0000","last_peered":"2023-11-28T02:52:44.457589+0000","last_clean":"2023-11-28T02:52:44.457589+0000","last_became_active":"2023-11-28T02:52:44.102609+0000","last_became_peered":"2023-11-28T02:52:44.102609+0000","last_unstale":"2023-11-28T02:52:44.457589+0000","last_undegraded":"2023-11-28T02:52:44.457589+0000","last_fullsized":"2023-11-28T02:52:44.457589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465782+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.465782+0000","last_peered":"2023-11-28T02:52:44.465782+0000","last_clean":"2023-11-28T02:52:44.465782+0000","last_became_active":"2023-11-28T02:52:44.460662+0000","last_became_peered":"2023-11-28T02:52:44.460662+0000","last_unstale":"2023-11-28T02:52:44.465782+0000","last_undegraded":"2023-11-28T02:52:44.465782+0000","last_fullsized":"2023-11-28T02:52:44.465782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469497+0000","last_change":"2023-11-28T02:52:44.104319+0000","last_active":"2023-11-28T02:52:44.469497+0000","last_peered":"2023-11-28T02:52:44.469497+0000","last_clean":"2023-11-28T02:52:44.469497+0000","last_became_active":"2023-11-28T02:52:44.104055+0000","last_became_peered":"2023-11-28T02:52:44.104055+0000","last_unstale":"2023-11-28T02:52:44.469497+0000","last_undegraded":"2023-11-28T02:52:44.469497+0000","last_fullsized":"2023-11-28T02:52:44.469497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466536+0000","last_change":"2023-11-28T02:52:44.460800+0000","last_active":"2023-11-28T02:52:44.466536+0000","last_peered":"2023-11-28T02:52:44.466536+0000","last_clean":"2023-11-28T02:52:44.466536+0000","last_became_active":"2023-11-28T02:52:44.460518+0000","last_became_peered":"2023-11-28T02:52:44.460518+0000","last_unstale":"2023-11-28T02:52:44.466536+0000","last_undegraded":"2023-11-28T02:52:44.466536+0000","last_fullsized":"2023-11-28T02:52:44.466536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466150+0000","last_change":"2023-11-28T02:52:44.093970+0000","last_active":"2023-11-28T02:52:44.466150+0000","last_peered":"2023-11-28T02:52:44.466150+0000","last_clean":"2023-11-28T02:52:44.466150+0000","last_became_active":"2023-11-28T02:52:44.093701+0000","last_became_peered":"2023-11-28T02:52:44.093701+0000","last_unstale":"2023-11-28T02:52:44.466150+0000","last_undegraded":"2023-11-28T02:52:44.466150+0000","last_fullsized":"2023-11-28T02:52:44.466150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475064+0000","last_change":"2023-11-28T02:52:44.469053+0000","last_active":"2023-11-28T02:52:44.475064+0000","last_peered":"2023-11-28T02:52:44.475064+0000","last_clean":"2023-11-28T02:52:44.475064+0000","last_became_active":"2023-11-28T02:52:44.468580+0000","last_became_peered":"2023-11-28T02:52:44.468580+0000","last_unstale":"2023-11-28T02:52:44.475064+0000","last_undegraded":"2023-11-28T02:52:44.475064+0000","last_fullsized":"2023-11-28T02:52:44.475064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464009+0000","last_change":"2023-11-28T02:52:44.099449+0000","last_active":"2023-11-28T02:52:44.464009+0000","last_peered":"2023-11-28T02:52:44.464009+0000","last_clean":"2023-11-28T02:52:44.464009+0000","last_became_active":"2023-11-28T02:52:44.099097+0000","last_became_peered":"2023-11-28T02:52:44.099097+0000","last_unstale":"2023-11-28T02:52:44.464009+0000","last_undegraded":"2023-11-28T02:52:44.464009+0000","last_fullsized":"2023-11-28T02:52:44.464009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470464+0000","last_change":"2023-11-28T02:52:44.467418+0000","last_active":"2023-11-28T02:52:44.470464+0000","last_peered":"2023-11-28T02:52:44.470464+0000","last_clean":"2023-11-28T02:52:44.470464+0000","last_became_active":"2023-11-28T02:52:44.465656+0000","last_became_peered":"2023-11-28T02:52:44.465656+0000","last_unstale":"2023-11-28T02:52:44.470464+0000","last_undegraded":"2023-11-28T02:52:44.470464+0000","last_fullsized":"2023-11-28T02:52:44.470464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462541+0000","last_change":"2023-11-28T02:52:44.095567+0000","last_active":"2023-11-28T02:52:44.462541+0000","last_peered":"2023-11-28T02:52:44.462541+0000","last_clean":"2023-11-28T02:52:44.462541+0000","last_became_active":"2023-11-28T02:52:44.095290+0000","last_became_peered":"2023-11-28T02:52:44.095290+0000","last_unstale":"2023-11-28T02:52:44.462541+0000","last_undegraded":"2023-11-28T02:52:44.462541+0000","last_fullsized":"2023-11-28T02:52:44.462541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467572+0000","last_change":"2023-11-28T02:52:44.464051+0000","last_active":"2023-11-28T02:52:44.467572+0000","last_peered":"2023-11-28T02:52:44.467572+0000","last_clean":"2023-11-28T02:52:44.467572+0000","last_became_active":"2023-11-28T02:52:44.463832+0000","last_became_peered":"2023-11-28T02:52:44.463832+0000","last_unstale":"2023-11-28T02:52:44.467572+0000","last_undegraded":"2023-11-28T02:52:44.467572+0000","last_fullsized":"2023-11-28T02:52:44.467572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448090+0000","last_change":"2023-11-28T02:52:44.092817+0000","last_active":"2023-11-28T02:52:44.448090+0000","last_peered":"2023-11-28T02:52:44.448090+0000","last_clean":"2023-11-28T02:52:44.448090+0000","last_became_active":"2023-11-28T02:52:44.092559+0000","last_became_peered":"2023-11-28T02:52:44.092559+0000","last_unstale":"2023-11-28T02:52:44.448090+0000","last_undegraded":"2023-11-28T02:52:44.448090+0000","last_fullsized":"2023-11-28T02:52:44.448090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477920+0000","last_change":"2023-11-28T02:52:44.476928+0000","last_active":"2023-11-28T02:52:44.477920+0000","last_peered":"2023-11-28T02:52:44.477920+0000","last_clean":"2023-11-28T02:52:44.477920+0000","last_became_active":"2023-11-28T02:52:44.476404+0000","last_became_peered":"2023-11-28T02:52:44.476404+0000","last_unstale":"2023-11-28T02:52:44.477920+0000","last_undegraded":"2023-11-28T02:52:44.477920+0000","last_fullsized":"2023-11-28T02:52:44.477920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461903+0000","last_change":"2023-11-28T02:52:44.100532+0000","last_active":"2023-11-28T02:52:44.461903+0000","last_peered":"2023-11-28T02:52:44.461903+0000","last_clean":"2023-11-28T02:52:44.461903+0000","last_became_active":"2023-11-28T02:52:44.100241+0000","last_became_peered":"2023-11-28T02:52:44.100241+0000","last_unstale":"2023-11-28T02:52:44.461903+0000","last_undegraded":"2023-11-28T02:52:44.461903+0000","last_fullsized":"2023-11-28T02:52:44.461903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477206+0000","last_change":"2023-11-28T02:52:44.476740+0000","last_active":"2023-11-28T02:52:44.477206+0000","last_peered":"2023-11-28T02:52:44.477206+0000","last_clean":"2023-11-28T02:52:44.477206+0000","last_became_active":"2023-11-28T02:52:44.476473+0000","last_became_peered":"2023-11-28T02:52:44.476473+0000","last_unstale":"2023-11-28T02:52:44.477206+0000","last_undegraded":"2023-11-28T02:52:44.477206+0000","last_fullsized":"2023-11-28T02:52:44.477206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452872+0000","last_change":"2023-11-28T02:52:44.101558+0000","last_active":"2023-11-28T02:52:44.452872+0000","last_peered":"2023-11-28T02:52:44.452872+0000","last_clean":"2023-11-28T02:52:44.452872+0000","last_became_active":"2023-11-28T02:52:44.101242+0000","last_became_peered":"2023-11-28T02:52:44.101242+0000","last_unstale":"2023-11-28T02:52:44.452872+0000","last_undegraded":"2023-11-28T02:52:44.452872+0000","last_fullsized":"2023-11-28T02:52:44.452872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476230+0000","last_change":"2023-11-28T02:52:44.475794+0000","last_active":"2023-11-28T02:52:44.476230+0000","last_peered":"2023-11-28T02:52:44.476230+0000","last_clean":"2023-11-28T02:52:44.476230+0000","last_became_active":"2023-11-28T02:52:44.475520+0000","last_became_peered":"2023-11-28T02:52:44.475520+0000","last_unstale":"2023-11-28T02:52:44.476230+0000","last_undegraded":"2023-11-28T02:52:44.476230+0000","last_fullsized":"2023-11-28T02:52:44.476230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454590+0000","last_change":"2023-11-28T02:52:44.102703+0000","last_active":"2023-11-28T02:52:44.454590+0000","last_peered":"2023-11-28T02:52:44.454590+0000","last_clean":"2023-11-28T02:52:44.454590+0000","last_became_active":"2023-11-28T02:52:44.102344+0000","last_became_peered":"2023-11-28T02:52:44.102344+0000","last_unstale":"2023-11-28T02:52:44.454590+0000","last_undegraded":"2023-11-28T02:52:44.454590+0000","last_fullsized":"2023-11-28T02:52:44.454590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469219+0000","last_change":"2023-11-28T02:52:44.467020+0000","last_active":"2023-11-28T02:52:44.469219+0000","last_peered":"2023-11-28T02:52:44.469219+0000","last_clean":"2023-11-28T02:52:44.469219+0000","last_became_active":"2023-11-28T02:52:44.466737+0000","last_became_peered":"2023-11-28T02:52:44.466737+0000","last_unstale":"2023-11-28T02:52:44.469219+0000","last_undegraded":"2023-11-28T02:52:44.469219+0000","last_fullsized":"2023-11-28T02:52:44.469219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462456+0000","last_change":"2023-11-28T02:52:44.095605+0000","last_active":"2023-11-28T02:52:44.462456+0000","last_peered":"2023-11-28T02:52:44.462456+0000","last_clean":"2023-11-28T02:52:44.462456+0000","last_became_active":"2023-11-28T02:52:44.093886+0000","last_became_peered":"2023-11-28T02:52:44.093886+0000","last_unstale":"2023-11-28T02:52:44.462456+0000","last_undegraded":"2023-11-28T02:52:44.462456+0000","last_fullsized":"2023-11-28T02:52:44.462456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469773+0000","last_change":"2023-11-28T02:52:44.466446+0000","last_active":"2023-11-28T02:52:44.469773+0000","last_peered":"2023-11-28T02:52:44.469773+0000","last_clean":"2023-11-28T02:52:44.469773+0000","last_became_active":"2023-11-28T02:52:44.466194+0000","last_became_peered":"2023-11-28T02:52:44.466194+0000","last_unstale":"2023-11-28T02:52:44.469773+0000","last_undegraded":"2023-11-28T02:52:44.469773+0000","last_fullsized":"2023-11-28T02:52:44.469773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T02:52:45.452769+0000","last_change":"2023-11-28T02:52:44.089689+0000","last_active":"2023-11-28T02:52:45.452769+0000","last_peered":"2023-11-28T02:52:45.452769+0000","last_clean":"2023-11-28T02:52:45.452769+0000","last_became_active":"2023-11-28T02:52:44.089421+0000","last_became_peered":"2023-11-28T02:52:44.089421+0000","last_unstale":"2023-11-28T02:52:45.452769+0000","last_undegraded":"2023-11-28T02:52:45.452769+0000","last_fullsized":"2023-11-28T02:52:45.452769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472331+0000","last_change":"2023-11-28T02:52:44.468436+0000","last_active":"2023-11-28T02:52:44.472331+0000","last_peered":"2023-11-28T02:52:44.472331+0000","last_clean":"2023-11-28T02:52:44.472331+0000","last_became_active":"2023-11-28T02:52:44.468102+0000","last_became_peered":"2023-11-28T02:52:44.468102+0000","last_unstale":"2023-11-28T02:52:44.472331+0000","last_undegraded":"2023-11-28T02:52:44.472331+0000","last_fullsized":"2023-11-28T02:52:44.472331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457616+0000","last_change":"2023-11-28T02:52:44.104653+0000","last_active":"2023-11-28T02:52:44.457616+0000","last_peered":"2023-11-28T02:52:44.457616+0000","last_clean":"2023-11-28T02:52:44.457616+0000","last_became_active":"2023-11-28T02:52:44.104404+0000","last_became_peered":"2023-11-28T02:52:44.104404+0000","last_unstale":"2023-11-28T02:52:44.457616+0000","last_undegraded":"2023-11-28T02:52:44.457616+0000","last_fullsized":"2023-11-28T02:52:44.457616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464296+0000","last_change":"2023-11-28T02:52:44.460305+0000","last_active":"2023-11-28T02:52:44.464296+0000","last_peered":"2023-11-28T02:52:44.464296+0000","last_clean":"2023-11-28T02:52:44.464296+0000","last_became_active":"2023-11-28T02:52:44.460040+0000","last_became_peered":"2023-11-28T02:52:44.460040+0000","last_unstale":"2023-11-28T02:52:44.464296+0000","last_undegraded":"2023-11-28T02:52:44.464296+0000","last_fullsized":"2023-11-28T02:52:44.464296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461498+0000","last_change":"2023-11-28T02:52:44.103715+0000","last_active":"2023-11-28T02:52:44.461498+0000","last_peered":"2023-11-28T02:52:44.461498+0000","last_clean":"2023-11-28T02:52:44.461498+0000","last_became_active":"2023-11-28T02:52:44.103451+0000","last_became_peered":"2023-11-28T02:52:44.103451+0000","last_unstale":"2023-11-28T02:52:44.461498+0000","last_undegraded":"2023-11-28T02:52:44.461498+0000","last_fullsized":"2023-11-28T02:52:44.461498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464165+0000","last_change":"2023-11-28T02:52:44.460282+0000","last_active":"2023-11-28T02:52:44.464165+0000","last_peered":"2023-11-28T02:52:44.464165+0000","last_clean":"2023-11-28T02:52:44.464165+0000","last_became_active":"2023-11-28T02:52:44.460011+0000","last_became_peered":"2023-11-28T02:52:44.460011+0000","last_unstale":"2023-11-28T02:52:44.464165+0000","last_undegraded":"2023-11-28T02:52:44.464165+0000","last_fullsized":"2023-11-28T02:52:44.464165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454784+0000","last_change":"2023-11-28T02:52:44.104288+0000","last_active":"2023-11-28T02:52:44.454784+0000","last_peered":"2023-11-28T02:52:44.454784+0000","last_clean":"2023-11-28T02:52:44.454784+0000","last_became_active":"2023-11-28T02:52:44.102746+0000","last_became_peered":"2023-11-28T02:52:44.102746+0000","last_unstale":"2023-11-28T02:52:44.454784+0000","last_undegraded":"2023-11-28T02:52:44.454784+0000","last_fullsized":"2023-11-28T02:52:44.454784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471224+0000","last_change":"2023-11-28T02:52:44.468068+0000","last_active":"2023-11-28T02:52:44.471224+0000","last_peered":"2023-11-28T02:52:44.471224+0000","last_clean":"2023-11-28T02:52:44.471224+0000","last_became_active":"2023-11-28T02:52:44.467788+0000","last_became_peered":"2023-11-28T02:52:44.467788+0000","last_unstale":"2023-11-28T02:52:44.471224+0000","last_undegraded":"2023-11-28T02:52:44.471224+0000","last_fullsized":"2023-11-28T02:52:44.471224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461485+0000","last_change":"2023-11-28T02:52:44.102690+0000","last_active":"2023-11-28T02:52:44.461485+0000","last_peered":"2023-11-28T02:52:44.461485+0000","last_clean":"2023-11-28T02:52:44.461485+0000","last_became_active":"2023-11-28T02:52:44.102408+0000","last_became_peered":"2023-11-28T02:52:44.102408+0000","last_unstale":"2023-11-28T02:52:44.461485+0000","last_undegraded":"2023-11-28T02:52:44.461485+0000","last_fullsized":"2023-11-28T02:52:44.461485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469649+0000","last_change":"2023-11-28T02:52:44.467827+0000","last_active":"2023-11-28T02:52:44.469649+0000","last_peered":"2023-11-28T02:52:44.469649+0000","last_clean":"2023-11-28T02:52:44.469649+0000","last_became_active":"2023-11-28T02:52:44.467547+0000","last_became_peered":"2023-11-28T02:52:44.467547+0000","last_unstale":"2023-11-28T02:52:44.469649+0000","last_undegraded":"2023-11-28T02:52:44.469649+0000","last_fullsized":"2023-11-28T02:52:44.469649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.460918+0000","last_change":"2023-11-28T02:52:44.097889+0000","last_active":"2023-11-28T02:52:44.460918+0000","last_peered":"2023-11-28T02:52:44.460918+0000","last_clean":"2023-11-28T02:52:44.460918+0000","last_became_active":"2023-11-28T02:52:44.097641+0000","last_became_peered":"2023-11-28T02:52:44.097641+0000","last_unstale":"2023-11-28T02:52:44.460918+0000","last_undegraded":"2023-11-28T02:52:44.460918+0000","last_fullsized":"2023-11-28T02:52:44.460918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479544+0000","last_change":"2023-11-28T02:52:44.477649+0000","last_active":"2023-11-28T02:52:44.479544+0000","last_peered":"2023-11-28T02:52:44.479544+0000","last_clean":"2023-11-28T02:52:44.479544+0000","last_became_active":"2023-11-28T02:52:44.477212+0000","last_became_peered":"2023-11-28T02:52:44.477212+0000","last_unstale":"2023-11-28T02:52:44.479544+0000","last_undegraded":"2023-11-28T02:52:44.479544+0000","last_fullsized":"2023-11-28T02:52:44.479544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461323+0000","last_change":"2023-11-28T02:52:44.098078+0000","last_active":"2023-11-28T02:52:44.461323+0000","last_peered":"2023-11-28T02:52:44.461323+0000","last_clean":"2023-11-28T02:52:44.461323+0000","last_became_active":"2023-11-28T02:52:44.097777+0000","last_became_peered":"2023-11-28T02:52:44.097777+0000","last_unstale":"2023-11-28T02:52:44.461323+0000","last_undegraded":"2023-11-28T02:52:44.461323+0000","last_fullsized":"2023-11-28T02:52:44.461323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470412+0000","last_change":"2023-11-28T02:52:44.467211+0000","last_active":"2023-11-28T02:52:44.470412+0000","last_peered":"2023-11-28T02:52:44.470412+0000","last_clean":"2023-11-28T02:52:44.470412+0000","last_became_active":"2023-11-28T02:52:44.466934+0000","last_became_peered":"2023-11-28T02:52:44.466934+0000","last_unstale":"2023-11-28T02:52:44.470412+0000","last_undegraded":"2023-11-28T02:52:44.470412+0000","last_fullsized":"2023-11-28T02:52:44.470412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479015+0000","last_change":"2023-11-28T02:52:44.097548+0000","last_active":"2023-11-28T02:52:44.479015+0000","last_peered":"2023-11-28T02:52:44.479015+0000","last_clean":"2023-11-28T02:52:44.479015+0000","last_became_active":"2023-11-28T02:52:44.097253+0000","last_became_peered":"2023-11-28T02:52:44.097253+0000","last_unstale":"2023-11-28T02:52:44.479015+0000","last_undegraded":"2023-11-28T02:52:44.479015+0000","last_fullsized":"2023-11-28T02:52:44.479015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469674+0000","last_change":"2023-11-28T02:52:44.463550+0000","last_active":"2023-11-28T02:52:44.469674+0000","last_peered":"2023-11-28T02:52:44.469674+0000","last_clean":"2023-11-28T02:52:44.469674+0000","last_became_active":"2023-11-28T02:52:44.463313+0000","last_became_peered":"2023-11-28T02:52:44.463313+0000","last_unstale":"2023-11-28T02:52:44.469674+0000","last_undegraded":"2023-11-28T02:52:44.469674+0000","last_fullsized":"2023-11-28T02:52:44.469674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448095+0000","last_change":"2023-11-28T02:52:44.093490+0000","last_active":"2023-11-28T02:52:44.448095+0000","last_peered":"2023-11-28T02:52:44.448095+0000","last_clean":"2023-11-28T02:52:44.448095+0000","last_became_active":"2023-11-28T02:52:44.093241+0000","last_became_peered":"2023-11-28T02:52:44.093241+0000","last_unstale":"2023-11-28T02:52:44.448095+0000","last_undegraded":"2023-11-28T02:52:44.448095+0000","last_fullsized":"2023-11-28T02:52:44.448095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485046+0000","last_change":"2023-11-28T02:52:44.480001+0000","last_active":"2023-11-28T02:52:44.485046+0000","last_peered":"2023-11-28T02:52:44.485046+0000","last_clean":"2023-11-28T02:52:44.485046+0000","last_became_active":"2023-11-28T02:52:44.479755+0000","last_became_peered":"2023-11-28T02:52:44.479755+0000","last_unstale":"2023-11-28T02:52:44.485046+0000","last_undegraded":"2023-11-28T02:52:44.485046+0000","last_fullsized":"2023-11-28T02:52:44.485046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463559+0000","last_change":"2023-11-28T02:52:44.102658+0000","last_active":"2023-11-28T02:52:44.463559+0000","last_peered":"2023-11-28T02:52:44.463559+0000","last_clean":"2023-11-28T02:52:44.463559+0000","last_became_active":"2023-11-28T02:52:44.102381+0000","last_became_peered":"2023-11-28T02:52:44.102381+0000","last_unstale":"2023-11-28T02:52:44.463559+0000","last_undegraded":"2023-11-28T02:52:44.463559+0000","last_fullsized":"2023-11-28T02:52:44.463559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464315+0000","last_change":"2023-11-28T02:52:44.460430+0000","last_active":"2023-11-28T02:52:44.464315+0000","last_peered":"2023-11-28T02:52:44.464315+0000","last_clean":"2023-11-28T02:52:44.464315+0000","last_became_active":"2023-11-28T02:52:44.460161+0000","last_became_peered":"2023-11-28T02:52:44.460161+0000","last_unstale":"2023-11-28T02:52:44.464315+0000","last_undegraded":"2023-11-28T02:52:44.464315+0000","last_fullsized":"2023-11-28T02:52:44.464315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472669+0000","last_change":"2023-11-28T02:52:44.099082+0000","last_active":"2023-11-28T02:52:44.472669+0000","last_peered":"2023-11-28T02:52:44.472669+0000","last_clean":"2023-11-28T02:52:44.472669+0000","last_became_active":"2023-11-28T02:52:44.098444+0000","last_became_peered":"2023-11-28T02:52:44.098444+0000","last_unstale":"2023-11-28T02:52:44.472669+0000","last_undegraded":"2023-11-28T02:52:44.472669+0000","last_fullsized":"2023-11-28T02:52:44.472669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472505+0000","last_change":"2023-11-28T02:52:44.467312+0000","last_active":"2023-11-28T02:52:44.472505+0000","last_peered":"2023-11-28T02:52:44.472505+0000","last_clean":"2023-11-28T02:52:44.472505+0000","last_became_active":"2023-11-28T02:52:44.467066+0000","last_became_peered":"2023-11-28T02:52:44.467066+0000","last_unstale":"2023-11-28T02:52:44.472505+0000","last_undegraded":"2023-11-28T02:52:44.472505+0000","last_fullsized":"2023-11-28T02:52:44.472505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462586+0000","last_change":"2023-11-28T02:52:44.089495+0000","last_active":"2023-11-28T02:52:44.462586+0000","last_peered":"2023-11-28T02:52:44.462586+0000","last_clean":"2023-11-28T02:52:44.462586+0000","last_became_active":"2023-11-28T02:52:44.089229+0000","last_became_peered":"2023-11-28T02:52:44.089229+0000","last_unstale":"2023-11-28T02:52:44.462586+0000","last_undegraded":"2023-11-28T02:52:44.462586+0000","last_fullsized":"2023-11-28T02:52:44.462586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469877+0000","last_change":"2023-11-28T02:52:44.465836+0000","last_active":"2023-11-28T02:52:44.469877+0000","last_peered":"2023-11-28T02:52:44.469877+0000","last_clean":"2023-11-28T02:52:44.469877+0000","last_became_active":"2023-11-28T02:52:44.465430+0000","last_became_peered":"2023-11-28T02:52:44.465430+0000","last_unstale":"2023-11-28T02:52:44.469877+0000","last_undegraded":"2023-11-28T02:52:44.469877+0000","last_fullsized":"2023-11-28T02:52:44.469877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453296+0000","last_change":"2023-11-28T02:52:44.100936+0000","last_active":"2023-11-28T02:52:44.453296+0000","last_peered":"2023-11-28T02:52:44.453296+0000","last_clean":"2023-11-28T02:52:44.453296+0000","last_became_active":"2023-11-28T02:52:44.100435+0000","last_became_peered":"2023-11-28T02:52:44.100435+0000","last_unstale":"2023-11-28T02:52:44.453296+0000","last_undegraded":"2023-11-28T02:52:44.453296+0000","last_fullsized":"2023-11-28T02:52:44.453296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468382+0000","last_change":"2023-11-28T02:52:44.464061+0000","last_active":"2023-11-28T02:52:44.468382+0000","last_peered":"2023-11-28T02:52:44.468382+0000","last_clean":"2023-11-28T02:52:44.468382+0000","last_became_active":"2023-11-28T02:52:44.463811+0000","last_became_peered":"2023-11-28T02:52:44.463811+0000","last_unstale":"2023-11-28T02:52:44.468382+0000","last_undegraded":"2023-11-28T02:52:44.468382+0000","last_fullsized":"2023-11-28T02:52:44.468382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466528+0000","last_change":"2023-11-28T02:52:44.095761+0000","last_active":"2023-11-28T02:52:44.466528+0000","last_peered":"2023-11-28T02:52:44.466528+0000","last_clean":"2023-11-28T02:52:44.466528+0000","last_became_active":"2023-11-28T02:52:44.095568+0000","last_became_peered":"2023-11-28T02:52:44.095568+0000","last_unstale":"2023-11-28T02:52:44.466528+0000","last_undegraded":"2023-11-28T02:52:44.466528+0000","last_fullsized":"2023-11-28T02:52:44.466528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462346+0000","last_change":"2023-11-28T02:52:44.460026+0000","last_active":"2023-11-28T02:52:44.462346+0000","last_peered":"2023-11-28T02:52:44.462346+0000","last_clean":"2023-11-28T02:52:44.462346+0000","last_became_active":"2023-11-28T02:52:44.459771+0000","last_became_peered":"2023-11-28T02:52:44.459771+0000","last_unstale":"2023-11-28T02:52:44.462346+0000","last_undegraded":"2023-11-28T02:52:44.462346+0000","last_fullsized":"2023-11-28T02:52:44.462346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479985+0000","last_change":"2023-11-28T02:52:44.101052+0000","last_active":"2023-11-28T02:52:44.479985+0000","last_peered":"2023-11-28T02:52:44.479985+0000","last_clean":"2023-11-28T02:52:44.479985+0000","last_became_active":"2023-11-28T02:52:44.099171+0000","last_became_peered":"2023-11-28T02:52:44.099171+0000","last_unstale":"2023-11-28T02:52:44.479985+0000","last_undegraded":"2023-11-28T02:52:44.479985+0000","last_fullsized":"2023-11-28T02:52:44.479985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471983+0000","last_change":"2023-11-28T02:52:44.464434+0000","last_active":"2023-11-28T02:52:44.471983+0000","last_peered":"2023-11-28T02:52:44.471983+0000","last_clean":"2023-11-28T02:52:44.471983+0000","last_became_active":"2023-11-28T02:52:44.463883+0000","last_became_peered":"2023-11-28T02:52:44.463883+0000","last_unstale":"2023-11-28T02:52:44.471983+0000","last_undegraded":"2023-11-28T02:52:44.471983+0000","last_fullsized":"2023-11-28T02:52:44.471983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462246+0000","last_change":"2023-11-28T02:52:44.099394+0000","last_active":"2023-11-28T02:52:44.462246+0000","last_peered":"2023-11-28T02:52:44.462246+0000","last_clean":"2023-11-28T02:52:44.462246+0000","last_became_active":"2023-11-28T02:52:44.099099+0000","last_became_peered":"2023-11-28T02:52:44.099099+0000","last_unstale":"2023-11-28T02:52:44.462246+0000","last_undegraded":"2023-11-28T02:52:44.462246+0000","last_fullsized":"2023-11-28T02:52:44.462246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480138+0000","last_change":"2023-11-28T02:52:44.478063+0000","last_active":"2023-11-28T02:52:44.480138+0000","last_peered":"2023-11-28T02:52:44.480138+0000","last_clean":"2023-11-28T02:52:44.480138+0000","last_became_active":"2023-11-28T02:52:44.477777+0000","last_became_peered":"2023-11-28T02:52:44.477777+0000","last_unstale":"2023-11-28T02:52:44.480138+0000","last_undegraded":"2023-11-28T02:52:44.480138+0000","last_fullsized":"2023-11-28T02:52:44.480138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461922+0000","last_change":"2023-11-28T02:52:44.099389+0000","last_active":"2023-11-28T02:52:44.461922+0000","last_peered":"2023-11-28T02:52:44.461922+0000","last_clean":"2023-11-28T02:52:44.461922+0000","last_became_active":"2023-11-28T02:52:44.099084+0000","last_became_peered":"2023-11-28T02:52:44.099084+0000","last_unstale":"2023-11-28T02:52:44.461922+0000","last_undegraded":"2023-11-28T02:52:44.461922+0000","last_fullsized":"2023-11-28T02:52:44.461922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459683+0000","last_change":"2023-11-28T02:52:44.454584+0000","last_active":"2023-11-28T02:52:44.459683+0000","last_peered":"2023-11-28T02:52:44.459683+0000","last_clean":"2023-11-28T02:52:44.459683+0000","last_became_active":"2023-11-28T02:52:44.454393+0000","last_became_peered":"2023-11-28T02:52:44.454393+0000","last_unstale":"2023-11-28T02:52:44.459683+0000","last_undegraded":"2023-11-28T02:52:44.459683+0000","last_fullsized":"2023-11-28T02:52:44.459683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478337+0000","last_change":"2023-11-28T02:52:44.098086+0000","last_active":"2023-11-28T02:52:44.478337+0000","last_peered":"2023-11-28T02:52:44.478337+0000","last_clean":"2023-11-28T02:52:44.478337+0000","last_became_active":"2023-11-28T02:52:44.097733+0000","last_became_peered":"2023-11-28T02:52:44.097733+0000","last_unstale":"2023-11-28T02:52:44.478337+0000","last_undegraded":"2023-11-28T02:52:44.478337+0000","last_fullsized":"2023-11-28T02:52:44.478337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463401+0000","last_change":"2023-11-28T02:52:44.460723+0000","last_active":"2023-11-28T02:52:44.463401+0000","last_peered":"2023-11-28T02:52:44.463401+0000","last_clean":"2023-11-28T02:52:44.463401+0000","last_became_active":"2023-11-28T02:52:44.460443+0000","last_became_peered":"2023-11-28T02:52:44.460443+0000","last_unstale":"2023-11-28T02:52:44.463401+0000","last_undegraded":"2023-11-28T02:52:44.463401+0000","last_fullsized":"2023-11-28T02:52:44.463401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451507+0000","last_change":"2023-11-28T02:52:44.094043+0000","last_active":"2023-11-28T02:52:44.451507+0000","last_peered":"2023-11-28T02:52:44.451507+0000","last_clean":"2023-11-28T02:52:44.451507+0000","last_became_active":"2023-11-28T02:52:44.093761+0000","last_became_peered":"2023-11-28T02:52:44.093761+0000","last_unstale":"2023-11-28T02:52:44.451507+0000","last_undegraded":"2023-11-28T02:52:44.451507+0000","last_fullsized":"2023-11-28T02:52:44.451507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473075+0000","last_change":"2023-11-28T02:52:44.469033+0000","last_active":"2023-11-28T02:52:44.473075+0000","last_peered":"2023-11-28T02:52:44.473075+0000","last_clean":"2023-11-28T02:52:44.473075+0000","last_became_active":"2023-11-28T02:52:44.468811+0000","last_became_peered":"2023-11-28T02:52:44.468811+0000","last_unstale":"2023-11-28T02:52:44.473075+0000","last_undegraded":"2023-11-28T02:52:44.473075+0000","last_fullsized":"2023-11-28T02:52:44.473075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475940+0000","last_change":"2023-11-28T02:52:44.097577+0000","last_active":"2023-11-28T02:52:44.475940+0000","last_peered":"2023-11-28T02:52:44.475940+0000","last_clean":"2023-11-28T02:52:44.475940+0000","last_became_active":"2023-11-28T02:52:44.097283+0000","last_became_peered":"2023-11-28T02:52:44.097283+0000","last_unstale":"2023-11-28T02:52:44.475940+0000","last_undegraded":"2023-11-28T02:52:44.475940+0000","last_fullsized":"2023-11-28T02:52:44.475940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464026+0000","last_change":"2023-11-28T02:52:44.459558+0000","last_active":"2023-11-28T02:52:44.464026+0000","last_peered":"2023-11-28T02:52:44.464026+0000","last_clean":"2023-11-28T02:52:44.464026+0000","last_became_active":"2023-11-28T02:52:44.459236+0000","last_became_peered":"2023-11-28T02:52:44.459236+0000","last_unstale":"2023-11-28T02:52:44.464026+0000","last_undegraded":"2023-11-28T02:52:44.464026+0000","last_fullsized":"2023-11-28T02:52:44.464026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469470+0000","last_change":"2023-11-28T02:52:44.098691+0000","last_active":"2023-11-28T02:52:44.469470+0000","last_peered":"2023-11-28T02:52:44.469470+0000","last_clean":"2023-11-28T02:52:44.469470+0000","last_became_active":"2023-11-28T02:52:44.098401+0000","last_became_peered":"2023-11-28T02:52:44.098401+0000","last_unstale":"2023-11-28T02:52:44.469470+0000","last_undegraded":"2023-11-28T02:52:44.469470+0000","last_fullsized":"2023-11-28T02:52:44.469470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469747+0000","last_change":"2023-11-28T02:52:44.464788+0000","last_active":"2023-11-28T02:52:44.469747+0000","last_peered":"2023-11-28T02:52:44.469747+0000","last_clean":"2023-11-28T02:52:44.469747+0000","last_became_active":"2023-11-28T02:52:44.464491+0000","last_became_peered":"2023-11-28T02:52:44.464491+0000","last_unstale":"2023-11-28T02:52:44.469747+0000","last_undegraded":"2023-11-28T02:52:44.469747+0000","last_fullsized":"2023-11-28T02:52:44.469747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477600+0000","last_change":"2023-11-28T02:52:44.095969+0000","last_active":"2023-11-28T02:52:44.477600+0000","last_peered":"2023-11-28T02:52:44.477600+0000","last_clean":"2023-11-28T02:52:44.477600+0000","last_became_active":"2023-11-28T02:52:44.095693+0000","last_became_peered":"2023-11-28T02:52:44.095693+0000","last_unstale":"2023-11-28T02:52:44.477600+0000","last_undegraded":"2023-11-28T02:52:44.477600+0000","last_fullsized":"2023-11-28T02:52:44.477600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470142+0000","last_change":"2023-11-28T02:52:44.462943+0000","last_active":"2023-11-28T02:52:44.470142+0000","last_peered":"2023-11-28T02:52:44.470142+0000","last_clean":"2023-11-28T02:52:44.470142+0000","last_became_active":"2023-11-28T02:52:44.462685+0000","last_became_peered":"2023-11-28T02:52:44.462685+0000","last_unstale":"2023-11-28T02:52:44.470142+0000","last_undegraded":"2023-11-28T02:52:44.470142+0000","last_fullsized":"2023-11-28T02:52:44.470142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469567+0000","last_change":"2023-11-28T02:52:44.100318+0000","last_active":"2023-11-28T02:52:44.469567+0000","last_peered":"2023-11-28T02:52:44.469567+0000","last_clean":"2023-11-28T02:52:44.469567+0000","last_became_active":"2023-11-28T02:52:44.100056+0000","last_became_peered":"2023-11-28T02:52:44.100056+0000","last_unstale":"2023-11-28T02:52:44.469567+0000","last_undegraded":"2023-11-28T02:52:44.469567+0000","last_fullsized":"2023-11-28T02:52:44.469567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463773+0000","last_change":"2023-11-28T02:52:44.460144+0000","last_active":"2023-11-28T02:52:44.463773+0000","last_peered":"2023-11-28T02:52:44.463773+0000","last_clean":"2023-11-28T02:52:44.463773+0000","last_became_active":"2023-11-28T02:52:44.459861+0000","last_became_peered":"2023-11-28T02:52:44.459861+0000","last_unstale":"2023-11-28T02:52:44.463773+0000","last_undegraded":"2023-11-28T02:52:44.463773+0000","last_fullsized":"2023-11-28T02:52:44.463773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450475+0000","last_change":"2023-11-28T02:52:44.101546+0000","last_active":"2023-11-28T02:52:44.450475+0000","last_peered":"2023-11-28T02:52:44.450475+0000","last_clean":"2023-11-28T02:52:44.450475+0000","last_became_active":"2023-11-28T02:52:44.101254+0000","last_became_peered":"2023-11-28T02:52:44.101254+0000","last_unstale":"2023-11-28T02:52:44.450475+0000","last_undegraded":"2023-11-28T02:52:44.450475+0000","last_fullsized":"2023-11-28T02:52:44.450475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479536+0000","last_change":"2023-11-28T02:52:44.477542+0000","last_active":"2023-11-28T02:52:44.479536+0000","last_peered":"2023-11-28T02:52:44.479536+0000","last_clean":"2023-11-28T02:52:44.479536+0000","last_became_active":"2023-11-28T02:52:44.477191+0000","last_became_peered":"2023-11-28T02:52:44.477191+0000","last_unstale":"2023-11-28T02:52:44.479536+0000","last_undegraded":"2023-11-28T02:52:44.479536+0000","last_fullsized":"2023-11-28T02:52:44.479536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464374+0000","last_change":"2023-11-28T02:52:44.097950+0000","last_active":"2023-11-28T02:52:44.464374+0000","last_peered":"2023-11-28T02:52:44.464374+0000","last_clean":"2023-11-28T02:52:44.464374+0000","last_became_active":"2023-11-28T02:52:44.097703+0000","last_became_peered":"2023-11-28T02:52:44.097703+0000","last_unstale":"2023-11-28T02:52:44.464374+0000","last_undegraded":"2023-11-28T02:52:44.464374+0000","last_fullsized":"2023-11-28T02:52:44.464374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472612+0000","last_change":"2023-11-28T02:52:44.466320+0000","last_active":"2023-11-28T02:52:44.472612+0000","last_peered":"2023-11-28T02:52:44.472612+0000","last_clean":"2023-11-28T02:52:44.472612+0000","last_became_active":"2023-11-28T02:52:44.466039+0000","last_became_peered":"2023-11-28T02:52:44.466039+0000","last_unstale":"2023-11-28T02:52:44.472612+0000","last_undegraded":"2023-11-28T02:52:44.472612+0000","last_fullsized":"2023-11-28T02:52:44.472612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485236+0000","last_change":"2023-11-28T02:52:44.093193+0000","last_active":"2023-11-28T02:52:44.485236+0000","last_peered":"2023-11-28T02:52:44.485236+0000","last_clean":"2023-11-28T02:52:44.485236+0000","last_became_active":"2023-11-28T02:52:44.092869+0000","last_became_peered":"2023-11-28T02:52:44.092869+0000","last_unstale":"2023-11-28T02:52:44.485236+0000","last_undegraded":"2023-11-28T02:52:44.485236+0000","last_fullsized":"2023-11-28T02:52:44.485236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297348,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074492,"statfs":{"total":96636764160,"available":96332279808,"internally_reserved":0,"allocated":258048,"data_stored":86755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":86787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297340,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074500,"statfs":{"total":96636764160,"available":96332288000,"internally_reserved":0,"allocated":249856,"data_stored":86933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:52:54.071 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:52:54.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.214+0000 7f2dc7a2f700 1 Processor -- start 2023-11-28T02:52:54.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.214+0000 7f2dc7a2f700 1 -- start start 2023-11-28T02:52:54.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc7a2f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2dc00caab0 0x7f2dc00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc7a2f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc00a9a90 con 0x7f2dc00cae90 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc7a2f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc0154ce0 con 0x7f2dc00ca180 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc7a2f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc00a8e80 con 0x7f2dc00caab0 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc4fca700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2dc00cae90 0x7f2dc00c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44904/0 (socket says 172.21.15.73:44904) 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.215+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3979120638 learned_addr learned my addr 172.21.15.73:0/3979120638 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 24213108 0 0) 0x7f2dc00a9a90 con 0x7f2dc00cae90 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2da80033f0 con 0x7f2dc00cae90 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc57cb700 1 --2- 172.21.15.73:0/3979120638 >> v2:172.21.15.150:3300/0 conn(0x7f2dc00caab0 0x7f2dc00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3634513432 0 0) 0x7f2da80033f0 con 0x7f2dc00cae90 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 >> v2:172.21.15.150:3300/0 conn(0x7f2dc00caab0 msgr2=0x7f2dc00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 --2- 172.21.15.73:0/3979120638 >> v2:172.21.15.150:3300/0 conn(0x7f2dc00caab0 0x7f2dc00aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 >> v1:172.21.15.150:6789/0 conn(0x7f2dc00ca180 legacy=0x7f2dc015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00a9160 con 0x7f2dc00cae90 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2db4001470 con 0x7f2dc00cae90 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 >> v1:172.21.15.73:6789/0 conn(0x7f2dc00cae90 legacy=0x7f2dc00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da80038e0 0x7f2da8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 0x7f2da8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8005000 0x7f2da800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00a8e80 con 0x7f2da8004690 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2da800bf60 con 0x7f2da80038e0 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2da800c0f0 con 0x7f2da8005000 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 0x7f2da8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc57cb700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da80038e0 0x7f2da8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc5fcc700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8005000 0x7f2da800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8005000 msgr2=0x7f2da800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8005000 0x7f2da800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da80038e0 msgr2=0x7f2da8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2da80038e0 0x7f2da8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3979120638 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2da80033f0 con 0x7f2da8004690 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.216+0000 7f2dc5fcc700 1 -- 172.21.15.73:0/3979120638 reap_dead start 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 0x7f2da8004b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2db400c190 tx=0x7f2db4001e50).ready entity=mon.0 client_cookie=c6b405e28f0ff748 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db4007a20 con 0x7f2da8004690 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db400ca80 con 0x7f2da8004690 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dbffff700 1 -- 172.21.15.73:0/3979120638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400ce00 con 0x7f2da8004690 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 msgr2=0x7f2da8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 0x7f2da8004b10 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7f2db400c190 tx=0x7f2db4001e50).stop 2023-11-28T02:52:54.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3979120638 shutdown_connections 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3979120638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2da8004690 0x7f2da8004b10 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3979120638 >> 172.21.15.73:0/3979120638 conn(0x7f2dc0094920 msgr2=0x7f2dc006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3979120638 shutdown_connections 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.217+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3979120638 wait complete. 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.218+0000 7f2dc7a2f700 1 Processor -- start 2023-11-28T02:52:54.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 -- start start 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8004690 0x7f2dc01480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 0x7f2dc00aa510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00abe80 con 0x7f2dc0156bc0 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc0157530 con 0x7f2dc00aa090 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc7a2f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc00aade0 con 0x7f2da8004690 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.219+0000 7f2dc4fca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44876/0 (socket says 172.21.15.73:44876) 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3664626916 learned_addr learned my addr 172.21.15.73:0/3664626916 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc57cb700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8004690 0x7f2dc01480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc5fcc700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 0x7f2dc00aa510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8004690 msgr2=0x7f2dc01480d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8004690 0x7f2dc01480d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 msgr2=0x7f2dc00aa510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 0x7f2dc00aa510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 -- 172.21.15.73:0/3664626916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc00ab0c0 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc5fcc700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 0x7f2dc00aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc4fca700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2db4001ed0 tx=0x7f2db4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40073b0 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db4007510 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc00ab3a0 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db40048e0 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.220+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc00ab960 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.221+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc006c860 con 0x7f2dc0156bc0 2023-11-28T02:52:54.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.221+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2db401c070 con 0x7f2dc0156bc0 2023-11-28T02:52:54.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.222+0000 7f2dbd7fa700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 0x7f2dac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.222+0000 7f2dc57cb700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 0x7f2dac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.222+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2db407a920 con 0x7f2dc0156bc0 2023-11-28T02:52:54.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.222+0000 7f2dc57cb700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 0x7f2dac061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2dc0060fb0 tx=0x7f2db0006f40).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.226+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2db404bbf0 con 0x7f2dc0156bc0 2023-11-28T02:52:54.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.386+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2dc00a9440 con 0x7f2dac05f150 2023-11-28T02:52:54.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.394+0000 7f2dbd7fa700 1 -- 172.21.15.73:0/3664626916 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336341 (secure 0 0 0) 0x7f2dc00a9440 con 0x7f2dac05f150 2023-11-28T02:52:54.400 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:54.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 msgr2=0x7f2dac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 0x7f2dac061610 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f2dc0060fb0 tx=0x7f2db0006f40).stop 2023-11-28T02:52:54.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 msgr2=0x7f2dc0157040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f2db4001ed0 tx=0x7f2db4015000).stop 2023-11-28T02:52:54.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 shutdown_connections 2023-11-28T02:52:54.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2dac05f150 0x7f2dac061610 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dc00aa090 0x7f2dc00aa510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2da8004690 0x7f2dc01480d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 --2- 172.21.15.73:0/3664626916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dc0156bc0 0x7f2dc0157040 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 >> 172.21.15.73:0/3664626916 conn(0x7f2dc0094920 msgr2=0x7f2dc00a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 shutdown_connections 2023-11-28T02:52:54.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.398+0000 7f2dc7a2f700 1 -- 172.21.15.73:0/3664626916 wait complete. 2023-11-28T02:52:54.405 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:52:54.411 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T02:52:53.391060+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2378624,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752596096,"statfs":{"total":773094113280,"available":770658402304,"internally_reserved":0,"allocated":1900544,"data_stored":687340,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001465"},"pg_stats":[{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470198+0000","last_change":"2023-11-28T02:52:44.467253+0000","last_active":"2023-11-28T02:52:44.470198+0000","last_peered":"2023-11-28T02:52:44.470198+0000","last_clean":"2023-11-28T02:52:44.470198+0000","last_became_active":"2023-11-28T02:52:44.466977+0000","last_became_peered":"2023-11-28T02:52:44.466977+0000","last_unstale":"2023-11-28T02:52:44.470198+0000","last_undegraded":"2023-11-28T02:52:44.470198+0000","last_fullsized":"2023-11-28T02:52:44.470198+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455029+0000","last_change":"2023-11-28T02:52:44.098094+0000","last_active":"2023-11-28T02:52:44.455029+0000","last_peered":"2023-11-28T02:52:44.455029+0000","last_clean":"2023-11-28T02:52:44.455029+0000","last_became_active":"2023-11-28T02:52:44.097826+0000","last_became_peered":"2023-11-28T02:52:44.097826+0000","last_unstale":"2023-11-28T02:52:44.455029+0000","last_undegraded":"2023-11-28T02:52:44.455029+0000","last_fullsized":"2023-11-28T02:52:44.455029+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479071+0000","last_change":"2023-11-28T02:52:44.478096+0000","last_active":"2023-11-28T02:52:44.479071+0000","last_peered":"2023-11-28T02:52:44.479071+0000","last_clean":"2023-11-28T02:52:44.479071+0000","last_became_active":"2023-11-28T02:52:44.477817+0000","last_became_peered":"2023-11-28T02:52:44.477817+0000","last_unstale":"2023-11-28T02:52:44.479071+0000","last_undegraded":"2023-11-28T02:52:44.479071+0000","last_fullsized":"2023-11-28T02:52:44.479071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462484+0000","last_change":"2023-11-28T02:52:44.087580+0000","last_active":"2023-11-28T02:52:44.462484+0000","last_peered":"2023-11-28T02:52:44.462484+0000","last_clean":"2023-11-28T02:52:44.462484+0000","last_became_active":"2023-11-28T02:52:44.087310+0000","last_became_peered":"2023-11-28T02:52:44.087310+0000","last_unstale":"2023-11-28T02:52:44.462484+0000","last_undegraded":"2023-11-28T02:52:44.462484+0000","last_fullsized":"2023-11-28T02:52:44.462484+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472269+0000","last_change":"2023-11-28T02:52:44.466469+0000","last_active":"2023-11-28T02:52:44.472269+0000","last_peered":"2023-11-28T02:52:44.472269+0000","last_clean":"2023-11-28T02:52:44.472269+0000","last_became_active":"2023-11-28T02:52:44.465822+0000","last_became_peered":"2023-11-28T02:52:44.465822+0000","last_unstale":"2023-11-28T02:52:44.472269+0000","last_undegraded":"2023-11-28T02:52:44.472269+0000","last_fullsized":"2023-11-28T02:52:44.472269+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451606+0000","last_change":"2023-11-28T02:52:44.097696+0000","last_active":"2023-11-28T02:52:44.451606+0000","last_peered":"2023-11-28T02:52:44.451606+0000","last_clean":"2023-11-28T02:52:44.451606+0000","last_became_active":"2023-11-28T02:52:44.097423+0000","last_became_peered":"2023-11-28T02:52:44.097423+0000","last_unstale":"2023-11-28T02:52:44.451606+0000","last_undegraded":"2023-11-28T02:52:44.451606+0000","last_fullsized":"2023-11-28T02:52:44.451606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469722+0000","last_change":"2023-11-28T02:52:44.465794+0000","last_active":"2023-11-28T02:52:44.469722+0000","last_peered":"2023-11-28T02:52:44.469722+0000","last_clean":"2023-11-28T02:52:44.469722+0000","last_became_active":"2023-11-28T02:52:44.465545+0000","last_became_peered":"2023-11-28T02:52:44.465545+0000","last_unstale":"2023-11-28T02:52:44.469722+0000","last_undegraded":"2023-11-28T02:52:44.469722+0000","last_fullsized":"2023-11-28T02:52:44.469722+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450984+0000","last_change":"2023-11-28T02:52:44.099493+0000","last_active":"2023-11-28T02:52:44.450984+0000","last_peered":"2023-11-28T02:52:44.450984+0000","last_clean":"2023-11-28T02:52:44.450984+0000","last_became_active":"2023-11-28T02:52:44.099248+0000","last_became_peered":"2023-11-28T02:52:44.099248+0000","last_unstale":"2023-11-28T02:52:44.450984+0000","last_undegraded":"2023-11-28T02:52:44.450984+0000","last_fullsized":"2023-11-28T02:52:44.450984+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479353+0000","last_change":"2023-11-28T02:52:44.477723+0000","last_active":"2023-11-28T02:52:44.479353+0000","last_peered":"2023-11-28T02:52:44.479353+0000","last_clean":"2023-11-28T02:52:44.479353+0000","last_became_active":"2023-11-28T02:52:44.477430+0000","last_became_peered":"2023-11-28T02:52:44.477430+0000","last_unstale":"2023-11-28T02:52:44.479353+0000","last_undegraded":"2023-11-28T02:52:44.479353+0000","last_fullsized":"2023-11-28T02:52:44.479353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462583+0000","last_change":"2023-11-28T02:52:44.102758+0000","last_active":"2023-11-28T02:52:44.462583+0000","last_peered":"2023-11-28T02:52:44.462583+0000","last_clean":"2023-11-28T02:52:44.462583+0000","last_became_active":"2023-11-28T02:52:44.102441+0000","last_became_peered":"2023-11-28T02:52:44.102441+0000","last_unstale":"2023-11-28T02:52:44.462583+0000","last_undegraded":"2023-11-28T02:52:44.462583+0000","last_fullsized":"2023-11-28T02:52:44.462583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468903+0000","last_change":"2023-11-28T02:52:44.466352+0000","last_active":"2023-11-28T02:52:44.468903+0000","last_peered":"2023-11-28T02:52:44.468903+0000","last_clean":"2023-11-28T02:52:44.468903+0000","last_became_active":"2023-11-28T02:52:44.466023+0000","last_became_peered":"2023-11-28T02:52:44.466023+0000","last_unstale":"2023-11-28T02:52:44.468903+0000","last_undegraded":"2023-11-28T02:52:44.468903+0000","last_fullsized":"2023-11-28T02:52:44.468903+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479820+0000","last_change":"2023-11-28T02:52:44.100407+0000","last_active":"2023-11-28T02:52:44.479820+0000","last_peered":"2023-11-28T02:52:44.479820+0000","last_clean":"2023-11-28T02:52:44.479820+0000","last_became_active":"2023-11-28T02:52:44.100184+0000","last_became_peered":"2023-11-28T02:52:44.100184+0000","last_unstale":"2023-11-28T02:52:44.479820+0000","last_undegraded":"2023-11-28T02:52:44.479820+0000","last_fullsized":"2023-11-28T02:52:44.479820+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469727+0000","last_change":"2023-11-28T02:52:44.467942+0000","last_active":"2023-11-28T02:52:44.469727+0000","last_peered":"2023-11-28T02:52:44.469727+0000","last_clean":"2023-11-28T02:52:44.469727+0000","last_became_active":"2023-11-28T02:52:44.467659+0000","last_became_peered":"2023-11-28T02:52:44.467659+0000","last_unstale":"2023-11-28T02:52:44.469727+0000","last_undegraded":"2023-11-28T02:52:44.469727+0000","last_fullsized":"2023-11-28T02:52:44.469727+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.14","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472685+0000","last_change":"2023-11-28T02:52:44.094177+0000","last_active":"2023-11-28T02:52:44.472685+0000","last_peered":"2023-11-28T02:52:44.472685+0000","last_clean":"2023-11-28T02:52:44.472685+0000","last_became_active":"2023-11-28T02:52:44.093901+0000","last_became_peered":"2023-11-28T02:52:44.093901+0000","last_unstale":"2023-11-28T02:52:44.472685+0000","last_undegraded":"2023-11-28T02:52:44.472685+0000","last_fullsized":"2023-11-28T02:52:44.472685+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470448+0000","last_change":"2023-11-28T02:52:44.467833+0000","last_active":"2023-11-28T02:52:44.470448+0000","last_peered":"2023-11-28T02:52:44.470448+0000","last_clean":"2023-11-28T02:52:44.470448+0000","last_became_active":"2023-11-28T02:52:44.467517+0000","last_became_peered":"2023-11-28T02:52:44.467517+0000","last_unstale":"2023-11-28T02:52:44.470448+0000","last_undegraded":"2023-11-28T02:52:44.470448+0000","last_fullsized":"2023-11-28T02:52:44.470448+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485164+0000","last_change":"2023-11-28T02:52:44.104338+0000","last_active":"2023-11-28T02:52:44.485164+0000","last_peered":"2023-11-28T02:52:44.485164+0000","last_clean":"2023-11-28T02:52:44.485164+0000","last_became_active":"2023-11-28T02:52:44.104073+0000","last_became_peered":"2023-11-28T02:52:44.104073+0000","last_unstale":"2023-11-28T02:52:44.485164+0000","last_undegraded":"2023-11-28T02:52:44.485164+0000","last_fullsized":"2023-11-28T02:52:44.485164+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477889+0000","last_change":"2023-11-28T02:52:44.476733+0000","last_active":"2023-11-28T02:52:44.477889+0000","last_peered":"2023-11-28T02:52:44.477889+0000","last_clean":"2023-11-28T02:52:44.477889+0000","last_became_active":"2023-11-28T02:52:44.476400+0000","last_became_peered":"2023-11-28T02:52:44.476400+0000","last_unstale":"2023-11-28T02:52:44.477889+0000","last_undegraded":"2023-11-28T02:52:44.477889+0000","last_fullsized":"2023-11-28T02:52:44.477889+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453202+0000","last_change":"2023-11-28T02:52:44.096921+0000","last_active":"2023-11-28T02:52:44.453202+0000","last_peered":"2023-11-28T02:52:44.453202+0000","last_clean":"2023-11-28T02:52:44.453202+0000","last_became_active":"2023-11-28T02:52:44.096687+0000","last_became_peered":"2023-11-28T02:52:44.096687+0000","last_unstale":"2023-11-28T02:52:44.453202+0000","last_undegraded":"2023-11-28T02:52:44.453202+0000","last_fullsized":"2023-11-28T02:52:44.453202+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466146+0000","last_change":"2023-11-28T02:52:44.461227+0000","last_active":"2023-11-28T02:52:44.466146+0000","last_peered":"2023-11-28T02:52:44.466146+0000","last_clean":"2023-11-28T02:52:44.466146+0000","last_became_active":"2023-11-28T02:52:44.460957+0000","last_became_peered":"2023-11-28T02:52:44.460957+0000","last_unstale":"2023-11-28T02:52:44.466146+0000","last_undegraded":"2023-11-28T02:52:44.466146+0000","last_fullsized":"2023-11-28T02:52:44.466146+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475829+0000","last_change":"2023-11-28T02:52:44.106278+0000","last_active":"2023-11-28T02:52:44.475829+0000","last_peered":"2023-11-28T02:52:44.475829+0000","last_clean":"2023-11-28T02:52:44.475829+0000","last_became_active":"2023-11-28T02:52:44.106026+0000","last_became_peered":"2023-11-28T02:52:44.106026+0000","last_unstale":"2023-11-28T02:52:44.475829+0000","last_undegraded":"2023-11-28T02:52:44.475829+0000","last_fullsized":"2023-11-28T02:52:44.475829+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480394+0000","last_change":"2023-11-28T02:52:44.479895+0000","last_active":"2023-11-28T02:52:44.480394+0000","last_peered":"2023-11-28T02:52:44.480394+0000","last_clean":"2023-11-28T02:52:44.480394+0000","last_became_active":"2023-11-28T02:52:44.479685+0000","last_became_peered":"2023-11-28T02:52:44.479685+0000","last_unstale":"2023-11-28T02:52:44.480394+0000","last_undegraded":"2023-11-28T02:52:44.480394+0000","last_fullsized":"2023-11-28T02:52:44.480394+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.10","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470565+0000","last_change":"2023-11-28T02:52:44.097431+0000","last_active":"2023-11-28T02:52:44.470565+0000","last_peered":"2023-11-28T02:52:44.470565+0000","last_clean":"2023-11-28T02:52:44.470565+0000","last_became_active":"2023-11-28T02:52:44.097043+0000","last_became_peered":"2023-11-28T02:52:44.097043+0000","last_unstale":"2023-11-28T02:52:44.470565+0000","last_undegraded":"2023-11-28T02:52:44.470565+0000","last_fullsized":"2023-11-28T02:52:44.470565+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478662+0000","last_change":"2023-11-28T02:52:44.477784+0000","last_active":"2023-11-28T02:52:44.478662+0000","last_peered":"2023-11-28T02:52:44.478662+0000","last_clean":"2023-11-28T02:52:44.478662+0000","last_became_active":"2023-11-28T02:52:44.477515+0000","last_became_peered":"2023-11-28T02:52:44.477515+0000","last_unstale":"2023-11-28T02:52:44.478662+0000","last_undegraded":"2023-11-28T02:52:44.478662+0000","last_fullsized":"2023-11-28T02:52:44.478662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485148+0000","last_change":"2023-11-28T02:52:44.104964+0000","last_active":"2023-11-28T02:52:44.485148+0000","last_peered":"2023-11-28T02:52:44.485148+0000","last_clean":"2023-11-28T02:52:44.485148+0000","last_became_active":"2023-11-28T02:52:44.104688+0000","last_became_peered":"2023-11-28T02:52:44.104688+0000","last_unstale":"2023-11-28T02:52:44.485148+0000","last_undegraded":"2023-11-28T02:52:44.485148+0000","last_fullsized":"2023-11-28T02:52:44.485148+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469058+0000","last_change":"2023-11-28T02:52:44.465011+0000","last_active":"2023-11-28T02:52:44.469058+0000","last_peered":"2023-11-28T02:52:44.469058+0000","last_clean":"2023-11-28T02:52:44.469058+0000","last_became_active":"2023-11-28T02:52:44.464725+0000","last_became_peered":"2023-11-28T02:52:44.464725+0000","last_unstale":"2023-11-28T02:52:44.469058+0000","last_undegraded":"2023-11-28T02:52:44.469058+0000","last_fullsized":"2023-11-28T02:52:44.469058+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.482297+0000","last_change":"2023-11-28T02:52:44.104530+0000","last_active":"2023-11-28T02:52:44.482297+0000","last_peered":"2023-11-28T02:52:44.482297+0000","last_clean":"2023-11-28T02:52:44.482297+0000","last_became_active":"2023-11-28T02:52:44.104035+0000","last_became_peered":"2023-11-28T02:52:44.104035+0000","last_unstale":"2023-11-28T02:52:44.482297+0000","last_undegraded":"2023-11-28T02:52:44.482297+0000","last_fullsized":"2023-11-28T02:52:44.482297+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474584+0000","last_change":"2023-11-28T02:52:44.469216+0000","last_active":"2023-11-28T02:52:44.474584+0000","last_peered":"2023-11-28T02:52:44.474584+0000","last_clean":"2023-11-28T02:52:44.474584+0000","last_became_active":"2023-11-28T02:52:44.467199+0000","last_became_peered":"2023-11-28T02:52:44.467199+0000","last_unstale":"2023-11-28T02:52:44.474584+0000","last_undegraded":"2023-11-28T02:52:44.474584+0000","last_fullsized":"2023-11-28T02:52:44.474584+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474622+0000","last_change":"2023-11-28T02:52:44.099749+0000","last_active":"2023-11-28T02:52:44.474622+0000","last_peered":"2023-11-28T02:52:44.474622+0000","last_clean":"2023-11-28T02:52:44.474622+0000","last_became_active":"2023-11-28T02:52:44.099525+0000","last_became_peered":"2023-11-28T02:52:44.099525+0000","last_unstale":"2023-11-28T02:52:44.474622+0000","last_undegraded":"2023-11-28T02:52:44.474622+0000","last_fullsized":"2023-11-28T02:52:44.474622+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470191+0000","last_change":"2023-11-28T02:52:44.467373+0000","last_active":"2023-11-28T02:52:44.470191+0000","last_peered":"2023-11-28T02:52:44.470191+0000","last_clean":"2023-11-28T02:52:44.470191+0000","last_became_active":"2023-11-28T02:52:44.464529+0000","last_became_peered":"2023-11-28T02:52:44.464529+0000","last_unstale":"2023-11-28T02:52:44.470191+0000","last_undegraded":"2023-11-28T02:52:44.470191+0000","last_fullsized":"2023-11-28T02:52:44.470191+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457475+0000","last_change":"2023-11-28T02:52:44.099774+0000","last_active":"2023-11-28T02:52:44.457475+0000","last_peered":"2023-11-28T02:52:44.457475+0000","last_clean":"2023-11-28T02:52:44.457475+0000","last_became_active":"2023-11-28T02:52:44.099484+0000","last_became_peered":"2023-11-28T02:52:44.099484+0000","last_unstale":"2023-11-28T02:52:44.457475+0000","last_undegraded":"2023-11-28T02:52:44.457475+0000","last_fullsized":"2023-11-28T02:52:44.457475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469631+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.469631+0000","last_peered":"2023-11-28T02:52:44.469631+0000","last_clean":"2023-11-28T02:52:44.469631+0000","last_became_active":"2023-11-28T02:52:44.460712+0000","last_became_peered":"2023-11-28T02:52:44.460712+0000","last_unstale":"2023-11-28T02:52:44.469631+0000","last_undegraded":"2023-11-28T02:52:44.469631+0000","last_fullsized":"2023-11-28T02:52:44.469631+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465456+0000","last_change":"2023-11-28T02:52:44.098187+0000","last_active":"2023-11-28T02:52:44.465456+0000","last_peered":"2023-11-28T02:52:44.465456+0000","last_clean":"2023-11-28T02:52:44.465456+0000","last_became_active":"2023-11-28T02:52:44.097937+0000","last_became_peered":"2023-11-28T02:52:44.097937+0000","last_unstale":"2023-11-28T02:52:44.465456+0000","last_undegraded":"2023-11-28T02:52:44.465456+0000","last_fullsized":"2023-11-28T02:52:44.465456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464006+0000","last_change":"2023-11-28T02:52:44.458708+0000","last_active":"2023-11-28T02:52:44.464006+0000","last_peered":"2023-11-28T02:52:44.464006+0000","last_clean":"2023-11-28T02:52:44.464006+0000","last_became_active":"2023-11-28T02:52:44.458372+0000","last_became_peered":"2023-11-28T02:52:44.458372+0000","last_unstale":"2023-11-28T02:52:44.464006+0000","last_undegraded":"2023-11-28T02:52:44.464006+0000","last_fullsized":"2023-11-28T02:52:44.464006+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463865+0000","last_change":"2023-11-28T02:52:44.098081+0000","last_active":"2023-11-28T02:52:44.463865+0000","last_peered":"2023-11-28T02:52:44.463865+0000","last_clean":"2023-11-28T02:52:44.463865+0000","last_became_active":"2023-11-28T02:52:44.097763+0000","last_became_peered":"2023-11-28T02:52:44.097763+0000","last_unstale":"2023-11-28T02:52:44.463865+0000","last_undegraded":"2023-11-28T02:52:44.463865+0000","last_fullsized":"2023-11-28T02:52:44.463865+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468716+0000","last_change":"2023-11-28T02:52:44.464973+0000","last_active":"2023-11-28T02:52:44.468716+0000","last_peered":"2023-11-28T02:52:44.468716+0000","last_clean":"2023-11-28T02:52:44.468716+0000","last_became_active":"2023-11-28T02:52:44.464692+0000","last_became_peered":"2023-11-28T02:52:44.464692+0000","last_unstale":"2023-11-28T02:52:44.468716+0000","last_undegraded":"2023-11-28T02:52:44.468716+0000","last_fullsized":"2023-11-28T02:52:44.468716+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454758+0000","last_change":"2023-11-28T02:52:44.096316+0000","last_active":"2023-11-28T02:52:44.454758+0000","last_peered":"2023-11-28T02:52:44.454758+0000","last_clean":"2023-11-28T02:52:44.454758+0000","last_became_active":"2023-11-28T02:52:44.090973+0000","last_became_peered":"2023-11-28T02:52:44.090973+0000","last_unstale":"2023-11-28T02:52:44.454758+0000","last_undegraded":"2023-11-28T02:52:44.454758+0000","last_fullsized":"2023-11-28T02:52:44.454758+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467774+0000","last_change":"2023-11-28T02:52:44.463807+0000","last_active":"2023-11-28T02:52:44.467774+0000","last_peered":"2023-11-28T02:52:44.467774+0000","last_clean":"2023-11-28T02:52:44.467774+0000","last_became_active":"2023-11-28T02:52:44.463532+0000","last_became_peered":"2023-11-28T02:52:44.463532+0000","last_unstale":"2023-11-28T02:52:44.467774+0000","last_undegraded":"2023-11-28T02:52:44.467774+0000","last_fullsized":"2023-11-28T02:52:44.467774+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451394+0000","last_change":"2023-11-28T02:52:44.099397+0000","last_active":"2023-11-28T02:52:44.451394+0000","last_peered":"2023-11-28T02:52:44.451394+0000","last_clean":"2023-11-28T02:52:44.451394+0000","last_became_active":"2023-11-28T02:52:44.099225+0000","last_became_peered":"2023-11-28T02:52:44.099225+0000","last_unstale":"2023-11-28T02:52:44.451394+0000","last_undegraded":"2023-11-28T02:52:44.451394+0000","last_fullsized":"2023-11-28T02:52:44.451394+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464258+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:52:44.464258+0000","last_peered":"2023-11-28T02:52:44.464258+0000","last_clean":"2023-11-28T02:52:44.464258+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:52:44.464258+0000","last_undegraded":"2023-11-28T02:52:44.464258+0000","last_fullsized":"2023-11-28T02:52:44.464258+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461086+0000","last_change":"2023-11-28T02:52:44.099940+0000","last_active":"2023-11-28T02:52:44.461086+0000","last_peered":"2023-11-28T02:52:44.461086+0000","last_clean":"2023-11-28T02:52:44.461086+0000","last_became_active":"2023-11-28T02:52:44.099735+0000","last_became_peered":"2023-11-28T02:52:44.099735+0000","last_unstale":"2023-11-28T02:52:44.461086+0000","last_undegraded":"2023-11-28T02:52:44.461086+0000","last_fullsized":"2023-11-28T02:52:44.461086+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473095+0000","last_change":"2023-11-28T02:52:44.472735+0000","last_active":"2023-11-28T02:52:44.473095+0000","last_peered":"2023-11-28T02:52:44.473095+0000","last_clean":"2023-11-28T02:52:44.473095+0000","last_became_active":"2023-11-28T02:52:44.472418+0000","last_became_peered":"2023-11-28T02:52:44.472418+0000","last_unstale":"2023-11-28T02:52:44.473095+0000","last_undegraded":"2023-11-28T02:52:44.473095+0000","last_fullsized":"2023-11-28T02:52:44.473095+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459288+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:52:44.459288+0000","last_peered":"2023-11-28T02:52:44.459288+0000","last_clean":"2023-11-28T02:52:44.459288+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:52:44.459288+0000","last_undegraded":"2023-11-28T02:52:44.459288+0000","last_fullsized":"2023-11-28T02:52:44.459288+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.455813+0000","last_change":"2023-11-28T02:52:44.100602+0000","last_active":"2023-11-28T02:52:44.455813+0000","last_peered":"2023-11-28T02:52:44.455813+0000","last_clean":"2023-11-28T02:52:44.455813+0000","last_became_active":"2023-11-28T02:52:44.100337+0000","last_became_peered":"2023-11-28T02:52:44.100337+0000","last_unstale":"2023-11-28T02:52:44.455813+0000","last_undegraded":"2023-11-28T02:52:44.455813+0000","last_fullsized":"2023-11-28T02:52:44.455813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468871+0000","last_change":"2023-11-28T02:52:44.465676+0000","last_active":"2023-11-28T02:52:44.468871+0000","last_peered":"2023-11-28T02:52:44.468871+0000","last_clean":"2023-11-28T02:52:44.468871+0000","last_became_active":"2023-11-28T02:52:44.465401+0000","last_became_peered":"2023-11-28T02:52:44.465401+0000","last_unstale":"2023-11-28T02:52:44.468871+0000","last_undegraded":"2023-11-28T02:52:44.468871+0000","last_fullsized":"2023-11-28T02:52:44.468871+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471562+0000","last_change":"2023-11-28T02:52:44.470158+0000","last_active":"2023-11-28T02:52:44.471562+0000","last_peered":"2023-11-28T02:52:44.471562+0000","last_clean":"2023-11-28T02:52:44.471562+0000","last_became_active":"2023-11-28T02:52:44.469795+0000","last_became_peered":"2023-11-28T02:52:44.469795+0000","last_unstale":"2023-11-28T02:52:44.471562+0000","last_undegraded":"2023-11-28T02:52:44.471562+0000","last_fullsized":"2023-11-28T02:52:44.471562+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448512+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:52:44.448512+0000","last_peered":"2023-11-28T02:52:44.448512+0000","last_clean":"2023-11-28T02:52:44.448512+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:52:44.448512+0000","last_undegraded":"2023-11-28T02:52:44.448512+0000","last_fullsized":"2023-11-28T02:52:44.448512+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462396+0000","last_change":"2023-11-28T02:52:44.091912+0000","last_active":"2023-11-28T02:52:44.462396+0000","last_peered":"2023-11-28T02:52:44.462396+0000","last_clean":"2023-11-28T02:52:44.462396+0000","last_became_active":"2023-11-28T02:52:44.091630+0000","last_became_peered":"2023-11-28T02:52:44.091630+0000","last_unstale":"2023-11-28T02:52:44.462396+0000","last_undegraded":"2023-11-28T02:52:44.462396+0000","last_fullsized":"2023-11-28T02:52:44.462396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472298+0000","last_change":"2023-11-28T02:52:44.468106+0000","last_active":"2023-11-28T02:52:44.472298+0000","last_peered":"2023-11-28T02:52:44.472298+0000","last_clean":"2023-11-28T02:52:44.472298+0000","last_became_active":"2023-11-28T02:52:44.467833+0000","last_became_peered":"2023-11-28T02:52:44.467833+0000","last_unstale":"2023-11-28T02:52:44.472298+0000","last_undegraded":"2023-11-28T02:52:44.472298+0000","last_fullsized":"2023-11-28T02:52:44.472298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461405+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:52:44.461405+0000","last_peered":"2023-11-28T02:52:44.461405+0000","last_clean":"2023-11-28T02:52:44.461405+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:52:44.461405+0000","last_undegraded":"2023-11-28T02:52:44.461405+0000","last_fullsized":"2023-11-28T02:52:44.461405+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476368+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:52:44.476368+0000","last_peered":"2023-11-28T02:52:44.476368+0000","last_clean":"2023-11-28T02:52:44.476368+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:52:44.476368+0000","last_undegraded":"2023-11-28T02:52:44.476368+0000","last_fullsized":"2023-11-28T02:52:44.476368+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453788+0000","last_change":"2023-11-28T02:52:44.095277+0000","last_active":"2023-11-28T02:52:44.453788+0000","last_peered":"2023-11-28T02:52:44.453788+0000","last_clean":"2023-11-28T02:52:44.453788+0000","last_became_active":"2023-11-28T02:52:44.095021+0000","last_became_peered":"2023-11-28T02:52:44.095021+0000","last_unstale":"2023-11-28T02:52:44.453788+0000","last_undegraded":"2023-11-28T02:52:44.453788+0000","last_fullsized":"2023-11-28T02:52:44.453788+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469245+0000","last_change":"2023-11-28T02:52:44.464858+0000","last_active":"2023-11-28T02:52:44.469245+0000","last_peered":"2023-11-28T02:52:44.469245+0000","last_clean":"2023-11-28T02:52:44.469245+0000","last_became_active":"2023-11-28T02:52:44.464352+0000","last_became_peered":"2023-11-28T02:52:44.464352+0000","last_unstale":"2023-11-28T02:52:44.469245+0000","last_undegraded":"2023-11-28T02:52:44.469245+0000","last_fullsized":"2023-11-28T02:52:44.469245+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476767+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:52:44.476767+0000","last_peered":"2023-11-28T02:52:44.476767+0000","last_clean":"2023-11-28T02:52:44.476767+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:52:44.476767+0000","last_undegraded":"2023-11-28T02:52:44.476767+0000","last_fullsized":"2023-11-28T02:52:44.476767+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450488+0000","last_change":"2023-11-28T02:52:44.105713+0000","last_active":"2023-11-28T02:52:44.450488+0000","last_peered":"2023-11-28T02:52:44.450488+0000","last_clean":"2023-11-28T02:52:44.450488+0000","last_became_active":"2023-11-28T02:52:44.105352+0000","last_became_peered":"2023-11-28T02:52:44.105352+0000","last_unstale":"2023-11-28T02:52:44.450488+0000","last_undegraded":"2023-11-28T02:52:44.450488+0000","last_fullsized":"2023-11-28T02:52:44.450488+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.474530+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:52:44.474530+0000","last_peered":"2023-11-28T02:52:44.474530+0000","last_clean":"2023-11-28T02:52:44.474530+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:52:44.474530+0000","last_undegraded":"2023-11-28T02:52:44.474530+0000","last_fullsized":"2023-11-28T02:52:44.474530+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464373+0000","last_change":"2023-11-28T02:52:44.096162+0000","last_active":"2023-11-28T02:52:44.464373+0000","last_peered":"2023-11-28T02:52:44.464373+0000","last_clean":"2023-11-28T02:52:44.464373+0000","last_became_active":"2023-11-28T02:52:44.095900+0000","last_became_peered":"2023-11-28T02:52:44.095900+0000","last_unstale":"2023-11-28T02:52:44.464373+0000","last_undegraded":"2023-11-28T02:52:44.464373+0000","last_fullsized":"2023-11-28T02:52:44.464373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470164+0000","last_change":"2023-11-28T02:52:44.464449+0000","last_active":"2023-11-28T02:52:44.470164+0000","last_peered":"2023-11-28T02:52:44.470164+0000","last_clean":"2023-11-28T02:52:44.470164+0000","last_became_active":"2023-11-28T02:52:44.464196+0000","last_became_peered":"2023-11-28T02:52:44.464196+0000","last_unstale":"2023-11-28T02:52:44.470164+0000","last_undegraded":"2023-11-28T02:52:44.470164+0000","last_fullsized":"2023-11-28T02:52:44.470164+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478810+0000","last_change":"2023-11-28T02:52:44.094471+0000","last_active":"2023-11-28T02:52:44.478810+0000","last_peered":"2023-11-28T02:52:44.478810+0000","last_clean":"2023-11-28T02:52:44.478810+0000","last_became_active":"2023-11-28T02:52:44.094193+0000","last_became_peered":"2023-11-28T02:52:44.094193+0000","last_unstale":"2023-11-28T02:52:44.478810+0000","last_undegraded":"2023-11-28T02:52:44.478810+0000","last_fullsized":"2023-11-28T02:52:44.478810+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464206+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:52:44.464206+0000","last_peered":"2023-11-28T02:52:44.464206+0000","last_clean":"2023-11-28T02:52:44.464206+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:52:44.464206+0000","last_undegraded":"2023-11-28T02:52:44.464206+0000","last_fullsized":"2023-11-28T02:52:44.464206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479397+0000","last_change":"2023-11-28T02:52:44.478383+0000","last_active":"2023-11-28T02:52:44.479397+0000","last_peered":"2023-11-28T02:52:44.479397+0000","last_clean":"2023-11-28T02:52:44.479397+0000","last_became_active":"2023-11-28T02:52:44.478122+0000","last_became_peered":"2023-11-28T02:52:44.478122+0000","last_unstale":"2023-11-28T02:52:44.479397+0000","last_undegraded":"2023-11-28T02:52:44.479397+0000","last_fullsized":"2023-11-28T02:52:44.479397+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463683+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:52:44.463683+0000","last_peered":"2023-11-28T02:52:44.463683+0000","last_clean":"2023-11-28T02:52:44.463683+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:52:44.463683+0000","last_undegraded":"2023-11-28T02:52:44.463683+0000","last_fullsized":"2023-11-28T02:52:44.463683+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451382+0000","last_change":"2023-11-28T02:52:44.099058+0000","last_active":"2023-11-28T02:52:44.451382+0000","last_peered":"2023-11-28T02:52:44.451382+0000","last_clean":"2023-11-28T02:52:44.451382+0000","last_became_active":"2023-11-28T02:52:44.098489+0000","last_became_peered":"2023-11-28T02:52:44.098489+0000","last_unstale":"2023-11-28T02:52:44.451382+0000","last_undegraded":"2023-11-28T02:52:44.451382+0000","last_fullsized":"2023-11-28T02:52:44.451382+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475427+0000","last_change":"2023-11-28T02:52:44.473356+0000","last_active":"2023-11-28T02:52:44.475427+0000","last_peered":"2023-11-28T02:52:44.475427+0000","last_clean":"2023-11-28T02:52:44.475427+0000","last_became_active":"2023-11-28T02:52:44.473032+0000","last_became_peered":"2023-11-28T02:52:44.473032+0000","last_unstale":"2023-11-28T02:52:44.475427+0000","last_undegraded":"2023-11-28T02:52:44.475427+0000","last_fullsized":"2023-11-28T02:52:44.475427+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471249+0000","last_change":"2023-11-28T02:52:44.469050+0000","last_active":"2023-11-28T02:52:44.471249+0000","last_peered":"2023-11-28T02:52:44.471249+0000","last_clean":"2023-11-28T02:52:44.471249+0000","last_became_active":"2023-11-28T02:52:44.468695+0000","last_became_peered":"2023-11-28T02:52:44.468695+0000","last_unstale":"2023-11-28T02:52:44.471249+0000","last_undegraded":"2023-11-28T02:52:44.471249+0000","last_fullsized":"2023-11-28T02:52:44.471249+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452775+0000","last_change":"2023-11-28T02:52:44.105717+0000","last_active":"2023-11-28T02:52:44.452775+0000","last_peered":"2023-11-28T02:52:44.452775+0000","last_clean":"2023-11-28T02:52:44.452775+0000","last_became_active":"2023-11-28T02:52:44.105369+0000","last_became_peered":"2023-11-28T02:52:44.105369+0000","last_unstale":"2023-11-28T02:52:44.452775+0000","last_undegraded":"2023-11-28T02:52:44.452775+0000","last_fullsized":"2023-11-28T02:52:44.452775+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465888+0000","last_change":"2023-11-28T02:52:44.463547+0000","last_active":"2023-11-28T02:52:44.465888+0000","last_peered":"2023-11-28T02:52:44.465888+0000","last_clean":"2023-11-28T02:52:44.465888+0000","last_became_active":"2023-11-28T02:52:44.463254+0000","last_became_peered":"2023-11-28T02:52:44.463254+0000","last_unstale":"2023-11-28T02:52:44.465888+0000","last_undegraded":"2023-11-28T02:52:44.465888+0000","last_fullsized":"2023-11-28T02:52:44.465888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457589+0000","last_change":"2023-11-28T02:52:44.102948+0000","last_active":"2023-11-28T02:52:44.457589+0000","last_peered":"2023-11-28T02:52:44.457589+0000","last_clean":"2023-11-28T02:52:44.457589+0000","last_became_active":"2023-11-28T02:52:44.102609+0000","last_became_peered":"2023-11-28T02:52:44.102609+0000","last_unstale":"2023-11-28T02:52:44.457589+0000","last_undegraded":"2023-11-28T02:52:44.457589+0000","last_fullsized":"2023-11-28T02:52:44.457589+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.465782+0000","last_change":"2023-11-28T02:52:44.460962+0000","last_active":"2023-11-28T02:52:44.465782+0000","last_peered":"2023-11-28T02:52:44.465782+0000","last_clean":"2023-11-28T02:52:44.465782+0000","last_became_active":"2023-11-28T02:52:44.460662+0000","last_became_peered":"2023-11-28T02:52:44.460662+0000","last_unstale":"2023-11-28T02:52:44.465782+0000","last_undegraded":"2023-11-28T02:52:44.465782+0000","last_fullsized":"2023-11-28T02:52:44.465782+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469497+0000","last_change":"2023-11-28T02:52:44.104319+0000","last_active":"2023-11-28T02:52:44.469497+0000","last_peered":"2023-11-28T02:52:44.469497+0000","last_clean":"2023-11-28T02:52:44.469497+0000","last_became_active":"2023-11-28T02:52:44.104055+0000","last_became_peered":"2023-11-28T02:52:44.104055+0000","last_unstale":"2023-11-28T02:52:44.469497+0000","last_undegraded":"2023-11-28T02:52:44.469497+0000","last_fullsized":"2023-11-28T02:52:44.469497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466536+0000","last_change":"2023-11-28T02:52:44.460800+0000","last_active":"2023-11-28T02:52:44.466536+0000","last_peered":"2023-11-28T02:52:44.466536+0000","last_clean":"2023-11-28T02:52:44.466536+0000","last_became_active":"2023-11-28T02:52:44.460518+0000","last_became_peered":"2023-11-28T02:52:44.460518+0000","last_unstale":"2023-11-28T02:52:44.466536+0000","last_undegraded":"2023-11-28T02:52:44.466536+0000","last_fullsized":"2023-11-28T02:52:44.466536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466150+0000","last_change":"2023-11-28T02:52:44.093970+0000","last_active":"2023-11-28T02:52:44.466150+0000","last_peered":"2023-11-28T02:52:44.466150+0000","last_clean":"2023-11-28T02:52:44.466150+0000","last_became_active":"2023-11-28T02:52:44.093701+0000","last_became_peered":"2023-11-28T02:52:44.093701+0000","last_unstale":"2023-11-28T02:52:44.466150+0000","last_undegraded":"2023-11-28T02:52:44.466150+0000","last_fullsized":"2023-11-28T02:52:44.466150+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475064+0000","last_change":"2023-11-28T02:52:44.469053+0000","last_active":"2023-11-28T02:52:44.475064+0000","last_peered":"2023-11-28T02:52:44.475064+0000","last_clean":"2023-11-28T02:52:44.475064+0000","last_became_active":"2023-11-28T02:52:44.468580+0000","last_became_peered":"2023-11-28T02:52:44.468580+0000","last_unstale":"2023-11-28T02:52:44.475064+0000","last_undegraded":"2023-11-28T02:52:44.475064+0000","last_fullsized":"2023-11-28T02:52:44.475064+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1f","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464009+0000","last_change":"2023-11-28T02:52:44.099449+0000","last_active":"2023-11-28T02:52:44.464009+0000","last_peered":"2023-11-28T02:52:44.464009+0000","last_clean":"2023-11-28T02:52:44.464009+0000","last_became_active":"2023-11-28T02:52:44.099097+0000","last_became_peered":"2023-11-28T02:52:44.099097+0000","last_unstale":"2023-11-28T02:52:44.464009+0000","last_undegraded":"2023-11-28T02:52:44.464009+0000","last_fullsized":"2023-11-28T02:52:44.464009+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470464+0000","last_change":"2023-11-28T02:52:44.467418+0000","last_active":"2023-11-28T02:52:44.470464+0000","last_peered":"2023-11-28T02:52:44.470464+0000","last_clean":"2023-11-28T02:52:44.470464+0000","last_became_active":"2023-11-28T02:52:44.465656+0000","last_became_peered":"2023-11-28T02:52:44.465656+0000","last_unstale":"2023-11-28T02:52:44.470464+0000","last_undegraded":"2023-11-28T02:52:44.470464+0000","last_fullsized":"2023-11-28T02:52:44.470464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462541+0000","last_change":"2023-11-28T02:52:44.095567+0000","last_active":"2023-11-28T02:52:44.462541+0000","last_peered":"2023-11-28T02:52:44.462541+0000","last_clean":"2023-11-28T02:52:44.462541+0000","last_became_active":"2023-11-28T02:52:44.095290+0000","last_became_peered":"2023-11-28T02:52:44.095290+0000","last_unstale":"2023-11-28T02:52:44.462541+0000","last_undegraded":"2023-11-28T02:52:44.462541+0000","last_fullsized":"2023-11-28T02:52:44.462541+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.467572+0000","last_change":"2023-11-28T02:52:44.464051+0000","last_active":"2023-11-28T02:52:44.467572+0000","last_peered":"2023-11-28T02:52:44.467572+0000","last_clean":"2023-11-28T02:52:44.467572+0000","last_became_active":"2023-11-28T02:52:44.463832+0000","last_became_peered":"2023-11-28T02:52:44.463832+0000","last_unstale":"2023-11-28T02:52:44.467572+0000","last_undegraded":"2023-11-28T02:52:44.467572+0000","last_fullsized":"2023-11-28T02:52:44.467572+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448090+0000","last_change":"2023-11-28T02:52:44.092817+0000","last_active":"2023-11-28T02:52:44.448090+0000","last_peered":"2023-11-28T02:52:44.448090+0000","last_clean":"2023-11-28T02:52:44.448090+0000","last_became_active":"2023-11-28T02:52:44.092559+0000","last_became_peered":"2023-11-28T02:52:44.092559+0000","last_unstale":"2023-11-28T02:52:44.448090+0000","last_undegraded":"2023-11-28T02:52:44.448090+0000","last_fullsized":"2023-11-28T02:52:44.448090+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477920+0000","last_change":"2023-11-28T02:52:44.476928+0000","last_active":"2023-11-28T02:52:44.477920+0000","last_peered":"2023-11-28T02:52:44.477920+0000","last_clean":"2023-11-28T02:52:44.477920+0000","last_became_active":"2023-11-28T02:52:44.476404+0000","last_became_peered":"2023-11-28T02:52:44.476404+0000","last_unstale":"2023-11-28T02:52:44.477920+0000","last_undegraded":"2023-11-28T02:52:44.477920+0000","last_fullsized":"2023-11-28T02:52:44.477920+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461903+0000","last_change":"2023-11-28T02:52:44.100532+0000","last_active":"2023-11-28T02:52:44.461903+0000","last_peered":"2023-11-28T02:52:44.461903+0000","last_clean":"2023-11-28T02:52:44.461903+0000","last_became_active":"2023-11-28T02:52:44.100241+0000","last_became_peered":"2023-11-28T02:52:44.100241+0000","last_unstale":"2023-11-28T02:52:44.461903+0000","last_undegraded":"2023-11-28T02:52:44.461903+0000","last_fullsized":"2023-11-28T02:52:44.461903+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477206+0000","last_change":"2023-11-28T02:52:44.476740+0000","last_active":"2023-11-28T02:52:44.477206+0000","last_peered":"2023-11-28T02:52:44.477206+0000","last_clean":"2023-11-28T02:52:44.477206+0000","last_became_active":"2023-11-28T02:52:44.476473+0000","last_became_peered":"2023-11-28T02:52:44.476473+0000","last_unstale":"2023-11-28T02:52:44.477206+0000","last_undegraded":"2023-11-28T02:52:44.477206+0000","last_fullsized":"2023-11-28T02:52:44.477206+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.452872+0000","last_change":"2023-11-28T02:52:44.101558+0000","last_active":"2023-11-28T02:52:44.452872+0000","last_peered":"2023-11-28T02:52:44.452872+0000","last_clean":"2023-11-28T02:52:44.452872+0000","last_became_active":"2023-11-28T02:52:44.101242+0000","last_became_peered":"2023-11-28T02:52:44.101242+0000","last_unstale":"2023-11-28T02:52:44.452872+0000","last_undegraded":"2023-11-28T02:52:44.452872+0000","last_fullsized":"2023-11-28T02:52:44.452872+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.476230+0000","last_change":"2023-11-28T02:52:44.475794+0000","last_active":"2023-11-28T02:52:44.476230+0000","last_peered":"2023-11-28T02:52:44.476230+0000","last_clean":"2023-11-28T02:52:44.476230+0000","last_became_active":"2023-11-28T02:52:44.475520+0000","last_became_peered":"2023-11-28T02:52:44.475520+0000","last_unstale":"2023-11-28T02:52:44.476230+0000","last_undegraded":"2023-11-28T02:52:44.476230+0000","last_fullsized":"2023-11-28T02:52:44.476230+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454590+0000","last_change":"2023-11-28T02:52:44.102703+0000","last_active":"2023-11-28T02:52:44.454590+0000","last_peered":"2023-11-28T02:52:44.454590+0000","last_clean":"2023-11-28T02:52:44.454590+0000","last_became_active":"2023-11-28T02:52:44.102344+0000","last_became_peered":"2023-11-28T02:52:44.102344+0000","last_unstale":"2023-11-28T02:52:44.454590+0000","last_undegraded":"2023-11-28T02:52:44.454590+0000","last_fullsized":"2023-11-28T02:52:44.454590+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469219+0000","last_change":"2023-11-28T02:52:44.467020+0000","last_active":"2023-11-28T02:52:44.469219+0000","last_peered":"2023-11-28T02:52:44.469219+0000","last_clean":"2023-11-28T02:52:44.469219+0000","last_became_active":"2023-11-28T02:52:44.466737+0000","last_became_peered":"2023-11-28T02:52:44.466737+0000","last_unstale":"2023-11-28T02:52:44.469219+0000","last_undegraded":"2023-11-28T02:52:44.469219+0000","last_fullsized":"2023-11-28T02:52:44.469219+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462456+0000","last_change":"2023-11-28T02:52:44.095605+0000","last_active":"2023-11-28T02:52:44.462456+0000","last_peered":"2023-11-28T02:52:44.462456+0000","last_clean":"2023-11-28T02:52:44.462456+0000","last_became_active":"2023-11-28T02:52:44.093886+0000","last_became_peered":"2023-11-28T02:52:44.093886+0000","last_unstale":"2023-11-28T02:52:44.462456+0000","last_undegraded":"2023-11-28T02:52:44.462456+0000","last_fullsized":"2023-11-28T02:52:44.462456+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469773+0000","last_change":"2023-11-28T02:52:44.466446+0000","last_active":"2023-11-28T02:52:44.469773+0000","last_peered":"2023-11-28T02:52:44.469773+0000","last_clean":"2023-11-28T02:52:44.469773+0000","last_became_active":"2023-11-28T02:52:44.466194+0000","last_became_peered":"2023-11-28T02:52:44.466194+0000","last_unstale":"2023-11-28T02:52:44.469773+0000","last_undegraded":"2023-11-28T02:52:44.469773+0000","last_fullsized":"2023-11-28T02:52:44.469773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-11-28T02:52:45.452769+0000","last_change":"2023-11-28T02:52:44.089689+0000","last_active":"2023-11-28T02:52:45.452769+0000","last_peered":"2023-11-28T02:52:45.452769+0000","last_clean":"2023-11-28T02:52:45.452769+0000","last_became_active":"2023-11-28T02:52:44.089421+0000","last_became_peered":"2023-11-28T02:52:44.089421+0000","last_unstale":"2023-11-28T02:52:45.452769+0000","last_undegraded":"2023-11-28T02:52:45.452769+0000","last_fullsized":"2023-11-28T02:52:45.452769+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472331+0000","last_change":"2023-11-28T02:52:44.468436+0000","last_active":"2023-11-28T02:52:44.472331+0000","last_peered":"2023-11-28T02:52:44.472331+0000","last_clean":"2023-11-28T02:52:44.472331+0000","last_became_active":"2023-11-28T02:52:44.468102+0000","last_became_peered":"2023-11-28T02:52:44.468102+0000","last_unstale":"2023-11-28T02:52:44.472331+0000","last_undegraded":"2023-11-28T02:52:44.472331+0000","last_fullsized":"2023-11-28T02:52:44.472331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.457616+0000","last_change":"2023-11-28T02:52:44.104653+0000","last_active":"2023-11-28T02:52:44.457616+0000","last_peered":"2023-11-28T02:52:44.457616+0000","last_clean":"2023-11-28T02:52:44.457616+0000","last_became_active":"2023-11-28T02:52:44.104404+0000","last_became_peered":"2023-11-28T02:52:44.104404+0000","last_unstale":"2023-11-28T02:52:44.457616+0000","last_undegraded":"2023-11-28T02:52:44.457616+0000","last_fullsized":"2023-11-28T02:52:44.457616+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464296+0000","last_change":"2023-11-28T02:52:44.460305+0000","last_active":"2023-11-28T02:52:44.464296+0000","last_peered":"2023-11-28T02:52:44.464296+0000","last_clean":"2023-11-28T02:52:44.464296+0000","last_became_active":"2023-11-28T02:52:44.460040+0000","last_became_peered":"2023-11-28T02:52:44.460040+0000","last_unstale":"2023-11-28T02:52:44.464296+0000","last_undegraded":"2023-11-28T02:52:44.464296+0000","last_fullsized":"2023-11-28T02:52:44.464296+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461498+0000","last_change":"2023-11-28T02:52:44.103715+0000","last_active":"2023-11-28T02:52:44.461498+0000","last_peered":"2023-11-28T02:52:44.461498+0000","last_clean":"2023-11-28T02:52:44.461498+0000","last_became_active":"2023-11-28T02:52:44.103451+0000","last_became_peered":"2023-11-28T02:52:44.103451+0000","last_unstale":"2023-11-28T02:52:44.461498+0000","last_undegraded":"2023-11-28T02:52:44.461498+0000","last_fullsized":"2023-11-28T02:52:44.461498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464165+0000","last_change":"2023-11-28T02:52:44.460282+0000","last_active":"2023-11-28T02:52:44.464165+0000","last_peered":"2023-11-28T02:52:44.464165+0000","last_clean":"2023-11-28T02:52:44.464165+0000","last_became_active":"2023-11-28T02:52:44.460011+0000","last_became_peered":"2023-11-28T02:52:44.460011+0000","last_unstale":"2023-11-28T02:52:44.464165+0000","last_undegraded":"2023-11-28T02:52:44.464165+0000","last_fullsized":"2023-11-28T02:52:44.464165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.454784+0000","last_change":"2023-11-28T02:52:44.104288+0000","last_active":"2023-11-28T02:52:44.454784+0000","last_peered":"2023-11-28T02:52:44.454784+0000","last_clean":"2023-11-28T02:52:44.454784+0000","last_became_active":"2023-11-28T02:52:44.102746+0000","last_became_peered":"2023-11-28T02:52:44.102746+0000","last_unstale":"2023-11-28T02:52:44.454784+0000","last_undegraded":"2023-11-28T02:52:44.454784+0000","last_fullsized":"2023-11-28T02:52:44.454784+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471224+0000","last_change":"2023-11-28T02:52:44.468068+0000","last_active":"2023-11-28T02:52:44.471224+0000","last_peered":"2023-11-28T02:52:44.471224+0000","last_clean":"2023-11-28T02:52:44.471224+0000","last_became_active":"2023-11-28T02:52:44.467788+0000","last_became_peered":"2023-11-28T02:52:44.467788+0000","last_unstale":"2023-11-28T02:52:44.471224+0000","last_undegraded":"2023-11-28T02:52:44.471224+0000","last_fullsized":"2023-11-28T02:52:44.471224+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461485+0000","last_change":"2023-11-28T02:52:44.102690+0000","last_active":"2023-11-28T02:52:44.461485+0000","last_peered":"2023-11-28T02:52:44.461485+0000","last_clean":"2023-11-28T02:52:44.461485+0000","last_became_active":"2023-11-28T02:52:44.102408+0000","last_became_peered":"2023-11-28T02:52:44.102408+0000","last_unstale":"2023-11-28T02:52:44.461485+0000","last_undegraded":"2023-11-28T02:52:44.461485+0000","last_fullsized":"2023-11-28T02:52:44.461485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469649+0000","last_change":"2023-11-28T02:52:44.467827+0000","last_active":"2023-11-28T02:52:44.469649+0000","last_peered":"2023-11-28T02:52:44.469649+0000","last_clean":"2023-11-28T02:52:44.469649+0000","last_became_active":"2023-11-28T02:52:44.467547+0000","last_became_peered":"2023-11-28T02:52:44.467547+0000","last_unstale":"2023-11-28T02:52:44.469649+0000","last_undegraded":"2023-11-28T02:52:44.469649+0000","last_fullsized":"2023-11-28T02:52:44.469649+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.460918+0000","last_change":"2023-11-28T02:52:44.097889+0000","last_active":"2023-11-28T02:52:44.460918+0000","last_peered":"2023-11-28T02:52:44.460918+0000","last_clean":"2023-11-28T02:52:44.460918+0000","last_became_active":"2023-11-28T02:52:44.097641+0000","last_became_peered":"2023-11-28T02:52:44.097641+0000","last_unstale":"2023-11-28T02:52:44.460918+0000","last_undegraded":"2023-11-28T02:52:44.460918+0000","last_fullsized":"2023-11-28T02:52:44.460918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479544+0000","last_change":"2023-11-28T02:52:44.477649+0000","last_active":"2023-11-28T02:52:44.479544+0000","last_peered":"2023-11-28T02:52:44.479544+0000","last_clean":"2023-11-28T02:52:44.479544+0000","last_became_active":"2023-11-28T02:52:44.477212+0000","last_became_peered":"2023-11-28T02:52:44.477212+0000","last_unstale":"2023-11-28T02:52:44.479544+0000","last_undegraded":"2023-11-28T02:52:44.479544+0000","last_fullsized":"2023-11-28T02:52:44.479544+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461323+0000","last_change":"2023-11-28T02:52:44.098078+0000","last_active":"2023-11-28T02:52:44.461323+0000","last_peered":"2023-11-28T02:52:44.461323+0000","last_clean":"2023-11-28T02:52:44.461323+0000","last_became_active":"2023-11-28T02:52:44.097777+0000","last_became_peered":"2023-11-28T02:52:44.097777+0000","last_unstale":"2023-11-28T02:52:44.461323+0000","last_undegraded":"2023-11-28T02:52:44.461323+0000","last_fullsized":"2023-11-28T02:52:44.461323+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470412+0000","last_change":"2023-11-28T02:52:44.467211+0000","last_active":"2023-11-28T02:52:44.470412+0000","last_peered":"2023-11-28T02:52:44.470412+0000","last_clean":"2023-11-28T02:52:44.470412+0000","last_became_active":"2023-11-28T02:52:44.466934+0000","last_became_peered":"2023-11-28T02:52:44.466934+0000","last_unstale":"2023-11-28T02:52:44.470412+0000","last_undegraded":"2023-11-28T02:52:44.470412+0000","last_fullsized":"2023-11-28T02:52:44.470412+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2d","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479015+0000","last_change":"2023-11-28T02:52:44.097548+0000","last_active":"2023-11-28T02:52:44.479015+0000","last_peered":"2023-11-28T02:52:44.479015+0000","last_clean":"2023-11-28T02:52:44.479015+0000","last_became_active":"2023-11-28T02:52:44.097253+0000","last_became_peered":"2023-11-28T02:52:44.097253+0000","last_unstale":"2023-11-28T02:52:44.479015+0000","last_undegraded":"2023-11-28T02:52:44.479015+0000","last_fullsized":"2023-11-28T02:52:44.479015+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469674+0000","last_change":"2023-11-28T02:52:44.463550+0000","last_active":"2023-11-28T02:52:44.469674+0000","last_peered":"2023-11-28T02:52:44.469674+0000","last_clean":"2023-11-28T02:52:44.469674+0000","last_became_active":"2023-11-28T02:52:44.463313+0000","last_became_peered":"2023-11-28T02:52:44.463313+0000","last_unstale":"2023-11-28T02:52:44.469674+0000","last_undegraded":"2023-11-28T02:52:44.469674+0000","last_fullsized":"2023-11-28T02:52:44.469674+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.448095+0000","last_change":"2023-11-28T02:52:44.093490+0000","last_active":"2023-11-28T02:52:44.448095+0000","last_peered":"2023-11-28T02:52:44.448095+0000","last_clean":"2023-11-28T02:52:44.448095+0000","last_became_active":"2023-11-28T02:52:44.093241+0000","last_became_peered":"2023-11-28T02:52:44.093241+0000","last_unstale":"2023-11-28T02:52:44.448095+0000","last_undegraded":"2023-11-28T02:52:44.448095+0000","last_fullsized":"2023-11-28T02:52:44.448095+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485046+0000","last_change":"2023-11-28T02:52:44.480001+0000","last_active":"2023-11-28T02:52:44.485046+0000","last_peered":"2023-11-28T02:52:44.485046+0000","last_clean":"2023-11-28T02:52:44.485046+0000","last_became_active":"2023-11-28T02:52:44.479755+0000","last_became_peered":"2023-11-28T02:52:44.479755+0000","last_unstale":"2023-11-28T02:52:44.485046+0000","last_undegraded":"2023-11-28T02:52:44.485046+0000","last_fullsized":"2023-11-28T02:52:44.485046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463559+0000","last_change":"2023-11-28T02:52:44.102658+0000","last_active":"2023-11-28T02:52:44.463559+0000","last_peered":"2023-11-28T02:52:44.463559+0000","last_clean":"2023-11-28T02:52:44.463559+0000","last_became_active":"2023-11-28T02:52:44.102381+0000","last_became_peered":"2023-11-28T02:52:44.102381+0000","last_unstale":"2023-11-28T02:52:44.463559+0000","last_undegraded":"2023-11-28T02:52:44.463559+0000","last_fullsized":"2023-11-28T02:52:44.463559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464315+0000","last_change":"2023-11-28T02:52:44.460430+0000","last_active":"2023-11-28T02:52:44.464315+0000","last_peered":"2023-11-28T02:52:44.464315+0000","last_clean":"2023-11-28T02:52:44.464315+0000","last_became_active":"2023-11-28T02:52:44.460161+0000","last_became_peered":"2023-11-28T02:52:44.460161+0000","last_unstale":"2023-11-28T02:52:44.464315+0000","last_undegraded":"2023-11-28T02:52:44.464315+0000","last_fullsized":"2023-11-28T02:52:44.464315+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472669+0000","last_change":"2023-11-28T02:52:44.099082+0000","last_active":"2023-11-28T02:52:44.472669+0000","last_peered":"2023-11-28T02:52:44.472669+0000","last_clean":"2023-11-28T02:52:44.472669+0000","last_became_active":"2023-11-28T02:52:44.098444+0000","last_became_peered":"2023-11-28T02:52:44.098444+0000","last_unstale":"2023-11-28T02:52:44.472669+0000","last_undegraded":"2023-11-28T02:52:44.472669+0000","last_fullsized":"2023-11-28T02:52:44.472669+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472505+0000","last_change":"2023-11-28T02:52:44.467312+0000","last_active":"2023-11-28T02:52:44.472505+0000","last_peered":"2023-11-28T02:52:44.472505+0000","last_clean":"2023-11-28T02:52:44.472505+0000","last_became_active":"2023-11-28T02:52:44.467066+0000","last_became_peered":"2023-11-28T02:52:44.467066+0000","last_unstale":"2023-11-28T02:52:44.472505+0000","last_undegraded":"2023-11-28T02:52:44.472505+0000","last_fullsized":"2023-11-28T02:52:44.472505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462586+0000","last_change":"2023-11-28T02:52:44.089495+0000","last_active":"2023-11-28T02:52:44.462586+0000","last_peered":"2023-11-28T02:52:44.462586+0000","last_clean":"2023-11-28T02:52:44.462586+0000","last_became_active":"2023-11-28T02:52:44.089229+0000","last_became_peered":"2023-11-28T02:52:44.089229+0000","last_unstale":"2023-11-28T02:52:44.462586+0000","last_undegraded":"2023-11-28T02:52:44.462586+0000","last_fullsized":"2023-11-28T02:52:44.462586+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469877+0000","last_change":"2023-11-28T02:52:44.465836+0000","last_active":"2023-11-28T02:52:44.469877+0000","last_peered":"2023-11-28T02:52:44.469877+0000","last_clean":"2023-11-28T02:52:44.469877+0000","last_became_active":"2023-11-28T02:52:44.465430+0000","last_became_peered":"2023-11-28T02:52:44.465430+0000","last_unstale":"2023-11-28T02:52:44.469877+0000","last_undegraded":"2023-11-28T02:52:44.469877+0000","last_fullsized":"2023-11-28T02:52:44.469877+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.453296+0000","last_change":"2023-11-28T02:52:44.100936+0000","last_active":"2023-11-28T02:52:44.453296+0000","last_peered":"2023-11-28T02:52:44.453296+0000","last_clean":"2023-11-28T02:52:44.453296+0000","last_became_active":"2023-11-28T02:52:44.100435+0000","last_became_peered":"2023-11-28T02:52:44.100435+0000","last_unstale":"2023-11-28T02:52:44.453296+0000","last_undegraded":"2023-11-28T02:52:44.453296+0000","last_fullsized":"2023-11-28T02:52:44.453296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.468382+0000","last_change":"2023-11-28T02:52:44.464061+0000","last_active":"2023-11-28T02:52:44.468382+0000","last_peered":"2023-11-28T02:52:44.468382+0000","last_clean":"2023-11-28T02:52:44.468382+0000","last_became_active":"2023-11-28T02:52:44.463811+0000","last_became_peered":"2023-11-28T02:52:44.463811+0000","last_unstale":"2023-11-28T02:52:44.468382+0000","last_undegraded":"2023-11-28T02:52:44.468382+0000","last_fullsized":"2023-11-28T02:52:44.468382+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.466528+0000","last_change":"2023-11-28T02:52:44.095761+0000","last_active":"2023-11-28T02:52:44.466528+0000","last_peered":"2023-11-28T02:52:44.466528+0000","last_clean":"2023-11-28T02:52:44.466528+0000","last_became_active":"2023-11-28T02:52:44.095568+0000","last_became_peered":"2023-11-28T02:52:44.095568+0000","last_unstale":"2023-11-28T02:52:44.466528+0000","last_undegraded":"2023-11-28T02:52:44.466528+0000","last_fullsized":"2023-11-28T02:52:44.466528+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462346+0000","last_change":"2023-11-28T02:52:44.460026+0000","last_active":"2023-11-28T02:52:44.462346+0000","last_peered":"2023-11-28T02:52:44.462346+0000","last_clean":"2023-11-28T02:52:44.462346+0000","last_became_active":"2023-11-28T02:52:44.459771+0000","last_became_peered":"2023-11-28T02:52:44.459771+0000","last_unstale":"2023-11-28T02:52:44.462346+0000","last_undegraded":"2023-11-28T02:52:44.462346+0000","last_fullsized":"2023-11-28T02:52:44.462346+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.34","version":"21'4","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479985+0000","last_change":"2023-11-28T02:52:44.101052+0000","last_active":"2023-11-28T02:52:44.479985+0000","last_peered":"2023-11-28T02:52:44.479985+0000","last_clean":"2023-11-28T02:52:44.479985+0000","last_became_active":"2023-11-28T02:52:44.099171+0000","last_became_peered":"2023-11-28T02:52:44.099171+0000","last_unstale":"2023-11-28T02:52:44.479985+0000","last_undegraded":"2023-11-28T02:52:44.479985+0000","last_fullsized":"2023-11-28T02:52:44.479985+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.471983+0000","last_change":"2023-11-28T02:52:44.464434+0000","last_active":"2023-11-28T02:52:44.471983+0000","last_peered":"2023-11-28T02:52:44.471983+0000","last_clean":"2023-11-28T02:52:44.471983+0000","last_became_active":"2023-11-28T02:52:44.463883+0000","last_became_peered":"2023-11-28T02:52:44.463883+0000","last_unstale":"2023-11-28T02:52:44.471983+0000","last_undegraded":"2023-11-28T02:52:44.471983+0000","last_fullsized":"2023-11-28T02:52:44.471983+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.462246+0000","last_change":"2023-11-28T02:52:44.099394+0000","last_active":"2023-11-28T02:52:44.462246+0000","last_peered":"2023-11-28T02:52:44.462246+0000","last_clean":"2023-11-28T02:52:44.462246+0000","last_became_active":"2023-11-28T02:52:44.099099+0000","last_became_peered":"2023-11-28T02:52:44.099099+0000","last_unstale":"2023-11-28T02:52:44.462246+0000","last_undegraded":"2023-11-28T02:52:44.462246+0000","last_fullsized":"2023-11-28T02:52:44.462246+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.480138+0000","last_change":"2023-11-28T02:52:44.478063+0000","last_active":"2023-11-28T02:52:44.480138+0000","last_peered":"2023-11-28T02:52:44.480138+0000","last_clean":"2023-11-28T02:52:44.480138+0000","last_became_active":"2023-11-28T02:52:44.477777+0000","last_became_peered":"2023-11-28T02:52:44.477777+0000","last_unstale":"2023-11-28T02:52:44.480138+0000","last_undegraded":"2023-11-28T02:52:44.480138+0000","last_fullsized":"2023-11-28T02:52:44.480138+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.461922+0000","last_change":"2023-11-28T02:52:44.099389+0000","last_active":"2023-11-28T02:52:44.461922+0000","last_peered":"2023-11-28T02:52:44.461922+0000","last_clean":"2023-11-28T02:52:44.461922+0000","last_became_active":"2023-11-28T02:52:44.099084+0000","last_became_peered":"2023-11-28T02:52:44.099084+0000","last_unstale":"2023-11-28T02:52:44.461922+0000","last_undegraded":"2023-11-28T02:52:44.461922+0000","last_fullsized":"2023-11-28T02:52:44.461922+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.459683+0000","last_change":"2023-11-28T02:52:44.454584+0000","last_active":"2023-11-28T02:52:44.459683+0000","last_peered":"2023-11-28T02:52:44.459683+0000","last_clean":"2023-11-28T02:52:44.459683+0000","last_became_active":"2023-11-28T02:52:44.454393+0000","last_became_peered":"2023-11-28T02:52:44.454393+0000","last_unstale":"2023-11-28T02:52:44.459683+0000","last_undegraded":"2023-11-28T02:52:44.459683+0000","last_fullsized":"2023-11-28T02:52:44.459683+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.478337+0000","last_change":"2023-11-28T02:52:44.098086+0000","last_active":"2023-11-28T02:52:44.478337+0000","last_peered":"2023-11-28T02:52:44.478337+0000","last_clean":"2023-11-28T02:52:44.478337+0000","last_became_active":"2023-11-28T02:52:44.097733+0000","last_became_peered":"2023-11-28T02:52:44.097733+0000","last_unstale":"2023-11-28T02:52:44.478337+0000","last_undegraded":"2023-11-28T02:52:44.478337+0000","last_fullsized":"2023-11-28T02:52:44.478337+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463401+0000","last_change":"2023-11-28T02:52:44.460723+0000","last_active":"2023-11-28T02:52:44.463401+0000","last_peered":"2023-11-28T02:52:44.463401+0000","last_clean":"2023-11-28T02:52:44.463401+0000","last_became_active":"2023-11-28T02:52:44.460443+0000","last_became_peered":"2023-11-28T02:52:44.460443+0000","last_unstale":"2023-11-28T02:52:44.463401+0000","last_undegraded":"2023-11-28T02:52:44.463401+0000","last_fullsized":"2023-11-28T02:52:44.463401+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.451507+0000","last_change":"2023-11-28T02:52:44.094043+0000","last_active":"2023-11-28T02:52:44.451507+0000","last_peered":"2023-11-28T02:52:44.451507+0000","last_clean":"2023-11-28T02:52:44.451507+0000","last_became_active":"2023-11-28T02:52:44.093761+0000","last_became_peered":"2023-11-28T02:52:44.093761+0000","last_unstale":"2023-11-28T02:52:44.451507+0000","last_undegraded":"2023-11-28T02:52:44.451507+0000","last_fullsized":"2023-11-28T02:52:44.451507+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.473075+0000","last_change":"2023-11-28T02:52:44.469033+0000","last_active":"2023-11-28T02:52:44.473075+0000","last_peered":"2023-11-28T02:52:44.473075+0000","last_clean":"2023-11-28T02:52:44.473075+0000","last_became_active":"2023-11-28T02:52:44.468811+0000","last_became_peered":"2023-11-28T02:52:44.468811+0000","last_unstale":"2023-11-28T02:52:44.473075+0000","last_undegraded":"2023-11-28T02:52:44.473075+0000","last_fullsized":"2023-11-28T02:52:44.473075+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.475940+0000","last_change":"2023-11-28T02:52:44.097577+0000","last_active":"2023-11-28T02:52:44.475940+0000","last_peered":"2023-11-28T02:52:44.475940+0000","last_clean":"2023-11-28T02:52:44.475940+0000","last_became_active":"2023-11-28T02:52:44.097283+0000","last_became_peered":"2023-11-28T02:52:44.097283+0000","last_unstale":"2023-11-28T02:52:44.475940+0000","last_undegraded":"2023-11-28T02:52:44.475940+0000","last_fullsized":"2023-11-28T02:52:44.475940+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464026+0000","last_change":"2023-11-28T02:52:44.459558+0000","last_active":"2023-11-28T02:52:44.464026+0000","last_peered":"2023-11-28T02:52:44.464026+0000","last_clean":"2023-11-28T02:52:44.464026+0000","last_became_active":"2023-11-28T02:52:44.459236+0000","last_became_peered":"2023-11-28T02:52:44.459236+0000","last_unstale":"2023-11-28T02:52:44.464026+0000","last_undegraded":"2023-11-28T02:52:44.464026+0000","last_fullsized":"2023-11-28T02:52:44.464026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3a","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469470+0000","last_change":"2023-11-28T02:52:44.098691+0000","last_active":"2023-11-28T02:52:44.469470+0000","last_peered":"2023-11-28T02:52:44.469470+0000","last_clean":"2023-11-28T02:52:44.469470+0000","last_became_active":"2023-11-28T02:52:44.098401+0000","last_became_peered":"2023-11-28T02:52:44.098401+0000","last_unstale":"2023-11-28T02:52:44.469470+0000","last_undegraded":"2023-11-28T02:52:44.469470+0000","last_fullsized":"2023-11-28T02:52:44.469470+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469747+0000","last_change":"2023-11-28T02:52:44.464788+0000","last_active":"2023-11-28T02:52:44.469747+0000","last_peered":"2023-11-28T02:52:44.469747+0000","last_clean":"2023-11-28T02:52:44.469747+0000","last_became_active":"2023-11-28T02:52:44.464491+0000","last_became_peered":"2023-11-28T02:52:44.464491+0000","last_unstale":"2023-11-28T02:52:44.469747+0000","last_undegraded":"2023-11-28T02:52:44.469747+0000","last_fullsized":"2023-11-28T02:52:44.469747+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3b","version":"21'1","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.477600+0000","last_change":"2023-11-28T02:52:44.095969+0000","last_active":"2023-11-28T02:52:44.477600+0000","last_peered":"2023-11-28T02:52:44.477600+0000","last_clean":"2023-11-28T02:52:44.477600+0000","last_became_active":"2023-11-28T02:52:44.095693+0000","last_became_peered":"2023-11-28T02:52:44.095693+0000","last_unstale":"2023-11-28T02:52:44.477600+0000","last_undegraded":"2023-11-28T02:52:44.477600+0000","last_fullsized":"2023-11-28T02:52:44.477600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.470142+0000","last_change":"2023-11-28T02:52:44.462943+0000","last_active":"2023-11-28T02:52:44.470142+0000","last_peered":"2023-11-28T02:52:44.470142+0000","last_clean":"2023-11-28T02:52:44.470142+0000","last_became_active":"2023-11-28T02:52:44.462685+0000","last_became_peered":"2023-11-28T02:52:44.462685+0000","last_unstale":"2023-11-28T02:52:44.470142+0000","last_undegraded":"2023-11-28T02:52:44.470142+0000","last_fullsized":"2023-11-28T02:52:44.470142+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"21'2","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.469567+0000","last_change":"2023-11-28T02:52:44.100318+0000","last_active":"2023-11-28T02:52:44.469567+0000","last_peered":"2023-11-28T02:52:44.469567+0000","last_clean":"2023-11-28T02:52:44.469567+0000","last_became_active":"2023-11-28T02:52:44.100056+0000","last_became_peered":"2023-11-28T02:52:44.100056+0000","last_unstale":"2023-11-28T02:52:44.469567+0000","last_undegraded":"2023-11-28T02:52:44.469567+0000","last_fullsized":"2023-11-28T02:52:44.469567+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.463773+0000","last_change":"2023-11-28T02:52:44.460144+0000","last_active":"2023-11-28T02:52:44.463773+0000","last_peered":"2023-11-28T02:52:44.463773+0000","last_clean":"2023-11-28T02:52:44.463773+0000","last_became_active":"2023-11-28T02:52:44.459861+0000","last_became_peered":"2023-11-28T02:52:44.459861+0000","last_unstale":"2023-11-28T02:52:44.463773+0000","last_undegraded":"2023-11-28T02:52:44.463773+0000","last_fullsized":"2023-11-28T02:52:44.463773+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.450475+0000","last_change":"2023-11-28T02:52:44.101546+0000","last_active":"2023-11-28T02:52:44.450475+0000","last_peered":"2023-11-28T02:52:44.450475+0000","last_clean":"2023-11-28T02:52:44.450475+0000","last_became_active":"2023-11-28T02:52:44.101254+0000","last_became_peered":"2023-11-28T02:52:44.101254+0000","last_unstale":"2023-11-28T02:52:44.450475+0000","last_undegraded":"2023-11-28T02:52:44.450475+0000","last_fullsized":"2023-11-28T02:52:44.450475+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.479536+0000","last_change":"2023-11-28T02:52:44.477542+0000","last_active":"2023-11-28T02:52:44.479536+0000","last_peered":"2023-11-28T02:52:44.479536+0000","last_clean":"2023-11-28T02:52:44.479536+0000","last_became_active":"2023-11-28T02:52:44.477191+0000","last_became_peered":"2023-11-28T02:52:44.477191+0000","last_unstale":"2023-11-28T02:52:44.479536+0000","last_undegraded":"2023-11-28T02:52:44.479536+0000","last_fullsized":"2023-11-28T02:52:44.479536+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.464374+0000","last_change":"2023-11-28T02:52:44.097950+0000","last_active":"2023-11-28T02:52:44.464374+0000","last_peered":"2023-11-28T02:52:44.464374+0000","last_clean":"2023-11-28T02:52:44.464374+0000","last_became_active":"2023-11-28T02:52:44.097703+0000","last_became_peered":"2023-11-28T02:52:44.097703+0000","last_unstale":"2023-11-28T02:52:44.464374+0000","last_undegraded":"2023-11-28T02:52:44.464374+0000","last_fullsized":"2023-11-28T02:52:44.464374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.472612+0000","last_change":"2023-11-28T02:52:44.466320+0000","last_active":"2023-11-28T02:52:44.472612+0000","last_peered":"2023-11-28T02:52:44.472612+0000","last_clean":"2023-11-28T02:52:44.472612+0000","last_became_active":"2023-11-28T02:52:44.466039+0000","last_became_peered":"2023-11-28T02:52:44.466039+0000","last_unstale":"2023-11-28T02:52:44.472612+0000","last_undegraded":"2023-11-28T02:52:44.472612+0000","last_fullsized":"2023-11-28T02:52:44.472612+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:44.069437+0000","last_clean_scrub_stamp":"2023-11-28T02:52:44.069437+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3f","version":"21'3","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-11-28T02:52:44.485236+0000","last_change":"2023-11-28T02:52:44.093193+0000","last_active":"2023-11-28T02:52:44.485236+0000","last_peered":"2023-11-28T02:52:44.485236+0000","last_clean":"2023-11-28T02:52:44.485236+0000","last_became_active":"2023-11-28T02:52:44.092869+0000","last_became_peered":"2023-11-28T02:52:44.092869+0000","last_unstale":"2023-11-28T02:52:44.485236+0000","last_undegraded":"2023-11-28T02:52:44.485236+0000","last_fullsized":"2023-11-28T02:52:44.485236+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:43.067314+0000","last_clean_scrub_stamp":"2023-11-28T02:52:43.067314+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297348,"kb_used_data":252,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074492,"statfs":{"total":96636764160,"available":96332279808,"internally_reserved":0,"allocated":258048,"data_stored":86755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297328,"kb_used_data":232,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074512,"statfs":{"total":96636764160,"available":96332300288,"internally_reserved":0,"allocated":237568,"data_stored":86787,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38900000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65300000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71899999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.24199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85411,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.20799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297324,"kb_used_data":228,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074516,"statfs":{"total":96636764160,"available":96332304384,"internally_reserved":0,"allocated":233472,"data_stored":85431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29699999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574853,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297340,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074500,"statfs":{"total":96636764160,"available":96332288000,"internally_reserved":0,"allocated":249856,"data_stored":86933,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297320,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074520,"statfs":{"total":96636764160,"available":96332308480,"internally_reserved":0,"allocated":229376,"data_stored":85341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50900000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1414,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:52:54.417 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T02:52:54.418 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-11-28T02:52:54.418 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-11-28T02:52:54.418 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-11-28T02:52:54.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.562+0000 7f6698f0b700 1 Processor -- start 2023-11-28T02:52:54.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.562+0000 7f6698f0b700 1 -- start start 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6698f0b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 0x7f66940c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6698f0b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 0x7f66940c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6698f0b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66940bfc00 con 0x7f6694154120 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6698f0b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66940c49b0 con 0x7f66940c7db0 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6698f0b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940beff0 con 0x7f6694154500 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6693fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 0x7f66940c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 0x7f66940c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f66937fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6694154120 0x7f66940be510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:49886/0 (socket says 172.21.15.73:49886) 2023-11-28T02:52:54.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.563+0000 7f66937fe700 1 -- 172.21.15.73:0/3859732068 learned_addr learned my addr 172.21.15.73:0/3859732068 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6692ffd700 1 -- 172.21.15.73:0/3859732068 >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 msgr2=0x7f66940c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6692ffd700 1 --2- 172.21.15.73:0/3859732068 >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 0x7f66940c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6692ffd700 1 -- 172.21.15.73:0/3859732068 >> v1:172.21.15.150:6789/0 conn(0x7f6694154120 legacy=0x7f66940be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f66927fc700 1 -- 172.21.15.73:0/3859732068 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1862977403 0 0) 0x7f66940bfc00 con 0x7f6694154120 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6692ffd700 1 -- 172.21.15.73:0/3859732068 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66940bf2d0 con 0x7f6694154500 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6693fff700 1 --2- 172.21.15.73:0/3859732068 >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 0x7f66940c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6692ffd700 1 --2- 172.21.15.73:0/3859732068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 0x7f66940c3fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6688009760 tx=0x7f6688009060).ready entity=mon.2 client_cookie=452159538c5ada25 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f66927fc700 1 -- 172.21.15.73:0/3859732068 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800f510 con 0x7f6694154500 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f66927fc700 1 -- 172.21.15.73:0/3859732068 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800f670 con 0x7f6694154500 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f66927fc700 1 -- 172.21.15.73:0/3859732068 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800f9f0 con 0x7f6694154500 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 -- 172.21.15.73:0/3859732068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 msgr2=0x7f66940c3fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3859732068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 0x7f66940c3fd0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6688009760 tx=0x7f6688009060).stop 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 -- 172.21.15.73:0/3859732068 shutdown_connections 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3859732068 >> v2:172.21.15.150:3300/0 conn(0x7f66940c7db0 0x7f66940c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3859732068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6694154500 0x7f66940c3fd0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 -- 172.21.15.73:0/3859732068 >> 172.21.15.73:0/3859732068 conn(0x7f6694083000 msgr2=0x7f6694092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 -- 172.21.15.73:0/3859732068 shutdown_connections 2023-11-28T02:52:54.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.564+0000 7f6698f0b700 1 -- 172.21.15.73:0/3859732068 wait complete. 2023-11-28T02:52:54.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 Processor -- start 2023-11-28T02:52:54.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 -- start start 2023-11-28T02:52:54.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66940c7db0 0x7f669414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694154500 0x7f66940acce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66940aab70 con 0x7f6694154120 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6694156db0 con 0x7f6694154500 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6698f0b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66940adc00 con 0x7f66940c7db0 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.566+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44890/0 (socket says 172.21.15.73:44890) 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 -- 172.21.15.73:0/3876681740 learned_addr learned my addr 172.21.15.73:0/3876681740 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6693fff700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694154500 0x7f66940acce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f66937fe700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66940c7db0 0x7f669414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 -- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66940c7db0 msgr2=0x7f669414fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66940c7db0 0x7f669414fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 -- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694154500 msgr2=0x7f66940acce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694154500 0x7f66940acce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 -- 172.21.15.73:0/3876681740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f668c0092a0 con 0x7f6694154120 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6692ffd700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6688009e00 tx=0x7f6688017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668800f7e0 con 0x7f6694154120 2023-11-28T02:52:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f668800f940 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66880092a0 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668801b660 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.567+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66940ad640 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.568+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f669406c7c0 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.568+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f668801ba50 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.569+0000 7f667b7fe700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 0x7f6680061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.569+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f668807bc60 con 0x7f6694154120 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.569+0000 7f66937fe700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 0x7f6680061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.569+0000 7f66937fe700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 0x7f6680061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f6684001fd0 tx=0x7f6684009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.573+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f668804cfe0 con 0x7f6694154120 2023-11-28T02:52:54.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.771+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f669406c130 con 0x7f6694154120 2023-11-28T02:52:54.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.771+0000 7f667b7fe700 1 -- 172.21.15.73:0/3876681740 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+45 (secure 0 0 0) 0x7f668801f080 con 0x7f6694154120 2023-11-28T02:52:54.773 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 msgr2=0x7f6680061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 0x7f6680061610 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 rx=0x7f6684001fd0 tx=0x7f6684009040).stop 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 msgr2=0x7f66940ac7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f6688009e00 tx=0x7f6688017000).stop 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 shutdown_connections 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f668005f150 0x7f6680061610 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6694154500 0x7f66940acce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66940c7db0 0x7f669414fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 --2- 172.21.15.73:0/3876681740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6694154120 0x7f66940ac7f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 >> 172.21.15.73:0/3876681740 conn(0x7f6694083000 msgr2=0x7f6694067e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.773+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 shutdown_connections 2023-11-28T02:52:54.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.774+0000 7f6698f0b700 1 -- 172.21.15.73:0/3876681740 wait complete. 2023-11-28T02:52:54.786 INFO:teuthology.orchestra.run.smithi073.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-11-28T02:52:54.787 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-11-28T02:52:54.787 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:54.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.898+0000 7f9f35b20700 1 Processor -- start 2023-11-28T02:52:54.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.898+0000 7f9f35b20700 1 -- start start 2023-11-28T02:52:54.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.899+0000 7f9f35b20700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f30150890 0x7f9f300bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.899+0000 7f9f35b20700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f301544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.899+0000 7f9f35b20700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f300c28c0 con 0x7f9f300bc320 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.899+0000 7f9f35b20700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f301552e0 con 0x7f9f30150890 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.899+0000 7f9f35b20700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f3006a4b0 con 0x7f9f3006a010 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f301544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2f7fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f30150890 0x7f9f300bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2effd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9f300bc320 0x7f9f30069900 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:49890/0 (socket says 172.21.15.73:49890) 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2effd700 1 -- 172.21.15.73:0/2882646696 learned_addr learned my addr 172.21.15.73:0/2882646696 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 -- 172.21.15.73:0/2882646696 >> v2:172.21.15.150:3300/0 conn(0x7f9f30150890 msgr2=0x7f9f300bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:54.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 --2- 172.21.15.73:0/2882646696 >> v2:172.21.15.150:3300/0 conn(0x7f9f30150890 0x7f9f300bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 -- 172.21.15.73:0/2882646696 >> v1:172.21.15.150:6789/0 conn(0x7f9f300bc320 legacy=0x7f9f30069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 -- 172.21.15.73:0/2882646696 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300c1f90 con 0x7f9f3006a010 2023-11-28T02:52:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2ffff700 1 --2- 172.21.15.73:0/2882646696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f301544f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9f28009760 tx=0x7f9f28009060).ready entity=mon.2 client_cookie=7ccbe6b0e9ff0a6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2e7fc700 1 -- 172.21.15.73:0/2882646696 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800f510 con 0x7f9f3006a010 2023-11-28T02:52:54.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2e7fc700 1 -- 172.21.15.73:0/2882646696 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f2800f670 con 0x7f9f3006a010 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.900+0000 7f9f2e7fc700 1 -- 172.21.15.73:0/2882646696 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2800f9f0 con 0x7f9f3006a010 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 -- 172.21.15.73:0/2882646696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 msgr2=0x7f9f301544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 --2- 172.21.15.73:0/2882646696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f301544f0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f9f28009760 tx=0x7f9f28009060).stop 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 -- 172.21.15.73:0/2882646696 shutdown_connections 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 --2- 172.21.15.73:0/2882646696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f301544f0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 --2- 172.21.15.73:0/2882646696 >> v2:172.21.15.150:3300/0 conn(0x7f9f30150890 0x7f9f300bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 -- 172.21.15.73:0/2882646696 >> 172.21.15.73:0/2882646696 conn(0x7f9f300631d0 msgr2=0x7f9f300c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:54.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 -- 172.21.15.73:0/2882646696 shutdown_connections 2023-11-28T02:52:54.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.901+0000 7f9f35b20700 1 -- 172.21.15.73:0/2882646696 wait complete. 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.902+0000 7f9f35b20700 1 Processor -- start 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.902+0000 7f9f35b20700 1 -- start start 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f300bf490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f30150890 0x7f9f3014ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300bfa50 con 0x7f9f300bc320 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300bfd30 con 0x7f9f30150890 2023-11-28T02:52:54.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f35b20700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f300c0010 con 0x7f9f3006a010 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44904/0 (socket says 172.21.15.73:44904) 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 -- 172.21.15.73:0/368182149 learned_addr learned my addr 172.21.15.73:0/368182149 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2ffff700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f30150890 0x7f9f3014ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2f7fe700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f300bf490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 -- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 msgr2=0x7f9f300bf490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f300bf490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 -- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f30150890 msgr2=0x7f9f3014ba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f30150890 0x7f9f3014ba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:54.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 -- 172.21.15.73:0/368182149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f280092a0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.903+0000 7f9f2effd700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9f2400c4c0 tx=0x7f9f24007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.904+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f2400ce10 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.904+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f240072e0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.904+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f300c02f0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.904+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f24007660 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.904+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f300a4ca0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f3006c860 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9f240059b0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f177fe700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 0x7f9f1c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9f240797b0 con 0x7f9f300bc320 2023-11-28T02:52:54.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f2f7fe700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 0x7f9f1c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.905+0000 7f9f2f7fe700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 0x7f9f1c061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9f2000f550 tx=0x7f9f2000b040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:54.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:54.909+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9f2404aa20 con 0x7f9f300bc320 2023-11-28T02:52:55.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.073+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f300c1cd0 con 0x7f9f300bc320 2023-11-28T02:52:55.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.074+0000 7f9f177fe700 1 -- 172.21.15.73:0/368182149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f9f2401b300 con 0x7f9f300bc320 2023-11-28T02:52:55.075 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:55.076 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 msgr2=0x7f9f1c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 0x7f9f1c061610 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 rx=0x7f9f2000f550 tx=0x7f9f2000b040).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 msgr2=0x7f9f300a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f9f2400c4c0 tx=0x7f9f24007dc0).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 shutdown_connections 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f1c05f150 0x7f9f1c061610 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f30150890 0x7f9f3014ba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f3006a010 0x7f9f300bf490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 --2- 172.21.15.73:0/368182149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f300bc320 0x7f9f300a47b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 >> 172.21.15.73:0/368182149 conn(0x7f9f300631d0 msgr2=0x7f9f300a5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 shutdown_connections 2023-11-28T02:52:55.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.076+0000 7f9f35b20700 1 -- 172.21.15.73:0/368182149 wait complete. 2023-11-28T02:52:55.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:52:55.090 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:52:55.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.203+0000 7f93cc59d700 1 Processor -- start 2023-11-28T02:52:55.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.203+0000 7f93cc59d700 1 -- start start 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93cc59d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93c414ed10 0x7f93c40a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93cc59d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c4067770 con 0x7f93c40a8280 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93cc59d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93c40be8d0 con 0x7f93c4158160 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93cc59d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93c4159180 con 0x7f93c414ed10 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93cab3a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f93c414ed10 0x7f93c40a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93ca339700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f93c40a8280 0x7f93c4155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44912/0 (socket says 172.21.15.73:44912) 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.204+0000 7f93ca339700 1 -- 172.21.15.73:0/4199106139 learned_addr learned my addr 172.21.15.73:0/4199106139 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cab3a700 1 -- 172.21.15.73:0/4199106139 >> v1:172.21.15.150:6789/0 conn(0x7f93c4158160 legacy=0x7f93c414e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cab3a700 1 -- 172.21.15.73:0/4199106139 >> v1:172.21.15.73:6789/0 conn(0x7f93c40a8280 legacy=0x7f93c4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cab3a700 1 -- 172.21.15.73:0/4199106139 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93c4158600 con 0x7f93c414ed10 2023-11-28T02:52:55.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cab3a700 1 --2- 172.21.15.73:0/4199106139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c414ed10 0x7f93c40a8660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f93bc009e60 tx=0x7f93bc009780).ready entity=mon.0 client_cookie=5bbc3de0d8156abc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93c9337700 1 -- 172.21.15.73:0/4199106139 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc00e070 con 0x7f93c414ed10 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93c9337700 1 -- 172.21.15.73:0/4199106139 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc00bc30 con 0x7f93c414ed10 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93c9337700 1 -- 172.21.15.73:0/4199106139 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc012410 con 0x7f93c414ed10 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 -- 172.21.15.73:0/4199106139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c414ed10 msgr2=0x7f93c40a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 --2- 172.21.15.73:0/4199106139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c414ed10 0x7f93c40a8660 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7f93bc009e60 tx=0x7f93bc009780).stop 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 -- 172.21.15.73:0/4199106139 shutdown_connections 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 --2- 172.21.15.73:0/4199106139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c414ed10 0x7f93c40a8660 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 -- 172.21.15.73:0/4199106139 >> 172.21.15.73:0/4199106139 conn(0x7f93c4062700 msgr2=0x7f93c4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 -- 172.21.15.73:0/4199106139 shutdown_connections 2023-11-28T02:52:55.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.205+0000 7f93cc59d700 1 -- 172.21.15.73:0/4199106139 wait complete. 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.206+0000 7f93cc59d700 1 Processor -- start 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 -- start start 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93c40a8280 0x7f93c41566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93c414ed10 0x7f93c4156be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c40c01e0 con 0x7f93c4158160 2023-11-28T02:52:55.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93c41573b0 con 0x7f93c40a8280 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cc59d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93c4157690 con 0x7f93c414ed10 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cab3a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cab3a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44914/0 (socket says 172.21.15.73:44914) 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93cab3a700 1 -- 172.21.15.73:0/145633093 learned_addr learned my addr 172.21.15.73:0/145633093 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.207+0000 7f93ca339700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93c40a8280 0x7f93c41566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93c9b38700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93c414ed10 0x7f93c4156be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 -- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93c414ed10 msgr2=0x7f93c4156be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93c414ed10 0x7f93c4156be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 -- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93c40a8280 msgr2=0x7f93c41566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93c40a8280 0x7f93c41566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 -- 172.21.15.73:0/145633093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93bc009a20 con 0x7f93c4158160 2023-11-28T02:52:55.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cab3a700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f93c4067900 tx=0x7f93c40bea60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc0117d0 con 0x7f93c4158160 2023-11-28T02:52:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93bc011930 con 0x7f93c4158160 2023-11-28T02:52:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93bc011cb0 con 0x7f93c4158160 2023-11-28T02:52:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93c4157970 con 0x7f93c4158160 2023-11-28T02:52:55.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.208+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93c41570d0 con 0x7f93c4158160 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.209+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93c40c0390 con 0x7f93c4158160 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.209+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f93bc01d630 con 0x7f93c4158160 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.210+0000 7f93ba7fc700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 0x7f93a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.210+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f93bc07c360 con 0x7f93c4158160 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.210+0000 7f93ca339700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 0x7f93a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.210+0000 7f93ca339700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 0x7f93a8061610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f93c0007fd0 tx=0x7f93c0009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.214+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f93bc04d630 con 0x7f93c4158160 2023-11-28T02:52:55.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.377+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f93c406c860 con 0x7f93c4158160 2023-11-28T02:52:55.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.378+0000 7f93ba7fc700 1 -- 172.21.15.73:0/145633093 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f93bc050c50 con 0x7f93c4158160 2023-11-28T02:52:55.379 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 msgr2=0x7f93a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 0x7f93a8061610 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 rx=0x7f93c0007fd0 tx=0x7f93c0009040).stop 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 msgr2=0x7f93c40a99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f93c4067900 tx=0x7f93c40bea60).stop 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 shutdown_connections 2023-11-28T02:52:55.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f93a805f150 0x7f93a8061610 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93c4158160 0x7f93c40a99b0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93c40a8280 0x7f93c41566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 --2- 172.21.15.73:0/145633093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93c414ed10 0x7f93c4156be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.379+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 >> 172.21.15.73:0/145633093 conn(0x7f93c4062700 msgr2=0x7f93c40c8880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.380+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 shutdown_connections 2023-11-28T02:52:55.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.380+0000 7f93cc59d700 1 -- 172.21.15.73:0/145633093 wait complete. 2023-11-28T02:52:55.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:52:55.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.510+0000 7f36bb73a700 1 Processor -- start 2023-11-28T02:52:55.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.510+0000 7f36bb73a700 1 -- start start 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36bb73a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 0x7f36b40c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36bb73a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36b40c8780 0x7f36b40bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36bb73a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36b40c7500 con 0x7f36b40bd430 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36bb73a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b413c200 con 0x7f36b40c8780 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36bb73a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36b40c68f0 con 0x7f36b413be20 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36b8cd5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36b40c8780 0x7f36b40bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36b9cd7700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f36b40bd430 0x7f36b4062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44920/0 (socket says 172.21.15.73:44920) 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.511+0000 7f36b9cd7700 1 -- 172.21.15.73:0/860253536 learned_addr learned my addr 172.21.15.73:0/860253536 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b94d6700 1 --2- 172.21.15.73:0/860253536 >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 0x7f36b40c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b8cd5700 1 -- 172.21.15.73:0/860253536 >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 msgr2=0x7f36b40c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b8cd5700 1 --2- 172.21.15.73:0/860253536 >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 0x7f36b40c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b8cd5700 1 -- 172.21.15.73:0/860253536 >> v1:172.21.15.73:6789/0 conn(0x7f36b40bd430 legacy=0x7f36b4062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b8cd5700 1 -- 172.21.15.73:0/860253536 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36b40c6bd0 con 0x7f36b40c8780 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b3fff700 1 -- 172.21.15.73:0/860253536 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140386208 0 0) 0x7f36b40c7500 con 0x7f36b40bd430 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b94d6700 1 --2- 172.21.15.73:0/860253536 >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 0x7f36b40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b8cd5700 1 --2- 172.21.15.73:0/860253536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40bcf40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f36b405f7f0 tx=0x7f36a8009060).ready entity=mon.0 client_cookie=306fe917c4a15dec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b3fff700 1 -- 172.21.15.73:0/860253536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800b580 con 0x7f36b40c8780 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b3fff700 1 -- 172.21.15.73:0/860253536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a800b6e0 con 0x7f36b40c8780 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36b3fff700 1 -- 172.21.15.73:0/860253536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800ba60 con 0x7f36b40c8780 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 -- 172.21.15.73:0/860253536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 msgr2=0x7f36b40bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 --2- 172.21.15.73:0/860253536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40bcf40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7f36b405f7f0 tx=0x7f36a8009060).stop 2023-11-28T02:52:55.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 -- 172.21.15.73:0/860253536 shutdown_connections 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 --2- 172.21.15.73:0/860253536 >> v2:172.21.15.150:3300/0 conn(0x7f36b413be20 0x7f36b40c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 --2- 172.21.15.73:0/860253536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40bcf40 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.512+0000 7f36bb73a700 1 -- 172.21.15.73:0/860253536 >> 172.21.15.73:0/860253536 conn(0x7f36b40642d0 msgr2=0x7f36b40c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.513+0000 7f36bb73a700 1 -- 172.21.15.73:0/860253536 shutdown_connections 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.513+0000 7f36bb73a700 1 -- 172.21.15.73:0/860253536 wait complete. 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.514+0000 7f36bb73a700 1 Processor -- start 2023-11-28T02:52:55.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.514+0000 7f36bb73a700 1 -- start start 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36b40bd430 0x7f36b4138fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 0x7f36b40c1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40aaf90 con 0x7f36b40c8780 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40c29f0 con 0x7f36b40bd430 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36b40aa1f0 con 0x7f36b413be20 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44944/0 (socket says 172.21.15.73:44944) 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 -- 172.21.15.73:0/3709747876 learned_addr learned my addr 172.21.15.73:0/3709747876 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b94d6700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36b40bd430 0x7f36b4138fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b9cd7700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 0x7f36b40c1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 -- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 msgr2=0x7f36b40c1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 0x7f36b40c1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 -- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36b40bd430 msgr2=0x7f36b4138fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36b40bd430 0x7f36b4138fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 -- 172.21.15.73:0/3709747876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36a80092a0 con 0x7f36b40c8780 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b9cd7700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 0x7f36b40c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b8cd5700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f36a8011000 tx=0x7f36a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a800b850 con 0x7f36b40c8780 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36a800b9b0 con 0x7f36b40c8780 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36b40aa4d0 con 0x7f36b40c8780 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36a801b660 con 0x7f36b40c8780 2023-11-28T02:52:55.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.515+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36b40c2680 con 0x7f36b40c8780 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.516+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36b4139490 con 0x7f36b40c8780 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.517+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f36a801b9d0 con 0x7f36b40c8780 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.517+0000 7f36b17fa700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 0x7f36a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.517+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f36a807bcc0 con 0x7f36b40c8780 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.517+0000 7f36b94d6700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 0x7f36a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.517+0000 7f36b94d6700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 0x7f36a0061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f36a400e4b0 tx=0x7f36a4007670).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.521+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f36a804cf90 con 0x7f36b40c8780 2023-11-28T02:52:55.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.684+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f36b406c860 con 0x7f36b40c8780 2023-11-28T02:52:55.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.684+0000 7f36b17fa700 1 -- 172.21.15.73:0/3709747876 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f36a801f020 con 0x7f36b40c8780 2023-11-28T02:52:55.685 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 msgr2=0x7f36a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 0x7f36a0061610 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 rx=0x7f36a400e4b0 tx=0x7f36a4007670).stop 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 msgr2=0x7f36b40a7980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f36a8011000 tx=0x7f36a8017000).stop 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 shutdown_connections 2023-11-28T02:52:55.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f36a005f150 0x7f36a0061610 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36b413be20 0x7f36b40c1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36b40bd430 0x7f36b4138fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 --2- 172.21.15.73:0/3709747876 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36b40c8780 0x7f36b40a7980 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 >> 172.21.15.73:0/3709747876 conn(0x7f36b40642d0 msgr2=0x7f36b40a5160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 shutdown_connections 2023-11-28T02:52:55.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.686+0000 7f36bb73a700 1 -- 172.21.15.73:0/3709747876 wait complete. 2023-11-28T02:52:55.699 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:52:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.814+0000 7fc6cb7c7700 1 Processor -- start 2023-11-28T02:52:55.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 -- start start 2023-11-28T02:52:55.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c4098d10 0x7fc6c40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6c4158320 0x7fc6c41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6c4158cc0 con 0x7fc6c40c6190 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6c40c72d0 con 0x7fc6c4158320 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6cb7c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c4064570 con 0x7fc6c4098d10 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6c9d64700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6c4158320 0x7fc6c41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6c8d62700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc6c40c6190 0x7fc6c4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44928/0 (socket says 172.21.15.73:44928) 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.815+0000 7fc6c8d62700 1 -- 172.21.15.73:0/821186099 learned_addr learned my addr 172.21.15.73:0/821186099 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9563700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c4098d10 0x7fc6c40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9d64700 1 -- 172.21.15.73:0/821186099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c4098d10 msgr2=0x7fc6c40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9d64700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c4098d10 0x7fc6c40c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c3fff700 1 -- 172.21.15.73:0/821186099 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107472714 0 0) 0x7fc6b8001110 con 0x7fc6c40c6190 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9d64700 1 -- 172.21.15.73:0/821186099 >> v1:172.21.15.73:6789/0 conn(0x7fc6c40c6190 legacy=0x7fc6c4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9d64700 1 -- 172.21.15.73:0/821186099 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6c4154bc0 con 0x7fc6c4158320 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c9d64700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4158320 0x7fc6c41587d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc6bc009760 tx=0x7fc6bc009060).ready entity=mon.0 client_cookie=201ebb5c02c2e0d7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c3fff700 1 -- 172.21.15.73:0/821186099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00b580 con 0x7fc6c4158320 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c3fff700 1 -- 172.21.15.73:0/821186099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00b6e0 con 0x7fc6c4158320 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6c3fff700 1 -- 172.21.15.73:0/821186099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00ba60 con 0x7fc6c4158320 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/821186099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4158320 msgr2=0x7fc6c41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.816+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4158320 0x7fc6c41587d0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fc6bc009760 tx=0x7fc6bc009060).stop 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/821186099 shutdown_connections 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c4098d10 0x7fc6c40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/821186099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4158320 0x7fc6c41587d0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/821186099 >> 172.21.15.73:0/821186099 conn(0x7fc6c40cb310 msgr2=0x7fc6c40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/821186099 shutdown_connections 2023-11-28T02:52:55.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.817+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/821186099 wait complete. 2023-11-28T02:52:55.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.818+0000 7fc6cb7c7700 1 Processor -- start 2023-11-28T02:52:55.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.818+0000 7fc6cb7c7700 1 -- start start 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 0x7fc6c414f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6c4158320 0x7fc6c40c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c40c2d80 con 0x7fc6c4098d10 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c41551f0 con 0x7fc6c4158320 2023-11-28T02:52:55.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6c414fc40 con 0x7fc6c40c6190 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44970/0 (socket says 172.21.15.73:44970) 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 -- 172.21.15.73:0/823228249 learned_addr learned my addr 172.21.15.73:0/823228249 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9d64700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6c4158320 0x7fc6c40c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c8d62700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 0x7fc6c414f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 -- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 msgr2=0x7fc6c414f3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 0x7fc6c414f3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 -- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6c4158320 msgr2=0x7fc6c40c4140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6c4158320 0x7fc6c40c4140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 -- 172.21.15.73:0/823228249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0092a0 con 0x7fc6c4098d10 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c8d62700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 0x7fc6c414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:55.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c9563700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fc6b400cfc0 tx=0x7fc6b4007890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b4016400 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b4016560 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b40168e0 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6c414ff80 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.819+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6c414f8e0 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.820+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6c406c860 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.821+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc6b4009030 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.821+0000 7fc6c17fa700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 0x7fc6b00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.821+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc6b407aa50 con 0x7fc6c4098d10 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.821+0000 7fc6c8d62700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 0x7fc6b00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:55.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.821+0000 7fc6c8d62700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 0x7fc6b00615e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc6c4084960 tx=0x7fc6b8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:55.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.825+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc6b404bcc0 con 0x7fc6c4098d10 2023-11-28T02:52:55.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.988+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc6c40c66c0 con 0x7fc6c4098d10 2023-11-28T02:52:55.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.989+0000 7fc6c17fa700 1 -- 172.21.15.73:0/823228249 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc6b404f2e0 con 0x7fc6c4098d10 2023-11-28T02:52:55.990 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 msgr2=0x7fc6b00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 0x7fc6b00615e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fc6c4084960 tx=0x7fc6b8009040).stop 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 msgr2=0x7fc6c40c2890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fc6b400cfc0 tx=0x7fc6b4007890).stop 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 shutdown_connections 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc6b005f120 0x7fc6b00615e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6c4158320 0x7fc6c40c4140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6c4098d10 0x7fc6c40c2890 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 --2- 172.21.15.73:0/823228249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6c40c6190 0x7fc6c414f3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:55.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 >> 172.21.15.73:0/823228249 conn(0x7fc6c40cb310 msgr2=0x7fc6c40c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:55.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.990+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 shutdown_connections 2023-11-28T02:52:55.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:55.991+0000 7fc6cb7c7700 1 -- 172.21.15.73:0/823228249 wait complete. 2023-11-28T02:52:56.004 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:52:56.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.119+0000 7f358c570700 1 Processor -- start 2023-11-28T02:52:56.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.119+0000 7f358c570700 1 -- start start 2023-11-28T02:52:56.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f358c570700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3584158160 0x7f35840a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f358c570700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f358414e600 0x7f358414ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f358c570700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3584067770 con 0x7f35840a8280 2023-11-28T02:52:56.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f358c570700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3584159180 con 0x7f3584158160 2023-11-28T02:52:56.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f358c570700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35840be8d0 con 0x7f358414e600 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f3589b0b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3584158160 0x7f35840a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f3589b0b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3584158160 0x7f35840a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44972/0 (socket says 172.21.15.73:44972) 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.120+0000 7f3589b0b700 1 -- 172.21.15.73:0/2806684823 learned_addr learned my addr 172.21.15.73:0/2806684823 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358ab0d700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f358414e600 0x7f358414ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f3589b0b700 1 -- 172.21.15.73:0/2806684823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f358414e600 msgr2=0x7f358414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f3589b0b700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f358414e600 0x7f358414ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f3589b0b700 1 -- 172.21.15.73:0/2806684823 >> v1:172.21.15.150:6789/0 conn(0x7f35840a8280 legacy=0x7f3584155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f3589b0b700 1 -- 172.21.15.73:0/2806684823 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3584158600 con 0x7f3584158160 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f3589b0b700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3584158160 0x7f35840a8660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3574006fd0 tx=0x7f35740090f0).ready entity=mon.0 client_cookie=c112020f06459bde server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358930a700 1 -- 172.21.15.73:0/2806684823 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400b600 con 0x7f3584158160 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358930a700 1 -- 172.21.15.73:0/2806684823 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400b760 con 0x7f3584158160 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358930a700 1 -- 172.21.15.73:0/2806684823 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400bae0 con 0x7f3584158160 2023-11-28T02:52:56.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358c570700 1 -- 172.21.15.73:0/2806684823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3584158160 msgr2=0x7f35840a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.121+0000 7f358c570700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3584158160 0x7f35840a8660 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f3574006fd0 tx=0x7f35740090f0).stop 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 -- 172.21.15.73:0/2806684823 shutdown_connections 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f358414e600 0x7f358414ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 --2- 172.21.15.73:0/2806684823 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3584158160 0x7f35840a8660 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 -- 172.21.15.73:0/2806684823 >> 172.21.15.73:0/2806684823 conn(0x7f3584062700 msgr2=0x7f3584062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 -- 172.21.15.73:0/2806684823 shutdown_connections 2023-11-28T02:52:56.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.122+0000 7f358c570700 1 -- 172.21.15.73:0/2806684823 wait complete. 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 Processor -- start 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 -- start start 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35840a8280 0x7f3584157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3584158160 0x7f35841566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35840a8c80 con 0x7f358414e600 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f358414c2e0 con 0x7f35840a8280 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358c570700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3584156f60 con 0x7f3584158160 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f3589b0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f3589b0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44978/0 (socket says 172.21.15.73:44978) 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f3589b0b700 1 -- 172.21.15.73:0/1518276289 learned_addr learned my addr 172.21.15.73:0/1518276289 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.123+0000 7f358a30c700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35840a8280 0x7f3584157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f358ab0d700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3584158160 0x7f35841566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 -- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3584158160 msgr2=0x7f35841566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3584158160 0x7f35841566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 -- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35840a8280 msgr2=0x7f3584157bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35840a8280 0x7f3584157bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 -- 172.21.15.73:0/1518276289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3574009370 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f3589b0b700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3584065f50 tx=0x7f3574017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400b8d0 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f357400ba30 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35840a8f20 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357401c670 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.124+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3584157240 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.125+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35840a9440 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.125+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f357401c9e0 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.126+0000 7f357a7fc700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 0x7f35680615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.126+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f357407be00 con 0x7f358414e600 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.126+0000 7f358a30c700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 0x7f35680615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.126+0000 7f358a30c700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 0x7f35680615e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3580003920 tx=0x7f3580000bc0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.130+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f35840a9440 con 0x7f358414e600 2023-11-28T02:52:56.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.294+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f35840a9440 con 0x7f358414e600 2023-11-28T02:52:56.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.295+0000 7f357a7fc700 1 -- 172.21.15.73:0/1518276289 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f35840a9440 con 0x7f358414e600 2023-11-28T02:52:56.297 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:52:56.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 msgr2=0x7f35680615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 0x7f35680615e0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7f3580003920 tx=0x7f3580000bc0).stop 2023-11-28T02:52:56.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 msgr2=0x7f358414bdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f3584065f50 tx=0x7f3574017000).stop 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 shutdown_connections 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f356805f120 0x7f35680615e0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3584158160 0x7f35841566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35840a8280 0x7f3584157bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 --2- 172.21.15.73:0/1518276289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f358414e600 0x7f358414bdf0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 >> 172.21.15.73:0/1518276289 conn(0x7f3584062700 msgr2=0x7f358414ce40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.296+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 shutdown_connections 2023-11-28T02:52:56.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.297+0000 7f358c570700 1 -- 172.21.15.73:0/1518276289 wait complete. 2023-11-28T02:52:56.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:52:56.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.423+0000 7fab3e1c7700 1 Processor -- start 2023-11-28T02:52:56.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.423+0000 7fab3e1c7700 1 -- start start 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab3e1c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab3e1c7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fab380ca180 0x7fab3815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab3e1c7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab380a9a90 con 0x7fab380cae90 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab3e1c7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab38154ce0 con 0x7fab380ca180 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab3e1c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380a8e80 con 0x7fab380caab0 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab37fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fab380ca180 0x7fab3815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab377fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab36ffd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fab380cae90 0x7fab380c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:49924/0 (socket says 172.21.15.73:49924) 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.424+0000 7fab36ffd700 1 -- 172.21.15.73:0/78492174 learned_addr learned my addr 172.21.15.73:0/78492174 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab37fff700 1 -- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 msgr2=0x7fab380aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab37fff700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab37fff700 1 -- 172.21.15.73:0/78492174 >> v1:172.21.15.150:6789/0 conn(0x7fab380cae90 legacy=0x7fab380c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab367fc700 1 -- 172.21.15.73:0/78492174 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4137907114 0 0) 0x7fab380a9a90 con 0x7fab380cae90 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab37fff700 1 -- 172.21.15.73:0/78492174 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab380a9160 con 0x7fab380ca180 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab377fe700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab37fff700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab3815e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fab30009760 tx=0x7fab30009060).ready entity=mon.1 client_cookie=f87161a4d5c179ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab367fc700 1 -- 172.21.15.73:0/78492174 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000f510 con 0x7fab380ca180 2023-11-28T02:52:56.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab367fc700 1 -- 172.21.15.73:0/78492174 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000f670 con 0x7fab380ca180 2023-11-28T02:52:56.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab367fc700 1 -- 172.21.15.73:0/78492174 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000f9f0 con 0x7fab380ca180 2023-11-28T02:52:56.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 -- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 msgr2=0x7fab3815e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab3815e3c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fab30009760 tx=0x7fab30009060).stop 2023-11-28T02:52:56.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 -- 172.21.15.73:0/78492174 shutdown_connections 2023-11-28T02:52:56.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab3815e3c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/78492174 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.425+0000 7fab3e1c7700 1 -- 172.21.15.73:0/78492174 >> 172.21.15.73:0/78492174 conn(0x7fab38094920 msgr2=0x7fab3806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.426+0000 7fab3e1c7700 1 -- 172.21.15.73:0/78492174 shutdown_connections 2023-11-28T02:52:56.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.426+0000 7fab3e1c7700 1 -- 172.21.15.73:0/78492174 wait complete. 2023-11-28T02:52:56.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 Processor -- start 2023-11-28T02:52:56.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 -- start start 2023-11-28T02:52:56.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab38163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380ae220 con 0x7fab380cae90 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab380ae470 con 0x7fab380ca180 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.427+0000 7fab3e1c7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab380ae750 con 0x7fab380caab0 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44992/0 (socket says 172.21.15.73:44992) 2023-11-28T02:52:56.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 -- 172.21.15.73:0/4109957213 learned_addr learned my addr 172.21.15.73:0/4109957213 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab377fe700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab38163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab36ffd700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 -- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 msgr2=0x7fab380c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 -- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 msgr2=0x7fab38163a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab38163a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 -- 172.21.15.73:0/4109957213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab28009a20 con 0x7fab380cae90 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab377fe700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab38163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab37fff700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fab30009e00 tx=0x7fab30017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3000f7e0 con 0x7fab380cae90 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab3000f940 con 0x7fab380cae90 2023-11-28T02:52:56.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab300092a0 con 0x7fab380cae90 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab3001b660 con 0x7fab380cae90 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.428+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab380aed10 con 0x7fab380cae90 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.429+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab380af080 con 0x7fab380cae90 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.429+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fab3001b9d0 con 0x7fab380cae90 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.430+0000 7fab1f7fe700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 0x7fab24061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.430+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fab3007bb20 con 0x7fab380cae90 2023-11-28T02:52:56.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.430+0000 7fab377fe700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 0x7fab24061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.430+0000 7fab377fe700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 0x7fab24061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fab2800eb30 tx=0x7fab2800b040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.434+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fab3004cdf0 con 0x7fab380cae90 2023-11-28T02:52:56.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.640+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fab3806c860 con 0x7fab380cae90 2023-11-28T02:52:56.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.641+0000 7fab1f7fe700 1 -- 172.21.15.73:0/4109957213 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7fab3001f020 con 0x7fab380cae90 2023-11-28T02:52:56.642 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:56.642 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:52:56.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 msgr2=0x7fab24061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 0x7fab24061610 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7fab2800eb30 tx=0x7fab2800b040).stop 2023-11-28T02:52:56.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 msgr2=0x7fab380c4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7fab30009e00 tx=0x7fab30017000).stop 2023-11-28T02:52:56.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 shutdown_connections 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab2405f150 0x7fab24061610 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab380cae90 0x7fab380c4490 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab380ca180 0x7fab38163a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 --2- 172.21.15.73:0/4109957213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab380caab0 0x7fab380c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 >> 172.21.15.73:0/4109957213 conn(0x7fab38094920 msgr2=0x7fab380c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.642+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 shutdown_connections 2023-11-28T02:52:56.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.643+0000 7fab3e1c7700 1 -- 172.21.15.73:0/4109957213 wait complete. 2023-11-28T02:52:56.647 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:52:56.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.767+0000 7fdb944e7700 1 Processor -- start 2023-11-28T02:52:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.767+0000 7fdb944e7700 1 -- start start 2023-11-28T02:52:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb944e7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb944e7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb8c0ca180 0x7fdb8c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb944e7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb8c0a9a90 con 0x7fdb8c0caab0 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb944e7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c1490 con 0x7fdb8c0ca180 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb944e7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c154ce0 con 0x7fdb8c0cae90 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb92a84700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdb8c0ca180 0x7fdb8c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb92283700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdb8c0caab0 0x7fdb8c0aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44936/0 (socket says 172.21.15.73:44936) 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.768+0000 7fdb92283700 1 -- 172.21.15.73:0/1849186643 learned_addr learned my addr 172.21.15.73:0/1849186643 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb91a82700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb92a84700 1 -- 172.21.15.73:0/1849186643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 msgr2=0x7fdb8c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb92a84700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb92a84700 1 -- 172.21.15.73:0/1849186643 >> v1:172.21.15.73:6789/0 conn(0x7fdb8c0caab0 legacy=0x7fdb8c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb91281700 1 -- 172.21.15.73:0/1849186643 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1156176751 0 0) 0x7fdb8c0a9a90 con 0x7fdb8c0caab0 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb92a84700 1 -- 172.21.15.73:0/1849186643 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb8c0aab80 con 0x7fdb8c0ca180 2023-11-28T02:52:56.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb92a84700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c0c0fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fdb84009e60 tx=0x7fdb84009780).ready entity=mon.0 client_cookie=634377a337baac2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb91281700 1 -- 172.21.15.73:0/1849186643 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb8400e070 con 0x7fdb8c0ca180 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb91281700 1 -- 172.21.15.73:0/1849186643 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb8400bc30 con 0x7fdb8c0ca180 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb91281700 1 -- 172.21.15.73:0/1849186643 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb84012410 con 0x7fdb8c0ca180 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb944e7700 1 -- 172.21.15.73:0/1849186643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 msgr2=0x7fdb8c0c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.769+0000 7fdb944e7700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c0c0fa0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7fdb84009e60 tx=0x7fdb84009780).stop 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 -- 172.21.15.73:0/1849186643 shutdown_connections 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c0c0fa0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 --2- 172.21.15.73:0/1849186643 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 -- 172.21.15.73:0/1849186643 >> 172.21.15.73:0/1849186643 conn(0x7fdb8c094920 msgr2=0x7fdb8c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 -- 172.21.15.73:0/1849186643 shutdown_connections 2023-11-28T02:52:56.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.770+0000 7fdb944e7700 1 -- 172.21.15.73:0/1849186643 wait complete. 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 Processor -- start 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 -- start start 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb8c0caab0 0x7fdb8c0c2540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c3e80 con 0x7fdb8c0ca180 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c3270 con 0x7fdb8c0caab0 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb944e7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb8c0c3550 con 0x7fdb8c0cae90 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.771+0000 7fdb92283700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45006/0 (socket says 172.21.15.73:45006) 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 -- 172.21.15.73:0/4129441056 learned_addr learned my addr 172.21.15.73:0/4129441056 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 -- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 msgr2=0x7fdb8c0c2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 -- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb8c0caab0 msgr2=0x7fdb8c0c2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb8c0caab0 0x7fdb8c0c2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 -- 172.21.15.73:0/4129441056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb84009a20 con 0x7fdb8c0ca180 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb92283700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fdb88001360 tx=0x7fdb88007b00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880072e0 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb88007440 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb8c0c3830 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb880058a0 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.772+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb8c0c2f20 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.773+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb8c06c860 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.773+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdb88005c50 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.773+0000 7fdb827fc700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 0x7fdb70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.774+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdb88079ac0 con 0x7fdb8c0ca180 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.774+0000 7fdb91a82700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 0x7fdb70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:56.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.774+0000 7fdb91a82700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 0x7fdb70061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fdb8c152d10 tx=0x7fdb7c008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:56.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.778+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdb8804ad90 con 0x7fdb8c0ca180 2023-11-28T02:52:56.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.942+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdb8c0a8e80 con 0x7fdb8c0ca180 2023-11-28T02:52:56.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.943+0000 7fdb827fc700 1 -- 172.21.15.73:0/4129441056 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fdb88079e60 con 0x7fdb8c0ca180 2023-11-28T02:52:56.945 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:56.945 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:56.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 msgr2=0x7fdb70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 0x7fdb70061610 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 rx=0x7fdb8c152d10 tx=0x7fdb7c008040).stop 2023-11-28T02:52:56.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 msgr2=0x7fdb8c1560b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:56.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7fdb88001360 tx=0x7fdb88007b00).stop 2023-11-28T02:52:56.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 shutdown_connections 2023-11-28T02:52:56.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb7005f150 0x7fdb70061610 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdb8c0cae90 0x7fdb8c0c2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:56.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdb8c0ca180 0x7fdb8c1560b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:56.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 --2- 172.21.15.73:0/4129441056 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdb8c0caab0 0x7fdb8c0c2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:56.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 >> 172.21.15.73:0/4129441056 conn(0x7fdb8c094920 msgr2=0x7fdb8c0aea50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:56.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 shutdown_connections 2023-11-28T02:52:56.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:56.945+0000 7fdb944e7700 1 -- 172.21.15.73:0/4129441056 wait complete. 2023-11-28T02:52:56.959 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:52:56.959 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.073+0000 7f98ca11f700 1 Processor -- start 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.073+0000 7f98ca11f700 1 -- start start 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98ca11f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 0x7f98c40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98ca11f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f98c4158320 0x7f98c41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98ca11f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98c4158cc0 con 0x7f98c4098d10 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98ca11f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98c40c72d0 con 0x7f98c4158320 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98ca11f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c4064570 con 0x7f98c40c6190 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c37fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f98c4098d10 0x7f98c4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44946/0 (socket says 172.21.15.73:44946) 2023-11-28T02:52:57.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c37fe700 1 -- 172.21.15.73:0/3959292547 learned_addr learned my addr 172.21.15.73:0/3959292547 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c2ffd700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 0x7f98c40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c3fff700 1 --2- 172.21.15.73:0/3959292547 >> v2:172.21.15.150:3300/0 conn(0x7f98c4158320 0x7f98c41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3067376488 0 0) 0x7f98b4001110 con 0x7f98c4098d10 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.074+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98ac0033f0 con 0x7f98c4098d10 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2554286606 0 0) 0x7f98ac0033f0 con 0x7f98c4098d10 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c3fff700 1 -- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 msgr2=0x7f98c40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c3fff700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 0x7f98c40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c3fff700 1 -- 172.21.15.73:0/3959292547 >> v1:172.21.15.73:6789/0 conn(0x7f98c4098d10 legacy=0x7f98c4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c3fff700 1 -- 172.21.15.73:0/3959292547 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c4154bc0 con 0x7f98c4158320 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c2ffd700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 0x7f98c40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c3fff700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4158320 0x7f98c41587d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f98bc009e60 tx=0x7f98bc009780).ready entity=mon.1 client_cookie=49c6d0eb1061f734 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98bc00e070 con 0x7f98c4158320 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98bc00b7a0 con 0x7f98c4158320 2023-11-28T02:52:57.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98c27fc700 1 -- 172.21.15.73:0/3959292547 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b4001600 con 0x7f98c4158320 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 -- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4158320 msgr2=0x7f98c41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4158320 0x7f98c41587d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7f98bc009e60 tx=0x7f98bc009780).stop 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 -- 172.21.15.73:0/3959292547 shutdown_connections 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4158320 0x7f98c41587d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3959292547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c40c6190 0x7f98c40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 -- 172.21.15.73:0/3959292547 >> 172.21.15.73:0/3959292547 conn(0x7f98c40cb310 msgr2=0x7f98c40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 -- 172.21.15.73:0/3959292547 shutdown_connections 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.075+0000 7f98ca11f700 1 -- 172.21.15.73:0/3959292547 wait complete. 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 Processor -- start 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 -- start start 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 0x7f98c40b02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 0x7f98c40c9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40afe50 con 0x7f98c40c6190 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40af240 con 0x7f98c4098d10 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98ca11f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98c40af520 con 0x7f98c4158320 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45010/0 (socket says 172.21.15.73:45010) 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.077+0000 7f98c2ffd700 1 -- 172.21.15.73:0/3329981737 learned_addr learned my addr 172.21.15.73:0/3329981737 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c3fff700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 0x7f98c40c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c37fe700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 0x7f98c40b02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 -- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 msgr2=0x7f98c40c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 0x7f98c40c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 -- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 msgr2=0x7f98c40b02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 0x7f98c40b02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 -- 172.21.15.73:0/3329981737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80092a0 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c37fe700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 0x7f98c40b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c3fff700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 0x7f98c40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98c2ffd700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f98b800e9d0 tx=0x7f98b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800c850 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98b800c9b0 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98bc009a20 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98b800cd30 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.078+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98c40afae0 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.079+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98c40affc0 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.079+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f98b8012600 con 0x7f98c40c6190 2023-11-28T02:52:57.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.080+0000 7f98ab7fe700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 0x7f98b00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.080+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f98b807a800 con 0x7f98c40c6190 2023-11-28T02:52:57.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.080+0000 7f98c37fe700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 0x7f98b00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.080+0000 7f98c37fe700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 0x7f98b00615e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f98c40c1960 tx=0x7f98b4009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.084+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f98b804bad0 con 0x7f98c40c6190 2023-11-28T02:52:57.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.246+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f98c406c860 con 0x7f98c40c6190 2023-11-28T02:52:57.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.246+0000 7f98ab7fe700 1 -- 172.21.15.73:0/3329981737 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f98b80160b0 con 0x7f98c40c6190 2023-11-28T02:52:57.248 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 msgr2=0x7f98b00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 0x7f98b00615e0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 rx=0x7f98c40c1960 tx=0x7f98b4009040).stop 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 msgr2=0x7f98c41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f98b800e9d0 tx=0x7f98b8009060).stop 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 shutdown_connections 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f98b005f120 0x7f98b00615e0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98c4158320 0x7f98c40c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f98c4098d10 0x7f98c40b02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 --2- 172.21.15.73:0/3329981737 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98c40c6190 0x7f98c41560d0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 >> 172.21.15.73:0/3329981737 conn(0x7f98c40cb310 msgr2=0x7f98c41538c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 shutdown_connections 2023-11-28T02:52:57.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.248+0000 7f98ca11f700 1 -- 172.21.15.73:0/3329981737 wait complete. 2023-11-28T02:52:57.261 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:52:57.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.376+0000 7fce47fff700 1 Processor -- start 2023-11-28T02:52:57.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.376+0000 7fce47fff700 1 -- start start 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce480a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce47fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce481514b0 0x7fce480c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce47fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce48063b40 con 0x7fce480bcea0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce47fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce480c2ce0 con 0x7fce481514b0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce47fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480c2fc0 con 0x7fce481510d0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce477fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fce480bcea0 0x7fce480bd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44962/0 (socket says 172.21.15.73:44962) 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce477fe700 1 -- 172.21.15.73:0/2342047818 learned_addr learned my addr 172.21.15.73:0/2342047818 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce467fc700 1 --2- 172.21.15.73:0/2342047818 >> v2:172.21.15.150:3300/0 conn(0x7fce481514b0 0x7fce480c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce46ffd700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce480a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3027048655 0 0) 0x7fce48063b40 con 0x7fce480bcea0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce300033f0 con 0x7fce480bcea0 2023-11-28T02:52:57.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1220492322 0 0) 0x7fce300033f0 con 0x7fce480bcea0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce467fc700 1 -- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 msgr2=0x7fce480a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce467fc700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce480a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce467fc700 1 -- 172.21.15.73:0/2342047818 >> v1:172.21.15.73:6789/0 conn(0x7fce480bcea0 legacy=0x7fce480bd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.377+0000 7fce467fc700 1 -- 172.21.15.73:0/2342047818 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce480c32a0 con 0x7fce481514b0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce467fc700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce481514b0 0x7fce480c37c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fce3c009790 tx=0x7fce3c009060).ready entity=mon.1 client_cookie=8ba006793f9424bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce400017f0 con 0x7fce481514b0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce3c011400 con 0x7fce481514b0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce45ffb700 1 -- 172.21.15.73:0/2342047818 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce3c011780 con 0x7fce481514b0 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 -- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce481514b0 msgr2=0x7fce480c37c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce481514b0 0x7fce480c37c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fce3c009790 tx=0x7fce3c009060).stop 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 -- 172.21.15.73:0/2342047818 shutdown_connections 2023-11-28T02:52:57.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce480a6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 --2- 172.21.15.73:0/2342047818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce481514b0 0x7fce480c37c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 -- 172.21.15.73:0/2342047818 >> 172.21.15.73:0/2342047818 conn(0x7fce480c4010 msgr2=0x7fce480c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 -- 172.21.15.73:0/2342047818 shutdown_connections 2023-11-28T02:52:57.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.378+0000 7fce47fff700 1 -- 172.21.15.73:0/2342047818 wait complete. 2023-11-28T02:52:57.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.379+0000 7fce47fff700 1 Processor -- start 2023-11-28T02:52:57.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 -- start start 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 0x7fce480a9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce48148d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480a94d0 con 0x7fce481514b0 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce480a88c0 con 0x7fce480bcea0 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce47fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce480a8ba0 con 0x7fce481510d0 2023-11-28T02:52:57.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce477fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce46ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 0x7fce480a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce477fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45018/0 (socket says 172.21.15.73:45018) 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.380+0000 7fce477fe700 1 -- 172.21.15.73:0/3398248150 learned_addr learned my addr 172.21.15.73:0/3398248150 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce467fc700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce48148d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 -- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 msgr2=0x7fce48148d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce48148d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 -- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 msgr2=0x7fce480a9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 0x7fce480a9960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 -- 172.21.15.73:0/3398248150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce3c0092a0 con 0x7fce481514b0 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce46ffd700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 0x7fce480a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce477fe700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fce4000cb90 tx=0x7fce40001d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce4000cea0 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce40004c40 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce480a8e80 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce40007170 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.381+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce480a9e50 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.382+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce48154c10 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.382+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fce400074e0 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.383+0000 7fce2effd700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 0x7fce34061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.383+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce40079f40 con 0x7fce481514b0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.383+0000 7fce46ffd700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 0x7fce34061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.383+0000 7fce46ffd700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 0x7fce34061640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fce38009f90 tx=0x7fce38009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.387+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce4004b210 con 0x7fce481514b0 2023-11-28T02:52:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.552+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fce4806c7c0 con 0x7fce481514b0 2023-11-28T02:52:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.552+0000 7fce2effd700 1 -- 172.21.15.73:0/3398248150 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7fce40001ec0 con 0x7fce481514b0 2023-11-28T02:52:57.553 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:52:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 msgr2=0x7fce34061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 0x7fce34061640 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fce38009f90 tx=0x7fce38009060).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 msgr2=0x7fce48147380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 rx=0x7fce4000cb90 tx=0x7fce40001d80).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 shutdown_connections 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce3405f180 0x7fce34061640 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce481514b0 0x7fce48147380 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce480bcea0 0x7fce480a9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 --2- 172.21.15.73:0/3398248150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce481510d0 0x7fce48148d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 >> 172.21.15.73:0/3398248150 conn(0x7fce480c4010 msgr2=0x7fce48066260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 shutdown_connections 2023-11-28T02:52:57.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.554+0000 7fce47fff700 1 -- 172.21.15.73:0/3398248150 wait complete. 2023-11-28T02:52:57.567 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:52:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.683+0000 7fa5389d8700 1 Processor -- start 2023-11-28T02:52:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.683+0000 7fa5389d8700 1 -- start start 2023-11-28T02:52:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa5389d8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa534098d10 0x7fa5340c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa5389d8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa5389d8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5340c72d0 con 0x7fa534064350 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa5389d8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa534154bc0 con 0x7fa534098d10 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa5389d8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5340c66c0 con 0x7fa5340c6190 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa532d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa534064350 0x7fa534158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44972/0 (socket says 172.21.15.73:44972) 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa532d9d700 1 -- 172.21.15.73:0/3271768950 learned_addr learned my addr 172.21.15.73:0/3271768950 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa53259c700 1 --2- 172.21.15.73:0/3271768950 >> v2:172.21.15.150:3300/0 conn(0x7fa534098d10 0x7fa5340c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa531d9b700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2314808951 0 0) 0x7fa5340c72d0 con 0x7fa534064350 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.684+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa51c0033f0 con 0x7fa534064350 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3814159176 0 0) 0x7fa51c0033f0 con 0x7fa534064350 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53259c700 1 -- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 msgr2=0x7fa534063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53259c700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53259c700 1 -- 172.21.15.73:0/3271768950 >> v1:172.21.15.73:6789/0 conn(0x7fa534064350 legacy=0x7fa534158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53259c700 1 -- 172.21.15.73:0/3271768950 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5340c69a0 con 0x7fa534098d10 2023-11-28T02:52:57.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa531d9b700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53259c700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa5340c7760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa524009e60 tx=0x7fa524009780).ready entity=mon.1 client_cookie=7a6df11c262b0d91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52400e070 con 0x7fa534098d10 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52400b7a0 con 0x7fa534098d10 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa53159a700 1 -- 172.21.15.73:0/3271768950 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c001470 con 0x7fa534098d10 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 -- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 msgr2=0x7fa5340c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa5340c7760 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 rx=0x7fa524009e60 tx=0x7fa524009780).stop 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 -- 172.21.15.73:0/3271768950 shutdown_connections 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa5340c7760 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 --2- 172.21.15.73:0/3271768950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.685+0000 7fa5389d8700 1 -- 172.21.15.73:0/3271768950 >> 172.21.15.73:0/3271768950 conn(0x7fa5340cb310 msgr2=0x7fa5340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.686+0000 7fa5389d8700 1 -- 172.21.15.73:0/3271768950 shutdown_connections 2023-11-28T02:52:57.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.686+0000 7fa5389d8700 1 -- 172.21.15.73:0/3271768950 wait complete. 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 Processor -- start 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 -- start start 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa534167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534168460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534157c50 con 0x7fa534064350 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa534150560 con 0x7fa534098d10 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa5389d8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa53414ff30 con 0x7fa5340c6190 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa53259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa53259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45022/0 (socket says 172.21.15.73:45022) 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.687+0000 7fa53259c700 1 -- 172.21.15.73:0/2604989893 learned_addr learned my addr 172.21.15.73:0/2604989893 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 -- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 msgr2=0x7fa534168460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534168460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa531d9b700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa534167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 -- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 msgr2=0x7fa534167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa534167f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 -- 172.21.15.73:0/2604989893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5280092a0 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa53259c700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa52400b610 tx=0x7fa524009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa524001030 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52401d420 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa524009a20 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52401d7c0 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.688+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa534168a30 con 0x7fa534064350 2023-11-28T02:52:57.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.689+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa53414fad0 con 0x7fa534064350 2023-11-28T02:52:57.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.689+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa524003040 con 0x7fa534064350 2023-11-28T02:52:57.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.689+0000 7fa5227fc700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 0x7fa514061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.690+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa52407bda0 con 0x7fa534064350 2023-11-28T02:52:57.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.690+0000 7fa531d9b700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 0x7fa514061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.690+0000 7fa531d9b700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 0x7fa514061610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa528006d80 tx=0x7fa528008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.694+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa52404d070 con 0x7fa534064350 2023-11-28T02:52:57.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.857+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa53406c860 con 0x7fa534064350 2023-11-28T02:52:57.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.858+0000 7fa5227fc700 1 -- 172.21.15.73:0/2604989893 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa524050690 con 0x7fa534064350 2023-11-28T02:52:57.859 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:52:57.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 msgr2=0x7fa514061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 0x7fa514061610 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fa528006d80 tx=0x7fa528008040).stop 2023-11-28T02:52:57.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 msgr2=0x7fa534157760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 rx=0x7fa52400b610 tx=0x7fa524009780).stop 2023-11-28T02:52:57.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 shutdown_connections 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa51405f150 0x7fa514061610 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5340c6190 0x7fa534168460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa534064350 0x7fa534157760 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 --2- 172.21.15.73:0/2604989893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa534098d10 0x7fa534167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.859+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 >> 172.21.15.73:0/2604989893 conn(0x7fa5340cb310 msgr2=0x7fa5340ca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.860+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 shutdown_connections 2023-11-28T02:52:57.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.860+0000 7fa5389d8700 1 -- 172.21.15.73:0/2604989893 wait complete. 2023-11-28T02:52:57.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:52:57.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.988+0000 7ff0b0eed700 1 Processor -- start 2023-11-28T02:52:57.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.988+0000 7ff0b0eed700 1 -- start start 2023-11-28T02:52:57.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0b0eed700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff0ac14e530 0x7ff0ac0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0b0eed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff0ac14e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0b0eed700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0ac0c7bc0 con 0x7ff0ac158570 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0b0eed700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0ac155920 con 0x7ff0ac14e530 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0b0eed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac154dd0 con 0x7ff0ac154950 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0abfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff0ac158570 0x7ff0ac0aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44976/0 (socket says 172.21.15.73:44976) 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0abfff700 1 -- 172.21.15.73:0/560416539 learned_addr learned my addr 172.21.15.73:0/560416539 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0ab7fe700 1 --2- 172.21.15.73:0/560416539 >> v2:172.21.15.150:3300/0 conn(0x7ff0ac14e530 0x7ff0ac0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff0ac14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2768998446 0 0) 0x7ff0ac0c7bc0 con 0x7ff0ac158570 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0940033f0 con 0x7ff0ac158570 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3591772615 0 0) 0x7ff0940033f0 con 0x7ff0ac158570 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 msgr2=0x7ff0ac14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff0ac14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 >> v2:172.21.15.150:3300/0 conn(0x7ff0ac14e530 msgr2=0x7ff0ac0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 --2- 172.21.15.73:0/560416539 >> v2:172.21.15.150:3300/0 conn(0x7ff0ac14e530 0x7ff0ac0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac155080 con 0x7ff0ac158570 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff0a40017f0 con 0x7ff0ac158570 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 >> v1:172.21.15.73:6789/0 conn(0x7ff0ac158570 legacy=0x7ff0ac0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094003960 0x7ff094003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff094004690 0x7ff094004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff094005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac155920 con 0x7ff094004690 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac154dd0 con 0x7ff094003960 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.989+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff09400b990 con 0x7ff0ac154950 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff094004690 0x7ff094004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0ab7fe700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094003960 0x7ff094003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0abfff700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff094005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 -- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 msgr2=0x7ff094005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac154950 0x7ff094005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 -- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094003960 msgr2=0x7ff094003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094003960 0x7ff094003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:57.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 -- 172.21.15.73:0/560416539 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0940033f0 con 0x7ff094004690 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff094004690 0x7ff094004b10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff09c009a80 tx=0x7ff09c009060).ready entity=mon.0 client_cookie=4b644caf9e3d4d29 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c009430 con 0x7ff094004690 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff09c00e640 con 0x7ff094004690 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0aa7fc700 1 -- 172.21.15.73:0/560416539 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff09c00e9c0 con 0x7ff094004690 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0b0eed700 1 -- 172.21.15.73:0/560416539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff094004690 msgr2=0x7ff094004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/560416539 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff094004690 0x7ff094004b10 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7ff09c009a80 tx=0x7ff09c009060).stop 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.990+0000 7ff0abfff700 1 -- 172.21.15.73:0/560416539 reap_dead start 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.991+0000 7ff0b0eed700 1 -- 172.21.15.73:0/560416539 shutdown_connections 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.991+0000 7ff0b0eed700 1 -- 172.21.15.73:0/560416539 >> 172.21.15.73:0/560416539 conn(0x7ff0ac064990 msgr2=0x7ff0ac064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.991+0000 7ff0b0eed700 1 -- 172.21.15.73:0/560416539 shutdown_connections 2023-11-28T02:52:57.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.991+0000 7ff0b0eed700 1 -- 172.21.15.73:0/560416539 wait complete. 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.992+0000 7ff0b0eed700 1 Processor -- start 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.992+0000 7ff0b0eed700 1 -- start start 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.992+0000 7ff0b0eed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094004690 0x7ff0ac164fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac0c4ac0 0x7ff0ac0c4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac165730 con 0x7ff0ac0af340 2023-11-28T02:52:57.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac1654d0 con 0x7ff094004690 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0ac0c5430 con 0x7ff0ac0c4ac0 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45034/0 (socket says 172.21.15.73:45034) 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 -- 172.21.15.73:0/4157626899 learned_addr learned my addr 172.21.15.73:0/4157626899 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0ab7fe700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094004690 0x7ff0ac164fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0aaffd700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac0c4ac0 0x7ff0ac0c4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 -- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac0c4ac0 msgr2=0x7ff0ac0c4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac0c4ac0 0x7ff0ac0c4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 -- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094004690 msgr2=0x7ff0ac164fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094004690 0x7ff0ac164fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 -- 172.21.15.73:0/4157626899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0ac0b00a0 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0abfff700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff0a4000f80 tx=0x7ff0a4001cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400f8a0 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0a400fa00 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0a400fda0 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0ac0b0380 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.993+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0ac0b0940 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.994+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0ac062ee0 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.994+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff0a4015620 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.995+0000 7ff0937fe700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 0x7ff098061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.995+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff0a40799e0 con 0x7ff0ac0af340 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.995+0000 7ff0ab7fe700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 0x7ff098061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.995+0000 7ff0ab7fe700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 0x7ff098061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff0a0006d80 tx=0x7ff0a0009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:57.999+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff0a404ac50 con 0x7ff0ac0af340 2023-11-28T02:52:58.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.166+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff0ac06c7c0 con 0x7ff0ac0af340 2023-11-28T02:52:58.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.166+0000 7ff0937fe700 1 -- 172.21.15.73:0/4157626899 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff0a401d030 con 0x7ff0ac0af340 2023-11-28T02:52:58.169 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:52:58.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 msgr2=0x7ff098061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 0x7ff098061610 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 rx=0x7ff0a0006d80 tx=0x7ff0a0009060).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 msgr2=0x7ff0ac0af7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7ff0a4000f80 tx=0x7ff0a4001cc0).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 shutdown_connections 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff09805f150 0x7ff098061610 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff0ac0af340 0x7ff0ac0af7c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff094004690 0x7ff0ac164fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 --2- 172.21.15.73:0/4157626899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff0ac0c4ac0 0x7ff0ac0c4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 >> 172.21.15.73:0/4157626899 conn(0x7ff0ac064990 msgr2=0x7ff0ac0c1080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 shutdown_connections 2023-11-28T02:52:58.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.168+0000 7ff0b0eed700 1 -- 172.21.15.73:0/4157626899 wait complete. 2023-11-28T02:52:58.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:52:58.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.294+0000 7f90bb5d2700 1 Processor -- start 2023-11-28T02:52:58.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.295+0000 7f90bb5d2700 1 -- start start 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.295+0000 7f90bb5d2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f90b4069900 0x7f90b4069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.295+0000 7f90bb5d2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b406a270 con 0x7f90b41544f0 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.295+0000 7f90bb5d2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90b40c28c0 con 0x7f90b4150890 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.295+0000 7f90bb5d2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90b41552e0 con 0x7f90b4069900 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b9b6f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f90b41544f0 0x7f90b40c2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44992/0 (socket says 172.21.15.73:44992) 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b9b6f700 1 -- 172.21.15.73:0/2840313691 learned_addr learned my addr 172.21.15.73:0/2840313691 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916329532 0 0) 0x7f90ac001490 con 0x7f90b41544f0 2023-11-28T02:52:58.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b8b6d700 1 --2- 172.21.15.73:0/2840313691 >> v2:172.21.15.150:3300/0 conn(0x7f90b4069900 0x7f90b4069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90a00033f0 con 0x7f90b41544f0 2023-11-28T02:52:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 16532604 0 0) 0x7f90a00033f0 con 0x7f90b41544f0 2023-11-28T02:52:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 >> v2:172.21.15.150:3300/0 conn(0x7f90b4069900 msgr2=0x7f90b4069d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 --2- 172.21.15.73:0/2840313691 >> v2:172.21.15.150:3300/0 conn(0x7f90b4069900 0x7f90b4069d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 >> v1:172.21.15.150:6789/0 conn(0x7f90b4150890 legacy=0x7f90b40bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b40bc540 con 0x7f90b41544f0 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f90ac001980 con 0x7f90b41544f0 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 >> v1:172.21.15.73:6789/0 conn(0x7f90b41544f0 legacy=0x7f90b40c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a00038e0 0x7f90a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a0004690 0x7f90a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b41552e0 con 0x7f90a0005000 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a000beb0 con 0x7f90a00038e0 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a000c070 con 0x7f90a0004690 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b936e700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a00038e0 0x7f90a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b8b6d700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a0004690 0x7f90a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b9b6f700 1 -- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a0004690 msgr2=0x7f90a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.296+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a0004690 0x7f90a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90b9b6f700 1 -- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a00038e0 msgr2=0x7f90a0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a00038e0 0x7f90a0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90b9b6f700 1 -- 172.21.15.73:0/2840313691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a00033f0 con 0x7f90a0005000 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90b9b6f700 1 -- 172.21.15.73:0/2840313691 reap_dead start 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90a000b8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f90ac00c660 tx=0x7f90ac007c50).ready entity=mon.0 client_cookie=43d5f8c09a25b553 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac005b20 con 0x7f90a0005000 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90ac005c80 con 0x7f90a0005000 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90abfff700 1 -- 172.21.15.73:0/2840313691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90ac007110 con 0x7f90a0005000 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 -- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 msgr2=0x7f90a000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90a000b8e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f90ac00c660 tx=0x7f90ac007c50).stop 2023-11-28T02:52:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 -- 172.21.15.73:0/2840313691 shutdown_connections 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/2840313691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90a000b8e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 -- 172.21.15.73:0/2840313691 >> 172.21.15.73:0/2840313691 conn(0x7f90b40631d0 msgr2=0x7f90b40c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 -- 172.21.15.73:0/2840313691 shutdown_connections 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.297+0000 7f90bb5d2700 1 -- 172.21.15.73:0/2840313691 wait complete. 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.298+0000 7f90bb5d2700 1 Processor -- start 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.298+0000 7f90bb5d2700 1 -- start start 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4153570 0x7f90b40bf0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 0x7f90b40bfa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c7b90 con 0x7f90a0005000 2023-11-28T02:52:58.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40bff50 con 0x7f90b4153570 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90bb5d2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90b40c7310 con 0x7f90b40bf5e0 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90b936e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.299+0000 7f90b936e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45062/0 (socket says 172.21.15.73:45062) 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 -- 172.21.15.73:0/620492243 learned_addr learned my addr 172.21.15.73:0/620492243 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b8b6d700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4153570 0x7f90b40bf0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 0x7f90b40bfa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 -- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 msgr2=0x7f90b40bfa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 0x7f90b40bfa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 -- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4153570 msgr2=0x7f90b40bf0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4153570 0x7f90b40bf0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 -- 172.21.15.73:0/620492243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90b4091f20 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b9b6f700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 0x7f90b40bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90b936e700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f90b00028c0 tx=0x7f90b0007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0011590 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90b00116f0 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90b4092200 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90b0011a70 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.300+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90b40927c0 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.301+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90b40929e0 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.301+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f90b001c3b0 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.302+0000 7f90a97fa700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 0x7f9098061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.302+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f90b0079be0 con 0x7f90a0005000 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.302+0000 7f90b8b6d700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 0x7f9098061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.302+0000 7f90b8b6d700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 0x7f9098061610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f90b40c0d20 tx=0x7f90a4009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.306+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f90b0049af0 con 0x7f90a0005000 2023-11-28T02:52:58.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.509+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f90b406c860 con 0x7f90a0005000 2023-11-28T02:52:58.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.509+0000 7f90a97fa700 1 -- 172.21.15.73:0/620492243 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f90b0018030 con 0x7f90a0005000 2023-11-28T02:52:58.511 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:58.511 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:52:58.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.511+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 msgr2=0x7f9098061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.511+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 0x7f9098061610 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7f90b40c0d20 tx=0x7f90a4009060).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.511+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 msgr2=0x7f90b4143dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.511+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7f90b00028c0 tx=0x7f90b0007bb0).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 shutdown_connections 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f909805f150 0x7f9098061610 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90b40bf5e0 0x7f90b40bfa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a0005000 0x7f90b4143dc0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 --2- 172.21.15.73:0/620492243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90b4153570 0x7f90b40bf0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 >> 172.21.15.73:0/620492243 conn(0x7f90b40631d0 msgr2=0x7f90b40c2c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 shutdown_connections 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.512+0000 7f90bb5d2700 1 -- 172.21.15.73:0/620492243 wait complete. 2023-11-28T02:52:58.514 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:52:58.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:52:58.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.637+0000 7f23847bd700 1 Processor -- start 2023-11-28T02:52:58.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.637+0000 7f23847bd700 1 -- start start 2023-11-28T02:52:58.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f23847bd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f237c1544e0 0x7f237c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f23847bd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c6e30 0x7f237c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f23847bd700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f237c0c69a0 con 0x7f237c06a610 2023-11-28T02:52:58.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f23847bd700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f237c0c13a0 con 0x7f237c1544e0 2023-11-28T02:52:58.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f23847bd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0c1680 con 0x7f237c0c6e30 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f2381d58700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f237c06a610 0x7f237c0c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45000/0 (socket says 172.21.15.73:45000) 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f2381d58700 1 -- 172.21.15.73:0/2479599206 learned_addr learned my addr 172.21.15.73:0/2479599206 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f2382d5a700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c6e30 0x7f237c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f2382559700 1 --2- 172.21.15.73:0/2479599206 >> v2:172.21.15.150:3300/0 conn(0x7f237c1544e0 0x7f237c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.638+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4034884506 0 0) 0x7f237c0c69a0 con 0x7f237c06a610 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f23640033f0 con 0x7f237c06a610 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 621487824 0 0) 0x7f23640033f0 con 0x7f237c06a610 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c6e30 msgr2=0x7f237c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c6e30 0x7f237c0c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 >> v2:172.21.15.150:3300/0 conn(0x7f237c1544e0 msgr2=0x7f237c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 --2- 172.21.15.73:0/2479599206 >> v2:172.21.15.150:3300/0 conn(0x7f237c1544e0 0x7f237c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0c6630 con 0x7f237c06a610 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2370001470 con 0x7f237c06a610 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 >> v1:172.21.15.73:6789/0 conn(0x7f237c06a610 legacy=0x7f237c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23640038e0 0x7f2364003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2364004690 0x7f2364004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364005000 0x7f236400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0c13a0 con 0x7f2364004690 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c0c1680 con 0x7f23640038e0 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236400bdd0 con 0x7f2364005000 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2382559700 1 --2- 172.21.15.73:0/2479599206 >> v2:172.21.15.150:3300/0 conn(0x7f237c1544e0 0x7f237c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:58.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2382d5a700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f237c0c6e30 0x7f237c0c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:52:58.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2364004690 0x7f2364004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2382559700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23640038e0 0x7f2364003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2382d5a700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364005000 0x7f236400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 -- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364005000 msgr2=0x7f236400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364005000 0x7f236400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 -- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23640038e0 msgr2=0x7f2364003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23640038e0 0x7f2364003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 -- 172.21.15.73:0/2479599206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23640033f0 con 0x7f2364004690 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2382559700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f23640038e0 0x7f2364003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381d58700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2364004690 0x7f2364004b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f237000c210 tx=0x7f2370001e90).ready entity=mon.0 client_cookie=44abddc5cc2b78a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2370007a60 con 0x7f2364004690 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f237000caa0 con 0x7f2364004690 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.639+0000 7f2381557700 1 -- 172.21.15.73:0/2479599206 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f237000ce40 con 0x7f2364004690 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 -- 172.21.15.73:0/2479599206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2364004690 msgr2=0x7f2364004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 --2- 172.21.15.73:0/2479599206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2364004690 0x7f2364004b10 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 rx=0x7f237000c210 tx=0x7f2370001e90).stop 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f2382d5a700 1 -- 172.21.15.73:0/2479599206 reap_dead start 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 -- 172.21.15.73:0/2479599206 shutdown_connections 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 -- 172.21.15.73:0/2479599206 >> 172.21.15.73:0/2479599206 conn(0x7f237c085020 msgr2=0x7f237c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 -- 172.21.15.73:0/2479599206 shutdown_connections 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.640+0000 7f23847bd700 1 -- 172.21.15.73:0/2479599206 wait complete. 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.641+0000 7f23847bd700 1 Processor -- start 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.641+0000 7f23847bd700 1 -- start start 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364004690 0x7f237c1698e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f237c097420 0x7f237c152ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c098940 con 0x7f237c0978a0 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f237c097d30 con 0x7f237c097420 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f237c098010 con 0x7f2364004690 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45088/0 (socket says 172.21.15.73:45088) 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 -- 172.21.15.73:0/3548759730 learned_addr learned my addr 172.21.15.73:0/3548759730 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:58.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 -- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364004690 msgr2=0x7f237c1698e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364004690 0x7f237c1698e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 -- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f237c097420 msgr2=0x7f237c152ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f237c097420 0x7f237c152ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 -- 172.21.15.73:0/3548759730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f237c0982f0 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f2382d5a700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f237c062ec0 tx=0x7f2374008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374013550 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23740136b0 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f237c0985d0 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2374013a30 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.642+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f237c095640 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.643+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f237c153c90 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.644+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2374015440 con 0x7f237c0978a0 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.644+0000 7f237a7fc700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 0x7f2368061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.644+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2374079da0 con 0x7f237c0978a0 2023-11-28T02:52:58.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.644+0000 7f2382559700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 0x7f2368061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.644+0000 7f2382559700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 0x7f2368061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f236c0094d0 tx=0x7f236c00b040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.648+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f237404b0c0 con 0x7f237c0978a0 2023-11-28T02:52:58.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.811+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f237c06c860 con 0x7f237c0978a0 2023-11-28T02:52:58.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.812+0000 7f237a7fc700 1 -- 172.21.15.73:0/3548759730 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f2374013e60 con 0x7f237c0978a0 2023-11-28T02:52:58.814 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:58.814 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 msgr2=0x7f2368061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 0x7f2368061610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 rx=0x7f236c0094d0 tx=0x7f236c00b040).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 msgr2=0x7f237c1533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 rx=0x7f237c062ec0 tx=0x7f2374008040).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 shutdown_connections 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f236805f150 0x7f2368061610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f237c0978a0 0x7f237c1533d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2364004690 0x7f237c1698e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 --2- 172.21.15.73:0/3548759730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f237c097420 0x7f237c152ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:58.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 >> 172.21.15.73:0/3548759730 conn(0x7f237c085020 msgr2=0x7f237c0c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 shutdown_connections 2023-11-28T02:52:58.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.814+0000 7f23847bd700 1 -- 172.21.15.73:0/3548759730 wait complete. 2023-11-28T02:52:58.828 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:52:58.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 Processor -- start 2023-11-28T02:52:58.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 -- start start 2023-11-28T02:52:58.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff4380aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4380cae90 0x7ff4380c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4380a9a90 con 0x7ff4380aa960 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff438154ce0 con 0x7ff4380cae90 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.939+0000 7ff43d9c4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4380a8e80 con 0x7ff4380caab0 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4380cae90 0x7ff4380c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4377fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff4380aa960 0x7ff4380c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45012/0 (socket says 172.21.15.73:45012) 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4377fe700 1 -- 172.21.15.73:0/1799439669 learned_addr learned my addr 172.21.15.73:0/1799439669 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff436ffd700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff4380aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 -- 172.21.15.73:0/1799439669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 msgr2=0x7ff4380aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff4380aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 -- 172.21.15.73:0/1799439669 >> v1:172.21.15.73:6789/0 conn(0x7ff4380aa960 legacy=0x7ff4380c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff435ffb700 1 -- 172.21.15.73:0/1799439669 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 582265950 0 0) 0x7ff4380a9a90 con 0x7ff4380aa960 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 -- 172.21.15.73:0/1799439669 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4380a9160 con 0x7ff4380cae90 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff436ffd700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff4380aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff4367fc700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380c0ab0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7ff420009760 tx=0x7ff420009060).ready entity=mon.0 client_cookie=2d4bf32e7565c8b4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff435ffb700 1 -- 172.21.15.73:0/1799439669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000b580 con 0x7ff4380cae90 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff435ffb700 1 -- 172.21.15.73:0/1799439669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff42000b6e0 con 0x7ff4380cae90 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.940+0000 7ff435ffb700 1 -- 172.21.15.73:0/1799439669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000ba60 con 0x7ff4380cae90 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 -- 172.21.15.73:0/1799439669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 msgr2=0x7ff4380c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380c0ab0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7ff420009760 tx=0x7ff420009060).stop 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 -- 172.21.15.73:0/1799439669 shutdown_connections 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff4380aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/1799439669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380c0ab0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 -- 172.21.15.73:0/1799439669 >> 172.21.15.73:0/1799439669 conn(0x7ff438094920 msgr2=0x7ff43806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:58.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 -- 172.21.15.73:0/1799439669 shutdown_connections 2023-11-28T02:52:58.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.941+0000 7ff43d9c4700 1 -- 172.21.15.73:0/1799439669 wait complete. 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.942+0000 7ff43d9c4700 1 Processor -- start 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.942+0000 7ff43d9c4700 1 -- start start 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff438157b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4380b05c0 0x7ff4380b0a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438154e80 con 0x7ff4380cae90 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff438155160 con 0x7ff4380b05c0 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff438159420 con 0x7ff4380caab0 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45104/0 (socket says 172.21.15.73:45104) 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 -- 172.21.15.73:0/615886369 learned_addr learned my addr 172.21.15.73:0/615886369 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4377fe700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4380b05c0 0x7ff4380b0a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff436ffd700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff438157b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 -- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 msgr2=0x7ff438157b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff438157b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 -- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4380b05c0 msgr2=0x7ff4380b0a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4380b05c0 0x7ff4380b0a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 -- 172.21.15.73:0/615886369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4200092a0 con 0x7ff4380cae90 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff4367fc700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff438083c30 tx=0x7ff420017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42000ba60 con 0x7ff4380cae90 2023-11-28T02:52:58.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff420000c90 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4380b0f10 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff42001b640 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.943+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4380af090 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.944+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff43806c860 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.944+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff42001ba70 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.945+0000 7ff42effd700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 0x7ff418061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.945+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff42007bce0 con 0x7ff4380cae90 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.945+0000 7ff436ffd700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 0x7ff418061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:58.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.945+0000 7ff436ffd700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 0x7ff418061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff428007fd0 tx=0x7ff428007760).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:58.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:58.949+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff42004cfb0 con 0x7ff4380cae90 2023-11-28T02:52:59.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.150+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff4381552a0 con 0x7ff4380cae90 2023-11-28T02:52:59.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.151+0000 7ff42effd700 1 -- 172.21.15.73:0/615886369 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7ff4200505d0 con 0x7ff4380cae90 2023-11-28T02:52:59.152 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:52:59.152 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:52:59.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.152+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 msgr2=0x7ff418061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.152+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 0x7ff418061610 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 rx=0x7ff428007fd0 tx=0x7ff428007760).stop 2023-11-28T02:52:59.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 msgr2=0x7ff4380afd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7ff438083c30 tx=0x7ff420017000).stop 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 shutdown_connections 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff41805f150 0x7ff418061610 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4380b05c0 0x7ff4380b0a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4380caab0 0x7ff438157b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 --2- 172.21.15.73:0/615886369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4380cae90 0x7ff4380afd40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 >> 172.21.15.73:0/615886369 conn(0x7ff438094920 msgr2=0x7ff4380a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 shutdown_connections 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.153+0000 7ff43d9c4700 1 -- 172.21.15.73:0/615886369 wait complete. 2023-11-28T02:52:59.155 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:52:59.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T02:52:44.444957+0000', 'modified': '2023-11-28T02:52:46.458206+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4281}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4281': {'gid': 4281, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/2149449616', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6833', 'nonce': 2149449616}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 2149449616}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T02:52:59.569 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T02:52:59.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-11-28T02:52:59.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.682+0000 7efe61dcc700 1 Processor -- start 2023-11-28T02:52:59.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.682+0000 7efe61dcc700 1 -- start start 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe61dcc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 0x7efe5c069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe61dcc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe5c069a50 0x7efe5c0c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe61dcc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe5c14f1e0 con 0x7efe5c0c3520 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe61dcc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe5c0bcd60 con 0x7efe5c069a50 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe61dcc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c069ed0 con 0x7efe5c14ee00 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe5c069a50 0x7efe5c0c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe615cb700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe5c0c3520 0x7efe5c154cf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45018/0 (socket says 172.21.15.73:45018) 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe615cb700 1 -- 172.21.15.73:0/4203125638 learned_addr learned my addr 172.21.15.73:0/4203125638 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe60dca700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 0x7efe5c069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 -- 172.21.15.73:0/4203125638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 msgr2=0x7efe5c069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 0x7efe5c069560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 -- 172.21.15.73:0/4203125638 >> v1:172.21.15.73:6789/0 conn(0x7efe5c0c3520 legacy=0x7efe5c154cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 -- 172.21.15.73:0/4203125638 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe5c0c39d0 con 0x7efe5c069a50 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5b7fe700 1 -- 172.21.15.73:0/4203125638 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121132706 0 0) 0x7efe5c14f1e0 con 0x7efe5c0c3520 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe60dca700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 0x7efe5c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5bfff700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c069a50 0x7efe5c0c3030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7efe4c009760 tx=0x7efe4c009060).ready entity=mon.0 client_cookie=e427d2ac5454a5e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5b7fe700 1 -- 172.21.15.73:0/4203125638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00b580 con 0x7efe5c069a50 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5b7fe700 1 -- 172.21.15.73:0/4203125638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c00b6e0 con 0x7efe5c069a50 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.683+0000 7efe5b7fe700 1 -- 172.21.15.73:0/4203125638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00ba60 con 0x7efe5c069a50 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 -- 172.21.15.73:0/4203125638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c069a50 msgr2=0x7efe5c0c3030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c069a50 0x7efe5c0c3030 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0x7efe4c009760 tx=0x7efe4c009060).stop 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 -- 172.21.15.73:0/4203125638 shutdown_connections 2023-11-28T02:52:59.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c14ee00 0x7efe5c069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4203125638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c069a50 0x7efe5c0c3030 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 -- 172.21.15.73:0/4203125638 >> 172.21.15.73:0/4203125638 conn(0x7efe5c064320 msgr2=0x7efe5c0c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 -- 172.21.15.73:0/4203125638 shutdown_connections 2023-11-28T02:52:59.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.684+0000 7efe61dcc700 1 -- 172.21.15.73:0/4203125638 wait complete. 2023-11-28T02:52:59.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.685+0000 7efe61dcc700 1 Processor -- start 2023-11-28T02:52:59.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.685+0000 7efe61dcc700 1 -- start start 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5c069a50 0x7efe5c0c0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c0c3520 0x7efe5c14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0aca20 con 0x7efe5c14ee00 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c15c0 con 0x7efe5c069a50 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe61dcc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe5c0c18a0 con 0x7efe5c0c3520 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45118/0 (socket says 172.21.15.73:45118) 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 -- 172.21.15.73:0/4132081870 learned_addr learned my addr 172.21.15.73:0/4132081870 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe60dca700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5c069a50 0x7efe5c0c0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe5bfff700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c0c3520 0x7efe5c14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 -- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c0c3520 msgr2=0x7efe5c14cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c0c3520 0x7efe5c14cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 -- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5c069a50 msgr2=0x7efe5c0c0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5c069a50 0x7efe5c0c0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.686+0000 7efe615cb700 1 -- 172.21.15.73:0/4132081870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4c0092a0 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe615cb700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7efe4400c490 tx=0x7efe44007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe58ff9700 1 -- 172.21.15.73:0/4132081870 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400ce00 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe58ff9700 1 -- 172.21.15.73:0/4132081870 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe440072e0 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe5c0c1b80 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe58ff9700 1 -- 172.21.15.73:0/4132081870 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44007660 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.687+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0bee10 con 0x7efe5c14ee00 2023-11-28T02:52:59.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.688+0000 7efe58ff9700 1 -- 172.21.15.73:0/4132081870 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efe440059b0 con 0x7efe5c14ee00 2023-11-28T02:52:59.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.688+0000 7efe58ff9700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 0x7efe3c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.688+0000 7efe58ff9700 1 -- 172.21.15.73:0/4132081870 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efe440799c0 con 0x7efe5c14ee00 2023-11-28T02:52:59.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.688+0000 7efe60dca700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 0x7efe3c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.689+0000 7efe60dca700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 0x7efe3c061610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7efe5400d6e0 tx=0x7efe54007850).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 Processor -- start 2023-11-28T02:52:59.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 -- start start 2023-11-28T02:52:59.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e464e0 0x559cd8dfdd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8df87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8dc3060 con 0x559cd8bb9e40 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8d3a1f0 con 0x559cd8e25cf0 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cd8d4f440 con 0x559cd8e464e0 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe60dca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe60dca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45132/0 (socket says 172.21.15.73:45132) 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.691+0000 7efe60dca700 1 -- 172.21.15.73:0/3299245440 learned_addr learned my addr 172.21.15.73:0/3299245440 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 -- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e464e0 msgr2=0x559cd8dfdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e464e0 0x559cd8dfdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 -- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 msgr2=0x559cd8df87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8df87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 -- 172.21.15.73:0/3299245440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cd8d48620 con 0x559cd8bb9e40 2023-11-28T02:52:59.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe60dca700 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efe5400c8d0 tx=0x7efe54014000).ready entity=mon.0 client_cookie=efe84299b33b9517 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe51ffb700 1 -- 172.21.15.73:0/3299245440 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54012070 con 0x559cd8bb9e40 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe51ffb700 1 -- 172.21.15.73:0/3299245440 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe54007b30 con 0x559cd8bb9e40 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe51ffb700 1 -- 172.21.15.73:0/3299245440 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe54019cd0 con 0x559cd8bb9e40 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 -- 172.21.15.73:0/3299245440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 msgr2=0x559cd8dfd880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7efe5400c8d0 tx=0x7efe54014000).stop 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 -- 172.21.15.73:0/3299245440 shutdown_connections 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8df87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8bb9e40 0x559cd8dfd880 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 --2- 172.21.15.73:0/3299245440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e464e0 0x559cd8dfdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 -- 172.21.15.73:0/3299245440 >> 172.21.15.73:0/3299245440 conn(0x559cd8c674a0 msgr2=0x559cd8e46970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 -- 172.21.15.73:0/3299245440 shutdown_connections 2023-11-28T02:52:59.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.692+0000 7efe749c6b80 1 -- 172.21.15.73:0/3299245440 wait complete. 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.693+0000 7efe749c6b80 1 -- start start 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.693+0000 7efe749c6b80 1 Processor -- start 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e22ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8e26400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8d4f580 con 0x559cd8e464e0 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8d4f800 con 0x559cd8e25cf0 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cd8d1ae10 con 0x559cd8bb9e40 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe749c6b80 10 client.0 ll_register_callbacks cb 0x559cd8dbd640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45146/0 (socket says 172.21.15.73:45146) 2023-11-28T02:52:59.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 -- 172.21.15.73:0/1437252568 learned_addr learned my addr 172.21.15.73:0/1437252568 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe5bfff700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8e26400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe60dca700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e22ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 -- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 msgr2=0x559cd8e22ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e22ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 -- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 msgr2=0x559cd8e26400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8e26400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe615cb700 1 -- 172.21.15.73:0/1437252568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5400c550 con 0x559cd8e464e0 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe5bfff700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8e26400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.694+0000 7efe60dca700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e22ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe615cb700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7efe4404ad90 tx=0x7efe4400f000).ready entity=mon.0 client_cookie=ef6d50b825eb44ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T02:52:59.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44005e70 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efe4404e830 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559cd8df1720 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe4404df90 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 10 client.4668 fetch_fsmap learned FSMap version 6 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x559cd8d439c0 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559cd8df1720 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efe4404a680 con 0x559cd8e464e0 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 10 client.4668 fetch_fsmap finished waiting for FSMap version 6 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 10 client.4668 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 20 client.4668 populate_metadata read hostname 'smithi073' 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e2bd90 0x559cd8e415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 4 client.4668 mds_command: new command op to 4281 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T02:52:59.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 --> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559cd8bb5ca0 con 0x559cd8e2bd90 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.695+0000 7efe60dca700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e2bd90 0x559cd8e415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe3affd700 10 client.4668 ms_handle_connect on v2:172.21.15.150:6833/2149449616 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe60dca700 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e2bd90 0x559cd8e415c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=57d10ffbdd9df385 server_cookie=f0283667ff7aba8a in_seq=0 out_seq=0 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe3affd700 1 -- 172.21.15.73:0/1437252568 <== mds.0 v2:172.21.15.150:6833/2149449616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x559cd8bb5ca0 con 0x559cd8e2bd90 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe3affd700 10 client.4668 handle_command_reply: tid=0 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe749c6b80 1 client.4668 shutdown 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 msgr2=0x559cd8df8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.696+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7efe4404ad90 tx=0x7efe4400f000).stop 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 shutdown_connections 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e2bd90 0x559cd8e415c0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8df8de0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe3affd700 0 client.4668 ms_handle_reset on v2:172.21.15.150:6833/2149449616 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e22ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1437252568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e25cf0 0x559cd8e26400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 >> 172.21.15.73:0/1437252568 conn(0x559cd8c674a0 msgr2=0x559cd8dfeb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 shutdown_connections 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 1 -- 172.21.15.73:0/1437252568 wait complete. 2023-11-28T02:52:59.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.697+0000 7efe749c6b80 20 client.4668 trim_cache size 0 max 16384 2023-11-28T02:52:59.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.713+0000 7efe749c6b80 1 Processor -- start 2023-11-28T02:52:59.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.713+0000 7efe749c6b80 1 -- start start 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e41d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e464e0 0x559cd8e26610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8e42390 con 0x559cd8e25cf0 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8dfa530 con 0x559cd8e464e0 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cd8e42d10 con 0x559cd8bb9e40 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45154/0 (socket says 172.21.15.73:45154) 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 -- 172.21.15.73:0/350524495 learned_addr learned my addr 172.21.15.73:0/350524495 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe615cb700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e464e0 0x559cd8e26610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe60dca700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e41d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 -- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 msgr2=0x559cd8e41d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e41d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 -- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e464e0 msgr2=0x559cd8e26610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e464e0 0x559cd8e26610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 -- 172.21.15.73:0/350524495 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559cd8d43710 con 0x559cd8e25cf0 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe60dca700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe5bfff700 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efe5c064760 tx=0x7efe4c017000).ready entity=mon.0 client_cookie=48ff02e0983107de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe527fc700 1 -- 172.21.15.73:0/350524495 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c006cd0 con 0x559cd8e25cf0 2023-11-28T02:52:59.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe527fc700 1 -- 172.21.15.73:0/350524495 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4c000bc0 con 0x559cd8e25cf0 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.714+0000 7efe527fc700 1 -- 172.21.15.73:0/350524495 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4c00b850 con 0x559cd8e25cf0 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- 172.21.15.73:0/350524495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 msgr2=0x559cd8df8d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7efe5c064760 tx=0x7efe4c017000).stop 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- 172.21.15.73:0/350524495 shutdown_connections 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8e464e0 0x559cd8e26610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8bb9e40 0x559cd8e41d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 --2- 172.21.15.73:0/350524495 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e25cf0 0x559cd8df8d50 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- 172.21.15.73:0/350524495 >> 172.21.15.73:0/350524495 conn(0x559cd8c674a0 msgr2=0x559cd8dfd880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- 172.21.15.73:0/350524495 shutdown_connections 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- 172.21.15.73:0/350524495 wait complete. 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 -- start start 2023-11-28T02:52:59.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.715+0000 7efe749c6b80 1 Processor -- start 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8bb9e40 0x559cd8ea6f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e25cf0 0x559cd8e468e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8e472c0 con 0x559cd8e464e0 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x559cd8e27da0 con 0x559cd8bb9e40 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x559cd8e45520 con 0x559cd8e25cf0 2023-11-28T02:52:59.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe749c6b80 10 client.0 ll_register_callbacks cb 0x559cd8dbd640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45170/0 (socket says 172.21.15.73:45170) 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 -- 172.21.15.73:0/1390810237 learned_addr learned my addr 172.21.15.73:0/1390810237 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe60dca700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8bb9e40 0x559cd8ea6f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe5bfff700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e25cf0 0x559cd8e468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 -- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e25cf0 msgr2=0x559cd8e468e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e25cf0 0x559cd8e468e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 -- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8bb9e40 msgr2=0x559cd8ea6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.716+0000 7efe615cb700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8bb9e40 0x559cd8ea6f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe615cb700 1 -- 172.21.15.73:0/1390810237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe5c0c39d0 con 0x559cd8e464e0 2023-11-28T02:52:59.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe615cb700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x559cd8e2ca60 tx=0x7efe44061000).ready entity=mon.0 client_cookie=37062f1091dfc802 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4406d070 con 0x559cd8e464e0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efe4404e830 con 0x559cd8e464e0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7efe54007b30 con 0x559cd8e464e0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7efe44000f90 con 0x559cd8e464e0 2023-11-28T02:52:59.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 10 client.4674 fetch_fsmap learned FSMap version 6 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x559cd8bbaf80 con 0x559cd8e464e0 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe54007b30 con 0x559cd8e464e0 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 6) v1 ==== 3100+0+0 (secure 0 0 0) 0x7efe4404a680 con 0x559cd8e464e0 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 10 client.4674 fetch_fsmap finished waiting for FSMap version 6 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 10 client.4674 resolve_mds: resolved name 'd' to daemon mds.d 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 20 client.4674 populate_metadata read hostname 'smithi073' 2023-11-28T02:52:59.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e234b0 0x559cd8e23890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 4 client.4674 mds_command: new command op to 4281 tid=0[{"prefix": "status"}] 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 --> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559cd8becca0 con 0x559cd8e234b0 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.717+0000 7efe60dca700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e234b0 0x559cd8e23890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe3affd700 10 client.4674 ms_handle_connect on v2:172.21.15.150:6833/2149449616 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe60dca700 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e234b0 0x559cd8e23890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1e2139d4b6331721 server_cookie=c464fb6971db02f0 in_seq=0 out_seq=0 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe3affd700 1 -- 172.21.15.73:0/1390810237 <== mds.0 v2:172.21.15.150:6833/2149449616 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x559cd8becca0 con 0x559cd8e234b0 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe3affd700 10 client.4674 handle_command_reply: tid=0 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 client.4674 shutdown 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 msgr2=0x559cd8e46dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x559cd8e2ca60 tx=0x7efe44061000).stop 2023-11-28T02:52:59.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 shutdown_connections 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:6833/2149449616,v1:172.21.15.150:6835/2149449616] conn(0x559cd8e234b0 0x559cd8e23890 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x559cd8e464e0 0x559cd8e46dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe3affd700 0 client.4674 ms_handle_reset on v2:172.21.15.150:6833/2149449616 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x559cd8bb9e40 0x559cd8ea6f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 --2- 172.21.15.73:0/1390810237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x559cd8e25cf0 0x559cd8e468e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 >> 172.21.15.73:0/1390810237 conn(0x559cd8c674a0 msgr2=0x559cd8e412b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 shutdown_connections 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 1 -- 172.21.15.73:0/1390810237 wait complete. 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.718+0000 7efe749c6b80 20 client.4674 trim_cache size 0 max 16384 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4281, 2023-11-28T02:52:59.731 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 15.272774012999999, 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 6, 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 22, 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 22, 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 20.171341088999998 2023-11-28T02:52:59.732 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T02:52:59.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 msgr2=0x7efe3c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 0x7efe3c061610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 rx=0x7efe5400d6e0 tx=0x7efe54007850).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 msgr2=0x7efe5c1501b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7efe4400c490 tx=0x7efe44007c10).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 shutdown_connections 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe3c05f150 0x7efe3c061610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe5c14ee00 0x7efe5c1501b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe5c069a50 0x7efe5c0c0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 --2- 172.21.15.73:0/4132081870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe5c0c3520 0x7efe5c14cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 >> 172.21.15.73:0/4132081870 conn(0x7efe5c064320 msgr2=0x7efe5c14e170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 shutdown_connections 2023-11-28T02:52:59.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:52:59.719+0000 7efe61dcc700 1 -- 172.21.15.73:0/4132081870 wait complete. 2023-11-28T02:52:59.735 INFO:teuthology.run_tasks:Running task kclient... 2023-11-28T02:52:59.761 INFO:tasks.kclient:Mounting kernel clients... 2023-11-28T02:52:59.762 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-11-28T02:52:59.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.762 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T02:52:59.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2023-11-28T02:52:59.844 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2023-11-28T02:52:59.845 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:52:59.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.846 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T02:52:59.900 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2023-11-28T02:52:59.965 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2023-11-28T02:52:59.966 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:52:59.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.967 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T02:52:59.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:52:59.986 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-28T02:53:00.058 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2023-11-28T02:53:00.059 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:53:00.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.060 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T02:53:00.116 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.117 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-28T02:53:00.186 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T02:53:00.186 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2023-11-28T02:53:00.186 INFO:tasks.kclient:client.0 config is {} 2023-11-28T02:53:00.187 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T02:53:00.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.286+0000 7fccec493700 1 Processor -- start 2023-11-28T02:53:00.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.286+0000 7fccec493700 1 -- start start 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccec493700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcce41513c0 0x7fcce40a9fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccec493700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcce40bdfb0 0x7fcce40be420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccec493700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcce40be910 con 0x7fcce4150fe0 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccec493700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40aab70 con 0x7fcce40bdfb0 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccec493700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcce40aa5b0 con 0x7fcce41513c0 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccea22f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcce4150fe0 0x7fcce406a620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45030/0 (socket says 172.21.15.73:45030) 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fccea22f700 1 -- 172.21.15.73:0/1283874775 learned_addr learned my addr 172.21.15.73:0/1283874775 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcceaa30700 1 --2- 172.21.15.73:0/1283874775 >> v2:172.21.15.73:3300/0 conn(0x7fcce40bdfb0 0x7fcce40be420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcce9a2e700 1 --2- 172.21.15.73:0/1283874775 >> v2:172.21.15.150:3300/0 conn(0x7fcce41513c0 0x7fcce40a9fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcce922d700 1 -- 172.21.15.73:0/1283874775 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79550187 0 0) 0x7fcce40be910 con 0x7fcce4150fe0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcce922d700 1 -- 172.21.15.73:0/1283874775 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcccc0033f0 con 0x7fcce4150fe0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcceaa30700 1 -- 172.21.15.73:0/1283874775 >> v2:172.21.15.150:3300/0 conn(0x7fcce41513c0 msgr2=0x7fcce40a9fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcceaa30700 1 --2- 172.21.15.73:0/1283874775 >> v2:172.21.15.150:3300/0 conn(0x7fcce41513c0 0x7fcce40a9fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcceaa30700 1 -- 172.21.15.73:0/1283874775 >> v1:172.21.15.73:6789/0 conn(0x7fcce4150fe0 legacy=0x7fcce406a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.287+0000 7fcceaa30700 1 -- 172.21.15.73:0/1283874775 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce40aa890 con 0x7fcce40bdfb0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fcceaa30700 1 --2- 172.21.15.73:0/1283874775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce40be420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fcce4069260 tx=0x7fccd4009060).ready entity=mon.0 client_cookie=478464efebed4d55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fcce922d700 1 -- 172.21.15.73:0/1283874775 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400d4a0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fcce922d700 1 -- 172.21.15.73:0/1283874775 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccd400d600 con 0x7fcce40bdfb0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fcce922d700 1 -- 172.21.15.73:0/1283874775 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccd400d980 con 0x7fcce40bdfb0 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 -- 172.21.15.73:0/1283874775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 msgr2=0x7fcce40be420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 --2- 172.21.15.73:0/1283874775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce40be420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7fcce4069260 tx=0x7fccd4009060).stop 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 -- 172.21.15.73:0/1283874775 shutdown_connections 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 --2- 172.21.15.73:0/1283874775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce40be420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 --2- 172.21.15.73:0/1283874775 >> v2:172.21.15.150:3300/0 conn(0x7fcce41513c0 0x7fcce40a9fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 -- 172.21.15.73:0/1283874775 >> 172.21.15.73:0/1283874775 conn(0x7fcce4064360 msgr2=0x7fcce4065900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 -- 172.21.15.73:0/1283874775 shutdown_connections 2023-11-28T02:53:00.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.288+0000 7fccec493700 1 -- 172.21.15.73:0/1283874775 wait complete. 2023-11-28T02:53:00.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.289+0000 7fccec493700 1 Processor -- start 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.289+0000 7fccec493700 1 -- start start 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcce4150fe0 0x7fcce40ad820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcce41513c0 0x7fcce40ac8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40c01c0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40bf5b0 con 0x7fcce4150fe0 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccec493700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcce40bf890 con 0x7fcce41513c0 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccea22f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccea22f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45180/0 (socket says 172.21.15.73:45180) 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fccea22f700 1 -- 172.21.15.73:0/2719456187 learned_addr learned my addr 172.21.15.73:0/2719456187 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fcce9a2e700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcce4150fe0 0x7fcce40ad820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.290+0000 7fcceaa30700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcce41513c0 0x7fcce40ac8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 -- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcce41513c0 msgr2=0x7fcce40ac8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcce41513c0 0x7fcce40ac8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 -- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcce4150fe0 msgr2=0x7fcce40ad820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcce4150fe0 0x7fcce40ad820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 -- 172.21.15.73:0/2719456187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccd40092a0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccea22f700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fccdc00c820 tx=0x7fccdc001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc005d30 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc005e90 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcce40bfb70 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc007340 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.291+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcce413bb10 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.292+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcce40475e0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.292+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fccdc007730 con 0x7fcce40bdfb0 2023-11-28T02:53:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.292+0000 7fcce27fc700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 0x7fccd0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.293+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fccdc07a1d0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.293+0000 7fcce9a2e700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 0x7fccd0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.293+0000 7fcce9a2e700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 0x7fccd0061610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fcce4144900 tx=0x7fccd8009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:00.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.297+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fccdc04b440 con 0x7fcce40bdfb0 2023-11-28T02:53:00.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.510+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fcce40a7cc0 con 0x7fcce40bdfb0 2023-11-28T02:53:00.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.510+0000 7fcce27fc700 1 -- 172.21.15.73:0/2719456187 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fccdc04ea60 con 0x7fcce40bdfb0 2023-11-28T02:53:00.512 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 msgr2=0x7fccd0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 0x7fccd0061610 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 rx=0x7fcce4144900 tx=0x7fccd8009060).stop 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 msgr2=0x7fcce413b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fccdc00c820 tx=0x7fccdc001650).stop 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 shutdown_connections 2023-11-28T02:53:00.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccd005f150 0x7fccd0061610 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcce41513c0 0x7fcce40ac8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcce40bdfb0 0x7fcce413b620 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 --2- 172.21.15.73:0/2719456187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcce4150fe0 0x7fcce40ad820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.512+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 >> 172.21.15.73:0/2719456187 conn(0x7fcce4064360 msgr2=0x7fcce4144d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.513+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 shutdown_connections 2023-11-28T02:53:00.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:00.513+0000 7fccec493700 1 -- 172.21.15.73:0/2719456187 wait complete. 2023-11-28T02:53:00.524 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T02:53:00.525 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T02:53:00.526 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T02:53:00.526 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T02:53:00.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.526 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T02:53:00.542 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:00.542 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:00.542 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:00.542 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:00.543 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T02:53:00.544 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:00.544 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T02:53:00.544 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42499sec preferred_lft 42499sec 2023-11-28T02:53:00.544 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T02:53:00.544 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:00.544 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T02:53:00.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add name ceph-brx type bridge 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr flush dev ceph-brx 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set ceph-brx up 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T02:53:00.545 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:00.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.701 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T02:53:00.742 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T02:53:00.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.744 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T02:53:00.815 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T02:53:00.815 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T02:53:00.815 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T02:53:00.815 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T02:53:00.815 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T02:53:00.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T02:53:00.816 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:01.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.056 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T02:53:01.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.071 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2023-11-28T02:53:01.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.126 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:01.126 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:01.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T02:53:01.126 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-11-28T02:53:01.127 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:01.214 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-11-28T02:53:01.215 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-11-28T02:53:01.215 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:01.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.446 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:01.446 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:01.446 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T02:53:01.446 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T02:53:01.446 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:01.539 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.539 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:01.554 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T02:53:01.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:01.698 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:01.699 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:01.793 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:01.793 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:01.796 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T02:53:01.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:01.796 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:01.822 INFO:tasks.kclient:client.2 config is {} 2023-11-28T02:53:01.822 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T02:53:01.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.961+0000 7f9356df7700 1 Processor -- start 2023-11-28T02:53:01.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.961+0000 7f9356df7700 1 -- start start 2023-11-28T02:53:01.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f9356df7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9350154520 0x7f935006a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f9356df7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93500c67b0 con 0x7f9350154900 2023-11-28T02:53:01.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f9356df7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9350064eb0 con 0x7f93500c7980 2023-11-28T02:53:01.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f9356df7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93500c8650 con 0x7f9350154520 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f934ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9350154900 0x7f93500a7f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45038/0 (socket says 172.21.15.73:45038) 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.962+0000 7f934ffff700 1 -- 172.21.15.73:0/2903567446 learned_addr learned my addr 172.21.15.73:0/2903567446 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1956105182 0 0) 0x7f93500c67b0 con 0x7f9350154900 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9354b93700 1 --2- 172.21.15.73:0/2903567446 >> v2:172.21.15.150:3300/0 conn(0x7f9350154520 0x7f935006a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93380033f0 con 0x7f9350154900 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3803817464 0 0) 0x7f93380033f0 con 0x7f9350154900 2023-11-28T02:53:01.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 >> v2:172.21.15.150:3300/0 conn(0x7f9350154520 msgr2=0x7f935006a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 --2- 172.21.15.73:0/2903567446 >> v2:172.21.15.150:3300/0 conn(0x7f9350154520 0x7f935006a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 >> v1:172.21.15.150:6789/0 conn(0x7f93500c7980 legacy=0x7f93500c6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500c61a0 con 0x7f9350154900 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f9344001470 con 0x7f9350154900 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 >> v1:172.21.15.73:6789/0 conn(0x7f9350154900 legacy=0x7f93500a7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93380038e0 0x7f9338003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9338004690 0x7f9338004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 0x7f933800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500c8650 con 0x7f9338005000 2023-11-28T02:53:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f933800bf60 con 0x7f93380038e0 2023-11-28T02:53:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f933800c0f0 con 0x7f9338004690 2023-11-28T02:53:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 0x7f933800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9354b93700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93380038e0 0x7f9338003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934ffff700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9338004690 0x7f9338004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 -- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9338004690 msgr2=0x7f9338004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9338004690 0x7f9338004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 -- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93380038e0 msgr2=0x7f9338003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93380038e0 0x7f9338003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 -- 172.21.15.73:0/2903567446 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93380033f0 con 0x7f9338005000 2023-11-28T02:53:01.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f934ffff700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9338004690 0x7f9338004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.963+0000 7f9355394700 1 -- 172.21.15.73:0/2903567446 reap_dead start 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9355394700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 0x7f933800b8e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f9348001730 tx=0x7f9348007bb0).ready entity=mon.0 client_cookie=60814a258e524a93 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348007020 con 0x7f9338005000 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348007180 con 0x7f9338005000 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f934f7fe700 1 -- 172.21.15.73:0/2903567446 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348007520 con 0x7f9338005000 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 -- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 msgr2=0x7f933800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:01.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 0x7f933800b8e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 rx=0x7f9348001730 tx=0x7f9348007bb0).stop 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 -- 172.21.15.73:0/2903567446 shutdown_connections 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 --2- 172.21.15.73:0/2903567446 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9338005000 0x7f933800b8e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 -- 172.21.15.73:0/2903567446 >> 172.21.15.73:0/2903567446 conn(0x7f9350082f80 msgr2=0x7f9350094870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 -- 172.21.15.73:0/2903567446 shutdown_connections 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.964+0000 7f9356df7700 1 -- 172.21.15.73:0/2903567446 wait complete. 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.965+0000 7f9356df7700 1 Processor -- start 2023-11-28T02:53:01.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 -- start start 2023-11-28T02:53:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9338005000 0x7f93500ab160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500ca0a0 0x7f93500ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500ca520 con 0x7f93500c9810 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93500ab820 con 0x7f9338005000 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9356df7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93500abb00 con 0x7f93500ca0a0 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9355394700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9355394700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45192/0 (socket says 172.21.15.73:45192) 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.966+0000 7f9355394700 1 -- 172.21.15.73:0/3231956809 learned_addr learned my addr 172.21.15.73:0/3231956809 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:01.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9354b93700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9338005000 0x7f93500ab160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f934ffff700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500ca0a0 0x7f93500ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 -- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500ca0a0 msgr2=0x7f93500ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500ca0a0 0x7f93500ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 -- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9338005000 msgr2=0x7f93500ab160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9338005000 0x7f93500ab160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 -- 172.21.15.73:0/3231956809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93500c8d30 con 0x7f93500c9810 2023-11-28T02:53:01.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9355394700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9350083f30 tx=0x7f9348015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348007020 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9348013670 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93500c9010 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9348013a10 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.967+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93500adaa0 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.968+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93500ca4a0 con 0x7f93500c9810 2023-11-28T02:53:01.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.968+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f934801f3e0 con 0x7f93500c9810 2023-11-28T02:53:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.969+0000 7f934cff9700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 0x7f933c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.969+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f934807aba0 con 0x7f93500c9810 2023-11-28T02:53:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.969+0000 7f9354b93700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 0x7f933c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.969+0000 7f9354b93700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 0x7f933c061610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9340009fd0 tx=0x7f934000b040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:01.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:01.973+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f934804be10 con 0x7f93500c9810 2023-11-28T02:53:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.178+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f935005f460 con 0x7f93500c9810 2023-11-28T02:53:02.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.179+0000 7f934cff9700 1 -- 172.21.15.73:0/3231956809 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f9348025020 con 0x7f93500c9810 2023-11-28T02:53:02.180 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T02:53:02.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 msgr2=0x7f933c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:02.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 0x7f933c061610 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f9340009fd0 tx=0x7f934000b040).stop 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 msgr2=0x7f93500afdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 rx=0x7f9350083f30 tx=0x7f9348015000).stop 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 shutdown_connections 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f933c05f150 0x7f933c061610 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f93500c9810 0x7f93500afdc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9338005000 0x7f93500ab160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:02.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 --2- 172.21.15.73:0/3231956809 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93500ca0a0 0x7f93500ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.180+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 >> 172.21.15.73:0/3231956809 conn(0x7f9350082f80 msgr2=0x7f93501532c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.181+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 shutdown_connections 2023-11-28T02:53:02.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:02.181+0000 7f9356df7700 1 -- 172.21.15.73:0/3231956809 wait complete. 2023-11-28T02:53:02.193 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T02:53:02.193 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T02:53:02.194 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-11-28T02:53:02.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.195 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:02.211 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42497sec preferred_lft 42497sec 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:02.212 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link tentative 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link tentative 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.213 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T02:53:02.274 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T02:53:02.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.275 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T02:53:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T02:53:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T02:53:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T02:53:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T02:53:02.331 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T02:53:02.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T02:53:02.332 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:02.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.431 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T02:53:02.445 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T02:53:02.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.446 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list-id 2023-11-28T02:53:02.501 INFO:teuthology.orchestra.run.smithi073.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-11-28T02:53:02.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.501 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:02.502 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:02.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T02:53:02.502 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-11-28T02:53:02.502 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:02.589 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.626 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T02:53:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether a6:d8:81:6e:ad:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T02:53:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T02:53:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::a4d8:81ff:fe6e:adad/64 scope link tentative 2023-11-28T02:53:02.627 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether a6:d8:81:6e:ad:ad brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T02:53:02.709 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T02:53:02.710 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.710 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::a4d8:81ff:fe6e:adad/64 scope link tentative 2023-11-28T02:53:02.710 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:02.730 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-11-28T02:53:02.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-11-28T02:53:02.731 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:02.936 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:02.936 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:02.936 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:02.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.1 up 2023-11-28T02:53:02.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T02:53:02.936 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:03.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.017 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.2 2023-11-28T02:53:03.031 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T02:53:03.032 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:03.111 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:03.111 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:03.136 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=2,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:03.136 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=2,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:03.138 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T02:53:03.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.139 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-11-28T02:53:03.204 INFO:tasks.kclient:client.1 config is {} 2023-11-28T02:53:03.204 DEBUG:teuthology.orchestra.run.smithi150:> ceph fs ls 2023-11-28T02:53:03.318 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.317+0000 7fe1ce780700 1 Processor -- start 2023-11-28T02:53:03.318 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.317+0000 7fe1ce780700 1 -- start start 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1ce780700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 0x7fe1c80aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1ce780700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c80c6d80 con 0x7fe1c8154700 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1ce780700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1c8154ae0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1ce780700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c8064c30 con 0x7fe1c80c8360 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1c77fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe1c80aa1a0 0x7fe1c80c7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:46692/0 (socket says 172.21.15.150:46692) 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1c77fe700 1 -- 172.21.15.150:0/2430484153 learned_addr learned my addr 172.21.15.150:0/2430484153 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.318+0000 7fe1ccd1d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 0x7fe1c80aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1c6ffd700 1 -- 172.21.15.150:0/2430484153 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3931685933 0 0) 0x7fe1c8154ae0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.320 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1c6ffd700 1 -- 172.21.15.150:0/2430484153 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1b00033f0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ccd1d700 1 -- 172.21.15.150:0/2430484153 >> v1:172.21.15.150:6789/0 conn(0x7fe1c80aa1a0 legacy=0x7fe1c80c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ccd1d700 1 -- 172.21.15.150:0/2430484153 >> v1:172.21.15.73:6789/0 conn(0x7fe1c8154700 legacy=0x7fe1c80c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ccd1d700 1 -- 172.21.15.150:0/2430484153 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c80c6450 con 0x7fe1c80c8360 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ccd1d700 1 --2- 172.21.15.150:0/2430484153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 0x7fe1c80aa620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe1c0009e90 tx=0x7fe1c0009780).ready entity=mon.2 client_cookie=eb16bce9174199a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1c6ffd700 1 -- 172.21.15.150:0/2430484153 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c000e070 con 0x7fe1c80c8360 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1c6ffd700 1 -- 172.21.15.150:0/2430484153 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1c000bb60 con 0x7fe1c80c8360 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1c6ffd700 1 -- 172.21.15.150:0/2430484153 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1c0012410 con 0x7fe1c80c8360 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ce780700 1 -- 172.21.15.150:0/2430484153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 msgr2=0x7fe1c80aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.319+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2430484153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 0x7fe1c80aa620 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe1c0009e90 tx=0x7fe1c0009780).stop 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.320+0000 7fe1ce780700 1 -- 172.21.15.150:0/2430484153 shutdown_connections 2023-11-28T02:53:03.321 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.320+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2430484153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80c8360 0x7fe1c80aa620 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.322 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.320+0000 7fe1ce780700 1 -- 172.21.15.150:0/2430484153 >> 172.21.15.150:0/2430484153 conn(0x7fe1c80948b0 msgr2=0x7fe1c806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:03.322 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.320+0000 7fe1ce780700 1 -- 172.21.15.150:0/2430484153 shutdown_connections 2023-11-28T02:53:03.322 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.320+0000 7fe1ce780700 1 -- 172.21.15.150:0/2430484153 wait complete. 2023-11-28T02:53:03.325 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.321+0000 7fe1ce780700 1 Processor -- start 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.321+0000 7fe1ce780700 1 -- start start 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1c80c8360 0x7fe1c814e200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c8154700 0x7fe1c814ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80af700 con 0x7fe1c80c8360 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80ae540 con 0x7fe1c8154700 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1c80affe0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ccd1d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c8154700 0x7fe1c814ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:47290/0 (socket says 172.21.15.150:47290) 2023-11-28T02:53:03.326 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 -- 172.21.15.150:0/2160962965 learned_addr learned my addr 172.21.15.150:0/2160962965 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c77fe700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1c80c8360 0x7fe1c814e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 -- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c8154700 msgr2=0x7fe1c814ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c8154700 0x7fe1c814ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 -- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1c80c8360 msgr2=0x7fe1c814e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1c80c8360 0x7fe1c814e200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 -- 172.21.15.150:0/2160962965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1c0009a20 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1c7fff700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe1b800c440 tx=0x7fe1b8007e00).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b800ce40 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1b8007020 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1c80b02c0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b80073c0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.327 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.322+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1c80b0900 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.323+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1c80475e0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.324+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe1b80068b0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.324+0000 7fe1affff700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 0x7fe1b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.324+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe1b8079480 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.324+0000 7fe1c77fe700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 0x7fe1b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:03.328 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.325+0000 7fe1c77fe700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 0x7fe1b4061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe1bc007fd0 tx=0x7fe1bc009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:03.329 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.329+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1b8047af0 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.536 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.535+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fe1c80c6730 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.536 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.536+0000 7fe1affff700 1 -- 172.21.15.150:0/2160962965 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fe1b804a750 con 0x7fe1c80aa1a0 2023-11-28T02:53:03.536 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T02:53:03.538 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 msgr2=0x7fe1b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 0x7fe1b4061610 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fe1bc007fd0 tx=0x7fe1bc009040).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 msgr2=0x7fe1c80af210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fe1b800c440 tx=0x7fe1b8007e00).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 shutdown_connections 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe1b405f150 0x7fe1b4061610 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1c8154700 0x7fe1c814ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1c80aa1a0 0x7fe1c80af210 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 --2- 172.21.15.150:0/2160962965 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1c80c8360 0x7fe1c814e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 >> 172.21.15.150:0/2160962965 conn(0x7fe1c80948b0 msgr2=0x7fe1c80640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:03.539 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 shutdown_connections 2023-11-28T02:53:03.540 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:03.538+0000 7fe1ce780700 1 -- 172.21.15.150:0/2160962965 wait complete. 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T02:53:03.549 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T02:53:03.550 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-11-28T02:53:03.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.550 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2023-11-28T02:53:03.565 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42503sec preferred_lft 42503sec 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2023-11-28T02:53:03.566 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:03.567 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-11-28T02:53:03.567 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add name ceph-brx type bridge 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr flush dev ceph-brx 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set ceph-brx up 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-11-28T02:53:03.567 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:03.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.744 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T02:53:03.787 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-28T02:53:03.788 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.789 DEBUG:teuthology.orchestra.run.smithi150:> route 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T02:53:03.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T02:53:03.840 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:04.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.070 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T02:53:04.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.085 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2023-11-28T02:53:04.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.142 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:04.142 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:04.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T02:53:04.142 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-11-28T02:53:04.143 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:04.231 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-11-28T02:53:04.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-11-28T02:53:04.232 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:04.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.429 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:04.429 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:04.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.0 up 2023-11-28T02:53:04.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.0 master ceph-brx 2023-11-28T02:53:04.429 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:04.522 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.522 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p /home/ubuntu/cephtest/mnt.1 2023-11-28T02:53:04.540 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T02:53:04.540 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:04.632 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:04.632 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:04.724 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,name=1,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:04.724 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,name=1,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:04.727 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T02:53:04.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:04.727 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-11-28T02:53:04.752 INFO:tasks.kclient:client.3 config is {} 2023-11-28T02:53:04.752 DEBUG:teuthology.orchestra.run.smithi150:> ceph fs ls 2023-11-28T02:53:04.893 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.893+0000 7f3d15591700 1 Processor -- start 2023-11-28T02:53:04.893 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.893+0000 7f3d15591700 1 -- start start 2023-11-28T02:53:04.901 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.893+0000 7f3d15591700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3d10154700 0x7f3d100c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:04.901 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.893+0000 7f3d15591700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d100c6d80 con 0x7f3d100c7700 2023-11-28T02:53:04.901 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.893+0000 7f3d15591700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d10154ae0 con 0x7f3d100c7c50 2023-11-28T02:53:04.901 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d15591700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d10064c30 con 0x7f3d10154700 2023-11-28T02:53:04.902 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0f7fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f3d100c7c50 0x7f3d100a7f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.150:46706/0 (socket says 172.21.15.150:46706) 2023-11-28T02:53:04.902 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0f7fe700 1 -- 172.21.15.150:0/1968402436 learned_addr learned my addr 172.21.15.150:0/1968402436 (peer_addr_for_me v1:172.21.15.150:0/0) 2023-11-28T02:53:04.902 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0effd700 1 --2- 172.21.15.150:0/1968402436 >> v2:172.21.15.150:3300/0 conn(0x7f3d10154700 0x7f3d100c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:04.903 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0dffb700 1 -- 172.21.15.150:0/1968402436 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412726522 0 0) 0x7f3d10154ae0 con 0x7f3d100c7c50 2023-11-28T02:53:04.903 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0dffb700 1 -- 172.21.15.150:0/1968402436 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3cf80033f0 con 0x7f3d100c7c50 2023-11-28T02:53:04.903 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0effd700 1 -- 172.21.15.150:0/1968402436 >> v1:172.21.15.150:6789/0 conn(0x7f3d100c7c50 legacy=0x7f3d100a7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:04.904 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0effd700 1 -- 172.21.15.150:0/1968402436 >> v1:172.21.15.73:6789/0 conn(0x7f3d100c7700 legacy=0x7f3d100aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:04.904 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0effd700 1 -- 172.21.15.150:0/1968402436 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d100c6450 con 0x7f3d10154700 2023-11-28T02:53:04.904 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0effd700 1 --2- 172.21.15.150:0/1968402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d100c7210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3d00009760 tx=0x7f3d00009060).ready entity=mon.1 client_cookie=85e9ee50ea94ab52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:04.904 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0dffb700 1 -- 172.21.15.150:0/1968402436 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000f510 con 0x7f3d10154700 2023-11-28T02:53:04.905 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0dffb700 1 -- 172.21.15.150:0/1968402436 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000f670 con 0x7f3d10154700 2023-11-28T02:53:04.905 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.894+0000 7f3d0dffb700 1 -- 172.21.15.150:0/1968402436 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000f9f0 con 0x7f3d10154700 2023-11-28T02:53:04.905 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 -- 172.21.15.150:0/1968402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 msgr2=0x7f3d100c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:04.905 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 --2- 172.21.15.150:0/1968402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d100c7210 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 rx=0x7f3d00009760 tx=0x7f3d00009060).stop 2023-11-28T02:53:04.906 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 -- 172.21.15.150:0/1968402436 shutdown_connections 2023-11-28T02:53:04.906 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 --2- 172.21.15.150:0/1968402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d100c7210 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:04.906 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 -- 172.21.15.150:0/1968402436 >> 172.21.15.150:0/1968402436 conn(0x7f3d100948b0 msgr2=0x7f3d1006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:04.906 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 -- 172.21.15.150:0/1968402436 shutdown_connections 2023-11-28T02:53:04.907 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.895+0000 7f3d15591700 1 -- 172.21.15.150:0/1968402436 wait complete. 2023-11-28T02:53:04.907 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.896+0000 7f3d15591700 1 Processor -- start 2023-11-28T02:53:04.907 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.896+0000 7f3d15591700 1 -- start start 2023-11-28T02:53:04.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:04.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d100c7c50 0x7f3d10154cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:04.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d101551b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:04.908 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d101556a0 con 0x7f3d100c7c50 2023-11-28T02:53:04.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10156490 con 0x7f3d10154700 2023-11-28T02:53:04.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d15591700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d10168f10 con 0x7f3d100c7700 2023-11-28T02:53:04.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0f7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d101551b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:04.909 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:04.910 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.150:47294/0 (socket says 172.21.15.150:47294) 2023-11-28T02:53:04.910 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 -- 172.21.15.150:0/3469113504 learned_addr learned my addr 172.21.15.150:0/3469113504 (peer_addr_for_me v2:172.21.15.150:0/0) 2023-11-28T02:53:04.911 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 -- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 msgr2=0x7f3d101551b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:04.911 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d101551b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:04.912 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0e7fc700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d100c7c50 0x7f3d10154cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:04.912 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 -- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d100c7c50 msgr2=0x7f3d10154cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:04.913 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d100c7c50 0x7f3d10154cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:04.913 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0effd700 1 -- 172.21.15.150:0/3469113504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d000092a0 con 0x7f3d100c7700 2023-11-28T02:53:04.914 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.897+0000 7f3d0f7fe700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d101551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:04.914 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3d0effd700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3d00009e00 tx=0x7f3d00017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0000f7e0 con 0x7f3d100c7700 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d0000f940 con 0x7f3d100c7700 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d10155880 con 0x7f3d100c7700 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d0001b690 con 0x7f3d100c7700 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.898+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d10155e40 con 0x7f3d100c7700 2023-11-28T02:53:04.915 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.899+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d100475e0 con 0x7f3d100c7700 2023-11-28T02:53:04.916 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.899+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3d0001ba00 con 0x7f3d100c7700 2023-11-28T02:53:04.916 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.899+0000 7f3cf6ffd700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 0x7f3cfc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:04.916 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.899+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3d0007b880 con 0x7f3d100c7700 2023-11-28T02:53:04.916 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.899+0000 7f3d0e7fc700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 0x7f3cfc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:04.916 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.900+0000 7f3d0e7fc700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 0x7f3cfc061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f3d100c5690 tx=0x7f3d04009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:04.917 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:04.903+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3d0004cb50 con 0x7f3d100c7700 2023-11-28T02:53:05.108 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.107+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3d100c6730 con 0x7f3d100c7700 2023-11-28T02:53:05.108 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.108+0000 7f3cf6ffd700 1 -- 172.21.15.150:0/3469113504 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3d00028020 con 0x7f3d100c7700 2023-11-28T02:53:05.109 INFO:teuthology.orchestra.run.smithi150.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T02:53:05.111 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.109+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 msgr2=0x7f3cfc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:05.111 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.109+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 0x7f3cfc061610 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f3d100c5690 tx=0x7f3d04009040).stop 2023-11-28T02:53:05.111 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.109+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 msgr2=0x7f3d10142190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:05.111 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.109+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f3d00009e00 tx=0x7f3d00017000).stop 2023-11-28T02:53:05.111 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.109+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 shutdown_connections 2023-11-28T02:53:05.112 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cfc05f150 0x7f3cfc061610 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:05.112 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3d10154700 0x7f3d101551b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:05.112 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3d100c7700 0x7f3d10142190 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:05.112 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 --2- 172.21.15.150:0/3469113504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3d100c7c50 0x7f3d10154cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:05.112 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 >> 172.21.15.150:0/3469113504 conn(0x7f3d100948b0 msgr2=0x7f3d100ca840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:05.113 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 shutdown_connections 2023-11-28T02:53:05.113 INFO:teuthology.orchestra.run.smithi150.stderr:2023-11-28T02:53:05.110+0000 7f3d15591700 1 -- 172.21.15.150:0/3469113504 wait complete. 2023-11-28T02:53:05.120 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi150.front.sepia.ceph.com 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T02:53:05.121 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-11-28T02:53:05.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.121 DEBUG:teuthology.orchestra.run.smithi150:> ip addr 2023-11-28T02:53:05.137 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:05.138 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:05.138 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:05.138 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bc brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f0 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:d9:da:bd brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: altname enp7s0f1 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:34 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T02:53:05.139 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 0c:c4:7a:88:6e:35 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: inet 172.21.15.150/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft 42502sec preferred_lft 42502sec 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::ec4:7aff:fe88:6e35/64 scope link 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 82:34:a1:9a:d5:07 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::bc9c:2dff:fee7:ac49/64 scope link tentative 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T02:53:05.140 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 82:34:a1:9a:d5:07 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T02:53:05.141 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::8034:a1ff:fe9a:d507/64 scope link tentative 2023-11-28T02:53:05.141 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.141 DEBUG:teuthology.orchestra.run.smithi150:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T02:53:05.202 INFO:teuthology.orchestra.run.smithi150.stdout:1 2023-11-28T02:53:05.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.204 DEBUG:teuthology.orchestra.run.smithi150:> route 2023-11-28T02:53:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:Kernel IP routing table 2023-11-28T02:53:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T02:53:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T02:53:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T02:53:05.260 INFO:teuthology.orchestra.run.smithi150.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T02:53:05.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T02:53:05.261 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:05.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.363 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T02:53:05.378 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T02:53:05.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.379 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list-id 2023-11-28T02:53:05.433 INFO:teuthology.orchestra.run.smithi150.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-11-28T02:53:05.434 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.434 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:05.435 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:05.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T02:53:05.435 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-11-28T02:53:05.435 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:05.524 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.524 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.558 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T02:53:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 46:c9:43:07:99:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T02:53:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T02:53:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::44c9:43ff:fe07:9960/64 scope link tentative 2023-11-28T02:53:05.559 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.566 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.566 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.644 INFO:teuthology.orchestra.run.smithi150.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T02:53:05.645 INFO:teuthology.orchestra.run.smithi150.stdout: link/ether 46:c9:43:07:99:60 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-11-28T02:53:05.645 INFO:teuthology.orchestra.run.smithi150.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-11-28T02:53:05.645 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.645 INFO:teuthology.orchestra.run.smithi150.stdout: inet6 fe80::44c9:43ff:fe07:9960/64 scope link tentative 2023-11-28T02:53:05.645 INFO:teuthology.orchestra.run.smithi150.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:05.658 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-11-28T02:53:05.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.658 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:05.658 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-11-28T02:53:05.659 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:05.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.891 DEBUG:teuthology.orchestra.run.smithi150:> (cd / && exec bash -c ' 2023-11-28T02:53:05.892 DEBUG:teuthology.orchestra.run.smithi150:> set -e 2023-11-28T02:53:05.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set brx.1 up 2023-11-28T02:53:05.892 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link set dev brx.1 master ceph-brx 2023-11-28T02:53:05.892 DEBUG:teuthology.orchestra.run.smithi150:> ') 2023-11-28T02:53:05.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:05.973 DEBUG:teuthology.orchestra.run.smithi150:> mkdir -p /home/ubuntu/cephtest/mnt.3 2023-11-28T02:53:05.988 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T02:53:05.988 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:06.066 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:06.067 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:06.092 INFO:teuthology.orchestra.run.smithi150.stdout:parsing options: rw,norequire_active_mds,name=3,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:06.093 INFO:teuthology.orchestra.run.smithi150.stdout:mount.ceph: options "norequire_active_mds,name=3,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:06.095 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T02:53:06.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:06.095 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-11-28T02:53:06.161 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-11-28T02:53:06.195 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:06.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.317+0000 7f61cbcd3700 1 Processor -- start 2023-11-28T02:53:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.318+0000 7f61cbcd3700 1 -- start start 2023-11-28T02:53:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.318+0000 7f61cbcd3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61c40c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.318+0000 7f61cbcd3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c40a7490 con 0x7f61c4067cb0 2023-11-28T02:53:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.318+0000 7f61cbcd3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61c4154d20 con 0x7f61c40c45d0 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.318+0000 7f61cbcd3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4151670 con 0x7f61c4068090 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c9a6f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f61c4067cb0 0x7f61c40c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47796/0 (socket says 172.21.15.73:47796) 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c9a6f700 1 -- 172.21.15.73:0/593185593 learned_addr learned my addr 172.21.15.73:0/593185593 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61ca270700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61c40c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4249266766 0 0) 0x7f61c0001110 con 0x7f61c4067cb0 2023-11-28T02:53:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b00033f0 con 0x7f61c4067cb0 2023-11-28T02:53:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1722920238 0 0) 0x7f61c4154d20 con 0x7f61c40c45d0 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61c0001110 con 0x7f61c40c45d0 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2917617741 0 0) 0x7f61b00033f0 con 0x7f61c4067cb0 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 msgr2=0x7f61c40c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61c40c2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 >> v1:172.21.15.150:6789/0 conn(0x7f61c40c45d0 legacy=0x7f61c40c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c41555c0 con 0x7f61c4067cb0 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f61c0001600 con 0x7f61c4067cb0 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 >> v1:172.21.15.73:6789/0 conn(0x7f61c4067cb0 legacy=0x7f61c40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61b0003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61b0003fb0 0x7f61b0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61b0004b80 0x7f61b0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b000b990 con 0x7f61b0004b80 2023-11-28T02:53:06.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61b000bad0 con 0x7f61b0003fb0 2023-11-28T02:53:06.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61b000bc90 con 0x7f61c4068090 2023-11-28T02:53:06.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61ca270700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61b0004b80 0x7f61b0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61b0003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.319+0000 7f61c926e700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61b0003fb0 0x7f61b0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 -- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61b0003fb0 msgr2=0x7f61b0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61b0003fb0 0x7f61b0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 -- 172.21.15.73:0/593185593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61b0004b80 msgr2=0x7f61b0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61b0004b80 0x7f61b0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 -- 172.21.15.73:0/593185593 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61b00033f0 con 0x7f61c4068090 2023-11-28T02:53:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c926e700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61b0003fb0 0x7f61b0004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61ca270700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61b0004b80 0x7f61b0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61b0003aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f61c000c2a0 tx=0x7f61c4155530).ready entity=mon.2 client_cookie=25f463628780cc34 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0007b50 con 0x7f61c4068090 2023-11-28T02:53:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c000cbd0 con 0x7f61c4068090 2023-11-28T02:53:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61c8a6d700 1 -- 172.21.15.73:0/593185593 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00070d0 con 0x7f61c4068090 2023-11-28T02:53:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61cbcd3700 1 -- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 msgr2=0x7f61b0003aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/593185593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c4068090 0x7f61b0003aa0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f61c000c2a0 tx=0x7f61c4155530).stop 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61ca270700 1 -- 172.21.15.73:0/593185593 reap_dead start 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61cbcd3700 1 -- 172.21.15.73:0/593185593 shutdown_connections 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.320+0000 7f61cbcd3700 1 -- 172.21.15.73:0/593185593 >> 172.21.15.73:0/593185593 conn(0x7f61c40c7fd0 msgr2=0x7f61c40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.321+0000 7f61cbcd3700 1 -- 172.21.15.73:0/593185593 shutdown_connections 2023-11-28T02:53:06.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.321+0000 7f61cbcd3700 1 -- 172.21.15.73:0/593185593 wait complete. 2023-11-28T02:53:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 Processor -- start 2023-11-28T02:53:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 -- start start 2023-11-28T02:53:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c41535f0 0x7f61c4153a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40c67f0 0x7f61c40c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4154370 con 0x7f61c4068090 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61c4144600 con 0x7f61c40c67f0 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.322+0000 7f61cbcd3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61c40c7160 con 0x7f61c41535f0 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34850/0 (socket says 172.21.15.73:34850) 2023-11-28T02:53:06.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 -- 172.21.15.73:0/3359015819 learned_addr learned my addr 172.21.15.73:0/3359015819 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61ca270700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40c67f0 0x7f61c40c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c926e700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c41535f0 0x7f61c4153a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 -- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c41535f0 msgr2=0x7f61c4153a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c41535f0 0x7f61c4153a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 -- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40c67f0 msgr2=0x7f61c40c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40c67f0 0x7f61c40c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 -- 172.21.15.73:0/3359015819 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c4143ff0 con 0x7f61c4068090 2023-11-28T02:53:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61c9a6f700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f61c000c890 tx=0x7f61c0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0001710 con 0x7f61c4068090 2023-11-28T02:53:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c0001870 con 0x7f61c4068090 2023-11-28T02:53:06.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61c4154090 con 0x7f61c4068090 2023-11-28T02:53:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c0004900 con 0x7f61c4068090 2023-11-28T02:53:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.323+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61c41602f0 con 0x7f61c4068090 2023-11-28T02:53:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.324+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61c406c860 con 0x7f61c4068090 2023-11-28T02:53:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.325+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f61c0004cf0 con 0x7f61c4068090 2023-11-28T02:53:06.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.325+0000 7f61b9ffb700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 0x7f61a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.325+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f61c001c040 con 0x7f61c4068090 2023-11-28T02:53:06.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.325+0000 7f61c926e700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 0x7f61a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.325+0000 7f61c926e700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 0x7f61a8061610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f61b4000c00 tx=0x7f61b4007bb0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.329+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61c004c110 con 0x7f61c4068090 2023-11-28T02:53:06.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.494+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f61c40a7490 con 0x7f61c4068090 2023-11-28T02:53:06.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.495+0000 7f61b9ffb700 1 -- 172.21.15.73:0/3359015819 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f61c40a7490 con 0x7f61c4068090 2023-11-28T02:53:06.497 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:06.497 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:06.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 msgr2=0x7f61a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 0x7f61a8061610 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f61b4000c00 tx=0x7f61b4007bb0).stop 2023-11-28T02:53:06.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 msgr2=0x7f61c4161d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 rx=0x7f61c000c890 tx=0x7f61c0015000).stop 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 shutdown_connections 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61a805f150 0x7f61a8061610 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61c40c67f0 0x7f61c40c6c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61c4068090 0x7f61c4161d50 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 --2- 172.21.15.73:0/3359015819 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61c41535f0 0x7f61c4153a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 >> 172.21.15.73:0/3359015819 conn(0x7f61c40c7fd0 msgr2=0x7f61c4150800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 shutdown_connections 2023-11-28T02:53:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.497+0000 7f61cbcd3700 1 -- 172.21.15.73:0/3359015819 wait complete. 2023-11-28T02:53:06.512 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:53:06.513 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:06.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.627+0000 7f7963363700 1 Processor -- start 2023-11-28T02:53:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.627+0000 7f7963363700 1 -- start start 2023-11-28T02:53:06.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7963363700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f795c0c66f0 0x7f795c154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7963363700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c154e40 con 0x7f795c158570 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7963363700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f795c0c7bc0 con 0x7f795c14e530 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7963363700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f795c0c6ba0 con 0x7f795c0c66f0 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7962b62700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f795c158570 0x7f795c0aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47802/0 (socket says 172.21.15.73:47802) 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7962b62700 1 -- 172.21.15.73:0/4083726229 learned_addr learned my addr 172.21.15.73:0/4083726229 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> v2:172.21.15.150:3300/0 conn(0x7f795c0c66f0 0x7f795c154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 557679963 0 0) 0x7f7954001490 con 0x7f795c158570 2023-11-28T02:53:06.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.628+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79480033f0 con 0x7f795c158570 2023-11-28T02:53:06.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4069608962 0 0) 0x7f79480033f0 con 0x7f795c158570 2023-11-28T02:53:06.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 >> v2:172.21.15.150:3300/0 conn(0x7f795c0c66f0 msgr2=0x7f795c154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 --2- 172.21.15.73:0/4083726229 >> v2:172.21.15.150:3300/0 conn(0x7f795c0c66f0 0x7f795c154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 >> v1:172.21.15.150:6789/0 conn(0x7f795c14e530 legacy=0x7f795c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c155920 con 0x7f795c158570 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7954001980 con 0x7f795c158570 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> v2:172.21.15.150:3300/0 conn(0x7f795c0c66f0 0x7f795c154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 >> v1:172.21.15.73:6789/0 conn(0x7f795c158570 legacy=0x7f795c0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79480038e0 0x7f7948003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 0x7f7948004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948005000 0x7f794800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0c6ba0 con 0x7f7948004690 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794800beb0 con 0x7f7948005000 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f794800c070 con 0x7f79480038e0 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 0x7f7948004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7962361700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79480038e0 0x7f7948003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7962b62700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948005000 0x7f794800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 -- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79480038e0 msgr2=0x7f7948003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79480038e0 0x7f7948003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 -- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948005000 msgr2=0x7f794800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948005000 0x7f794800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 -- 172.21.15.73:0/4083726229 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79480033f0 con 0x7f7948004690 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7962b62700 1 -- 172.21.15.73:0/4083726229 reap_dead start 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f7961b60700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 0x7f7948004b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f794c009c70 tx=0x7f794c009060).ready entity=mon.0 client_cookie=6ad285861b2fe08d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c0095c0 con 0x7f7948004690 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00e690 con 0x7f7948004690 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.629+0000 7f796135f700 1 -- 172.21.15.73:0/4083726229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c00ea10 con 0x7f7948004690 2023-11-28T02:53:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 -- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 msgr2=0x7f7948004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 0x7f7948004b10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 rx=0x7f794c009c70 tx=0x7f794c009060).stop 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 -- 172.21.15.73:0/4083726229 shutdown_connections 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 --2- 172.21.15.73:0/4083726229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7948004690 0x7f7948004b10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 -- 172.21.15.73:0/4083726229 >> 172.21.15.73:0/4083726229 conn(0x7f795c064990 msgr2=0x7f795c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 -- 172.21.15.73:0/4083726229 shutdown_connections 2023-11-28T02:53:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.630+0000 7f7963363700 1 -- 172.21.15.73:0/4083726229 wait complete. 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.631+0000 7f7963363700 1 Processor -- start 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.631+0000 7f7963363700 1 -- start start 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948004690 0x7f795c14f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f795c168240 0x7f795c0af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c1686c0 con 0x7f795c14f550 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f795c0aa550 con 0x7f7948004690 2023-11-28T02:53:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7963363700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795c167b00 con 0x7f795c168240 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34868/0 (socket says 172.21.15.73:34868) 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 -- 172.21.15.73:0/1472686768 learned_addr learned my addr 172.21.15.73:0/1472686768 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7962b62700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f795c168240 0x7f795c0af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7962361700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948004690 0x7f795c14f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 -- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f795c168240 msgr2=0x7f795c0af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f795c168240 0x7f795c0af2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 -- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948004690 msgr2=0x7f795c14f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948004690 0x7f795c14f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 -- 172.21.15.73:0/1472686768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f795c0b0090 con 0x7f795c14f550 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.632+0000 7f7961b60700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f794c009260 tx=0x7f794c01a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.633+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c000cb0 con 0x7f795c14f550 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.633+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f794c00e800 con 0x7f795c14f550 2023-11-28T02:53:06.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.633+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f794c025440 con 0x7f795c14f550 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.633+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f795c0b0370 con 0x7f795c14f550 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.633+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f795c0b0930 con 0x7f795c14f550 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.634+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f795c062ee0 con 0x7f795c14f550 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.634+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f794c02e440 con 0x7f795c14f550 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.634+0000 7f79527fc700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 0x7f7940061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.634+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f794c07bcf0 con 0x7f795c14f550 2023-11-28T02:53:06.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.634+0000 7f7962361700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 0x7f7940061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.635+0000 7f7962361700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 0x7f7940061640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7958003b10 tx=0x7f7958007b90).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.638+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f794c04cf60 con 0x7f795c14f550 2023-11-28T02:53:06.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.801+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f795c06c7c0 con 0x7f795c14f550 2023-11-28T02:53:06.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.801+0000 7f79527fc700 1 -- 172.21.15.73:0/1472686768 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f794c016020 con 0x7f795c14f550 2023-11-28T02:53:06.803 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:06.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 msgr2=0x7f7940061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 0x7f7940061640 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 rx=0x7f7958003b10 tx=0x7f7958007b90).stop 2023-11-28T02:53:06.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 msgr2=0x7f795c167d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f794c009260 tx=0x7f794c01a000).stop 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 shutdown_connections 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f794005f180 0x7f7940061640 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f795c168240 0x7f795c0af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7948004690 0x7f795c14f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 --2- 172.21.15.73:0/1472686768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f795c14f550 0x7f795c167d50 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 >> 172.21.15.73:0/1472686768 conn(0x7f795c064990 msgr2=0x7f795c156440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 shutdown_connections 2023-11-28T02:53:06.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.803+0000 7f7963363700 1 -- 172.21.15.73:0/1472686768 wait complete. 2023-11-28T02:53:06.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:06.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.930+0000 7f16c9d8e700 1 Processor -- start 2023-11-28T02:53:06.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.930+0000 7f16c9d8e700 1 -- start start 2023-11-28T02:53:06.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c9d8e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4098d10 0x7f16c40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c9d8e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16c40c6190 0x7f16c4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c9d8e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16c40c72d0 con 0x7f16c4064350 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c9d8e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16c4154bc0 con 0x7f16c40c6190 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c9d8e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40c66c0 con 0x7f16c4098d10 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16c40c6190 0x7f16c4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16c40c6190 0x7f16c4063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34874/0 (socket says 172.21.15.73:34874) 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c2ffd700 1 -- 172.21.15.73:0/3938843811 learned_addr learned my addr 172.21.15.73:0/3938843811 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:06.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.931+0000 7f16c37fe700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4098d10 0x7f16c40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c2ffd700 1 -- 172.21.15.73:0/3938843811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4098d10 msgr2=0x7f16c40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c2ffd700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4098d10 0x7f16c40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c2ffd700 1 -- 172.21.15.73:0/3938843811 >> v1:172.21.15.150:6789/0 conn(0x7f16c4064350 legacy=0x7f16c4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c2ffd700 1 -- 172.21.15.73:0/3938843811 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c40c69a0 con 0x7f16c40c6190 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c2ffd700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c40c6190 0x7f16c4063e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f16b8006fd0 tx=0x7f16b80090f0).ready entity=mon.0 client_cookie=d6af93fae77da442 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c27fc700 1 -- 172.21.15.73:0/3938843811 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800b580 con 0x7f16c40c6190 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c27fc700 1 -- 172.21.15.73:0/3938843811 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800b6e0 con 0x7f16c40c6190 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c27fc700 1 -- 172.21.15.73:0/3938843811 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b800ba60 con 0x7f16c40c6190 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 -- 172.21.15.73:0/3938843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c40c6190 msgr2=0x7f16c4063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c40c6190 0x7f16c4063e60 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 rx=0x7f16b8006fd0 tx=0x7f16b80090f0).stop 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 -- 172.21.15.73:0/3938843811 shutdown_connections 2023-11-28T02:53:06.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4098d10 0x7f16c40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/3938843811 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c40c6190 0x7f16c4063e60 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 -- 172.21.15.73:0/3938843811 >> 172.21.15.73:0/3938843811 conn(0x7f16c40cb310 msgr2=0x7f16c40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 -- 172.21.15.73:0/3938843811 shutdown_connections 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.932+0000 7f16c9d8e700 1 -- 172.21.15.73:0/3938843811 wait complete. 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 Processor -- start 2023-11-28T02:53:06.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 -- start start 2023-11-28T02:53:06.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4064350 0x7f16c4150a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c40c6190 0x7f16c40abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40aad20 con 0x7f16c4098d10 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4156be0 con 0x7f16c40c6190 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c9d8e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c40aaf70 con 0x7f16c4064350 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.934+0000 7f16c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4064350 0x7f16c4150a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34878/0 (socket says 172.21.15.73:34878) 2023-11-28T02:53:06.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 -- 172.21.15.73:0/4003223640 learned_addr learned my addr 172.21.15.73:0/4003223640 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c3fff700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c40c6190 0x7f16c40abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 -- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4064350 msgr2=0x7f16c4150a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4064350 0x7f16c4150a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 -- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c40c6190 msgr2=0x7f16c40abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c40c6190 0x7f16c40abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 -- 172.21.15.73:0/4003223640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16b8009370 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c2ffd700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f16b8009130 tx=0x7f16b8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8000bc0 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16b800b850 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c40ab250 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b8000d20 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.935+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c4150f40 con 0x7f16c4098d10 2023-11-28T02:53:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.936+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c406c860 con 0x7f16c4098d10 2023-11-28T02:53:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.936+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f16b801b8f0 con 0x7f16c4098d10 2023-11-28T02:53:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.937+0000 7f16ab7fe700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 0x7f16b0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.937+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f16b807bac0 con 0x7f16c4098d10 2023-11-28T02:53:06.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.937+0000 7f16c37fe700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 0x7f16b0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.937+0000 7f16c37fe700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 0x7f16b0061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f16b4009fd0 tx=0x7f16b400b040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:06.941+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f16b804cd30 con 0x7f16c4098d10 2023-11-28T02:53:07.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.102+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f16c40c6c80 con 0x7f16c4098d10 2023-11-28T02:53:07.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.103+0000 7f16ab7fe700 1 -- 172.21.15.73:0/4003223640 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f16b801f070 con 0x7f16c4098d10 2023-11-28T02:53:07.105 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.104+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 msgr2=0x7f16b0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.104+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 0x7f16b0061610 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 rx=0x7f16b4009fd0 tx=0x7f16b400b040).stop 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.104+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 msgr2=0x7f16c40aba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.104+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 rx=0x7f16b8009130 tx=0x7f16b8017000).stop 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 shutdown_connections 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b005f150 0x7f16b0061610 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16c40c6190 0x7f16c40abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16c4064350 0x7f16c4150a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 --2- 172.21.15.73:0/4003223640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16c4098d10 0x7f16c40aba50 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 >> 172.21.15.73:0/4003223640 conn(0x7f16c40cb310 msgr2=0x7f16c40aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 shutdown_connections 2023-11-28T02:53:07.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.105+0000 7f16c9d8e700 1 -- 172.21.15.73:0/4003223640 wait complete. 2023-11-28T02:53:07.118 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:53:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.232+0000 7f4315e75700 1 Processor -- start 2023-11-28T02:53:07.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.232+0000 7f4315e75700 1 -- start start 2023-11-28T02:53:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f4315e75700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4310098d10 0x7f43100c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f4315e75700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f4310063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f4315e75700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43100c72d0 con 0x7f4310064350 2023-11-28T02:53:07.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f4315e75700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4310154bc0 con 0x7f4310098d10 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f4315e75700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100c66c0 con 0x7f43100c6190 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f430ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4310064350 0x7f4310158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47808/0 (socket says 172.21.15.73:47808) 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f430ffff700 1 -- 172.21.15.73:0/2074009755 learned_addr learned my addr 172.21.15.73:0/2074009755 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f430f7fe700 1 --2- 172.21.15.73:0/2074009755 >> v2:172.21.15.150:3300/0 conn(0x7f4310098d10 0x7f43100c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f430effd700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f4310063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.233+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4076355936 0 0) 0x7f43100c72d0 con 0x7f4310064350 2023-11-28T02:53:07.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f80033f0 con 0x7f4310064350 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1375937517 0 0) 0x7f42f80033f0 con 0x7f4310064350 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 msgr2=0x7f4310063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f4310063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 >> v2:172.21.15.150:3300/0 conn(0x7f4310098d10 msgr2=0x7f43100c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 --2- 172.21.15.73:0/2074009755 >> v2:172.21.15.150:3300/0 conn(0x7f4310098d10 0x7f43100c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43100c69a0 con 0x7f4310064350 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f4308001470 con 0x7f4310064350 2023-11-28T02:53:07.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 >> v1:172.21.15.73:6789/0 conn(0x7f4310064350 legacy=0x7f4310158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f80038e0 0x7f42f8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f42f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f42f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4310154bc0 con 0x7f42f8004b80 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100c66c0 con 0x7f42f80038e0 2023-11-28T02:53:07.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42f800b8e0 con 0x7f43100c6190 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f42f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430f7fe700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f80038e0 0x7f42f8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430effd700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f42f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 -- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 msgr2=0x7f42f8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c6190 0x7f42f8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 -- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f80038e0 msgr2=0x7f42f8003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42f80038e0 0x7f42f8003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 -- 172.21.15.73:0/2074009755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f80033f0 con 0x7f42f8004b80 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.234+0000 7f430ffff700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f42f8005000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f430800c210 tx=0x7f4308001e90).ready entity=mon.0 client_cookie=f2b660237bedbfdc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4308007a60 con 0x7f42f8004b80 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430800cae0 con 0x7f42f8004b80 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f430e7fc700 1 -- 172.21.15.73:0/2074009755 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430800ce60 con 0x7f42f8004b80 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 -- 172.21.15.73:0/2074009755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 msgr2=0x7f42f8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 --2- 172.21.15.73:0/2074009755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f42f8005000 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f430800c210 tx=0x7f4308001e90).stop 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f430ffff700 1 -- 172.21.15.73:0/2074009755 reap_dead start 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 -- 172.21.15.73:0/2074009755 shutdown_connections 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 -- 172.21.15.73:0/2074009755 >> 172.21.15.73:0/2074009755 conn(0x7f43100cb310 msgr2=0x7f43100cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 -- 172.21.15.73:0/2074009755 shutdown_connections 2023-11-28T02:53:07.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.235+0000 7f4315e75700 1 -- 172.21.15.73:0/2074009755 wait complete. 2023-11-28T02:53:07.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.236+0000 7f4315e75700 1 Processor -- start 2023-11-28T02:53:07.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.236+0000 7f4315e75700 1 -- start start 2023-11-28T02:53:07.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 0x7f43100c4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 0x7f43100c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100c2c90 con 0x7f42f8004b80 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100c7d10 con 0x7f43100ad750 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f4315e75700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100adbd0 con 0x7f43100c4580 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34906/0 (socket says 172.21.15.73:34906) 2023-11-28T02:53:07.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.237+0000 7f430f7fe700 1 -- 172.21.15.73:0/1302156148 learned_addr learned my addr 172.21.15.73:0/1302156148 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430ffff700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 0x7f43100c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430effd700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 0x7f43100c4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 -- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 msgr2=0x7f43100c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 0x7f43100c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 -- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 msgr2=0x7f43100c4090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 0x7f43100c4090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 -- 172.21.15.73:0/1302156148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310153a60 con 0x7f42f8004b80 2023-11-28T02:53:07.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430ffff700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 0x7f43100c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:07.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430effd700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 0x7f43100c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:07.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f430f7fe700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4300011e80 tx=0x7f430000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f430000e040 con 0x7f42f8004b80 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f430000f040 con 0x7f42f8004b80 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43100c7f60 con 0x7f42f8004b80 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4300015450 con 0x7f42f8004b80 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.238+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43100c8520 con 0x7f42f8004b80 2023-11-28T02:53:07.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.239+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43100c8890 con 0x7f42f8004b80 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.239+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4300015880 con 0x7f42f8004b80 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.239+0000 7f42f77fe700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 0x7f42fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.240+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f430007b180 con 0x7f42f8004b80 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.240+0000 7f430effd700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 0x7f42fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.240+0000 7f430effd700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 0x7f42fc061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f4304009270 tx=0x7f4304008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.243+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f430004c450 con 0x7f42f8004b80 2023-11-28T02:53:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.406+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f431006c860 con 0x7f42f8004b80 2023-11-28T02:53:07.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.406+0000 7f42f77fe700 1 -- 172.21.15.73:0/1302156148 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f43000190a0 con 0x7f42f8004b80 2023-11-28T02:53:07.408 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:07.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 msgr2=0x7f42fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 0x7f42fc061610 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7f4304009270 tx=0x7f4304008040).stop 2023-11-28T02:53:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 msgr2=0x7f43100c27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 rx=0x7f4300011e80 tx=0x7f430000b040).stop 2023-11-28T02:53:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 shutdown_connections 2023-11-28T02:53:07.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f42fc05f150 0x7f42fc061610 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43100c4580 0x7f43100c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42f8004b80 0x7f43100c27a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 --2- 172.21.15.73:0/1302156148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43100ad750 0x7f43100c4090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 >> 172.21.15.73:0/1302156148 conn(0x7f43100cb310 msgr2=0x7f4310154d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 shutdown_connections 2023-11-28T02:53:07.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.408+0000 7f4315e75700 1 -- 172.21.15.73:0/1302156148 wait complete. 2023-11-28T02:53:07.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.536+0000 7fb7610de700 1 Processor -- start 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.536+0000 7fb7610de700 1 -- start start 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb7610de700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb75c0c6190 0x7fb75c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb7610de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb75c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb7610de700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb75c158cc0 con 0x7fb75c098d10 2023-11-28T02:53:07.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb7610de700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb75c0c72d0 con 0x7fb75c0c6190 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb7610de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c064570 con 0x7fb75c158320 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75ad9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb75c098d10 0x7fb75c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47812/0 (socket says 172.21.15.73:47812) 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75ad9d700 1 -- 172.21.15.73:0/3957526102 learned_addr learned my addr 172.21.15.73:0/3957526102 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75259c700 1 --2- 172.21.15.73:0/3957526102 >> v2:172.21.15.150:3300/0 conn(0x7fb75c0c6190 0x7fb75c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2166675020 0 0) 0x7fb74c001110 con 0x7fb75c098d10 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75b59e700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb75c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.537+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7480033f0 con 0x7fb75c098d10 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4155087994 0 0) 0x7fb7480033f0 con 0x7fb75c098d10 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 msgr2=0x7fb75c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb75c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 >> v2:172.21.15.150:3300/0 conn(0x7fb75c0c6190 msgr2=0x7fb75c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 --2- 172.21.15.73:0/3957526102 >> v2:172.21.15.150:3300/0 conn(0x7fb75c0c6190 0x7fb75c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c154bc0 con 0x7fb75c098d10 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb74c001600 con 0x7fb75c098d10 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 >> v1:172.21.15.73:6789/0 conn(0x7fb75c098d10 legacy=0x7fb75c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb7480038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb748004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb748004b80 0x7fb748005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0c72d0 con 0x7fb748003e10 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c064570 con 0x7fb748004b80 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb74800b8e0 con 0x7fb75c158320 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb748004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75ad9d700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb7480038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75b59e700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb748004b80 0x7fb748005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 -- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 msgr2=0x7fb7480038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c158320 0x7fb7480038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 -- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb748004b80 msgr2=0x7fb748005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb748004b80 0x7fb748005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 -- 172.21.15.73:0/3957526102 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7480033f0 con 0x7fb748003e10 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75259c700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb748004690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fb744009a80 tx=0x7fb744009060).ready entity=mon.0 client_cookie=f63266a3f51b6614 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb744009430 con 0x7fb748003e10 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74400e640 con 0x7fb748003e10 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.538+0000 7fb75a59c700 1 -- 172.21.15.73:0/3957526102 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74400e9c0 con 0x7fb748003e10 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 -- 172.21.15.73:0/3957526102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 msgr2=0x7fb748004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 --2- 172.21.15.73:0/3957526102 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb748004690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 rx=0x7fb744009a80 tx=0x7fb744009060).stop 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb75b59e700 1 -- 172.21.15.73:0/3957526102 reap_dead start 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 -- 172.21.15.73:0/3957526102 shutdown_connections 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 -- 172.21.15.73:0/3957526102 >> 172.21.15.73:0/3957526102 conn(0x7fb75c0cb310 msgr2=0x7fb75c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 -- 172.21.15.73:0/3957526102 shutdown_connections 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.539+0000 7fb7610de700 1 -- 172.21.15.73:0/3957526102 wait complete. 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.540+0000 7fb7610de700 1 Processor -- start 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.540+0000 7fb7610de700 1 -- start start 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c0afe10 0x7fb75c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb75c1514c0 0x7fb75c0638c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c165690 con 0x7fb748003e10 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c0ae3f0 con 0x7fb75c1514c0 2023-11-28T02:53:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7610de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb75c151f50 con 0x7fb75c0afe10 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34926/0 (socket says 172.21.15.73:34926) 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 -- 172.21.15.73:0/708025500 learned_addr learned my addr 172.21.15.73:0/708025500 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75259c700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c0afe10 0x7fb75c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75b59e700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb75c1514c0 0x7fb75c0638c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 -- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c0afe10 msgr2=0x7fb75c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c0afe10 0x7fb75c0c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 -- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb75c1514c0 msgr2=0x7fb75c0638c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb75c1514c0 0x7fb75c0638c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 -- 172.21.15.73:0/708025500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb75c152230 con 0x7fb748003e10 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb75ad9d700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb74c004a10 tx=0x7fb75c0a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c004e80 con 0x7fb748003e10 2023-11-28T02:53:07.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.541+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb74c00f850 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.542+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb74c00fbd0 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.542+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb75c152510 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.542+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb75c0c28a0 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.542+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb75c06c860 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.543+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb74c01d3b0 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.543+0000 7fb7537fe700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 0x7fb73c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.543+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb74c0799f0 con 0x7fb748003e10 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.543+0000 7fb75259c700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 0x7fb73c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.544+0000 7fb75259c700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 0x7fb73c0615e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb75c1544a0 tx=0x7fb74401a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.547+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb74c04ad10 con 0x7fb748003e10 2023-11-28T02:53:07.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.710+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb75c158cc0 con 0x7fb748003e10 2023-11-28T02:53:07.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.711+0000 7fb7537fe700 1 -- 172.21.15.73:0/708025500 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb75c158cc0 con 0x7fb748003e10 2023-11-28T02:53:07.713 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:07.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 msgr2=0x7fb73c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 0x7fb73c0615e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7fb75c1544a0 tx=0x7fb74401a000).stop 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 msgr2=0x7fb75c1650b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 rx=0x7fb74c004a10 tx=0x7fb75c0a8040).stop 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 shutdown_connections 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb73c05f120 0x7fb73c0615e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb75c1514c0 0x7fb75c0638c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb748003e10 0x7fb75c1650b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.712+0000 7fb7610de700 1 --2- 172.21.15.73:0/708025500 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb75c0afe10 0x7fb75c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.713+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 >> 172.21.15.73:0/708025500 conn(0x7fb75c0cb310 msgr2=0x7fb75c155330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.713+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 shutdown_connections 2023-11-28T02:53:07.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.713+0000 7fb7610de700 1 -- 172.21.15.73:0/708025500 wait complete. 2023-11-28T02:53:07.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:07.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.838+0000 7fc227fff700 1 Processor -- start 2023-11-28T02:53:07.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.838+0000 7fc227fff700 1 -- start start 2023-11-28T02:53:07.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.838+0000 7fc227fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc2280c0180 0x7fc228157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc227fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2280c12c0 con 0x7fc2280c1750 2023-11-28T02:53:07.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc227fff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2280c06b0 con 0x7fc228157660 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc227fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2280c0990 con 0x7fc2280c0180 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc2277fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc2280c1750 0x7fc2280648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47818/0 (socket says 172.21.15.73:47818) 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc2277fe700 1 -- 172.21.15.73:0/1595243000 learned_addr learned my addr 172.21.15.73:0/1595243000 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc2267fc700 1 --2- 172.21.15.73:0/1595243000 >> v2:172.21.15.150:3300/0 conn(0x7fc2280c0180 0x7fc228157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705788923 0 0) 0x7fc2280c12c0 con 0x7fc2280c1750 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2100033f0 con 0x7fc2280c1750 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4293233978 0 0) 0x7fc2100033f0 con 0x7fc2280c1750 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 >> v2:172.21.15.150:3300/0 conn(0x7fc2280c0180 msgr2=0x7fc228157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 --2- 172.21.15.73:0/1595243000 >> v2:172.21.15.150:3300/0 conn(0x7fc2280c0180 0x7fc228157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 >> v1:172.21.15.150:6789/0 conn(0x7fc228157660 legacy=0x7fc2281589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2280c0c70 con 0x7fc2280c1750 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc2200017f0 con 0x7fc2280c1750 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 >> v1:172.21.15.73:6789/0 conn(0x7fc2280c1750 legacy=0x7fc2280648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2100038e0 0x7fc210003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc210004690 0x7fc210004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 0x7fc21000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2280c0990 con 0x7fc210005000 2023-11-28T02:53:07.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc21000bf60 con 0x7fc210004690 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc21000c0f0 con 0x7fc2100038e0 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.839+0000 7fc2277fe700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 0x7fc21000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2267fc700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc210004690 0x7fc210004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc226ffd700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2100038e0 0x7fc210003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 -- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2100038e0 msgr2=0x7fc210003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2100038e0 0x7fc210003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 -- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc210004690 msgr2=0x7fc210004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc210004690 0x7fc210004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 -- 172.21.15.73:0/1595243000 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2100033f0 con 0x7fc210005000 2023-11-28T02:53:07.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2267fc700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc210004690 0x7fc210004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 -- 172.21.15.73:0/1595243000 reap_dead start 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc2277fe700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 0x7fc21000b8e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc22000c480 tx=0x7fc22000c790).ready entity=mon.0 client_cookie=61cdec19a558f870 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220002030 con 0x7fc210005000 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc22000ce20 con 0x7fc210005000 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc225ffb700 1 -- 172.21.15.73:0/1595243000 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220005d30 con 0x7fc210005000 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc227fff700 1 -- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 msgr2=0x7fc21000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc227fff700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 0x7fc21000b8e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 rx=0x7fc22000c480 tx=0x7fc22000c790).stop 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc227fff700 1 -- 172.21.15.73:0/1595243000 shutdown_connections 2023-11-28T02:53:07.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc227fff700 1 --2- 172.21.15.73:0/1595243000 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc210005000 0x7fc21000b8e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.840+0000 7fc227fff700 1 -- 172.21.15.73:0/1595243000 >> 172.21.15.73:0/1595243000 conn(0x7fc2280cb000 msgr2=0x7fc2280cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.841+0000 7fc227fff700 1 -- 172.21.15.73:0/1595243000 shutdown_connections 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.841+0000 7fc227fff700 1 -- 172.21.15.73:0/1595243000 wait complete. 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 Processor -- start 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 -- start start 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210005000 0x7fc2281685a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 0x7fc228151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228168110 con 0x7fc2281562a0 2023-11-28T02:53:07.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc228156bf0 con 0x7fc228153cb0 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.842+0000 7fc227fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc228167ac0 con 0x7fc210005000 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34950/0 (socket says 172.21.15.73:34950) 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 -- 172.21.15.73:0/368298188 learned_addr learned my addr 172.21.15.73:0/368298188 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc226ffd700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210005000 0x7fc2281685a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2267fc700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 0x7fc228151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 -- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210005000 msgr2=0x7fc2281685a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210005000 0x7fc2281685a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 -- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 msgr2=0x7fc228151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:07.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 0x7fc228151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 -- 172.21.15.73:0/368298188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc228167da0 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2267fc700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 0x7fc228151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc2277fe700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fc220000f80 tx=0x7fc220015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc220002030 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2200017b0 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc228156ed0 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2200138b0 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.843+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc228156720 con 0x7fc2281562a0 2023-11-28T02:53:07.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.844+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc228167f30 con 0x7fc2281562a0 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.844+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc220013c20 con 0x7fc2281562a0 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.845+0000 7fc20effd700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 0x7fc214061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.845+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc22007b460 con 0x7fc2281562a0 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.845+0000 7fc226ffd700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 0x7fc214061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.845+0000 7fc226ffd700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 0x7fc214061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc218007fd0 tx=0x7fc218007bb0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:07.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:07.849+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc22004c730 con 0x7fc2281562a0 2023-11-28T02:53:08.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.013+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc228156970 con 0x7fc2281562a0 2023-11-28T02:53:08.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.014+0000 7fc20effd700 1 -- 172.21.15.73:0/368298188 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fc22001a070 con 0x7fc2281562a0 2023-11-28T02:53:08.015 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:08.016 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:08.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 msgr2=0x7fc214061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 0x7fc214061610 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7fc218007fd0 tx=0x7fc218007bb0).stop 2023-11-28T02:53:08.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 msgr2=0x7fc2280abaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 rx=0x7fc220000f80 tx=0x7fc220015000).stop 2023-11-28T02:53:08.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 shutdown_connections 2023-11-28T02:53:08.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc21405f150 0x7fc214061610 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2281562a0 0x7fc2280abaa0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc210005000 0x7fc2281685a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 --2- 172.21.15.73:0/368298188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc228153cb0 0x7fc228151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.015+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 >> 172.21.15.73:0/368298188 conn(0x7fc2280cb000 msgr2=0x7fc2280aaf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.016+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 shutdown_connections 2023-11-28T02:53:08.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.016+0000 7fc227fff700 1 -- 172.21.15.73:0/368298188 wait complete. 2023-11-28T02:53:08.030 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:53:08.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:08.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.148+0000 7f5930dea700 1 Processor -- start 2023-11-28T02:53:08.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.148+0000 7f5930dea700 1 -- start start 2023-11-28T02:53:08.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f5930dea700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f592c098d10 0x7f592c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f5930dea700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f592c158320 0x7f592c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f5930dea700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f592c158cc0 con 0x7f592c0c6190 2023-11-28T02:53:08.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f5930dea700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c0c72d0 con 0x7f592c098d10 2023-11-28T02:53:08.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f5930dea700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f592c064570 con 0x7f592c158320 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f592c098d10 0x7f592c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f592c098d10 0x7f592c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34958/0 (socket says 172.21.15.73:34958) 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 -- 172.21.15.73:0/1022015184 learned_addr learned my addr 172.21.15.73:0/1022015184 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 -- 172.21.15.73:0/1022015184 >> v2:172.21.15.150:3300/0 conn(0x7f592c158320 msgr2=0x7f592c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 --2- 172.21.15.73:0/1022015184 >> v2:172.21.15.150:3300/0 conn(0x7f592c158320 0x7f592c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 -- 172.21.15.73:0/1022015184 >> v1:172.21.15.150:6789/0 conn(0x7f592c0c6190 legacy=0x7f592c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:08.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.149+0000 7f592a59c700 1 -- 172.21.15.73:0/1022015184 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f592c154bc0 con 0x7f592c098d10 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f592a59c700 1 --2- 172.21.15.73:0/1022015184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c098d10 0x7f592c0c7760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f592c0657a0 tx=0x7f591c009840).ready entity=mon.0 client_cookie=7fa9483b86e7608a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f592959a700 1 -- 172.21.15.73:0/1022015184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00e070 con 0x7f592c098d10 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f592959a700 1 -- 172.21.15.73:0/1022015184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f591c00ba10 con 0x7f592c098d10 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f592959a700 1 -- 172.21.15.73:0/1022015184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f591c00bd90 con 0x7f592c098d10 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 -- 172.21.15.73:0/1022015184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c098d10 msgr2=0x7f592c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 --2- 172.21.15.73:0/1022015184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c098d10 0x7f592c0c7760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 rx=0x7f592c0657a0 tx=0x7f591c009840).stop 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 -- 172.21.15.73:0/1022015184 shutdown_connections 2023-11-28T02:53:08.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 --2- 172.21.15.73:0/1022015184 >> v2:172.21.15.150:3300/0 conn(0x7f592c158320 0x7f592c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 --2- 172.21.15.73:0/1022015184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c098d10 0x7f592c0c7760 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 -- 172.21.15.73:0/1022015184 >> 172.21.15.73:0/1022015184 conn(0x7f592c0cb310 msgr2=0x7f592c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 -- 172.21.15.73:0/1022015184 shutdown_connections 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.150+0000 7f5930dea700 1 -- 172.21.15.73:0/1022015184 wait complete. 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.151+0000 7f5930dea700 1 Processor -- start 2023-11-28T02:53:08.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 -- start start 2023-11-28T02:53:08.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 0x7f592c0c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f592c158320 0x7f592c159a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15aee0 con 0x7f592c0c6190 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15a2d0 con 0x7f592c098d10 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5930dea700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f592c15a5b0 con 0x7f592c158320 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5929d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5929d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34960/0 (socket says 172.21.15.73:34960) 2023-11-28T02:53:08.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.152+0000 7f5929d9b700 1 -- 172.21.15.73:0/1752479064 learned_addr learned my addr 172.21.15.73:0/1752479064 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:08.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f592a59c700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 0x7f592c0c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f592ad9d700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f592c158320 0x7f592c159a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 -- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f592c158320 msgr2=0x7f592c159a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f592c158320 0x7f592c159a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 -- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 msgr2=0x7f592c0c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 0x7f592c0c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 -- 172.21.15.73:0/1752479064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c009ae0 con 0x7f592c0c6190 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f592a59c700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 0x7f592c0c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5929d9b700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f591400b470 tx=0x7f5914006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914012410 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5914012590 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f592c15a890 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5914012950 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.153+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f592c159f40 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.154+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f592c154d50 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.154+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5914012d00 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.155+0000 7f59227fc700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 0x7f590c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.155+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5914079eb0 con 0x7f592c0c6190 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.155+0000 7f592a59c700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 0x7f590c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.155+0000 7f592a59c700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 0x7f590c061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f591c009880 tx=0x7f592c0a8040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.159+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f592c154d50 con 0x7f592c0c6190 2023-11-28T02:53:08.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.321+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f592c154d50 con 0x7f592c0c6190 2023-11-28T02:53:08.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.322+0000 7f59227fc700 1 -- 172.21.15.73:0/1752479064 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f592c154d50 con 0x7f592c0c6190 2023-11-28T02:53:08.323 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 msgr2=0x7f590c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 0x7f590c061610 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f591c009880 tx=0x7f592c0a8040).stop 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 msgr2=0x7f592c159560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 rx=0x7f591400b470 tx=0x7f5914006f90).stop 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 shutdown_connections 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f590c05f150 0x7f590c061610 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f592c158320 0x7f592c159a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f592c098d10 0x7f592c0c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 --2- 172.21.15.73:0/1752479064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f592c0c6190 0x7f592c159560 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.323+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 >> 172.21.15.73:0/1752479064 conn(0x7f592c0cb310 msgr2=0x7f592c0ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.324+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 shutdown_connections 2023-11-28T02:53:08.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.324+0000 7f5930dea700 1 -- 172.21.15.73:0/1752479064 wait complete. 2023-11-28T02:53:08.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.450+0000 7efd2133c700 1 Processor -- start 2023-11-28T02:53:08.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 -- start start 2023-11-28T02:53:08.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efd1c14e530 0x7efd1c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd1c158570 0x7efd1c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd1c0c7bc0 con 0x7efd1c154950 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c155920 con 0x7efd1c158570 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.451+0000 7efd2133c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd1c154e00 con 0x7efd1c14e530 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd1c158570 0x7efd1c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efd1c154950 0x7efd1c14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47824/0 (socket says 172.21.15.73:47824) 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1b7fe700 1 -- 172.21.15.73:0/935343264 learned_addr learned my addr 172.21.15.73:0/935343264 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1bfff700 1 --2- 172.21.15.73:0/935343264 >> v2:172.21.15.150:3300/0 conn(0x7efd1c14e530 0x7efd1c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 -- 172.21.15.73:0/935343264 >> v2:172.21.15.150:3300/0 conn(0x7efd1c14e530 msgr2=0x7efd1c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 --2- 172.21.15.73:0/935343264 >> v2:172.21.15.150:3300/0 conn(0x7efd1c14e530 0x7efd1c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 -- 172.21.15.73:0/935343264 >> v1:172.21.15.73:6789/0 conn(0x7efd1c154950 legacy=0x7efd1c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1affd700 1 -- 172.21.15.73:0/935343264 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4281806238 0 0) 0x7efd1c0c7bc0 con 0x7efd1c154950 2023-11-28T02:53:08.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 -- 172.21.15.73:0/935343264 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd1c155080 con 0x7efd1c158570 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd20b3b700 1 --2- 172.21.15.73:0/935343264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c158570 0x7efd1c158a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7efd1c064450 tx=0x7efd14009780).ready entity=mon.0 client_cookie=471e3a7186a648db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1affd700 1 -- 172.21.15.73:0/935343264 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1400e070 con 0x7efd1c158570 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1affd700 1 -- 172.21.15.73:0/935343264 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1400bb60 con 0x7efd1c158570 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.452+0000 7efd1affd700 1 -- 172.21.15.73:0/935343264 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd14012410 con 0x7efd1c158570 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 -- 172.21.15.73:0/935343264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c158570 msgr2=0x7efd1c158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 --2- 172.21.15.73:0/935343264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c158570 0x7efd1c158a00 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 rx=0x7efd1c064450 tx=0x7efd14009780).stop 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 -- 172.21.15.73:0/935343264 shutdown_connections 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 --2- 172.21.15.73:0/935343264 >> v2:172.21.15.150:3300/0 conn(0x7efd1c14e530 0x7efd1c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 --2- 172.21.15.73:0/935343264 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c158570 0x7efd1c158a00 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 -- 172.21.15.73:0/935343264 >> 172.21.15.73:0/935343264 conn(0x7efd1c064990 msgr2=0x7efd1c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 -- 172.21.15.73:0/935343264 shutdown_connections 2023-11-28T02:53:08.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.453+0000 7efd2133c700 1 -- 172.21.15.73:0/935343264 wait complete. 2023-11-28T02:53:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.454+0000 7efd2133c700 1 Processor -- start 2023-11-28T02:53:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.454+0000 7efd2133c700 1 -- start start 2023-11-28T02:53:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd1c14e530 0x7efd1c14de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 0x7efd1c0c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0c2af0 con 0x7efd1c154950 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c14e330 con 0x7efd1c14e530 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd1c0c4ca0 con 0x7efd1c158570 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34980/0 (socket says 172.21.15.73:34980) 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 -- 172.21.15.73:0/1817766270 learned_addr learned my addr 172.21.15.73:0/1817766270 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1bfff700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd1c14e530 0x7efd1c14de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd20b3b700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 0x7efd1c0c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 -- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 msgr2=0x7efd1c0c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 0x7efd1c0c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 -- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd1c14e530 msgr2=0x7efd1c14de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd1c14e530 0x7efd1c14de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 -- 172.21.15.73:0/1817766270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd14009a20 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd20b3b700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 0x7efd1c0c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd1b7fe700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7efd1000c4e0 tx=0x7efd10007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100072e0 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd10007440 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd1c0c21c0 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd100057a0 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.455+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd1c0c2780 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.456+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd1c158b90 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.457+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efd10005bd0 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.457+0000 7efd03fff700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 0x7efd08061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.457+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efd10079ab0 con 0x7efd1c154950 2023-11-28T02:53:08.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.457+0000 7efd1bfff700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 0x7efd08061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.457+0000 7efd1bfff700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 0x7efd08061610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7efd0c009c30 tx=0x7efd0c009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.461+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd1004ad80 con 0x7efd1c154950 2023-11-28T02:53:08.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.627+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efd1c062ee0 con 0x7efd1c154950 2023-11-28T02:53:08.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.628+0000 7efd03fff700 1 -- 172.21.15.73:0/1817766270 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7efd10018030 con 0x7efd1c154950 2023-11-28T02:53:08.629 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:08.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.629+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 msgr2=0x7efd08061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.629+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 0x7efd08061610 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7efd0c009c30 tx=0x7efd0c009060).stop 2023-11-28T02:53:08.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 msgr2=0x7efd1c0c2f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 rx=0x7efd1000c4e0 tx=0x7efd10007c70).stop 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 shutdown_connections 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd0805f150 0x7efd08061610 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd1c158570 0x7efd1c0c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd1c14e530 0x7efd1c14de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 --2- 172.21.15.73:0/1817766270 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd1c154950 0x7efd1c0c2f80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 >> 172.21.15.73:0/1817766270 conn(0x7efd1c064990 msgr2=0x7efd1c0cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 shutdown_connections 2023-11-28T02:53:08.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.630+0000 7efd2133c700 1 -- 172.21.15.73:0/1817766270 wait complete. 2023-11-28T02:53:08.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:53:08.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.758+0000 7f15b74fc700 1 Processor -- start 2023-11-28T02:53:08.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.758+0000 7f15b74fc700 1 -- start start 2023-11-28T02:53:08.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b74fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15b00c6190 0x7f15b00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b74fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f15b0158320 0x7f15b01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b74fc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15b0158cc0 con 0x7f15b0098d10 2023-11-28T02:53:08.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b74fc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b00c72d0 con 0x7f15b0158320 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b74fc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15b0064570 con 0x7f15b00c6190 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b5a99700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f15b0158320 0x7f15b01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b5298700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f15b0098d10 0x7f15b0063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47828/0 (socket says 172.21.15.73:47828) 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.759+0000 7f15b5298700 1 -- 172.21.15.73:0/1558896414 learned_addr learned my addr 172.21.15.73:0/1558896414 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b4a97700 1 --2- 172.21.15.73:0/1558896414 >> v2:172.21.15.150:3300/0 conn(0x7f15b00c6190 0x7f15b00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b5a99700 1 -- 172.21.15.73:0/1558896414 >> v2:172.21.15.150:3300/0 conn(0x7f15b00c6190 msgr2=0x7f15b00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:08.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b5a99700 1 --2- 172.21.15.73:0/1558896414 >> v2:172.21.15.150:3300/0 conn(0x7f15b00c6190 0x7f15b00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b5a99700 1 -- 172.21.15.73:0/1558896414 >> v1:172.21.15.73:6789/0 conn(0x7f15b0098d10 legacy=0x7f15b0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15affff700 1 -- 172.21.15.73:0/1558896414 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1961587546 0 0) 0x7f15a0001490 con 0x7f15b0098d10 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b5a99700 1 -- 172.21.15.73:0/1558896414 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15b0154bc0 con 0x7f15b0158320 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b5a99700 1 --2- 172.21.15.73:0/1558896414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b01587d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f15b00c5b40 tx=0x7f15a8009060).ready entity=mon.0 client_cookie=35c4c9b13e18dc43 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15affff700 1 -- 172.21.15.73:0/1558896414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800b580 con 0x7f15b0158320 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15affff700 1 -- 172.21.15.73:0/1558896414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a800b6e0 con 0x7f15b0158320 2023-11-28T02:53:08.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15affff700 1 -- 172.21.15.73:0/1558896414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800ba60 con 0x7f15b0158320 2023-11-28T02:53:08.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 -- 172.21.15.73:0/1558896414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 msgr2=0x7f15b01587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1558896414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b01587d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f15b00c5b40 tx=0x7f15a8009060).stop 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 -- 172.21.15.73:0/1558896414 shutdown_connections 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1558896414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b01587d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1558896414 >> v2:172.21.15.150:3300/0 conn(0x7f15b00c6190 0x7f15b00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.760+0000 7f15b74fc700 1 -- 172.21.15.73:0/1558896414 >> 172.21.15.73:0/1558896414 conn(0x7f15b00cb310 msgr2=0x7f15b00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.761+0000 7f15b74fc700 1 -- 172.21.15.73:0/1558896414 shutdown_connections 2023-11-28T02:53:08.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.761+0000 7f15b74fc700 1 -- 172.21.15.73:0/1558896414 wait complete. 2023-11-28T02:53:08.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.762+0000 7f15b74fc700 1 Processor -- start 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.762+0000 7f15b74fc700 1 -- start start 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b0098d10 0x7f15b00c4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15b00c6190 0x7f15b014ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b00c2580 con 0x7f15b0158320 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15b014f2a0 con 0x7f15b0098d10 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15b014f580 con 0x7f15b00c6190 2023-11-28T02:53:08.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34998/0 (socket says 172.21.15.73:34998) 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 -- 172.21.15.73:0/1268470050 learned_addr learned my addr 172.21.15.73:0/1268470050 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5298700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b0098d10 0x7f15b00c4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b4a97700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15b00c6190 0x7f15b014ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 -- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15b00c6190 msgr2=0x7f15b014ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15b00c6190 0x7f15b014ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 -- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b0098d10 msgr2=0x7f15b00c4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b0098d10 0x7f15b00c4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 -- 172.21.15.73:0/1268470050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15a80092a0 con 0x7f15b0158320 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b5a99700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f15b00c2360 tx=0x7f15a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a800ba60 con 0x7f15b0158320 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15a801b410 con 0x7f15b0158320 2023-11-28T02:53:08.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15b00c27d0 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15a801b790 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.763+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15b014ef00 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.764+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15b00c2cf0 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.764+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f15a801bbc0 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.765+0000 7f15ad7fa700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 0x7f159c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.765+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f15a807bf10 con 0x7f15b0158320 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.765+0000 7f15b5298700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 0x7f159c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.765+0000 7f15b5298700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 0x7f159c061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f15a0001e20 tx=0x7f15a0009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:08.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.769+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f15b00c2cf0 con 0x7f15b0158320 2023-11-28T02:53:08.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.930+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f15b00c2cf0 con 0x7f15b0158320 2023-11-28T02:53:08.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.930+0000 7f15ad7fa700 1 -- 172.21.15.73:0/1268470050 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f15b00c2cf0 con 0x7f15b0158320 2023-11-28T02:53:08.932 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 msgr2=0x7f159c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 0x7f159c061610 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f15a0001e20 tx=0x7f15a0009040).stop 2023-11-28T02:53:08.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 msgr2=0x7f15b0147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 rx=0x7f15b00c2360 tx=0x7f15a8017000).stop 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 shutdown_connections 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f159c05f150 0x7f159c061610 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15b0158320 0x7f15b0147f00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15b0098d10 0x7f15b00c4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 --2- 172.21.15.73:0/1268470050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15b00c6190 0x7f15b014ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 >> 172.21.15.73:0/1268470050 conn(0x7f15b00cb310 msgr2=0x7f15b00c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 shutdown_connections 2023-11-28T02:53:08.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:08.932+0000 7f15b74fc700 1 -- 172.21.15.73:0/1268470050 wait complete. 2023-11-28T02:53:08.945 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:53:09.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.058+0000 7f981e933700 1 Processor -- start 2023-11-28T02:53:09.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.058+0000 7f981e933700 1 -- start start 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981e933700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9818098d10 0x7f98180c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981e933700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9818158320 0x7f98181587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981e933700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9818158cc0 con 0x7f98180c6190 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981e933700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98180c72d0 con 0x7f9818158320 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981e933700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9818064570 con 0x7f9818098d10 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981ced0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9818158320 0x7f98181587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981ced0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9818158320 0x7f98181587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35006/0 (socket says 172.21.15.73:35006) 2023-11-28T02:53:09.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981ced0700 1 -- 172.21.15.73:0/1461621226 learned_addr learned my addr 172.21.15.73:0/1461621226 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f981ced0700 1 -- 172.21.15.73:0/1461621226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9818098d10 msgr2=0x7f98180c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.059+0000 7f9817fff700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9818098d10 0x7f98180c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981ced0700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9818098d10 0x7f98180c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981ced0700 1 -- 172.21.15.73:0/1461621226 >> v1:172.21.15.150:6789/0 conn(0x7f98180c6190 legacy=0x7f9818063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981ced0700 1 -- 172.21.15.73:0/1461621226 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9818154bc0 con 0x7f9818158320 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981ced0700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f98181587d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f9810009f20 tx=0x7f9810009840).ready entity=mon.0 client_cookie=55bf2de19dc1552 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f9816ffd700 1 -- 172.21.15.73:0/1461621226 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000e070 con 0x7f9818158320 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f9816ffd700 1 -- 172.21.15.73:0/1461621226 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f981000ba40 con 0x7f9818158320 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f9816ffd700 1 -- 172.21.15.73:0/1461621226 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f981000bdc0 con 0x7f9818158320 2023-11-28T02:53:09.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 -- 172.21.15.73:0/1461621226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 msgr2=0x7f98181587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f98181587d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 rx=0x7f9810009f20 tx=0x7f9810009840).stop 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 -- 172.21.15.73:0/1461621226 shutdown_connections 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f98181587d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 --2- 172.21.15.73:0/1461621226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9818098d10 0x7f98180c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 -- 172.21.15.73:0/1461621226 >> 172.21.15.73:0/1461621226 conn(0x7f98180cb310 msgr2=0x7f98180cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 -- 172.21.15.73:0/1461621226 shutdown_connections 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.060+0000 7f981e933700 1 -- 172.21.15.73:0/1461621226 wait complete. 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.061+0000 7f981e933700 1 Processor -- start 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 -- start start 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9818098d10 0x7f9818154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98180c6190 0x7f9818156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9818155240 con 0x7f9818158320 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98180b02b0 con 0x7f9818098d10 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981e933700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9818168d10 con 0x7f98180c6190 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.062+0000 7f981ced0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35008/0 (socket says 172.21.15.73:35008) 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 -- 172.21.15.73:0/1906146941 learned_addr learned my addr 172.21.15.73:0/1906146941 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f98177fe700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98180c6190 0x7f9818156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f9817fff700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9818098d10 0x7f9818154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 -- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98180c6190 msgr2=0x7f9818156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98180c6190 0x7f9818156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 -- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9818098d10 msgr2=0x7f9818154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9818098d10 0x7f9818154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 -- 172.21.15.73:0/1906146941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9810009ae0 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981ced0700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f98180c2360 tx=0x7f98180a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98100092e0 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9810007610 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98180afc60 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98100079d0 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.063+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98180af910 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.064+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9818046de0 con 0x7f9818158320 2023-11-28T02:53:09.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.064+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9810003040 con 0x7f9818158320 2023-11-28T02:53:09.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.065+0000 7f97fffff700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 0x7f9804061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.065+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f981007bb50 con 0x7f9818158320 2023-11-28T02:53:09.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.065+0000 7f9817fff700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 0x7f9804061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.065+0000 7f9817fff700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 0x7f9804061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f980800ab30 tx=0x7f9808008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.069+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f981004ce20 con 0x7f9818158320 2023-11-28T02:53:09.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.232+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9818158cc0 con 0x7f9818158320 2023-11-28T02:53:09.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.232+0000 7f97fffff700 1 -- 172.21.15.73:0/1906146941 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9818158cc0 con 0x7f9818158320 2023-11-28T02:53:09.234 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 msgr2=0x7f9804061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 0x7f9804061610 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 rx=0x7f980800ab30 tx=0x7f9808008040).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 msgr2=0x7f9818152150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 rx=0x7f98180c2360 tx=0x7f98180a8040).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 shutdown_connections 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f980405f150 0x7f9804061610 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9818158320 0x7f9818152150 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9818098d10 0x7f9818154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 --2- 172.21.15.73:0/1906146941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98180c6190 0x7f9818156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 >> 172.21.15.73:0/1906146941 conn(0x7f98180cb310 msgr2=0x7f98181536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 shutdown_connections 2023-11-28T02:53:09.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.234+0000 7f981e933700 1 -- 172.21.15.73:0/1906146941 wait complete. 2023-11-28T02:53:09.247 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:09.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.359+0000 7f9f83fff700 1 Processor -- start 2023-11-28T02:53:09.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.359+0000 7f9f83fff700 1 -- start start 2023-11-28T02:53:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f84157660 0x7f9f84157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 0x7f9f841589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c1750 0x7f9f840c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f840c12c0 con 0x7f9f84157660 2023-11-28T02:53:09.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f840c06b0 con 0x7f9f840c0180 2023-11-28T02:53:09.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f83fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840c0990 con 0x7f9f840c1750 2023-11-28T02:53:09.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f82ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f84157660 0x7f9f84157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f82ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9f84157660 0x7f9f84157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35014/0 (socket says 172.21.15.73:35014) 2023-11-28T02:53:09.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f82ffd700 1 -- 172.21.15.73:0/4051068878 learned_addr learned my addr 172.21.15.73:0/4051068878 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f827fc700 1 --2- 172.21.15.73:0/4051068878 >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 0x7f9f841589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f82ffd700 1 -- 172.21.15.73:0/4051068878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c1750 msgr2=0x7f9f840c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f837fe700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c1750 0x7f9f840c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.360+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c1750 0x7f9f840c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f82ffd700 1 -- 172.21.15.73:0/4051068878 >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 msgr2=0x7f9f841589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/4051068878 >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 0x7f9f841589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f82ffd700 1 -- 172.21.15.73:0/4051068878 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f840c0c70 con 0x7f9f84157660 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f827fc700 1 --2- 172.21.15.73:0/4051068878 >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 0x7f9f841589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:09.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f84157a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f9f840c5820 tx=0x7f9f740090f0).ready entity=mon.0 client_cookie=6f9e5afa4c5636a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f81ffb700 1 -- 172.21.15.73:0/4051068878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400b600 con 0x7f9f84157660 2023-11-28T02:53:09.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f81ffb700 1 -- 172.21.15.73:0/4051068878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7400b760 con 0x7f9f84157660 2023-11-28T02:53:09.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f81ffb700 1 -- 172.21.15.73:0/4051068878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7400bae0 con 0x7f9f84157660 2023-11-28T02:53:09.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 -- 172.21.15.73:0/4051068878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 msgr2=0x7f9f84157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f84157a40 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 rx=0x7f9f840c5820 tx=0x7f9f740090f0).stop 2023-11-28T02:53:09.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 -- 172.21.15.73:0/4051068878 shutdown_connections 2023-11-28T02:53:09.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f84157a40 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 --2- 172.21.15.73:0/4051068878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c1750 0x7f9f840c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 --2- 172.21.15.73:0/4051068878 >> v2:172.21.15.150:3300/0 conn(0x7f9f840c0180 0x7f9f841589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 -- 172.21.15.73:0/4051068878 >> 172.21.15.73:0/4051068878 conn(0x7f9f840cb000 msgr2=0x7f9f840cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 -- 172.21.15.73:0/4051068878 shutdown_connections 2023-11-28T02:53:09.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.361+0000 7f9f83fff700 1 -- 172.21.15.73:0/4051068878 wait complete. 2023-11-28T02:53:09.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.362+0000 7f9f83fff700 1 Processor -- start 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 -- start start 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c0180 0x7f9f8413c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f840c1750 0x7f9f840adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840af1f0 con 0x7f9f84157660 2023-11-28T02:53:09.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840c1df0 con 0x7f9f840c1750 2023-11-28T02:53:09.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f83fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f840ae950 con 0x7f9f840c0180 2023-11-28T02:53:09.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35018/0 (socket says 172.21.15.73:35018) 2023-11-28T02:53:09.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f837fe700 1 -- 172.21.15.73:0/3759426055 learned_addr learned my addr 172.21.15.73:0/3759426055 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c0180 0x7f9f8413c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.363+0000 7f9f827fc700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f840c1750 0x7f9f840adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 -- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c0180 msgr2=0x7f9f8413c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c0180 0x7f9f8413c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 -- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f840c1750 msgr2=0x7f9f840adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f840c1750 0x7f9f840adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 -- 172.21.15.73:0/3759426055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f74009370 con 0x7f9f84157660 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f837fe700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9f7c00cee0 tx=0x7f9f7c007870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c00ad50 con 0x7f9f84157660 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f7c016400 con 0x7f9f84157660 2023-11-28T02:53:09.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f840aec30 con 0x7f9f84157660 2023-11-28T02:53:09.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f7c016780 con 0x7f9f84157660 2023-11-28T02:53:09.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.364+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f8413c6a0 con 0x7f9f84157660 2023-11-28T02:53:09.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.365+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9f840c1bc0 con 0x7f9f84157660 2023-11-28T02:53:09.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.365+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9f7c009030 con 0x7f9f84157660 2023-11-28T02:53:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.366+0000 7f9f6affd700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 0x7f9f70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.366+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 0x7f9f70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.366+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9f7c07a850 con 0x7f9f84157660 2023-11-28T02:53:09.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.366+0000 7f9f82ffd700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 0x7f9f70061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f9f74009130 tx=0x7f9f74017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.370+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9f840c1bc0 con 0x7f9f84157660 2023-11-28T02:53:09.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.534+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9f840c1bc0 con 0x7f9f84157660 2023-11-28T02:53:09.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.534+0000 7f9f6affd700 1 -- 172.21.15.73:0/3759426055 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f9f840c1bc0 con 0x7f9f84157660 2023-11-28T02:53:09.537 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:09.537 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 msgr2=0x7f9f70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 0x7f9f70061610 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 rx=0x7f9f74009130 tx=0x7f9f74017000).stop 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 msgr2=0x7f9f840ab6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 rx=0x7f9f7c00cee0 tx=0x7f9f7c007870).stop 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 shutdown_connections 2023-11-28T02:53:09.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9f7005f150 0x7f9f70061610 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9f84157660 0x7f9f840ab6b0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9f840c0180 0x7f9f8413c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 --2- 172.21.15.73:0/3759426055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9f840c1750 0x7f9f840adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 >> 172.21.15.73:0/3759426055 conn(0x7f9f840cb000 msgr2=0x7f9f840acab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 shutdown_connections 2023-11-28T02:53:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.537+0000 7f9f83fff700 1 -- 172.21.15.73:0/3759426055 wait complete. 2023-11-28T02:53:09.551 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:53:09.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:09.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.666+0000 7f78dd2f7700 1 Processor -- start 2023-11-28T02:53:09.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.666+0000 7f78dd2f7700 1 -- start start 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dd2f7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78d814e530 0x7f78d80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dd2f7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d8158570 0x7f78d8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dd2f7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78d80c7bc0 con 0x7f78d8154950 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dd2f7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78d8155920 con 0x7f78d814e530 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dd2f7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d8154e00 con 0x7f78d8158570 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78d7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78d814e530 0x7f78d80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78d77fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78d8154950 0x7f78d814e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47830/0 (socket says 172.21.15.73:47830) 2023-11-28T02:53:09.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78d77fe700 1 -- 172.21.15.73:0/3645844581 learned_addr learned my addr 172.21.15.73:0/3645844581 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.667+0000 7f78dcaf6700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d8158570 0x7f78d8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d7fff700 1 -- 172.21.15.73:0/3645844581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d8158570 msgr2=0x7f78d8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d7fff700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d8158570 0x7f78d8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d7fff700 1 -- 172.21.15.73:0/3645844581 >> v1:172.21.15.73:6789/0 conn(0x7f78d8154950 legacy=0x7f78d814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d7fff700 1 -- 172.21.15.73:0/3645844581 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8155080 con 0x7f78d814e530 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d6ffd700 1 -- 172.21.15.73:0/3645844581 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 154100502 0 0) 0x7f78d80c7bc0 con 0x7f78d8154950 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d7fff700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d814e530 0x7f78d80c66f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f78c8009760 tx=0x7f78c8009060).ready entity=mon.0 client_cookie=a9740f855f808819 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d6ffd700 1 -- 172.21.15.73:0/3645844581 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800f510 con 0x7f78d814e530 2023-11-28T02:53:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d6ffd700 1 -- 172.21.15.73:0/3645844581 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c800f670 con 0x7f78d814e530 2023-11-28T02:53:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78d6ffd700 1 -- 172.21.15.73:0/3645844581 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c800f9f0 con 0x7f78d814e530 2023-11-28T02:53:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 -- 172.21.15.73:0/3645844581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d814e530 msgr2=0x7f78d80c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d814e530 0x7f78d80c66f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 rx=0x7f78c8009760 tx=0x7f78c8009060).stop 2023-11-28T02:53:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 -- 172.21.15.73:0/3645844581 shutdown_connections 2023-11-28T02:53:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d8158570 0x7f78d8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/3645844581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d814e530 0x7f78d80c66f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 -- 172.21.15.73:0/3645844581 >> 172.21.15.73:0/3645844581 conn(0x7f78d8064990 msgr2=0x7f78d8064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 -- 172.21.15.73:0/3645844581 shutdown_connections 2023-11-28T02:53:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.668+0000 7f78dd2f7700 1 -- 172.21.15.73:0/3645844581 wait complete. 2023-11-28T02:53:09.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.669+0000 7f78dd2f7700 1 Processor -- start 2023-11-28T02:53:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 -- start start 2023-11-28T02:53:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d814e530 0x7f78d813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d8154950 0x7f78d80bffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c09c0 con 0x7f78d8158570 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c2ed0 con 0x7f78d8154950 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.670+0000 7f78dd2f7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78d80c22c0 con 0x7f78d814e530 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35038/0 (socket says 172.21.15.73:35038) 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 -- 172.21.15.73:0/410941865 learned_addr learned my addr 172.21.15.73:0/410941865 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78d7fff700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d814e530 0x7f78d813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78d77fe700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d8154950 0x7f78d80bffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 -- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d814e530 msgr2=0x7f78d813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d814e530 0x7f78d813c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 -- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d8154950 msgr2=0x7f78d80bffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d8154950 0x7f78d80bffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 -- 172.21.15.73:0/410941865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c80092a0 con 0x7f78d8158570 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dcaf6700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f78d0009df0 tx=0x7f78d000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000caa0 con 0x7f78d8158570 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d000f040 con 0x7f78d8158570 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78d80c25a0 con 0x7f78d8158570 2023-11-28T02:53:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d000ce10 con 0x7f78d8158570 2023-11-28T02:53:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.671+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78d80c2b60 con 0x7f78d8158570 2023-11-28T02:53:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.672+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78d8062ee0 con 0x7f78d8158570 2023-11-28T02:53:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.672+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f78d0013610 con 0x7f78d8158570 2023-11-28T02:53:09.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.673+0000 7f78bffff700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 0x7f78c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.673+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f78d007a8f0 con 0x7f78d8158570 2023-11-28T02:53:09.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.673+0000 7f78d7fff700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 0x7f78c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.673+0000 7f78d7fff700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 0x7f78c4061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f78c8009fd0 tx=0x7f78c8017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.677+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78d004bbc0 con 0x7f78d8158570 2023-11-28T02:53:09.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.839+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f78d806c7c0 con 0x7f78d8158570 2023-11-28T02:53:09.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.840+0000 7f78bffff700 1 -- 172.21.15.73:0/410941865 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f78d004f1e0 con 0x7f78d8158570 2023-11-28T02:53:09.841 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:09.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 msgr2=0x7f78c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 0x7f78c4061610 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7f78c8009fd0 tx=0x7f78c8017000).stop 2023-11-28T02:53:09.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 msgr2=0x7f78d80c04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 rx=0x7f78d0009df0 tx=0x7f78d000b040).stop 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 shutdown_connections 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f78c405f150 0x7f78c4061610 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78d8158570 0x7f78d80c04d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78d814e530 0x7f78d813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 --2- 172.21.15.73:0/410941865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78d8154950 0x7f78d80bffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 >> 172.21.15.73:0/410941865 conn(0x7f78d8064990 msgr2=0x7f78d81565a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 shutdown_connections 2023-11-28T02:53:09.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.842+0000 7f78dd2f7700 1 -- 172.21.15.73:0/410941865 wait complete. 2023-11-28T02:53:09.855 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:09.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.969+0000 7f916c27f700 1 Processor -- start 2023-11-28T02:53:09.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.969+0000 7f916c27f700 1 -- start start 2023-11-28T02:53:09.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916c27f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916c27f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f91640c6190 0x7f9164063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916c27f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91640c72d0 con 0x7f9164064350 2023-11-28T02:53:09.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916c27f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9164154bc0 con 0x7f91640c6190 2023-11-28T02:53:09.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916c27f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c66c0 con 0x7f9164098d10 2023-11-28T02:53:09.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916a81c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9164064350 0x7f9164158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47836/0 (socket says 172.21.15.73:47836) 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916a81c700 1 -- 172.21.15.73:0/3060337973 learned_addr learned my addr 172.21.15.73:0/3060337973 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916a01b700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> v2:172.21.15.150:3300/0 conn(0x7f91640c6190 0x7f9164063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2329536393 0 0) 0x7f91640c72d0 con 0x7f9164064350 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.970+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91500033f0 con 0x7f9164064350 2023-11-28T02:53:09.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2415759222 0 0) 0x7f91500033f0 con 0x7f9164064350 2023-11-28T02:53:09.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 msgr2=0x7f91640c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91640c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 >> v2:172.21.15.150:3300/0 conn(0x7f91640c6190 msgr2=0x7f9164063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 --2- 172.21.15.73:0/3060337973 >> v2:172.21.15.150:3300/0 conn(0x7f91640c6190 0x7f9164063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640c69a0 con 0x7f9164064350 2023-11-28T02:53:09.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f915c001470 con 0x7f9164064350 2023-11-28T02:53:09.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> v2:172.21.15.150:3300/0 conn(0x7f91640c6190 0x7f9164063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:09.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 >> v1:172.21.15.73:6789/0 conn(0x7f9164064350 legacy=0x7f9164158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91500038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f9150004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9150004b80 0x7f9150005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9164154bc0 con 0x7f9150003e10 2023-11-28T02:53:09.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c66c0 con 0x7f9150004b80 2023-11-28T02:53:09.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915000b8e0 con 0x7f9164098d10 2023-11-28T02:53:09.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f9150004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916a01b700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91500038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916a81c700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9150004b80 0x7f9150005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 -- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 msgr2=0x7f91500038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91500038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 -- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9150004b80 msgr2=0x7f9150005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9150004b80 0x7f9150005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 -- 172.21.15.73:0/3060337973 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91500033f0 con 0x7f9150003e10 2023-11-28T02:53:09.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916a01b700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9164098d10 0x7f91500038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:09.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916a81c700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9150004b80 0x7f9150005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:09.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f916981a700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f9150004690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f9154009c70 tx=0x7f9154009060).ready entity=mon.0 client_cookie=f7554913a3102b9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91540095c0 con 0x7f9150003e10 2023-11-28T02:53:09.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f915400e6f0 con 0x7f9150003e10 2023-11-28T02:53:09.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.971+0000 7f9169019700 1 -- 172.21.15.73:0/3060337973 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f915400ea70 con 0x7f9150003e10 2023-11-28T02:53:09.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 -- 172.21.15.73:0/3060337973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 msgr2=0x7f9150004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 --2- 172.21.15.73:0/3060337973 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f9150004690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 rx=0x7f9154009c70 tx=0x7f9154009060).stop 2023-11-28T02:53:09.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916a81c700 1 -- 172.21.15.73:0/3060337973 reap_dead start 2023-11-28T02:53:09.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 -- 172.21.15.73:0/3060337973 shutdown_connections 2023-11-28T02:53:09.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 -- 172.21.15.73:0/3060337973 >> 172.21.15.73:0/3060337973 conn(0x7f91640cb310 msgr2=0x7f91640cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 -- 172.21.15.73:0/3060337973 shutdown_connections 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.972+0000 7f916c27f700 1 -- 172.21.15.73:0/3060337973 wait complete. 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.973+0000 7f916c27f700 1 Processor -- start 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.973+0000 7f916c27f700 1 -- start start 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91640a8ba0 0x7f91640aa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 0x7f91641532f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f916414dae0 con 0x7f9150003e10 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c7660 con 0x7f91640aad70 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916c27f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91640c81e0 con 0x7f91640a8ba0 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35064/0 (socket says 172.21.15.73:35064) 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 -- 172.21.15.73:0/197436663 learned_addr learned my addr 172.21.15.73:0/197436663 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a81c700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 0x7f91641532f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 -- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91640a8ba0 msgr2=0x7f91640aa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91640a8ba0 0x7f91640aa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 -- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 msgr2=0x7f91641532f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:09.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 0x7f91641532f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:09.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 -- 172.21.15.73:0/197436663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91640a9020 con 0x7f9150003e10 2023-11-28T02:53:09.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a81c700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 0x7f91641532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:09.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.974+0000 7f916a01b700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f91600105a0 tx=0x7f916000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.975+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f916000e030 con 0x7f9150003e10 2023-11-28T02:53:09.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.975+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f916000f040 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.975+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91640aa8e0 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.975+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9160015690 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.975+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91640c7b90 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.976+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f916406c860 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.976+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9160015ac0 con 0x7f9150003e10 2023-11-28T02:53:09.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.976+0000 7f915a7fc700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 0x7f9148061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:09.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.976+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f916007aed0 con 0x7f9150003e10 2023-11-28T02:53:09.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.976+0000 7f916981a700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 0x7f9148061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:09.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.977+0000 7f916981a700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 0x7f9148061640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f9164153ea0 tx=0x7f9154019000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:09.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:09.980+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f916004c1a0 con 0x7f9150003e10 2023-11-28T02:53:10.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.141+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f91640c6c80 con 0x7f9150003e10 2023-11-28T02:53:10.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.141+0000 7f915a7fc700 1 -- 172.21.15.73:0/197436663 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f916004f7c0 con 0x7f9150003e10 2023-11-28T02:53:10.143 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:10.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 msgr2=0x7f9148061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 0x7f9148061640 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 rx=0x7f9164153ea0 tx=0x7f9154019000).stop 2023-11-28T02:53:10.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 msgr2=0x7f916414d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 rx=0x7f91600105a0 tx=0x7f916000b040).stop 2023-11-28T02:53:10.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 shutdown_connections 2023-11-28T02:53:10.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f914805f180 0x7f9148061640 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91640aad70 0x7f91641532f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9150003e10 0x7f916414d5f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 --2- 172.21.15.73:0/197436663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91640a8ba0 0x7f91640aa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:10.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 >> 172.21.15.73:0/197436663 conn(0x7f91640cb310 msgr2=0x7f91641518e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 shutdown_connections 2023-11-28T02:53:10.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.143+0000 7f916c27f700 1 -- 172.21.15.73:0/197436663 wait complete. 2023-11-28T02:53:10.156 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:53:10.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.271+0000 7f7fda4de700 1 Processor -- start 2023-11-28T02:53:10.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.271+0000 7f7fda4de700 1 -- start start 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.271+0000 7f7fda4de700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fd414e530 0x7f7fd40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fda4de700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd40c7bc0 con 0x7f7fd4158570 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fda4de700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fd4155920 con 0x7f7fd4154950 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fda4de700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fd4154e00 con 0x7f7fd414e530 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd9cdd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7fd4158570 0x7f7fd40aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47842/0 (socket says 172.21.15.73:47842) 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd94dc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fd414e530 0x7f7fd40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd9cdd700 1 -- 172.21.15.73:0/1511345775 learned_addr learned my addr 172.21.15.73:0/1511345775 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd94dc700 1 -- 172.21.15.73:0/1511345775 >> v1:172.21.15.150:6789/0 conn(0x7f7fd4154950 legacy=0x7f7fd414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd94dc700 1 -- 172.21.15.73:0/1511345775 >> v1:172.21.15.73:6789/0 conn(0x7f7fd4158570 legacy=0x7f7fd40aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd3fff700 1 -- 172.21.15.73:0/1511345775 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066225892 0 0) 0x7f7fd40c7bc0 con 0x7f7fd4158570 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd94dc700 1 -- 172.21.15.73:0/1511345775 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4155080 con 0x7f7fd414e530 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd94dc700 1 --2- 172.21.15.73:0/1511345775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd40c66f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f7fc4009e60 tx=0x7f7fc4009780).ready entity=mon.0 client_cookie=cc0506600bba5983 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd3fff700 1 -- 172.21.15.73:0/1511345775 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400e070 con 0x7f7fd414e530 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd3fff700 1 -- 172.21.15.73:0/1511345775 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc400bc30 con 0x7f7fd414e530 2023-11-28T02:53:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.272+0000 7f7fd3fff700 1 -- 172.21.15.73:0/1511345775 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc4012410 con 0x7f7fd414e530 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 -- 172.21.15.73:0/1511345775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 msgr2=0x7f7fd40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 --2- 172.21.15.73:0/1511345775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd40c66f0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 rx=0x7f7fc4009e60 tx=0x7f7fc4009780).stop 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 -- 172.21.15.73:0/1511345775 shutdown_connections 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 --2- 172.21.15.73:0/1511345775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd40c66f0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 -- 172.21.15.73:0/1511345775 >> 172.21.15.73:0/1511345775 conn(0x7f7fd4064990 msgr2=0x7f7fd4064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 -- 172.21.15.73:0/1511345775 shutdown_connections 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.273+0000 7f7fda4de700 1 -- 172.21.15.73:0/1511345775 wait complete. 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.274+0000 7f7fda4de700 1 Processor -- start 2023-11-28T02:53:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.274+0000 7f7fda4de700 1 -- start start 2023-11-28T02:53:10.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 0x7f7fd40c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd4158570 0x7f7fd41681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd413d6d0 con 0x7f7fd414e530 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40c4530 con 0x7f7fd4154950 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fda4de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fd40c3920 con 0x7f7fd4158570 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35084/0 (socket says 172.21.15.73:35084) 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 -- 172.21.15.73:0/3722330613 learned_addr learned my addr 172.21.15.73:0/3722330613 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 -- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd4158570 msgr2=0x7f7fd41681b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd8cdb700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 0x7f7fd40c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd9cdd700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd4158570 0x7f7fd41681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd4158570 0x7f7fd41681b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 -- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 msgr2=0x7f7fd40c3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 0x7f7fd40c3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 -- 172.21.15.73:0/3722330613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009a20 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd8cdb700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 0x7f7fd40c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.275+0000 7f7fd94dc700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7fc400b6c0 tx=0x7f7fc4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.276+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc401d630 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.276+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc401d790 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.276+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fd40c3c00 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.276+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc401db10 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.276+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fd40c41c0 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.277+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fd4062ee0 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.277+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7fc4027460 con 0x7f7fd414e530 2023-11-28T02:53:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.277+0000 7f7fd17fa700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 0x7f7fc0061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.277+0000 7f7fd8cdb700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 0x7f7fc0061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.277+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7fc407c4e0 con 0x7f7fd414e530 2023-11-28T02:53:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.278+0000 7f7fd8cdb700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 0x7f7fc0061640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7fd414f3f0 tx=0x7f7fc8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.281+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7fc404d750 con 0x7f7fd414e530 2023-11-28T02:53:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.444+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7fd406c7c0 con 0x7f7fd414e530 2023-11-28T02:53:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.445+0000 7f7fd17fa700 1 -- 172.21.15.73:0/3722330613 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7fc4050d70 con 0x7f7fd414e530 2023-11-28T02:53:10.446 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.446+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 msgr2=0x7f7fc0061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.446+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 0x7f7fc0061640 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 rx=0x7f7fd414f3f0 tx=0x7f7fc8009040).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 msgr2=0x7f7fd414ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 rx=0x7f7fc400b6c0 tx=0x7f7fc4009780).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 shutdown_connections 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fc005f180 0x7f7fc0061640 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fd4158570 0x7f7fd41681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fd414e530 0x7f7fd414ef00 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 --2- 172.21.15.73:0/3722330613 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fd4154950 0x7f7fd40c3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 >> 172.21.15.73:0/3722330613 conn(0x7f7fd4064990 msgr2=0x7f7fd40ad180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 shutdown_connections 2023-11-28T02:53:10.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.447+0000 7f7fda4de700 1 -- 172.21.15.73:0/3722330613 wait complete. 2023-11-28T02:53:10.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:53:10.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.575+0000 7f8af0929700 1 Processor -- start 2023-11-28T02:53:10.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.575+0000 7f8af0929700 1 -- start start 2023-11-28T02:53:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8af0929700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8aec098d10 0x7f8aec0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8af0929700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec0c72d0 con 0x7f8aec158320 2023-11-28T02:53:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8af0929700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec064570 con 0x7f8aec0c6190 2023-11-28T02:53:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8af0929700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec154bc0 con 0x7f8aec098d10 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8aead9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8aec158320 0x7f8aec0a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47846/0 (socket says 172.21.15.73:47846) 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8aead9d700 1 -- 172.21.15.73:0/1379612555 learned_addr learned my addr 172.21.15.73:0/1379612555 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750135475 0 0) 0x7f8aec0c72d0 con 0x7f8aec158320 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ad40033f0 con 0x7f8aec158320 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.576+0000 7f8aea59c700 1 --2- 172.21.15.73:0/1379612555 >> v2:172.21.15.150:3300/0 conn(0x7f8aec098d10 0x7f8aec0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 859848405 0 0) 0x7f8ad40033f0 con 0x7f8aec158320 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 >> v2:172.21.15.150:3300/0 conn(0x7f8aec098d10 msgr2=0x7f8aec0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 --2- 172.21.15.73:0/1379612555 >> v2:172.21.15.150:3300/0 conn(0x7f8aec098d10 0x7f8aec0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 >> v1:172.21.15.150:6789/0 conn(0x7f8aec0c6190 legacy=0x7f8aec063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec0c66c0 con 0x7f8aec158320 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8ae40017f0 con 0x7f8aec158320 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 >> v1:172.21.15.73:6789/0 conn(0x7f8aec158320 legacy=0x7f8aec0a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8ad4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4004690 0x7f8ad4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad4005050 0x7f8ad400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec154bc0 con 0x7f8ad40038e0 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400bf30 con 0x7f8ad4005050 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad400c0f0 con 0x7f8ad4004690 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8ad4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aead9d700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad4005050 0x7f8ad400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae9d9b700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4004690 0x7f8ad4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 -- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4004690 msgr2=0x7f8ad4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4004690 0x7f8ad4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 -- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad4005050 msgr2=0x7f8ad400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad4005050 0x7f8ad400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 -- 172.21.15.73:0/1379612555 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40033f0 con 0x7f8ad40038e0 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aead9d700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad4005050 0x7f8ad400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aead9d700 1 -- 172.21.15.73:0/1379612555 reap_dead start 2023-11-28T02:53:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8aea59c700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8ad4003d00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f8adc00ba00 tx=0x7f8adc008040).ready entity=mon.0 client_cookie=934a8701b6a36f35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc00b340 con 0x7f8ad40038e0 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc013480 con 0x7f8ad40038e0 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.577+0000 7f8ae959a700 1 -- 172.21.15.73:0/1379612555 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc013800 con 0x7f8ad40038e0 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 -- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 msgr2=0x7f8ad4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8ad4003d00 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 rx=0x7f8adc00ba00 tx=0x7f8adc008040).stop 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 -- 172.21.15.73:0/1379612555 shutdown_connections 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 --2- 172.21.15.73:0/1379612555 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8ad4003d00 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 -- 172.21.15.73:0/1379612555 >> 172.21.15.73:0/1379612555 conn(0x7f8aec0cb310 msgr2=0x7f8aec0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 -- 172.21.15.73:0/1379612555 shutdown_connections 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.578+0000 7f8af0929700 1 -- 172.21.15.73:0/1379612555 wait complete. 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.579+0000 7f8af0929700 1 Processor -- start 2023-11-28T02:53:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.579+0000 7f8af0929700 1 -- start start 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0a84b0 0x7f8aec0a8930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec156980 0x7f8aec156e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec1564f0 con 0x7f8ad40038e0 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0a9e70 con 0x7f8aec156980 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec14c800 con 0x7f8aec0a84b0 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35104/0 (socket says 172.21.15.73:35104) 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 -- 172.21.15.73:0/3070667421 learned_addr learned my addr 172.21.15.73:0/3070667421 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 -- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0a84b0 msgr2=0x7f8aec0a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0a84b0 0x7f8aec0a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 -- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec156980 msgr2=0x7f8aec156e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec156980 0x7f8aec156e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 -- 172.21.15.73:0/3070667421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec0a8f70 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8aea59c700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f8adc00d160 tx=0x7f8adc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012bd0 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8adc012d30 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec0a9250 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8adc012ea0 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.580+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec165550 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.581+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec1567d0 con 0x7f8ad40038e0 2023-11-28T02:53:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.581+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8adc01a070 con 0x7f8ad40038e0 2023-11-28T02:53:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.582+0000 7f8ada7fc700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 0x7f8acc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.582+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8adc07ad50 con 0x7f8ad40038e0 2023-11-28T02:53:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.582+0000 7f8ae9d9b700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 0x7f8acc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.582+0000 7f8ae9d9b700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 0x7f8acc061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8ae0007fd0 tx=0x7f8ae0009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.586+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8adc07e030 con 0x7f8ad40038e0 2023-11-28T02:53:10.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.753+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8aec156340 con 0x7f8ad40038e0 2023-11-28T02:53:10.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.753+0000 7f8ada7fc700 1 -- 172.21.15.73:0/3070667421 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8adc04c020 con 0x7f8ad40038e0 2023-11-28T02:53:10.754 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:10.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 msgr2=0x7f8acc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 0x7f8acc061610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f8ae0007fd0 tx=0x7f8ae0009040).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 msgr2=0x7f8aec14c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 rx=0x7f8adc00d160 tx=0x7f8adc008040).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 shutdown_connections 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8acc05f150 0x7f8acc061610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec156980 0x7f8aec156e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ad40038e0 0x7f8aec14c310 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 --2- 172.21.15.73:0/3070667421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0a84b0 0x7f8aec0a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 >> 172.21.15.73:0/3070667421 conn(0x7f8aec0cb310 msgr2=0x7f8aec0ca0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 shutdown_connections 2023-11-28T02:53:10.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.755+0000 7f8af0929700 1 -- 172.21.15.73:0/3070667421 wait complete. 2023-11-28T02:53:10.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:10.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.882+0000 7f5de2888700 1 Processor -- start 2023-11-28T02:53:10.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.882+0000 7f5de2888700 1 -- start start 2023-11-28T02:53:10.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.883+0000 7f5de2888700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.883+0000 7f5de2888700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.883+0000 7f5de2888700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ddc096800 con 0x7f5ddc1544d0 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.883+0000 7f5de2888700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ddc067650 con 0x7f5ddc095e80 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.883+0000 7f5de2888700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0683d0 con 0x7f5ddc064e20 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5de0e25700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5de0e25700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:42836/0 (socket says 172.21.15.73:42836) 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5de0e25700 1 -- 172.21.15.73:0/2604298224 learned_addr learned my addr 172.21.15.73:0/2604298224 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:10.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 -- 172.21.15.73:0/2604298224 >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 msgr2=0x7f5ddc096310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:10.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 --2- 172.21.15.73:0/2604298224 >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 -- 172.21.15.73:0/2604298224 >> v1:172.21.15.150:6789/0 conn(0x7f5ddc1544d0 legacy=0x7f5ddc066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 -- 172.21.15.73:0/2604298224 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ddc067850 con 0x7f5ddc064e20 2023-11-28T02:53:10.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5de0e25700 1 --2- 172.21.15.73:0/2604298224 >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:10.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddb7fe700 1 --2- 172.21.15.73:0/2604298224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc155750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5dd0009760 tx=0x7f5dd0009060).ready entity=mon.2 client_cookie=116047b24a856493 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddaffd700 1 -- 172.21.15.73:0/2604298224 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000f510 con 0x7f5ddc064e20 2023-11-28T02:53:10.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddaffd700 1 -- 172.21.15.73:0/2604298224 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd000f670 con 0x7f5ddc064e20 2023-11-28T02:53:10.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.884+0000 7f5ddaffd700 1 -- 172.21.15.73:0/2604298224 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000f9f0 con 0x7f5ddc064e20 2023-11-28T02:53:10.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 -- 172.21.15.73:0/2604298224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 msgr2=0x7f5ddc155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 --2- 172.21.15.73:0/2604298224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc155750 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f5dd0009760 tx=0x7f5dd0009060).stop 2023-11-28T02:53:10.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 -- 172.21.15.73:0/2604298224 shutdown_connections 2023-11-28T02:53:10.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 --2- 172.21.15.73:0/2604298224 >> v2:172.21.15.150:3300/0 conn(0x7f5ddc095e80 0x7f5ddc096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 --2- 172.21.15.73:0/2604298224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc155750 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 -- 172.21.15.73:0/2604298224 >> 172.21.15.73:0/2604298224 conn(0x7f5ddc0631d0 msgr2=0x7f5ddc0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:10.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 -- 172.21.15.73:0/2604298224 shutdown_connections 2023-11-28T02:53:10.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.885+0000 7f5de2888700 1 -- 172.21.15.73:0/2604298224 wait complete. 2023-11-28T02:53:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.886+0000 7f5de2888700 1 Processor -- start 2023-11-28T02:53:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.886+0000 7f5de2888700 1 -- start start 2023-11-28T02:53:10.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc164510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc095e80 0x7f5ddc167370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0af230 con 0x7f5ddc1544d0 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc0a97b0 con 0x7f5ddc095e80 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de2888700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ddc167860 con 0x7f5ddc064e20 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35120/0 (socket says 172.21.15.73:35120) 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 -- 172.21.15.73:0/2799015122 learned_addr learned my addr 172.21.15.73:0/2799015122 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:10.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5ddb7fe700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc095e80 0x7f5ddc167370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5ddbfff700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc164510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 -- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 msgr2=0x7f5ddc164510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc164510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 -- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc095e80 msgr2=0x7f5ddc167370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc095e80 0x7f5ddc167370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.887+0000 7f5de0e25700 1 -- 172.21.15.73:0/2799015122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd4009ae0 con 0x7f5ddc1544d0 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5de0e25700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f5dd400c0c0 tx=0x7f5dd400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40092e0 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd400f040 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd00092a0 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd400cc00 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.888+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ddc0aef50 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.889+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ddc06c860 con 0x7f5ddc1544d0 2023-11-28T02:53:10.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.889+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5dd4007400 con 0x7f5ddc1544d0 2023-11-28T02:53:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.889+0000 7f5dc3fff700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 0x7f5dc8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.889+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5dd407a1c0 con 0x7f5ddc1544d0 2023-11-28T02:53:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.889+0000 7f5ddbfff700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 0x7f5dc8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.890+0000 7f5ddbfff700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 0x7f5dc8061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5dcc007fd0 tx=0x7f5dcc007d90).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:10.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:10.893+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5dd404aaf0 con 0x7f5ddc1544d0 2023-11-28T02:53:11.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.060+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ddc067b30 con 0x7f5ddc1544d0 2023-11-28T02:53:11.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.061+0000 7f5dc3fff700 1 -- 172.21.15.73:0/2799015122 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f5dd404b4e0 con 0x7f5ddc1544d0 2023-11-28T02:53:11.063 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:11.065 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 msgr2=0x7f5dc8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 0x7f5dc8061610 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7f5dcc007fd0 tx=0x7f5dcc007d90).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 msgr2=0x7f5ddc0ae200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 rx=0x7f5dd400c0c0 tx=0x7f5dd400b040).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 shutdown_connections 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5dc805f150 0x7f5dc8061610 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5ddc1544d0 0x7f5ddc0ae200 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5ddc064e20 0x7f5ddc164510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 --2- 172.21.15.73:0/2799015122 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5ddc095e80 0x7f5ddc167370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 >> 172.21.15.73:0/2799015122 conn(0x7f5ddc0631d0 msgr2=0x7f5ddc0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 shutdown_connections 2023-11-28T02:53:11.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.063+0000 7f5de2888700 1 -- 172.21.15.73:0/2799015122 wait complete. 2023-11-28T02:53:11.078 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:53:11.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 Processor -- start 2023-11-28T02:53:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 -- start start 2023-11-28T02:53:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd800c20c0 0x7fdd800c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd8006a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd801554d0 con 0x7fdd800bd680 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd8006ab10 con 0x7fdd800c20c0 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.195+0000 7fdd85080700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801548c0 con 0x7fdd800bdb30 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7e59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdd800bd680 0x7fdd80069f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47848/0 (socket says 172.21.15.73:47848) 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7e59c700 1 -- 172.21.15.73:0/1241557511 learned_addr learned my addr 172.21.15.73:0/1241557511 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2060184954 0 0) 0x7fdd801554d0 con 0x7fdd800bd680 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdd6c0033f0 con 0x7fdd800bd680 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1241557511 >> v2:172.21.15.150:3300/0 conn(0x7fdd800c20c0 0x7fdd800c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7f59e700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd8006a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3418538266 0 0) 0x7fdd6c0033f0 con 0x7fdd800bd680 2023-11-28T02:53:11.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 msgr2=0x7fdd8006a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd8006a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 >> v2:172.21.15.150:3300/0 conn(0x7fdd800c20c0 msgr2=0x7fdd800c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 --2- 172.21.15.73:0/1241557511 >> v2:172.21.15.150:3300/0 conn(0x7fdd800c20c0 0x7fdd800c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80154ba0 con 0x7fdd800bd680 2023-11-28T02:53:11.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fdd68001470 con 0x7fdd800bd680 2023-11-28T02:53:11.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 >> v1:172.21.15.73:6789/0 conn(0x7fdd800bd680 legacy=0x7fdd80069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd6c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c004690 0x7fdd6c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd6c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd8006ab10 con 0x7fdd6c0038e0 2023-11-28T02:53:11.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd801548c0 con 0x7fdd6c004690 2023-11-28T02:53:11.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd6c0078c0 con 0x7fdd800bdb30 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd6c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7f59e700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd6c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.196+0000 7fdd7e59c700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c004690 0x7fdd6c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 msgr2=0x7fdd6c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd6c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c004690 msgr2=0x7fdd6c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c004690 0x7fdd6c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1241557511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd6c0033f0 con 0x7fdd6c0038e0 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7e59c700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd6c004690 0x7fdd6c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7f59e700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd800bdb30 0x7fdd6c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd6c003d00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdd7000a2c0 tx=0x7fdd7000a890).ready entity=mon.0 client_cookie=8899257c5a75a180 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7000acc0 con 0x7fdd6c0038e0 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70010040 con 0x7fdd6c0038e0 2023-11-28T02:53:11.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7dd9b700 1 -- 172.21.15.73:0/1241557511 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70009dc0 con 0x7fdd6c0038e0 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 -- 172.21.15.73:0/1241557511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 msgr2=0x7fdd6c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 --2- 172.21.15.73:0/1241557511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd6c003d00 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 rx=0x7fdd7000a2c0 tx=0x7fdd7000a890).stop 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd7f59e700 1 -- 172.21.15.73:0/1241557511 reap_dead start 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 -- 172.21.15.73:0/1241557511 shutdown_connections 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 -- 172.21.15.73:0/1241557511 >> 172.21.15.73:0/1241557511 conn(0x7fdd80062700 msgr2=0x7fdd80062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 -- 172.21.15.73:0/1241557511 shutdown_connections 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.197+0000 7fdd85080700 1 -- 172.21.15.73:0/1241557511 wait complete. 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 Processor -- start 2023-11-28T02:53:11.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 -- start start 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd80093dc0 0x7fdd800bdf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd80153340 0x7fdd801537c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800aaf10 con 0x7fdd6c0038e0 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd80094240 con 0x7fdd80093dc0 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd85080700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd800aac10 con 0x7fdd80153340 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd7ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd7ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35134/0 (socket says 172.21.15.73:35134) 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.199+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1168969839 learned_addr learned my addr 172.21.15.73:0/1168969839 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:11.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7e59c700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd80093dc0 0x7fdd800bdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7f59e700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd80153340 0x7fdd801537c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd80153340 msgr2=0x7fdd801537c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd80153340 0x7fdd801537c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd80093dc0 msgr2=0x7fdd800bdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd80093dc0 0x7fdd800bdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 -- 172.21.15.73:0/1168969839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd80153cb0 con 0x7fdd6c0038e0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd7ed9d700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fdd70009a20 tx=0x7fdd70019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd70015d30 con 0x7fdd6c0038e0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd70015e90 con 0x7fdd6c0038e0 2023-11-28T02:53:11.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7001f670 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd800be440 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.200+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd800be9d0 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.201+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd800aaa70 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.201+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdd70003040 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.202+0000 7fdd76ffd700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 0x7fdd60061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.202+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdd7000ae40 con 0x7fdd6c0038e0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.202+0000 7fdd7e59c700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 0x7fdd60061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.202+0000 7fdd7e59c700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 0x7fdd60061610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fdd68001600 tx=0x7fdd68007040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.206+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd800aaa70 con 0x7fdd6c0038e0 2023-11-28T02:53:11.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.370+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fdd800aaa70 con 0x7fdd6c0038e0 2023-11-28T02:53:11.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.370+0000 7fdd76ffd700 1 -- 172.21.15.73:0/1168969839 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7fdd800aaa70 con 0x7fdd6c0038e0 2023-11-28T02:53:11.372 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 msgr2=0x7fdd60061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 0x7fdd60061610 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7fdd68001600 tx=0x7fdd68007040).stop 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 msgr2=0x7fdd80150ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 rx=0x7fdd70009a20 tx=0x7fdd70019000).stop 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 shutdown_connections 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6005f150 0x7fdd60061610 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd80153340 0x7fdd801537c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd6c0038e0 0x7fdd80150ee0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 --2- 172.21.15.73:0/1168969839 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd80093dc0 0x7fdd800bdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 >> 172.21.15.73:0/1168969839 conn(0x7fdd80062700 msgr2=0x7fdd8006a2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.372+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 shutdown_connections 2023-11-28T02:53:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.373+0000 7fdd85080700 1 -- 172.21.15.73:0/1168969839 wait complete. 2023-11-28T02:53:11.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:11.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.501+0000 7f8f7a2e6700 1 Processor -- start 2023-11-28T02:53:11.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.501+0000 7f8f7a2e6700 1 -- start start 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f7a2e6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8f74098d10 0x7f8f740c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f7a2e6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f740c72d0 con 0x7f8f740c6190 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f7a2e6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f74064570 con 0x7f8f74158320 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f7a2e6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f74154bc0 con 0x7f8f74098d10 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f737fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f740c6190 0x7f8f74063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47858/0 (socket says 172.21.15.73:47858) 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.502+0000 7f8f737fe700 1 -- 172.21.15.73:0/657984586 learned_addr learned my addr 172.21.15.73:0/657984586 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2397755747 0 0) 0x7f8f740c72d0 con 0x7f8f740c6190 2023-11-28T02:53:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 --2- 172.21.15.73:0/657984586 >> v2:172.21.15.150:3300/0 conn(0x7f8f74098d10 0x7f8f740c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f5c0033f0 con 0x7f8f740c6190 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 771189895 0 0) 0x7f8f5c0033f0 con 0x7f8f740c6190 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 >> v2:172.21.15.150:3300/0 conn(0x7f8f74098d10 msgr2=0x7f8f740c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 --2- 172.21.15.73:0/657984586 >> v2:172.21.15.150:3300/0 conn(0x7f8f74098d10 0x7f8f740c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 >> v1:172.21.15.150:6789/0 conn(0x7f8f74158320 legacy=0x7f8f740a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740c66c0 con 0x7f8f740c6190 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8f68001470 con 0x7f8f740c6190 2023-11-28T02:53:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 >> v1:172.21.15.73:6789/0 conn(0x7f8f740c6190 legacy=0x7f8f74063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f5c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f5c004690 0x7f8f5c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f5c005000 0x7f8f5c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f74154bc0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c00beb0 con 0x7f8f5c004690 2023-11-28T02:53:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c00c070 con 0x7f8f5c005000 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f5c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f737fe700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f5c004690 0x7f8f5c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f78883700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f5c005000 0x7f8f5c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 -- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f5c005000 msgr2=0x7f8f5c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f5c005000 0x7f8f5c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 -- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f5c004690 msgr2=0x7f8f5c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f5c004690 0x7f8f5c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f78883700 1 -- 172.21.15.73:0/657984586 reap_dead start 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 -- 172.21.15.73:0/657984586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0033f0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f737fe700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f5c004690 0x7f8f5c004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.503+0000 7f8f73fff700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f5c003d00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8f64009a80 tx=0x7f8f64009060).ready entity=mon.0 client_cookie=d2e90ca1b8d7f5a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009430 con 0x7f8f5c0038e0 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400e640 con 0x7f8f5c0038e0 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f72ffd700 1 -- 172.21.15.73:0/657984586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400e9c0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 msgr2=0x7f8f5c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f5c003d00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 rx=0x7f8f64009a80 tx=0x7f8f64009060).stop 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/657984586 shutdown_connections 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/657984586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f5c003d00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/657984586 >> 172.21.15.73:0/657984586 conn(0x7f8f740cb310 msgr2=0x7f8f740cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/657984586 shutdown_connections 2023-11-28T02:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.504+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/657984586 wait complete. 2023-11-28T02:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.505+0000 7f8f7a2e6700 1 Processor -- start 2023-11-28T02:53:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.505+0000 7f8f7a2e6700 1 -- start start 2023-11-28T02:53:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f740b0120 0x7f8f7414b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f74167e50 0x7f8f741682d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740ac810 con 0x7f8f5c0038e0 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f741687c0 con 0x7f8f74167e50 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f7a2e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f740ac010 con 0x7f8f740b0120 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f73fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f73fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35164/0 (socket says 172.21.15.73:35164) 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f73fff700 1 -- 172.21.15.73:0/603499516 learned_addr learned my addr 172.21.15.73:0/603499516 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f78883700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f74167e50 0x7f8f741682d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.506+0000 7f8f737fe700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f740b0120 0x7f8f7414b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 -- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f740b0120 msgr2=0x7f8f7414b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f740b0120 0x7f8f7414b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 -- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f74167e50 msgr2=0x7f8f741682d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f74167e50 0x7f8f741682d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 -- 172.21.15.73:0/603499516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f740b0620 con 0x7f8f5c0038e0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f73fff700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f8f6401bfd0 tx=0x7f8f6401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f64009ec0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f6400ed70 con 0x7f8f5c0038e0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f740b0900 con 0x7f8f5c0038e0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f6400eee0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.507+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f740ac4a0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.508+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f740c9ef0 con 0x7f8f5c0038e0 2023-11-28T02:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.508+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8f64024600 con 0x7f8f5c0038e0 2023-11-28T02:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.508+0000 7f8f5bfff700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 0x7f8f600615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.509+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8f6407b370 con 0x7f8f5c0038e0 2023-11-28T02:53:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.509+0000 7f8f737fe700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 0x7f8f600615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.509+0000 7f8f737fe700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 0x7f8f600615e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8f68001ce0 tx=0x7f8f68009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.513+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8f6404c690 con 0x7f8f5c0038e0 2023-11-28T02:53:11.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.677+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f8f7406c860 con 0x7f8f5c0038e0 2023-11-28T02:53:11.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.678+0000 7f8f5bfff700 1 -- 172.21.15.73:0/603499516 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7f8f64016070 con 0x7f8f5c0038e0 2023-11-28T02:53:11.679 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 msgr2=0x7f8f600615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 0x7f8f600615e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7f8f68001ce0 tx=0x7f8f68009040).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 msgr2=0x7f8f740aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 rx=0x7f8f6401bfd0 tx=0x7f8f6401a000).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 shutdown_connections 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f6005f120 0x7f8f600615e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f74167e50 0x7f8f741682d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f5c0038e0 0x7f8f740aa600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 --2- 172.21.15.73:0/603499516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f740b0120 0x7f8f7414b2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 >> 172.21.15.73:0/603499516 conn(0x7f8f740cb310 msgr2=0x7f8f7414c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.679+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 shutdown_connections 2023-11-28T02:53:11.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.680+0000 7f8f7a2e6700 1 -- 172.21.15.73:0/603499516 wait complete. 2023-11-28T02:53:11.692 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:53:11.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.806+0000 7fe993de0700 1 Processor -- start 2023-11-28T02:53:11.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.806+0000 7fe993de0700 1 -- start start 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe98c157c10 0x7fe98c0a8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe98c0c0250 0x7fe98c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c0a8570 0x7fe98c0ca3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0c12f0 con 0x7fe98c157c10 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe98c0c06e0 con 0x7fe98c0c0250 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe993de0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0c09c0 con 0x7fe98c0a8570 2023-11-28T02:53:11.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.807+0000 7fe991b7c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe98c157c10 0x7fe98c0a8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe98c157c10 0x7fe98c0a8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35172/0 (socket says 172.21.15.73:35172) 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 -- 172.21.15.73:0/4041563467 learned_addr learned my addr 172.21.15.73:0/4041563467 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe99237d700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c0a8570 0x7fe98c0ca3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe99137b700 1 --2- 172.21.15.73:0/4041563467 >> v2:172.21.15.150:3300/0 conn(0x7fe98c0c0250 0x7fe98c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 -- 172.21.15.73:0/4041563467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c0a8570 msgr2=0x7fe98c0ca3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c0a8570 0x7fe98c0ca3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 -- 172.21.15.73:0/4041563467 >> v2:172.21.15.150:3300/0 conn(0x7fe98c0c0250 msgr2=0x7fe98c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 --2- 172.21.15.73:0/4041563467 >> v2:172.21.15.150:3300/0 conn(0x7fe98c0c0250 0x7fe98c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 -- 172.21.15.73:0/4041563467 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe98c0c0ca0 con 0x7fe98c157c10 2023-11-28T02:53:11.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe991b7c700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0a8080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe98c083630 tx=0x7fe988009840).ready entity=mon.0 client_cookie=24f85f42488af84c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe990b7a700 1 -- 172.21.15.73:0/4041563467 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800e070 con 0x7fe98c157c10 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe990b7a700 1 -- 172.21.15.73:0/4041563467 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800bad0 con 0x7fe98c157c10 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.808+0000 7fe990b7a700 1 -- 172.21.15.73:0/4041563467 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800be50 con 0x7fe98c157c10 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 -- 172.21.15.73:0/4041563467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 msgr2=0x7fe98c0a8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0a8080 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fe98c083630 tx=0x7fe988009840).stop 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 -- 172.21.15.73:0/4041563467 shutdown_connections 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0a8080 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 --2- 172.21.15.73:0/4041563467 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c0a8570 0x7fe98c0ca3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 --2- 172.21.15.73:0/4041563467 >> v2:172.21.15.150:3300/0 conn(0x7fe98c0c0250 0x7fe98c0c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 -- 172.21.15.73:0/4041563467 >> 172.21.15.73:0/4041563467 conn(0x7fe98c06a610 msgr2=0x7fe98c0cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 -- 172.21.15.73:0/4041563467 shutdown_connections 2023-11-28T02:53:11.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.809+0000 7fe993de0700 1 -- 172.21.15.73:0/4041563467 wait complete. 2023-11-28T02:53:11.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.810+0000 7fe993de0700 1 Processor -- start 2023-11-28T02:53:11.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.810+0000 7fe993de0700 1 -- start start 2023-11-28T02:53:11.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.810+0000 7fe993de0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.810+0000 7fe993de0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe98c0ac9e0 0x7fe98c151150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c152320 0x7fe98c1527a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c151c10 con 0x7fe98c157c10 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c0c3f70 con 0x7fe98c0ac9e0 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe98c152c90 con 0x7fe98c152320 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35186/0 (socket says 172.21.15.73:35186) 2023-11-28T02:53:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 -- 172.21.15.73:0/2834959115 learned_addr learned my addr 172.21.15.73:0/2834959115 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe99137b700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe98c0ac9e0 0x7fe98c151150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe99237d700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c152320 0x7fe98c1527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 -- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c152320 msgr2=0x7fe98c1527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c152320 0x7fe98c1527a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 -- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe98c0ac9e0 msgr2=0x7fe98c151150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe98c0ac9e0 0x7fe98c151150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 -- 172.21.15.73:0/2834959115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe988009ae0 con 0x7fe98c157c10 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe991b7c700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe9880011d0 tx=0x7fe988009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98800b7a0 con 0x7fe98c157c10 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe98800bc40 con 0x7fe98c157c10 2023-11-28T02:53:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe98c151640 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe98801e6a0 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.811+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe98c0c3640 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.812+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe98c0c4250 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.813+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe988003040 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.813+0000 7fe981ffb700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 0x7fe970061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.813+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe98807b620 con 0x7fe98c157c10 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.813+0000 7fe99137b700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 0x7fe970061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.813+0000 7fe99137b700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 0x7fe970061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe98c0aacc0 tx=0x7fe97c008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:11.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.817+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe98804c8f0 con 0x7fe98c157c10 2023-11-28T02:53:11.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.978+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe98c06c860 con 0x7fe98c157c10 2023-11-28T02:53:11.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.979+0000 7fe981ffb700 1 -- 172.21.15.73:0/2834959115 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe988023020 con 0x7fe98c157c10 2023-11-28T02:53:11.980 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:11.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 msgr2=0x7fe970061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 0x7fe970061610 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 rx=0x7fe98c0aacc0 tx=0x7fe97c008040).stop 2023-11-28T02:53:11.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 msgr2=0x7fe98c0c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:11.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7fe9880011d0 tx=0x7fe988009840).stop 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 shutdown_connections 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe97005f150 0x7fe970061610 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe98c152320 0x7fe98c1527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe98c157c10 0x7fe98c0c2d50 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 --2- 172.21.15.73:0/2834959115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe98c0ac9e0 0x7fe98c151150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 >> 172.21.15.73:0/2834959115 conn(0x7fe98c06a610 msgr2=0x7fe98c153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 shutdown_connections 2023-11-28T02:53:11.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:11.981+0000 7fe993de0700 1 -- 172.21.15.73:0/2834959115 wait complete. 2023-11-28T02:53:11.994 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:53:12.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.108+0000 7fb2d9b92700 1 Processor -- start 2023-11-28T02:53:12.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.108+0000 7fb2d9b92700 1 -- start start 2023-11-28T02:53:12.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d9b92700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb2d4069d10 0x7fb2d406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d9b92700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d40bd660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d9b92700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d40c3470 con 0x7fb2d4151a20 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d9b92700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d40c48f0 con 0x7fb2d4069d10 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d9b92700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d40bdb50 con 0x7fb2d40bd1f0 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d2ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb2d4151a20 0x7fb2d40be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47868/0 (socket says 172.21.15.73:47868) 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d2ffd700 1 -- 172.21.15.73:0/25436264 learned_addr learned my addr 172.21.15.73:0/25436264 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d3fff700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d40bd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/25436264 >> v2:172.21.15.150:3300/0 conn(0x7fb2d4069d10 0x7fb2d406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2614732288 0 0) 0x7fb2d40c3470 con 0x7fb2d4151a20 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.109+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2bc0033f0 con 0x7fb2d4151a20 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d37fe700 1 -- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 msgr2=0x7fb2d40bd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d40bd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2143793516 0 0) 0x7fb2bc0033f0 con 0x7fb2d4151a20 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d37fe700 1 -- 172.21.15.73:0/25436264 >> v1:172.21.15.73:6789/0 conn(0x7fb2d4151a20 legacy=0x7fb2d40be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d37fe700 1 -- 172.21.15.73:0/25436264 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d4151ea0 con 0x7fb2d4069d10 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d3fff700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d40bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d406a0f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb2c4009e60 tx=0x7fb2c4009780).ready entity=mon.1 client_cookie=b0a2cd715e3339af server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c400e070 con 0x7fb2d4069d10 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c400b7a0 con 0x7fb2d4069d10 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d27fc700 1 -- 172.21.15.73:0/25436264 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c8001470 con 0x7fb2d4069d10 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d9b92700 1 -- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 msgr2=0x7fb2d406a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d406a0f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 rx=0x7fb2c4009e60 tx=0x7fb2c4009780).stop 2023-11-28T02:53:12.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d9b92700 1 -- 172.21.15.73:0/25436264 shutdown_connections 2023-11-28T02:53:12.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d40bd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.110+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/25436264 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d406a0f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.111+0000 7fb2d9b92700 1 -- 172.21.15.73:0/25436264 >> 172.21.15.73:0/25436264 conn(0x7fb2d40642d0 msgr2=0x7fb2d40c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:12.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.111+0000 7fb2d9b92700 1 -- 172.21.15.73:0/25436264 shutdown_connections 2023-11-28T02:53:12.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.111+0000 7fb2d9b92700 1 -- 172.21.15.73:0/25436264 wait complete. 2023-11-28T02:53:12.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.112+0000 7fb2d9b92700 1 Processor -- start 2023-11-28T02:53:12.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.112+0000 7fb2d9b92700 1 -- start start 2023-11-28T02:53:12.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.112+0000 7fb2d9b92700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d40a5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d4164d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4168660 con 0x7fb2d4151a20 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4168310 con 0x7fb2d4069d10 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d4165590 con 0x7fb2d40bd1f0 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35192/0 (socket says 172.21.15.73:35192) 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 -- 172.21.15.73:0/1882663297 learned_addr learned my addr 172.21.15.73:0/1882663297 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d2ffd700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d4164d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d40a5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 -- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 msgr2=0x7fb2d4164d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d4164d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 -- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 msgr2=0x7fb2d40a5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d40a5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 -- 172.21.15.73:0/1882663297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2cc0092a0 con 0x7fb2d4151a20 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d3fff700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb2cc00e900 tx=0x7fb2cc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc012600 con 0x7fb2d4151a20 2023-11-28T02:53:12.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2cc012760 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2c4009a20 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2cc012b00 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.113+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d4165b50 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.114+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d4168810 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.114+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb2cc01b400 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.115+0000 7fb2bb7fe700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 0x7fb2c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.115+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb2cc07a630 con 0x7fb2d4151a20 2023-11-28T02:53:12.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.115+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 0x7fb2c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.115+0000 7fb2d37fe700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 0x7fb2c0061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fb2c40099f0 tx=0x7fb2c4009780).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.119+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2d4168810 con 0x7fb2d4151a20 2023-11-28T02:53:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.280+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2d406c860 con 0x7fb2d4151a20 2023-11-28T02:53:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.280+0000 7fb2bb7fe700 1 -- 172.21.15.73:0/1882663297 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2cc016020 con 0x7fb2d4151a20 2023-11-28T02:53:12.282 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 msgr2=0x7fb2c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 0x7fb2c0061610 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 rx=0x7fb2c40099f0 tx=0x7fb2c4009780).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 msgr2=0x7fb2d414b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 rx=0x7fb2cc00e900 tx=0x7fb2cc009060).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 shutdown_connections 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2c005f150 0x7fb2c0061610 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2d4151a20 0x7fb2d414b620 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2d4069d10 0x7fb2d40a5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 --2- 172.21.15.73:0/1882663297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2d40bd1f0 0x7fb2d4164d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 >> 172.21.15.73:0/1882663297 conn(0x7fb2d40642d0 msgr2=0x7fb2d40c6390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:12.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 shutdown_connections 2023-11-28T02:53:12.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.282+0000 7fb2d9b92700 1 -- 172.21.15.73:0/1882663297 wait complete. 2023-11-28T02:53:12.308 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-11-28T02:53:12.308 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T02:53:12.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-11-28T02:53:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.420+0000 7f91cc642700 1 Processor -- start 2023-11-28T02:53:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.420+0000 7f91cc642700 1 -- start start 2023-11-28T02:53:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91cc642700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91c41544e0 0x7f91c41548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91cc642700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c40c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91cc642700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91c40c69a0 con 0x7f91c406a610 2023-11-28T02:53:12.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91cc642700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91c40c13a0 con 0x7f91c41544e0 2023-11-28T02:53:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91cc642700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40c1680 con 0x7f91c40c6e30 2023-11-28T02:53:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91ca3de700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91c41544e0 0x7f91c41548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91c9bdd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f91c406a610 0x7f91c40c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47874/0 (socket says 172.21.15.73:47874) 2023-11-28T02:53:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.421+0000 7f91c9bdd700 1 -- 172.21.15.73:0/1069657894 learned_addr learned my addr 172.21.15.73:0/1069657894 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:12.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91ca3de700 1 -- 172.21.15.73:0/1069657894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 msgr2=0x7f91c40c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:12.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cabdf700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c40c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91c93dc700 1 -- 172.21.15.73:0/1069657894 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904426052 0 0) 0x7f91c40c69a0 con 0x7f91c406a610 2023-11-28T02:53:12.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91ca3de700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c40c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91ca3de700 1 -- 172.21.15.73:0/1069657894 >> v1:172.21.15.73:6789/0 conn(0x7f91c406a610 legacy=0x7f91c40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91ca3de700 1 -- 172.21.15.73:0/1069657894 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c40c6630 con 0x7f91c41544e0 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cabdf700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c40c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91ca3de700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c41548c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f91c0006fd0 tx=0x7f91c0006f40).ready entity=mon.0 client_cookie=30854fbabe567220 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91c93dc700 1 -- 172.21.15.73:0/1069657894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0005880 con 0x7f91c41544e0 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91c93dc700 1 -- 172.21.15.73:0/1069657894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91c00059e0 con 0x7f91c41544e0 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91c93dc700 1 -- 172.21.15.73:0/1069657894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91c0005d60 con 0x7f91c41544e0 2023-11-28T02:53:12.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cc642700 1 -- 172.21.15.73:0/1069657894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 msgr2=0x7f91c41548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cc642700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c41548c0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 rx=0x7f91c0006fd0 tx=0x7f91c0006f40).stop 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cc642700 1 -- 172.21.15.73:0/1069657894 shutdown_connections 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cc642700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c40c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.422+0000 7f91cc642700 1 --2- 172.21.15.73:0/1069657894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c41548c0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.423+0000 7f91cc642700 1 -- 172.21.15.73:0/1069657894 >> 172.21.15.73:0/1069657894 conn(0x7f91c4085020 msgr2=0x7f91c4094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.423+0000 7f91cc642700 1 -- 172.21.15.73:0/1069657894 shutdown_connections 2023-11-28T02:53:12.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.423+0000 7f91cc642700 1 -- 172.21.15.73:0/1069657894 wait complete. 2023-11-28T02:53:12.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 Processor -- start 2023-11-28T02:53:12.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 -- start start 2023-11-28T02:53:12.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 0x7f91c40ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c4098580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4096070 con 0x7f91c41544e0 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91c40979a0 con 0x7f91c406a610 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.424+0000 7f91cc642700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91c4095460 con 0x7f91c40c6e30 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35208/0 (socket says 172.21.15.73:35208) 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 -- 172.21.15.73:0/2411881958 learned_addr learned my addr 172.21.15.73:0/2411881958 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91c9bdd700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c4098580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91ca3de700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 0x7f91c40ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 -- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 msgr2=0x7f91c4098580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c4098580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 -- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 msgr2=0x7f91c40ad670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:12.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 0x7f91c40ad670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:12.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 -- 172.21.15.73:0/2411881958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91c0009250 con 0x7f91c41544e0 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91ca3de700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 0x7f91c40ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cabdf700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f91bc00cef0 tx=0x7f91bc007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc00ae00 con 0x7f91c41544e0 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91bc016400 con 0x7f91c41544e0 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91c4095740 con 0x7f91c41544e0 2023-11-28T02:53:12.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91bc016780 con 0x7f91c41544e0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.425+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91c4095d00 con 0x7f91c41544e0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.426+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91c4096220 con 0x7f91c41544e0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.426+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f91bc01f3f0 con 0x7f91c41544e0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.427+0000 7f91ba7fc700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 0x7f91a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.427+0000 7f91ca3de700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 0x7f91a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.427+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f91bc07a940 con 0x7f91c41544e0 2023-11-28T02:53:12.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.427+0000 7f91ca3de700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 0x7f91a8061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f91c0009fd0 tx=0x7f91c0016000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:12.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.431+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f91c4096220 con 0x7f91c41544e0 2023-11-28T02:53:12.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:12.662+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f91c40961b0 con 0x7f91c41544e0 2023-11-28T02:53:13.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.537+0000 7f91ba7fc700 1 -- 172.21.15.73:0/2411881958 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v46) v1 ==== 161+0+0 (secure 0 0 0) 0x7f91bc04bbb0 con 0x7f91c41544e0 2023-11-28T02:53:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 msgr2=0x7f91a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 0x7f91a8061610 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 rx=0x7f91c0009fd0 tx=0x7f91c0016000).stop 2023-11-28T02:53:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 msgr2=0x7f91c4096500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 rx=0x7f91bc00cef0 tx=0x7f91bc007860).stop 2023-11-28T02:53:13.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 shutdown_connections 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f91a805f150 0x7f91a8061610 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91c41544e0 0x7f91c4096500 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91c406a610 0x7f91c40ad670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 --2- 172.21.15.73:0/2411881958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91c40c6e30 0x7f91c4098580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 >> 172.21.15.73:0/2411881958 conn(0x7f91c4085020 msgr2=0x7f91c40c9890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 shutdown_connections 2023-11-28T02:53:13.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.539+0000 7f91cc642700 1 -- 172.21.15.73:0/2411881958 wait complete. 2023-11-28T02:53:13.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T02:53:13.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.671+0000 7f38f2482700 1 Processor -- start 2023-11-28T02:53:13.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.671+0000 7f38f2482700 1 -- start start 2023-11-28T02:53:13.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f2482700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38ec0c66f0 0x7f38ec154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f2482700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec158570 0x7f38ec158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f2482700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec154e40 con 0x7f38ec14e530 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f2482700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0c6ba0 con 0x7f38ec0c66f0 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f2482700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0c7bc0 con 0x7f38ec158570 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f1480700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38ec14e530 0x7f38ec14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54796/0 (socket says 172.21.15.73:54796) 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38ec0c66f0 0x7f38ec154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f1480700 1 -- 172.21.15.73:0/777826020 learned_addr learned my addr 172.21.15.73:0/777826020 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f1c81700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec158570 0x7f38ec158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 -- 172.21.15.73:0/777826020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec158570 msgr2=0x7f38ec158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec158570 0x7f38ec158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 -- 172.21.15.73:0/777826020 >> v1:172.21.15.73:6789/0 conn(0x7f38ec14e530 legacy=0x7f38ec14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38e3fff700 1 -- 172.21.15.73:0/777826020 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1102683486 0 0) 0x7f38e8001110 con 0x7f38ec14e530 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 -- 172.21.15.73:0/777826020 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec155920 con 0x7f38ec0c66f0 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38f0c7f700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec0c66f0 0x7f38ec154950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f38dc009760 tx=0x7f38dc009060).ready entity=mon.0 client_cookie=e36e6d4fbfdf02e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.672+0000 7f38e3fff700 1 -- 172.21.15.73:0/777826020 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00b580 con 0x7f38ec0c66f0 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38e3fff700 1 -- 172.21.15.73:0/777826020 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38dc00b6e0 con 0x7f38ec0c66f0 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38e3fff700 1 -- 172.21.15.73:0/777826020 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38dc00ba60 con 0x7f38ec0c66f0 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 -- 172.21.15.73:0/777826020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec0c66f0 msgr2=0x7f38ec154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec0c66f0 0x7f38ec154950 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 rx=0x7f38dc009760 tx=0x7f38dc009060).stop 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 -- 172.21.15.73:0/777826020 shutdown_connections 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec158570 0x7f38ec158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 --2- 172.21.15.73:0/777826020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec0c66f0 0x7f38ec154950 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 -- 172.21.15.73:0/777826020 >> 172.21.15.73:0/777826020 conn(0x7f38ec064990 msgr2=0x7f38ec064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:13.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 -- 172.21.15.73:0/777826020 shutdown_connections 2023-11-28T02:53:13.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.673+0000 7f38f2482700 1 -- 172.21.15.73:0/777826020 wait complete. 2023-11-28T02:53:13.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.674+0000 7f38f2482700 1 Processor -- start 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.674+0000 7f38f2482700 1 -- start start 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec0c66f0 0x7f38ec0c2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 0x7f38ec0c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0afab0 con 0x7f38ec158570 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0af180 con 0x7f38ec14e530 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f2482700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0af460 con 0x7f38ec0c66f0 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f1c81700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f1c81700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51944/0 (socket says 172.21.15.73:51944) 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f1c81700 1 -- 172.21.15.73:0/4027188646 learned_addr learned my addr 172.21.15.73:0/4027188646 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:13.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.675+0000 7f38f1480700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec0c66f0 0x7f38ec0c2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f0c7f700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 0x7f38ec0c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 -- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec0c66f0 msgr2=0x7f38ec0c2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec0c66f0 0x7f38ec0c2fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 -- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 msgr2=0x7f38ec0c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 0x7f38ec0c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 -- 172.21.15.73:0/4027188646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38dc0092a0 con 0x7f38ec158570 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f0c7f700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 0x7f38ec0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f1c81700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f38e4007f20 tx=0x7f38e4009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e400eca0 con 0x7f38ec158570 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e400ee00 con 0x7f38ec158570 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec0af740 con 0x7f38ec158570 2023-11-28T02:53:13.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e4014610 con 0x7f38ec158570 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.676+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec14ded0 con 0x7f38ec158570 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.677+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec14e150 con 0x7f38ec158570 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.677+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f38e4014980 con 0x7f38ec158570 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.677+0000 7f38e17fa700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 0x7f38d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.678+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f38e407a7e0 con 0x7f38ec158570 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.678+0000 7f38f1480700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 0x7f38d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.678+0000 7f38f1480700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 0x7f38d0061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f38e8001720 tx=0x7f38e8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:13.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.681+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f38e404b9f0 con 0x7f38ec158570 2023-11-28T02:53:13.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.844+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f38ec062ee0 con 0x7f38ec158570 2023-11-28T02:53:13.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.847+0000 7f38e17fa700 1 -- 172.21.15.73:0/4027188646 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v2)=0 v2) v1 ==== 122+0+0 (secure 0 0 0) 0x7f38e404f010 con 0x7f38ec158570 2023-11-28T02:53:13.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 msgr2=0x7f38d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 0x7f38d0061610 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 rx=0x7f38e8001720 tx=0x7f38e8009040).stop 2023-11-28T02:53:13.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 msgr2=0x7f38ec14d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:13.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 rx=0x7f38e4007f20 tx=0x7f38e4009860).stop 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 shutdown_connections 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f38d005f150 0x7f38d0061610 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38ec158570 0x7f38ec14d9e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38ec0c66f0 0x7f38ec0c2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 --2- 172.21.15.73:0/4027188646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38ec14e530 0x7f38ec0c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:13.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 >> 172.21.15.73:0/4027188646 conn(0x7f38ec064990 msgr2=0x7f38ec156c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:13.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 shutdown_connections 2023-11-28T02:53:13.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:13.849+0000 7f38f2482700 1 -- 172.21.15.73:0/4027188646 wait complete. 2023-11-28T02:53:13.862 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T02:53:13.863 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T02:53:13.902 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T02:53:13.951 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T02:53:13.951 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T02:53:13.951 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:13.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-11-28T02:53:13.990 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:53:13.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:13.990 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T02:53:14.045 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-11-28T02:53:14.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:14.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-11-28T02:53:14.087 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:53:14.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:14.087 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-11-28T02:53:14.143 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-11-28T02:53:14.143 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:14.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-11-28T02:53:14.182 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T02:53:14.182 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:14.182 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-11-28T02:53:14.237 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-11-28T02:53:14.237 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:14.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-11-28T02:53:14.274 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T02:53:14.274 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:14.274 DEBUG:teuthology.orchestra.run.smithi150:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-11-28T02:53:14.330 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T02:53:14.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.441+0000 7f1ef942c700 1 Processor -- start 2023-11-28T02:53:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.441+0000 7f1ef942c700 1 -- start start 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef942c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 0x7f1ef4064f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef942c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 0x7f1ef40c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef942c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef4066240 con 0x7f1ef40c5df0 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef942c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef4065440 con 0x7f1ef414ada0 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef942c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef40666d0 con 0x7f1ef4151880 2023-11-28T02:53:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef2ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 0x7f1ef4064f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1eea7fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 0x7f1ef40c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef37fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1ef40c5df0 0x7f1ef40c2f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:47170/0 (socket says 172.21.15.73:47170) 2023-11-28T02:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.442+0000 7f1ef37fe700 1 -- 172.21.15.73:0/454558986 learned_addr learned my addr 172.21.15.73:0/454558986 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1eea7fc700 1 -- 172.21.15.73:0/454558986 >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 msgr2=0x7f1ef4064f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:14.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1eea7fc700 1 --2- 172.21.15.73:0/454558986 >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 0x7f1ef4064f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1eea7fc700 1 -- 172.21.15.73:0/454558986 >> v1:172.21.15.150:6789/0 conn(0x7f1ef40c5df0 legacy=0x7f1ef40c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1eea7fc700 1 -- 172.21.15.73:0/454558986 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef40c6280 con 0x7f1ef4151880 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef2ffd700 1 --2- 172.21.15.73:0/454558986 >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 0x7f1ef4064f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1eea7fc700 1 --2- 172.21.15.73:0/454558986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 0x7f1ef40c5900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f1edc009760 tx=0x7f1edc009060).ready entity=mon.2 client_cookie=22ca0cff730ee5c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef27fc700 1 -- 172.21.15.73:0/454558986 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00f510 con 0x7f1ef4151880 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef27fc700 1 -- 172.21.15.73:0/454558986 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00f670 con 0x7f1ef4151880 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef27fc700 1 -- 172.21.15.73:0/454558986 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00f9f0 con 0x7f1ef4151880 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef942c700 1 -- 172.21.15.73:0/454558986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 msgr2=0x7f1ef40c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.443+0000 7f1ef942c700 1 --2- 172.21.15.73:0/454558986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 0x7f1ef40c5900 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f1edc009760 tx=0x7f1edc009060).stop 2023-11-28T02:53:14.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 -- 172.21.15.73:0/454558986 shutdown_connections 2023-11-28T02:53:14.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 --2- 172.21.15.73:0/454558986 >> v2:172.21.15.150:3300/0 conn(0x7f1ef414ada0 0x7f1ef4064f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 --2- 172.21.15.73:0/454558986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef4151880 0x7f1ef40c5900 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 -- 172.21.15.73:0/454558986 >> 172.21.15.73:0/454558986 conn(0x7f1ef40c3f20 msgr2=0x7f1ef40c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:14.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 -- 172.21.15.73:0/454558986 shutdown_connections 2023-11-28T02:53:14.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.444+0000 7f1ef942c700 1 -- 172.21.15.73:0/454558986 wait complete. 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.445+0000 7f1ef942c700 1 Processor -- start 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.445+0000 7f1ef942c700 1 -- start start 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef40c5df0 0x7f1ef40bf9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef4151880 0x7f1ef414d880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4164480 con 0x7f1ef414ada0 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4163800 con 0x7f1ef4151880 2023-11-28T02:53:14.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef4163ae0 con 0x7f1ef40c5df0 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51946/0 (socket says 172.21.15.73:51946) 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 -- 172.21.15.73:0/2685016877 learned_addr learned my addr 172.21.15.73:0/2685016877 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef2ffd700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef40c5df0 0x7f1ef40bf9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef37fe700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef4151880 0x7f1ef414d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 -- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef40c5df0 msgr2=0x7f1ef40bf9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef40c5df0 0x7f1ef40bf9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 -- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef4151880 msgr2=0x7f1ef414d880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef4151880 0x7f1ef414d880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 -- 172.21.15.73:0/2685016877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40092a0 con 0x7f1ef414ada0 2023-11-28T02:53:14.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eea7fc700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f1edc009e00 tx=0x7f1edc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc00f510 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1edc00f7e0 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1edc0092a0 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1edc01b410 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.446+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef41640a0 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.447+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ef406c7c0 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.448+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1edc01b780 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.448+0000 7f1eeb7fe700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 0x7f1ed4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.448+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1edc07b900 con 0x7f1ef414ada0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.448+0000 7f1ef2ffd700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 0x7f1ed4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.448+0000 7f1ef2ffd700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 0x7f1ed4061640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1ee4009dc0 tx=0x7f1ee4009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.452+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1edc04cbd0 con 0x7f1ef414ada0 2023-11-28T02:53:14.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.658+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1ef4064b00 con 0x7f1ef414ada0 2023-11-28T02:53:14.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.659+0000 7f1eeb7fe700 1 -- 172.21.15.73:0/2685016877 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f1edc01f020 con 0x7f1ef414ada0 2023-11-28T02:53:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:14.661 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:53:14.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 msgr2=0x7f1ed4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 0x7f1ed4061640 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 rx=0x7f1ee4009dc0 tx=0x7f1ee4009060).stop 2023-11-28T02:53:14.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 msgr2=0x7f1ef414a150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 rx=0x7f1edc009e00 tx=0x7f1edc017000).stop 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 shutdown_connections 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed405f180 0x7f1ed4061640 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef4151880 0x7f1ef414d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef40c5df0 0x7f1ef40bf9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 --2- 172.21.15.73:0/2685016877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef414ada0 0x7f1ef414a150 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 >> 172.21.15.73:0/2685016877 conn(0x7f1ef40c3f20 msgr2=0x7f1ef40c3280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 shutdown_connections 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.661+0000 7f1ef942c700 1 -- 172.21.15.73:0/2685016877 wait complete. 2023-11-28T02:53:14.663 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:53:14.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:14.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.786+0000 7fdf2f220700 1 Processor -- start 2023-11-28T02:53:14.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.786+0000 7fdf2f220700 1 -- start start 2023-11-28T02:53:14.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2f220700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2f220700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf28155510 0x7fdf280694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2f220700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf28069e60 con 0x7fdf280699d0 2023-11-28T02:53:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2f220700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf28066e60 con 0x7fdf28155510 2023-11-28T02:53:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2f220700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28067140 con 0x7fdf2813a8d0 2023-11-28T02:53:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2da1d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf28155510 0x7fdf280694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2e21e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2ea1f700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fdf280699d0 0x7fdf280c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:47182/0 (socket says 172.21.15.73:47182) 2023-11-28T02:53:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2da1d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf28155510 0x7fdf280694e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:35250/0 (socket says 172.21.15.73:35250) 2023-11-28T02:53:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2e21e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:32850/0 (socket says 172.21.15.73:32850) 2023-11-28T02:53:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.787+0000 7fdf2ea1f700 1 -- 172.21.15.73:0/3646687234 learned_addr learned my addr 172.21.15.73:0/3646687234 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2da1d700 1 -- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 msgr2=0x7fdf28066880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2da1d700 1 -- 172.21.15.73:0/3646687234 >> v1:172.21.15.150:6789/0 conn(0x7fdf280699d0 legacy=0x7fdf280c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2d21c700 1 -- 172.21.15.73:0/3646687234 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390919396 0 0) 0x7fdf28069e60 con 0x7fdf280699d0 2023-11-28T02:53:14.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2da1d700 1 -- 172.21.15.73:0/3646687234 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf28067420 con 0x7fdf28155510 2023-11-28T02:53:14.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2e21e700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:14.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf280694e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fdf18009760 tx=0x7fdf18009060).ready entity=mon.1 client_cookie=9ab2c0ab3045de4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2d21c700 1 -- 172.21.15.73:0/3646687234 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800f510 con 0x7fdf28155510 2023-11-28T02:53:14.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2d21c700 1 -- 172.21.15.73:0/3646687234 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf1800f670 con 0x7fdf28155510 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2d21c700 1 -- 172.21.15.73:0/3646687234 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800f9f0 con 0x7fdf28155510 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 -- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 msgr2=0x7fdf280694e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf280694e0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 rx=0x7fdf18009760 tx=0x7fdf18009060).stop 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 -- 172.21.15.73:0/3646687234 shutdown_connections 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf28066880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 --2- 172.21.15.73:0/3646687234 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf280694e0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 -- 172.21.15.73:0/3646687234 >> 172.21.15.73:0/3646687234 conn(0x7fdf280c7ca0 msgr2=0x7fdf280c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 -- 172.21.15.73:0/3646687234 shutdown_connections 2023-11-28T02:53:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.788+0000 7fdf2f220700 1 -- 172.21.15.73:0/3646687234 wait complete. 2023-11-28T02:53:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 Processor -- start 2023-11-28T02:53:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 -- start start 2023-11-28T02:53:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf280a0ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf28164a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28153140 con 0x7fdf280699d0 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28152530 con 0x7fdf28155510 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2f220700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf28152810 con 0x7fdf2813a8d0 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2e21e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2e21e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51958/0 (socket says 172.21.15.73:51958) 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.790+0000 7fdf2e21e700 1 -- 172.21.15.73:0/218715366 learned_addr learned my addr 172.21.15.73:0/218715366 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf280a0ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2ea1f700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf28164a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 -- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 msgr2=0x7fdf280a0ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf280a0ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 -- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 msgr2=0x7fdf28164a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf28164a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 -- 172.21.15.73:0/218715366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf2400b540 con 0x7fdf280699d0 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2ea1f700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf28164a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf280a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2e21e700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fdf2400ede0 tx=0x7fdf2400b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf2400be50 con 0x7fdf280699d0 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf2400dde0 con 0x7fdf280699d0 2023-11-28T02:53:14.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf180092a0 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf24010650 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.791+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf28152dd0 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.792+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf280c6db0 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.792+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdf240109c0 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.793+0000 7fdf1e7fc700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 0x7fdf0c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.793+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf2407acc0 con 0x7fdf280699d0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.793+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 0x7fdf0c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.793+0000 7fdf2da1d700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 0x7fdf0c061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fdf18009760 tx=0x7fdf18017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:14.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.797+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdf2404bf90 con 0x7fdf280699d0 2023-11-28T02:53:14.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.959+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf2806c7c0 con 0x7fdf280699d0 2023-11-28T02:53:14.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.960+0000 7fdf1e7fc700 1 -- 172.21.15.73:0/218715366 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fdf24014020 con 0x7fdf280699d0 2023-11-28T02:53:14.962 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:14.962 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 msgr2=0x7fdf0c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 0x7fdf0c061610 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 rx=0x7fdf18009760 tx=0x7fdf18017000).stop 2023-11-28T02:53:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 msgr2=0x7fdf28139860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 rx=0x7fdf2400ede0 tx=0x7fdf2400b300).stop 2023-11-28T02:53:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 shutdown_connections 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf0c05f150 0x7fdf0c061610 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf28155510 0x7fdf28164a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf280699d0 0x7fdf28139860 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 --2- 172.21.15.73:0/218715366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf2813a8d0 0x7fdf280a0ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 >> 172.21.15.73:0/218715366 conn(0x7fdf280c7ca0 msgr2=0x7fdf280bcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:14.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 shutdown_connections 2023-11-28T02:53:14.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:14.961+0000 7fdf2f220700 1 -- 172.21.15.73:0/218715366 wait complete. 2023-11-28T02:53:14.975 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:52:42.251100+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:52:43.421590+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T02:53:14.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:15.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.093+0000 7f2171e4b700 1 Processor -- start 2023-11-28T02:53:15.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.093+0000 7f2171e4b700 1 -- start start 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f2171e4b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f216c158a30 0x7f216c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f2171e4b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0c72d0 con 0x7f216c0c6190 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f2171e4b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c064570 con 0x7f216c098d10 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f2171e4b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c154bc0 con 0x7f216c158a30 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f216c158a30 0x7f216c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f216c0c6190 0x7f216c158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54798/0 (socket says 172.21.15.73:54798) 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216affd700 1 -- 172.21.15.73:0/2539723908 learned_addr learned my addr 172.21.15.73:0/2539723908 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216bfff700 1 -- 172.21.15.73:0/2539723908 >> v1:172.21.15.150:6789/0 conn(0x7f216c098d10 legacy=0x7f216c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216bfff700 1 -- 172.21.15.73:0/2539723908 >> v1:172.21.15.73:6789/0 conn(0x7f216c0c6190 legacy=0x7f216c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.094+0000 7f216a7fc700 1 -- 172.21.15.73:0/2539723908 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 19605714 0 0) 0x7f216c0c72d0 con 0x7f216c0c6190 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f216bfff700 1 -- 172.21.15.73:0/2539723908 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c0c66c0 con 0x7f216c158a30 2023-11-28T02:53:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f216bfff700 1 --2- 172.21.15.73:0/2539723908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c0c7760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f2164009760 tx=0x7f2164009060).ready entity=mon.0 client_cookie=11527091b16f51fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f216a7fc700 1 -- 172.21.15.73:0/2539723908 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400b580 con 0x7f216c158a30 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f216a7fc700 1 -- 172.21.15.73:0/2539723908 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400b6e0 con 0x7f216c158a30 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f216a7fc700 1 -- 172.21.15.73:0/2539723908 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400ba60 con 0x7f216c158a30 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 -- 172.21.15.73:0/2539723908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 msgr2=0x7f216c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 --2- 172.21.15.73:0/2539723908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c0c7760 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 rx=0x7f2164009760 tx=0x7f2164009060).stop 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 -- 172.21.15.73:0/2539723908 shutdown_connections 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 --2- 172.21.15.73:0/2539723908 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c0c7760 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 -- 172.21.15.73:0/2539723908 >> 172.21.15.73:0/2539723908 conn(0x7f216c0cb310 msgr2=0x7f216c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 -- 172.21.15.73:0/2539723908 shutdown_connections 2023-11-28T02:53:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.095+0000 7f2171e4b700 1 -- 172.21.15.73:0/2539723908 wait complete. 2023-11-28T02:53:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 Processor -- start 2023-11-28T02:53:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 -- start start 2023-11-28T02:53:15.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f216c098d10 0x7f216c14bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 0x7f216c0ac160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0abcd0 con 0x7f216c158a30 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0ab0c0 con 0x7f216c0c6190 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f2171e4b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0ab3a0 con 0x7f216c098d10 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f216bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f216bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51966/0 (socket says 172.21.15.73:51966) 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.097+0000 7f216bfff700 1 -- 172.21.15.73:0/406440799 learned_addr learned my addr 172.21.15.73:0/406440799 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216b7fe700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f216c098d10 0x7f216c14bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216affd700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 0x7f216c0ac160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 -- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f216c098d10 msgr2=0x7f216c14bb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f216c098d10 0x7f216c14bb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 -- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 msgr2=0x7f216c0ac160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 0x7f216c0ac160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 -- 172.21.15.73:0/406440799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21640092a0 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216affd700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 0x7f216c0ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f216bfff700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f216c0c2360 tx=0x7f2164017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216400ba60 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216401b410 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c0ab680 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f216401b790 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.098+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c0ac650 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.099+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c06c860 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.099+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f216401bbc0 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.100+0000 7f21537fe700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 0x7f21580615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.100+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f216407bd40 con 0x7f216c158a30 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.100+0000 7f216b7fe700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 0x7f21580615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.100+0000 7f216b7fe700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 0x7f21580615e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f215c004290 tx=0x7f215c001560).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.104+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f216404d010 con 0x7f216c158a30 2023-11-28T02:53:15.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.266+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f216c0c69a0 con 0x7f216c158a30 2023-11-28T02:53:15.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.267+0000 7f21537fe700 1 -- 172.21.15.73:0/406440799 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v22) v1 ==== 110+0+10 (secure 0 0 0) 0x7f2164050630 con 0x7f216c158a30 2023-11-28T02:53:15.268 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:15.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 msgr2=0x7f21580615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 0x7f21580615e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f215c004290 tx=0x7f215c001560).stop 2023-11-28T02:53:15.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 msgr2=0x7f216c157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 rx=0x7f216c0c2360 tx=0x7f2164017000).stop 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 shutdown_connections 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f215805f120 0x7f21580615e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f216c158a30 0x7f216c157500 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f216c098d10 0x7f216c14bb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 --2- 172.21.15.73:0/406440799 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f216c0c6190 0x7f216c0ac160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 >> 172.21.15.73:0/406440799 conn(0x7f216c0cb310 msgr2=0x7f216c0ca600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 shutdown_connections 2023-11-28T02:53:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.269+0000 7f2171e4b700 1 -- 172.21.15.73:0/406440799 wait complete. 2023-11-28T02:53:15.282 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.397+0000 7faf34dd2700 1 Processor -- start 2023-11-28T02:53:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.397+0000 7faf34dd2700 1 -- start start 2023-11-28T02:53:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 0x7faf300c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf30062700 0x7faf30062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf300630a0 con 0x7faf30062700 2023-11-28T02:53:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf300c7e30 con 0x7faf300c86f0 2023-11-28T02:53:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf34dd2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf300bcf40 con 0x7faf301547f0 2023-11-28T02:53:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf30062700 0x7faf30062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf30062700 0x7faf30062bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51972/0 (socket says 172.21.15.73:51972) 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 -- 172.21.15.73:0/2791970556 learned_addr learned my addr 172.21.15.73:0/2791970556 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2e59c700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2dd9b700 1 --2- 172.21.15.73:0/2791970556 >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 0x7faf300c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 -- 172.21.15.73:0/2791970556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 msgr2=0x7faf300a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 -- 172.21.15.73:0/2791970556 >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 msgr2=0x7faf300c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 --2- 172.21.15.73:0/2791970556 >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 0x7faf300c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 -- 172.21.15.73:0/2791970556 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf300c6b20 con 0x7faf30062700 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2e59c700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2dd9b700 1 --2- 172.21.15.73:0/2791970556 >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 0x7faf300c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.398+0000 7faf2ed9d700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30062bb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7faf300600b0 tx=0x7faf28009840).ready entity=mon.0 client_cookie=8b0aa111a180ddb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf2d59a700 1 -- 172.21.15.73:0/2791970556 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800e070 con 0x7faf30062700 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf2d59a700 1 -- 172.21.15.73:0/2791970556 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2800ba50 con 0x7faf30062700 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf2d59a700 1 -- 172.21.15.73:0/2791970556 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800bdd0 con 0x7faf30062700 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 -- 172.21.15.73:0/2791970556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 msgr2=0x7faf30062bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30062bb0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 rx=0x7faf300600b0 tx=0x7faf28009840).stop 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 -- 172.21.15.73:0/2791970556 shutdown_connections 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30062bb0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2791970556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2791970556 >> v2:172.21.15.150:3300/0 conn(0x7faf300c86f0 0x7faf300c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 -- 172.21.15.73:0/2791970556 >> 172.21.15.73:0/2791970556 conn(0x7faf300642d0 msgr2=0x7faf300c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 -- 172.21.15.73:0/2791970556 shutdown_connections 2023-11-28T02:53:15.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.399+0000 7faf34dd2700 1 -- 172.21.15.73:0/2791970556 wait complete. 2023-11-28T02:53:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.400+0000 7faf34dd2700 1 Processor -- start 2023-11-28T02:53:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.400+0000 7faf34dd2700 1 -- start start 2023-11-28T02:53:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf300c86f0 0x7faf300acd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300ad210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf300ac890 con 0x7faf30062700 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf30146440 con 0x7faf300c86f0 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf34dd2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf30153aa0 con 0x7faf301547f0 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51986/0 (socket says 172.21.15.73:51986) 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 -- 172.21.15.73:0/2606545901 learned_addr learned my addr 172.21.15.73:0/2606545901 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2dd9b700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf300c86f0 0x7faf300acd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2ed9d700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300ad210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 -- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 msgr2=0x7faf300ad210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300ad210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 -- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf300c86f0 msgr2=0x7faf300acd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf300c86f0 0x7faf300acd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.401+0000 7faf2e59c700 1 -- 172.21.15.73:0/2606545901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28009ae0 con 0x7faf30062700 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf2ed9d700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf2e59c700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7faf2000c880 tx=0x7faf20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20010580 con 0x7faf30062700 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf200106e0 con 0x7faf30062700 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf30153d80 con 0x7faf30062700 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf20010a60 con 0x7faf30062700 2023-11-28T02:53:15.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.402+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf300ac140 con 0x7faf30062700 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.403+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf300acb70 con 0x7faf30062700 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.403+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faf2001b460 con 0x7faf30062700 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.403+0000 7faf1e7fc700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 0x7faf100615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.404+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faf2007a0d0 con 0x7faf30062700 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.404+0000 7faf2dd9b700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 0x7faf100615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.404+0000 7faf2dd9b700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 0x7faf100615e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7faf300674c0 tx=0x7faf24008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.408+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf2004b340 con 0x7faf30062700 2023-11-28T02:53:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.570+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faf3006c860 con 0x7faf30062700 2023-11-28T02:53:15.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.570+0000 7faf1e7fc700 1 -- 172.21.15.73:0/2606545901 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v22) v1 ==== 92+0+10 (secure 0 0 0) 0x7faf2004e960 con 0x7faf30062700 2023-11-28T02:53:15.571 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 msgr2=0x7faf100615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 0x7faf100615e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7faf300674c0 tx=0x7faf24008040).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 msgr2=0x7faf30145f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 rx=0x7faf2000c880 tx=0x7faf20008040).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 shutdown_connections 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf1005f120 0x7faf100615e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf301547f0 0x7faf300ad210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf30062700 0x7faf30145f50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 --2- 172.21.15.73:0/2606545901 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf300c86f0 0x7faf300acd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 >> 172.21.15.73:0/2606545901 conn(0x7faf300642d0 msgr2=0x7faf300c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 shutdown_connections 2023-11-28T02:53:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.572+0000 7faf34dd2700 1 -- 172.21.15.73:0/2606545901 wait complete. 2023-11-28T02:53:15.585 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T02:53:15.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.699+0000 7efd15a5a700 1 Processor -- start 2023-11-28T02:53:15.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.699+0000 7efd15a5a700 1 -- start start 2023-11-28T02:53:15.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd15a5a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd10098d10 0x7efd100c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd15a5a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd100c72d0 con 0x7efd10158320 2023-11-28T02:53:15.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd15a5a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd10064570 con 0x7efd100c6190 2023-11-28T02:53:15.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd15a5a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd10154bc0 con 0x7efd10098d10 2023-11-28T02:53:15.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd10098d10 0x7efd100c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efd10158320 0x7efd100a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54806/0 (socket says 172.21.15.73:54806) 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd10098d10 0x7efd100c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52002/0 (socket says 172.21.15.73:52002) 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0f7fe700 1 -- 172.21.15.73:0/1152706669 learned_addr learned my addr 172.21.15.73:0/1152706669 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0effd700 1 -- 172.21.15.73:0/1152706669 >> v1:172.21.15.150:6789/0 conn(0x7efd100c6190 legacy=0x7efd10063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0effd700 1 -- 172.21.15.73:0/1152706669 >> v1:172.21.15.73:6789/0 conn(0x7efd10158320 legacy=0x7efd100a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.700+0000 7efd0e7fc700 1 -- 172.21.15.73:0/1152706669 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976194803 0 0) 0x7efd100c72d0 con 0x7efd10158320 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd0effd700 1 -- 172.21.15.73:0/1152706669 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd100c66c0 con 0x7efd10098d10 2023-11-28T02:53:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd0effd700 1 --2- 172.21.15.73:0/1152706669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10098d10 0x7efd100c7760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7efd00009760 tx=0x7efd00009060).ready entity=mon.0 client_cookie=a65662c5c9865566 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd0e7fc700 1 -- 172.21.15.73:0/1152706669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000b520 con 0x7efd10098d10 2023-11-28T02:53:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd0e7fc700 1 -- 172.21.15.73:0/1152706669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd0000b680 con 0x7efd10098d10 2023-11-28T02:53:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd0e7fc700 1 -- 172.21.15.73:0/1152706669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd0000ba00 con 0x7efd10098d10 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 -- 172.21.15.73:0/1152706669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10098d10 msgr2=0x7efd100c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 --2- 172.21.15.73:0/1152706669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10098d10 0x7efd100c7760 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 rx=0x7efd00009760 tx=0x7efd00009060).stop 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 -- 172.21.15.73:0/1152706669 shutdown_connections 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 --2- 172.21.15.73:0/1152706669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10098d10 0x7efd100c7760 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 -- 172.21.15.73:0/1152706669 >> 172.21.15.73:0/1152706669 conn(0x7efd100cb310 msgr2=0x7efd100cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 -- 172.21.15.73:0/1152706669 shutdown_connections 2023-11-28T02:53:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.701+0000 7efd15a5a700 1 -- 172.21.15.73:0/1152706669 wait complete. 2023-11-28T02:53:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 Processor -- start 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 -- start start 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 0x7efd100c4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd100c6190 0x7efd100c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd10148e00 con 0x7efd10158320 2023-11-28T02:53:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd101571d0 con 0x7efd10098d10 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.703+0000 7efd15a5a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd10156c10 con 0x7efd100c6190 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52018/0 (socket says 172.21.15.73:52018) 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 -- 172.21.15.73:0/961286715 learned_addr learned my addr 172.21.15.73:0/961286715 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0effd700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 0x7efd100c4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd067fc700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd100c6190 0x7efd100c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 -- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd100c6190 msgr2=0x7efd100c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd100c6190 0x7efd100c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 -- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 msgr2=0x7efd100c4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 0x7efd100c4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 -- 172.21.15.73:0/961286715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd000092a0 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0effd700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 0x7efd100c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd0f7fe700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7efd0800c730 tx=0x7efd08001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08005d30 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd08005e90 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd08007340 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd10156ef0 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.704+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd101568a0 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.705+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd10148fb0 con 0x7efd10158320 2023-11-28T02:53:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.705+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efd080076f0 con 0x7efd10158320 2023-11-28T02:53:15.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.706+0000 7efd077fe700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 0x7efcf00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:15.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.706+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efd0807a190 con 0x7efd10158320 2023-11-28T02:53:15.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.706+0000 7efd0effd700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 0x7efcf00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:15.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.706+0000 7efd0effd700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 0x7efcf00615e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7efd0000b2e0 tx=0x7efd00017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:15.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.710+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd10148fb0 con 0x7efd10158320 2023-11-28T02:53:15.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.874+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7efd10148fb0 con 0x7efd10158320 2023-11-28T02:53:15.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.874+0000 7efd077fe700 1 -- 172.21.15.73:0/961286715 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7efd10148fb0 con 0x7efd10158320 2023-11-28T02:53:15.876 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 msgr2=0x7efcf00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 0x7efcf00615e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7efd0000b2e0 tx=0x7efd00017000).stop 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 msgr2=0x7efd100c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 rx=0x7efd0800c730 tx=0x7efd08001520).stop 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 shutdown_connections 2023-11-28T02:53:15.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efcf005f120 0x7efcf00615e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd10158320 0x7efd100c32b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd10098d10 0x7efd100c4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 --2- 172.21.15.73:0/961286715 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd100c6190 0x7efd100c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 >> 172.21.15.73:0/961286715 conn(0x7efd100cb310 msgr2=0x7efd1014bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 shutdown_connections 2023-11-28T02:53:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:15.876+0000 7efd15a5a700 1 -- 172.21.15.73:0/961286715 wait complete. 2023-11-28T02:53:15.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T02:53:16.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.004+0000 7f66c15d2700 1 Processor -- start 2023-11-28T02:53:16.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.004+0000 7f66c15d2700 1 -- start start 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66c15d2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 0x7f66bc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66c15d2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66bc0c6190 0x7f66bc063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66c15d2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66bc0c72d0 con 0x7f66bc064350 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66c15d2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc154bc0 con 0x7f66bc0c6190 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66c15d2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66bc0c66c0 con 0x7f66bc098d10 2023-11-28T02:53:16.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66bc0c6190 0x7f66bc063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66bc0c6190 0x7f66bc063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52020/0 (socket says 172.21.15.73:52020) 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1183443414 learned_addr learned my addr 172.21.15.73:0/1183443414 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66baffd700 1 --2- 172.21.15.73:0/1183443414 >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 0x7f66bc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1183443414 >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 msgr2=0x7f66bc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 --2- 172.21.15.73:0/1183443414 >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 0x7f66bc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1183443414 >> v1:172.21.15.150:6789/0 conn(0x7f66bc064350 legacy=0x7f66bc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1183443414 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0c69a0 con 0x7f66bc0c6190 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66baffd700 1 --2- 172.21.15.73:0/1183443414 >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 0x7f66bc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66ba7fc700 1 --2- 172.21.15.73:0/1183443414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0c6190 0x7f66bc063e60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f66bc083f80 tx=0x7f66b00090f0).ready entity=mon.0 client_cookie=8691af6872e18349 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66b9ffb700 1 -- 172.21.15.73:0/1183443414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000b5a0 con 0x7f66bc0c6190 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66b9ffb700 1 -- 172.21.15.73:0/1183443414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b000b700 con 0x7f66bc0c6190 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.005+0000 7f66b9ffb700 1 -- 172.21.15.73:0/1183443414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000ba80 con 0x7f66bc0c6190 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 -- 172.21.15.73:0/1183443414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0c6190 msgr2=0x7f66bc063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1183443414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0c6190 0x7f66bc063e60 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 rx=0x7f66bc083f80 tx=0x7f66b00090f0).stop 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 -- 172.21.15.73:0/1183443414 shutdown_connections 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1183443414 >> v2:172.21.15.150:3300/0 conn(0x7f66bc098d10 0x7f66bc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1183443414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc0c6190 0x7f66bc063e60 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 -- 172.21.15.73:0/1183443414 >> 172.21.15.73:0/1183443414 conn(0x7f66bc0cb310 msgr2=0x7f66bc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 -- 172.21.15.73:0/1183443414 shutdown_connections 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.006+0000 7f66c15d2700 1 -- 172.21.15.73:0/1183443414 wait complete. 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.007+0000 7f66c15d2700 1 Processor -- start 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.007+0000 7f66c15d2700 1 -- start start 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 0x7f66bc155e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 0x7f66bc1687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc156f90 con 0x7f66bc098d10 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc168cd0 con 0x7f66bc064350 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66c15d2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66bc1566f0 con 0x7f66bc0c6190 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52026/0 (socket says 172.21.15.73:52026) 2023-11-28T02:53:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1243407800 learned_addr learned my addr 172.21.15.73:0/1243407800 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66baffd700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 0x7f66bc155e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66bb7fe700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 0x7f66bc1687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 msgr2=0x7f66bc1687e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 0x7f66bc1687e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 msgr2=0x7f66bc155e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 0x7f66bc155e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 -- 172.21.15.73:0/1243407800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b0009370 con 0x7f66bc098d10 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66baffd700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 0x7f66bc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66bb7fe700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 0x7f66bc1687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.008+0000 7f66ba7fc700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f66b0009130 tx=0x7f66b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.009+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b000ba80 con 0x7f66bc098d10 2023-11-28T02:53:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.009+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b0000c90 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.009+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66bc1569d0 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.009+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b001b660 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.009+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66bc0aef20 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.010+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66bc0af110 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.010+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f66b001b9d0 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.010+0000 7f66a2ffd700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.010+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f66b007bce0 con 0x7f66bc098d10 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.010+0000 7f66baffd700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.011+0000 7f66baffd700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f66b4008990 tx=0x7f66b4008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.014+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66b004cfb0 con 0x7f66bc098d10 2023-11-28T02:53:16.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.183+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f66bc06c860 con 0x7f66bc098d10 2023-11-28T02:53:16.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.184+0000 7f66a2ffd700 1 -- 172.21.15.73:0/1243407800 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f66b0029020 con 0x7f66bc098d10 2023-11-28T02:53:16.185 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 msgr2=0x7f66a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f66b4008990 tx=0x7f66b4008040).stop 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 msgr2=0x7f66bc1682f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 rx=0x7f66b0009130 tx=0x7f66b0015000).stop 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 shutdown_connections 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66bc0c6190 0x7f66bc1687e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66bc064350 0x7f66bc155e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 --2- 172.21.15.73:0/1243407800 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66bc098d10 0x7f66bc1682f0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.185+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 >> 172.21.15.73:0/1243407800 conn(0x7f66bc0cb310 msgr2=0x7f66bc152620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.186+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 shutdown_connections 2023-11-28T02:53:16.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.186+0000 7f66c15d2700 1 -- 172.21.15.73:0/1243407800 wait complete. 2023-11-28T02:53:16.199 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.311+0000 7f215f87e700 1 Processor -- start 2023-11-28T02:53:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.311+0000 7f215f87e700 1 -- start start 2023-11-28T02:53:16.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215f87e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21580caab0 0x7f21580aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215f87e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21580a9a90 con 0x7f21580ca180 2023-11-28T02:53:16.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215f87e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2158154ce0 con 0x7f21580cae90 2023-11-28T02:53:16.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215f87e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21580a8e80 con 0x7f21580caab0 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215de1b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21580ca180 0x7f215815e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54810/0 (socket says 172.21.15.73:54810) 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215de1b700 1 -- 172.21.15.73:0/2259460369 learned_addr learned my addr 172.21.15.73:0/2259460369 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1501122865 0 0) 0x7f21580a9a90 con 0x7f21580ca180 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21400033f0 con 0x7f21580ca180 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.312+0000 7f215d61a700 1 --2- 172.21.15.73:0/2259460369 >> v2:172.21.15.150:3300/0 conn(0x7f21580caab0 0x7f21580aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1624424685 0 0) 0x7f21400033f0 con 0x7f21580ca180 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 >> v2:172.21.15.150:3300/0 conn(0x7f21580caab0 msgr2=0x7f21580aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 --2- 172.21.15.73:0/2259460369 >> v2:172.21.15.150:3300/0 conn(0x7f21580caab0 0x7f21580aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 >> v1:172.21.15.150:6789/0 conn(0x7f21580cae90 legacy=0x7f21580c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21580a9160 con 0x7f21580ca180 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2150001470 con 0x7f21580ca180 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 >> v1:172.21.15.73:6789/0 conn(0x7f21580ca180 legacy=0x7f215815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 0x7f2140003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2140004690 0x7f2140004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2140005000 0x7f214000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21580a8e80 con 0x7f21400038e0 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f214000bf60 con 0x7f2140004690 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f214000c0f0 con 0x7f2140005000 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 0x7f2140003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215ce19700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2140004690 0x7f2140004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215de1b700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2140005000 0x7f214000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 -- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2140005000 msgr2=0x7f214000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2140005000 0x7f214000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 -- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2140004690 msgr2=0x7f2140004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2140004690 0x7f2140004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 -- 172.21.15.73:0/2259460369 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21400033f0 con 0x7f21400038e0 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215de1b700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2140005000 0x7f214000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215de1b700 1 -- 172.21.15.73:0/2259460369 reap_dead start 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f215d61a700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 0x7f2140003d00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f214800e210 tx=0x7f214800b040).ready entity=mon.0 client_cookie=99a54b1aff38eff3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2148009b10 con 0x7f21400038e0 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214800f040 con 0x7f21400038e0 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.313+0000 7f2157fff700 1 -- 172.21.15.73:0/2259460369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214800ece0 con 0x7f21400038e0 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 -- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 msgr2=0x7f2140003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 0x7f2140003d00 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 rx=0x7f214800e210 tx=0x7f214800b040).stop 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 -- 172.21.15.73:0/2259460369 shutdown_connections 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 --2- 172.21.15.73:0/2259460369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21400038e0 0x7f2140003d00 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 -- 172.21.15.73:0/2259460369 >> 172.21.15.73:0/2259460369 conn(0x7f2158094920 msgr2=0x7f215806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 -- 172.21.15.73:0/2259460369 shutdown_connections 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.314+0000 7f215f87e700 1 -- 172.21.15.73:0/2259460369 wait complete. 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.315+0000 7f215f87e700 1 Processor -- start 2023-11-28T02:53:16.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.315+0000 7f215f87e700 1 -- start start 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21400038e0 0x7f21580c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2158167c60 0x7f21581680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21580c7c20 con 0x7f21580c77a0 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2158165070 con 0x7f21400038e0 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215f87e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21580c8a30 con 0x7f2158167c60 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52040/0 (socket says 172.21.15.73:52040) 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 -- 172.21.15.73:0/3517163868 learned_addr learned my addr 172.21.15.73:0/3517163868 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 -- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2158167c60 msgr2=0x7f21581680c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2158167c60 0x7f21581680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 -- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21400038e0 msgr2=0x7f21580c72b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21400038e0 0x7f21580c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 -- 172.21.15.73:0/3517163868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21581685b0 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.316+0000 7f215ce19700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f214c0028f0 tx=0x7f214c007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.317+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00f6c0 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.317+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f214c00f820 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.317+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f214c00fba0 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.317+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21580c8ec0 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.317+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21580c8100 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.318+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f215806c860 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.318+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f214c014450 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.318+0000 7f21557fa700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 0x7f2144061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.318+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f214c079e70 con 0x7f21580c77a0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.318+0000 7f215d61a700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 0x7f2144061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.319+0000 7f215d61a700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 0x7f2144061610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f214800e210 tx=0x7f214801a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.322+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f214c04b190 con 0x7f21580c77a0 2023-11-28T02:53:16.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.530+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21580c8ba0 con 0x7f21580c77a0 2023-11-28T02:53:16.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.531+0000 7f21557fa700 1 -- 172.21.15.73:0/3517163868 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f214c04e7b0 con 0x7f21580c77a0 2023-11-28T02:53:16.532 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:16.532 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 msgr2=0x7f2144061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 0x7f2144061610 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7f214800e210 tx=0x7f214801a000).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 msgr2=0x7f2158167770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 rx=0x7f214c0028f0 tx=0x7f214c007bb0).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 shutdown_connections 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214405f150 0x7f2144061610 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2158167c60 0x7f21581680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21400038e0 0x7f21580c72b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 --2- 172.21.15.73:0/3517163868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21580c77a0 0x7f2158167770 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 >> 172.21.15.73:0/3517163868 conn(0x7f2158094920 msgr2=0x7f21580b0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 shutdown_connections 2023-11-28T02:53:16.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.533+0000 7f215f87e700 1 -- 172.21.15.73:0/3517163868 wait complete. 2023-11-28T02:53:16.536 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:53:16.546 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.657+0000 7f8af150e700 1 Processor -- start 2023-11-28T02:53:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.657+0000 7f8af150e700 1 -- start start 2023-11-28T02:53:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8af150e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aec0cae90 0x7f8aec0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8af150e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec0a9a90 con 0x7f8aec0ca180 2023-11-28T02:53:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8af150e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aec154ce0 con 0x7f8aec0caab0 2023-11-28T02:53:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8af150e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aec0aab80 con 0x7f8aec0cae90 2023-11-28T02:53:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aea7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aec0cae90 0x7f8aec0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aeb7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8aec0ca180 0x7f8aec15e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54826/0 (socket says 172.21.15.73:54826) 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aeb7fe700 1 -- 172.21.15.73:0/2046609348 learned_addr learned my addr 172.21.15.73:0/2046609348 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aea7fc700 1 -- 172.21.15.73:0/2046609348 >> v1:172.21.15.150:6789/0 conn(0x7f8aec0caab0 legacy=0x7f8aec0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aea7fc700 1 -- 172.21.15.73:0/2046609348 >> v1:172.21.15.73:6789/0 conn(0x7f8aec0ca180 legacy=0x7f8aec15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aea7fc700 1 -- 172.21.15.73:0/2046609348 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aec0a8e80 con 0x7f8aec0cae90 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8aea7fc700 1 --2- 172.21.15.73:0/2046609348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0cae90 0x7f8aec0c0ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8ae0009e60 tx=0x7f8ae0009780).ready entity=mon.0 client_cookie=958beee3d5b41ed9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8ae9ffb700 1 -- 172.21.15.73:0/2046609348 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae000e070 con 0x7f8aec0cae90 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8ae9ffb700 1 -- 172.21.15.73:0/2046609348 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae000bc30 con 0x7f8aec0cae90 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.658+0000 7f8ae9ffb700 1 -- 172.21.15.73:0/2046609348 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae0012410 con 0x7f8aec0cae90 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 -- 172.21.15.73:0/2046609348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0cae90 msgr2=0x7f8aec0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 --2- 172.21.15.73:0/2046609348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0cae90 0x7f8aec0c0ab0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 rx=0x7f8ae0009e60 tx=0x7f8ae0009780).stop 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 -- 172.21.15.73:0/2046609348 shutdown_connections 2023-11-28T02:53:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 --2- 172.21.15.73:0/2046609348 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0cae90 0x7f8aec0c0ab0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 -- 172.21.15.73:0/2046609348 >> 172.21.15.73:0/2046609348 conn(0x7f8aec094920 msgr2=0x7f8aec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 -- 172.21.15.73:0/2046609348 shutdown_connections 2023-11-28T02:53:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.659+0000 7f8af150e700 1 -- 172.21.15.73:0/2046609348 wait complete. 2023-11-28T02:53:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.660+0000 7f8af150e700 1 Processor -- start 2023-11-28T02:53:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.660+0000 7f8af150e700 1 -- start start 2023-11-28T02:53:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0caab0 0x7f8aec0abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 0x7f8aec0ac430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec0ac920 con 0x7f8aec0ca180 2023-11-28T02:53:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec168870 con 0x7f8aec0cae90 2023-11-28T02:53:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aec167f80 con 0x7f8aec0caab0 2023-11-28T02:53:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52062/0 (socket says 172.21.15.73:52062) 2023-11-28T02:53:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 -- 172.21.15.73:0/2677597855 learned_addr learned my addr 172.21.15.73:0/2677597855 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeb7fe700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 0x7f8aec0ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 -- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0caab0 msgr2=0x7f8aec0abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0caab0 0x7f8aec0abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 -- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 msgr2=0x7f8aec0ac430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 0x7f8aec0ac430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 -- 172.21.15.73:0/2677597855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae0009a20 con 0x7f8aec0ca180 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeb7fe700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 0x7f8aec0ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8aeaffd700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8ae400c4c0 tx=0x7f8ae4007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4007020 con 0x7f8aec0ca180 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ae4007180 con 0x7f8aec0ca180 2023-11-28T02:53:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aec157ee0 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ae4007520 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.661+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aec168500 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.662+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aec0991a0 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.663+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8ae4006900 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.663+0000 7f8ad2ffd700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 0x7f8ad80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.663+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8ae4079ca0 con 0x7f8aec0ca180 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.663+0000 7f8aea7fc700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 0x7f8ad80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.663+0000 7f8aea7fc700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 0x7f8ad80615e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8aec0b0400 tx=0x7f8ae0009780).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.668+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8ae404af70 con 0x7f8aec0ca180 2023-11-28T02:53:16.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.831+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8aec06c860 con 0x7f8aec0ca180 2023-11-28T02:53:16.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.832+0000 7f8ad2ffd700 1 -- 172.21.15.73:0/2677597855 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f8ae404ee60 con 0x7f8aec0ca180 2023-11-28T02:53:16.833 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:16.834 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 msgr2=0x7f8ad80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 0x7f8ad80615e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f8aec0b0400 tx=0x7f8ae0009780).stop 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 msgr2=0x7f8aec140610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 rx=0x7f8ae400c4c0 tx=0x7f8ae4007df0).stop 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 shutdown_connections 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ad805f120 0x7f8ad80615e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aec0cae90 0x7f8aec0ac430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aec0ca180 0x7f8aec140610 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 --2- 172.21.15.73:0/2677597855 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aec0caab0 0x7f8aec0abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:16.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.833+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 >> 172.21.15.73:0/2677597855 conn(0x7f8aec094920 msgr2=0x7f8aec0ab800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.834+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 shutdown_connections 2023-11-28T02:53:16.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.834+0000 7f8af150e700 1 -- 172.21.15.73:0/2677597855 wait complete. 2023-11-28T02:53:16.847 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T02:53:16.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:16.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.959+0000 7f6cc2bf9700 1 Processor -- start 2023-11-28T02:53:16.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.959+0000 7f6cc2bf9700 1 -- start start 2023-11-28T02:53:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc2bf9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6cbc0c0180 0x7f6cbc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc2bf9700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0c12c0 con 0x7f6cbc0c1750 2023-11-28T02:53:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc2bf9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cbc0c06b0 con 0x7f6cbc157660 2023-11-28T02:53:16.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc2bf9700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6cbc0c0990 con 0x7f6cbc0c0180 2023-11-28T02:53:16.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc23f8700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6cbc0c1750 0x7f6cbc0648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54828/0 (socket says 172.21.15.73:54828) 2023-11-28T02:53:16.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc23f8700 1 -- 172.21.15.73:0/747946722 learned_addr learned my addr 172.21.15.73:0/747946722 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:16.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/747946722 >> v2:172.21.15.150:3300/0 conn(0x7f6cbc0c0180 0x7f6cbc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2448642766 0 0) 0x7f6cbc0c12c0 con 0x7f6cbc0c1750 2023-11-28T02:53:16.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ca80033f0 con 0x7f6cbc0c1750 2023-11-28T02:53:16.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1598593427 0 0) 0x7f6ca80033f0 con 0x7f6cbc0c1750 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 >> v2:172.21.15.150:3300/0 conn(0x7f6cbc0c0180 msgr2=0x7f6cbc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 --2- 172.21.15.73:0/747946722 >> v2:172.21.15.150:3300/0 conn(0x7f6cbc0c0180 0x7f6cbc157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 >> v1:172.21.15.150:6789/0 conn(0x7f6cbc157660 legacy=0x7f6cbc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0c0c70 con 0x7f6cbc0c1750 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.960+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6cb40017f0 con 0x7f6cbc0c1750 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 >> v1:172.21.15.73:6789/0 conn(0x7f6cbc0c1750 legacy=0x7f6cbc0648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca80038e0 0x7f6ca8003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 0x7f6ca8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca8005000 0x7f6ca800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0c0990 con 0x7f6ca8004690 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca800beb0 con 0x7f6ca8005000 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ca800c070 con 0x7f6ca80038e0 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 0x7f6ca8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc23f8700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca8005000 0x7f6ca800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc1bf7700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca80038e0 0x7f6ca8003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 -- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca80038e0 msgr2=0x7f6ca8003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca80038e0 0x7f6ca8003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 -- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca8005000 msgr2=0x7f6ca800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ca8005000 0x7f6ca800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc23f8700 1 -- 172.21.15.73:0/747946722 reap_dead start 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 -- 172.21.15.73:0/747946722 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ca80033f0 con 0x7f6ca8004690 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 0x7f6ca8004b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f6cac009a80 tx=0x7f6cac009060).ready entity=mon.0 client_cookie=1d0a77a4c041ac7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac009430 con 0x7f6ca8004690 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cac00e640 con 0x7f6ca8004690 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.961+0000 7f6cc0bf5700 1 -- 172.21.15.73:0/747946722 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac00e9c0 con 0x7f6ca8004690 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 msgr2=0x7f6ca8004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 0x7f6ca8004b10 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f6cac009a80 tx=0x7f6cac009060).stop 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/747946722 shutdown_connections 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/747946722 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ca8004690 0x7f6ca8004b10 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/747946722 >> 172.21.15.73:0/747946722 conn(0x7f6cbc0cb000 msgr2=0x7f6cbc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/747946722 shutdown_connections 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.962+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/747946722 wait complete. 2023-11-28T02:53:16.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.963+0000 7f6cc2bf9700 1 Processor -- start 2023-11-28T02:53:16.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.963+0000 7f6cc2bf9700 1 -- start start 2023-11-28T02:53:16.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca8004690 0x7f6cbc0c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 0x7f6cbc0adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc156570 con 0x7f6cbc155a80 2023-11-28T02:53:16.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0c2550 con 0x7f6cbc155600 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc0c4a20 con 0x7f6ca8004690 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52088/0 (socket says 172.21.15.73:52088) 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 -- 172.21.15.73:0/179319911 learned_addr learned my addr 172.21.15.73:0/179319911 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc1bf7700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca8004690 0x7f6cbc0c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 0x7f6cbc0adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 -- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca8004690 msgr2=0x7f6cbc0c4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca8004690 0x7f6cbc0c4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 -- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 msgr2=0x7f6cbc0adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:16.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 0x7f6cbc0adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 -- 172.21.15.73:0/179319911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cbc0ae990 con 0x7f6cbc155a80 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc13f6700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 0x7f6cbc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc23f8700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6cb4000f80 tx=0x7f6cb4001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400f8a0 con 0x7f6cbc155a80 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cb400fa00 con 0x7f6cbc155a80 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cb400fda0 con 0x7f6cbc155a80 2023-11-28T02:53:16.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.964+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc155fb0 con 0x7f6cbc155a80 2023-11-28T02:53:16.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.965+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc0c21e0 con 0x7f6cbc155a80 2023-11-28T02:53:16.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc0c2830 con 0x7f6cbc155a80 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6cb4012560 con 0x7f6cbc155a80 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cb1ffb700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 0x7f6ca0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6cb4079800 con 0x7f6cbc155a80 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cc1bf7700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 0x7f6ca0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.966+0000 7f6cc1bf7700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 0x7f6ca0061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6cb8000c00 tx=0x7f6cb8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:16.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:16.971+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6cb4049af0 con 0x7f6cbc155a80 2023-11-28T02:53:17.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.173+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cbc06c7c0 con 0x7f6cbc155a80 2023-11-28T02:53:17.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.174+0000 7f6cb1ffb700 1 -- 172.21.15.73:0/179319911 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f6cb4019020 con 0x7f6cbc155a80 2023-11-28T02:53:17.175 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:17.175 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 msgr2=0x7f6ca0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 0x7f6ca0061610 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 rx=0x7f6cb8000c00 tx=0x7f6cb8009040).stop 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 msgr2=0x7f6cbc0ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 rx=0x7f6cb4000f80 tx=0x7f6cb4001d90).stop 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 shutdown_connections 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ca005f150 0x7f6ca0061610 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6cbc155a80 0x7f6cbc0ae4a0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ca8004690 0x7f6cbc0c4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 --2- 172.21.15.73:0/179319911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6cbc155600 0x7f6cbc0adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.175+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 >> 172.21.15.73:0/179319911 conn(0x7f6cbc0cb000 msgr2=0x7f6cbc0aae90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.176+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 shutdown_connections 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.176+0000 7f6cc2bf9700 1 -- 172.21.15.73:0/179319911 wait complete. 2023-11-28T02:53:17.178 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:53:17.189 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:17.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.299+0000 7f914f61e700 1 Processor -- start 2023-11-28T02:53:17.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.299+0000 7f914f61e700 1 -- start start 2023-11-28T02:53:17.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914f61e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f91480a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914f61e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480c69a0 con 0x7f91481548c0 2023-11-28T02:53:17.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914f61e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91480c13a0 con 0x7f91481544e0 2023-11-28T02:53:17.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914f61e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91480c1680 con 0x7f914806a610 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914cbb9700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f91481548c0 0x7f91480c6e30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54844/0 (socket says 172.21.15.73:54844) 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914cbb9700 1 -- 172.21.15.73:0/3387004873 learned_addr learned my addr 172.21.15.73:0/3387004873 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.300+0000 7f914dbbb700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f91480a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3201162586 0 0) 0x7f91480c69a0 con 0x7f91481548c0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91300033f0 con 0x7f91481548c0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3041121852 0 0) 0x7f91480c13a0 con 0x7f91481544e0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91480c69a0 con 0x7f91481544e0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1605440225 0 0) 0x7f91300033f0 con 0x7f91481548c0 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 msgr2=0x7f91480a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f91480a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 >> v1:172.21.15.150:6789/0 conn(0x7f91481544e0 legacy=0x7f91480c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91480c6630 con 0x7f91481548c0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f913c001470 con 0x7f91481548c0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 >> v1:172.21.15.73:6789/0 conn(0x7f91481548c0 legacy=0x7f91480c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130003aa0 0x7f9130003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9130004690 0x7f9130004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f9130005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480c1680 con 0x7f9130004690 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f913000b910 con 0x7f9130003aa0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f913000bad0 con 0x7f914806a610 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9130004690 0x7f9130004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914d3ba700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130003aa0 0x7f9130003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914dbbb700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f9130005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 -- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 msgr2=0x7f9130005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f914806a610 0x7f9130005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 -- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130003aa0 msgr2=0x7f9130003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130003aa0 0x7f9130003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 -- 172.21.15.73:0/3387004873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91300033f0 con 0x7f9130004690 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f914cbb9700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9130004690 0x7f9130004b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f913c00c110 tx=0x7f913c001e10).ready entity=mon.0 client_cookie=7bd6295ce3145327 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.301+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c00ca40 con 0x7f9130004690 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c00cba0 con 0x7f9130004690 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f9147fff700 1 -- 172.21.15.73:0/3387004873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c0070d0 con 0x7f9130004690 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 -- 172.21.15.73:0/3387004873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9130004690 msgr2=0x7f9130004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 --2- 172.21.15.73:0/3387004873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9130004690 0x7f9130004b10 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 rx=0x7f913c00c110 tx=0x7f913c001e10).stop 2023-11-28T02:53:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914dbbb700 1 -- 172.21.15.73:0/3387004873 reap_dead start 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 -- 172.21.15.73:0/3387004873 shutdown_connections 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 -- 172.21.15.73:0/3387004873 >> 172.21.15.73:0/3387004873 conn(0x7f9148085020 msgr2=0x7f9148094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 -- 172.21.15.73:0/3387004873 shutdown_connections 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.302+0000 7f914f61e700 1 -- 172.21.15.73:0/3387004873 wait complete. 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.303+0000 7f914f61e700 1 Processor -- start 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.303+0000 7f914f61e700 1 -- start start 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 0x7f914813c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91480c4ec0 0x7f9148098360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9148151470 con 0x7f914813c6f0 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91480c5320 con 0x7f9130004690 2023-11-28T02:53:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914f61e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9148150860 con 0x7f91480c4ec0 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914cbb9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914cbb9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52112/0 (socket says 172.21.15.73:52112) 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914cbb9700 1 -- 172.21.15.73:0/59386639 learned_addr learned my addr 172.21.15.73:0/59386639 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.304+0000 7f914d3ba700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 0x7f914813c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914dbbb700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91480c4ec0 0x7f9148098360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 -- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91480c4ec0 msgr2=0x7f9148098360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91480c4ec0 0x7f9148098360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 -- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 msgr2=0x7f914813c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 0x7f914813c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 -- 172.21.15.73:0/59386639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9148150b40 con 0x7f914813c6f0 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914d3ba700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 0x7f914813c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914cbb9700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f913000b8e0 tx=0x7f913c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c001640 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f913c0046b0 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9148150e20 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f913c004a30 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.305+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9148098850 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.306+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f914806acd0 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.306+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f913c022470 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.307+0000 7f91457fa700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 0x7f9134061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.307+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f913c01c040 con 0x7f914813c6f0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.307+0000 7f914d3ba700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 0x7f9134061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.307+0000 7f914d3ba700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 0x7f9134061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9138007af0 tx=0x7f9138007a80).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.311+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f913c04c870 con 0x7f914813c6f0 2023-11-28T02:53:17.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.513+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91480c7540 con 0x7f914813c6f0 2023-11-28T02:53:17.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.513+0000 7f91457fa700 1 -- 172.21.15.73:0/59386639 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4483 (secure 0 0 0) 0x7f913c04fe90 con 0x7f914813c6f0 2023-11-28T02:53:17.515 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:17.515 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4266,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/697186067","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":697186067},{"type":"v1","addr":"172.21.15.73:6836","nonce":697186067}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4269,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1597706048","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1597706048},{"type":"v1","addr":"172.21.15.73:6837","nonce":1597706048}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4278,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/449713501","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":449713501},{"type":"v1","addr":"172.21.15.150:6834","nonce":449713501}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:52:44.444957+0000","modified":"2023-11-28T02:52:46.458206+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4281},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4281":{"gid":4281,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/2149449616","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2149449616},{"type":"v1","addr":"172.21.15.150:6835","nonce":2149449616}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-11-28T02:53:17.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 msgr2=0x7f9134061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 0x7f9134061610 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 rx=0x7f9138007af0 tx=0x7f9138007a80).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 msgr2=0x7f91480c49d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 rx=0x7f913000b8e0 tx=0x7f913c015000).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 shutdown_connections 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f913405f150 0x7f9134061610 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91480c4ec0 0x7f9148098360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9130004690 0x7f914813c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 --2- 172.21.15.73:0/59386639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f914813c6f0 0x7f91480c49d0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 >> 172.21.15.73:0/59386639 conn(0x7f9148085020 msgr2=0x7f91480b0c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 shutdown_connections 2023-11-28T02:53:17.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.515+0000 7f914f61e700 1 -- 172.21.15.73:0/59386639 wait complete. 2023-11-28T02:53:17.518 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 6 2023-11-28T02:53:17.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:17.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.640+0000 7f98064e6700 1 Processor -- start 2023-11-28T02:53:17.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.640+0000 7f98064e6700 1 -- start start 2023-11-28T02:53:17.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98001544e0 0x7f98001548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980006a610 0x7f98000c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98000c69a0 con 0x7f98000c6e30 2023-11-28T02:53:17.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98000c13a0 con 0x7f98001544e0 2023-11-28T02:53:17.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98000c1680 con 0x7f980006a610 2023-11-28T02:53:17.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98001544e0 0x7f98001548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f98001544e0 0x7f98001548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52122/0 (socket says 172.21.15.73:52122) 2023-11-28T02:53:17.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 -- 172.21.15.73:0/2752795161 learned_addr learned my addr 172.21.15.73:0/2752795161 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:17.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97ff7fe700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980006a610 0x7f98000c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 -- 172.21.15.73:0/2752795161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980006a610 msgr2=0x7f98000c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980006a610 0x7f98000c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 -- 172.21.15.73:0/2752795161 >> v1:172.21.15.150:6789/0 conn(0x7f98000c6e30 legacy=0x7f98000a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 -- 172.21.15.73:0/2752795161 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98000c6630 con 0x7f98001544e0 2023-11-28T02:53:17.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97fffff700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98001544e0 0x7f98001548c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f97f0009810 tx=0x7f97f0009060).ready entity=mon.0 client_cookie=b3e5f9f02f62d518 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97feffd700 1 -- 172.21.15.73:0/2752795161 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0005880 con 0x7f98001544e0 2023-11-28T02:53:17.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97feffd700 1 -- 172.21.15.73:0/2752795161 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f00059e0 con 0x7f98001544e0 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f97feffd700 1 -- 172.21.15.73:0/2752795161 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f0005d60 con 0x7f98001544e0 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- 172.21.15.73:0/2752795161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98001544e0 msgr2=0x7f98001548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98001544e0 0x7f98001548c0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 rx=0x7f97f0009810 tx=0x7f97f0009060).stop 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- 172.21.15.73:0/2752795161 shutdown_connections 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98001544e0 0x7f98001548c0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 --2- 172.21.15.73:0/2752795161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f980006a610 0x7f98000c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- 172.21.15.73:0/2752795161 >> 172.21.15.73:0/2752795161 conn(0x7f9800085020 msgr2=0x7f9800094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- 172.21.15.73:0/2752795161 shutdown_connections 2023-11-28T02:53:17.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.642+0000 7f98064e6700 1 -- 172.21.15.73:0/2752795161 wait complete. 2023-11-28T02:53:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.644+0000 7f98064e6700 1 Processor -- start 2023-11-28T02:53:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.644+0000 7f98064e6700 1 -- start start 2023-11-28T02:53:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980006a610 0x7f98000afea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 0x7f98000aaa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9800097c60 con 0x7f98000c6e30 2023-11-28T02:53:17.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98000a81e0 con 0x7f980006a610 2023-11-28T02:53:17.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9800097050 con 0x7f98001544e0 2023-11-28T02:53:17.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52124/0 (socket says 172.21.15.73:52124) 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 -- 172.21.15.73:0/2261563826 learned_addr learned my addr 172.21.15.73:0/2261563826 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97fffff700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980006a610 0x7f98000afea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f9804a83700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 0x7f98000aaa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 -- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 msgr2=0x7f98000aaa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 0x7f98000aaa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 -- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980006a610 msgr2=0x7f98000afea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980006a610 0x7f98000afea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 -- 172.21.15.73:0/2261563826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97f0009300 con 0x7f98000c6e30 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f9804a83700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 0x7f98000aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:17.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97ff7fe700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f97f400c650 tx=0x7f97f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4010580 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97f40106e0 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9800097330 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97f4010a60 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.645+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98000978f0 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.646+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f980006c860 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.646+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f97f401b460 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.647+0000 7f97e7fff700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.647+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f97f407a040 con 0x7f98000c6e30 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.647+0000 7f97fffff700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.647+0000 7f97fffff700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f97f0009fd0 tx=0x7f97f0016000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.652+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f97f404b2b0 con 0x7f98000c6e30 2023-11-28T02:53:17.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.816+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98000ae310 con 0x7f98000c6e30 2023-11-28T02:53:17.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.817+0000 7f97e7fff700 1 -- 172.21.15.73:0/2261563826 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f97f404e8d0 con 0x7f98000c6e30 2023-11-28T02:53:17.818 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:17.818 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:52:45.446125+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:52:42.251100+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-11-28T02:52:43.421590+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 msgr2=0x7f97ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f97f0009fd0 tx=0x7f97f0016000).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 msgr2=0x7f9800096a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 rx=0x7f97f400c650 tx=0x7f97f4008040).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 shutdown_connections 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97ec05f150 0x7f97ec061610 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f98001544e0 0x7f98000aaa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f980006a610 0x7f98000afea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 --2- 172.21.15.73:0/2261563826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f98000c6e30 0x7f9800096a90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 >> 172.21.15.73:0/2261563826 conn(0x7f9800085020 msgr2=0x7f98000c4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 shutdown_connections 2023-11-28T02:53:17.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.819+0000 7f98064e6700 1 -- 172.21.15.73:0/2261563826 wait complete. 2023-11-28T02:53:17.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T02:53:17.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.947+0000 7f672b109700 1 Processor -- start 2023-11-28T02:53:17.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.947+0000 7f672b109700 1 -- start start 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f672b109700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6724098d10 0x7f67240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f672b109700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f67240c6190 0x7f6724063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f672b109700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67240c72d0 con 0x7f6724064350 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f672b109700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6724154bc0 con 0x7f6724098d10 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f672b109700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67240c66c0 con 0x7f67240c6190 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f6728ea5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6724098d10 0x7f67240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f67296a6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6724064350 0x7f6724158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54854/0 (socket says 172.21.15.73:54854) 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f67296a6700 1 -- 172.21.15.73:0/1050916917 learned_addr learned my addr 172.21.15.73:0/1050916917 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:17.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f6723fff700 1 --2- 172.21.15.73:0/1050916917 >> v2:172.21.15.150:3300/0 conn(0x7f67240c6190 0x7f6724063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f6728ea5700 1 -- 172.21.15.73:0/1050916917 >> v2:172.21.15.150:3300/0 conn(0x7f67240c6190 msgr2=0x7f6724063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f6728ea5700 1 --2- 172.21.15.73:0/1050916917 >> v2:172.21.15.150:3300/0 conn(0x7f67240c6190 0x7f6724063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.948+0000 7f6728ea5700 1 -- 172.21.15.73:0/1050916917 >> v1:172.21.15.73:6789/0 conn(0x7f6724064350 legacy=0x7f6724158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f6728ea5700 1 -- 172.21.15.73:0/1050916917 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240c69a0 con 0x7f6724098d10 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f67237fe700 1 -- 172.21.15.73:0/1050916917 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2077306893 0 0) 0x7f67240c72d0 con 0x7f6724064350 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f6728ea5700 1 --2- 172.21.15.73:0/1050916917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6724098d10 0x7f67240c7760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f67240c5b40 tx=0x7f6714009060).ready entity=mon.0 client_cookie=36e327771adc08ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f67237fe700 1 -- 172.21.15.73:0/1050916917 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400b580 con 0x7f6724098d10 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f67237fe700 1 -- 172.21.15.73:0/1050916917 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671400b6e0 con 0x7f6724098d10 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f67237fe700 1 -- 172.21.15.73:0/1050916917 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671400ba60 con 0x7f6724098d10 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 -- 172.21.15.73:0/1050916917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6724098d10 msgr2=0x7f67240c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 --2- 172.21.15.73:0/1050916917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6724098d10 0x7f67240c7760 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 rx=0x7f67240c5b40 tx=0x7f6714009060).stop 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 -- 172.21.15.73:0/1050916917 shutdown_connections 2023-11-28T02:53:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 --2- 172.21.15.73:0/1050916917 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6724098d10 0x7f67240c7760 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 --2- 172.21.15.73:0/1050916917 >> v2:172.21.15.150:3300/0 conn(0x7f67240c6190 0x7f6724063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 -- 172.21.15.73:0/1050916917 >> 172.21.15.73:0/1050916917 conn(0x7f67240cb310 msgr2=0x7f67240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:17.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 -- 172.21.15.73:0/1050916917 shutdown_connections 2023-11-28T02:53:17.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.949+0000 7f672b109700 1 -- 172.21.15.73:0/1050916917 wait complete. 2023-11-28T02:53:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.950+0000 7f672b109700 1 Processor -- start 2023-11-28T02:53:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 -- start start 2023-11-28T02:53:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6724064350 0x7f6724155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6724098d10 0x7f672414b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f672414b9c0 con 0x7f67240c6190 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67240aad70 con 0x7f6724064350 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f672b109700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67240afc60 con 0x7f6724098d10 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f67296a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f67296a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52144/0 (socket says 172.21.15.73:52144) 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.951+0000 7f67296a6700 1 -- 172.21.15.73:0/1867340314 learned_addr learned my addr 172.21.15.73:0/1867340314 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f6728ea5700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6724064350 0x7f6724155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f6723fff700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6724098d10 0x7f672414b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 -- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6724098d10 msgr2=0x7f672414b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6724098d10 0x7f672414b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 -- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6724064350 msgr2=0x7f6724155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6724064350 0x7f6724155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 -- 172.21.15.73:0/1867340314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67140092a0 con 0x7f67240c6190 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f67296a6700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f671c00c810 tx=0x7f671c007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c005d30 con 0x7f67240c6190 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f671c005e90 con 0x7f67240c6190 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6724155a30 con 0x7f67240c6190 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f671c007360 con 0x7f67240c6190 2023-11-28T02:53:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.952+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6724155ff0 con 0x7f67240c6190 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.953+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67240afa30 con 0x7f67240c6190 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.953+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f671c007710 con 0x7f67240c6190 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.954+0000 7f6720ff9700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 0x7f67040615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.954+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f671c07a080 con 0x7f67240c6190 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.954+0000 7f6728ea5700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 0x7f67040615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.954+0000 7f6728ea5700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 0x7f67040615e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6714009fd0 tx=0x7f6714017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:17.958+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f67240afa30 con 0x7f67240c6190 2023-11-28T02:53:18.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.160+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f67240afa30 con 0x7f67240c6190 2023-11-28T02:53:18.549 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T02:53:18.549+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T02:53:18.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.554+0000 7f6720ff9700 1 -- 172.21.15.73:0/1867340314 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7f67240afa30 con 0x7f67240c6190 2023-11-28T02:53:18.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 msgr2=0x7f67040615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:18.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 0x7f67040615e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 rx=0x7f6714009fd0 tx=0x7f6714017000).stop 2023-11-28T02:53:18.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 msgr2=0x7f6724157600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 rx=0x7f671c00c810 tx=0x7f671c007f90).stop 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 shutdown_connections 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f670405f120 0x7f67040615e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f67240c6190 0x7f6724157600 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6724064350 0x7f6724155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 --2- 172.21.15.73:0/1867340314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6724098d10 0x7f672414b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 >> 172.21.15.73:0/1867340314 conn(0x7f67240cb310 msgr2=0x7f6724159c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 shutdown_connections 2023-11-28T02:53:18.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.556+0000 7f672b109700 1 -- 172.21.15.73:0/1867340314 wait complete. 2023-11-28T02:53:18.558 INFO:tasks.ceph.mds.d.smithi150.stderr: -1755> 2023-11-28T02:52:45.556+0000 7fdbd0bc1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T02:53:18.558 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T02:53:18.569 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T02:53:18.579 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T02:53:18.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.683+0000 7f8452e73700 1 Processor -- start 2023-11-28T02:53:18.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.683+0000 7f8452e73700 1 -- start start 2023-11-28T02:53:18.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c098d10 0x7f844c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f844c0c6190 0x7f844c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 0x7f844c158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844c0c72d0 con 0x7f844c0c6190 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f844c154bc0 con 0x7f844c064350 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8452e73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0c66c0 con 0x7f844c098d10 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f844c0c6190 0x7f844c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f844c0c6190 0x7f844c063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52158/0 (socket says 172.21.15.73:52158) 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 -- 172.21.15.73:0/2385404448 learned_addr learned my addr 172.21.15.73:0/2385404448 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8451410700 1 --2- 172.21.15.73:0/2385404448 >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 0x7f844c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8450c0f700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c098d10 0x7f844c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 -- 172.21.15.73:0/2385404448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c098d10 msgr2=0x7f844c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c098d10 0x7f844c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 -- 172.21.15.73:0/2385404448 >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 msgr2=0x7f844c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:18.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 --2- 172.21.15.73:0/2385404448 >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 0x7f844c158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 -- 172.21.15.73:0/2385404448 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f844c0c69a0 con 0x7f844c0c6190 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f8451410700 1 --2- 172.21.15.73:0/2385404448 >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 0x7f844c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844bfff700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c0c6190 0x7f844c063e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f844c084570 tx=0x7f843c009840).ready entity=mon.0 client_cookie=c4c4e2af20ec6de9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844b7fe700 1 -- 172.21.15.73:0/2385404448 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c00e070 con 0x7f844c0c6190 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844b7fe700 1 -- 172.21.15.73:0/2385404448 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f843c00ba50 con 0x7f844c0c6190 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.684+0000 7f844b7fe700 1 -- 172.21.15.73:0/2385404448 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f843c00bdd0 con 0x7f844c0c6190 2023-11-28T02:53:18.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 -- 172.21.15.73:0/2385404448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c0c6190 msgr2=0x7f844c063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c0c6190 0x7f844c063e60 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 rx=0x7f844c084570 tx=0x7f843c009840).stop 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 -- 172.21.15.73:0/2385404448 shutdown_connections 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 --2- 172.21.15.73:0/2385404448 >> v2:172.21.15.150:3300/0 conn(0x7f844c064350 0x7f844c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c098d10 0x7f844c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 --2- 172.21.15.73:0/2385404448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c0c6190 0x7f844c063e60 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 -- 172.21.15.73:0/2385404448 >> 172.21.15.73:0/2385404448 conn(0x7f844c0cb310 msgr2=0x7f844c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 -- 172.21.15.73:0/2385404448 shutdown_connections 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.685+0000 7f8452e73700 1 -- 172.21.15.73:0/2385404448 wait complete. 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.686+0000 7f8452e73700 1 Processor -- start 2023-11-28T02:53:18.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.686+0000 7f8452e73700 1 -- start start 2023-11-28T02:53:18.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 0x7f844c0b06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c0c6190 0x7f844c0b0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0c46d0 con 0x7f844c064350 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0aeb80 con 0x7f844c098d10 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8452e73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f844c0aee60 con 0x7f844c0c6190 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52160/0 (socket says 172.21.15.73:52160) 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 -- 172.21.15.73:0/396955570 learned_addr learned my addr 172.21.15.73:0/396955570 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f844bfff700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 0x7f844c0b06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8451410700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c0c6190 0x7f844c0b0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 -- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c0c6190 msgr2=0x7f844c0b0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c0c6190 0x7f844c0b0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 -- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 msgr2=0x7f844c0b06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 0x7f844c0b06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 -- 172.21.15.73:0/396955570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f843c009ae0 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f844bfff700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 0x7f844c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.687+0000 7f8450c0f700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f844400d890 tx=0x7f8444008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.688+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84440125f0 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.688+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8444012750 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.688+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f844c0ada00 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.688+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8444012ad0 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.688+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f844c0b1080 con 0x7f844c064350 2023-11-28T02:53:18.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.689+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f844c098950 con 0x7f844c064350 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.689+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8444013480 con 0x7f844c064350 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.689+0000 7f8448ff9700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 0x7f842c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.689+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6207+0+0 (secure 0 0 0) 0x7f844407a420 con 0x7f844c064350 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.689+0000 7f844bfff700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 0x7f842c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.690+0000 7f844bfff700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 0x7f842c061610 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f844c084570 tx=0x7f843c009840).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:18.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.693+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f844404b640 con 0x7f844c064350 2023-11-28T02:53:18.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:18.897+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f844c06c860 con 0x7f844c064350 2023-11-28T02:53:19.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.559+0000 7f8448ff9700 1 -- 172.21.15.73:0/396955570 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f844404ec60 con 0x7f844c064350 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 msgr2=0x7f842c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 0x7f842c061610 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f844c084570 tx=0x7f843c009840).stop 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 msgr2=0x7f844c148340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7f844400d890 tx=0x7f8444008040).stop 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 shutdown_connections 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f842c05f150 0x7f842c061610 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f844c0c6190 0x7f844c0b0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f844c064350 0x7f844c148340 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 --2- 172.21.15.73:0/396955570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f844c098d10 0x7f844c0b06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 >> 172.21.15.73:0/396955570 conn(0x7f844c0cb310 msgr2=0x7f844c148c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:19.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 shutdown_connections 2023-11-28T02:53:19.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.561+0000 7f8452e73700 1 -- 172.21.15.73:0/396955570 wait complete. 2023-11-28T02:53:19.576 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T02:53:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.699+0000 7faacaccb700 1 Processor -- start 2023-11-28T02:53:19.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.699+0000 7faacaccb700 1 -- start start 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 0x7faac4154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac40a7ff0 0x7faac40c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac40aa220 0x7faac40aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40c6390 con 0x7faac40a7ff0 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac40c6630 con 0x7faac4154530 2023-11-28T02:53:19.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faacaccb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac40aaba0 con 0x7faac40aa220 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac40a7ff0 0x7faac40c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac40a7ff0 0x7faac40c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52172/0 (socket says 172.21.15.73:52172) 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac3fff700 1 -- 172.21.15.73:0/3853445890 learned_addr learned my addr 172.21.15.73:0/3853445890 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac3fff700 1 -- 172.21.15.73:0/3853445890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac40aa220 msgr2=0x7faac40aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac8a67700 1 --2- 172.21.15.73:0/3853445890 >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 0x7faac4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac9268700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac40aa220 0x7faac40aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.700+0000 7faac3fff700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac40aa220 0x7faac40aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac3fff700 1 -- 172.21.15.73:0/3853445890 >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 msgr2=0x7faac4154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac3fff700 1 --2- 172.21.15.73:0/3853445890 >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 0x7faac4154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac3fff700 1 -- 172.21.15.73:0/3853445890 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac40a8470 con 0x7faac40a7ff0 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac8a67700 1 --2- 172.21.15.73:0/3853445890 >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 0x7faac4154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac3fff700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40a7ff0 0x7faac40c8580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7faac40c9080 tx=0x7faab80090f0).ready entity=mon.0 client_cookie=14a85f35a1643cce server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:19.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac37fe700 1 -- 172.21.15.73:0/3853445890 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800b600 con 0x7faac40a7ff0 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac37fe700 1 -- 172.21.15.73:0/3853445890 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800b760 con 0x7faac40a7ff0 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faac37fe700 1 -- 172.21.15.73:0/3853445890 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800bae0 con 0x7faac40a7ff0 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 -- 172.21.15.73:0/3853445890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40a7ff0 msgr2=0x7faac40c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40a7ff0 0x7faac40c8580 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 rx=0x7faac40c9080 tx=0x7faab80090f0).stop 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 -- 172.21.15.73:0/3853445890 shutdown_connections 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac40aa220 0x7faac40aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 --2- 172.21.15.73:0/3853445890 >> v2:172.21.15.150:3300/0 conn(0x7faac4154530 0x7faac4154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 --2- 172.21.15.73:0/3853445890 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40a7ff0 0x7faac40c8580 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 -- 172.21.15.73:0/3853445890 >> 172.21.15.73:0/3853445890 conn(0x7faac4094930 msgr2=0x7faac406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 -- 172.21.15.73:0/3853445890 shutdown_connections 2023-11-28T02:53:19.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.701+0000 7faacaccb700 1 -- 172.21.15.73:0/3853445890 wait complete. 2023-11-28T02:53:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.702+0000 7faacaccb700 1 Processor -- start 2023-11-28T02:53:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 -- start start 2023-11-28T02:53:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac40a7ff0 0x7faac414dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 0x7faac414d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac40cab60 con 0x7faac40aa220 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac4151650 con 0x7faac40a7ff0 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faacaccb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac4151890 con 0x7faac4154530 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faac3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faac3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52176/0 (socket says 172.21.15.73:52176) 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faac3fff700 1 -- 172.21.15.73:0/2019701360 learned_addr learned my addr 172.21.15.73:0/2019701360 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.703+0000 7faac8a67700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac40a7ff0 0x7faac414dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac9268700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 0x7faac414d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 -- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 msgr2=0x7faac414d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 0x7faac414d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 -- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac40a7ff0 msgr2=0x7faac414dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:19.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac40a7ff0 0x7faac414dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 -- 172.21.15.73:0/2019701360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab8009370 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac9268700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 0x7faac414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac3fff700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7faab8009130 tx=0x7faab8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab800b8d0 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab800ba70 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac40abe10 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab801c670 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.704+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac40ac3d0 con 0x7faac40aa220 2023-11-28T02:53:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.705+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faac40a93a0 con 0x7faac40aa220 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.705+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faab801c9e0 con 0x7faac40aa220 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.706+0000 7faac0ff9700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 0x7faab0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.706+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6207+0+0 (secure 0 0 0) 0x7faab807c020 con 0x7faac40aa220 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.706+0000 7faac8a67700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 0x7faab0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.706+0000 7faac8a67700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 0x7faab0061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7faab400d650 tx=0x7faab4007800).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.710+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faab804d2a0 con 0x7faac40aa220 2023-11-28T02:53:19.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:19.875+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7faac4084570 con 0x7faac40aa220 2023-11-28T02:53:20.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.565+0000 7faac0ff9700 1 -- 172.21.15.73:0/2019701360 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7faab802a070 con 0x7faac40aa220 2023-11-28T02:53:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 msgr2=0x7faab0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 0x7faab0061610 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 rx=0x7faab400d650 tx=0x7faab4007800).stop 2023-11-28T02:53:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 msgr2=0x7faac4151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 rx=0x7faab8009130 tx=0x7faab8017000).stop 2023-11-28T02:53:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 shutdown_connections 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faab005f150 0x7faab0061610 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac4154530 0x7faac414d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.567+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac40a7ff0 0x7faac414dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.568+0000 7faacaccb700 1 --2- 172.21.15.73:0/2019701360 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac40aa220 0x7faac4151160 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.568+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 >> 172.21.15.73:0/2019701360 conn(0x7faac4094930 msgr2=0x7faac414ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.568+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 shutdown_connections 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.568+0000 7faacaccb700 1 -- 172.21.15.73:0/2019701360 wait complete. 2023-11-28T02:53:20.569 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T02:53:20.581 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T02:53:20.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:53:20.626 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:53:20.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.701+0000 7f4e7ae0d700 1 Processor -- start 2023-11-28T02:53:20.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 -- start start 2023-11-28T02:53:20.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 0x7f4e741582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e740aa1c0 0x7f4e740aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e74159210 con 0x7f4e740ab3b0 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e740c0c70 con 0x7f4e740aa1c0 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.702+0000 7f4e7ae0d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e740c2110 con 0x7f4e74154610 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e740aa1c0 0x7f4e740aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4e740aa1c0 0x7f4e740aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52190/0 (socket says 172.21.15.73:52190) 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 -- 172.21.15.73:0/2496922131 learned_addr learned my addr 172.21.15.73:0/2496922131 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e78ba9700 1 --2- 172.21.15.73:0/2496922131 >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 0x7f4e741582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 -- 172.21.15.73:0/2496922131 >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 msgr2=0x7f4e741582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 --2- 172.21.15.73:0/2496922131 >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 0x7f4e741582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 -- 172.21.15.73:0/2496922131 >> v1:172.21.15.150:6789/0 conn(0x7f4e740ab3b0 legacy=0x7f4e740c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 -- 172.21.15.73:0/2496922131 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e740aaaf0 con 0x7f4e740aa1c0 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e78ba9700 1 --2- 172.21.15.73:0/2496922131 >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 0x7f4e741582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:20.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e793aa700 1 --2- 172.21.15.73:0/2496922131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e740aa600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4e74062e70 tx=0x7f4e6c0090f0).ready entity=mon.0 client_cookie=27b0afd1e30bf4b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/2496922131 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00b600 con 0x7f4e740aa1c0 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/2496922131 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e6c00b760 con 0x7f4e740aa1c0 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.703+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/2496922131 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e6c00bae0 con 0x7f4e740aa1c0 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/2496922131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 msgr2=0x7f4e740aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/2496922131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e740aa600 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7f4e74062e70 tx=0x7f4e6c0090f0).stop 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/2496922131 shutdown_connections 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/2496922131 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e740aa600 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/2496922131 >> v2:172.21.15.150:3300/0 conn(0x7f4e74154610 0x7f4e741582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/2496922131 >> 172.21.15.73:0/2496922131 conn(0x7f4e740655a0 msgr2=0x7f4e74069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/2496922131 shutdown_connections 2023-11-28T02:53:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.704+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/2496922131 wait complete. 2023-11-28T02:53:20.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.705+0000 7f4e7ae0d700 1 Processor -- start 2023-11-28T02:53:20.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.705+0000 7f4e7ae0d700 1 -- start start 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e74154610 0x7f4e740abf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 0x7f4e74156170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74169040 con 0x7f4e740aa1c0 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e74168bb0 con 0x7f4e740ac430 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e740ac8b0 con 0x7f4e74154610 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52196/0 (socket says 172.21.15.73:52196) 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 -- 172.21.15.73:0/50139454 learned_addr learned my addr 172.21.15.73:0/50139454 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:20.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e793aa700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 0x7f4e74156170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e74154610 0x7f4e740abf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 -- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e74154610 msgr2=0x7f4e740abf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e74154610 0x7f4e740abf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 -- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 msgr2=0x7f4e74156170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 0x7f4e74156170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 -- 172.21.15.73:0/50139454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c009370 con 0x7f4e740aa1c0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e793aa700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 0x7f4e74156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e78ba9700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4e7000d1b0 tx=0x7f4e700099f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e7000ddf0 con 0x7f4e740aa1c0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e7000f040 con 0x7f4e740aa1c0 2023-11-28T02:53:20.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e740c2aa0 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e70013600 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.706+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e74168560 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.707+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e7406c860 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.708+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4e700139f0 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.708+0000 7f4e68ff9700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 0x7f4e60061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.708+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5769+0+0 (secure 0 0 0) 0x7f4e7007a780 con 0x7f4e740aa1c0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.708+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 0x7f4e60061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.708+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 0x7f4e60061610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4e74168060 tx=0x7f4e64009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:20.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.712+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e7004bb50 con 0x7f4e740aa1c0 2023-11-28T02:53:20.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:20.881+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4e74158970 con 0x7f4e740aa1c0 2023-11-28T02:53:21.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.565+0000 7f4e68ff9700 1 -- 172.21.15.73:0/50139454 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7f4e7004f170 con 0x7f4e740aa1c0 2023-11-28T02:53:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 msgr2=0x7f4e60061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 0x7f4e60061610 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f4e74168060 tx=0x7f4e64009040).stop 2023-11-28T02:53:21.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 msgr2=0x7f4e74144f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7f4e7000d1b0 tx=0x7f4e700099f0).stop 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 shutdown_connections 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e6005f150 0x7f4e60061610 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e740ac430 0x7f4e74156170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e740aa1c0 0x7f4e74144f90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 --2- 172.21.15.73:0/50139454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e74154610 0x7f4e740abf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 >> 172.21.15.73:0/50139454 conn(0x7f4e740655a0 msgr2=0x7f4e7414eea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 shutdown_connections 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.568+0000 7f4e7ae0d700 1 -- 172.21.15.73:0/50139454 wait complete. 2023-11-28T02:53:21.569 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T02:53:21.582 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T02:53:21.583 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T02:53:21.583 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T02:53:21.583 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T02:53:21.583 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T02:53:21.583 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T02:53:21.583 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T02:53:21.583 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T02:53:21.584 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T02:53:21.584 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T02:53:21.584 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T02:53:21.584 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T02:53:21.584 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T02:53:21.584 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T02:53:21.584 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T02:53:21.585 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T02:53:21.595 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T02:53:21.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T02:53:21.596 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T02:53:21.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T02:53:21.631 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T02:53:21.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T02:53:21.632 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T02:53:21.632 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T02:53:21.638 INFO:tasks.ceph.mds.b:Started 2023-11-28T02:53:21.639 DEBUG:teuthology.parallel:result is None 2023-11-28T02:53:21.639 INFO:tasks.ceph.mds.d:Started 2023-11-28T02:53:21.639 DEBUG:teuthology.parallel:result is None 2023-11-28T02:53:21.673 INFO:tasks.ceph.mds.a:Started 2023-11-28T02:53:21.673 DEBUG:teuthology.parallel:result is None 2023-11-28T02:53:21.674 INFO:tasks.ceph.mds.c:Started 2023-11-28T02:53:21.674 DEBUG:teuthology.parallel:result is None 2023-11-28T02:53:21.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T02:53:21.720 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T02:53:21.721 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T02:53:21.723 INFO:tasks.ceph.mds.b.smithi150.stderr:2023-11-28T02:53:21.718+0000 7ff9480d0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-28T02:53:21.752 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T02:53:21.752 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T02:53:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.805+0000 7f855d5e4700 1 Processor -- start 2023-11-28T02:53:21.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.805+0000 7f855d5e4700 1 -- start start 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f855d5e4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f85581502a0 0x7f8558150680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f855d5e4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8558069f00 con 0x7f8558068c30 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f855d5e4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85580692f0 con 0x7f85580bc440 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f855d5e4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85580c29a0 con 0x7f85581502a0 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f85577fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8558068c30 0x7f85580c2290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54900/0 (socket says 172.21.15.73:54900) 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f85577fe700 1 -- 172.21.15.73:0/3571873314 learned_addr learned my addr 172.21.15.73:0/3571873314 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8557fff700 1 --2- 172.21.15.73:0/3571873314 >> v2:172.21.15.150:3300/0 conn(0x7f85581502a0 0x7f8558150680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3000803031 0 0) 0x7f8558069f00 con 0x7f8558068c30 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85400033f0 con 0x7f8558068c30 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2970117188 0 0) 0x7f85580692f0 con 0x7f85580bc440 2023-11-28T02:53:21.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8558069f00 con 0x7f85580bc440 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8557fff700 1 -- 172.21.15.73:0/3571873314 >> v1:172.21.15.150:6789/0 conn(0x7f85580bc440 legacy=0x7f85580bc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8557fff700 1 -- 172.21.15.73:0/3571873314 >> v1:172.21.15.73:6789/0 conn(0x7f8558068c30 legacy=0x7f85580c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.806+0000 7f8557fff700 1 -- 172.21.15.73:0/3571873314 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85580695d0 con 0x7f85581502a0 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f8557fff700 1 --2- 172.21.15.73:0/3571873314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85581502a0 0x7f8558150680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8548009760 tx=0x7f8548009060).ready entity=mon.1 client_cookie=3d420cba819e2e7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480108a0 con 0x7f85581502a0 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8548010a00 con 0x7f85581502a0 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f8556ffd700 1 -- 172.21.15.73:0/3571873314 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548010b70 con 0x7f85581502a0 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 -- 172.21.15.73:0/3571873314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85581502a0 msgr2=0x7f8558150680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 --2- 172.21.15.73:0/3571873314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85581502a0 0x7f8558150680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 rx=0x7f8548009760 tx=0x7f8548009060).stop 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 -- 172.21.15.73:0/3571873314 shutdown_connections 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 --2- 172.21.15.73:0/3571873314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85581502a0 0x7f8558150680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 -- 172.21.15.73:0/3571873314 >> 172.21.15.73:0/3571873314 conn(0x7f85580c6c50 msgr2=0x7f85580c7060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:21.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 -- 172.21.15.73:0/3571873314 shutdown_connections 2023-11-28T02:53:21.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.807+0000 7f855d5e4700 1 -- 172.21.15.73:0/3571873314 wait complete. 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.808+0000 7f855d5e4700 1 Processor -- start 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 -- start start 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85580bc440 0x7f85580abd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85581502a0 0x7f85580ac220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8558160af0 con 0x7f8558068c30 2023-11-28T02:53:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85580ac710 con 0x7f85580bc440 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.809+0000 7f855d5e4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85580bd6a0 con 0x7f85581502a0 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52230/0 (socket says 172.21.15.73:52230) 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 -- 172.21.15.73:0/2095028858 learned_addr learned my addr 172.21.15.73:0/2095028858 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f85577fe700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85580bc440 0x7f85580abd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f855cde3700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85581502a0 0x7f85580ac220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 -- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85581502a0 msgr2=0x7f85580ac220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85581502a0 0x7f85580ac220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 -- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85580bc440 msgr2=0x7f85580abd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85580bc440 0x7f85580abd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 -- 172.21.15.73:0/2095028858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85480092a0 con 0x7f8558068c30 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f8557fff700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8558068240 tx=0x7f8548018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548020070 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854801b080 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480108a0 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85580bd980 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.810+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85580bdf40 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.811+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855814f950 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.812+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8548016020 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.812+0000 7f853ffff700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 0x7f8544061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.812+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(29..29 src has 1..29) v4 ==== 5355+0+0 (secure 0 0 0) 0x7f854807e360 con 0x7f8558068c30 2023-11-28T02:53:21.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.812+0000 7f85577fe700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 0x7f8544061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:21.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.812+0000 7f85577fe700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 0x7f8544061610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f85580bfef0 tx=0x7f854c009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:21.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:21.816+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f854804f930 con 0x7f8558068c30 2023-11-28T02:53:22.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.000+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f855806c7c0 con 0x7f8558068c30 2023-11-28T02:53:22.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.791+0000 7f853ffff700 1 -- 172.21.15.73:0/2095028858 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v30) v1 ==== 97+0+0 (secure 0 0 0) 0x7f8548052f50 con 0x7f8558068c30 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 msgr2=0x7f8544061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 0x7f8544061610 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 rx=0x7f85580bfef0 tx=0x7f854c009040).stop 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 msgr2=0x7f8558149540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 rx=0x7f8558068240 tx=0x7f8548018000).stop 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 shutdown_connections 2023-11-28T02:53:22.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f854405f150 0x7f8544061610 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85581502a0 0x7f85580ac220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8558068c30 0x7f8558149540 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 --2- 172.21.15.73:0/2095028858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85580bc440 0x7f85580abd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 >> 172.21.15.73:0/2095028858 conn(0x7f85580c6c50 msgr2=0x7f85580c14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 shutdown_connections 2023-11-28T02:53:22.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.793+0000 7f855d5e4700 1 -- 172.21.15.73:0/2095028858 wait complete. 2023-11-28T02:53:22.797 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T02:53:22.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T02:53:22.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.920+0000 7f1da5a8b700 1 Processor -- start 2023-11-28T02:53:22.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.920+0000 7f1da5a8b700 1 -- start start 2023-11-28T02:53:22.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1da5a8b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1da00caab0 0x7f1da00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1da5a8b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1da00ca180 0x7f1da015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1da5a8b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1da00a9a90 con 0x7f1da00cae90 2023-11-28T02:53:22.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1da5a8b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da0154ce0 con 0x7f1da00ca180 2023-11-28T02:53:22.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1da5a8b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1da00a8e80 con 0x7f1da00caab0 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1da00ca180 0x7f1da015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1da00ca180 0x7f1da015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52242/0 (socket says 172.21.15.73:52242) 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 -- 172.21.15.73:0/3884417592 learned_addr learned my addr 172.21.15.73:0/3884417592 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9f7fe700 1 --2- 172.21.15.73:0/3884417592 >> v2:172.21.15.150:3300/0 conn(0x7f1da00caab0 0x7f1da00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 -- 172.21.15.73:0/3884417592 >> v2:172.21.15.150:3300/0 conn(0x7f1da00caab0 msgr2=0x7f1da00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 --2- 172.21.15.73:0/3884417592 >> v2:172.21.15.150:3300/0 conn(0x7f1da00caab0 0x7f1da00aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 -- 172.21.15.73:0/3884417592 >> v1:172.21.15.150:6789/0 conn(0x7f1da00cae90 legacy=0x7f1da00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.921+0000 7f1d9ffff700 1 -- 172.21.15.73:0/3884417592 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1da00a9160 con 0x7f1da00ca180 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1d9ffff700 1 --2- 172.21.15.73:0/3884417592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00ca180 0x7f1da015e3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1da00848c0 tx=0x7f1d98009840).ready entity=mon.0 client_cookie=705a5223b225a440 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:22.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1d9e7fc700 1 -- 172.21.15.73:0/3884417592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800e070 con 0x7f1da00ca180 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1d9e7fc700 1 -- 172.21.15.73:0/3884417592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9800ba20 con 0x7f1da00ca180 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1d9e7fc700 1 -- 172.21.15.73:0/3884417592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9800bda0 con 0x7f1da00ca180 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 -- 172.21.15.73:0/3884417592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00ca180 msgr2=0x7f1da015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/3884417592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00ca180 0x7f1da015e3c0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 rx=0x7f1da00848c0 tx=0x7f1d98009840).stop 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 -- 172.21.15.73:0/3884417592 shutdown_connections 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/3884417592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00ca180 0x7f1da015e3c0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/3884417592 >> v2:172.21.15.150:3300/0 conn(0x7f1da00caab0 0x7f1da00aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 -- 172.21.15.73:0/3884417592 >> 172.21.15.73:0/3884417592 conn(0x7f1da0094920 msgr2=0x7f1da006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:22.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 -- 172.21.15.73:0/3884417592 shutdown_connections 2023-11-28T02:53:22.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.922+0000 7f1da5a8b700 1 -- 172.21.15.73:0/3884417592 wait complete. 2023-11-28T02:53:22.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.923+0000 7f1da5a8b700 1 Processor -- start 2023-11-28T02:53:22.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 -- start start 2023-11-28T02:53:22.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da00ca180 0x7f1da0153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 0x7f1da00c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c32e0 con 0x7f1da00cae90 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c3580 con 0x7f1da00caab0 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1da5a8b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1da00c3860 con 0x7f1da00ca180 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1d9ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52256/0 (socket says 172.21.15.73:52256) 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1d9ffff700 1 -- 172.21.15.73:0/2310587739 learned_addr learned my addr 172.21.15.73:0/2310587739 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1d9effd700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 0x7f1da00c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.924+0000 7f1d9f7fe700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da00ca180 0x7f1da0153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 -- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da00ca180 msgr2=0x7f1da0153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da00ca180 0x7f1da0153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 -- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 msgr2=0x7f1da00c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 0x7f1da00c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 -- 172.21.15.73:0/2310587739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d98009ae0 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9effd700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 0x7f1da00c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d9ffff700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f1d9800b6c0 tx=0x7f1d98009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d980092e0 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d9801e470 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1da00c3b40 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d9801e830 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.925+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1da00c4100 con 0x7f1da00cae90 2023-11-28T02:53:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.926+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1da00c4470 con 0x7f1da00cae90 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.926+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1d98003040 con 0x7f1da00cae90 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.927+0000 7f1d877fe700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 0x7f1d8c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.927+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1d9807b2a0 con 0x7f1da00cae90 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.927+0000 7f1d9f7fe700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 0x7f1d8c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.927+0000 7f1d9f7fe700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 0x7f1d8c061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1d90006fd0 tx=0x7f1d90006f40).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:22.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:22.931+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1d9804c970 con 0x7f1da00cae90 2023-11-28T02:53:23.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.154+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1da006c860 con 0x7f1da00cae90 2023-11-28T02:53:23.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.155+0000 7f1d877fe700 1 -- 172.21.15.73:0/2310587739 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f1d98023070 con 0x7f1da00cae90 2023-11-28T02:53:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:23.156 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T02:53:23.156 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.157 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T02:53:23.158 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.159 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.160 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T02:53:23.161 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.162 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.163 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T02:53:23.164 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.165 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.166 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T02:53:23.167 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.168 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.169 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mds", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVlLXXuERAAVMPrP9scOz3AwhWjnGMVGw==", 2023-11-28T02:53:23.170 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mds" 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-mgr", 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVlAn/uERAA2UUZoQ5cvYW9oPo0dNExiQ==", 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-mgr" 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.171 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-osd", 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVlW4buERAAhCmBv5+K5rr+cXx4vG6gdw==", 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-osd" 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd", 2023-11-28T02:53:23.173 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVlsY3uERAAnsftgidaPcySsQQYK0aL5A==", 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd" 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVlNZXuERAAFFI7XEDibRqPuogNO5gOUw==", 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.174 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.bootstrap-rgw", 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDqVWVl85zuERAAZqdFuHZVch6vdl7ZYlua9w==", 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile bootstrap-rgw" 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T02:53:23.175 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T02:53:23.176 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T02:53:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T02:53:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:23.177 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T02:53:23.177 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 msgr2=0x7f1d8c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 0x7f1d8c061610 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f1d90006fd0 tx=0x7f1d90006f40).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 msgr2=0x7f1da00c7550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 rx=0x7f1d9800b6c0 tx=0x7f1d98009840).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 shutdown_connections 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1d8c05f150 0x7f1d8c061610 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1da00cae90 0x7f1da00c7550 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1da00ca180 0x7f1da0153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 --2- 172.21.15.73:0/2310587739 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1da00caab0 0x7f1da00c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 >> 172.21.15.73:0/2310587739 conn(0x7f1da0094920 msgr2=0x7f1da00b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:23.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 shutdown_connections 2023-11-28T02:53:23.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.157+0000 7f1da5a8b700 1 -- 172.21.15.73:0/2310587739 wait complete. 2023-11-28T02:53:23.180 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-11-28T02:53:23.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.292+0000 7f8202b58700 1 Processor -- start 2023-11-28T02:53:23.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.292+0000 7f8202b58700 1 -- start start 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202b58700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81fc0c66f0 0x7f81fc154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202b58700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81fc158570 0x7f81fc158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202b58700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81fc154e40 con 0x7f81fc14e530 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202b58700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0c6ba0 con 0x7f81fc158570 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202b58700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81fc0c7bc0 con 0x7f81fc0c66f0 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8201b56700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f81fc14e530 0x7f81fc14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54906/0 (socket says 172.21.15.73:54906) 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81fc158570 0x7f81fc158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8201b56700 1 -- 172.21.15.73:0/678360802 learned_addr learned my addr 172.21.15.73:0/678360802 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:23.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8201355700 1 --2- 172.21.15.73:0/678360802 >> v2:172.21.15.150:3300/0 conn(0x7f81fc0c66f0 0x7f81fc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 -- 172.21.15.73:0/678360802 >> v2:172.21.15.150:3300/0 conn(0x7f81fc0c66f0 msgr2=0x7f81fc154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 --2- 172.21.15.73:0/678360802 >> v2:172.21.15.150:3300/0 conn(0x7f81fc0c66f0 0x7f81fc154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 -- 172.21.15.73:0/678360802 >> v1:172.21.15.73:6789/0 conn(0x7f81fc14e530 legacy=0x7f81fc14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 -- 172.21.15.73:0/678360802 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81fc155920 con 0x7f81fc158570 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8200b54700 1 -- 172.21.15.73:0/678360802 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3123347517 0 0) 0x7f81f8001110 con 0x7f81fc14e530 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8202357700 1 --2- 172.21.15.73:0/678360802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc158570 0x7f81fc158a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f81fc0ab0a0 tx=0x7f81f4009780).ready entity=mon.0 client_cookie=d9faa88a46b7495 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8200b54700 1 -- 172.21.15.73:0/678360802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f400e070 con 0x7f81fc158570 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8200b54700 1 -- 172.21.15.73:0/678360802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81f400bc30 con 0x7f81fc158570 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.293+0000 7f8200b54700 1 -- 172.21.15.73:0/678360802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81f4012410 con 0x7f81fc158570 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 -- 172.21.15.73:0/678360802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc158570 msgr2=0x7f81fc158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 --2- 172.21.15.73:0/678360802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc158570 0x7f81fc158a00 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 rx=0x7f81fc0ab0a0 tx=0x7f81f4009780).stop 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 -- 172.21.15.73:0/678360802 shutdown_connections 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 --2- 172.21.15.73:0/678360802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc158570 0x7f81fc158a00 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 --2- 172.21.15.73:0/678360802 >> v2:172.21.15.150:3300/0 conn(0x7f81fc0c66f0 0x7f81fc154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 -- 172.21.15.73:0/678360802 >> 172.21.15.73:0/678360802 conn(0x7f81fc064990 msgr2=0x7f81fc064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:23.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 -- 172.21.15.73:0/678360802 shutdown_connections 2023-11-28T02:53:23.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.294+0000 7f8202b58700 1 -- 172.21.15.73:0/678360802 wait complete. 2023-11-28T02:53:23.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.295+0000 7f8202b58700 1 Processor -- start 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.295+0000 7f8202b58700 1 -- start start 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81fc0c66f0 0x7f81fc14dc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81fc158570 0x7f81fc14d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c32f0 con 0x7f81fc14e530 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c26e0 con 0x7f81fc158570 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202b58700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81fc0c29c0 con 0x7f81fc0c66f0 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52280/0 (socket says 172.21.15.73:52280) 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 -- 172.21.15.73:0/1725799303 learned_addr learned my addr 172.21.15.73:0/1725799303 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201b56700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81fc0c66f0 0x7f81fc14dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8202357700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81fc158570 0x7f81fc14d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 -- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81fc0c66f0 msgr2=0x7f81fc14dc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81fc0c66f0 0x7f81fc14dc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 -- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81fc158570 msgr2=0x7f81fc14d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81fc158570 0x7f81fc14d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 -- 172.21.15.73:0/1725799303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f4009a20 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.296+0000 7f8201355700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f81ec00d890 tx=0x7f81ec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.297+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec011590 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.297+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec0116f0 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.297+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81fc0c2ca0 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.297+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec011a70 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.297+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81fc14e160 con 0x7f81fc14e530 2023-11-28T02:53:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.298+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81fc062ee0 con 0x7f81fc14e530 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.298+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f81ec012420 con 0x7f81fc14e530 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.298+0000 7f81f1ffb700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 0x7f81e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.298+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f81ec079790 con 0x7f81fc14e530 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.298+0000 7f8201b56700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 0x7f81e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.299+0000 7f8201b56700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 0x7f81e0061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f81f80075c0 tx=0x7f81f8007a10).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.302+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81ec04ad50 con 0x7f81fc14e530 2023-11-28T02:53:23.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.530+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f81fc06c7c0 con 0x7f81fc14e530 2023-11-28T02:53:23.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.534+0000 7f81f1ffb700 1 -- 172.21.15.73:0/1725799303 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 updated v3) v1 ==== 97+0+0 (secure 0 0 0) 0x7f81ec011e60 con 0x7f81fc14e530 2023-11-28T02:53:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.535+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 msgr2=0x7f81e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.535+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 0x7f81e0061610 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7f81f80075c0 tx=0x7f81f8007a10).stop 2023-11-28T02:53:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.535+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 msgr2=0x7f81fc1570d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.535+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 rx=0x7f81ec00d890 tx=0x7f81ec008040).stop 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 shutdown_connections 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81e005f150 0x7f81e0061610 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81fc158570 0x7f81fc14d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81fc0c66f0 0x7f81fc14dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 --2- 172.21.15.73:0/1725799303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81fc14e530 0x7f81fc1570d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 >> 172.21.15.73:0/1725799303 conn(0x7f81fc064990 msgr2=0x7f81fc0c0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 shutdown_connections 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.536+0000 7f8202b58700 1 -- 172.21.15.73:0/1725799303 wait complete. 2023-11-28T02:53:23.538 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:23.549 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-11-28T02:53:23.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.661+0000 7f13fffff700 1 Processor -- start 2023-11-28T02:53:23.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.661+0000 7f13fffff700 1 -- start start 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14000c66f0 0x7f1400154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f1400158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fffff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1400154e40 con 0x7f140014e530 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fffff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14000c6ba0 con 0x7f14000c66f0 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c7bc0 con 0x7f1400158570 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14000c66f0 0x7f1400154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13feffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f140014e530 0x7f140014e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58838/0 (socket says 172.21.15.73:58838) 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f14000c66f0 0x7f1400154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45568/0 (socket says 172.21.15.73:45568) 2023-11-28T02:53:23.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13feffd700 1 -- 172.21.15.73:0/1751861125 learned_addr learned my addr 172.21.15.73:0/1751861125 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13ff7fe700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f1400158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 -- 172.21.15.73:0/1751861125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 msgr2=0x7f1400158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f1400158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 -- 172.21.15.73:0/1751861125 >> v1:172.21.15.73:6789/0 conn(0x7f140014e530 legacy=0x7f140014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13fe7fc700 1 -- 172.21.15.73:0/1751861125 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1050805237 0 0) 0x7f13f0001110 con 0x7f140014e530 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13f67fc700 1 -- 172.21.15.73:0/1751861125 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1400155920 con 0x7f14000c66f0 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.662+0000 7f13ff7fe700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f1400158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13f67fc700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f1400154950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f13e8009760 tx=0x7f13e8009060).ready entity=mon.0 client_cookie=6307bfc9976547d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fe7fc700 1 -- 172.21.15.73:0/1751861125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800b580 con 0x7f14000c66f0 2023-11-28T02:53:23.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fe7fc700 1 -- 172.21.15.73:0/1751861125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e800b6e0 con 0x7f14000c66f0 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fe7fc700 1 -- 172.21.15.73:0/1751861125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e800ba60 con 0x7f14000c66f0 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 -- 172.21.15.73:0/1751861125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 msgr2=0x7f1400154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f1400154950 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 rx=0x7f13e8009760 tx=0x7f13e8009060).stop 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 -- 172.21.15.73:0/1751861125 shutdown_connections 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f1400158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 --2- 172.21.15.73:0/1751861125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f1400154950 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 -- 172.21.15.73:0/1751861125 >> 172.21.15.73:0/1751861125 conn(0x7f1400064990 msgr2=0x7f1400064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 -- 172.21.15.73:0/1751861125 shutdown_connections 2023-11-28T02:53:23.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.663+0000 7f13fffff700 1 -- 172.21.15.73:0/1751861125 wait complete. 2023-11-28T02:53:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.664+0000 7f13fffff700 1 Processor -- start 2023-11-28T02:53:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.664+0000 7f13fffff700 1 -- start start 2023-11-28T02:53:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140014e530 0x7f1400153130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f14000c36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c3230 con 0x7f14000c66f0 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c2620 con 0x7f140014e530 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13fffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14000c2900 con 0x7f1400158570 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13feffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13feffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45574/0 (socket says 172.21.15.73:45574) 2023-11-28T02:53:23.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13feffd700 1 -- 172.21.15.73:0/516523842 learned_addr learned my addr 172.21.15.73:0/516523842 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:23.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13ff7fe700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f14000c36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.665+0000 7f13f67fc700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140014e530 0x7f1400153130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 -- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 msgr2=0x7f14000c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f14000c36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 -- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140014e530 msgr2=0x7f1400153130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140014e530 0x7f1400153130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 -- 172.21.15.73:0/516523842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13e80092a0 con 0x7f14000c66f0 2023-11-28T02:53:23.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13feffd700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f13f000c510 tx=0x7f13f0007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f000ce80 con 0x7f14000c66f0 2023-11-28T02:53:23.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13f00072e0 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14000c2be0 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13f0007660 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.666+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f140014e2c0 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.667+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14000c3510 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.667+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f13f00059b0 con 0x7f14000c66f0 2023-11-28T02:53:23.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.667+0000 7f13f77fe700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 0x7f13e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:23.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.668+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f13f00794b0 con 0x7f14000c66f0 2023-11-28T02:53:23.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.668+0000 7f13f67fc700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 0x7f13e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:23.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.668+0000 7f13f67fc700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 0x7f13e0061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f13e800b340 tx=0x7f13e8017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:23.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.672+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f13f004aa10 con 0x7f14000c66f0 2023-11-28T02:53:23.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.898+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f1400154e40 con 0x7f14000c66f0 2023-11-28T02:53:23.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.901+0000 7f13f77fe700 1 -- 172.21.15.73:0/516523842 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 updated v4) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1400154e40 con 0x7f14000c66f0 2023-11-28T02:53:23.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.903+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 msgr2=0x7f13e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.903+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 0x7f13e0061610 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7f13e800b340 tx=0x7f13e8017000).stop 2023-11-28T02:53:23.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.903+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 msgr2=0x7f140014ddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:23.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.903+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 rx=0x7f13f000c510 tx=0x7f13f0007ad0).stop 2023-11-28T02:53:23.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 shutdown_connections 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f13e005f150 0x7f13e0061610 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1400158570 0x7f14000c36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f14000c66f0 0x7f140014ddd0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 --2- 172.21.15.73:0/516523842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f140014e530 0x7f1400153130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 >> 172.21.15.73:0/516523842 conn(0x7f1400064990 msgr2=0x7f1400153c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 shutdown_connections 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:23.904+0000 7f13fffff700 1 -- 172.21.15.73:0/516523842 wait complete. 2023-11-28T02:53:23.906 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:23.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-11-28T02:53:24.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.030+0000 7f9958c37700 1 Processor -- start 2023-11-28T02:53:24.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.030+0000 7f9958c37700 1 -- start start 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9958c37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f99540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9958c37700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9954158320 0x7f99541587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9958c37700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9954158cc0 con 0x7f9954098d10 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9958c37700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99540c72d0 con 0x7f9954158320 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9958c37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9954064570 con 0x7f99540c6190 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9954098d10 0x7f9954063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58848/0 (socket says 172.21.15.73:58848) 2023-11-28T02:53:24.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995259c700 1 -- 172.21.15.73:0/4172240928 learned_addr learned my addr 172.21.15.73:0/4172240928 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> v2:172.21.15.150:3300/0 conn(0x7f9954158320 0x7f99541587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9951d9b700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f99540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 169156766 0 0) 0x7f9944001490 con 0x7f9954098d10 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f993c0033f0 con 0x7f9954098d10 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2286487672 0 0) 0x7f993c0033f0 con 0x7f9954098d10 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 msgr2=0x7f99540c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f99540c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 >> v2:172.21.15.150:3300/0 conn(0x7f9954158320 msgr2=0x7f99541587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 --2- 172.21.15.73:0/4172240928 >> v2:172.21.15.150:3300/0 conn(0x7f9954158320 0x7f99541587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9954154bc0 con 0x7f9954098d10 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f9944001980 con 0x7f9954098d10 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> v2:172.21.15.150:3300/0 conn(0x7f9954158320 0x7f99541587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 >> v1:172.21.15.73:6789/0 conn(0x7f9954098d10 legacy=0x7f9954063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f993c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993c003e10 0x7f993c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f993c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540c72d0 con 0x7f993c004b80 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9954064570 con 0x7f993c003e10 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f993c00b8e0 con 0x7f99540c6190 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.031+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f993c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9951d9b700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993c003e10 0x7f993c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f995259c700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f993c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 -- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 msgr2=0x7f993c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540c6190 0x7f993c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 -- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993c003e10 msgr2=0x7f993c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f993c003e10 0x7f993c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 -- 172.21.15.73:0/4172240928 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f993c0033f0 con 0x7f993c004b80 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9952d9d700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f993c005000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f994c009ca0 tx=0x7f994c009060).ready entity=mon.0 client_cookie=a7c764d4883e9c37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c0095c0 con 0x7f993c004b80 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994c00e770 con 0x7f993c004b80 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f995159a700 1 -- 172.21.15.73:0/4172240928 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994c00eaf0 con 0x7f993c004b80 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9958c37700 1 -- 172.21.15.73:0/4172240928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 msgr2=0x7f993c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.032+0000 7f9958c37700 1 --2- 172.21.15.73:0/4172240928 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f993c005000 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 rx=0x7f994c009ca0 tx=0x7f994c009060).stop 2023-11-28T02:53:24.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.033+0000 7f9952d9d700 1 -- 172.21.15.73:0/4172240928 reap_dead start 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.033+0000 7f9958c37700 1 -- 172.21.15.73:0/4172240928 shutdown_connections 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.033+0000 7f9958c37700 1 -- 172.21.15.73:0/4172240928 >> 172.21.15.73:0/4172240928 conn(0x7f99540cb310 msgr2=0x7f99540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.033+0000 7f9958c37700 1 -- 172.21.15.73:0/4172240928 shutdown_connections 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.033+0000 7f9958c37700 1 -- 172.21.15.73:0/4172240928 wait complete. 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.034+0000 7f9958c37700 1 Processor -- start 2023-11-28T02:53:24.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.034+0000 7f9958c37700 1 -- start start 2023-11-28T02:53:24.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540ad8d0 0x7f99540aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99540abeb0 0x7f99540ac330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540aae10 con 0x7f993c004b80 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ab0f0 con 0x7f99540abeb0 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99540ab3d0 con 0x7f99540ad8d0 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45604/0 (socket says 172.21.15.73:45604) 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 -- 172.21.15.73:0/3552631897 learned_addr learned my addr 172.21.15.73:0/3552631897 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9951d9b700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540ad8d0 0x7f99540aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9952d9d700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99540abeb0 0x7f99540ac330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 -- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540ad8d0 msgr2=0x7f99540aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540ad8d0 0x7f99540aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 -- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99540abeb0 msgr2=0x7f99540ac330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99540abeb0 0x7f99540ac330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 -- 172.21.15.73:0/3552631897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99540ab6b0 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f995259c700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9944007710 tx=0x7f99540a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400f870 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f994400f9d0 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f994400fd70 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99540ab990 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.035+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99541516f0 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.036+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99540abd00 con 0x7f993c004b80 2023-11-28T02:53:24.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.037+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9944012560 con 0x7f993c004b80 2023-11-28T02:53:24.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.037+0000 7f99427fc700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 0x7f99340615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.037+0000 7f9951d9b700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 0x7f99340615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.037+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9944079660 con 0x7f993c004b80 2023-11-28T02:53:24.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.037+0000 7f9951d9b700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 0x7f99340615e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f99541567c0 tx=0x7f9948009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.041+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f994404acd0 con 0x7f993c004b80 2023-11-28T02:53:24.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.270+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f9954158cc0 con 0x7f993c004b80 2023-11-28T02:53:24.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.274+0000 7f99427fc700 1 -- 172.21.15.73:0/3552631897 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 updated v5) v1 ==== 97+0+0 (secure 0 0 0) 0x7f9954158cc0 con 0x7f993c004b80 2023-11-28T02:53:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 msgr2=0x7f99340615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 0x7f99340615e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 rx=0x7f99541567c0 tx=0x7f9948009060).stop 2023-11-28T02:53:24.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 msgr2=0x7f9954151040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 rx=0x7f9944007710 tx=0x7f99540a8040).stop 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 shutdown_connections 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f993405f120 0x7f99340615e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f99540abeb0 0x7f99540ac330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f993c004b80 0x7f9954151040 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 --2- 172.21.15.73:0/3552631897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f99540ad8d0 0x7f99540aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 >> 172.21.15.73:0/3552631897 conn(0x7f99540cb310 msgr2=0x7f99540aa5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 shutdown_connections 2023-11-28T02:53:24.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.276+0000 7f9958c37700 1 -- 172.21.15.73:0/3552631897 wait complete. 2023-11-28T02:53:24.279 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:24.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-11-28T02:53:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.400+0000 7fc42ae76700 1 Processor -- start 2023-11-28T02:53:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.400+0000 7fc42ae76700 1 -- start start 2023-11-28T02:53:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc42ae76700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc424098d10 0x7fc4240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc42ae76700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4240c72d0 con 0x7fc4240c6190 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc42ae76700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc424064570 con 0x7fc424158320 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc42ae76700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc424154bc0 con 0x7fc424098d10 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc428c12700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc424098d10 0x7fc4240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc41bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc4240c6190 0x7fc424063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58852/0 (socket says 172.21.15.73:58852) 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc428c12700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc424098d10 0x7fc4240c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45608/0 (socket says 172.21.15.73:45608) 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.401+0000 7fc41bfff700 1 -- 172.21.15.73:0/467688705 learned_addr learned my addr 172.21.15.73:0/467688705 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc428c12700 1 -- 172.21.15.73:0/467688705 >> v1:172.21.15.150:6789/0 conn(0x7fc424158320 legacy=0x7fc4240a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc428c12700 1 -- 172.21.15.73:0/467688705 >> v1:172.21.15.73:6789/0 conn(0x7fc4240c6190 legacy=0x7fc424063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc428c12700 1 -- 172.21.15.73:0/467688705 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4240c66c0 con 0x7fc424098d10 2023-11-28T02:53:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc41b7fe700 1 -- 172.21.15.73:0/467688705 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 368882584 0 0) 0x7fc4240c72d0 con 0x7fc4240c6190 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc428c12700 1 --2- 172.21.15.73:0/467688705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc4240c7760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fc420009e60 tx=0x7fc420009780).ready entity=mon.0 client_cookie=eb0c74535123dc3a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc41b7fe700 1 -- 172.21.15.73:0/467688705 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000e070 con 0x7fc424098d10 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc41b7fe700 1 -- 172.21.15.73:0/467688705 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000bbf0 con 0x7fc424098d10 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc41b7fe700 1 -- 172.21.15.73:0/467688705 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420012410 con 0x7fc424098d10 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 -- 172.21.15.73:0/467688705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 msgr2=0x7fc4240c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 --2- 172.21.15.73:0/467688705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc4240c7760 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 rx=0x7fc420009e60 tx=0x7fc420009780).stop 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 -- 172.21.15.73:0/467688705 shutdown_connections 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 --2- 172.21.15.73:0/467688705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc4240c7760 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 -- 172.21.15.73:0/467688705 >> 172.21.15.73:0/467688705 conn(0x7fc4240cb310 msgr2=0x7fc4240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 -- 172.21.15.73:0/467688705 shutdown_connections 2023-11-28T02:53:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.402+0000 7fc42ae76700 1 -- 172.21.15.73:0/467688705 wait complete. 2023-11-28T02:53:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.403+0000 7fc42ae76700 1 Processor -- start 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 -- start start 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4240c6190 0x7fc4240c0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 0x7fc4240c0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4241689d0 con 0x7fc424098d10 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4240c0c50 con 0x7fc4240c6190 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc42ae76700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc424169520 con 0x7fc424158320 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.404+0000 7fc428c12700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45614/0 (socket says 172.21.15.73:45614) 2023-11-28T02:53:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 -- 172.21.15.73:0/8702905 learned_addr learned my addr 172.21.15.73:0/8702905 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc41bfff700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4240c6190 0x7fc4240c0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc429413700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 0x7fc4240c0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 -- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 msgr2=0x7fc4240c0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 0x7fc4240c0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 -- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4240c6190 msgr2=0x7fc4240c0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4240c6190 0x7fc4240c0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 -- 172.21.15.73:0/8702905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc420009a20 con 0x7fc424098d10 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc429413700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 0x7fc4240c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc428c12700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fc42000b6c0 tx=0x7fc420009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001d630 con 0x7fc424098d10 2023-11-28T02:53:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42001d790 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4240c47f0 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42001db30 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.405+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4240c4db0 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.406+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc42406c860 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.406+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc420026410 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.407+0000 7fc418ff9700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 0x7fc410061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.407+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc42007be10 con 0x7fc424098d10 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.407+0000 7fc41bfff700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 0x7fc410061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.407+0000 7fc41bfff700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 0x7fc410061610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc4240c9550 tx=0x7fc414007bb0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.411+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc42004d3d0 con 0x7fc424098d10 2023-11-28T02:53:24.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.640+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7fc4240c69a0 con 0x7fc424098d10 2023-11-28T02:53:24.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.643+0000 7fc418ff9700 1 -- 172.21.15.73:0/8702905 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 updated v6) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc4200509f0 con 0x7fc424098d10 2023-11-28T02:53:24.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 msgr2=0x7fc410061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 0x7fc410061610 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 rx=0x7fc4240c9550 tx=0x7fc414007bb0).stop 2023-11-28T02:53:24.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 msgr2=0x7fc424169030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 rx=0x7fc42000b6c0 tx=0x7fc420009780).stop 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 shutdown_connections 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41005f150 0x7fc410061610 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc424158320 0x7fc4240c0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc424098d10 0x7fc424169030 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 --2- 172.21.15.73:0/8702905 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4240c6190 0x7fc4240c0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 >> 172.21.15.73:0/8702905 conn(0x7fc4240cb310 msgr2=0x7fc424153c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 shutdown_connections 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.645+0000 7fc42ae76700 1 -- 172.21.15.73:0/8702905 wait complete. 2023-11-28T02:53:24.647 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:24.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-11-28T02:53:24.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.771+0000 7f33d893e700 1 Processor -- start 2023-11-28T02:53:24.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.771+0000 7f33d893e700 1 -- start start 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d893e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f33d4151a20 0x7f33d406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d893e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33d40bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d893e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33d40c3470 con 0x7f33d4069d10 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d893e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33d40c48f0 con 0x7f33d4151a20 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d893e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40bdb70 con 0x7f33d40bd1f0 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f33d4069d10 0x7f33d40be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58858/0 (socket says 172.21.15.73:58858) 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d259c700 1 -- 172.21.15.73:0/3249086646 learned_addr learned my addr 172.21.15.73:0/3249086646 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245221772 0 0) 0x7f33d40c3470 con 0x7f33d4069d10 2023-11-28T02:53:24.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33c00033f0 con 0x7f33d4069d10 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.772+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/3249086646 >> v2:172.21.15.150:3300/0 conn(0x7f33d4151a20 0x7f33d406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d2d9d700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33d40bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3645246536 0 0) 0x7f33c00033f0 con 0x7f33d4069d10 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 msgr2=0x7f33d40bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33d40bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 >> v2:172.21.15.150:3300/0 conn(0x7f33d4151a20 msgr2=0x7f33d406a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 --2- 172.21.15.73:0/3249086646 >> v2:172.21.15.150:3300/0 conn(0x7f33d4151a20 0x7f33d406a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4151ea0 con 0x7f33d4069d10 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f33c4001470 con 0x7f33d4069d10 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d2d9d700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33d40bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 >> v1:172.21.15.73:6789/0 conn(0x7f33d4069d10 legacy=0x7f33d40be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33c0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c0004690 0x7f33c0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33c0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40c48f0 con 0x7f33c00038e0 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40bdb70 con 0x7f33c0004690 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33c000b8e0 con 0x7f33d40bd1f0 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33c0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c0004690 0x7f33c0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d2d9d700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33c0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 -- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 msgr2=0x7f33c0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40bd1f0 0x7f33c0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 -- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c0004690 msgr2=0x7f33c0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33c0004690 0x7f33c0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 -- 172.21.15.73:0/3249086646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33c00033f0 con 0x7f33c00038e0 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d259c700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33c0003d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f33c400c210 tx=0x7f33c4001e90).ready entity=mon.0 client_cookie=cb26f42b2837f64d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.773+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4007a60 con 0x7f33c00038e0 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c400cae0 con 0x7f33c00038e0 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d1d9b700 1 -- 172.21.15.73:0/3249086646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c400ce80 con 0x7f33c00038e0 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 -- 172.21.15.73:0/3249086646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 msgr2=0x7f33c0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 --2- 172.21.15.73:0/3249086646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33c0003d00 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7f33c400c210 tx=0x7f33c4001e90).stop 2023-11-28T02:53:24.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d2d9d700 1 -- 172.21.15.73:0/3249086646 reap_dead start 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 -- 172.21.15.73:0/3249086646 shutdown_connections 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 -- 172.21.15.73:0/3249086646 >> 172.21.15.73:0/3249086646 conn(0x7f33d40642d0 msgr2=0x7f33d40c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 -- 172.21.15.73:0/3249086646 shutdown_connections 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.774+0000 7f33d893e700 1 -- 172.21.15.73:0/3249086646 wait complete. 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.775+0000 7f33d893e700 1 Processor -- start 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.775+0000 7f33d893e700 1 -- start start 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 0x7f33d40a69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 0x7f33d41643b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d4154aa0 con 0x7f33c00038e0 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33d40a60c0 con 0x7f33d4163f30 2023-11-28T02:53:24.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d893e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33d4145ce0 con 0x7f33d40a6550 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45626/0 (socket says 172.21.15.73:45626) 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 -- 172.21.15.73:0/2030590295 learned_addr learned my addr 172.21.15.73:0/2030590295 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 0x7f33d40a69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d2d9d700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 0x7f33d41643b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 -- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 msgr2=0x7f33d40a69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 0x7f33d40a69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 -- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 msgr2=0x7f33d41643b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 0x7f33d41643b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 -- 172.21.15.73:0/2030590295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d4164a30 con 0x7f33c00038e0 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d2d9d700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 0x7f33d41643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:24.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 0x7f33d40a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.776+0000 7f33d259c700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f33c400c800 tx=0x7f33c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.777+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c40075c0 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.777+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33c4007720 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.777+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33d4164d10 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.777+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004900 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.777+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33d40a5d50 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.778+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33d40a63a0 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.778+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f33c4004c70 con 0x7f33c00038e0 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.778+0000 7f33caffd700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 0x7f33b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:24.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.778+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f33c4019020 con 0x7f33c00038e0 2023-11-28T02:53:24.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.778+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 0x7f33b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:24.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.779+0000 7f33c9d9b700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 0x7f33b4061610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f33bc006c40 tx=0x7f33bc008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:24.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:24.782+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f33c404c660 con 0x7f33c00038e0 2023-11-28T02:53:25.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.008+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f33d406aa70 con 0x7f33c00038e0 2023-11-28T02:53:25.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.012+0000 7f33caffd700 1 -- 172.21.15.73:0/2030590295 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 updated v7) v1 ==== 104+0+0 (secure 0 0 0) 0x7f33c404fc80 con 0x7f33c00038e0 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 msgr2=0x7f33b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 0x7f33b4061610 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f33bc006c40 tx=0x7f33bc008040).stop 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 msgr2=0x7f33d41457f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 rx=0x7f33c400c800 tx=0x7f33c4015000).stop 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 shutdown_connections 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f33b405f150 0x7f33b4061610 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33d4163f30 0x7f33d41643b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33c00038e0 0x7f33d41457f0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 --2- 172.21.15.73:0/2030590295 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f33d40a6550 0x7f33d40a69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 >> 172.21.15.73:0/2030590295 conn(0x7f33d40642d0 msgr2=0x7f33d40c7330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 shutdown_connections 2023-11-28T02:53:25.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.014+0000 7f33d893e700 1 -- 172.21.15.73:0/2030590295 wait complete. 2023-11-28T02:53:25.019 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:25.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-11-28T02:53:25.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.140+0000 7fc83d785700 1 Processor -- start 2023-11-28T02:53:25.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.140+0000 7fc83d785700 1 -- start start 2023-11-28T02:53:25.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc83d785700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc838098d10 0x7fc8380c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc83d785700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc838063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc83d785700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8380c72d0 con 0x7fc838064350 2023-11-28T02:53:25.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc83d785700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc838154bc0 con 0x7fc838098d10 2023-11-28T02:53:25.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc83d785700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380c66c0 con 0x7fc8380c6190 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8377fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc838064350 0x7fc838158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58862/0 (socket says 172.21.15.73:58862) 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8377fe700 1 -- 172.21.15.73:0/1963602313 learned_addr learned my addr 172.21.15.73:0/1963602313 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc836ffd700 1 --2- 172.21.15.73:0/1963602313 >> v2:172.21.15.150:3300/0 conn(0x7fc838098d10 0x7fc8380c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc82e7fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc838063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1885491013 0 0) 0x7fc8380c72d0 con 0x7fc838064350 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8240033f0 con 0x7fc838064350 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 556146296 0 0) 0x7fc8240033f0 con 0x7fc838064350 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 msgr2=0x7fc838063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc838063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 >> v2:172.21.15.150:3300/0 conn(0x7fc838098d10 msgr2=0x7fc8380c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:25.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 --2- 172.21.15.73:0/1963602313 >> v2:172.21.15.150:3300/0 conn(0x7fc838098d10 0x7fc8380c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380c69a0 con 0x7fc838064350 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc8300017f0 con 0x7fc838064350 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 >> v1:172.21.15.73:6789/0 conn(0x7fc838064350 legacy=0x7fc838158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8240038e0 0x7fc824003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824004690 0x7fc824004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc824005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc838154bc0 con 0x7fc8240038e0 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380c66c0 con 0x7fc824004690 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.141+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc82400b8e0 con 0x7fc8380c6190 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8240038e0 0x7fc824003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc82e7fc700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824004690 0x7fc824004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc8377fe700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc824005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 -- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 msgr2=0x7fc824005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8380c6190 0x7fc824005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 -- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824004690 msgr2=0x7fc824004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc824004690 0x7fc824004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 -- 172.21.15.73:0/1963602313 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240033f0 con 0x7fc8240038e0 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc836ffd700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8240038e0 0x7fc824003d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc828009a80 tx=0x7fc828009060).ready entity=mon.0 client_cookie=bb389c071637bde7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc828009430 con 0x7fc8240038e0 2023-11-28T02:53:25.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc82800e640 con 0x7fc8240038e0 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc8367fc700 1 -- 172.21.15.73:0/1963602313 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc82800e9c0 con 0x7fc8240038e0 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc83d785700 1 -- 172.21.15.73:0/1963602313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8240038e0 msgr2=0x7fc824003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.142+0000 7fc83d785700 1 --2- 172.21.15.73:0/1963602313 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8240038e0 0x7fc824003d00 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 rx=0x7fc828009a80 tx=0x7fc828009060).stop 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.143+0000 7fc8377fe700 1 -- 172.21.15.73:0/1963602313 reap_dead start 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.143+0000 7fc83d785700 1 -- 172.21.15.73:0/1963602313 shutdown_connections 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.143+0000 7fc83d785700 1 -- 172.21.15.73:0/1963602313 >> 172.21.15.73:0/1963602313 conn(0x7fc8380cb310 msgr2=0x7fc8380cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.143+0000 7fc83d785700 1 -- 172.21.15.73:0/1963602313 shutdown_connections 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.143+0000 7fc83d785700 1 -- 172.21.15.73:0/1963602313 wait complete. 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.144+0000 7fc83d785700 1 Processor -- start 2023-11-28T02:53:25.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.144+0000 7fc83d785700 1 -- start start 2023-11-28T02:53:25.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8240038e0 0x7fc8380ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc838150960 0x7fc838148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380ac1d0 con 0x7fc8380aad70 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc838148660 con 0x7fc8240038e0 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8380ab5c0 con 0x7fc838150960 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45640/0 (socket says 172.21.15.73:45640) 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 -- 172.21.15.73:0/237214303 learned_addr learned my addr 172.21.15.73:0/237214303 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc836ffd700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8240038e0 0x7fc8380ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc82e7fc700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc838150960 0x7fc838148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 -- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc838150960 msgr2=0x7fc838148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc838150960 0x7fc838148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 -- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8240038e0 msgr2=0x7fc8380ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8240038e0 0x7fc8380ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 -- 172.21.15.73:0/237214303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8380ab8a0 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc8377fe700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc830000f80 tx=0x7fc830001890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8300079e0 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc83000e860 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8380abb80 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc83000ec20 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.145+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8380ab1f0 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.146+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc83806c860 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.147+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc8300133e0 con 0x7fc8380aad70 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.147+0000 7fc82f7fe700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 0x7fc818061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.147+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc830079a80 con 0x7fc8380aad70 2023-11-28T02:53:25.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.147+0000 7fc836ffd700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 0x7fc818061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.147+0000 7fc836ffd700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 0x7fc818061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc828006d50 tx=0x7fc82801a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.151+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc83004b040 con 0x7fc8380aad70 2023-11-28T02:53:25.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.376+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fc8380c6c80 con 0x7fc8380aad70 2023-11-28T02:53:25.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.379+0000 7fc82f7fe700 1 -- 172.21.15.73:0/237214303 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 updated v8) v1 ==== 97+0+0 (secure 0 0 0) 0x7fc83004e660 con 0x7fc8380aad70 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 msgr2=0x7fc818061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 0x7fc818061610 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 rx=0x7fc828006d50 tx=0x7fc82801a000).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 msgr2=0x7fc838151260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 rx=0x7fc830000f80 tx=0x7fc830001890).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 shutdown_connections 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc81805f150 0x7fc818061610 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8380aad70 0x7fc838151260 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8240038e0 0x7fc8380ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 --2- 172.21.15.73:0/237214303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc838150960 0x7fc838148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 >> 172.21.15.73:0/237214303 conn(0x7fc8380cb310 msgr2=0x7fc838063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 shutdown_connections 2023-11-28T02:53:25.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.381+0000 7fc83d785700 1 -- 172.21.15.73:0/237214303 wait complete. 2023-11-28T02:53:25.384 INFO:teuthology.orchestra.run.smithi073.stderr:updated 2023-11-28T02:53:25.394 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:25.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.505+0000 7f10e66ca700 1 Processor -- start 2023-11-28T02:53:25.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.505+0000 7f10e66ca700 1 -- start start 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e66ca700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f10e01544e0 0x7f10e01548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e66ca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10e006a610 0x7f10e00c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e66ca700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e00c69a0 con 0x7f10e00c6e30 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e66ca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e00c13a0 con 0x7f10e006a610 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e66ca700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e00c1680 con 0x7f10e01544e0 2023-11-28T02:53:25.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10df7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10e006a610 0x7f10e00c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10df7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f10e006a610 0x7f10e00c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45654/0 (socket says 172.21.15.73:45654) 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10df7fe700 1 -- 172.21.15.73:0/189848330 learned_addr learned my addr 172.21.15.73:0/189848330 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.506+0000 7f10e4c67700 1 --1- 172.21.15.73:0/189848330 >> v1:172.21.15.73:6789/0 conn(0x7f10e00c6e30 0x7f10e00a8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58868/0 (socket says 172.21.15.73:58868) 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10dffff700 1 --2- 172.21.15.73:0/189848330 >> v2:172.21.15.150:3300/0 conn(0x7f10e01544e0 0x7f10e01548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10df7fe700 1 -- 172.21.15.73:0/189848330 >> v2:172.21.15.150:3300/0 conn(0x7f10e01544e0 msgr2=0x7f10e01548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10df7fe700 1 --2- 172.21.15.73:0/189848330 >> v2:172.21.15.150:3300/0 conn(0x7f10e01544e0 0x7f10e01548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10df7fe700 1 -- 172.21.15.73:0/189848330 >> v1:172.21.15.73:6789/0 conn(0x7f10e00c6e30 legacy=0x7f10e00a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10deffd700 1 -- 172.21.15.73:0/189848330 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688897415 0 0) 0x7f10e00c69a0 con 0x7f10e00c6e30 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10df7fe700 1 -- 172.21.15.73:0/189848330 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e00c6630 con 0x7f10e006a610 2023-11-28T02:53:25.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10df7fe700 1 --2- 172.21.15.73:0/189848330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e006a610 0x7f10e00c0bc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f10e0085760 tx=0x7f10d40090f0).ready entity=mon.0 client_cookie=5d2de9c7ede851fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10deffd700 1 -- 172.21.15.73:0/189848330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d400b600 con 0x7f10e006a610 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10deffd700 1 -- 172.21.15.73:0/189848330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d400b760 con 0x7f10e006a610 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10deffd700 1 -- 172.21.15.73:0/189848330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d400bae0 con 0x7f10e006a610 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 -- 172.21.15.73:0/189848330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e006a610 msgr2=0x7f10e00c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 --2- 172.21.15.73:0/189848330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e006a610 0x7f10e00c0bc0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 rx=0x7f10e0085760 tx=0x7f10d40090f0).stop 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 -- 172.21.15.73:0/189848330 shutdown_connections 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 --2- 172.21.15.73:0/189848330 >> v2:172.21.15.150:3300/0 conn(0x7f10e01544e0 0x7f10e01548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 --2- 172.21.15.73:0/189848330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e006a610 0x7f10e00c0bc0 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.507+0000 7f10e66ca700 1 -- 172.21.15.73:0/189848330 >> 172.21.15.73:0/189848330 conn(0x7f10e0085020 msgr2=0x7f10e0094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.508+0000 7f10e66ca700 1 -- 172.21.15.73:0/189848330 shutdown_connections 2023-11-28T02:53:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.508+0000 7f10e66ca700 1 -- 172.21.15.73:0/189848330 wait complete. 2023-11-28T02:53:25.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.509+0000 7f10e66ca700 1 Processor -- start 2023-11-28T02:53:25.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.509+0000 7f10e66ca700 1 -- start start 2023-11-28T02:53:25.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.509+0000 7f10e66ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 0x7f10e00980e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e01544e0 0x7f10e0145f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0094b70 con 0x7f10e00c6e30 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0098f60 con 0x7f10e01544e0 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e00a84f0 con 0x7f10e006a610 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45660/0 (socket says 172.21.15.73:45660) 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 -- 172.21.15.73:0/1626484993 learned_addr learned my addr 172.21.15.73:0/1626484993 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e4c67700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e01544e0 0x7f10e0145f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10dffff700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 0x7f10e00980e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 -- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 msgr2=0x7f10e00980e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 0x7f10e00980e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 -- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e01544e0 msgr2=0x7f10e0145f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e01544e0 0x7f10e0145f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 -- 172.21.15.73:0/1626484993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4009370 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10dffff700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 0x7f10e00980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10df7fe700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f10e005fb30 tx=0x7f10d4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d400b8d0 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d400ba30 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e00986c0 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d401b660 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.510+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e0098c80 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.511+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e0094940 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.511+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f10d401ba10 con 0x7f10e00c6e30 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.512+0000 7f10c7fff700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 0x7f10cc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.512+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f10d407b7b0 con 0x7f10e00c6e30 2023-11-28T02:53:25.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.512+0000 7f10dffff700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 0x7f10cc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.512+0000 7f10dffff700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 0x7f10cc061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f10d0006920 tx=0x7f10d0008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.516+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f10e0094940 con 0x7f10e00c6e30 2023-11-28T02:53:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.683+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10e0094940 con 0x7f10e00c6e30 2023-11-28T02:53:25.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.683+0000 7f10c7fff700 1 -- 172.21.15.73:0/1626484993 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13712 (secure 0 0 0) 0x7f10e0094940 con 0x7f10e00c6e30 2023-11-28T02:53:25.685 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:25.685 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":30,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:53:22.784783+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":20,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 msgr2=0x7f10cc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 0x7f10cc061610 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7f10d0006920 tx=0x7f10d0008040).stop 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 msgr2=0x7f10e00a8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 rx=0x7f10e005fb30 tx=0x7f10d4017000).stop 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 shutdown_connections 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f10cc05f150 0x7f10cc061610 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10e01544e0 0x7f10e0145f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f10e006a610 0x7f10e00980e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 --2- 172.21.15.73:0/1626484993 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10e00c6e30 0x7f10e00a8000 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.685+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 >> 172.21.15.73:0/1626484993 conn(0x7f10e0085020 msgr2=0x7f10e00c9910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.686+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 shutdown_connections 2023-11-28T02:53:25.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.686+0000 7f10e66ca700 1 -- 172.21.15.73:0/1626484993 wait complete. 2023-11-28T02:53:25.700 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T02:53:25.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T02:53:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.814+0000 7fea4c0b2700 1 Processor -- start 2023-11-28T02:53:25.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.815+0000 7fea4c0b2700 1 -- start start 2023-11-28T02:53:25.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.815+0000 7fea4c0b2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea44158a30 0x7fea440c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.815+0000 7fea4c0b2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea440c72d0 con 0x7fea44098d10 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4c0b2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea44064570 con 0x7fea440c6190 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4c0b2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea44154bc0 con 0x7fea44158a30 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea49e4e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fea44098d10 0x7fea44063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58882/0 (socket says 172.21.15.73:58882) 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea49e4e700 1 -- 172.21.15.73:0/975801972 learned_addr learned my addr 172.21.15.73:0/975801972 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4a64f700 1 --2- 172.21.15.73:0/975801972 >> v2:172.21.15.150:3300/0 conn(0x7fea44158a30 0x7fea440c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098785400 0 0) 0x7fea440c72d0 con 0x7fea44098d10 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fea300033f0 con 0x7fea44098d10 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2149317728 0 0) 0x7fea300033f0 con 0x7fea44098d10 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4a64f700 1 -- 172.21.15.73:0/975801972 >> v1:172.21.15.150:6789/0 conn(0x7fea440c6190 legacy=0x7fea44158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4a64f700 1 -- 172.21.15.73:0/975801972 >> v1:172.21.15.73:6789/0 conn(0x7fea44098d10 legacy=0x7fea44063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4a64f700 1 -- 172.21.15.73:0/975801972 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea440c66c0 con 0x7fea44158a30 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea4a64f700 1 --2- 172.21.15.73:0/975801972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44158a30 0x7fea440c7760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fea3c009760 tx=0x7fea3c009060).ready entity=mon.1 client_cookie=34ac1c394c324e73 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.816+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea400017f0 con 0x7fea44158a30 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c011400 con 0x7fea44158a30 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea48e4c700 1 -- 172.21.15.73:0/975801972 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c011780 con 0x7fea44158a30 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 -- 172.21.15.73:0/975801972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44158a30 msgr2=0x7fea440c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/975801972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44158a30 0x7fea440c7760 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 rx=0x7fea3c009760 tx=0x7fea3c009060).stop 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 -- 172.21.15.73:0/975801972 shutdown_connections 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/975801972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44158a30 0x7fea440c7760 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 -- 172.21.15.73:0/975801972 >> 172.21.15.73:0/975801972 conn(0x7fea440cb310 msgr2=0x7fea440cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 -- 172.21.15.73:0/975801972 shutdown_connections 2023-11-28T02:53:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.817+0000 7fea4c0b2700 1 -- 172.21.15.73:0/975801972 wait complete. 2023-11-28T02:53:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.818+0000 7fea4c0b2700 1 Processor -- start 2023-11-28T02:53:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.818+0000 7fea4c0b2700 1 -- start start 2023-11-28T02:53:25.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44098d10 0x7fea4413af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea440c6190 0x7fea44156f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea44164ad0 con 0x7fea44158a30 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea441578f0 con 0x7fea44098d10 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4c0b2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea441658d0 con 0x7fea440c6190 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45670/0 (socket says 172.21.15.73:45670) 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 -- 172.21.15.73:0/4070674600 learned_addr learned my addr 172.21.15.73:0/4070674600 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4964d700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea440c6190 0x7fea44156f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea49e4e700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44098d10 0x7fea4413af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 -- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea440c6190 msgr2=0x7fea44156f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea440c6190 0x7fea44156f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 -- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44098d10 msgr2=0x7fea4413af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44098d10 0x7fea4413af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 -- 172.21.15.73:0/4070674600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea3c0092a0 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.819+0000 7fea4a64f700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fea440c2360 tx=0x7fea3c014000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.820+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c010c00 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.820+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea3c010d60 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.820+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea44164cc0 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.820+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea3c01d680 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.820+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea44165280 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.821+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea4406c860 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.821+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fea3c01da70 con 0x7fea44158a30 2023-11-28T02:53:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.821+0000 7fea39ffb700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 0x7fea28061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:25.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.822+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fea3c07b780 con 0x7fea44158a30 2023-11-28T02:53:25.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.822+0000 7fea49e4e700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 0x7fea28061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:25.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.822+0000 7fea49e4e700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 0x7fea28061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fea40001560 tx=0x7fea40000f40).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:25.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.826+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea3c04cce0 con 0x7fea44158a30 2023-11-28T02:53:25.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.988+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fea440c69a0 con 0x7fea44158a30 2023-11-28T02:53:25.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.988+0000 7fea39ffb700 1 -- 172.21.15.73:0/4070674600 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v30) v1 ==== 110+0+10 (secure 0 0 0) 0x7fea3c016030 con 0x7fea44158a30 2023-11-28T02:53:25.991 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 msgr2=0x7fea28061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 0x7fea28061610 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7fea40001560 tx=0x7fea40000f40).stop 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 msgr2=0x7fea44157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 rx=0x7fea440c2360 tx=0x7fea3c014000).stop 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 shutdown_connections 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fea2805f150 0x7fea28061610 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea44158a30 0x7fea44157400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea44098d10 0x7fea4413af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 --2- 172.21.15.73:0/4070674600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea440c6190 0x7fea44156f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:25.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 >> 172.21.15.73:0/4070674600 conn(0x7fea440cb310 msgr2=0x7fea440ca670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:25.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 shutdown_connections 2023-11-28T02:53:25.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:25.990+0000 7fea4c0b2700 1 -- 172.21.15.73:0/4070674600 wait complete. 2023-11-28T02:53:26.003 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T02:53:26.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.119+0000 7fd4a95e4700 1 Processor -- start 2023-11-28T02:53:26.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.119+0000 7fd4a95e4700 1 -- start start 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4a40c6190 0x7fd4a4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4a4064350 0x7fd4a4158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a40c72d0 con 0x7fd4a40c6190 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4a4154bc0 con 0x7fd4a4064350 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a95e4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40c66c0 con 0x7fd4a4098d10 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4a40c6190 0x7fd4a4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4a40c6190 0x7fd4a4063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45672/0 (socket says 172.21.15.73:45672) 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 -- 172.21.15.73:0/3485989788 learned_addr learned my addr 172.21.15.73:0/3485989788 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:26.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a37fe700 1 --2- 172.21.15.73:0/3485989788 >> v2:172.21.15.150:3300/0 conn(0x7fd4a4064350 0x7fd4a4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd4a2ffd700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 -- 172.21.15.73:0/3485989788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 msgr2=0x7fd4a40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 -- 172.21.15.73:0/3485989788 >> v2:172.21.15.150:3300/0 conn(0x7fd4a4064350 msgr2=0x7fd4a4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3485989788 >> v2:172.21.15.150:3300/0 conn(0x7fd4a4064350 0x7fd4a4158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.120+0000 7fd49a7fc700 1 -- 172.21.15.73:0/3485989788 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4a40c69a0 con 0x7fd4a40c6190 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a40c6190 0x7fd4a4063e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd4a4084570 tx=0x7fd48c0090f0).ready entity=mon.0 client_cookie=98ece6086e33405d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a27fc700 1 -- 172.21.15.73:0/3485989788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00b600 con 0x7fd4a40c6190 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a27fc700 1 -- 172.21.15.73:0/3485989788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd48c00b760 con 0x7fd4a40c6190 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a27fc700 1 -- 172.21.15.73:0/3485989788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd48c00bae0 con 0x7fd4a40c6190 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3485989788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a40c6190 msgr2=0x7fd4a4063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a40c6190 0x7fd4a4063e60 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 rx=0x7fd4a4084570 tx=0x7fd48c0090f0).stop 2023-11-28T02:53:26.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3485989788 shutdown_connections 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3485989788 >> v2:172.21.15.150:3300/0 conn(0x7fd4a4064350 0x7fd4a4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3485989788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a40c6190 0x7fd4a4063e60 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3485989788 >> 172.21.15.73:0/3485989788 conn(0x7fd4a40cb310 msgr2=0x7fd4a40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3485989788 shutdown_connections 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.121+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3485989788 wait complete. 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 Processor -- start 2023-11-28T02:53:26.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 -- start start 2023-11-28T02:53:26.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40c6190 0x7fd4a4158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40afff0 con 0x7fd4a4064350 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40b02d0 con 0x7fd4a40c6190 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.123+0000 7fd4a95e4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4a40b05b0 con 0x7fd4a4098d10 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45678/0 (socket says 172.21.15.73:45678) 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 -- 172.21.15.73:0/3572381161 learned_addr learned my addr 172.21.15.73:0/3572381161 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a37fe700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40c6190 0x7fd4a4158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 -- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 msgr2=0x7fd4a40af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 -- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40c6190 msgr2=0x7fd4a4158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40c6190 0x7fd4a4158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 -- 172.21.15.73:0/3572381161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd48c009370 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a2ffd700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd49400d890 tx=0x7fd494008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494011590 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4940116f0 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4a40b0890 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd494011a70 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.124+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4a4148760 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.125+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4a406c860 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.125+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd494012420 con 0x7fd4a4064350 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.126+0000 7fd49b7fe700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 0x7fd4840615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.126+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd4940798b0 con 0x7fd4a4064350 2023-11-28T02:53:26.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.126+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 0x7fd4840615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.126+0000 7fd49a7fc700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 0x7fd4840615e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd4a4084570 tx=0x7fd48c017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.130+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd49404af10 con 0x7fd4a4064350 2023-11-28T02:53:26.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.292+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fd4a40c6c80 con 0x7fd4a4064350 2023-11-28T02:53:26.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.292+0000 7fd49b7fe700 1 -- 172.21.15.73:0/3572381161 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7fd49404e530 con 0x7fd4a4064350 2023-11-28T02:53:26.294 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 msgr2=0x7fd4840615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 0x7fd4840615e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7fd4a4084570 tx=0x7fd48c017000).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 msgr2=0x7fd4a4148270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 rx=0x7fd49400d890 tx=0x7fd494008040).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 shutdown_connections 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd48405f120 0x7fd4840615e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4a40c6190 0x7fd4a4158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4a4064350 0x7fd4a4148270 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 --2- 172.21.15.73:0/3572381161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4a4098d10 0x7fd4a40af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 >> 172.21.15.73:0/3572381161 conn(0x7fd4a40cb310 msgr2=0x7fd4a40cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:26.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 shutdown_connections 2023-11-28T02:53:26.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.294+0000 7fd4a95e4700 1 -- 172.21.15.73:0/3572381161 wait complete. 2023-11-28T02:53:26.308 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T02:53:26.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.420+0000 7f3e1e529700 1 Processor -- start 2023-11-28T02:53:26.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.420+0000 7f3e1e529700 1 -- start start 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1e529700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e18064e20 0x7f3e18155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1e529700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 0x7f3e18096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1e529700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e18096800 con 0x7f3e181544d0 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1e529700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e18067650 con 0x7f3e18064e20 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1e529700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180683d0 con 0x7f3e18095e80 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e18064e20 0x7f3e18155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e17fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3e181544d0 0x7f3e18066f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58896/0 (socket says 172.21.15.73:58896) 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e17fff700 1 -- 172.21.15.73:0/540770197 learned_addr learned my addr 172.21.15.73:0/540770197 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 --2- 172.21.15.73:0/540770197 >> v2:172.21.15.73:3300/0 conn(0x7f3e18064e20 0x7f3e18155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45682/0 (socket says 172.21.15.73:45682) 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1cac6700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 0x7f3e18096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 -- 172.21.15.73:0/540770197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 msgr2=0x7f3e18096310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 0x7f3e18096310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 -- 172.21.15.73:0/540770197 >> v1:172.21.15.73:6789/0 conn(0x7f3e181544d0 legacy=0x7f3e18066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e16ffd700 1 -- 172.21.15.73:0/540770197 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3752034197 0 0) 0x7f3e18096800 con 0x7f3e181544d0 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 -- 172.21.15.73:0/540770197 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e18067850 con 0x7f3e18064e20 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e1cac6700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 0x7f3e18096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.421+0000 7f3e177fe700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18064e20 0x7f3e18155750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f3e0c009760 tx=0x7f3e0c009060).ready entity=mon.0 client_cookie=9fb2ed6039336ec0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e16ffd700 1 -- 172.21.15.73:0/540770197 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00b580 con 0x7f3e18064e20 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e16ffd700 1 -- 172.21.15.73:0/540770197 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c00b6e0 con 0x7f3e18064e20 2023-11-28T02:53:26.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e16ffd700 1 -- 172.21.15.73:0/540770197 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00ba60 con 0x7f3e18064e20 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 -- 172.21.15.73:0/540770197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18064e20 msgr2=0x7f3e18155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18064e20 0x7f3e18155750 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 rx=0x7f3e0c009760 tx=0x7f3e0c009060).stop 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 -- 172.21.15.73:0/540770197 shutdown_connections 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e18095e80 0x7f3e18096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 --2- 172.21.15.73:0/540770197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18064e20 0x7f3e18155750 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 -- 172.21.15.73:0/540770197 >> 172.21.15.73:0/540770197 conn(0x7f3e180631d0 msgr2=0x7f3e180c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 -- 172.21.15.73:0/540770197 shutdown_connections 2023-11-28T02:53:26.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.422+0000 7f3e1e529700 1 -- 172.21.15.73:0/540770197 wait complete. 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 Processor -- start 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 -- start start 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e18064e20 0x7f3e180c7930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 0x7f3e1814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18156d50 con 0x7f3e18095e80 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e18156440 con 0x7f3e18064e20 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.424+0000 7f3e1e529700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e180c7f10 con 0x7f3e181544d0 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45698/0 (socket says 172.21.15.73:45698) 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 -- 172.21.15.73:0/342802772 learned_addr learned my addr 172.21.15.73:0/342802772 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e17fff700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e18064e20 0x7f3e180c7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e1cac6700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 0x7f3e1814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 -- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 msgr2=0x7f3e1814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 0x7f3e1814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 -- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e18064e20 msgr2=0x7f3e180c7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e18064e20 0x7f3e180c7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 -- 172.21.15.73:0/342802772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c0092a0 con 0x7f3e18095e80 2023-11-28T02:53:26.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e1cac6700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 0x7f3e1814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e177fe700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f3e0c009eb0 tx=0x7f3e0c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c00ba60 con 0x7f3e18095e80 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c000c90 con 0x7f3e18095e80 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e18156700 con 0x7f3e18095e80 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c01b660 con 0x7f3e18095e80 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.425+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e181578f0 con 0x7f3e18095e80 2023-11-28T02:53:26.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.426+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e1806c860 con 0x7f3e18095e80 2023-11-28T02:53:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.427+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3e0c01ba10 con 0x7f3e18095e80 2023-11-28T02:53:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.428+0000 7f3dfffff700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 0x7f3e04061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.428+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f3e0c07bf90 con 0x7f3e18095e80 2023-11-28T02:53:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.428+0000 7f3e17fff700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 0x7f3e04061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.428+0000 7f3e17fff700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 0x7f3e04061610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3e08002670 tx=0x7f3e08009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.431+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e0c04d5f0 con 0x7f3e18095e80 2023-11-28T02:53:26.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.597+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3e18067b30 con 0x7f3e18095e80 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.597+0000 7f3dfffff700 1 -- 172.21.15.73:0/342802772 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+143 (secure 0 0 0) 0x7f3e0c029020 con 0x7f3e18095e80 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T02:53:26.599 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T02:53:26.600 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:26.600 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T02:53:26.600 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T02:53:26.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:26.600 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 msgr2=0x7f3e04061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 0x7f3e04061610 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f3e08002670 tx=0x7f3e08009040).stop 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 msgr2=0x7f3e18157400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 rx=0x7f3e0c009eb0 tx=0x7f3e0c017000).stop 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 shutdown_connections 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e0405f150 0x7f3e04061610 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e181544d0 0x7f3e1814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e18064e20 0x7f3e180c7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 --2- 172.21.15.73:0/342802772 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e18095e80 0x7f3e18157400 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 >> 172.21.15.73:0/342802772 conn(0x7f3e180631d0 msgr2=0x7f3e180a7570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:26.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 shutdown_connections 2023-11-28T02:53:26.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.599+0000 7f3e1e529700 1 -- 172.21.15.73:0/342802772 wait complete. 2023-11-28T02:53:26.612 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T02:53:26.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T02:53:26.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.729+0000 7ff4baef4700 1 Processor -- start 2023-11-28T02:53:26.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.729+0000 7ff4baef4700 1 -- start start 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4baef4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff4b40aa930 0x7ff4b40a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4baef4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b40c6630 con 0x7ff4b4154530 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4baef4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b4064230 con 0x7ff4b4154910 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4baef4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b40c6390 con 0x7ff4b40aa930 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b8c90700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff4b4154530 0x7ff4b40c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58910/0 (socket says 172.21.15.73:58910) 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b8c90700 1 -- 172.21.15.73:0/1552345983 learned_addr learned my addr 172.21.15.73:0/1552345983 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b9491700 1 --2- 172.21.15.73:0/1552345983 >> v2:172.21.15.150:3300/0 conn(0x7ff4b40aa930 0x7ff4b40a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2036905055 0 0) 0x7ff4b40c6630 con 0x7ff4b4154530 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4a40033f0 con 0x7ff4b4154530 2023-11-28T02:53:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.730+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4017724732 0 0) 0x7ff4a40033f0 con 0x7ff4b4154530 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 >> v2:172.21.15.150:3300/0 conn(0x7ff4b40aa930 msgr2=0x7ff4b40a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 --2- 172.21.15.73:0/1552345983 >> v2:172.21.15.150:3300/0 conn(0x7ff4b40aa930 0x7ff4b40a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 >> v1:172.21.15.150:6789/0 conn(0x7ff4b4154910 legacy=0x7ff4b40aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b4064510 con 0x7ff4b4154530 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff4a80015f0 con 0x7ff4b4154530 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 >> v1:172.21.15.73:6789/0 conn(0x7ff4b4154530 legacy=0x7ff4b40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a40038e0 0x7ff4a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a4004690 0x7ff4a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 0x7ff4a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40c6390 con 0x7ff4a4005000 2023-11-28T02:53:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400bf60 con 0x7ff4a4004690 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4a400c0f0 con 0x7ff4a40038e0 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 0x7ff4a400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b3fff700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a4004690 0x7ff4a4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b8c90700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a40038e0 0x7ff4a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 -- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a40038e0 msgr2=0x7ff4a4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a40038e0 0x7ff4a4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 -- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a4004690 msgr2=0x7ff4a4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4a4004690 0x7ff4a4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 -- 172.21.15.73:0/1552345983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4a40033f0 con 0x7ff4a4005000 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b8c90700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a40038e0 0x7ff4a4003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 -- 172.21.15.73:0/1552345983 reap_dead start 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b9491700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 0x7ff4a400b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7ff4b40a9fe0 tx=0x7ff4ac00b040).ready entity=mon.0 client_cookie=4a8d7b09f2dd01f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac009ac0 con 0x7ff4a4005000 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac00f040 con 0x7ff4a4005000 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.731+0000 7ff4b37fe700 1 -- 172.21.15.73:0/1552345983 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00ece0 con 0x7ff4a4005000 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 -- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 msgr2=0x7ff4a400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 0x7ff4a400b8e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 rx=0x7ff4b40a9fe0 tx=0x7ff4ac00b040).stop 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 -- 172.21.15.73:0/1552345983 shutdown_connections 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 --2- 172.21.15.73:0/1552345983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4a4005000 0x7ff4a400b8e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 -- 172.21.15.73:0/1552345983 >> 172.21.15.73:0/1552345983 conn(0x7ff4b4094930 msgr2=0x7ff4b406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 -- 172.21.15.73:0/1552345983 shutdown_connections 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.732+0000 7ff4baef4700 1 -- 172.21.15.73:0/1552345983 wait complete. 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.733+0000 7ff4baef4700 1 Processor -- start 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.733+0000 7ff4baef4700 1 -- start start 2023-11-28T02:53:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a4005000 0x7ff4b414f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40b1030 0x7ff4b4167e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b40ac620 con 0x7ff4b4168340 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b41687c0 con 0x7ff4b40b1030 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4baef4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b414f9d0 con 0x7ff4a4005000 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45716/0 (socket says 172.21.15.73:45716) 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 -- 172.21.15.73:0/3032347589 learned_addr learned my addr 172.21.15.73:0/3032347589 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 -- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a4005000 msgr2=0x7ff4b414f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a4005000 0x7ff4b414f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 -- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40b1030 msgr2=0x7ff4b4167e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40b1030 0x7ff4b4167e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 -- 172.21.15.73:0/3032347589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b40c4e80 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b9491700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7ff4b40848a0 tx=0x7ff4ac018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.734+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac00eed0 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.735+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4ac01d460 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.735+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4ac01d800 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.735+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b40abcf0 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.735+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b40ac2b0 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.736+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b40ac900 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.736+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff4ac003020 con 0x7ff4b4168340 2023-11-28T02:53:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.736+0000 7ff4b0ff9700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 0x7ff498061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:26.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.736+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff4ac07bdb0 con 0x7ff4b4168340 2023-11-28T02:53:26.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.736+0000 7ff4b8c90700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 0x7ff498061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:26.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.737+0000 7ff4b8c90700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 0x7ff498061610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7ff4a8000e10 tx=0x7ff4a8009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:26.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.740+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff4ac04d3d0 con 0x7ff4b4168340 2023-11-28T02:53:26.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:26.906+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff4b40ac760 con 0x7ff4b4168340 2023-11-28T02:53:27.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.593+0000 7ff4b0ff9700 1 -- 172.21.15.73:0/3032347589 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff4b40ac760 con 0x7ff4b4168340 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 msgr2=0x7ff498061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 0x7ff498061610 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 rx=0x7ff4a8000e10 tx=0x7ff4a8009040).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 msgr2=0x7ff4b40acab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 rx=0x7ff4b40848a0 tx=0x7ff4ac018000).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 shutdown_connections 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49805f150 0x7ff498061610 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b4168340 0x7ff4b40acab0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4a4005000 0x7ff4b414f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 --2- 172.21.15.73:0/3032347589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b40b1030 0x7ff4b4167e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 >> 172.21.15.73:0/3032347589 conn(0x7ff4b4094930 msgr2=0x7ff4b414fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 shutdown_connections 2023-11-28T02:53:27.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.596+0000 7ff4baef4700 1 -- 172.21.15.73:0/3032347589 wait complete. 2023-11-28T02:53:27.609 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T02:53:27.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T02:53:27.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.733+0000 7f4898da8700 1 Processor -- start 2023-11-28T02:53:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.733+0000 7f4898da8700 1 -- start start 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4898da8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f48940c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4898da8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48940c72d0 con 0x7f48940c6190 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4898da8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4894064570 con 0x7f4894158320 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4898da8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4894154bc0 con 0x7f4894098d10 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f488bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f48940c6190 0x7f4894063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58916/0 (socket says 172.21.15.73:58916) 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f488bfff700 1 -- 172.21.15.73:0/1724639967 learned_addr learned my addr 172.21.15.73:0/1724639967 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f489259c700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f48940c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3947733201 0 0) 0x7f48940c72d0 con 0x7f48940c6190 2023-11-28T02:53:27.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.734+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48800033f0 con 0x7f48940c6190 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1329536805 0 0) 0x7f48800033f0 con 0x7f48940c6190 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 msgr2=0x7f48940c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f48940c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 >> v1:172.21.15.150:6789/0 conn(0x7f4894158320 legacy=0x7f48940a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48940c66c0 con 0x7f48940c6190 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f487c001470 con 0x7f48940c6190 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 >> v1:172.21.15.73:6789/0 conn(0x7f48940c6190 legacy=0x7f4894063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4880003910 0x7f4880003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f4880004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4880004b80 0x7f4880005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894154bc0 con 0x7f4880004b80 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f488000b8e0 con 0x7f4880003910 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f488000ba50 con 0x7f4894098d10 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4880004b80 0x7f4880005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f488bfff700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f4880004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f489259c700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4880003910 0x7f4880003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 -- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 msgr2=0x7f4880004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4894098d10 0x7f4880004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 -- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4880003910 msgr2=0x7f4880003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4880003910 0x7f4880003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 -- 172.21.15.73:0/1724639967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48800033f0 con 0x7f4880004b80 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4880004b80 0x7f4880005000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f488c001fe0 tx=0x7f488c007f30).ready entity=mon.0 client_cookie=6a9fde68a1077952 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:27.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00c810 con 0x7f4880004b80 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c003030 con 0x7f4880004b80 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.735+0000 7f4891d9b700 1 -- 172.21.15.73:0/1724639967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c00cbc0 con 0x7f4880004b80 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 -- 172.21.15.73:0/1724639967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4880004b80 msgr2=0x7f4880005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 --2- 172.21.15.73:0/1724639967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4880004b80 0x7f4880005000 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 rx=0x7f488c001fe0 tx=0x7f488c007f30).stop 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4892d9d700 1 -- 172.21.15.73:0/1724639967 reap_dead start 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 -- 172.21.15.73:0/1724639967 shutdown_connections 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 -- 172.21.15.73:0/1724639967 >> 172.21.15.73:0/1724639967 conn(0x7f48940cb310 msgr2=0x7f48940cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 -- 172.21.15.73:0/1724639967 shutdown_connections 2023-11-28T02:53:27.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.736+0000 7f4898da8700 1 -- 172.21.15.73:0/1724639967 wait complete. 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.737+0000 7f4898da8700 1 Processor -- start 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.737+0000 7f4898da8700 1 -- start start 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 0x7f489414fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 0x7f48940c2c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940c27c0 con 0x7f48941515d0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4894150640 con 0x7f48941501c0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4894150920 con 0x7f4880004b80 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45736/0 (socket says 172.21.15.73:45736) 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 -- 172.21.15.73:0/1291467979 learned_addr learned my addr 172.21.15.73:0/1291467979 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f489259c700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 0x7f489414fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f488bfff700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 0x7f48940c2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 -- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 msgr2=0x7f489414fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 0x7f489414fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 -- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 msgr2=0x7f48940c2c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 0x7f48940c2c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 -- 172.21.15.73:0/1291467979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894150c00 con 0x7f48941515d0 2023-11-28T02:53:27.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f489259c700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 0x7f489414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f488bfff700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 0x7f48940c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4892d9d700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f488c010000 tx=0x7f488c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c000fb0 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c00c810 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.738+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48940c2450 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.739+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c01d410 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.739+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48940c3140 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.740+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48940c2930 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.740+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f488c0263b0 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.740+0000 7f488a7fc700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 0x7f4874061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.740+0000 7f489259c700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 0x7f4874061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.740+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f488c07ab50 con 0x7f48941515d0 2023-11-28T02:53:27.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.741+0000 7f489259c700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 0x7f4874061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4884009270 tx=0x7f4884008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:27.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.745+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f488c04c0f0 con 0x7f48941515d0 2023-11-28T02:53:27.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:27.909+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f489406c860 con 0x7f48941515d0 2023-11-28T02:53:28.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.594+0000 7f488a7fc700 1 -- 172.21.15.73:0/1291467979 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f488c022070 con 0x7f48941515d0 2023-11-28T02:53:28.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 msgr2=0x7f4874061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 0x7f4874061610 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 rx=0x7f4884009270 tx=0x7f4884008040).stop 2023-11-28T02:53:28.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 msgr2=0x7f4894156480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f488c010000 tx=0x7f488c016000).stop 2023-11-28T02:53:28.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 shutdown_connections 2023-11-28T02:53:28.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f487405f150 0x7f4874061610 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48941515d0 0x7f4894156480 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4880004b80 0x7f489414fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 --2- 172.21.15.73:0/1291467979 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48941501c0 0x7f48940c2c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.601+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 >> 172.21.15.73:0/1291467979 conn(0x7f48940cb310 msgr2=0x7f48940aa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:28.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.609+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 shutdown_connections 2023-11-28T02:53:28.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.610+0000 7f4898da8700 1 -- 172.21.15.73:0/1291467979 wait complete. 2023-11-28T02:53:28.619 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T02:53:28.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T02:53:28.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.744+0000 7fa313f38700 1 Processor -- start 2023-11-28T02:53:28.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 -- start start 2023-11-28T02:53:28.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa30c1547f0 0x7fa30c0a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa30c0c7e30 con 0x7fa30c062700 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0bcf40 con 0x7fa30c0c86f0 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.745+0000 7fa313f38700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa30c0c6b20 con 0x7fa30c1547f0 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45750/0 (socket says 172.21.15.73:45750) 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 -- 172.21.15.73:0/1084491628 learned_addr learned my addr 172.21.15.73:0/1084491628 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa311cd4700 1 --2- 172.21.15.73:0/1084491628 >> v2:172.21.15.150:3300/0 conn(0x7fa30c1547f0 0x7fa30c0a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 -- 172.21.15.73:0/1084491628 >> v2:172.21.15.150:3300/0 conn(0x7fa30c1547f0 msgr2=0x7fa30c0a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:28.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 --2- 172.21.15.73:0/1084491628 >> v2:172.21.15.150:3300/0 conn(0x7fa30c1547f0 0x7fa30c0a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 -- 172.21.15.73:0/1084491628 >> v1:172.21.15.150:6789/0 conn(0x7fa30c062700 legacy=0x7fa30c0a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 -- 172.21.15.73:0/1084491628 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c0a77d0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa3114d3700 1 --2- 172.21.15.73:0/1084491628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa30c05f7f0 tx=0x7fa2fc0090f0).ready entity=mon.0 client_cookie=518cacda8f20ec8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa310cd2700 1 -- 172.21.15.73:0/1084491628 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00b5a0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa310cd2700 1 -- 172.21.15.73:0/1084491628 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc00b700 con 0x7fa30c0c86f0 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.746+0000 7fa310cd2700 1 -- 172.21.15.73:0/1084491628 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00baa0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 -- 172.21.15.73:0/1084491628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 msgr2=0x7fa30c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 --2- 172.21.15.73:0/1084491628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 rx=0x7fa30c05f7f0 tx=0x7fa2fc0090f0).stop 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 -- 172.21.15.73:0/1084491628 shutdown_connections 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 --2- 172.21.15.73:0/1084491628 >> v2:172.21.15.150:3300/0 conn(0x7fa30c1547f0 0x7fa30c0a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 --2- 172.21.15.73:0/1084491628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0c6fb0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 -- 172.21.15.73:0/1084491628 >> 172.21.15.73:0/1084491628 conn(0x7fa30c0642d0 msgr2=0x7fa30c0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 -- 172.21.15.73:0/1084491628 shutdown_connections 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.747+0000 7fa313f38700 1 -- 172.21.15.73:0/1084491628 wait complete. 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.748+0000 7fa313f38700 1 Processor -- start 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.748+0000 7fa313f38700 1 -- start start 2023-11-28T02:53:28.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa30c062700 0x7fa30c148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 0x7fa30c0bf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0c08a0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0aa4f0 con 0x7fa30c1547f0 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa30c0bfc90 con 0x7fa30c062700 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45766/0 (socket says 172.21.15.73:45766) 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 -- 172.21.15.73:0/3561980205 learned_addr learned my addr 172.21.15.73:0/3561980205 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:28.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3124d5700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 0x7fa30c0bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa311cd4700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa30c062700 0x7fa30c148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 -- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa30c062700 msgr2=0x7fa30c148660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa30c062700 0x7fa30c148660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 -- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 msgr2=0x7fa30c0bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 0x7fa30c0bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 -- 172.21.15.73:0/3561980205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2fc009370 con 0x7fa30c0c86f0 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3124d5700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 0x7fa30c0bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa3114d3700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa2fc011000 tx=0x7fa2fc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc00bee0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2fc00b870 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2fc0069b0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.749+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa30c0bff70 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.750+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa30c0c0530 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.751+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa30c0c0a10 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.751+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa2fc01c630 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.751+0000 7fa301ffb700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 0x7fa2f00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.751+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fa2fc07bbd0 con 0x7fa30c0c86f0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.751+0000 7fa311cd4700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 0x7fa2f00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:28.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.752+0000 7fa311cd4700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 0x7fa2f00615e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa3080075a0 tx=0x7fa3080097a0).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:28.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.756+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa2fc04cf00 con 0x7fa30c0c86f0 2023-11-28T02:53:28.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.961+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa30c06c860 con 0x7fa30c0c86f0 2023-11-28T02:53:28.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.968+0000 7fa301ffb700 1 -- 172.21.15.73:0/3561980205 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v10) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa2fc020070 con 0x7fa30c0c86f0 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 msgr2=0x7fa2f00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 0x7fa2f00615e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7fa3080075a0 tx=0x7fa3080097a0).stop 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 msgr2=0x7fa30c0bf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 rx=0x7fa2fc011000 tx=0x7fa2fc015000).stop 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 shutdown_connections 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa2f005f120 0x7fa2f00615e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa30c1547f0 0x7fa30c0bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa30c062700 0x7fa30c148660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 --2- 172.21.15.73:0/3561980205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa30c0c86f0 0x7fa30c0bf120 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:28.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.973+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 >> 172.21.15.73:0/3561980205 conn(0x7fa30c0642d0 msgr2=0x7fa30c153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:28.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.978+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 shutdown_connections 2023-11-28T02:53:28.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:28.978+0000 7fa313f38700 1 -- 172.21.15.73:0/3561980205 wait complete. 2023-11-28T02:53:28.981 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 5 and data pool 6 2023-11-28T02:53:28.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.111+0000 7fad629f4700 1 Processor -- start 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.111+0000 7fad629f4700 1 -- start start 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad629f4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad5c0c0180 0x7fad5c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad629f4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad5c0c1750 0x7fad5c0c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad629f4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5c0c12c0 con 0x7fad5c157660 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad629f4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c0c06b0 con 0x7fad5c0c1750 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad629f4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5c0c0990 con 0x7fad5c0c0180 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad5c0c1750 0x7fad5c0c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad5c0c1750 0x7fad5c0c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45768/0 (socket says 172.21.15.73:45768) 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 -- 172.21.15.73:0/3342413630 learned_addr learned my addr 172.21.15.73:0/3342413630 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad611f1700 1 --2- 172.21.15.73:0/3342413630 >> v2:172.21.15.150:3300/0 conn(0x7fad5c0c0180 0x7fad5c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 -- 172.21.15.73:0/3342413630 >> v2:172.21.15.150:3300/0 conn(0x7fad5c0c0180 msgr2=0x7fad5c157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 --2- 172.21.15.73:0/3342413630 >> v2:172.21.15.150:3300/0 conn(0x7fad5c0c0180 0x7fad5c157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 -- 172.21.15.73:0/3342413630 >> v1:172.21.15.150:6789/0 conn(0x7fad5c157660 legacy=0x7fad5c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.112+0000 7fad621f3700 1 -- 172.21.15.73:0/3342413630 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5c0c0c70 con 0x7fad5c0c1750 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad621f3700 1 --2- 172.21.15.73:0/3342413630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c1750 0x7fad5c0c1b70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fad5c0c5820 tx=0x7fad540090f0).ready entity=mon.0 client_cookie=c07e4deec666e861 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad609f0700 1 -- 172.21.15.73:0/3342413630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400b5a0 con 0x7fad5c0c1750 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad609f0700 1 -- 172.21.15.73:0/3342413630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5400b700 con 0x7fad5c0c1750 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad609f0700 1 -- 172.21.15.73:0/3342413630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5400ba80 con 0x7fad5c0c1750 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 -- 172.21.15.73:0/3342413630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c1750 msgr2=0x7fad5c0c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 --2- 172.21.15.73:0/3342413630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c1750 0x7fad5c0c1b70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 rx=0x7fad5c0c5820 tx=0x7fad540090f0).stop 2023-11-28T02:53:29.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 -- 172.21.15.73:0/3342413630 shutdown_connections 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 --2- 172.21.15.73:0/3342413630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c1750 0x7fad5c0c1b70 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 --2- 172.21.15.73:0/3342413630 >> v2:172.21.15.150:3300/0 conn(0x7fad5c0c0180 0x7fad5c157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 -- 172.21.15.73:0/3342413630 >> 172.21.15.73:0/3342413630 conn(0x7fad5c0cb000 msgr2=0x7fad5c0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 -- 172.21.15.73:0/3342413630 shutdown_connections 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.113+0000 7fad629f4700 1 -- 172.21.15.73:0/3342413630 wait complete. 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.114+0000 7fad629f4700 1 Processor -- start 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.114+0000 7fad629f4700 1 -- start start 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad5c0c1750 0x7fad5c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad5c157660 0x7fad5c0ae980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0c20c0 con 0x7fad5c0c0180 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c0c2c40 con 0x7fad5c0c1750 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad629f4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5c1512e0 con 0x7fad5c157660 2023-11-28T02:53:29.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad619f2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad619f2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45772/0 (socket says 172.21.15.73:45772) 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad619f2700 1 -- 172.21.15.73:0/1604764939 learned_addr learned my addr 172.21.15.73:0/1604764939 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad621f3700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad5c157660 0x7fad5c0ae980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.115+0000 7fad611f1700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad5c0c1750 0x7fad5c1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 -- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad5c157660 msgr2=0x7fad5c0ae980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad5c157660 0x7fad5c0ae980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 -- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad5c0c1750 msgr2=0x7fad5c1645d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad5c0c1750 0x7fad5c1645d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 -- 172.21.15.73:0/1604764939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad54009370 con 0x7fad5c0c0180 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad619f2700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fad5800b810 tx=0x7fad58009310).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800d5a0 con 0x7fad5c0c0180 2023-11-28T02:53:29.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad5800d740 con 0x7fad5c0c0180 2023-11-28T02:53:29.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5c0c2310 con 0x7fad5c0c0180 2023-11-28T02:53:29.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad5800db00 con 0x7fad5c0c0180 2023-11-28T02:53:29.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.116+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5c0c28d0 con 0x7fad5c0c0180 2023-11-28T02:53:29.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.117+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5c06c7c0 con 0x7fad5c0c0180 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.117+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fad58019460 con 0x7fad5c0c0180 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.118+0000 7fad51ffb700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 0x7fad40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.118+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fad5807a170 con 0x7fad5c0c0180 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.118+0000 7fad611f1700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 0x7fad40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.118+0000 7fad611f1700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 0x7fad40061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fad5c063f70 tx=0x7fad4c008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.122+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fad5804b3e0 con 0x7fad5c0c0180 2023-11-28T02:53:29.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.286+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad5c0c0f50 con 0x7fad5c0c0180 2023-11-28T02:53:29.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.287+0000 7fad51ffb700 1 -- 172.21.15.73:0/1604764939 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16941 (secure 0 0 0) 0x7fad5804ea00 con 0x7fad5c0c0180 2023-11-28T02:53:29.289 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:29.289 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:53:28.963319+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 msgr2=0x7fad40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 0x7fad40061610 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 rx=0x7fad5c063f70 tx=0x7fad4c008040).stop 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 msgr2=0x7fad5c150df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 rx=0x7fad5800b810 tx=0x7fad58009310).stop 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 shutdown_connections 2023-11-28T02:53:29.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad4005f150 0x7fad40061610 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad5c157660 0x7fad5c0ae980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad5c0c0180 0x7fad5c150df0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 --2- 172.21.15.73:0/1604764939 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad5c0c1750 0x7fad5c1645d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 >> 172.21.15.73:0/1604764939 conn(0x7fad5c0cb000 msgr2=0x7fad5c0c9410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 shutdown_connections 2023-11-28T02:53:29.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.289+0000 7fad629f4700 1 -- 172.21.15.73:0/1604764939 wait complete. 2023-11-28T02:53:29.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:29.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.416+0000 7f73920c9700 1 Processor -- start 2023-11-28T02:53:29.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.416+0000 7f73920c9700 1 -- start start 2023-11-28T02:53:29.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c1544d0 0x7f738c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 0x7f738c095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c0673c0 0x7f738c096370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c096860 con 0x7f738c1544d0 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f738c0683d0 con 0x7f738c066f40 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f73920c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c067850 con 0x7f738c0673c0 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.417+0000 7f738b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c1544d0 0x7f738c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f738c1544d0 0x7f738c155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45786/0 (socket says 172.21.15.73:45786) 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 -- 172.21.15.73:0/4146638487 learned_addr learned my addr 172.21.15.73:0/4146638487 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738affd700 1 --2- 172.21.15.73:0/4146638487 >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 0x7f738c095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 -- 172.21.15.73:0/4146638487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c0673c0 msgr2=0x7f738c096370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4146638487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c0673c0 0x7f738c096370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 -- 172.21.15.73:0/4146638487 >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 msgr2=0x7f738c095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4146638487 >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 0x7f738c095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 -- 172.21.15.73:0/4146638487 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f738c067b30 con 0x7f738c1544d0 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738affd700 1 --2- 172.21.15.73:0/4146638487 >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 0x7f738c095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:29.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4146638487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c1544d0 0x7f738c155750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f738c06a7d0 tx=0x7f737c009840).ready entity=mon.0 client_cookie=3e135c4dd9d5dee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738a7fc700 1 -- 172.21.15.73:0/4146638487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00e070 con 0x7f738c1544d0 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738a7fc700 1 -- 172.21.15.73:0/4146638487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00bad0 con 0x7f738c1544d0 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f738a7fc700 1 -- 172.21.15.73:0/4146638487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00be50 con 0x7f738c1544d0 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 -- 172.21.15.73:0/4146638487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c1544d0 msgr2=0x7f738c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 --2- 172.21.15.73:0/4146638487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c1544d0 0x7f738c155750 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 rx=0x7f738c06a7d0 tx=0x7f737c009840).stop 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 -- 172.21.15.73:0/4146638487 shutdown_connections 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 --2- 172.21.15.73:0/4146638487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c1544d0 0x7f738c155750 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 --2- 172.21.15.73:0/4146638487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c0673c0 0x7f738c096370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 --2- 172.21.15.73:0/4146638487 >> v2:172.21.15.150:3300/0 conn(0x7f738c066f40 0x7f738c095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.418+0000 7f73920c9700 1 -- 172.21.15.73:0/4146638487 >> 172.21.15.73:0/4146638487 conn(0x7f738c0631d0 msgr2=0x7f738c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.419+0000 7f73920c9700 1 -- 172.21.15.73:0/4146638487 shutdown_connections 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.419+0000 7f73920c9700 1 -- 172.21.15.73:0/4146638487 wait complete. 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.420+0000 7f73920c9700 1 Processor -- start 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.420+0000 7f73920c9700 1 -- start start 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.420+0000 7f73920c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c1544d0 0x7f738c0a93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 0x7f738c157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0acbb0 con 0x7f738c066f40 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738c156400 con 0x7f738c14edd0 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f738c0a98c0 con 0x7f738c1544d0 2023-11-28T02:53:29.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45790/0 (socket says 172.21.15.73:45790) 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 -- 172.21.15.73:0/4141843630 learned_addr learned my addr 172.21.15.73:0/4141843630 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738bfff700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 0x7f738c157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738affd700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c1544d0 0x7f738c0a93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 -- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c1544d0 msgr2=0x7f738c0a93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c1544d0 0x7f738c0a93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 -- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 msgr2=0x7f738c157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 0x7f738c157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 -- 172.21.15.73:0/4141843630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c009ae0 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738bfff700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 0x7f738c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f738b7fe700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f738c06c7f0 tx=0x7f737c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00be50 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c01f480 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f738c156ad0 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c01f840 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.421+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738c155b60 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.422+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f738c06c860 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.423+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f737c003040 con 0x7f738c066f40 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.423+0000 7f73737fe700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 0x7f7378061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:29.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.423+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f737c07bd80 con 0x7f738c066f40 2023-11-28T02:53:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.423+0000 7f738affd700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 0x7f7378061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.423+0000 7f738affd700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 0x7f7378061610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f738c0a7cd0 tx=0x7f7380008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:29.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.427+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f737c04d100 con 0x7f738c066f40 2023-11-28T02:53:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.591+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f738c14f5d0 con 0x7f738c066f40 2023-11-28T02:53:29.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.591+0000 7f73737fe700 1 -- 172.21.15.73:0/4141843630 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f737c024030 con 0x7f738c066f40 2023-11-28T02:53:29.593 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:29.593 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":33,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:53:28.963319+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:29.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 msgr2=0x7f7378061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 0x7f7378061610 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f738c0a7cd0 tx=0x7f7380008040).stop 2023-11-28T02:53:29.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 msgr2=0x7f738c0ac6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:29.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 rx=0x7f738c06c7f0 tx=0x7f737c009840).stop 2023-11-28T02:53:29.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 shutdown_connections 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f737805f150 0x7f7378061610 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f738c14edd0 0x7f738c157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f738c066f40 0x7f738c0ac6c0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 --2- 172.21.15.73:0/4141843630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f738c1544d0 0x7f738c0a93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 >> 172.21.15.73:0/4141843630 conn(0x7f738c0631d0 msgr2=0x7f738c0641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 shutdown_connections 2023-11-28T02:53:29.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.593+0000 7f73920c9700 1 -- 172.21.15.73:0/4141843630 wait complete. 2023-11-28T02:53:29.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T02:53:29.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.720+0000 7f8224693700 1 Processor -- start 2023-11-28T02:53:34.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.720+0000 7f8224693700 1 -- start start 2023-11-28T02:53:34.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8224693700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f821c158a30 0x7f821c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8224693700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c0c72d0 con 0x7f821c098d10 2023-11-28T02:53:34.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8224693700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f821c064570 con 0x7f821c0c6190 2023-11-28T02:53:34.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8224693700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f821c154bc0 con 0x7f821c158a30 2023-11-28T02:53:34.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8222c30700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f821c158a30 0x7f821c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f822242f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f821c098d10 0x7f821c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:58924/0 (socket says 172.21.15.73:58924) 2023-11-28T02:53:34.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f8222c30700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f821c158a30 0x7f821c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45794/0 (socket says 172.21.15.73:45794) 2023-11-28T02:53:34.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.721+0000 7f822242f700 1 -- 172.21.15.73:0/2275879284 learned_addr learned my addr 172.21.15.73:0/2275879284 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:34.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8222c30700 1 -- 172.21.15.73:0/2275879284 >> v1:172.21.15.150:6789/0 conn(0x7f821c0c6190 legacy=0x7f821c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8222c30700 1 -- 172.21.15.73:0/2275879284 >> v1:172.21.15.73:6789/0 conn(0x7f821c098d10 legacy=0x7f821c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8222c30700 1 -- 172.21.15.73:0/2275879284 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f821c0c66c0 con 0x7f821c158a30 2023-11-28T02:53:34.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8222c30700 1 --2- 172.21.15.73:0/2275879284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0c7760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f8214009760 tx=0x7f8214009060).ready entity=mon.0 client_cookie=7955a3696561fdb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f822142d700 1 -- 172.21.15.73:0/2275879284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400f510 con 0x7f821c158a30 2023-11-28T02:53:34.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f822142d700 1 -- 172.21.15.73:0/2275879284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821400f670 con 0x7f821c158a30 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f822142d700 1 -- 172.21.15.73:0/2275879284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400f9f0 con 0x7f821c158a30 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8224693700 1 -- 172.21.15.73:0/2275879284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 msgr2=0x7f821c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8224693700 1 --2- 172.21.15.73:0/2275879284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0c7760 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 rx=0x7f8214009760 tx=0x7f8214009060).stop 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8224693700 1 -- 172.21.15.73:0/2275879284 shutdown_connections 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8224693700 1 --2- 172.21.15.73:0/2275879284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0c7760 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.722+0000 7f8224693700 1 -- 172.21.15.73:0/2275879284 >> 172.21.15.73:0/2275879284 conn(0x7f821c0cb310 msgr2=0x7f821c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.723+0000 7f8224693700 1 -- 172.21.15.73:0/2275879284 shutdown_connections 2023-11-28T02:53:34.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.723+0000 7f8224693700 1 -- 172.21.15.73:0/2275879284 wait complete. 2023-11-28T02:53:34.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 Processor -- start 2023-11-28T02:53:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 -- start start 2023-11-28T02:53:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 0x7f821c1515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f821c0c6190 0x7f821c0c3560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b0e90 con 0x7f821c158a30 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b1320 con 0x7f821c098d10 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8224693700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f821c0b0280 con 0x7f821c0c6190 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.724+0000 7f8222c30700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45806/0 (socket says 172.21.15.73:45806) 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 -- 172.21.15.73:0/708114903 learned_addr learned my addr 172.21.15.73:0/708114903 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:34.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f822242f700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 0x7f821c1515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8221c2e700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f821c0c6190 0x7f821c0c3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 -- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f821c0c6190 msgr2=0x7f821c0c3560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f821c0c6190 0x7f821c0c3560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 -- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 msgr2=0x7f821c1515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 0x7f821c1515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 -- 172.21.15.73:0/708114903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82140092a0 con 0x7f821c158a30 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f822242f700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 0x7f821c1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8222c30700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f821c0c2360 tx=0x7f8214017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821400f9f0 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821401b410 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f821c0b0560 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821401b7d0 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.725+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f821c0b0b20 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.726+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f821c0b1040 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.726+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f821401bb40 con 0x7f821c158a30 2023-11-28T02:53:34.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.727+0000 7f82127fc700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 0x7f82000615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.727+0000 7f822242f700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 0x7f82000615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.727+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f821407bc90 con 0x7f821c158a30 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.727+0000 7f822242f700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 0x7f82000615e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f821c0a9a50 tx=0x7f8218009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.731+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f821404cf60 con 0x7f821c158a30 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.937+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f821c06c860 con 0x7f821c158a30 2023-11-28T02:53:34.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.971+0000 7f82127fc700 1 -- 172.21.15.73:0/708114903 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v11) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8214029020 con 0x7f821c158a30 2023-11-28T02:53:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 msgr2=0x7f82000615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 0x7f82000615e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f821c0a9a50 tx=0x7f8218009040).stop 2023-11-28T02:53:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 msgr2=0x7f821c0a8ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7f821c0c2360 tx=0x7f8214017000).stop 2023-11-28T02:53:34.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 shutdown_connections 2023-11-28T02:53:34.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f820005f120 0x7f82000615e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f821c158a30 0x7f821c0a8ef0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f821c098d10 0x7f821c1515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 --2- 172.21.15.73:0/708114903 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f821c0c6190 0x7f821c0c3560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 >> 172.21.15.73:0/708114903 conn(0x7f821c0cb310 msgr2=0x7f821c063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 shutdown_connections 2023-11-28T02:53:34.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:29.974+0000 7f8224693700 1 -- 172.21.15.73:0/708114903 wait complete. 2023-11-28T02:53:34.058 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T02:53:34.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T02:53:34.059 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T02:53:32.581+0000 7f0f5b60d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T02:53:34.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.184+0000 7fdf92468700 1 Processor -- start 2023-11-28T02:53:34.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.184+0000 7fdf92468700 1 -- start start 2023-11-28T02:53:34.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.184+0000 7fdf92468700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.184+0000 7fdf92468700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 0x7fdf8c15c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf92468700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf8c064090 con 0x7fdf8c152510 2023-11-28T02:53:34.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf92468700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf8c063480 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf92468700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c063760 con 0x7fdf8c0c6950 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf90a05700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 0x7fdf8c15c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35910/0 (socket says 172.21.15.73:35910) 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf90a05700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 0x7fdf8c15c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:51562/0 (socket says 172.21.15.73:51562) 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 -- 172.21.15.73:0/1160169425 learned_addr learned my addr 172.21.15.73:0/1160169425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8bfff700 1 --1- 172.21.15.73:0/1160169425 >> v1:172.21.15.150:6789/0 conn(0x7fdf8c152510 0x7fdf8c0a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59028/0 (socket says 172.21.15.73:59028) 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8affd700 1 -- 172.21.15.73:0/1160169425 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42448313 0 0) 0x7fdf8c064090 con 0x7fdf8c152510 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8affd700 1 -- 172.21.15.73:0/1160169425 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdf740033f0 con 0x7fdf8c152510 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 -- 172.21.15.73:0/1160169425 >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 msgr2=0x7fdf8c15c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 --2- 172.21.15.73:0/1160169425 >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 0x7fdf8c15c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 -- 172.21.15.73:0/1160169425 >> v1:172.21.15.150:6789/0 conn(0x7fdf8c152510 legacy=0x7fdf8c0a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.185+0000 7fdf8b7fe700 1 -- 172.21.15.73:0/1160169425 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf8c063a40 con 0x7fdf8c0c6950 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf8b7fe700 1 --2- 172.21.15.73:0/1160169425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdf80009810 tx=0x7fdf80009060).ready entity=mon.2 client_cookie=7bb1ce27bde5d6b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf8affd700 1 -- 172.21.15.73:0/1160169425 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80005880 con 0x7fdf8c0c6950 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf8affd700 1 -- 172.21.15.73:0/1160169425 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf800059e0 con 0x7fdf8c0c6950 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf8affd700 1 -- 172.21.15.73:0/1160169425 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf80005d60 con 0x7fdf8c0c6950 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 -- 172.21.15.73:0/1160169425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 msgr2=0x7fdf8c0bc6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 --2- 172.21.15.73:0/1160169425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdf80009810 tx=0x7fdf80009060).stop 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 -- 172.21.15.73:0/1160169425 shutdown_connections 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 --2- 172.21.15.73:0/1160169425 >> v2:172.21.15.150:3300/0 conn(0x7fdf8c0bcbb0 0x7fdf8c15c170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 --2- 172.21.15.73:0/1160169425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c0bc6c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 -- 172.21.15.73:0/1160169425 >> 172.21.15.73:0/1160169425 conn(0x7fdf8c0c8230 msgr2=0x7fdf8c0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 -- 172.21.15.73:0/1160169425 shutdown_connections 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.186+0000 7fdf92468700 1 -- 172.21.15.73:0/1160169425 wait complete. 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.187+0000 7fdf92468700 1 Processor -- start 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 -- start start 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c149da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 0x7fdf8c0ab120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c153ae0 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c155790 con 0x7fdf8c152510 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.188+0000 7fdf92468700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf8c168490 con 0x7fdf8c0c6950 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38746/0 (socket says 172.21.15.73:38746) 2023-11-28T02:53:34.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 -- 172.21.15.73:0/1335516253 learned_addr learned my addr 172.21.15.73:0/1335516253 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8b7fe700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c149da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf90a05700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 0x7fdf8c0ab120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 -- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 msgr2=0x7fdf8c149da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c149da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 -- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 msgr2=0x7fdf8c0ab120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 0x7fdf8c0ab120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 -- 172.21.15.73:0/1335516253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf80009300 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf90a05700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 0x7fdf8c0ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf8bfff700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fdf7c00c440 tx=0x7fdf7c007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c00cea0 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf7c0072e0 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf8c0ab610 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c005730 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.189+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf8c153800 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.190+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf8c168770 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.191+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdf7c005b20 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.191+0000 7fdf73fff700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 0x7fdf78061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.191+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdf7c0799d0 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.191+0000 7fdf8b7fe700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 0x7fdf78061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.191+0000 7fdf8b7fe700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 0x7fdf78061610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdf80009fd0 tx=0x7fdf80016000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.195+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdf7c04aca0 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.397+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdf8c06c860 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.459+0000 7fdf73fff700 1 -- 172.21.15.73:0/1335516253 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v12) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdf7c018070 con 0x7fdf8c0bcbb0 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 msgr2=0x7fdf78061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 0x7fdf78061610 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7fdf80009fd0 tx=0x7fdf80016000).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 msgr2=0x7fdf8c0c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 rx=0x7fdf7c00c440 tx=0x7fdf7c007c70).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 shutdown_connections 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdf7805f150 0x7fdf78061610 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdf8c152510 0x7fdf8c0ab120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf8c0bcbb0 0x7fdf8c0c39a0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 --2- 172.21.15.73:0/1335516253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf8c0c6950 0x7fdf8c149da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 >> 172.21.15.73:0/1335516253 conn(0x7fdf8c0c8230 msgr2=0x7fdf8c150cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 shutdown_connections 2023-11-28T02:53:34.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.461+0000 7fdf92468700 1 -- 172.21.15.73:0/1335516253 wait complete. 2023-11-28T02:53:34.474 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T02:53:34.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:34.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.585+0000 7f4236a9d700 1 Processor -- start 2023-11-28T02:53:34.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.585+0000 7f4236a9d700 1 -- start start 2023-11-28T02:53:34.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4236a9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42300c0180 0x7f4230157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4236a9d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42300c12c0 con 0x7f42300c1750 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4236a9d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42300c06b0 con 0x7f4230157660 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4236a9d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42300c0990 con 0x7f42300c0180 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42300c0180 0x7f4230157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423629c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f42300c1750 0x7f42300648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33732/0 (socket says 172.21.15.73:33732) 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f42300c0180 0x7f4230157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38762/0 (socket says 172.21.15.73:38762) 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423629c700 1 -- 172.21.15.73:0/3417390959 learned_addr learned my addr 172.21.15.73:0/3417390959 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 -- 172.21.15.73:0/3417390959 >> v1:172.21.15.150:6789/0 conn(0x7f4230157660 legacy=0x7f42301589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 -- 172.21.15.73:0/3417390959 >> v1:172.21.15.73:6789/0 conn(0x7f42300c1750 legacy=0x7f42300648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 -- 172.21.15.73:0/3417390959 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42300c0c70 con 0x7f42300c0180 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4234a99700 1 -- 172.21.15.73:0/3417390959 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1428603206 0 0) 0x7f42300c12c0 con 0x7f42300c1750 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f423529a700 1 --2- 172.21.15.73:0/3417390959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42300c0180 0x7f4230157a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f4220009760 tx=0x7f4220009060).ready entity=mon.0 client_cookie=dc0fb8e4a82c136a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4234a99700 1 -- 172.21.15.73:0/3417390959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000b580 con 0x7f42300c0180 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.586+0000 7f4234a99700 1 -- 172.21.15.73:0/3417390959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f422000b6e0 con 0x7f42300c0180 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4234a99700 1 -- 172.21.15.73:0/3417390959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f422000ba60 con 0x7f42300c0180 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 -- 172.21.15.73:0/3417390959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42300c0180 msgr2=0x7f4230157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 --2- 172.21.15.73:0/3417390959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42300c0180 0x7f4230157a40 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 rx=0x7f4220009760 tx=0x7f4220009060).stop 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 -- 172.21.15.73:0/3417390959 shutdown_connections 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 --2- 172.21.15.73:0/3417390959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f42300c0180 0x7f4230157a40 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 -- 172.21.15.73:0/3417390959 >> 172.21.15.73:0/3417390959 conn(0x7f42300cb000 msgr2=0x7f42300cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 -- 172.21.15.73:0/3417390959 shutdown_connections 2023-11-28T02:53:34.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.587+0000 7f4236a9d700 1 -- 172.21.15.73:0/3417390959 wait complete. 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.588+0000 7f4236a9d700 1 Processor -- start 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.588+0000 7f4236a9d700 1 -- start start 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 0x7f4230168900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42300c1750 0x7f42301649e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42300aaa80 con 0x7f4230157660 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4230165930 con 0x7f42300c0180 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4236a9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4230168470 con 0x7f42300c1750 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38768/0 (socket says 172.21.15.73:38768) 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 -- 172.21.15.73:0/1986448848 learned_addr learned my addr 172.21.15.73:0/1986448848 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 -- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42300c1750 msgr2=0x7f42301649e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42300c1750 0x7f42301649e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4235a9b700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 0x7f4230168900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 -- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 msgr2=0x7f4230168900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 0x7f4230168900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 -- 172.21.15.73:0/1986448848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42200092a0 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f4235a9b700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 0x7f4230168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.589+0000 7f423629c700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f422800c7e0 tx=0x7f4228007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.590+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228005d30 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.590+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4228005e90 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.590+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4230165370 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.590+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4228007340 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.590+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4230168100 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.591+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f42280076f0 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.591+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42301685e0 con 0x7f4230157660 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.591+0000 7f4225ffb700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 0x7f4214061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.591+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f422807a200 con 0x7f4230157660 2023-11-28T02:53:34.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.591+0000 7f4235a9b700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 0x7f4214061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.592+0000 7f4235a9b700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 0x7f4214061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f422c0016a0 tx=0x7f422c009040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.595+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f422804b4d0 con 0x7f4230157660 2023-11-28T02:53:34.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.798+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f423006c7c0 con 0x7f4230157660 2023-11-28T02:53:34.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.798+0000 7f4225ffb700 1 -- 172.21.15.73:0/1986448848 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f422804eaf0 con 0x7f4230157660 2023-11-28T02:53:34.799 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:34.800 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T02:53:34.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 msgr2=0x7f4214061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 0x7f4214061610 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f422c0016a0 tx=0x7f422c009040).stop 2023-11-28T02:53:34.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 msgr2=0x7f4230152d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 rx=0x7f422800c7e0 tx=0x7f4228007f90).stop 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 shutdown_connections 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f421405f150 0x7f4214061610 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4230157660 0x7f4230152d50 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f42300c0180 0x7f4230168900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 --2- 172.21.15.73:0/1986448848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f42300c1750 0x7f42301649e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 >> 172.21.15.73:0/1986448848 conn(0x7f42300cb000 msgr2=0x7f42300abc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 shutdown_connections 2023-11-28T02:53:34.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.800+0000 7f4236a9d700 1 -- 172.21.15.73:0/1986448848 wait complete. 2023-11-28T02:53:34.803 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T02:53:34.813 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T02:53:34.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.924+0000 7fafc57a7700 1 Processor -- start 2023-11-28T02:53:34.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.924+0000 7fafc57a7700 1 -- start start 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafc57a7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fafc01520c0 0x7fafc0066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafc57a7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafc0066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafc57a7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc00be110 con 0x7fafc01524a0 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafc57a7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc0066fc0 con 0x7fafc01520c0 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafc57a7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00672a0 con 0x7fafc0068150 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafb67fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fafc01524a0 0x7fafc0067a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33742/0 (socket says 172.21.15.73:33742) 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.925+0000 7fafb67fc700 1 -- 172.21.15.73:0/2780674159 learned_addr learned my addr 172.21.15.73:0/2780674159 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:34.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> v2:172.21.15.150:3300/0 conn(0x7fafc01520c0 0x7fafc0066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbf7fe700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafc0066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1123081634 0 0) 0x7fafc00be110 con 0x7fafc01524a0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafac0033f0 con 0x7fafc01524a0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 187458035 0 0) 0x7fafac0033f0 con 0x7fafc01524a0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 msgr2=0x7fafc0066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafc0066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 >> v2:172.21.15.150:3300/0 conn(0x7fafc01520c0 msgr2=0x7fafc0066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/2780674159 >> v2:172.21.15.150:3300/0 conn(0x7fafc01520c0 0x7fafc0066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00be380 con 0x7fafc01524a0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fafa8001470 con 0x7fafc01524a0 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> v2:172.21.15.150:3300/0 conn(0x7fafc01520c0 0x7fafc0066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:34.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 >> v1:172.21.15.73:6789/0 conn(0x7fafc01524a0 legacy=0x7fafc0067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac0038e0 0x7fafac003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac004690 0x7fafac004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafac005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0066fc0 con 0x7fafac0038e0 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc00672a0 con 0x7fafac004690 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafac00b8e0 con 0x7fafc0068150 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac0038e0 0x7fafac003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafb67fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac004690 0x7fafac004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbf7fe700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafac005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 -- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 msgr2=0x7fafac005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafac005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 -- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac004690 msgr2=0x7fafac004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac004690 0x7fafac004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 -- 172.21.15.73:0/2780674159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafac0033f0 con 0x7fafac0038e0 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbf7fe700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0068150 0x7fafac005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafb67fc700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac004690 0x7fafac004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.926+0000 7fafbeffd700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac0038e0 0x7fafac003d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fafb0009d00 tx=0x7fafb0006f40).ready entity=mon.0 client_cookie=44046535e910ee54 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb0009e60 con 0x7fafac0038e0 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb0009670 con 0x7fafac0038e0 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafbe7fc700 1 -- 172.21.15.73:0/2780674159 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb000eb30 con 0x7fafac0038e0 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 -- 172.21.15.73:0/2780674159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac0038e0 msgr2=0x7fafac003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 --2- 172.21.15.73:0/2780674159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafac0038e0 0x7fafac003d00 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 rx=0x7fafb0009d00 tx=0x7fafb0006f40).stop 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafbf7fe700 1 -- 172.21.15.73:0/2780674159 reap_dead start 2023-11-28T02:53:34.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 -- 172.21.15.73:0/2780674159 shutdown_connections 2023-11-28T02:53:34.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 -- 172.21.15.73:0/2780674159 >> 172.21.15.73:0/2780674159 conn(0x7fafc00631d0 msgr2=0x7fafc00c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:34.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 -- 172.21.15.73:0/2780674159 shutdown_connections 2023-11-28T02:53:34.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.927+0000 7fafc57a7700 1 -- 172.21.15.73:0/2780674159 wait complete. 2023-11-28T02:53:34.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.928+0000 7fafc57a7700 1 Processor -- start 2023-11-28T02:53:34.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.928+0000 7fafc57a7700 1 -- start start 2023-11-28T02:53:34.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 0x7fafc014d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0162da0 0x7fafc00c75f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0151830 con 0x7fafc014fde0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc0150eb0 con 0x7fafac0038e0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafc57a7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc01502a0 con 0x7fafc0162da0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafb67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafb67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38796/0 (socket says 172.21.15.73:38796) 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafb67fc700 1 -- 172.21.15.73:0/224339723 learned_addr learned my addr 172.21.15.73:0/224339723 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafbeffd700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 0x7fafc014d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.929+0000 7fafbf7fe700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0162da0 0x7fafc00c75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 -- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0162da0 msgr2=0x7fafc00c75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0162da0 0x7fafc00c75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 -- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 msgr2=0x7fafc014d830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 0x7fafc014d830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 -- 172.21.15.73:0/224339723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc0150580 con 0x7fafc014fde0 2023-11-28T02:53:34.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafbeffd700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 0x7fafc014d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb67fc700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fafa800e7b0 tx=0x7fafa8001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8010520 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafa8010680 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc0150860 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafa8010a00 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.930+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc014e730 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.931+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc0151040 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.931+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fafa8014430 con 0x7fafc014fde0 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.931+0000 7fafb77fe700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 0x7fafa0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:34.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.932+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fafa8079cf0 con 0x7fafc014fde0 2023-11-28T02:53:34.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.932+0000 7fafbeffd700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 0x7fafa0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:34.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.932+0000 7fafbeffd700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 0x7fafa0061610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fafb0009820 tx=0x7fafb0019000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:34.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:34.935+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fafa8049af0 con 0x7fafc014fde0 2023-11-28T02:53:35.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.098+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fafc006c860 con 0x7fafc014fde0 2023-11-28T02:53:35.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.099+0000 7fafb77fe700 1 -- 172.21.15.73:0/224339723 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16928 (secure 0 0 0) 0x7fafa807e070 con 0x7fafc014fde0 2023-11-28T02:53:35.101 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:35.101 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T02:53:29.965179+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 msgr2=0x7fafa0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 0x7fafa0061610 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7fafb0009820 tx=0x7fafb0019000).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 msgr2=0x7fafc0151340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 rx=0x7fafa800e7b0 tx=0x7fafa8001e90).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 shutdown_connections 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa005f150 0x7fafa0061610 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc0162da0 0x7fafc00c75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafac0038e0 0x7fafc014d830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 --2- 172.21.15.73:0/224339723 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc014fde0 0x7fafc0151340 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 >> 172.21.15.73:0/224339723 conn(0x7fafc00631d0 msgr2=0x7fafc00c21c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 shutdown_connections 2023-11-28T02:53:35.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.101+0000 7fafc57a7700 1 -- 172.21.15.73:0/224339723 wait complete. 2023-11-28T02:53:35.115 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T02:53:35.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:53:35.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.226+0000 7f7fc66b3700 1 Processor -- start 2023-11-28T02:53:35.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.226+0000 7f7fc66b3700 1 -- start start 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc66b3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fc0157660 0x7f7fc0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc66b3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fc00c1750 0x7f7fc00c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc66b3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fc00c12c0 con 0x7f7fc00c0180 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc66b3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc00c06b0 con 0x7f7fc00c1750 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc66b3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fc00c0990 con 0x7f7fc0157660 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fc00c1750 0x7f7fc00c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc4eb0700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7fc00c0180 0x7f7fc01589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33750/0 (socket says 172.21.15.73:33750) 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fc00c1750 0x7f7fc00c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38812/0 (socket says 172.21.15.73:38812) 2023-11-28T02:53:35.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc4eb0700 1 -- 172.21.15.73:0/1168845675 learned_addr learned my addr 172.21.15.73:0/1168845675 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc56b1700 1 --2- 172.21.15.73:0/1168845675 >> v2:172.21.15.150:3300/0 conn(0x7f7fc0157660 0x7f7fc0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/1168845675 >> v2:172.21.15.150:3300/0 conn(0x7f7fc0157660 msgr2=0x7f7fc0157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 --2- 172.21.15.73:0/1168845675 >> v2:172.21.15.150:3300/0 conn(0x7f7fc0157660 0x7f7fc0157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/1168845675 >> v1:172.21.15.73:6789/0 conn(0x7f7fc00c0180 legacy=0x7f7fc01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fb7fff700 1 -- 172.21.15.73:0/1168845675 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796558777 0 0) 0x7f7fc00c12c0 con 0x7f7fc00c0180 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.227+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/1168845675 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc00c0c70 con 0x7f7fc00c1750 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc5eb2700 1 --2- 172.21.15.73:0/1168845675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc00c1750 0x7f7fc00c1b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f7fc00839a0 tx=0x7f7fb8009780).ready entity=mon.0 client_cookie=e57749efdf54ab3d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fb7fff700 1 -- 172.21.15.73:0/1168845675 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb800e070 con 0x7f7fc00c1750 2023-11-28T02:53:35.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fb7fff700 1 -- 172.21.15.73:0/1168845675 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb800bc30 con 0x7f7fc00c1750 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fb7fff700 1 -- 172.21.15.73:0/1168845675 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8012410 con 0x7f7fc00c1750 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 -- 172.21.15.73:0/1168845675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc00c1750 msgr2=0x7f7fc00c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/1168845675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc00c1750 0x7f7fc00c1b70 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 rx=0x7f7fc00839a0 tx=0x7f7fb8009780).stop 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 -- 172.21.15.73:0/1168845675 shutdown_connections 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/1168845675 >> v2:172.21.15.150:3300/0 conn(0x7f7fc0157660 0x7f7fc0157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/1168845675 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc00c1750 0x7f7fc00c1b70 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 -- 172.21.15.73:0/1168845675 >> 172.21.15.73:0/1168845675 conn(0x7f7fc00cb000 msgr2=0x7f7fc00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 -- 172.21.15.73:0/1168845675 shutdown_connections 2023-11-28T02:53:35.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.228+0000 7f7fc66b3700 1 -- 172.21.15.73:0/1168845675 wait complete. 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.229+0000 7f7fc66b3700 1 Processor -- start 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.229+0000 7f7fc66b3700 1 -- start start 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fc00c0180 0x7f7fc00b0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 0x7f7fc0147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc0148360 con 0x7f7fc0157660 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00c4550 con 0x7f7fc00c0180 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc66b3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fc00c3940 con 0x7f7fc00c1750 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc5eb2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc5eb2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38820/0 (socket says 172.21.15.73:38820) 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.230+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/737522025 learned_addr learned my addr 172.21.15.73:0/737522025 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc56b1700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fc00c0180 0x7f7fc00b0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc4eb0700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 0x7f7fc0147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 msgr2=0x7f7fc0147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 0x7f7fc0147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fc00c0180 msgr2=0x7f7fc00b0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fc00c0180 0x7f7fc00b0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 -- 172.21.15.73:0/737522025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fb8009a20 con 0x7f7fc0157660 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc4eb0700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 0x7f7fc0147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc5eb2700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f7fc0084570 tx=0x7f7fc00c1230).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb80116d0 con 0x7f7fc0157660 2023-11-28T02:53:35.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fb8011830 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fb8011bf0 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fc00c3c20 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.231+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fc00c41e0 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.232+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fc00c46e0 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.233+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7fb8003040 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.233+0000 7f7fb57fa700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 0x7f7fa4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.233+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7fb807cc60 con 0x7f7fc0157660 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.233+0000 7f7fc56b1700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 0x7f7fa4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.234+0000 7f7fc56b1700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 0x7f7fa4061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f7fbc00bb10 tx=0x7f7fbc008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.237+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7fb804de70 con 0x7f7fc0157660 2023-11-28T02:53:35.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.396+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7fc006c7c0 con 0x7f7fa405f150 2023-11-28T02:53:35.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.405+0000 7f7fb57fa700 1 -- 172.21.15.73:0/737522025 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336455 (secure 0 0 0) 0x7f7fc006c7c0 con 0x7f7fa405f150 2023-11-28T02:53:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 msgr2=0x7f7fa4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 0x7f7fa4061610 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f7fbc00bb10 tx=0x7f7fbc008040).stop 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 msgr2=0x7f7fc00c3160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 rx=0x7f7fc0084570 tx=0x7f7fc00c1230).stop 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 shutdown_connections 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fa405f150 0x7f7fa4061610 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fc0157660 0x7f7fc00c3160 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fc00c0180 0x7f7fc00b0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 --2- 172.21.15.73:0/737522025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fc00c1750 0x7f7fc0147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 >> 172.21.15.73:0/737522025 conn(0x7f7fc00cb000 msgr2=0x7f7fc00ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 shutdown_connections 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.409+0000 7f7fc66b3700 1 -- 172.21.15.73:0/737522025 wait complete. 2023-11-28T02:53:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:53:35.422 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":56,"stamp":"2023-11-28T02:53:33.399348+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":237,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379840,"kb_used_data":3072,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594880,"statfs":{"total":773094113280,"available":770657157120,"internally_reserved":0,"allocated":3145728,"data_stored":1274244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4305,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.168820"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994329+0000","last_change":"2023-11-28T02:53:28.987670+0000","last_active":"2023-11-28T02:53:28.994329+0000","last_peered":"2023-11-28T02:53:28.994329+0000","last_clean":"2023-11-28T02:53:28.994329+0000","last_became_active":"2023-11-28T02:53:28.987421+0000","last_became_peered":"2023-11-28T02:53:28.987421+0000","last_unstale":"2023-11-28T02:53:28.994329+0000","last_undegraded":"2023-11-28T02:53:28.994329+0000","last_fullsized":"2023-11-28T02:53:28.994329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997134+0000","last_change":"2023-11-28T02:53:28.615436+0000","last_active":"2023-11-28T02:53:28.997134+0000","last_peered":"2023-11-28T02:53:28.997134+0000","last_clean":"2023-11-28T02:53:28.997134+0000","last_became_active":"2023-11-28T02:53:28.615171+0000","last_became_peered":"2023-11-28T02:53:28.615171+0000","last_unstale":"2023-11-28T02:53:28.997134+0000","last_undegraded":"2023-11-28T02:53:28.997134+0000","last_fullsized":"2023-11-28T02:53:28.997134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991501+0000","last_change":"2023-11-28T02:53:28.988359+0000","last_active":"2023-11-28T02:53:28.991501+0000","last_peered":"2023-11-28T02:53:28.991501+0000","last_clean":"2023-11-28T02:53:28.991501+0000","last_became_active":"2023-11-28T02:53:28.987894+0000","last_became_peered":"2023-11-28T02:53:28.987894+0000","last_unstale":"2023-11-28T02:53:28.991501+0000","last_undegraded":"2023-11-28T02:53:28.991501+0000","last_fullsized":"2023-11-28T02:53:28.991501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974610+0000","last_change":"2023-11-28T02:53:28.607829+0000","last_active":"2023-11-28T02:53:28.974610+0000","last_peered":"2023-11-28T02:53:28.974610+0000","last_clean":"2023-11-28T02:53:28.974610+0000","last_became_active":"2023-11-28T02:53:28.607574+0000","last_became_peered":"2023-11-28T02:53:28.607574+0000","last_unstale":"2023-11-28T02:53:28.974610+0000","last_undegraded":"2023-11-28T02:53:28.974610+0000","last_fullsized":"2023-11-28T02:53:28.974610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987889+0000","last_change":"2023-11-28T02:53:28.982914+0000","last_active":"2023-11-28T02:53:28.987889+0000","last_peered":"2023-11-28T02:53:28.987889+0000","last_clean":"2023-11-28T02:53:28.987889+0000","last_became_active":"2023-11-28T02:53:28.982648+0000","last_became_peered":"2023-11-28T02:53:28.982648+0000","last_unstale":"2023-11-28T02:53:28.987889+0000","last_undegraded":"2023-11-28T02:53:28.987889+0000","last_fullsized":"2023-11-28T02:53:28.987889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.981403+0000","last_change":"2023-11-28T02:53:28.614003+0000","last_active":"2023-11-28T02:53:28.981403+0000","last_peered":"2023-11-28T02:53:28.981403+0000","last_clean":"2023-11-28T02:53:28.981403+0000","last_became_active":"2023-11-28T02:53:28.613504+0000","last_became_peered":"2023-11-28T02:53:28.613504+0000","last_unstale":"2023-11-28T02:53:28.981403+0000","last_undegraded":"2023-11-28T02:53:28.981403+0000","last_fullsized":"2023-11-28T02:53:28.981403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986511+0000","last_change":"2023-11-28T02:53:28.980516+0000","last_active":"2023-11-28T02:53:28.986511+0000","last_peered":"2023-11-28T02:53:28.986511+0000","last_clean":"2023-11-28T02:53:28.986511+0000","last_became_active":"2023-11-28T02:53:28.980254+0000","last_became_peered":"2023-11-28T02:53:28.980254+0000","last_unstale":"2023-11-28T02:53:28.986511+0000","last_undegraded":"2023-11-28T02:53:28.986511+0000","last_fullsized":"2023-11-28T02:53:28.986511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990633+0000","last_change":"2023-11-28T02:53:28.625815+0000","last_active":"2023-11-28T02:53:28.990633+0000","last_peered":"2023-11-28T02:53:28.990633+0000","last_clean":"2023-11-28T02:53:28.990633+0000","last_became_active":"2023-11-28T02:53:28.625542+0000","last_became_peered":"2023-11-28T02:53:28.625542+0000","last_unstale":"2023-11-28T02:53:28.990633+0000","last_undegraded":"2023-11-28T02:53:28.990633+0000","last_fullsized":"2023-11-28T02:53:28.990633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994331+0000","last_change":"2023-11-28T02:53:28.991711+0000","last_active":"2023-11-28T02:53:28.994331+0000","last_peered":"2023-11-28T02:53:28.994331+0000","last_clean":"2023-11-28T02:53:28.994331+0000","last_became_active":"2023-11-28T02:53:28.991447+0000","last_became_peered":"2023-11-28T02:53:28.991447+0000","last_unstale":"2023-11-28T02:53:28.994331+0000","last_undegraded":"2023-11-28T02:53:28.994331+0000","last_fullsized":"2023-11-28T02:53:28.994331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994166+0000","last_change":"2023-11-28T02:53:28.606830+0000","last_active":"2023-11-28T02:53:28.994166+0000","last_peered":"2023-11-28T02:53:28.994166+0000","last_clean":"2023-11-28T02:53:28.994166+0000","last_became_active":"2023-11-28T02:53:28.606253+0000","last_became_peered":"2023-11-28T02:53:28.606253+0000","last_unstale":"2023-11-28T02:53:28.994166+0000","last_undegraded":"2023-11-28T02:53:28.994166+0000","last_fullsized":"2023-11-28T02:53:28.994166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993557+0000","last_change":"2023-11-28T02:53:28.987243+0000","last_active":"2023-11-28T02:53:28.993557+0000","last_peered":"2023-11-28T02:53:28.993557+0000","last_clean":"2023-11-28T02:53:28.993557+0000","last_became_active":"2023-11-28T02:53:28.986679+0000","last_became_peered":"2023-11-28T02:53:28.986679+0000","last_unstale":"2023-11-28T02:53:28.993557+0000","last_undegraded":"2023-11-28T02:53:28.993557+0000","last_fullsized":"2023-11-28T02:53:28.993557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989015+0000","last_change":"2023-11-28T02:53:28.614186+0000","last_active":"2023-11-28T02:53:28.989015+0000","last_peered":"2023-11-28T02:53:28.989015+0000","last_clean":"2023-11-28T02:53:28.989015+0000","last_became_active":"2023-11-28T02:53:28.613904+0000","last_became_peered":"2023-11-28T02:53:28.613904+0000","last_unstale":"2023-11-28T02:53:28.989015+0000","last_undegraded":"2023-11-28T02:53:28.989015+0000","last_fullsized":"2023-11-28T02:53:28.989015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.982877+0000","last_change":"2023-11-28T02:53:28.978510+0000","last_active":"2023-11-28T02:53:28.982877+0000","last_peered":"2023-11-28T02:53:28.982877+0000","last_clean":"2023-11-28T02:53:28.982877+0000","last_became_active":"2023-11-28T02:53:28.978249+0000","last_became_peered":"2023-11-28T02:53:28.978249+0000","last_unstale":"2023-11-28T02:53:28.982877+0000","last_undegraded":"2023-11-28T02:53:28.982877+0000","last_fullsized":"2023-11-28T02:53:28.982877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973327+0000","last_change":"2023-11-28T02:53:28.621237+0000","last_active":"2023-11-28T02:53:28.973327+0000","last_peered":"2023-11-28T02:53:28.973327+0000","last_clean":"2023-11-28T02:53:28.973327+0000","last_became_active":"2023-11-28T02:53:28.620850+0000","last_became_peered":"2023-11-28T02:53:28.620850+0000","last_unstale":"2023-11-28T02:53:28.973327+0000","last_undegraded":"2023-11-28T02:53:28.973327+0000","last_fullsized":"2023-11-28T02:53:28.973327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994316+0000","last_change":"2023-11-28T02:53:28.987573+0000","last_active":"2023-11-28T02:53:28.994316+0000","last_peered":"2023-11-28T02:53:28.994316+0000","last_clean":"2023-11-28T02:53:28.994316+0000","last_became_active":"2023-11-28T02:53:28.987293+0000","last_became_peered":"2023-11-28T02:53:28.987293+0000","last_unstale":"2023-11-28T02:53:28.994316+0000","last_undegraded":"2023-11-28T02:53:28.994316+0000","last_fullsized":"2023-11-28T02:53:28.994316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973596+0000","last_change":"2023-11-28T02:53:28.619795+0000","last_active":"2023-11-28T02:53:28.973596+0000","last_peered":"2023-11-28T02:53:28.973596+0000","last_clean":"2023-11-28T02:53:28.973596+0000","last_became_active":"2023-11-28T02:53:28.619452+0000","last_became_peered":"2023-11-28T02:53:28.619452+0000","last_unstale":"2023-11-28T02:53:28.973596+0000","last_undegraded":"2023-11-28T02:53:28.973596+0000","last_fullsized":"2023-11-28T02:53:28.973596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990559+0000","last_change":"2023-11-28T02:53:28.986425+0000","last_active":"2023-11-28T02:53:28.990559+0000","last_peered":"2023-11-28T02:53:28.990559+0000","last_clean":"2023-11-28T02:53:28.990559+0000","last_became_active":"2023-11-28T02:53:28.986130+0000","last_became_peered":"2023-11-28T02:53:28.986130+0000","last_unstale":"2023-11-28T02:53:28.990559+0000","last_undegraded":"2023-11-28T02:53:28.990559+0000","last_fullsized":"2023-11-28T02:53:28.990559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968733+0000","last_change":"2023-11-28T02:53:28.615665+0000","last_active":"2023-11-28T02:53:28.968733+0000","last_peered":"2023-11-28T02:53:28.968733+0000","last_clean":"2023-11-28T02:53:28.968733+0000","last_became_active":"2023-11-28T02:53:28.615238+0000","last_became_peered":"2023-11-28T02:53:28.615238+0000","last_unstale":"2023-11-28T02:53:28.968733+0000","last_undegraded":"2023-11-28T02:53:28.968733+0000","last_fullsized":"2023-11-28T02:53:28.968733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996054+0000","last_change":"2023-11-28T02:53:28.992310+0000","last_active":"2023-11-28T02:53:28.996054+0000","last_peered":"2023-11-28T02:53:28.996054+0000","last_clean":"2023-11-28T02:53:28.996054+0000","last_became_active":"2023-11-28T02:53:28.991941+0000","last_became_peered":"2023-11-28T02:53:28.991941+0000","last_unstale":"2023-11-28T02:53:28.996054+0000","last_undegraded":"2023-11-28T02:53:28.996054+0000","last_fullsized":"2023-11-28T02:53:28.996054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968891+0000","last_change":"2023-11-28T02:53:28.616239+0000","last_active":"2023-11-28T02:53:28.968891+0000","last_peered":"2023-11-28T02:53:28.968891+0000","last_clean":"2023-11-28T02:53:28.968891+0000","last_became_active":"2023-11-28T02:53:28.615922+0000","last_became_peered":"2023-11-28T02:53:28.615922+0000","last_unstale":"2023-11-28T02:53:28.968891+0000","last_undegraded":"2023-11-28T02:53:28.968891+0000","last_fullsized":"2023-11-28T02:53:28.968891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996992+0000","last_change":"2023-11-28T02:53:28.994692+0000","last_active":"2023-11-28T02:53:28.996992+0000","last_peered":"2023-11-28T02:53:28.996992+0000","last_clean":"2023-11-28T02:53:28.996992+0000","last_became_active":"2023-11-28T02:53:28.994376+0000","last_became_peered":"2023-11-28T02:53:28.994376+0000","last_unstale":"2023-11-28T02:53:28.996992+0000","last_undegraded":"2023-11-28T02:53:28.996992+0000","last_fullsized":"2023-11-28T02:53:28.996992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974544+0000","last_change":"2023-11-28T02:53:28.623298+0000","last_active":"2023-11-28T02:53:28.974544+0000","last_peered":"2023-11-28T02:53:28.974544+0000","last_clean":"2023-11-28T02:53:28.974544+0000","last_became_active":"2023-11-28T02:53:28.623015+0000","last_became_peered":"2023-11-28T02:53:28.623015+0000","last_unstale":"2023-11-28T02:53:28.974544+0000","last_undegraded":"2023-11-28T02:53:28.974544+0000","last_fullsized":"2023-11-28T02:53:28.974544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997432+0000","last_change":"2023-11-28T02:53:28.991999+0000","last_active":"2023-11-28T02:53:28.997432+0000","last_peered":"2023-11-28T02:53:28.997432+0000","last_clean":"2023-11-28T02:53:28.997432+0000","last_became_active":"2023-11-28T02:53:28.991669+0000","last_became_peered":"2023-11-28T02:53:28.991669+0000","last_unstale":"2023-11-28T02:53:28.997432+0000","last_undegraded":"2023-11-28T02:53:28.997432+0000","last_fullsized":"2023-11-28T02:53:28.997432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:29.002034+0000","last_change":"2023-11-28T02:53:28.614001+0000","last_active":"2023-11-28T02:53:29.002034+0000","last_peered":"2023-11-28T02:53:29.002034+0000","last_clean":"2023-11-28T02:53:29.002034+0000","last_became_active":"2023-11-28T02:53:28.613640+0000","last_became_peered":"2023-11-28T02:53:28.613640+0000","last_unstale":"2023-11-28T02:53:29.002034+0000","last_undegraded":"2023-11-28T02:53:29.002034+0000","last_fullsized":"2023-11-28T02:53:29.002034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997550+0000","last_change":"2023-11-28T02:53:28.995577+0000","last_active":"2023-11-28T02:53:28.997550+0000","last_peered":"2023-11-28T02:53:28.997550+0000","last_clean":"2023-11-28T02:53:28.997550+0000","last_became_active":"2023-11-28T02:53:28.995316+0000","last_became_peered":"2023-11-28T02:53:28.995316+0000","last_unstale":"2023-11-28T02:53:28.997550+0000","last_undegraded":"2023-11-28T02:53:28.997550+0000","last_fullsized":"2023-11-28T02:53:28.997550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992441+0000","last_change":"2023-11-28T02:53:28.612256+0000","last_active":"2023-11-28T02:53:28.992441+0000","last_peered":"2023-11-28T02:53:28.992441+0000","last_clean":"2023-11-28T02:53:28.992441+0000","last_became_active":"2023-11-28T02:53:28.611877+0000","last_became_peered":"2023-11-28T02:53:28.611877+0000","last_unstale":"2023-11-28T02:53:28.992441+0000","last_undegraded":"2023-11-28T02:53:28.992441+0000","last_fullsized":"2023-11-28T02:53:28.992441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993574+0000","last_change":"2023-11-28T02:53:28.989222+0000","last_active":"2023-11-28T02:53:28.993574+0000","last_peered":"2023-11-28T02:53:28.993574+0000","last_clean":"2023-11-28T02:53:28.993574+0000","last_became_active":"2023-11-28T02:53:28.987099+0000","last_became_peered":"2023-11-28T02:53:28.987099+0000","last_unstale":"2023-11-28T02:53:28.993574+0000","last_undegraded":"2023-11-28T02:53:28.993574+0000","last_fullsized":"2023-11-28T02:53:28.993574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974957+0000","last_change":"2023-11-28T02:53:28.604483+0000","last_active":"2023-11-28T02:53:28.974957+0000","last_peered":"2023-11-28T02:53:28.974957+0000","last_clean":"2023-11-28T02:53:28.974957+0000","last_became_active":"2023-11-28T02:53:28.604198+0000","last_became_peered":"2023-11-28T02:53:28.604198+0000","last_unstale":"2023-11-28T02:53:28.974957+0000","last_undegraded":"2023-11-28T02:53:28.974957+0000","last_fullsized":"2023-11-28T02:53:28.974957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986280+0000","last_change":"2023-11-28T02:53:28.978915+0000","last_active":"2023-11-28T02:53:28.986280+0000","last_peered":"2023-11-28T02:53:28.986280+0000","last_clean":"2023-11-28T02:53:28.986280+0000","last_became_active":"2023-11-28T02:53:28.978436+0000","last_became_peered":"2023-11-28T02:53:28.978436+0000","last_unstale":"2023-11-28T02:53:28.986280+0000","last_undegraded":"2023-11-28T02:53:28.986280+0000","last_fullsized":"2023-11-28T02:53:28.986280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977737+0000","last_change":"2023-11-28T02:53:28.614088+0000","last_active":"2023-11-28T02:53:28.977737+0000","last_peered":"2023-11-28T02:53:28.977737+0000","last_clean":"2023-11-28T02:53:28.977737+0000","last_became_active":"2023-11-28T02:53:28.613799+0000","last_became_peered":"2023-11-28T02:53:28.613799+0000","last_unstale":"2023-11-28T02:53:28.977737+0000","last_undegraded":"2023-11-28T02:53:28.977737+0000","last_fullsized":"2023-11-28T02:53:28.977737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993749+0000","last_change":"2023-11-28T02:53:28.990079+0000","last_active":"2023-11-28T02:53:28.993749+0000","last_peered":"2023-11-28T02:53:28.993749+0000","last_clean":"2023-11-28T02:53:28.993749+0000","last_became_active":"2023-11-28T02:53:28.989796+0000","last_became_peered":"2023-11-28T02:53:28.989796+0000","last_unstale":"2023-11-28T02:53:28.993749+0000","last_undegraded":"2023-11-28T02:53:28.993749+0000","last_fullsized":"2023-11-28T02:53:28.993749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990305+0000","last_change":"2023-11-28T02:53:28.619663+0000","last_active":"2023-11-28T02:53:28.990305+0000","last_peered":"2023-11-28T02:53:28.990305+0000","last_clean":"2023-11-28T02:53:28.990305+0000","last_became_active":"2023-11-28T02:53:28.619099+0000","last_became_peered":"2023-11-28T02:53:28.619099+0000","last_unstale":"2023-11-28T02:53:28.990305+0000","last_undegraded":"2023-11-28T02:53:28.990305+0000","last_fullsized":"2023-11-28T02:53:28.990305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983843+0000","last_change":"2023-11-28T02:53:28.981838+0000","last_active":"2023-11-28T02:53:28.983843+0000","last_peered":"2023-11-28T02:53:28.983843+0000","last_clean":"2023-11-28T02:53:28.983843+0000","last_became_active":"2023-11-28T02:53:28.981157+0000","last_became_peered":"2023-11-28T02:53:28.981157+0000","last_unstale":"2023-11-28T02:53:28.983843+0000","last_undegraded":"2023-11-28T02:53:28.983843+0000","last_fullsized":"2023-11-28T02:53:28.983843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972511+0000","last_change":"2023-11-28T02:53:28.625948+0000","last_active":"2023-11-28T02:53:28.972511+0000","last_peered":"2023-11-28T02:53:28.972511+0000","last_clean":"2023-11-28T02:53:28.972511+0000","last_became_active":"2023-11-28T02:53:28.625727+0000","last_became_peered":"2023-11-28T02:53:28.625727+0000","last_unstale":"2023-11-28T02:53:28.972511+0000","last_undegraded":"2023-11-28T02:53:28.972511+0000","last_fullsized":"2023-11-28T02:53:28.972511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994166+0000","last_change":"2023-11-28T02:53:28.985735+0000","last_active":"2023-11-28T02:53:28.994166+0000","last_peered":"2023-11-28T02:53:28.994166+0000","last_clean":"2023-11-28T02:53:28.994166+0000","last_became_active":"2023-11-28T02:53:28.985480+0000","last_became_peered":"2023-11-28T02:53:28.985480+0000","last_unstale":"2023-11-28T02:53:28.994166+0000","last_undegraded":"2023-11-28T02:53:28.994166+0000","last_fullsized":"2023-11-28T02:53:28.994166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974322+0000","last_change":"2023-11-28T02:53:28.613536+0000","last_active":"2023-11-28T02:53:28.974322+0000","last_peered":"2023-11-28T02:53:28.974322+0000","last_clean":"2023-11-28T02:53:28.974322+0000","last_became_active":"2023-11-28T02:53:28.613217+0000","last_became_peered":"2023-11-28T02:53:28.613217+0000","last_unstale":"2023-11-28T02:53:28.974322+0000","last_undegraded":"2023-11-28T02:53:28.974322+0000","last_fullsized":"2023-11-28T02:53:28.974322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989513+0000","last_change":"2023-11-28T02:53:28.983979+0000","last_active":"2023-11-28T02:53:28.989513+0000","last_peered":"2023-11-28T02:53:28.989513+0000","last_clean":"2023-11-28T02:53:28.989513+0000","last_became_active":"2023-11-28T02:53:28.983627+0000","last_became_peered":"2023-11-28T02:53:28.983627+0000","last_unstale":"2023-11-28T02:53:28.989513+0000","last_undegraded":"2023-11-28T02:53:28.989513+0000","last_fullsized":"2023-11-28T02:53:28.989513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991928+0000","last_change":"2023-11-28T02:53:28.613321+0000","last_active":"2023-11-28T02:53:28.991928+0000","last_peered":"2023-11-28T02:53:28.991928+0000","last_clean":"2023-11-28T02:53:28.991928+0000","last_became_active":"2023-11-28T02:53:28.613039+0000","last_became_peered":"2023-11-28T02:53:28.613039+0000","last_unstale":"2023-11-28T02:53:28.991928+0000","last_undegraded":"2023-11-28T02:53:28.991928+0000","last_fullsized":"2023-11-28T02:53:28.991928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996877+0000","last_change":"2023-11-28T02:53:28.994024+0000","last_active":"2023-11-28T02:53:28.996877+0000","last_peered":"2023-11-28T02:53:28.996877+0000","last_clean":"2023-11-28T02:53:28.996877+0000","last_became_active":"2023-11-28T02:53:28.993749+0000","last_became_peered":"2023-11-28T02:53:28.993749+0000","last_unstale":"2023-11-28T02:53:28.996877+0000","last_undegraded":"2023-11-28T02:53:28.996877+0000","last_fullsized":"2023-11-28T02:53:28.996877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984340+0000","last_change":"2023-11-28T02:53:28.615797+0000","last_active":"2023-11-28T02:53:28.984340+0000","last_peered":"2023-11-28T02:53:28.984340+0000","last_clean":"2023-11-28T02:53:28.984340+0000","last_became_active":"2023-11-28T02:53:28.615499+0000","last_became_peered":"2023-11-28T02:53:28.615499+0000","last_unstale":"2023-11-28T02:53:28.984340+0000","last_undegraded":"2023-11-28T02:53:28.984340+0000","last_fullsized":"2023-11-28T02:53:28.984340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997447+0000","last_change":"2023-11-28T02:53:28.994091+0000","last_active":"2023-11-28T02:53:28.997447+0000","last_peered":"2023-11-28T02:53:28.997447+0000","last_clean":"2023-11-28T02:53:28.997447+0000","last_became_active":"2023-11-28T02:53:28.993809+0000","last_became_peered":"2023-11-28T02:53:28.993809+0000","last_unstale":"2023-11-28T02:53:28.997447+0000","last_undegraded":"2023-11-28T02:53:28.997447+0000","last_fullsized":"2023-11-28T02:53:28.997447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991158+0000","last_change":"2023-11-28T02:53:28.622046+0000","last_active":"2023-11-28T02:53:28.991158+0000","last_peered":"2023-11-28T02:53:28.991158+0000","last_clean":"2023-11-28T02:53:28.991158+0000","last_became_active":"2023-11-28T02:53:28.621813+0000","last_became_peered":"2023-11-28T02:53:28.621813+0000","last_unstale":"2023-11-28T02:53:28.991158+0000","last_undegraded":"2023-11-28T02:53:28.991158+0000","last_fullsized":"2023-11-28T02:53:28.991158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995692+0000","last_change":"2023-11-28T02:53:28.990175+0000","last_active":"2023-11-28T02:53:28.995692+0000","last_peered":"2023-11-28T02:53:28.995692+0000","last_clean":"2023-11-28T02:53:28.995692+0000","last_became_active":"2023-11-28T02:53:28.988430+0000","last_became_peered":"2023-11-28T02:53:28.988430+0000","last_unstale":"2023-11-28T02:53:28.995692+0000","last_undegraded":"2023-11-28T02:53:28.995692+0000","last_fullsized":"2023-11-28T02:53:28.995692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969384+0000","last_change":"2023-11-28T02:53:28.624148+0000","last_active":"2023-11-28T02:53:28.969384+0000","last_peered":"2023-11-28T02:53:28.969384+0000","last_clean":"2023-11-28T02:53:28.969384+0000","last_became_active":"2023-11-28T02:53:28.622802+0000","last_became_peered":"2023-11-28T02:53:28.622802+0000","last_unstale":"2023-11-28T02:53:28.969384+0000","last_undegraded":"2023-11-28T02:53:28.969384+0000","last_fullsized":"2023-11-28T02:53:28.969384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996093+0000","last_change":"2023-11-28T02:53:28.992937+0000","last_active":"2023-11-28T02:53:28.996093+0000","last_peered":"2023-11-28T02:53:28.996093+0000","last_clean":"2023-11-28T02:53:28.996093+0000","last_became_active":"2023-11-28T02:53:28.992636+0000","last_became_peered":"2023-11-28T02:53:28.992636+0000","last_unstale":"2023-11-28T02:53:28.996093+0000","last_undegraded":"2023-11-28T02:53:28.996093+0000","last_fullsized":"2023-11-28T02:53:28.996093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977354+0000","last_change":"2023-11-28T02:53:28.619570+0000","last_active":"2023-11-28T02:53:28.977354+0000","last_peered":"2023-11-28T02:53:28.977354+0000","last_clean":"2023-11-28T02:53:28.977354+0000","last_became_active":"2023-11-28T02:53:28.619323+0000","last_became_peered":"2023-11-28T02:53:28.619323+0000","last_unstale":"2023-11-28T02:53:28.977354+0000","last_undegraded":"2023-11-28T02:53:28.977354+0000","last_fullsized":"2023-11-28T02:53:28.977354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987789+0000","last_change":"2023-11-28T02:53:28.984950+0000","last_active":"2023-11-28T02:53:28.987789+0000","last_peered":"2023-11-28T02:53:28.987789+0000","last_clean":"2023-11-28T02:53:28.987789+0000","last_became_active":"2023-11-28T02:53:28.984627+0000","last_became_peered":"2023-11-28T02:53:28.984627+0000","last_unstale":"2023-11-28T02:53:28.987789+0000","last_undegraded":"2023-11-28T02:53:28.987789+0000","last_fullsized":"2023-11-28T02:53:28.987789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991026+0000","last_change":"2023-11-28T02:53:28.612552+0000","last_active":"2023-11-28T02:53:28.991026+0000","last_peered":"2023-11-28T02:53:28.991026+0000","last_clean":"2023-11-28T02:53:28.991026+0000","last_became_active":"2023-11-28T02:53:28.612193+0000","last_became_peered":"2023-11-28T02:53:28.612193+0000","last_unstale":"2023-11-28T02:53:28.991026+0000","last_undegraded":"2023-11-28T02:53:28.991026+0000","last_fullsized":"2023-11-28T02:53:28.991026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996305+0000","last_change":"2023-11-28T02:53:28.992515+0000","last_active":"2023-11-28T02:53:28.996305+0000","last_peered":"2023-11-28T02:53:28.996305+0000","last_clean":"2023-11-28T02:53:28.996305+0000","last_became_active":"2023-11-28T02:53:28.989851+0000","last_became_peered":"2023-11-28T02:53:28.989851+0000","last_unstale":"2023-11-28T02:53:28.996305+0000","last_undegraded":"2023-11-28T02:53:28.996305+0000","last_fullsized":"2023-11-28T02:53:28.996305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969075+0000","last_change":"2023-11-28T02:53:28.618220+0000","last_active":"2023-11-28T02:53:28.969075+0000","last_peered":"2023-11-28T02:53:28.969075+0000","last_clean":"2023-11-28T02:53:28.969075+0000","last_became_active":"2023-11-28T02:53:28.617971+0000","last_became_peered":"2023-11-28T02:53:28.617971+0000","last_unstale":"2023-11-28T02:53:28.969075+0000","last_undegraded":"2023-11-28T02:53:28.969075+0000","last_fullsized":"2023-11-28T02:53:28.969075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993916+0000","last_change":"2023-11-28T02:53:28.989630+0000","last_active":"2023-11-28T02:53:28.993916+0000","last_peered":"2023-11-28T02:53:28.993916+0000","last_clean":"2023-11-28T02:53:28.993916+0000","last_became_active":"2023-11-28T02:53:28.989322+0000","last_became_peered":"2023-11-28T02:53:28.989322+0000","last_unstale":"2023-11-28T02:53:28.993916+0000","last_undegraded":"2023-11-28T02:53:28.993916+0000","last_fullsized":"2023-11-28T02:53:28.993916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974420+0000","last_change":"2023-11-28T02:53:28.615301+0000","last_active":"2023-11-28T02:53:28.974420+0000","last_peered":"2023-11-28T02:53:28.974420+0000","last_clean":"2023-11-28T02:53:28.974420+0000","last_became_active":"2023-11-28T02:53:28.614882+0000","last_became_peered":"2023-11-28T02:53:28.614882+0000","last_unstale":"2023-11-28T02:53:28.974420+0000","last_undegraded":"2023-11-28T02:53:28.974420+0000","last_fullsized":"2023-11-28T02:53:28.974420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994461+0000","last_change":"2023-11-28T02:53:28.989438+0000","last_active":"2023-11-28T02:53:28.994461+0000","last_peered":"2023-11-28T02:53:28.994461+0000","last_clean":"2023-11-28T02:53:28.994461+0000","last_became_active":"2023-11-28T02:53:28.988433+0000","last_became_peered":"2023-11-28T02:53:28.988433+0000","last_unstale":"2023-11-28T02:53:28.994461+0000","last_undegraded":"2023-11-28T02:53:28.994461+0000","last_fullsized":"2023-11-28T02:53:28.994461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969581+0000","last_change":"2023-11-28T02:53:28.624478+0000","last_active":"2023-11-28T02:53:28.969581+0000","last_peered":"2023-11-28T02:53:28.969581+0000","last_clean":"2023-11-28T02:53:28.969581+0000","last_became_active":"2023-11-28T02:53:28.624184+0000","last_became_peered":"2023-11-28T02:53:28.624184+0000","last_unstale":"2023-11-28T02:53:28.969581+0000","last_undegraded":"2023-11-28T02:53:28.969581+0000","last_fullsized":"2023-11-28T02:53:28.969581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983881+0000","last_change":"2023-11-28T02:53:28.980304+0000","last_active":"2023-11-28T02:53:28.983881+0000","last_peered":"2023-11-28T02:53:28.983881+0000","last_clean":"2023-11-28T02:53:28.983881+0000","last_became_active":"2023-11-28T02:53:28.980010+0000","last_became_peered":"2023-11-28T02:53:28.980010+0000","last_unstale":"2023-11-28T02:53:28.983881+0000","last_undegraded":"2023-11-28T02:53:28.983881+0000","last_fullsized":"2023-11-28T02:53:28.983881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.982139+0000","last_change":"2023-11-28T02:53:28.614011+0000","last_active":"2023-11-28T02:53:28.982139+0000","last_peered":"2023-11-28T02:53:28.982139+0000","last_clean":"2023-11-28T02:53:28.982139+0000","last_became_active":"2023-11-28T02:53:28.613693+0000","last_became_peered":"2023-11-28T02:53:28.613693+0000","last_unstale":"2023-11-28T02:53:28.982139+0000","last_undegraded":"2023-11-28T02:53:28.982139+0000","last_fullsized":"2023-11-28T02:53:28.982139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994527+0000","last_change":"2023-11-28T02:53:28.989670+0000","last_active":"2023-11-28T02:53:28.994527+0000","last_peered":"2023-11-28T02:53:28.994527+0000","last_clean":"2023-11-28T02:53:28.994527+0000","last_became_active":"2023-11-28T02:53:28.989404+0000","last_became_peered":"2023-11-28T02:53:28.989404+0000","last_unstale":"2023-11-28T02:53:28.994527+0000","last_undegraded":"2023-11-28T02:53:28.994527+0000","last_fullsized":"2023-11-28T02:53:28.994527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968290+0000","last_change":"2023-11-28T02:53:28.622136+0000","last_active":"2023-11-28T02:53:28.968290+0000","last_peered":"2023-11-28T02:53:28.968290+0000","last_clean":"2023-11-28T02:53:28.968290+0000","last_became_active":"2023-11-28T02:53:28.615426+0000","last_became_peered":"2023-11-28T02:53:28.615426+0000","last_unstale":"2023-11-28T02:53:28.968290+0000","last_undegraded":"2023-11-28T02:53:28.968290+0000","last_fullsized":"2023-11-28T02:53:28.968290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996526+0000","last_change":"2023-11-28T02:53:28.993084+0000","last_active":"2023-11-28T02:53:28.996526+0000","last_peered":"2023-11-28T02:53:28.996526+0000","last_clean":"2023-11-28T02:53:28.996526+0000","last_became_active":"2023-11-28T02:53:28.989889+0000","last_became_peered":"2023-11-28T02:53:28.989889+0000","last_unstale":"2023-11-28T02:53:28.996526+0000","last_undegraded":"2023-11-28T02:53:28.996526+0000","last_fullsized":"2023-11-28T02:53:28.996526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973080+0000","last_change":"2023-11-28T02:53:28.623918+0000","last_active":"2023-11-28T02:53:28.973080+0000","last_peered":"2023-11-28T02:53:28.973080+0000","last_clean":"2023-11-28T02:53:28.973080+0000","last_became_active":"2023-11-28T02:53:28.623641+0000","last_became_peered":"2023-11-28T02:53:28.623641+0000","last_unstale":"2023-11-28T02:53:28.973080+0000","last_undegraded":"2023-11-28T02:53:28.973080+0000","last_fullsized":"2023-11-28T02:53:28.973080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992377+0000","last_change":"2023-11-28T02:53:28.987144+0000","last_active":"2023-11-28T02:53:28.992377+0000","last_peered":"2023-11-28T02:53:28.992377+0000","last_clean":"2023-11-28T02:53:28.992377+0000","last_became_active":"2023-11-28T02:53:28.985075+0000","last_became_peered":"2023-11-28T02:53:28.985075+0000","last_unstale":"2023-11-28T02:53:28.992377+0000","last_undegraded":"2023-11-28T02:53:28.992377+0000","last_fullsized":"2023-11-28T02:53:28.992377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975794+0000","last_change":"2023-11-28T02:53:28.606475+0000","last_active":"2023-11-28T02:53:28.975794+0000","last_peered":"2023-11-28T02:53:28.975794+0000","last_clean":"2023-11-28T02:53:28.975794+0000","last_became_active":"2023-11-28T02:53:28.606177+0000","last_became_peered":"2023-11-28T02:53:28.606177+0000","last_unstale":"2023-11-28T02:53:28.975794+0000","last_undegraded":"2023-11-28T02:53:28.975794+0000","last_fullsized":"2023-11-28T02:53:28.975794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.978259+0000","last_change":"2023-11-28T02:53:28.977105+0000","last_active":"2023-11-28T02:53:28.978259+0000","last_peered":"2023-11-28T02:53:28.978259+0000","last_clean":"2023-11-28T02:53:28.978259+0000","last_became_active":"2023-11-28T02:53:28.976061+0000","last_became_peered":"2023-11-28T02:53:28.976061+0000","last_unstale":"2023-11-28T02:53:28.978259+0000","last_undegraded":"2023-11-28T02:53:28.978259+0000","last_fullsized":"2023-11-28T02:53:28.978259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972986+0000","last_change":"2023-11-28T02:53:28.627570+0000","last_active":"2023-11-28T02:53:28.972986+0000","last_peered":"2023-11-28T02:53:28.972986+0000","last_clean":"2023-11-28T02:53:28.972986+0000","last_became_active":"2023-11-28T02:53:28.627207+0000","last_became_peered":"2023-11-28T02:53:28.627207+0000","last_unstale":"2023-11-28T02:53:28.972986+0000","last_undegraded":"2023-11-28T02:53:28.972986+0000","last_fullsized":"2023-11-28T02:53:28.972986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995547+0000","last_change":"2023-11-28T02:53:28.988822+0000","last_active":"2023-11-28T02:53:28.995547+0000","last_peered":"2023-11-28T02:53:28.995547+0000","last_clean":"2023-11-28T02:53:28.995547+0000","last_became_active":"2023-11-28T02:53:28.988328+0000","last_became_peered":"2023-11-28T02:53:28.988328+0000","last_unstale":"2023-11-28T02:53:28.995547+0000","last_undegraded":"2023-11-28T02:53:28.995547+0000","last_fullsized":"2023-11-28T02:53:28.995547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994154+0000","last_change":"2023-11-28T02:53:28.610805+0000","last_active":"2023-11-28T02:53:28.994154+0000","last_peered":"2023-11-28T02:53:28.994154+0000","last_clean":"2023-11-28T02:53:28.994154+0000","last_became_active":"2023-11-28T02:53:28.610370+0000","last_became_peered":"2023-11-28T02:53:28.610370+0000","last_unstale":"2023-11-28T02:53:28.994154+0000","last_undegraded":"2023-11-28T02:53:28.994154+0000","last_fullsized":"2023-11-28T02:53:28.994154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997253+0000","last_change":"2023-11-28T02:53:28.994943+0000","last_active":"2023-11-28T02:53:28.997253+0000","last_peered":"2023-11-28T02:53:28.997253+0000","last_clean":"2023-11-28T02:53:28.997253+0000","last_became_active":"2023-11-28T02:53:28.994619+0000","last_became_peered":"2023-11-28T02:53:28.994619+0000","last_unstale":"2023-11-28T02:53:28.997253+0000","last_undegraded":"2023-11-28T02:53:28.997253+0000","last_fullsized":"2023-11-28T02:53:28.997253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993626+0000","last_change":"2023-11-28T02:53:28.613771+0000","last_active":"2023-11-28T02:53:28.993626+0000","last_peered":"2023-11-28T02:53:28.993626+0000","last_clean":"2023-11-28T02:53:28.993626+0000","last_became_active":"2023-11-28T02:53:28.613520+0000","last_became_peered":"2023-11-28T02:53:28.613520+0000","last_unstale":"2023-11-28T02:53:28.993626+0000","last_undegraded":"2023-11-28T02:53:28.993626+0000","last_fullsized":"2023-11-28T02:53:28.993626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.988754+0000","last_change":"2023-11-28T02:53:28.980028+0000","last_active":"2023-11-28T02:53:28.988754+0000","last_peered":"2023-11-28T02:53:28.988754+0000","last_clean":"2023-11-28T02:53:28.988754+0000","last_became_active":"2023-11-28T02:53:28.979301+0000","last_became_peered":"2023-11-28T02:53:28.979301+0000","last_unstale":"2023-11-28T02:53:28.988754+0000","last_undegraded":"2023-11-28T02:53:28.988754+0000","last_fullsized":"2023-11-28T02:53:28.988754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987042+0000","last_change":"2023-11-28T02:53:28.624374+0000","last_active":"2023-11-28T02:53:28.987042+0000","last_peered":"2023-11-28T02:53:28.987042+0000","last_clean":"2023-11-28T02:53:28.987042+0000","last_became_active":"2023-11-28T02:53:28.624177+0000","last_became_peered":"2023-11-28T02:53:28.624177+0000","last_unstale":"2023-11-28T02:53:28.987042+0000","last_undegraded":"2023-11-28T02:53:28.987042+0000","last_fullsized":"2023-11-28T02:53:28.987042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989750+0000","last_change":"2023-11-28T02:53:28.986339+0000","last_active":"2023-11-28T02:53:28.989750+0000","last_peered":"2023-11-28T02:53:28.989750+0000","last_clean":"2023-11-28T02:53:28.989750+0000","last_became_active":"2023-11-28T02:53:28.986068+0000","last_became_peered":"2023-11-28T02:53:28.986068+0000","last_unstale":"2023-11-28T02:53:28.989750+0000","last_undegraded":"2023-11-28T02:53:28.989750+0000","last_fullsized":"2023-11-28T02:53:28.989750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968753+0000","last_change":"2023-11-28T02:53:28.609904+0000","last_active":"2023-11-28T02:53:28.968753+0000","last_peered":"2023-11-28T02:53:28.968753+0000","last_clean":"2023-11-28T02:53:28.968753+0000","last_became_active":"2023-11-28T02:53:28.609574+0000","last_became_peered":"2023-11-28T02:53:28.609574+0000","last_unstale":"2023-11-28T02:53:28.968753+0000","last_undegraded":"2023-11-28T02:53:28.968753+0000","last_fullsized":"2023-11-28T02:53:28.968753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984706+0000","last_change":"2023-11-28T02:53:28.979178+0000","last_active":"2023-11-28T02:53:28.984706+0000","last_peered":"2023-11-28T02:53:28.984706+0000","last_clean":"2023-11-28T02:53:28.984706+0000","last_became_active":"2023-11-28T02:53:28.978772+0000","last_became_peered":"2023-11-28T02:53:28.978772+0000","last_unstale":"2023-11-28T02:53:28.984706+0000","last_undegraded":"2023-11-28T02:53:28.984706+0000","last_fullsized":"2023-11-28T02:53:28.984706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977227+0000","last_change":"2023-11-28T02:53:28.613015+0000","last_active":"2023-11-28T02:53:28.977227+0000","last_peered":"2023-11-28T02:53:28.977227+0000","last_clean":"2023-11-28T02:53:28.977227+0000","last_became_active":"2023-11-28T02:53:28.612713+0000","last_became_peered":"2023-11-28T02:53:28.612713+0000","last_unstale":"2023-11-28T02:53:28.977227+0000","last_undegraded":"2023-11-28T02:53:28.977227+0000","last_fullsized":"2023-11-28T02:53:28.977227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993535+0000","last_change":"2023-11-28T02:53:28.987025+0000","last_active":"2023-11-28T02:53:28.993535+0000","last_peered":"2023-11-28T02:53:28.993535+0000","last_clean":"2023-11-28T02:53:28.993535+0000","last_became_active":"2023-11-28T02:53:28.985857+0000","last_became_peered":"2023-11-28T02:53:28.985857+0000","last_unstale":"2023-11-28T02:53:28.993535+0000","last_undegraded":"2023-11-28T02:53:28.993535+0000","last_fullsized":"2023-11-28T02:53:28.993535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972966+0000","last_change":"2023-11-28T02:53:28.625797+0000","last_active":"2023-11-28T02:53:28.972966+0000","last_peered":"2023-11-28T02:53:28.972966+0000","last_clean":"2023-11-28T02:53:28.972966+0000","last_became_active":"2023-11-28T02:53:28.625489+0000","last_became_peered":"2023-11-28T02:53:28.625489+0000","last_unstale":"2023-11-28T02:53:28.972966+0000","last_undegraded":"2023-11-28T02:53:28.972966+0000","last_fullsized":"2023-11-28T02:53:28.972966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990553+0000","last_change":"2023-11-28T02:53:28.988300+0000","last_active":"2023-11-28T02:53:28.990553+0000","last_peered":"2023-11-28T02:53:28.990553+0000","last_clean":"2023-11-28T02:53:28.990553+0000","last_became_active":"2023-11-28T02:53:28.988014+0000","last_became_peered":"2023-11-28T02:53:28.988014+0000","last_unstale":"2023-11-28T02:53:28.990553+0000","last_undegraded":"2023-11-28T02:53:28.990553+0000","last_fullsized":"2023-11-28T02:53:28.990553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968813+0000","last_change":"2023-11-28T02:53:28.618562+0000","last_active":"2023-11-28T02:53:28.968813+0000","last_peered":"2023-11-28T02:53:28.968813+0000","last_clean":"2023-11-28T02:53:28.968813+0000","last_became_active":"2023-11-28T02:53:28.618218+0000","last_became_peered":"2023-11-28T02:53:28.618218+0000","last_unstale":"2023-11-28T02:53:28.968813+0000","last_undegraded":"2023-11-28T02:53:28.968813+0000","last_fullsized":"2023-11-28T02:53:28.968813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994696+0000","last_change":"2023-11-28T02:53:28.992384+0000","last_active":"2023-11-28T02:53:28.994696+0000","last_peered":"2023-11-28T02:53:28.994696+0000","last_clean":"2023-11-28T02:53:28.994696+0000","last_became_active":"2023-11-28T02:53:28.992058+0000","last_became_peered":"2023-11-28T02:53:28.992058+0000","last_unstale":"2023-11-28T02:53:28.994696+0000","last_undegraded":"2023-11-28T02:53:28.994696+0000","last_fullsized":"2023-11-28T02:53:28.994696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968089+0000","last_change":"2023-11-28T02:53:28.614987+0000","last_active":"2023-11-28T02:53:28.968089+0000","last_peered":"2023-11-28T02:53:28.968089+0000","last_clean":"2023-11-28T02:53:28.968089+0000","last_became_active":"2023-11-28T02:53:28.614665+0000","last_became_peered":"2023-11-28T02:53:28.614665+0000","last_unstale":"2023-11-28T02:53:28.968089+0000","last_undegraded":"2023-11-28T02:53:28.968089+0000","last_fullsized":"2023-11-28T02:53:28.968089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994348+0000","last_change":"2023-11-28T02:53:28.987703+0000","last_active":"2023-11-28T02:53:28.994348+0000","last_peered":"2023-11-28T02:53:28.994348+0000","last_clean":"2023-11-28T02:53:28.994348+0000","last_became_active":"2023-11-28T02:53:28.987426+0000","last_became_peered":"2023-11-28T02:53:28.987426+0000","last_unstale":"2023-11-28T02:53:28.994348+0000","last_undegraded":"2023-11-28T02:53:28.994348+0000","last_fullsized":"2023-11-28T02:53:28.994348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974546+0000","last_change":"2023-11-28T02:53:28.611862+0000","last_active":"2023-11-28T02:53:28.974546+0000","last_peered":"2023-11-28T02:53:28.974546+0000","last_clean":"2023-11-28T02:53:28.974546+0000","last_became_active":"2023-11-28T02:53:28.611598+0000","last_became_peered":"2023-11-28T02:53:28.611598+0000","last_unstale":"2023-11-28T02:53:28.974546+0000","last_undegraded":"2023-11-28T02:53:28.974546+0000","last_fullsized":"2023-11-28T02:53:28.974546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989285+0000","last_change":"2023-11-28T02:53:28.987367+0000","last_active":"2023-11-28T02:53:28.989285+0000","last_peered":"2023-11-28T02:53:28.989285+0000","last_clean":"2023-11-28T02:53:28.989285+0000","last_became_active":"2023-11-28T02:53:28.986325+0000","last_became_peered":"2023-11-28T02:53:28.986325+0000","last_unstale":"2023-11-28T02:53:28.989285+0000","last_undegraded":"2023-11-28T02:53:28.989285+0000","last_fullsized":"2023-11-28T02:53:28.989285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.979734+0000","last_change":"2023-11-28T02:53:28.612981+0000","last_active":"2023-11-28T02:53:28.979734+0000","last_peered":"2023-11-28T02:53:28.979734+0000","last_clean":"2023-11-28T02:53:28.979734+0000","last_became_active":"2023-11-28T02:53:28.612549+0000","last_became_peered":"2023-11-28T02:53:28.612549+0000","last_unstale":"2023-11-28T02:53:28.979734+0000","last_undegraded":"2023-11-28T02:53:28.979734+0000","last_fullsized":"2023-11-28T02:53:28.979734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.979772+0000","last_change":"2023-11-28T02:53:28.978113+0000","last_active":"2023-11-28T02:53:28.979772+0000","last_peered":"2023-11-28T02:53:28.979772+0000","last_clean":"2023-11-28T02:53:28.979772+0000","last_became_active":"2023-11-28T02:53:28.977855+0000","last_became_peered":"2023-11-28T02:53:28.977855+0000","last_unstale":"2023-11-28T02:53:28.979772+0000","last_undegraded":"2023-11-28T02:53:28.979772+0000","last_fullsized":"2023-11-28T02:53:28.979772+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994921+0000","last_change":"2023-11-28T02:53:28.614226+0000","last_active":"2023-11-28T02:53:28.994921+0000","last_peered":"2023-11-28T02:53:28.994921+0000","last_clean":"2023-11-28T02:53:28.994921+0000","last_became_active":"2023-11-28T02:53:28.613474+0000","last_became_peered":"2023-11-28T02:53:28.613474+0000","last_unstale":"2023-11-28T02:53:28.994921+0000","last_undegraded":"2023-11-28T02:53:28.994921+0000","last_fullsized":"2023-11-28T02:53:28.994921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995239+0000","last_change":"2023-11-28T02:53:28.992998+0000","last_active":"2023-11-28T02:53:28.995239+0000","last_peered":"2023-11-28T02:53:28.995239+0000","last_clean":"2023-11-28T02:53:28.995239+0000","last_became_active":"2023-11-28T02:53:28.992705+0000","last_became_peered":"2023-11-28T02:53:28.992705+0000","last_unstale":"2023-11-28T02:53:28.995239+0000","last_undegraded":"2023-11-28T02:53:28.995239+0000","last_fullsized":"2023-11-28T02:53:28.995239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992195+0000","last_change":"2023-11-28T02:53:28.617731+0000","last_active":"2023-11-28T02:53:28.992195+0000","last_peered":"2023-11-28T02:53:28.992195+0000","last_clean":"2023-11-28T02:53:28.992195+0000","last_became_active":"2023-11-28T02:53:28.617429+0000","last_became_peered":"2023-11-28T02:53:28.617429+0000","last_unstale":"2023-11-28T02:53:28.992195+0000","last_undegraded":"2023-11-28T02:53:28.992195+0000","last_fullsized":"2023-11-28T02:53:28.992195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997723+0000","last_change":"2023-11-28T02:53:28.995450+0000","last_active":"2023-11-28T02:53:28.997723+0000","last_peered":"2023-11-28T02:53:28.997723+0000","last_clean":"2023-11-28T02:53:28.997723+0000","last_became_active":"2023-11-28T02:53:28.995190+0000","last_became_peered":"2023-11-28T02:53:28.995190+0000","last_unstale":"2023-11-28T02:53:28.997723+0000","last_undegraded":"2023-11-28T02:53:28.997723+0000","last_fullsized":"2023-11-28T02:53:28.997723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985316+0000","last_change":"2023-11-28T02:53:28.610107+0000","last_active":"2023-11-28T02:53:28.985316+0000","last_peered":"2023-11-28T02:53:28.985316+0000","last_clean":"2023-11-28T02:53:28.985316+0000","last_became_active":"2023-11-28T02:53:28.609440+0000","last_became_peered":"2023-11-28T02:53:28.609440+0000","last_unstale":"2023-11-28T02:53:28.985316+0000","last_undegraded":"2023-11-28T02:53:28.985316+0000","last_fullsized":"2023-11-28T02:53:28.985316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985337+0000","last_change":"2023-11-28T02:53:28.980510+0000","last_active":"2023-11-28T02:53:28.985337+0000","last_peered":"2023-11-28T02:53:28.985337+0000","last_clean":"2023-11-28T02:53:28.985337+0000","last_became_active":"2023-11-28T02:53:28.980225+0000","last_became_peered":"2023-11-28T02:53:28.980225+0000","last_unstale":"2023-11-28T02:53:28.985337+0000","last_undegraded":"2023-11-28T02:53:28.985337+0000","last_fullsized":"2023-11-28T02:53:28.985337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968728+0000","last_change":"2023-11-28T02:53:28.616013+0000","last_active":"2023-11-28T02:53:28.968728+0000","last_peered":"2023-11-28T02:53:28.968728+0000","last_clean":"2023-11-28T02:53:28.968728+0000","last_became_active":"2023-11-28T02:53:28.615730+0000","last_became_peered":"2023-11-28T02:53:28.615730+0000","last_unstale":"2023-11-28T02:53:28.968728+0000","last_undegraded":"2023-11-28T02:53:28.968728+0000","last_fullsized":"2023-11-28T02:53:28.968728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989391+0000","last_change":"2023-11-28T02:53:28.983758+0000","last_active":"2023-11-28T02:53:28.989391+0000","last_peered":"2023-11-28T02:53:28.989391+0000","last_clean":"2023-11-28T02:53:28.989391+0000","last_became_active":"2023-11-28T02:53:28.983409+0000","last_became_peered":"2023-11-28T02:53:28.983409+0000","last_unstale":"2023-11-28T02:53:28.989391+0000","last_undegraded":"2023-11-28T02:53:28.989391+0000","last_fullsized":"2023-11-28T02:53:28.989391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975684+0000","last_change":"2023-11-28T02:53:28.613797+0000","last_active":"2023-11-28T02:53:28.975684+0000","last_peered":"2023-11-28T02:53:28.975684+0000","last_clean":"2023-11-28T02:53:28.975684+0000","last_became_active":"2023-11-28T02:53:28.613504+0000","last_became_peered":"2023-11-28T02:53:28.613504+0000","last_unstale":"2023-11-28T02:53:28.975684+0000","last_undegraded":"2023-11-28T02:53:28.975684+0000","last_fullsized":"2023-11-28T02:53:28.975684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986230+0000","last_change":"2023-11-28T02:53:28.980548+0000","last_active":"2023-11-28T02:53:28.986230+0000","last_peered":"2023-11-28T02:53:28.986230+0000","last_clean":"2023-11-28T02:53:28.986230+0000","last_became_active":"2023-11-28T02:53:28.980262+0000","last_became_peered":"2023-11-28T02:53:28.980262+0000","last_unstale":"2023-11-28T02:53:28.986230+0000","last_undegraded":"2023-11-28T02:53:28.986230+0000","last_fullsized":"2023-11-28T02:53:28.986230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995281+0000","last_change":"2023-11-28T02:53:28.627331+0000","last_active":"2023-11-28T02:53:28.995281+0000","last_peered":"2023-11-28T02:53:28.995281+0000","last_clean":"2023-11-28T02:53:28.995281+0000","last_became_active":"2023-11-28T02:53:28.627083+0000","last_became_peered":"2023-11-28T02:53:28.627083+0000","last_unstale":"2023-11-28T02:53:28.995281+0000","last_undegraded":"2023-11-28T02:53:28.995281+0000","last_fullsized":"2023-11-28T02:53:28.995281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989720+0000","last_change":"2023-11-28T02:53:28.985192+0000","last_active":"2023-11-28T02:53:28.989720+0000","last_peered":"2023-11-28T02:53:28.989720+0000","last_clean":"2023-11-28T02:53:28.989720+0000","last_became_active":"2023-11-28T02:53:28.985003+0000","last_became_peered":"2023-11-28T02:53:28.985003+0000","last_unstale":"2023-11-28T02:53:28.989720+0000","last_undegraded":"2023-11-28T02:53:28.989720+0000","last_fullsized":"2023-11-28T02:53:28.989720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994276+0000","last_change":"2023-11-28T02:53:28.623609+0000","last_active":"2023-11-28T02:53:28.994276+0000","last_peered":"2023-11-28T02:53:28.994276+0000","last_clean":"2023-11-28T02:53:28.994276+0000","last_became_active":"2023-11-28T02:53:28.623324+0000","last_became_peered":"2023-11-28T02:53:28.623324+0000","last_unstale":"2023-11-28T02:53:28.994276+0000","last_undegraded":"2023-11-28T02:53:28.994276+0000","last_fullsized":"2023-11-28T02:53:28.994276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996976+0000","last_change":"2023-11-28T02:53:28.994540+0000","last_active":"2023-11-28T02:53:28.996976+0000","last_peered":"2023-11-28T02:53:28.996976+0000","last_clean":"2023-11-28T02:53:28.996976+0000","last_became_active":"2023-11-28T02:53:28.994234+0000","last_became_peered":"2023-11-28T02:53:28.994234+0000","last_unstale":"2023-11-28T02:53:28.996976+0000","last_undegraded":"2023-11-28T02:53:28.996976+0000","last_fullsized":"2023-11-28T02:53:28.996976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991199+0000","last_change":"2023-11-28T02:53:28.623468+0000","last_active":"2023-11-28T02:53:28.991199+0000","last_peered":"2023-11-28T02:53:28.991199+0000","last_clean":"2023-11-28T02:53:28.991199+0000","last_became_active":"2023-11-28T02:53:28.623187+0000","last_became_peered":"2023-11-28T02:53:28.623187+0000","last_unstale":"2023-11-28T02:53:28.991199+0000","last_undegraded":"2023-11-28T02:53:28.991199+0000","last_fullsized":"2023-11-28T02:53:28.991199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.988775+0000","last_change":"2023-11-28T02:53:28.980195+0000","last_active":"2023-11-28T02:53:28.988775+0000","last_peered":"2023-11-28T02:53:28.988775+0000","last_clean":"2023-11-28T02:53:28.988775+0000","last_became_active":"2023-11-28T02:53:28.977563+0000","last_became_peered":"2023-11-28T02:53:28.977563+0000","last_unstale":"2023-11-28T02:53:28.988775+0000","last_undegraded":"2023-11-28T02:53:28.988775+0000","last_fullsized":"2023-11-28T02:53:28.988775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983410+0000","last_change":"2023-11-28T02:53:28.621174+0000","last_active":"2023-11-28T02:53:28.983410+0000","last_peered":"2023-11-28T02:53:28.983410+0000","last_clean":"2023-11-28T02:53:28.983410+0000","last_became_active":"2023-11-28T02:53:28.620891+0000","last_became_peered":"2023-11-28T02:53:28.620891+0000","last_unstale":"2023-11-28T02:53:28.983410+0000","last_undegraded":"2023-11-28T02:53:28.983410+0000","last_fullsized":"2023-11-28T02:53:28.983410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994187+0000","last_change":"2023-11-28T02:53:28.985257+0000","last_active":"2023-11-28T02:53:28.994187+0000","last_peered":"2023-11-28T02:53:28.994187+0000","last_clean":"2023-11-28T02:53:28.994187+0000","last_became_active":"2023-11-28T02:53:28.984948+0000","last_became_peered":"2023-11-28T02:53:28.984948+0000","last_unstale":"2023-11-28T02:53:28.994187+0000","last_undegraded":"2023-11-28T02:53:28.994187+0000","last_fullsized":"2023-11-28T02:53:28.994187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968857+0000","last_change":"2023-11-28T02:53:28.624514+0000","last_active":"2023-11-28T02:53:28.968857+0000","last_peered":"2023-11-28T02:53:28.968857+0000","last_clean":"2023-11-28T02:53:28.968857+0000","last_became_active":"2023-11-28T02:53:28.624248+0000","last_became_peered":"2023-11-28T02:53:28.624248+0000","last_unstale":"2023-11-28T02:53:28.968857+0000","last_undegraded":"2023-11-28T02:53:28.968857+0000","last_fullsized":"2023-11-28T02:53:28.968857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993712+0000","last_change":"2023-11-28T02:53:28.986667+0000","last_active":"2023-11-28T02:53:28.993712+0000","last_peered":"2023-11-28T02:53:28.993712+0000","last_clean":"2023-11-28T02:53:28.993712+0000","last_became_active":"2023-11-28T02:53:28.986374+0000","last_became_peered":"2023-11-28T02:53:28.986374+0000","last_unstale":"2023-11-28T02:53:28.993712+0000","last_undegraded":"2023-11-28T02:53:28.993712+0000","last_fullsized":"2023-11-28T02:53:28.993712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994268+0000","last_change":"2023-11-28T02:53:28.606978+0000","last_active":"2023-11-28T02:53:28.994268+0000","last_peered":"2023-11-28T02:53:28.994268+0000","last_clean":"2023-11-28T02:53:28.994268+0000","last_became_active":"2023-11-28T02:53:28.606572+0000","last_became_peered":"2023-11-28T02:53:28.606572+0000","last_unstale":"2023-11-28T02:53:28.994268+0000","last_undegraded":"2023-11-28T02:53:28.994268+0000","last_fullsized":"2023-11-28T02:53:28.994268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991834+0000","last_change":"2023-11-28T02:53:28.988706+0000","last_active":"2023-11-28T02:53:28.991834+0000","last_peered":"2023-11-28T02:53:28.991834+0000","last_clean":"2023-11-28T02:53:28.991834+0000","last_became_active":"2023-11-28T02:53:28.988486+0000","last_became_peered":"2023-11-28T02:53:28.988486+0000","last_unstale":"2023-11-28T02:53:28.991834+0000","last_undegraded":"2023-11-28T02:53:28.991834+0000","last_fullsized":"2023-11-28T02:53:28.991834+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983319+0000","last_change":"2023-11-28T02:53:28.612430+0000","last_active":"2023-11-28T02:53:28.983319+0000","last_peered":"2023-11-28T02:53:28.983319+0000","last_clean":"2023-11-28T02:53:28.983319+0000","last_became_active":"2023-11-28T02:53:28.609736+0000","last_became_peered":"2023-11-28T02:53:28.609736+0000","last_unstale":"2023-11-28T02:53:28.983319+0000","last_undegraded":"2023-11-28T02:53:28.983319+0000","last_fullsized":"2023-11-28T02:53:28.983319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997090+0000","last_change":"2023-11-28T02:53:28.994751+0000","last_active":"2023-11-28T02:53:28.997090+0000","last_peered":"2023-11-28T02:53:28.997090+0000","last_clean":"2023-11-28T02:53:28.997090+0000","last_became_active":"2023-11-28T02:53:28.994461+0000","last_became_peered":"2023-11-28T02:53:28.994461+0000","last_unstale":"2023-11-28T02:53:28.997090+0000","last_undegraded":"2023-11-28T02:53:28.997090+0000","last_fullsized":"2023-11-28T02:53:28.997090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974138+0000","last_change":"2023-11-28T02:53:28.621440+0000","last_active":"2023-11-28T02:53:28.974138+0000","last_peered":"2023-11-28T02:53:28.974138+0000","last_clean":"2023-11-28T02:53:28.974138+0000","last_became_active":"2023-11-28T02:53:28.621181+0000","last_became_peered":"2023-11-28T02:53:28.621181+0000","last_unstale":"2023-11-28T02:53:28.974138+0000","last_undegraded":"2023-11-28T02:53:28.974138+0000","last_fullsized":"2023-11-28T02:53:28.974138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992308+0000","last_change":"2023-11-28T02:53:28.989879+0000","last_active":"2023-11-28T02:53:28.992308+0000","last_peered":"2023-11-28T02:53:28.992308+0000","last_clean":"2023-11-28T02:53:28.992308+0000","last_became_active":"2023-11-28T02:53:28.989420+0000","last_became_peered":"2023-11-28T02:53:28.989420+0000","last_unstale":"2023-11-28T02:53:28.992308+0000","last_undegraded":"2023-11-28T02:53:28.992308+0000","last_fullsized":"2023-11-28T02:53:28.992308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983358+0000","last_change":"2023-11-28T02:53:28.607190+0000","last_active":"2023-11-28T02:53:28.983358+0000","last_peered":"2023-11-28T02:53:28.983358+0000","last_clean":"2023-11-28T02:53:28.983358+0000","last_became_active":"2023-11-28T02:53:28.606693+0000","last_became_peered":"2023-11-28T02:53:28.606693+0000","last_unstale":"2023-11-28T02:53:28.983358+0000","last_undegraded":"2023-11-28T02:53:28.983358+0000","last_fullsized":"2023-11-28T02:53:28.983358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984686+0000","last_change":"2023-11-28T02:53:28.979154+0000","last_active":"2023-11-28T02:53:28.984686+0000","last_peered":"2023-11-28T02:53:28.984686+0000","last_clean":"2023-11-28T02:53:28.984686+0000","last_became_active":"2023-11-28T02:53:28.978734+0000","last_became_peered":"2023-11-28T02:53:28.978734+0000","last_unstale":"2023-11-28T02:53:28.984686+0000","last_undegraded":"2023-11-28T02:53:28.984686+0000","last_fullsized":"2023-11-28T02:53:28.984686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977591+0000","last_change":"2023-11-28T02:53:28.609677+0000","last_active":"2023-11-28T02:53:28.977591+0000","last_peered":"2023-11-28T02:53:28.977591+0000","last_clean":"2023-11-28T02:53:28.977591+0000","last_became_active":"2023-11-28T02:53:28.609414+0000","last_became_peered":"2023-11-28T02:53:28.609414+0000","last_unstale":"2023-11-28T02:53:28.977591+0000","last_undegraded":"2023-11-28T02:53:28.977591+0000","last_fullsized":"2023-11-28T02:53:28.977591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968934+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:53:28.968934+0000","last_peered":"2023-11-28T02:53:28.968934+0000","last_clean":"2023-11-28T02:53:28.968934+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:53:28.968934+0000","last_undegraded":"2023-11-28T02:53:28.968934+0000","last_fullsized":"2023-11-28T02:53:28.968934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987865+0000","last_change":"2023-11-28T02:53:28.982245+0000","last_active":"2023-11-28T02:53:28.987865+0000","last_peered":"2023-11-28T02:53:28.987865+0000","last_clean":"2023-11-28T02:53:28.987865+0000","last_became_active":"2023-11-28T02:53:28.981794+0000","last_became_peered":"2023-11-28T02:53:28.981794+0000","last_unstale":"2023-11-28T02:53:28.987865+0000","last_undegraded":"2023-11-28T02:53:28.987865+0000","last_fullsized":"2023-11-28T02:53:28.987865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977302+0000","last_change":"2023-11-28T02:53:28.613224+0000","last_active":"2023-11-28T02:53:28.977302+0000","last_peered":"2023-11-28T02:53:28.977302+0000","last_clean":"2023-11-28T02:53:28.977302+0000","last_became_active":"2023-11-28T02:53:28.612943+0000","last_became_peered":"2023-11-28T02:53:28.612943+0000","last_unstale":"2023-11-28T02:53:28.977302+0000","last_undegraded":"2023-11-28T02:53:28.977302+0000","last_fullsized":"2023-11-28T02:53:28.977302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.978785+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:53:28.978785+0000","last_peered":"2023-11-28T02:53:28.978785+0000","last_clean":"2023-11-28T02:53:28.978785+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:53:28.978785+0000","last_undegraded":"2023-11-28T02:53:28.978785+0000","last_fullsized":"2023-11-28T02:53:28.978785+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989265+0000","last_change":"2023-11-28T02:53:28.987145+0000","last_active":"2023-11-28T02:53:28.989265+0000","last_peered":"2023-11-28T02:53:28.989265+0000","last_clean":"2023-11-28T02:53:28.989265+0000","last_became_active":"2023-11-28T02:53:28.986548+0000","last_became_peered":"2023-11-28T02:53:28.986548+0000","last_unstale":"2023-11-28T02:53:28.989265+0000","last_undegraded":"2023-11-28T02:53:28.989265+0000","last_fullsized":"2023-11-28T02:53:28.989265+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984195+0000","last_change":"2023-11-28T02:53:28.615680+0000","last_active":"2023-11-28T02:53:28.984195+0000","last_peered":"2023-11-28T02:53:28.984195+0000","last_clean":"2023-11-28T02:53:28.984195+0000","last_became_active":"2023-11-28T02:53:28.615386+0000","last_became_peered":"2023-11-28T02:53:28.615386+0000","last_unstale":"2023-11-28T02:53:28.984195+0000","last_undegraded":"2023-11-28T02:53:28.984195+0000","last_fullsized":"2023-11-28T02:53:28.984195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985160+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:53:28.985160+0000","last_peered":"2023-11-28T02:53:28.985160+0000","last_clean":"2023-11-28T02:53:28.985160+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:53:28.985160+0000","last_undegraded":"2023-11-28T02:53:28.985160+0000","last_fullsized":"2023-11-28T02:53:28.985160+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996472+0000","last_change":"2023-11-28T02:53:28.993278+0000","last_active":"2023-11-28T02:53:28.996472+0000","last_peered":"2023-11-28T02:53:28.996472+0000","last_clean":"2023-11-28T02:53:28.996472+0000","last_became_active":"2023-11-28T02:53:28.990902+0000","last_became_peered":"2023-11-28T02:53:28.990902+0000","last_unstale":"2023-11-28T02:53:28.996472+0000","last_undegraded":"2023-11-28T02:53:28.996472+0000","last_fullsized":"2023-11-28T02:53:28.996472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973405+0000","last_change":"2023-11-28T02:53:28.621358+0000","last_active":"2023-11-28T02:53:28.973405+0000","last_peered":"2023-11-28T02:53:28.973405+0000","last_clean":"2023-11-28T02:53:28.973405+0000","last_became_active":"2023-11-28T02:53:28.621059+0000","last_became_peered":"2023-11-28T02:53:28.621059+0000","last_unstale":"2023-11-28T02:53:28.973405+0000","last_undegraded":"2023-11-28T02:53:28.973405+0000","last_fullsized":"2023-11-28T02:53:28.973405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977109+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:53:28.977109+0000","last_peered":"2023-11-28T02:53:28.977109+0000","last_clean":"2023-11-28T02:53:28.977109+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:53:28.977109+0000","last_undegraded":"2023-11-28T02:53:28.977109+0000","last_fullsized":"2023-11-28T02:53:28.977109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973086+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:53:28.973086+0000","last_peered":"2023-11-28T02:53:28.973086+0000","last_clean":"2023-11-28T02:53:28.973086+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:53:28.973086+0000","last_undegraded":"2023-11-28T02:53:28.973086+0000","last_fullsized":"2023-11-28T02:53:28.973086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994320+0000","last_change":"2023-11-28T02:53:28.988844+0000","last_active":"2023-11-28T02:53:28.994320+0000","last_peered":"2023-11-28T02:53:28.994320+0000","last_clean":"2023-11-28T02:53:28.994320+0000","last_became_active":"2023-11-28T02:53:28.988625+0000","last_became_peered":"2023-11-28T02:53:28.988625+0000","last_unstale":"2023-11-28T02:53:28.994320+0000","last_undegraded":"2023-11-28T02:53:28.994320+0000","last_fullsized":"2023-11-28T02:53:28.994320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995566+0000","last_change":"2023-11-28T02:53:28.613163+0000","last_active":"2023-11-28T02:53:28.995566+0000","last_peered":"2023-11-28T02:53:28.995566+0000","last_clean":"2023-11-28T02:53:28.995566+0000","last_became_active":"2023-11-28T02:53:28.612845+0000","last_became_peered":"2023-11-28T02:53:28.612845+0000","last_unstale":"2023-11-28T02:53:28.995566+0000","last_undegraded":"2023-11-28T02:53:28.995566+0000","last_fullsized":"2023-11-28T02:53:28.995566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973376+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:53:28.973376+0000","last_peered":"2023-11-28T02:53:28.973376+0000","last_clean":"2023-11-28T02:53:28.973376+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:53:28.973376+0000","last_undegraded":"2023-11-28T02:53:28.973376+0000","last_fullsized":"2023-11-28T02:53:28.973376+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991570+0000","last_change":"2023-11-28T02:53:28.985510+0000","last_active":"2023-11-28T02:53:28.991570+0000","last_peered":"2023-11-28T02:53:28.991570+0000","last_clean":"2023-11-28T02:53:28.991570+0000","last_became_active":"2023-11-28T02:53:28.985063+0000","last_became_peered":"2023-11-28T02:53:28.985063+0000","last_unstale":"2023-11-28T02:53:28.991570+0000","last_undegraded":"2023-11-28T02:53:28.991570+0000","last_fullsized":"2023-11-28T02:53:28.991570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975072+0000","last_change":"2023-11-28T02:53:28.608083+0000","last_active":"2023-11-28T02:53:28.975072+0000","last_peered":"2023-11-28T02:53:28.975072+0000","last_clean":"2023-11-28T02:53:28.975072+0000","last_became_active":"2023-11-28T02:53:28.607764+0000","last_became_peered":"2023-11-28T02:53:28.607764+0000","last_unstale":"2023-11-28T02:53:28.975072+0000","last_undegraded":"2023-11-28T02:53:28.975072+0000","last_fullsized":"2023-11-28T02:53:28.975072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974637+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:53:28.974637+0000","last_peered":"2023-11-28T02:53:28.974637+0000","last_clean":"2023-11-28T02:53:28.974637+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:53:28.974637+0000","last_undegraded":"2023-11-28T02:53:28.974637+0000","last_fullsized":"2023-11-28T02:53:28.974637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986257+0000","last_change":"2023-11-28T02:53:28.978737+0000","last_active":"2023-11-28T02:53:28.986257+0000","last_peered":"2023-11-28T02:53:28.986257+0000","last_clean":"2023-11-28T02:53:28.986257+0000","last_became_active":"2023-11-28T02:53:28.978006+0000","last_became_peered":"2023-11-28T02:53:28.978006+0000","last_unstale":"2023-11-28T02:53:28.986257+0000","last_undegraded":"2023-11-28T02:53:28.986257+0000","last_fullsized":"2023-11-28T02:53:28.986257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968251+0000","last_change":"2023-11-28T02:53:28.610884+0000","last_active":"2023-11-28T02:53:28.968251+0000","last_peered":"2023-11-28T02:53:28.968251+0000","last_clean":"2023-11-28T02:53:28.968251+0000","last_became_active":"2023-11-28T02:53:28.609394+0000","last_became_peered":"2023-11-28T02:53:28.609394+0000","last_unstale":"2023-11-28T02:53:28.968251+0000","last_undegraded":"2023-11-28T02:53:28.968251+0000","last_fullsized":"2023-11-28T02:53:28.968251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968936+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:53:28.968936+0000","last_peered":"2023-11-28T02:53:28.968936+0000","last_clean":"2023-11-28T02:53:28.968936+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:53:28.968936+0000","last_undegraded":"2023-11-28T02:53:28.968936+0000","last_fullsized":"2023-11-28T02:53:28.968936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991609+0000","last_change":"2023-11-28T02:53:28.987538+0000","last_active":"2023-11-28T02:53:28.991609+0000","last_peered":"2023-11-28T02:53:28.991609+0000","last_clean":"2023-11-28T02:53:28.991609+0000","last_became_active":"2023-11-28T02:53:28.986197+0000","last_became_peered":"2023-11-28T02:53:28.986197+0000","last_unstale":"2023-11-28T02:53:28.991609+0000","last_undegraded":"2023-11-28T02:53:28.991609+0000","last_fullsized":"2023-11-28T02:53:28.991609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975923+0000","last_change":"2023-11-28T02:53:28.611177+0000","last_active":"2023-11-28T02:53:28.975923+0000","last_peered":"2023-11-28T02:53:28.975923+0000","last_clean":"2023-11-28T02:53:28.975923+0000","last_became_active":"2023-11-28T02:53:28.610893+0000","last_became_peered":"2023-11-28T02:53:28.610893+0000","last_unstale":"2023-11-28T02:53:28.975923+0000","last_undegraded":"2023-11-28T02:53:28.975923+0000","last_fullsized":"2023-11-28T02:53:28.975923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975174+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:53:28.975174+0000","last_peered":"2023-11-28T02:53:28.975174+0000","last_clean":"2023-11-28T02:53:28.975174+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:53:28.975174+0000","last_undegraded":"2023-11-28T02:53:28.975174+0000","last_fullsized":"2023-11-28T02:53:28.975174+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":393216,"data_stored":159348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":159258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":393216,"data_stored":158864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":158738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":385024,"data_stored":158704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":160344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":158726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":160262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:53:35.430 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T02:53:35.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.574+0000 7f5e41971700 1 Processor -- start 2023-11-28T02:53:35.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.574+0000 7f5e41971700 1 -- start start 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41971700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c152190 0x7f5e3c0a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41971700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c0c5ff0 0x7f5e3c0bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41971700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0bf0e0 con 0x7f5e3c0bfa60 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41971700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c0be510 con 0x7f5e3c152190 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41971700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c152570 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c152190 0x7f5e3c0a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41170700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5e3c0bfa60 0x7f5e3c0a5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33758/0 (socket says 172.21.15.73:33758) 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c152190 0x7f5e3c0a8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38830/0 (socket says 172.21.15.73:38830) 2023-11-28T02:53:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e41170700 1 -- 172.21.15.73:0/2003126818 learned_addr learned my addr 172.21.15.73:0/2003126818 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e3bfff700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c0c5ff0 0x7f5e3c0bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 -- 172.21.15.73:0/2003126818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c0c5ff0 msgr2=0x7f5e3c0bf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c0c5ff0 0x7f5e3c0bf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 -- 172.21.15.73:0/2003126818 >> v1:172.21.15.73:6789/0 conn(0x7f5e3c0bfa60 legacy=0x7f5e3c0a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e3b7fe700 1 -- 172.21.15.73:0/2003126818 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2428055199 0 0) 0x7f5e3c0bf0e0 con 0x7f5e3c0bfa60 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 -- 172.21.15.73:0/2003126818 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c0be7b0 con 0x7f5e3c152190 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.575+0000 7f5e4096f700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c152190 0x7f5e3c0a8380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f5e34006fd0 tx=0x7f5e34006f40).ready entity=mon.0 client_cookie=b27be210fc80d02f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e3b7fe700 1 -- 172.21.15.73:0/2003126818 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34005880 con 0x7f5e3c152190 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e3b7fe700 1 -- 172.21.15.73:0/2003126818 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e340059e0 con 0x7f5e3c152190 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e3b7fe700 1 -- 172.21.15.73:0/2003126818 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e34005d60 con 0x7f5e3c152190 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 -- 172.21.15.73:0/2003126818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c152190 msgr2=0x7f5e3c0a8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c152190 0x7f5e3c0a8380 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 rx=0x7f5e34006fd0 tx=0x7f5e34006f40).stop 2023-11-28T02:53:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 -- 172.21.15.73:0/2003126818 shutdown_connections 2023-11-28T02:53:35.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c152190 0x7f5e3c0a8380 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 --2- 172.21.15.73:0/2003126818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c0c5ff0 0x7f5e3c0bf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 -- 172.21.15.73:0/2003126818 >> 172.21.15.73:0/2003126818 conn(0x7f5e3c083000 msgr2=0x7f5e3c092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 -- 172.21.15.73:0/2003126818 shutdown_connections 2023-11-28T02:53:35.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.576+0000 7f5e41971700 1 -- 172.21.15.73:0/2003126818 wait complete. 2023-11-28T02:53:35.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.577+0000 7f5e41971700 1 Processor -- start 2023-11-28T02:53:35.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.577+0000 7f5e41971700 1 -- start start 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c0bfa60 0x7f5e3c0c9320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c152190 0x7f5e3c1577b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c157ca0 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c7220 con 0x7f5e3c0bfa60 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e41971700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0c9810 con 0x7f5e3c152190 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38838/0 (socket says 172.21.15.73:38838) 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 -- 172.21.15.73:0/2435090071 learned_addr learned my addr 172.21.15.73:0/2435090071 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 -- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c152190 msgr2=0x7f5e3c1577b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c152190 0x7f5e3c1577b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 -- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c0bfa60 msgr2=0x7f5e3c0c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.578+0000 7f5e3bfff700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c0bfa60 0x7f5e3c0c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e3bfff700 1 -- 172.21.15.73:0/2435090071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e34009250 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e3bfff700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5e2c00e190 tx=0x7f5e2c009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00edd0 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c014400 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0c0ec0 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c014780 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.579+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0c1480 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.580+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c06c7c0 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.580+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5e2c014b30 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.580+0000 7f5e38ff9700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 0x7f5e1c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.580+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5e2c07a8f0 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.580+0000 7f5e4096f700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 0x7f5e1c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.581+0000 7f5e4096f700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 0x7f5e1c0615e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f5e34009fd0 tx=0x7f5e34016000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.584+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5e2c04bb60 con 0x7f5e3c0c5ff0 2023-11-28T02:53:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.744+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5e3c06c130 con 0x7f5e1c05f120 2023-11-28T02:53:35.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.753+0000 7f5e38ff9700 1 -- 172.21.15.73:0/2435090071 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+336455 (secure 0 0 0) 0x7f5e3c06c130 con 0x7f5e1c05f120 2023-11-28T02:53:35.758 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 msgr2=0x7f5e1c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 0x7f5e1c0615e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7f5e34009fd0 tx=0x7f5e34016000).stop 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 msgr2=0x7f5e3c1572c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 rx=0x7f5e2c00e190 tx=0x7f5e2c009900).stop 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 shutdown_connections 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e1c05f120 0x7f5e1c0615e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c152190 0x7f5e3c1577b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c0bfa60 0x7f5e3c0c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 --2- 172.21.15.73:0/2435090071 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c0c5ff0 0x7f5e3c1572c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 >> 172.21.15.73:0/2435090071 conn(0x7f5e3c083000 msgr2=0x7f5e3c14d370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 shutdown_connections 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.756+0000 7f5e41971700 1 -- 172.21.15.73:0/2435090071 wait complete. 2023-11-28T02:53:35.762 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T02:53:35.769 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":57,"stamp":"2023-11-28T02:53:35.406372+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":237,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2379840,"kb_used_data":3072,"kb_used_omap":0,"kb_used_meta":2376704,"kb_avail":752594880,"statfs":{"total":773094113280,"available":770657157120,"internally_reserved":0,"allocated":3145728,"data_stored":1274244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2433744896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4305,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.175407"},"pg_stats":[{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994329+0000","last_change":"2023-11-28T02:53:28.987670+0000","last_active":"2023-11-28T02:53:28.994329+0000","last_peered":"2023-11-28T02:53:28.994329+0000","last_clean":"2023-11-28T02:53:28.994329+0000","last_became_active":"2023-11-28T02:53:28.987421+0000","last_became_peered":"2023-11-28T02:53:28.987421+0000","last_unstale":"2023-11-28T02:53:28.994329+0000","last_undegraded":"2023-11-28T02:53:28.994329+0000","last_fullsized":"2023-11-28T02:53:28.994329+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3f","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997134+0000","last_change":"2023-11-28T02:53:28.615436+0000","last_active":"2023-11-28T02:53:28.997134+0000","last_peered":"2023-11-28T02:53:28.997134+0000","last_clean":"2023-11-28T02:53:28.997134+0000","last_became_active":"2023-11-28T02:53:28.615171+0000","last_became_peered":"2023-11-28T02:53:28.615171+0000","last_unstale":"2023-11-28T02:53:28.997134+0000","last_undegraded":"2023-11-28T02:53:28.997134+0000","last_fullsized":"2023-11-28T02:53:28.997134+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991501+0000","last_change":"2023-11-28T02:53:28.988359+0000","last_active":"2023-11-28T02:53:28.991501+0000","last_peered":"2023-11-28T02:53:28.991501+0000","last_clean":"2023-11-28T02:53:28.991501+0000","last_became_active":"2023-11-28T02:53:28.987894+0000","last_became_peered":"2023-11-28T02:53:28.987894+0000","last_unstale":"2023-11-28T02:53:28.991501+0000","last_undegraded":"2023-11-28T02:53:28.991501+0000","last_fullsized":"2023-11-28T02:53:28.991501+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974610+0000","last_change":"2023-11-28T02:53:28.607829+0000","last_active":"2023-11-28T02:53:28.974610+0000","last_peered":"2023-11-28T02:53:28.974610+0000","last_clean":"2023-11-28T02:53:28.974610+0000","last_became_active":"2023-11-28T02:53:28.607574+0000","last_became_peered":"2023-11-28T02:53:28.607574+0000","last_unstale":"2023-11-28T02:53:28.974610+0000","last_undegraded":"2023-11-28T02:53:28.974610+0000","last_fullsized":"2023-11-28T02:53:28.974610+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987889+0000","last_change":"2023-11-28T02:53:28.982914+0000","last_active":"2023-11-28T02:53:28.987889+0000","last_peered":"2023-11-28T02:53:28.987889+0000","last_clean":"2023-11-28T02:53:28.987889+0000","last_became_active":"2023-11-28T02:53:28.982648+0000","last_became_peered":"2023-11-28T02:53:28.982648+0000","last_unstale":"2023-11-28T02:53:28.987889+0000","last_undegraded":"2023-11-28T02:53:28.987889+0000","last_fullsized":"2023-11-28T02:53:28.987889+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.981403+0000","last_change":"2023-11-28T02:53:28.614003+0000","last_active":"2023-11-28T02:53:28.981403+0000","last_peered":"2023-11-28T02:53:28.981403+0000","last_clean":"2023-11-28T02:53:28.981403+0000","last_became_active":"2023-11-28T02:53:28.613504+0000","last_became_peered":"2023-11-28T02:53:28.613504+0000","last_unstale":"2023-11-28T02:53:28.981403+0000","last_undegraded":"2023-11-28T02:53:28.981403+0000","last_fullsized":"2023-11-28T02:53:28.981403+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986511+0000","last_change":"2023-11-28T02:53:28.980516+0000","last_active":"2023-11-28T02:53:28.986511+0000","last_peered":"2023-11-28T02:53:28.986511+0000","last_clean":"2023-11-28T02:53:28.986511+0000","last_became_active":"2023-11-28T02:53:28.980254+0000","last_became_peered":"2023-11-28T02:53:28.980254+0000","last_unstale":"2023-11-28T02:53:28.986511+0000","last_undegraded":"2023-11-28T02:53:28.986511+0000","last_fullsized":"2023-11-28T02:53:28.986511+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990633+0000","last_change":"2023-11-28T02:53:28.625815+0000","last_active":"2023-11-28T02:53:28.990633+0000","last_peered":"2023-11-28T02:53:28.990633+0000","last_clean":"2023-11-28T02:53:28.990633+0000","last_became_active":"2023-11-28T02:53:28.625542+0000","last_became_peered":"2023-11-28T02:53:28.625542+0000","last_unstale":"2023-11-28T02:53:28.990633+0000","last_undegraded":"2023-11-28T02:53:28.990633+0000","last_fullsized":"2023-11-28T02:53:28.990633+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994331+0000","last_change":"2023-11-28T02:53:28.991711+0000","last_active":"2023-11-28T02:53:28.994331+0000","last_peered":"2023-11-28T02:53:28.994331+0000","last_clean":"2023-11-28T02:53:28.994331+0000","last_became_active":"2023-11-28T02:53:28.991447+0000","last_became_peered":"2023-11-28T02:53:28.991447+0000","last_unstale":"2023-11-28T02:53:28.994331+0000","last_undegraded":"2023-11-28T02:53:28.994331+0000","last_fullsized":"2023-11-28T02:53:28.994331+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994166+0000","last_change":"2023-11-28T02:53:28.606830+0000","last_active":"2023-11-28T02:53:28.994166+0000","last_peered":"2023-11-28T02:53:28.994166+0000","last_clean":"2023-11-28T02:53:28.994166+0000","last_became_active":"2023-11-28T02:53:28.606253+0000","last_became_peered":"2023-11-28T02:53:28.606253+0000","last_unstale":"2023-11-28T02:53:28.994166+0000","last_undegraded":"2023-11-28T02:53:28.994166+0000","last_fullsized":"2023-11-28T02:53:28.994166+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993557+0000","last_change":"2023-11-28T02:53:28.987243+0000","last_active":"2023-11-28T02:53:28.993557+0000","last_peered":"2023-11-28T02:53:28.993557+0000","last_clean":"2023-11-28T02:53:28.993557+0000","last_became_active":"2023-11-28T02:53:28.986679+0000","last_became_peered":"2023-11-28T02:53:28.986679+0000","last_unstale":"2023-11-28T02:53:28.993557+0000","last_undegraded":"2023-11-28T02:53:28.993557+0000","last_fullsized":"2023-11-28T02:53:28.993557+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989015+0000","last_change":"2023-11-28T02:53:28.614186+0000","last_active":"2023-11-28T02:53:28.989015+0000","last_peered":"2023-11-28T02:53:28.989015+0000","last_clean":"2023-11-28T02:53:28.989015+0000","last_became_active":"2023-11-28T02:53:28.613904+0000","last_became_peered":"2023-11-28T02:53:28.613904+0000","last_unstale":"2023-11-28T02:53:28.989015+0000","last_undegraded":"2023-11-28T02:53:28.989015+0000","last_fullsized":"2023-11-28T02:53:28.989015+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.982877+0000","last_change":"2023-11-28T02:53:28.978510+0000","last_active":"2023-11-28T02:53:28.982877+0000","last_peered":"2023-11-28T02:53:28.982877+0000","last_clean":"2023-11-28T02:53:28.982877+0000","last_became_active":"2023-11-28T02:53:28.978249+0000","last_became_peered":"2023-11-28T02:53:28.978249+0000","last_unstale":"2023-11-28T02:53:28.982877+0000","last_undegraded":"2023-11-28T02:53:28.982877+0000","last_fullsized":"2023-11-28T02:53:28.982877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973327+0000","last_change":"2023-11-28T02:53:28.621237+0000","last_active":"2023-11-28T02:53:28.973327+0000","last_peered":"2023-11-28T02:53:28.973327+0000","last_clean":"2023-11-28T02:53:28.973327+0000","last_became_active":"2023-11-28T02:53:28.620850+0000","last_became_peered":"2023-11-28T02:53:28.620850+0000","last_unstale":"2023-11-28T02:53:28.973327+0000","last_undegraded":"2023-11-28T02:53:28.973327+0000","last_fullsized":"2023-11-28T02:53:28.973327+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994316+0000","last_change":"2023-11-28T02:53:28.987573+0000","last_active":"2023-11-28T02:53:28.994316+0000","last_peered":"2023-11-28T02:53:28.994316+0000","last_clean":"2023-11-28T02:53:28.994316+0000","last_became_active":"2023-11-28T02:53:28.987293+0000","last_became_peered":"2023-11-28T02:53:28.987293+0000","last_unstale":"2023-11-28T02:53:28.994316+0000","last_undegraded":"2023-11-28T02:53:28.994316+0000","last_fullsized":"2023-11-28T02:53:28.994316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973596+0000","last_change":"2023-11-28T02:53:28.619795+0000","last_active":"2023-11-28T02:53:28.973596+0000","last_peered":"2023-11-28T02:53:28.973596+0000","last_clean":"2023-11-28T02:53:28.973596+0000","last_became_active":"2023-11-28T02:53:28.619452+0000","last_became_peered":"2023-11-28T02:53:28.619452+0000","last_unstale":"2023-11-28T02:53:28.973596+0000","last_undegraded":"2023-11-28T02:53:28.973596+0000","last_fullsized":"2023-11-28T02:53:28.973596+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990559+0000","last_change":"2023-11-28T02:53:28.986425+0000","last_active":"2023-11-28T02:53:28.990559+0000","last_peered":"2023-11-28T02:53:28.990559+0000","last_clean":"2023-11-28T02:53:28.990559+0000","last_became_active":"2023-11-28T02:53:28.986130+0000","last_became_peered":"2023-11-28T02:53:28.986130+0000","last_unstale":"2023-11-28T02:53:28.990559+0000","last_undegraded":"2023-11-28T02:53:28.990559+0000","last_fullsized":"2023-11-28T02:53:28.990559+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968733+0000","last_change":"2023-11-28T02:53:28.615665+0000","last_active":"2023-11-28T02:53:28.968733+0000","last_peered":"2023-11-28T02:53:28.968733+0000","last_clean":"2023-11-28T02:53:28.968733+0000","last_became_active":"2023-11-28T02:53:28.615238+0000","last_became_peered":"2023-11-28T02:53:28.615238+0000","last_unstale":"2023-11-28T02:53:28.968733+0000","last_undegraded":"2023-11-28T02:53:28.968733+0000","last_fullsized":"2023-11-28T02:53:28.968733+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996054+0000","last_change":"2023-11-28T02:53:28.992310+0000","last_active":"2023-11-28T02:53:28.996054+0000","last_peered":"2023-11-28T02:53:28.996054+0000","last_clean":"2023-11-28T02:53:28.996054+0000","last_became_active":"2023-11-28T02:53:28.991941+0000","last_became_peered":"2023-11-28T02:53:28.991941+0000","last_unstale":"2023-11-28T02:53:28.996054+0000","last_undegraded":"2023-11-28T02:53:28.996054+0000","last_fullsized":"2023-11-28T02:53:28.996054+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968891+0000","last_change":"2023-11-28T02:53:28.616239+0000","last_active":"2023-11-28T02:53:28.968891+0000","last_peered":"2023-11-28T02:53:28.968891+0000","last_clean":"2023-11-28T02:53:28.968891+0000","last_became_active":"2023-11-28T02:53:28.615922+0000","last_became_peered":"2023-11-28T02:53:28.615922+0000","last_unstale":"2023-11-28T02:53:28.968891+0000","last_undegraded":"2023-11-28T02:53:28.968891+0000","last_fullsized":"2023-11-28T02:53:28.968891+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996992+0000","last_change":"2023-11-28T02:53:28.994692+0000","last_active":"2023-11-28T02:53:28.996992+0000","last_peered":"2023-11-28T02:53:28.996992+0000","last_clean":"2023-11-28T02:53:28.996992+0000","last_became_active":"2023-11-28T02:53:28.994376+0000","last_became_peered":"2023-11-28T02:53:28.994376+0000","last_unstale":"2023-11-28T02:53:28.996992+0000","last_undegraded":"2023-11-28T02:53:28.996992+0000","last_fullsized":"2023-11-28T02:53:28.996992+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974544+0000","last_change":"2023-11-28T02:53:28.623298+0000","last_active":"2023-11-28T02:53:28.974544+0000","last_peered":"2023-11-28T02:53:28.974544+0000","last_clean":"2023-11-28T02:53:28.974544+0000","last_became_active":"2023-11-28T02:53:28.623015+0000","last_became_peered":"2023-11-28T02:53:28.623015+0000","last_unstale":"2023-11-28T02:53:28.974544+0000","last_undegraded":"2023-11-28T02:53:28.974544+0000","last_fullsized":"2023-11-28T02:53:28.974544+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997432+0000","last_change":"2023-11-28T02:53:28.991999+0000","last_active":"2023-11-28T02:53:28.997432+0000","last_peered":"2023-11-28T02:53:28.997432+0000","last_clean":"2023-11-28T02:53:28.997432+0000","last_became_active":"2023-11-28T02:53:28.991669+0000","last_became_peered":"2023-11-28T02:53:28.991669+0000","last_unstale":"2023-11-28T02:53:28.997432+0000","last_undegraded":"2023-11-28T02:53:28.997432+0000","last_fullsized":"2023-11-28T02:53:28.997432+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.34","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:29.002034+0000","last_change":"2023-11-28T02:53:28.614001+0000","last_active":"2023-11-28T02:53:29.002034+0000","last_peered":"2023-11-28T02:53:29.002034+0000","last_clean":"2023-11-28T02:53:29.002034+0000","last_became_active":"2023-11-28T02:53:28.613640+0000","last_became_peered":"2023-11-28T02:53:28.613640+0000","last_unstale":"2023-11-28T02:53:29.002034+0000","last_undegraded":"2023-11-28T02:53:29.002034+0000","last_fullsized":"2023-11-28T02:53:29.002034+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997550+0000","last_change":"2023-11-28T02:53:28.995577+0000","last_active":"2023-11-28T02:53:28.997550+0000","last_peered":"2023-11-28T02:53:28.997550+0000","last_clean":"2023-11-28T02:53:28.997550+0000","last_became_active":"2023-11-28T02:53:28.995316+0000","last_became_peered":"2023-11-28T02:53:28.995316+0000","last_unstale":"2023-11-28T02:53:28.997550+0000","last_undegraded":"2023-11-28T02:53:28.997550+0000","last_fullsized":"2023-11-28T02:53:28.997550+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.33","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992441+0000","last_change":"2023-11-28T02:53:28.612256+0000","last_active":"2023-11-28T02:53:28.992441+0000","last_peered":"2023-11-28T02:53:28.992441+0000","last_clean":"2023-11-28T02:53:28.992441+0000","last_became_active":"2023-11-28T02:53:28.611877+0000","last_became_peered":"2023-11-28T02:53:28.611877+0000","last_unstale":"2023-11-28T02:53:28.992441+0000","last_undegraded":"2023-11-28T02:53:28.992441+0000","last_fullsized":"2023-11-28T02:53:28.992441+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993574+0000","last_change":"2023-11-28T02:53:28.989222+0000","last_active":"2023-11-28T02:53:28.993574+0000","last_peered":"2023-11-28T02:53:28.993574+0000","last_clean":"2023-11-28T02:53:28.993574+0000","last_became_active":"2023-11-28T02:53:28.987099+0000","last_became_peered":"2023-11-28T02:53:28.987099+0000","last_unstale":"2023-11-28T02:53:28.993574+0000","last_undegraded":"2023-11-28T02:53:28.993574+0000","last_fullsized":"2023-11-28T02:53:28.993574+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974957+0000","last_change":"2023-11-28T02:53:28.604483+0000","last_active":"2023-11-28T02:53:28.974957+0000","last_peered":"2023-11-28T02:53:28.974957+0000","last_clean":"2023-11-28T02:53:28.974957+0000","last_became_active":"2023-11-28T02:53:28.604198+0000","last_became_peered":"2023-11-28T02:53:28.604198+0000","last_unstale":"2023-11-28T02:53:28.974957+0000","last_undegraded":"2023-11-28T02:53:28.974957+0000","last_fullsized":"2023-11-28T02:53:28.974957+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986280+0000","last_change":"2023-11-28T02:53:28.978915+0000","last_active":"2023-11-28T02:53:28.986280+0000","last_peered":"2023-11-28T02:53:28.986280+0000","last_clean":"2023-11-28T02:53:28.986280+0000","last_became_active":"2023-11-28T02:53:28.978436+0000","last_became_peered":"2023-11-28T02:53:28.978436+0000","last_unstale":"2023-11-28T02:53:28.986280+0000","last_undegraded":"2023-11-28T02:53:28.986280+0000","last_fullsized":"2023-11-28T02:53:28.986280+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977737+0000","last_change":"2023-11-28T02:53:28.614088+0000","last_active":"2023-11-28T02:53:28.977737+0000","last_peered":"2023-11-28T02:53:28.977737+0000","last_clean":"2023-11-28T02:53:28.977737+0000","last_became_active":"2023-11-28T02:53:28.613799+0000","last_became_peered":"2023-11-28T02:53:28.613799+0000","last_unstale":"2023-11-28T02:53:28.977737+0000","last_undegraded":"2023-11-28T02:53:28.977737+0000","last_fullsized":"2023-11-28T02:53:28.977737+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993749+0000","last_change":"2023-11-28T02:53:28.990079+0000","last_active":"2023-11-28T02:53:28.993749+0000","last_peered":"2023-11-28T02:53:28.993749+0000","last_clean":"2023-11-28T02:53:28.993749+0000","last_became_active":"2023-11-28T02:53:28.989796+0000","last_became_peered":"2023-11-28T02:53:28.989796+0000","last_unstale":"2023-11-28T02:53:28.993749+0000","last_undegraded":"2023-11-28T02:53:28.993749+0000","last_fullsized":"2023-11-28T02:53:28.993749+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990305+0000","last_change":"2023-11-28T02:53:28.619663+0000","last_active":"2023-11-28T02:53:28.990305+0000","last_peered":"2023-11-28T02:53:28.990305+0000","last_clean":"2023-11-28T02:53:28.990305+0000","last_became_active":"2023-11-28T02:53:28.619099+0000","last_became_peered":"2023-11-28T02:53:28.619099+0000","last_unstale":"2023-11-28T02:53:28.990305+0000","last_undegraded":"2023-11-28T02:53:28.990305+0000","last_fullsized":"2023-11-28T02:53:28.990305+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983843+0000","last_change":"2023-11-28T02:53:28.981838+0000","last_active":"2023-11-28T02:53:28.983843+0000","last_peered":"2023-11-28T02:53:28.983843+0000","last_clean":"2023-11-28T02:53:28.983843+0000","last_became_active":"2023-11-28T02:53:28.981157+0000","last_became_peered":"2023-11-28T02:53:28.981157+0000","last_unstale":"2023-11-28T02:53:28.983843+0000","last_undegraded":"2023-11-28T02:53:28.983843+0000","last_fullsized":"2023-11-28T02:53:28.983843+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972511+0000","last_change":"2023-11-28T02:53:28.625948+0000","last_active":"2023-11-28T02:53:28.972511+0000","last_peered":"2023-11-28T02:53:28.972511+0000","last_clean":"2023-11-28T02:53:28.972511+0000","last_became_active":"2023-11-28T02:53:28.625727+0000","last_became_peered":"2023-11-28T02:53:28.625727+0000","last_unstale":"2023-11-28T02:53:28.972511+0000","last_undegraded":"2023-11-28T02:53:28.972511+0000","last_fullsized":"2023-11-28T02:53:28.972511+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994166+0000","last_change":"2023-11-28T02:53:28.985735+0000","last_active":"2023-11-28T02:53:28.994166+0000","last_peered":"2023-11-28T02:53:28.994166+0000","last_clean":"2023-11-28T02:53:28.994166+0000","last_became_active":"2023-11-28T02:53:28.985480+0000","last_became_peered":"2023-11-28T02:53:28.985480+0000","last_unstale":"2023-11-28T02:53:28.994166+0000","last_undegraded":"2023-11-28T02:53:28.994166+0000","last_fullsized":"2023-11-28T02:53:28.994166+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974322+0000","last_change":"2023-11-28T02:53:28.613536+0000","last_active":"2023-11-28T02:53:28.974322+0000","last_peered":"2023-11-28T02:53:28.974322+0000","last_clean":"2023-11-28T02:53:28.974322+0000","last_became_active":"2023-11-28T02:53:28.613217+0000","last_became_peered":"2023-11-28T02:53:28.613217+0000","last_unstale":"2023-11-28T02:53:28.974322+0000","last_undegraded":"2023-11-28T02:53:28.974322+0000","last_fullsized":"2023-11-28T02:53:28.974322+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989513+0000","last_change":"2023-11-28T02:53:28.983979+0000","last_active":"2023-11-28T02:53:28.989513+0000","last_peered":"2023-11-28T02:53:28.989513+0000","last_clean":"2023-11-28T02:53:28.989513+0000","last_became_active":"2023-11-28T02:53:28.983627+0000","last_became_peered":"2023-11-28T02:53:28.983627+0000","last_unstale":"2023-11-28T02:53:28.989513+0000","last_undegraded":"2023-11-28T02:53:28.989513+0000","last_fullsized":"2023-11-28T02:53:28.989513+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991928+0000","last_change":"2023-11-28T02:53:28.613321+0000","last_active":"2023-11-28T02:53:28.991928+0000","last_peered":"2023-11-28T02:53:28.991928+0000","last_clean":"2023-11-28T02:53:28.991928+0000","last_became_active":"2023-11-28T02:53:28.613039+0000","last_became_peered":"2023-11-28T02:53:28.613039+0000","last_unstale":"2023-11-28T02:53:28.991928+0000","last_undegraded":"2023-11-28T02:53:28.991928+0000","last_fullsized":"2023-11-28T02:53:28.991928+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996877+0000","last_change":"2023-11-28T02:53:28.994024+0000","last_active":"2023-11-28T02:53:28.996877+0000","last_peered":"2023-11-28T02:53:28.996877+0000","last_clean":"2023-11-28T02:53:28.996877+0000","last_became_active":"2023-11-28T02:53:28.993749+0000","last_became_peered":"2023-11-28T02:53:28.993749+0000","last_unstale":"2023-11-28T02:53:28.996877+0000","last_undegraded":"2023-11-28T02:53:28.996877+0000","last_fullsized":"2023-11-28T02:53:28.996877+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984340+0000","last_change":"2023-11-28T02:53:28.615797+0000","last_active":"2023-11-28T02:53:28.984340+0000","last_peered":"2023-11-28T02:53:28.984340+0000","last_clean":"2023-11-28T02:53:28.984340+0000","last_became_active":"2023-11-28T02:53:28.615499+0000","last_became_peered":"2023-11-28T02:53:28.615499+0000","last_unstale":"2023-11-28T02:53:28.984340+0000","last_undegraded":"2023-11-28T02:53:28.984340+0000","last_fullsized":"2023-11-28T02:53:28.984340+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997447+0000","last_change":"2023-11-28T02:53:28.994091+0000","last_active":"2023-11-28T02:53:28.997447+0000","last_peered":"2023-11-28T02:53:28.997447+0000","last_clean":"2023-11-28T02:53:28.997447+0000","last_became_active":"2023-11-28T02:53:28.993809+0000","last_became_peered":"2023-11-28T02:53:28.993809+0000","last_unstale":"2023-11-28T02:53:28.997447+0000","last_undegraded":"2023-11-28T02:53:28.997447+0000","last_fullsized":"2023-11-28T02:53:28.997447+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991158+0000","last_change":"2023-11-28T02:53:28.622046+0000","last_active":"2023-11-28T02:53:28.991158+0000","last_peered":"2023-11-28T02:53:28.991158+0000","last_clean":"2023-11-28T02:53:28.991158+0000","last_became_active":"2023-11-28T02:53:28.621813+0000","last_became_peered":"2023-11-28T02:53:28.621813+0000","last_unstale":"2023-11-28T02:53:28.991158+0000","last_undegraded":"2023-11-28T02:53:28.991158+0000","last_fullsized":"2023-11-28T02:53:28.991158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995692+0000","last_change":"2023-11-28T02:53:28.990175+0000","last_active":"2023-11-28T02:53:28.995692+0000","last_peered":"2023-11-28T02:53:28.995692+0000","last_clean":"2023-11-28T02:53:28.995692+0000","last_became_active":"2023-11-28T02:53:28.988430+0000","last_became_peered":"2023-11-28T02:53:28.988430+0000","last_unstale":"2023-11-28T02:53:28.995692+0000","last_undegraded":"2023-11-28T02:53:28.995692+0000","last_fullsized":"2023-11-28T02:53:28.995692+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969384+0000","last_change":"2023-11-28T02:53:28.624148+0000","last_active":"2023-11-28T02:53:28.969384+0000","last_peered":"2023-11-28T02:53:28.969384+0000","last_clean":"2023-11-28T02:53:28.969384+0000","last_became_active":"2023-11-28T02:53:28.622802+0000","last_became_peered":"2023-11-28T02:53:28.622802+0000","last_unstale":"2023-11-28T02:53:28.969384+0000","last_undegraded":"2023-11-28T02:53:28.969384+0000","last_fullsized":"2023-11-28T02:53:28.969384+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996093+0000","last_change":"2023-11-28T02:53:28.992937+0000","last_active":"2023-11-28T02:53:28.996093+0000","last_peered":"2023-11-28T02:53:28.996093+0000","last_clean":"2023-11-28T02:53:28.996093+0000","last_became_active":"2023-11-28T02:53:28.992636+0000","last_became_peered":"2023-11-28T02:53:28.992636+0000","last_unstale":"2023-11-28T02:53:28.996093+0000","last_undegraded":"2023-11-28T02:53:28.996093+0000","last_fullsized":"2023-11-28T02:53:28.996093+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977354+0000","last_change":"2023-11-28T02:53:28.619570+0000","last_active":"2023-11-28T02:53:28.977354+0000","last_peered":"2023-11-28T02:53:28.977354+0000","last_clean":"2023-11-28T02:53:28.977354+0000","last_became_active":"2023-11-28T02:53:28.619323+0000","last_became_peered":"2023-11-28T02:53:28.619323+0000","last_unstale":"2023-11-28T02:53:28.977354+0000","last_undegraded":"2023-11-28T02:53:28.977354+0000","last_fullsized":"2023-11-28T02:53:28.977354+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987789+0000","last_change":"2023-11-28T02:53:28.984950+0000","last_active":"2023-11-28T02:53:28.987789+0000","last_peered":"2023-11-28T02:53:28.987789+0000","last_clean":"2023-11-28T02:53:28.987789+0000","last_became_active":"2023-11-28T02:53:28.984627+0000","last_became_peered":"2023-11-28T02:53:28.984627+0000","last_unstale":"2023-11-28T02:53:28.987789+0000","last_undegraded":"2023-11-28T02:53:28.987789+0000","last_fullsized":"2023-11-28T02:53:28.987789+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991026+0000","last_change":"2023-11-28T02:53:28.612552+0000","last_active":"2023-11-28T02:53:28.991026+0000","last_peered":"2023-11-28T02:53:28.991026+0000","last_clean":"2023-11-28T02:53:28.991026+0000","last_became_active":"2023-11-28T02:53:28.612193+0000","last_became_peered":"2023-11-28T02:53:28.612193+0000","last_unstale":"2023-11-28T02:53:28.991026+0000","last_undegraded":"2023-11-28T02:53:28.991026+0000","last_fullsized":"2023-11-28T02:53:28.991026+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996305+0000","last_change":"2023-11-28T02:53:28.992515+0000","last_active":"2023-11-28T02:53:28.996305+0000","last_peered":"2023-11-28T02:53:28.996305+0000","last_clean":"2023-11-28T02:53:28.996305+0000","last_became_active":"2023-11-28T02:53:28.989851+0000","last_became_peered":"2023-11-28T02:53:28.989851+0000","last_unstale":"2023-11-28T02:53:28.996305+0000","last_undegraded":"2023-11-28T02:53:28.996305+0000","last_fullsized":"2023-11-28T02:53:28.996305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969075+0000","last_change":"2023-11-28T02:53:28.618220+0000","last_active":"2023-11-28T02:53:28.969075+0000","last_peered":"2023-11-28T02:53:28.969075+0000","last_clean":"2023-11-28T02:53:28.969075+0000","last_became_active":"2023-11-28T02:53:28.617971+0000","last_became_peered":"2023-11-28T02:53:28.617971+0000","last_unstale":"2023-11-28T02:53:28.969075+0000","last_undegraded":"2023-11-28T02:53:28.969075+0000","last_fullsized":"2023-11-28T02:53:28.969075+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993916+0000","last_change":"2023-11-28T02:53:28.989630+0000","last_active":"2023-11-28T02:53:28.993916+0000","last_peered":"2023-11-28T02:53:28.993916+0000","last_clean":"2023-11-28T02:53:28.993916+0000","last_became_active":"2023-11-28T02:53:28.989322+0000","last_became_peered":"2023-11-28T02:53:28.989322+0000","last_unstale":"2023-11-28T02:53:28.993916+0000","last_undegraded":"2023-11-28T02:53:28.993916+0000","last_fullsized":"2023-11-28T02:53:28.993916+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974420+0000","last_change":"2023-11-28T02:53:28.615301+0000","last_active":"2023-11-28T02:53:28.974420+0000","last_peered":"2023-11-28T02:53:28.974420+0000","last_clean":"2023-11-28T02:53:28.974420+0000","last_became_active":"2023-11-28T02:53:28.614882+0000","last_became_peered":"2023-11-28T02:53:28.614882+0000","last_unstale":"2023-11-28T02:53:28.974420+0000","last_undegraded":"2023-11-28T02:53:28.974420+0000","last_fullsized":"2023-11-28T02:53:28.974420+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994461+0000","last_change":"2023-11-28T02:53:28.989438+0000","last_active":"2023-11-28T02:53:28.994461+0000","last_peered":"2023-11-28T02:53:28.994461+0000","last_clean":"2023-11-28T02:53:28.994461+0000","last_became_active":"2023-11-28T02:53:28.988433+0000","last_became_peered":"2023-11-28T02:53:28.988433+0000","last_unstale":"2023-11-28T02:53:28.994461+0000","last_undegraded":"2023-11-28T02:53:28.994461+0000","last_fullsized":"2023-11-28T02:53:28.994461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.969581+0000","last_change":"2023-11-28T02:53:28.624478+0000","last_active":"2023-11-28T02:53:28.969581+0000","last_peered":"2023-11-28T02:53:28.969581+0000","last_clean":"2023-11-28T02:53:28.969581+0000","last_became_active":"2023-11-28T02:53:28.624184+0000","last_became_peered":"2023-11-28T02:53:28.624184+0000","last_unstale":"2023-11-28T02:53:28.969581+0000","last_undegraded":"2023-11-28T02:53:28.969581+0000","last_fullsized":"2023-11-28T02:53:28.969581+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983881+0000","last_change":"2023-11-28T02:53:28.980304+0000","last_active":"2023-11-28T02:53:28.983881+0000","last_peered":"2023-11-28T02:53:28.983881+0000","last_clean":"2023-11-28T02:53:28.983881+0000","last_became_active":"2023-11-28T02:53:28.980010+0000","last_became_peered":"2023-11-28T02:53:28.980010+0000","last_unstale":"2023-11-28T02:53:28.983881+0000","last_undegraded":"2023-11-28T02:53:28.983881+0000","last_fullsized":"2023-11-28T02:53:28.983881+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.982139+0000","last_change":"2023-11-28T02:53:28.614011+0000","last_active":"2023-11-28T02:53:28.982139+0000","last_peered":"2023-11-28T02:53:28.982139+0000","last_clean":"2023-11-28T02:53:28.982139+0000","last_became_active":"2023-11-28T02:53:28.613693+0000","last_became_peered":"2023-11-28T02:53:28.613693+0000","last_unstale":"2023-11-28T02:53:28.982139+0000","last_undegraded":"2023-11-28T02:53:28.982139+0000","last_fullsized":"2023-11-28T02:53:28.982139+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994527+0000","last_change":"2023-11-28T02:53:28.989670+0000","last_active":"2023-11-28T02:53:28.994527+0000","last_peered":"2023-11-28T02:53:28.994527+0000","last_clean":"2023-11-28T02:53:28.994527+0000","last_became_active":"2023-11-28T02:53:28.989404+0000","last_became_peered":"2023-11-28T02:53:28.989404+0000","last_unstale":"2023-11-28T02:53:28.994527+0000","last_undegraded":"2023-11-28T02:53:28.994527+0000","last_fullsized":"2023-11-28T02:53:28.994527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968290+0000","last_change":"2023-11-28T02:53:28.622136+0000","last_active":"2023-11-28T02:53:28.968290+0000","last_peered":"2023-11-28T02:53:28.968290+0000","last_clean":"2023-11-28T02:53:28.968290+0000","last_became_active":"2023-11-28T02:53:28.615426+0000","last_became_peered":"2023-11-28T02:53:28.615426+0000","last_unstale":"2023-11-28T02:53:28.968290+0000","last_undegraded":"2023-11-28T02:53:28.968290+0000","last_fullsized":"2023-11-28T02:53:28.968290+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996526+0000","last_change":"2023-11-28T02:53:28.993084+0000","last_active":"2023-11-28T02:53:28.996526+0000","last_peered":"2023-11-28T02:53:28.996526+0000","last_clean":"2023-11-28T02:53:28.996526+0000","last_became_active":"2023-11-28T02:53:28.989889+0000","last_became_peered":"2023-11-28T02:53:28.989889+0000","last_unstale":"2023-11-28T02:53:28.996526+0000","last_undegraded":"2023-11-28T02:53:28.996526+0000","last_fullsized":"2023-11-28T02:53:28.996526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973080+0000","last_change":"2023-11-28T02:53:28.623918+0000","last_active":"2023-11-28T02:53:28.973080+0000","last_peered":"2023-11-28T02:53:28.973080+0000","last_clean":"2023-11-28T02:53:28.973080+0000","last_became_active":"2023-11-28T02:53:28.623641+0000","last_became_peered":"2023-11-28T02:53:28.623641+0000","last_unstale":"2023-11-28T02:53:28.973080+0000","last_undegraded":"2023-11-28T02:53:28.973080+0000","last_fullsized":"2023-11-28T02:53:28.973080+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992377+0000","last_change":"2023-11-28T02:53:28.987144+0000","last_active":"2023-11-28T02:53:28.992377+0000","last_peered":"2023-11-28T02:53:28.992377+0000","last_clean":"2023-11-28T02:53:28.992377+0000","last_became_active":"2023-11-28T02:53:28.985075+0000","last_became_peered":"2023-11-28T02:53:28.985075+0000","last_unstale":"2023-11-28T02:53:28.992377+0000","last_undegraded":"2023-11-28T02:53:28.992377+0000","last_fullsized":"2023-11-28T02:53:28.992377+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975794+0000","last_change":"2023-11-28T02:53:28.606475+0000","last_active":"2023-11-28T02:53:28.975794+0000","last_peered":"2023-11-28T02:53:28.975794+0000","last_clean":"2023-11-28T02:53:28.975794+0000","last_became_active":"2023-11-28T02:53:28.606177+0000","last_became_peered":"2023-11-28T02:53:28.606177+0000","last_unstale":"2023-11-28T02:53:28.975794+0000","last_undegraded":"2023-11-28T02:53:28.975794+0000","last_fullsized":"2023-11-28T02:53:28.975794+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.978259+0000","last_change":"2023-11-28T02:53:28.977105+0000","last_active":"2023-11-28T02:53:28.978259+0000","last_peered":"2023-11-28T02:53:28.978259+0000","last_clean":"2023-11-28T02:53:28.978259+0000","last_became_active":"2023-11-28T02:53:28.976061+0000","last_became_peered":"2023-11-28T02:53:28.976061+0000","last_unstale":"2023-11-28T02:53:28.978259+0000","last_undegraded":"2023-11-28T02:53:28.978259+0000","last_fullsized":"2023-11-28T02:53:28.978259+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972986+0000","last_change":"2023-11-28T02:53:28.627570+0000","last_active":"2023-11-28T02:53:28.972986+0000","last_peered":"2023-11-28T02:53:28.972986+0000","last_clean":"2023-11-28T02:53:28.972986+0000","last_became_active":"2023-11-28T02:53:28.627207+0000","last_became_peered":"2023-11-28T02:53:28.627207+0000","last_unstale":"2023-11-28T02:53:28.972986+0000","last_undegraded":"2023-11-28T02:53:28.972986+0000","last_fullsized":"2023-11-28T02:53:28.972986+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995547+0000","last_change":"2023-11-28T02:53:28.988822+0000","last_active":"2023-11-28T02:53:28.995547+0000","last_peered":"2023-11-28T02:53:28.995547+0000","last_clean":"2023-11-28T02:53:28.995547+0000","last_became_active":"2023-11-28T02:53:28.988328+0000","last_became_peered":"2023-11-28T02:53:28.988328+0000","last_unstale":"2023-11-28T02:53:28.995547+0000","last_undegraded":"2023-11-28T02:53:28.995547+0000","last_fullsized":"2023-11-28T02:53:28.995547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994154+0000","last_change":"2023-11-28T02:53:28.610805+0000","last_active":"2023-11-28T02:53:28.994154+0000","last_peered":"2023-11-28T02:53:28.994154+0000","last_clean":"2023-11-28T02:53:28.994154+0000","last_became_active":"2023-11-28T02:53:28.610370+0000","last_became_peered":"2023-11-28T02:53:28.610370+0000","last_unstale":"2023-11-28T02:53:28.994154+0000","last_undegraded":"2023-11-28T02:53:28.994154+0000","last_fullsized":"2023-11-28T02:53:28.994154+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997253+0000","last_change":"2023-11-28T02:53:28.994943+0000","last_active":"2023-11-28T02:53:28.997253+0000","last_peered":"2023-11-28T02:53:28.997253+0000","last_clean":"2023-11-28T02:53:28.997253+0000","last_became_active":"2023-11-28T02:53:28.994619+0000","last_became_peered":"2023-11-28T02:53:28.994619+0000","last_unstale":"2023-11-28T02:53:28.997253+0000","last_undegraded":"2023-11-28T02:53:28.997253+0000","last_fullsized":"2023-11-28T02:53:28.997253+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993626+0000","last_change":"2023-11-28T02:53:28.613771+0000","last_active":"2023-11-28T02:53:28.993626+0000","last_peered":"2023-11-28T02:53:28.993626+0000","last_clean":"2023-11-28T02:53:28.993626+0000","last_became_active":"2023-11-28T02:53:28.613520+0000","last_became_peered":"2023-11-28T02:53:28.613520+0000","last_unstale":"2023-11-28T02:53:28.993626+0000","last_undegraded":"2023-11-28T02:53:28.993626+0000","last_fullsized":"2023-11-28T02:53:28.993626+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.988754+0000","last_change":"2023-11-28T02:53:28.980028+0000","last_active":"2023-11-28T02:53:28.988754+0000","last_peered":"2023-11-28T02:53:28.988754+0000","last_clean":"2023-11-28T02:53:28.988754+0000","last_became_active":"2023-11-28T02:53:28.979301+0000","last_became_peered":"2023-11-28T02:53:28.979301+0000","last_unstale":"2023-11-28T02:53:28.988754+0000","last_undegraded":"2023-11-28T02:53:28.988754+0000","last_fullsized":"2023-11-28T02:53:28.988754+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987042+0000","last_change":"2023-11-28T02:53:28.624374+0000","last_active":"2023-11-28T02:53:28.987042+0000","last_peered":"2023-11-28T02:53:28.987042+0000","last_clean":"2023-11-28T02:53:28.987042+0000","last_became_active":"2023-11-28T02:53:28.624177+0000","last_became_peered":"2023-11-28T02:53:28.624177+0000","last_unstale":"2023-11-28T02:53:28.987042+0000","last_undegraded":"2023-11-28T02:53:28.987042+0000","last_fullsized":"2023-11-28T02:53:28.987042+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989750+0000","last_change":"2023-11-28T02:53:28.986339+0000","last_active":"2023-11-28T02:53:28.989750+0000","last_peered":"2023-11-28T02:53:28.989750+0000","last_clean":"2023-11-28T02:53:28.989750+0000","last_became_active":"2023-11-28T02:53:28.986068+0000","last_became_peered":"2023-11-28T02:53:28.986068+0000","last_unstale":"2023-11-28T02:53:28.989750+0000","last_undegraded":"2023-11-28T02:53:28.989750+0000","last_fullsized":"2023-11-28T02:53:28.989750+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968753+0000","last_change":"2023-11-28T02:53:28.609904+0000","last_active":"2023-11-28T02:53:28.968753+0000","last_peered":"2023-11-28T02:53:28.968753+0000","last_clean":"2023-11-28T02:53:28.968753+0000","last_became_active":"2023-11-28T02:53:28.609574+0000","last_became_peered":"2023-11-28T02:53:28.609574+0000","last_unstale":"2023-11-28T02:53:28.968753+0000","last_undegraded":"2023-11-28T02:53:28.968753+0000","last_fullsized":"2023-11-28T02:53:28.968753+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984706+0000","last_change":"2023-11-28T02:53:28.979178+0000","last_active":"2023-11-28T02:53:28.984706+0000","last_peered":"2023-11-28T02:53:28.984706+0000","last_clean":"2023-11-28T02:53:28.984706+0000","last_became_active":"2023-11-28T02:53:28.978772+0000","last_became_peered":"2023-11-28T02:53:28.978772+0000","last_unstale":"2023-11-28T02:53:28.984706+0000","last_undegraded":"2023-11-28T02:53:28.984706+0000","last_fullsized":"2023-11-28T02:53:28.984706+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977227+0000","last_change":"2023-11-28T02:53:28.613015+0000","last_active":"2023-11-28T02:53:28.977227+0000","last_peered":"2023-11-28T02:53:28.977227+0000","last_clean":"2023-11-28T02:53:28.977227+0000","last_became_active":"2023-11-28T02:53:28.612713+0000","last_became_peered":"2023-11-28T02:53:28.612713+0000","last_unstale":"2023-11-28T02:53:28.977227+0000","last_undegraded":"2023-11-28T02:53:28.977227+0000","last_fullsized":"2023-11-28T02:53:28.977227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993535+0000","last_change":"2023-11-28T02:53:28.987025+0000","last_active":"2023-11-28T02:53:28.993535+0000","last_peered":"2023-11-28T02:53:28.993535+0000","last_clean":"2023-11-28T02:53:28.993535+0000","last_became_active":"2023-11-28T02:53:28.985857+0000","last_became_peered":"2023-11-28T02:53:28.985857+0000","last_unstale":"2023-11-28T02:53:28.993535+0000","last_undegraded":"2023-11-28T02:53:28.993535+0000","last_fullsized":"2023-11-28T02:53:28.993535+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.972966+0000","last_change":"2023-11-28T02:53:28.625797+0000","last_active":"2023-11-28T02:53:28.972966+0000","last_peered":"2023-11-28T02:53:28.972966+0000","last_clean":"2023-11-28T02:53:28.972966+0000","last_became_active":"2023-11-28T02:53:28.625489+0000","last_became_peered":"2023-11-28T02:53:28.625489+0000","last_unstale":"2023-11-28T02:53:28.972966+0000","last_undegraded":"2023-11-28T02:53:28.972966+0000","last_fullsized":"2023-11-28T02:53:28.972966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.990553+0000","last_change":"2023-11-28T02:53:28.988300+0000","last_active":"2023-11-28T02:53:28.990553+0000","last_peered":"2023-11-28T02:53:28.990553+0000","last_clean":"2023-11-28T02:53:28.990553+0000","last_became_active":"2023-11-28T02:53:28.988014+0000","last_became_peered":"2023-11-28T02:53:28.988014+0000","last_unstale":"2023-11-28T02:53:28.990553+0000","last_undegraded":"2023-11-28T02:53:28.990553+0000","last_fullsized":"2023-11-28T02:53:28.990553+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968813+0000","last_change":"2023-11-28T02:53:28.618562+0000","last_active":"2023-11-28T02:53:28.968813+0000","last_peered":"2023-11-28T02:53:28.968813+0000","last_clean":"2023-11-28T02:53:28.968813+0000","last_became_active":"2023-11-28T02:53:28.618218+0000","last_became_peered":"2023-11-28T02:53:28.618218+0000","last_unstale":"2023-11-28T02:53:28.968813+0000","last_undegraded":"2023-11-28T02:53:28.968813+0000","last_fullsized":"2023-11-28T02:53:28.968813+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994696+0000","last_change":"2023-11-28T02:53:28.992384+0000","last_active":"2023-11-28T02:53:28.994696+0000","last_peered":"2023-11-28T02:53:28.994696+0000","last_clean":"2023-11-28T02:53:28.994696+0000","last_became_active":"2023-11-28T02:53:28.992058+0000","last_became_peered":"2023-11-28T02:53:28.992058+0000","last_unstale":"2023-11-28T02:53:28.994696+0000","last_undegraded":"2023-11-28T02:53:28.994696+0000","last_fullsized":"2023-11-28T02:53:28.994696+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968089+0000","last_change":"2023-11-28T02:53:28.614987+0000","last_active":"2023-11-28T02:53:28.968089+0000","last_peered":"2023-11-28T02:53:28.968089+0000","last_clean":"2023-11-28T02:53:28.968089+0000","last_became_active":"2023-11-28T02:53:28.614665+0000","last_became_peered":"2023-11-28T02:53:28.614665+0000","last_unstale":"2023-11-28T02:53:28.968089+0000","last_undegraded":"2023-11-28T02:53:28.968089+0000","last_fullsized":"2023-11-28T02:53:28.968089+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994348+0000","last_change":"2023-11-28T02:53:28.987703+0000","last_active":"2023-11-28T02:53:28.994348+0000","last_peered":"2023-11-28T02:53:28.994348+0000","last_clean":"2023-11-28T02:53:28.994348+0000","last_became_active":"2023-11-28T02:53:28.987426+0000","last_became_peered":"2023-11-28T02:53:28.987426+0000","last_unstale":"2023-11-28T02:53:28.994348+0000","last_undegraded":"2023-11-28T02:53:28.994348+0000","last_fullsized":"2023-11-28T02:53:28.994348+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974546+0000","last_change":"2023-11-28T02:53:28.611862+0000","last_active":"2023-11-28T02:53:28.974546+0000","last_peered":"2023-11-28T02:53:28.974546+0000","last_clean":"2023-11-28T02:53:28.974546+0000","last_became_active":"2023-11-28T02:53:28.611598+0000","last_became_peered":"2023-11-28T02:53:28.611598+0000","last_unstale":"2023-11-28T02:53:28.974546+0000","last_undegraded":"2023-11-28T02:53:28.974546+0000","last_fullsized":"2023-11-28T02:53:28.974546+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989285+0000","last_change":"2023-11-28T02:53:28.987367+0000","last_active":"2023-11-28T02:53:28.989285+0000","last_peered":"2023-11-28T02:53:28.989285+0000","last_clean":"2023-11-28T02:53:28.989285+0000","last_became_active":"2023-11-28T02:53:28.986325+0000","last_became_peered":"2023-11-28T02:53:28.986325+0000","last_unstale":"2023-11-28T02:53:28.989285+0000","last_undegraded":"2023-11-28T02:53:28.989285+0000","last_fullsized":"2023-11-28T02:53:28.989285+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.979734+0000","last_change":"2023-11-28T02:53:28.612981+0000","last_active":"2023-11-28T02:53:28.979734+0000","last_peered":"2023-11-28T02:53:28.979734+0000","last_clean":"2023-11-28T02:53:28.979734+0000","last_became_active":"2023-11-28T02:53:28.612549+0000","last_became_peered":"2023-11-28T02:53:28.612549+0000","last_unstale":"2023-11-28T02:53:28.979734+0000","last_undegraded":"2023-11-28T02:53:28.979734+0000","last_fullsized":"2023-11-28T02:53:28.979734+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.979772+0000","last_change":"2023-11-28T02:53:28.978113+0000","last_active":"2023-11-28T02:53:28.979772+0000","last_peered":"2023-11-28T02:53:28.979772+0000","last_clean":"2023-11-28T02:53:28.979772+0000","last_became_active":"2023-11-28T02:53:28.977855+0000","last_became_peered":"2023-11-28T02:53:28.977855+0000","last_unstale":"2023-11-28T02:53:28.979772+0000","last_undegraded":"2023-11-28T02:53:28.979772+0000","last_fullsized":"2023-11-28T02:53:28.979772+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994921+0000","last_change":"2023-11-28T02:53:28.614226+0000","last_active":"2023-11-28T02:53:28.994921+0000","last_peered":"2023-11-28T02:53:28.994921+0000","last_clean":"2023-11-28T02:53:28.994921+0000","last_became_active":"2023-11-28T02:53:28.613474+0000","last_became_peered":"2023-11-28T02:53:28.613474+0000","last_unstale":"2023-11-28T02:53:28.994921+0000","last_undegraded":"2023-11-28T02:53:28.994921+0000","last_fullsized":"2023-11-28T02:53:28.994921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995239+0000","last_change":"2023-11-28T02:53:28.992998+0000","last_active":"2023-11-28T02:53:28.995239+0000","last_peered":"2023-11-28T02:53:28.995239+0000","last_clean":"2023-11-28T02:53:28.995239+0000","last_became_active":"2023-11-28T02:53:28.992705+0000","last_became_peered":"2023-11-28T02:53:28.992705+0000","last_unstale":"2023-11-28T02:53:28.995239+0000","last_undegraded":"2023-11-28T02:53:28.995239+0000","last_fullsized":"2023-11-28T02:53:28.995239+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992195+0000","last_change":"2023-11-28T02:53:28.617731+0000","last_active":"2023-11-28T02:53:28.992195+0000","last_peered":"2023-11-28T02:53:28.992195+0000","last_clean":"2023-11-28T02:53:28.992195+0000","last_became_active":"2023-11-28T02:53:28.617429+0000","last_became_peered":"2023-11-28T02:53:28.617429+0000","last_unstale":"2023-11-28T02:53:28.992195+0000","last_undegraded":"2023-11-28T02:53:28.992195+0000","last_fullsized":"2023-11-28T02:53:28.992195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997723+0000","last_change":"2023-11-28T02:53:28.995450+0000","last_active":"2023-11-28T02:53:28.997723+0000","last_peered":"2023-11-28T02:53:28.997723+0000","last_clean":"2023-11-28T02:53:28.997723+0000","last_became_active":"2023-11-28T02:53:28.995190+0000","last_became_peered":"2023-11-28T02:53:28.995190+0000","last_unstale":"2023-11-28T02:53:28.997723+0000","last_undegraded":"2023-11-28T02:53:28.997723+0000","last_fullsized":"2023-11-28T02:53:28.997723+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985316+0000","last_change":"2023-11-28T02:53:28.610107+0000","last_active":"2023-11-28T02:53:28.985316+0000","last_peered":"2023-11-28T02:53:28.985316+0000","last_clean":"2023-11-28T02:53:28.985316+0000","last_became_active":"2023-11-28T02:53:28.609440+0000","last_became_peered":"2023-11-28T02:53:28.609440+0000","last_unstale":"2023-11-28T02:53:28.985316+0000","last_undegraded":"2023-11-28T02:53:28.985316+0000","last_fullsized":"2023-11-28T02:53:28.985316+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985337+0000","last_change":"2023-11-28T02:53:28.980510+0000","last_active":"2023-11-28T02:53:28.985337+0000","last_peered":"2023-11-28T02:53:28.985337+0000","last_clean":"2023-11-28T02:53:28.985337+0000","last_became_active":"2023-11-28T02:53:28.980225+0000","last_became_peered":"2023-11-28T02:53:28.980225+0000","last_unstale":"2023-11-28T02:53:28.985337+0000","last_undegraded":"2023-11-28T02:53:28.985337+0000","last_fullsized":"2023-11-28T02:53:28.985337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968728+0000","last_change":"2023-11-28T02:53:28.616013+0000","last_active":"2023-11-28T02:53:28.968728+0000","last_peered":"2023-11-28T02:53:28.968728+0000","last_clean":"2023-11-28T02:53:28.968728+0000","last_became_active":"2023-11-28T02:53:28.615730+0000","last_became_peered":"2023-11-28T02:53:28.615730+0000","last_unstale":"2023-11-28T02:53:28.968728+0000","last_undegraded":"2023-11-28T02:53:28.968728+0000","last_fullsized":"2023-11-28T02:53:28.968728+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989391+0000","last_change":"2023-11-28T02:53:28.983758+0000","last_active":"2023-11-28T02:53:28.989391+0000","last_peered":"2023-11-28T02:53:28.989391+0000","last_clean":"2023-11-28T02:53:28.989391+0000","last_became_active":"2023-11-28T02:53:28.983409+0000","last_became_peered":"2023-11-28T02:53:28.983409+0000","last_unstale":"2023-11-28T02:53:28.989391+0000","last_undegraded":"2023-11-28T02:53:28.989391+0000","last_fullsized":"2023-11-28T02:53:28.989391+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975684+0000","last_change":"2023-11-28T02:53:28.613797+0000","last_active":"2023-11-28T02:53:28.975684+0000","last_peered":"2023-11-28T02:53:28.975684+0000","last_clean":"2023-11-28T02:53:28.975684+0000","last_became_active":"2023-11-28T02:53:28.613504+0000","last_became_peered":"2023-11-28T02:53:28.613504+0000","last_unstale":"2023-11-28T02:53:28.975684+0000","last_undegraded":"2023-11-28T02:53:28.975684+0000","last_fullsized":"2023-11-28T02:53:28.975684+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986230+0000","last_change":"2023-11-28T02:53:28.980548+0000","last_active":"2023-11-28T02:53:28.986230+0000","last_peered":"2023-11-28T02:53:28.986230+0000","last_clean":"2023-11-28T02:53:28.986230+0000","last_became_active":"2023-11-28T02:53:28.980262+0000","last_became_peered":"2023-11-28T02:53:28.980262+0000","last_unstale":"2023-11-28T02:53:28.986230+0000","last_undegraded":"2023-11-28T02:53:28.986230+0000","last_fullsized":"2023-11-28T02:53:28.986230+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995281+0000","last_change":"2023-11-28T02:53:28.627331+0000","last_active":"2023-11-28T02:53:28.995281+0000","last_peered":"2023-11-28T02:53:28.995281+0000","last_clean":"2023-11-28T02:53:28.995281+0000","last_became_active":"2023-11-28T02:53:28.627083+0000","last_became_peered":"2023-11-28T02:53:28.627083+0000","last_unstale":"2023-11-28T02:53:28.995281+0000","last_undegraded":"2023-11-28T02:53:28.995281+0000","last_fullsized":"2023-11-28T02:53:28.995281+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989720+0000","last_change":"2023-11-28T02:53:28.985192+0000","last_active":"2023-11-28T02:53:28.989720+0000","last_peered":"2023-11-28T02:53:28.989720+0000","last_clean":"2023-11-28T02:53:28.989720+0000","last_became_active":"2023-11-28T02:53:28.985003+0000","last_became_peered":"2023-11-28T02:53:28.985003+0000","last_unstale":"2023-11-28T02:53:28.989720+0000","last_undegraded":"2023-11-28T02:53:28.989720+0000","last_fullsized":"2023-11-28T02:53:28.989720+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994276+0000","last_change":"2023-11-28T02:53:28.623609+0000","last_active":"2023-11-28T02:53:28.994276+0000","last_peered":"2023-11-28T02:53:28.994276+0000","last_clean":"2023-11-28T02:53:28.994276+0000","last_became_active":"2023-11-28T02:53:28.623324+0000","last_became_peered":"2023-11-28T02:53:28.623324+0000","last_unstale":"2023-11-28T02:53:28.994276+0000","last_undegraded":"2023-11-28T02:53:28.994276+0000","last_fullsized":"2023-11-28T02:53:28.994276+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996976+0000","last_change":"2023-11-28T02:53:28.994540+0000","last_active":"2023-11-28T02:53:28.996976+0000","last_peered":"2023-11-28T02:53:28.996976+0000","last_clean":"2023-11-28T02:53:28.996976+0000","last_became_active":"2023-11-28T02:53:28.994234+0000","last_became_peered":"2023-11-28T02:53:28.994234+0000","last_unstale":"2023-11-28T02:53:28.996976+0000","last_undegraded":"2023-11-28T02:53:28.996976+0000","last_fullsized":"2023-11-28T02:53:28.996976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991199+0000","last_change":"2023-11-28T02:53:28.623468+0000","last_active":"2023-11-28T02:53:28.991199+0000","last_peered":"2023-11-28T02:53:28.991199+0000","last_clean":"2023-11-28T02:53:28.991199+0000","last_became_active":"2023-11-28T02:53:28.623187+0000","last_became_peered":"2023-11-28T02:53:28.623187+0000","last_unstale":"2023-11-28T02:53:28.991199+0000","last_undegraded":"2023-11-28T02:53:28.991199+0000","last_fullsized":"2023-11-28T02:53:28.991199+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.988775+0000","last_change":"2023-11-28T02:53:28.980195+0000","last_active":"2023-11-28T02:53:28.988775+0000","last_peered":"2023-11-28T02:53:28.988775+0000","last_clean":"2023-11-28T02:53:28.988775+0000","last_became_active":"2023-11-28T02:53:28.977563+0000","last_became_peered":"2023-11-28T02:53:28.977563+0000","last_unstale":"2023-11-28T02:53:28.988775+0000","last_undegraded":"2023-11-28T02:53:28.988775+0000","last_fullsized":"2023-11-28T02:53:28.988775+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983410+0000","last_change":"2023-11-28T02:53:28.621174+0000","last_active":"2023-11-28T02:53:28.983410+0000","last_peered":"2023-11-28T02:53:28.983410+0000","last_clean":"2023-11-28T02:53:28.983410+0000","last_became_active":"2023-11-28T02:53:28.620891+0000","last_became_peered":"2023-11-28T02:53:28.620891+0000","last_unstale":"2023-11-28T02:53:28.983410+0000","last_undegraded":"2023-11-28T02:53:28.983410+0000","last_fullsized":"2023-11-28T02:53:28.983410+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994187+0000","last_change":"2023-11-28T02:53:28.985257+0000","last_active":"2023-11-28T02:53:28.994187+0000","last_peered":"2023-11-28T02:53:28.994187+0000","last_clean":"2023-11-28T02:53:28.994187+0000","last_became_active":"2023-11-28T02:53:28.984948+0000","last_became_peered":"2023-11-28T02:53:28.984948+0000","last_unstale":"2023-11-28T02:53:28.994187+0000","last_undegraded":"2023-11-28T02:53:28.994187+0000","last_fullsized":"2023-11-28T02:53:28.994187+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968857+0000","last_change":"2023-11-28T02:53:28.624514+0000","last_active":"2023-11-28T02:53:28.968857+0000","last_peered":"2023-11-28T02:53:28.968857+0000","last_clean":"2023-11-28T02:53:28.968857+0000","last_became_active":"2023-11-28T02:53:28.624248+0000","last_became_peered":"2023-11-28T02:53:28.624248+0000","last_unstale":"2023-11-28T02:53:28.968857+0000","last_undegraded":"2023-11-28T02:53:28.968857+0000","last_fullsized":"2023-11-28T02:53:28.968857+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.993712+0000","last_change":"2023-11-28T02:53:28.986667+0000","last_active":"2023-11-28T02:53:28.993712+0000","last_peered":"2023-11-28T02:53:28.993712+0000","last_clean":"2023-11-28T02:53:28.993712+0000","last_became_active":"2023-11-28T02:53:28.986374+0000","last_became_peered":"2023-11-28T02:53:28.986374+0000","last_unstale":"2023-11-28T02:53:28.993712+0000","last_undegraded":"2023-11-28T02:53:28.993712+0000","last_fullsized":"2023-11-28T02:53:28.993712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994268+0000","last_change":"2023-11-28T02:53:28.606978+0000","last_active":"2023-11-28T02:53:28.994268+0000","last_peered":"2023-11-28T02:53:28.994268+0000","last_clean":"2023-11-28T02:53:28.994268+0000","last_became_active":"2023-11-28T02:53:28.606572+0000","last_became_peered":"2023-11-28T02:53:28.606572+0000","last_unstale":"2023-11-28T02:53:28.994268+0000","last_undegraded":"2023-11-28T02:53:28.994268+0000","last_fullsized":"2023-11-28T02:53:28.994268+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991834+0000","last_change":"2023-11-28T02:53:28.988706+0000","last_active":"2023-11-28T02:53:28.991834+0000","last_peered":"2023-11-28T02:53:28.991834+0000","last_clean":"2023-11-28T02:53:28.991834+0000","last_became_active":"2023-11-28T02:53:28.988486+0000","last_became_peered":"2023-11-28T02:53:28.988486+0000","last_unstale":"2023-11-28T02:53:28.991834+0000","last_undegraded":"2023-11-28T02:53:28.991834+0000","last_fullsized":"2023-11-28T02:53:28.991834+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983319+0000","last_change":"2023-11-28T02:53:28.612430+0000","last_active":"2023-11-28T02:53:28.983319+0000","last_peered":"2023-11-28T02:53:28.983319+0000","last_clean":"2023-11-28T02:53:28.983319+0000","last_became_active":"2023-11-28T02:53:28.609736+0000","last_became_peered":"2023-11-28T02:53:28.609736+0000","last_unstale":"2023-11-28T02:53:28.983319+0000","last_undegraded":"2023-11-28T02:53:28.983319+0000","last_fullsized":"2023-11-28T02:53:28.983319+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.997090+0000","last_change":"2023-11-28T02:53:28.994751+0000","last_active":"2023-11-28T02:53:28.997090+0000","last_peered":"2023-11-28T02:53:28.997090+0000","last_clean":"2023-11-28T02:53:28.997090+0000","last_became_active":"2023-11-28T02:53:28.994461+0000","last_became_peered":"2023-11-28T02:53:28.994461+0000","last_unstale":"2023-11-28T02:53:28.997090+0000","last_undegraded":"2023-11-28T02:53:28.997090+0000","last_fullsized":"2023-11-28T02:53:28.997090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974138+0000","last_change":"2023-11-28T02:53:28.621440+0000","last_active":"2023-11-28T02:53:28.974138+0000","last_peered":"2023-11-28T02:53:28.974138+0000","last_clean":"2023-11-28T02:53:28.974138+0000","last_became_active":"2023-11-28T02:53:28.621181+0000","last_became_peered":"2023-11-28T02:53:28.621181+0000","last_unstale":"2023-11-28T02:53:28.974138+0000","last_undegraded":"2023-11-28T02:53:28.974138+0000","last_fullsized":"2023-11-28T02:53:28.974138+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.992308+0000","last_change":"2023-11-28T02:53:28.989879+0000","last_active":"2023-11-28T02:53:28.992308+0000","last_peered":"2023-11-28T02:53:28.992308+0000","last_clean":"2023-11-28T02:53:28.992308+0000","last_became_active":"2023-11-28T02:53:28.989420+0000","last_became_peered":"2023-11-28T02:53:28.989420+0000","last_unstale":"2023-11-28T02:53:28.992308+0000","last_undegraded":"2023-11-28T02:53:28.992308+0000","last_fullsized":"2023-11-28T02:53:28.992308+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.983358+0000","last_change":"2023-11-28T02:53:28.607190+0000","last_active":"2023-11-28T02:53:28.983358+0000","last_peered":"2023-11-28T02:53:28.983358+0000","last_clean":"2023-11-28T02:53:28.983358+0000","last_became_active":"2023-11-28T02:53:28.606693+0000","last_became_peered":"2023-11-28T02:53:28.606693+0000","last_unstale":"2023-11-28T02:53:28.983358+0000","last_undegraded":"2023-11-28T02:53:28.983358+0000","last_fullsized":"2023-11-28T02:53:28.983358+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984686+0000","last_change":"2023-11-28T02:53:28.979154+0000","last_active":"2023-11-28T02:53:28.984686+0000","last_peered":"2023-11-28T02:53:28.984686+0000","last_clean":"2023-11-28T02:53:28.984686+0000","last_became_active":"2023-11-28T02:53:28.978734+0000","last_became_peered":"2023-11-28T02:53:28.978734+0000","last_unstale":"2023-11-28T02:53:28.984686+0000","last_undegraded":"2023-11-28T02:53:28.984686+0000","last_fullsized":"2023-11-28T02:53:28.984686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977591+0000","last_change":"2023-11-28T02:53:28.609677+0000","last_active":"2023-11-28T02:53:28.977591+0000","last_peered":"2023-11-28T02:53:28.977591+0000","last_clean":"2023-11-28T02:53:28.977591+0000","last_became_active":"2023-11-28T02:53:28.609414+0000","last_became_peered":"2023-11-28T02:53:28.609414+0000","last_unstale":"2023-11-28T02:53:28.977591+0000","last_undegraded":"2023-11-28T02:53:28.977591+0000","last_fullsized":"2023-11-28T02:53:28.977591+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968934+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T02:53:28.968934+0000","last_peered":"2023-11-28T02:53:28.968934+0000","last_clean":"2023-11-28T02:53:28.968934+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T02:53:28.968934+0000","last_undegraded":"2023-11-28T02:53:28.968934+0000","last_fullsized":"2023-11-28T02:53:28.968934+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.987865+0000","last_change":"2023-11-28T02:53:28.982245+0000","last_active":"2023-11-28T02:53:28.987865+0000","last_peered":"2023-11-28T02:53:28.987865+0000","last_clean":"2023-11-28T02:53:28.987865+0000","last_became_active":"2023-11-28T02:53:28.981794+0000","last_became_peered":"2023-11-28T02:53:28.981794+0000","last_unstale":"2023-11-28T02:53:28.987865+0000","last_undegraded":"2023-11-28T02:53:28.987865+0000","last_fullsized":"2023-11-28T02:53:28.987865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977302+0000","last_change":"2023-11-28T02:53:28.613224+0000","last_active":"2023-11-28T02:53:28.977302+0000","last_peered":"2023-11-28T02:53:28.977302+0000","last_clean":"2023-11-28T02:53:28.977302+0000","last_became_active":"2023-11-28T02:53:28.612943+0000","last_became_peered":"2023-11-28T02:53:28.612943+0000","last_unstale":"2023-11-28T02:53:28.977302+0000","last_undegraded":"2023-11-28T02:53:28.977302+0000","last_fullsized":"2023-11-28T02:53:28.977302+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.978785+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T02:53:28.978785+0000","last_peered":"2023-11-28T02:53:28.978785+0000","last_clean":"2023-11-28T02:53:28.978785+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T02:53:28.978785+0000","last_undegraded":"2023-11-28T02:53:28.978785+0000","last_fullsized":"2023-11-28T02:53:28.978785+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.989265+0000","last_change":"2023-11-28T02:53:28.987145+0000","last_active":"2023-11-28T02:53:28.989265+0000","last_peered":"2023-11-28T02:53:28.989265+0000","last_clean":"2023-11-28T02:53:28.989265+0000","last_became_active":"2023-11-28T02:53:28.986548+0000","last_became_peered":"2023-11-28T02:53:28.986548+0000","last_unstale":"2023-11-28T02:53:28.989265+0000","last_undegraded":"2023-11-28T02:53:28.989265+0000","last_fullsized":"2023-11-28T02:53:28.989265+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.984195+0000","last_change":"2023-11-28T02:53:28.615680+0000","last_active":"2023-11-28T02:53:28.984195+0000","last_peered":"2023-11-28T02:53:28.984195+0000","last_clean":"2023-11-28T02:53:28.984195+0000","last_became_active":"2023-11-28T02:53:28.615386+0000","last_became_peered":"2023-11-28T02:53:28.615386+0000","last_unstale":"2023-11-28T02:53:28.984195+0000","last_undegraded":"2023-11-28T02:53:28.984195+0000","last_fullsized":"2023-11-28T02:53:28.984195+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.985160+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T02:53:28.985160+0000","last_peered":"2023-11-28T02:53:28.985160+0000","last_clean":"2023-11-28T02:53:28.985160+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T02:53:28.985160+0000","last_undegraded":"2023-11-28T02:53:28.985160+0000","last_fullsized":"2023-11-28T02:53:28.985160+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.996472+0000","last_change":"2023-11-28T02:53:28.993278+0000","last_active":"2023-11-28T02:53:28.996472+0000","last_peered":"2023-11-28T02:53:28.996472+0000","last_clean":"2023-11-28T02:53:28.996472+0000","last_became_active":"2023-11-28T02:53:28.990902+0000","last_became_peered":"2023-11-28T02:53:28.990902+0000","last_unstale":"2023-11-28T02:53:28.996472+0000","last_undegraded":"2023-11-28T02:53:28.996472+0000","last_fullsized":"2023-11-28T02:53:28.996472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973405+0000","last_change":"2023-11-28T02:53:28.621358+0000","last_active":"2023-11-28T02:53:28.973405+0000","last_peered":"2023-11-28T02:53:28.973405+0000","last_clean":"2023-11-28T02:53:28.973405+0000","last_became_active":"2023-11-28T02:53:28.621059+0000","last_became_peered":"2023-11-28T02:53:28.621059+0000","last_unstale":"2023-11-28T02:53:28.973405+0000","last_undegraded":"2023-11-28T02:53:28.973405+0000","last_fullsized":"2023-11-28T02:53:28.973405+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.977109+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T02:53:28.977109+0000","last_peered":"2023-11-28T02:53:28.977109+0000","last_clean":"2023-11-28T02:53:28.977109+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T02:53:28.977109+0000","last_undegraded":"2023-11-28T02:53:28.977109+0000","last_fullsized":"2023-11-28T02:53:28.977109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":24,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973086+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T02:53:28.973086+0000","last_peered":"2023-11-28T02:53:28.973086+0000","last_clean":"2023-11-28T02:53:28.973086+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T02:53:28.973086+0000","last_undegraded":"2023-11-28T02:53:28.973086+0000","last_fullsized":"2023-11-28T02:53:28.973086+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.994320+0000","last_change":"2023-11-28T02:53:28.988844+0000","last_active":"2023-11-28T02:53:28.994320+0000","last_peered":"2023-11-28T02:53:28.994320+0000","last_clean":"2023-11-28T02:53:28.994320+0000","last_became_active":"2023-11-28T02:53:28.988625+0000","last_became_peered":"2023-11-28T02:53:28.988625+0000","last_unstale":"2023-11-28T02:53:28.994320+0000","last_undegraded":"2023-11-28T02:53:28.994320+0000","last_fullsized":"2023-11-28T02:53:28.994320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.995566+0000","last_change":"2023-11-28T02:53:28.613163+0000","last_active":"2023-11-28T02:53:28.995566+0000","last_peered":"2023-11-28T02:53:28.995566+0000","last_clean":"2023-11-28T02:53:28.995566+0000","last_became_active":"2023-11-28T02:53:28.612845+0000","last_became_peered":"2023-11-28T02:53:28.612845+0000","last_unstale":"2023-11-28T02:53:28.995566+0000","last_undegraded":"2023-11-28T02:53:28.995566+0000","last_fullsized":"2023-11-28T02:53:28.995566+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.973376+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T02:53:28.973376+0000","last_peered":"2023-11-28T02:53:28.973376+0000","last_clean":"2023-11-28T02:53:28.973376+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T02:53:28.973376+0000","last_undegraded":"2023-11-28T02:53:28.973376+0000","last_fullsized":"2023-11-28T02:53:28.973376+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991570+0000","last_change":"2023-11-28T02:53:28.985510+0000","last_active":"2023-11-28T02:53:28.991570+0000","last_peered":"2023-11-28T02:53:28.991570+0000","last_clean":"2023-11-28T02:53:28.991570+0000","last_became_active":"2023-11-28T02:53:28.985063+0000","last_became_peered":"2023-11-28T02:53:28.985063+0000","last_unstale":"2023-11-28T02:53:28.991570+0000","last_undegraded":"2023-11-28T02:53:28.991570+0000","last_fullsized":"2023-11-28T02:53:28.991570+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975072+0000","last_change":"2023-11-28T02:53:28.608083+0000","last_active":"2023-11-28T02:53:28.975072+0000","last_peered":"2023-11-28T02:53:28.975072+0000","last_clean":"2023-11-28T02:53:28.975072+0000","last_became_active":"2023-11-28T02:53:28.607764+0000","last_became_peered":"2023-11-28T02:53:28.607764+0000","last_unstale":"2023-11-28T02:53:28.975072+0000","last_undegraded":"2023-11-28T02:53:28.975072+0000","last_fullsized":"2023-11-28T02:53:28.975072+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.974637+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T02:53:28.974637+0000","last_peered":"2023-11-28T02:53:28.974637+0000","last_clean":"2023-11-28T02:53:28.974637+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T02:53:28.974637+0000","last_undegraded":"2023-11-28T02:53:28.974637+0000","last_fullsized":"2023-11-28T02:53:28.974637+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.986257+0000","last_change":"2023-11-28T02:53:28.978737+0000","last_active":"2023-11-28T02:53:28.986257+0000","last_peered":"2023-11-28T02:53:28.986257+0000","last_clean":"2023-11-28T02:53:28.986257+0000","last_became_active":"2023-11-28T02:53:28.978006+0000","last_became_peered":"2023-11-28T02:53:28.978006+0000","last_unstale":"2023-11-28T02:53:28.986257+0000","last_undegraded":"2023-11-28T02:53:28.986257+0000","last_fullsized":"2023-11-28T02:53:28.986257+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968251+0000","last_change":"2023-11-28T02:53:28.610884+0000","last_active":"2023-11-28T02:53:28.968251+0000","last_peered":"2023-11-28T02:53:28.968251+0000","last_clean":"2023-11-28T02:53:28.968251+0000","last_became_active":"2023-11-28T02:53:28.609394+0000","last_became_peered":"2023-11-28T02:53:28.609394+0000","last_unstale":"2023-11-28T02:53:28.968251+0000","last_undegraded":"2023-11-28T02:53:28.968251+0000","last_fullsized":"2023-11-28T02:53:28.968251+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.968936+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T02:53:28.968936+0000","last_peered":"2023-11-28T02:53:28.968936+0000","last_clean":"2023-11-28T02:53:28.968936+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T02:53:28.968936+0000","last_undegraded":"2023-11-28T02:53:28.968936+0000","last_fullsized":"2023-11-28T02:53:28.968936+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.991609+0000","last_change":"2023-11-28T02:53:28.987538+0000","last_active":"2023-11-28T02:53:28.991609+0000","last_peered":"2023-11-28T02:53:28.991609+0000","last_clean":"2023-11-28T02:53:28.991609+0000","last_became_active":"2023-11-28T02:53:28.986197+0000","last_became_peered":"2023-11-28T02:53:28.986197+0000","last_unstale":"2023-11-28T02:53:28.991609+0000","last_undegraded":"2023-11-28T02:53:28.991609+0000","last_fullsized":"2023-11-28T02:53:28.991609+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:28.590869+0000","last_clean_scrub_stamp":"2023-11-28T02:53:28.590869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975923+0000","last_change":"2023-11-28T02:53:28.611177+0000","last_active":"2023-11-28T02:53:28.975923+0000","last_peered":"2023-11-28T02:53:28.975923+0000","last_clean":"2023-11-28T02:53:28.975923+0000","last_became_active":"2023-11-28T02:53:28.610893+0000","last_became_peered":"2023-11-28T02:53:28.610893+0000","last_unstale":"2023-11-28T02:53:28.975923+0000","last_undegraded":"2023-11-28T02:53:28.975923+0000","last_fullsized":"2023-11-28T02:53:28.975923+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:53:27.586664+0000","last_clean_scrub_stamp":"2023-11-28T02:53:27.586664+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":25,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-11-28T02:53:28.975174+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T02:53:28.975174+0000","last_peered":"2023-11-28T02:53:28.975174+0000","last_clean":"2023-11-28T02:53:28.975174+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T02:53:28.975174+0000","last_undegraded":"2023-11-28T02:53:28.975174+0000","last_fullsized":"2023-11-28T02:53:28.975174+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":393216,"data_stored":159348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":159258,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.442}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53700000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297480,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074360,"statfs":{"total":96636764160,"available":96332144640,"internally_reserved":0,"allocated":393216,"data_stored":158864,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25900000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64900000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55900000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":158738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28899999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48899999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297472,"kb_used_data":376,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074368,"statfs":{"total":96636764160,"available":96332152832,"internally_reserved":0,"allocated":385024,"data_stored":158704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297492,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074348,"statfs":{"total":96636764160,"available":96332132352,"internally_reserved":0,"allocated":405504,"data_stored":160344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29399999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57299999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574861,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297476,"kb_used_data":380,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074364,"statfs":{"total":96636764160,"available":96332148736,"internally_reserved":0,"allocated":389120,"data_stored":158726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61699999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.433}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574863,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297488,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074352,"statfs":{"total":96636764160,"available":96332136448,"internally_reserved":0,"allocated":401408,"data_stored":160262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57999999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1640,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T02:53:35.775 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T02:53:35.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T02:53:35.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.928+0000 7feec3312700 1 Processor -- start 2023-11-28T02:53:35.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.928+0000 7feec3312700 1 -- start start 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec3312700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec3312700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 0x7feebc063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec3312700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feebc0c72d0 con 0x7feebc064350 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec3312700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feebc154bc0 con 0x7feebc098d10 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec3312700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c66c0 con 0x7feebc0c6190 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec08ad700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 0x7feebc063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec10ae700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec18af700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7feebc064350 0x7feebc158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59046/0 (socket says 172.21.15.73:59046) 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec10ae700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:51636/0 (socket says 172.21.15.73:51636) 2023-11-28T02:53:35.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.929+0000 7feec18af700 1 -- 172.21.15.73:0/879605260 learned_addr learned my addr 172.21.15.73:0/879605260 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec08ad700 1 -- 172.21.15.73:0/879605260 >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 msgr2=0x7feebc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec08ad700 1 --2- 172.21.15.73:0/879605260 >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec08ad700 1 -- 172.21.15.73:0/879605260 >> v1:172.21.15.150:6789/0 conn(0x7feebc064350 legacy=0x7feebc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feebbfff700 1 -- 172.21.15.73:0/879605260 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536045248 0 0) 0x7feebc0c72d0 con 0x7feebc064350 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec08ad700 1 -- 172.21.15.73:0/879605260 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feebc0c69a0 con 0x7feebc0c6190 2023-11-28T02:53:35.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec10ae700 1 --2- 172.21.15.73:0/879605260 >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec08ad700 1 --2- 172.21.15.73:0/879605260 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 0x7feebc063e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7feeb0009760 tx=0x7feeb0009060).ready entity=mon.2 client_cookie=241501aeadef7040 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feebbfff700 1 -- 172.21.15.73:0/879605260 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000f510 con 0x7feebc0c6190 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feebbfff700 1 -- 172.21.15.73:0/879605260 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb000f670 con 0x7feebc0c6190 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feebbfff700 1 -- 172.21.15.73:0/879605260 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000f9f0 con 0x7feebc0c6190 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 -- 172.21.15.73:0/879605260 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 msgr2=0x7feebc063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 --2- 172.21.15.73:0/879605260 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 0x7feebc063e60 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7feeb0009760 tx=0x7feeb0009060).stop 2023-11-28T02:53:35.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 -- 172.21.15.73:0/879605260 shutdown_connections 2023-11-28T02:53:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 --2- 172.21.15.73:0/879605260 >> v2:172.21.15.150:3300/0 conn(0x7feebc098d10 0x7feebc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 --2- 172.21.15.73:0/879605260 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc0c6190 0x7feebc063e60 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.930+0000 7feec3312700 1 -- 172.21.15.73:0/879605260 >> 172.21.15.73:0/879605260 conn(0x7feebc0cb310 msgr2=0x7feebc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.931+0000 7feec3312700 1 -- 172.21.15.73:0/879605260 shutdown_connections 2023-11-28T02:53:35.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.931+0000 7feec3312700 1 -- 172.21.15.73:0/879605260 wait complete. 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.932+0000 7feec3312700 1 Processor -- start 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.932+0000 7feec3312700 1 -- start start 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.932+0000 7feec3312700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc064350 0x7feebc14f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.932+0000 7feec3312700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 0x7feebc0c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0af490 con 0x7feebc0c6190 2023-11-28T02:53:35.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c30b0 con 0x7feebc098d10 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feebc0c3390 con 0x7feebc064350 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38854/0 (socket says 172.21.15.73:38854) 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 -- 172.21.15.73:0/2857640931 learned_addr learned my addr 172.21.15.73:0/2857640931 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec08ad700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 0x7feebc0c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec10ae700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc064350 0x7feebc14f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 -- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc064350 msgr2=0x7feebc14f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc064350 0x7feebc14f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 -- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 msgr2=0x7feebc0c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 0x7feebc0c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 -- 172.21.15.73:0/2857640931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeac0092a0 con 0x7feebc0c6190 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec08ad700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 0x7feebc0c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec18af700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7feeb400c7e0 tx=0x7feeb4007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4005d30 con 0x7feebc0c6190 2023-11-28T02:53:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb4005e90 con 0x7feebc0c6190 2023-11-28T02:53:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb4007320 con 0x7feebc0c6190 2023-11-28T02:53:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feeb00092a0 con 0x7feebc0c6190 2023-11-28T02:53:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.933+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feebc0c3950 con 0x7feebc0c6190 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.934+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feebc0af260 con 0x7feebc0c6190 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.935+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feeb4007710 con 0x7feebc0c6190 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.935+0000 7feeb97fa700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 0x7feea8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.935+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feeb407a0f0 con 0x7feebc0c6190 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.935+0000 7feec10ae700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 0x7feea8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:35.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.935+0000 7feec10ae700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 0x7feea8061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7feeac009dc0 tx=0x7feeac009060).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:35.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:35.939+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feebc0af260 con 0x7feebc0c6190 2023-11-28T02:53:36.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.167+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7feebc0af260 con 0x7feebc0c6190 2023-11-28T02:53:36.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.171+0000 7feeb97fa700 1 -- 172.21.15.73:0/2857640931 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7feebc0af260 con 0x7feebc0c6190 2023-11-28T02:53:36.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 msgr2=0x7feea8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 0x7feea8061610 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7feeac009dc0 tx=0x7feeac009060).stop 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 msgr2=0x7feebc0c2b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 rx=0x7feeb400c7e0 tx=0x7feeb4007f90).stop 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 shutdown_connections 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feea805f150 0x7feea8061610 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feebc0c6190 0x7feebc0c2b20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feebc064350 0x7feebc14f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 --2- 172.21.15.73:0/2857640931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feebc098d10 0x7feebc0c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 >> 172.21.15.73:0/2857640931 conn(0x7feebc0cb310 msgr2=0x7feebc0c0270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:36.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 shutdown_connections 2023-11-28T02:53:36.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.173+0000 7feec3312700 1 -- 172.21.15.73:0/2857640931 wait complete. 2023-11-28T02:53:36.177 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T02:53:36.186 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T02:53:36.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.299+0000 7f7652cf1700 1 Processor -- start 2023-11-28T02:53:36.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.299+0000 7f7652cf1700 1 -- start start 2023-11-28T02:53:36.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f7652cf1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f764c0cae90 0x7f764c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f7652cf1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c0a9a90 con 0x7f764c0ca180 2023-11-28T02:53:36.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f7652cf1700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764c154ce0 con 0x7f764c0caab0 2023-11-28T02:53:36.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f7652cf1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764c0aab80 con 0x7f764c0cae90 2023-11-28T02:53:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f765128e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f764c0ca180 0x7f764c15e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33762/0 (socket says 172.21.15.73:33762) 2023-11-28T02:53:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f765128e700 1 -- 172.21.15.73:0/277239970 learned_addr learned my addr 172.21.15.73:0/277239970 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764bfff700 1 --2- 172.21.15.73:0/277239970 >> v2:172.21.15.150:3300/0 conn(0x7f764c0cae90 0x7f764c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767129351 0 0) 0x7f764c0a9a90 con 0x7f764c0ca180 2023-11-28T02:53:36.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76380033f0 con 0x7f764c0ca180 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 66389450 0 0) 0x7f76380033f0 con 0x7f764c0ca180 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 >> v2:172.21.15.150:3300/0 conn(0x7f764c0cae90 msgr2=0x7f764c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 --2- 172.21.15.73:0/277239970 >> v2:172.21.15.150:3300/0 conn(0x7f764c0cae90 0x7f764c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 >> v1:172.21.15.150:6789/0 conn(0x7f764c0caab0 legacy=0x7f764c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c0a8e80 con 0x7f764c0ca180 2023-11-28T02:53:36.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7634001470 con 0x7f764c0ca180 2023-11-28T02:53:36.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 >> v1:172.21.15.73:6789/0 conn(0x7f764c0ca180 legacy=0x7f764c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76380038e0 0x7f7638003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f7638004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7638005000 0x7f763800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0aab80 con 0x7f7638004690 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763800beb0 con 0x7f76380038e0 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.300+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763800c070 con 0x7f7638005000 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f7638004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f7650a8d700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76380038e0 0x7f7638003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f765128e700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7638005000 0x7f763800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 -- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7638005000 msgr2=0x7f763800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7638005000 0x7f763800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 -- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76380038e0 msgr2=0x7f7638003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76380038e0 0x7f7638003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 -- 172.21.15.73:0/277239970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76380033f0 con 0x7f7638004690 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f765128e700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7638005000 0x7f763800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:36.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f7650a8d700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76380038e0 0x7f7638003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f765128e700 1 -- 172.21.15.73:0/277239970 reap_dead start 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764bfff700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f7638004b10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f763c00e210 tx=0x7f763c00b040).ready entity=mon.0 client_cookie=ecc61b37d1354b32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c009ac0 con 0x7f7638004690 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00f040 con 0x7f7638004690 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.301+0000 7f764b7fe700 1 -- 172.21.15.73:0/277239970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00ecd0 con 0x7f7638004690 2023-11-28T02:53:36.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 -- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 msgr2=0x7f7638004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f7638004b10 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7f763c00e210 tx=0x7f763c00b040).stop 2023-11-28T02:53:36.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 -- 172.21.15.73:0/277239970 shutdown_connections 2023-11-28T02:53:36.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 --2- 172.21.15.73:0/277239970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f7638004b10 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 -- 172.21.15.73:0/277239970 >> 172.21.15.73:0/277239970 conn(0x7f764c094920 msgr2=0x7f764c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:36.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 -- 172.21.15.73:0/277239970 shutdown_connections 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.302+0000 7f7652cf1700 1 -- 172.21.15.73:0/277239970 wait complete. 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.303+0000 7f7652cf1700 1 Processor -- start 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.303+0000 7f7652cf1700 1 -- start start 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764c0aae00 0x7f764c164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f764c164b50 0x7f764c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c66b0 con 0x7f7638004690 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c9140 con 0x7f764c164b50 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f764c0c6380 con 0x7f764c0aae00 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38872/0 (socket says 172.21.15.73:38872) 2023-11-28T02:53:36.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 -- 172.21.15.73:0/694838554 learned_addr learned my addr 172.21.15.73:0/694838554 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f765128e700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f764c164b50 0x7f764c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f764bfff700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764c0aae00 0x7f764c164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 -- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764c0aae00 msgr2=0x7f764c164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764c0aae00 0x7f764c164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 -- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f764c164b50 msgr2=0x7f764c0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f764c164b50 0x7f764c0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 -- 172.21.15.73:0/694838554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764c164fd0 con 0x7f7638004690 2023-11-28T02:53:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7650a8d700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7644007950 tx=0x7f7644009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400d610 con 0x7f7638004690 2023-11-28T02:53:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f764400d770 con 0x7f7638004690 2023-11-28T02:53:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f764400daf0 con 0x7f7638004690 2023-11-28T02:53:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.304+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764c1652b0 con 0x7f7638004690 2023-11-28T02:53:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.305+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764c0c7030 con 0x7f7638004690 2023-11-28T02:53:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.305+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764c0c6990 con 0x7f7638004690 2023-11-28T02:53:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.306+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f76440133f0 con 0x7f7638004690 2023-11-28T02:53:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.306+0000 7f7648ff9700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 0x7f762c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:36.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.306+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7644079ad0 con 0x7f7638004690 2023-11-28T02:53:36.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.306+0000 7f764bfff700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 0x7f762c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:36.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.306+0000 7f764bfff700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 0x7f762c061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f763c00afd0 tx=0x7f763c01a000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:36.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.310+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7644049af0 con 0x7f7638004690 2023-11-28T02:53:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.517+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f764c0c7310 con 0x7f7638004690 2023-11-28T02:53:36.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.518+0000 7f7648ff9700 1 -- 172.21.15.73:0/694838554 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f764c0c7310 con 0x7f7638004690 2023-11-28T02:53:36.519 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T02:53:36.520 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T02:53:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 msgr2=0x7f762c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 0x7f762c061610 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 rx=0x7f763c00afd0 tx=0x7f763c01a000).stop 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 msgr2=0x7f764c0c8c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 rx=0x7f7644007950 tx=0x7f7644009040).stop 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 shutdown_connections 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f762c05f150 0x7f762c061610 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f764c164b50 0x7f764c0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7638004690 0x7f764c0c8c50 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 --2- 172.21.15.73:0/694838554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f764c0aae00 0x7f764c164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 >> 172.21.15.73:0/694838554 conn(0x7f764c094920 msgr2=0x7f764c0ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 shutdown_connections 2023-11-28T02:53:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:36.520+0000 7f7652cf1700 1 -- 172.21.15.73:0/694838554 wait complete. 2023-11-28T02:53:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T02:53:36.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 12, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T02:53:28.963671+0000', 'modified': '2023-11-28T02:53:34.452717+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4974}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4974': {'gid': 4974, 'name': 'c', 'rank': 0, 'incarnation': 10, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/1459377503', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 1459377503}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 1459377503}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T02:53:36.936 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T02:53:36.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T02:53:37.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.052+0000 7f139270c700 1 Processor -- start 2023-11-28T02:53:37.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.052+0000 7f139270c700 1 -- start start 2023-11-28T02:53:37.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f138c098d10 0x7f138c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 0x7f138c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c064350 0x7f138c158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c0c72d0 con 0x7f138c098d10 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f138c154bc0 con 0x7f138c0c6190 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f139270c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c66c0 con 0x7f138c064350 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f138c098d10 0x7f138c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f138c098d10 0x7f138c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38882/0 (socket says 172.21.15.73:38882) 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 -- 172.21.15.73:0/1374316304 learned_addr learned my addr 172.21.15.73:0/1374316304 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 -- 172.21.15.73:0/1374316304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c064350 msgr2=0x7f138c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138b7fe700 1 --2- 172.21.15.73:0/1374316304 >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 0x7f138c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f1390ca9700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c064350 0x7f138c158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c064350 0x7f138c158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 -- 172.21.15.73:0/1374316304 >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 msgr2=0x7f138c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 --2- 172.21.15.73:0/1374316304 >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 0x7f138c063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 -- 172.21.15.73:0/1374316304 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f138c0c69a0 con 0x7f138c098d10 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138b7fe700 1 --2- 172.21.15.73:0/1374316304 >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 0x7f138c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.053+0000 7f138bfff700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c098d10 0x7f138c0c7760 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f138c0c5b40 tx=0x7f137c0090f0).ready entity=mon.0 client_cookie=dcd7a2872e83c4b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f138affd700 1 -- 172.21.15.73:0/1374316304 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00b600 con 0x7f138c098d10 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f138affd700 1 -- 172.21.15.73:0/1374316304 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f137c00b760 con 0x7f138c098d10 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f138affd700 1 -- 172.21.15.73:0/1374316304 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c00bae0 con 0x7f138c098d10 2023-11-28T02:53:37.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 -- 172.21.15.73:0/1374316304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c098d10 msgr2=0x7f138c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c098d10 0x7f138c0c7760 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 rx=0x7f138c0c5b40 tx=0x7f137c0090f0).stop 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 -- 172.21.15.73:0/1374316304 shutdown_connections 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c098d10 0x7f138c0c7760 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 --2- 172.21.15.73:0/1374316304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c064350 0x7f138c158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 --2- 172.21.15.73:0/1374316304 >> v2:172.21.15.150:3300/0 conn(0x7f138c0c6190 0x7f138c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 -- 172.21.15.73:0/1374316304 >> 172.21.15.73:0/1374316304 conn(0x7f138c0cb310 msgr2=0x7f138c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 -- 172.21.15.73:0/1374316304 shutdown_connections 2023-11-28T02:53:37.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.054+0000 7f139270c700 1 -- 172.21.15.73:0/1374316304 wait complete. 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.055+0000 7f139270c700 1 Processor -- start 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.055+0000 7f139270c700 1 -- start start 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 0x7f138c14b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 0x7f138c0ca370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0aab70 con 0x7f138c0c6190 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0cad50 con 0x7f138c064350 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f139270c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f138c0c27f0 con 0x7f138c098d10 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f1390ca9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f1390ca9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38896/0 (socket says 172.21.15.73:38896) 2023-11-28T02:53:37.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f1390ca9700 1 -- 172.21.15.73:0/2513837043 learned_addr learned my addr 172.21.15.73:0/2513837043 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f138b7fe700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 0x7f138c0ca370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.056+0000 7f138bfff700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 0x7f138c14b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 -- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 msgr2=0x7f138c0ca370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 0x7f138c0ca370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 -- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 msgr2=0x7f138c14b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 0x7f138c14b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 -- 172.21.15.73:0/2513837043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f137c009370 con 0x7f138c0c6190 2023-11-28T02:53:37.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f138bfff700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 0x7f138c14b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f138b7fe700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 0x7f138c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1390ca9700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f138400ceb0 tx=0x7f1384007870).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1373fff700 1 -- 172.21.15.73:0/2513837043 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f138400ae40 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1373fff700 1 -- 172.21.15.73:0/2513837043 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1384016400 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f138c165b50 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f1373fff700 1 -- 172.21.15.73:0/2513837043 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384016780 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.057+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138c0aa2d0 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.058+0000 7f1373fff700 1 -- 172.21.15.73:0/2513837043 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1384009030 con 0x7f138c0c6190 2023-11-28T02:53:37.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.058+0000 7f1373fff700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 0x7f1378061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.059+0000 7f1373fff700 1 -- 172.21.15.73:0/2513837043 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f138407aaf0 con 0x7f138c0c6190 2023-11-28T02:53:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.059+0000 7f138bfff700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 0x7f1378061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.059+0000 7f138bfff700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 0x7f1378061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f137c009fd0 tx=0x7f137c017000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.061+0000 7f13a40a4b80 1 Processor -- start 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.061+0000 7f13a40a4b80 1 -- start start 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cffdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0cf9b060 con 0x562b0d01eea0 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0cf121f0 con 0x562b0cfd5cb0 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b0cf27440 con 0x562b0cd91e40 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38900/0 (socket says 172.21.15.73:38900) 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 -- 172.21.15.73:0/2532011978 learned_addr learned my addr 172.21.15.73:0/2532011978 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f1390ca9700 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138bfff700 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cffdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 -- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 msgr2=0x562b0cffdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cffdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 -- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cfd5cb0 msgr2=0x562b0cfd0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 -- 172.21.15.73:0/2532011978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b0cf20620 con 0x562b0d01eea0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f138b7fe700 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f138000b8d0 tx=0x7f1380008040).ready entity=mon.0 client_cookie=ac76078bbffd4eea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13717fa700 1 -- 172.21.15.73:0/2532011978 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13800115f0 con 0x562b0d01eea0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13717fa700 1 -- 172.21.15.73:0/2532011978 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000c040 con 0x562b0d01eea0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.062+0000 7f13717fa700 1 -- 172.21.15.73:0/2532011978 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011960 con 0x562b0d01eea0 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 -- 172.21.15.73:0/2532011978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 msgr2=0x562b0cfd57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 rx=0x7f138000b8d0 tx=0x7f1380008040).stop 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 -- 172.21.15.73:0/2532011978 shutdown_connections 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cffdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/2532011978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0cfd57c0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 -- 172.21.15.73:0/2532011978 >> 172.21.15.73:0/2532011978 conn(0x562b0ce3f4a0 msgr2=0x562b0d01e790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 -- 172.21.15.73:0/2532011978 shutdown_connections 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.063+0000 7f13a40a4b80 1 -- 172.21.15.73:0/2532011978 wait complete. 2023-11-28T02:53:37.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 -- start start 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 Processor -- start 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cd91e40 0x562b0cfeefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d015990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0cf27580 con 0x562b0d01eea0 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0cf27800 con 0x562b0cd91e40 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b0cef2e10 con 0x562b0cfd5cb0 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.064+0000 7f13a40a4b80 10 client.0 ll_register_callbacks cb 0x562b0cf95460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38904/0 (socket says 172.21.15.73:38904) 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 -- 172.21.15.73:0/1846618774 learned_addr learned my addr 172.21.15.73:0/1846618774 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f138bfff700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cd91e40 0x562b0cfeefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f138b7fe700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d015990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 -- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 msgr2=0x562b0d015990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d015990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 -- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cd91e40 msgr2=0x562b0cfeefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cd91e40 0x562b0cfeefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 -- 172.21.15.73:0/1846618774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f138000b500 con 0x562b0d01eea0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f1390ca9700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f138404f780 tx=0x7f138400f000).ready entity=mon.0 client_cookie=4d21b4010adbe101 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1384054030 con 0x562b0d01eea0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562b0cfc9720 con 0x562b0d01eea0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f138404c2e0 con 0x562b0d01eea0 2023-11-28T02:53:37.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1384016b30 con 0x562b0d01eea0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 10 client.5157 fetch_fsmap learned FSMap version 12 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x562b0cf1b9c0 con 0x562b0d01eea0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562b0cfc9720 con 0x562b0d01eea0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 12) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1384016c90 con 0x562b0d01eea0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 10 client.5157 fetch_fsmap finished waiting for FSMap version 12 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 10 client.5157 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 20 client.5157 populate_metadata read hostname 'smithi073' 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d017ed0 0x562b0d0182b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 4 client.5157 mds_command: new command op to 4974 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.065+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 --> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562b0cd8dca0 con 0x562b0d017ed0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f138bfff700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d017ed0 0x562b0d0182b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f136a7fc700 10 client.5157 ms_handle_connect on v2:172.21.15.73:6835/1459377503 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f138bfff700 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d017ed0 0x562b0d0182b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b7a94a60b801ec9f server_cookie=a9999b2da0d1decf in_seq=0 out_seq=0 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f136a7fc700 1 -- 172.21.15.73:0/1846618774 <== mds.0 v2:172.21.15.73:6835/1459377503 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x562b0cd8dca0 con 0x562b0d017ed0 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f136a7fc700 10 client.5157 handle_command_reply: tid=0 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f13a40a4b80 1 client.5157 shutdown 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 msgr2=0x562b0d015e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.066+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 rx=0x7f138404f780 tx=0x7f138400f000).stop 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 shutdown_connections 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0d01eea0 0x562b0d015e80 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d017ed0 0x562b0d0182b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0cd91e40 0x562b0cfeefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/1846618774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d015990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f136a7fc700 0 client.5157 ms_handle_reset on v2:172.21.15.73:6835/1459377503 2023-11-28T02:53:37.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 >> 172.21.15.73:0/1846618774 conn(0x562b0ce3f4a0 msgr2=0x562b0cfd6aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 shutdown_connections 2023-11-28T02:53:37.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 1 -- 172.21.15.73:0/1846618774 wait complete. 2023-11-28T02:53:37.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.067+0000 7f13a40a4b80 20 client.5157 trim_cache size 0 max 16384 2023-11-28T02:53:37.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.083+0000 7f13a40a4b80 1 Processor -- start 2023-11-28T02:53:37.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.083+0000 7f13a40a4b80 1 -- start start 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cfd36f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0cfd4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0cfc9180 con 0x562b0cfd5cb0 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0d0045a0 con 0x562b0d01eea0 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b0d014f90 con 0x562b0cd91e40 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38920/0 (socket says 172.21.15.73:38920) 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 -- 172.21.15.73:0/3028875349 learned_addr learned my addr 172.21.15.73:0/3028875349 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f1390ca9700 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0cfd4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138bfff700 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cfd36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 -- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 msgr2=0x562b0cfd36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cfd36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 -- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 msgr2=0x562b0cfd4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0cfd4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 -- 172.21.15.73:0/3028875349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562b0cfc9340 con 0x562b0cfd5cb0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.084+0000 7f138b7fe700 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f138000b770 tx=0x7f1380008040).ready entity=mon.0 client_cookie=a53f3c322017b325 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f1371ffb700 1 -- 172.21.15.73:0/3028875349 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380000d00 con 0x562b0cfd5cb0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f1371ffb700 1 -- 172.21.15.73:0/3028875349 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1380011d10 con 0x562b0cfd5cb0 2023-11-28T02:53:37.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f1371ffb700 1 -- 172.21.15.73:0/3028875349 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1380011800 con 0x562b0cfd5cb0 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 -- 172.21.15.73:0/3028875349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 msgr2=0x562b0cfd0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f138000b770 tx=0x7f1380008040).stop 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 -- 172.21.15.73:0/3028875349 shutdown_connections 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0cfd4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cd91e40 0x562b0cfd36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/3028875349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cfd5cb0 0x562b0cfd0930 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 -- 172.21.15.73:0/3028875349 >> 172.21.15.73:0/3028875349 conn(0x562b0ce3f4a0 msgr2=0x562b0d0158b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 -- 172.21.15.73:0/3028875349 shutdown_connections 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.085+0000 7f13a40a4b80 1 -- 172.21.15.73:0/3028875349 wait complete. 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.086+0000 7f13a40a4b80 1 -- start start 2023-11-28T02:53:37.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.086+0000 7f13a40a4b80 1 Processor -- start 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.086+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.086+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d039a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0d039f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0d002ea0 con 0x562b0cd91e40 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x562b0d07fc80 con 0x562b0d01eea0 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x562b0d03a960 con 0x562b0cfd5cb0 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 10 client.0 ll_register_callbacks cb 0x562b0cffb8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38922/0 (socket says 172.21.15.73:38922) 2023-11-28T02:53:37.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 -- 172.21.15.73:0/908770511 learned_addr learned my addr 172.21.15.73:0/908770511 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f1390ca9700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0d039f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138b7fe700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d039a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 -- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 msgr2=0x562b0d039a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d039a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 -- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 msgr2=0x562b0d039f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0d039f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 -- 172.21.15.73:0/908770511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562b0cf20620 con 0x562b0cd91e40 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138b7fe700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d039a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f138bfff700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f138c0c2360 tx=0x7f137c028040).ready entity=mon.0 client_cookie=ea73ae97f7cc0290 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f137c02c040 con 0x562b0cd91e40 2023-11-28T02:53:37.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f137c031660 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f138000c040 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=12) v2 ==== 24+0+0 (secure 0 0 0) 0x7f137c030280 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 10 client.5163 fetch_fsmap learned FSMap version 12 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=12,monmap=2+}) v3 -- 0x562b0cf1b710 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f138000c040 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.087+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 12) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f137c031a30 con 0x562b0cd91e40 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 10 client.5163 fetch_fsmap finished waiting for FSMap version 12 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 10 client.5163 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 20 client.5163 populate_metadata read hostname 'smithi073' 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d0335f0 0x562b0d0339d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 4 client.5163 mds_command: new command op to 4974 tid=0[{"prefix": "status"}] 2023-11-28T02:53:37.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 --> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562b0cdc4ca0 con 0x562b0d0335f0 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f138b7fe700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d0335f0 0x562b0d0339d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f136a7fc700 10 client.5163 ms_handle_connect on v2:172.21.15.73:6835/1459377503 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f138b7fe700 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d0335f0 0x562b0d0339d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d684de7600421ca4 server_cookie=b802e1f7aa7a03a1 in_seq=0 out_seq=0 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f136a7fc700 1 -- 172.21.15.73:0/908770511 <== mds.0 v2:172.21.15.73:6835/1459377503 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562b0cdc4ca0 con 0x562b0d0335f0 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f136a7fc700 10 client.5163 handle_command_reply: tid=0 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 client.5163 shutdown 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 msgr2=0x562b0d07f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 rx=0x7f138c0c2360 tx=0x7f137c028040).stop 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 shutdown_connections 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:6835/1459377503,v1:172.21.15.73:6837/1459377503] conn(0x562b0d0335f0 0x562b0d0339d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x562b0d01eea0 0x562b0d039f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x562b0cd91e40 0x562b0d07f790 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f136a7fc700 0 client.5163 ms_handle_reset on v2:172.21.15.73:6835/1459377503 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 --2- 172.21.15.73:0/908770511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x562b0cfd5cb0 0x562b0d039a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 >> 172.21.15.73:0/908770511 conn(0x562b0ce3f4a0 msgr2=0x562b0cfd36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 shutdown_connections 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 1 -- 172.21.15.73:0/908770511 wait complete. 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.088+0000 7f13a40a4b80 20 client.5163 trim_cache size 0 max 16384 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 4974, 2023-11-28T02:53:37.097 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 8.1243770489999996, 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 12, 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 34, 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 34, 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 15.336291128999999 2023-11-28T02:53:37.098 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 msgr2=0x7f1378061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 0x7f1378061610 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0x7f137c009fd0 tx=0x7f137c017000).stop 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 msgr2=0x7f138c0ca860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 rx=0x7f138400ceb0 tx=0x7f1384007870).stop 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 shutdown_connections 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f137805f150 0x7f1378061610 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f138c0c6190 0x7f138c0ca860 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f138c064350 0x7f138c14b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 --2- 172.21.15.73:0/2513837043 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f138c098d10 0x7f138c0ca370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 >> 172.21.15.73:0/2513837043 conn(0x7f138c0cb310 msgr2=0x7f138c14c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 shutdown_connections 2023-11-28T02:53:37.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.089+0000 7f139270c700 1 -- 172.21.15.73:0/2513837043 wait complete. 2023-11-28T02:53:37.103 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T02:53:37.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.203+0000 7f64e702c700 1 Processor -- start 2023-11-28T02:53:37.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.203+0000 7f64e702c700 1 -- start start 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64e702c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 0x7f64e00c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64e702c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64e00c7700 0x7f64e00aa1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64e702c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64e00c6d80 con 0x7f64e00aa690 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64e702c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e0154ae0 con 0x7f64e00c7700 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64e702c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64e0064c30 con 0x7f64e0154700 2023-11-28T02:53:37.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64d7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64e00c7700 0x7f64e00aa1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64d7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f64e00c7700 0x7f64e00aa1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38930/0 (socket says 172.21.15.73:38930) 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.204+0000 7f64d7fff700 1 -- 172.21.15.73:0/3557863706 learned_addr learned my addr 172.21.15.73:0/3557863706 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e4dc8700 1 --2- 172.21.15.73:0/3557863706 >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 0x7f64e00c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d7fff700 1 -- 172.21.15.73:0/3557863706 >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 msgr2=0x7f64e00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d7fff700 1 --2- 172.21.15.73:0/3557863706 >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 0x7f64e00c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d7fff700 1 -- 172.21.15.73:0/3557863706 >> v1:172.21.15.150:6789/0 conn(0x7f64e00aa690 legacy=0x7f64e00c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d7fff700 1 -- 172.21.15.73:0/3557863706 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e00c6450 con 0x7f64e00c7700 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e4dc8700 1 --2- 172.21.15.73:0/3557863706 >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 0x7f64e00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d7fff700 1 --2- 172.21.15.73:0/3557863706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e00c7700 0x7f64e00aa1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f64e00cc2b0 tx=0x7f64d00090f0).ready entity=mon.0 client_cookie=c32a1812f480072c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d77fe700 1 -- 172.21.15.73:0/3557863706 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000b5a0 con 0x7f64e00c7700 2023-11-28T02:53:37.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d77fe700 1 -- 172.21.15.73:0/3557863706 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d000b700 con 0x7f64e00c7700 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64d77fe700 1 -- 172.21.15.73:0/3557863706 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d000ba80 con 0x7f64e00c7700 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 -- 172.21.15.73:0/3557863706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e00c7700 msgr2=0x7f64e00aa1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 --2- 172.21.15.73:0/3557863706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e00c7700 0x7f64e00aa1a0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 rx=0x7f64e00cc2b0 tx=0x7f64d00090f0).stop 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 -- 172.21.15.73:0/3557863706 shutdown_connections 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 --2- 172.21.15.73:0/3557863706 >> v2:172.21.15.150:3300/0 conn(0x7f64e0154700 0x7f64e00c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 --2- 172.21.15.73:0/3557863706 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e00c7700 0x7f64e00aa1a0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 -- 172.21.15.73:0/3557863706 >> 172.21.15.73:0/3557863706 conn(0x7f64e00948b0 msgr2=0x7f64e006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 -- 172.21.15.73:0/3557863706 shutdown_connections 2023-11-28T02:53:37.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.205+0000 7f64e702c700 1 -- 172.21.15.73:0/3557863706 wait complete. 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 Processor -- start 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 -- start start 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e00aa690 0x7f64e00ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64e00c7700 0x7f64e00aca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00adbe0 con 0x7f64e0154700 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64e01555c0 con 0x7f64e00aa690 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.207+0000 7f64e702c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64e00ad340 con 0x7f64e00c7700 2023-11-28T02:53:37.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38942/0 (socket says 172.21.15.73:38942) 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 -- 172.21.15.73:0/873961126 learned_addr learned my addr 172.21.15.73:0/873961126 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e4dc8700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e00aa690 0x7f64e00ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64d7fff700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64e00c7700 0x7f64e00aca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 -- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64e00c7700 msgr2=0x7f64e00aca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64e00c7700 0x7f64e00aca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 -- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e00aa690 msgr2=0x7f64e00ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e00aa690 0x7f64e00ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 -- 172.21.15.73:0/873961126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0009370 con 0x7f64e0154700 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e55c9700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f64d800b810 tx=0x7f64d8009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800d500 con 0x7f64e0154700 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64d800d660 con 0x7f64e0154700 2023-11-28T02:53:37.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64e00ad620 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64d800da20 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.208+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64e014eab0 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.209+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64e0155390 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.209+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f64d8011460 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.210+0000 7f64d4ff9700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 0x7f64cc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.210+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f64d8079c20 con 0x7f64e0154700 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.210+0000 7f64e4dc8700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 0x7f64cc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:37.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.210+0000 7f64e4dc8700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 0x7f64cc061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f64dc00abd0 tx=0x7f64dc008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:37.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.214+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f64d8047af0 con 0x7f64e0154700 2023-11-28T02:53:37.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.423+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64e00475e0 con 0x7f64e0154700 2023-11-28T02:53:37.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.423+0000 7f64d4ff9700 1 -- 172.21.15.73:0/873961126 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v12) v1 ==== 53+0+73 (secure 0 0 0) 0x7f64e0155390 con 0x7f64e0154700 2023-11-28T02:53:37.425 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 msgr2=0x7f64cc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 0x7f64cc061610 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f64dc00abd0 tx=0x7f64dc008040).stop 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 msgr2=0x7f64e00ae1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f64d800b810 tx=0x7f64d8009290).stop 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 shutdown_connections 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64cc05f150 0x7f64cc061610 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64e0154700 0x7f64e00ae1f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64e00aa690 0x7f64e00ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 --2- 172.21.15.73:0/873961126 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64e00c7700 0x7f64e00aca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:37.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 >> 172.21.15.73:0/873961126 conn(0x7f64e00948b0 msgr2=0x7f64e00ca240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:37.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 shutdown_connections 2023-11-28T02:53:37.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:37.425+0000 7f64e702c700 1 -- 172.21.15.73:0/873961126 wait complete. 2023-11-28T02:53:37.437 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T02:53:37.437 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T02:53:37.437 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T02:53:37.437 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T02:53:37.438 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:37.438 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T02:53:37.438 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T02:53:37.438 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T02:53:37.438 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T02:53:37.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.439 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T02:53:37.454 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T02:53:37.454 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T02:53:37.454 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T02:53:37.454 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.455 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T02:53:37.455 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.455 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:37.455 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T02:53:37.456 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 42462sec preferred_lft 42462sec 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T02:53:37.457 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.458 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T02:53:37.459 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T02:53:37.459 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T02:53:37.460 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T02:53:37.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.460 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T02:53:37.518 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T02:53:37.519 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.519 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T02:53:37.575 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T02:53:37.576 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T02:53:37.576 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T02:53:37.576 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T02:53:37.576 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T02:53:37.576 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.576 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T02:53:37.576 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T02:53:37.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T02:53:37.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T02:53:37.577 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T02:53:37.577 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T02:53:37.675 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.675 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T02:53:37.690 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T02:53:37.690 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T02:53:37.690 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T02:53:37.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.691 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T02:53:37.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.755 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:37.809 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T02:53:37.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:37.887 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:37.888 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:37.913 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T02:53:37.913 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T02:53:37.916 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T02:53:37.916 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T02:53:37.917 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T02:53:37.981 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T02:53:38.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.134+0000 7fcf6350d700 1 Processor -- start 2023-11-28T02:53:38.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.134+0000 7fcf6350d700 1 -- start start 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf6350d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0a69f0 0x7fcf5c0bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf6350d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf5c062ef0 0x7fcf5c155de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf6350d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf5c0bd030 con 0x7fcf5c0bcbb0 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf6350d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf5c063510 con 0x7fcf5c062ef0 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf6350d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c0637f0 con 0x7fcf5c0a69f0 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf61aaa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf5c062ef0 0x7fcf5c155de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf61aaa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcf5c062ef0 0x7fcf5c155de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38954/0 (socket says 172.21.15.73:38954) 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.135+0000 7fcf61aaa700 1 -- 172.21.15.73:0/661838921 learned_addr learned my addr 172.21.15.73:0/661838921 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf61aaa700 1 -- 172.21.15.73:0/661838921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0a69f0 msgr2=0x7fcf5c0bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:38.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf61aaa700 1 --2- 172.21.15.73:0/661838921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0a69f0 0x7fcf5c0bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf61aaa700 1 -- 172.21.15.73:0/661838921 >> v1:172.21.15.150:6789/0 conn(0x7fcf5c0bcbb0 legacy=0x7fcf5c0627e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf61aaa700 1 -- 172.21.15.73:0/661838921 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf5c063ad0 con 0x7fcf5c062ef0 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf61aaa700 1 --2- 172.21.15.73:0/661838921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fcf54006fd0 tx=0x7fcf540090f0).ready entity=mon.0 client_cookie=dd4db0ceb9f4899 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf53fff700 1 -- 172.21.15.73:0/661838921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400b600 con 0x7fcf5c062ef0 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf53fff700 1 -- 172.21.15.73:0/661838921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5400b760 con 0x7fcf5c062ef0 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf53fff700 1 -- 172.21.15.73:0/661838921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5400bb00 con 0x7fcf5c062ef0 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 -- 172.21.15.73:0/661838921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 msgr2=0x7fcf5c155de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 --2- 172.21.15.73:0/661838921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155de0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 rx=0x7fcf54006fd0 tx=0x7fcf540090f0).stop 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 -- 172.21.15.73:0/661838921 shutdown_connections 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 --2- 172.21.15.73:0/661838921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155de0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 --2- 172.21.15.73:0/661838921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0a69f0 0x7fcf5c0bc6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:38.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.136+0000 7fcf6350d700 1 -- 172.21.15.73:0/661838921 >> 172.21.15.73:0/661838921 conn(0x7fcf5c0c8230 msgr2=0x7fcf5c0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:38.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.137+0000 7fcf6350d700 1 -- 172.21.15.73:0/661838921 shutdown_connections 2023-11-28T02:53:38.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.137+0000 7fcf6350d700 1 -- 172.21.15.73:0/661838921 wait complete. 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.138+0000 7fcf6350d700 1 Processor -- start 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.138+0000 7fcf6350d700 1 -- start start 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 0x7fcf5c16cd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 0x7fcf5c158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c142550 con 0x7fcf5c062ef0 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c141940 con 0x7fcf5c0a69f0 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf5c141c20 con 0x7fcf5c0bcbb0 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38958/0 (socket says 172.21.15.73:38958) 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 -- 172.21.15.73:0/43494439 learned_addr learned my addr 172.21.15.73:0/43494439 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:38.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf60aa8700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 0x7fcf5c16cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf61aaa700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 0x7fcf5c158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 -- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 msgr2=0x7fcf5c158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 0x7fcf5c158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 -- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 msgr2=0x7fcf5c16cd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 0x7fcf5c16cd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 -- 172.21.15.73:0/43494439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf54009370 con 0x7fcf5c062ef0 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf60aa8700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 0x7fcf5c16cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf61aaa700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 0x7fcf5c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:38.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf612a9700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fcf5800bc30 tx=0x7fcf58009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800c7a0 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf58010070 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf5c141f00 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800c960 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.139+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf5c156200 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.140+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf5c142830 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.141+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcf5800ccd0 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.141+0000 7fcf517fa700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 0x7fcf40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.141+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcf58079a80 con 0x7fcf5c062ef0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.141+0000 7fcf60aa8700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 0x7fcf40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.141+0000 7fcf60aa8700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 0x7fcf40061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcf5c065c90 tx=0x7fcf4c008040).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.145+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcf5804ad50 con 0x7fcf5c062ef0 2023-11-28T02:53:38.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.333+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf5c06c860 con 0x7fcf4005f150 2023-11-28T02:53:38.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.335+0000 7fcf517fa700 1 -- 172.21.15.73:0/43494439 <== mgr.4107 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcf5c06c860 con 0x7fcf4005f150 2023-11-28T02:53:38.336 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T02:53:38.336 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T02:53:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T02:53:38.337 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T02:53:38.337 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T02:53:38.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 msgr2=0x7fcf40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 0x7fcf40061610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7fcf5c065c90 tx=0x7fcf4c008040).stop 2023-11-28T02:53:38.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 msgr2=0x7fcf5c155d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 rx=0x7fcf5800bc30 tx=0x7fcf58009840).stop 2023-11-28T02:53:38.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 shutdown_connections 2023-11-28T02:53:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcf4005f150 0x7fcf40061610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcf5c0bcbb0 0x7fcf5c158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcf5c062ef0 0x7fcf5c155d10 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 --2- 172.21.15.73:0/43494439 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcf5c0a69f0 0x7fcf5c16cd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 >> 172.21.15.73:0/43494439 conn(0x7fcf5c0c8230 msgr2=0x7fcf5c13ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 shutdown_connections 2023-11-28T02:53:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.337+0000 7fcf6350d700 1 -- 172.21.15.73:0/43494439 wait complete. 2023-11-28T02:53:38.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T02:53:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.463+0000 7f81f7af7700 1 Processor -- start 2023-11-28T02:53:38.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.463+0000 7f81f7af7700 1 -- start start 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f7af7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81f00c6190 0x7f81f00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f7af7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00c72d0 con 0x7f81f0098d10 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f7af7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f0064570 con 0x7f81f0158320 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f7af7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f0154bc0 con 0x7f81f00c6190 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f5893700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f81f0098d10 0x7f81f0063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33778/0 (socket says 172.21.15.73:33778) 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f5893700 1 -- 172.21.15.73:0/2869517316 learned_addr learned my addr 172.21.15.73:0/2869517316 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2503641769 0 0) 0x7f81f00c72d0 con 0x7f81f0098d10 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f5092700 1 --2- 172.21.15.73:0/2869517316 >> v2:172.21.15.150:3300/0 conn(0x7f81f00c6190 0x7f81f00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81dc0033f0 con 0x7f81f0098d10 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3825997502 0 0) 0x7f81dc0033f0 con 0x7f81f0098d10 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 >> v2:172.21.15.150:3300/0 conn(0x7f81f00c6190 msgr2=0x7f81f00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 --2- 172.21.15.73:0/2869517316 >> v2:172.21.15.150:3300/0 conn(0x7f81f00c6190 0x7f81f00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 >> v1:172.21.15.150:6789/0 conn(0x7f81f0158320 legacy=0x7f81f00a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00c66c0 con 0x7f81f0098d10 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.464+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f81ec001470 con 0x7f81f0098d10 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 >> v1:172.21.15.73:6789/0 conn(0x7f81f0098d10 legacy=0x7f81f0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 0x7f81dc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81dc004690 0x7f81dc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc005000 0x7f81dc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0154bc0 con 0x7f81dc0038e0 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc00beb0 con 0x7f81dc005000 2023-11-28T02:53:38.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc00c070 con 0x7f81dc004690 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 0x7f81dc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f6094700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc005000 0x7f81dc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5092700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81dc004690 0x7f81dc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 -- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81dc004690 msgr2=0x7f81dc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81dc004690 0x7f81dc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 -- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc005000 msgr2=0x7f81dc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc005000 0x7f81dc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 -- 172.21.15.73:0/2869517316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc0033f0 con 0x7f81dc0038e0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f6094700 1 -- 172.21.15.73:0/2869517316 reap_dead start 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f5893700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 0x7f81dc003d00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f81ec00c190 tx=0x7f81ec001e50).ready entity=mon.0 client_cookie=6a78e5982470db65 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec007a20 con 0x7f81dc0038e0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81ec00cac0 con 0x7f81dc0038e0 2023-11-28T02:53:38.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.465+0000 7f81f4891700 1 -- 172.21.15.73:0/2869517316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81ec00ce40 con 0x7f81dc0038e0 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 -- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 msgr2=0x7f81dc003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 0x7f81dc003d00 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 rx=0x7f81ec00c190 tx=0x7f81ec001e50).stop 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 -- 172.21.15.73:0/2869517316 shutdown_connections 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 --2- 172.21.15.73:0/2869517316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81dc0038e0 0x7f81dc003d00 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 -- 172.21.15.73:0/2869517316 >> 172.21.15.73:0/2869517316 conn(0x7f81f00cb310 msgr2=0x7f81f00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 -- 172.21.15.73:0/2869517316 shutdown_connections 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.466+0000 7f81f7af7700 1 -- 172.21.15.73:0/2869517316 wait complete. 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.467+0000 7f81f7af7700 1 Processor -- start 2023-11-28T02:53:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.467+0000 7f81f7af7700 1 -- start start 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc0038e0 0x7f81f014b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c27a0 0x7f81f0153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f01547d0 con 0x7f81f00c2c20 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f014a2f0 con 0x7f81dc0038e0 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00c5010 con 0x7f81f00c27a0 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38968/0 (socket says 172.21.15.73:38968) 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 -- 172.21.15.73:0/118704349 learned_addr learned my addr 172.21.15.73:0/118704349 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f5893700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc0038e0 0x7f81f014b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f5092700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c27a0 0x7f81f0153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 -- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c27a0 msgr2=0x7f81f0153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c27a0 0x7f81f0153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 -- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc0038e0 msgr2=0x7f81f014b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc0038e0 0x7f81f014b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 -- 172.21.15.73:0/118704349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f014a540 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f6094700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f81f0062c90 tx=0x7f81e8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80030d0 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8003230 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f014a820 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.468+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8006a20 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.469+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f014b510 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f014ad40 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 4) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f81e8007030 con 0x7f81f00c2c20 2023-11-28T02:53:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81e5ffb700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 0x7f81d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f81e8079530 con 0x7f81f00c2c20 2023-11-28T02:53:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81f5893700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 0x7f81d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.470+0000 7f81f5893700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 0x7f81d4061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f81ec001ed0 tx=0x7f81ec015000).ready entity=mgr.4107 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.474+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81f014ad40 con 0x7f81f00c2c20 2023-11-28T02:53:38.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:38.681+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f81f014ad40 con 0x7f81f00c2c20 2023-11-28T02:53:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.486+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f81e8077300 con 0x7f81f00c2c20 2023-11-28T02:53:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.487+0000 7f81e5ffb700 1 -- 172.21.15.73:0/118704349 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v5) v1 ==== 92+0+0 (secure 0 0 0) 0x7f81e8071b40 con 0x7f81f00c2c20 2023-11-28T02:53:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 msgr2=0x7f81d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 0x7f81d4061610 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 rx=0x7f81ec001ed0 tx=0x7f81ec015000).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 msgr2=0x7f81f01542e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 rx=0x7f81f0062c90 tx=0x7f81e8009040).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 shutdown_connections 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f81d405f150 0x7f81d4061610 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f00c2c20 0x7f81f01542e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81dc0038e0 0x7f81f014b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 --2- 172.21.15.73:0/118704349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c27a0 0x7f81f0153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 >> 172.21.15.73:0/118704349 conn(0x7f81f00cb310 msgr2=0x7f81f00ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 shutdown_connections 2023-11-28T02:53:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.490+0000 7f81f7af7700 1 -- 172.21.15.73:0/118704349 wait complete. 2023-11-28T02:53:39.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T02:53:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.626+0000 7f4a2bb0d700 1 Processor -- start 2023-11-28T02:53:39.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.626+0000 7f4a2bb0d700 1 -- start start 2023-11-28T02:53:39.628 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:39.626+0000 7f9a1262c680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2bb0d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2bb0d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4a240c6190 0x7f4a24063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2bb0d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a240c72d0 con 0x7f4a24064350 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2bb0d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a24154bc0 con 0x7f4a240c6190 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2bb0d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240c66c0 con 0x7f4a24098d10 2023-11-28T02:53:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a290a8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4a240c6190 0x7f4a24063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a298a9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a290a8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4a240c6190 0x7f4a24063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:51838/0 (socket says 172.21.15.73:51838) 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a298a9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:36136/0 (socket says 172.21.15.73:36136) 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a2a0aa700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4a24064350 0x7f4a24158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59078/0 (socket says 172.21.15.73:59078) 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.627+0000 7f4a290a8700 1 -- 172.21.15.73:0/3828253429 learned_addr learned my addr 172.21.15.73:0/3828253429 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a290a8700 1 -- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 msgr2=0x7f4a240c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a290a8700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a290a8700 1 -- 172.21.15.73:0/3828253429 >> v1:172.21.15.150:6789/0 conn(0x7f4a24064350 legacy=0x7f4a24158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a290a8700 1 -- 172.21.15.73:0/3828253429 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a240c69a0 con 0x7f4a240c6190 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a288a7700 1 -- 172.21.15.73:0/3828253429 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3060389944 0 0) 0x7f4a240c72d0 con 0x7f4a24064350 2023-11-28T02:53:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a298a9700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:39.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a290a8700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240c6190 0x7f4a24063e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f4a20006fd0 tx=0x7f4a200090f0).ready entity=mon.1 client_cookie=33264450de05ea2e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:39.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a288a7700 1 -- 172.21.15.73:0/3828253429 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000b580 con 0x7f4a240c6190 2023-11-28T02:53:39.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a288a7700 1 -- 172.21.15.73:0/3828253429 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000b6e0 con 0x7f4a240c6190 2023-11-28T02:53:39.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f4a288a7700 1 -- 172.21.15.73:0/3828253429 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a2000ba60 con 0x7f4a240c6190 2023-11-28T02:53:39.632 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:39.628+0000 7f2bdc330680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T02:53:39.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240c6190 msgr2=0x7f4a24063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240c6190 0x7f4a24063e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f4a20006fd0 tx=0x7f4a200090f0).stop 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/3828253429 shutdown_connections 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24098d10 0x7f4a240c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/3828253429 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a240c6190 0x7f4a24063e60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/3828253429 >> 172.21.15.73:0/3828253429 conn(0x7f4a240cb310 msgr2=0x7f4a240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/3828253429 shutdown_connections 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.629+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/3828253429 wait complete. 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.630+0000 7f4a2bb0d700 1 Processor -- start 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.630+0000 7f4a2bb0d700 1 -- start start 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 0x7f4a24154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 0x7f4a240ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a240c6190 0x7f4a241422c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a240ade80 con 0x7f4a240c6190 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24156170 con 0x7f4a24098d10 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2bb0d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a24155560 con 0x7f4a24064350 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a298a9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 0x7f4a24154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a2a0aa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a240c6190 0x7f4a241422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 0x7f4a240ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a298a9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 0x7f4a24154ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:36140/0 (socket says 172.21.15.73:36140) 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a298a9700 1 -- 172.21.15.73:0/2855521735 learned_addr learned my addr 172.21.15.73:0/2855521735 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 -- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 msgr2=0x7f4a24154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 0x7f4a24154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 -- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a240c6190 msgr2=0x7f4a241422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a240c6190 0x7f4a241422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 -- 172.21.15.73:0/2855521735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a140092a0 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a290a8700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 0x7f4a240ca610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4a20014000 tx=0x7f4a20017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20000bc0 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.631+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a2000b850 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.632+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a20009370 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.632+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a20000d20 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.632+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a24155b20 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a24158fd0 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f4a2001b8f0 con 0x7f4a24098d10 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a19ffb700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 0x7f4a08061690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:39.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a298a9700 1 -- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 msgr2=0x7f4a08061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:39.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4a2007b9b0 con 0x7f4a24098d10 2023-11-28T02:53:39.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.633+0000 7f4a298a9700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 0x7f4a08061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T02:53:39.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.638+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a2004cc80 con 0x7f4a24098d10 2023-11-28T02:53:39.692 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:39.691+0000 7f9a1262c680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T02:53:39.696 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:39.694+0000 7f2bdc330680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T02:53:39.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.810+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4a2406c860 con 0x7f4a24098d10 2023-11-28T02:53:39.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.813+0000 7f4a19ffb700 1 -- 172.21.15.73:0/2855521735 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v3)=0 v3) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4a200502a0 con 0x7f4a24098d10 2023-11-28T02:53:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 msgr2=0x7f4a08061690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T02:53:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 0x7f4a08061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:39.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 msgr2=0x7f4a240ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 0x7f4a240ca610 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f4a20014000 tx=0x7f4a20017000).stop 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 shutdown_connections 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a0805f1d0 0x7f4a08061690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a240c6190 0x7f4a241422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a24064350 0x7f4a24154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 --2- 172.21.15.73:0/2855521735 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a24098d10 0x7f4a240ca610 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 >> 172.21.15.73:0/2855521735 conn(0x7f4a240cb310 msgr2=0x7f4a24143520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 shutdown_connections 2023-11-28T02:53:39.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:39.816+0000 7f4a2bb0d700 1 -- 172.21.15.73:0/2855521735 wait complete. 2023-11-28T02:53:39.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:39.829 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T02:53:39.849 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:39.849 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-11-28T02:53:39.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir0 1M 2023-11-28T02:53:39.969 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:39.968+0000 7f9a1262c680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T02:53:39.984 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:39.983+0000 7f2bdc330680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T02:53:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.067+0000 7f2ab473b700 1 Processor -- start 2023-11-28T02:53:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.067+0000 7f2ab473b700 1 -- start start 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab473b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2aac154610 0x7f2aac1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab473b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac0aa1c0 0x7f2aac0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab473b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aac159210 con 0x7f2aac0ab3b0 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab473b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aac0c0c70 con 0x7f2aac154610 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab473b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0c2110 con 0x7f2aac0aa1c0 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2aac154610 0x7f2aac1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2aac154610 0x7f2aac1582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39004/0 (socket says 172.21.15.73:39004) 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 -- 172.21.15.73:0/2111655059 learned_addr learned my addr 172.21.15.73:0/2111655059 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab2cd8700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac0aa1c0 0x7f2aac0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:40.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 -- 172.21.15.73:0/2111655059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac0aa1c0 msgr2=0x7f2aac0aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac0aa1c0 0x7f2aac0aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 -- 172.21.15.73:0/2111655059 >> v1:172.21.15.150:6789/0 conn(0x7f2aac0ab3b0 legacy=0x7f2aac0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.068+0000 7f2ab24d7700 1 -- 172.21.15.73:0/2111655059 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aac0aaaf0 con 0x7f2aac154610 2023-11-28T02:53:40.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.069+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac154610 0x7f2aac1582e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2a9c006fd0 tx=0x7f2a9c0090f0).ready entity=mon.0 client_cookie=1ae428ab06c39faa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:40.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.069+0000 7f2ab14d5700 1 -- 172.21.15.73:0/2111655059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00b580 con 0x7f2aac154610 2023-11-28T02:53:40.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.069+0000 7f2ab14d5700 1 -- 172.21.15.73:0/2111655059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c00b6e0 con 0x7f2aac154610 2023-11-28T02:53:40.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.069+0000 7f2ab14d5700 1 -- 172.21.15.73:0/2111655059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c00ba60 con 0x7f2aac154610 2023-11-28T02:53:40.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 -- 172.21.15.73:0/2111655059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac154610 msgr2=0x7f2aac1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:40.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac154610 0x7f2aac1582e0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 rx=0x7f2a9c006fd0 tx=0x7f2a9c0090f0).stop 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 -- 172.21.15.73:0/2111655059 shutdown_connections 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac0aa1c0 0x7f2aac0aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 --2- 172.21.15.73:0/2111655059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac154610 0x7f2aac1582e0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 -- 172.21.15.73:0/2111655059 >> 172.21.15.73:0/2111655059 conn(0x7f2aac0655a0 msgr2=0x7f2aac069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 -- 172.21.15.73:0/2111655059 shutdown_connections 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.070+0000 7f2ab473b700 1 -- 172.21.15.73:0/2111655059 wait complete. 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.071+0000 7f2ab473b700 1 Processor -- start 2023-11-28T02:53:40.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.071+0000 7f2ab473b700 1 -- start start 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 0x7f2aac1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac154610 0x7f2aac0ae5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac156150 con 0x7f2aac0aeab0 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0aef80 con 0x7f2aac0aa1c0 2023-11-28T02:53:40.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab473b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aac0af260 con 0x7f2aac154610 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39016/0 (socket says 172.21.15.73:39016) 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 -- 172.21.15.73:0/623675620 learned_addr learned my addr 172.21.15.73:0/623675620 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 0x7f2aac1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab1cd6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac154610 0x7f2aac0ae5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac154610 msgr2=0x7f2aac0ae5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac154610 0x7f2aac0ae5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 msgr2=0x7f2aac1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 0x7f2aac1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c009370 con 0x7f2aac0aeab0 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 0x7f2aac1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:40.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2ab2cd8700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2aa4009e90 tx=0x7f2aa400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.072+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400caa0 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.073+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa400f040 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.073+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aac0af540 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.073+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa400ce10 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.073+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aac145960 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.074+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aac156300 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.074+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 5) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f2aa4013610 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.074+0000 7f2aa27fc700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.074+0000 7f2ab24d7700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.075+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2aa407aab0 con 0x7f2aac0aeab0 2023-11-28T02:53:40.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.075+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T02:53:40.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.079+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2aac156300 con 0x7f2aac0aeab0 2023-11-28T02:53:40.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.273+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aac156300 con 0x7f2a9005f1a0 2023-11-28T02:53:40.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.276+0000 7f2ab24d7700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:40.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.276+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T02:53:40.584 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:40.583+0000 7f9a1262c680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T02:53:40.609 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:40.608+0000 7f2bdc330680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T02:53:40.656 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:40.655+0000 7f9a1262c680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T02:53:40.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.677+0000 7f2ab24d7700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:40.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:40.677+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T02:53:40.681 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:40.680+0000 7f2bdc330680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T02:53:40.842 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:40.840+0000 7f9a1262c680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T02:53:40.867 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:40.866+0000 7f2bdc330680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T02:53:41.264 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.263+0000 7f9a1262c680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T02:53:41.285 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.284+0000 7f2bdc330680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T02:53:41.367 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.366+0000 7f9a1262c680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:53:41.389 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.388+0000 7f2bdc330680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:53:41.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:41.478+0000 7f2ab24d7700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:41.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:41.478+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T02:53:41.520 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.519+0000 7f9a1262c680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T02:53:41.540 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.539+0000 7f2bdc330680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T02:53:41.595 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.594+0000 7f9a1262c680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T02:53:41.615 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.613+0000 7f2bdc330680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T02:53:41.662 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.661+0000 7f9a1262c680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T02:53:41.681 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.680+0000 7f2bdc330680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T02:53:41.920 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:41.919+0000 7f9a1262c680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T02:53:41.940 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:41.939+0000 7f2bdc330680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T02:53:42.102 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:42.101+0000 7f9a1262c680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T02:53:42.120 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:42.119+0000 7f2bdc330680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T02:53:42.755 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:42.753+0000 7f9a1262c680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T02:53:42.766 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:42.765+0000 7f2bdc330680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T02:53:42.822 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:42.821+0000 7f9a1262c680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T02:53:42.833 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:42.832+0000 7f2bdc330680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T02:53:42.897 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:42.896+0000 7f9a1262c680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T02:53:42.908 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:42.906+0000 7f2bdc330680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T02:53:43.043 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.042+0000 7f9a1262c680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T02:53:43.052 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.051+0000 7f2bdc330680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T02:53:43.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:43.079+0000 7f2ab24d7700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T02:53:43.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:43.079+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T02:53:43.113 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.112+0000 7f9a1262c680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T02:53:43.121 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.120+0000 7f2bdc330680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T02:53:43.228 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.226+0000 7f9a1262c680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T02:53:43.235 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.234+0000 7f2bdc330680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T02:53:43.335 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.334+0000 7f9a1262c680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:53:43.342 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.341+0000 7f2bdc330680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T02:53:43.627 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.626+0000 7f9a1262c680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T02:53:43.627 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.626+0000 7f2bdc330680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T02:53:43.697 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.695+0000 7f2bdc330680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T02:53:43.697 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T02:53:43.696+0000 7f9a1262c680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T02:53:43.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:43.715+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 6) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f2aa4078880 con 0x7f2aac0aeab0 2023-11-28T02:53:43.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:43.715+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a90061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T02:53:43.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:43.715+0000 7f2aa27fc700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a90061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b50ab6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b50ab6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b50ab6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b50ab6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b50ab6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b532bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b532bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b532bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b532bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:43.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:53:43.781+0000 7f2b532bb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:53:44.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.718+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2aa4078560 con 0x7f2aac0aeab0 2023-11-28T02:53:44.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.718+0000 7f2aa27fc700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a900615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.718+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aac156300 con 0x7f2a9005f1a0 2023-11-28T02:53:44.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.720+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a900615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.722+0000 7f2ab24d7700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a900615e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f2a9c009fd0 tx=0x7f2a9c017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.745+0000 7f2aa27fc700 1 -- 172.21.15.73:0/623675620 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f2aac156300 con 0x7f2a9005f1a0 2023-11-28T02:53:44.746 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir0 2023-11-28T02:53:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 msgr2=0x7f2a900615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a900615e0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7f2a9c009fd0 tx=0x7f2a9c017000).stop 2023-11-28T02:53:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 msgr2=0x7f2aac0ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 rx=0x7f2aa4009e90 tx=0x7f2aa400b040).stop 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 shutdown_connections 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2a9005f1a0 0x7f2a900615e0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2aac0aeab0 0x7f2aac0ac800 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aac0aa1c0 0x7f2aac1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 --2- 172.21.15.73:0/623675620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aac154610 0x7f2aac0ae5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 >> 172.21.15.73:0/623675620 conn(0x7f2aac0655a0 msgr2=0x7f2aac153540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 shutdown_connections 2023-11-28T02:53:44.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.747+0000 7f2ab473b700 1 -- 172.21.15.73:0/623675620 wait complete. 2023-11-28T02:53:44.762 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-11-28T02:53:44.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:44.762 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-11-28T02:53:44.819 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir1 1M 2023-11-28T02:53:44.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.977+0000 7ff9b77d3700 1 Processor -- start 2023-11-28T02:53:44.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b77d3700 1 -- start start 2023-11-28T02:53:44.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b77d3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b00c5f30 0x7ff9b00c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b77d3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b00c68d0 con 0x7ff9b0063af0 2023-11-28T02:53:44.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b77d3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9b0158e20 con 0x7ff9b01373a0 2023-11-28T02:53:44.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b77d3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9b01582a0 con 0x7ff9b00c5f30 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.978+0000 7ff9b4d6e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b00c5f30 0x7ff9b00c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff9b00c5f30 0x7ff9b00c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36494/0 (socket says 172.21.15.73:36494) 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 -- 172.21.15.73:0/839496006 learned_addr learned my addr 172.21.15.73:0/839496006 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 -- 172.21.15.73:0/839496006 >> v1:172.21.15.150:6789/0 conn(0x7ff9b01373a0 legacy=0x7ff9b00c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 -- 172.21.15.73:0/839496006 >> v1:172.21.15.73:6789/0 conn(0x7ff9b0063af0 legacy=0x7ff9b0063f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 -- 172.21.15.73:0/839496006 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9b0158580 con 0x7ff9b00c5f30 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9b4d6e700 1 --2- 172.21.15.73:0/839496006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b00c5f30 0x7ff9b00c63e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff9a0006fd0 tx=0x7ff9a00090f0).ready entity=mon.0 client_cookie=5a174ed07812dcc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9a7fff700 1 -- 172.21.15.73:0/839496006 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000b600 con 0x7ff9b00c5f30 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9a7fff700 1 -- 172.21.15.73:0/839496006 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9a000b760 con 0x7ff9b00c5f30 2023-11-28T02:53:44.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.979+0000 7ff9a7fff700 1 -- 172.21.15.73:0/839496006 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9a000bae0 con 0x7ff9b00c5f30 2023-11-28T02:53:44.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 -- 172.21.15.73:0/839496006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b00c5f30 msgr2=0x7ff9b00c63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/839496006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b00c5f30 0x7ff9b00c63e0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7ff9a0006fd0 tx=0x7ff9a00090f0).stop 2023-11-28T02:53:44.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 -- 172.21.15.73:0/839496006 shutdown_connections 2023-11-28T02:53:44.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/839496006 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b00c5f30 0x7ff9b00c63e0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 -- 172.21.15.73:0/839496006 >> 172.21.15.73:0/839496006 conn(0x7ff9b00628f0 msgr2=0x7ff9b0062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:44.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 -- 172.21.15.73:0/839496006 shutdown_connections 2023-11-28T02:53:44.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.980+0000 7ff9b77d3700 1 -- 172.21.15.73:0/839496006 wait complete. 2023-11-28T02:53:44.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.981+0000 7ff9b77d3700 1 Processor -- start 2023-11-28T02:53:44.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.981+0000 7ff9b77d3700 1 -- start start 2023-11-28T02:53:44.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9b01373a0 0x7ff9b0164a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9b0150780 0x7ff9b00ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b01518e0 con 0x7ff9b0063af0 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b0165c70 con 0x7ff9b01373a0 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b77d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9b00ac3b0 con 0x7ff9b0150780 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36506/0 (socket says 172.21.15.73:36506) 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 -- 172.21.15.73:0/1727186596 learned_addr learned my addr 172.21.15.73:0/1727186596 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b4d6e700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9b01373a0 0x7ff9b0164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b5d70700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9b0150780 0x7ff9b00ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 -- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9b0150780 msgr2=0x7ff9b00ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9b0150780 0x7ff9b00ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 -- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9b01373a0 msgr2=0x7ff9b0164a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:44.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9b01373a0 0x7ff9b0164a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 -- 172.21.15.73:0/1727186596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9a0009370 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9b556f700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff9ac00c470 tx=0x7ff9ac007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.982+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac007020 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.983+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9ac007180 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.983+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9b0165360 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.983+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac007520 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.983+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9b0165920 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.984+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9b006c860 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.984+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7ff9ac0068b0 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.984+0000 7ff9a57fa700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 0x7ff994061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.985+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff9ac079260 con 0x7ff9b0063af0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.985+0000 7ff9b4d6e700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 0x7ff994061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:44.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.985+0000 7ff9b4d6e700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 0x7ff994061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff9b00c6310 tx=0x7ff9a0017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:44.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:44.989+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff9ac047af0 con 0x7ff9b0063af0 2023-11-28T02:53:45.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.177+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9b0158860 con 0x7ff99405f050 2023-11-28T02:53:45.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.181+0000 7ff9a57fa700 1 -- 172.21.15.73:0/1727186596 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7ff9b0158860 con 0x7ff99405f050 2023-11-28T02:53:45.182 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir1 2023-11-28T02:53:45.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 msgr2=0x7ff994061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 0x7ff994061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff9b00c6310 tx=0x7ff9a0017000).stop 2023-11-28T02:53:45.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 msgr2=0x7ff9b01356e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 rx=0x7ff9ac00c470 tx=0x7ff9ac007e00).stop 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 shutdown_connections 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff99405f050 0x7ff994061510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9b0150780 0x7ff9b00ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9b0063af0 0x7ff9b01356e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.182+0000 7ff9b77d3700 1 --2- 172.21.15.73:0/1727186596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9b01373a0 0x7ff9b0164a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.183+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 >> 172.21.15.73:0/1727186596 conn(0x7ff9b00628f0 msgr2=0x7ff9b00c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.183+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 shutdown_connections 2023-11-28T02:53:45.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.183+0000 7ff9b77d3700 1 -- 172.21.15.73:0/1727186596 wait complete. 2023-11-28T02:53:45.196 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-11-28T02:53:45.196 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:45.196 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-11-28T02:53:45.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir2 1M 2023-11-28T02:53:45.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.368+0000 7fc4ae608700 1 Processor -- start 2023-11-28T02:53:45.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.368+0000 7fc4ae608700 1 -- start start 2023-11-28T02:53:45.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4ae608700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc4a80bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4ae608700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4a80bd3e0 0x7fc4a80c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4ae608700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a8067840 con 0x7fc4a8154520 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4ae608700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a80bd870 con 0x7fc4a80bd3e0 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4ae608700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8066c30 con 0x7fc4a8154900 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4a7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc4a8154520 0x7fc4a806a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43408/0 (socket says 172.21.15.73:43408) 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4a7fff700 1 -- 172.21.15.73:0/1718338329 learned_addr learned my addr 172.21.15.73:0/1718338329 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.369+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc4a80bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4acba5700 1 --2- 172.21.15.73:0/1718338329 >> v2:172.21.15.150:3300/0 conn(0x7fc4a80bd3e0 0x7fc4a80c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063518775 0 0) 0x7fc4a8067840 con 0x7fc4a8154520 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4900033f0 con 0x7fc4a8154520 2023-11-28T02:53:45.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1342802260 0 0) 0x7fc4900033f0 con 0x7fc4a8154520 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 msgr2=0x7fc4a80bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc4a80bcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 >> v2:172.21.15.150:3300/0 conn(0x7fc4a80bd3e0 msgr2=0x7fc4a80c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 --2- 172.21.15.73:0/1718338329 >> v2:172.21.15.150:3300/0 conn(0x7fc4a80bd3e0 0x7fc4a80c40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8066f10 con 0x7fc4a8154520 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc498001470 con 0x7fc4a8154520 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 >> v1:172.21.15.73:6789/0 conn(0x7fc4a8154520 legacy=0x7fc4a806a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4900038e0 0x7fc490003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc490004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc490005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80bd870 con 0x7fc490004690 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8066c30 con 0x7fc4900038e0 2023-11-28T02:53:45.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc49000b8e0 con 0x7fc4a8154900 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc490004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a7fff700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4900038e0 0x7fc490003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4acba5700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc490005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 -- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 msgr2=0x7fc490005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a8154900 0x7fc490005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 -- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4900038e0 msgr2=0x7fc490003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4900038e0 0x7fc490003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 -- 172.21.15.73:0/1718338329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4900033f0 con 0x7fc490004690 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.370+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc490004b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fc49c009a80 tx=0x7fc49c009060).ready entity=mon.0 client_cookie=4555db7a7a226ea1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c009430 con 0x7fc490004690 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c00e640 con 0x7fc490004690 2023-11-28T02:53:45.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4a6ffd700 1 -- 172.21.15.73:0/1718338329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c00e9c0 con 0x7fc490004690 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 -- 172.21.15.73:0/1718338329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 msgr2=0x7fc490004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 --2- 172.21.15.73:0/1718338329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc490004b10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 rx=0x7fc49c009a80 tx=0x7fc49c009060).stop 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4acba5700 1 -- 172.21.15.73:0/1718338329 reap_dead start 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 -- 172.21.15.73:0/1718338329 shutdown_connections 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 -- 172.21.15.73:0/1718338329 >> 172.21.15.73:0/1718338329 conn(0x7fc4a8063810 msgr2=0x7fc4a8063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 -- 172.21.15.73:0/1718338329 shutdown_connections 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.371+0000 7fc4ae608700 1 -- 172.21.15.73:0/1718338329 wait complete. 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.372+0000 7fc4ae608700 1 Processor -- start 2023-11-28T02:53:45.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.372+0000 7fc4ae608700 1 -- start start 2023-11-28T02:53:45.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 0x7fc4a8168790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a814eb70 0x7fc4a814efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80a9700 con 0x7fc490004690 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80a8690 con 0x7fc4a814eb70 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4ae608700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8168c80 con 0x7fc4a80c3e00 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36518/0 (socket says 172.21.15.73:36518) 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 -- 172.21.15.73:0/837604911 learned_addr learned my addr 172.21.15.73:0/837604911 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 0x7fc4a8168790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4acba5700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a814eb70 0x7fc4a814efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 -- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 msgr2=0x7fc4a8168790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 0x7fc4a8168790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 -- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a814eb70 msgr2=0x7fc4a814efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a814eb70 0x7fc4a814efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a7fff700 1 -- 172.21.15.73:0/837604911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a80a88a0 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.373+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 0x7fc4a8168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc4a7fff700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc498004730 tx=0x7fc498001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011590 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4980116f0 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a80a8b80 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498011a70 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.374+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a80a9140 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.375+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a806c860 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.375+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fc49801d3b0 con 0x7fc490004690 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.375+0000 7fc48ffff700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 0x7fc494061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.376+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 0x7fc494061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.376+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fc498079990 con 0x7fc490004690 2023-11-28T02:53:45.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.376+0000 7fc4a77fe700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 0x7fc494061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc49c006d50 tx=0x7fc49c01a000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.380+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc49804add0 con 0x7fc490004690 2023-11-28T02:53:45.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.569+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4a80671f0 con 0x7fc49405f080 2023-11-28T02:53:45.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.572+0000 7fc48ffff700 1 -- 172.21.15.73:0/837604911 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fc4a80671f0 con 0x7fc49405f080 2023-11-28T02:53:45.574 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir2 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 msgr2=0x7fc494061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 0x7fc494061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fc49c006d50 tx=0x7fc49c01a000).stop 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 msgr2=0x7fc4a80a81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 rx=0x7fc498004730 tx=0x7fc498001e90).stop 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 shutdown_connections 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc49405f080 0x7fc494061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4a814eb70 0x7fc4a814efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc490004690 0x7fc4a80a81a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 --2- 172.21.15.73:0/837604911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4a80c3e00 0x7fc4a8168790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 >> 172.21.15.73:0/837604911 conn(0x7fc4a8063810 msgr2=0x7fc4a80a6800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 shutdown_connections 2023-11-28T02:53:45.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.574+0000 7fc4ae608700 1 -- 172.21.15.73:0/837604911 wait complete. 2023-11-28T02:53:45.588 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-11-28T02:53:45.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:45.588 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-11-28T02:53:45.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir3 1M 2023-11-28T02:53:45.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.761+0000 7f996df60700 1 Processor -- start 2023-11-28T02:53:45.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.761+0000 7f996df60700 1 -- start start 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f996df60700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99680aa930 0x7f99680a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f996df60700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99680c6630 con 0x7f9968154530 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f996df60700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9968064230 con 0x7f9968154910 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f996df60700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99680c6390 con 0x7f99680aa930 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f9967fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99680aa930 0x7f99680a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f9967fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f99680aa930 0x7f99680a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36530/0 (socket says 172.21.15.73:36530) 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.762+0000 7f9967fff700 1 -- 172.21.15.73:0/4222450241 learned_addr learned my addr 172.21.15.73:0/4222450241 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f9967fff700 1 -- 172.21.15.73:0/4222450241 >> v1:172.21.15.150:6789/0 conn(0x7f9968154910 legacy=0x7f99680aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f9967fff700 1 -- 172.21.15.73:0/4222450241 >> v1:172.21.15.73:6789/0 conn(0x7f9968154530 legacy=0x7f99680c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f9967fff700 1 -- 172.21.15.73:0/4222450241 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9968064510 con 0x7f99680aa930 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f9967fff700 1 --2- 172.21.15.73:0/4222450241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f99680a7ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9960006fd0 tx=0x7f99600090f0).ready entity=mon.0 client_cookie=ce6fad70520c1843 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f99667fc700 1 -- 172.21.15.73:0/4222450241 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000b600 con 0x7f99680aa930 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f99667fc700 1 -- 172.21.15.73:0/4222450241 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996000b760 con 0x7f99680aa930 2023-11-28T02:53:45.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f99667fc700 1 -- 172.21.15.73:0/4222450241 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996000bae0 con 0x7f99680aa930 2023-11-28T02:53:45.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f996df60700 1 -- 172.21.15.73:0/4222450241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 msgr2=0x7f99680a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f996df60700 1 --2- 172.21.15.73:0/4222450241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f99680a7ff0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 rx=0x7f9960006fd0 tx=0x7f99600090f0).stop 2023-11-28T02:53:45.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f996df60700 1 -- 172.21.15.73:0/4222450241 shutdown_connections 2023-11-28T02:53:45.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f996df60700 1 --2- 172.21.15.73:0/4222450241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f99680a7ff0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.763+0000 7f996df60700 1 -- 172.21.15.73:0/4222450241 >> 172.21.15.73:0/4222450241 conn(0x7f9968094930 msgr2=0x7f996806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.764+0000 7f996df60700 1 -- 172.21.15.73:0/4222450241 shutdown_connections 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.764+0000 7f996df60700 1 -- 172.21.15.73:0/4222450241 wait complete. 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.765+0000 7f996df60700 1 Processor -- start 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.765+0000 7f996df60700 1 -- start start 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 0x7f99680ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9968154910 0x7f99680ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680caa60 con 0x7f99680aa930 2023-11-28T02:53:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99680aad30 con 0x7f9968154910 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99680ab5d0 con 0x7f9968154530 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36540/0 (socket says 172.21.15.73:36540) 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 -- 172.21.15.73:0/3679961366 learned_addr learned my addr 172.21.15.73:0/3679961366 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f9967fff700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9968154910 0x7f99680ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f9966ffd700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 0x7f99680ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 -- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 msgr2=0x7f99680ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 0x7f99680ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 -- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9968154910 msgr2=0x7f99680ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9968154910 0x7f99680ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 -- 172.21.15.73:0/3679961366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9960009370 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f9966ffd700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 0x7f99680ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f99677fe700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f995800c870 tx=0x7f9958001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800e4a0 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f995800e600 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9968151c20 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f995800e9a0 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.766+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99680ab000 con 0x7f99680aa930 2023-11-28T02:53:45.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.767+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f9958003070 con 0x7f99680aa930 2023-11-28T02:53:45.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.767+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f99680ab380 con 0x7f99680aa930 2023-11-28T02:53:45.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.768+0000 7f994f7fe700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 0x7f9954061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:45.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.768+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9958079860 con 0x7f99680aa930 2023-11-28T02:53:45.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.768+0000 7f9966ffd700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 0x7f9954061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:45.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.768+0000 7f9966ffd700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 0x7f9954061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f995c006fd0 tx=0x7f995c008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:45.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.772+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f995804ac50 con 0x7f99680aa930 2023-11-28T02:53:45.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.960+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f99680ab710 con 0x7f995405f080 2023-11-28T02:53:45.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.964+0000 7f994f7fe700 1 -- 172.21.15.73:0/3679961366 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f995c00be30 con 0x7f995405f080 2023-11-28T02:53:45.965 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir3 2023-11-28T02:53:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 msgr2=0x7f9954061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 0x7f9954061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f995c006fd0 tx=0x7f995c008040).stop 2023-11-28T02:53:45.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 msgr2=0x7f9968151730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 rx=0x7f995800c870 tx=0x7f9958001650).stop 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 shutdown_connections 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f995405f080 0x7f9954061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9968154910 0x7f99680ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f99680aa930 0x7f9968151730 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 --2- 172.21.15.73:0/3679961366 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9968154530 0x7f99680ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 >> 172.21.15.73:0/3679961366 conn(0x7f9968094930 msgr2=0x7f99681534a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 shutdown_connections 2023-11-28T02:53:45.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:45.966+0000 7f996df60700 1 -- 172.21.15.73:0/3679961366 wait complete. 2023-11-28T02:53:45.979 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-11-28T02:53:45.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:45.980 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-11-28T02:53:45.997 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir4 1M 2023-11-28T02:53:46.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.153+0000 7fe4608ba700 1 Processor -- start 2023-11-28T02:53:46.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 -- start start 2023-11-28T02:53:46.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe45c0c6530 0x7fe45c0640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0c57e0 0x7fe45c0be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe45c0bf1f0 con 0x7fe45c0bed70 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c065680 con 0x7fe45c0c6530 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe4608ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c6910 con 0x7fe45c0c57e0 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.154+0000 7fe45ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe45c0c6530 0x7fe45c0640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe45c0c6530 0x7fe45c0640b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36554/0 (socket says 172.21.15.73:36554) 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 -- 172.21.15.73:0/3238740229 learned_addr learned my addr 172.21.15.73:0/3238740229 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45259c700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0c57e0 0x7fe45c0be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 -- 172.21.15.73:0/3238740229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0c57e0 msgr2=0x7fe45c0be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0c57e0 0x7fe45c0be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 -- 172.21.15.73:0/3238740229 >> v1:172.21.15.73:6789/0 conn(0x7fe45c0bed70 legacy=0x7fe45c1549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 -- 172.21.15.73:0/3238740229 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c064a00 con 0x7fe45c0c6530 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c6530 0x7fe45c0640b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe44c009f20 tx=0x7fe44c009840).ready entity=mon.0 client_cookie=b19e64a8e54e573c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45a59c700 1 -- 172.21.15.73:0/3238740229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00e070 con 0x7fe45c0c6530 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45a59c700 1 -- 172.21.15.73:0/3238740229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44c00b9d0 con 0x7fe45c0c6530 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe45a59c700 1 -- 172.21.15.73:0/3238740229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44c00bd50 con 0x7fe45c0c6530 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 -- 172.21.15.73:0/3238740229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c6530 msgr2=0x7fe45c0640b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c6530 0x7fe45c0640b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 rx=0x7fe44c009f20 tx=0x7fe44c009840).stop 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 -- 172.21.15.73:0/3238740229 shutdown_connections 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c6530 0x7fe45c0640b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 --2- 172.21.15.73:0/3238740229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0c57e0 0x7fe45c0be880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.155+0000 7fe4608ba700 1 -- 172.21.15.73:0/3238740229 >> 172.21.15.73:0/3238740229 conn(0x7fe45c092520 msgr2=0x7fe45c062ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.156+0000 7fe4608ba700 1 -- 172.21.15.73:0/3238740229 shutdown_connections 2023-11-28T02:53:46.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.156+0000 7fe4608ba700 1 -- 172.21.15.73:0/3238740229 wait complete. 2023-11-28T02:53:46.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.157+0000 7fe4608ba700 1 Processor -- start 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.157+0000 7fe4608ba700 1 -- start start 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.157+0000 7fe4608ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 0x7fe45c152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.157+0000 7fe4608ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe45c0c6530 0x7fe45c0965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c1688a0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c167c90 con 0x7fe45c0c6530 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c167f70 con 0x7fe45c0bed70 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36556/0 (socket says 172.21.15.73:36556) 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 -- 172.21.15.73:0/2469338932 learned_addr learned my addr 172.21.15.73:0/2469338932 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 0x7fe45c152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45b59e700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe45c0c6530 0x7fe45c0965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 -- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 msgr2=0x7fe45c152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 0x7fe45c152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 -- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe45c0c6530 msgr2=0x7fe45c0965f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe45c0c6530 0x7fe45c0965f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 -- 172.21.15.73:0/2469338932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe44c009ae0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 0x7fe45c152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:46.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe45259c700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe44400e950 tx=0x7fe444009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44400c850 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe44400c9b0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c168250 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe44400cd30 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.158+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c153490 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.159+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c06c7c0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.160+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fe444012600 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.160+0000 7fe4537fe700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 0x7fe43c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.160+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe44407a6d0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.160+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 0x7fe43c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.160+0000 7fe45ad9d700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 0x7fe43c061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe45c0ad270 tx=0x7fe44c009840).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.164+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe44404bac0 con 0x7fe45c0c57e0 2023-11-28T02:53:46.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.351+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe45c064ce0 con 0x7fe43c05f080 2023-11-28T02:53:46.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.354+0000 7fe4537fe700 1 -- 172.21.15.73:0/2469338932 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe45c064ce0 con 0x7fe43c05f080 2023-11-28T02:53:46.356 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir4 2023-11-28T02:53:46.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 msgr2=0x7fe43c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 0x7fe43c061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe45c0ad270 tx=0x7fe44c009840).stop 2023-11-28T02:53:46.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 msgr2=0x7fe45c096100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 rx=0x7fe44400e950 tx=0x7fe444009060).stop 2023-11-28T02:53:46.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 shutdown_connections 2023-11-28T02:53:46.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe43c05f080 0x7fe43c061540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe45c0c6530 0x7fe45c0965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe45c0bed70 0x7fe45c152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 --2- 172.21.15.73:0/2469338932 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe45c0c57e0 0x7fe45c096100 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 >> 172.21.15.73:0/2469338932 conn(0x7fe45c092520 msgr2=0x7fe45c1549a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 shutdown_connections 2023-11-28T02:53:46.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.356+0000 7fe4608ba700 1 -- 172.21.15.73:0/2469338932 wait complete. 2023-11-28T02:53:46.370 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-11-28T02:53:46.370 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:46.371 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-11-28T02:53:46.387 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir5 1M 2023-11-28T02:53:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.544+0000 7f8940945700 1 Processor -- start 2023-11-28T02:53:46.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.544+0000 7f8940945700 1 -- start start 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8940945700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8940945700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f893c0aa220 0x7f893c0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8940945700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f893c0c6630 con 0x7f893c154530 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8940945700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f893c0c6390 con 0x7f893c0aa220 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8940945700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c0a84e0 con 0x7f893c154910 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f893c0aa220 0x7f893c0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f893c0aa220 0x7f893c0aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36562/0 (socket says 172.21.15.73:36562) 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 -- 172.21.15.73:0/3608673526 learned_addr learned my addr 172.21.15.73:0/3608673526 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f8939d9b700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 -- 172.21.15.73:0/3608673526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 msgr2=0x7f893c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:46.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 -- 172.21.15.73:0/3608673526 >> v1:172.21.15.73:6789/0 conn(0x7f893c154530 legacy=0x7f893c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 -- 172.21.15.73:0/3608673526 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0aaba0 con 0x7f893c0aa220 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893ad9d700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0aa220 0x7f893c0aa6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f8934009f20 tx=0x7f8934009840).ready entity=mon.0 client_cookie=a0b6586243f2645e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893959a700 1 -- 172.21.15.73:0/3608673526 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400e070 con 0x7f893c0aa220 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893959a700 1 -- 172.21.15.73:0/3608673526 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f893400ba80 con 0x7f893c0aa220 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.545+0000 7f893959a700 1 -- 172.21.15.73:0/3608673526 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f893400be00 con 0x7f893c0aa220 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 -- 172.21.15.73:0/3608673526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0aa220 msgr2=0x7f893c0aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0aa220 0x7f893c0aa6b0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 rx=0x7f8934009f20 tx=0x7f8934009840).stop 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 -- 172.21.15.73:0/3608673526 shutdown_connections 2023-11-28T02:53:46.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0aa220 0x7f893c0aa6b0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 --2- 172.21.15.73:0/3608673526 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 -- 172.21.15.73:0/3608673526 >> 172.21.15.73:0/3608673526 conn(0x7f893c094930 msgr2=0x7f893c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:46.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 -- 172.21.15.73:0/3608673526 shutdown_connections 2023-11-28T02:53:46.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.546+0000 7f8940945700 1 -- 172.21.15.73:0/3608673526 wait complete. 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.547+0000 7f8940945700 1 Processor -- start 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.547+0000 7f8940945700 1 -- start start 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c0aa220 0x7f893c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0c0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c158480 con 0x7f893c154530 2023-11-28T02:53:46.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c157ec0 con 0x7f893c0aa220 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c1581a0 con 0x7f893c154910 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36570/0 (socket says 172.21.15.73:36570) 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 -- 172.21.15.73:0/44408601 learned_addr learned my addr 172.21.15.73:0/44408601 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f893ad9d700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0c0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f893a59c700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c0aa220 0x7f893c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 -- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 msgr2=0x7f893c0c0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0c0410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 -- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c0aa220 msgr2=0x7f893c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c0aa220 0x7f893c157900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 -- 172.21.15.73:0/44408601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8934009ae0 con 0x7f893c154530 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f893ad9d700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8939d9b700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f893000d860 tx=0x7f8930008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930011590 con 0x7f893c154530 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89300116f0 con 0x7f893c154530 2023-11-28T02:53:46.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.548+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f893c0c3810 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.549+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8930011a70 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.549+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f893c168cd0 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.549+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f893c0c3670 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.550+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 7) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f8930012420 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.550+0000 7f892a7fc700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 0x7f891c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.550+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f8930079b50 con 0x7f893c154530 2023-11-28T02:53:46.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.550+0000 7f893a59c700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 0x7f891c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.550+0000 7f893a59c700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 0x7f891c061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f892c0015e0 tx=0x7f892c009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.554+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f893c0c3670 con 0x7f893c154530 2023-11-28T02:53:46.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.724+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f8930077a40 con 0x7f893c154530 2023-11-28T02:53:46.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.743+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f893c0c3d30 con 0x7f891c05f080 2023-11-28T02:53:46.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.746+0000 7f892a7fc700 1 -- 172.21.15.73:0/44408601 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f893c0c3d30 con 0x7f891c05f080 2023-11-28T02:53:46.748 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir5 2023-11-28T02:53:46.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 msgr2=0x7f891c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 0x7f891c061540 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f892c0015e0 tx=0x7f892c009040).stop 2023-11-28T02:53:46.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 msgr2=0x7f893c1687e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 rx=0x7f893000d860 tx=0x7f8930008040).stop 2023-11-28T02:53:46.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 shutdown_connections 2023-11-28T02:53:46.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f891c05f080 0x7f891c061540 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c154910 0x7f893c0c0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c0aa220 0x7f893c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 --2- 172.21.15.73:0/44408601 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c154530 0x7f893c1687e0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 >> 172.21.15.73:0/44408601 conn(0x7f893c094930 msgr2=0x7f893c153040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:46.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 shutdown_connections 2023-11-28T02:53:46.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.748+0000 7f8940945700 1 -- 172.21.15.73:0/44408601 wait complete. 2023-11-28T02:53:46.762 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-11-28T02:53:46.762 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:46.762 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-11-28T02:53:46.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir6 1M 2023-11-28T02:53:46.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.932+0000 7f6eafd48700 1 Processor -- start 2023-11-28T02:53:46.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.933+0000 7f6eafd48700 1 -- start start 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.933+0000 7f6eafd48700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.933+0000 7f6eafd48700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ea80c6550 0x7f6ea80c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eafd48700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea8156d50 con 0x7f6ea813acd0 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eafd48700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ea80c6970 con 0x7f6ea80c6550 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eafd48700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea8156140 con 0x7f6ea813b0b0 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ea80c6550 0x7f6ea80c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ea80c6550 0x7f6ea80c7100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:54632/0 (socket says 172.21.15.73:54632) 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 -- 172.21.15.73:0/4021015042 learned_addr learned my addr 172.21.15.73:0/4021015042 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eadae4700 1 --1- 172.21.15.73:0/4021015042 >> v1:172.21.15.150:6789/0 conn(0x7f6ea813acd0 0x7f6ea813b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:55280/0 (socket says 172.21.15.73:55280) 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6ead2e3700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 -- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 msgr2=0x7f6ea80c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:46.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 -- 172.21.15.73:0/4021015042 >> v1:172.21.15.150:6789/0 conn(0x7f6ea813acd0 legacy=0x7f6ea813b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 -- 172.21.15.73:0/4021015042 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea8156420 con 0x7f6ea80c6550 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6ead2e3700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.934+0000 7f6eae2e5700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea80c7100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6ea0006fd0 tx=0x7f6ea00090f0).ready entity=mon.1 client_cookie=833bc20f83965923 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eacae2700 1 -- 172.21.15.73:0/4021015042 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000b580 con 0x7f6ea80c6550 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eacae2700 1 -- 172.21.15.73:0/4021015042 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea000b6e0 con 0x7f6ea80c6550 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eacae2700 1 -- 172.21.15.73:0/4021015042 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea000ba80 con 0x7f6ea80c6550 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 -- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 msgr2=0x7f6ea80c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea80c7100 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 rx=0x7f6ea0006fd0 tx=0x7f6ea00090f0).stop 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 -- 172.21.15.73:0/4021015042 shutdown_connections 2023-11-28T02:53:46.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea80c7100 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4021015042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 -- 172.21.15.73:0/4021015042 >> 172.21.15.73:0/4021015042 conn(0x7f6ea80628f0 msgr2=0x7f6ea8062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 -- 172.21.15.73:0/4021015042 shutdown_connections 2023-11-28T02:53:46.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.935+0000 7f6eafd48700 1 -- 172.21.15.73:0/4021015042 wait complete. 2023-11-28T02:53:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.936+0000 7f6eafd48700 1 Processor -- start 2023-11-28T02:53:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.936+0000 7f6eafd48700 1 -- start start 2023-11-28T02:53:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea812d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea812cbe0 con 0x7f6ea813acd0 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea812bfd0 con 0x7f6ea80c6550 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eafd48700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea812c2b0 con 0x7f6ea813b0b0 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6ead2e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6ead2e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36576/0 (socket says 172.21.15.73:36576) 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6ead2e3700 1 -- 172.21.15.73:0/4124920694 learned_addr learned my addr 172.21.15.73:0/4124920694 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eae2e5700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.937+0000 7f6eadae4700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea812d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 -- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 msgr2=0x7f6ea80c9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 -- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 msgr2=0x7f6ea812d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea812d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 -- 172.21.15.73:0/4124920694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e980092a0 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6ead2e3700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f6e9800e980 tx=0x7f6e98009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800ee70 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9800c850 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9800cbd0 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea0009370 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.938+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea812c870 con 0x7f6ea813acd0 2023-11-28T02:53:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.939+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f6e9801b400 con 0x7f6ea813acd0 2023-11-28T02:53:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.939+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea812cd50 con 0x7f6ea813acd0 2023-11-28T02:53:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.940+0000 7f6e9dffb700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 0x7f6e8c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.940+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f6e9807aaa0 con 0x7f6ea813acd0 2023-11-28T02:53:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.940+0000 7f6eadae4700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 0x7f6e8c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:46.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.940+0000 7f6eadae4700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 0x7f6e8c061810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6ea4001a10 tx=0x7f6ea4001560).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:46.944+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6e9804bb10 con 0x7f6ea813acd0 2023-11-28T02:53:47.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.131+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6ea806c860 con 0x7f6e8c05f350 2023-11-28T02:53:47.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.134+0000 7f6e9dffb700 1 -- 172.21.15.73:0/4124920694 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f6ea806c860 con 0x7f6e8c05f350 2023-11-28T02:53:47.136 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir6 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 msgr2=0x7f6e8c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 0x7f6e8c061810 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f6ea4001a10 tx=0x7f6ea4001560).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 msgr2=0x7f6ea80aad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 rx=0x7f6e9800e980 tx=0x7f6e98009060).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 shutdown_connections 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f350 0x7f6e8c061810 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea813b0b0 0x7f6ea80c9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80c6550 0x7f6ea812d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 --2- 172.21.15.73:0/4124920694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea813acd0 0x7f6ea80aad50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 >> 172.21.15.73:0/4124920694 conn(0x7f6ea80628f0 msgr2=0x7f6ea80bf8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:47.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 shutdown_connections 2023-11-28T02:53:47.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.136+0000 7f6eafd48700 1 -- 172.21.15.73:0/4124920694 wait complete. 2023-11-28T02:53:47.149 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-11-28T02:53:47.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:47.150 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-11-28T02:53:47.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir7 1M 2023-11-28T02:53:47.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.322+0000 7f1ec01d6700 1 Processor -- start 2023-11-28T02:53:47.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.322+0000 7f1ec01d6700 1 -- start start 2023-11-28T02:53:47.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ec01d6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1eb80c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ec01d6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb8158e20 con 0x7f1eb81373a0 2023-11-28T02:53:47.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ec01d6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1eb80c68a0 con 0x7f1eb8063af0 2023-11-28T02:53:47.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ec01d6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81582a0 con 0x7f1eb80c5f30 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ebdf72700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1eb81373a0 0x7f1eb80c72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43442/0 (socket says 172.21.15.73:43442) 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.323+0000 7f1ebdf72700 1 -- 172.21.15.73:0/607296730 learned_addr learned my addr 172.21.15.73:0/607296730 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015893849 0 0) 0x7f1eb8158e20 con 0x7f1eb81373a0 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebd771700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1eb80c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea40033f0 con 0x7f1eb81373a0 2023-11-28T02:53:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 97511701 0 0) 0x7f1ea40033f0 con 0x7f1eb81373a0 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 msgr2=0x7f1eb80c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1eb80c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 >> v1:172.21.15.150:6789/0 conn(0x7f1eb8063af0 legacy=0x7f1eb8063f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8158580 con 0x7f1eb81373a0 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f1eb4001470 con 0x7f1eb81373a0 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 >> v1:172.21.15.73:6789/0 conn(0x7f1eb81373a0 legacy=0x7f1eb80c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1ea4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1ea4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ea4004b80 0x7f1ea4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81582a0 con 0x7f1ea4003e00 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea400b8e0 con 0x7f1ea4004b80 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea400baa0 con 0x7f1eb80c5f30 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebd771700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1ea4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebdf72700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1ea4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebe773700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ea4004b80 0x7f1ea4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebd771700 1 -- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 msgr2=0x7f1ea4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.324+0000 7f1ebd771700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb80c5f30 0x7f1ea4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebd771700 1 -- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ea4004b80 msgr2=0x7f1ea4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebd771700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ea4004b80 0x7f1ea4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebd771700 1 -- 172.21.15.73:0/607296730 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea40033f0 con 0x7f1ea4003e00 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebd771700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1ea4004690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f1ea8009a80 tx=0x7f1ea8009060).ready entity=mon.0 client_cookie=a5e33b513b0c4fc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea8009430 con 0x7f1ea4003e00 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ea800e640 con 0x7f1ea4003e00 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebcf70700 1 -- 172.21.15.73:0/607296730 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ea800e9c0 con 0x7f1ea4003e00 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 -- 172.21.15.73:0/607296730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 msgr2=0x7f1ea4004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/607296730 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1ea4004690 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 rx=0x7f1ea8009a80 tx=0x7f1ea8009060).stop 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ebe773700 1 -- 172.21.15.73:0/607296730 reap_dead start 2023-11-28T02:53:47.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 -- 172.21.15.73:0/607296730 shutdown_connections 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 -- 172.21.15.73:0/607296730 >> 172.21.15.73:0/607296730 conn(0x7f1eb80628f0 msgr2=0x7f1eb8062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 -- 172.21.15.73:0/607296730 shutdown_connections 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.325+0000 7f1ec01d6700 1 -- 172.21.15.73:0/607296730 wait complete. 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.326+0000 7f1ec01d6700 1 Processor -- start 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.326+0000 7f1ec01d6700 1 -- start start 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 0x7f1eb8138fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb8165370 0x7f1eb80bf1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb81542a0 con 0x7f1ea4003e00 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8139a20 con 0x7f1eb8165370 2023-11-28T02:53:47.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ec01d6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1eb8139d00 con 0x7f1eb8152620 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ebdf72700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ebdf72700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36594/0 (socket says 172.21.15.73:36594) 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ebdf72700 1 -- 172.21.15.73:0/1682740019 learned_addr learned my addr 172.21.15.73:0/1682740019 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ebd771700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 0x7f1eb8138fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.327+0000 7f1ebe773700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb8165370 0x7f1eb80bf1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 -- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 msgr2=0x7f1eb8138fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 0x7f1eb8138fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 -- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb8165370 msgr2=0x7f1eb80bf1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb8165370 0x7f1eb80bf1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 -- 172.21.15.73:0/1682740019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1eb8139fe0 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebd771700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 0x7f1eb8138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ebdf72700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1eb4004730 tx=0x7f1eb4001e70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011590 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb40116f0 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1eb813a2c0 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4011a70 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.328+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1eb81394c0 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.329+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1eb806c860 con 0x7f1ea4003e00 2023-11-28T02:53:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.329+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f1eb401d3b0 con 0x7f1ea4003e00 2023-11-28T02:53:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.329+0000 7f1eadffb700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 0x7f1e9c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.330+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f1eb4079e50 con 0x7f1ea4003e00 2023-11-28T02:53:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.330+0000 7f1ebd771700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 0x7f1e9c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.330+0000 7f1ebd771700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 0x7f1e9c061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1eb80ab520 tx=0x7f1ea801a000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.333+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1eb404aec0 con 0x7f1ea4003e00 2023-11-28T02:53:47.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.518+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1eb8136cf0 con 0x7f1e9c05f350 2023-11-28T02:53:47.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.521+0000 7f1eadffb700 1 -- 172.21.15.73:0/1682740019 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f1eb8136cf0 con 0x7f1e9c05f350 2023-11-28T02:53:47.523 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir7 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 msgr2=0x7f1e9c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 0x7f1e9c061810 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1eb80ab520 tx=0x7f1ea801a000).stop 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 msgr2=0x7f1eb81365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 rx=0x7f1eb4004730 tx=0x7f1eb4001e70).stop 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 shutdown_connections 2023-11-28T02:53:47.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e9c05f350 0x7f1e9c061810 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1eb8165370 0x7f1eb80bf1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ea4003e00 0x7f1eb81365c0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 --2- 172.21.15.73:0/1682740019 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1eb8152620 0x7f1eb8138fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 >> 172.21.15.73:0/1682740019 conn(0x7f1eb80628f0 msgr2=0x7f1eb8064070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 shutdown_connections 2023-11-28T02:53:47.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.523+0000 7f1ec01d6700 1 -- 172.21.15.73:0/1682740019 wait complete. 2023-11-28T02:53:47.536 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-11-28T02:53:47.536 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:47.537 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-11-28T02:53:47.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir8 1M 2023-11-28T02:53:47.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.709+0000 7fa6fe03b700 1 Processor -- start 2023-11-28T02:53:47.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.709+0000 7fa6fe03b700 1 -- start start 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6fe03b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6f8154910 0x7fa6f80a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6fe03b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6f80aa220 0x7fa6f80aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6fe03b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6f80c6630 con 0x7fa6f8154530 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6fe03b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80c6390 con 0x7fa6f80aa220 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6fe03b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6f80a84e0 con 0x7fa6f8154910 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6f80aa220 0x7fa6f80aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6f80aa220 0x7fa6f80aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36604/0 (socket says 172.21.15.73:36604) 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3055473491 learned_addr learned my addr 172.21.15.73:0/3055473491 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f6ffd700 1 --2- 172.21.15.73:0/3055473491 >> v2:172.21.15.150:3300/0 conn(0x7fa6f8154910 0x7fa6f80a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3055473491 >> v2:172.21.15.150:3300/0 conn(0x7fa6f8154910 msgr2=0x7fa6f80a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 --2- 172.21.15.73:0/3055473491 >> v2:172.21.15.150:3300/0 conn(0x7fa6f8154910 0x7fa6f80a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3055473491 >> v1:172.21.15.150:6789/0 conn(0x7fa6f8154530 legacy=0x7fa6f80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3055473491 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f80aab80 con 0x7fa6f80aa220 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f7fff700 1 --2- 172.21.15.73:0/3055473491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f80aa220 0x7fa6f80aa690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fa6f80adfd0 tx=0x7fa6f00090f0).ready entity=mon.0 client_cookie=9ebc7bc0d5d1f728 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f67fc700 1 -- 172.21.15.73:0/3055473491 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000b600 con 0x7fa6f80aa220 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f67fc700 1 -- 172.21.15.73:0/3055473491 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f000b760 con 0x7fa6f80aa220 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.710+0000 7fa6f67fc700 1 -- 172.21.15.73:0/3055473491 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000bae0 con 0x7fa6f80aa220 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3055473491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f80aa220 msgr2=0x7fa6f80aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3055473491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f80aa220 0x7fa6f80aa690 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 rx=0x7fa6f80adfd0 tx=0x7fa6f00090f0).stop 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3055473491 shutdown_connections 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3055473491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f80aa220 0x7fa6f80aa690 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3055473491 >> v2:172.21.15.150:3300/0 conn(0x7fa6f8154910 0x7fa6f80a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3055473491 >> 172.21.15.73:0/3055473491 conn(0x7fa6f8094930 msgr2=0x7fa6f806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:47.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3055473491 shutdown_connections 2023-11-28T02:53:47.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.711+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3055473491 wait complete. 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.712+0000 7fa6fe03b700 1 Processor -- start 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.712+0000 7fa6fe03b700 1 -- start start 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 0x7fa6f80acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6f8154530 0x7fa6f80aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80ad170 con 0x7fa6f8154910 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c51d0 con 0x7fa6f80aa220 2023-11-28T02:53:47.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6fe03b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6f80c4370 con 0x7fa6f8154530 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6f7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36610/0 (socket says 172.21.15.73:36610) 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3298004035 learned_addr learned my addr 172.21.15.73:0/3298004035 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6f77fe700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 0x7fa6f80acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.713+0000 7fa6f6ffd700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6f8154530 0x7fa6f80aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6f8154530 msgr2=0x7fa6f80aad60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6f8154530 0x7fa6f80aad60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 msgr2=0x7fa6f80acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 0x7fa6f80acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 -- 172.21.15.73:0/3298004035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6f0009370 con 0x7fa6f8154910 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f77fe700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 0x7fa6f80acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6f7fff700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa6f806ae10 tx=0x7fa6f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f000bae0 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f80ab250 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6f80c48a0 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f001c630 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.714+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f001c7a0 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.715+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6f806c860 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.715+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fa6f001cbd0 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.716+0000 7fa6df7fe700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 0x7fa6e4061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.716+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa6f007c8c0 con 0x7fa6f8154910 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.716+0000 7fa6f77fe700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 0x7fa6e4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:47.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.716+0000 7fa6f77fe700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 0x7fa6e4061810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa6e8007fd0 tx=0x7fa6e8009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:47.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.720+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6f004d960 con 0x7fa6f8154910 2023-11-28T02:53:47.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.907+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6f8064290 con 0x7fa6e405f350 2023-11-28T02:53:47.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.911+0000 7fa6df7fe700 1 -- 172.21.15.73:0/3298004035 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fa6f8064290 con 0x7fa6e405f350 2023-11-28T02:53:47.912 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir8 2023-11-28T02:53:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 msgr2=0x7fa6e4061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 0x7fa6e4061810 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fa6e8007fd0 tx=0x7fa6e8009040).stop 2023-11-28T02:53:47.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 msgr2=0x7fa6f814d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 rx=0x7fa6f806ae10 tx=0x7fa6f0017000).stop 2023-11-28T02:53:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 shutdown_connections 2023-11-28T02:53:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6e405f350 0x7fa6e4061810 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6f8154910 0x7fa6f814d8e0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6f80aa220 0x7fa6f80acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 --2- 172.21.15.73:0/3298004035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6f8154530 0x7fa6f80aad60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 >> 172.21.15.73:0/3298004035 conn(0x7fa6f8094930 msgr2=0x7fa6f814fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:47.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 shutdown_connections 2023-11-28T02:53:47.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:47.913+0000 7fa6fe03b700 1 -- 172.21.15.73:0/3298004035 wait complete. 2023-11-28T02:53:47.926 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-11-28T02:53:47.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T02:53:47.927 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-11-28T02:53:47.943 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/dir9 1M 2023-11-28T02:53:48.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.100+0000 7f0ad4ee4700 1 Processor -- start 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.100+0000 7f0ad4ee4700 1 -- start start 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.101+0000 7f0ad4ee4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0ad0154910 0x7f0ad00a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.101+0000 7f0ad4ee4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad00c6630 con 0x7f0ad00aa220 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.101+0000 7f0ad4ee4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ad0064230 con 0x7f0ad0154530 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.101+0000 7f0ad4ee4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ad00c6390 con 0x7f0ad0154910 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0aced9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0ad00aa220 0x7f0ad00c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43450/0 (socket says 172.21.15.73:43450) 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0aced9d700 1 -- 172.21.15.73:0/1881522533 learned_addr learned my addr 172.21.15.73:0/1881522533 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acdd9b700 1 --2- 172.21.15.73:0/1881522533 >> v2:172.21.15.150:3300/0 conn(0x7f0ad0154910 0x7f0ad00a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3656887752 0 0) 0x7f0ad00c6630 con 0x7f0ad00aa220 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ab80033f0 con 0x7f0ad00aa220 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376978762 0 0) 0x7f0ad0064230 con 0x7f0ad0154530 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ad00c6630 con 0x7f0ad0154530 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2058086979 0 0) 0x7f0ab80033f0 con 0x7f0ad00aa220 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 >> v2:172.21.15.150:3300/0 conn(0x7f0ad0154910 msgr2=0x7f0ad00a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 --2- 172.21.15.73:0/1881522533 >> v2:172.21.15.150:3300/0 conn(0x7f0ad0154910 0x7f0ad00a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 >> v1:172.21.15.150:6789/0 conn(0x7f0ad0154530 legacy=0x7f0ad00c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad0064510 con 0x7f0ad00aa220 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f0ac8001470 con 0x7f0ad00aa220 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 >> v1:172.21.15.73:6789/0 conn(0x7f0ad00aa220 legacy=0x7f0ad00c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 0x7f0ab8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8004690 0x7f0ab8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab8005000 0x7f0ab800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00c6390 con 0x7f0ab8003a70 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab800bf60 con 0x7f0ab8005000 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab800c0d0 con 0x7f0ab8004690 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 0x7f0ab8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0acdd9b700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8004690 0x7f0ab8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0aced9d700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab8005000 0x7f0ab800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 -- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8004690 msgr2=0x7f0ab8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8004690 0x7f0ab8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 -- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab8005000 msgr2=0x7f0ab800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ab8005000 0x7f0ab800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0ace59c700 1 -- 172.21.15.73:0/1881522533 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab80033f0 con 0x7f0ab8003a70 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.102+0000 7f0aced9d700 1 -- 172.21.15.73:0/1881522533 reap_dead start 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ace59c700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 0x7f0ab8003e70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f0ac0001f60 tx=0x7f0ac000c0e0).ready entity=mon.0 client_cookie=8f275b0784f7850d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0001970 con 0x7f0ab8003a70 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac0006a70 con 0x7f0ab8003a70 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0acd59a700 1 -- 172.21.15.73:0/1881522533 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac0006e10 con 0x7f0ab8003a70 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 msgr2=0x7f0ab8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 0x7f0ab8003e70 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 rx=0x7f0ac0001f60 tx=0x7f0ac000c0e0).stop 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/1881522533 shutdown_connections 2023-11-28T02:53:48.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/1881522533 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ab8003a70 0x7f0ab8003e70 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/1881522533 >> 172.21.15.73:0/1881522533 conn(0x7f0ad0094930 msgr2=0x7f0ad006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/1881522533 shutdown_connections 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.103+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/1881522533 wait complete. 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.104+0000 7f0ad4ee4700 1 Processor -- start 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.104+0000 7f0ad4ee4700 1 -- start start 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8003a70 0x7f0ad00c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 0x7f0ad014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00abf40 con 0x7f0ad00aac50 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad014d9a0 con 0x7f0ad014d030 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ad4ee4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ad00c5270 con 0x7f0ab8003a70 2023-11-28T02:53:48.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0acdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0acdd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36622/0 (socket says 172.21.15.73:36622) 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0acdd9b700 1 -- 172.21.15.73:0/399873338 learned_addr learned my addr 172.21.15.73:0/399873338 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0ace59c700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8003a70 0x7f0ad00c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0aced9d700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 0x7f0ad014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0acdd9b700 1 -- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8003a70 msgr2=0x7f0ad00c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.105+0000 7f0acdd9b700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8003a70 0x7f0ad00c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0acdd9b700 1 -- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 msgr2=0x7f0ad014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0acdd9b700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 0x7f0ad014d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0acdd9b700 1 -- 172.21.15.73:0/399873338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad00c3cd0 con 0x7f0ad00aac50 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0aced9d700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 0x7f0ad014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0acdd9b700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0ac40116e0 tx=0x7f0ac4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4013620 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ac4013780 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ac4013b00 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ad00c4660 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.106+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ad00c4c20 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.107+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ad006c860 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.107+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f0ac4015480 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.108+0000 7f0abe7fc700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 0x7f0ab0061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.108+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f0ac407ac80 con 0x7f0ad00aac50 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.108+0000 7f0ace59c700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 0x7f0ab0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.108+0000 7f0ace59c700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 0x7f0ab0061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f0ac000c120 tx=0x7f0ac0016000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.112+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0ac404bc40 con 0x7f0ad00aac50 2023-11-28T02:53:48.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.298+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0ad00647f0 con 0x7f0ab005f350 2023-11-28T02:53:48.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.302+0000 7f0abe7fc700 1 -- 172.21.15.73:0/399873338 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f0ad00647f0 con 0x7f0ab005f350 2023-11-28T02:53:48.303 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/dir9 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.303+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 msgr2=0x7f0ab0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.303+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 0x7f0ab0061810 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7f0ac000c120 tx=0x7f0ac0016000).stop 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.303+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 msgr2=0x7f0ad00aba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.303+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 rx=0x7f0ac40116e0 tx=0x7f0ac4008040).stop 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 shutdown_connections 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ab005f350 0x7f0ab0061810 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ad014d030 0x7f0ad014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ab8003a70 0x7f0ad00c3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 --2- 172.21.15.73:0/399873338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ad00aac50 0x7f0ad00aba50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 >> 172.21.15.73:0/399873338 conn(0x7f0ad0094930 msgr2=0x7f0ad00c05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 shutdown_connections 2023-11-28T02:53:48.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.304+0000 7f0ad4ee4700 1 -- 172.21.15.73:0/399873338 wait complete. 2023-11-28T02:53:48.317 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-11-28T02:53:48.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir0 1M 2023-11-28T02:53:48.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.434+0000 7f51025e7700 1 Processor -- start 2023-11-28T02:53:48.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.434+0000 7f51025e7700 1 -- start start 2023-11-28T02:53:48.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f51025e7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50fc154530 0x7f50fc154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f51025e7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50fc0a7ff0 0x7f50fc0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f51025e7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50fc0c6630 con 0x7f50fc0aa220 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f51025e7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc0c6390 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f51025e7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50fc0a8470 con 0x7f50fc154530 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f50fb7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50fc0a7ff0 0x7f50fc0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f50fb7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f50fc0a7ff0 0x7f50fc0c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36634/0 (socket says 172.21.15.73:36634) 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f50fb7fe700 1 -- 172.21.15.73:0/3884210944 learned_addr learned my addr 172.21.15.73:0/3884210944 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.435+0000 7f50fbfff700 1 --2- 172.21.15.73:0/3884210944 >> v2:172.21.15.150:3300/0 conn(0x7f50fc154530 0x7f50fc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50fb7fe700 1 -- 172.21.15.73:0/3884210944 >> v2:172.21.15.150:3300/0 conn(0x7f50fc154530 msgr2=0x7f50fc154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50fb7fe700 1 --2- 172.21.15.73:0/3884210944 >> v2:172.21.15.150:3300/0 conn(0x7f50fc154530 0x7f50fc154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50fb7fe700 1 -- 172.21.15.73:0/3884210944 >> v1:172.21.15.150:6789/0 conn(0x7f50fc0aa220 legacy=0x7f50fc0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50fb7fe700 1 -- 172.21.15.73:0/3884210944 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50fc064230 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50fb7fe700 1 --2- 172.21.15.73:0/3884210944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0a7ff0 0x7f50fc0c8580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f50fc0c9080 tx=0x7f50f00090f0).ready entity=mon.0 client_cookie=b75c2115a5929ba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50faffd700 1 -- 172.21.15.73:0/3884210944 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000b600 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50faffd700 1 -- 172.21.15.73:0/3884210944 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000b760 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f50faffd700 1 -- 172.21.15.73:0/3884210944 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000bae0 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 -- 172.21.15.73:0/3884210944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0a7ff0 msgr2=0x7f50fc0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 --2- 172.21.15.73:0/3884210944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0a7ff0 0x7f50fc0c8580 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 rx=0x7f50fc0c9080 tx=0x7f50f00090f0).stop 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 -- 172.21.15.73:0/3884210944 shutdown_connections 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 --2- 172.21.15.73:0/3884210944 >> v2:172.21.15.150:3300/0 conn(0x7f50fc154530 0x7f50fc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 --2- 172.21.15.73:0/3884210944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0a7ff0 0x7f50fc0c8580 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 -- 172.21.15.73:0/3884210944 >> 172.21.15.73:0/3884210944 conn(0x7f50fc094930 msgr2=0x7f50fc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 -- 172.21.15.73:0/3884210944 shutdown_connections 2023-11-28T02:53:48.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.436+0000 7f51025e7700 1 -- 172.21.15.73:0/3884210944 wait complete. 2023-11-28T02:53:48.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 Processor -- start 2023-11-28T02:53:48.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 -- start start 2023-11-28T02:53:48.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50fc0a7ff0 0x7f50fc0aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50fc154530 0x7f50fc0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0ad700 con 0x7f50fc0aa220 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0aae80 con 0x7f50fc154530 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.438+0000 7f51025e7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50fc0ad420 con 0x7f50fc0a7ff0 2023-11-28T02:53:48.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36648/0 (socket says 172.21.15.73:36648) 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 -- 172.21.15.73:0/1080046031 learned_addr learned my addr 172.21.15.73:0/1080046031 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fbfff700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50fc0a7ff0 0x7f50fc0aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f5100b84700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50fc154530 0x7f50fc0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 -- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50fc0a7ff0 msgr2=0x7f50fc0aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50fc0a7ff0 0x7f50fc0aca90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 -- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50fc154530 msgr2=0x7f50fc0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50fc154530 0x7f50fc0c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 -- 172.21.15.73:0/1080046031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f0009370 con 0x7f50fc0aa220 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50fb7fe700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f50f0009130 tx=0x7f50f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f000bea0 con 0x7f50fc0aa220 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f000b8d0 con 0x7f50fc0aa220 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50fc0aa830 con 0x7f50fc0aa220 2023-11-28T02:53:48.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f001c670 con 0x7f50fc0aa220 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.439+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50fc14f250 con 0x7f50fc0aa220 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.440+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50fc05f760 con 0x7f50fc0aa220 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.440+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f50f001c9e0 con 0x7f50fc0aa220 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.441+0000 7f50e3fff700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 0x7f50e8061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.441+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f50f007c250 con 0x7f50fc0aa220 2023-11-28T02:53:48.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.441+0000 7f50fbfff700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 0x7f50e8061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.441+0000 7f50fbfff700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 0x7f50e8061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f50fc099170 tx=0x7f50ec008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.445+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f50fc05f760 con 0x7f50fc0aa220 2023-11-28T02:53:48.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.636+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f50fc14f440 con 0x7f50e805f350 2023-11-28T02:53:48.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.641+0000 7f50e3fff700 1 -- 172.21.15.73:0/1080046031 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f50ec00b500 con 0x7f50e805f350 2023-11-28T02:53:48.642 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-11-28T02:53:48.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 msgr2=0x7f50e8061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 0x7f50e8061810 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f50fc099170 tx=0x7f50ec008040).stop 2023-11-28T02:53:48.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 msgr2=0x7f50fc0c2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 rx=0x7f50f0009130 tx=0x7f50f0017000).stop 2023-11-28T02:53:48.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 shutdown_connections 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f50e805f350 0x7f50e8061810 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50fc154530 0x7f50fc0c2810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50fc0a7ff0 0x7f50fc0aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 --2- 172.21.15.73:0/1080046031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50fc0aa220 0x7f50fc0c2320 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 >> 172.21.15.73:0/1080046031 conn(0x7f50fc094930 msgr2=0x7f50fc0c0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 shutdown_connections 2023-11-28T02:53:48.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.643+0000 7f51025e7700 1 -- 172.21.15.73:0/1080046031 wait complete. 2023-11-28T02:53:48.656 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-11-28T02:53:48.656 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir1 1M 2023-11-28T02:53:48.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.773+0000 7f9c5153d700 1 Processor -- start 2023-11-28T02:53:48.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.773+0000 7f9c5153d700 1 -- start start 2023-11-28T02:53:48.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c5153d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c5153d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c158d50 con 0x7f9c4c0c66d0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c5153d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c4c158140 con 0x7f9c4c1546b0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c5153d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c4c0c6b60 con 0x7f9c4c1542d0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36656/0 (socket says 172.21.15.73:36656) 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 -- 172.21.15.73:0/4116162424 learned_addr learned my addr 172.21.15.73:0/4116162424 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 -- 172.21.15.73:0/4116162424 >> v1:172.21.15.150:6789/0 conn(0x7f9c4c1546b0 legacy=0x7f9c4c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 -- 172.21.15.73:0/4116162424 >> v1:172.21.15.73:6789/0 conn(0x7f9c4c0c66d0 legacy=0x7f9c4c0c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 -- 172.21.15.73:0/4116162424 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c4c158420 con 0x7f9c4c1542d0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4bfff700 1 --2- 172.21.15.73:0/4116162424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9c3c006fd0 tx=0x7f9c3c0090f0).ready entity=mon.0 client_cookie=128c73ec3bebdc8b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.774+0000 7f9c4affd700 1 -- 172.21.15.73:0/4116162424 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00b600 con 0x7f9c4c1542d0 2023-11-28T02:53:48.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c4affd700 1 -- 172.21.15.73:0/4116162424 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3c00b760 con 0x7f9c4c1542d0 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c4affd700 1 -- 172.21.15.73:0/4116162424 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3c00bae0 con 0x7f9c4c1542d0 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 -- 172.21.15.73:0/4116162424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 msgr2=0x7f9c4c0a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 --2- 172.21.15.73:0/4116162424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 rx=0x7f9c3c006fd0 tx=0x7f9c3c0090f0).stop 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 -- 172.21.15.73:0/4116162424 shutdown_connections 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 --2- 172.21.15.73:0/4116162424 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c0a9f70 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 -- 172.21.15.73:0/4116162424 >> 172.21.15.73:0/4116162424 conn(0x7f9c4c0cb010 msgr2=0x7f9c4c0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 -- 172.21.15.73:0/4116162424 shutdown_connections 2023-11-28T02:53:48.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.775+0000 7f9c5153d700 1 -- 172.21.15.73:0/4116162424 wait complete. 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.776+0000 7f9c5153d700 1 Processor -- start 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.776+0000 7f9c5153d700 1 -- start start 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c4c0c66d0 0x7f9c4c1557f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4c1546b0 0x7f9c4c14ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c155ce0 con 0x7f9c4c1542d0 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0ae180 con 0x7f9c4c0c66d0 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c5153d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c4c0ae460 con 0x7f9c4c1546b0 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c4b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36670/0 (socket says 172.21.15.73:36670) 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c4b7fe700 1 -- 172.21.15.73:0/522244394 learned_addr learned my addr 172.21.15.73:0/522244394 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c4bfff700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c4c0c66d0 0x7f9c4c1557f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c4b7fe700 1 -- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4c1546b0 msgr2=0x7f9c4c14ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.777+0000 7f9c50d3c700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4c1546b0 0x7f9c4c14ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c4b7fe700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4c1546b0 0x7f9c4c14ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c4b7fe700 1 -- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c4c0c66d0 msgr2=0x7f9c4c1557f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c4b7fe700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c4c0c66d0 0x7f9c4c1557f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c4b7fe700 1 -- 172.21.15.73:0/522244394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c3c009370 con 0x7f9c4c1542d0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c4b7fe700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f9c4000c440 tx=0x7f9c40007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40007020 con 0x7f9c4c1542d0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c40007180 con 0x7f9c4c1542d0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c4c0ae740 con 0x7f9c4c1542d0 2023-11-28T02:53:48.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c40007520 con 0x7f9c4c1542d0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.778+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4c153750 con 0x7f9c4c1542d0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.779+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c4c168130 con 0x7f9c4c1542d0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.779+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f9c400068b0 con 0x7f9c4c1542d0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.779+0000 7f9c33fff700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 0x7f9c38061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.780+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9c40079900 con 0x7f9c4c1542d0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.780+0000 7f9c4bfff700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 0x7f9c38061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:48.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.780+0000 7f9c4bfff700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 0x7f9c38061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9c3c00b3c0 tx=0x7f9c3c017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:48.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.784+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9c40047af0 con 0x7f9c4c1542d0 2023-11-28T02:53:48.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.973+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c4c06c7c0 con 0x7f9c3805f350 2023-11-28T02:53:48.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.977+0000 7f9c33fff700 1 -- 172.21.15.73:0/522244394 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9c4c06c7c0 con 0x7f9c3805f350 2023-11-28T02:53:48.979 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 msgr2=0x7f9c38061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 0x7f9c38061810 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7f9c3c00b3c0 tx=0x7f9c3c017000).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 msgr2=0x7f9c4c153260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 rx=0x7f9c4000c440 tx=0x7f9c40007e00).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 shutdown_connections 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c3805f350 0x7f9c38061810 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c4c1546b0 0x7f9c4c14ced0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c4c0c66d0 0x7f9c4c1557f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 --2- 172.21.15.73:0/522244394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4c1542d0 0x7f9c4c153260 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 >> 172.21.15.73:0/522244394 conn(0x7f9c4c0cb010 msgr2=0x7f9c4c064880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:48.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 shutdown_connections 2023-11-28T02:53:48.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:48.979+0000 7f9c5153d700 1 -- 172.21.15.73:0/522244394 wait complete. 2023-11-28T02:53:48.992 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-11-28T02:53:48.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir2 1M 2023-11-28T02:53:49.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.109+0000 7f9acebca700 1 Processor -- start 2023-11-28T02:53:49.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.109+0000 7f9acebca700 1 -- start start 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acebca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac8154610 0x7f9ac81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acebca700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac8159210 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acebca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ac80c0c70 con 0x7f9ac80ab3b0 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acebca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ac8158970 con 0x7f9ac8154610 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acc966700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac8154610 0x7f9ac81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acc966700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9ac8154610 0x7f9ac81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36680/0 (socket says 172.21.15.73:36680) 2023-11-28T02:53:49.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acd167700 1 --1- 172.21.15.73:0/948778472 >> v1:172.21.15.73:6789/0 conn(0x7f9ac80aa1c0 0x7f9ac80c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43466/0 (socket says 172.21.15.73:43466) 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.110+0000 7f9acc966700 1 -- 172.21.15.73:0/948778472 learned_addr learned my addr 172.21.15.73:0/948778472 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acc966700 1 -- 172.21.15.73:0/948778472 >> v1:172.21.15.150:6789/0 conn(0x7f9ac80ab3b0 legacy=0x7f9ac80c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acc966700 1 -- 172.21.15.73:0/948778472 >> v1:172.21.15.73:6789/0 conn(0x7f9ac80aa1c0 legacy=0x7f9ac80c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9abf7fe700 1 -- 172.21.15.73:0/948778472 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2657928319 0 0) 0x7f9ac8159210 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acc966700 1 -- 172.21.15.73:0/948778472 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac8158c50 con 0x7f9ac8154610 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acc966700 1 --2- 172.21.15.73:0/948778472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac8154610 0x7f9ac81582e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f9ac4009f20 tx=0x7f9ac4009840).ready entity=mon.0 client_cookie=530e548757672176 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9abf7fe700 1 -- 172.21.15.73:0/948778472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400e070 con 0x7f9ac8154610 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9abf7fe700 1 -- 172.21.15.73:0/948778472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac400ba80 con 0x7f9ac8154610 2023-11-28T02:53:49.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9abf7fe700 1 -- 172.21.15.73:0/948778472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400be00 con 0x7f9ac8154610 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acebca700 1 -- 172.21.15.73:0/948778472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac8154610 msgr2=0x7f9ac81582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acebca700 1 --2- 172.21.15.73:0/948778472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac8154610 0x7f9ac81582e0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 rx=0x7f9ac4009f20 tx=0x7f9ac4009840).stop 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acebca700 1 -- 172.21.15.73:0/948778472 shutdown_connections 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acebca700 1 --2- 172.21.15.73:0/948778472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac8154610 0x7f9ac81582e0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.111+0000 7f9acebca700 1 -- 172.21.15.73:0/948778472 >> 172.21.15.73:0/948778472 conn(0x7f9ac80655a0 msgr2=0x7f9ac8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.112+0000 7f9acebca700 1 -- 172.21.15.73:0/948778472 shutdown_connections 2023-11-28T02:53:49.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.112+0000 7f9acebca700 1 -- 172.21.15.73:0/948778472 wait complete. 2023-11-28T02:53:49.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.113+0000 7f9acebca700 1 Processor -- start 2023-11-28T02:53:49.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.113+0000 7f9acebca700 1 -- start start 2023-11-28T02:53:49.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.113+0000 7f9acebca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.113+0000 7f9acebca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ac8154610 0x7f9ac80c1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.113+0000 7f9acebca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ac80c1b20 0x7f9ac8156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acebca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80af480 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acebca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80c1fa0 con 0x7f9ac8154610 2023-11-28T02:53:49.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acebca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ac80ae870 con 0x7f9ac80c1b20 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36688/0 (socket says 172.21.15.73:36688) 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 -- 172.21.15.73:0/3780549068 learned_addr learned my addr 172.21.15.73:0/3780549068 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9abffff700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ac8154610 0x7f9ac80c1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acd167700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ac80c1b20 0x7f9ac8156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 -- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ac80c1b20 msgr2=0x7f9ac8156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ac80c1b20 0x7f9ac8156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 -- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ac8154610 msgr2=0x7f9ac80c1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ac8154610 0x7f9ac80c1630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 -- 172.21.15.73:0/3780549068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009ae0 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acc966700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f9ac80cc2d0 tx=0x7f9ac4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac400be00 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac401c400 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ac80aeb50 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac401c7a0 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.114+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ac80af110 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.115+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f9ac401cb90 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.115+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ac806c860 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.116+0000 7f9abcff9700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 0x7f9ab4061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.116+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9ac407bf90 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.116+0000 7f9abffff700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 0x7f9ab4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.116+0000 7f9abffff700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 0x7f9ab4061810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9ac8153e50 tx=0x7f9ab8009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.120+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ac404cea0 con 0x7f9ac80aa1c0 2023-11-28T02:53:49.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.309+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9ac80af5c0 con 0x7f9ab405f350 2023-11-28T02:53:49.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.312+0000 7f9abcff9700 1 -- 172.21.15.73:0/3780549068 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f9ab8001240 con 0x7f9ab405f350 2023-11-28T02:53:49.314 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 msgr2=0x7f9ab4061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 0x7f9ab4061810 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f9ac8153e50 tx=0x7f9ab8009040).stop 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 msgr2=0x7f9ac8144d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 rx=0x7f9ac80cc2d0 tx=0x7f9ac4009840).stop 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 shutdown_connections 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9ab405f350 0x7f9ab4061810 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ac80c1b20 0x7f9ac8156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ac80aa1c0 0x7f9ac8144d80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 --2- 172.21.15.73:0/3780549068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ac8154610 0x7f9ac80c1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 >> 172.21.15.73:0/3780549068 conn(0x7f9ac80655a0 msgr2=0x7f9ac814e650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 shutdown_connections 2023-11-28T02:53:49.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.314+0000 7f9acebca700 1 -- 172.21.15.73:0/3780549068 wait complete. 2023-11-28T02:53:49.327 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-11-28T02:53:49.328 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir3 1M 2023-11-28T02:53:49.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.444+0000 7f4a84e80700 1 Processor -- start 2023-11-28T02:53:49.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.444+0000 7f4a84e80700 1 -- start start 2023-11-28T02:53:49.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a84e80700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a80154910 0x7f4a800a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a84e80700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a800c6630 con 0x7f4a800aa220 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a84e80700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a80064230 con 0x7f4a80154530 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a84e80700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a800c6390 con 0x7f4a80154910 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a80154910 0x7f4a800a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a80154910 0x7f4a800a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36692/0 (socket says 172.21.15.73:36692) 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 -- 172.21.15.73:0/631806114 learned_addr learned my addr 172.21.15.73:0/631806114 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 -- 172.21.15.73:0/631806114 >> v1:172.21.15.150:6789/0 conn(0x7f4a80154530 legacy=0x7f4a800c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 -- 172.21.15.73:0/631806114 >> v1:172.21.15.73:6789/0 conn(0x7f4a800aa220 legacy=0x7f4a800c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.445+0000 7f4a7dd9b700 1 -- 172.21.15.73:0/631806114 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a80064510 con 0x7f4a80154910 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a7dd9b700 1 --2- 172.21.15.73:0/631806114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800a7ff0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4a74006fd0 tx=0x7f4a740090f0).ready entity=mon.0 client_cookie=51010360ef06f6ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a7d59a700 1 -- 172.21.15.73:0/631806114 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400b600 con 0x7f4a80154910 2023-11-28T02:53:49.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a7d59a700 1 -- 172.21.15.73:0/631806114 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7400b760 con 0x7f4a80154910 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a7d59a700 1 -- 172.21.15.73:0/631806114 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7400bae0 con 0x7f4a80154910 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 -- 172.21.15.73:0/631806114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 msgr2=0x7f4a800a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 --2- 172.21.15.73:0/631806114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800a7ff0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 rx=0x7f4a74006fd0 tx=0x7f4a740090f0).stop 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 -- 172.21.15.73:0/631806114 shutdown_connections 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 --2- 172.21.15.73:0/631806114 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800a7ff0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 -- 172.21.15.73:0/631806114 >> 172.21.15.73:0/631806114 conn(0x7f4a80094930 msgr2=0x7f4a8006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 -- 172.21.15.73:0/631806114 shutdown_connections 2023-11-28T02:53:49.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.446+0000 7f4a84e80700 1 -- 172.21.15.73:0/631806114 wait complete. 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.447+0000 7f4a84e80700 1 Processor -- start 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.447+0000 7f4a84e80700 1 -- start start 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 0x7f4a80151730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 0x7f4a800c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800c45e0 con 0x7f4a80154910 2023-11-28T02:53:49.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a800ac530 con 0x7f4a80154530 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a84e80700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a80168d60 con 0x7f4a800aa220 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a7ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a7ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36696/0 (socket says 172.21.15.73:36696) 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.448+0000 7f4a7ed9d700 1 -- 172.21.15.73:0/2450263629 learned_addr learned my addr 172.21.15.73:0/2450263629 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7e59c700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 0x7f4a80151730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7dd9b700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 0x7f4a800c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 -- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 msgr2=0x7f4a80151730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 0x7f4a80151730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 -- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 msgr2=0x7f4a800c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 0x7f4a800c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 -- 172.21.15.73:0/2450263629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a74009370 con 0x7f4a80154910 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7dd9b700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 0x7f4a800c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:49.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7e59c700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 0x7f4a80151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:49.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a7ed9d700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f4a7800c470 tx=0x7f4a78007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800f470 con 0x7f4a80154910 2023-11-28T02:53:49.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a78001e90 con 0x7f4a80154910 2023-11-28T02:53:49.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800f7e0 con 0x7f4a80154910 2023-11-28T02:53:49.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a80151c20 con 0x7f4a80154910 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.449+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a80168280 con 0x7f4a80154910 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.450+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8006c860 con 0x7f4a80154910 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.450+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f4a7800fbd0 con 0x7f4a80154910 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.450+0000 7f4a6e7fc700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 0x7f4a60061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.451+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f4a780798c0 con 0x7f4a80154910 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.451+0000 7f4a7e59c700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 0x7f4a60061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.451+0000 7f4a7e59c700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 0x7f4a60061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4a70002e30 tx=0x7f4a70009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.454+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a78047af0 con 0x7f4a80154910 2023-11-28T02:53:49.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.642+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a80063050 con 0x7f4a6005f350 2023-11-28T02:53:49.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.645+0000 7f4a6e7fc700 1 -- 172.21.15.73:0/2450263629 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4a80063050 con 0x7f4a6005f350 2023-11-28T02:53:49.646 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-11-28T02:53:49.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 msgr2=0x7f4a60061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 0x7f4a60061810 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f4a70002e30 tx=0x7f4a70009040).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 msgr2=0x7f4a800c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 rx=0x7f4a7800c470 tx=0x7f4a78007d70).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 shutdown_connections 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a6005f350 0x7f4a60061810 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a80154910 0x7f4a800c40f0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a800aa220 0x7f4a80151730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 --2- 172.21.15.73:0/2450263629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a80154530 0x7f4a800c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 >> 172.21.15.73:0/2450263629 conn(0x7f4a80094930 msgr2=0x7f4a80153430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 shutdown_connections 2023-11-28T02:53:49.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.647+0000 7f4a84e80700 1 -- 172.21.15.73:0/2450263629 wait complete. 2023-11-28T02:53:49.660 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-11-28T02:53:49.660 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir4 1M 2023-11-28T02:53:49.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.778+0000 7f3a422ed700 1 Processor -- start 2023-11-28T02:53:49.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.778+0000 7f3a422ed700 1 -- start start 2023-11-28T02:53:49.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a422ed700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a3c13b0b0 0x7f3a3c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a422ed700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c156d50 con 0x7f3a3c13acd0 2023-11-28T02:53:49.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a422ed700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a3c0c69a0 con 0x7f3a3c0c6550 2023-11-28T02:53:49.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a422ed700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3c156140 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a3bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3a3c13acd0 0x7f3a3c13b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43478/0 (socket says 172.21.15.73:43478) 2023-11-28T02:53:49.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a3bfff700 1 -- 172.21.15.73:0/547637225 learned_addr learned my addr 172.21.15.73:0/547637225 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:49.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.779+0000 7f3a3b7fe700 1 --2- 172.21.15.73:0/547637225 >> v2:172.21.15.73:3300/0 conn(0x7f3a3c13b0b0 0x7f3a3c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/547637225 >> v1:172.21.15.150:6789/0 conn(0x7f3a3c0c6550 legacy=0x7f3a3c0c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:49.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/547637225 >> v1:172.21.15.73:6789/0 conn(0x7f3a3c13acd0 legacy=0x7f3a3c13b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/547637225 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a3c156420 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3b7fe700 1 --2- 172.21.15.73:0/547637225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13b0b0 0x7f3a3c0c6060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f3a30009760 tx=0x7f3a30009060).ready entity=mon.0 client_cookie=b73a28c639b02604 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3affd700 1 -- 172.21.15.73:0/547637225 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a300090d0 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3affd700 1 -- 172.21.15.73:0/547637225 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3000b520 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a3affd700 1 -- 172.21.15.73:0/547637225 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000b8a0 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 -- 172.21.15.73:0/547637225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13b0b0 msgr2=0x7f3a3c0c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 --2- 172.21.15.73:0/547637225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13b0b0 0x7f3a3c0c6060 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 rx=0x7f3a30009760 tx=0x7f3a30009060).stop 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 -- 172.21.15.73:0/547637225 shutdown_connections 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 --2- 172.21.15.73:0/547637225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13b0b0 0x7f3a3c0c6060 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 -- 172.21.15.73:0/547637225 >> 172.21.15.73:0/547637225 conn(0x7f3a3c0628f0 msgr2=0x7f3a3c062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 -- 172.21.15.73:0/547637225 shutdown_connections 2023-11-28T02:53:49.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.780+0000 7f3a422ed700 1 -- 172.21.15.73:0/547637225 wait complete. 2023-11-28T02:53:49.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.781+0000 7f3a422ed700 1 Processor -- start 2023-11-28T02:53:49.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 -- start start 2023-11-28T02:53:49.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0c6550 0x7f3a3c167f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c13b0b0 0x7f3a3c0aa940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c1688f0 con 0x7f3a3c13acd0 2023-11-28T02:53:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c065150 con 0x7f3a3c0c6550 2023-11-28T02:53:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a422ed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a3c166290 con 0x7f3a3c13b0b0 2023-11-28T02:53:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.782+0000 7f3a3b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36714/0 (socket says 172.21.15.73:36714) 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/710620088 learned_addr learned my addr 172.21.15.73:0/710620088 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a4088a700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c13b0b0 0x7f3a3c0aa940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3bfff700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0c6550 0x7f3a3c167f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c13b0b0 msgr2=0x7f3a3c0aa940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c13b0b0 0x7f3a3c0aa940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0c6550 msgr2=0x7f3a3c167f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0c6550 0x7f3a3c167f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 -- 172.21.15.73:0/710620088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a300092a0 con 0x7f3a3c13acd0 2023-11-28T02:53:49.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a3b7fe700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f3a30009eb0 tx=0x7f3a30017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3000b690 con 0x7f3a3c13acd0 2023-11-28T02:53:49.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a3000b7f0 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a3001b670 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a3c064e70 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.783+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a3c165f20 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.784+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3c06c860 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.784+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f3a3001b9e0 con 0x7f3a3c13acd0 2023-11-28T02:53:49.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.785+0000 7f3a23fff700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 0x7f3a28061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.785+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f3a3007bf80 con 0x7f3a3c13acd0 2023-11-28T02:53:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.785+0000 7f3a3bfff700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 0x7f3a28061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.785+0000 7f3a3bfff700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 0x7f3a28061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3a3c0c4240 tx=0x7f3a2c009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:49.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.789+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a3004cfa0 con 0x7f3a3c13acd0 2023-11-28T02:53:49.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.977+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a3c156700 con 0x7f3a2805f350 2023-11-28T02:53:49.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.981+0000 7f3a23fff700 1 -- 172.21.15.73:0/710620088 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f3a3c156700 con 0x7f3a2805f350 2023-11-28T02:53:49.982 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-11-28T02:53:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.982+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 msgr2=0x7f3a28061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.982+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 0x7f3a28061810 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f3a3c0c4240 tx=0x7f3a2c009040).stop 2023-11-28T02:53:49.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 msgr2=0x7f3a3c168400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 rx=0x7f3a30009eb0 tx=0x7f3a30017000).stop 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 shutdown_connections 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a2805f350 0x7f3a28061810 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3c13b0b0 0x7f3a3c0aa940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3c0c6550 0x7f3a3c167f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 --2- 172.21.15.73:0/710620088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3c13acd0 0x7f3a3c168400 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 >> 172.21.15.73:0/710620088 conn(0x7f3a3c0628f0 msgr2=0x7f3a3c139c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 shutdown_connections 2023-11-28T02:53:49.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:49.983+0000 7f3a422ed700 1 -- 172.21.15.73:0/710620088 wait complete. 2023-11-28T02:53:49.997 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-11-28T02:53:49.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir5 1M 2023-11-28T02:53:50.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.114+0000 7ff5aed0c700 1 Processor -- start 2023-11-28T02:53:50.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.115+0000 7ff5aed0c700 1 -- start start 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.115+0000 7ff5aed0c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff5a80c3c90 0x7ff5a80c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.115+0000 7ff5aed0c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80c2520 con 0x7ff5a8066a90 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.115+0000 7ff5aed0c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80bd430 con 0x7ff5a80be030 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.115+0000 7ff5aed0c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a80bdcd0 con 0x7ff5a80c3c90 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5ae50b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff5a8066a90 0x7ff5a813a390 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43482/0 (socket says 172.21.15.73:43482) 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5ae50b700 1 -- 172.21.15.73:0/1644694243 learned_addr learned my addr 172.21.15.73:0/1644694243 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5add0a700 1 --2- 172.21.15.73:0/1644694243 >> v2:172.21.15.150:3300/0 conn(0x7ff5a80c3c90 0x7ff5a80c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197735183 0 0) 0x7ff5a80c2520 con 0x7ff5a8066a90 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5940033f0 con 0x7ff5a8066a90 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1396126644 0 0) 0x7ff5a80bd430 con 0x7ff5a80be030 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff5a80c2520 con 0x7ff5a80be030 2023-11-28T02:53:50.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4107551195 0 0) 0x7ff5940033f0 con 0x7ff5a8066a90 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 >> v2:172.21.15.150:3300/0 conn(0x7ff5a80c3c90 msgr2=0x7ff5a80c4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 --2- 172.21.15.73:0/1644694243 >> v2:172.21.15.150:3300/0 conn(0x7ff5a80c3c90 0x7ff5a80c4070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 >> v1:172.21.15.150:6789/0 conn(0x7ff5a80be030 legacy=0x7ff5a80bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80bd710 con 0x7ff5a8066a90 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff5a00017f0 con 0x7ff5a8066a90 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 >> v1:172.21.15.73:6789/0 conn(0x7ff5a8066a90 legacy=0x7ff5a813a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594003a70 0x7ff594003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 0x7ff594004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff594005000 0x7ff59400daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80bdcd0 con 0x7ff594004690 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59400e170 con 0x7ff594005000 2023-11-28T02:53:50.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff59400e2e0 con 0x7ff594003a70 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5ad509700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 0x7ff594004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5add0a700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594003a70 0x7ff594003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.116+0000 7ff5ae50b700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff594005000 0x7ff59400daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 -- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594003a70 msgr2=0x7ff594003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594003a70 0x7ff594003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 -- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff594005000 msgr2=0x7ff59400daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff594005000 0x7ff59400daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ae50b700 1 -- 172.21.15.73:0/1644694243 reap_dead start 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 -- 172.21.15.73:0/1644694243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5940033f0 con 0x7ff594004690 2023-11-28T02:53:50.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5ad509700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 0x7ff594004b10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7ff598001860 tx=0x7ff598007bb0).ready entity=mon.0 client_cookie=67fd6d6e3e429eb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598007020 con 0x7ff594004690 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598007180 con 0x7ff594004690 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5acd08700 1 -- 172.21.15.73:0/1644694243 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598007500 con 0x7ff594004690 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 -- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 msgr2=0x7ff594004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 0x7ff594004b10 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 rx=0x7ff598001860 tx=0x7ff598007bb0).stop 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 -- 172.21.15.73:0/1644694243 shutdown_connections 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/1644694243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff594004690 0x7ff594004b10 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 -- 172.21.15.73:0/1644694243 >> 172.21.15.73:0/1644694243 conn(0x7ff5a80642e0 msgr2=0x7ff5a80c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 -- 172.21.15.73:0/1644694243 shutdown_connections 2023-11-28T02:53:50.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.117+0000 7ff5aed0c700 1 -- 172.21.15.73:0/1644694243 wait complete. 2023-11-28T02:53:50.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 Processor -- start 2023-11-28T02:53:50.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 -- start start 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594004690 0x7ff5a81355f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a80c1980 0x7ff5a80ac6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80aa620 con 0x7ff5a80aaca0 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80ab130 con 0x7ff5a80c1980 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5aed0c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80ab410 con 0x7ff594004690 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5ad509700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5ad509700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36722/0 (socket says 172.21.15.73:36722) 2023-11-28T02:53:50.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5ad509700 1 -- 172.21.15.73:0/273837038 learned_addr learned my addr 172.21.15.73:0/273837038 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5add0a700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594004690 0x7ff5a81355f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.119+0000 7ff5ae50b700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a80c1980 0x7ff5a80ac6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 -- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594004690 msgr2=0x7ff5a81355f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594004690 0x7ff5a81355f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 -- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a80c1980 msgr2=0x7ff5a80ac6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a80c1980 0x7ff5a80ac6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 -- 172.21.15.73:0/273837038 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a80ab6f0 con 0x7ff5a80aaca0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5ad509700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7ff5940045c0 tx=0x7ff598015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598012610 con 0x7ff5a80aaca0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff598012770 con 0x7ff5a80aaca0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff598012900 con 0x7ff5a80aaca0 2023-11-28T02:53:50.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a80ab9d0 con 0x7ff5a80aaca0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.120+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a8135ae0 con 0x7ff5a80aaca0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.121+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a806c7c0 con 0x7ff5a80aaca0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.121+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7ff598018020 con 0x7ff5a80aaca0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.122+0000 7ff59dffb700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 0x7ff58c061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.122+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff59807a860 con 0x7ff5a80aaca0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.122+0000 7ff5add0a700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 0x7ff58c061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.122+0000 7ff5add0a700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 0x7ff58c061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff5a4006d80 tx=0x7ff5a4009060).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.126+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff59804b8d0 con 0x7ff5a80aaca0 2023-11-28T02:53:50.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.315+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff5a80bd9f0 con 0x7ff58c05f350 2023-11-28T02:53:50.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.318+0000 7ff59dffb700 1 -- 172.21.15.73:0/273837038 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7ff5a80bd9f0 con 0x7ff58c05f350 2023-11-28T02:53:50.320 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 msgr2=0x7ff58c061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 0x7ff58c061810 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7ff5a4006d80 tx=0x7ff5a4009060).stop 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 msgr2=0x7ff5a80ac1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7ff5940045c0 tx=0x7ff598015000).stop 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 shutdown_connections 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58c05f350 0x7ff58c061810 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a80c1980 0x7ff5a80ac6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff594004690 0x7ff5a81355f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 --2- 172.21.15.73:0/273837038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a80aaca0 0x7ff5a80ac1d0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 >> 172.21.15.73:0/273837038 conn(0x7ff5a80642e0 msgr2=0x7ff5a80be510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 shutdown_connections 2023-11-28T02:53:50.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.320+0000 7ff5aed0c700 1 -- 172.21.15.73:0/273837038 wait complete. 2023-11-28T02:53:50.333 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-11-28T02:53:50.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir6 1M 2023-11-28T02:53:50.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.449+0000 7f2269937700 1 Processor -- start 2023-11-28T02:53:50.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.449+0000 7f2269937700 1 -- start start 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2269937700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2264154910 0x7f22640a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2269937700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f22640aa220 0x7f22640aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2269937700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22640c6630 con 0x7f2264154530 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2269937700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640c6390 con 0x7f2264154910 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2269937700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22640a84e0 con 0x7f22640aa220 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2262ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2264154530 0x7f22640c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43486/0 (socket says 172.21.15.73:43486) 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f2262ffd700 1 -- 172.21.15.73:0/3543244877 learned_addr learned my addr 172.21.15.73:0/3543244877 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3543244877 >> v2:172.21.15.73:3300/0 conn(0x7f2264154910 0x7f22640a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f225a7fc700 1 -- 172.21.15.73:0/3543244877 >> v2:172.21.15.150:3300/0 conn(0x7f22640aa220 msgr2=0x7f22640aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3543244877 >> v2:172.21.15.150:3300/0 conn(0x7f22640aa220 0x7f22640aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:50.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f225a7fc700 1 -- 172.21.15.73:0/3543244877 >> v1:172.21.15.73:6789/0 conn(0x7f2264154530 legacy=0x7f22640c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f22627fc700 1 -- 172.21.15.73:0/3543244877 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2203719469 0 0) 0x7f22640c6630 con 0x7f2264154530 2023-11-28T02:53:50.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.450+0000 7f225a7fc700 1 -- 172.21.15.73:0/3543244877 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22640aab80 con 0x7f2264154910 2023-11-28T02:53:50.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3543244877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154910 0x7f22640a7ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2264084a60 tx=0x7f224c009060).ready entity=mon.0 client_cookie=1ba4ce39d32ba8c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f22627fc700 1 -- 172.21.15.73:0/3543244877 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00b580 con 0x7f2264154910 2023-11-28T02:53:50.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f22627fc700 1 -- 172.21.15.73:0/3543244877 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c00b6e0 con 0x7f2264154910 2023-11-28T02:53:50.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f22627fc700 1 -- 172.21.15.73:0/3543244877 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00ba60 con 0x7f2264154910 2023-11-28T02:53:50.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 -- 172.21.15.73:0/3543244877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154910 msgr2=0x7f22640a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 --2- 172.21.15.73:0/3543244877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154910 0x7f22640a7ff0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 rx=0x7f2264084a60 tx=0x7f224c009060).stop 2023-11-28T02:53:50.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 -- 172.21.15.73:0/3543244877 shutdown_connections 2023-11-28T02:53:50.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 --2- 172.21.15.73:0/3543244877 >> v2:172.21.15.150:3300/0 conn(0x7f22640aa220 0x7f22640aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T02:53:50.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 --2- 172.21.15.73:0/3543244877 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154910 0x7f22640a7ff0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 -- 172.21.15.73:0/3543244877 >> 172.21.15.73:0/3543244877 conn(0x7f2264094930 msgr2=0x7f226406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 -- 172.21.15.73:0/3543244877 shutdown_connections 2023-11-28T02:53:50.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.451+0000 7f2269937700 1 -- 172.21.15.73:0/3543244877 wait complete. 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.452+0000 7f2269937700 1 Processor -- start 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 -- start start 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 0x7f2264146130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 0x7f22641478f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22640c4670 con 0x7f2264154530 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22640c3a60 con 0x7f22640aa220 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f2269937700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22640c3d40 con 0x7f2264154910 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f225a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f225a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36734/0 (socket says 172.21.15.73:36734) 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.453+0000 7f225a7fc700 1 -- 172.21.15.73:0/3141859594 learned_addr learned my addr 172.21.15.73:0/3141859594 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f2262ffd700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 0x7f2264146130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f22637fe700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 0x7f22641478f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 -- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 msgr2=0x7f22641478f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 0x7f22641478f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 -- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 msgr2=0x7f2264146130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 0x7f2264146130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 -- 172.21.15.73:0/3141859594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0092a0 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f2262ffd700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 0x7f2264146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f22637fe700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 0x7f22641478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225a7fc700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f226406ae10 tx=0x7f224c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00b850 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c00b9b0 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22640c4020 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c01b640 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.454+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2264150c70 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.455+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226405f760 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.455+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f224c01ba70 con 0x7f2264154530 2023-11-28T02:53:50.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.456+0000 7f225b7fe700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 0x7f2244061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.456+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f224c07bfd0 con 0x7f2264154530 2023-11-28T02:53:50.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.456+0000 7f2262ffd700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 0x7f2244061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.456+0000 7f2262ffd700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 0x7f2244061810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f22540016e0 tx=0x7f2254009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.460+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f226405f760 con 0x7f2264154530 2023-11-28T02:53:50.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.647+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f226405f760 con 0x7f224405f350 2023-11-28T02:53:50.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.651+0000 7f225b7fe700 1 -- 172.21.15.73:0/3141859594 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f226405f760 con 0x7f224405f350 2023-11-28T02:53:50.652 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-11-28T02:53:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 msgr2=0x7f2244061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 0x7f2244061810 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f22540016e0 tx=0x7f2254009040).stop 2023-11-28T02:53:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 msgr2=0x7f22640c4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 rx=0x7f226406ae10 tx=0x7f224c017000).stop 2023-11-28T02:53:50.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 shutdown_connections 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f224405f350 0x7f2244061810 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2264154910 0x7f22641478f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f22640aa220 0x7f2264146130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 --2- 172.21.15.73:0/3141859594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2264154530 0x7f22640c4b00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 >> 172.21.15.73:0/3141859594 conn(0x7f2264094930 msgr2=0x7f2264148fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 shutdown_connections 2023-11-28T02:53:50.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.653+0000 7f2269937700 1 -- 172.21.15.73:0/3141859594 wait complete. 2023-11-28T02:53:50.666 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-11-28T02:53:50.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir7 1M 2023-11-28T02:53:50.690 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:53:50.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:53:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.786+0000 7fa6156ca700 1 Processor -- start 2023-11-28T02:53:50.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.786+0000 7fa6156ca700 1 -- start start 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6156ca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa61014cda0 0x7fa610066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6156ca700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6100bcef0 0x7fa6100bd350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6156ca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa610152e10 con 0x7fa61014c9c0 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6156ca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6101530f0 con 0x7fa61014cda0 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6156ca700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6101533d0 con 0x7fa6100bcef0 2023-11-28T02:53:50.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6067fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa61014cda0 0x7fa610066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6067fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa61014cda0 0x7fa610066b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36742/0 (socket says 172.21.15.73:36742) 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.787+0000 7fa6067fc700 1 -- 172.21.15.73:0/2757487541 learned_addr learned my addr 172.21.15.73:0/2757487541 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa60f7fe700 1 --2- 172.21.15.73:0/2757487541 >> v2:172.21.15.150:3300/0 conn(0x7fa6100bcef0 0x7fa6100bd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6067fc700 1 -- 172.21.15.73:0/2757487541 >> v2:172.21.15.150:3300/0 conn(0x7fa6100bcef0 msgr2=0x7fa6100bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6067fc700 1 --2- 172.21.15.73:0/2757487541 >> v2:172.21.15.150:3300/0 conn(0x7fa6100bcef0 0x7fa6100bd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6067fc700 1 -- 172.21.15.73:0/2757487541 >> v1:172.21.15.150:6789/0 conn(0x7fa61014c9c0 legacy=0x7fa6100c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6067fc700 1 -- 172.21.15.73:0/2757487541 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6101536b0 con 0x7fa61014cda0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6067fc700 1 --2- 172.21.15.73:0/2757487541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014cda0 0x7fa610066b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fa6100aacd0 tx=0x7fa5f80090f0).ready entity=mon.0 client_cookie=22d7d0658206d4e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa60e7fc700 1 -- 172.21.15.73:0/2757487541 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800b600 con 0x7fa61014cda0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa60e7fc700 1 -- 172.21.15.73:0/2757487541 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f800b760 con 0x7fa61014cda0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa60e7fc700 1 -- 172.21.15.73:0/2757487541 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800bb00 con 0x7fa61014cda0 2023-11-28T02:53:50.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6156ca700 1 -- 172.21.15.73:0/2757487541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014cda0 msgr2=0x7fa610066b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.788+0000 7fa6156ca700 1 --2- 172.21.15.73:0/2757487541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014cda0 0x7fa610066b90 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 rx=0x7fa6100aacd0 tx=0x7fa5f80090f0).stop 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 -- 172.21.15.73:0/2757487541 shutdown_connections 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 --2- 172.21.15.73:0/2757487541 >> v2:172.21.15.150:3300/0 conn(0x7fa6100bcef0 0x7fa6100bd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 --2- 172.21.15.73:0/2757487541 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014cda0 0x7fa610066b90 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 -- 172.21.15.73:0/2757487541 >> 172.21.15.73:0/2757487541 conn(0x7fa610063810 msgr2=0x7fa610063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 -- 172.21.15.73:0/2757487541 shutdown_connections 2023-11-28T02:53:50.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.789+0000 7fa6156ca700 1 -- 172.21.15.73:0/2757487541 wait complete. 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.790+0000 7fa6156ca700 1 Processor -- start 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.790+0000 7fa6156ca700 1 -- start start 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100bcef0 0x7fa610164fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 0x7fa6100c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610149200 con 0x7fa61014c9c0 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa610165600 con 0x7fa61014cda0 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6101658e0 con 0x7fa6100bcef0 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36754/0 (socket says 172.21.15.73:36754) 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 -- 172.21.15.73:0/1238792364 learned_addr learned my addr 172.21.15.73:0/1238792364 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa60f7fe700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 0x7fa6100c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa60effd700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100bcef0 0x7fa610164fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 -- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100bcef0 msgr2=0x7fa610164fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100bcef0 0x7fa610164fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 -- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 msgr2=0x7fa6100c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 0x7fa6100c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 -- 172.21.15.73:0/1238792364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f8009370 con 0x7fa61014c9c0 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa60f7fe700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 0x7fa6100c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6067fc700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa6100637a0 tx=0x7fa5f8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f800bec0 con 0x7fa61014c9c0 2023-11-28T02:53:50.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5f800b8d0 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa610165bc0 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f801d640 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.791+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa610165ea0 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.792+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6100c2870 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.792+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fa5f801da70 con 0x7fa61014c9c0 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.793+0000 7fa6077fe700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 0x7fa5f0061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:50.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.793+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa5f807c210 con 0x7fa61014c9c0 2023-11-28T02:53:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.793+0000 7fa60effd700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 0x7fa5f0061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.793+0000 7fa60effd700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 0x7fa5f0061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa600003900 tx=0x7fa600000bc0).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:50.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.797+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6100c2870 con 0x7fa61014c9c0 2023-11-28T02:53:50.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.987+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6100c2870 con 0x7fa5f005f350 2023-11-28T02:53:50.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.990+0000 7fa6077fe700 1 -- 172.21.15.73:0/1238792364 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fa6100c2870 con 0x7fa5f005f350 2023-11-28T02:53:50.991 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-11-28T02:53:50.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 msgr2=0x7fa5f0061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 0x7fa5f0061810 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa600003900 tx=0x7fa600000bc0).stop 2023-11-28T02:53:50.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 msgr2=0x7fa610148d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:50.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 rx=0x7fa6100637a0 tx=0x7fa5f8017000).stop 2023-11-28T02:53:50.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 shutdown_connections 2023-11-28T02:53:50.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5f005f350 0x7fa5f0061810 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa61014cda0 0x7fa6100c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6100bcef0 0x7fa610164fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 --2- 172.21.15.73:0/1238792364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa61014c9c0 0x7fa610148d10 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 >> 172.21.15.73:0/1238792364 conn(0x7fa610063810 msgr2=0x7fa610149a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 shutdown_connections 2023-11-28T02:53:50.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:50.992+0000 7fa6156ca700 1 -- 172.21.15.73:0/1238792364 wait complete. 2023-11-28T02:53:51.005 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-11-28T02:53:51.006 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir8 1M 2023-11-28T02:53:51.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.163+0000 7efeee4bd700 1 Processor -- start 2023-11-28T02:53:51.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.163+0000 7efeee4bd700 1 -- start start 2023-11-28T02:53:51.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeee4bd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efee8154610 0x7efee81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeee4bd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efee80aa1c0 0x7efee80aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeee4bd700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efee8159210 con 0x7efee80ab3b0 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeee4bd700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80c0c70 con 0x7efee8154610 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeee4bd700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efee80c2110 con 0x7efee80aa1c0 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efee8154610 0x7efee81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efee8154610 0x7efee81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36760/0 (socket says 172.21.15.73:36760) 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 -- 172.21.15.73:0/3855978558 learned_addr learned my addr 172.21.15.73:0/3855978558 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efeeca5a700 1 --2- 172.21.15.73:0/3855978558 >> v2:172.21.15.150:3300/0 conn(0x7efee80aa1c0 0x7efee80aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 -- 172.21.15.73:0/3855978558 >> v2:172.21.15.150:3300/0 conn(0x7efee80aa1c0 msgr2=0x7efee80aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 --2- 172.21.15.73:0/3855978558 >> v2:172.21.15.150:3300/0 conn(0x7efee80aa1c0 0x7efee80aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 -- 172.21.15.73:0/3855978558 >> v1:172.21.15.150:6789/0 conn(0x7efee80ab3b0 legacy=0x7efee80c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 -- 172.21.15.73:0/3855978558 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee80aaaf0 con 0x7efee8154610 2023-11-28T02:53:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee7fff700 1 --2- 172.21.15.73:0/3855978558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee8154610 0x7efee81582e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7efee80cc270 tx=0x7efed80090f0).ready entity=mon.0 client_cookie=ec8ac7e7646d0430 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee6ffd700 1 -- 172.21.15.73:0/3855978558 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800b600 con 0x7efee8154610 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee6ffd700 1 -- 172.21.15.73:0/3855978558 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efed800b760 con 0x7efee8154610 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.164+0000 7efee6ffd700 1 -- 172.21.15.73:0/3855978558 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efed800bae0 con 0x7efee8154610 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 -- 172.21.15.73:0/3855978558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee8154610 msgr2=0x7efee81582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 --2- 172.21.15.73:0/3855978558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee8154610 0x7efee81582e0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 rx=0x7efee80cc270 tx=0x7efed80090f0).stop 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 -- 172.21.15.73:0/3855978558 shutdown_connections 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 --2- 172.21.15.73:0/3855978558 >> v2:172.21.15.150:3300/0 conn(0x7efee80aa1c0 0x7efee80aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 --2- 172.21.15.73:0/3855978558 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee8154610 0x7efee81582e0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 -- 172.21.15.73:0/3855978558 >> 172.21.15.73:0/3855978558 conn(0x7efee80655a0 msgr2=0x7efee8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 -- 172.21.15.73:0/3855978558 shutdown_connections 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.165+0000 7efeee4bd700 1 -- 172.21.15.73:0/3855978558 wait complete. 2023-11-28T02:53:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.166+0000 7efeee4bd700 1 Processor -- start 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.166+0000 7efeee4bd700 1 -- start start 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee80aa1c0 0x7efee8168b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efee8154610 0x7efee8156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80ae970 con 0x7efee80c2300 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efee80af800 con 0x7efee80aa1c0 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeee4bd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee8169020 con 0x7efee8154610 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36770/0 (socket says 172.21.15.73:36770) 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 -- 172.21.15.73:0/1297256268 learned_addr learned my addr 172.21.15.73:0/1297256268 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efee7fff700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee80aa1c0 0x7efee8168b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efee77fe700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efee8154610 0x7efee8156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 -- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efee8154610 msgr2=0x7efee8156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efee8154610 0x7efee8156d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 -- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee80aa1c0 msgr2=0x7efee8168b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee80aa1c0 0x7efee8168b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.167+0000 7efeeca5a700 1 -- 172.21.15.73:0/1297256268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efed8009370 con 0x7efee80c2300 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efeeca5a700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7efee0009fa0 tx=0x7efee0009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee000ebc0 con 0x7efee80c2300 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee000ed20 con 0x7efee80c2300 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efee80aebf0 con 0x7efee80c2300 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee0014610 con 0x7efee80c2300 2023-11-28T02:53:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.168+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efee80af1b0 con 0x7efee80c2300 2023-11-28T02:53:51.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.169+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efee806c860 con 0x7efee80c2300 2023-11-28T02:53:51.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.169+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7efee001e440 con 0x7efee80c2300 2023-11-28T02:53:51.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.169+0000 7efecffff700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 0x7efed4061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.169+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7efee007a9b0 con 0x7efee80c2300 2023-11-28T02:53:51.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.169+0000 7efee7fff700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 0x7efed4061810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.170+0000 7efee7fff700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 0x7efed4061810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efed8009130 tx=0x7efed8017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.173+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efee004b970 con 0x7efee80c2300 2023-11-28T02:53:51.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.362+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efee80c0540 con 0x7efed405f350 2023-11-28T02:53:51.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.365+0000 7efecffff700 1 -- 172.21.15.73:0/1297256268 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7efee80c0540 con 0x7efed405f350 2023-11-28T02:53:51.367 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-11-28T02:53:51.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 msgr2=0x7efed4061810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 0x7efed4061810 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7efed8009130 tx=0x7efed8017000).stop 2023-11-28T02:53:51.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 msgr2=0x7efee80b0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 rx=0x7efee0009fa0 tx=0x7efee0009900).stop 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 shutdown_connections 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efed405f350 0x7efed4061810 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efee80c2300 0x7efee80b0550 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efee80aa1c0 0x7efee8168b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 --2- 172.21.15.73:0/1297256268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efee8154610 0x7efee8156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 >> 172.21.15.73:0/1297256268 conn(0x7efee80655a0 msgr2=0x7efee80ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:51.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 shutdown_connections 2023-11-28T02:53:51.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.367+0000 7efeee4bd700 1 -- 172.21.15.73:0/1297256268 wait complete. 2023-11-28T02:53:51.380 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-11-28T02:53:51.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/dir9 1M 2023-11-28T02:53:51.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.497+0000 7fd854802700 1 Processor -- start 2023-11-28T02:53:51.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.497+0000 7fd854802700 1 -- start start 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd854802700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd8501552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd854802700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 0x7fd850062b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd854802700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8500657d0 con 0x7fd850069420 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd854802700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8500c3580 con 0x7fd850062700 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd854802700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd850152330 con 0x7fd850151f50 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd84f59e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd850069420 0x7fd8500698d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:43492/0 (socket says 172.21.15.73:43492) 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.498+0000 7fd84f59e700 1 -- 172.21.15.73:0/1420998157 learned_addr learned my addr 172.21.15.73:0/1420998157 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/1420998157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd8501552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84659c700 1 --2- 172.21.15.73:0/1420998157 >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 0x7fd850062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84e59c700 1 -- 172.21.15.73:0/1420998157 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1689731183 0 0) 0x7fd8500657d0 con 0x7fd850069420 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84e59c700 1 -- 172.21.15.73:0/1420998157 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd83c0033f0 con 0x7fd850069420 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 -- 172.21.15.73:0/1420998157 >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 msgr2=0x7fd850062b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/1420998157 >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 0x7fd850062b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 -- 172.21.15.73:0/1420998157 >> v1:172.21.15.73:6789/0 conn(0x7fd850069420 legacy=0x7fd8500698d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 -- 172.21.15.73:0/1420998157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850063070 con 0x7fd850151f50 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84659c700 1 --2- 172.21.15.73:0/1420998157 >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 0x7fd850062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T02:53:51.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/1420998157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd8501552f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd840009760 tx=0x7fd840009060).ready entity=mon.2 client_cookie=97c7140a3ecf91c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84e59c700 1 -- 172.21.15.73:0/1420998157 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000d4a0 con 0x7fd850151f50 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84e59c700 1 -- 172.21.15.73:0/1420998157 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84000d600 con 0x7fd850151f50 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.499+0000 7fd84e59c700 1 -- 172.21.15.73:0/1420998157 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84000d980 con 0x7fd850151f50 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 -- 172.21.15.73:0/1420998157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 msgr2=0x7fd8501552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 --2- 172.21.15.73:0/1420998157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd8501552f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd840009760 tx=0x7fd840009060).stop 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 -- 172.21.15.73:0/1420998157 shutdown_connections 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 --2- 172.21.15.73:0/1420998157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd8501552f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 --2- 172.21.15.73:0/1420998157 >> v2:172.21.15.150:3300/0 conn(0x7fd850062700 0x7fd850062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 -- 172.21.15.73:0/1420998157 >> 172.21.15.73:0/1420998157 conn(0x7fd8500642e0 msgr2=0x7fd8500c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 -- 172.21.15.73:0/1420998157 shutdown_connections 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.500+0000 7fd854802700 1 -- 172.21.15.73:0/1420998157 wait complete. 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.501+0000 7fd854802700 1 Processor -- start 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.501+0000 7fd854802700 1 -- start start 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd850062700 0x7fd850144750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd850155ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500c8260 con 0x7fd850069420 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500c7320 con 0x7fd850062700 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8500c7a50 con 0x7fd850151f50 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36778/0 (socket says 172.21.15.73:36778) 2023-11-28T02:53:51.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 -- 172.21.15.73:0/2746486582 learned_addr learned my addr 172.21.15.73:0/2746486582 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd850062700 0x7fd850144750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84f59e700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd850155ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 -- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 msgr2=0x7fd850155ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd850155ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 -- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd850062700 msgr2=0x7fd850144750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd850062700 0x7fd850144750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 -- 172.21.15.73:0/2746486582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8380092a0 con 0x7fd850069420 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd84659c700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fd83800ea00 tx=0x7fd838009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838012600 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd838012760 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8400092a0 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.502+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd838012ae0 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.503+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8500c7ca0 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.503+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85006c7c0 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.504+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 8) v1 ==== 75622+0+0 (secure 0 0 0) 0x7fd83801b400 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.504+0000 7fd8477fe700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 0x7fd8300617e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.504+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd83807aa50 con 0x7fd850069420 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.504+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 0x7fd8300617e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T02:53:51.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.504+0000 7fd84ed9d700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 0x7fd8300617e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd84000d260 tx=0x7fd840016000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T02:53:51.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.508+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd83804ba10 con 0x7fd850069420 2023-11-28T02:53:51.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.696+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd85006c1a0 con 0x7fd83005f320 2023-11-28T02:53:51.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.700+0000 7fd8477fe700 1 -- 172.21.15.73:0/2746486582 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fd85006c1a0 con 0x7fd83005f320 2023-11-28T02:53:51.702 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 msgr2=0x7fd8300617e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 0x7fd8300617e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fd84000d260 tx=0x7fd840016000).stop 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 msgr2=0x7fd850067320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 rx=0x7fd83800ea00 tx=0x7fd838009060).stop 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 shutdown_connections 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd83005f320 0x7fd8300617e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd850151f50 0x7fd850155ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd850062700 0x7fd850144750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 --2- 172.21.15.73:0/2746486582 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd850069420 0x7fd850067320 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T02:53:51.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 >> 172.21.15.73:0/2746486582 conn(0x7fd8500642e0 msgr2=0x7fd850153cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T02:53:51.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 shutdown_connections 2023-11-28T02:53:51.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T02:53:51.702+0000 7fd854802700 1 -- 172.21.15.73:0/2746486582 wait complete. 2023-11-28T02:53:51.715 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-11-28T02:54:00.318 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:54:00.316+0000 7f2b42a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:54:00.318 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:54:00.316+0000 7f2b42a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:54:00.318 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:54:00.316+0000 7f2b42a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:54:00.318 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:54:00.316+0000 7f2b42a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:54:00.318 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T02:54:00.316+0000 7f2b42a9a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T02:54:20.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:54:20.778 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:54:50.845 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:54:50.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:55:20.931 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:55:20.935 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:55:50.999 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:55:51.002 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:56:21.062 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:56:21.065 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:56:51.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:56:51.158 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:57:21.233 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:57:21.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:57:51.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:57:51.308 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:58:21.364 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:58:21.367 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:58:51.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:58:51.428 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:59:21.508 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:59:21.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:59:51.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T02:59:51.581 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:00:21.644 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:00:21.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:00:51.708 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:00:51.711 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:01:21.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:01:21.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:01:51.844 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:01:51.848 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:02:21.910 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:02:21.914 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:02:51.989 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:02:51.992 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:03:22.054 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:03:22.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:03:52.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:03:52.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:04:18.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir0 1M 2023-11-28T03:04:18.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.846+0000 7fedca534700 1 Processor -- start 2023-11-28T03:04:18.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 -- start start 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154900 0x7fedc40bcef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedc40bd3e0 0x7fedc40c40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc4067840 con 0x7fedc4154520 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedc40bd870 con 0x7fedc40bd3e0 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.847+0000 7fedca534700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4066c30 con 0x7fedc4154900 2023-11-28T03:04:18.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fedc4154520 0x7fedc406a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50942/0 (socket says 172.21.15.73:50942) 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fedc40bd3e0 0x7fedc40c40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc3fff700 1 -- 172.21.15.73:0/2082986380 learned_addr learned my addr 172.21.15.73:0/2082986380 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc37fe700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154900 0x7fedc40bcef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc2ffd700 1 -- 172.21.15.73:0/2082986380 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 793704984 0 0) 0x7fedc4067840 con 0x7fedc4154520 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc2ffd700 1 -- 172.21.15.73:0/2082986380 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedac0033f0 con 0x7fedc4154520 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2082986380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154900 msgr2=0x7fedc40bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154900 0x7fedc40bcef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2082986380 >> v1:172.21.15.73:6789/0 conn(0x7fedc4154520 legacy=0x7fedc406a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2082986380 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4066f10 con 0x7fedc40bd3e0 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc8ad1700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c40d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fedbc009e60 tx=0x7fedbc009780).ready entity=mon.0 client_cookie=3f18b3bc611e2e02 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc2ffd700 1 -- 172.21.15.73:0/2082986380 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00e070 con 0x7fedc40bd3e0 2023-11-28T03:04:18.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc2ffd700 1 -- 172.21.15.73:0/2082986380 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc00bab0 con 0x7fedc40bd3e0 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.848+0000 7fedc2ffd700 1 -- 172.21.15.73:0/2082986380 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc00be30 con 0x7fedc40bd3e0 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 -- 172.21.15.73:0/2082986380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc40bd3e0 msgr2=0x7fedc40c40d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c40d0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fedbc009e60 tx=0x7fedbc009780).stop 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 -- 172.21.15.73:0/2082986380 shutdown_connections 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c40d0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 --2- 172.21.15.73:0/2082986380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154900 0x7fedc40bcef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 -- 172.21.15.73:0/2082986380 >> 172.21.15.73:0/2082986380 conn(0x7fedc4063810 msgr2=0x7fedc4063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 -- 172.21.15.73:0/2082986380 shutdown_connections 2023-11-28T03:04:18.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.849+0000 7fedca534700 1 -- 172.21.15.73:0/2082986380 wait complete. 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.850+0000 7fedca534700 1 Processor -- start 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.850+0000 7fedca534700 1 -- start start 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154520 0x7fedc40c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40a7c00 con 0x7fedc4154900 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc4145520 con 0x7fedc40bd3e0 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedca534700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40c75d0 con 0x7fedc4154520 2023-11-28T03:04:18.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedc8ad1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedc8ad1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56676/0 (socket says 172.21.15.73:56676) 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2741381581 learned_addr learned my addr 172.21.15.73:0/2741381581 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedc37fe700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154520 0x7fedc40c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.851+0000 7fedc3fff700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154520 msgr2=0x7fedc40c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154520 0x7fedc40c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc40bd3e0 msgr2=0x7fedc40c38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 -- 172.21.15.73:0/2741381581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedbc009a20 con 0x7fedc4154900 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedc8ad1700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fedc40637a0 tx=0x7fedbc009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01d630 con 0x7fedc4154900 2023-11-28T03:04:18.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc01d790 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc40c78b0 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01db50 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.852+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc40c7e70 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.853+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc406c860 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.853+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fedbc026410 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.853+0000 7fedabfff700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 0x7fedb0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.854+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fedbc07c3a0 con 0x7fedc4154900 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.854+0000 7fedc3fff700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 0x7fedb0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.854+0000 7fedc3fff700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 0x7fedb0061780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fedb4005750 tx=0x7fedb4009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:18.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:18.858+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fedbc04d3c0 con 0x7fedc4154900 2023-11-28T03:04:19.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.048+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fedc40671f0 con 0x7fedb005f2c0 2023-11-28T03:04:19.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.052+0000 7fedabfff700 1 -- 172.21.15.73:0/2741381581 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fedc40671f0 con 0x7fedb005f2c0 2023-11-28T03:04:19.053 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-11-28T03:04:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 msgr2=0x7fedb0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 0x7fedb0061780 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fedb4005750 tx=0x7fedb4009040).stop 2023-11-28T03:04:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 msgr2=0x7fedc4145030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 rx=0x7fedc40637a0 tx=0x7fedbc009780).stop 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 shutdown_connections 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fedb005f2c0 0x7fedb0061780 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fedc4154900 0x7fedc4145030 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fedc40bd3e0 0x7fedc40c38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 --2- 172.21.15.73:0/2741381581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fedc4154520 0x7fedc40c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 >> 172.21.15.73:0/2741381581 conn(0x7fedc4063810 msgr2=0x7fedc406a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 shutdown_connections 2023-11-28T03:04:19.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.054+0000 7fedca534700 1 -- 172.21.15.73:0/2741381581 wait complete. 2023-11-28T03:04:19.066 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-11-28T03:04:19.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir1 1M 2023-11-28T03:04:19.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.182+0000 7f6ff28b1700 1 Processor -- start 2023-11-28T03:04:19.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 -- start start 2023-11-28T03:04:19.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6fec154910 0x7f6fec0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fec0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec0c6630 con 0x7f6fec154530 2023-11-28T03:04:19.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0c6390 con 0x7f6fec154910 2023-11-28T03:04:19.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6ff28b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0a84e0 con 0x7f6fec0aa220 2023-11-28T03:04:19.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6febfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6fec154530 0x7f6fec0c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50944/0 (socket says 172.21.15.73:50944) 2023-11-28T03:04:19.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.183+0000 7f6febfff700 1 -- 172.21.15.73:0/306323244 learned_addr learned my addr 172.21.15.73:0/306323244 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:19.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6ff0e4e700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fec0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/306323244 >> v2:172.21.15.150:3300/0 conn(0x7f6fec154910 0x7f6fec0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2651280296 0 0) 0x7f6fec0c6630 con 0x7f6fec154530 2023-11-28T03:04:19.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6fd40033f0 con 0x7f6fec154530 2023-11-28T03:04:19.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3820246453 0 0) 0x7f6fd40033f0 con 0x7f6fec154530 2023-11-28T03:04:19.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 msgr2=0x7f6fec0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fec0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 >> v2:172.21.15.150:3300/0 conn(0x7f6fec154910 msgr2=0x7f6fec0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 --2- 172.21.15.73:0/306323244 >> v2:172.21.15.150:3300/0 conn(0x7f6fec154910 0x7f6fec0a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0aaba0 con 0x7f6fec154530 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6fdc0015f0 con 0x7f6fec154530 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 >> v1:172.21.15.73:6789/0 conn(0x7f6fec154530 legacy=0x7f6fec0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fd40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fd4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fd4004b80 0x7f6fd4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0c6390 con 0x7f6fd4003e10 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0a84e0 con 0x7f6fd4004b80 2023-11-28T03:04:19.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fd400b8e0 con 0x7f6fec0aa220 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fd4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6febfff700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fd40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6ff0e4e700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fd4004b80 0x7f6fd4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 -- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 msgr2=0x7f6fd40038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0aa220 0x7f6fd40038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 -- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fd4004b80 msgr2=0x7f6fd4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fd4004b80 0x7f6fd4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.184+0000 7f6feb7fe700 1 -- 172.21.15.73:0/306323244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fd40033f0 con 0x7f6fd4003e10 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fd4004690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f6fe0009ab0 tx=0x7f6fe0009060).ready entity=mon.0 client_cookie=1289b38852ad13a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe0009430 con 0x7f6fd4003e10 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe000e640 con 0x7f6fd4003e10 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6feaffd700 1 -- 172.21.15.73:0/306323244 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe000e9c0 con 0x7f6fd4003e10 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 -- 172.21.15.73:0/306323244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 msgr2=0x7f6fd4004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/306323244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fd4004690 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 rx=0x7f6fe0009ab0 tx=0x7f6fe0009060).stop 2023-11-28T03:04:19.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff0e4e700 1 -- 172.21.15.73:0/306323244 reap_dead start 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 -- 172.21.15.73:0/306323244 shutdown_connections 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 -- 172.21.15.73:0/306323244 >> 172.21.15.73:0/306323244 conn(0x7f6fec094930 msgr2=0x7f6fec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 -- 172.21.15.73:0/306323244 shutdown_connections 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.185+0000 7f6ff28b1700 1 -- 172.21.15.73:0/306323244 wait complete. 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.186+0000 7f6ff28b1700 1 Processor -- start 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 -- start start 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 0x7f6fec0a8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 0x7f6fec0c32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0abd20 con 0x7f6fd4003e10 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec0a8fa0 con 0x7f6fec14ecb0 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6ff28b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec14f6f0 con 0x7f6fec0a8b40 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6febfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56696/0 (socket says 172.21.15.73:56696) 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.187+0000 7f6febfff700 1 -- 172.21.15.73:0/4176460241 learned_addr learned my addr 172.21.15.73:0/4176460241 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:19.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6ff0e4e700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 0x7f6fec0c32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 0x7f6fec0a8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 -- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 msgr2=0x7f6fec0c32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 0x7f6fec0c32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 -- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 msgr2=0x7f6fec0a8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 0x7f6fec0a8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 -- 172.21.15.73:0/4176460241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec0ab110 con 0x7f6fd4003e10 2023-11-28T03:04:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6ff0e4e700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 0x7f6fec0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 0x7f6fec0a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6febfff700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f6fdc00f8b0 tx=0x7f6fdc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc0070d0 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc007230 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec0ab3f0 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc005880 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.188+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec0ab9b0 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.189+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec06c860 con 0x7f6fd4003e10 2023-11-28T03:04:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.189+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f6fdc005bf0 con 0x7f6fd4003e10 2023-11-28T03:04:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.190+0000 7f6fd3fff700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 0x7f6fd8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.190+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f6fdc079f10 con 0x7f6fd4003e10 2023-11-28T03:04:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.190+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 0x7f6fd8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.190+0000 7f6feb7fe700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 0x7f6fd8061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6fec0a9390 tx=0x7f6fe001a000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.194+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6fdc04b050 con 0x7f6fd4003e10 2023-11-28T03:04:19.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.382+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6fec0abe60 con 0x7f6fd805f2c0 2023-11-28T03:04:19.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.386+0000 7f6fd3fff700 1 -- 172.21.15.73:0/4176460241 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6fe0000c50 con 0x7f6fd805f2c0 2023-11-28T03:04:19.387 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.387+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 msgr2=0x7f6fd8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.387+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 0x7f6fd8061780 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f6fec0a9390 tx=0x7f6fe001a000).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.387+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 msgr2=0x7f6fec1651f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.387+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7f6fdc00f8b0 tx=0x7f6fdc009040).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.387+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 shutdown_connections 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fd805f2c0 0x7f6fd8061780 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec0a8b40 0x7f6fec0c32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fd4003e10 0x7f6fec1651f0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 --2- 172.21.15.73:0/4176460241 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec14ecb0 0x7f6fec0a8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 >> 172.21.15.73:0/4176460241 conn(0x7f6fec094930 msgr2=0x7f6fec14fc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 shutdown_connections 2023-11-28T03:04:19.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.388+0000 7f6ff28b1700 1 -- 172.21.15.73:0/4176460241 wait complete. 2023-11-28T03:04:19.400 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-11-28T03:04:19.400 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir2 1M 2023-11-28T03:04:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.518+0000 7fddd7156700 1 Processor -- start 2023-11-28T03:04:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.518+0000 7fddd7156700 1 -- start start 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd7156700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddd01373a0 0x7fddd00c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd7156700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd7156700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddd0158e20 con 0x7fddd00669a0 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd7156700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddd00c68a0 con 0x7fddd01373a0 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd7156700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd01582a0 con 0x7fddd00c5f30 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddd01373a0 0x7fddd00c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fddd01373a0 0x7fddd00c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56708/0 (socket says 172.21.15.73:56708) 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 -- 172.21.15.73:0/2755188346 learned_addr learned my addr 172.21.15.73:0/2755188346 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddcffff700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 -- 172.21.15.73:0/2755188346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 msgr2=0x7fddd00c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 -- 172.21.15.73:0/2755188346 >> v1:172.21.15.150:6789/0 conn(0x7fddd00669a0 legacy=0x7fddd0066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 -- 172.21.15.73:0/2755188346 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd0158580 con 0x7fddd01373a0 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddd4ef2700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00c72e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fddc4006fd0 tx=0x7fddc40090f0).ready entity=mon.0 client_cookie=ab5e9f7c74f98c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddcf7fe700 1 -- 172.21.15.73:0/2755188346 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc400b600 con 0x7fddd01373a0 2023-11-28T03:04:19.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddcf7fe700 1 -- 172.21.15.73:0/2755188346 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc400b760 con 0x7fddd01373a0 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.519+0000 7fddcf7fe700 1 -- 172.21.15.73:0/2755188346 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc400bae0 con 0x7fddd01373a0 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 -- 172.21.15.73:0/2755188346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 msgr2=0x7fddd00c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00c72e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 rx=0x7fddc4006fd0 tx=0x7fddc40090f0).stop 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 -- 172.21.15.73:0/2755188346 shutdown_connections 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00c72e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 --2- 172.21.15.73:0/2755188346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 -- 172.21.15.73:0/2755188346 >> 172.21.15.73:0/2755188346 conn(0x7fddd00628f0 msgr2=0x7fddd0062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 -- 172.21.15.73:0/2755188346 shutdown_connections 2023-11-28T03:04:19.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.520+0000 7fddd7156700 1 -- 172.21.15.73:0/2755188346 wait complete. 2023-11-28T03:04:19.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.521+0000 7fddd7156700 1 Processor -- start 2023-11-28T03:04:19.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.521+0000 7fddd7156700 1 -- start start 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd00669a0 0x7fddd014de80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd0138860 con 0x7fddd01373a0 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00a9560 con 0x7fddd00669a0 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd7156700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd00a9840 con 0x7fddd00c5f30 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56710/0 (socket says 172.21.15.73:56710) 2023-11-28T03:04:19.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 -- 172.21.15.73:0/1188097167 learned_addr learned my addr 172.21.15.73:0/1188097167 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddcffff700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd4ef2700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd00669a0 0x7fddd014de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 -- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 msgr2=0x7fddd00a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 -- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd00669a0 msgr2=0x7fddd014de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd00669a0 0x7fddd014de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 -- 172.21.15.73:0/1188097167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddc4009370 con 0x7fddd01373a0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddcffff700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.522+0000 7fddd56f3700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fddc800c810 tx=0x7fddc8002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.523+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8010510 con 0x7fddd01373a0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.523+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddc80106b0 con 0x7fddd01373a0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.523+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd00a9b20 con 0x7fddd01373a0 2023-11-28T03:04:19.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.523+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8010a70 con 0x7fddd01373a0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.523+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd00aa0e0 con 0x7fddd01373a0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.524+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd00aa300 con 0x7fddd01373a0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.524+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fddc801b400 con 0x7fddd01373a0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.524+0000 7fddccff9700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 0x7fddb4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.525+0000 7fddd4ef2700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 0x7fddb4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.525+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fddc807a370 con 0x7fddd01373a0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.525+0000 7fddd4ef2700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 0x7fddb4061780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fddc4009130 tx=0x7fddc4017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.529+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fddc804b390 con 0x7fddd01373a0 2023-11-28T03:04:19.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.715+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd006c860 con 0x7fddb405f2c0 2023-11-28T03:04:19.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.719+0000 7fddccff9700 1 -- 172.21.15.73:0/1188097167 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fddd006c860 con 0x7fddb405f2c0 2023-11-28T03:04:19.720 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-11-28T03:04:19.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.720+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 msgr2=0x7fddb4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.720+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 0x7fddb4061780 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fddc4009130 tx=0x7fddc4017000).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.720+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 msgr2=0x7fddd00bee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.720+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 rx=0x7fddc800c810 tx=0x7fddc8002560).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 shutdown_connections 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fddb405f2c0 0x7fddb4061780 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fddd01373a0 0x7fddd00bee40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fddd00669a0 0x7fddd014de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 --2- 172.21.15.73:0/1188097167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fddd00c5f30 0x7fddd00a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 >> 172.21.15.73:0/1188097167 conn(0x7fddd00628f0 msgr2=0x7fddd0063a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 shutdown_connections 2023-11-28T03:04:19.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.721+0000 7fddd7156700 1 -- 172.21.15.73:0/1188097167 wait complete. 2023-11-28T03:04:19.733 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-11-28T03:04:19.733 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir3 1M 2023-11-28T03:04:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.851+0000 7f9365b20700 1 Processor -- start 2023-11-28T03:04:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.851+0000 7f9365b20700 1 -- start start 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f9365b20700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f93600aa930 0x7f93600a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f9365b20700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600c6630 con 0x7f9360154910 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f9365b20700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9360064230 con 0x7f9360154530 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f9365b20700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600c6390 con 0x7f93600aa930 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f935effd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9360154910 0x7f93600aa220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50956/0 (socket says 172.21.15.73:50956) 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f935effd700 1 -- 172.21.15.73:0/3466252055 learned_addr learned my addr 172.21.15.73:0/3466252055 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2346727687 0 0) 0x7f93600c6630 con 0x7f9360154910 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f93480033f0 con 0x7f9360154910 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.852+0000 7f935ffff700 1 --2- 172.21.15.73:0/3466252055 >> v2:172.21.15.150:3300/0 conn(0x7f93600aa930 0x7f93600a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3534565628 0 0) 0x7f93480033f0 con 0x7f9360154910 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 >> v2:172.21.15.150:3300/0 conn(0x7f93600aa930 msgr2=0x7f93600a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 --2- 172.21.15.73:0/3466252055 >> v2:172.21.15.150:3300/0 conn(0x7f93600aa930 0x7f93600a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 >> v1:172.21.15.150:6789/0 conn(0x7f9360154530 legacy=0x7f93600c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360064510 con 0x7f9360154910 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f9354001470 con 0x7f9360154910 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 >> v1:172.21.15.73:6789/0 conn(0x7f9360154910 legacy=0x7f93600aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93480038e0 0x7f9348003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 0x7f9348004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9348005000 0x7f934800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93600c6390 con 0x7f9348004690 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f934800bf60 con 0x7f9348005000 2023-11-28T03:04:19.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f934800c0f0 con 0x7f93480038e0 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 0x7f9348004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935f7fe700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93480038e0 0x7f9348003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935ffff700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9348005000 0x7f934800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 -- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93480038e0 msgr2=0x7f9348003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f93480038e0 0x7f9348003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 -- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9348005000 msgr2=0x7f934800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9348005000 0x7f934800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 -- 172.21.15.73:0/3466252055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93480033f0 con 0x7f9348004690 2023-11-28T03:04:19.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935ffff700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9348005000 0x7f934800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935ffff700 1 -- 172.21.15.73:0/3466252055 reap_dead start 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935effd700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 0x7f9348004b10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f935400c190 tx=0x7f9354001e50).ready entity=mon.0 client_cookie=2d375cb86673e883 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9354007a20 con 0x7f9348004690 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f935400cac0 con 0x7f9348004690 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.853+0000 7f935e7fc700 1 -- 172.21.15.73:0/3466252055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f935400ce40 con 0x7f9348004690 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 -- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 msgr2=0x7f9348004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 0x7f9348004b10 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 rx=0x7f935400c190 tx=0x7f9354001e50).stop 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 -- 172.21.15.73:0/3466252055 shutdown_connections 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 --2- 172.21.15.73:0/3466252055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9348004690 0x7f9348004b10 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 -- 172.21.15.73:0/3466252055 >> 172.21.15.73:0/3466252055 conn(0x7f9360094930 msgr2=0x7f936006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:19.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 -- 172.21.15.73:0/3466252055 shutdown_connections 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.854+0000 7f9365b20700 1 -- 172.21.15.73:0/3466252055 wait complete. 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.855+0000 7f9365b20700 1 Processor -- start 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.855+0000 7f9365b20700 1 -- start start 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9348004690 0x7f93600ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 0x7f93601513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601694f0 con 0x7f9360168550 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601689d0 con 0x7f9360150f70 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f9365b20700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9360168c50 con 0x7f9348004690 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56720/0 (socket says 172.21.15.73:56720) 2023-11-28T03:04:19.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 -- 172.21.15.73:0/582865323 learned_addr learned my addr 172.21.15.73:0/582865323 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935ffff700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 0x7f93601513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935f7fe700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9348004690 0x7f93600ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 -- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9348004690 msgr2=0x7f93600ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9348004690 0x7f93600ab770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 -- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 msgr2=0x7f93601513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 0x7f93601513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 -- 172.21.15.73:0/582865323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9360168f30 con 0x7f9360168550 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935ffff700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 0x7f93601513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.856+0000 7f935effd700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f9354001ed0 tx=0x7f9354015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.857+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93540073b0 con 0x7f9360168550 2023-11-28T03:04:19.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.857+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9354007510 con 0x7f9360168550 2023-11-28T03:04:19.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.857+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9360169210 con 0x7f9360168550 2023-11-28T03:04:19.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.857+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93540048e0 con 0x7f9360168550 2023-11-28T03:04:19.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.857+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9360148b90 con 0x7f9360168550 2023-11-28T03:04:19.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.858+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93600af450 con 0x7f9360168550 2023-11-28T03:04:19.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.858+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f935401c070 con 0x7f9360168550 2023-11-28T03:04:19.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.858+0000 7f93477fe700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 0x7f934c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:19.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.859+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f935407ac10 con 0x7f9360168550 2023-11-28T03:04:19.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.859+0000 7f935f7fe700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 0x7f934c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:19.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.859+0000 7f935f7fe700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 0x7f934c061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9350007e10 tx=0x7f9350007a80).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:19.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:19.863+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f93600af450 con 0x7f9360168550 2023-11-28T03:04:20.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.051+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f93600af450 con 0x7f934c05f2c0 2023-11-28T03:04:20.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.055+0000 7f93477fe700 1 -- 172.21.15.73:0/582865323 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f93600af450 con 0x7f934c05f2c0 2023-11-28T03:04:20.056 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-11-28T03:04:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 msgr2=0x7f934c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 0x7f934c061780 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f9350007e10 tx=0x7f9350007a80).stop 2023-11-28T03:04:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 msgr2=0x7f93601486a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 rx=0x7f9354001ed0 tx=0x7f9354015000).stop 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 shutdown_connections 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f934c05f2c0 0x7f934c061780 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9360150f70 0x7f93601513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9348004690 0x7f93600ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 --2- 172.21.15.73:0/582865323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9360168550 0x7f93601486a0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 >> 172.21.15.73:0/582865323 conn(0x7f9360094930 msgr2=0x7f93600c69c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 shutdown_connections 2023-11-28T03:04:20.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.057+0000 7f9365b20700 1 -- 172.21.15.73:0/582865323 wait complete. 2023-11-28T03:04:20.069 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-11-28T03:04:20.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir4 1M 2023-11-28T03:04:20.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.186+0000 7f040a6d0700 1 Processor -- start 2023-11-28T03:04:20.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.186+0000 7f040a6d0700 1 -- start start 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f040a6d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0404154530 0x7f0404154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f040a6d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f04040aa220 0x7f04040aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f040a6d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04040c6630 con 0x7f04040a7ff0 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f040a6d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040c6390 con 0x7f04040aa220 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f040a6d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04040a8470 con 0x7f0404154530 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f04040aa220 0x7f04040aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f04040aa220 0x7f04040aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56730/0 (socket says 172.21.15.73:56730) 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 -- 172.21.15.73:0/963587854 learned_addr learned my addr 172.21.15.73:0/963587854 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0403fff700 1 --2- 172.21.15.73:0/963587854 >> v2:172.21.15.150:3300/0 conn(0x7f0404154530 0x7f0404154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 -- 172.21.15.73:0/963587854 >> v2:172.21.15.150:3300/0 conn(0x7f0404154530 msgr2=0x7f0404154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 --2- 172.21.15.73:0/963587854 >> v2:172.21.15.150:3300/0 conn(0x7f0404154530 0x7f0404154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 -- 172.21.15.73:0/963587854 >> v1:172.21.15.150:6789/0 conn(0x7f04040a7ff0 legacy=0x7f04040c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 -- 172.21.15.73:0/963587854 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04040aab80 con 0x7f04040aa220 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.187+0000 7f0408c6d700 1 --2- 172.21.15.73:0/963587854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040aa220 0x7f04040aa690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f04040c9080 tx=0x7f03fc009840).ready entity=mon.0 client_cookie=a9a5f0bb742a43e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f0402ffd700 1 -- 172.21.15.73:0/963587854 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00e070 con 0x7f04040aa220 2023-11-28T03:04:20.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f0402ffd700 1 -- 172.21.15.73:0/963587854 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03fc00ba10 con 0x7f04040aa220 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f0402ffd700 1 -- 172.21.15.73:0/963587854 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03fc00bd90 con 0x7f04040aa220 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 -- 172.21.15.73:0/963587854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040aa220 msgr2=0x7f04040aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 --2- 172.21.15.73:0/963587854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040aa220 0x7f04040aa690 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 rx=0x7f04040c9080 tx=0x7f03fc009840).stop 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 -- 172.21.15.73:0/963587854 shutdown_connections 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 --2- 172.21.15.73:0/963587854 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040aa220 0x7f04040aa690 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 --2- 172.21.15.73:0/963587854 >> v2:172.21.15.150:3300/0 conn(0x7f0404154530 0x7f0404154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 -- 172.21.15.73:0/963587854 >> 172.21.15.73:0/963587854 conn(0x7f0404094930 msgr2=0x7f040406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 -- 172.21.15.73:0/963587854 shutdown_connections 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.188+0000 7f040a6d0700 1 -- 172.21.15.73:0/963587854 wait complete. 2023-11-28T03:04:20.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.189+0000 7f040a6d0700 1 Processor -- start 2023-11-28T03:04:20.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.189+0000 7f040a6d0700 1 -- start start 2023-11-28T03:04:20.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04040aa220 0x7f04040c3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0404154530 0x7f04040c4240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0404151790 con 0x7f04040a7ff0 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04040c4730 con 0x7f0404154530 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f040a6d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0404168a20 con 0x7f04040aa220 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f0403fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f0403fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56744/0 (socket says 172.21.15.73:56744) 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f0403fff700 1 -- 172.21.15.73:0/2425121748 learned_addr learned my addr 172.21.15.73:0/2425121748 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:20.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f04037fe700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04040aa220 0x7f04040c3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f0408c6d700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0404154530 0x7f04040c4240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.190+0000 7f0403fff700 1 -- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04040aa220 msgr2=0x7f04040c3d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f0403fff700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04040aa220 0x7f04040c3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f0403fff700 1 -- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0404154530 msgr2=0x7f04040c4240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f0403fff700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0404154530 0x7f04040c4240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f0403fff700 1 -- 172.21.15.73:0/2425121748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03fc009ae0 con 0x7f04040a7ff0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f0403fff700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f03f400d6b0 tx=0x7f03f4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f40125f0 con 0x7f04040a7ff0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03f4012750 con 0x7f04040a7ff0 2023-11-28T03:04:20.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04041680f0 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03f4012ad0 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.191+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04041686b0 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.192+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f040406c860 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.192+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f03f4013480 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.192+0000 7f03ebfff700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 0x7f03f0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.192+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f03f407a900 con 0x7f04040a7ff0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.192+0000 7f04037fe700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 0x7f03f0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.193+0000 7f04037fe700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 0x7f03f0061780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f03f8007fd0 tx=0x7f03f8009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.196+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f03f404b920 con 0x7f04040a7ff0 2023-11-28T03:04:20.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.388+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0404168b60 con 0x7f03f005f2c0 2023-11-28T03:04:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.391+0000 7f03ebfff700 1 -- 172.21.15.73:0/2425121748 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f03f8002210 con 0x7f03f005f2c0 2023-11-28T03:04:20.392 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-11-28T03:04:20.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 msgr2=0x7f03f0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 0x7f03f0061780 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f03f8007fd0 tx=0x7f03f8009040).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 msgr2=0x7f0404153270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 rx=0x7f03f400d6b0 tx=0x7f03f4008040).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 shutdown_connections 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f03f005f2c0 0x7f03f0061780 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0404154530 0x7f04040c4240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f04040a7ff0 0x7f0404153270 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 --2- 172.21.15.73:0/2425121748 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f04040aa220 0x7f04040c3d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 >> 172.21.15.73:0/2425121748 conn(0x7f0404094930 msgr2=0x7f040414ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 shutdown_connections 2023-11-28T03:04:20.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.393+0000 7f040a6d0700 1 -- 172.21.15.73:0/2425121748 wait complete. 2023-11-28T03:04:20.406 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-11-28T03:04:20.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir5 1M 2023-11-28T03:04:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.522+0000 7ff1e0a58700 1 Processor -- start 2023-11-28T03:04:20.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.522+0000 7ff1e0a58700 1 -- start start 2023-11-28T03:04:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1e0a58700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1dc154610 0x7ff1dc1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1e0a58700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc159210 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1e0a58700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1dc0c0c70 con 0x7ff1dc0ab3b0 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1e0a58700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1dc158970 con 0x7ff1dc154610 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1da59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1dc154610 0x7ff1dc1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1dad9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1dc0aa1c0 0x7ff1dc0c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50972/0 (socket says 172.21.15.73:50972) 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.523+0000 7ff1dad9d700 1 -- 172.21.15.73:0/3312038205 learned_addr learned my addr 172.21.15.73:0/3312038205 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1da59c700 1 -- 172.21.15.73:0/3312038205 >> v1:172.21.15.150:6789/0 conn(0x7ff1dc0ab3b0 legacy=0x7ff1dc0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1da59c700 1 -- 172.21.15.73:0/3312038205 >> v1:172.21.15.73:6789/0 conn(0x7ff1dc0aa1c0 legacy=0x7ff1dc0c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1d959a700 1 -- 172.21.15.73:0/3312038205 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 37041596 0 0) 0x7ff1dc159210 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1da59c700 1 -- 172.21.15.73:0/3312038205 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1dc158c50 con 0x7ff1dc154610 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1da59c700 1 --2- 172.21.15.73:0/3312038205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc154610 0x7ff1dc1582e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff1d4009e60 tx=0x7ff1d4009780).ready entity=mon.0 client_cookie=54625956b82a4c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1d959a700 1 -- 172.21.15.73:0/3312038205 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d400e070 con 0x7ff1dc154610 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1d959a700 1 -- 172.21.15.73:0/3312038205 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d400bbf0 con 0x7ff1dc154610 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1d959a700 1 -- 172.21.15.73:0/3312038205 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4012410 con 0x7ff1dc154610 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 -- 172.21.15.73:0/3312038205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc154610 msgr2=0x7ff1dc1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/3312038205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc154610 0x7ff1dc1582e0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7ff1d4009e60 tx=0x7ff1d4009780).stop 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 -- 172.21.15.73:0/3312038205 shutdown_connections 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/3312038205 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc154610 0x7ff1dc1582e0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 -- 172.21.15.73:0/3312038205 >> 172.21.15.73:0/3312038205 conn(0x7ff1dc0655a0 msgr2=0x7ff1dc069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 -- 172.21.15.73:0/3312038205 shutdown_connections 2023-11-28T03:04:20.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.524+0000 7ff1e0a58700 1 -- 172.21.15.73:0/3312038205 wait complete. 2023-11-28T03:04:20.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.525+0000 7ff1e0a58700 1 Processor -- start 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 -- start start 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc154610 0x7ff1dc154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc144f90 0x7ff1dc14d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc156350 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc1554c0 con 0x7ff1dc144f90 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1e0a58700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1dc155740 con 0x7ff1dc154610 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1da59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56770/0 (socket says 172.21.15.73:56770) 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.526+0000 7ff1da59c700 1 -- 172.21.15.73:0/1502435325 learned_addr learned my addr 172.21.15.73:0/1502435325 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 -- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc154610 msgr2=0x7ff1dc154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc154610 0x7ff1dc154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 -- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc144f90 msgr2=0x7ff1dc14d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc144f90 0x7ff1dc14d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 -- 172.21.15.73:0/1502435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4009a20 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1da59c700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff1dc0cc2d0 tx=0x7ff1d4009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d40116d0 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d4011830 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1dc155a20 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4011bd0 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.527+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1dc155fe0 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.528+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1dc0aa600 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.528+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7ff1d401d420 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.529+0000 7ff1ca7fc700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 0x7ff1bc061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.529+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7ff1d407c4e0 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.529+0000 7ff1d9d9b700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 0x7ff1bc061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.529+0000 7ff1d9d9b700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 0x7ff1bc061750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff1dc1538a0 tx=0x7ff1d0006b60).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.533+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1d404d500 con 0x7ff1dc0aa1c0 2023-11-28T03:04:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.720+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1dc158f30 con 0x7ff1bc05f290 2023-11-28T03:04:20.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.723+0000 7ff1ca7fc700 1 -- 172.21.15.73:0/1502435325 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff1dc158f30 con 0x7ff1bc05f290 2023-11-28T03:04:20.724 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 msgr2=0x7ff1bc061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 0x7ff1bc061750 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7ff1dc1538a0 tx=0x7ff1d0006b60).stop 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 msgr2=0x7ff1dc0ae210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 rx=0x7ff1dc0cc2d0 tx=0x7ff1d4009780).stop 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 shutdown_connections 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1bc05f290 0x7ff1bc061750 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1dc144f90 0x7ff1dc14d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1dc0aa1c0 0x7ff1dc0ae210 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 --2- 172.21.15.73:0/1502435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1dc154610 0x7ff1dc154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:20.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 >> 172.21.15.73:0/1502435325 conn(0x7ff1dc0655a0 msgr2=0x7ff1dc063e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 shutdown_connections 2023-11-28T03:04:20.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.726+0000 7ff1e0a58700 1 -- 172.21.15.73:0/1502435325 wait complete. 2023-11-28T03:04:20.738 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-11-28T03:04:20.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir6 1M 2023-11-28T03:04:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.855+0000 7f1ac92b2700 1 Processor -- start 2023-11-28T03:04:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.855+0000 7f1ac92b2700 1 -- start start 2023-11-28T03:04:20.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac92b2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ac4154910 0x7f1ac40a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac92b2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac40c6630 con 0x7f1ac4154530 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac92b2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ac4064230 con 0x7f1ac40aa220 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac92b2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ac40c6390 con 0x7f1ac4154910 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac2ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ac4154530 0x7f1ac40c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50980/0 (socket says 172.21.15.73:50980) 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac2ffd700 1 -- 172.21.15.73:0/1029702043 learned_addr learned my addr 172.21.15.73:0/1029702043 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.856+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/1029702043 >> v2:172.21.15.150:3300/0 conn(0x7f1ac4154910 0x7f1ac40a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749322845 0 0) 0x7f1ac40c6630 con 0x7f1ac4154530 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1aac0033f0 con 0x7f1ac4154530 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2570003064 0 0) 0x7f1aac0033f0 con 0x7f1ac4154530 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac27fc700 1 -- 172.21.15.73:0/1029702043 >> v1:172.21.15.150:6789/0 conn(0x7f1ac40aa220 legacy=0x7f1ac40c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac27fc700 1 -- 172.21.15.73:0/1029702043 >> v1:172.21.15.73:6789/0 conn(0x7f1ac4154530 legacy=0x7f1ac40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac27fc700 1 -- 172.21.15.73:0/1029702043 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac4064510 con 0x7f1ac4154910 2023-11-28T03:04:20.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/1029702043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac40a7ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1ab8009760 tx=0x7f1ab8009060).ready entity=mon.1 client_cookie=968f3dd1520c790a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4001970 con 0x7f1ac4154910 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab8011400 con 0x7f1ac4154910 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac1ffb700 1 -- 172.21.15.73:0/1029702043 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab8011780 con 0x7f1ac4154910 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac92b2700 1 -- 172.21.15.73:0/1029702043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 msgr2=0x7f1ac40a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.857+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/1029702043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac40a7ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 rx=0x7f1ab8009760 tx=0x7f1ab8009060).stop 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.858+0000 7f1ac92b2700 1 -- 172.21.15.73:0/1029702043 shutdown_connections 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.858+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/1029702043 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac40a7ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.858+0000 7f1ac92b2700 1 -- 172.21.15.73:0/1029702043 >> 172.21.15.73:0/1029702043 conn(0x7f1ac4094930 msgr2=0x7f1ac406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.858+0000 7f1ac92b2700 1 -- 172.21.15.73:0/1029702043 shutdown_connections 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.858+0000 7f1ac92b2700 1 -- 172.21.15.73:0/1029702043 wait complete. 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.859+0000 7f1ac92b2700 1 Processor -- start 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.859+0000 7f1ac92b2700 1 -- start start 2023-11-28T03:04:20.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 0x7f1ac40c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac4151c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40c3a70 con 0x7f1ac40aa220 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac41529e0 con 0x7f1ac4154910 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ac40c3180 con 0x7f1ac4154530 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56776/0 (socket says 172.21.15.73:56776) 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 -- 172.21.15.73:0/3123919381 learned_addr learned my addr 172.21.15.73:0/3123919381 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 0x7f1ac40c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac37fe700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac4151c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 -- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 msgr2=0x7f1ac40c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 0x7f1ac40c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 -- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 msgr2=0x7f1ac4151c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac4151c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 -- 172.21.15.73:0/3123919381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80092a0 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 0x7f1ac40c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac37fe700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac4151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac2ffd700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f1ab4001460 tx=0x7f1ab40017c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4005c00 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab4005da0 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ac41519f0 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.860+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab4007360 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.861+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ac40c3420 con 0x7f1ac40aa220 2023-11-28T03:04:20.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.862+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac406c860 con 0x7f1ac40aa220 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.862+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f1ab40076d0 con 0x7f1ac40aa220 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.862+0000 7f1aaaffd700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 0x7f1ab0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.862+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f1ab407a920 con 0x7f1ac40aa220 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.862+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 0x7f1ab0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.863+0000 7f1ac27fc700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 0x7f1ab0061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f1ac40c4c30 tx=0x7f1ab8014000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:20.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:20.866+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1ab404b9a0 con 0x7f1ac40aa220 2023-11-28T03:04:21.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.056+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1ac40c3bb0 con 0x7f1ab005f2c0 2023-11-28T03:04:21.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.059+0000 7f1aaaffd700 1 -- 172.21.15.73:0/3123919381 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f1ab8000ce0 con 0x7f1ab005f2c0 2023-11-28T03:04:21.061 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 msgr2=0x7f1ab0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 0x7f1ab0061780 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f1ac40c4c30 tx=0x7f1ab8014000).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 msgr2=0x7f1ac4151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 rx=0x7f1ab4001460 tx=0x7f1ab40017c0).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 shutdown_connections 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ab005f2c0 0x7f1ab0061780 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ac4154910 0x7f1ac4151c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ac40aa220 0x7f1ac4151500 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 --2- 172.21.15.73:0/3123919381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ac4154530 0x7f1ac40c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 >> 172.21.15.73:0/3123919381 conn(0x7f1ac4094930 msgr2=0x7f1ac40c0bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 shutdown_connections 2023-11-28T03:04:21.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.061+0000 7f1ac92b2700 1 -- 172.21.15.73:0/3123919381 wait complete. 2023-11-28T03:04:21.074 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-11-28T03:04:21.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir7 1M 2023-11-28T03:04:21.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.190+0000 7faa124b8700 1 Processor -- start 2023-11-28T03:04:21.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.190+0000 7faa124b8700 1 -- start start 2023-11-28T03:04:21.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa124b8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faa0c154910 0x7faa0c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa124b8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 0x7faa0c0aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa124b8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa0c0c6630 con 0x7faa0c154530 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa124b8700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c0c6390 con 0x7faa0c154910 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa124b8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa0c0a84e0 con 0x7faa0c0aa220 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faa0c154910 0x7faa0c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faa0c154910 0x7faa0c0a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56782/0 (socket says 172.21.15.73:56782) 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 -- 172.21.15.73:0/3943973139 learned_addr learned my addr 172.21.15.73:0/3943973139 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa10a55700 1 --2- 172.21.15.73:0/3943973139 >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 0x7faa0c0aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 -- 172.21.15.73:0/3943973139 >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 msgr2=0x7faa0c0aa690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 --2- 172.21.15.73:0/3943973139 >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 0x7faa0c0aa690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 -- 172.21.15.73:0/3943973139 >> v1:172.21.15.150:6789/0 conn(0x7faa0c154530 legacy=0x7faa0c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa0b7fe700 1 -- 172.21.15.73:0/3943973139 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0c0aab80 con 0x7faa0c154910 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.191+0000 7faa10a55700 1 --2- 172.21.15.73:0/3943973139 >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 0x7faa0c0aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa0b7fe700 1 --2- 172.21.15.73:0/3943973139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154910 0x7faa0c0a7ff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7faa0c084a60 tx=0x7faa000090f0).ready entity=mon.0 client_cookie=1336d7fc9e60f9d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa0affd700 1 -- 172.21.15.73:0/3943973139 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000b5a0 con 0x7faa0c154910 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa0affd700 1 -- 172.21.15.73:0/3943973139 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa0000b700 con 0x7faa0c154910 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa0affd700 1 -- 172.21.15.73:0/3943973139 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000ba80 con 0x7faa0c154910 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 -- 172.21.15.73:0/3943973139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154910 msgr2=0x7faa0c0a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 --2- 172.21.15.73:0/3943973139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154910 0x7faa0c0a7ff0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 rx=0x7faa0c084a60 tx=0x7faa000090f0).stop 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 -- 172.21.15.73:0/3943973139 shutdown_connections 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 --2- 172.21.15.73:0/3943973139 >> v2:172.21.15.150:3300/0 conn(0x7faa0c0aa220 0x7faa0c0aa690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 --2- 172.21.15.73:0/3943973139 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154910 0x7faa0c0a7ff0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 -- 172.21.15.73:0/3943973139 >> 172.21.15.73:0/3943973139 conn(0x7faa0c094930 msgr2=0x7faa0c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 -- 172.21.15.73:0/3943973139 shutdown_connections 2023-11-28T03:04:21.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.192+0000 7faa124b8700 1 -- 172.21.15.73:0/3943973139 wait complete. 2023-11-28T03:04:21.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.193+0000 7faa124b8700 1 Processor -- start 2023-11-28T03:04:21.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.193+0000 7faa124b8700 1 -- start start 2023-11-28T03:04:21.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0aa220 0x7faa0c151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0c154910 0x7faa0c0c7f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0c8470 con 0x7faa0c154530 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0affe0 con 0x7faa0c0aa220 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa124b8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa0c0afb50 con 0x7faa0c154910 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa0b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa0b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56788/0 (socket says 172.21.15.73:56788) 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa0b7fe700 1 -- 172.21.15.73:0/811338641 learned_addr learned my addr 172.21.15.73:0/811338641 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.194+0000 7faa10a55700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0c154910 0x7faa0c0c7f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0bfff700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0aa220 0x7faa0c151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 -- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0c154910 msgr2=0x7faa0c0c7f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0c154910 0x7faa0c0c7f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 -- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0aa220 msgr2=0x7faa0c151270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0aa220 0x7faa0c151270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 -- 172.21.15.73:0/811338641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa00009370 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa0b7fe700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7faa0c06ae10 tx=0x7faa00015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0000ba80 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0c0aef40 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa00000c90 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa0001c670 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.195+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa0c0af500 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.196+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7faa0001c9e0 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.196+0000 7fa9f3fff700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 0x7fa9f8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.197+0000 7faa0bfff700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 0x7fa9f8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.197+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa0c063a10 con 0x7faa0c154530 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.197+0000 7faa0bfff700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 0x7fa9f8061780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa9fc00b870 tx=0x7fa9fc008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.197+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7faa00050fb0 con 0x7faa0c154530 2023-11-28T03:04:21.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.202+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faa0004d350 con 0x7faa0c154530 2023-11-28T03:04:21.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.394+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faa0c06c860 con 0x7fa9f805f2c0 2023-11-28T03:04:21.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.397+0000 7fa9f3fff700 1 -- 172.21.15.73:0/811338641 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7faa0c06c860 con 0x7fa9f805f2c0 2023-11-28T03:04:21.398 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-11-28T03:04:21.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 msgr2=0x7fa9f8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 0x7fa9f8061780 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa9fc00b870 tx=0x7fa9fc008040).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 msgr2=0x7faa0c0c7a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 rx=0x7faa0c06ae10 tx=0x7faa00015000).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 shutdown_connections 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9f805f2c0 0x7fa9f8061780 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa0c154910 0x7faa0c0c7f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa0c0aa220 0x7faa0c151270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 --2- 172.21.15.73:0/811338641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa0c154530 0x7faa0c0c7a90 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 >> 172.21.15.73:0/811338641 conn(0x7faa0c094930 msgr2=0x7faa0c1530d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 shutdown_connections 2023-11-28T03:04:21.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.399+0000 7faa124b8700 1 -- 172.21.15.73:0/811338641 wait complete. 2023-11-28T03:04:21.412 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-11-28T03:04:21.412 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir8 1M 2023-11-28T03:04:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.529+0000 7fc4f759e700 1 Processor -- start 2023-11-28T03:04:21.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 -- start start 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc4f81542d0 0x7fc4f80a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f81546b0 0x7fc4f80aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4f80c5fc0 0x7fc4f80c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f80aa950 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4f8158d50 con 0x7fc4f81542d0 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f8158140 con 0x7fc4f81546b0 2023-11-28T03:04:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.530+0000 7fc4f6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4f80c5fc0 0x7fc4f80c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc4f80c5fc0 0x7fc4f80c6450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56798/0 (socket says 172.21.15.73:56798) 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2426278107 learned_addr learned my addr 172.21.15.73:0/2426278107 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4edd9b700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f81546b0 0x7fc4f80aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f659c700 1 --2- 172.21.15.73:0/2426278107 >> v2:172.21.15.150:3300/0 conn(0x7fc4f81542d0 0x7fc4f80a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2426278107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f81546b0 msgr2=0x7fc4f80aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f81546b0 0x7fc4f80aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2426278107 >> v2:172.21.15.150:3300/0 conn(0x7fc4f81542d0 msgr2=0x7fc4f80a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2426278107 >> v2:172.21.15.150:3300/0 conn(0x7fc4f81542d0 0x7fc4f80a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2426278107 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f8158420 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f80c5fc0 0x7fc4f80c6450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fc4f80c07e0 tx=0x7fc4f0009840).ready entity=mon.0 client_cookie=e4814c92379294ba server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f5d9b700 1 -- 172.21.15.73:0/2426278107 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000e070 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f5d9b700 1 -- 172.21.15.73:0/2426278107 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f000ba50 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.531+0000 7fc4f5d9b700 1 -- 172.21.15.73:0/2426278107 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f000bdd0 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 -- 172.21.15.73:0/2426278107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f80c5fc0 msgr2=0x7fc4f80c6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f80c5fc0 0x7fc4f80c6450 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 rx=0x7fc4f80c07e0 tx=0x7fc4f0009840).stop 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 -- 172.21.15.73:0/2426278107 shutdown_connections 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f80c5fc0 0x7fc4f80c6450 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2426278107 >> v2:172.21.15.150:3300/0 conn(0x7fc4f81542d0 0x7fc4f80a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2426278107 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f81546b0 0x7fc4f80aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 -- 172.21.15.73:0/2426278107 >> 172.21.15.73:0/2426278107 conn(0x7fc4f80cb010 msgr2=0x7fc4f80cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 -- 172.21.15.73:0/2426278107 shutdown_connections 2023-11-28T03:04:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.532+0000 7fc4f759e700 1 -- 172.21.15.73:0/2426278107 wait complete. 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.533+0000 7fc4f759e700 1 Processor -- start 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.533+0000 7fc4f759e700 1 -- start start 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f80c5fc0 0x7fc4f8147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f81542d0 0x7fc4f80ae9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c20b0 con 0x7fc4f81546b0 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c2390 con 0x7fc4f81542d0 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4f80c2670 con 0x7fc4f80c5fc0 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56804/0 (socket says 172.21.15.73:56804) 2023-11-28T03:04:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2646324896 learned_addr learned my addr 172.21.15.73:0/2646324896 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4edd9b700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f81542d0 0x7fc4f80ae9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f659c700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f80c5fc0 0x7fc4f8147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f80c5fc0 msgr2=0x7fc4f8147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f80c5fc0 0x7fc4f8147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f81542d0 msgr2=0x7fc4f80ae9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f81542d0 0x7fc4f80ae9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 -- 172.21.15.73:0/2646324896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4f0009ae0 con 0x7fc4f81546b0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f6d9d700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc4f0009fd0 tx=0x7fc4f0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f0013e60 con 0x7fc4f81546b0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4f00092e0 con 0x7fc4f81546b0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4f80c2950 con 0x7fc4f81546b0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4f001e6c0 con 0x7fc4f81546b0 2023-11-28T03:04:21.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.534+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4f80c2f10 con 0x7fc4f81546b0 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.535+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4f80c3280 con 0x7fc4f81546b0 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.536+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc4f0003040 con 0x7fc4f81546b0 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.536+0000 7fc4eeffd700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 0x7fc4d8061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.536+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fc4f007ba80 con 0x7fc4f81546b0 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.536+0000 7fc4f659c700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 0x7fc4d8061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.536+0000 7fc4f659c700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 0x7fc4d8061750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc4e800bbc0 tx=0x7fc4e8008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.540+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc4f004cb00 con 0x7fc4f81546b0 2023-11-28T03:04:21.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.726+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4f806c7c0 con 0x7fc4d805f290 2023-11-28T03:04:21.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.729+0000 7fc4eeffd700 1 -- 172.21.15.73:0/2646324896 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc4f806c7c0 con 0x7fc4d805f290 2023-11-28T03:04:21.730 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 msgr2=0x7fc4d8061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 0x7fc4d8061750 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc4e800bbc0 tx=0x7fc4e8008040).stop 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 msgr2=0x7fc4f80aeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7fc4f0009fd0 tx=0x7fc4f0009840).stop 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 shutdown_connections 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4d805f290 0x7fc4d8061750 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4f81546b0 0x7fc4f80aeea0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4f80c5fc0 0x7fc4f8147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 --2- 172.21.15.73:0/2646324896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4f81542d0 0x7fc4f80ae9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.731+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 >> 172.21.15.73:0/2646324896 conn(0x7fc4f80cb010 msgr2=0x7fc4f80c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.732+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 shutdown_connections 2023-11-28T03:04:21.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.732+0000 7fc4f759e700 1 -- 172.21.15.73:0/2646324896 wait complete. 2023-11-28T03:04:21.744 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-11-28T03:04:21.744 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/dir9 1M 2023-11-28T03:04:21.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.861+0000 7ff147272700 1 Processor -- start 2023-11-28T03:04:21.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 -- start start 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff1400c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff140063af0 0x7ff140063f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff140158e20 con 0x7ff1401373a0 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1400c68a0 con 0x7ff140063af0 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff147272700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1401582a0 con 0x7ff1400c5f30 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.862+0000 7ff14580f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff140063af0 0x7ff140063f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff140063af0 0x7ff140063f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56812/0 (socket says 172.21.15.73:56812) 2023-11-28T03:04:21.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 -- 172.21.15.73:0/3467877157 learned_addr learned my addr 172.21.15.73:0/3467877157 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14480d700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff1400c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 -- 172.21.15.73:0/3467877157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 msgr2=0x7ff1400c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff1400c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 -- 172.21.15.73:0/3467877157 >> v1:172.21.15.150:6789/0 conn(0x7ff1401373a0 legacy=0x7ff1400c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 -- 172.21.15.73:0/3467877157 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140158580 con 0x7ff140063af0 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14480d700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff1400c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff14580f700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff140063f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7ff138009f20 tx=0x7ff138009840).ready entity=mon.0 client_cookie=176eaa53e72691f8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff137fff700 1 -- 172.21.15.73:0/3467877157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800e070 con 0x7ff140063af0 2023-11-28T03:04:21.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff137fff700 1 -- 172.21.15.73:0/3467877157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13800ba80 con 0x7ff140063af0 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.863+0000 7ff137fff700 1 -- 172.21.15.73:0/3467877157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13800be00 con 0x7ff140063af0 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 -- 172.21.15.73:0/3467877157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 msgr2=0x7ff140063f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff140063f80 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 rx=0x7ff138009f20 tx=0x7ff138009840).stop 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 -- 172.21.15.73:0/3467877157 shutdown_connections 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff140063f80 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 --2- 172.21.15.73:0/3467877157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff1400c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 -- 172.21.15.73:0/3467877157 >> 172.21.15.73:0/3467877157 conn(0x7ff1400628f0 msgr2=0x7ff140062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 -- 172.21.15.73:0/3467877157 shutdown_connections 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.864+0000 7ff147272700 1 -- 172.21.15.73:0/3467877157 wait complete. 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.865+0000 7ff147272700 1 Processor -- start 2023-11-28T03:04:21.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.865+0000 7ff147272700 1 -- start start 2023-11-28T03:04:21.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff140151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1401373a0 0x7ff140151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff140150600 con 0x7ff140063af0 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff14014f9f0 con 0x7ff1401373a0 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff14014fcd0 con 0x7ff1400c5f30 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56824/0 (socket says 172.21.15.73:56824) 2023-11-28T03:04:21.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 -- 172.21.15.73:0/45861072 learned_addr learned my addr 172.21.15.73:0/45861072 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:21.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14480d700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff140151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14580f700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1401373a0 0x7ff140151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 -- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 msgr2=0x7ff140151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff140151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 -- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1401373a0 msgr2=0x7ff140151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1401373a0 0x7ff140151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 -- 172.21.15.73:0/45861072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff138009ae0 con 0x7ff140063af0 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff14500e700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff13c00c440 tx=0x7ff13c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c007020 con 0x7ff140063af0 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff13c007180 con 0x7ff140063af0 2023-11-28T03:04:21.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff14014ffb0 con 0x7ff140063af0 2023-11-28T03:04:21.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.866+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff13c007520 con 0x7ff140063af0 2023-11-28T03:04:21.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.867+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff14014f6d0 con 0x7ff140063af0 2023-11-28T03:04:21.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff14006c860 con 0x7ff140063af0 2023-11-28T03:04:21.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7ff13c0068b0 con 0x7ff140063af0 2023-11-28T03:04:21.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff1357fa700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 0x7ff1240617b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:21.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7ff13c079730 con 0x7ff140063af0 2023-11-28T03:04:21.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff14480d700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 0x7ff1240617b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:21.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.868+0000 7ff14480d700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 0x7ff1240617b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff1400abd80 tx=0x7ff130008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:21.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:21.872+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff13c047af0 con 0x7ff140063af0 2023-11-28T03:04:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.062+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff140138200 con 0x7ff12405f2f0 2023-11-28T03:04:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.065+0000 7ff1357fa700 1 -- 172.21.15.73:0/45861072 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff140138200 con 0x7ff12405f2f0 2023-11-28T03:04:22.067 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-11-28T03:04:22.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 msgr2=0x7ff1240617b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 0x7ff1240617b0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff1400abd80 tx=0x7ff130008040).stop 2023-11-28T03:04:22.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 msgr2=0x7ff1400a8cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 rx=0x7ff13c00c440 tx=0x7ff13c007e00).stop 2023-11-28T03:04:22.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 shutdown_connections 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff12405f2f0 0x7ff1240617b0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1401373a0 0x7ff140151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff140063af0 0x7ff1400a8cd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 --2- 172.21.15.73:0/45861072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1400c5f30 0x7ff140151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 >> 172.21.15.73:0/45861072 conn(0x7ff1400628f0 msgr2=0x7ff1400646f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 shutdown_connections 2023-11-28T03:04:22.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.067+0000 7ff147272700 1 -- 172.21.15.73:0/45861072 wait complete. 2023-11-28T03:04:22.079 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-11-28T03:04:22.080 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:22.080 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-11-28T03:04:22.096 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:22.096 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:22.096 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:22.096 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:22.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:22.097 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:22.097 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir0 2023-11-28T03:04:22.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.251+0000 7f0259faa700 1 Processor -- start 2023-11-28T03:04:22.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.251+0000 7f0259faa700 1 -- start start 2023-11-28T03:04:22.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f0259faa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f02541373a0 0x7f02540c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f0259faa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f02540c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f0259faa700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02540c68d0 con 0x7f02540c5f30 2023-11-28T03:04:22.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f0259faa700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0254158e20 con 0x7f02541373a0 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f0259faa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02541582a0 con 0x7f0254064b00 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f02537fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f02541373a0 0x7f02540c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f02537fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f02541373a0 0x7f02540c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56828/0 (socket says 172.21.15.73:56828) 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.252+0000 7f02537fe700 1 -- 172.21.15.73:0/397911984 learned_addr learned my addr 172.21.15.73:0/397911984 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f0253fff700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f02540c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02537fe700 1 -- 172.21.15.73:0/397911984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 msgr2=0x7f02540c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02537fe700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f02540c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02537fe700 1 -- 172.21.15.73:0/397911984 >> v1:172.21.15.150:6789/0 conn(0x7f02540c5f30 legacy=0x7f02540669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02537fe700 1 -- 172.21.15.73:0/397911984 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254158580 con 0x7f02541373a0 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02537fe700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f02540c72e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f0244009f20 tx=0x7f0244009840).ready entity=mon.0 client_cookie=7999208481e8f8c6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02527fc700 1 -- 172.21.15.73:0/397911984 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400e070 con 0x7f02541373a0 2023-11-28T03:04:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02527fc700 1 -- 172.21.15.73:0/397911984 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024400ba40 con 0x7f02541373a0 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f02527fc700 1 -- 172.21.15.73:0/397911984 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024400bdc0 con 0x7f02541373a0 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f0259faa700 1 -- 172.21.15.73:0/397911984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 msgr2=0x7f02540c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.253+0000 7f0259faa700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f02540c72e0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 rx=0x7f0244009f20 tx=0x7f0244009840).stop 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 -- 172.21.15.73:0/397911984 shutdown_connections 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f02540c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 --2- 172.21.15.73:0/397911984 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f02540c72e0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 -- 172.21.15.73:0/397911984 >> 172.21.15.73:0/397911984 conn(0x7f02540628f0 msgr2=0x7f0254062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:22.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 -- 172.21.15.73:0/397911984 shutdown_connections 2023-11-28T03:04:22.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.254+0000 7f0259faa700 1 -- 172.21.15.73:0/397911984 wait complete. 2023-11-28T03:04:22.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.255+0000 7f0259faa700 1 Processor -- start 2023-11-28T03:04:22.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.255+0000 7f0259faa700 1 -- start start 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f0254136b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 0x7f0254138350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254137080 con 0x7f02541373a0 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0254139ba0 con 0x7f02540c5f30 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02541518d0 con 0x7f0254064b00 2023-11-28T03:04:22.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56838/0 (socket says 172.21.15.73:56838) 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 -- 172.21.15.73:0/2405137849 learned_addr learned my addr 172.21.15.73:0/2405137849 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f02537fe700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f0254136b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0252ffd700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 0x7f0254138350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 -- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 msgr2=0x7f0254136b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f0254136b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 -- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 msgr2=0x7f0254138350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 0x7f0254138350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 -- 172.21.15.73:0/2405137849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0244009ae0 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0252ffd700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 0x7f0254138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0253fff700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f024c00c650 tx=0x7f024c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c010580 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c0106e0 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c010a60 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.256+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0254138d30 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.257+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0254139010 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.257+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025406c860 con 0x7f02541373a0 2023-11-28T03:04:22.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.259+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f024c01b460 con 0x7f02541373a0 2023-11-28T03:04:22.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.260+0000 7f023b7fe700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 0x7f0240061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.260+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f024c07ad30 con 0x7f02541373a0 2023-11-28T03:04:22.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.260+0000 7f02537fe700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 0x7f0240061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.260+0000 7f02537fe700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 0x7f0240061750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f024400c010 tx=0x7f0244009840).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:04:22.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.263+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f024c04be10 con 0x7f02541373a0 2023-11-28T03:04:22.269 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:04:22.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.455+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0254158860 con 0x7f024005f290 2023-11-28T03:04:22.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.457+0000 7f023b7fe700 1 -- 172.21.15.73:0/2405137849 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f0254158860 con 0x7f024005f290 2023-11-28T03:04:22.458 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:22.458 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:44", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:22.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 msgr2=0x7f0240061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 0x7f0240061750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f024400c010 tx=0x7f0244009840).stop 2023-11-28T03:04:22.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 msgr2=0x7f0254138840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 rx=0x7f024c00c650 tx=0x7f024c008040).stop 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 shutdown_connections 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f024005f290 0x7f0240061750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f02541373a0 0x7f0254138840 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0254064b00 0x7f0254136b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 --2- 172.21.15.73:0/2405137849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f02540c5f30 0x7f0254138350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 >> 172.21.15.73:0/2405137849 conn(0x7f02540628f0 msgr2=0x7f02540aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 shutdown_connections 2023-11-28T03:04:22.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.459+0000 7f0259faa700 1 -- 172.21.15.73:0/2405137849 wait complete. 2023-11-28T03:04:22.472 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:44", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:22.472 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:44', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:22.472 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:22.472 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:22.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:22.472 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:22.488 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:22.489 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:22.489 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:22.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir1 2023-11-28T03:04:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.643+0000 7f7f75d78700 1 Processor -- start 2023-11-28T03:04:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 -- start start 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f70154530 0x7f7f70154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f700aa220 0x7f7f700aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f700c6630 con 0x7f7f700a7ff0 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f700c6390 con 0x7f7f70154530 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.644+0000 7f7f75d78700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f700aaba0 con 0x7f7f700aa220 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7f70154530 0x7f7f70154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f67fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7f700a7ff0 0x7f7f700c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50984/0 (socket says 172.21.15.73:50984) 2023-11-28T03:04:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f67fff700 1 -- 172.21.15.73:0/391686483 learned_addr learned my addr 172.21.15.73:0/391686483 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 -- 172.21.15.73:0/391686483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f700aa220 msgr2=0x7f7f700aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 --2- 172.21.15.73:0/391686483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f700aa220 0x7f7f700aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 -- 172.21.15.73:0/391686483 >> v1:172.21.15.73:6789/0 conn(0x7f7f700a7ff0 legacy=0x7f7f700c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6effd700 1 -- 172.21.15.73:0/391686483 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996207495 0 0) 0x7f7f700c6630 con 0x7f7f700a7ff0 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 -- 172.21.15.73:0/391686483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f700a8470 con 0x7f7f70154530 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6f7fe700 1 --2- 172.21.15.73:0/391686483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f70154530 0x7f7f70154910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f7f60006fd0 tx=0x7f7f60006f40).ready entity=mon.0 client_cookie=7d326065f7e5d826 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6effd700 1 -- 172.21.15.73:0/391686483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60005880 con 0x7f7f70154530 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6effd700 1 -- 172.21.15.73:0/391686483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f600059e0 con 0x7f7f70154530 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.645+0000 7f7f6effd700 1 -- 172.21.15.73:0/391686483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f60005d60 con 0x7f7f70154530 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 -- 172.21.15.73:0/391686483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f70154530 msgr2=0x7f7f70154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 --2- 172.21.15.73:0/391686483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f70154530 0x7f7f70154910 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 rx=0x7f7f60006fd0 tx=0x7f7f60006f40).stop 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 -- 172.21.15.73:0/391686483 shutdown_connections 2023-11-28T03:04:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 --2- 172.21.15.73:0/391686483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f700aa220 0x7f7f700aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 --2- 172.21.15.73:0/391686483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f70154530 0x7f7f70154910 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 -- 172.21.15.73:0/391686483 >> 172.21.15.73:0/391686483 conn(0x7f7f70094930 msgr2=0x7f7f7006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 -- 172.21.15.73:0/391686483 shutdown_connections 2023-11-28T03:04:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.646+0000 7f7f75d78700 1 -- 172.21.15.73:0/391686483 wait complete. 2023-11-28T03:04:22.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.647+0000 7f7f75d78700 1 Processor -- start 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.647+0000 7f7f75d78700 1 -- start start 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700a7ff0 0x7f7f70164660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f70154530 0x7f7f7014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f7014f180 con 0x7f7f700aa220 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f70165ec0 con 0x7f7f700a7ff0 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f701661a0 con 0x7f7f70154530 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56854/0 (socket says 172.21.15.73:56854) 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 -- 172.21.15.73:0/3597988088 learned_addr learned my addr 172.21.15.73:0/3597988088 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f6f7fe700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700a7ff0 0x7f7f70164660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f6ffff700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f70154530 0x7f7f7014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 -- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f70154530 msgr2=0x7f7f7014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f70154530 0x7f7f7014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 -- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700a7ff0 msgr2=0x7f7f70164660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700a7ff0 0x7f7f70164660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 -- 172.21.15.73:0/3597988088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f60009250 con 0x7f7f700aa220 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f67fff700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f7f5800c4e0 tx=0x7f7f58007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f580072e0 con 0x7f7f700aa220 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f58007440 con 0x7f7f700aa220 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f70166480 con 0x7f7f700aa220 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f580057a0 con 0x7f7f700aa220 2023-11-28T03:04:22.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.648+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70166a40 con 0x7f7f700aa220 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.649+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f70166c60 con 0x7f7f700aa220 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.649+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f7f58005bd0 con 0x7f7f700aa220 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.650+0000 7f7f677fe700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 0x7f7f50061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.650+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f7f58079f60 con 0x7f7f700aa220 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.650+0000 7f7f6f7fe700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 0x7f7f50061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:22.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.650+0000 7f7f6f7fe700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 0x7f7f50061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7f60006f80 tx=0x7f7f60016000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:22.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.654+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7f5804afe0 con 0x7f7f700aa220 2023-11-28T03:04:22.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.838+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7f7006c860 con 0x7f7f5005f2c0 2023-11-28T03:04:22.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.840+0000 7f7f677fe700 1 -- 172.21.15.73:0/3597988088 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7f7006c860 con 0x7f7f5005f2c0 2023-11-28T03:04:22.841 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:22.841 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 msgr2=0x7f7f50061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 0x7f7f50061780 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f7f60006f80 tx=0x7f7f60016000).stop 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 msgr2=0x7f7f70165850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 rx=0x7f7f5800c4e0 tx=0x7f7f58007c70).stop 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 shutdown_connections 2023-11-28T03:04:22.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f5005f2c0 0x7f7f50061780 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f70154530 0x7f7f7014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f700a7ff0 0x7f7f70164660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 --2- 172.21.15.73:0/3597988088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f700aa220 0x7f7f70165850 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 >> 172.21.15.73:0/3597988088 conn(0x7f7f70094930 msgr2=0x7f7f7014fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 shutdown_connections 2023-11-28T03:04:22.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:22.842+0000 7f7f75d78700 1 -- 172.21.15.73:0/3597988088 wait complete. 2023-11-28T03:04:22.855 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:22.855 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:45', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:22.855 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:22.855 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:22.855 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:22.855 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-11-28T03:04:22.871 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:22.871 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:22.871 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:22.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:22.873 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:22.873 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir2 2023-11-28T03:04:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.026+0000 7fcb1d02f700 1 Processor -- start 2023-11-28T03:04:23.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.026+0000 7fcb1d02f700 1 -- start start 2023-11-28T03:04:23.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1d02f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcb18154530 0x7fcb18154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1d02f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb180aa220 0x7fcb180aa690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1d02f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb180c6630 con 0x7fcb180a7ff0 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1d02f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180c6390 con 0x7fcb180aa220 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1d02f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb180a8470 con 0x7fcb18154530 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb180aa220 0x7fcb180aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb180aa220 0x7fcb180aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56858/0 (socket says 172.21.15.73:56858) 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 -- 172.21.15.73:0/2770055888 learned_addr learned my addr 172.21.15.73:0/2770055888 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb16d9d700 1 --2- 172.21.15.73:0/2770055888 >> v2:172.21.15.150:3300/0 conn(0x7fcb18154530 0x7fcb18154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 -- 172.21.15.73:0/2770055888 >> v2:172.21.15.150:3300/0 conn(0x7fcb18154530 msgr2=0x7fcb18154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 --2- 172.21.15.73:0/2770055888 >> v2:172.21.15.150:3300/0 conn(0x7fcb18154530 0x7fcb18154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 -- 172.21.15.73:0/2770055888 >> v1:172.21.15.150:6789/0 conn(0x7fcb180a7ff0 legacy=0x7fcb180c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 -- 172.21.15.73:0/2770055888 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb180aab80 con 0x7fcb180aa220 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb1759e700 1 --2- 172.21.15.73:0/2770055888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb180aa690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fcb180c9080 tx=0x7fcb10009840).ready entity=mon.0 client_cookie=b78737603e6f2cd7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb15d9b700 1 -- 172.21.15.73:0/2770055888 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000e070 con 0x7fcb180aa220 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb15d9b700 1 -- 172.21.15.73:0/2770055888 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb1000ba10 con 0x7fcb180aa220 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.027+0000 7fcb15d9b700 1 -- 172.21.15.73:0/2770055888 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb1000bd90 con 0x7fcb180aa220 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 -- 172.21.15.73:0/2770055888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 msgr2=0x7fcb180aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/2770055888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb180aa690 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 rx=0x7fcb180c9080 tx=0x7fcb10009840).stop 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 -- 172.21.15.73:0/2770055888 shutdown_connections 2023-11-28T03:04:23.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/2770055888 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb180aa690 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/2770055888 >> v2:172.21.15.150:3300/0 conn(0x7fcb18154530 0x7fcb18154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 -- 172.21.15.73:0/2770055888 >> 172.21.15.73:0/2770055888 conn(0x7fcb18094930 msgr2=0x7fcb1806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 -- 172.21.15.73:0/2770055888 shutdown_connections 2023-11-28T03:04:23.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.028+0000 7fcb1d02f700 1 -- 172.21.15.73:0/2770055888 wait complete. 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.029+0000 7fcb1d02f700 1 Processor -- start 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.029+0000 7fcb1d02f700 1 -- start start 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 0x7fcb180c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 0x7fcb180acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180abbf0 con 0x7fcb180aa220 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180ac400 con 0x7fcb180a7ff0 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1d02f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb180c24e0 con 0x7fcb18154530 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56870/0 (socket says 172.21.15.73:56870) 2023-11-28T03:04:23.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 -- 172.21.15.73:0/369398037 learned_addr learned my addr 172.21.15.73:0/369398037 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb16d9d700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 0x7fcb180c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1759e700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 0x7fcb180acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 -- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 msgr2=0x7fcb180acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 0x7fcb180acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 -- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 msgr2=0x7fcb180c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 0x7fcb180c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 -- 172.21.15.73:0/369398037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb10009ae0 con 0x7fcb180aa220 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb16d9d700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 0x7fcb180c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1759e700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 0x7fcb180acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.030+0000 7fcb1659c700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fcb0c00b4c0 tx=0x7fcb0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.031+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0090d0 con 0x7fcb180aa220 2023-11-28T03:04:23.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.031+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb0c012410 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.031+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb0c0127d0 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.031+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb180c4ff0 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.031+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb180ac120 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.032+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb1806c860 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.032+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fcb0c012b80 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.032+0000 7fcafeffd700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 0x7fcb04061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.032+0000 7fcb16d9d700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 0x7fcb04061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.032+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7fcb0c07a110 con 0x7fcb180aa220 2023-11-28T03:04:23.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.033+0000 7fcb16d9d700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 0x7fcb04061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcb0800a960 tx=0x7fcb08008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.036+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb0c04b190 con 0x7fcb180aa220 2023-11-28T03:04:23.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.220+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcb180c2620 con 0x7fcb0405f2c0 2023-11-28T03:04:23.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.222+0000 7fcafeffd700 1 -- 172.21.15.73:0/369398037 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fcb180c2620 con 0x7fcb0405f2c0 2023-11-28T03:04:23.223 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:23.223 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:23.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 msgr2=0x7fcb04061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 0x7fcb04061780 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fcb0800a960 tx=0x7fcb08008040).stop 2023-11-28T03:04:23.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 msgr2=0x7fcb18151500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fcb0c00b4c0 tx=0x7fcb0c009060).stop 2023-11-28T03:04:23.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 shutdown_connections 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb0405f2c0 0x7fcb04061780 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb18154530 0x7fcb180acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb180a7ff0 0x7fcb180c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 --2- 172.21.15.73:0/369398037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb180aa220 0x7fcb18151500 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 >> 172.21.15.73:0/369398037 conn(0x7fcb18094930 msgr2=0x7fcb18152d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 shutdown_connections 2023-11-28T03:04:23.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.224+0000 7fcb1d02f700 1 -- 172.21.15.73:0/369398037 wait complete. 2023-11-28T03:04:23.238 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:23.238 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:45', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:23.238 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:23.238 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:23.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:23.238 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:23.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:23.255 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:23.255 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:23.255 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:23.255 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:23.255 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:23.255 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:23.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir3 2023-11-28T03:04:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.410+0000 7f37cd18c700 1 Processor -- start 2023-11-28T03:04:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.410+0000 7f37cd18c700 1 -- start start 2023-11-28T03:04:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c80a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37c81546b0 0x7f37c80aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f37c80c5fc0 0x7f37c80c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c80aa950 con 0x7f37c81546b0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37c8158d50 con 0x7f37c80c5fc0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cd18c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8158140 con 0x7f37c81542d0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37c81546b0 0x7f37c80aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f37c81546b0 0x7f37c80aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49736/0 (socket says 172.21.15.73:49736) 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 -- 172.21.15.73:0/2845475686 learned_addr learned my addr 172.21.15.73:0/2845475686 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c7fff700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c80a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 -- 172.21.15.73:0/2845475686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 msgr2=0x7f37c80a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c80a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37cc98b700 1 --2- 172.21.15.73:0/2845475686 >> v2:172.21.15.150:3300/0 conn(0x7f37c80c5fc0 0x7f37c80c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 -- 172.21.15.73:0/2845475686 >> v2:172.21.15.150:3300/0 conn(0x7f37c80c5fc0 msgr2=0x7f37c80c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2845475686 >> v2:172.21.15.150:3300/0 conn(0x7f37c80c5fc0 0x7f37c80c6470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 -- 172.21.15.73:0/2845475686 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37c8158420 con 0x7f37c81546b0 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c81546b0 0x7f37c80aa460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f37c8154b30 tx=0x7f37bc0090f0).ready entity=mon.0 client_cookie=2a515fd6df9167db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.411+0000 7f37c6ffd700 1 -- 172.21.15.73:0/2845475686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00b600 con 0x7f37c81546b0 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37c6ffd700 1 -- 172.21.15.73:0/2845475686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37bc00b760 con 0x7f37c81546b0 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37c6ffd700 1 -- 172.21.15.73:0/2845475686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37bc00bae0 con 0x7f37c81546b0 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 -- 172.21.15.73:0/2845475686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c81546b0 msgr2=0x7f37c80aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c81546b0 0x7f37c80aa460 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 rx=0x7f37c8154b30 tx=0x7f37bc0090f0).stop 2023-11-28T03:04:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 -- 172.21.15.73:0/2845475686 shutdown_connections 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2845475686 >> v2:172.21.15.150:3300/0 conn(0x7f37c80c5fc0 0x7f37c80c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c80a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2845475686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c81546b0 0x7f37c80aa460 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 -- 172.21.15.73:0/2845475686 >> 172.21.15.73:0/2845475686 conn(0x7f37c80cb010 msgr2=0x7f37c80cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 -- 172.21.15.73:0/2845475686 shutdown_connections 2023-11-28T03:04:23.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.412+0000 7f37cd18c700 1 -- 172.21.15.73:0/2845475686 wait complete. 2023-11-28T03:04:23.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.413+0000 7f37cd18c700 1 Processor -- start 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.413+0000 7f37cd18c700 1 -- start start 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c814fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c81546b0 0x7f37c814e510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c814e270 con 0x7f37c80c5fc0 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37c813cf00 con 0x7f37c81546b0 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cd18c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37c8152630 con 0x7f37c81542d0 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49740/0 (socket says 172.21.15.73:49740) 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 -- 172.21.15.73:0/2162414148 learned_addr learned my addr 172.21.15.73:0/2162414148 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c814fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37cc98b700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c81546b0 0x7f37c814e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 -- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 msgr2=0x7f37c814fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c814fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 -- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c81546b0 msgr2=0x7f37c814e510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c81546b0 0x7f37c814e510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.414+0000 7f37c7fff700 1 -- 172.21.15.73:0/2162414148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37bc009370 con 0x7f37c80c5fc0 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37c7fff700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f37b800c650 tx=0x7f37b8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8010580 con 0x7f37c80c5fc0 2023-11-28T03:04:23.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37b80106e0 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c8152910 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37b8010a60 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.415+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37c813cb90 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.416+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37c806c7c0 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.416+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f37b801b460 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.416+0000 7f37affff700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 0x7f37b4061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.417+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 0x7f37b4061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.417+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f37b807a400 con 0x7f37c80c5fc0 2023-11-28T03:04:23.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.417+0000 7f37c77fe700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 0x7f37b4061750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f37c8154b30 tx=0x7f37bc017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.421+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f37b804b420 con 0x7f37c80c5fc0 2023-11-28T03:04:23.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.609+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f37c80c0120 con 0x7f37b405f290 2023-11-28T03:04:23.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.610+0000 7f37affff700 1 -- 172.21.15.73:0/2162414148 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f37c80c0120 con 0x7f37b405f290 2023-11-28T03:04:23.611 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:23.612 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 msgr2=0x7f37b4061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 0x7f37b4061750 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f37c8154b30 tx=0x7f37bc017000).stop 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 msgr2=0x7f37c813c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7f37b800c650 tx=0x7f37b8008040).stop 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 shutdown_connections 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37b405f290 0x7f37b4061750 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37c81546b0 0x7f37c814e510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37c80c5fc0 0x7f37c813c540 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 --2- 172.21.15.73:0/2162414148 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37c81542d0 0x7f37c814fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 >> 172.21.15.73:0/2162414148 conn(0x7f37c80cb010 msgr2=0x7f37c8153100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 shutdown_connections 2023-11-28T03:04:23.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.612+0000 7f37cd18c700 1 -- 172.21.15.73:0/2162414148 wait complete. 2023-11-28T03:04:23.625 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:45", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:23.625 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:45', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:23.625 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:23.625 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:23.626 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:23.626 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-11-28T03:04:23.641 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:23.642 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:23.642 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:23.642 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:23.645 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:23.645 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:23.645 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:23.645 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:23.645 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:23.646 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:23.646 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:23.646 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:23.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir4 2023-11-28T03:04:23.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.797+0000 7f4a888c5700 1 Processor -- start 2023-11-28T03:04:23.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.797+0000 7f4a888c5700 1 -- start start 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a888c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a841542d0 0x7f4a840a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a888c5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a841546b0 0x7f4a840aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a888c5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a84158d50 con 0x7f4a840c5fc0 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a888c5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a840aa950 con 0x7f4a841546b0 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a888c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84158140 con 0x7f4a841542d0 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a841546b0 0x7f4a840aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a841546b0 0x7f4a840aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49742/0 (socket says 172.21.15.73:49742) 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 -- 172.21.15.73:0/552014972 learned_addr learned my addr 172.21.15.73:0/552014972 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a837fe700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a841542d0 0x7f4a840a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 -- 172.21.15.73:0/552014972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a841542d0 msgr2=0x7f4a840a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a841542d0 0x7f4a840a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.798+0000 7f4a82ffd700 1 -- 172.21.15.73:0/552014972 >> v1:172.21.15.150:6789/0 conn(0x7f4a840c5fc0 legacy=0x7f4a840c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a82ffd700 1 -- 172.21.15.73:0/552014972 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a84158420 con 0x7f4a841546b0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a82ffd700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841546b0 0x7f4a840aa460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4a78006fd0 tx=0x7f4a780090f0).ready entity=mon.0 client_cookie=e15ff5adc07885f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a827fc700 1 -- 172.21.15.73:0/552014972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800b600 con 0x7f4a841546b0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a827fc700 1 -- 172.21.15.73:0/552014972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800b760 con 0x7f4a841546b0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a827fc700 1 -- 172.21.15.73:0/552014972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800bae0 con 0x7f4a841546b0 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 -- 172.21.15.73:0/552014972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841546b0 msgr2=0x7f4a840aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841546b0 0x7f4a840aa460 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 rx=0x7f4a78006fd0 tx=0x7f4a780090f0).stop 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 -- 172.21.15.73:0/552014972 shutdown_connections 2023-11-28T03:04:23.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a841542d0 0x7f4a840a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 --2- 172.21.15.73:0/552014972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841546b0 0x7f4a840aa460 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 -- 172.21.15.73:0/552014972 >> 172.21.15.73:0/552014972 conn(0x7f4a840cb010 msgr2=0x7f4a840cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 -- 172.21.15.73:0/552014972 shutdown_connections 2023-11-28T03:04:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.799+0000 7f4a888c5700 1 -- 172.21.15.73:0/552014972 wait complete. 2023-11-28T03:04:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 Processor -- start 2023-11-28T03:04:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 -- start start 2023-11-28T03:04:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 0x7f4a84144e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a841546b0 0x7f4a840ac990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b0f70 con 0x7f4a841542d0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84145340 con 0x7f4a841546b0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a888c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a84155490 con 0x7f4a840c5fc0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.801+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49758/0 (socket says 172.21.15.73:49758) 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 -- 172.21.15.73:0/1820402154 learned_addr learned my addr 172.21.15.73:0/1820402154 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a837fe700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 0x7f4a84144e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a83fff700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a841546b0 0x7f4a840ac990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 -- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 msgr2=0x7f4a84144e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 0x7f4a84144e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 -- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a841546b0 msgr2=0x7f4a840ac990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a841546b0 0x7f4a840ac990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 -- 172.21.15.73:0/1820402154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a78009370 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a837fe700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 0x7f4a84144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a82ffd700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4a84060090 tx=0x7f4a78017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7800b8d0 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a7800ba30 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840c4bb0 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a7801b660 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.802+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a84154e40 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.803+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8406c7c0 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.803+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f4a7801b9d0 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.804+0000 7f4a6b7fe700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.804+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f4a7807bec0 con 0x7f4a841542d0 2023-11-28T03:04:23.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.804+0000 7f4a837fe700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.804+0000 7f4a837fe700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 0x7f4a70061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4a840aa290 tx=0x7f4a7400b040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:23.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.808+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4a7804cf40 con 0x7f4a841542d0 2023-11-28T03:04:23.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.993+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4a841555d0 con 0x7f4a7005f2c0 2023-11-28T03:04:23.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.995+0000 7f4a6b7fe700 1 -- 172.21.15.73:0/1820402154 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f4a841555d0 con 0x7f4a7005f2c0 2023-11-28T03:04:23.996 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:23.996 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:46", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 msgr2=0x7f4a70061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 0x7f4a70061780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f4a840aa290 tx=0x7f4a7400b040).stop 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 msgr2=0x7f4a840adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 rx=0x7f4a84060090 tx=0x7f4a78017000).stop 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 shutdown_connections 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4a7005f2c0 0x7f4a70061780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a841546b0 0x7f4a840ac990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a840c5fc0 0x7f4a84144e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 --2- 172.21.15.73:0/1820402154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a841542d0 0x7f4a840adfc0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:23.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 >> 172.21.15.73:0/1820402154 conn(0x7f4a840cb010 msgr2=0x7f4a840ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:23.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 shutdown_connections 2023-11-28T03:04:23.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:23.997+0000 7f4a888c5700 1 -- 172.21.15.73:0/1820402154 wait complete. 2023-11-28T03:04:24.009 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:46", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:24.010 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:46', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:24.010 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:24.010 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:24.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:24.010 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:24.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:24.027 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:24.027 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:24.027 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir5 2023-11-28T03:04:24.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.180+0000 7f54c3bc6700 1 Processor -- start 2023-11-28T03:04:24.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.180+0000 7f54c3bc6700 1 -- start start 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c3bc6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54bc154910 0x7f54bc0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c3bc6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54bc0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c3bc6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54bc0c6630 con 0x7f54bc154530 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c3bc6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54bc0c6390 con 0x7f54bc154910 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c3bc6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0a84e0 con 0x7f54bc0aa220 2023-11-28T03:04:24.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c1962700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f54bc154530 0x7f54bc0c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48154/0 (socket says 172.21.15.73:48154) 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c1962700 1 -- 172.21.15.73:0/988299575 learned_addr learned my addr 172.21.15.73:0/988299575 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c1161700 1 --2- 172.21.15.73:0/988299575 >> v2:172.21.15.150:3300/0 conn(0x7f54bc154910 0x7f54bc0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c2163700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54bc0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656245355 0 0) 0x7f54bc0c6630 con 0x7f54bc154530 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54a80033f0 con 0x7f54bc154530 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 654134196 0 0) 0x7f54a80033f0 con 0x7f54bc154530 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 msgr2=0x7f54bc0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54bc0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 >> v2:172.21.15.150:3300/0 conn(0x7f54bc154910 msgr2=0x7f54bc0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 --2- 172.21.15.73:0/988299575 >> v2:172.21.15.150:3300/0 conn(0x7f54bc154910 0x7f54bc0a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0aaba0 con 0x7f54bc154530 2023-11-28T03:04:24.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.181+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f54b80015f0 con 0x7f54bc154530 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 >> v1:172.21.15.73:6789/0 conn(0x7f54bc154530 legacy=0x7f54bc0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54a80038e0 0x7f54a8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54a8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a8004b80 0x7f54a8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0c6390 con 0x7f54a80038e0 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0a84e0 con 0x7f54a8004b80 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54a800b8e0 con 0x7f54bc0aa220 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54a80038e0 0x7f54a8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1161700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54a8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c2163700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a8004b80 0x7f54a8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 -- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 msgr2=0x7f54a8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0aa220 0x7f54a8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 -- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a8004b80 msgr2=0x7f54a8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a8004b80 0x7f54a8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 -- 172.21.15.73:0/988299575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54a80033f0 con 0x7f54a80038e0 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c2163700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a8004b80 0x7f54a8005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c1962700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54a80038e0 0x7f54a8003d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f54b800c210 tx=0x7f54b80077a0).ready entity=mon.0 client_cookie=817ec4014cf5474 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b80079d0 con 0x7f54a80038e0 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b800cae0 con 0x7f54a80038e0 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.182+0000 7f54c0960700 1 -- 172.21.15.73:0/988299575 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b800ce60 con 0x7f54a80038e0 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 -- 172.21.15.73:0/988299575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54a80038e0 msgr2=0x7f54a8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/988299575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54a80038e0 0x7f54a8003d00 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 rx=0x7f54b800c210 tx=0x7f54b80077a0).stop 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c2163700 1 -- 172.21.15.73:0/988299575 reap_dead start 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 -- 172.21.15.73:0/988299575 shutdown_connections 2023-11-28T03:04:24.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 -- 172.21.15.73:0/988299575 >> 172.21.15.73:0/988299575 conn(0x7f54bc094930 msgr2=0x7f54bc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:24.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 -- 172.21.15.73:0/988299575 shutdown_connections 2023-11-28T03:04:24.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.183+0000 7f54c3bc6700 1 -- 172.21.15.73:0/988299575 wait complete. 2023-11-28T03:04:24.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.184+0000 7f54c3bc6700 1 Processor -- start 2023-11-28T03:04:24.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.184+0000 7f54c3bc6700 1 -- start start 2023-11-28T03:04:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a80038e0 0x7f54bc14ff60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0c4e70 0x7f54bc0ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc150450 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0c2df0 con 0x7f54a80038e0 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54bc0c30d0 con 0x7f54bc0c4e70 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49774/0 (socket says 172.21.15.73:49774) 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 -- 172.21.15.73:0/2752813691 learned_addr learned my addr 172.21.15.73:0/2752813691 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c1161700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0c4e70 0x7f54bc0ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c1962700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a80038e0 0x7f54bc14ff60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 -- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0c4e70 msgr2=0x7f54bc0ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0c4e70 0x7f54bc0ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 -- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a80038e0 msgr2=0x7f54bc14ff60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a80038e0 0x7f54bc14ff60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 -- 172.21.15.73:0/2752813691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54bc0c33b0 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c2163700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f54bc0c6180 tx=0x7f54b400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b400e030 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54b400f040 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54bc0c4470 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.185+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54b4010d00 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.186+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54bc0abb30 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.186+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54bc0c35d0 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.187+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f54b4015690 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.187+0000 7f54b1ffb700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 0x7f54a0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.187+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f54b407ad60 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.187+0000 7f54c1962700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 0x7f54a0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.187+0000 7f54c1962700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 0x7f54a0061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f54b80012a0 tx=0x7f54b8013000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.191+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f54b404bde0 con 0x7f54bc0ad1f0 2023-11-28T03:04:24.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.383+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54bc065310 con 0x7f54a005f2c0 2023-11-28T03:04:24.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.384+0000 7f54b1ffb700 1 -- 172.21.15.73:0/2752813691 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f54bc065310 con 0x7f54a005f2c0 2023-11-28T03:04:24.385 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:24.386 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:46", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:24.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 msgr2=0x7f54a0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 0x7f54a0061780 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f54b80012a0 tx=0x7f54b8013000).stop 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 msgr2=0x7f54bc0ad670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f54bc0c6180 tx=0x7f54b400b040).stop 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 shutdown_connections 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54a005f2c0 0x7f54a0061780 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54bc0ad1f0 0x7f54bc0ad670 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54a80038e0 0x7f54bc14ff60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 --2- 172.21.15.73:0/2752813691 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54bc0c4e70 0x7f54bc0ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 >> 172.21.15.73:0/2752813691 conn(0x7f54bc094930 msgr2=0x7f54bc1534f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 shutdown_connections 2023-11-28T03:04:24.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.386+0000 7f54c3bc6700 1 -- 172.21.15.73:0/2752813691 wait complete. 2023-11-28T03:04:24.399 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:46", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:24.399 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:46', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:24.399 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:24.399 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:24.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:24.400 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-11-28T03:04:24.416 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:24.417 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:24.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:24.419 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:24.419 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir6 2023-11-28T03:04:24.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.571+0000 7f3e5a85c700 1 Processor -- start 2023-11-28T03:04:24.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.571+0000 7f3e5a85c700 1 -- start start 2023-11-28T03:04:24.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5a85c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 0x7f3e540a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5a85c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e541546b0 0x7f3e540aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5a85c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e54158d50 con 0x7f3e540c5fc0 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5a85c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540c6b60 con 0x7f3e541546b0 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5a85c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e540aa950 con 0x7f3e541542d0 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e541546b0 0x7f3e540aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e541546b0 0x7f3e540aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49786/0 (socket says 172.21.15.73:49786) 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 -- 172.21.15.73:0/1795994275 learned_addr learned my addr 172.21.15.73:0/1795994275 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e5985a700 1 --2- 172.21.15.73:0/1795994275 >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 0x7f3e540a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 -- 172.21.15.73:0/1795994275 >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 msgr2=0x7f3e540a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 --2- 172.21.15.73:0/1795994275 >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 0x7f3e540a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.572+0000 7f3e59059700 1 -- 172.21.15.73:0/1795994275 >> v1:172.21.15.150:6789/0 conn(0x7f3e540c5fc0 legacy=0x7f3e540c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e59059700 1 -- 172.21.15.73:0/1795994275 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e54158140 con 0x7f3e541546b0 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5985a700 1 --2- 172.21.15.73:0/1795994275 >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 0x7f3e540a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e59059700 1 --2- 172.21.15.73:0/1795994275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541546b0 0x7f3e540aa460 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f3e540c07e0 tx=0x7f3e440090f0).ready entity=mon.0 client_cookie=74ce08ffb22b5a0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e58858700 1 -- 172.21.15.73:0/1795994275 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400b5a0 con 0x7f3e541546b0 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e58858700 1 -- 172.21.15.73:0/1795994275 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b700 con 0x7f3e541546b0 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e58858700 1 -- 172.21.15.73:0/1795994275 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400ba80 con 0x7f3e541546b0 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 -- 172.21.15.73:0/1795994275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541546b0 msgr2=0x7f3e540aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/1795994275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541546b0 0x7f3e540aa460 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 rx=0x7f3e540c07e0 tx=0x7f3e440090f0).stop 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 -- 172.21.15.73:0/1795994275 shutdown_connections 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/1795994275 >> v2:172.21.15.150:3300/0 conn(0x7f3e541542d0 0x7f3e540a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/1795994275 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541546b0 0x7f3e540aa460 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 -- 172.21.15.73:0/1795994275 >> 172.21.15.73:0/1795994275 conn(0x7f3e540cb010 msgr2=0x7f3e540cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 -- 172.21.15.73:0/1795994275 shutdown_connections 2023-11-28T03:04:24.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.573+0000 7f3e5a85c700 1 -- 172.21.15.73:0/1795994275 wait complete. 2023-11-28T03:04:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.574+0000 7f3e5a85c700 1 Processor -- start 2023-11-28T03:04:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 -- start start 2023-11-28T03:04:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e540c5fc0 0x7f3e5414de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e541546b0 0x7f3e540c0810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e541552c0 con 0x7f3e541542d0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e5414e340 con 0x7f3e541546b0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a85c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e540aef40 con 0x7f3e540c5fc0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e59059700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e59059700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49790/0 (socket says 172.21.15.73:49790) 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e59059700 1 -- 172.21.15.73:0/522559232 learned_addr learned my addr 172.21.15.73:0/522559232 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5985a700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e540c5fc0 0x7f3e5414de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.575+0000 7f3e5a05b700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e541546b0 0x7f3e540c0810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 -- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e540c5fc0 msgr2=0x7f3e5414de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e540c5fc0 0x7f3e5414de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 -- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e541546b0 msgr2=0x7f3e540c0810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e541546b0 0x7f3e540c0810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 -- 172.21.15.73:0/522559232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e44009370 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e59059700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f3e44009130 tx=0x7f3e44015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4400b870 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4400b9d0 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e540ae4b0 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4401c670 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.576+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e54154c70 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.577+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e54154a90 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.577+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f3e4401c9e0 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.578+0000 7f3e49ffb700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 0x7f3e38061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.578+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f3e4407c1b0 con 0x7f3e541542d0 2023-11-28T03:04:24.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.578+0000 7f3e5985a700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 0x7f3e38061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.578+0000 7f3e5985a700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 0x7f3e38061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3e5000abd0 tx=0x7f3e50008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.582+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e4404d230 con 0x7f3e541542d0 2023-11-28T03:04:24.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.772+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e5406c7c0 con 0x7f3e3805f2c0 2023-11-28T03:04:24.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.773+0000 7f3e49ffb700 1 -- 172.21.15.73:0/522559232 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3e5406c7c0 con 0x7f3e3805f2c0 2023-11-28T03:04:24.774 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:24.774 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:24.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 msgr2=0x7f3e38061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 0x7f3e38061780 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f3e5000abd0 tx=0x7f3e50008040).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 msgr2=0x7f3e540adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 rx=0x7f3e44009130 tx=0x7f3e44015000).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 shutdown_connections 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3805f2c0 0x7f3e38061780 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e541546b0 0x7f3e540c0810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e540c5fc0 0x7f3e5414de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 --2- 172.21.15.73:0/522559232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e541542d0 0x7f3e540adfc0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 >> 172.21.15.73:0/522559232 conn(0x7f3e540cb010 msgr2=0x7f3e5414e5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 shutdown_connections 2023-11-28T03:04:24.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.775+0000 7f3e5a85c700 1 -- 172.21.15.73:0/522559232 wait complete. 2023-11-28T03:04:24.788 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:24.788 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:47', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:24.788 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:24.788 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:24.789 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:24.789 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-11-28T03:04:24.804 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_01_UTC 2023-11-28T03:04:24.804 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:24.804 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:24.804 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:24.804 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:24.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:24.805 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_01_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:24.805 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir7 2023-11-28T03:04:24.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.960+0000 7f3e5599c700 1 Processor -- start 2023-11-28T03:04:24.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.960+0000 7f3e5599c700 1 -- start start 2023-11-28T03:04:24.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5599c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5599c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e501546b0 0x7f3e500c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5599c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e50158d50 con 0x7f3e501542d0 2023-11-28T03:04:24.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5599c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e50158140 con 0x7f3e501546b0 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5599c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e50158420 con 0x7f3e500aa680 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e5519b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3e501546b0 0x7f3e500c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:57398/0 (socket says 172.21.15.73:57398) 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 -- 172.21.15.73:0/553486065 learned_addr learned my addr 172.21.15.73:0/553486065 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 -- 172.21.15.73:0/553486065 >> v2:172.21.15.150:3300/0 conn(0x7f3e501546b0 msgr2=0x7f3e500c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 --2- 172.21.15.73:0/553486065 >> v2:172.21.15.150:3300/0 conn(0x7f3e501546b0 0x7f3e500c64b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 -- 172.21.15.73:0/553486065 >> v1:172.21.15.150:6789/0 conn(0x7f3e501542d0 legacy=0x7f3e500a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 -- 172.21.15.73:0/553486065 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e50158700 con 0x7f3e500aa680 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.961+0000 7f3e47fff700 1 --2- 172.21.15.73:0/553486065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3e40006fd0 tx=0x7f3e400090f0).ready entity=mon.2 client_cookie=c35c95665939ba5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e477fe700 1 -- 172.21.15.73:0/553486065 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000b580 con 0x7f3e500aa680 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e477fe700 1 -- 172.21.15.73:0/553486065 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4000b6e0 con 0x7f3e500aa680 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e477fe700 1 -- 172.21.15.73:0/553486065 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4000ba60 con 0x7f3e500aa680 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 -- 172.21.15.73:0/553486065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 msgr2=0x7f3e500c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 --2- 172.21.15.73:0/553486065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f3e40006fd0 tx=0x7f3e400090f0).stop 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 -- 172.21.15.73:0/553486065 shutdown_connections 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 --2- 172.21.15.73:0/553486065 >> v2:172.21.15.150:3300/0 conn(0x7f3e501546b0 0x7f3e500c64b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 --2- 172.21.15.73:0/553486065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e500aa680 0x7f3e500c5fc0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 -- 172.21.15.73:0/553486065 >> 172.21.15.73:0/553486065 conn(0x7f3e500cb010 msgr2=0x7f3e500cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 -- 172.21.15.73:0/553486065 shutdown_connections 2023-11-28T03:04:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.962+0000 7f3e5599c700 1 -- 172.21.15.73:0/553486065 wait complete. 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.963+0000 7f3e5599c700 1 Processor -- start 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.963+0000 7f3e5599c700 1 -- start start 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 0x7f3e50167e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e501546b0 0x7f3e50168330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500adfc0 con 0x7f3e500aa680 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e500ae580 con 0x7f3e501542d0 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5599c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e501482e0 con 0x7f3e501546b0 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5499a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5499a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49798/0 (socket says 172.21.15.73:49798) 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5499a700 1 -- 172.21.15.73:0/1399980921 learned_addr learned my addr 172.21.15.73:0/1399980921 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e5519b700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e501546b0 0x7f3e50168330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.964+0000 7f3e47fff700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 0x7f3e50167e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 -- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e501546b0 msgr2=0x7f3e50168330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e501546b0 0x7f3e50168330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 -- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 msgr2=0x7f3e50167e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 0x7f3e50167e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 -- 172.21.15.73:0/1399980921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e40009370 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e47fff700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 0x7f3e50167e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5499a700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3e4c00c870 tx=0x7f3e4c001660).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c006a20 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e4c006b80 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e50168900 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e4c007020 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.965+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e500ae210 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.966+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e5006c7c0 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.966+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f3e4c0073d0 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.966+0000 7f3e44ff9700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 0x7f3e3c061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.967+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f3e4c079dd0 con 0x7f3e500aa680 2023-11-28T03:04:24.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.967+0000 7f3e47fff700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 0x7f3e3c061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:24.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.967+0000 7f3e47fff700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 0x7f3e3c061750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3e500aaa80 tx=0x7f3e40017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:24.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:24.971+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e4c047af0 con 0x7f3e500aa680 2023-11-28T03:04:25.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.158+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3e500c0120 con 0x7f3e3c05f290 2023-11-28T03:04:25.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.160+0000 7f3e44ff9700 1 -- 172.21.15.73:0/1399980921 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f3e500c0120 con 0x7f3e3c05f290 2023-11-28T03:04:25.161 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:25.161 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:01", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 msgr2=0x7f3e3c061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 0x7f3e3c061750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3e500aaa80 tx=0x7f3e40017000).stop 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 msgr2=0x7f3e50147df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 rx=0x7f3e4c00c870 tx=0x7f3e4c001660).stop 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 shutdown_connections 2023-11-28T03:04:25.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e3c05f290 0x7f3e3c061750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e501546b0 0x7f3e50168330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e500aa680 0x7f3e50147df0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 --2- 172.21.15.73:0/1399980921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e501542d0 0x7f3e50167e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 >> 172.21.15.73:0/1399980921 conn(0x7f3e500cb010 msgr2=0x7f3e500c1770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 shutdown_connections 2023-11-28T03:04:25.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.162+0000 7f3e5599c700 1 -- 172.21.15.73:0/1399980921 wait complete. 2023-11-28T03:04:25.174 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:01", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.175 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:47', 'first': '2023-11-28T02:54:01', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:25.175 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:25.175 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:25.175 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:25.175 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-11-28T03:04:25.191 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:25.191 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:25.191 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:25.191 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:25.191 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:25.192 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:25.192 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:25.193 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir8 2023-11-28T03:04:25.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.346+0000 7f96b5163700 1 Processor -- start 2023-11-28T03:04:25.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.346+0000 7f96b5163700 1 -- start start 2023-11-28T03:04:25.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96b5163700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f96b006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96b5163700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0067840 con 0x7f96b00bd600 2023-11-28T03:04:25.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96b5163700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b0066c30 con 0x7f96b0154900 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96b5163700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0066f10 con 0x7f96b0154520 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96af59e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f96b00bd600 0x7f96b00c40d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48170/0 (socket says 172.21.15.73:48170) 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96af59e700 1 -- 172.21.15.73:0/1988772941 learned_addr learned my addr 172.21.15.73:0/1988772941 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f96b006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1626227412 0 0) 0x7f96b0067840 con 0x7f96b00bd600 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.347+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969c0033f0 con 0x7f96b00bd600 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1493141763 0 0) 0x7f969c0033f0 con 0x7f96b00bd600 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 msgr2=0x7f96b006a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f96b006a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 >> v1:172.21.15.150:6789/0 conn(0x7f96b0154900 legacy=0x7f96b00bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00671f0 con 0x7f96b00bd600 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f96a80017f0 con 0x7f96b00bd600 2023-11-28T03:04:25.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 >> v1:172.21.15.73:6789/0 conn(0x7f96b00bd600 legacy=0x7f96b00c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003910 0x7f969c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f969c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0066f10 con 0x7f969c004b80 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00b9c0 con 0x7f969c003910 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00bb80 con 0x7f96b0154520 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003910 0x7f969c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96a659c700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f969c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 -- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 msgr2=0x7f969c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b0154520 0x7f969c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 -- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003910 msgr2=0x7f969c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:25.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003910 0x7f969c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 -- 172.21.15.73:0/1988772941 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0033f0 con 0x7f969c004b80 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96af59e700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f96b00c5d90 tx=0x7f96a800c730).ready entity=mon.0 client_cookie=df341a8f2eaa916c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8002030 con 0x7f969c004b80 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800cd90 con 0x7f969c004b80 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.348+0000 7f96ae59c700 1 -- 172.21.15.73:0/1988772941 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8005d70 con 0x7f969c004b80 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 -- 172.21.15.73:0/1988772941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 msgr2=0x7f969c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 --2- 172.21.15.73:0/1988772941 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 rx=0x7f96b00c5d90 tx=0x7f96a800c730).stop 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96af59e700 1 -- 172.21.15.73:0/1988772941 reap_dead start 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 -- 172.21.15.73:0/1988772941 shutdown_connections 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 -- 172.21.15.73:0/1988772941 >> 172.21.15.73:0/1988772941 conn(0x7f96b0063810 msgr2=0x7f96b0063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 -- 172.21.15.73:0/1988772941 shutdown_connections 2023-11-28T03:04:25.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.349+0000 7f96b5163700 1 -- 172.21.15.73:0/1988772941 wait complete. 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.350+0000 7f96b5163700 1 Processor -- start 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.350+0000 7f96b5163700 1 -- start start 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b00c5150 0x7f96b00c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00a8c80 0x7f96b00a90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c4cc0 con 0x7f969c004b80 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c4110 con 0x7f96b00c5150 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96b5163700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c4390 con 0x7f96b00a8c80 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49812/0 (socket says 172.21.15.73:49812) 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 -- 172.21.15.73:0/2842059046 learned_addr learned my addr 172.21.15.73:0/2842059046 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 -- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00a8c80 msgr2=0x7f96b00a90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96af59e700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00a8c80 0x7f96b00a90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96a659c700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b00c5150 0x7f96b00c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00a8c80 0x7f96b00a90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 -- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b00c5150 msgr2=0x7f96b00c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b00c5150 0x7f96b00c55d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 -- 172.21.15.73:0/2842059046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00c4670 con 0x7f969c004b80 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.351+0000 7f96aed9d700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f96a0009fd0 tx=0x7f96a0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.352+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a000f940 con 0x7f969c004b80 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.352+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a000faa0 con 0x7f969c004b80 2023-11-28T03:04:25.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.352+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b00c4950 con 0x7f969c004b80 2023-11-28T03:04:25.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.352+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a000fe20 con 0x7f969c004b80 2023-11-28T03:04:25.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.352+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b00c27f0 con 0x7f969c004b80 2023-11-28T03:04:25.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.353+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b00c4fa0 con 0x7f969c004b80 2023-11-28T03:04:25.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.353+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f96a0014650 con 0x7f969c004b80 2023-11-28T03:04:25.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.353+0000 7f96a77fe700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 0x7f9690061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.353+0000 7f96a659c700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 0x7f9690061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.353+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f96a007b120 con 0x7f969c004b80 2023-11-28T03:04:25.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.354+0000 7f96a659c700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 0x7f9690061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f96b00c2190 tx=0x7f9698009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.357+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f96a004c140 con 0x7f969c004b80 2023-11-28T03:04:25.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.546+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96b00c4e00 con 0x7f969005f2c0 2023-11-28T03:04:25.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.547+0000 7f96a77fe700 1 -- 172.21.15.73:0/2842059046 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f9698003570 con 0x7f969005f2c0 2023-11-28T03:04:25.548 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:25.548 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 msgr2=0x7f9690061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 0x7f9690061780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f96b00c2190 tx=0x7f9698009040).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 msgr2=0x7f96b014dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 rx=0x7f96a0009fd0 tx=0x7f96a0009060).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 shutdown_connections 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f2c0 0x7f9690061780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00a8c80 0x7f96b00a90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f96b014dcf0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 --2- 172.21.15.73:0/2842059046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b00c5150 0x7f96b00c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 >> 172.21.15.73:0/2842059046 conn(0x7f96b0063810 msgr2=0x7f96b00c88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 shutdown_connections 2023-11-28T03:04:25.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.549+0000 7f96b5163700 1 -- 172.21.15.73:0/2842059046 wait complete. 2023-11-28T03:04:25.562 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:47", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.562 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:47', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:25.562 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:25.562 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:25.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:25.562 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:25.578 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:25.579 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:25.579 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:25.579 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:25.579 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-02_54_00_UTC', 'scheduled-2023-11-28-02_55_00_UTC', 'scheduled-2023-11-28-02_56_00_UTC', 'scheduled-2023-11-28-02_57_00_UTC', 'scheduled-2023-11-28-02_58_00_UTC', 'scheduled-2023-11-28-02_59_00_UTC', 'scheduled-2023-11-28-03_00_00_UTC', 'scheduled-2023-11-28-03_01_00_UTC', 'scheduled-2023-11-28-03_02_00_UTC', 'scheduled-2023-11-28-03_03_00_UTC', 'scheduled-2023-11-28-03_04_00_UTC'] 2023-11-28T03:04:25.579 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/dir9 2023-11-28T03:04:25.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.733+0000 7f26cd4c0700 1 Processor -- start 2023-11-28T03:04:25.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.733+0000 7f26cd4c0700 1 -- start start 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26cd4c0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f26c80aa930 0x7f26c80a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26cd4c0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c80c6630 con 0x7f26c8154910 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26cd4c0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26c8064230 con 0x7f26c8154530 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26cd4c0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26c80c6390 con 0x7f26c80aa930 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26be7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f26c8154910 0x7f26c80aa220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48178/0 (socket says 172.21.15.73:48178) 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26be7fc700 1 -- 172.21.15.73:0/2238835081 learned_addr learned my addr 172.21.15.73:0/2238835081 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26c77fe700 1 --2- 172.21.15.73:0/2238835081 >> v2:172.21.15.150:3300/0 conn(0x7f26c80aa930 0x7f26c80a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 801802320 0 0) 0x7f26c80c6630 con 0x7f26c8154910 2023-11-28T03:04:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.734+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f26b40033f0 con 0x7f26c8154910 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3946921175 0 0) 0x7f26b40033f0 con 0x7f26c8154910 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c77fe700 1 -- 172.21.15.73:0/2238835081 >> v1:172.21.15.150:6789/0 conn(0x7f26c8154530 legacy=0x7f26c80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c77fe700 1 -- 172.21.15.73:0/2238835081 >> v1:172.21.15.73:6789/0 conn(0x7f26c8154910 legacy=0x7f26c80aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c77fe700 1 -- 172.21.15.73:0/2238835081 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c8064510 con 0x7f26c80aa930 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c77fe700 1 --2- 172.21.15.73:0/2238835081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80a7ff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f26c0009e60 tx=0x7f26c0009780).ready entity=mon.1 client_cookie=4954ea2f26631067 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c000e070 con 0x7f26c80aa930 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26c000b850 con 0x7f26c80aa930 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26c67fc700 1 -- 172.21.15.73:0/2238835081 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0001470 con 0x7f26c80aa930 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2238835081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 msgr2=0x7f26c80a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.735+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2238835081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80a7ff0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 rx=0x7f26c0009e60 tx=0x7f26c0009780).stop 2023-11-28T03:04:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.736+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2238835081 shutdown_connections 2023-11-28T03:04:25.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.736+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2238835081 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80a7ff0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.736+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2238835081 >> 172.21.15.73:0/2238835081 conn(0x7f26c8094930 msgr2=0x7f26c806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:25.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.736+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2238835081 shutdown_connections 2023-11-28T03:04:25.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.736+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2238835081 wait complete. 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.737+0000 7f26cd4c0700 1 Processor -- start 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.737+0000 7f26cd4c0700 1 -- start start 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26c8154910 0x7f26c8150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80aad10 con 0x7f26c8154530 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80b03c0 con 0x7f26c80aa930 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26c80ae810 con 0x7f26c8154910 2023-11-28T03:04:25.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49820/0 (socket says 172.21.15.73:49820) 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 -- 172.21.15.73:0/2548883527 learned_addr learned my addr 172.21.15.73:0/2548883527 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26c6ffd700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26c77fe700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26c8154910 0x7f26c8150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 -- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26c8154910 msgr2=0x7f26c8150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26c8154910 0x7f26c8150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 -- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 msgr2=0x7f26c80ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 -- 172.21.15.73:0/2548883527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0009a20 con 0x7f26c8154530 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26be7fc700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f26b000c810 tx=0x7f26b0001b50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b0007170 con 0x7f26c8154530 2023-11-28T03:04:25.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26b00072d0 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26c80aeaf0 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26b00048c0 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.738+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26c80af0b0 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.739+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26c80aaec0 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.740+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f26b0004c70 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.740+0000 7f26bf7fe700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 0x7f26a8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.740+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f26b007a570 con 0x7f26c8154530 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.740+0000 7f26c6ffd700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 0x7f26a8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:25.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.741+0000 7f26c6ffd700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 0x7f26a8061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f26b8007fd0 tx=0x7f26b8007d30).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:25.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.744+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f26c80aaec0 con 0x7f26c8154530 2023-11-28T03:04:25.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.931+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26c80aaec0 con 0x7f26a805f2c0 2023-11-28T03:04:25.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.932+0000 7f26bf7fe700 1 -- 172.21.15.73:0/2548883527 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f26c80aaec0 con 0x7f26a805f2c0 2023-11-28T03:04:25.933 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:25.933 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:48", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 msgr2=0x7f26a8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 0x7f26a8061780 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f26b8007fd0 tx=0x7f26b8007d30).stop 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 msgr2=0x7f26c80afed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 rx=0x7f26b000c810 tx=0x7f26b0001b50).stop 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 shutdown_connections 2023-11-28T03:04:25.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f26a805f2c0 0x7f26a8061780 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f26c8154910 0x7f26c8150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f26c80aa930 0x7f26c80ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 --2- 172.21.15.73:0/2548883527 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f26c8154530 0x7f26c80afed0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 >> 172.21.15.73:0/2548883527 conn(0x7f26c8094930 msgr2=0x7f26c80c01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 shutdown_connections 2023-11-28T03:04:25.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:25.934+0000 7f26cd4c0700 1 -- 172.21.15.73:0/2548883527 wait complete. 2023-11-28T03:04:25.947 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T02:53:48", "first": "2023-11-28T02:54:00", "last": "2023-11-28T03:04:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-11-28T03:04:25.947 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T02:53:48', 'first': '2023-11-28T02:54:00', 'last': '2023-11-28T03:04:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-11-28T03:04:25.947 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T03:04:25.947 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T03:04:25.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir0 1M 2023-11-28T03:04:26.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.064+0000 7f0b644f0700 1 Processor -- start 2023-11-28T03:04:26.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.064+0000 7f0b644f0700 1 -- start start 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b644f0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0b5c154610 0x7f0b5c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b644f0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b5c0ab3b0 0x7f0b5c0c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b644f0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b5c159210 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b644f0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c0c0c70 con 0x7f0b5c0ab3b0 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b644f0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b5c0c2110 con 0x7f0b5c154610 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b5c0ab3b0 0x7f0b5c0c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:26.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b62a8d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b5c0aa1c0 0x7f0b5c063550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48180/0 (socket says 172.21.15.73:48180) 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b62a8d700 1 -- 172.21.15.73:0/974563953 learned_addr learned my addr 172.21.15.73:0/974563953 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b6228c700 1 --2- 172.21.15.73:0/974563953 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c154610 0x7f0b5c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 -- 172.21.15.73:0/974563953 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c154610 msgr2=0x7f0b5c1582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 --2- 172.21.15.73:0/974563953 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c154610 0x7f0b5c1582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b6128a700 1 -- 172.21.15.73:0/974563953 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4112111789 0 0) 0x7f0b5c159210 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 -- 172.21.15.73:0/974563953 >> v1:172.21.15.73:6789/0 conn(0x7f0b5c0aa1c0 legacy=0x7f0b5c063550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 -- 172.21.15.73:0/974563953 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b5c158970 con 0x7f0b5c0ab3b0 2023-11-28T03:04:26.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.065+0000 7f0b61a8b700 1 --2- 172.21.15.73:0/974563953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0ab3b0 0x7f0b5c0c80c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f0b5c062e70 tx=0x7f0b4c009060).ready entity=mon.0 client_cookie=44ec0c68b6992f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b6128a700 1 -- 172.21.15.73:0/974563953 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00f510 con 0x7f0b5c0ab3b0 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b6128a700 1 -- 172.21.15.73:0/974563953 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4c00f670 con 0x7f0b5c0ab3b0 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b6128a700 1 -- 172.21.15.73:0/974563953 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4c00f9f0 con 0x7f0b5c0ab3b0 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 -- 172.21.15.73:0/974563953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0ab3b0 msgr2=0x7f0b5c0c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 --2- 172.21.15.73:0/974563953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0ab3b0 0x7f0b5c0c80c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f0b5c062e70 tx=0x7f0b4c009060).stop 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 -- 172.21.15.73:0/974563953 shutdown_connections 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 --2- 172.21.15.73:0/974563953 >> v2:172.21.15.150:3300/0 conn(0x7f0b5c154610 0x7f0b5c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:26.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 --2- 172.21.15.73:0/974563953 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0ab3b0 0x7f0b5c0c80c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 -- 172.21.15.73:0/974563953 >> 172.21.15.73:0/974563953 conn(0x7f0b5c0655a0 msgr2=0x7f0b5c069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 -- 172.21.15.73:0/974563953 shutdown_connections 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.066+0000 7f0b644f0700 1 -- 172.21.15.73:0/974563953 wait complete. 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.067+0000 7f0b644f0700 1 Processor -- start 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.067+0000 7f0b644f0700 1 -- start start 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c154610 0x7f0b5c0b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c0b0da0 0x7f0b5c0afb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c0b1220 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c155a00 con 0x7f0b5c0b0da0 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b644f0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b5c154b10 con 0x7f0b5c154610 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:26.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49844/0 (socket says 172.21.15.73:49844) 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 -- 172.21.15.73:0/322502654 learned_addr learned my addr 172.21.15.73:0/322502654 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 -- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c154610 msgr2=0x7f0b5c0b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c154610 0x7f0b5c0b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 -- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c0b0da0 msgr2=0x7f0b5c0afb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c0b0da0 0x7f0b5c0afb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.068+0000 7f0b6228c700 1 -- 172.21.15.73:0/322502654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b4c0092a0 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b6228c700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f0b5800e900 tx=0x7f0b58009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800c640 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5800c7a0 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5c154df0 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5800cb20 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.069+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b5c1553b0 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.070+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5c06c860 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.070+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f0b5801b400 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.070+0000 7f0b527fc700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 0x7f0b40061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:26.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.070+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f0b5807a940 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.070+0000 7f0b61a8b700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 0x7f0b40061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:26.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.071+0000 7f0b61a8b700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 0x7f0b40061780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0b5c062e70 tx=0x7f0b4c017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:26.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.074+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0b5804b960 con 0x7f0b5c0aa1c0 2023-11-28T03:04:26.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.263+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b5c158c50 con 0x7f0b4005f2c0 2023-11-28T03:04:26.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.267+0000 7f0b527fc700 1 -- 172.21.15.73:0/322502654 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0b5c158c50 con 0x7f0b4005f2c0 2023-11-28T03:04:26.268 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-11-28T03:04:26.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 msgr2=0x7f0b40061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 0x7f0b40061780 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 rx=0x7f0b5c062e70 tx=0x7f0b4c017000).stop 2023-11-28T03:04:26.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 msgr2=0x7f0b5c137bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 rx=0x7f0b5800e900 tx=0x7f0b58009060).stop 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 shutdown_connections 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f2c0 0x7f0b40061780 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b5c0b0da0 0x7f0b5c0afb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b5c0aa1c0 0x7f0b5c137bc0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:26.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 --2- 172.21.15.73:0/322502654 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b5c154610 0x7f0b5c0b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 >> 172.21.15.73:0/322502654 conn(0x7f0b5c0655a0 msgr2=0x7f0b5c0c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 shutdown_connections 2023-11-28T03:04:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:26.269+0000 7f0b644f0700 1 -- 172.21.15.73:0/322502654 wait complete. 2023-11-28T03:04:26.284 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-11-28T03:04:26.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.284 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-11-28T03:04:26.299 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:26.299 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:26.299 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:26.299 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:26.299 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:26.300 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:26.300 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:26.300 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:26.300 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:26.300 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:26.301 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:26.301 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:26.302 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.303 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:26.364 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:26.365 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.365 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:26.426 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:26.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.426 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:26.486 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:26.486 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.486 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:26.547 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:26.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.547 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:26.609 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:26.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.609 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:26.670 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:26.670 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.670 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:26.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:26.730 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.731 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:26.792 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:26.792 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.792 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:26.852 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:26.852 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.853 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:26.913 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:26.913 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.913 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:26.977 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:26.977 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-11-28T03:04:27.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir1 1M 2023-11-28T03:04:27.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.191+0000 7f48a939c700 1 Processor -- start 2023-11-28T03:04:27.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.191+0000 7f48a939c700 1 -- start start 2023-11-28T03:04:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a4154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48a40a7ff0 0x7f48a40c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f48a40aa220 0x7f48a40aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40c6390 con 0x7f48a40a7ff0 2023-11-28T03:04:27.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48a40aabc0 con 0x7f48a40aa220 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a939c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40c6630 con 0x7f48a4154530 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48a40a7ff0 0x7f48a40c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f48a40a7ff0 0x7f48a40c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49858/0 (socket says 172.21.15.73:49858) 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 -- 172.21.15.73:0/1138156597 learned_addr learned my addr 172.21.15.73:0/1138156597 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a37fe700 1 --2- 172.21.15.73:0/1138156597 >> v2:172.21.15.150:3300/0 conn(0x7f48a40aa220 0x7f48a40aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a2ffd700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 -- 172.21.15.73:0/1138156597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 msgr2=0x7f48a4154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a4154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 -- 172.21.15.73:0/1138156597 >> v2:172.21.15.150:3300/0 conn(0x7f48a40aa220 msgr2=0x7f48a40aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1138156597 >> v2:172.21.15.150:3300/0 conn(0x7f48a40aa220 0x7f48a40aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.192+0000 7f48a27fc700 1 -- 172.21.15.73:0/1138156597 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a40a8470 con 0x7f48a40a7ff0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40a7ff0 0x7f48a40c8580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f48a40a9fc0 tx=0x7f48980090f0).ready entity=mon.0 client_cookie=e7325783975955cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a1ffb700 1 -- 172.21.15.73:0/1138156597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800b600 con 0x7f48a40a7ff0 2023-11-28T03:04:27.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a1ffb700 1 -- 172.21.15.73:0/1138156597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800b760 con 0x7f48a40a7ff0 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a1ffb700 1 -- 172.21.15.73:0/1138156597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800bae0 con 0x7f48a40a7ff0 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 -- 172.21.15.73:0/1138156597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40a7ff0 msgr2=0x7f48a40c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40a7ff0 0x7f48a40c8580 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 rx=0x7f48a40a9fc0 tx=0x7f48980090f0).stop 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 -- 172.21.15.73:0/1138156597 shutdown_connections 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 --2- 172.21.15.73:0/1138156597 >> v2:172.21.15.150:3300/0 conn(0x7f48a40aa220 0x7f48a40aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a4154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 --2- 172.21.15.73:0/1138156597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40a7ff0 0x7f48a40c8580 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 -- 172.21.15.73:0/1138156597 >> 172.21.15.73:0/1138156597 conn(0x7f48a4094930 msgr2=0x7f48a406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 -- 172.21.15.73:0/1138156597 shutdown_connections 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.193+0000 7f48a939c700 1 -- 172.21.15.73:0/1138156597 wait complete. 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.194+0000 7f48a939c700 1 Processor -- start 2023-11-28T03:04:27.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.194+0000 7f48a939c700 1 -- start start 2023-11-28T03:04:27.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48a40a7ff0 0x7f48a40ab770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a40c4470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40aece0 con 0x7f48a40aa220 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40ae3b0 con 0x7f48a40a7ff0 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a939c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48a40ae690 con 0x7f48a4154530 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49860/0 (socket says 172.21.15.73:49860) 2023-11-28T03:04:27.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a27fc700 1 -- 172.21.15.73:0/1922750213 learned_addr learned my addr 172.21.15.73:0/1922750213 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a2ffd700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48a40a7ff0 0x7f48a40ab770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.195+0000 7f48a37fe700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a40c4470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 -- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 msgr2=0x7f48a40c4470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a40c4470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 -- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48a40a7ff0 msgr2=0x7f48a40ab770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48a40a7ff0 0x7f48a40ab770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 -- 172.21.15.73:0/1922750213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4898009370 con 0x7f48a40aa220 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a37fe700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a40c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a27fc700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f4898009130 tx=0x7f4898017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489800b8d0 con 0x7f48a40aa220 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f489800ba30 con 0x7f48a40aa220 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f489801c670 con 0x7f48a40aa220 2023-11-28T03:04:27.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48a40ae970 con 0x7f48a40aa220 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.196+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48a4153600 con 0x7f48a40aa220 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.197+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48a41537f0 con 0x7f48a40aa220 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.197+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f489801ca20 con 0x7f48a40aa220 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.197+0000 7f488affd700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 0x7f4890061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.198+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 6322+0+0 (secure 0 0 0) 0x7f489807c270 con 0x7f48a40aa220 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.198+0000 7f48a2ffd700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 0x7f4890061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.198+0000 7f48a2ffd700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 0x7f4890061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4894007560 tx=0x7f4894009800).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:27.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.202+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f489804d2f0 con 0x7f48a40aa220 2023-11-28T03:04:27.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.402+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f48a406c860 con 0x7f489005f2c0 2023-11-28T03:04:27.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.405+0000 7f488affd700 1 -- 172.21.15.73:0/1922750213 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f48a406c860 con 0x7f489005f2c0 2023-11-28T03:04:27.407 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 msgr2=0x7f4890061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 0x7f4890061780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f4894007560 tx=0x7f4894009800).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 msgr2=0x7f48a4153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 rx=0x7f4898009130 tx=0x7f4898017000).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 shutdown_connections 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f489005f2c0 0x7f4890061780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f48a4154530 0x7f48a40c4470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48a40a7ff0 0x7f48a40ab770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 --2- 172.21.15.73:0/1922750213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48a40aa220 0x7f48a4153110 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 >> 172.21.15.73:0/1922750213 conn(0x7f48a4094930 msgr2=0x7f48a40c9490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:27.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 shutdown_connections 2023-11-28T03:04:27.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:27.407+0000 7f48a939c700 1 -- 172.21.15.73:0/1922750213 wait complete. 2023-11-28T03:04:27.420 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-11-28T03:04:27.420 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.420 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:27.436 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:27.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:27.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.437 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:27.504 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:27.504 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.505 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:27.565 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:27.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.566 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:27.628 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:27.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.628 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:27.690 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:27.690 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.691 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:27.751 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:27.751 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.751 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:27.811 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:27.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.811 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:27.871 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:27.871 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.871 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:27.931 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:27.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.932 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:27.992 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:27.992 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:27.992 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:28.052 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:28.052 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.052 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:28.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.112 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-11-28T03:04:28.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir2 1M 2023-11-28T03:04:28.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.333+0000 7f8cd3fff700 1 Processor -- start 2023-11-28T03:04:28.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.333+0000 7f8cd3fff700 1 -- start start 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd3fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8cd41542d0 0x7f8cd40a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cd40aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd3fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd4158d50 con 0x7f8cd41546b0 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd3fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd40c6b60 con 0x7f8cd41542d0 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd3fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40aa950 con 0x7f8cd40c66d0 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd27fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8cd41546b0 0x7f8cd40c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48192/0 (socket says 172.21.15.73:48192) 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd27fc700 1 -- 172.21.15.73:0/1456342642 learned_addr learned my addr 172.21.15.73:0/1456342642 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:28.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.334+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194664169 0 0) 0x7f8cd4158d50 con 0x7f8cd41546b0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd37fe700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cd40aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> v2:172.21.15.150:3300/0 conn(0x7f8cd41542d0 0x7f8cd40a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8cc00033f0 con 0x7f8cd41546b0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2555545767 0 0) 0x7f8cc00033f0 con 0x7f8cd41546b0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 msgr2=0x7f8cd40aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cd40aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 >> v2:172.21.15.150:3300/0 conn(0x7f8cd41542d0 msgr2=0x7f8cd40a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 --2- 172.21.15.73:0/1456342642 >> v2:172.21.15.150:3300/0 conn(0x7f8cd41542d0 0x7f8cd40a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4158140 con 0x7f8cd41546b0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8cbc001470 con 0x7f8cd41546b0 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> v2:172.21.15.150:3300/0 conn(0x7f8cd41542d0 0x7f8cd40a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 >> v1:172.21.15.73:6789/0 conn(0x7f8cd41546b0 legacy=0x7f8cd40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cc00038e0 0x7f8cc0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cc0004690 0x7f8cc0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cc0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40aa950 con 0x7f8cc00038e0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc000b8e0 con 0x7f8cc0004690 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cc000baa0 con 0x7f8cd40c66d0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cc00038e0 0x7f8cc0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd27fc700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cc0004690 0x7f8cc0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd37fe700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cc0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 -- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 msgr2=0x7f8cc0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cd40c66d0 0x7f8cc0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 -- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cc0004690 msgr2=0x7f8cc0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cc0004690 0x7f8cc0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 -- 172.21.15.73:0/1456342642 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cc00033f0 con 0x7f8cc00038e0 2023-11-28T03:04:28.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.335+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cc00038e0 0x7f8cc0003d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f8ccc00bad0 tx=0x7f8ccc008040).ready entity=mon.0 client_cookie=e2ad4c7e34405112 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00b410 con 0x7f8cc00038e0 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00d5d0 con 0x7f8cc00038e0 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd1ffb700 1 -- 172.21.15.73:0/1456342642 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00d950 con 0x7f8cc00038e0 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 -- 172.21.15.73:0/1456342642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cc00038e0 msgr2=0x7f8cc0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/1456342642 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cc00038e0 0x7f8cc0003d00 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 rx=0x7f8ccc00bad0 tx=0x7f8ccc008040).stop 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd37fe700 1 -- 172.21.15.73:0/1456342642 reap_dead start 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 -- 172.21.15.73:0/1456342642 shutdown_connections 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 -- 172.21.15.73:0/1456342642 >> 172.21.15.73:0/1456342642 conn(0x7f8cd40cb010 msgr2=0x7f8cd40cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 -- 172.21.15.73:0/1456342642 shutdown_connections 2023-11-28T03:04:28.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.336+0000 7f8cd3fff700 1 -- 172.21.15.73:0/1456342642 wait complete. 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.337+0000 7f8cd3fff700 1 Processor -- start 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.337+0000 7f8cd3fff700 1 -- start start 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cc00038e0 0x7f8cd40adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cd40af950 0x7f8cd4147bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40af400 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40cab20 con 0x7f8cd40af950 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd3fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40ae7f0 con 0x7f8cc00038e0 2023-11-28T03:04:28.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49874/0 (socket says 172.21.15.73:49874) 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd37fe700 1 -- 172.21.15.73:0/2670267409 learned_addr learned my addr 172.21.15.73:0/2670267409 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd27fc700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cd40af950 0x7f8cd4147bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.338+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cc00038e0 0x7f8cd40adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 -- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cc00038e0 msgr2=0x7f8cd40adfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cc00038e0 0x7f8cd40adfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 -- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cd40af950 msgr2=0x7f8cd4147bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cd40af950 0x7f8cd4147bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 -- 172.21.15.73:0/2670267409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd40aead0 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd37fe700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8cd4158000 tx=0x7f8cc4009710).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4009b10 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cc4002bd0 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd40aedb0 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cc4016430 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.339+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40ae490 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.340+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40af790 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.340+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f8cc4003040 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.340+0000 7f8ccaffd700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 0x7f8cb4061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.341+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6602+0+0 (secure 0 0 0) 0x7f8cc407add0 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.341+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 0x7f8cb4061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.341+0000 7f8cd2ffd700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 0x7f8cb4061750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8ccc008010 tx=0x7f8ccc008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:28.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.345+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8cd40af790 con 0x7f8cd40ca1d0 2023-11-28T03:04:28.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.546+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cd406c7c0 con 0x7f8cb405f290 2023-11-28T03:04:28.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.550+0000 7f8ccaffd700 1 -- 172.21.15.73:0/2670267409 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8cd406c7c0 con 0x7f8cb405f290 2023-11-28T03:04:28.551 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 msgr2=0x7f8cb4061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 0x7f8cb4061750 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f8ccc008010 tx=0x7f8ccc008040).stop 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 msgr2=0x7f8cd40ca630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 rx=0x7f8cd4158000 tx=0x7f8cc4009710).stop 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 shutdown_connections 2023-11-28T03:04:28.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8cb405f290 0x7f8cb4061750 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8cd40ca1d0 0x7f8cd40ca630 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8cc00038e0 0x7f8cd40adfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 --2- 172.21.15.73:0/2670267409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8cd40af950 0x7f8cd4147bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 >> 172.21.15.73:0/2670267409 conn(0x7f8cd40cb010 msgr2=0x7f8cd4063250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 shutdown_connections 2023-11-28T03:04:28.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:28.552+0000 7f8cd3fff700 1 -- 172.21.15.73:0/2670267409 wait complete. 2023-11-28T03:04:28.564 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-11-28T03:04:28.564 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.565 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-11-28T03:04:28.580 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:28.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:28.582 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.582 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:28.645 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:28.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.645 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:28.706 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:28.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.706 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:28.766 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:28.766 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.767 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:28.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:28.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.827 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:28.887 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:28.887 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.887 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:28.950 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:28.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:28.951 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:29.012 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:29.012 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.012 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:29.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:29.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.073 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:29.133 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:29.133 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.134 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:29.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:29.194 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.195 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:29.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.255 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-11-28T03:04:29.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir3 1M 2023-11-28T03:04:29.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.477+0000 7f5d7182e700 1 Processor -- start 2023-11-28T03:04:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.477+0000 7f5d7182e700 1 -- start start 2023-11-28T03:04:29.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.478+0000 7f5d7182e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5d6c154610 0x7f5d6c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.478+0000 7f5d7182e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.478+0000 7f5d7182e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d6c159210 con 0x7f5d6c0ab3b0 2023-11-28T03:04:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.478+0000 7f5d7182e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d6c0c0c70 con 0x7f5d6c154610 2023-11-28T03:04:29.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.478+0000 7f5d7182e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0c2110 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:57492/0 (socket says 172.21.15.73:57492) 2023-11-28T03:04:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/961243134 learned_addr learned my addr 172.21.15.73:0/961243134 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:29.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/961243134 >> v2:172.21.15.150:3300/0 conn(0x7f5d6c154610 msgr2=0x7f5d6c1582e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 --2- 172.21.15.73:0/961243134 >> v2:172.21.15.150:3300/0 conn(0x7f5d6c154610 0x7f5d6c1582e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/961243134 >> v1:172.21.15.150:6789/0 conn(0x7f5d6c0ab3b0 legacy=0x7f5d6c0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.479+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/961243134 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c0aaaf0 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.480+0000 7f5d6b7fe700 1 --2- 172.21.15.73:0/961243134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5d64009f20 tx=0x7f5d64009840).ready entity=mon.2 client_cookie=94041bcde36cd6eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.480+0000 7f5d69ffb700 1 -- 172.21.15.73:0/961243134 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400e070 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.480+0000 7f5d69ffb700 1 -- 172.21.15.73:0/961243134 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6400ba40 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.480+0000 7f5d69ffb700 1 -- 172.21.15.73:0/961243134 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6400bdc0 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 -- 172.21.15.73:0/961243134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 msgr2=0x7f5d6c0aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:29.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 --2- 172.21.15.73:0/961243134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f5d64009f20 tx=0x7f5d64009840).stop 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 -- 172.21.15.73:0/961243134 shutdown_connections 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 --2- 172.21.15.73:0/961243134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c0aa1c0 0x7f5d6c0aa600 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 --2- 172.21.15.73:0/961243134 >> v2:172.21.15.150:3300/0 conn(0x7f5d6c154610 0x7f5d6c1582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 -- 172.21.15.73:0/961243134 >> 172.21.15.73:0/961243134 conn(0x7f5d6c0655a0 msgr2=0x7f5d6c069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.481+0000 7f5d7182e700 1 -- 172.21.15.73:0/961243134 shutdown_connections 2023-11-28T03:04:29.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.482+0000 7f5d7182e700 1 -- 172.21.15.73:0/961243134 wait complete. 2023-11-28T03:04:29.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.483+0000 7f5d7182e700 1 Processor -- start 2023-11-28T03:04:29.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.484+0000 7f5d7182e700 1 -- start start 2023-11-28T03:04:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d6c0aa1c0 0x7f5d6c157210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c154610 0x7f5d6c1524a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0ac720 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0aca00 con 0x7f5d6c0aa1c0 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d7182e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d6c0acce0 con 0x7f5d6c154610 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49888/0 (socket says 172.21.15.73:49888) 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/849026974 learned_addr learned my addr 172.21.15.73:0/849026974 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6affd700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d6c0aa1c0 0x7f5d6c157210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6a7fc700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c154610 0x7f5d6c1524a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c154610 msgr2=0x7f5d6c1524a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c154610 0x7f5d6c1524a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d6c0aa1c0 msgr2=0x7f5d6c157210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:29.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d6c0aa1c0 0x7f5d6c157210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.485+0000 7f5d6b7fe700 1 -- 172.21.15.73:0/849026974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d64009ae0 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d6b7fe700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f5d6c05e620 tx=0x7f5d64009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:29.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d64007170 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d640092e0 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d6c0acfc0 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6401d660 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.486+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d6c1569f0 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.487+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d4c005120 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.488+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f5d6401da50 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.488+0000 7f5d52ffd700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 0x7f5d58061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.488+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6858+0+0 (secure 0 0 0) 0x7f5d6407c290 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.488+0000 7f5d6affd700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 0x7f5d58061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:29.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.489+0000 7f5d6affd700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 0x7f5d58061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5d5c00b9b0 tx=0x7f5d5c008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:29.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.494+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5d6404d100 con 0x7f5d6c0ac1b0 2023-11-28T03:04:29.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.691+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5d4c005120 con 0x7f5d5805f2c0 2023-11-28T03:04:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.699+0000 7f5d52ffd700 1 -- 172.21.15.73:0/849026974 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f5d4c005120 con 0x7f5d5805f2c0 2023-11-28T03:04:29.703 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-11-28T03:04:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.703+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 msgr2=0x7f5d58061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.703+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 0x7f5d58061780 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f5d5c00b9b0 tx=0x7f5d5c008040).stop 2023-11-28T03:04:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.703+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 msgr2=0x7f5d6c152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:29.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.703+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 rx=0x7f5d6c05e620 tx=0x7f5d64009840).stop 2023-11-28T03:04:29.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 shutdown_connections 2023-11-28T03:04:29.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d5805f2c0 0x7f5d58061780 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d6c0ac1b0 0x7f5d6c152990 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d6c0aa1c0 0x7f5d6c157210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 --2- 172.21.15.73:0/849026974 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d6c154610 0x7f5d6c1524a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:29.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.704+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 >> 172.21.15.73:0/849026974 conn(0x7f5d6c0655a0 msgr2=0x7f5d6c14dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:29.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.705+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 shutdown_connections 2023-11-28T03:04:29.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:29.706+0000 7f5d7182e700 1 -- 172.21.15.73:0/849026974 wait complete. 2023-11-28T03:04:29.725 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-11-28T03:04:29.725 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.725 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-11-28T03:04:29.741 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:29.741 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:29.741 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:29.741 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:29.741 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:29.742 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:29.743 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:29.743 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.743 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:29.803 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:29.803 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.803 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:29.863 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:29.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.864 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:29.924 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:29.924 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.924 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:29.983 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:29.984 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:29.984 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:30.046 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:30.047 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.047 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:30.107 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:30.107 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.107 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:30.166 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:30.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.167 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:30.227 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:30.227 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.227 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:30.287 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:30.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.287 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:30.362 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:30.362 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.362 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:30.426 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.426 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-11-28T03:04:30.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir4 1M 2023-11-28T03:04:30.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.639+0000 7f193e486700 1 Processor -- start 2023-11-28T03:04:30.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.639+0000 7f193e486700 1 -- start start 2023-11-28T03:04:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193e486700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193e486700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193e486700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19380c6630 con 0x7f1938154530 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193e486700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19380c6390 con 0x7f19380aa220 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193e486700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380a84e0 con 0x7f1938154910 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193ca23700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193ca23700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:47778/0 (socket says 172.21.15.73:47778) 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:57514/0 (socket says 172.21.15.73:57514) 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193ca23700 1 -- 172.21.15.73:0/772321976 learned_addr learned my addr 172.21.15.73:0/772321976 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 -- 172.21.15.73:0/772321976 >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 msgr2=0x7f19380aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 --2- 172.21.15.73:0/772321976 >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 -- 172.21.15.73:0/772321976 >> v1:172.21.15.150:6789/0 conn(0x7f1938154530 legacy=0x7f19380c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f1936ffd700 1 -- 172.21.15.73:0/772321976 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3138252923 0 0) 0x7f19380c6630 con 0x7f1938154530 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f19377fe700 1 -- 172.21.15.73:0/772321976 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380aaba0 con 0x7f1938154910 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.640+0000 7f193ca23700 1 --2- 172.21.15.73:0/772321976 >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f19377fe700 1 --2- 172.21.15.73:0/772321976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f192c009760 tx=0x7f192c009060).ready entity=mon.2 client_cookie=fdc58ce9723a0e70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f1936ffd700 1 -- 172.21.15.73:0/772321976 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f510 con 0x7f1938154910 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f1936ffd700 1 -- 172.21.15.73:0/772321976 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00f670 con 0x7f1938154910 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f1936ffd700 1 -- 172.21.15.73:0/772321976 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f9f0 con 0x7f1938154910 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 -- 172.21.15.73:0/772321976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 msgr2=0x7f19380a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 --2- 172.21.15.73:0/772321976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f192c009760 tx=0x7f192c009060).stop 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 -- 172.21.15.73:0/772321976 shutdown_connections 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 --2- 172.21.15.73:0/772321976 >> v2:172.21.15.150:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 --2- 172.21.15.73:0/772321976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 -- 172.21.15.73:0/772321976 >> 172.21.15.73:0/772321976 conn(0x7f1938094930 msgr2=0x7f193806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:30.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 -- 172.21.15.73:0/772321976 shutdown_connections 2023-11-28T03:04:30.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.641+0000 7f193e486700 1 -- 172.21.15.73:0/772321976 wait complete. 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.642+0000 7f193e486700 1 Processor -- start 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 -- start start 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19380aa220 0x7f19381517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1938154910 0x7f19380c2cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1938151cc0 con 0x7f1938154530 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19381507c0 con 0x7f1938154910 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f193e486700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1938151340 con 0x7f19380aa220 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f19377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f19377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49896/0 (socket says 172.21.15.73:49896) 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.643+0000 7f19377fe700 1 -- 172.21.15.73:0/3727512157 learned_addr learned my addr 172.21.15.73:0/3727512157 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f1937fff700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19380aa220 0x7f19381517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f193ca23700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1938154910 0x7f19380c2cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 -- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19380aa220 msgr2=0x7f19381517d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19380aa220 0x7f19381517d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 -- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1938154910 msgr2=0x7f19380c2cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1938154910 0x7f19380c2cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 -- 172.21.15.73:0/3727512157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930009ae0 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f19377fe700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f192c009e00 tx=0x7f192c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c00f7e0 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f192c00f940 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f192c0092a0 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f192c01b660 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.644+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1938150cf0 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.645+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19381514f0 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.645+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f192c01ba10 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.646+0000 7f191ffff700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 0x7f1924061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.646+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 7050+0+0 (secure 0 0 0) 0x7f192c07c010 con 0x7f1938154530 2023-11-28T03:04:30.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.646+0000 7f1937fff700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 0x7f1924061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:30.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.646+0000 7f1937fff700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 0x7f1924061750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1928003780 tx=0x7f1928009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:30.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.650+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19381514f0 con 0x7f1938154530 2023-11-28T03:04:30.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.847+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f19381514b0 con 0x7f192405f290 2023-11-28T03:04:30.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.850+0000 7f191ffff700 1 -- 172.21.15.73:0/3727512157 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f19381514b0 con 0x7f192405f290 2023-11-28T03:04:30.852 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 msgr2=0x7f1924061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 0x7f1924061750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7f1928003780 tx=0x7f1928009040).stop 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 msgr2=0x7f1938145020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 rx=0x7f192c009e00 tx=0x7f192c017000).stop 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 shutdown_connections 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f192405f290 0x7f1924061750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1938154910 0x7f19380c2cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19380aa220 0x7f19381517d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 --2- 172.21.15.73:0/3727512157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f1938145020 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 >> 172.21.15.73:0/3727512157 conn(0x7f1938094930 msgr2=0x7f193814fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 shutdown_connections 2023-11-28T03:04:30.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:30.852+0000 7f193e486700 1 -- 172.21.15.73:0/3727512157 wait complete. 2023-11-28T03:04:30.865 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-11-28T03:04:30.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.865 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:30.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:30.882 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:30.882 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:30.882 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:30.882 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:30.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:30.882 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.882 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:30.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:30.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:30.943 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:31.003 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:31.003 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.003 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:31.064 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:31.064 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.064 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:31.124 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:31.124 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.124 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:31.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:31.184 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.184 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:31.250 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:31.250 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.250 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:31.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:31.310 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.310 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:31.371 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:31.371 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.371 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:31.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:31.443 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.443 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:31.508 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:31.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.508 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:31.569 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:31.569 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-11-28T03:04:31.627 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir5 1M 2023-11-28T03:04:31.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.784+0000 7f7a8759e700 1 Processor -- start 2023-11-28T03:04:31.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.784+0000 7f7a8759e700 1 -- start start 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a8759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 0x7f7a88065fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a8759e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a88062e10 0x7f7a880bcec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a8759e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a88069e40 con 0x7f7a88153100 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a8759e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a88069550 con 0x7f7a88062e10 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a8759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a88153580 con 0x7f7a881386f0 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.785+0000 7f7a86d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a88062e10 0x7f7a880bcec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a88062e10 0x7f7a880bcec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49904/0 (socket says 172.21.15.73:49904) 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 -- 172.21.15.73:0/3302245904 learned_addr learned my addr 172.21.15.73:0/3302245904 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a8659c700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 0x7f7a88065fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 -- 172.21.15.73:0/3302245904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 msgr2=0x7f7a88065fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 0x7f7a88065fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 -- 172.21.15.73:0/3302245904 >> v1:172.21.15.150:6789/0 conn(0x7f7a88153100 legacy=0x7f7a88062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 -- 172.21.15.73:0/3302245904 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a880697f0 con 0x7f7a88062e10 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a8659c700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 0x7f7a88065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a86d9d700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88062e10 0x7f7a880bcec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f7a80009f20 tx=0x7f7a80009840).ready entity=mon.0 client_cookie=b678a0817903724b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a85d9b700 1 -- 172.21.15.73:0/3302245904 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000e070 con 0x7f7a88062e10 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a85d9b700 1 -- 172.21.15.73:0/3302245904 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8000ba80 con 0x7f7a88062e10 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.786+0000 7f7a85d9b700 1 -- 172.21.15.73:0/3302245904 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8000be00 con 0x7f7a88062e10 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 -- 172.21.15.73:0/3302245904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88062e10 msgr2=0x7f7a880bcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88062e10 0x7f7a880bcec0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 rx=0x7f7a80009f20 tx=0x7f7a80009840).stop 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 -- 172.21.15.73:0/3302245904 shutdown_connections 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88062e10 0x7f7a880bcec0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 --2- 172.21.15.73:0/3302245904 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a881386f0 0x7f7a88065fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 -- 172.21.15.73:0/3302245904 >> 172.21.15.73:0/3302245904 conn(0x7f7a880642e0 msgr2=0x7f7a880c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 -- 172.21.15.73:0/3302245904 shutdown_connections 2023-11-28T03:04:31.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.787+0000 7f7a8759e700 1 -- 172.21.15.73:0/3302245904 wait complete. 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.788+0000 7f7a8759e700 1 Processor -- start 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.788+0000 7f7a8759e700 1 -- start start 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a88062e10 0x7f7a880c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a881386f0 0x7f7a880ac280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880acad0 con 0x7f7a88153100 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880acdb0 con 0x7f7a881386f0 2023-11-28T03:04:31.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a880ad090 con 0x7f7a88062e10 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49918/0 (socket says 172.21.15.73:49918) 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8659c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a88062e10 0x7f7a880c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 -- 172.21.15.73:0/2607702680 learned_addr learned my addr 172.21.15.73:0/2607702680 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a7dd9b700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a881386f0 0x7f7a880ac280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 -- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a88062e10 msgr2=0x7f7a880c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a88062e10 0x7f7a880c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 -- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a881386f0 msgr2=0x7f7a880ac280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a881386f0 0x7f7a880ac280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 -- 172.21.15.73:0/2607702680 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a80009ae0 con 0x7f7a88153100 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a86d9d700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f7a880646f0 tx=0x7f7a80009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a800032f0 con 0x7f7a88153100 2023-11-28T03:04:31.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a80013d80 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a880ad370 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8001d670 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.789+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a880ac770 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.790+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a8806c7c0 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.791+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f7a80003040 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.791+0000 7f7a7effd700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 0x7f7a68061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.791+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 7190+0+0 (secure 0 0 0) 0x7f7a8007bac0 con 0x7f7a88153100 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.791+0000 7f7a8659c700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 0x7f7a68061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:31.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.791+0000 7f7a8659c700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 0x7f7a68061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7a78006d80 tx=0x7f7a78008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:31.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.796+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a8004c830 con 0x7f7a88153100 2023-11-28T03:04:31.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.982+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7a8805e8e0 con 0x7f7a6805f2c0 2023-11-28T03:04:31.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.985+0000 7f7a7effd700 1 -- 172.21.15.73:0/2607702680 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f7a8805e8e0 con 0x7f7a6805f2c0 2023-11-28T03:04:31.987 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-11-28T03:04:31.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 msgr2=0x7f7a68061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 0x7f7a68061780 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 rx=0x7f7a78006d80 tx=0x7f7a78008040).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 msgr2=0x7f7a880bfc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 rx=0x7f7a880646f0 tx=0x7f7a80009840).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 shutdown_connections 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a6805f2c0 0x7f7a68061780 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a88153100 0x7f7a880bfc70 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a88062e10 0x7f7a880c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 --2- 172.21.15.73:0/2607702680 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a881386f0 0x7f7a880ac280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 >> 172.21.15.73:0/2607702680 conn(0x7f7a880642e0 msgr2=0x7f7a88067a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:31.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 shutdown_connections 2023-11-28T03:04:31.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:31.987+0000 7f7a8759e700 1 -- 172.21.15.73:0/2607702680 wait complete. 2023-11-28T03:04:32.000 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-11-28T03:04:32.000 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.000 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-11-28T03:04:32.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:32.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:32.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:32.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:32.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:32.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:32.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.017 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:32.079 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:32.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.079 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:32.140 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:32.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.141 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:32.203 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:32.204 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.204 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:32.266 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:32.266 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.267 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:32.329 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:32.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.329 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:32.393 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:32.393 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.393 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:32.470 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:32.470 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.470 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:32.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:32.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.531 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:32.591 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:32.591 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.591 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:32.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:32.653 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.653 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:32.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:32.714 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-11-28T03:04:32.771 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir6 1M 2023-11-28T03:04:32.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.929+0000 7f46c9af7700 1 Processor -- start 2023-11-28T03:04:32.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.929+0000 7f46c9af7700 1 -- start start 2023-11-28T03:04:32.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c9af7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46c4066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c9af7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f46c414cda0 0x7f46c40c4ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c9af7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46c4152e10 con 0x7f46c40c4fb0 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c9af7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46c40c5430 con 0x7f46c414cda0 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c9af7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41530f0 con 0x7f46c414c9c0 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46c40c4fb0 0x7f46c40bcef0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48208/0 (socket says 172.21.15.73:48208) 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c3fff700 1 -- 172.21.15.73:0/997049659 learned_addr learned my addr 172.21.15.73:0/997049659 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:32.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 926593910 0 0) 0x7f46c4152e10 con 0x7f46c40c4fb0 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f46ac0033f0 con 0x7f46c40c4fb0 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c37fe700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46c4066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.930+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/997049659 >> v2:172.21.15.150:3300/0 conn(0x7f46c414cda0 0x7f46c40c4ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118510252 0 0) 0x7f46ac0033f0 con 0x7f46c40c4fb0 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 msgr2=0x7f46c4066b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46c4066b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 >> v2:172.21.15.150:3300/0 conn(0x7f46c414cda0 msgr2=0x7f46c40c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:32.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 --2- 172.21.15.73:0/997049659 >> v2:172.21.15.150:3300/0 conn(0x7f46c414cda0 0x7f46c40c4ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c41533d0 con 0x7f46c40c4fb0 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f46bc0017f0 con 0x7f46c40c4fb0 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 >> v1:172.21.15.73:6789/0 conn(0x7f46c40c4fb0 legacy=0x7f46c40bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46ac0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46ac003e10 0x7f46ac004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac004b80 0x7f46ac005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40c5430 con 0x7f46ac003e10 2023-11-28T03:04:32.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c41530f0 con 0x7f46ac004b80 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46ac00b8e0 con 0x7f46c414c9c0 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46ac003e10 0x7f46ac004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c37fe700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46ac0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c3fff700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac004b80 0x7f46ac005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 -- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 msgr2=0x7f46ac0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46c414c9c0 0x7f46ac0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 -- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac004b80 msgr2=0x7f46ac005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:32.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46ac004b80 0x7f46ac005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 -- 172.21.15.73:0/997049659 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46ac0033f0 con 0x7f46ac003e10 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46ac003e10 0x7f46ac004690 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f46c40befe0 tx=0x7f46b8009060).ready entity=mon.0 client_cookie=70ac4e943e06703f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b8009e90 con 0x7f46ac003e10 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46b8009430 con 0x7f46ac003e10 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.931+0000 7f46c27fc700 1 -- 172.21.15.73:0/997049659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46b800e820 con 0x7f46ac003e10 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 -- 172.21.15.73:0/997049659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46ac003e10 msgr2=0x7f46ac004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 --2- 172.21.15.73:0/997049659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46ac003e10 0x7f46ac004690 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7f46c40befe0 tx=0x7f46b8009060).stop 2023-11-28T03:04:32.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c3fff700 1 -- 172.21.15.73:0/997049659 reap_dead start 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 -- 172.21.15.73:0/997049659 shutdown_connections 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 -- 172.21.15.73:0/997049659 >> 172.21.15.73:0/997049659 conn(0x7f46c4063810 msgr2=0x7f46c4063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 -- 172.21.15.73:0/997049659 shutdown_connections 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.932+0000 7f46c9af7700 1 -- 172.21.15.73:0/997049659 wait complete. 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.933+0000 7f46c9af7700 1 Processor -- start 2023-11-28T03:04:32.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.933+0000 7f46c9af7700 1 -- start start 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46ac003e10 0x7f46c4161ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46c40acec0 0x7f46c40ad340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c4144550 con 0x7f46c4067110 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40c6290 con 0x7f46c40acec0 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c9af7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46c40ad830 con 0x7f46ac003e10 2023-11-28T03:04:32.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49944/0 (socket says 172.21.15.73:49944) 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 -- 172.21.15.73:0/1168028721 learned_addr learned my addr 172.21.15.73:0/1168028721 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c2ffd700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46c40acec0 0x7f46c40ad340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c37fe700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46ac003e10 0x7f46c4161ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 -- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46ac003e10 msgr2=0x7f46c4161ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46ac003e10 0x7f46c4161ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 -- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46c40acec0 msgr2=0x7f46c40ad340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:32.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46c40acec0 0x7f46c40ad340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 -- 172.21.15.73:0/1168028721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46c4067a80 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.934+0000 7f46c3fff700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f46bc001350 tx=0x7f46bc001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.935+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00f970 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.935+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46bc00fad0 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.935+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46bc00fe70 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.935+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46c40c5f40 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.935+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46c40add60 con 0x7f46c4067110 2023-11-28T03:04:32.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.936+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46c40c6420 con 0x7f46c4067110 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.936+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f46bc0133a0 con 0x7f46c4067110 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.936+0000 7f46ab7fe700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 0x7f46b0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.936+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 7114+0+0 (secure 0 0 0) 0x7f46bc07a470 con 0x7f46c4067110 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.936+0000 7f46c37fe700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 0x7f46b0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.937+0000 7f46c37fe700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 0x7f46b0061780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f46b4006c40 tx=0x7f46b4008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:32.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:32.940+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f46bc04b180 con 0x7f46c4067110 2023-11-28T03:04:33.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.132+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f46c40adf50 con 0x7f46b005f2c0 2023-11-28T03:04:33.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.136+0000 7f46ab7fe700 1 -- 172.21.15.73:0/1168028721 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f46c40adf50 con 0x7f46b005f2c0 2023-11-28T03:04:33.136 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 msgr2=0x7f46b0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 0x7f46b0061780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 rx=0x7f46b4006c40 tx=0x7f46b4008040).stop 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 msgr2=0x7f46c4067590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 rx=0x7f46bc001350 tx=0x7f46bc001d90).stop 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 shutdown_connections 2023-11-28T03:04:33.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f46b005f2c0 0x7f46b0061780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46c4067110 0x7f46c4067590 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46ac003e10 0x7f46c4161ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 --2- 172.21.15.73:0/1168028721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46c40acec0 0x7f46c40ad340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 >> 172.21.15.73:0/1168028721 conn(0x7f46c4063810 msgr2=0x7f46c4155540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 shutdown_connections 2023-11-28T03:04:33.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:33.138+0000 7f46c9af7700 1 -- 172.21.15.73:0/1168028721 wait complete. 2023-11-28T03:04:33.149 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-11-28T03:04:33.150 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.150 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:33.166 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:33.167 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:33.167 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:33.167 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:33.167 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.167 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:33.233 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:33.233 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.234 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:33.294 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:33.294 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.295 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:33.356 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:33.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.356 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:33.417 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:33.418 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.418 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:33.479 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:33.479 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.479 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:33.539 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:33.539 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.539 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:33.601 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:33.601 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.601 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:33.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:33.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.673 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:33.734 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:33.734 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.734 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:33.795 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:33.795 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.795 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:33.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:33.857 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-11-28T03:04:33.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir7 1M 2023-11-28T03:04:34.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.072+0000 7f19cd2c2700 1 Processor -- start 2023-11-28T03:04:34.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.072+0000 7f19cd2c2700 1 -- start start 2023-11-28T03:04:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 0x7f19c8067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 0x7f19c80bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19c8154fe0 0x7f19c8155490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c8067c70 con 0x7f19c8154fe0 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19c80bddb0 con 0x7f19c813af40 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19cd2c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c8155980 con 0x7f19c813ab60 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19c8154fe0 0x7f19c8155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19c8154fe0 0x7f19c8155490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51638/0 (socket says 172.21.15.73:51638) 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 -- 172.21.15.73:0/2645695678 learned_addr learned my addr 172.21.15.73:0/2645695678 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c6ffd700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 0x7f19c8067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c67fc700 1 --2- 172.21.15.73:0/2645695678 >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 0x7f19c80bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 -- 172.21.15.73:0/2645695678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 msgr2=0x7f19c8067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 0x7f19c8067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 -- 172.21.15.73:0/2645695678 >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 msgr2=0x7f19c80bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 --2- 172.21.15.73:0/2645695678 >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 0x7f19c80bd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 -- 172.21.15.73:0/2645695678 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c80c3510 con 0x7f19c8154fe0 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c6ffd700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 0x7f19c8067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c67fc700 1 --2- 172.21.15.73:0/2645695678 >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 0x7f19c80bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c77fe700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c8154fe0 0x7f19c8155490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f19c8069220 tx=0x7f19b80090f0).ready entity=mon.0 client_cookie=36e5a9ad64379043 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c5ffb700 1 -- 172.21.15.73:0/2645695678 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800b600 con 0x7f19c8154fe0 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c5ffb700 1 -- 172.21.15.73:0/2645695678 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19b800b760 con 0x7f19c8154fe0 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.073+0000 7f19c5ffb700 1 -- 172.21.15.73:0/2645695678 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19b800bae0 con 0x7f19c8154fe0 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2645695678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c8154fe0 msgr2=0x7f19c8155490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c8154fe0 0x7f19c8155490 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 rx=0x7f19c8069220 tx=0x7f19b80090f0).stop 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2645695678 shutdown_connections 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c8154fe0 0x7f19c8155490 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2645695678 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c813ab60 0x7f19c8067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2645695678 >> v2:172.21.15.150:3300/0 conn(0x7f19c813af40 0x7f19c80bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2645695678 >> 172.21.15.73:0/2645695678 conn(0x7f19c8063810 msgr2=0x7f19c8063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2645695678 shutdown_connections 2023-11-28T03:04:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.074+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2645695678 wait complete. 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.075+0000 7f19cd2c2700 1 Processor -- start 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.075+0000 7f19cd2c2700 1 -- start start 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19c813af40 0x7f19c80c8140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c8154fe0 0x7f19c8165700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c42a0 con 0x7f19c813ab60 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c7cb0 con 0x7f19c813af40 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19cd2c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19c80c8630 con 0x7f19c8154fe0 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51644/0 (socket says 172.21.15.73:51644) 2023-11-28T03:04:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 -- 172.21.15.73:0/2539991252 learned_addr learned my addr 172.21.15.73:0/2539991252 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 -- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c8154fe0 msgr2=0x7f19c8165700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c8154fe0 0x7f19c8165700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 -- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19c813af40 msgr2=0x7f19c80c8140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19c813af40 0x7f19c80c8140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 -- 172.21.15.73:0/2539991252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19b8009370 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.076+0000 7f19c6ffd700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f19c000d200 tx=0x7f19c0009a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.077+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c000dea0 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.077+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19c000f040 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.077+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19c80aa7f0 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.077+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19c0013600 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.077+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19c80c7940 con 0x7f19c813ab60 2023-11-28T03:04:34.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.078+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19c80c7f90 con 0x7f19c813ab60 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.078+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f19c00139b0 con 0x7f19c813ab60 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.078+0000 7f19aeffd700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 0x7f19b4061750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.078+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6954+0+0 (secure 0 0 0) 0x7f19c007ade0 con 0x7f19c813ab60 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.078+0000 7f19c67fc700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 0x7f19b4061750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.079+0000 7f19c67fc700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 0x7f19b4061750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f19bc006e00 tx=0x7f19bc008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:34.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.082+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19c004bb90 con 0x7f19c813ab60 2023-11-28T03:04:34.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.285+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f19c806c860 con 0x7f19b405f290 2023-11-28T03:04:34.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.291+0000 7f19aeffd700 1 -- 172.21.15.73:0/2539991252 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f19c806c860 con 0x7f19b405f290 2023-11-28T03:04:34.291 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 msgr2=0x7f19b4061750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 0x7f19b4061750 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f19bc006e00 tx=0x7f19bc008040).stop 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 msgr2=0x7f19c80a7190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 rx=0x7f19c000d200 tx=0x7f19c0009a90).stop 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 shutdown_connections 2023-11-28T03:04:34.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19b405f290 0x7f19b4061750 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19c8154fe0 0x7f19c8165700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19c813ab60 0x7f19c80a7190 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 --2- 172.21.15.73:0/2539991252 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19c813af40 0x7f19c80c8140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 >> 172.21.15.73:0/2539991252 conn(0x7f19c8063810 msgr2=0x7f19c8154320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 shutdown_connections 2023-11-28T03:04:34.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:34.293+0000 7f19cd2c2700 1 -- 172.21.15.73:0/2539991252 wait complete. 2023-11-28T03:04:34.304 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-11-28T03:04:34.304 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.305 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-11-28T03:04:34.322 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_01_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:34.323 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:34.324 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:34.324 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:34.324 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:34.324 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:34.324 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_54_01_UTC 2023-11-28T03:04:34.325 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.325 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_54_01_UTC) 2023-11-28T03:04:34.389 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:34.390 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.390 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:34.454 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:34.454 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.455 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:34.516 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:34.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.516 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:34.577 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:34.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.577 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:34.638 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:34.639 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.639 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:34.700 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:34.700 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.701 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:34.772 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:34.772 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.772 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:34.833 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:34.833 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.833 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:34.894 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:34.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.894 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:34.955 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:34.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:34.955 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:35.016 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.016 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-11-28T03:04:35.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir8 1M 2023-11-28T03:04:35.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.249+0000 7fc4487b3700 1 Processor -- start 2023-11-28T03:04:35.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.249+0000 7fc4487b3700 1 -- start start 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.250+0000 7fc4487b3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc44014c9c0 0x7fc440066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.250+0000 7fc4487b3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc440152e10 con 0x7fc4400bcef0 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.250+0000 7fc4487b3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4401530f0 con 0x7fc44014cda0 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.250+0000 7fc4487b3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4401533d0 con 0x7fc44014c9c0 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc446d50700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc4400bcef0 0x7fc4400bd350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47982/0 (socket says 172.21.15.73:47982) 2023-11-28T03:04:35.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc446d50700 1 -- 172.21.15.73:0/799958893 learned_addr learned my addr 172.21.15.73:0/799958893 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44654f700 1 --2- 172.21.15.73:0/799958893 >> v2:172.21.15.150:3300/0 conn(0x7fc44014c9c0 0x7fc440066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44554d700 1 -- 172.21.15.73:0/799958893 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1849849344 0 0) 0x7fc440152e10 con 0x7fc4400bcef0 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44554d700 1 -- 172.21.15.73:0/799958893 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc42c0033f0 con 0x7fc4400bcef0 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44654f700 1 -- 172.21.15.73:0/799958893 >> v1:172.21.15.150:6789/0 conn(0x7fc44014cda0 legacy=0x7fc4400c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44654f700 1 -- 172.21.15.73:0/799958893 >> v1:172.21.15.73:6789/0 conn(0x7fc4400bcef0 legacy=0x7fc4400bd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44654f700 1 -- 172.21.15.73:0/799958893 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4401536b0 con 0x7fc44014c9c0 2023-11-28T03:04:35.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44654f700 1 --2- 172.21.15.73:0/799958893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440066b90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fc430009760 tx=0x7fc430009060).ready entity=mon.1 client_cookie=5e412896169c2649 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.251+0000 7fc44554d700 1 -- 172.21.15.73:0/799958893 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4300090d0 con 0x7fc44014c9c0 2023-11-28T03:04:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.252+0000 7fc44554d700 1 -- 172.21.15.73:0/799958893 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc43000f510 con 0x7fc44014c9c0 2023-11-28T03:04:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.252+0000 7fc44554d700 1 -- 172.21.15.73:0/799958893 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000f890 con 0x7fc44014c9c0 2023-11-28T03:04:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 -- 172.21.15.73:0/799958893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 msgr2=0x7fc440066b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 --2- 172.21.15.73:0/799958893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440066b90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7fc430009760 tx=0x7fc430009060).stop 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 -- 172.21.15.73:0/799958893 shutdown_connections 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 --2- 172.21.15.73:0/799958893 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440066b90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 -- 172.21.15.73:0/799958893 >> 172.21.15.73:0/799958893 conn(0x7fc440063810 msgr2=0x7fc440063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 -- 172.21.15.73:0/799958893 shutdown_connections 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.253+0000 7fc4487b3700 1 -- 172.21.15.73:0/799958893 wait complete. 2023-11-28T03:04:35.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.255+0000 7fc4487b3700 1 Processor -- start 2023-11-28T03:04:35.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.255+0000 7fc4487b3700 1 -- start start 2023-11-28T03:04:35.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:35.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440144420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:35.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44014cda0 0x7fc440142e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:35.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401371f0 con 0x7fc4400bcef0 2023-11-28T03:04:35.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4401408d0 con 0x7fc44014c9c0 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc4487b3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4400c6540 con 0x7fc44014cda0 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc44654f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc44654f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51652/0 (socket says 172.21.15.73:51652) 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc44654f700 1 -- 172.21.15.73:0/1763580466 learned_addr learned my addr 172.21.15.73:0/1763580466 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc445d4e700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440144420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.256+0000 7fc446d50700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44014cda0 0x7fc440142e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 -- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44014cda0 msgr2=0x7fc440142e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44014cda0 0x7fc440142e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 -- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 msgr2=0x7fc440144420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440144420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 -- 172.21.15.73:0/1763580466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4300092a0 con 0x7fc4400bcef0 2023-11-28T03:04:35.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc44654f700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc430009110 tx=0x7fc430017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:35.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000f890 con 0x7fc4400bcef0 2023-11-28T03:04:35.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc430006910 con 0x7fc4400bcef0 2023-11-28T03:04:35.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4401405f0 con 0x7fc4400bcef0 2023-11-28T03:04:35.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43000fe80 con 0x7fc4400bcef0 2023-11-28T03:04:35.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.257+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4400c61d0 con 0x7fc4400bcef0 2023-11-28T03:04:35.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.258+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc44006c860 con 0x7fc4400bcef0 2023-11-28T03:04:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.259+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc43001b620 con 0x7fc4400bcef0 2023-11-28T03:04:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.259+0000 7fc4367fc700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 0x7fc424061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.259+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7082+0+0 (secure 0 0 0) 0x7fc43007c1c0 con 0x7fc4400bcef0 2023-11-28T03:04:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.259+0000 7fc445d4e700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 0x7fc424061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.259+0000 7fc445d4e700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 0x7fc424061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc44013f210 tx=0x7fc43c009040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:35.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.263+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc43004cf50 con 0x7fc4400bcef0 2023-11-28T03:04:35.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.485+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc4400c6680 con 0x7fc42405f2c0 2023-11-28T03:04:35.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.489+0000 7fc4367fc700 1 -- 172.21.15.73:0/1763580466 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fc43c0032e0 con 0x7fc42405f2c0 2023-11-28T03:04:35.489 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 msgr2=0x7fc424061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 0x7fc424061780 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fc44013f210 tx=0x7fc43c009040).stop 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 msgr2=0x7fc44013fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 rx=0x7fc430009110 tx=0x7fc430017000).stop 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 shutdown_connections 2023-11-28T03:04:35.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc42405f2c0 0x7fc424061780 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc44014cda0 0x7fc440142e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4400bcef0 0x7fc44013fdc0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 --2- 172.21.15.73:0/1763580466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc44014c9c0 0x7fc440144420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 >> 172.21.15.73:0/1763580466 conn(0x7fc440063810 msgr2=0x7fc440152580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 shutdown_connections 2023-11-28T03:04:35.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:35.491+0000 7fc4487b3700 1 -- 172.21.15.73:0/1763580466 wait complete. 2023-11-28T03:04:35.502 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-11-28T03:04:35.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.503 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-11-28T03:04:35.518 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:35.518 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:35.519 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:35.519 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:35.520 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.520 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:35.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:35.581 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.581 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:35.641 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:35.641 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.642 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:35.702 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:35.703 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.703 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:35.764 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:35.765 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.765 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:35.829 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:35.829 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.829 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:35.890 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:35.890 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.890 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:35.952 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:35.952 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:35.952 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:36.013 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:36.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.014 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:36.076 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:36.076 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.077 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:36.141 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:36.141 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.141 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:36.202 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.202 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-11-28T03:04:36.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/dir9 1M 2023-11-28T03:04:36.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.436+0000 7faab124a700 1 Processor -- start 2023-11-28T03:04:36.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.436+0000 7faab124a700 1 -- start start 2023-11-28T03:04:36.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faaac1542d0 0x7faaac0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 0x7faaac0aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaac0aa950 con 0x7faaac1542d0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faaac158d50 con 0x7faaac1546b0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab124a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac158140 con 0x7faaac0c5fc0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faaac1542d0 0x7faaac0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faaac1542d0 0x7faaac0a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51664/0 (socket says 172.21.15.73:51664) 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 -- 172.21.15.73:0/100771693 learned_addr learned my addr 172.21.15.73:0/100771693 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaab7fe700 1 --2- 172.21.15.73:0/100771693 >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 0x7faaac0aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faab0a49700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 -- 172.21.15.73:0/100771693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 msgr2=0x7faaac0c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 -- 172.21.15.73:0/100771693 >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 msgr2=0x7faaac0aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 --2- 172.21.15.73:0/100771693 >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 0x7faaac0aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 -- 172.21.15.73:0/100771693 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaac158420 con 0x7faaac1542d0 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaab7fe700 1 --2- 172.21.15.73:0/100771693 >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 0x7faaac0aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaabfff700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1542d0 0x7faaac0a9f70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7faaac0add40 tx=0x7faa9c0090f0).ready entity=mon.0 client_cookie=b3fca6eb50758863 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaaaffd700 1 -- 172.21.15.73:0/100771693 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00b600 con 0x7faaac1542d0 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaaaffd700 1 -- 172.21.15.73:0/100771693 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa9c00b760 con 0x7faaac1542d0 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.437+0000 7faaaaffd700 1 -- 172.21.15.73:0/100771693 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa9c00bae0 con 0x7faaac1542d0 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 -- 172.21.15.73:0/100771693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1542d0 msgr2=0x7faaac0a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1542d0 0x7faaac0a9f70 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 rx=0x7faaac0add40 tx=0x7faa9c0090f0).stop 2023-11-28T03:04:36.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 -- 172.21.15.73:0/100771693 shutdown_connections 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1542d0 0x7faaac0a9f70 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 --2- 172.21.15.73:0/100771693 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 --2- 172.21.15.73:0/100771693 >> v2:172.21.15.150:3300/0 conn(0x7faaac1546b0 0x7faaac0aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 -- 172.21.15.73:0/100771693 >> 172.21.15.73:0/100771693 conn(0x7faaac0cb010 msgr2=0x7faaac0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 -- 172.21.15.73:0/100771693 shutdown_connections 2023-11-28T03:04:36.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.438+0000 7faab124a700 1 -- 172.21.15.73:0/100771693 wait complete. 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.439+0000 7faab124a700 1 Processor -- start 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.439+0000 7faab124a700 1 -- start start 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0ae760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac1542d0 0x7faaac0c3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac167b80 con 0x7faaac1546b0 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faaac167e60 con 0x7faaac1542d0 2023-11-28T03:04:36.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab124a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faaac168140 con 0x7faaac0c5fc0 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51674/0 (socket says 172.21.15.73:51674) 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 -- 172.21.15.73:0/51050083 learned_addr learned my addr 172.21.15.73:0/51050083 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faaabfff700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0ae760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faaab7fe700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac1542d0 0x7faaac0c3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 -- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 msgr2=0x7faaac0ae760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0ae760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 -- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac1542d0 msgr2=0x7faaac0c3ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac1542d0 0x7faaac0c3ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 -- 172.21.15.73:0/51050083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9c009370 con 0x7faaac1546b0 2023-11-28T03:04:36.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.440+0000 7faab0a49700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7faaa4009fa0 tx=0x7faaa4009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.441+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa400ebc0 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.441+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa400ed20 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.441+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa4014610 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.441+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faaac168420 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.441+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faaac1689e0 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.442+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faaac06c7c0 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.442+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7faaa401d3b0 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.442+0000 7faa93fff700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 0x7faa98061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.442+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6922+0+0 (secure 0 0 0) 0x7faaa407abc0 con 0x7faaac1546b0 2023-11-28T03:04:36.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.442+0000 7faaabfff700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 0x7faa98061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:36.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.443+0000 7faaabfff700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 0x7faa98061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7faa9c009130 tx=0x7faa9c017000).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:36.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.447+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faaa404b990 con 0x7faaac1546b0 2023-11-28T03:04:36.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.634+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faaac0c0120 con 0x7faa9805f2c0 2023-11-28T03:04:36.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.637+0000 7faa93fff700 1 -- 172.21.15.73:0/51050083 <== mgr.5196 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7faaac0c0120 con 0x7faa9805f2c0 2023-11-28T03:04:36.637 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-11-28T03:04:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.640+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 msgr2=0x7faa98061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.640+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 0x7faa98061780 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7faa9c009130 tx=0x7faa9c017000).stop 2023-11-28T03:04:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.640+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 msgr2=0x7faaac152f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:36.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.640+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 rx=0x7faaa4009fa0 tx=0x7faaa4009900).stop 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 shutdown_connections 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa9805f2c0 0x7faa98061780 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faaac1546b0 0x7faaac152f20 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faaac0c5fc0 0x7faaac0ae760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 --2- 172.21.15.73:0/51050083 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faaac1542d0 0x7faaac0c3ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:36.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.641+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 >> 172.21.15.73:0/51050083 conn(0x7faaac0cb010 msgr2=0x7faaac14e490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:36.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.642+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 shutdown_connections 2023-11-28T03:04:36.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:36.642+0000 7faab124a700 1 -- 172.21.15.73:0/51050083 wait complete. 2023-11-28T03:04:36.658 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-11-28T03:04:36.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.658 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-11-28T03:04:36.677 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:36.677 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:36.677 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:36.678 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:36.679 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:36.679 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:36.680 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_54_00_UTC 2023-11-28T03:04:36.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.680 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_54_00_UTC) 2023-11-28T03:04:36.744 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_55_00_UTC 2023-11-28T03:04:36.744 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.744 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_55_00_UTC) 2023-11-28T03:04:36.805 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_56_00_UTC 2023-11-28T03:04:36.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.806 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_56_00_UTC) 2023-11-28T03:04:36.865 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_57_00_UTC 2023-11-28T03:04:36.865 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.866 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_57_00_UTC) 2023-11-28T03:04:36.928 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_58_00_UTC 2023-11-28T03:04:36.928 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.928 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_58_00_UTC) 2023-11-28T03:04:36.989 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_59_00_UTC 2023-11-28T03:04:36.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:36.989 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-02_59_00_UTC) 2023-11-28T03:04:37.049 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_00_00_UTC 2023-11-28T03:04:37.049 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.049 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_00_00_UTC) 2023-11-28T03:04:37.110 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_01_00_UTC 2023-11-28T03:04:37.110 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.110 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_01_00_UTC) 2023-11-28T03:04:37.172 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_02_00_UTC 2023-11-28T03:04:37.172 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.172 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_02_00_UTC) 2023-11-28T03:04:37.254 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_03_00_UTC 2023-11-28T03:04:37.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.254 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_03_00_UTC) 2023-11-28T03:04:37.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_04_00_UTC 2023-11-28T03:04:37.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.316 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-11-28-03_04_00_UTC) 2023-11-28T03:04:37.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:37.376 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-11-28T03:04:37.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:04:37.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.600+0000 7ff22134b700 1 Processor -- start 2023-11-28T03:04:37.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.600+0000 7ff22134b700 1 -- start start 2023-11-28T03:04:37.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff21c153390 0x7ff21c0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff21c156690 0x7ff21c156b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c153990 con 0x7ff21c153390 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff21c155f20 con 0x7ff21c156690 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff22134b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c156200 con 0x7ff21c0c5ac0 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff2127fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff21c153390 0x7ff21c0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff2127fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff21c153390 0x7ff21c0c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51688/0 (socket says 172.21.15.73:51688) 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff2127fc700 1 -- 172.21.15.73:0/1194306586 learned_addr learned my addr 172.21.15.73:0/1194306586 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff21b7fe700 1 --2- 172.21.15.73:0/1194306586 >> v2:172.21.15.150:3300/0 conn(0x7ff21c156690 0x7ff21c156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.601+0000 7ff21affd700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 -- 172.21.15.73:0/1194306586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 msgr2=0x7ff21c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 -- 172.21.15.73:0/1194306586 >> v2:172.21.15.150:3300/0 conn(0x7ff21c156690 msgr2=0x7ff21c156b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 --2- 172.21.15.73:0/1194306586 >> v2:172.21.15.150:3300/0 conn(0x7ff21c156690 0x7ff21c156b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 -- 172.21.15.73:0/1194306586 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff21c0bff70 con 0x7ff21c153390 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff21affd700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff2127fc700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c153390 0x7ff21c0c5ea0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ff21c155e60 tx=0x7ff2040090f0).ready entity=mon.0 client_cookie=ec131ba284ade8c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff21a7fc700 1 -- 172.21.15.73:0/1194306586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20400b520 con 0x7ff21c153390 2023-11-28T03:04:37.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff21a7fc700 1 -- 172.21.15.73:0/1194306586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff20400b680 con 0x7ff21c153390 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff21a7fc700 1 -- 172.21.15.73:0/1194306586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff20400ba20 con 0x7ff21c153390 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 -- 172.21.15.73:0/1194306586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c153390 msgr2=0x7ff21c0c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c153390 0x7ff21c0c5ea0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 rx=0x7ff21c155e60 tx=0x7ff2040090f0).stop 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 -- 172.21.15.73:0/1194306586 shutdown_connections 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 --2- 172.21.15.73:0/1194306586 >> v2:172.21.15.150:3300/0 conn(0x7ff21c156690 0x7ff21c156b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 --2- 172.21.15.73:0/1194306586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c153390 0x7ff21c0c5ea0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 -- 172.21.15.73:0/1194306586 >> 172.21.15.73:0/1194306586 conn(0x7ff21c062700 msgr2=0x7ff21c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 -- 172.21.15.73:0/1194306586 shutdown_connections 2023-11-28T03:04:37.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.602+0000 7ff22134b700 1 -- 172.21.15.73:0/1194306586 wait complete. 2023-11-28T03:04:37.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.604+0000 7ff22134b700 1 Processor -- start 2023-11-28T03:04:37.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.604+0000 7ff22134b700 1 -- start start 2023-11-28T03:04:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.604+0000 7ff22134b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c2550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.604+0000 7ff22134b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21c153390 0x7ff21c0c0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.604+0000 7ff22134b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff22134b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0ad740 con 0x7ff21c156690 2023-11-28T03:04:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff22134b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0ad0b0 con 0x7ff21c153390 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff22134b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff21c0c10a0 con 0x7ff21c0c5ac0 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51694/0 (socket says 172.21.15.73:51694) 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 -- 172.21.15.73:0/816259249 learned_addr learned my addr 172.21.15.73:0/816259249 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21affd700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff2127fc700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21c153390 0x7ff21c0c0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 -- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 msgr2=0x7ff21c0c2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 -- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21c153390 msgr2=0x7ff21c0c0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:37.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21c153390 0x7ff21c0c0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 -- 172.21.15.73:0/816259249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff204009370 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff21b7fe700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7ff214009df0 tx=0x7ff21400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400caa0 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff21400f040 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff21c0c2b60 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff21400ce10 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.605+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff21c0c3120 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.606+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21c0ad8b0 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.607+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 10) v1 ==== 75577+0+0 (secure 0 0 0) 0x7ff214013610 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.607+0000 7ff2137fe700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 0x7ff1fc061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.607+0000 7ff21affd700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 0x7ff1fc061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.607+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6906+0+0 (secure 0 0 0) 0x7ff21407ae50 con 0x7ff21c156690 2023-11-28T03:04:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.607+0000 7ff21affd700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 0x7ff1fc061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ff20c006d80 tx=0x7ff20c008040).ready entity=mgr.5196 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:37.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.611+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff21c0ad8b0 con 0x7ff21c156690 2023-11-28T03:04:37.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:37.821+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7ff21c067860 con 0x7ff21c156690 2023-11-28T03:04:38.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.344+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7ff21407e030 con 0x7ff21c156690 2023-11-28T03:04:38.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.345+0000 7ff2137fe700 1 -- 172.21.15.73:0/816259249 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v11) v1 ==== 93+0+0 (secure 0 0 0) 0x7ff214072750 con 0x7ff21c156690 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 msgr2=0x7ff1fc061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 0x7ff1fc061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7ff20c006d80 tx=0x7ff20c008040).stop 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 msgr2=0x7ff21c14fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 rx=0x7ff214009df0 tx=0x7ff21400b040).stop 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 shutdown_connections 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1fc05f2c0 0x7ff1fc061780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff21c156690 0x7ff21c14fb40 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff21c0c5ac0 0x7ff21c0c2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 --2- 172.21.15.73:0/816259249 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff21c153390 0x7ff21c0c0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 >> 172.21.15.73:0/816259249 conn(0x7ff21c062700 msgr2=0x7ff21c066980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:38.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 shutdown_connections 2023-11-28T03:04:38.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.348+0000 7ff22134b700 1 -- 172.21.15.73:0/816259249 wait complete. 2023-11-28T03:04:38.363 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:04:38.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:04:38.400 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:04:38.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:04:38.445 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:04:38.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:04:38.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:04:38.489 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:04:38.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:04:38.489 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:04:38.491 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:38.490+0000 7f6916946680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:04:38.497 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:38.496+0000 7f9dc53e2680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:04:38.506 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:04:38.556 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:38.555+0000 7f6916946680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:04:38.560 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:38.559+0000 7f9dc53e2680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:04:38.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.668+0000 7fcab8a1f700 1 Processor -- start 2023-11-28T03:04:38.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.668+0000 7fcab8a1f700 1 -- start start 2023-11-28T03:04:38.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab8a1f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab40671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab8a1f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcab4067980 0x7fcab4067e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab8a1f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcab40bd590 con 0x7fcab4155e40 2023-11-28T03:04:38.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab8a1f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcab40a6090 con 0x7fcab4067980 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab8a1f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4068300 con 0x7fcab4066e00 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab2d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcab4067980 0x7fcab4067e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab1d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcab4155e40 0x7fcab40a5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47994/0 (socket says 172.21.15.73:47994) 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab1d9b700 1 -- 172.21.15.73:0/944146303 learned_addr learned my addr 172.21.15.73:0/944146303 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.669+0000 7fcab259c700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab40671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab2d9d700 1 -- 172.21.15.73:0/944146303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 msgr2=0x7fcab40671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab2d9d700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab40671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab2d9d700 1 -- 172.21.15.73:0/944146303 >> v1:172.21.15.73:6789/0 conn(0x7fcab4155e40 legacy=0x7fcab40a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab2d9d700 1 -- 172.21.15.73:0/944146303 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcab4156370 con 0x7fcab4067980 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab159a700 1 -- 172.21.15.73:0/944146303 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3590301549 0 0) 0x7fcab40bd590 con 0x7fcab4155e40 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab259c700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab40671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:38.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab2d9d700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab4067e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fcaac009e60 tx=0x7fcaac009780).ready entity=mon.0 client_cookie=9e8997be3c6ec6bb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab159a700 1 -- 172.21.15.73:0/944146303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac00e070 con 0x7fcab4067980 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab159a700 1 -- 172.21.15.73:0/944146303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaac00bc30 con 0x7fcab4067980 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab159a700 1 -- 172.21.15.73:0/944146303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaac012410 con 0x7fcab4067980 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab8a1f700 1 -- 172.21.15.73:0/944146303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 msgr2=0x7fcab4067e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.670+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab4067e10 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7fcaac009e60 tx=0x7fcaac009780).stop 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 -- 172.21.15.73:0/944146303 shutdown_connections 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab40671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/944146303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab4067e10 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 -- 172.21.15.73:0/944146303 >> 172.21.15.73:0/944146303 conn(0x7fcab40c7fd0 msgr2=0x7fcab40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 -- 172.21.15.73:0/944146303 shutdown_connections 2023-11-28T03:04:38.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.671+0000 7fcab8a1f700 1 -- 172.21.15.73:0/944146303 wait complete. 2023-11-28T03:04:38.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 Processor -- start 2023-11-28T03:04:38.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 -- start start 2023-11-28T03:04:38.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab4145ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcab4155e40 0x7fcab40a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab4146090 con 0x7fcab4067980 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40abd30 con 0x7fcab4155e40 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.672+0000 7fcab8a1f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcab40ab120 con 0x7fcab4066e00 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51740/0 (socket says 172.21.15.73:51740) 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 -- 172.21.15.73:0/1066630181 learned_addr learned my addr 172.21.15.73:0/1066630181 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab259c700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab4145ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab2d9d700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcab4155e40 0x7fcab40a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 msgr2=0x7fcab4145ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab4145ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcab4155e40 msgr2=0x7fcab40a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcab4155e40 0x7fcab40a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 -- 172.21.15.73:0/1066630181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcaac009a20 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab1d9b700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fcaa800c490 tx=0x7fcaa8007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80070d0 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcaa8007230 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcab40ab400 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcaa80075b0 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.673+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcab40ab9c0 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.674+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcab406c860 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.675+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fcaa80059b0 con 0x7fcab4067980 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.675+0000 7fcaa27fc700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 0x7fca94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.675+0000 7fcab259c700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 msgr2=0x7fca94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:38.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.675+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6714+0+0 (secure 0 0 0) 0x7fcaa8079b00 con 0x7fcab4067980 2023-11-28T03:04:38.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.675+0000 7fcab259c700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 0x7fca94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:38.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.679+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcaa804a9a0 con 0x7fcab4067980 2023-11-28T03:04:38.844 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:38.843+0000 7f9dc53e2680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:04:38.860 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:38.859+0000 7f6916946680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:04:38.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.876+0000 7fcab259c700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 msgr2=0x7fca94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:38.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.876+0000 7fcab259c700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 0x7fca94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:38.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.904+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcab4156650 con 0x7fcab4067980 2023-11-28T03:04:38.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.905+0000 7fcaa27fc700 1 -- 172.21.15.73:0/1066630181 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7fcaa8018030 con 0x7fcab4067980 2023-11-28T03:04:38.906 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:38.906 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T03:04:38.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 msgr2=0x7fca94061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:38.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 0x7fca94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 msgr2=0x7fcab40c2f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 rx=0x7fcaa800c490 tx=0x7fcaa8007e30).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 shutdown_connections 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fca9405f270 0x7fca94061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcab4155e40 0x7fcab40a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcab4066e00 0x7fcab4145ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 --2- 172.21.15.73:0/1066630181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcab4067980 0x7fcab40c2f10 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 >> 172.21.15.73:0/1066630181 conn(0x7fcab40c7fd0 msgr2=0x7fcab40c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 shutdown_connections 2023-11-28T03:04:38.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:38.907+0000 7fcab8a1f700 1 -- 172.21.15.73:0/1066630181 wait complete. 2023-11-28T03:04:38.910 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T03:04:38.920 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:04:39.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.075+0000 7f2f0b60c700 1 Processor -- start 2023-11-28T03:04:39.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.075+0000 7f2f0b60c700 1 -- start start 2023-11-28T03:04:39.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f0b60c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f0b60c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f04066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f0b60c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f040630a0 con 0x7f2f041542d0 2023-11-28T03:04:39.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f0b60c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f04065760 con 0x7f2f04062700 2023-11-28T03:04:39.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f0b60c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c3250 con 0x7f2f040661b0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f093a8700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2f041542d0 0x7f2f040697f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48008/0 (socket says 172.21.15.73:48008) 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f093a8700 1 -- 172.21.15.73:0/1077533608 learned_addr learned my addr 172.21.15.73:0/1077533608 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.076+0000 7f2f09ba9700 1 --2- 172.21.15.73:0/1077533608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f04066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f08ba7700 1 --2- 172.21.15.73:0/1077533608 >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2efbfff700 1 -- 172.21.15.73:0/1077533608 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1342634418 0 0) 0x7f2ef4001110 con 0x7f2f041542d0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2efbfff700 1 -- 172.21.15.73:0/1077533608 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ef00033f0 con 0x7f2f041542d0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f09ba9700 1 -- 172.21.15.73:0/1077533608 >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 msgr2=0x7f2f04062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f09ba9700 1 --2- 172.21.15.73:0/1077533608 >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f09ba9700 1 -- 172.21.15.73:0/1077533608 >> v1:172.21.15.73:6789/0 conn(0x7f2f041542d0 legacy=0x7f2f040697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f09ba9700 1 -- 172.21.15.73:0/1077533608 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0406a340 con 0x7f2f040661b0 2023-11-28T03:04:39.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f08ba7700 1 --2- 172.21.15.73:0/1077533608 >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2f09ba9700 1 --2- 172.21.15.73:0/1077533608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f04066640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2efc009e60 tx=0x7f2efc009780).ready entity=mon.2 client_cookie=12a12c395c3889e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2efbfff700 1 -- 172.21.15.73:0/1077533608 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e070 con 0x7f2f040661b0 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2efbfff700 1 -- 172.21.15.73:0/1077533608 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00bab0 con 0x7f2f040661b0 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.077+0000 7f2efbfff700 1 -- 172.21.15.73:0/1077533608 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00be30 con 0x7f2f040661b0 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 -- 172.21.15.73:0/1077533608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 msgr2=0x7f2f04066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/1077533608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f04066640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2efc009e60 tx=0x7f2efc009780).stop 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 -- 172.21.15.73:0/1077533608 shutdown_connections 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/1077533608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f04066640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/1077533608 >> v2:172.21.15.150:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 -- 172.21.15.73:0/1077533608 >> 172.21.15.73:0/1077533608 conn(0x7f2f040c7fd0 msgr2=0x7f2f040c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 -- 172.21.15.73:0/1077533608 shutdown_connections 2023-11-28T03:04:39.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.078+0000 7f2f0b60c700 1 -- 172.21.15.73:0/1077533608 wait complete. 2023-11-28T03:04:39.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.079+0000 7f2f0b60c700 1 Processor -- start 2023-11-28T03:04:39.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.079+0000 7f2f0b60c700 1 -- start start 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f041514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f041542d0 0x7f2f040a7a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040a8a20 con 0x7f2f04062700 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c12d0 con 0x7f2f041542d0 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f0b60c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c03d0 con 0x7f2f040661b0 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51750/0 (socket says 172.21.15.73:51750) 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 -- 172.21.15.73:0/2777150150 learned_addr learned my addr 172.21.15.73:0/2777150150 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f08ba7700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f041514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f09ba9700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f041542d0 0x7f2f040a7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 -- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 msgr2=0x7f2f041514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f041514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 -- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f041542d0 msgr2=0x7f2f040a7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f041542d0 0x7f2f040a7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.080+0000 7f2f093a8700 1 -- 172.21.15.73:0/2777150150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f000092a0 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2f093a8700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2ef400c440 tx=0x7f2ef4007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40070d0 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef4007230 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2efc009a20 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef40075d0 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.081+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f040a8460 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.082+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f040c0c30 con 0x7f2f04062700 2023-11-28T03:04:39.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.083+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2ef40059b0 con 0x7f2f04062700 2023-11-28T03:04:39.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.083+0000 7f2ef97fa700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 0x7f2ee8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.083+0000 7f2f08ba7700 1 -- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 msgr2=0x7f2ee8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:39.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.083+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6714+0+0 (secure 0 0 0) 0x7f2ef4079d50 con 0x7f2f04062700 2023-11-28T03:04:39.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.083+0000 7f2f08ba7700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 0x7f2ee8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:39.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.088+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ef4049af0 con 0x7f2f04062700 2023-11-28T03:04:39.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.265+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f040630a0 con 0x7f2f04062700 2023-11-28T03:04:39.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.266+0000 7f2ef97fa700 1 -- 172.21.15.73:0/2777150150 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+17672 (secure 0 0 0) 0x7f2f040630a0 con 0x7f2f04062700 2023-11-28T03:04:39.268 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:39.268 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":50,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:04:39.208007+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"50","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":220,"snap_epoch":50,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:6833/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/160657230":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/1072502020":"2023-11-29T02:53:43.710017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":2,"length":3},{"begin":6,"length":11},{"begin":18,"length":9},{"begin":28,"length":10},{"begin":39,"length":9},{"begin":49,"length":9},{"begin":59,"length":9},{"begin":69,"length":9},{"begin":79,"length":9},{"begin":89,"length":9},{"begin":99,"length":12},{"begin":122,"length":99}]}],"new_removed_snaps":[{"pool":6,"snaps":[{"begin":81,"length":1},{"begin":91,"length":1},{"begin":101,"length":1},{"begin":218,"length":3}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:04:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 msgr2=0x7f2ee8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 0x7f2ee8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 msgr2=0x7f2f040c0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 rx=0x7f2ef400c440 tx=0x7f2ef4007af0).stop 2023-11-28T03:04:39.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 shutdown_connections 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f270 0x7f2ee8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f041542d0 0x7f2f040a7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f040c0de0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 --2- 172.21.15.73:0/2777150150 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f041514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.268+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 >> 172.21.15.73:0/2777150150 conn(0x7f2f040c7fd0 msgr2=0x7f2f040c5da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.269+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 shutdown_connections 2023-11-28T03:04:39.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.269+0000 7f2f0b60c700 1 -- 172.21.15.73:0/2777150150 wait complete. 2023-11-28T03:04:39.282 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '37', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T02:53:26.929672+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T02:53:27.932055+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 220, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:04:39.282 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:04:39.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.410+0000 7fcfd18d5700 1 Processor -- start 2023-11-28T03:04:39.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.410+0000 7fcfd18d5700 1 -- start start 2023-11-28T03:04:39.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfd18d5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 0x7fcfcc095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfd18d5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 0x7fcfcc0673d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfd18d5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfcc154c00 con 0x7fcfcc0678c0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfd18d5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfcc14f190 con 0x7fcfcc14edb0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfd18d5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc155770 con 0x7fcfcc096370 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfcb7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcfcc0678c0 0x7fcfcc0bebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48018/0 (socket says 172.21.15.73:48018) 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfcb7fe700 1 -- 172.21.15.73:0/1470697573 learned_addr learned my addr 172.21.15.73:0/1470697573 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1470697573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 0x7fcfcc0673d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfcaffd700 1 --2- 172.21.15.73:0/1470697573 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 0x7fcfcc095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfc9ffb700 1 -- 172.21.15.73:0/1470697573 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 990445326 0 0) 0x7fcfc4001490 con 0x7fcfcc0678c0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfc9ffb700 1 -- 172.21.15.73:0/1470697573 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfb40033f0 con 0x7fcfcc0678c0 2023-11-28T03:04:39.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1470697573 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 msgr2=0x7fcfcc095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1470697573 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 0x7fcfcc095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1470697573 >> v1:172.21.15.73:6789/0 conn(0x7fcfcc0678c0 legacy=0x7fcfcc0bebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1470697573 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc0967f0 con 0x7fcfcc096370 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfcaffd700 1 --2- 172.21.15.73:0/1470697573 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 0x7fcfcc095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.411+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1470697573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 0x7fcfcc0673d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcfc0009760 tx=0x7fcfc0009060).ready entity=mon.2 client_cookie=37d26f8cd1ea2251 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfc9ffb700 1 -- 172.21.15.73:0/1470697573 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000d4a0 con 0x7fcfcc096370 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfc9ffb700 1 -- 172.21.15.73:0/1470697573 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc000d600 con 0x7fcfcc096370 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfc9ffb700 1 -- 172.21.15.73:0/1470697573 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000d980 con 0x7fcfcc096370 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1470697573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 msgr2=0x7fcfcc0673d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1470697573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 0x7fcfcc0673d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcfc0009760 tx=0x7fcfc0009060).stop 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1470697573 shutdown_connections 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1470697573 >> v2:172.21.15.150:3300/0 conn(0x7fcfcc14edb0 0x7fcfcc095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1470697573 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc096370 0x7fcfcc0673d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1470697573 >> 172.21.15.73:0/1470697573 conn(0x7fcfcc0631d0 msgr2=0x7fcfcc0c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1470697573 shutdown_connections 2023-11-28T03:04:39.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.412+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1470697573 wait complete. 2023-11-28T03:04:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.413+0000 7fcfd18d5700 1 Processor -- start 2023-11-28T03:04:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.413+0000 7fcfd18d5700 1 -- start start 2023-11-28T03:04:39.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc0678c0 0x7fcfcc0ac3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc14edb0 0x7fcfcc14d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0ae660 con 0x7fcfcc096370 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0ae940 con 0x7fcfcc14edb0 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfd18d5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfcc0aec20 con 0x7fcfcc0678c0 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfca7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfca7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51756/0 (socket says 172.21.15.73:51756) 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1594534705 learned_addr learned my addr 172.21.15.73:0/1594534705 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfcaffd700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc0678c0 0x7fcfcc0ac3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.414+0000 7fcfcb7fe700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc14edb0 0x7fcfcc14d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc0678c0 msgr2=0x7fcfcc0ac3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc0678c0 0x7fcfcc0ac3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc14edb0 msgr2=0x7fcfcc14d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc14edb0 0x7fcfcc14d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 -- 172.21.15.73:0/1594534705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfbc0092a0 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfca7fc700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fcfc0009d50 tx=0x7fcfc0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000dd40 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc000d770 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfc00092a0 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc000dea0 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.415+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfcc0af1e0 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.416+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfcc06c860 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.416+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fcfc0018070 con 0x7fcfcc096370 2023-11-28T03:04:39.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.417+0000 7fcfb2ffd700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 0x7fcfb8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.417+0000 7fcfcaffd700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 msgr2=0x7fcfb8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:39.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.417+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fcfc007bea0 con 0x7fcfcc096370 2023-11-28T03:04:39.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.417+0000 7fcfcaffd700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 0x7fcfb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:39.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.421+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfc004cec0 con 0x7fcfcc096370 2023-11-28T03:04:39.496 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:39.495+0000 7f9dc53e2680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:04:39.521 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:39.509+0000 7f6916946680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:04:39.571 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:39.570+0000 7f9dc53e2680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:04:39.587 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:39.586+0000 7f6916946680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:04:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.618+0000 7fcfcaffd700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 msgr2=0x7fcfb8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:39.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.618+0000 7fcfcaffd700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 0x7fcfb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:39.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.629+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fcfcc153590 con 0x7fcfcc096370 2023-11-28T03:04:39.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.630+0000 7fcfb2ffd700 1 -- 172.21.15.73:0/1594534705 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v50) v1 ==== 110+0+10 (secure 0 0 0) 0x7fcfc00210b0 con 0x7fcfcc096370 2023-11-28T03:04:39.631 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:04:39.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 msgr2=0x7fcfb8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:39.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 0x7fcfb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 msgr2=0x7fcfcc151cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 rx=0x7fcfc0009d50 tx=0x7fcfc0016000).stop 2023-11-28T03:04:39.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 shutdown_connections 2023-11-28T03:04:39.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfb805f270 0x7fcfb8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfcc14edb0 0x7fcfcc14d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfcc0678c0 0x7fcfcc0ac3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 --2- 172.21.15.73:0/1594534705 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfcc096370 0x7fcfcc151cb0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 >> 172.21.15.73:0/1594534705 conn(0x7fcfcc0631d0 msgr2=0x7fcfcc0c70f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 shutdown_connections 2023-11-28T03:04:39.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.632+0000 7fcfd18d5700 1 -- 172.21.15.73:0/1594534705 wait complete. 2023-11-28T03:04:39.646 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:04:39.760 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:39.759+0000 7f9dc53e2680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:04:39.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 Processor -- start 2023-11-28T03:04:39.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 -- start start 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faefc098d10 0x7faefc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0c72d0 con 0x7faefc0c6190 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc064570 con 0x7faefc158320 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.770+0000 7faf0186f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc154bc0 con 0x7faefc098d10 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefa7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faefc0c6190 0x7faefc063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48028/0 (socket says 172.21.15.73:48028) 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefa7fc700 1 -- 172.21.15.73:0/2217044345 learned_addr learned my addr 172.21.15.73:0/2217044345 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefaffd700 1 --2- 172.21.15.73:0/2217044345 >> v2:172.21.15.150:3300/0 conn(0x7faefc098d10 0x7faefc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 374483924 0 0) 0x7faefc0c72d0 con 0x7faefc0c6190 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee80033f0 con 0x7faefc0c6190 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3278152333 0 0) 0x7faefc064570 con 0x7faefc158320 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faefc0c72d0 con 0x7faefc158320 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefaffd700 1 -- 172.21.15.73:0/2217044345 >> v1:172.21.15.150:6789/0 conn(0x7faefc158320 legacy=0x7faefc0a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefaffd700 1 -- 172.21.15.73:0/2217044345 >> v1:172.21.15.73:6789/0 conn(0x7faefc0c6190 legacy=0x7faefc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefaffd700 1 -- 172.21.15.73:0/2217044345 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0c66c0 con 0x7faefc098d10 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faefaffd700 1 --2- 172.21.15.73:0/2217044345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc098d10 0x7faefc0c7760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7faeec009760 tx=0x7faeec009060).ready entity=mon.1 client_cookie=363362d2766d033e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec00d4a0 con 0x7faefc098d10 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeec00d600 con 0x7faefc098d10 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.771+0000 7faef9ffb700 1 -- 172.21.15.73:0/2217044345 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeec00d980 con 0x7faefc098d10 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 -- 172.21.15.73:0/2217044345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc098d10 msgr2=0x7faefc0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 --2- 172.21.15.73:0/2217044345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc098d10 0x7faefc0c7760 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 rx=0x7faeec009760 tx=0x7faeec009060).stop 2023-11-28T03:04:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 -- 172.21.15.73:0/2217044345 shutdown_connections 2023-11-28T03:04:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 --2- 172.21.15.73:0/2217044345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc098d10 0x7faefc0c7760 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 -- 172.21.15.73:0/2217044345 >> 172.21.15.73:0/2217044345 conn(0x7faefc0cb310 msgr2=0x7faefc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 -- 172.21.15.73:0/2217044345 shutdown_connections 2023-11-28T03:04:39.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.772+0000 7faf0186f700 1 -- 172.21.15.73:0/2217044345 wait complete. 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.773+0000 7faf0186f700 1 Processor -- start 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.773+0000 7faf0186f700 1 -- start start 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faefc098d10 0x7faefc165070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc0c6190 0x7faefc168dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc168660 con 0x7faefc158320 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1658c0 con 0x7faefc0c6190 2023-11-28T03:04:39.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faf0186f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc165ba0 con 0x7faefc098d10 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51768/0 (socket says 172.21.15.73:51768) 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 -- 172.21.15.73:0/243075310 learned_addr learned my addr 172.21.15.73:0/243075310 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefa7fc700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc0c6190 0x7faefc168dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefaffd700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faefc098d10 0x7faefc165070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 -- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faefc098d10 msgr2=0x7faefc165070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faefc098d10 0x7faefc165070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 -- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc0c6190 msgr2=0x7faefc168dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc0c6190 0x7faefc168dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.774+0000 7faefb7fe700 1 -- 172.21.15.73:0/243075310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeec0092a0 con 0x7faefc158320 2023-11-28T03:04:39.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faefb7fe700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7faef400c7c0 tx=0x7faef400cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4002020 con 0x7faefc158320 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef4002180 con 0x7faefc158320 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc165e80 con 0x7faefc158320 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4001590 con 0x7faefc158320 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.775+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc165560 con 0x7faefc158320 2023-11-28T03:04:39.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.776+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc168af0 con 0x7faefc158320 2023-11-28T03:04:39.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.776+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7faef4001980 con 0x7faefc158320 2023-11-28T03:04:39.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.777+0000 7faef2ffd700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 0x7faedc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:39.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.777+0000 7faefaffd700 1 -- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 msgr2=0x7faedc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:39.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.777+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6602+0+0 (secure 0 0 0) 0x7faef4079bb0 con 0x7faefc158320 2023-11-28T03:04:39.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.777+0000 7faefaffd700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 0x7faedc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:39.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.781+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faef4047af0 con 0x7faefc158320 2023-11-28T03:04:39.783 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:39.782+0000 7f6916946680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:04:39.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.952+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7faefc06c860 con 0x7faefc158320 2023-11-28T03:04:39.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.953+0000 7faef2ffd700 1 -- 172.21.15.73:0/243075310 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v50) v1 ==== 92+0+10 (secure 0 0 0) 0x7faef4079020 con 0x7faefc158320 2023-11-28T03:04:39.954 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 msgr2=0x7faedc061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 0x7faedc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 msgr2=0x7faefc156270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7faef400c7c0 tx=0x7faef400cad0).stop 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 shutdown_connections 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faedc05f270 0x7faedc061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:39.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faefc158320 0x7faefc156270 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faefc098d10 0x7faefc165070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 --2- 172.21.15.73:0/243075310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faefc0c6190 0x7faefc168dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:39.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 >> 172.21.15.73:0/243075310 conn(0x7faefc0cb310 msgr2=0x7faefc0c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:39.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 shutdown_connections 2023-11-28T03:04:39.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:39.955+0000 7faf0186f700 1 -- 172.21.15.73:0/243075310 wait complete. 2023-11-28T03:04:39.969 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:04:40.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.089+0000 7fd83b29a700 1 Processor -- start 2023-11-28T03:04:40.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.089+0000 7fd83b29a700 1 -- start start 2023-11-28T03:04:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd83b29a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 0x7fd8340c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd83b29a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 0x7fd834063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd83b29a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8340c72d0 con 0x7fd834064350 2023-11-28T03:04:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd83b29a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd834154bc0 con 0x7fd8340c6190 2023-11-28T03:04:40.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd83b29a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8340c66c0 con 0x7fd834098d10 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd839036700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 0x7fd8340c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd838835700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 0x7fd834063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd839837700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fd834064350 0x7fd834158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:52816/0 (socket says 172.21.15.73:52816) 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.090+0000 7fd839837700 1 -- 172.21.15.73:0/2705873227 learned_addr learned my addr 172.21.15.73:0/2705873227 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd839036700 1 -- 172.21.15.73:0/2705873227 >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 msgr2=0x7fd834063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd839036700 1 --2- 172.21.15.73:0/2705873227 >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 0x7fd834063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd839036700 1 -- 172.21.15.73:0/2705873227 >> v1:172.21.15.150:6789/0 conn(0x7fd834064350 legacy=0x7fd834158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd839036700 1 -- 172.21.15.73:0/2705873227 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8340c69a0 con 0x7fd834098d10 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd838835700 1 --2- 172.21.15.73:0/2705873227 >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 0x7fd834063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd839036700 1 --2- 172.21.15.73:0/2705873227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 0x7fd8340c7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd824009760 tx=0x7fd824009060).ready entity=mon.2 client_cookie=8cc0a3124323b51f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd82bfff700 1 -- 172.21.15.73:0/2705873227 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82400f510 con 0x7fd834098d10 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd82bfff700 1 -- 172.21.15.73:0/2705873227 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82400f670 con 0x7fd834098d10 2023-11-28T03:04:40.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.091+0000 7fd82bfff700 1 -- 172.21.15.73:0/2705873227 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82400f9f0 con 0x7fd834098d10 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 -- 172.21.15.73:0/2705873227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 msgr2=0x7fd8340c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2705873227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 0x7fd8340c7760 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fd824009760 tx=0x7fd824009060).stop 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 -- 172.21.15.73:0/2705873227 shutdown_connections 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2705873227 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834098d10 0x7fd8340c7760 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2705873227 >> v2:172.21.15.150:3300/0 conn(0x7fd8340c6190 0x7fd834063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 -- 172.21.15.73:0/2705873227 >> 172.21.15.73:0/2705873227 conn(0x7fd8340cb310 msgr2=0x7fd8340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 -- 172.21.15.73:0/2705873227 shutdown_connections 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.092+0000 7fd83b29a700 1 -- 172.21.15.73:0/2705873227 wait complete. 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.093+0000 7fd83b29a700 1 Processor -- start 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.093+0000 7fd83b29a700 1 -- start start 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 0x7fd8340abec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 0x7fd834155f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341572c0 con 0x7fd8340c6190 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd834156450 con 0x7fd834098d10 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd83b29a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8341566b0 con 0x7fd834064350 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51772/0 (socket says 172.21.15.73:51772) 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 -- 172.21.15.73:0/2592013636 learned_addr learned my addr 172.21.15.73:0/2592013636 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:40.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd838835700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 0x7fd834155f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839036700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 0x7fd8340abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 -- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 msgr2=0x7fd8340abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 0x7fd8340abec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 -- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 msgr2=0x7fd834155f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 0x7fd834155f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 -- 172.21.15.73:0/2592013636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8300092a0 con 0x7fd8340c6190 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd838835700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 0x7fd834155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839036700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 0x7fd8340abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.094+0000 7fd839837700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd82c00c790 tx=0x7fd82c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.095+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c005d30 con 0x7fd8340c6190 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.095+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd82c005e90 con 0x7fd8340c6190 2023-11-28T03:04:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.095+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8240092a0 con 0x7fd8340c6190 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.095+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd82c007320 con 0x7fd8340c6190 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.095+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd834156c70 con 0x7fd8340c6190 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.096+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd834157430 con 0x7fd8340c6190 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.096+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd82c0076d0 con 0x7fd8340c6190 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.097+0000 7fd8297fa700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 0x7fd818061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.097+0000 7fd839036700 1 -- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 msgr2=0x7fd818061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.097+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6602+0+0 (secure 0 0 0) 0x7fd82c07a440 con 0x7fd8340c6190 2023-11-28T03:04:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.097+0000 7fd839036700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 0x7fd818061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:40.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.101+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd82c04b3b0 con 0x7fd8340c6190 2023-11-28T03:04:40.182 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.181+0000 7f9dc53e2680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:04:40.222 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.221+0000 7f6916946680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:04:40.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.275+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd83406c860 con 0x7fd8340c6190 2023-11-28T03:04:40.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.275+0000 7fd8297fa700 1 -- 172.21.15.73:0/2592013636 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v51) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd82c04e9d0 con 0x7fd8340c6190 2023-11-28T03:04:40.279 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:04:40.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 msgr2=0x7fd818061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:40.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 0x7fd818061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 msgr2=0x7fd834167a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 rx=0x7fd82c00c790 tx=0x7fd82c0015e0).stop 2023-11-28T03:04:40.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 shutdown_connections 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd81805f270 0x7fd818061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8340c6190 0x7fd834167a20 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd834064350 0x7fd8340abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 --2- 172.21.15.73:0/2592013636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd834098d10 0x7fd834155f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 >> 172.21.15.73:0/2592013636 conn(0x7fd8340cb310 msgr2=0x7fd834148b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 shutdown_connections 2023-11-28T03:04:40.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.277+0000 7fd83b29a700 1 -- 172.21.15.73:0/2592013636 wait complete. 2023-11-28T03:04:40.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:04:40.292 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.291+0000 7f9dc53e2680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:04:40.331 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.330+0000 7f6916946680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:04:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.411+0000 7fecc0e25700 1 Processor -- start 2023-11-28T03:04:40.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.411+0000 7fecc0e25700 1 -- start start 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecc0e25700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecbc151a20 0x7fecbc06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecc0e25700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fecbc0bd1f0 0x7fecbc0bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecc0e25700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecbc0c3470 con 0x7fecbc069d10 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecc0e25700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0c48f0 con 0x7fecbc151a20 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecc0e25700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecbc0bdb70 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecbc151a20 0x7fecbc06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecbc151a20 0x7fecbc06a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51782/0 (socket says 172.21.15.73:51782) 2023-11-28T03:04:40.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 -- 172.21.15.73:0/393899866 learned_addr learned my addr 172.21.15.73:0/393899866 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:40.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecbad9d700 1 --2- 172.21.15.73:0/393899866 >> v2:172.21.15.150:3300/0 conn(0x7fecbc0bd1f0 0x7fecbc0bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 -- 172.21.15.73:0/393899866 >> v2:172.21.15.150:3300/0 conn(0x7fecbc0bd1f0 msgr2=0x7fecbc0bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:40.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 --2- 172.21.15.73:0/393899866 >> v2:172.21.15.150:3300/0 conn(0x7fecbc0bd1f0 0x7fecbc0bd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 -- 172.21.15.73:0/393899866 >> v1:172.21.15.150:6789/0 conn(0x7fecbc069d10 legacy=0x7fecbc0be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 -- 172.21.15.73:0/393899866 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecbc151ea0 con 0x7fecbc151a20 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.412+0000 7fecb9d9b700 1 --2- 172.21.15.73:0/393899866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc151a20 0x7fecbc06a0f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fecbc14f670 tx=0x7fecb000b3b0).ready entity=mon.0 client_cookie=32deabf0a65a7c7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecb959a700 1 -- 172.21.15.73:0/393899866 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb000da40 con 0x7fecbc151a20 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecb959a700 1 -- 172.21.15.73:0/393899866 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb000dba0 con 0x7fecbc151a20 2023-11-28T03:04:40.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecb959a700 1 -- 172.21.15.73:0/393899866 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0010470 con 0x7fecbc151a20 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 -- 172.21.15.73:0/393899866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc151a20 msgr2=0x7fecbc06a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 --2- 172.21.15.73:0/393899866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc151a20 0x7fecbc06a0f0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 rx=0x7fecbc14f670 tx=0x7fecb000b3b0).stop 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 -- 172.21.15.73:0/393899866 shutdown_connections 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 --2- 172.21.15.73:0/393899866 >> v2:172.21.15.150:3300/0 conn(0x7fecbc0bd1f0 0x7fecbc0bd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 --2- 172.21.15.73:0/393899866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc151a20 0x7fecbc06a0f0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 -- 172.21.15.73:0/393899866 >> 172.21.15.73:0/393899866 conn(0x7fecbc0642d0 msgr2=0x7fecbc0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 -- 172.21.15.73:0/393899866 shutdown_connections 2023-11-28T03:04:40.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.413+0000 7fecc0e25700 1 -- 172.21.15.73:0/393899866 wait complete. 2023-11-28T03:04:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 Processor -- start 2023-11-28T03:04:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 -- start start 2023-11-28T03:04:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecbc069d10 0x7fecbc1461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecbc151a20 0x7fecbc14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc14f2b0 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc14f890 con 0x7fecbc151a20 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecc0e25700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecbc14fb70 con 0x7fecbc069d10 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecb9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecb9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51794/0 (socket says 172.21.15.73:51794) 2023-11-28T03:04:40.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecb9d9b700 1 -- 172.21.15.73:0/1636398560 learned_addr learned my addr 172.21.15.73:0/1636398560 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecba59c700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecbc069d10 0x7fecbc1461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.415+0000 7fecbad9d700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecbc151a20 0x7fecbc14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 -- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecbc069d10 msgr2=0x7fecbc1461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecbc069d10 0x7fecbc1461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 -- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecbc151a20 msgr2=0x7fecbc14edc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecbc151a20 0x7fecbc14edc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 -- 172.21.15.73:0/1636398560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecb000b610 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecb9d9b700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fecbc0695b0 tx=0x7fecb000b3b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0019450 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecb001f040 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecbc1466d0 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb0019800 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.416+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecbc14bba0 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.417+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecbc06c860 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.417+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fecb0019b70 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.418+0000 7fecaa7fc700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 0x7fec9c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.418+0000 7fecba59c700 1 -- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 msgr2=0x7fec9c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:40.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.418+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6570+0+0 (secure 0 0 0) 0x7fecb007e480 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.418+0000 7fecba59c700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 0x7fec9c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:40.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.425+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fecb004f410 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.446+0000 7f9dc53e2680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:04:40.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.491+0000 7f6916946680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:04:40.523 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.522+0000 7f9dc53e2680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:04:40.574 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.573+0000 7f6916946680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:04:40.592 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.591+0000 7f9dc53e2680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:04:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.602+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fecbc14fcb0 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.603+0000 7fecaa7fc700 1 -- 172.21.15.73:0/1636398560 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v51) v1 ==== 100+0+11 (secure 0 0 0) 0x7fecb0052a30 con 0x7fecbc0bd1f0 2023-11-28T03:04:40.604 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:04:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 msgr2=0x7fec9c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 0x7fec9c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 msgr2=0x7fecbc14e8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 rx=0x7fecbc0695b0 tx=0x7fecb000b3b0).stop 2023-11-28T03:04:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 shutdown_connections 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fec9c05f270 0x7fec9c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecbc151a20 0x7fecbc14edc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecbc069d10 0x7fecbc1461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 --2- 172.21.15.73:0/1636398560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecbc0bd1f0 0x7fecbc14e8d0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.604+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 >> 172.21.15.73:0/1636398560 conn(0x7fecbc0642d0 msgr2=0x7fecbc0c7450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.605+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 shutdown_connections 2023-11-28T03:04:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.605+0000 7fecc0e25700 1 -- 172.21.15.73:0/1636398560 wait complete. 2023-11-28T03:04:40.618 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:04:40.645 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.644+0000 7f6916946680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:04:40.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.760+0000 7f8670ba5700 1 Processor -- start 2023-11-28T03:04:40.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.760+0000 7f8670ba5700 1 -- start start 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8670ba5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f866c154310 0x7f866c067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8670ba5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f866c0bdf90 0x7f866c0c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8670ba5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f866c0c3760 con 0x7f866c066a50 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8670ba5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c0be410 con 0x7f866c0bdf90 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8670ba5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f866c068580 con 0x7f866c154310 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8669d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f866c0bdf90 0x7f866c0c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8669d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f866c0bdf90 0x7f866c0c5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51796/0 (socket says 172.21.15.73:51796) 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f8669d9b700 1 -- 172.21.15.73:0/1420004067 learned_addr learned my addr 172.21.15.73:0/1420004067 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.761+0000 7f866a59c700 1 --2- 172.21.15.73:0/1420004067 >> v2:172.21.15.150:3300/0 conn(0x7f866c154310 0x7f866c067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8669d9b700 1 -- 172.21.15.73:0/1420004067 >> v2:172.21.15.150:3300/0 conn(0x7f866c154310 msgr2=0x7f866c067fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8669d9b700 1 --2- 172.21.15.73:0/1420004067 >> v2:172.21.15.150:3300/0 conn(0x7f866c154310 0x7f866c067fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8669d9b700 1 -- 172.21.15.73:0/1420004067 >> v1:172.21.15.150:6789/0 conn(0x7f866c066a50 legacy=0x7f866c0c3bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8669d9b700 1 -- 172.21.15.73:0/1420004067 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866c068860 con 0x7f866c0bdf90 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f866959a700 1 -- 172.21.15.73:0/1420004067 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835882873 0 0) 0x7f866c0c3760 con 0x7f866c066a50 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8669d9b700 1 --2- 172.21.15.73:0/1420004067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c0bdf90 0x7f866c0c5330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f866c069d60 tx=0x7f865c0090f0).ready entity=mon.0 client_cookie=dcd3d8c73d2e332c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f866959a700 1 -- 172.21.15.73:0/1420004067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00b520 con 0x7f866c0bdf90 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f866959a700 1 -- 172.21.15.73:0/1420004067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c00b680 con 0x7f866c0bdf90 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f866959a700 1 -- 172.21.15.73:0/1420004067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c00ba00 con 0x7f866c0bdf90 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8670ba5700 1 -- 172.21.15.73:0/1420004067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c0bdf90 msgr2=0x7f866c0c5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.762+0000 7f8670ba5700 1 --2- 172.21.15.73:0/1420004067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c0bdf90 0x7f866c0c5330 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 rx=0x7f866c069d60 tx=0x7f865c0090f0).stop 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 -- 172.21.15.73:0/1420004067 shutdown_connections 2023-11-28T03:04:40.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 --2- 172.21.15.73:0/1420004067 >> v2:172.21.15.150:3300/0 conn(0x7f866c154310 0x7f866c067fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 --2- 172.21.15.73:0/1420004067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c0bdf90 0x7f866c0c5330 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 -- 172.21.15.73:0/1420004067 >> 172.21.15.73:0/1420004067 conn(0x7f866c0c9030 msgr2=0x7f866c0c9440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 -- 172.21.15.73:0/1420004067 shutdown_connections 2023-11-28T03:04:40.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.763+0000 7f8670ba5700 1 -- 172.21.15.73:0/1420004067 wait complete. 2023-11-28T03:04:40.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 Processor -- start 2023-11-28T03:04:40.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 -- start start 2023-11-28T03:04:40.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 0x7f866c0a8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 0x7f866c14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0a9dd0 con 0x7f866c154310 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.764+0000 7f8670ba5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0abb30 con 0x7f866c0bdf90 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8670ba5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866c0a91c0 con 0x7f866c066a50 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 0x7f866c14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8669d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866a59c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 0x7f866c0a8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8669d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:48556/0 (socket says 172.21.15.73:48556) 2023-11-28T03:04:40.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8669d9b700 1 -- 172.21.15.73:0/2260402436 learned_addr learned my addr 172.21.15.73:0/2260402436 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 msgr2=0x7f866c0a8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 0x7f866c0a8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 msgr2=0x7f866c0ab640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 -- 172.21.15.73:0/2260402436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c009370 con 0x7f866c154310 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8669d9b700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866a59c700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 0x7f866c0a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f866ad9d700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 0x7f866c14c3f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f866400c7e0 tx=0x7f8664001480).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400f470 con 0x7f866c154310 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866400f5d0 con 0x7f866c154310 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866c0a94a0 con 0x7f866c154310 2023-11-28T03:04:40.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f866400f970 con 0x7f866c154310 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.765+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f866c0a9a60 con 0x7f866c154310 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.766+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866c06c830 con 0x7f866c154310 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.767+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f8664012400 con 0x7f866c154310 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.767+0000 7f865a7fc700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 0x7f864c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.767+0000 7f866a59c700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 msgr2=0x7f864c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.767+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6570+0+0 (secure 0 0 0) 0x7f8664079ad0 con 0x7f866c154310 2023-11-28T03:04:40.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.767+0000 7f866a59c700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 0x7f864c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:40.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.775+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8664048af0 con 0x7f866c154310 2023-11-28T03:04:40.854 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:40.853+0000 7f9dc53e2680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:04:40.918 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:40.917+0000 7f6916946680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:04:40.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.968+0000 7f866a59c700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 msgr2=0x7f864c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:40.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.968+0000 7f866a59c700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 0x7f864c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:40.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.987+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f866c067740 con 0x7f866c154310 2023-11-28T03:04:40.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.988+0000 7f865a7fc700 1 -- 172.21.15.73:0/2260402436 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f866404aab0 con 0x7f866c154310 2023-11-28T03:04:40.989 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:40.989 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T03:04:40.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 msgr2=0x7f864c061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 0x7f864c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 msgr2=0x7f866c14c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 0x7f866c14c3f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 rx=0x7f866400c7e0 tx=0x7f8664001480).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 shutdown_connections 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f864c05f240 0x7f864c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f866c154310 0x7f866c14c3f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f866c066a50 0x7f866c0a8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 --2- 172.21.15.73:0/2260402436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f866c0bdf90 0x7f866c0ab640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 >> 172.21.15.73:0/2260402436 conn(0x7f866c0c9030 msgr2=0x7f866c063810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 shutdown_connections 2023-11-28T03:04:40.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:40.990+0000 7f8670ba5700 1 -- 172.21.15.73:0/2260402436 wait complete. 2023-11-28T03:04:40.993 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T03:04:41.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:04:41.040 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:41.039+0000 7f9dc53e2680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:04:41.107 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:41.106+0000 7f6916946680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:04:41.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.123+0000 7f3ae4918700 1 Processor -- start 2023-11-28T03:04:41.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.123+0000 7f3ae4918700 1 -- start start 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ae4918700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3ae0066e00 0x7f3ae00671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ae4918700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ae0155e40 0x7f3ae00a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ae4918700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ae00bd590 con 0x7f3ae00a5e70 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ae4918700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae0156370 con 0x7f3ae0066e00 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ae4918700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ae0156650 con 0x7f3ae0155e40 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3ade59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3ae0066e00 0x7f3ae00671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3aded9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3ae00a5e70 0x7f3ae0067980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48044/0 (socket says 172.21.15.73:48044) 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.124+0000 7f3aded9d700 1 -- 172.21.15.73:0/2084023438 learned_addr learned my addr 172.21.15.73:0/2084023438 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3addd9b700 1 --2- 172.21.15.73:0/2084023438 >> v2:172.21.15.150:3300/0 conn(0x7f3ae0155e40 0x7f3ae00a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3ade59c700 1 -- 172.21.15.73:0/2084023438 >> v2:172.21.15.150:3300/0 conn(0x7f3ae0155e40 msgr2=0x7f3ae00a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:41.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3ade59c700 1 --2- 172.21.15.73:0/2084023438 >> v2:172.21.15.150:3300/0 conn(0x7f3ae0155e40 0x7f3ae00a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3ade59c700 1 -- 172.21.15.73:0/2084023438 >> v1:172.21.15.73:6789/0 conn(0x7f3ae00a5e70 legacy=0x7f3ae0067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3ade59c700 1 -- 172.21.15.73:0/2084023438 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ae0156930 con 0x7f3ae0066e00 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3ade59c700 1 --2- 172.21.15.73:0/2084023438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae0066e00 0x7f3ae00671e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f3ae0063510 tx=0x7f3ad8009780).ready entity=mon.0 client_cookie=fd247c9758516017 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3add59a700 1 -- 172.21.15.73:0/2084023438 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad800e070 con 0x7f3ae0066e00 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3add59a700 1 -- 172.21.15.73:0/2084023438 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad800bc30 con 0x7f3ae0066e00 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.125+0000 7f3add59a700 1 -- 172.21.15.73:0/2084023438 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad8012410 con 0x7f3ae0066e00 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 -- 172.21.15.73:0/2084023438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae0066e00 msgr2=0x7f3ae00671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2084023438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae0066e00 0x7f3ae00671e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 rx=0x7f3ae0063510 tx=0x7f3ad8009780).stop 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 -- 172.21.15.73:0/2084023438 shutdown_connections 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2084023438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae0066e00 0x7f3ae00671e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2084023438 >> v2:172.21.15.150:3300/0 conn(0x7f3ae0155e40 0x7f3ae00a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 -- 172.21.15.73:0/2084023438 >> 172.21.15.73:0/2084023438 conn(0x7f3ae00c7fd0 msgr2=0x7f3ae00c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:41.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 -- 172.21.15.73:0/2084023438 shutdown_connections 2023-11-28T03:04:41.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.126+0000 7f3ae4918700 1 -- 172.21.15.73:0/2084023438 wait complete. 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.127+0000 7f3ae4918700 1 Processor -- start 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.127+0000 7f3ae4918700 1 -- start start 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ae0066e00 0x7f3ae0142ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae0155e40 0x7f3ae0165ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0142fc0 con 0x7f3ae01663b0 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0166980 con 0x7f3ae0155e40 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ae0166c60 con 0x7f3ae0066e00 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51818/0 (socket says 172.21.15.73:51818) 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 -- 172.21.15.73:0/2287445651 learned_addr learned my addr 172.21.15.73:0/2287445651 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3addd9b700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae0155e40 0x7f3ae0165ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ade59c700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ae0066e00 0x7f3ae0142ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ae0066e00 msgr2=0x7f3ae0142ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ae0066e00 0x7f3ae0142ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae0155e40 msgr2=0x7f3ae0165ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae0155e40 0x7f3ae0165ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 -- 172.21.15.73:0/2287445651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad8009a20 con 0x7f3ae01663b0 2023-11-28T03:04:41.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3aded9d700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3ad000c440 tx=0x7f3ad0007d40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00070d0 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ad0007230 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ad00075b0 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ae00a8b70 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.128+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ae0151290 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f3ad00059b0 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ae00a9090 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ad67fc700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 0x7f3ac0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ade59c700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 msgr2=0x7f3ac0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 6570+0+0 (secure 0 0 0) 0x7f3ad0079ab0 con 0x7f3ae01663b0 2023-11-28T03:04:41.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.130+0000 7f3ade59c700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 0x7f3ac0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:41.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.134+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3ae00a9090 con 0x7f3ae01663b0 2023-11-28T03:04:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.331+0000 7f3ade59c700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 msgr2=0x7f3ac0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:41.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.331+0000 7f3ade59c700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 0x7f3ac0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:41.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.336+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3ae00a9090 con 0x7f3ae01663b0 2023-11-28T03:04:41.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.337+0000 7f3ad67fc700 1 -- 172.21.15.73:0/2287445651 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18921 (secure 0 0 0) 0x7f3ae00a9090 con 0x7f3ae01663b0 2023-11-28T03:04:41.338 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:41.338 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:04:41.200517+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:6833/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/160657230":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/1072502020":"2023-11-29T02:53:43.710017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":1},{"begin":7,"length":2},{"begin":11,"length":1},{"begin":13,"length":1},{"begin":16,"length":1},{"begin":19,"length":2},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":29,"length":2},{"begin":34,"length":1},{"begin":37,"length":1},{"begin":40,"length":2},{"begin":44,"length":1},{"begin":47,"length":1},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":57,"length":1},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":67,"length":1},{"begin":70,"length":2},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":80,"length":2},{"begin":84,"length":1},{"begin":86,"length":2},{"begin":90,"length":2},{"begin":94,"length":1},{"begin":96,"length":2},{"begin":100,"length":2},{"begin":104,"length":1},{"begin":106,"length":2},{"begin":110,"length":2},{"begin":169,"length":53}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2},{"begin":6,"length":1},{"begin":9,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":1},{"begin":21,"length":2},{"begin":24,"length":2},{"begin":28,"length":1},{"begin":31,"length":3},{"begin":35,"length":2},{"begin":39,"length":1},{"begin":42,"length":2},{"begin":45,"length":2},{"begin":49,"length":1},{"begin":52,"length":2},{"begin":55,"length":2},{"begin":59,"length":1},{"begin":62,"length":2},{"begin":65,"length":2},{"begin":69,"length":1},{"begin":72,"length":2},{"begin":75,"length":2},{"begin":79,"length":1},{"begin":82,"length":2},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":92,"length":2},{"begin":95,"length":1},{"begin":99,"length":1},{"begin":102,"length":2},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":122,"length":47}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 msgr2=0x7f3ac0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 0x7f3ac0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 msgr2=0x7f3ae0065670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 rx=0x7f3ad000c440 tx=0x7f3ad0007d40).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 shutdown_connections 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ac005f270 0x7f3ac0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ae01663b0 0x7f3ae0065670 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ae0066e00 0x7f3ae0142ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 --2- 172.21.15.73:0/2287445651 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ae0155e40 0x7f3ae0165ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 >> 172.21.15.73:0/2287445651 conn(0x7f3ae00c7fd0 msgr2=0x7f3ae00acf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:41.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.338+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 shutdown_connections 2023-11-28T03:04:41.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.339+0000 7f3ae4918700 1 -- 172.21.15.73:0/2287445651 wait complete. 2023-11-28T03:04:41.352 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:04:41.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:04:41.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.470+0000 7f51e6a8d700 1 Processor -- start 2023-11-28T03:04:41.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.470+0000 7f51e6a8d700 1 -- start start 2023-11-28T03:04:41.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.470+0000 7f51e6a8d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00a81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.470+0000 7f51e6a8d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e00c60f0 0x7f51e00c7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e6a8d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e0158450 con 0x7f51e00a8690 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e6a8d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e00c77b0 con 0x7f51e00c60f0 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e6a8d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00c7940 con 0x7f51e0154270 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e00c60f0 0x7f51e00c7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e00c60f0 0x7f51e00c7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51832/0 (socket says 172.21.15.73:51832) 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 -- 172.21.15.73:0/1460533838 learned_addr learned my addr 172.21.15.73:0/1460533838 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51dffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f51e00a8690 0x7f51e00c59e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48048/0 (socket says 172.21.15.73:48048) 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e4829700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00a81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 -- 172.21.15.73:0/1460533838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 msgr2=0x7f51e00a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00a81a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 -- 172.21.15.73:0/1460533838 >> v1:172.21.15.73:6789/0 conn(0x7f51e00a8690 legacy=0x7f51e00c59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 -- 172.21.15.73:0/1460533838 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e0158730 con 0x7f51e00c60f0 2023-11-28T03:04:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51df7fe700 1 -- 172.21.15.73:0/1460533838 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3025037021 0 0) 0x7f51e0158450 con 0x7f51e00a8690 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51e502a700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e00c7d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f51d8009f20 tx=0x7f51d8009840).ready entity=mon.0 client_cookie=43ab04ed12f5ff84 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51df7fe700 1 -- 172.21.15.73:0/1460533838 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800e070 con 0x7f51e00c60f0 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51df7fe700 1 -- 172.21.15.73:0/1460533838 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800b9d0 con 0x7f51e00c60f0 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.471+0000 7f51df7fe700 1 -- 172.21.15.73:0/1460533838 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800bd50 con 0x7f51e00c60f0 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1460533838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 msgr2=0x7f51e00c7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e00c7d90 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 rx=0x7f51d8009f20 tx=0x7f51d8009840).stop 2023-11-28T03:04:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1460533838 shutdown_connections 2023-11-28T03:04:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00a81a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1460533838 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e00c7d90 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1460533838 >> 172.21.15.73:0/1460533838 conn(0x7f51e0062700 msgr2=0x7f51e0062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1460533838 shutdown_connections 2023-11-28T03:04:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.472+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1460533838 wait complete. 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.473+0000 7f51e6a8d700 1 Processor -- start 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.473+0000 7f51e6a8d700 1 -- start start 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e00a8690 0x7f51e01574d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00c49a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e00ad420 con 0x7f51e00c60f0 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0154ed0 con 0x7f51e00a8690 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e6a8d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e0155180 con 0x7f51e0154270 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51838/0 (socket says 172.21.15.73:51838) 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 -- 172.21.15.73:0/1942739619 learned_addr learned my addr 172.21.15.73:0/1942739619 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:41.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e502a700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00c49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51e4829700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e00a8690 0x7f51e01574d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 msgr2=0x7f51e00c49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00c49a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e00a8690 msgr2=0x7f51e01574d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e00a8690 0x7f51e01574d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 -- 172.21.15.73:0/1942739619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d8009ae0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.474+0000 7f51dffff700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f51cc00c490 tx=0x7f51cc007c00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.475+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc00ce40 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.475+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51cc0072e0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.475+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e00acaf0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.475+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51cc0056e0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.475+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e00ad0b0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f51cc005ad0 con 0x7f51e00c60f0 2023-11-28T03:04:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e00ad590 con 0x7f51e00c60f0 2023-11-28T03:04:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51dcff9700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 0x7f51c4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51e4829700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 msgr2=0x7f51c4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f51cc07a090 con 0x7f51e00c60f0 2023-11-28T03:04:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.476+0000 7f51e4829700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 0x7f51c4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.481+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51cc04acc0 con 0x7f51e00c60f0 2023-11-28T03:04:41.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.677+0000 7f51e4829700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 msgr2=0x7f51c4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:41.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.677+0000 7f51e4829700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 0x7f51c4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:41.704 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:41.703+0000 7f9dc53e2680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:04:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.743+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51e006c860 con 0x7f51e00c60f0 2023-11-28T03:04:41.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.744+0000 7f51dcff9700 1 -- 172.21.15.73:0/1942739619 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f51cc018070 con 0x7f51e00c60f0 2023-11-28T03:04:41.745 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:41.746 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T03:04:41.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 msgr2=0x7f51c4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:41.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 0x7f51c4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 msgr2=0x7f51e01549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 rx=0x7f51cc00c490 tx=0x7f51cc007c00).stop 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 shutdown_connections 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51c405f270 0x7f51c4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e0154270 0x7f51e00c49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e00a8690 0x7f51e01574d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 --2- 172.21.15.73:0/1942739619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e00c60f0 0x7f51e01549e0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 >> 172.21.15.73:0/1942739619 conn(0x7f51e0062700 msgr2=0x7f51e00bf6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 shutdown_connections 2023-11-28T03:04:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.746+0000 7f51e6a8d700 1 -- 172.21.15.73:0/1942739619 wait complete. 2023-11-28T03:04:41.749 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T03:04:41.763 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:04:41.774 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:41.773+0000 7f9dc53e2680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:04:41.848 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:41.847+0000 7f9dc53e2680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:04:41.852 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:41.852+0000 7f6916946680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:04:41.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.880+0000 7f9249d58700 1 Processor -- start 2023-11-28T03:04:41.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.880+0000 7f9249d58700 1 -- start start 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.880+0000 7f9249d58700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 0x7f9244155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f9249d58700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92440683d0 con 0x7f92440673c0 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f9249d58700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9244067850 con 0x7f9244066f40 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f9249d58700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9244067b30 con 0x7f92441544d0 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f9243fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92440673c0 0x7f924415e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48064/0 (socket says 172.21.15.73:48064) 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f9243fff700 1 -- 172.21.15.73:0/1225396717 learned_addr learned my addr 172.21.15.73:0/1225396717 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:41.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92437fe700 1 --2- 172.21.15.73:0/1225396717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 0x7f9244155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010244093 0 0) 0x7f92440683d0 con 0x7f92440673c0 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f922c0033f0 con 0x7f92440673c0 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2809483941 0 0) 0x7f9244067850 con 0x7f9244066f40 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92440683d0 con 0x7f9244066f40 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92437fe700 1 -- 172.21.15.73:0/1225396717 >> v1:172.21.15.150:6789/0 conn(0x7f9244066f40 legacy=0x7f9244095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92437fe700 1 -- 172.21.15.73:0/1225396717 >> v1:172.21.15.73:6789/0 conn(0x7f92440673c0 legacy=0x7f924415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92437fe700 1 -- 172.21.15.73:0/1225396717 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9244067e10 con 0x7f92441544d0 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92437fe700 1 --2- 172.21.15.73:0/1225396717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 0x7f9244155750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9234009760 tx=0x7f9234006f40).ready entity=mon.2 client_cookie=5075b5defe190897 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.881+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234005880 con 0x7f92441544d0 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92340059e0 con 0x7f92441544d0 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f92427fc700 1 -- 172.21.15.73:0/1225396717 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9234005d60 con 0x7f92441544d0 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 -- 172.21.15.73:0/1225396717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 msgr2=0x7f9244155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 --2- 172.21.15.73:0/1225396717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 0x7f9244155750 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9234009760 tx=0x7f9234006f40).stop 2023-11-28T03:04:41.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 -- 172.21.15.73:0/1225396717 shutdown_connections 2023-11-28T03:04:41.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 --2- 172.21.15.73:0/1225396717 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92441544d0 0x7f9244155750 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 -- 172.21.15.73:0/1225396717 >> 172.21.15.73:0/1225396717 conn(0x7f92440631d0 msgr2=0x7f92440c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:41.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 -- 172.21.15.73:0/1225396717 shutdown_connections 2023-11-28T03:04:41.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.882+0000 7f9249d58700 1 -- 172.21.15.73:0/1225396717 wait complete. 2023-11-28T03:04:41.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.883+0000 7f9249d58700 1 Processor -- start 2023-11-28T03:04:41.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.883+0000 7f9249d58700 1 -- start start 2023-11-28T03:04:41.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 0x7f9244159960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92440a76c0 0x7f924415a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92441524f0 con 0x7f92441544d0 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92441532f0 con 0x7f9244066f40 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9249d58700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9244152720 con 0x7f92440a76c0 2023-11-28T03:04:41.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9242ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9242ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51840/0 (socket says 172.21.15.73:51840) 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9242ffd700 1 -- 172.21.15.73:0/1260672630 learned_addr learned my addr 172.21.15.73:0/1260672630 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f92437fe700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 0x7f9244159960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.884+0000 7f9243fff700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92440a76c0 0x7f924415a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92440a76c0 msgr2=0x7f924415a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92440a76c0 0x7f924415a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 msgr2=0x7f9244159960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 0x7f9244159960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 -- 172.21.15.73:0/1260672630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9234009250 con 0x7f92441544d0 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f92437fe700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 0x7f9244159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9242ffd700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f923800c490 tx=0x7f9238007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800ce40 con 0x7f92441544d0 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923800f470 con 0x7f92441544d0 2023-11-28T03:04:41.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9244153780 con 0x7f92441544d0 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923800f830 con 0x7f92441544d0 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.885+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9244152ca0 con 0x7f92441544d0 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.886+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92441534a0 con 0x7f92441544d0 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.886+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f923800fbe0 con 0x7f92441544d0 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.887+0000 7f922b7fe700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 0x7f9230061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.887+0000 7f92437fe700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 msgr2=0x7f9230061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.887+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f9238079ea0 con 0x7f92441544d0 2023-11-28T03:04:41.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.887+0000 7f92437fe700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 0x7f9230061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:41.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:41.891+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92441534a0 con 0x7f92441544d0 2023-11-28T03:04:41.926 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:41.925+0000 7f6916946680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:04:41.995 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:41.994+0000 7f9dc53e2680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:04:42.003 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.003+0000 7f6916946680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:04:42.066 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:42.065+0000 7f9dc53e2680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:04:42.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.088+0000 7f92437fe700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 msgr2=0x7f9230061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:42.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.088+0000 7f92437fe700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 0x7f9230061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:42.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.102+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9244063a30 con 0x7f92441544d0 2023-11-28T03:04:42.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.103+0000 7f922b7fe700 1 -- 172.21.15.73:0/1260672630 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+4492 (secure 0 0 0) 0x7f923804aa10 con 0x7f92441544d0 2023-11-28T03:04:42.105 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:42.105 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":12,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":12,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T02:53:28.963671+0000","modified":"2023-11-28T02:53:34.452717+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4974},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4974":{"gid":4974,"name":"c","rank":0,"incarnation":10,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1459377503","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1459377503},{"type":"v1","addr":"172.21.15.73:6837","nonce":1459377503}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-11-28T03:04:42.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 msgr2=0x7f9230061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:42.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 0x7f9230061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:42.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 msgr2=0x7f92440a71d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 rx=0x7f923800c490 tx=0x7f9238007c60).stop 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 shutdown_connections 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f923005f270 0x7f9230061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92440a76c0 0x7f924415a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9244066f40 0x7f9244159960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 --2- 172.21.15.73:0/1260672630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92441544d0 0x7f92440a71d0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 >> 172.21.15.73:0/1260672630 conn(0x7f92440631d0 msgr2=0x7f9244095e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:42.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 shutdown_connections 2023-11-28T03:04:42.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.105+0000 7f9249d58700 1 -- 172.21.15.73:0/1260672630 wait complete. 2023-11-28T03:04:42.109 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 12 2023-11-28T03:04:42.119 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:04:42.155 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.154+0000 7f6916946680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:04:42.181 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:42.180+0000 7f9dc53e2680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:04:42.228 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.227+0000 7f6916946680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:04:42.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.234+0000 7f69054bf700 1 Processor -- start 2023-11-28T03:04:42.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.234+0000 7f69054bf700 1 -- start start 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f69054bf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000c51a0 0x7f69000c5580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f69054bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f69001544a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f69054bf700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6900157fe0 con 0x7f690006b8e0 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f69054bf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69001582c0 con 0x7f69000c51a0 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f69054bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69000628a0 con 0x7f6900064950 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f68feffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000c51a0 0x7f69000c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f68feffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000c51a0 0x7f69000c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51852/0 (socket says 172.21.15.73:51852) 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f68feffd700 1 -- 172.21.15.73:0/804699972 learned_addr learned my addr 172.21.15.73:0/804699972 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:42.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.235+0000 7f68ff7fe700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f69001544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68feffd700 1 -- 172.21.15.73:0/804699972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 msgr2=0x7f69001544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68feffd700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f69001544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68feffd700 1 -- 172.21.15.73:0/804699972 >> v1:172.21.15.150:6789/0 conn(0x7f690006b8e0 legacy=0x7f6900153d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68feffd700 1 -- 172.21.15.73:0/804699972 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900062b80 con 0x7f69000c51a0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68feffd700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f69000c5580 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f68f4006fd0 tx=0x7f68f40090f0).ready entity=mon.0 client_cookie=366bb6e7d9bc522f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68fdffb700 1 -- 172.21.15.73:0/804699972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400b580 con 0x7f69000c51a0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68fdffb700 1 -- 172.21.15.73:0/804699972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f400b6e0 con 0x7f69000c51a0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f68fdffb700 1 -- 172.21.15.73:0/804699972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f400ba60 con 0x7f69000c51a0 2023-11-28T03:04:42.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 -- 172.21.15.73:0/804699972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 msgr2=0x7f69000c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f69000c5580 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 rx=0x7f68f4006fd0 tx=0x7f68f40090f0).stop 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 -- 172.21.15.73:0/804699972 shutdown_connections 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f69001544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 --2- 172.21.15.73:0/804699972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f69000c5580 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 -- 172.21.15.73:0/804699972 >> 172.21.15.73:0/804699972 conn(0x7f69000c3f20 msgr2=0x7f69000c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 -- 172.21.15.73:0/804699972 shutdown_connections 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.236+0000 7f69054bf700 1 -- 172.21.15.73:0/804699972 wait complete. 2023-11-28T03:04:42.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 Processor -- start 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 -- start start 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f6900154530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 0x7f69000a9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6900155700 con 0x7f69000c51a0 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6900154af0 con 0x7f690006b8e0 2023-11-28T03:04:42.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.238+0000 7f69054bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6900154dd0 con 0x7f6900064950 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68feffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f6900154530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51858/0 (socket says 172.21.15.73:51858) 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 -- 172.21.15.73:0/123138862 learned_addr learned my addr 172.21.15.73:0/123138862 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68fe7fc700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 0x7f69000a9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 -- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 msgr2=0x7f6900154530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f6900154530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 -- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 msgr2=0x7f69000a9060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 0x7f69000a9060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 -- 172.21.15.73:0/123138862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f4009370 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68feffd700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f6900154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68fe7fc700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 0x7f69000a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68ff7fe700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f68f8009e90 tx=0x7f68f800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800caa0 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f800f040 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69001550b0 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f800ce10 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.239+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69000a9550 con 0x7f69000c51a0 2023-11-28T03:04:42.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.240+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69001558b0 con 0x7f69000c51a0 2023-11-28T03:04:42.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.241+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f68f8013610 con 0x7f69000c51a0 2023-11-28T03:04:42.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.241+0000 7f68e6ffd700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 0x7f68ec061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.241+0000 7f68feffd700 1 -- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 msgr2=0x7f68ec061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:42.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.241+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f68f807b130 con 0x7f69000c51a0 2023-11-28T03:04:42.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.241+0000 7f68feffd700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 0x7f68ec061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:42.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.249+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f69001558b0 con 0x7f69000c51a0 2023-11-28T03:04:42.289 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:42.288+0000 7f9dc53e2680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:04:42.351 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.350+0000 7f6916946680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:04:42.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.421+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f690006c7c0 con 0x7f69000c51a0 2023-11-28T03:04:42.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.422+0000 7f68e6ffd700 1 -- 172.21.15.73:0/123138862 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) v1 ==== 74+0+18921 (secure 0 0 0) 0x7f68f804f790 con 0x7f69000c51a0 2023-11-28T03:04:42.424 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:42.424 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:04:41.200517+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"37","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-11-28T02:53:26.929672+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-11-28T02:53:27.932055+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"51","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":51,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"6.24","mappings":[{"from":5,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6832/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:6833/90017":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/160657230":"2023-11-29T02:53:43.710017+0000","172.21.15.73:0/1072502020":"2023-11-29T02:53:43.710017+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":6,"snaps":[{"begin":4,"length":1},{"begin":7,"length":2},{"begin":11,"length":1},{"begin":13,"length":1},{"begin":16,"length":1},{"begin":19,"length":2},{"begin":23,"length":1},{"begin":26,"length":1},{"begin":29,"length":2},{"begin":34,"length":1},{"begin":37,"length":1},{"begin":40,"length":2},{"begin":44,"length":1},{"begin":47,"length":1},{"begin":50,"length":2},{"begin":54,"length":1},{"begin":57,"length":1},{"begin":60,"length":2},{"begin":64,"length":1},{"begin":67,"length":1},{"begin":70,"length":2},{"begin":74,"length":1},{"begin":77,"length":1},{"begin":80,"length":2},{"begin":84,"length":1},{"begin":86,"length":2},{"begin":90,"length":2},{"begin":94,"length":1},{"begin":96,"length":2},{"begin":100,"length":2},{"begin":104,"length":1},{"begin":106,"length":2},{"begin":110,"length":2},{"begin":169,"length":53}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":6,"snaps":[{"begin":2,"length":2},{"begin":6,"length":1},{"begin":9,"length":2},{"begin":12,"length":1},{"begin":14,"length":2},{"begin":18,"length":1},{"begin":21,"length":2},{"begin":24,"length":2},{"begin":28,"length":1},{"begin":31,"length":3},{"begin":35,"length":2},{"begin":39,"length":1},{"begin":42,"length":2},{"begin":45,"length":2},{"begin":49,"length":1},{"begin":52,"length":2},{"begin":55,"length":2},{"begin":59,"length":1},{"begin":62,"length":2},{"begin":65,"length":2},{"begin":69,"length":1},{"begin":72,"length":2},{"begin":75,"length":2},{"begin":79,"length":1},{"begin":82,"length":2},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":92,"length":2},{"begin":95,"length":1},{"begin":99,"length":1},{"begin":102,"length":2},{"begin":105,"length":1},{"begin":108,"length":2},{"begin":122,"length":47}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:04:42.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 msgr2=0x7f68ec061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:42.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 0x7f68ec061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:42.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 msgr2=0x7f6900150970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 rx=0x7f68f8009e90 tx=0x7f68f800b040).stop 2023-11-28T03:04:42.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 shutdown_connections 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68ec05f270 0x7f68ec061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000c51a0 0x7f6900150970 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6900064950 0x7f6900154530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 --2- 172.21.15.73:0/123138862 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f690006b8e0 0x7f69000a9060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 >> 172.21.15.73:0/123138862 conn(0x7f69000c3f20 msgr2=0x7f69000bcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 shutdown_connections 2023-11-28T03:04:42.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.425+0000 7f69054bf700 1 -- 172.21.15.73:0/123138862 wait complete. 2023-11-28T03:04:42.438 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:04:42.464 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.463+0000 7f6916946680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:04:42.582 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:42.581+0000 7f9dc53e2680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:04:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.584+0000 7f2bdd055700 1 Processor -- start 2023-11-28T03:04:42.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.584+0000 7f2bdd055700 1 -- start start 2023-11-28T03:04:42.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bdd055700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2bd80c3040 0x7f2bd80c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bdd055700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2bd80c63b0 0x7f2bd8069930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bdd055700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bd80c6860 con 0x7f2bd80c2c60 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bdd055700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80c3a90 con 0x7f2bd80c3040 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bdd055700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bd80c3d70 con 0x7f2bd80c63b0 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd6d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2bd80c2c60 0x7f2bd8152420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48070/0 (socket says 172.21.15.73:48070) 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2bd80c3040 0x7f2bd80c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd6d9d700 1 -- 172.21.15.73:0/1474231694 learned_addr learned my addr 172.21.15.73:0/1474231694 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd759e700 1 --2- 172.21.15.73:0/1474231694 >> v2:172.21.15.150:3300/0 conn(0x7f2bd80c63b0 0x7f2bd8069930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 -- 172.21.15.73:0/1474231694 >> v2:172.21.15.150:3300/0 conn(0x7f2bd80c63b0 msgr2=0x7f2bd8069930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 --2- 172.21.15.73:0/1474231694 >> v2:172.21.15.150:3300/0 conn(0x7f2bd80c63b0 0x7f2bd8069930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 -- 172.21.15.73:0/1474231694 >> v1:172.21.15.73:6789/0 conn(0x7f2bd80c2c60 legacy=0x7f2bd8152420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd5d9b700 1 -- 172.21.15.73:0/1474231694 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2243434008 0 0) 0x7f2bc0001110 con 0x7f2bd80c2c60 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 -- 172.21.15.73:0/1474231694 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bd80c4050 con 0x7f2bd80c3040 2023-11-28T03:04:42.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd659c700 1 --2- 172.21.15.73:0/1474231694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c5ec0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2bd8066c80 tx=0x7f2bc8009060).ready entity=mon.0 client_cookie=8d6cba1f84ae4837 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd5d9b700 1 -- 172.21.15.73:0/1474231694 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc80090d0 con 0x7f2bd80c3040 2023-11-28T03:04:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd5d9b700 1 -- 172.21.15.73:0/1474231694 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800f510 con 0x7f2bd80c3040 2023-11-28T03:04:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.586+0000 7f2bd5d9b700 1 -- 172.21.15.73:0/1474231694 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800f890 con 0x7f2bd80c3040 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 -- 172.21.15.73:0/1474231694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 msgr2=0x7f2bd80c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1474231694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c5ec0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 rx=0x7f2bd8066c80 tx=0x7f2bc8009060).stop 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 -- 172.21.15.73:0/1474231694 shutdown_connections 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1474231694 >> v2:172.21.15.150:3300/0 conn(0x7f2bd80c63b0 0x7f2bd8069930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1474231694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c5ec0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 -- 172.21.15.73:0/1474231694 >> 172.21.15.73:0/1474231694 conn(0x7f2bd8062700 msgr2=0x7f2bd8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 -- 172.21.15.73:0/1474231694 shutdown_connections 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.587+0000 7f2bdd055700 1 -- 172.21.15.73:0/1474231694 wait complete. 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.589+0000 7f2bdd055700 1 Processor -- start 2023-11-28T03:04:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.589+0000 7f2bdd055700 1 -- start start 2023-11-28T03:04:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.589+0000 7f2bdd055700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 0x7f2bd80bef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bd80c63b0 0x7f2bd8139ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80bfda0 con 0x7f2bd80c3040 2023-11-28T03:04:42.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80c05d0 con 0x7f2bd80c2c60 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bd80a7590 con 0x7f2bd80c63b0 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:51884/0 (socket says 172.21.15.73:51884) 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 -- 172.21.15.73:0/1719962183 learned_addr learned my addr 172.21.15.73:0/1719962183 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd6d9d700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 0x7f2bd80bef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd759e700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bd80c63b0 0x7f2bd8139ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bd80c63b0 msgr2=0x7f2bd8139ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bd80c63b0 0x7f2bd8139ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 msgr2=0x7f2bd80bef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:42.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 0x7f2bd80bef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 -- 172.21.15.73:0/1719962183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bc80092a0 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd6d9d700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 0x7f2bd80bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bd659c700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f2bc8009dd0 tx=0x7f2bc8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc800f890 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2bc800fc50 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bd80a7260 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2bc801b410 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.590+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bd80bf7e0 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bd806c860 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 11) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2bc801b840 con 0x7f2bd80c3040 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bceffd700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 0x7f2bb8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:42.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bd6d9d700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 msgr2=0x7f2bb8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:42.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 7434+0+0 (secure 0 0 0) 0x7f2bc807c1b0 con 0x7f2bd80c3040 2023-11-28T03:04:42.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.592+0000 7f2bd6d9d700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 0x7f2bb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:04:42.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.599+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2bc804cde0 con 0x7f2bd80c3040 2023-11-28T03:04:42.654 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:04:42.653+0000 7f9dc53e2680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:04:42.766 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.765+0000 7f6916946680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:04:42.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.793+0000 7f2bd6d9d700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 msgr2=0x7f2bb8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:04:42.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.793+0000 7f2bd6d9d700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 0x7f2bb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:04:42.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.812+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f2bd80a76d0 con 0x7f2bd80c3040 2023-11-28T03:04:42.839 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.838+0000 7f6916946680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:04:42.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.859+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f2bc80798e0 con 0x7f2bd80c3040 2023-11-28T03:04:42.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.859+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 msgr2=0x7f2bb8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:04:42.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:42.859+0000 7f2bceffd700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 0x7f2bb8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:42.861 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:04:42.861 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:04:42.861 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:04:42.877 INFO:tasks.ceph.mds.c.smithi073.stderr: -4> 2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:04:42.878 INFO:tasks.ceph.mds.c.smithi073.stderr: -3> 2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:04:42.878 INFO:tasks.ceph.mds.c.smithi073.stderr: -1> 2023-11-28T03:04:42.860+0000 7f0f57e06700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:04:42.897 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:04:42.916 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68940de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.916 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68940de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.916 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68940de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68940de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68940de700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68910d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68910d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68910d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68910d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:42.917 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:04:42.915+0000 7f68910d8700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:04:43.350 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:04:43.349+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:04:43.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.359+0000 7f2bceffd700 1 -- 172.21.15.73:0/1719962183 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v13) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2bc8055840 con 0x7f2bd80c3040 2023-11-28T03:04:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 msgr2=0x7f2bd80c0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 rx=0x7f2bc8009dd0 tx=0x7f2bc8017000).stop 2023-11-28T03:04:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 shutdown_connections 2023-11-28T03:04:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2bb805f270 0x7f2bb8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:43.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bd80c63b0 0x7f2bd8139ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bd80c2c60 0x7f2bd80bef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 --2- 172.21.15.73:0/1719962183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bd80c3040 0x7f2bd80c0940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 >> 172.21.15.73:0/1719962183 conn(0x7f2bd8062700 msgr2=0x7f2bd80a5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 shutdown_connections 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.361+0000 7f2bdd055700 1 -- 172.21.15.73:0/1719962183 wait complete. 2023-11-28T03:04:43.363 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:04:43.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:04:43.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.488+0000 7f903160a700 1 Processor -- start 2023-11-28T03:04:43.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.488+0000 7f903160a700 1 -- start start 2023-11-28T03:04:43.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f903160a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f902c154950 0x7f902c0ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:43.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f903160a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c0c7bc0 con 0x7f902c14e530 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f903160a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f902c155920 con 0x7f902c0c66f0 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f903160a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f902c0c6ba0 con 0x7f902c154950 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f902c14e530 0x7f902c14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53404/0 (socket says 172.21.15.73:53404) 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902bfff700 1 -- 172.21.15.73:0/2540826123 learned_addr learned my addr 172.21.15.73:0/2540826123 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f9030e09700 1 --2- 172.21.15.73:0/2540826123 >> v2:172.21.15.150:3300/0 conn(0x7f902c154950 0x7f902c0ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:43.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2760949336 0 0) 0x7f902c0c7bc0 con 0x7f902c14e530 2023-11-28T03:04:43.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f90140033f0 con 0x7f902c14e530 2023-11-28T03:04:43.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 266683877 0 0) 0x7f902c155920 con 0x7f902c0c66f0 2023-11-28T03:04:43.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f902c0c7bc0 con 0x7f902c0c66f0 2023-11-28T03:04:43.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f9030e09700 1 -- 172.21.15.73:0/2540826123 >> v1:172.21.15.150:6789/0 conn(0x7f902c0c66f0 legacy=0x7f902c158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f9030e09700 1 -- 172.21.15.73:0/2540826123 >> v1:172.21.15.73:6789/0 conn(0x7f902c14e530 legacy=0x7f902c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f9030e09700 1 -- 172.21.15.73:0/2540826123 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f902c154de0 con 0x7f902c154950 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.489+0000 7f9030e09700 1 --2- 172.21.15.73:0/2540826123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c154950 0x7f902c0ca540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9024009e60 tx=0x7f9024009780).ready entity=mon.1 client_cookie=41dd9b2df0786dcf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902400e070 con 0x7f902c154950 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f902400bab0 con 0x7f902c154950 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f902affd700 1 -- 172.21.15.73:0/2540826123 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f902400be30 con 0x7f902c154950 2023-11-28T03:04:43.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 -- 172.21.15.73:0/2540826123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c154950 msgr2=0x7f902c0ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 --2- 172.21.15.73:0/2540826123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c154950 0x7f902c0ca540 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 rx=0x7f9024009e60 tx=0x7f9024009780).stop 2023-11-28T03:04:43.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 -- 172.21.15.73:0/2540826123 shutdown_connections 2023-11-28T03:04:43.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 --2- 172.21.15.73:0/2540826123 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c154950 0x7f902c0ca540 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:43.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 -- 172.21.15.73:0/2540826123 >> 172.21.15.73:0/2540826123 conn(0x7f902c064990 msgr2=0x7f902c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:43.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 -- 172.21.15.73:0/2540826123 shutdown_connections 2023-11-28T03:04:43.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.490+0000 7f903160a700 1 -- 172.21.15.73:0/2540826123 wait complete. 2023-11-28T03:04:43.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.491+0000 7f903160a700 1 Processor -- start 2023-11-28T03:04:43.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.491+0000 7f903160a700 1 -- start start 2023-11-28T03:04:43.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:43.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c14e530 0x7f902c156b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:43.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f902c154950 0x7f902c157070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:43.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c157560 con 0x7f902c0c66f0 2023-11-28T03:04:43.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c50a0 con 0x7f902c14e530 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f903160a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f902c0c4490 con 0x7f902c154950 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f902bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f902bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57396/0 (socket says 172.21.15.73:57396) 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.492+0000 7f902bfff700 1 -- 172.21.15.73:0/1573026653 learned_addr learned my addr 172.21.15.73:0/1573026653 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902b7fe700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c14e530 0x7f902c156b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:43.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 -- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f902c154950 msgr2=0x7f902c157070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f902c154950 0x7f902c157070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:43.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 -- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c14e530 msgr2=0x7f902c156b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:43.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c14e530 0x7f902c156b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:43.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 -- 172.21.15.73:0/1573026653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9024009a20 con 0x7f902c0c66f0 2023-11-28T03:04:43.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f902bfff700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f901c00c560 tx=0x7f901c007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c0072e0 con 0x7f902c0c66f0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f901c007440 con 0x7f902c0c66f0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f902c0c4770 con 0x7f902c0c66f0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f901c0057a0 con 0x7f902c0c66f0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.493+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f902c0c4d30 con 0x7f902c0c66f0 2023-11-28T03:04:43.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.494+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f902c0ad3e0 con 0x7f902c0c66f0 2023-11-28T03:04:43.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.494+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 12) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f901c005b90 con 0x7f902c0c66f0 2023-11-28T03:04:43.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.495+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7136+0+0 (secure 0 0 0) 0x7f901c079c10 con 0x7f902c0c66f0 2023-11-28T03:04:43.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.499+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f901c04acc0 con 0x7f902c0c66f0 2023-11-28T03:04:43.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.703+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f902c062ee0 con 0x7f902c0c66f0 2023-11-28T03:04:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.861+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f901c0777c0 con 0x7f902c0c66f0 2023-11-28T03:04:43.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.861+0000 7f9013fff700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 0x7f9018067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.862+0000 7f902b7fe700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 0x7f9018067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:43.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:43.862+0000 7f902b7fe700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 0x7f9018067ff0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f902c0b09d0 tx=0x7f9020009040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.360+0000 7f9013fff700 1 -- 172.21.15.73:0/1573026653 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v14) v1 ==== 104+0+0 (secure 0 0 0) 0x7f901c06e3c0 con 0x7f902c0c66f0 2023-11-28T03:04:44.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 msgr2=0x7f9018067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 0x7f9018067ff0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f902c0b09d0 tx=0x7f9020009040).stop 2023-11-28T03:04:44.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 msgr2=0x7f902c13ca90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 rx=0x7f901c00c560 tx=0x7f901c007d20).stop 2023-11-28T03:04:44.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 shutdown_connections 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f90180302c0 0x7f9018067ff0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f902c154950 0x7f902c157070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f902c0c66f0 0x7f902c13ca90 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 --2- 172.21.15.73:0/1573026653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f902c14e530 0x7f902c156b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 >> 172.21.15.73:0/1573026653 conn(0x7f902c064990 msgr2=0x7f902c0c0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 shutdown_connections 2023-11-28T03:04:44.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.362+0000 7f903160a700 1 -- 172.21.15.73:0/1573026653 wait complete. 2023-11-28T03:04:44.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:04:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.491+0000 7fd3a602d700 1 Processor -- start 2023-11-28T03:04:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.491+0000 7fd3a602d700 1 -- start start 2023-11-28T03:04:44.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd3a602d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3a0154610 0x7fd3a01582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd3a602d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a0159210 con 0x7fd3a00ab3b0 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd3a602d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3a00c0c70 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd3a602d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3a0158970 con 0x7fd3a0154610 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3a0154610 0x7fd3a01582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3a0154610 0x7fd3a01582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57398/0 (socket says 172.21.15.73:57398) 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 -- 172.21.15.73:0/1785803122 learned_addr learned my addr 172.21.15.73:0/1785803122 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 -- 172.21.15.73:0/1785803122 >> v1:172.21.15.150:6789/0 conn(0x7fd3a00aa1c0 legacy=0x7fd3a00c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:44.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 -- 172.21.15.73:0/1785803122 >> v1:172.21.15.73:6789/0 conn(0x7fd3a00ab3b0 legacy=0x7fd3a00c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 -- 172.21.15.73:0/1785803122 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0158c50 con 0x7fd3a0154610 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.492+0000 7fd39f7fe700 1 --2- 172.21.15.73:0/1785803122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a0154610 0x7fd3a01582e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fd390009f20 tx=0x7fd390009840).ready entity=mon.0 client_cookie=3ddde6adef307b0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd39e7fc700 1 -- 172.21.15.73:0/1785803122 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000e070 con 0x7fd3a0154610 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd39e7fc700 1 -- 172.21.15.73:0/1785803122 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39000ba40 con 0x7fd3a0154610 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd39e7fc700 1 -- 172.21.15.73:0/1785803122 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000bdc0 con 0x7fd3a0154610 2023-11-28T03:04:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 -- 172.21.15.73:0/1785803122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a0154610 msgr2=0x7fd3a01582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 --2- 172.21.15.73:0/1785803122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a0154610 0x7fd3a01582e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7fd390009f20 tx=0x7fd390009840).stop 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 -- 172.21.15.73:0/1785803122 shutdown_connections 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 --2- 172.21.15.73:0/1785803122 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a0154610 0x7fd3a01582e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 -- 172.21.15.73:0/1785803122 >> 172.21.15.73:0/1785803122 conn(0x7fd3a00655a0 msgr2=0x7fd3a0069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 -- 172.21.15.73:0/1785803122 shutdown_connections 2023-11-28T03:04:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.493+0000 7fd3a602d700 1 -- 172.21.15.73:0/1785803122 wait complete. 2023-11-28T03:04:44.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.494+0000 7fd3a602d700 1 Processor -- start 2023-11-28T03:04:44.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.494+0000 7fd3a602d700 1 -- start start 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3a0154610 0x7fd3a0153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 0x7fd3a0154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a01664b0 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00af5d0 con 0x7fd3a01549f0 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd3a602d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3a00aed10 con 0x7fd3a0154610 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd39f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd39f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57410/0 (socket says 172.21.15.73:57410) 2023-11-28T03:04:44.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.495+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3859525734 learned_addr learned my addr 172.21.15.73:0/3859525734 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39ffff700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 0x7fd3a0154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3a0154610 msgr2=0x7fd3a0153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39effd700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3a0154610 0x7fd3a0153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3a0154610 0x7fd3a0153780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 msgr2=0x7fd3a0154e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 0x7fd3a0154e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 -- 172.21.15.73:0/3859525734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd390009ae0 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39ffff700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 0x7fd3a0154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd39f7fe700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd390007280 tx=0x7fd390009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39000bdc0 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd39001c400 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3a0155360 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd39001c7c0 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.496+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3a00af260 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.497+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3a006c860 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.497+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd39001cb30 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.498+0000 7fd3877fe700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 0x7fd38c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.498+0000 7fd39effd700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 0x7fd38c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.498+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 7136+0+0 (secure 0 0 0) 0x7fd39007bd60 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.498+0000 7fd39effd700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 0x7fd38c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fd3a00c24d0 tx=0x7fd394009040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:44.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.502+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd39004ce10 con 0x7fd3a00aa1c0 2023-11-28T03:04:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:44.663+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd3a006c860 con 0x7fd3a00aa1c0 2023-11-28T03:04:45.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.366+0000 7fd3877fe700 1 -- 172.21.15.73:0/3859525734 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v55) v1 ==== 181+0+0 (secure 0 0 0) 0x7fd390050430 con 0x7fd3a00aa1c0 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 msgr2=0x7fd38c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 0x7fd38c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fd3a00c24d0 tx=0x7fd394009040).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 msgr2=0x7fd3a0165fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 rx=0x7fd390007280 tx=0x7fd390009840).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 shutdown_connections 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd38c05f030 0x7fd38c0614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3a01549f0 0x7fd3a0154e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3a00aa1c0 0x7fd3a0165fc0 secure :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 rx=0x7fd390007280 tx=0x7fd390009840).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 --2- 172.21.15.73:0/3859525734 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3a0154610 0x7fd3a0153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.369+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 >> 172.21.15.73:0/3859525734 conn(0x7fd3a00655a0 msgr2=0x7fd3a014e160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:45.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.370+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 shutdown_connections 2023-11-28T03:04:45.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.370+0000 7fd3a602d700 1 -- 172.21.15.73:0/3859525734 wait complete. 2023-11-28T03:04:45.372 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:04:45.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:04:45.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.502+0000 7ff20f59e700 1 Processor -- start 2023-11-28T03:04:45.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.502+0000 7ff20f59e700 1 -- start start 2023-11-28T03:04:45.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff2100c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:45.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20f59e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff210158d50 con 0x7ff2101546b0 2023-11-28T03:04:45.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20f59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff210158140 con 0x7ff2101542d0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff210158420 con 0x7ff2100aa680 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20ed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff2101546b0 0x7ff2100c64b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53416/0 (socket says 172.21.15.73:53416) 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20ed9d700 1 -- 172.21.15.73:0/1219411402 learned_addr learned my addr 172.21.15.73:0/1219411402 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20dd9b700 1 --2- 172.21.15.73:0/1219411402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff2100c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1997232015 0 0) 0x7ff210158d50 con 0x7ff2101546b0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1fc0033f0 con 0x7ff2101546b0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149238554 0 0) 0x7ff210158140 con 0x7ff2101542d0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff210158d50 con 0x7ff2101542d0 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20dd9b700 1 -- 172.21.15.73:0/1219411402 >> v1:172.21.15.150:6789/0 conn(0x7ff2101542d0 legacy=0x7ff2100a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20dd9b700 1 -- 172.21.15.73:0/1219411402 >> v1:172.21.15.73:6789/0 conn(0x7ff2101546b0 legacy=0x7ff2100c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.503+0000 7ff20dd9b700 1 -- 172.21.15.73:0/1219411402 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff210158700 con 0x7ff2100aa680 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20dd9b700 1 --2- 172.21.15.73:0/1219411402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff2100c5fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff1f8009790 tx=0x7ff1f8009060).ready entity=mon.2 client_cookie=888ead3c7be6609b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800f510 con 0x7ff2100aa680 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1f800f670 con 0x7ff2100aa680 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20d59a700 1 -- 172.21.15.73:0/1219411402 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1f800f9f0 con 0x7ff2100aa680 2023-11-28T03:04:45.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 -- 172.21.15.73:0/1219411402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 msgr2=0x7ff2100c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1219411402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff2100c5fc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff1f8009790 tx=0x7ff1f8009060).stop 2023-11-28T03:04:45.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 -- 172.21.15.73:0/1219411402 shutdown_connections 2023-11-28T03:04:45.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1219411402 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff2100c5fc0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 -- 172.21.15.73:0/1219411402 >> 172.21.15.73:0/1219411402 conn(0x7ff2100cb010 msgr2=0x7ff2100cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:45.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 -- 172.21.15.73:0/1219411402 shutdown_connections 2023-11-28T03:04:45.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.504+0000 7ff20f59e700 1 -- 172.21.15.73:0/1219411402 wait complete. 2023-11-28T03:04:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 Processor -- start 2023-11-28T03:04:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 -- start start 2023-11-28T03:04:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff210147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2101542d0 0x7ff2100c2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:45.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100c2700 con 0x7ff2101546b0 2023-11-28T03:04:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2101482e0 con 0x7ff2101542d0 2023-11-28T03:04:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2100aec80 con 0x7ff2100aa680 2023-11-28T03:04:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:45.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57416/0 (socket says 172.21.15.73:57416) 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20ed9d700 1 -- 172.21.15.73:0/1590112784 learned_addr learned my addr 172.21.15.73:0/1590112784 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20dd9b700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2101542d0 0x7ff2100c2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.506+0000 7ff20e59c700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff210147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 -- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 msgr2=0x7ff210147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff210147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 -- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2101542d0 msgr2=0x7ff2100c2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2101542d0 0x7ff2100c2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 -- 172.21.15.73:0/1590112784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f80092a0 con 0x7ff2101546b0 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20ed9d700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7ff20800c3f0 tx=0x7ff208007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080070d0 con 0x7ff2101546b0 2023-11-28T03:04:45.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff208007230 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2100aef60 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2080075d0 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.507+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2100c2390 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.508+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff21006c7c0 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.508+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 13) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff2080059b0 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.508+0000 7ff2067fc700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 0x7ff1f00614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.509+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(55..55 src has 1..55) v4 ==== 6698+0+0 (secure 0 0 0) 0x7ff2080797c0 con 0x7ff2101546b0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.509+0000 7ff20e59c700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 0x7ff1f00614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:45.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.509+0000 7ff20e59c700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 0x7ff1f00614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff200000f80 tx=0x7ff200007f30).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:45.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.512+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff20804aa20 con 0x7ff2101546b0 2023-11-28T03:04:45.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.674+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff2100c0120 con 0x7ff2101546b0 2023-11-28T03:04:45.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:45.876+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff208005ce0 con 0x7ff2101546b0 2023-11-28T03:04:46.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.206+0000 7ff2067fc700 1 -- 172.21.15.73:0/1590112784 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v56) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff20801d430 con 0x7ff2101546b0 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.209+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 msgr2=0x7ff1f00614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.209+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 0x7ff1f00614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7ff200000f80 tx=0x7ff200007f30).stop 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.209+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 msgr2=0x7ff210063370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.209+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 rx=0x7ff20800c3f0 tx=0x7ff208007c30).stop 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 shutdown_connections 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1f005f030 0x7ff1f00614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2101546b0 0x7ff210063370 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2100aa680 0x7ff210147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 --2- 172.21.15.73:0/1590112784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2101542d0 0x7ff2100c2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 >> 172.21.15.73:0/1590112784 conn(0x7ff2100cb010 msgr2=0x7ff2100acc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.210+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 shutdown_connections 2023-11-28T03:04:46.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.211+0000 7ff20f59e700 1 -- 172.21.15.73:0/1590112784 wait complete. 2023-11-28T03:04:46.214 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:04:46.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:04:46.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.339+0000 7f533d5d0700 1 Processor -- start 2023-11-28T03:04:46.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.339+0000 7f533d5d0700 1 -- start start 2023-11-28T03:04:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533d5d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5338066a30 0x7f5338066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533d5d0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53381586a0 con 0x7f53380c6eb0 2023-11-28T03:04:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533d5d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5338157de0 con 0x7f53380c6a30 2023-11-28T03:04:46.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533d5d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5338158050 con 0x7f5338066a30 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533cdcf700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f53380c6eb0 0x7f5338063810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53418/0 (socket says 172.21.15.73:53418) 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f533cdcf700 1 -- 172.21.15.73:0/2806849454 learned_addr learned my addr 172.21.15.73:0/2806849454 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5337fff700 1 --2- 172.21.15.73:0/2806849454 >> v2:172.21.15.150:3300/0 conn(0x7f5338066a30 0x7f5338066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 999111970 0 0) 0x7f53381586a0 con 0x7f53380c6eb0 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53200033f0 con 0x7f53380c6eb0 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 842802804 0 0) 0x7f53200033f0 con 0x7f53380c6eb0 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5337fff700 1 -- 172.21.15.73:0/2806849454 >> v1:172.21.15.150:6789/0 conn(0x7f53380c6a30 legacy=0x7f533813d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5337fff700 1 -- 172.21.15.73:0/2806849454 >> v1:172.21.15.73:6789/0 conn(0x7f53380c6eb0 legacy=0x7f5338063810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.340+0000 7f5337fff700 1 -- 172.21.15.73:0/2806849454 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5338158330 con 0x7f5338066a30 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f5337fff700 1 --2- 172.21.15.73:0/2806849454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5338066a30 0x7f5338066e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5328009760 tx=0x7f5328009060).ready entity=mon.1 client_cookie=8d3e5d8bbde7c93f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53300017f0 con 0x7f5338066a30 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5328009130 con 0x7f5338066a30 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f5336ffd700 1 -- 172.21.15.73:0/2806849454 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5328012620 con 0x7f5338066a30 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 -- 172.21.15.73:0/2806849454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5338066a30 msgr2=0x7f5338066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 --2- 172.21.15.73:0/2806849454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5338066a30 0x7f5338066e10 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7f5328009760 tx=0x7f5328009060).stop 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 -- 172.21.15.73:0/2806849454 shutdown_connections 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 --2- 172.21.15.73:0/2806849454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5338066a30 0x7f5338066e10 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 -- 172.21.15.73:0/2806849454 >> 172.21.15.73:0/2806849454 conn(0x7f53380c8ed0 msgr2=0x7f53380c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 -- 172.21.15.73:0/2806849454 shutdown_connections 2023-11-28T03:04:46.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.341+0000 7f533d5d0700 1 -- 172.21.15.73:0/2806849454 wait complete. 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.342+0000 7f533d5d0700 1 Processor -- start 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 -- start start 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5338066a30 0x7f53381533b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53380c6eb0 0x7f5338156ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5338157500 con 0x7f53380c6a30 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53380ac670 con 0x7f53380c6eb0 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f533d5d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5338157220 con 0x7f5338066a30 2023-11-28T03:04:46.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f53377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f53377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57418/0 (socket says 172.21.15.73:57418) 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.343+0000 7f53377fe700 1 -- 172.21.15.73:0/3669068728 learned_addr learned my addr 172.21.15.73:0/3669068728 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f5337fff700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5338066a30 0x7f53381533b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f533cdcf700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53380c6eb0 0x7f5338156ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 -- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5338066a30 msgr2=0x7f53381533b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5338066a30 0x7f53381533b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 -- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53380c6eb0 msgr2=0x7f5338156ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53380c6eb0 0x7f5338156ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 -- 172.21.15.73:0/3669068728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53280092a0 con 0x7f53380c6a30 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f53377fe700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f532c00c610 tx=0x7f532c007d80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00f470 con 0x7f53380c6a30 2023-11-28T03:04:46.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f532c00f5d0 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53380ac020 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f532c00f950 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.344+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5338153de0 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.345+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53380abe80 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.345+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f532c00fd40 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.346+0000 7f531ffff700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 0x7f5324061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.346+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f532c079920 con 0x7f53380c6a30 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.346+0000 7f5337fff700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 0x7f5324061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.346+0000 7f5337fff700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 0x7f5324061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f533806ac20 tx=0x7f5328015000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.350+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f532c047af0 con 0x7f53380c6a30 2023-11-28T03:04:46.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.517+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f533806c7c0 con 0x7f53380c6a30 2023-11-28T03:04:46.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.521+0000 7f531ffff700 1 -- 172.21.15.73:0/3669068728 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v4) v1 ==== 120+0+0 (secure 0 0 0) 0x7f53380abe80 con 0x7f53380c6a30 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 msgr2=0x7f5324061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 0x7f5324061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f533806ac20 tx=0x7f5328015000).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 msgr2=0x7f53381566b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 rx=0x7f532c00c610 tx=0x7f532c007d80).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 shutdown_connections 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532405f150 0x7f5324061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53380c6eb0 0x7f5338156ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5338066a30 0x7f53381533b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 --2- 172.21.15.73:0/3669068728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53380c6a30 0x7f53381566b0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 >> 172.21.15.73:0/3669068728 conn(0x7f53380c8ed0 msgr2=0x7f53380bef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 shutdown_connections 2023-11-28T03:04:46.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.523+0000 7f533d5d0700 1 -- 172.21.15.73:0/3669068728 wait complete. 2023-11-28T03:04:46.537 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:04:46.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.650+0000 7f60e3177700 1 Processor -- start 2023-11-28T03:04:46.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.650+0000 7f60e3177700 1 -- start start 2023-11-28T03:04:46.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e3177700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60dc098d10 0x7f60dc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e3177700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f60dc0c6190 0x7f60dc063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e3177700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60dc0c72d0 con 0x7f60dc064350 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e3177700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc154bc0 con 0x7f60dc098d10 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e3177700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60dc0c66c0 con 0x7f60dc0c6190 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60dc098d10 0x7f60dc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60dc098d10 0x7f60dc0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57434/0 (socket says 172.21.15.73:57434) 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 -- 172.21.15.73:0/627483369 learned_addr learned my addr 172.21.15.73:0/627483369 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 -- 172.21.15.73:0/627483369 >> v2:172.21.15.150:3300/0 conn(0x7f60dc0c6190 msgr2=0x7f60dc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 --2- 172.21.15.73:0/627483369 >> v2:172.21.15.150:3300/0 conn(0x7f60dc0c6190 0x7f60dc063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:46.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 -- 172.21.15.73:0/627483369 >> v1:172.21.15.150:6789/0 conn(0x7f60dc064350 legacy=0x7f60dc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.651+0000 7f60e0f13700 1 -- 172.21.15.73:0/627483369 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60dc0c69a0 con 0x7f60dc098d10 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e0f13700 1 --2- 172.21.15.73:0/627483369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0c7760 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f60dc0c5b40 tx=0x7f60d8009840).ready entity=mon.0 client_cookie=9c470c1c07be3ff0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60d37fe700 1 -- 172.21.15.73:0/627483369 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800e070 con 0x7f60dc098d10 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60d37fe700 1 -- 172.21.15.73:0/627483369 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60d800ba10 con 0x7f60dc098d10 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60d37fe700 1 -- 172.21.15.73:0/627483369 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60d800bd90 con 0x7f60dc098d10 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 -- 172.21.15.73:0/627483369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 msgr2=0x7f60dc0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 --2- 172.21.15.73:0/627483369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0c7760 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 rx=0x7f60dc0c5b40 tx=0x7f60d8009840).stop 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 -- 172.21.15.73:0/627483369 shutdown_connections 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 --2- 172.21.15.73:0/627483369 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0c7760 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 --2- 172.21.15.73:0/627483369 >> v2:172.21.15.150:3300/0 conn(0x7f60dc0c6190 0x7f60dc063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 -- 172.21.15.73:0/627483369 >> 172.21.15.73:0/627483369 conn(0x7f60dc0cb310 msgr2=0x7f60dc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 -- 172.21.15.73:0/627483369 shutdown_connections 2023-11-28T03:04:46.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.652+0000 7f60e3177700 1 -- 172.21.15.73:0/627483369 wait complete. 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.653+0000 7f60e3177700 1 Processor -- start 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 -- start start 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60dc064350 0x7f60dc1570f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60dc0c6190 0x7f60dc0abc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc156c60 con 0x7f60dc098d10 2023-11-28T03:04:46.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc156330 con 0x7f60dc064350 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60e3177700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60dc156610 con 0x7f60dc0c6190 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57440/0 (socket says 172.21.15.73:57440) 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.654+0000 7f60d3fff700 1 -- 172.21.15.73:0/3637964611 learned_addr learned my addr 172.21.15.73:0/3637964611 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60e0f13700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60dc064350 0x7f60dc1570f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60e1714700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60dc0c6190 0x7f60dc0abc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 -- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60dc0c6190 msgr2=0x7f60dc0abc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60dc0c6190 0x7f60dc0abc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 -- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60dc064350 msgr2=0x7f60dc1570f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60dc064350 0x7f60dc1570f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 -- 172.21.15.73:0/3637964611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60d8009ae0 con 0x7f60dc098d10 2023-11-28T03:04:46.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d3fff700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f60cc00b460 tx=0x7f60cc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc00be10 con 0x7f60dc098d10 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60cc002320 con 0x7f60dc098d10 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60dc1568f0 con 0x7f60dc098d10 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60cc0026e0 con 0x7f60dc098d10 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.655+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60dc1575e0 con 0x7f60dc098d10 2023-11-28T03:04:46.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.656+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60dc06c860 con 0x7f60dc098d10 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.656+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f60cc002a90 con 0x7f60dc098d10 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.657+0000 7f60d0ff9700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 0x7f60c80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.657+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f60cc079f40 con 0x7f60dc098d10 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.657+0000 7f60e0f13700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 0x7f60c80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.657+0000 7f60e0f13700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 0x7f60c80615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f60d8009880 tx=0x7f60dc0a8040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.661+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f60cc04b190 con 0x7f60dc098d10 2023-11-28T03:04:46.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.823+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f60dc0c6c80 con 0x7f60dc098d10 2023-11-28T03:04:46.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.827+0000 7f60d0ff9700 1 -- 172.21.15.73:0/3637964611 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v5) v1 ==== 104+0+0 (secure 0 0 0) 0x7f60cc04e7b0 con 0x7f60dc098d10 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 msgr2=0x7f60c80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 0x7f60c80615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f60d8009880 tx=0x7f60dc0a8040).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 msgr2=0x7f60dc0ab740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 rx=0x7f60cc00b460 tx=0x7f60cc009060).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 shutdown_connections 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f60c805f120 0x7f60c80615e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60dc0c6190 0x7f60dc0abc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60dc064350 0x7f60dc1570f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 --2- 172.21.15.73:0/3637964611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60dc098d10 0x7f60dc0ab740 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 >> 172.21.15.73:0/3637964611 conn(0x7f60dc0cb310 msgr2=0x7f60dc153d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 shutdown_connections 2023-11-28T03:04:46.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.829+0000 7f60e3177700 1 -- 172.21.15.73:0/3637964611 wait complete. 2023-11-28T03:04:46.842 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-11-28T03:04:46.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 Processor -- start 2023-11-28T03:04:46.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 -- start start 2023-11-28T03:04:46.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b5015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b500a9a90 con 0x7f8b500caab0 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b50154ce0 con 0x7f8b500cae90 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.960+0000 7f8b55300700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500aab80 con 0x7f8b500ca180 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4effd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8b500caab0 0x7f8b500aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53430/0 (socket says 172.21.15.73:53430) 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4effd700 1 -- 172.21.15.73:0/1857833084 learned_addr learned my addr 172.21.15.73:0/1857833084 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4f7fe700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b5015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511259606 0 0) 0x7f8b500a9a90 con 0x7f8b500caab0 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b380033f0 con 0x7f8b500caab0 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 795848192 0 0) 0x7f8b50154ce0 con 0x7f8b500cae90 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b500a9a90 con 0x7f8b500cae90 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 464206921 0 0) 0x7f8b380033f0 con 0x7f8b500caab0 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 msgr2=0x7f8b5015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:46.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b5015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 >> v1:172.21.15.150:6789/0 conn(0x7f8b500cae90 legacy=0x7f8b500c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b500a8e80 con 0x7f8b500caab0 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8b40001970 con 0x7f8b500caab0 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 >> v1:172.21.15.73:6789/0 conn(0x7f8b500caab0 legacy=0x7f8b500aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b38003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b38004690 0x7f8b38004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b38005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b500aab80 con 0x7f8b38003aa0 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800b940 con 0x7f8b38004690 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b3800bb00 con 0x7f8b500ca180 2023-11-28T03:04:46.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4effd700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b38003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4e7fc700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b38004690 0x7f8b38004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.961+0000 7f8b4f7fe700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b38005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 -- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 msgr2=0x7f8b38005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b38005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 -- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b38004690 msgr2=0x7f8b38004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b38004690 0x7f8b38004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 -- 172.21.15.73:0/1857833084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b380033f0 con 0x7f8b38003aa0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4f7fe700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b500ca180 0x7f8b38005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4effd700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b38003ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f8b4000c5f0 tx=0x7f8b40007aa0).ready entity=mon.0 client_cookie=27e85e22715789b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40007cd0 con 0x7f8b38003aa0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4000cdf0 con 0x7f8b38003aa0 2023-11-28T03:04:46.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4dffb700 1 -- 172.21.15.73:0/1857833084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b40005c20 con 0x7f8b38003aa0 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 -- 172.21.15.73:0/1857833084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 msgr2=0x7f8b38003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 --2- 172.21.15.73:0/1857833084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b38003ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 rx=0x7f8b4000c5f0 tx=0x7f8b40007aa0).stop 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b4f7fe700 1 -- 172.21.15.73:0/1857833084 reap_dead start 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 -- 172.21.15.73:0/1857833084 shutdown_connections 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 -- 172.21.15.73:0/1857833084 >> 172.21.15.73:0/1857833084 conn(0x7f8b50094920 msgr2=0x7f8b5006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 -- 172.21.15.73:0/1857833084 shutdown_connections 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.962+0000 7f8b55300700 1 -- 172.21.15.73:0/1857833084 wait complete. 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 Processor -- start 2023-11-28T03:04:46.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 -- start start 2023-11-28T03:04:46.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b50153670 0x7f8b50153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b50153f90 0x7f8b500c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50155e60 con 0x7f8b38003aa0 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b501543f0 con 0x7f8b50153f90 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.964+0000 7f8b55300700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b50155250 con 0x7f8b50153670 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57460/0 (socket says 172.21.15.73:57460) 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 -- 172.21.15.73:0/3144687026 learned_addr learned my addr 172.21.15.73:0/3144687026 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4f7fe700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b50153f90 0x7f8b500c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 -- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b50153670 msgr2=0x7f8b50153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b50153670 0x7f8b50153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 -- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b50153f90 msgr2=0x7f8b500c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b50153f90 0x7f8b500c3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:46.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 -- 172.21.15.73:0/3144687026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b50155530 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b4effd700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f8b40001270 tx=0x7f8b40015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400072e0 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b40007440 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b50155810 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b400077c0 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.965+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b500c4430 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.966+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5006c860 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.966+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8b40013660 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.967+0000 7f8b36ffd700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 0x7f8b3c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.967+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f8b4007b360 con 0x7f8b38003aa0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.967+0000 7f8b4e7fc700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 0x7f8b3c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:46.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.967+0000 7f8b4e7fc700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 0x7f8b3c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8b440010e0 tx=0x7f8b44007bb0).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:46.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:46.971+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b4004c5b0 con 0x7f8b38003aa0 2023-11-28T03:04:47.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.198+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7f8b500a9160 con 0x7f8b38003aa0 2023-11-28T03:04:47.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.367+0000 7f8b36ffd700 1 -- 172.21.15.73:0/3144687026 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v425) v1 ==== 158+0+0 (secure 0 0 0) 0x7f8b4004fbd0 con 0x7f8b38003aa0 2023-11-28T03:04:47.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 msgr2=0x7f8b3c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 0x7f8b3c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8b440010e0 tx=0x7f8b44007bb0).stop 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 msgr2=0x7f8b50153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 rx=0x7f8b40001270 tx=0x7f8b40015000).stop 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 shutdown_connections 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8b3c05f150 0x7f8b3c061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b50153f90 0x7f8b500c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b38003aa0 0x7f8b50153180 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 --2- 172.21.15.73:0/3144687026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b50153670 0x7f8b50153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:47.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 >> 172.21.15.73:0/3144687026 conn(0x7f8b50094920 msgr2=0x7f8b500ab6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:47.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 shutdown_connections 2023-11-28T03:04:47.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.369+0000 7f8b55300700 1 -- 172.21.15.73:0/3144687026 wait complete. 2023-11-28T03:04:47.382 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-11-28T03:04:47.382 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-11-28T03:04:47.382 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:04:47.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-11-28T03:04:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.496+0000 7f2d03409700 1 Processor -- start 2023-11-28T03:04:47.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.496+0000 7f2d03409700 1 -- start start 2023-11-28T03:04:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d03409700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2cfc1544d0 0x7f2cfc155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d03409700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2cfc066f40 0x7f2cfc095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d03409700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cfc0683d0 con 0x7f2cfc0673c0 2023-11-28T03:04:47.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d03409700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc067850 con 0x7f2cfc1544d0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d03409700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cfc067b30 con 0x7f2cfc066f40 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2cfc1544d0 0x7f2cfc155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2cfc1544d0 0x7f2cfc155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57472/0 (socket says 172.21.15.73:57472) 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 -- 172.21.15.73:0/4154766548 learned_addr learned my addr 172.21.15.73:0/4154766548 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 -- 172.21.15.73:0/4154766548 >> v2:172.21.15.150:3300/0 conn(0x7f2cfc066f40 msgr2=0x7f2cfc095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 --2- 172.21.15.73:0/4154766548 >> v2:172.21.15.150:3300/0 conn(0x7f2cfc066f40 0x7f2cfc095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 -- 172.21.15.73:0/4154766548 >> v1:172.21.15.150:6789/0 conn(0x7f2cfc0673c0 legacy=0x7f2cfc063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 -- 172.21.15.73:0/4154766548 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cfc067e10 con 0x7f2cfc1544d0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2d011a5700 1 --2- 172.21.15.73:0/4154766548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc155750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f2cfc06a7d0 tx=0x7f2cf4009840).ready entity=mon.0 client_cookie=af9a3196a9f51a51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2cf3fff700 1 -- 172.21.15.73:0/4154766548 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400e070 con 0x7f2cfc1544d0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2cf3fff700 1 -- 172.21.15.73:0/4154766548 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf400ba90 con 0x7f2cfc1544d0 2023-11-28T03:04:47.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.497+0000 7f2cf3fff700 1 -- 172.21.15.73:0/4154766548 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf400be10 con 0x7f2cfc1544d0 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 -- 172.21.15.73:0/4154766548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc1544d0 msgr2=0x7f2cfc155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 --2- 172.21.15.73:0/4154766548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc155750 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 rx=0x7f2cfc06a7d0 tx=0x7f2cf4009840).stop 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 -- 172.21.15.73:0/4154766548 shutdown_connections 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 --2- 172.21.15.73:0/4154766548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc155750 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 --2- 172.21.15.73:0/4154766548 >> v2:172.21.15.150:3300/0 conn(0x7f2cfc066f40 0x7f2cfc095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 -- 172.21.15.73:0/4154766548 >> 172.21.15.73:0/4154766548 conn(0x7f2cfc0631d0 msgr2=0x7f2cfc0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 -- 172.21.15.73:0/4154766548 shutdown_connections 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.498+0000 7f2d03409700 1 -- 172.21.15.73:0/4154766548 wait complete. 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.499+0000 7f2d03409700 1 Processor -- start 2023-11-28T03:04:47.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.499+0000 7f2d03409700 1 -- start start 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cfc066f40 0x7f2cfc0ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc0ac7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0aeee0 con 0x7f2cfc0acce0 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0ad160 con 0x7f2cfc1544d0 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d03409700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cfc0ac5b0 con 0x7f2cfc066f40 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57488/0 (socket says 172.21.15.73:57488) 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 -- 172.21.15.73:0/4036990132 learned_addr learned my addr 172.21.15.73:0/4036990132 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d009a4700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc0ac7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:47.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d011a5700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cfc066f40 0x7f2cfc0ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 -- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cfc066f40 msgr2=0x7f2cfc0ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cfc066f40 0x7f2cfc0ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 -- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 msgr2=0x7f2cfc0ac7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc0ac7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.500+0000 7f2d019a6700 1 -- 172.21.15.73:0/4036990132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cf4009ae0 con 0x7f2cfc0acce0 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2d009a4700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc0ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2d019a6700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2cf800b490 tx=0x7f2cf8006ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8012410 con 0x7f2cfc0acce0 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2cf80125b0 con 0x7f2cfc0acce0 2023-11-28T03:04:47.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cfc0ae2d0 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2cf8012970 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.501+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cfc0ae890 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.502+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cfc0c5570 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.502+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2cf8012d20 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.502+0000 7f2cf17fa700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 0x7f2ce0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.502+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7f2cf8079ef0 con 0x7f2cfc0acce0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.503+0000 7f2d011a5700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 0x7f2ce0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.503+0000 7f2d011a5700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 0x7f2ce0061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2cf4009fd0 tx=0x7f2cf4009840).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:47.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.506+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2cf804b140 con 0x7f2cfc0acce0 2023-11-28T03:04:47.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:47.734+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f2cfc06c860 con 0x7f2cfc0acce0 2023-11-28T03:04:48.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.368+0000 7f2cf17fa700 1 -- 172.21.15.73:0/4036990132 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v426) v1 ==== 157+0+0 (secure 0 0 0) 0x7f2cf804e760 con 0x7f2cfc0acce0 2023-11-28T03:04:48.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 msgr2=0x7f2ce0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 0x7f2ce0061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2cf4009fd0 tx=0x7f2cf4009840).stop 2023-11-28T03:04:48.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 msgr2=0x7f2cfc164fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 rx=0x7f2cf800b490 tx=0x7f2cf8006ef0).stop 2023-11-28T03:04:48.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 shutdown_connections 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ce005f150 0x7f2ce0061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cfc0acce0 0x7f2cfc164fc0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cfc066f40 0x7f2cfc0ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 --2- 172.21.15.73:0/4036990132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cfc1544d0 0x7f2cfc0ac7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 >> 172.21.15.73:0/4036990132 conn(0x7f2cfc0631d0 msgr2=0x7f2cfc0c4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 shutdown_connections 2023-11-28T03:04:48.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.370+0000 7f2d03409700 1 -- 172.21.15.73:0/4036990132 wait complete. 2023-11-28T03:04:48.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:04:48.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.498+0000 7fa85bcbb700 1 Processor -- start 2023-11-28T03:04:48.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.498+0000 7fa85bcbb700 1 -- start start 2023-11-28T03:04:48.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa85bcbb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa8540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa85bcbb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8540c72d0 con 0x7fa8540c6190 2023-11-28T03:04:48.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa85bcbb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa854064570 con 0x7fa854098d10 2023-11-28T03:04:48.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa85bcbb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa854154bc0 con 0x7fa854158a30 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa859256700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa8540c6190 0x7fa854158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:53434/0 (socket says 172.21.15.73:53434) 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa859256700 1 -- 172.21.15.73:0/1580196886 learned_addr learned my addr 172.21.15.73:0/1580196886 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa85a258700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa8540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.499+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3479103671 0 0) 0x7fa8540c72d0 con 0x7fa8540c6190 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8400033f0 con 0x7fa8540c6190 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3668195338 0 0) 0x7fa8400033f0 con 0x7fa8540c6190 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 msgr2=0x7fa8540c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:48.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa8540c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 >> v1:172.21.15.150:6789/0 conn(0x7fa854098d10 legacy=0x7fa854063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540c66c0 con 0x7fa8540c6190 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fa844001470 con 0x7fa8540c6190 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 >> v1:172.21.15.73:6789/0 conn(0x7fa8540c6190 legacy=0x7fa854158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa840003910 0x7fa840003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa840004690 0x7fa840004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa840005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa854154bc0 con 0x7fa840003910 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000b8e0 con 0x7fa840004690 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa84000baa0 con 0x7fa854158a30 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa840003910 0x7fa840003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859256700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa840004690 0x7fa840004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa85a258700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa840005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 -- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 msgr2=0x7fa840005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa840005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 -- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa840004690 msgr2=0x7fa840004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa840004690 0x7fa840004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 -- 172.21.15.73:0/1580196886 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8400033f0 con 0x7fa840003910 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859256700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa840004690 0x7fa840004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa85a258700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa854158a30 0x7fa840005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa859a57700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa840003910 0x7fa840003cf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa8500017e0 tx=0x7fa850007b90).ready entity=mon.0 client_cookie=ad0db92d8bbc8e5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa850007ea0 con 0x7fa840003910 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa850012400 con 0x7fa840003910 2023-11-28T03:04:48.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.500+0000 7fa858a55700 1 -- 172.21.15.73:0/1580196886 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8500127c0 con 0x7fa840003910 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 -- 172.21.15.73:0/1580196886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa840003910 msgr2=0x7fa840003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/1580196886 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa840003910 0x7fa840003cf0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 rx=0x7fa8500017e0 tx=0x7fa850007b90).stop 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85a258700 1 -- 172.21.15.73:0/1580196886 reap_dead start 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 -- 172.21.15.73:0/1580196886 shutdown_connections 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 -- 172.21.15.73:0/1580196886 >> 172.21.15.73:0/1580196886 conn(0x7fa8540cb310 msgr2=0x7fa8540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 -- 172.21.15.73:0/1580196886 shutdown_connections 2023-11-28T03:04:48.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.501+0000 7fa85bcbb700 1 -- 172.21.15.73:0/1580196886 wait complete. 2023-11-28T03:04:48.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.502+0000 7fa85bcbb700 1 Processor -- start 2023-11-28T03:04:48.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.502+0000 7fa85bcbb700 1 -- start start 2023-11-28T03:04:48.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa840003910 0x7fa8540c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 0x7fa8540c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540c8d60 con 0x7fa854153420 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540c8170 con 0x7fa8540c3db0 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85bcbb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8540c4720 con 0x7fa840003910 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57512/0 (socket says 172.21.15.73:57512) 2023-11-28T03:04:48.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 -- 172.21.15.73:0/809967045 learned_addr learned my addr 172.21.15.73:0/809967045 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85a258700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 0x7fa8540c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859a57700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa840003910 0x7fa8540c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 -- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa840003910 msgr2=0x7fa8540c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa840003910 0x7fa8540c9dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 -- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 msgr2=0x7fa8540c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 0x7fa8540c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 -- 172.21.15.73:0/809967045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8540c83c0 con 0x7fa854153420 2023-11-28T03:04:48.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa85a258700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 0x7fa8540c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.503+0000 7fa859256700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa84400e830 tx=0x7fa844001120).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.504+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844010520 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.504+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa844010680 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.504+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa844010a20 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.504+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8540c86a0 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.504+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8540c9180 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.505+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa85406c860 con 0x7fa854153420 2023-11-28T03:04:48.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.505+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa844014430 con 0x7fa854153420 2023-11-28T03:04:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.505+0000 7fa849ffb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 0x7fa838061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.505+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fa844079c60 con 0x7fa854153420 2023-11-28T03:04:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.505+0000 7fa859a57700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 0x7fa838061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.506+0000 7fa859a57700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 0x7fa838061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa8500015c0 tx=0x7fa850017000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.509+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa844049af0 con 0x7fa854153420 2023-11-28T03:04:48.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.674+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fa8540c69a0 con 0x7fa854153420 2023-11-28T03:04:48.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.678+0000 7fa849ffb700 1 -- 172.21.15.73:0/809967045 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v6)=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7fa84404ae50 con 0x7fa854153420 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.680+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 msgr2=0x7fa838061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.680+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 0x7fa838061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fa8500015c0 tx=0x7fa850017000).stop 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.680+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 msgr2=0x7fa854146e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.680+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 rx=0x7fa84400e830 tx=0x7fa844001120).stop 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 shutdown_connections 2023-11-28T03:04:48.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa83805f150 0x7fa838061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8540c3db0 0x7fa8540c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa840003910 0x7fa8540c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 --2- 172.21.15.73:0/809967045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa854153420 0x7fa854146e60 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 >> 172.21.15.73:0/809967045 conn(0x7fa8540cb310 msgr2=0x7fa854063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 shutdown_connections 2023-11-28T03:04:48.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.681+0000 7fa85bcbb700 1 -- 172.21.15.73:0/809967045 wait complete. 2023-11-28T03:04:48.693 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:04:48.694 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:04:48.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:04:48.768 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:04:48.769 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:04:48.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:04:48.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.883+0000 7fe2704b3700 1 Processor -- start 2023-11-28T03:04:48.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.883+0000 7fe2704b3700 1 -- start start 2023-11-28T03:04:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe2704b3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe268067cb0 0x7fe2680c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe2704b3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe2680c2d50 0x7fe2680c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe2704b3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe268154d20 con 0x7fe268068090 2023-11-28T03:04:48.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe2704b3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe268151670 con 0x7fe268067cb0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe2704b3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2680c48a0 con 0x7fe2680c2d50 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe268067cb0 0x7fe2680c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe268067cb0 0x7fe2680c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57518/0 (socket says 172.21.15.73:57518) 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 -- 172.21.15.73:0/1778596797 learned_addr learned my addr 172.21.15.73:0/1778596797 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26ea50700 1 --2- 172.21.15.73:0/1778596797 >> v2:172.21.15.150:3300/0 conn(0x7fe2680c2d50 0x7fe2680c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 -- 172.21.15.73:0/1778596797 >> v2:172.21.15.150:3300/0 conn(0x7fe2680c2d50 msgr2=0x7fe2680c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 --2- 172.21.15.73:0/1778596797 >> v2:172.21.15.150:3300/0 conn(0x7fe2680c2d50 0x7fe2680c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 -- 172.21.15.73:0/1778596797 >> v1:172.21.15.73:6789/0 conn(0x7fe268068090 legacy=0x7fe2680c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 -- 172.21.15.73:0/1778596797 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2680a7490 con 0x7fe268067cb0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26d24d700 1 -- 172.21.15.73:0/1778596797 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202767259 0 0) 0x7fe268154d20 con 0x7fe268068090 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26e24f700 1 --2- 172.21.15.73:0/1778596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268067cb0 0x7fe2680c3ec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fe2680aad00 tx=0x7fe2640090f0).ready entity=mon.0 client_cookie=f4bb3e738f18f50e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.884+0000 7fe26d24d700 1 -- 172.21.15.73:0/1778596797 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400b600 con 0x7fe268067cb0 2023-11-28T03:04:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe26d24d700 1 -- 172.21.15.73:0/1778596797 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe26400b760 con 0x7fe268067cb0 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe26d24d700 1 -- 172.21.15.73:0/1778596797 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe26400bae0 con 0x7fe268067cb0 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 -- 172.21.15.73:0/1778596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268067cb0 msgr2=0x7fe2680c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 --2- 172.21.15.73:0/1778596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268067cb0 0x7fe2680c3ec0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7fe2680aad00 tx=0x7fe2640090f0).stop 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 -- 172.21.15.73:0/1778596797 shutdown_connections 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 --2- 172.21.15.73:0/1778596797 >> v2:172.21.15.150:3300/0 conn(0x7fe2680c2d50 0x7fe2680c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 --2- 172.21.15.73:0/1778596797 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268067cb0 0x7fe2680c3ec0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 -- 172.21.15.73:0/1778596797 >> 172.21.15.73:0/1778596797 conn(0x7fe2680c7fd0 msgr2=0x7fe2680c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 -- 172.21.15.73:0/1778596797 shutdown_connections 2023-11-28T03:04:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.885+0000 7fe2704b3700 1 -- 172.21.15.73:0/1778596797 wait complete. 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.886+0000 7fe2704b3700 1 Processor -- start 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.886+0000 7fe2704b3700 1 -- start start 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 0x7fe26814fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680c2d50 0x7fe2680697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680aab00 con 0x7fe268068090 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680ab9c0 con 0x7fe268067cb0 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe2704b3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2680ab090 con 0x7fe2680c2d50 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:57520/0 (socket says 172.21.15.73:57520) 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 -- 172.21.15.73:0/2976126681 learned_addr learned my addr 172.21.15.73:0/2976126681 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26e24f700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 0x7fe26814fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26ea50700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680c2d50 0x7fe2680697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 -- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680c2d50 msgr2=0x7fe2680697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680c2d50 0x7fe2680697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 -- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 msgr2=0x7fe26814fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 0x7fe26814fea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26da4e700 1 -- 172.21.15.73:0/2976126681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe264009370 con 0x7fe268068090 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.887+0000 7fe26e24f700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 0x7fe26814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe26da4e700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fe25800c4e0 tx=0x7fe258007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800f470 con 0x7fe268068090 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe25800f5f0 con 0x7fe268068090 2023-11-28T03:04:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2680ab370 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe25800f9b0 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.888+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe268150390 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.889+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe25800fd20 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.889+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2680bef70 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.889+0000 7fe25e7fc700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 0x7fe24c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.889+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 6255+0+0 (secure 0 0 0) 0x7fe2580799c0 con 0x7fe268068090 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.889+0000 7fe26e24f700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 0x7fe24c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.890+0000 7fe26e24f700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 0x7fe24c061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe264009130 tx=0x7fe264017000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:48.893+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe2680bef70 con 0x7fe268068090 2023-11-28T03:04:49.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.099+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2680bef70 con 0x7fe268068090 2023-11-28T03:04:49.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.100+0000 7fe25e7fc700 1 -- 172.21.15.73:0/2976126681 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 14 v14) v1 ==== 94+0+3532 (secure 0 0 0) 0x7fe2680bef70 con 0x7fe268068090 2023-11-28T03:04:49.101 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:49.101 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":14,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4601,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2211802570","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2211802570},{"type":"v1","addr":"172.21.15.150:6835","nonce":2211802570}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4840,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3975722763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3975722763},{"type":"v1","addr":"172.21.15.150:6834","nonce":3975722763}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":4971,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3450935906","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3450935906},{"type":"v1","addr":"172.21.15.73:6836","nonce":3450935906}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":5428,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3139759669","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3139759669},{"type":"v1","addr":"172.21.15.73:6837","nonce":3139759669}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 msgr2=0x7fe24c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 0x7fe24c061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fe264009130 tx=0x7fe264017000).stop 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 msgr2=0x7fe2680abe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 rx=0x7fe25800c4e0 tx=0x7fe258007c70).stop 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 shutdown_connections 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe24c05f150 0x7fe24c061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:49.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe2680c2d50 0x7fe2680697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe268067cb0 0x7fe26814fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 --2- 172.21.15.73:0/2976126681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe268068090 0x7fe2680abe50 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 >> 172.21.15.73:0/2976126681 conn(0x7fe2680c7fd0 msgr2=0x7fe2680c62b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 shutdown_connections 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:49.102+0000 7fe2704b3700 1 -- 172.21.15.73:0/2976126681 wait complete. 2023-11-28T03:04:49.104 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 14 2023-11-28T03:04:49.115 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:04:49.115 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:04:49.115 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:04:49.115 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:04:49.115 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:04:49.115 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:04:49.116 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:04:49.116 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:04:49.116 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:04:49.116 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:04:49.116 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:04:49.116 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:04:49.116 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:04:49.117 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:04:49.117 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:04:49.117 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:04:49.124 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:04:49.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:04:49.130 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:04:49.131 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:04:49.131 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:04:49.132 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:04:49.167 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:04:49.167 DEBUG:teuthology.parallel:result is None 2023-11-28T03:04:49.173 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:04:49.173 DEBUG:teuthology.parallel:result is None 2023-11-28T03:04:49.174 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:04:49.174 DEBUG:teuthology.parallel:result is None 2023-11-28T03:04:49.237 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:04:49.255 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:04:49.255 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:04:51.893 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T03:04:52.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:04:52.356 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:04:55.125 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:04:55.125 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:04:55.128 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:04:55.128 DEBUG:teuthology.parallel:result is None 2023-11-28T03:04:55.128 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:04:55.211 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:04:55.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.256+0000 7ff26b710700 1 Processor -- start 2023-11-28T03:04:55.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 -- start start 2023-11-28T03:04:55.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff264150930 0x7ff2640c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff264150d10 0x7ff2640a6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2640bcfa0 0x7ff2640bd3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2640c2460 con 0x7ff264150930 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff264062910 con 0x7ff264150d10 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.257+0000 7ff26b710700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640bd8b0 con 0x7ff2640bcfa0 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff264150930 0x7ff2640c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff264150930 0x7ff2640c6620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52044/0 (socket says 172.21.15.73:52044) 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 -- 172.21.15.73:0/4025017592 learned_addr learned my addr 172.21.15.73:0/4025017592 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff269cad700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2640bcfa0 0x7ff2640bd3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff268cab700 1 --2- 172.21.15.73:0/4025017592 >> v2:172.21.15.150:3300/0 conn(0x7ff264150d10 0x7ff2640a6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 -- 172.21.15.73:0/4025017592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2640bcfa0 msgr2=0x7ff2640bd3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2640bcfa0 0x7ff2640bd3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 -- 172.21.15.73:0/4025017592 >> v2:172.21.15.150:3300/0 conn(0x7ff264150d10 msgr2=0x7ff2640a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:55.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 --2- 172.21.15.73:0/4025017592 >> v2:172.21.15.150:3300/0 conn(0x7ff264150d10 0x7ff2640a6ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 -- 172.21.15.73:0/4025017592 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2640c6c00 con 0x7ff264150930 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff2694ac700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff2640c6620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7ff2640c7700 tx=0x7ff254009060).ready entity=mon.0 client_cookie=bf4ee0fc29f10fea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff25bfff700 1 -- 172.21.15.73:0/4025017592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254005880 con 0x7ff264150930 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff25bfff700 1 -- 172.21.15.73:0/4025017592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2540059e0 con 0x7ff264150930 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.258+0000 7ff25bfff700 1 -- 172.21.15.73:0/4025017592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff254005d60 con 0x7ff264150930 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 -- 172.21.15.73:0/4025017592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 msgr2=0x7ff2640c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff2640c6620 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 rx=0x7ff2640c7700 tx=0x7ff254009060).stop 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 -- 172.21.15.73:0/4025017592 shutdown_connections 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff2640c6620 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 --2- 172.21.15.73:0/4025017592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2640bcfa0 0x7ff2640bd3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 --2- 172.21.15.73:0/4025017592 >> v2:172.21.15.150:3300/0 conn(0x7ff264150d10 0x7ff2640a6ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 -- 172.21.15.73:0/4025017592 >> 172.21.15.73:0/4025017592 conn(0x7ff2640c8230 msgr2=0x7ff2640c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:55.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 -- 172.21.15.73:0/4025017592 shutdown_connections 2023-11-28T03:04:55.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.259+0000 7ff26b710700 1 -- 172.21.15.73:0/4025017592 wait complete. 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.260+0000 7ff26b710700 1 Processor -- start 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.260+0000 7ff26b710700 1 -- start start 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 0x7ff26414ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 0x7ff26414ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640ad200 con 0x7ff264150930 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2640acd70 con 0x7ff2640bcfa0 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff26b710700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff26414b380 con 0x7ff264150d10 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52056/0 (socket says 172.21.15.73:52056) 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 -- 172.21.15.73:0/1075773068 learned_addr learned my addr 172.21.15.73:0/1075773068 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff2694ac700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 0x7ff26414ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 -- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 msgr2=0x7ff26414ddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff269cad700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 0x7ff26414ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 0x7ff26414ddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 -- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 msgr2=0x7ff26414ae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 0x7ff26414ae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:55.267 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff268cab700 1 -- 172.21.15.73:0/1075773068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff254009300 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff2694ac700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 0x7ff26414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.261+0000 7ff269cad700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 0x7ff26414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff268cab700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff26000e900 tx=0x7ff260006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000c640 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff26000c7a0 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2640aca40 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff26000cb20 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.262+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2640a6c70 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.263+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2640acf20 con 0x7ff264150930 2023-11-28T03:04:55.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.263+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff26001b400 con 0x7ff264150930 2023-11-28T03:04:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.263+0000 7ff2597fa700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 0x7ff248061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.264+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5699+0+0 (secure 0 0 0) 0x7ff26007a3e0 con 0x7ff264150930 2023-11-28T03:04:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.264+0000 7ff2694ac700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 0x7ff248061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.264+0000 7ff2694ac700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 0x7ff248061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff254009150 tx=0x7ff254016000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.268+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff2640acf20 con 0x7ff264150930 2023-11-28T03:04:55.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:55.451+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7ff2640acf20 con 0x7ff264150930 2023-11-28T03:04:56.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.386+0000 7ff2597fa700 1 -- 172.21.15.73:0/1075773068 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v61) v1 ==== 97+0+0 (secure 0 0 0) 0x7ff2640acf20 con 0x7ff264150930 2023-11-28T03:04:56.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 msgr2=0x7ff248061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 0x7ff248061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7ff254009150 tx=0x7ff254016000).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 msgr2=0x7ff26414b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 rx=0x7ff26000e900 tx=0x7ff260006f40).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 shutdown_connections 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff24805f150 0x7ff248061610 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff264150d10 0x7ff26414ddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2640bcfa0 0x7ff26414ae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 --2- 172.21.15.73:0/1075773068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff264150930 0x7ff26414b570 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 >> 172.21.15.73:0/1075773068 conn(0x7ff2640c8230 msgr2=0x7ff2640c37c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:56.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 shutdown_connections 2023-11-28T03:04:56.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.388+0000 7ff26b710700 1 -- 172.21.15.73:0/1075773068 wait complete. 2023-11-28T03:04:56.391 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:04:56.402 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:04:56.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.515+0000 7f66c7fff700 1 Processor -- start 2023-11-28T03:04:56.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.515+0000 7f66c7fff700 1 -- start start 2023-11-28T03:04:56.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c7fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66c80c0180 0x7f66c8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c7fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c80c12c0 con 0x7f66c8157660 2023-11-28T03:04:56.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c7fff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66c80c06b0 con 0x7f66c80c1750 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c7fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66c80c0990 con 0x7f66c80c0180 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c6ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f66c8157660 0x7f66c81589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42836/0 (socket says 172.21.15.73:42836) 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c6ffd700 1 -- 172.21.15.73:0/2051868662 learned_addr learned my addr 172.21.15.73:0/2051868662 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 625256676 0 0) 0x7f66c80c12c0 con 0x7f66c8157660 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.516+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66b40033f0 con 0x7f66c8157660 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66be7fc700 1 --2- 172.21.15.73:0/2051868662 >> v2:172.21.15.150:3300/0 conn(0x7f66c80c0180 0x7f66c8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2926808593 0 0) 0x7f66b40033f0 con 0x7f66c8157660 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 >> v2:172.21.15.150:3300/0 conn(0x7f66c80c0180 msgr2=0x7f66c8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 --2- 172.21.15.73:0/2051868662 >> v2:172.21.15.150:3300/0 conn(0x7f66c80c0180 0x7f66c8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 >> v1:172.21.15.150:6789/0 conn(0x7f66c80c1750 legacy=0x7f66c80648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c80c0c70 con 0x7f66c8157660 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f66b8001470 con 0x7f66c8157660 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 >> v1:172.21.15.73:6789/0 conn(0x7f66c8157660 legacy=0x7f66c81589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 0x7f66b4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b4004690 0x7f66b4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b4005050 0x7f66b400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80c0990 con 0x7f66b40038e0 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b400bfe0 con 0x7f66b4004690 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b400c1a0 con 0x7f66b4005050 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66be7fc700 1 --2- 172.21.15.73:0/2051868662 >> v2:172.21.15.150:3300/0 conn(0x7f66c80c0180 0x7f66c8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 0x7f66b4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66be7fc700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b4004690 0x7f66b4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c77fe700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b4005050 0x7f66b400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 -- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b4005050 msgr2=0x7f66b400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b4005050 0x7f66b400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 -- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b4004690 msgr2=0x7f66b4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66b4004690 0x7f66b4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 -- 172.21.15.73:0/2051868662 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40033f0 con 0x7f66b40038e0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c77fe700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b4005050 0x7f66b400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c77fe700 1 -- 172.21.15.73:0/2051868662 reap_dead start 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 0x7f66b4003d00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f66b800c190 tx=0x7f66b8001e50).ready entity=mon.0 client_cookie=dcc8f12ba582adae server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b8007a20 con 0x7f66b40038e0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66b800cac0 con 0x7f66b40038e0 2023-11-28T03:04:56.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.517+0000 7f66c67fc700 1 -- 172.21.15.73:0/2051868662 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66b800ce60 con 0x7f66b40038e0 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 -- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 msgr2=0x7f66b4003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 0x7f66b4003d00 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 rx=0x7f66b800c190 tx=0x7f66b8001e50).stop 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 -- 172.21.15.73:0/2051868662 shutdown_connections 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 --2- 172.21.15.73:0/2051868662 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66b40038e0 0x7f66b4003d00 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 -- 172.21.15.73:0/2051868662 >> 172.21.15.73:0/2051868662 conn(0x7f66c80cb000 msgr2=0x7f66c80cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 -- 172.21.15.73:0/2051868662 shutdown_connections 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.518+0000 7f66c7fff700 1 -- 172.21.15.73:0/2051868662 wait complete. 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.519+0000 7f66c7fff700 1 Processor -- start 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.519+0000 7f66c7fff700 1 -- start start 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b40038e0 0x7f66c80aea90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c80af8d0 0x7f66c80ca150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80cab30 con 0x7f66c80afd50 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80a1f60 con 0x7f66c80af8d0 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c7fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66c80ae600 con 0x7f66b40038e0 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52068/0 (socket says 172.21.15.73:52068) 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 -- 172.21.15.73:0/350393053 learned_addr learned my addr 172.21.15.73:0/350393053 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66be7fc700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c80af8d0 0x7f66c80ca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b40038e0 0x7f66c80aea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 -- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b40038e0 msgr2=0x7f66c80aea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b40038e0 0x7f66c80aea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 -- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c80af8d0 msgr2=0x7f66c80ca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c80af8d0 0x7f66c80ca150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 -- 172.21.15.73:0/350393053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66c81547a0 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.520+0000 7f66c77fe700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f66c8084120 tx=0x7f66c0009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.521+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0002490 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.521+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66c00025f0 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.521+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66c8154a80 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.521+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66c0002990 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.521+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66c80ae290 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.522+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66c80ae450 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.522+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f66c0003030 con 0x7f66c80afd50 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.522+0000 7f66bf7fe700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.522+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f66c00796f0 con 0x7f66c80afd50 2023-11-28T03:04:56.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.522+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.523+0000 7f66c6ffd700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66b800ca90 tx=0x7f66b8015000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.527+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66c0048af0 con 0x7f66c80afd50 2023-11-28T03:04:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.755+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f66c806c7c0 con 0x7f66c80afd50 2023-11-28T03:04:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.755+0000 7f66bf7fe700 1 -- 172.21.15.73:0/350393053 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v11) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f66c004acb0 con 0x7f66c80afd50 2023-11-28T03:04:56.756 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:56.756 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:04:56.756 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.757 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:04:56.758 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.759 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.760 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.761 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.762 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:04:56.763 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.764 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.765 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:04:56.766 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.767 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:04:56.768 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:04:56.769 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.770 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.771 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:04:56.772 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 msgr2=0x7f66a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f66b800ca90 tx=0x7f66b8015000).stop 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 msgr2=0x7f66c80ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7f66c8084120 tx=0x7f66c0009040).stop 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 shutdown_connections 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66a805f150 0x7f66a8061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66c80afd50 0x7f66c80ca640 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66b40038e0 0x7f66c80aea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 --2- 172.21.15.73:0/350393053 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66c80af8d0 0x7f66c80ca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 >> 172.21.15.73:0/350393053 conn(0x7f66c80cb000 msgr2=0x7f66c80ab050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 shutdown_connections 2023-11-28T03:04:56.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.757+0000 7f66c7fff700 1 -- 172.21.15.73:0/350393053 wait complete. 2023-11-28T03:04:56.775 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:04:56.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.886+0000 7f94a3e5d700 1 Processor -- start 2023-11-28T03:04:56.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.886+0000 7f94a3e5d700 1 -- start start 2023-11-28T03:04:56.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.887+0000 7f94a3e5d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 0x7f949c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.887+0000 7f94a3e5d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.887+0000 7f94a3e5d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f949c0c69a0 con 0x7f949c06a610 2023-11-28T03:04:56.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.887+0000 7f94a3e5d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f949c0c13a0 con 0x7f949c1544e0 2023-11-28T03:04:56.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.887+0000 7f94a3e5d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0c1680 con 0x7f949c0c6e30 2023-11-28T03:04:56.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a1bf9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 0x7f949c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35222/0 (socket says 172.21.15.73:35222) 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 -- 172.21.15.73:0/3600843108 learned_addr learned my addr 172.21.15.73:0/3600843108 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a13f8700 1 --1- 172.21.15.73:0/3600843108 >> v1:172.21.15.150:6789/0 conn(0x7f949c06a610 0x7f949c0c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43138/0 (socket says 172.21.15.73:43138) 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 -- 172.21.15.73:0/3600843108 >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 msgr2=0x7f949c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 --2- 172.21.15.73:0/3600843108 >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 0x7f949c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 -- 172.21.15.73:0/3600843108 >> v1:172.21.15.150:6789/0 conn(0x7f949c06a610 legacy=0x7f949c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 -- 172.21.15.73:0/3600843108 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f949c0c6630 con 0x7f949c0c6e30 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a1bf9700 1 --2- 172.21.15.73:0/3600843108 >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 0x7f949c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a23fa700 1 --2- 172.21.15.73:0/3600843108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9494009f20 tx=0x7f9494009840).ready entity=mon.2 client_cookie=8ec9ccbc8a1c01ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a0bf7700 1 -- 172.21.15.73:0/3600843108 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949400e070 con 0x7f949c0c6e30 2023-11-28T03:04:56.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a0bf7700 1 -- 172.21.15.73:0/3600843108 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949400b9d0 con 0x7f949c0c6e30 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.888+0000 7f94a0bf7700 1 -- 172.21.15.73:0/3600843108 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949400bd50 con 0x7f949c0c6e30 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3600843108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 msgr2=0x7f949c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3600843108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f9494009f20 tx=0x7f9494009840).stop 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3600843108 shutdown_connections 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3600843108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c0c6e30 0x7f949c0c72e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3600843108 >> v2:172.21.15.150:3300/0 conn(0x7f949c1544e0 0x7f949c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3600843108 >> 172.21.15.73:0/3600843108 conn(0x7f949c085020 msgr2=0x7f949c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3600843108 shutdown_connections 2023-11-28T03:04:56.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.889+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3600843108 wait complete. 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.890+0000 7f94a3e5d700 1 Processor -- start 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.890+0000 7f94a3e5d700 1 -- start start 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 0x7f949c153610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c1544e0 0x7f949c164130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0955f0 con 0x7f949c06a610 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f949c0949e0 con 0x7f949c0c6e30 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a3e5d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f949c094cc0 con 0x7f949c1544e0 2023-11-28T03:04:56.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52080/0 (socket says 172.21.15.73:52080) 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 -- 172.21.15.73:0/3804661444 learned_addr learned my addr 172.21.15.73:0/3804661444 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a23fa700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c1544e0 0x7f949c164130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a13f8700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 0x7f949c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 -- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c1544e0 msgr2=0x7f949c164130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c1544e0 0x7f949c164130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 -- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 msgr2=0x7f949c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 0x7f949c153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 -- 172.21.15.73:0/3804661444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9498009250 con 0x7f949c06a610 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a13f8700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 0x7f949c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:56.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.891+0000 7f94a1bf9700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f949800ec80 tx=0x7f9498006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.892+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94980096a0 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.892+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9498005880 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.892+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9494009ae0 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.892+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9498005c00 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.892+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f949c095280 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.893+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f949c0958d0 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.893+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f949800c850 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.893+0000 7f9491ffb700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 0x7f9480061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.893+0000 7f94a13f8700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 0x7f9480061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.893+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9498079fb0 con 0x7f949c06a610 2023-11-28T03:04:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.894+0000 7f94a13f8700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 0x7f9480061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f949c0c12d0 tx=0x7f948c007c90).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:56.898+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f949804b5d0 con 0x7f949c06a610 2023-11-28T03:04:57.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.061+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f949c0c6250 con 0x7f949c06a610 2023-11-28T03:04:57.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.061+0000 7f9491ffb700 1 -- 172.21.15.73:0/3804661444 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+13712 (secure 0 0 0) 0x7f9498016070 con 0x7f949c06a610 2023-11-28T03:04:57.062 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:57.063 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":61,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:04:56.383614+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":36,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 msgr2=0x7f9480061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 0x7f9480061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f949c0c12d0 tx=0x7f948c007c90).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 msgr2=0x7f949c095a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 rx=0x7f949800ec80 tx=0x7f9498006f40).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 shutdown_connections 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f948005f150 0x7f9480061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f949c1544e0 0x7f949c164130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f949c06a610 0x7f949c095a80 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 --2- 172.21.15.73:0/3804661444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f949c0c6e30 0x7f949c153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 >> 172.21.15.73:0/3804661444 conn(0x7f949c085020 msgr2=0x7f949c0ae510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 shutdown_connections 2023-11-28T03:04:57.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.063+0000 7f94a3e5d700 1 -- 172.21.15.73:0/3804661444 wait complete. 2023-11-28T03:04:57.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '56', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:04:57.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:04:57.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.191+0000 7f9c47d1b700 1 Processor -- start 2023-11-28T03:04:57.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.192+0000 7f9c47d1b700 1 -- start start 2023-11-28T03:04:57.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.192+0000 7f9c47d1b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c400a8280 0x7f9c400a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.192+0000 7f9c47d1b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 0x7f9c40155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.192+0000 7f9c47d1b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c40067770 con 0x7f9c4014e600 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c47d1b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c40159180 con 0x7f9c400a8280 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c47d1b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400be8d0 con 0x7f9c40158160 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c45ab7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c400a8280 0x7f9c400a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 0x7f9c40155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c462b8700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9c4014e600 0x7f9c4014ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43140/0 (socket says 172.21.15.73:43140) 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c462b8700 1 -- 172.21.15.73:0/487184141 learned_addr learned my addr 172.21.15.73:0/487184141 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 -- 172.21.15.73:0/487184141 >> v2:172.21.15.150:3300/0 conn(0x7f9c400a8280 msgr2=0x7f9c400a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 --2- 172.21.15.73:0/487184141 >> v2:172.21.15.150:3300/0 conn(0x7f9c400a8280 0x7f9c400a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 -- 172.21.15.73:0/487184141 >> v1:172.21.15.150:6789/0 conn(0x7f9c4014e600 legacy=0x7f9c4014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 -- 172.21.15.73:0/487184141 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c40158600 con 0x7f9c40158160 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.193+0000 7f9c452b6700 1 --2- 172.21.15.73:0/487184141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 0x7f9c40155500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9c30009760 tx=0x7f9c30009060).ready entity=mon.2 client_cookie=22eb3d4ff860503e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c44ab5700 1 -- 172.21.15.73:0/487184141 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000f510 con 0x7f9c40158160 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c44ab5700 1 -- 172.21.15.73:0/487184141 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000f670 con 0x7f9c40158160 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c44ab5700 1 -- 172.21.15.73:0/487184141 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000f9f0 con 0x7f9c40158160 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 -- 172.21.15.73:0/487184141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 msgr2=0x7f9c40155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/487184141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 0x7f9c40155500 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f9c30009760 tx=0x7f9c30009060).stop 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 -- 172.21.15.73:0/487184141 shutdown_connections 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/487184141 >> v2:172.21.15.150:3300/0 conn(0x7f9c400a8280 0x7f9c400a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/487184141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c40158160 0x7f9c40155500 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 -- 172.21.15.73:0/487184141 >> 172.21.15.73:0/487184141 conn(0x7f9c40062700 msgr2=0x7f9c40062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 -- 172.21.15.73:0/487184141 shutdown_connections 2023-11-28T03:04:57.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.194+0000 7f9c47d1b700 1 -- 172.21.15.73:0/487184141 wait complete. 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.195+0000 7f9c47d1b700 1 Processor -- start 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.195+0000 7f9c47d1b700 1 -- start start 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 0x7f9c400aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 0x7f9c40064a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400aa750 con 0x7f9c4014e600 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400a9b40 con 0x7f9c40158160 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c47d1b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c400a9e20 con 0x7f9c400a8280 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c452b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c452b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52092/0 (socket says 172.21.15.73:52092) 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c452b6700 1 -- 172.21.15.73:0/1158305957 learned_addr learned my addr 172.21.15.73:0/1158305957 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c45ab7700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 0x7f9c400aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.196+0000 7f9c462b8700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 0x7f9c40064a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 -- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 msgr2=0x7f9c400aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 0x7f9c400aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 -- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 msgr2=0x7f9c40064a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 0x7f9c40064a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 -- 172.21.15.73:0/1158305957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c300092a0 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c45ab7700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 0x7f9c400aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c462b8700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 0x7f9c40064a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c452b6700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f9c30009e00 tx=0x7f9c30017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000f9f0 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c30000c90 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c400aa100 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3001b660 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.197+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c4014d450 con 0x7f9c4014e600 2023-11-28T03:04:57.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.198+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c400aa8c0 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.198+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9c3001ba10 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.199+0000 7f9c35ffb700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 0x7f9c24061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.199+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9c3007b370 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.199+0000 7f9c45ab7700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 0x7f9c24061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.199+0000 7f9c45ab7700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 0x7f9c24061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f9c3c007780 tx=0x7f9c3c00b040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.203+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9c3004c990 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.364+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f9c4006c860 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.364+0000 7f9c35ffb700 1 -- 172.21.15.73:0/1158305957 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v61) v1 ==== 110+0+10 (secure 0 0 0) 0x7f9c3001f0b0 con 0x7f9c4014e600 2023-11-28T03:04:57.580 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 msgr2=0x7f9c24061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 0x7f9c24061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f9c3c007780 tx=0x7f9c3c00b040).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 msgr2=0x7f9c4014cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 rx=0x7f9c30009e00 tx=0x7f9c30017000).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 shutdown_connections 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c2405f150 0x7f9c24061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c40158160 0x7f9c40064a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c400a8280 0x7f9c400aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 --2- 172.21.15.73:0/1158305957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c4014e600 0x7f9c4014cf60 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 >> 172.21.15.73:0/1158305957 conn(0x7f9c40062700 msgr2=0x7f9c4014dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:57.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 shutdown_connections 2023-11-28T03:04:57.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.366+0000 7f9c47d1b700 1 -- 172.21.15.73:0/1158305957 wait complete. 2023-11-28T03:04:57.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:04:57.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.698+0000 7f0c72786700 1 Processor -- start 2023-11-28T03:04:57.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.698+0000 7f0c72786700 1 -- start start 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c72786700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0c6c098d10 0x7f0c6c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c72786700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0c6c0c6190 0x7f0c6c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c72786700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c6c0c72d0 con 0x7f0c6c064350 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c72786700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c154bc0 con 0x7f0c6c098d10 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c72786700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c6c0c66c0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c6bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0c6c098d10 0x7f0c6c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c70d23700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0c6c064350 0x7f0c6c158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42844/0 (socket says 172.21.15.73:42844) 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.699+0000 7f0c70d23700 1 -- 172.21.15.73:0/200780004 learned_addr learned my addr 172.21.15.73:0/200780004 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:57.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6bfff700 1 -- 172.21.15.73:0/200780004 >> v2:172.21.15.150:3300/0 conn(0x7f0c6c0c6190 msgr2=0x7f0c6c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6bfff700 1 --2- 172.21.15.73:0/200780004 >> v2:172.21.15.150:3300/0 conn(0x7f0c6c0c6190 0x7f0c6c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6bfff700 1 -- 172.21.15.73:0/200780004 >> v1:172.21.15.73:6789/0 conn(0x7f0c6c064350 legacy=0x7f0c6c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6affd700 1 -- 172.21.15.73:0/200780004 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133605945 0 0) 0x7f0c6c0c72d0 con 0x7f0c6c064350 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6bfff700 1 -- 172.21.15.73:0/200780004 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c6c0c69a0 con 0x7f0c6c098d10 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6bfff700 1 --2- 172.21.15.73:0/200780004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0c7760 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f0c6c0c5b40 tx=0x7f0c5c009060).ready entity=mon.0 client_cookie=669cd375e98c339c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6affd700 1 -- 172.21.15.73:0/200780004 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00b580 con 0x7f0c6c098d10 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6affd700 1 -- 172.21.15.73:0/200780004 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c5c00b6e0 con 0x7f0c6c098d10 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c6affd700 1 -- 172.21.15.73:0/200780004 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c5c00ba60 con 0x7f0c6c098d10 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 -- 172.21.15.73:0/200780004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c098d10 msgr2=0x7f0c6c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 --2- 172.21.15.73:0/200780004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0c7760 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 rx=0x7f0c6c0c5b40 tx=0x7f0c5c009060).stop 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 -- 172.21.15.73:0/200780004 shutdown_connections 2023-11-28T03:04:57.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 --2- 172.21.15.73:0/200780004 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0c7760 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 --2- 172.21.15.73:0/200780004 >> v2:172.21.15.150:3300/0 conn(0x7f0c6c0c6190 0x7f0c6c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:57.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 -- 172.21.15.73:0/200780004 >> 172.21.15.73:0/200780004 conn(0x7f0c6c0cb310 msgr2=0x7f0c6c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:57.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 -- 172.21.15.73:0/200780004 shutdown_connections 2023-11-28T03:04:57.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.700+0000 7f0c72786700 1 -- 172.21.15.73:0/200780004 wait complete. 2023-11-28T03:04:57.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.701+0000 7f0c72786700 1 Processor -- start 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 -- start start 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c6c064350 0x7f0c6c14c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c157330 con 0x7f0c6c0c6190 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c14cc60 con 0x7f0c6c098d10 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c72786700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c6c156a00 con 0x7f0c6c064350 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.702+0000 7f0c70d23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52110/0 (socket says 172.21.15.73:52110) 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 -- 172.21.15.73:0/218032774 learned_addr learned my addr 172.21.15.73:0/218032774 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c6bfff700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c6c064350 0x7f0c6c14c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c6b7fe700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 -- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c6c064350 msgr2=0x7f0c6c14c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c6c064350 0x7f0c6c14c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 -- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 msgr2=0x7f0c6c0aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 -- 172.21.15.73:0/218032774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c5c0092a0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c6b7fe700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c70d23700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f0c6400c810 tx=0x7f0c64007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64005d30 con 0x7f0c6c0c6190 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c64005e90 con 0x7f0c6c0c6190 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c6c156ce0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c64007320 con 0x7f0c6c0c6190 2023-11-28T03:04:57.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.703+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c6c1566f0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.704+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c6c1574e0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.704+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0c64007710 con 0x7f0c6c0c6190 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.705+0000 7f0c53fff700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 0x7f0c58061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.705+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0c64079f20 con 0x7f0c6c0c6190 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.705+0000 7f0c6bfff700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 0x7f0c58061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:57.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.705+0000 7f0c6bfff700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 0x7f0c58061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0c5c009fd0 tx=0x7f0c5c017000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:57.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.709+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0c6c1574e0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.871+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0c6c1576c0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.872+0000 7f0c53fff700 1 -- 172.21.15.73:0/218032774 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v61) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0c6404b4e0 con 0x7f0c6c0c6190 2023-11-28T03:04:57.873 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:04:57.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 msgr2=0x7f0c58061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 0x7f0c58061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0c5c009fd0 tx=0x7f0c5c017000).stop 2023-11-28T03:04:57.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 msgr2=0x7f0c6c1684c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:57.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 rx=0x7f0c6400c810 tx=0x7f0c64007f90).stop 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 shutdown_connections 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0c5805f150 0x7f0c58061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0c6c0c6190 0x7f0c6c1684c0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0c6c064350 0x7f0c6c14c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 --2- 172.21.15.73:0/218032774 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0c6c098d10 0x7f0c6c0aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 >> 172.21.15.73:0/218032774 conn(0x7f0c6c0cb310 msgr2=0x7f0c6c151a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 shutdown_connections 2023-11-28T03:04:57.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.874+0000 7f0c72786700 1 -- 172.21.15.73:0/218032774 wait complete. 2023-11-28T03:04:57.887 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:04:58.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.999+0000 7fd3f2c9c700 1 Processor -- start 2023-11-28T03:04:58.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:57.999+0000 7fd3f2c9c700 1 -- start start 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f2c9c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3ec0caab0 0x7fd3ec0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f2c9c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd3ec0ca180 0x7fd3ec15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f2c9c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3ec0a9a90 con 0x7fd3ec0cae90 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f2c9c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec154ce0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f2c9c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ec0a8e80 con 0x7fd3ec0ca180 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f0a38700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd3ec0caab0 0x7fd3ec0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3e3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd3ec0cae90 0x7fd3ec0c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42858/0 (socket says 172.21.15.73:42858) 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3e3fff700 1 -- 172.21.15.73:0/1415282474 learned_addr learned my addr 172.21.15.73:0/1415282474 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.000+0000 7fd3f1239700 1 --2- 172.21.15.73:0/1415282474 >> v2:172.21.15.150:3300/0 conn(0x7fd3ec0ca180 0x7fd3ec15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f0a38700 1 -- 172.21.15.73:0/1415282474 >> v2:172.21.15.150:3300/0 conn(0x7fd3ec0ca180 msgr2=0x7fd3ec15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/1415282474 >> v2:172.21.15.150:3300/0 conn(0x7fd3ec0ca180 0x7fd3ec15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f0a38700 1 -- 172.21.15.73:0/1415282474 >> v1:172.21.15.73:6789/0 conn(0x7fd3ec0cae90 legacy=0x7fd3ec0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f0a38700 1 -- 172.21.15.73:0/1415282474 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ec0a9160 con 0x7fd3ec0caab0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3e37fe700 1 -- 172.21.15.73:0/1415282474 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2737123346 0 0) 0x7fd3ec0a9a90 con 0x7fd3ec0cae90 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/1415282474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec0aa470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fd3ec0a9a00 tx=0x7fd3e8006f40).ready entity=mon.0 client_cookie=5f97efa9e190adcb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3e37fe700 1 -- 172.21.15.73:0/1415282474 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8005880 con 0x7fd3ec0caab0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3e37fe700 1 -- 172.21.15.73:0/1415282474 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80059e0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3e37fe700 1 -- 172.21.15.73:0/1415282474 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e8005d60 con 0x7fd3ec0caab0 2023-11-28T03:04:58.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/1415282474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 msgr2=0x7fd3ec0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/1415282474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec0aa470 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fd3ec0a9a00 tx=0x7fd3e8006f40).stop 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/1415282474 shutdown_connections 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/1415282474 >> v2:172.21.15.150:3300/0 conn(0x7fd3ec0ca180 0x7fd3ec15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/1415282474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec0aa470 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.001+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/1415282474 >> 172.21.15.73:0/1415282474 conn(0x7fd3ec094920 msgr2=0x7fd3ec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.002+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/1415282474 shutdown_connections 2023-11-28T03:04:58.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.002+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/1415282474 wait complete. 2023-11-28T03:04:58.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.003+0000 7fd3f2c9c700 1 Processor -- start 2023-11-28T03:04:58.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.003+0000 7fd3f2c9c700 1 -- start start 2023-11-28T03:04:58.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.003+0000 7fd3f2c9c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 0x7fd3ec163b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3ec0cae90 0x7fd3ec1553a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15b2d0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15a230 con 0x7fd3ec0cae90 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ec15a510 con 0x7fd3ec0ca180 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52132/0 (socket says 172.21.15.73:52132) 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 -- 172.21.15.73:0/970867042 learned_addr learned my addr 172.21.15.73:0/970867042 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 0x7fd3ec163b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f1239700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3ec0cae90 0x7fd3ec1553a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 -- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 msgr2=0x7fd3ec163b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 0x7fd3ec163b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 -- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3ec0cae90 msgr2=0x7fd3ec1553a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3ec0cae90 0x7fd3ec1553a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 -- 172.21.15.73:0/970867042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3e8009250 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 0x7fd3ec163b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e3fff700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fd3dc001eb0 tx=0x7fd3dc007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0072e0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3dc007440 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3dc0057a0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ec15a7f0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.004+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ec15ae30 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.005+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ec159fd0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.006+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd3dc005b50 con 0x7fd3ec0caab0 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.006+0000 7fd3e0ff9700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 0x7fd3d8061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.006+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd3dc0795b0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.006+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 0x7fd3d8061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.006+0000 7fd3f0a38700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 0x7fd3d8061640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd3e8009fd0 tx=0x7fd3e8016000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.010+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd3dc04abd0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.177+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd3ec15afc0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.177+0000 7fd3e0ff9700 1 -- 172.21.15.73:0/970867042 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v61) v1 ==== 84+0+143 (secure 0 0 0) 0x7fd3dc04e1f0 con 0x7fd3ec0caab0 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:04:58.179 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:04:58.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 msgr2=0x7fd3d8061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 0x7fd3d8061640 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fd3e8009fd0 tx=0x7fd3e8016000).stop 2023-11-28T03:04:58.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 msgr2=0x7fd3ec154eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 rx=0x7fd3dc001eb0 tx=0x7fd3dc007e90).stop 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 shutdown_connections 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd3d805f180 0x7fd3d8061640 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd3ec0cae90 0x7fd3ec1553a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd3ec0ca180 0x7fd3ec163b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 --2- 172.21.15.73:0/970867042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd3ec0caab0 0x7fd3ec154eb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 >> 172.21.15.73:0/970867042 conn(0x7fd3ec094920 msgr2=0x7fd3ec0af7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 shutdown_connections 2023-11-28T03:04:58.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.179+0000 7fd3f2c9c700 1 -- 172.21.15.73:0/970867042 wait complete. 2023-11-28T03:04:58.192 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:04:58.192 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:04:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.308+0000 7f8ef4b11700 1 Processor -- start 2023-11-28T03:04:58.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.308+0000 7f8ef4b11700 1 -- start start 2023-11-28T03:04:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ef4b11700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef01582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ef4b11700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ef4b11700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ef0159210 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ef4b11700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ef00c0c70 con 0x7f8ef00ab3b0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ef4b11700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00c2110 con 0x7f8ef0154610 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52138/0 (socket says 172.21.15.73:52138) 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 -- 172.21.15.73:0/1113205395 learned_addr learned my addr 172.21.15.73:0/1113205395 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 -- 172.21.15.73:0/1113205395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 msgr2=0x7f8ef01582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8eee59c700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef01582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef01582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 -- 172.21.15.73:0/1113205395 >> v1:172.21.15.150:6789/0 conn(0x7f8ef00aa1c0 legacy=0x7f8ef00aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 -- 172.21.15.73:0/1113205395 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ef0158970 con 0x7f8ef00ab3b0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.309+0000 7f8ee5d9b700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8ed8006fd0 tx=0x7f8ed80090f0).ready entity=mon.0 client_cookie=bf4042e8aad363b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8eedd9b700 1 -- 172.21.15.73:0/1113205395 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800b600 con 0x7f8ef00ab3b0 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8eedd9b700 1 -- 172.21.15.73:0/1113205395 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed800b760 con 0x7f8ef00ab3b0 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8eedd9b700 1 -- 172.21.15.73:0/1113205395 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed800bae0 con 0x7f8ef00ab3b0 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 -- 172.21.15.73:0/1113205395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00ab3b0 msgr2=0x7f8ef00c80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 rx=0x7f8ed8006fd0 tx=0x7f8ed80090f0).stop 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 -- 172.21.15.73:0/1113205395 shutdown_connections 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef01582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/1113205395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00ab3b0 0x7f8ef00c80c0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 -- 172.21.15.73:0/1113205395 >> 172.21.15.73:0/1113205395 conn(0x7f8ef00655a0 msgr2=0x7f8ef0069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 -- 172.21.15.73:0/1113205395 shutdown_connections 2023-11-28T03:04:58.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.310+0000 7f8ef4b11700 1 -- 172.21.15.73:0/1113205395 wait complete. 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.311+0000 7f8ef4b11700 1 Processor -- start 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.311+0000 7f8ef4b11700 1 -- start start 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef014aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ef0157280 0x7f8ef00aca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00ae1f0 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00aed70 con 0x7f8ef0157280 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8ef4b11700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ef00ae440 con 0x7f8ef0154610 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52152/0 (socket says 172.21.15.73:52152) 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 -- 172.21.15.73:0/4013618970 learned_addr learned my addr 172.21.15.73:0/4013618970 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 -- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 msgr2=0x7f8ef014aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef014aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 -- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ef0157280 msgr2=0x7f8ef00aca90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ef0157280 0x7f8ef00aca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.312+0000 7f8eee59c700 1 -- 172.21.15.73:0/4013618970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed8009370 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8eee59c700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f8ee000c8b0 tx=0x7f8ee0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0010580 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ee00106e0 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ef00ae720 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ee0010a60 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.313+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ef0156630 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.314+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ef00aebc0 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.314+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8ee001b460 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.314+0000 7f8ee6ffd700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 0x7f8ed0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:58.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.314+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8ee0079e60 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.314+0000 7f8ee5d9b700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 0x7f8ed0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:58.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.315+0000 7f8ee5d9b700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 0x7f8ed0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8ef00c7c60 tx=0x7f8ed8017000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:58.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.318+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8ee004b420 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:58.486+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f8ef00af100 con 0x7f8ef00aa1c0 2023-11-28T03:04:58.961 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-28T03:04:59.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.402+0000 7f8ee6ffd700 1 -- 172.21.15.73:0/4013618970 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v62) v1 ==== 152+0+0 (secure 0 0 0) 0x7f8ee004ea40 con 0x7f8ef00aa1c0 2023-11-28T03:04:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.405+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 msgr2=0x7f8ed0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.405+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 0x7f8ed0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f8ef00c7c60 tx=0x7f8ed8017000).stop 2023-11-28T03:04:59.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.405+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 msgr2=0x7f8ef0156140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.405+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 rx=0x7f8ee000c8b0 tx=0x7f8ee0008040).stop 2023-11-28T03:04:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 shutdown_connections 2023-11-28T03:04:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ed005f150 0x7f8ed0061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ef0157280 0x7f8ef00aca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:59.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ef00aa1c0 0x7f8ef0156140 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 --2- 172.21.15.73:0/4013618970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ef0154610 0x7f8ef014aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.406+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 >> 172.21.15.73:0/4013618970 conn(0x7f8ef00655a0 msgr2=0x7f8ef014e1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.407+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 shutdown_connections 2023-11-28T03:04:59.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.407+0000 7f8ef4b11700 1 -- 172.21.15.73:0/4013618970 wait complete. 2023-11-28T03:04:59.415 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:04:59.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:04:59.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.551+0000 7f2bb57f0700 1 Processor -- start 2023-11-28T03:04:59.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.551+0000 7f2bb57f0700 1 -- start start 2023-11-28T03:04:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.551+0000 7f2bb57f0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2bb0069d10 0x7f2bb006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:59.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bb57f0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb00c3470 con 0x7f2bb00bd1f0 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bb57f0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2bb0151ea0 con 0x7f2bb0151a20 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bb57f0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2bb0152180 con 0x7f2bb0069d10 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baf7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2bb00bd1f0 0x7f2bb00c38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42866/0 (socket says 172.21.15.73:42866) 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baf7fe700 1 -- 172.21.15.73:0/3363496006 learned_addr learned my addr 172.21.15.73:0/3363496006 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baeffd700 1 --2- 172.21.15.73:0/3363496006 >> v2:172.21.15.150:3300/0 conn(0x7f2bb0069d10 0x7f2bb006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1800986281 0 0) 0x7f2bb00c3470 con 0x7f2bb00bd1f0 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b9c0033f0 con 0x7f2bb00bd1f0 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3386131347 0 0) 0x7f2bb0151ea0 con 0x7f2bb0151a20 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2bb00c3470 con 0x7f2bb0151a20 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3560195151 0 0) 0x7f2b9c0033f0 con 0x7f2bb00bd1f0 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baeffd700 1 -- 172.21.15.73:0/3363496006 >> v1:172.21.15.150:6789/0 conn(0x7f2bb0151a20 legacy=0x7f2bb00be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baeffd700 1 -- 172.21.15.73:0/3363496006 >> v1:172.21.15.73:6789/0 conn(0x7f2bb00bd1f0 legacy=0x7f2bb00c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baeffd700 1 -- 172.21.15.73:0/3363496006 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2bb0152460 con 0x7f2bb0069d10 2023-11-28T03:04:59.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.552+0000 7f2baeffd700 1 --2- 172.21.15.73:0/3363496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0069d10 0x7f2bb006a0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f2ba0009760 tx=0x7f2ba0009060).ready entity=mon.1 client_cookie=8b0e42167df2f82f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba80017f0 con 0x7f2bb0069d10 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0011400 con 0x7f2bb0069d10 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bae7fc700 1 -- 172.21.15.73:0/3363496006 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0011780 con 0x7f2bb0069d10 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3363496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0069d10 msgr2=0x7f2bb006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3363496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0069d10 0x7f2bb006a0f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 rx=0x7f2ba0009760 tx=0x7f2ba0009060).stop 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3363496006 shutdown_connections 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3363496006 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb0069d10 0x7f2bb006a0f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3363496006 >> 172.21.15.73:0/3363496006 conn(0x7f2bb00642d0 msgr2=0x7f2bb00c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3363496006 shutdown_connections 2023-11-28T03:04:59.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.553+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3363496006 wait complete. 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.554+0000 7f2bb57f0700 1 Processor -- start 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.554+0000 7f2bb57f0700 1 -- start start 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb00bd1f0 0x7f2bb00c7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb0151a20 0x7f2bb0062700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00bfff0 con 0x7f2bb0069d10 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00c02d0 con 0x7f2bb00bd1f0 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2bb57f0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2bb00c05b0 con 0x7f2bb0151a20 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52164/0 (socket says 172.21.15.73:52164) 2023-11-28T03:04:59.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 -- 172.21.15.73:0/3826183396 learned_addr learned my addr 172.21.15.73:0/3826183396 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2ba7fff700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb00bd1f0 0x7f2bb00c7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 -- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb0151a20 msgr2=0x7f2bb0062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb0151a20 0x7f2bb0062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 -- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb00bd1f0 msgr2=0x7f2bb00c7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb00bd1f0 0x7f2bb00c7240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 -- 172.21.15.73:0/3826183396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ba00092a0 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.555+0000 7f2baeffd700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f2ba0009d50 tx=0x7f2ba0016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.556+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba00108e0 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.556+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ba0010a40 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.556+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2bb00c0890 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.556+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ba0010dc0 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.556+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2bb014e1c0 con 0x7f2bb0069d10 2023-11-28T03:04:59.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.557+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2bb006c860 con 0x7f2bb0069d10 2023-11-28T03:04:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.557+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2ba001d640 con 0x7f2bb0069d10 2023-11-28T03:04:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.557+0000 7f2ba6ffd700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 0x7f2b900615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:04:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.557+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f2ba007ab70 con 0x7f2bb0069d10 2023-11-28T03:04:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.557+0000 7f2ba7fff700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 0x7f2b900615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:04:59.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.558+0000 7f2ba7fff700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 0x7f2b900615e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2bb00ae050 tx=0x7f2b98009040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:04:59.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.562+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ba004c0d0 con 0x7f2bb0069d10 2023-11-28T03:04:59.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:04:59.726+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2bb00c5800 con 0x7f2bb0069d10 2023-11-28T03:05:00.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.407+0000 7f2ba6ffd700 1 -- 172.21.15.73:0/3826183396 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v63) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2ba0022030 con 0x7f2bb0069d10 2023-11-28T03:05:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.413+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 msgr2=0x7f2b900615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.413+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 0x7f2b900615e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2bb00ae050 tx=0x7f2b98009040).stop 2023-11-28T03:05:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.413+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 msgr2=0x7f2bb014dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.413+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f2ba0009d50 tx=0x7f2ba0016000).stop 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 shutdown_connections 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2b9005f120 0x7f2b900615e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2bb0151a20 0x7f2bb0062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2bb0069d10 0x7f2bb014dcd0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 --2- 172.21.15.73:0/3826183396 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2bb00bd1f0 0x7f2bb00c7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.419+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 >> 172.21.15.73:0/3826183396 conn(0x7f2bb00642d0 msgr2=0x7f2bb0154190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:00.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.420+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 shutdown_connections 2023-11-28T03:05:00.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.420+0000 7f2bb57f0700 1 -- 172.21.15.73:0/3826183396 wait complete. 2023-11-28T03:05:00.423 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:05:00.441 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:05:00.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.558+0000 7f92c3636700 1 Processor -- start 2023-11-28T03:05:00.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.558+0000 7f92c3636700 1 -- start start 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c3636700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f92bc098d10 0x7f92bc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c3636700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92bc063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c3636700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92bc0c72d0 con 0x7f92bc064350 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c3636700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92bc154bc0 con 0x7f92bc098d10 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c3636700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0c66c0 con 0x7f92bc0c6190 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c1bd3700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92bc064350 0x7f92bc158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42878/0 (socket says 172.21.15.73:42878) 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4213752345 learned_addr learned my addr 172.21.15.73:0/4213752345 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 282554315 0 0) 0x7f92bc0c72d0 con 0x7f92bc064350 2023-11-28T03:05:00.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92bc063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4213752345 >> v2:172.21.15.150:3300/0 conn(0x7f92bc098d10 0x7f92bc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a80033f0 con 0x7f92bc064350 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4221631540 0 0) 0x7f92a80033f0 con 0x7f92bc064350 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 msgr2=0x7f92bc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92bc063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 >> v2:172.21.15.150:3300/0 conn(0x7f92bc098d10 msgr2=0x7f92bc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 --2- 172.21.15.73:0/4213752345 >> v2:172.21.15.150:3300/0 conn(0x7f92bc098d10 0x7f92bc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0c69a0 con 0x7f92bc064350 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f92b40017f0 con 0x7f92bc064350 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 >> v1:172.21.15.73:6789/0 conn(0x7f92bc064350 legacy=0x7f92bc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92a80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a8003e10 0x7f92a8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a8004b80 0x7f92a8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc154bc0 con 0x7f92a8004b80 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0c66c0 con 0x7f92a8003e10 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.559+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a800b8e0 con 0x7f92bc0c6190 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92a80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a8003e10 0x7f92a8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c1bd3700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a8004b80 0x7f92a8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 -- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 msgr2=0x7f92a80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92bc0c6190 0x7f92a80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 -- 172.21.15.73:0/4213752345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a8004b80 msgr2=0x7f92a8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92a8004b80 0x7f92a8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 -- 172.21.15.73:0/4213752345 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a80033f0 con 0x7f92a8003e10 2023-11-28T03:05:00.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a8003e10 0x7f92a8004690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f92ac009a80 tx=0x7f92ac009060).ready entity=mon.1 client_cookie=54403ddeba3c5c7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac009430 con 0x7f92a8003e10 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac00e640 con 0x7f92a8003e10 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.560+0000 7f92b3fff700 1 -- 172.21.15.73:0/4213752345 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac00e9c0 con 0x7f92a8003e10 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 -- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a8003e10 msgr2=0x7f92a8004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 --2- 172.21.15.73:0/4213752345 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92a8003e10 0x7f92a8004690 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 rx=0x7f92ac009a80 tx=0x7f92ac009060).stop 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4213752345 reap_dead start 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 -- 172.21.15.73:0/4213752345 shutdown_connections 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 -- 172.21.15.73:0/4213752345 >> 172.21.15.73:0/4213752345 conn(0x7f92bc0cb310 msgr2=0x7f92bc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 -- 172.21.15.73:0/4213752345 shutdown_connections 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.561+0000 7f92c3636700 1 -- 172.21.15.73:0/4213752345 wait complete. 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.562+0000 7f92c3636700 1 Processor -- start 2023-11-28T03:05:00.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.562+0000 7f92c3636700 1 -- start start 2023-11-28T03:05:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 0x7f92bc150610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92bc0a94a0 0x7f92bc142f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0a81f0 con 0x7f92bc143450 2023-11-28T03:05:00.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0a9010 con 0x7f92bc0a94a0 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c3636700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92bc0a8400 con 0x7f92a8003e10 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52170/0 (socket says 172.21.15.73:52170) 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4256620258 learned_addr learned my addr 172.21.15.73:0/4256620258 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c0bd1700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92bc0a94a0 0x7f92bc142f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 0x7f92bc150610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 msgr2=0x7f92bc150610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 0x7f92bc150610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92bc0a94a0 msgr2=0x7f92bc142f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92bc0a94a0 0x7f92bc142f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 -- 172.21.15.73:0/4256620258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92bc0a86e0 con 0x7f92bc143450 2023-11-28T03:05:00.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 0x7f92bc150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92c1bd3700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f92b40013d0 tx=0x7f92b4002210).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b400f970 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.563+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b400faf0 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.564+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92bc0a89c0 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.564+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b400feb0 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.564+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92bc150b00 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.565+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92bc0a91a0 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.565+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f92b4015620 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.565+0000 7f92b17fa700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 0x7f92a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.566+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f92b407a070 con 0x7f92bc143450 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.566+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 0x7f92a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.566+0000 7f92c13d2700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 0x7f92a0061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f92b8009270 tx=0x7f92b8008040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.570+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92b404b340 con 0x7f92bc143450 2023-11-28T03:05:00.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.774+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f92bc06c860 con 0x7f92bc143450 2023-11-28T03:05:00.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.782+0000 7f92b17fa700 1 -- 172.21.15.73:0/4256620258 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 7 and data pool 8 v17) v1 ==== 172+0+0 (secure 0 0 0) 0x7f92b401b030 con 0x7f92bc143450 2023-11-28T03:05:00.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 msgr2=0x7f92a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 0x7f92a0061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f92b8009270 tx=0x7f92b8008040).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 msgr2=0x7f92bc0aad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f92b40013d0 tx=0x7f92b4002210).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 shutdown_connections 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92a005f150 0x7f92a0061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92bc143450 0x7f92bc0aad70 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92a8003e10 0x7f92bc150610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 --2- 172.21.15.73:0/4256620258 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92bc0a94a0 0x7f92bc142f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 >> 172.21.15.73:0/4256620258 conn(0x7f92bc0cb310 msgr2=0x7f92bc0c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 shutdown_connections 2023-11-28T03:05:00.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.787+0000 7f92c3636700 1 -- 172.21.15.73:0/4256620258 wait complete. 2023-11-28T03:05:00.807 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 7 and data pool 8 2023-11-28T03:05:00.818 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:05:00.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.931+0000 7f433baa7700 1 Processor -- start 2023-11-28T03:05:00.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.931+0000 7f433baa7700 1 -- start start 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433baa7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43341548c0 0x7f433406a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433baa7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43340c69a0 con 0x7f43340c6e30 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433baa7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43340c13a0 con 0x7f43341544e0 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433baa7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43340c1680 con 0x7f43341548c0 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4339042700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43341548c0 0x7f433406a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433a044700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f43340c6e30 0x7f43340a8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42890/0 (socket says 172.21.15.73:42890) 2023-11-28T03:05:00.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f433a044700 1 -- 172.21.15.73:0/1410231184 learned_addr learned my addr 172.21.15.73:0/1410231184 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4339042700 1 -- 172.21.15.73:0/1410231184 >> v1:172.21.15.150:6789/0 conn(0x7f43341544e0 legacy=0x7f43340c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4339042700 1 -- 172.21.15.73:0/1410231184 >> v1:172.21.15.73:6789/0 conn(0x7f43340c6e30 legacy=0x7f43340a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4338841700 1 -- 172.21.15.73:0/1410231184 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 187797645 0 0) 0x7f43340c69a0 con 0x7f43340c6e30 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4339042700 1 -- 172.21.15.73:0/1410231184 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340c6630 con 0x7f43341548c0 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4339042700 1 --2- 172.21.15.73:0/1410231184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43341548c0 0x7f433406a610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f4328009760 tx=0x7f4328009060).ready entity=mon.0 client_cookie=dc80fef03fc235b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4338841700 1 -- 172.21.15.73:0/1410231184 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800b520 con 0x7f43341548c0 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4338841700 1 -- 172.21.15.73:0/1410231184 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432800b680 con 0x7f43341548c0 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.932+0000 7f4338841700 1 -- 172.21.15.73:0/1410231184 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432800ba00 con 0x7f43341548c0 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 -- 172.21.15.73:0/1410231184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43341548c0 msgr2=0x7f433406a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 --2- 172.21.15.73:0/1410231184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43341548c0 0x7f433406a610 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 rx=0x7f4328009760 tx=0x7f4328009060).stop 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 -- 172.21.15.73:0/1410231184 shutdown_connections 2023-11-28T03:05:00.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 --2- 172.21.15.73:0/1410231184 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43341548c0 0x7f433406a610 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 -- 172.21.15.73:0/1410231184 >> 172.21.15.73:0/1410231184 conn(0x7f4334085020 msgr2=0x7f4334094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:00.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 -- 172.21.15.73:0/1410231184 shutdown_connections 2023-11-28T03:05:00.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.933+0000 7f433baa7700 1 -- 172.21.15.73:0/1410231184 wait complete. 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.934+0000 7f433baa7700 1 Processor -- start 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.934+0000 7f433baa7700 1 -- start start 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43341544e0 0x7f43340c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43341548c0 0x7f4334149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c30a0 con 0x7f43340c6e30 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c24d0 con 0x7f43341548c0 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43340c2770 con 0x7f43341544e0 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52182/0 (socket says 172.21.15.73:52182) 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 -- 172.21.15.73:0/3912121124 learned_addr learned my addr 172.21.15.73:0/3912121124 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433a044700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43341548c0 0x7f4334149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339042700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43341544e0 0x7f43340c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 -- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43341544e0 msgr2=0x7f43340c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43341544e0 0x7f43340c5090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 -- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43341548c0 msgr2=0x7f4334149f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43341548c0 0x7f4334149f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:00.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 -- 172.21.15.73:0/3912121124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43280092a0 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4339843700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f432400c7c0 tx=0x7f4324002620).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432400a580 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f432400a700 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43340c2a50 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.935+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f432400aac0 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.936+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43340c5580 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.937+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f433406c860 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.937+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f432401b400 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.937+0000 7f4331ffb700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 0x7f4320061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.937+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f432407a2d0 con 0x7f43340c6e30 2023-11-28T03:05:00.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.937+0000 7f4339042700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 0x7f4320061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:00.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.938+0000 7f4339042700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 0x7f4320061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f43340c41e0 tx=0x7f4328017000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:00.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:00.942+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f432404b540 con 0x7f43340c6e30 2023-11-28T03:05:01.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.105+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f43340c6250 con 0x7f43340c6e30 2023-11-28T03:05:01.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.106+0000 7f4331ffb700 1 -- 172.21.15.73:0/3912121124 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16941 (secure 0 0 0) 0x7f432404eb60 con 0x7f43340c6e30 2023-11-28T03:05:01.108 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:01.108 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:05:00.776258+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:05:01.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 msgr2=0x7f4320061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 0x7f4320061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f43340c41e0 tx=0x7f4328017000).stop 2023-11-28T03:05:01.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 msgr2=0x7f4334150350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 rx=0x7f432400c7c0 tx=0x7f4324002620).stop 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 shutdown_connections 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432005f150 0x7f4320061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43341548c0 0x7f4334149f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43340c6e30 0x7f4334150350 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 --2- 172.21.15.73:0/3912121124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43341544e0 0x7f43340c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 >> 172.21.15.73:0/3912121124 conn(0x7f4334085020 msgr2=0x7f43340ca9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 shutdown_connections 2023-11-28T03:05:01.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.108+0000 7f433baa7700 1 -- 172.21.15.73:0/3912121124 wait complete. 2023-11-28T03:05:01.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:05:01.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.236+0000 7eff86294700 1 Processor -- start 2023-11-28T03:05:01.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.236+0000 7eff86294700 1 -- start start 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff86294700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff800c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff86294700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff800ca180 0x7eff800c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff86294700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff800a9a90 con 0x7eff800caab0 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff86294700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff800c1490 con 0x7eff800ca180 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff86294700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff80154ce0 con 0x7eff800cae90 2023-11-28T03:05:01.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff7ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7eff800caab0 0x7eff800aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42898/0 (socket says 172.21.15.73:42898) 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff7ffff700 1 -- 172.21.15.73:0/370165462 learned_addr learned my addr 172.21.15.73:0/370165462 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 424432668 0 0) 0x7eff800a9a90 con 0x7eff800caab0 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.237+0000 7eff84831700 1 --2- 172.21.15.73:0/370165462 >> v2:172.21.15.150:3300/0 conn(0x7eff800ca180 0x7eff800c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff680033f0 con 0x7eff800caab0 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff800c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3025194178 0 0) 0x7eff680033f0 con 0x7eff800caab0 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 msgr2=0x7eff800c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff800c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 >> v2:172.21.15.150:3300/0 conn(0x7eff800ca180 msgr2=0x7eff800c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 --2- 172.21.15.73:0/370165462 >> v2:172.21.15.150:3300/0 conn(0x7eff800ca180 0x7eff800c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800aab80 con 0x7eff800caab0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7eff700015f0 con 0x7eff800caab0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 >> v1:172.21.15.73:6789/0 conn(0x7eff800caab0 legacy=0x7eff800aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff680038e0 0x7eff68003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff68004690 0x7eff68004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff68005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c1490 con 0x7eff68004690 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff80154ce0 con 0x7eff680038e0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff6800b8e0 con 0x7eff800cae90 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff68004690 0x7eff68004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff84831700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff68005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7ffff700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff680038e0 0x7eff68003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 -- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 msgr2=0x7eff68005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff800cae90 0x7eff68005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 -- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff680038e0 msgr2=0x7eff68003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff680038e0 0x7eff68003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 -- 172.21.15.73:0/370165462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff680033f0 con 0x7eff68004690 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7ffff700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff680038e0 0x7eff68003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff68004690 0x7eff68004b10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7eff74009a80 tx=0x7eff74009060).ready entity=mon.0 client_cookie=c018fe335150e946 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff74009430 con 0x7eff68004690 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff7400e640 con 0x7eff68004690 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.238+0000 7eff7effd700 1 -- 172.21.15.73:0/370165462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7400e9c0 con 0x7eff68004690 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 -- 172.21.15.73:0/370165462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff68004690 msgr2=0x7eff68004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 --2- 172.21.15.73:0/370165462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff68004690 0x7eff68004b10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 rx=0x7eff74009a80 tx=0x7eff74009060).stop 2023-11-28T03:05:01.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff84831700 1 -- 172.21.15.73:0/370165462 reap_dead start 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 -- 172.21.15.73:0/370165462 shutdown_connections 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 -- 172.21.15.73:0/370165462 >> 172.21.15.73:0/370165462 conn(0x7eff80094920 msgr2=0x7eff8006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 -- 172.21.15.73:0/370165462 shutdown_connections 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.239+0000 7eff86294700 1 -- 172.21.15.73:0/370165462 wait complete. 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.240+0000 7eff86294700 1 Processor -- start 2023-11-28T03:05:01.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.240+0000 7eff86294700 1 -- start start 2023-11-28T03:05:01.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68004690 0x7eff80147020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff80144720 0x7eff800c80f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c2ce0 con 0x7eff800c85e0 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c8a60 con 0x7eff68004690 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff86294700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff800c7a30 con 0x7eff80144720 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52214/0 (socket says 172.21.15.73:52214) 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 -- 172.21.15.73:0/3535512702 learned_addr learned my addr 172.21.15.73:0/3535512702 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff7f7fe700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff80144720 0x7eff800c80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff7ffff700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68004690 0x7eff80147020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 -- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff80144720 msgr2=0x7eff800c80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff80144720 0x7eff800c80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 -- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68004690 msgr2=0x7eff80147020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68004690 0x7eff80147020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.241+0000 7eff84831700 1 -- 172.21.15.73:0/3535512702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff800c23b0 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff84831700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7eff800a9a40 tx=0x7eff7800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff7800e070 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff78011c30 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff800c2690 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff78016450 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.242+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff80147510 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.243+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff8006c860 con 0x7eff800c85e0 2023-11-28T03:05:01.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.243+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff78016800 con 0x7eff800c85e0 2023-11-28T03:05:01.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.243+0000 7eff67fff700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 0x7eff6c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.243+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6121+0+0 (secure 0 0 0) 0x7eff7807a6f0 con 0x7eff800c85e0 2023-11-28T03:05:01.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.244+0000 7eff7ffff700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 0x7eff6c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.244+0000 7eff7ffff700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 0x7eff6c0615e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7eff70000e10 tx=0x7eff70009040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.248+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff7804ba10 con 0x7eff800c85e0 2023-11-28T03:05:01.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.414+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff800a8e80 con 0x7eff800c85e0 2023-11-28T03:05:01.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.415+0000 7eff67fff700 1 -- 172.21.15.73:0/3535512702 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16941 (secure 0 0 0) 0x7eff7804f030 con 0x7eff800c85e0 2023-11-28T03:05:01.416 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:01.416 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:05:00.776258+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:05:01.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 msgr2=0x7eff6c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 0x7eff6c0615e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7eff70000e10 tx=0x7eff70009040).stop 2023-11-28T03:05:01.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 msgr2=0x7eff800c6cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 rx=0x7eff800a9a40 tx=0x7eff7800b040).stop 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 shutdown_connections 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff6c05f120 0x7eff6c0615e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff800c85e0 0x7eff800c6cf0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff68004690 0x7eff80147020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 --2- 172.21.15.73:0/3535512702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff80144720 0x7eff800c80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 >> 172.21.15.73:0/3535512702 conn(0x7eff80094920 msgr2=0x7eff800b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 shutdown_connections 2023-11-28T03:05:01.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.417+0000 7eff86294700 1 -- 172.21.15.73:0/3535512702 wait complete. 2023-11-28T03:05:01.431 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:05:01.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.545+0000 7fdbf2038700 1 Processor -- start 2023-11-28T03:05:01.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.545+0000 7fdbf2038700 1 -- start start 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbf2038700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbec098d10 0x7fdbec0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbf2038700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec158320 0x7fdbec1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbf2038700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbec158cc0 con 0x7fdbec0c6190 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbf2038700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbec0c72d0 con 0x7fdbec098d10 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbf2038700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec064570 con 0x7fdbec158320 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbeb7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbec098d10 0x7fdbec0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbeaffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdbec0c6190 0x7fdbec063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42914/0 (socket says 172.21.15.73:42914) 2023-11-28T03:05:01.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbeaffd700 1 -- 172.21.15.73:0/2382230589 learned_addr learned my addr 172.21.15.73:0/2382230589 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.546+0000 7fdbebfff700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec158320 0x7fdbec1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbeb7fe700 1 -- 172.21.15.73:0/2382230589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec158320 msgr2=0x7fdbec1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbeb7fe700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec158320 0x7fdbec1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbeb7fe700 1 -- 172.21.15.73:0/2382230589 >> v1:172.21.15.73:6789/0 conn(0x7fdbec0c6190 legacy=0x7fdbec063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbea7fc700 1 -- 172.21.15.73:0/2382230589 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470959131 0 0) 0x7fdbe0001110 con 0x7fdbec0c6190 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbeb7fe700 1 -- 172.21.15.73:0/2382230589 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbec154bc0 con 0x7fdbec098d10 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbeb7fe700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec098d10 0x7fdbec0c7760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fdbdc009e60 tx=0x7fdbdc009780).ready entity=mon.0 client_cookie=2dc7ac8368ffdac5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbea7fc700 1 -- 172.21.15.73:0/2382230589 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc00e070 con 0x7fdbec098d10 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbea7fc700 1 -- 172.21.15.73:0/2382230589 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbdc00bb60 con 0x7fdbec098d10 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbea7fc700 1 -- 172.21.15.73:0/2382230589 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbdc012410 con 0x7fdbec098d10 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 -- 172.21.15.73:0/2382230589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec098d10 msgr2=0x7fdbec0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec098d10 0x7fdbec0c7760 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 rx=0x7fdbdc009e60 tx=0x7fdbdc009780).stop 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 -- 172.21.15.73:0/2382230589 shutdown_connections 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec158320 0x7fdbec1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2382230589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec098d10 0x7fdbec0c7760 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 -- 172.21.15.73:0/2382230589 >> 172.21.15.73:0/2382230589 conn(0x7fdbec0cb310 msgr2=0x7fdbec0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 -- 172.21.15.73:0/2382230589 shutdown_connections 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.547+0000 7fdbf2038700 1 -- 172.21.15.73:0/2382230589 wait complete. 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 Processor -- start 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 -- start start 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec098d10 0x7fdbec140aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbec158320 0x7fdbec141630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0af070 con 0x7fdbec0c6190 2023-11-28T03:05:01.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0ae460 con 0x7fdbec158320 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.549+0000 7fdbf2038700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbec0ae740 con 0x7fdbec098d10 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52228/0 (socket says 172.21.15.73:52228) 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 -- 172.21.15.73:0/2647843331 learned_addr learned my addr 172.21.15.73:0/2647843331 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeb7fe700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec098d10 0x7fdbec140aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbebfff700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbec158320 0x7fdbec141630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 -- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec098d10 msgr2=0x7fdbec140aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec098d10 0x7fdbec140aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 -- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbec158320 msgr2=0x7fdbec141630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbec158320 0x7fdbec141630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 -- 172.21.15.73:0/2647843331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbdc009a20 con 0x7fdbec0c6190 2023-11-28T03:05:01.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbeaffd700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fdbe000c4e0 tx=0x7fdbe0007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00072e0 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe0007440 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe00057a0 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbec0aea20 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.550+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbec0a99f0 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.551+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbec098960 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.551+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdbe0005b90 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.552+0000 7fdbd37fe700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 0x7fdbd8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.552+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdbe0079720 con 0x7fdbec0c6190 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.552+0000 7fdbeb7fe700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 0x7fdbd8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:01.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.552+0000 7fdbeb7fe700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 0x7fdbd8061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdbdc0097c0 tx=0x7fdbec0a8040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:01.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.556+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdbe004a9f0 con 0x7fdbec0c6190 2023-11-28T03:05:01.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:01.761+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdbec158cc0 con 0x7fdbec0c6190 2023-11-28T03:05:02.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.427+0000 7fdbd37fe700 1 -- 172.21.15.73:0/2647843331 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v19) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdbec158cc0 con 0x7fdbec0c6190 2023-11-28T03:05:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 msgr2=0x7fdbd8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 0x7fdbd8061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fdbdc0097c0 tx=0x7fdbec0a8040).stop 2023-11-28T03:05:02.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 msgr2=0x7fdbec141140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 rx=0x7fdbe000c4e0 tx=0x7fdbe0007ab0).stop 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 shutdown_connections 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbd805f150 0x7fdbd8061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbec158320 0x7fdbec141630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbec098d10 0x7fdbec140aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 --2- 172.21.15.73:0/2647843331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbec0c6190 0x7fdbec141140 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 >> 172.21.15.73:0/2647843331 conn(0x7fdbec0cb310 msgr2=0x7fdbec0ca7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 shutdown_connections 2023-11-28T03:05:02.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.429+0000 7fdbf2038700 1 -- 172.21.15.73:0/2647843331 wait complete. 2023-11-28T03:05:02.443 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:05:02.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:05:02.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.561+0000 7fc561967700 1 Processor -- start 2023-11-28T03:05:02.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.561+0000 7fc561967700 1 -- start start 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc561967700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc561967700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc561967700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc55c1527e0 con 0x7fc55c069a60 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc561967700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc55c0bea70 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc561967700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c6060 con 0x7fc55c151130 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55b7fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55affd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55affd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35370/0 (socket says 172.21.15.73:35370) 2023-11-28T03:05:02.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55a7fc700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc55c069a60 0x7fc55c069ee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43190/0 (socket says 172.21.15.73:43190) 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55affd700 1 -- 172.21.15.73:0/3462439311 learned_addr learned my addr 172.21.15.73:0/3462439311 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.562+0000 7fc55b7fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:43886/0 (socket says 172.21.15.73:43886) 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc559ffb700 1 -- 172.21.15.73:0/3462439311 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001518000 0 0) 0x7fc55c1527e0 con 0x7fc55c069a60 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc559ffb700 1 -- 172.21.15.73:0/3462439311 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5440033f0 con 0x7fc55c069a60 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55affd700 1 -- 172.21.15.73:0/3462439311 >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 msgr2=0x7fc55c0bd570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55affd700 1 --2- 172.21.15.73:0/3462439311 >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55affd700 1 -- 172.21.15.73:0/3462439311 >> v1:172.21.15.150:6789/0 conn(0x7fc55c069a60 legacy=0x7fc55c069ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55affd700 1 -- 172.21.15.73:0/3462439311 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc55c151bd0 con 0x7fc55c151130 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55b7fe700 1 --2- 172.21.15.73:0/3462439311 >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc55affd700 1 --2- 172.21.15.73:0/3462439311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc54c00d1f0 tx=0x7fc54c00ba50).ready entity=mon.2 client_cookie=821ffdd27c24675e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc559ffb700 1 -- 172.21.15.73:0/3462439311 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c010070 con 0x7fc55c151130 2023-11-28T03:05:02.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc559ffb700 1 -- 172.21.15.73:0/3462439311 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc54c00ddb0 con 0x7fc55c151130 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc559ffb700 1 -- 172.21.15.73:0/3462439311 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc54c00b2e0 con 0x7fc55c151130 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc561967700 1 -- 172.21.15.73:0/3462439311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 msgr2=0x7fc55c151510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.563+0000 7fc561967700 1 --2- 172.21.15.73:0/3462439311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7fc54c00d1f0 tx=0x7fc54c00ba50).stop 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 -- 172.21.15.73:0/3462439311 shutdown_connections 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 --2- 172.21.15.73:0/3462439311 >> v2:172.21.15.150:3300/0 conn(0x7fc55c0bd0c0 0x7fc55c0bd570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 --2- 172.21.15.73:0/3462439311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c151510 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 -- 172.21.15.73:0/3462439311 >> 172.21.15.73:0/3462439311 conn(0x7fc55c0c4c00 msgr2=0x7fc55c0c5010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 -- 172.21.15.73:0/3462439311 shutdown_connections 2023-11-28T03:05:02.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.564+0000 7fc561967700 1 -- 172.21.15.73:0/3462439311 wait complete. 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.565+0000 7fc561967700 1 Processor -- start 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.565+0000 7fc561967700 1 -- start start 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55c069a60 0x7fc55c0c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c161230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c1b00 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c0c10 con 0x7fc55c069a60 2023-11-28T03:05:02.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc55c0c0ef0 con 0x7fc55c151130 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52236/0 (socket says 172.21.15.73:52236) 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55affd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55c069a60 0x7fc55c0c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 -- 172.21.15.73:0/1501937089 learned_addr learned my addr 172.21.15.73:0/1501937089 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55b7fe700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 -- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 msgr2=0x7fc55c161230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c161230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 -- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55c069a60 msgr2=0x7fc55c0c1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55c069a60 0x7fc55c0c1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 -- 172.21.15.73:0/1501937089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5540092a0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc55a7fc700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc55000c4c0 tx=0x7fc550007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5500072e0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc550007440 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc54c00bcf0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc550005910 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.566+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc55c0c14b0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.567+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc55c06c830 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.568+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc550005d40 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.568+0000 7fc542ffd700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 0x7fc5480615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.568+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc5500799f0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.568+0000 7fc55affd700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 0x7fc5480615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:02.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.569+0000 7fc55affd700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 0x7fc5480615e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc54c00bb40 tx=0x7fc54c00ba50).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:02.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.572+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc55004acc0 con 0x7fc55c0bd0c0 2023-11-28T03:05:02.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:02.776+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc55c0aacc0 con 0x7fc55c0bd0c0 2023-11-28T03:05:03.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.429+0000 7fc542ffd700 1 -- 172.21.15.73:0/1501937089 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v20) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc55004e2e0 con 0x7fc55c0bd0c0 2023-11-28T03:05:03.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 msgr2=0x7fc5480615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 0x7fc5480615e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc54c00bb40 tx=0x7fc54c00ba50).stop 2023-11-28T03:05:03.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 msgr2=0x7fc55c14cec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 rx=0x7fc55000c4c0 tx=0x7fc550007c50).stop 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 shutdown_connections 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc54805f120 0x7fc5480615e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc55c151130 0x7fc55c161230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc55c069a60 0x7fc55c0c1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 --2- 172.21.15.73:0/1501937089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc55c0bd0c0 0x7fc55c14cec0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 >> 172.21.15.73:0/1501937089 conn(0x7fc55c0c4c00 msgr2=0x7fc55c0aa400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 shutdown_connections 2023-11-28T03:05:03.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.431+0000 7fc561967700 1 -- 172.21.15.73:0/1501937089 wait complete. 2023-11-28T03:05:03.445 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:05:03.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:05:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.559+0000 7f7d8f59e700 1 Processor -- start 2023-11-28T03:05:03.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.559+0000 7f7d8f59e700 1 -- start start 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8f59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7d90157660 0x7f7d90157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8f59e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900c12c0 con 0x7f7d900c0180 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8f59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d900c06b0 con 0x7f7d900c1750 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8f59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d900c0990 con 0x7f7d90157660 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8dd9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7d900c0180 0x7f7d901589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57382/0 (socket says 172.21.15.73:57382) 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8dd9b700 1 -- 172.21.15.73:0/2774022266 learned_addr learned my addr 172.21.15.73:0/2774022266 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8e59c700 1 --2- 172.21.15.73:0/2774022266 >> v2:172.21.15.150:3300/0 conn(0x7f7d90157660 0x7f7d90157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213123585 0 0) 0x7f7d900c12c0 con 0x7f7d900c0180 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d780033f0 con 0x7f7d900c0180 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2799509052 0 0) 0x7f7d780033f0 con 0x7f7d900c0180 2023-11-28T03:05:03.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 >> v2:172.21.15.150:3300/0 conn(0x7f7d90157660 msgr2=0x7f7d90157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 --2- 172.21.15.73:0/2774022266 >> v2:172.21.15.150:3300/0 conn(0x7f7d90157660 0x7f7d90157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 >> v1:172.21.15.150:6789/0 conn(0x7f7d900c1750 legacy=0x7f7d900648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900c0c70 con 0x7f7d900c0180 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7d84001470 con 0x7f7d900c0180 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.560+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 >> v1:172.21.15.73:6789/0 conn(0x7f7d900c0180 legacy=0x7f7d901589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d780038e0 0x7f7d78003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d78004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d78005000 0x7f7d7800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900c0990 con 0x7f7d78004690 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7800bf60 con 0x7f7d78005000 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d7800c0f0 con 0x7f7d780038e0 2023-11-28T03:05:03.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d78004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8e59c700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d780038e0 0x7f7d78003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8ed9d700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d78005000 0x7f7d7800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 -- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d780038e0 msgr2=0x7f7d78003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d780038e0 0x7f7d78003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 -- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d78005000 msgr2=0x7f7d7800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d78005000 0x7f7d7800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 -- 172.21.15.73:0/2774022266 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d780033f0 con 0x7f7d78004690 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8ed9d700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d78005000 0x7f7d7800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8ed9d700 1 -- 172.21.15.73:0/2774022266 reap_dead start 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d78004b10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f7d8400c190 tx=0x7f7d84001e50).ready entity=mon.0 client_cookie=53dfaf44b55a5594 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d84007a20 con 0x7f7d78004690 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8400cac0 con 0x7f7d78004690 2023-11-28T03:05:03.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.561+0000 7f7d8d59a700 1 -- 172.21.15.73:0/2774022266 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8400ce40 con 0x7f7d78004690 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 msgr2=0x7f7d78004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d78004b10 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 rx=0x7f7d8400c190 tx=0x7f7d84001e50).stop 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2774022266 shutdown_connections 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2774022266 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d78004b10 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2774022266 >> 172.21.15.73:0/2774022266 conn(0x7f7d900cb000 msgr2=0x7f7d900cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2774022266 shutdown_connections 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.562+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2774022266 wait complete. 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.563+0000 7f7d8f59e700 1 Processor -- start 2023-11-28T03:05:03.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.563+0000 7f7d8f59e700 1 -- start start 2023-11-28T03:05:03.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 0x7f7d90151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d900ae410 0x7f7d90151fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d900c2220 con 0x7f7d78004690 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90168840 con 0x7f7d900adfb0 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d90168b20 con 0x7f7d900ae410 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54432/0 (socket says 172.21.15.73:54432) 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 -- 172.21.15.73:0/2264961493 learned_addr learned my addr 172.21.15.73:0/2264961493 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8ed9d700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d900ae410 0x7f7d90151fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 0x7f7d90151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 -- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d900ae410 msgr2=0x7f7d90151fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d900ae410 0x7f7d90151fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 -- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 msgr2=0x7f7d90151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 0x7f7d90151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 -- 172.21.15.73:0/2264961493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d900c1bd0 con 0x7f7d78004690 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 0x7f7d90151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8e59c700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7d800119f0 tx=0x7f7d80009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000f940 con 0x7f7d78004690 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d8000faa0 con 0x7f7d78004690 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d900c1eb0 con 0x7f7d78004690 2023-11-28T03:05:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d8000fe20 con 0x7f7d78004690 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.564+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d90151720 con 0x7f7d78004690 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.565+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d9006c7c0 con 0x7f7d78004690 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.566+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7d80014650 con 0x7f7d78004690 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.566+0000 7f7d7e7fc700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 0x7f7d70061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.566+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7d8007ae50 con 0x7f7d78004690 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.566+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 0x7f7d70061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.566+0000 7f7d8dd9b700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 0x7f7d70061640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7d84001ed0 tx=0x7f7d84015000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.570+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d8004c120 con 0x7f7d78004690 2023-11-28T03:05:03.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.773+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d900c0f50 con 0x7f7d78004690 2023-11-28T03:05:03.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.774+0000 7f7d7e7fc700 1 -- 172.21.15.73:0/2264961493 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f7d8004f740 con 0x7f7d78004690 2023-11-28T03:05:03.775 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:03.775 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:05:03.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 msgr2=0x7f7d70061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 0x7f7d70061640 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f7d84001ed0 tx=0x7f7d84015000).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 msgr2=0x7f7d90151230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 rx=0x7f7d800119f0 tx=0x7f7d80009060).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 shutdown_connections 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d7005f180 0x7f7d70061640 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d900ae410 0x7f7d90151fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d78004690 0x7f7d90151230 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 --2- 172.21.15.73:0/2264961493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d900adfb0 0x7f7d90151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 >> 172.21.15.73:0/2264961493 conn(0x7f7d900cb000 msgr2=0x7f7d900aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 shutdown_connections 2023-11-28T03:05:03.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.775+0000 7f7d8f59e700 1 -- 172.21.15.73:0/2264961493 wait complete. 2023-11-28T03:05:03.778 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:05:03.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:05:03.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 Processor -- start 2023-11-28T03:05:03.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 -- start start 2023-11-28T03:05:03.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a64065760 con 0x7f8a641542d0 2023-11-28T03:05:03.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a64063040 con 0x7f8a64062700 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.901+0000 7f8a6a99a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a640c3250 con 0x7f8a64062bb0 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a63fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8a641542d0 0x7f8a640697f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57394/0 (socket says 172.21.15.73:57394) 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a63fff700 1 -- 172.21.15.73:0/3917102598 learned_addr learned my addr 172.21.15.73:0/3917102598 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a68f37700 1 --2- 172.21.15.73:0/3917102598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3664234251 0 0) 0x7f8a64065760 con 0x7f8a641542d0 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a4c0033f0 con 0x7f8a641542d0 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1535209058 0 0) 0x7f8a64063040 con 0x7f8a64062700 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a64065760 con 0x7f8a64062700 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4275936268 0 0) 0x7f8a4c0033f0 con 0x7f8a641542d0 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a68f37700 1 -- 172.21.15.73:0/3917102598 >> v1:172.21.15.150:6789/0 conn(0x7f8a64062700 legacy=0x7f8a640661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a68f37700 1 -- 172.21.15.73:0/3917102598 >> v1:172.21.15.73:6789/0 conn(0x7f8a641542d0 legacy=0x7f8a640697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a68f37700 1 -- 172.21.15.73:0/3917102598 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a6406a340 con 0x7f8a64062bb0 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a68f37700 1 --2- 172.21.15.73:0/3917102598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640668c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8a5c009e90 tx=0x7f8a5c009780).ready entity=mon.2 client_cookie=92c5f46cb94ed09b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a5c00e070 con 0x7f8a64062bb0 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a5c00b7a0 con 0x7f8a64062bb0 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.902+0000 7f8a62ffd700 1 -- 172.21.15.73:0/3917102598 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54001470 con 0x7f8a64062bb0 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3917102598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 msgr2=0x7f8a640668c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3917102598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640668c0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f8a5c009e90 tx=0x7f8a5c009780).stop 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3917102598 shutdown_connections 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3917102598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640668c0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3917102598 >> 172.21.15.73:0/3917102598 conn(0x7f8a640c7fd0 msgr2=0x7f8a640c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:03.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3917102598 shutdown_connections 2023-11-28T03:05:03.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.903+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3917102598 wait complete. 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.904+0000 7f8a6a99a700 1 Processor -- start 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.904+0000 7f8a6a99a700 1 -- start start 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 0x7f8a640a8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a641523a0 con 0x7f8a64062700 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a641525f0 con 0x7f8a641542d0 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a6a99a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a641528d0 con 0x7f8a64062bb0 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54434/0 (socket says 172.21.15.73:54434) 2023-11-28T03:05:03.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 -- 172.21.15.73:0/3576466304 learned_addr learned my addr 172.21.15.73:0/3576466304 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a637fe700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a68f37700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 0x7f8a640a8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 -- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 msgr2=0x7f8a640a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 -- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 msgr2=0x7f8a640a8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 0x7f8a640a8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a63fff700 1 -- 172.21.15.73:0/3576466304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a5c009a20 con 0x7f8a64062700 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a68f37700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 0x7f8a640a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.905+0000 7f8a637fe700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a63fff700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f8a5400c810 tx=0x7f8a540019e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a54007170 con 0x7f8a64062700 2023-11-28T03:05:03.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a540072f0 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a64152bb0 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a540048c0 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.906+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a640a95b0 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.907+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6406c860 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.907+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8a54004c70 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.907+0000 7f8a4bfff700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 0x7f8a50061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.907+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8a54079e20 con 0x7f8a64062700 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.908+0000 7f8a637fe700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 0x7f8a50061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.908+0000 7f8a637fe700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 0x7f8a50061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8a64067aa0 tx=0x7f8a58009040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:03.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:03.911+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8a5404b090 con 0x7f8a64062700 2023-11-28T03:05:04.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.075+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8a6406a620 con 0x7f8a64062700 2023-11-28T03:05:04.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.076+0000 7f8a4bfff700 1 -- 172.21.15.73:0/3576466304 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) v1 ==== 74+0+16928 (secure 0 0 0) 0x7f8a54019030 con 0x7f8a64062700 2023-11-28T03:05:04.077 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:04.078 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":65,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:05:01.778251+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:05:04.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 msgr2=0x7f8a50061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 0x7f8a50061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f8a64067aa0 tx=0x7f8a58009040).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 msgr2=0x7f8a640a7ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 rx=0x7f8a5400c810 tx=0x7f8a540019e0).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 shutdown_connections 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a5005f150 0x7f8a50061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a641542d0 0x7f8a640a8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a64062700 0x7f8a640a7ab0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 --2- 172.21.15.73:0/3576466304 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a64062bb0 0x7f8a640a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 >> 172.21.15.73:0/3576466304 conn(0x7f8a640c7fd0 msgr2=0x7f8a6414d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 shutdown_connections 2023-11-28T03:05:04.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.078+0000 7f8a6a99a700 1 -- 172.21.15.73:0/3576466304 wait complete. 2023-11-28T03:05:04.091 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:05:04.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:05:04.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.202+0000 7f6158069700 1 Processor -- start 2023-11-28T03:05:04.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.202+0000 7f6158069700 1 -- start start 2023-11-28T03:05:04.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.203+0000 7f6158069700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f61500aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.203+0000 7f6158069700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f61500ca180 0x7f615015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.203+0000 7f6158069700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61500a9a90 con 0x7f61500cae90 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.203+0000 7f6158069700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6150154ce0 con 0x7f61500ca180 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.203+0000 7f6158069700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61500a8e80 con 0x7f61500caab0 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6155604700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f61500cae90 0x7f61500c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57396/0 (socket says 172.21.15.73:57396) 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6155604700 1 -- 172.21.15.73:0/3247287578 learned_addr learned my addr 172.21.15.73:0/3247287578 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 --2- 172.21.15.73:0/3247287578 >> v2:172.21.15.150:3300/0 conn(0x7f61500ca180 0x7f615015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6155e05700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f61500aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6154e03700 1 -- 172.21.15.73:0/3247287578 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3136233169 0 0) 0x7f61500a9a90 con 0x7f61500cae90 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6154e03700 1 -- 172.21.15.73:0/3247287578 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61380033f0 con 0x7f61500cae90 2023-11-28T03:05:04.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 -- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 msgr2=0x7f61500aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f61500aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 -- 172.21.15.73:0/3247287578 >> v1:172.21.15.73:6789/0 conn(0x7f61500cae90 legacy=0x7f61500c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 -- 172.21.15.73:0/3247287578 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61500a9160 con 0x7f61500ca180 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6156606700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f615015e3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f6148009760 tx=0x7f6148009060).ready entity=mon.1 client_cookie=33e48a68f2e475e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6154e03700 1 -- 172.21.15.73:0/3247287578 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800d4a0 con 0x7f61500ca180 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6154e03700 1 -- 172.21.15.73:0/3247287578 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614800d600 con 0x7f61500ca180 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.204+0000 7f6154e03700 1 -- 172.21.15.73:0/3247287578 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800d980 con 0x7f61500ca180 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 -- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 msgr2=0x7f615015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f615015e3c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7f6148009760 tx=0x7f6148009060).stop 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 -- 172.21.15.73:0/3247287578 shutdown_connections 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f615015e3c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 --2- 172.21.15.73:0/3247287578 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f61500aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 -- 172.21.15.73:0/3247287578 >> 172.21.15.73:0/3247287578 conn(0x7f6150094920 msgr2=0x7f615006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 -- 172.21.15.73:0/3247287578 shutdown_connections 2023-11-28T03:05:04.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.205+0000 7f6158069700 1 -- 172.21.15.73:0/3247287578 wait complete. 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.206+0000 7f6158069700 1 Processor -- start 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.206+0000 7f6158069700 1 -- start start 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f6150155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f6150156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61500ae220 con 0x7f61500cae90 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61500aed30 con 0x7f61500ca180 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6158069700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61500ae400 con 0x7f61500caab0 2023-11-28T03:05:04.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54444/0 (socket says 172.21.15.73:54444) 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 -- 172.21.15.73:0/892520545 learned_addr learned my addr 172.21.15.73:0/892520545 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6155604700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f6150156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6155e05700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f6150155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 -- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 msgr2=0x7f6150156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f6150156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 -- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 msgr2=0x7f6150155bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f6150155bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6156606700 1 -- 172.21.15.73:0/892520545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61480092a0 con 0x7f61500cae90 2023-11-28T03:05:04.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6155604700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f6150156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.207+0000 7f6155e05700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f6150155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f6156606700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f6148009d50 tx=0x7f6148016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614800d980 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614800de00 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61500ae6e0 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614801d640 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.208+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6150158740 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.209+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61500ca560 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.209+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f614801d9b0 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.210+0000 7f614dffb700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 0x7f613c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.210+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f614807bbb0 con 0x7f61500cae90 2023-11-28T03:05:04.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.210+0000 7f6155e05700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 0x7f613c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.210+0000 7f6155e05700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 0x7f613c0615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f61500c8a10 tx=0x7f614000b040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.214+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f614804ce80 con 0x7f61500cae90 2023-11-28T03:05:04.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.372+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f615006c860 con 0x7f613c05f120 2023-11-28T03:05:04.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.382+0000 7f614dffb700 1 -- 172.21.15.73:0/892520545 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350360 (secure 0 0 0) 0x7f615006c860 con 0x7f613c05f120 2023-11-28T03:05:04.387 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 msgr2=0x7f613c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 0x7f613c0615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f61500c8a10 tx=0x7f614000b040).stop 2023-11-28T03:05:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 msgr2=0x7f6150152340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 rx=0x7f6148009d50 tx=0x7f6148016000).stop 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 shutdown_connections 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f613c05f120 0x7f613c0615e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61500cae90 0x7f6150152340 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61500ca180 0x7f6150155bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 --2- 172.21.15.73:0/892520545 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61500caab0 0x7f6150156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.385+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 >> 172.21.15.73:0/892520545 conn(0x7f6150094920 msgr2=0x7f6150064b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.386+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 shutdown_connections 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.386+0000 7f6158069700 1 -- 172.21.15.73:0/892520545 wait complete. 2023-11-28T03:05:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:05:04.399 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T03:05:02.869418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2407008,"kb_used_data":5664,"kb_used_omap":0,"kb_used_meta":2401280,"kb_avail":752567712,"statfs":{"total":773094113280,"available":770629337088,"internally_reserved":0,"allocated":5799936,"data_stored":2824072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2458910720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14996258,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-721,"num_write_kb":-15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.498335"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336636+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:04:55.336636+0000","last_peered":"2023-11-28T03:04:55.336636+0000","last_clean":"2023-11-28T03:04:55.336636+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:04:55.336636+0000","last_undegraded":"2023-11-28T03:04:55.336636+0000","last_fullsized":"2023-11-28T03:04:55.336636+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230109+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:04:55.230109+0000","last_peered":"2023-11-28T03:04:55.230109+0000","last_clean":"2023-11-28T03:04:55.230109+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:04:55.230109+0000","last_undegraded":"2023-11-28T03:04:55.230109+0000","last_fullsized":"2023-11-28T03:04:55.230109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.255301+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:04:55.255301+0000","last_peered":"2023-11-28T03:04:55.255301+0000","last_clean":"2023-11-28T03:04:55.255301+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:04:55.255301+0000","last_undegraded":"2023-11-28T03:04:55.255301+0000","last_fullsized":"2023-11-28T03:04:55.255301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230025+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:04:55.230025+0000","last_peered":"2023-11-28T03:04:55.230025+0000","last_clean":"2023-11-28T03:04:55.230025+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:04:55.230025+0000","last_undegraded":"2023-11-28T03:04:55.230025+0000","last_fullsized":"2023-11-28T03:04:55.230025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336901+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:04:55.336901+0000","last_peered":"2023-11-28T03:04:55.336901+0000","last_clean":"2023-11-28T03:04:55.336901+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:04:55.336901+0000","last_undegraded":"2023-11-28T03:04:55.336901+0000","last_fullsized":"2023-11-28T03:04:55.336901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336895+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:04:55.336895+0000","last_peered":"2023-11-28T03:04:55.336895+0000","last_clean":"2023-11-28T03:04:55.336895+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:04:55.336895+0000","last_undegraded":"2023-11-28T03:04:55.336895+0000","last_fullsized":"2023-11-28T03:04:55.336895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336928+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:04:55.336928+0000","last_peered":"2023-11-28T03:04:55.336928+0000","last_clean":"2023-11-28T03:04:55.336928+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:04:55.336928+0000","last_undegraded":"2023-11-28T03:04:55.336928+0000","last_fullsized":"2023-11-28T03:04:55.336928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336677+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:04:55.336677+0000","last_peered":"2023-11-28T03:04:55.336677+0000","last_clean":"2023-11-28T03:04:55.336677+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:04:55.336677+0000","last_undegraded":"2023-11-28T03:04:55.336677+0000","last_fullsized":"2023-11-28T03:04:55.336677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230025+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:04:55.230025+0000","last_peered":"2023-11-28T03:04:55.230025+0000","last_clean":"2023-11-28T03:04:55.230025+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:04:55.230025+0000","last_undegraded":"2023-11-28T03:04:55.230025+0000","last_fullsized":"2023-11-28T03:04:55.230025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73799999999999999,"5min":0.82399999999999995,"15min":0.85999999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.35499999999999998,"15min":0.32000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.42399999999999999,"5min":0.36899999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":0.749,"15min":0.749},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.77100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.495}]},{"osd":4,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.69699999999999995,"5min":0.77800000000000002,"15min":0.83099999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.496},"min":{"1min":0.32500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.81899999999999995},"last":0.32400000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48999999999999999,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.313},"max":{"1min":0.69099999999999995,"5min":0.72799999999999998,"15min":0.84499999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.34100000000000003},"max":{"1min":0.73899999999999999,"5min":0.83099999999999996,"15min":0.83199999999999996},"last":0.40000000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.84499999999999997},"last":0.48599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.496},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.379,"15min":0.32800000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.81000000000000005,"15min":0.84399999999999997},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.746,"15min":0.82499999999999996},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.317},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36099999999999999,"15min":0.33700000000000002},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.20599999999999999},"max":{"1min":0.46700000000000003,"5min":0.67700000000000005,"15min":0.70699999999999996},"last":0.245},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.36299999999999999},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.20300000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.30399999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40100000000000002,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.73499999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.42499999999999999,"15min":0.38800000000000001},"min":{"1min":0.32000000000000001,"5min":0.27600000000000002,"15min":0.19800000000000001},"max":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.72599999999999998},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.44},"min":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.749},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.433},"min":{"1min":0.376,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":1.6870000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83299999999999996,"5min":0.83899999999999997,"15min":1.0900000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.28699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.42399999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.505},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.49099999999999999,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.111},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.8080000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64300000000000002,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.073},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64200000000000002,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.48599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35499999999999998,"15min":0.33900000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.184},"max":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.75700000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.63,"15min":0.63},"last":0.317}]},{"osd":2,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.20100000000000001},"max":{"1min":0.48399999999999999,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.27100000000000002,"15min":0.26300000000000001},"max":{"1min":0.58599999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40699999999999997,"15min":0.38700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.434,"15min":0.41199999999999998},"min":{"1min":0.33900000000000002,"5min":0.312,"15min":0.251},"max":{"1min":0.54300000000000004,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.47699999999999998,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.48999999999999999,"5min":0.443,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":0.85499999999999998,"15min":0.91200000000000003},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94299999999999995},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.502,"5min":0.46899999999999997,"15min":0.36299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.374,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.20499999999999999},"max":{"1min":0.41099999999999998,"5min":0.57399999999999995,"15min":0.623},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.42199999999999999,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23599999999999999},"max":{"1min":0.47599999999999998,"5min":0.64700000000000002,"15min":0.65900000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.377},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51100000000000001,"5min":0.68600000000000005,"15min":0.73099999999999998},"last":0.32600000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.40600000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53600000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":1.004,"15min":1.004},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74399999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36899999999999999},"max":{"1min":0.73699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.45600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36799999999999999,"15min":0.34699999999999998},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.501,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38900000000000001,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.19900000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.60899999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.44800000000000001,"15min":0.42599999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.68200000000000005},"last":0.375}]},{"osd":4,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6000000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.621,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70699999999999996,"5min":0.94199999999999995,"15min":1.196},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.996,"15min":1.206},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.97199999999999998,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.56499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.372,"15min":0.27300000000000002},"max":{"1min":0.73099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.32800000000000001,"15min":0.314},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.30199999999999999},"max":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.38,"15min":0.33500000000000002},"max":{"1min":0.73799999999999999,"5min":0.76300000000000001,"15min":0.78100000000000003},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.39000000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.878,"15min":0.878},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.69199999999999995,"15min":0.71699999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.622,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.753}]},{"osd":7,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69199999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59599999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.377,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.76400000000000001,"15min":0.79800000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78700000000000003},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42599999999999999,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:05:04.406 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:05:04.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.550+0000 7fc51b610700 1 Processor -- start 2023-11-28T03:05:04.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.550+0000 7fc51b610700 1 -- start start 2023-11-28T03:05:04.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5140caab0 0x7fc5140aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc5140c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc5140aa960 0x7fc5140c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140c1490 con 0x7fc5140caab0 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5140a9a90 con 0x7fc5140aa960 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc51b610700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc514154ce0 con 0x7fc5140cae90 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc5193ac700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5140caab0 0x7fc5140aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc5193ac700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc5140caab0 0x7fc5140aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54452/0 (socket says 172.21.15.73:54452) 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.551+0000 7fc5193ac700 1 -- 172.21.15.73:0/609194638 learned_addr learned my addr 172.21.15.73:0/609194638 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:04.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc518bab700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc5140c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc519bad700 1 --2- 172.21.15.73:0/609194638 >> v2:172.21.15.150:3300/0 conn(0x7fc5140aa960 0x7fc5140c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 -- 172.21.15.73:0/609194638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 msgr2=0x7fc5140c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc5140c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 -- 172.21.15.73:0/609194638 >> v2:172.21.15.150:3300/0 conn(0x7fc5140aa960 msgr2=0x7fc5140c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 --2- 172.21.15.73:0/609194638 >> v2:172.21.15.150:3300/0 conn(0x7fc5140aa960 0x7fc5140c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 -- 172.21.15.73:0/609194638 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5140a8e80 con 0x7fc5140caab0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc5193ac700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5140aa470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fc51405e5a0 tx=0x7fc510009840).ready entity=mon.0 client_cookie=b379ce3b35c0d4fd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc50bfff700 1 -- 172.21.15.73:0/609194638 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000e070 con 0x7fc5140caab0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc50bfff700 1 -- 172.21.15.73:0/609194638 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000bad0 con 0x7fc5140caab0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc50bfff700 1 -- 172.21.15.73:0/609194638 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51000be50 con 0x7fc5140caab0 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 -- 172.21.15.73:0/609194638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 msgr2=0x7fc5140aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5140aa470 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 rx=0x7fc51405e5a0 tx=0x7fc510009840).stop 2023-11-28T03:05:04.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 -- 172.21.15.73:0/609194638 shutdown_connections 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5140aa470 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 --2- 172.21.15.73:0/609194638 >> v2:172.21.15.150:3300/0 conn(0x7fc5140aa960 0x7fc5140c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 --2- 172.21.15.73:0/609194638 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc5140c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.552+0000 7fc51b610700 1 -- 172.21.15.73:0/609194638 >> 172.21.15.73:0/609194638 conn(0x7fc514094920 msgr2=0x7fc51406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.553+0000 7fc51b610700 1 -- 172.21.15.73:0/609194638 shutdown_connections 2023-11-28T03:05:04.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.553+0000 7fc51b610700 1 -- 172.21.15.73:0/609194638 wait complete. 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.554+0000 7fc51b610700 1 Processor -- start 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.554+0000 7fc51b610700 1 -- start start 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.554+0000 7fc51b610700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.554+0000 7fc51b610700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc514164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.554+0000 7fc51b610700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc514150780 0x7fc514157520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc51b610700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc514156db0 con 0x7fc5140caab0 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc51b610700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc514165230 con 0x7fc514150780 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc51b610700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc514165510 con 0x7fc5140cae90 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54464/0 (socket says 172.21.15.73:54464) 2023-11-28T03:05:04.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 -- 172.21.15.73:0/2559467016 learned_addr learned my addr 172.21.15.73:0/2559467016 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc519bad700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc514150780 0x7fc514157520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc518bab700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc514164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 -- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 msgr2=0x7fc514164a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc514164a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 -- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc514150780 msgr2=0x7fc514157520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc514150780 0x7fc514157520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 -- 172.21.15.73:0/2559467016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc510009ae0 con 0x7fc5140caab0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5193ac700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fc514083c30 tx=0x7fc510009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5100092e0 con 0x7fc5140caab0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc51000bc40 con 0x7fc5140caab0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5141657f0 con 0x7fc5140caab0 2023-11-28T03:05:04.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc51001e6c0 con 0x7fc5140caab0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.555+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc514164f10 con 0x7fc5140caab0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.556+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc514157240 con 0x7fc5140caab0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.557+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc510003040 con 0x7fc5140caab0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.557+0000 7fc5097fa700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 0x7fc4f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.557+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc51007b550 con 0x7fc5140caab0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.557+0000 7fc518bab700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 0x7fc4f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.557+0000 7fc518bab700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 0x7fc4f8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc5140aef80 tx=0x7fc504008040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.561+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc51004c8d0 con 0x7fc5140caab0 2023-11-28T03:05:04.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.720+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc51406c860 con 0x7fc4f805f150 2023-11-28T03:05:04.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.729+0000 7fc5097fa700 1 -- 172.21.15.73:0/2559467016 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350360 (secure 0 0 0) 0x7fc51406c860 con 0x7fc4f805f150 2023-11-28T03:05:04.734 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 msgr2=0x7fc4f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 0x7fc4f8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fc5140aef80 tx=0x7fc504008040).stop 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 msgr2=0x7fc5141559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 rx=0x7fc514083c30 tx=0x7fc510009840).stop 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 shutdown_connections 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc4f805f150 0x7fc4f8061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc514150780 0x7fc514157520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc5140caab0 0x7fc5141559e0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 --2- 172.21.15.73:0/2559467016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc5140cae90 0x7fc514164a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 >> 172.21.15.73:0/2559467016 conn(0x7fc514094920 msgr2=0x7fc5140a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 shutdown_connections 2023-11-28T03:05:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.733+0000 7fc51b610700 1 -- 172.21.15.73:0/2559467016 wait complete. 2023-11-28T03:05:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:05:04.746 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-11-28T03:05:02.869418+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2407008,"kb_used_data":5664,"kb_used_omap":0,"kb_used_meta":2401280,"kb_avail":752567712,"statfs":{"total":773094113280,"available":770629337088,"internally_reserved":0,"allocated":5799936,"data_stored":2824072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2458910720},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14996258,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-721,"num_write_kb":-15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.498335"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336636+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:04:55.336636+0000","last_peered":"2023-11-28T03:04:55.336636+0000","last_clean":"2023-11-28T03:04:55.336636+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:04:55.336636+0000","last_undegraded":"2023-11-28T03:04:55.336636+0000","last_fullsized":"2023-11-28T03:04:55.336636+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230109+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:04:55.230109+0000","last_peered":"2023-11-28T03:04:55.230109+0000","last_clean":"2023-11-28T03:04:55.230109+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:04:55.230109+0000","last_undegraded":"2023-11-28T03:04:55.230109+0000","last_fullsized":"2023-11-28T03:04:55.230109+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.255301+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:04:55.255301+0000","last_peered":"2023-11-28T03:04:55.255301+0000","last_clean":"2023-11-28T03:04:55.255301+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:04:55.255301+0000","last_undegraded":"2023-11-28T03:04:55.255301+0000","last_fullsized":"2023-11-28T03:04:55.255301+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230025+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:04:55.230025+0000","last_peered":"2023-11-28T03:04:55.230025+0000","last_clean":"2023-11-28T03:04:55.230025+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:04:55.230025+0000","last_undegraded":"2023-11-28T03:04:55.230025+0000","last_fullsized":"2023-11-28T03:04:55.230025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":49,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336901+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:04:55.336901+0000","last_peered":"2023-11-28T03:04:55.336901+0000","last_clean":"2023-11-28T03:04:55.336901+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:04:55.336901+0000","last_undegraded":"2023-11-28T03:04:55.336901+0000","last_fullsized":"2023-11-28T03:04:55.336901+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336895+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:04:55.336895+0000","last_peered":"2023-11-28T03:04:55.336895+0000","last_clean":"2023-11-28T03:04:55.336895+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:04:55.336895+0000","last_undegraded":"2023-11-28T03:04:55.336895+0000","last_fullsized":"2023-11-28T03:04:55.336895+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336928+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:04:55.336928+0000","last_peered":"2023-11-28T03:04:55.336928+0000","last_clean":"2023-11-28T03:04:55.336928+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:04:55.336928+0000","last_undegraded":"2023-11-28T03:04:55.336928+0000","last_fullsized":"2023-11-28T03:04:55.336928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":46,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.336677+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:04:55.336677+0000","last_peered":"2023-11-28T03:04:55.336677+0000","last_clean":"2023-11-28T03:04:55.336677+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:04:55.336677+0000","last_undegraded":"2023-11-28T03:04:55.336677+0000","last_fullsized":"2023-11-28T03:04:55.336677+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:05:00.400291+0000","last_change":"2023-11-28T03:05:00.400291+0000","last_active":"2023-11-28T03:05:00.400291+0000","last_peered":"2023-11-28T03:05:00.400291+0000","last_clean":"2023-11-28T03:05:00.400291+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:05:00.400291+0000","last_undegraded":"2023-11-28T03:05:00.400291+0000","last_fullsized":"2023-11-28T03:05:00.400291+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:04:59.395780+0000","last_change":"2023-11-28T03:04:59.395780+0000","last_active":"2023-11-28T03:04:59.395780+0000","last_peered":"2023-11-28T03:04:59.395780+0000","last_clean":"2023-11-28T03:04:59.395780+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:04:59.395780+0000","last_undegraded":"2023-11-28T03:04:59.395780+0000","last_fullsized":"2023-11-28T03:04:59.395780+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":51,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-11-28T03:04:55.230025+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:04:55.230025+0000","last_peered":"2023-11-28T03:04:55.230025+0000","last_clean":"2023-11-28T03:04:55.230025+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:04:55.230025+0000","last_undegraded":"2023-11-28T03:04:55.230025+0000","last_fullsized":"2023-11-28T03:04:55.230025+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574998,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60799999999999998,"15min":0.58399999999999996},"min":{"1min":0.33400000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.73799999999999999,"5min":0.82399999999999995,"15min":0.85999999999999999},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.59599999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80300000000000005,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.51000000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.371,"5min":0.35499999999999998,"15min":0.32000000000000001},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.57199999999999995,"15min":0.55100000000000005},"min":{"1min":0.42399999999999999,"5min":0.36899999999999999,"15min":0.34999999999999998},"max":{"1min":0.72399999999999998,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54100000000000004},"min":{"1min":0.377,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.70499999999999996,"5min":0.749,"15min":0.749},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.55900000000000005},"min":{"1min":0.46000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.52500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57199999999999995,"15min":0.55500000000000005},"min":{"1min":0.41399999999999998,"5min":0.38700000000000001,"15min":0.36599999999999999},"max":{"1min":0.749,"5min":0.749,"15min":0.77100000000000002},"last":0.377},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.44800000000000001,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.495}]},{"osd":4,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.30299999999999999},"max":{"1min":0.69699999999999995,"5min":0.77800000000000002,"15min":0.83099999999999996},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48199999999999998,"15min":0.496},"min":{"1min":0.32500000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.67400000000000004,"5min":0.69699999999999995,"15min":0.81899999999999995},"last":0.32400000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.48999999999999999,"15min":0.51300000000000001},"min":{"1min":0.34599999999999997,"5min":0.34000000000000002,"15min":0.313},"max":{"1min":0.69099999999999995,"5min":0.72799999999999998,"15min":0.84499999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.54900000000000004},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.34100000000000003},"max":{"1min":0.73899999999999999,"5min":0.83099999999999996,"15min":0.83199999999999996},"last":0.40000000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:03:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56200000000000006,"15min":0.56699999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.55200000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.28999999999999998},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.84499999999999997},"last":0.48599999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574998,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.496},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.379,"15min":0.32800000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.61199999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.54600000000000004}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.81000000000000005,"15min":0.84399999999999997},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.746,"15min":0.82499999999999996},"last":0.745}]},{"osd":5,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.317},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.77100000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574997,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.36099999999999999,"15min":0.33700000000000002},"min":{"1min":0.25900000000000001,"5min":0.246,"15min":0.20599999999999999},"max":{"1min":0.46700000000000003,"5min":0.67700000000000005,"15min":0.70699999999999996},"last":0.245},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39700000000000002,"15min":0.36299999999999999},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.20300000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.30399999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.41999999999999998,"5min":0.40100000000000002,"15min":0.374},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.23100000000000001},"max":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.73499999999999999},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.42499999999999999,"15min":0.38800000000000001},"min":{"1min":0.32000000000000001,"5min":0.27600000000000002,"15min":0.19800000000000001},"max":{"1min":0.59099999999999997,"5min":0.63400000000000001,"15min":0.72599999999999998},"last":0.40400000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.48099999999999998,"15min":0.44},"min":{"1min":0.35299999999999998,"5min":0.34799999999999998,"15min":0.25800000000000001},"max":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.749},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.47799999999999998,"5min":0.45600000000000002,"15min":0.433},"min":{"1min":0.376,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.60899999999999999,"5min":0.65300000000000002,"15min":1.6870000000000001},"last":0.34000000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60999999999999999,"15min":0.58899999999999997},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.83299999999999996,"5min":0.83899999999999997,"15min":1.0900000000000001},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.625,"15min":0.59199999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.28699999999999998},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.42399999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.46300000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.505},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.49099999999999999,"5min":0.39900000000000002,"15min":0.38200000000000001},"max":{"1min":1.038,"5min":1.038,"15min":1.111},"last":0.53900000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":1.8080000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64300000000000002,"15min":0.61499999999999999},"min":{"1min":0.45000000000000001,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.073},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:03:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.64200000000000002,"15min":0.61899999999999999},"min":{"1min":0.46999999999999997,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.64200000000000002,"15min":0.63500000000000001},"min":{"1min":0.442,"5min":0.442,"15min":0.38700000000000001},"max":{"1min":0.84599999999999997,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.48599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574998,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297804,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94074036,"statfs":{"total":96636764160,"available":96331812864,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.33600000000000002,"5min":0.35499999999999998,"15min":0.33900000000000002},"min":{"1min":0.26400000000000001,"5min":0.247,"15min":0.184},"max":{"1min":0.51000000000000001,"5min":0.55200000000000005,"15min":0.75700000000000001},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.35899999999999999,"5min":0.38600000000000001,"15min":0.36399999999999999},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.502,"5min":0.63,"15min":0.63},"last":0.317}]},{"osd":2,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.38500000000000001,"15min":0.36599999999999999},"min":{"1min":0.254,"5min":0.254,"15min":0.20100000000000001},"max":{"1min":0.48399999999999999,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.46500000000000002,"15min":0.434},"min":{"1min":0.33500000000000002,"5min":0.27100000000000002,"15min":0.26300000000000001},"max":{"1min":0.58599999999999997,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.42299999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38700000000000001,"5min":0.40699999999999997,"15min":0.38700000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.24099999999999999},"max":{"1min":0.52000000000000002,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.434,"15min":0.41199999999999998},"min":{"1min":0.33900000000000002,"5min":0.312,"15min":0.251},"max":{"1min":0.54300000000000004,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.63600000000000001,"15min":0.61899999999999999},"min":{"1min":0.443,"5min":0.41399999999999998,"15min":0.35399999999999998},"max":{"1min":0.95699999999999996,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66100000000000003,"15min":0.63200000000000001},"min":{"1min":0.47699999999999998,"5min":0.46600000000000003,"15min":0.39900000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71499999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65800000000000003,"15min":0.63300000000000001},"min":{"1min":0.48599999999999999,"5min":0.40899999999999997,"15min":0.40400000000000003},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.65400000000000003,"15min":0.63200000000000001},"min":{"1min":0.48999999999999999,"5min":0.443,"15min":0.39000000000000001},"max":{"1min":0.877,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.61899999999999999,"15min":0.59899999999999998},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.38700000000000001},"max":{"1min":0.78900000000000003,"5min":0.85499999999999998,"15min":0.91200000000000003},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.63700000000000001,"15min":0.61399999999999999},"min":{"1min":0.44600000000000001,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.93999999999999995,"5min":0.93999999999999995,"15min":0.94299999999999995},"last":0.59899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:03:57 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.65300000000000002,"15min":0.626},"min":{"1min":0.502,"5min":0.46899999999999997,"15min":0.36299999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.628},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.69499999999999995,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.046,"5min":1.046,"15min":1.046},"last":0.72499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.374,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.20499999999999999},"max":{"1min":0.41099999999999998,"5min":0.57399999999999995,"15min":0.623},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.42199999999999999,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23599999999999999},"max":{"1min":0.47599999999999998,"5min":0.64700000000000002,"15min":0.65900000000000003},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.377},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51100000000000001,"5min":0.68600000000000005,"15min":0.73099999999999998},"last":0.32600000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.40600000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53600000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.316}]},{"osd":4,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.94599999999999995},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":1.004,"15min":1.004},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74399999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.497},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36899999999999999},"max":{"1min":0.73699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.45600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574998,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36799999999999999,"15min":0.34699999999999998},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.501,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38900000000000001,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.19900000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.60899999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.44800000000000001,"15min":0.42599999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.68200000000000005},"last":0.375}]},{"osd":4,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6000000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.621,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70699999999999996,"5min":0.94199999999999995,"15min":1.196},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.996,"15min":1.206},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.97199999999999998,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.56499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.372,"15min":0.27300000000000002},"max":{"1min":0.73099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.53300000000000003}]},{"osd":1,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.32800000000000001,"15min":0.314},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.60099999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.30199999999999999},"max":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.82199999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.38,"15min":0.33500000000000002},"max":{"1min":0.73799999999999999,"5min":0.76300000000000001,"15min":0.78100000000000003},"last":0.69799999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.39000000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.623},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.878,"15min":0.878},"last":0.628}]},{"osd":5,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.69199999999999995,"15min":0.71699999999999997},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.64400000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.622,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.753}]},{"osd":7,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.71099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574999,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069940,"statfs":{"total":96636764160,"available":96327618560,"internally_reserved":0,"allocated":724992,"data_stored":353009,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.46999999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.82699999999999996},"last":0.52300000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69199999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.59599999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.377,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.76400000000000001,"15min":0.79800000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78700000000000003},"last":0.53700000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42599999999999999,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.63600000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.624}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:05:04.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:05:04.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.900+0000 7ff4c031b700 1 Processor -- start 2023-11-28T03:05:04.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.900+0000 7ff4c031b700 1 -- start start 2023-11-28T03:05:04.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.901+0000 7ff4c031b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff4b80caab0 0x7ff4b80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.901+0000 7ff4c031b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4b80ca180 0x7ff4b815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.901+0000 7ff4c031b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4b80a9a90 con 0x7ff4b80cae90 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.901+0000 7ff4c031b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b8154ce0 con 0x7ff4b80ca180 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.901+0000 7ff4c031b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4b80a8e80 con 0x7ff4b80caab0 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff4b80ca180 0x7ff4b815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd8b6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff4b80cae90 0x7ff4b80c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57408/0 (socket says 172.21.15.73:57408) 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd8b6700 1 -- 172.21.15.73:0/697593484 learned_addr learned my addr 172.21.15.73:0/697593484 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be0b7700 1 --2- 172.21.15.73:0/697593484 >> v2:172.21.15.150:3300/0 conn(0x7ff4b80caab0 0x7ff4b80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 -- 172.21.15.73:0/697593484 >> v2:172.21.15.150:3300/0 conn(0x7ff4b80caab0 msgr2=0x7ff4b80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 --2- 172.21.15.73:0/697593484 >> v2:172.21.15.150:3300/0 conn(0x7ff4b80caab0 0x7ff4b80aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 -- 172.21.15.73:0/697593484 >> v1:172.21.15.73:6789/0 conn(0x7ff4b80cae90 legacy=0x7ff4b80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 -- 172.21.15.73:0/697593484 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b80a9160 con 0x7ff4b80ca180 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd0b5700 1 -- 172.21.15.73:0/697593484 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353596653 0 0) 0x7ff4b80a9a90 con 0x7ff4b80cae90 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4be8b8700 1 --2- 172.21.15.73:0/697593484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80ca180 0x7ff4b815e3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7ff4b80848c0 tx=0x7ff4b0009780).ready entity=mon.0 client_cookie=796e3622ff9153eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd0b5700 1 -- 172.21.15.73:0/697593484 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b000e070 con 0x7ff4b80ca180 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd0b5700 1 -- 172.21.15.73:0/697593484 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b000bc30 con 0x7ff4b80ca180 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.902+0000 7ff4bd0b5700 1 -- 172.21.15.73:0/697593484 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0012410 con 0x7ff4b80ca180 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 -- 172.21.15.73:0/697593484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80ca180 msgr2=0x7ff4b815e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 --2- 172.21.15.73:0/697593484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80ca180 0x7ff4b815e3c0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 rx=0x7ff4b80848c0 tx=0x7ff4b0009780).stop 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 -- 172.21.15.73:0/697593484 shutdown_connections 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 --2- 172.21.15.73:0/697593484 >> v2:172.21.15.150:3300/0 conn(0x7ff4b80caab0 0x7ff4b80aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 --2- 172.21.15.73:0/697593484 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80ca180 0x7ff4b815e3c0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 -- 172.21.15.73:0/697593484 >> 172.21.15.73:0/697593484 conn(0x7ff4b8094920 msgr2=0x7ff4b806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 -- 172.21.15.73:0/697593484 shutdown_connections 2023-11-28T03:05:04.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.903+0000 7ff4c031b700 1 -- 172.21.15.73:0/697593484 wait complete. 2023-11-28T03:05:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.904+0000 7ff4c031b700 1 Processor -- start 2023-11-28T03:05:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.904+0000 7ff4c031b700 1 -- start start 2023-11-28T03:05:04.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b80ca180 0x7ff4b81537d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b80caab0 0x7ff4b8143340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b8143830 con 0x7ff4b80cae90 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80abd40 con 0x7ff4b80ca180 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4b80c95f0 con 0x7ff4b80caab0 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54488/0 (socket says 172.21.15.73:54488) 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 -- 172.21.15.73:0/1418415626 learned_addr learned my addr 172.21.15.73:0/1418415626 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be0b7700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b80ca180 0x7ff4b81537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4bd8b6700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b80caab0 0x7ff4b8143340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 -- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b80caab0 msgr2=0x7ff4b8143340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b80caab0 0x7ff4b8143340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 -- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b80ca180 msgr2=0x7ff4b81537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b80ca180 0x7ff4b81537d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 -- 172.21.15.73:0/1418415626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4b0009a20 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4be8b8700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7ff4b8083c30 tx=0x7ff4b0009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0009440 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4b00117d0 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff4b80ab410 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4b0011b90 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.905+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4b80ab9d0 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.906+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff4b80abed0 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.907+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff4b0003040 con 0x7ff4b80cae90 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.907+0000 7ff4ae7fc700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 0x7ff49c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:04.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.907+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff4b007c1a0 con 0x7ff4b80cae90 2023-11-28T03:05:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.907+0000 7ff4be0b7700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 0x7ff49c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.907+0000 7ff4be0b7700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 0x7ff49c061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff4b400a9d0 tx=0x7ff4b4008040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:04.911+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff4b004d470 con 0x7ff4b80cae90 2023-11-28T03:05:05.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.070+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff4b806c860 con 0x7ff49c05f150 2023-11-28T03:05:05.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.079+0000 7ff4ae7fc700 1 -- 172.21.15.73:0/1418415626 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361534 (secure 0 0 0) 0x7ff4b806c860 con 0x7ff49c05f150 2023-11-28T03:05:05.085 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:05.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 msgr2=0x7ff49c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 0x7ff49c061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7ff4b400a9d0 tx=0x7ff4b4008040).stop 2023-11-28T03:05:05.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 msgr2=0x7ff4b8155930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 rx=0x7ff4b8083c30 tx=0x7ff4b0009780).stop 2023-11-28T03:05:05.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 shutdown_connections 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff49c05f150 0x7ff49c061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4b80cae90 0x7ff4b8155930 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4b80ca180 0x7ff4b81537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 --2- 172.21.15.73:0/1418415626 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4b80caab0 0x7ff4b8143340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.082+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 >> 172.21.15.73:0/1418415626 conn(0x7ff4b8094920 msgr2=0x7ff4b80b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.083+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 shutdown_connections 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.083+0000 7ff4c031b700 1 -- 172.21.15.73:0/1418415626 wait complete. 2023-11-28T03:05:05.089 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:05:05.095 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-28T03:05:04.869760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411552,"kb_used_data":6112,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752563168,"statfs":{"total":773094113280,"available":770624684032,"internally_reserved":0,"allocated":6258688,"data_stored":3042948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14996258,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-721,"num_write_kb":-15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.498335"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796416+0000","last_change":"2023-11-28T03:05:00.791917+0000","last_active":"2023-11-28T03:05:00.796416+0000","last_peered":"2023-11-28T03:05:00.796416+0000","last_clean":"2023-11-28T03:05:00.796416+0000","last_became_active":"2023-11-28T03:05:00.791612+0000","last_became_peered":"2023-11-28T03:05:00.791612+0000","last_unstale":"2023-11-28T03:05:00.796416+0000","last_undegraded":"2023-11-28T03:05:00.796416+0000","last_fullsized":"2023-11-28T03:05:00.796416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795306+0000","last_change":"2023-11-28T03:05:00.425524+0000","last_active":"2023-11-28T03:05:00.795306+0000","last_peered":"2023-11-28T03:05:00.795306+0000","last_clean":"2023-11-28T03:05:00.795306+0000","last_became_active":"2023-11-28T03:05:00.425163+0000","last_became_peered":"2023-11-28T03:05:00.425163+0000","last_unstale":"2023-11-28T03:05:00.795306+0000","last_undegraded":"2023-11-28T03:05:00.795306+0000","last_fullsized":"2023-11-28T03:05:00.795306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800549+0000","last_change":"2023-11-28T03:05:00.795761+0000","last_active":"2023-11-28T03:05:00.800549+0000","last_peered":"2023-11-28T03:05:00.800549+0000","last_clean":"2023-11-28T03:05:00.800549+0000","last_became_active":"2023-11-28T03:05:00.795500+0000","last_became_peered":"2023-11-28T03:05:00.795500+0000","last_unstale":"2023-11-28T03:05:00.800549+0000","last_undegraded":"2023-11-28T03:05:00.800549+0000","last_fullsized":"2023-11-28T03:05:00.800549+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784809+0000","last_change":"2023-11-28T03:05:00.438659+0000","last_active":"2023-11-28T03:05:00.784809+0000","last_peered":"2023-11-28T03:05:00.784809+0000","last_clean":"2023-11-28T03:05:00.784809+0000","last_became_active":"2023-11-28T03:05:00.438343+0000","last_became_peered":"2023-11-28T03:05:00.438343+0000","last_unstale":"2023-11-28T03:05:00.784809+0000","last_undegraded":"2023-11-28T03:05:00.784809+0000","last_fullsized":"2023-11-28T03:05:00.784809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807835+0000","last_change":"2023-11-28T03:05:00.804536+0000","last_active":"2023-11-28T03:05:00.807835+0000","last_peered":"2023-11-28T03:05:00.807835+0000","last_clean":"2023-11-28T03:05:00.807835+0000","last_became_active":"2023-11-28T03:05:00.804274+0000","last_became_peered":"2023-11-28T03:05:00.804274+0000","last_unstale":"2023-11-28T03:05:00.807835+0000","last_undegraded":"2023-11-28T03:05:00.807835+0000","last_fullsized":"2023-11-28T03:05:00.807835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783457+0000","last_change":"2023-11-28T03:05:00.413262+0000","last_active":"2023-11-28T03:05:00.783457+0000","last_peered":"2023-11-28T03:05:00.783457+0000","last_clean":"2023-11-28T03:05:00.783457+0000","last_became_active":"2023-11-28T03:05:00.412961+0000","last_became_peered":"2023-11-28T03:05:00.412961+0000","last_unstale":"2023-11-28T03:05:00.783457+0000","last_undegraded":"2023-11-28T03:05:00.783457+0000","last_fullsized":"2023-11-28T03:05:00.783457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796118+0000","last_change":"2023-11-28T03:05:00.791909+0000","last_active":"2023-11-28T03:05:00.796118+0000","last_peered":"2023-11-28T03:05:00.796118+0000","last_clean":"2023-11-28T03:05:00.796118+0000","last_became_active":"2023-11-28T03:05:00.791622+0000","last_became_peered":"2023-11-28T03:05:00.791622+0000","last_unstale":"2023-11-28T03:05:00.796118+0000","last_undegraded":"2023-11-28T03:05:00.796118+0000","last_fullsized":"2023-11-28T03:05:00.796118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801601+0000","last_change":"2023-11-28T03:05:00.436611+0000","last_active":"2023-11-28T03:05:00.801601+0000","last_peered":"2023-11-28T03:05:00.801601+0000","last_clean":"2023-11-28T03:05:00.801601+0000","last_became_active":"2023-11-28T03:05:00.436324+0000","last_became_peered":"2023-11-28T03:05:00.436324+0000","last_unstale":"2023-11-28T03:05:00.801601+0000","last_undegraded":"2023-11-28T03:05:00.801601+0000","last_fullsized":"2023-11-28T03:05:00.801601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804685+0000","last_change":"2023-11-28T03:05:00.791992+0000","last_active":"2023-11-28T03:05:00.804685+0000","last_peered":"2023-11-28T03:05:00.804685+0000","last_clean":"2023-11-28T03:05:00.804685+0000","last_became_active":"2023-11-28T03:05:00.791738+0000","last_became_peered":"2023-11-28T03:05:00.791738+0000","last_unstale":"2023-11-28T03:05:00.804685+0000","last_undegraded":"2023-11-28T03:05:00.804685+0000","last_fullsized":"2023-11-28T03:05:00.804685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804714+0000","last_change":"2023-11-28T03:05:00.437137+0000","last_active":"2023-11-28T03:05:00.804714+0000","last_peered":"2023-11-28T03:05:00.804714+0000","last_clean":"2023-11-28T03:05:00.804714+0000","last_became_active":"2023-11-28T03:05:00.436830+0000","last_became_peered":"2023-11-28T03:05:00.436830+0000","last_unstale":"2023-11-28T03:05:00.804714+0000","last_undegraded":"2023-11-28T03:05:00.804714+0000","last_fullsized":"2023-11-28T03:05:00.804714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801683+0000","last_change":"2023-11-28T03:05:00.798043+0000","last_active":"2023-11-28T03:05:00.801683+0000","last_peered":"2023-11-28T03:05:00.801683+0000","last_clean":"2023-11-28T03:05:00.801683+0000","last_became_active":"2023-11-28T03:05:00.797785+0000","last_became_peered":"2023-11-28T03:05:00.797785+0000","last_unstale":"2023-11-28T03:05:00.801683+0000","last_undegraded":"2023-11-28T03:05:00.801683+0000","last_fullsized":"2023-11-28T03:05:00.801683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794948+0000","last_change":"2023-11-28T03:05:00.425489+0000","last_active":"2023-11-28T03:05:00.794948+0000","last_peered":"2023-11-28T03:05:00.794948+0000","last_clean":"2023-11-28T03:05:00.794948+0000","last_became_active":"2023-11-28T03:05:00.425041+0000","last_became_peered":"2023-11-28T03:05:00.425041+0000","last_unstale":"2023-11-28T03:05:00.794948+0000","last_undegraded":"2023-11-28T03:05:00.794948+0000","last_fullsized":"2023-11-28T03:05:00.794948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796138+0000","last_change":"2023-11-28T03:05:00.792581+0000","last_active":"2023-11-28T03:05:00.796138+0000","last_peered":"2023-11-28T03:05:00.796138+0000","last_clean":"2023-11-28T03:05:00.796138+0000","last_became_active":"2023-11-28T03:05:00.792322+0000","last_became_peered":"2023-11-28T03:05:00.792322+0000","last_unstale":"2023-11-28T03:05:00.796138+0000","last_undegraded":"2023-11-28T03:05:00.796138+0000","last_fullsized":"2023-11-28T03:05:00.796138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787429+0000","last_change":"2023-11-28T03:05:00.431726+0000","last_active":"2023-11-28T03:05:00.787429+0000","last_peered":"2023-11-28T03:05:00.787429+0000","last_clean":"2023-11-28T03:05:00.787429+0000","last_became_active":"2023-11-28T03:05:00.431380+0000","last_became_peered":"2023-11-28T03:05:00.431380+0000","last_unstale":"2023-11-28T03:05:00.787429+0000","last_undegraded":"2023-11-28T03:05:00.787429+0000","last_fullsized":"2023-11-28T03:05:00.787429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807353+0000","last_change":"2023-11-28T03:05:00.803609+0000","last_active":"2023-11-28T03:05:00.807353+0000","last_peered":"2023-11-28T03:05:00.807353+0000","last_clean":"2023-11-28T03:05:00.807353+0000","last_became_active":"2023-11-28T03:05:00.803362+0000","last_became_peered":"2023-11-28T03:05:00.803362+0000","last_unstale":"2023-11-28T03:05:00.807353+0000","last_undegraded":"2023-11-28T03:05:00.807353+0000","last_fullsized":"2023-11-28T03:05:00.807353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785509+0000","last_change":"2023-11-28T03:05:00.420800+0000","last_active":"2023-11-28T03:05:00.785509+0000","last_peered":"2023-11-28T03:05:00.785509+0000","last_clean":"2023-11-28T03:05:00.785509+0000","last_became_active":"2023-11-28T03:05:00.420414+0000","last_became_peered":"2023-11-28T03:05:00.420414+0000","last_unstale":"2023-11-28T03:05:00.785509+0000","last_undegraded":"2023-11-28T03:05:00.785509+0000","last_fullsized":"2023-11-28T03:05:00.785509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805747+0000","last_change":"2023-11-28T03:05:00.795247+0000","last_active":"2023-11-28T03:05:00.805747+0000","last_peered":"2023-11-28T03:05:00.805747+0000","last_clean":"2023-11-28T03:05:00.805747+0000","last_became_active":"2023-11-28T03:05:00.795001+0000","last_became_peered":"2023-11-28T03:05:00.795001+0000","last_unstale":"2023-11-28T03:05:00.805747+0000","last_undegraded":"2023-11-28T03:05:00.805747+0000","last_fullsized":"2023-11-28T03:05:00.805747+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784311+0000","last_change":"2023-11-28T03:05:00.429277+0000","last_active":"2023-11-28T03:05:00.784311+0000","last_peered":"2023-11-28T03:05:00.784311+0000","last_clean":"2023-11-28T03:05:00.784311+0000","last_became_active":"2023-11-28T03:05:00.428938+0000","last_became_peered":"2023-11-28T03:05:00.428938+0000","last_unstale":"2023-11-28T03:05:00.784311+0000","last_undegraded":"2023-11-28T03:05:00.784311+0000","last_fullsized":"2023-11-28T03:05:00.784311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806756+0000","last_change":"2023-11-28T03:05:00.803701+0000","last_active":"2023-11-28T03:05:00.806756+0000","last_peered":"2023-11-28T03:05:00.806756+0000","last_clean":"2023-11-28T03:05:00.806756+0000","last_became_active":"2023-11-28T03:05:00.803424+0000","last_became_peered":"2023-11-28T03:05:00.803424+0000","last_unstale":"2023-11-28T03:05:00.806756+0000","last_undegraded":"2023-11-28T03:05:00.806756+0000","last_fullsized":"2023-11-28T03:05:00.806756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783367+0000","last_change":"2023-11-28T03:05:00.437611+0000","last_active":"2023-11-28T03:05:00.783367+0000","last_peered":"2023-11-28T03:05:00.783367+0000","last_clean":"2023-11-28T03:05:00.783367+0000","last_became_active":"2023-11-28T03:05:00.437351+0000","last_became_peered":"2023-11-28T03:05:00.437351+0000","last_unstale":"2023-11-28T03:05:00.783367+0000","last_undegraded":"2023-11-28T03:05:00.783367+0000","last_fullsized":"2023-11-28T03:05:00.783367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806347+0000","last_change":"2023-11-28T03:05:00.801846+0000","last_active":"2023-11-28T03:05:00.806347+0000","last_peered":"2023-11-28T03:05:00.806347+0000","last_clean":"2023-11-28T03:05:00.806347+0000","last_became_active":"2023-11-28T03:05:00.801594+0000","last_became_peered":"2023-11-28T03:05:00.801594+0000","last_unstale":"2023-11-28T03:05:00.806347+0000","last_undegraded":"2023-11-28T03:05:00.806347+0000","last_fullsized":"2023-11-28T03:05:00.806347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783455+0000","last_change":"2023-11-28T03:05:00.437201+0000","last_active":"2023-11-28T03:05:00.783455+0000","last_peered":"2023-11-28T03:05:00.783455+0000","last_clean":"2023-11-28T03:05:00.783455+0000","last_became_active":"2023-11-28T03:05:00.436875+0000","last_became_peered":"2023-11-28T03:05:00.436875+0000","last_unstale":"2023-11-28T03:05:00.783455+0000","last_undegraded":"2023-11-28T03:05:00.783455+0000","last_fullsized":"2023-11-28T03:05:00.783455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854418+0000","last_change":"2023-11-28T03:05:00.853613+0000","last_active":"2023-11-28T03:05:00.854418+0000","last_peered":"2023-11-28T03:05:00.854418+0000","last_clean":"2023-11-28T03:05:00.854418+0000","last_became_active":"2023-11-28T03:05:00.853268+0000","last_became_peered":"2023-11-28T03:05:00.853268+0000","last_unstale":"2023-11-28T03:05:00.854418+0000","last_undegraded":"2023-11-28T03:05:00.854418+0000","last_fullsized":"2023-11-28T03:05:00.854418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809619+0000","last_change":"2023-11-28T03:05:00.424153+0000","last_active":"2023-11-28T03:05:00.809619+0000","last_peered":"2023-11-28T03:05:00.809619+0000","last_clean":"2023-11-28T03:05:00.809619+0000","last_became_active":"2023-11-28T03:05:00.423700+0000","last_became_peered":"2023-11-28T03:05:00.423700+0000","last_unstale":"2023-11-28T03:05:00.809619+0000","last_undegraded":"2023-11-28T03:05:00.809619+0000","last_fullsized":"2023-11-28T03:05:00.809619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809404+0000","last_change":"2023-11-28T03:05:00.805354+0000","last_active":"2023-11-28T03:05:00.809404+0000","last_peered":"2023-11-28T03:05:00.809404+0000","last_clean":"2023-11-28T03:05:00.809404+0000","last_became_active":"2023-11-28T03:05:00.805055+0000","last_became_peered":"2023-11-28T03:05:00.805055+0000","last_unstale":"2023-11-28T03:05:00.809404+0000","last_undegraded":"2023-11-28T03:05:00.809404+0000","last_fullsized":"2023-11-28T03:05:00.809404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807391+0000","last_change":"2023-11-28T03:05:00.425518+0000","last_active":"2023-11-28T03:05:00.807391+0000","last_peered":"2023-11-28T03:05:00.807391+0000","last_clean":"2023-11-28T03:05:00.807391+0000","last_became_active":"2023-11-28T03:05:00.425264+0000","last_became_peered":"2023-11-28T03:05:00.425264+0000","last_unstale":"2023-11-28T03:05:00.807391+0000","last_undegraded":"2023-11-28T03:05:00.807391+0000","last_fullsized":"2023-11-28T03:05:00.807391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806750+0000","last_change":"2023-11-28T03:05:00.802976+0000","last_active":"2023-11-28T03:05:00.806750+0000","last_peered":"2023-11-28T03:05:00.806750+0000","last_clean":"2023-11-28T03:05:00.806750+0000","last_became_active":"2023-11-28T03:05:00.802705+0000","last_became_peered":"2023-11-28T03:05:00.802705+0000","last_unstale":"2023-11-28T03:05:00.806750+0000","last_undegraded":"2023-11-28T03:05:00.806750+0000","last_fullsized":"2023-11-28T03:05:00.806750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787102+0000","last_change":"2023-11-28T03:05:00.429504+0000","last_active":"2023-11-28T03:05:00.787102+0000","last_peered":"2023-11-28T03:05:00.787102+0000","last_clean":"2023-11-28T03:05:00.787102+0000","last_became_active":"2023-11-28T03:05:00.429272+0000","last_became_peered":"2023-11-28T03:05:00.429272+0000","last_unstale":"2023-11-28T03:05:00.787102+0000","last_undegraded":"2023-11-28T03:05:00.787102+0000","last_fullsized":"2023-11-28T03:05:00.787102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802325+0000","last_change":"2023-11-28T03:05:00.800107+0000","last_active":"2023-11-28T03:05:00.802325+0000","last_peered":"2023-11-28T03:05:00.802325+0000","last_clean":"2023-11-28T03:05:00.802325+0000","last_became_active":"2023-11-28T03:05:00.799586+0000","last_became_peered":"2023-11-28T03:05:00.799586+0000","last_unstale":"2023-11-28T03:05:00.802325+0000","last_undegraded":"2023-11-28T03:05:00.802325+0000","last_fullsized":"2023-11-28T03:05:00.802325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780802+0000","last_change":"2023-11-28T03:05:00.423474+0000","last_active":"2023-11-28T03:05:00.780802+0000","last_peered":"2023-11-28T03:05:00.780802+0000","last_clean":"2023-11-28T03:05:00.780802+0000","last_became_active":"2023-11-28T03:05:00.423230+0000","last_became_peered":"2023-11-28T03:05:00.423230+0000","last_unstale":"2023-11-28T03:05:00.780802+0000","last_undegraded":"2023-11-28T03:05:00.780802+0000","last_fullsized":"2023-11-28T03:05:00.780802+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799306+0000","last_change":"2023-11-28T03:05:00.797655+0000","last_active":"2023-11-28T03:05:00.799306+0000","last_peered":"2023-11-28T03:05:00.799306+0000","last_clean":"2023-11-28T03:05:00.799306+0000","last_became_active":"2023-11-28T03:05:00.797293+0000","last_became_peered":"2023-11-28T03:05:00.797293+0000","last_unstale":"2023-11-28T03:05:00.799306+0000","last_undegraded":"2023-11-28T03:05:00.799306+0000","last_fullsized":"2023-11-28T03:05:00.799306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805668+0000","last_change":"2023-11-28T03:05:00.423972+0000","last_active":"2023-11-28T03:05:00.805668+0000","last_peered":"2023-11-28T03:05:00.805668+0000","last_clean":"2023-11-28T03:05:00.805668+0000","last_became_active":"2023-11-28T03:05:00.423595+0000","last_became_peered":"2023-11-28T03:05:00.423595+0000","last_unstale":"2023-11-28T03:05:00.805668+0000","last_undegraded":"2023-11-28T03:05:00.805668+0000","last_fullsized":"2023-11-28T03:05:00.805668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801748+0000","last_change":"2023-11-28T03:05:00.799701+0000","last_active":"2023-11-28T03:05:00.801748+0000","last_peered":"2023-11-28T03:05:00.801748+0000","last_clean":"2023-11-28T03:05:00.801748+0000","last_became_active":"2023-11-28T03:05:00.797686+0000","last_became_peered":"2023-11-28T03:05:00.797686+0000","last_unstale":"2023-11-28T03:05:00.801748+0000","last_undegraded":"2023-11-28T03:05:00.801748+0000","last_fullsized":"2023-11-28T03:05:00.801748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783803+0000","last_change":"2023-11-28T03:05:00.422200+0000","last_active":"2023-11-28T03:05:00.783803+0000","last_peered":"2023-11-28T03:05:00.783803+0000","last_clean":"2023-11-28T03:05:00.783803+0000","last_became_active":"2023-11-28T03:05:00.421929+0000","last_became_peered":"2023-11-28T03:05:00.421929+0000","last_unstale":"2023-11-28T03:05:00.783803+0000","last_undegraded":"2023-11-28T03:05:00.783803+0000","last_fullsized":"2023-11-28T03:05:00.783803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806499+0000","last_change":"2023-11-28T03:05:00.800024+0000","last_active":"2023-11-28T03:05:00.806499+0000","last_peered":"2023-11-28T03:05:00.806499+0000","last_clean":"2023-11-28T03:05:00.806499+0000","last_became_active":"2023-11-28T03:05:00.799789+0000","last_became_peered":"2023-11-28T03:05:00.799789+0000","last_unstale":"2023-11-28T03:05:00.806499+0000","last_undegraded":"2023-11-28T03:05:00.806499+0000","last_fullsized":"2023-11-28T03:05:00.806499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783737+0000","last_change":"2023-11-28T03:05:00.436677+0000","last_active":"2023-11-28T03:05:00.783737+0000","last_peered":"2023-11-28T03:05:00.783737+0000","last_clean":"2023-11-28T03:05:00.783737+0000","last_became_active":"2023-11-28T03:05:00.436405+0000","last_became_peered":"2023-11-28T03:05:00.436405+0000","last_unstale":"2023-11-28T03:05:00.783737+0000","last_undegraded":"2023-11-28T03:05:00.783737+0000","last_fullsized":"2023-11-28T03:05:00.783737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801912+0000","last_change":"2023-11-28T03:05:00.796280+0000","last_active":"2023-11-28T03:05:00.801912+0000","last_peered":"2023-11-28T03:05:00.801912+0000","last_clean":"2023-11-28T03:05:00.801912+0000","last_became_active":"2023-11-28T03:05:00.795931+0000","last_became_peered":"2023-11-28T03:05:00.795931+0000","last_unstale":"2023-11-28T03:05:00.801912+0000","last_undegraded":"2023-11-28T03:05:00.801912+0000","last_fullsized":"2023-11-28T03:05:00.801912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805408+0000","last_change":"2023-11-28T03:05:00.437430+0000","last_active":"2023-11-28T03:05:00.805408+0000","last_peered":"2023-11-28T03:05:00.805408+0000","last_clean":"2023-11-28T03:05:00.805408+0000","last_became_active":"2023-11-28T03:05:00.437155+0000","last_became_peered":"2023-11-28T03:05:00.437155+0000","last_unstale":"2023-11-28T03:05:00.805408+0000","last_undegraded":"2023-11-28T03:05:00.805408+0000","last_fullsized":"2023-11-28T03:05:00.805408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806687+0000","last_change":"2023-11-28T03:05:00.803087+0000","last_active":"2023-11-28T03:05:00.806687+0000","last_peered":"2023-11-28T03:05:00.806687+0000","last_clean":"2023-11-28T03:05:00.806687+0000","last_became_active":"2023-11-28T03:05:00.801329+0000","last_became_peered":"2023-11-28T03:05:00.801329+0000","last_unstale":"2023-11-28T03:05:00.806687+0000","last_undegraded":"2023-11-28T03:05:00.806687+0000","last_fullsized":"2023-11-28T03:05:00.806687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.790507+0000","last_change":"2023-11-28T03:05:00.437255+0000","last_active":"2023-11-28T03:05:00.790507+0000","last_peered":"2023-11-28T03:05:00.790507+0000","last_clean":"2023-11-28T03:05:00.790507+0000","last_became_active":"2023-11-28T03:05:00.436973+0000","last_became_peered":"2023-11-28T03:05:00.436973+0000","last_unstale":"2023-11-28T03:05:00.790507+0000","last_undegraded":"2023-11-28T03:05:00.790507+0000","last_fullsized":"2023-11-28T03:05:00.790507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799336+0000","last_change":"2023-11-28T03:05:00.797320+0000","last_active":"2023-11-28T03:05:00.799336+0000","last_peered":"2023-11-28T03:05:00.799336+0000","last_clean":"2023-11-28T03:05:00.799336+0000","last_became_active":"2023-11-28T03:05:00.796972+0000","last_became_peered":"2023-11-28T03:05:00.796972+0000","last_unstale":"2023-11-28T03:05:00.799336+0000","last_undegraded":"2023-11-28T03:05:00.799336+0000","last_fullsized":"2023-11-28T03:05:00.799336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795672+0000","last_change":"2023-11-28T03:05:00.423051+0000","last_active":"2023-11-28T03:05:00.795672+0000","last_peered":"2023-11-28T03:05:00.795672+0000","last_clean":"2023-11-28T03:05:00.795672+0000","last_became_active":"2023-11-28T03:05:00.422598+0000","last_became_peered":"2023-11-28T03:05:00.422598+0000","last_unstale":"2023-11-28T03:05:00.795672+0000","last_undegraded":"2023-11-28T03:05:00.795672+0000","last_fullsized":"2023-11-28T03:05:00.795672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809795+0000","last_change":"2023-11-28T03:05:00.806914+0000","last_active":"2023-11-28T03:05:00.809795+0000","last_peered":"2023-11-28T03:05:00.809795+0000","last_clean":"2023-11-28T03:05:00.809795+0000","last_became_active":"2023-11-28T03:05:00.806635+0000","last_became_peered":"2023-11-28T03:05:00.806635+0000","last_unstale":"2023-11-28T03:05:00.809795+0000","last_undegraded":"2023-11-28T03:05:00.809795+0000","last_fullsized":"2023-11-28T03:05:00.809795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787095+0000","last_change":"2023-11-28T03:05:00.423259+0000","last_active":"2023-11-28T03:05:00.787095+0000","last_peered":"2023-11-28T03:05:00.787095+0000","last_clean":"2023-11-28T03:05:00.787095+0000","last_became_active":"2023-11-28T03:05:00.422970+0000","last_became_peered":"2023-11-28T03:05:00.422970+0000","last_unstale":"2023-11-28T03:05:00.787095+0000","last_undegraded":"2023-11-28T03:05:00.787095+0000","last_fullsized":"2023-11-28T03:05:00.787095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807191+0000","last_change":"2023-11-28T03:05:00.802199+0000","last_active":"2023-11-28T03:05:00.807191+0000","last_peered":"2023-11-28T03:05:00.807191+0000","last_clean":"2023-11-28T03:05:00.807191+0000","last_became_active":"2023-11-28T03:05:00.801950+0000","last_became_peered":"2023-11-28T03:05:00.801950+0000","last_unstale":"2023-11-28T03:05:00.807191+0000","last_undegraded":"2023-11-28T03:05:00.807191+0000","last_fullsized":"2023-11-28T03:05:00.807191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781349+0000","last_change":"2023-11-28T03:05:00.418282+0000","last_active":"2023-11-28T03:05:00.781349+0000","last_peered":"2023-11-28T03:05:00.781349+0000","last_clean":"2023-11-28T03:05:00.781349+0000","last_became_active":"2023-11-28T03:05:00.417976+0000","last_became_peered":"2023-11-28T03:05:00.417976+0000","last_unstale":"2023-11-28T03:05:00.781349+0000","last_undegraded":"2023-11-28T03:05:00.781349+0000","last_fullsized":"2023-11-28T03:05:00.781349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806696+0000","last_change":"2023-11-28T03:05:00.802071+0000","last_active":"2023-11-28T03:05:00.806696+0000","last_peered":"2023-11-28T03:05:00.806696+0000","last_clean":"2023-11-28T03:05:00.806696+0000","last_became_active":"2023-11-28T03:05:00.801430+0000","last_became_peered":"2023-11-28T03:05:00.801430+0000","last_unstale":"2023-11-28T03:05:00.806696+0000","last_undegraded":"2023-11-28T03:05:00.806696+0000","last_fullsized":"2023-11-28T03:05:00.806696+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781670+0000","last_change":"2023-11-28T03:05:00.418862+0000","last_active":"2023-11-28T03:05:00.781670+0000","last_peered":"2023-11-28T03:05:00.781670+0000","last_clean":"2023-11-28T03:05:00.781670+0000","last_became_active":"2023-11-28T03:05:00.418589+0000","last_became_peered":"2023-11-28T03:05:00.418589+0000","last_unstale":"2023-11-28T03:05:00.781670+0000","last_undegraded":"2023-11-28T03:05:00.781670+0000","last_fullsized":"2023-11-28T03:05:00.781670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806755+0000","last_change":"2023-11-28T03:05:00.803646+0000","last_active":"2023-11-28T03:05:00.806755+0000","last_peered":"2023-11-28T03:05:00.806755+0000","last_clean":"2023-11-28T03:05:00.806755+0000","last_became_active":"2023-11-28T03:05:00.803400+0000","last_became_peered":"2023-11-28T03:05:00.803400+0000","last_unstale":"2023-11-28T03:05:00.806755+0000","last_undegraded":"2023-11-28T03:05:00.806755+0000","last_fullsized":"2023-11-28T03:05:00.806755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781821+0000","last_change":"2023-11-28T03:05:00.428730+0000","last_active":"2023-11-28T03:05:00.781821+0000","last_peered":"2023-11-28T03:05:00.781821+0000","last_clean":"2023-11-28T03:05:00.781821+0000","last_became_active":"2023-11-28T03:05:00.428404+0000","last_became_peered":"2023-11-28T03:05:00.428404+0000","last_unstale":"2023-11-28T03:05:00.781821+0000","last_undegraded":"2023-11-28T03:05:00.781821+0000","last_fullsized":"2023-11-28T03:05:00.781821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803533+0000","last_change":"2023-11-28T03:05:00.802752+0000","last_active":"2023-11-28T03:05:00.803533+0000","last_peered":"2023-11-28T03:05:00.803533+0000","last_clean":"2023-11-28T03:05:00.803533+0000","last_became_active":"2023-11-28T03:05:00.802481+0000","last_became_peered":"2023-11-28T03:05:00.802481+0000","last_unstale":"2023-11-28T03:05:00.803533+0000","last_undegraded":"2023-11-28T03:05:00.803533+0000","last_fullsized":"2023-11-28T03:05:00.803533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.792184+0000","last_change":"2023-11-28T03:05:00.425905+0000","last_active":"2023-11-28T03:05:00.792184+0000","last_peered":"2023-11-28T03:05:00.792184+0000","last_clean":"2023-11-28T03:05:00.792184+0000","last_became_active":"2023-11-28T03:05:00.424944+0000","last_became_peered":"2023-11-28T03:05:00.424944+0000","last_unstale":"2023-11-28T03:05:00.792184+0000","last_undegraded":"2023-11-28T03:05:00.792184+0000","last_fullsized":"2023-11-28T03:05:00.792184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801767+0000","last_change":"2023-11-28T03:05:00.793768+0000","last_active":"2023-11-28T03:05:00.801767+0000","last_peered":"2023-11-28T03:05:00.801767+0000","last_clean":"2023-11-28T03:05:00.801767+0000","last_became_active":"2023-11-28T03:05:00.793493+0000","last_became_peered":"2023-11-28T03:05:00.793493+0000","last_unstale":"2023-11-28T03:05:00.801767+0000","last_undegraded":"2023-11-28T03:05:00.801767+0000","last_fullsized":"2023-11-28T03:05:00.801767+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783477+0000","last_change":"2023-11-28T03:05:00.418986+0000","last_active":"2023-11-28T03:05:00.783477+0000","last_peered":"2023-11-28T03:05:00.783477+0000","last_clean":"2023-11-28T03:05:00.783477+0000","last_became_active":"2023-11-28T03:05:00.418713+0000","last_became_peered":"2023-11-28T03:05:00.418713+0000","last_unstale":"2023-11-28T03:05:00.783477+0000","last_undegraded":"2023-11-28T03:05:00.783477+0000","last_fullsized":"2023-11-28T03:05:00.783477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799787+0000","last_change":"2023-11-28T03:05:00.798686+0000","last_active":"2023-11-28T03:05:00.799787+0000","last_peered":"2023-11-28T03:05:00.799787+0000","last_clean":"2023-11-28T03:05:00.799787+0000","last_became_active":"2023-11-28T03:05:00.798354+0000","last_became_peered":"2023-11-28T03:05:00.798354+0000","last_unstale":"2023-11-28T03:05:00.799787+0000","last_undegraded":"2023-11-28T03:05:00.799787+0000","last_fullsized":"2023-11-28T03:05:00.799787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785410+0000","last_change":"2023-11-28T03:05:00.423987+0000","last_active":"2023-11-28T03:05:00.785410+0000","last_peered":"2023-11-28T03:05:00.785410+0000","last_clean":"2023-11-28T03:05:00.785410+0000","last_became_active":"2023-11-28T03:05:00.423314+0000","last_became_peered":"2023-11-28T03:05:00.423314+0000","last_unstale":"2023-11-28T03:05:00.785410+0000","last_undegraded":"2023-11-28T03:05:00.785410+0000","last_fullsized":"2023-11-28T03:05:00.785410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795546+0000","last_change":"2023-11-28T03:05:00.792221+0000","last_active":"2023-11-28T03:05:00.795546+0000","last_peered":"2023-11-28T03:05:00.795546+0000","last_clean":"2023-11-28T03:05:00.795546+0000","last_became_active":"2023-11-28T03:05:00.791903+0000","last_became_peered":"2023-11-28T03:05:00.791903+0000","last_unstale":"2023-11-28T03:05:00.795546+0000","last_undegraded":"2023-11-28T03:05:00.795546+0000","last_fullsized":"2023-11-28T03:05:00.795546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782316+0000","last_change":"2023-11-28T03:05:00.428717+0000","last_active":"2023-11-28T03:05:00.782316+0000","last_peered":"2023-11-28T03:05:00.782316+0000","last_clean":"2023-11-28T03:05:00.782316+0000","last_became_active":"2023-11-28T03:05:00.428428+0000","last_became_peered":"2023-11-28T03:05:00.428428+0000","last_unstale":"2023-11-28T03:05:00.782316+0000","last_undegraded":"2023-11-28T03:05:00.782316+0000","last_fullsized":"2023-11-28T03:05:00.782316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854381+0000","last_change":"2023-11-28T03:05:00.853726+0000","last_active":"2023-11-28T03:05:00.854381+0000","last_peered":"2023-11-28T03:05:00.854381+0000","last_clean":"2023-11-28T03:05:00.854381+0000","last_became_active":"2023-11-28T03:05:00.853447+0000","last_became_peered":"2023-11-28T03:05:00.853447+0000","last_unstale":"2023-11-28T03:05:00.854381+0000","last_undegraded":"2023-11-28T03:05:00.854381+0000","last_fullsized":"2023-11-28T03:05:00.854381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783385+0000","last_change":"2023-11-28T03:05:00.420805+0000","last_active":"2023-11-28T03:05:00.783385+0000","last_peered":"2023-11-28T03:05:00.783385+0000","last_clean":"2023-11-28T03:05:00.783385+0000","last_became_active":"2023-11-28T03:05:00.420529+0000","last_became_peered":"2023-11-28T03:05:00.420529+0000","last_unstale":"2023-11-28T03:05:00.783385+0000","last_undegraded":"2023-11-28T03:05:00.783385+0000","last_fullsized":"2023-11-28T03:05:00.783385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802344+0000","last_change":"2023-11-28T03:05:00.800261+0000","last_active":"2023-11-28T03:05:00.802344+0000","last_peered":"2023-11-28T03:05:00.802344+0000","last_clean":"2023-11-28T03:05:00.802344+0000","last_became_active":"2023-11-28T03:05:00.799863+0000","last_became_peered":"2023-11-28T03:05:00.799863+0000","last_unstale":"2023-11-28T03:05:00.802344+0000","last_undegraded":"2023-11-28T03:05:00.802344+0000","last_fullsized":"2023-11-28T03:05:00.802344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.779918+0000","last_change":"2023-11-28T03:05:00.424347+0000","last_active":"2023-11-28T03:05:00.779918+0000","last_peered":"2023-11-28T03:05:00.779918+0000","last_clean":"2023-11-28T03:05:00.779918+0000","last_became_active":"2023-11-28T03:05:00.424002+0000","last_became_peered":"2023-11-28T03:05:00.424002+0000","last_unstale":"2023-11-28T03:05:00.779918+0000","last_undegraded":"2023-11-28T03:05:00.779918+0000","last_fullsized":"2023-11-28T03:05:00.779918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806717+0000","last_change":"2023-11-28T03:05:00.803173+0000","last_active":"2023-11-28T03:05:00.806717+0000","last_peered":"2023-11-28T03:05:00.806717+0000","last_clean":"2023-11-28T03:05:00.806717+0000","last_became_active":"2023-11-28T03:05:00.802493+0000","last_became_peered":"2023-11-28T03:05:00.802493+0000","last_unstale":"2023-11-28T03:05:00.806717+0000","last_undegraded":"2023-11-28T03:05:00.806717+0000","last_fullsized":"2023-11-28T03:05:00.806717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784497+0000","last_change":"2023-11-28T03:05:00.432485+0000","last_active":"2023-11-28T03:05:00.784497+0000","last_peered":"2023-11-28T03:05:00.784497+0000","last_clean":"2023-11-28T03:05:00.784497+0000","last_became_active":"2023-11-28T03:05:00.432202+0000","last_became_peered":"2023-11-28T03:05:00.432202+0000","last_unstale":"2023-11-28T03:05:00.784497+0000","last_undegraded":"2023-11-28T03:05:00.784497+0000","last_fullsized":"2023-11-28T03:05:00.784497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805558+0000","last_change":"2023-11-28T03:05:00.800890+0000","last_active":"2023-11-28T03:05:00.805558+0000","last_peered":"2023-11-28T03:05:00.805558+0000","last_clean":"2023-11-28T03:05:00.805558+0000","last_became_active":"2023-11-28T03:05:00.800607+0000","last_became_peered":"2023-11-28T03:05:00.800607+0000","last_unstale":"2023-11-28T03:05:00.805558+0000","last_undegraded":"2023-11-28T03:05:00.805558+0000","last_fullsized":"2023-11-28T03:05:00.805558+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795186+0000","last_change":"2023-11-28T03:05:00.420535+0000","last_active":"2023-11-28T03:05:00.795186+0000","last_peered":"2023-11-28T03:05:00.795186+0000","last_clean":"2023-11-28T03:05:00.795186+0000","last_became_active":"2023-11-28T03:05:00.419707+0000","last_became_peered":"2023-11-28T03:05:00.419707+0000","last_unstale":"2023-11-28T03:05:00.795186+0000","last_undegraded":"2023-11-28T03:05:00.795186+0000","last_fullsized":"2023-11-28T03:05:00.795186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802388+0000","last_change":"2023-11-28T03:05:00.801453+0000","last_active":"2023-11-28T03:05:00.802388+0000","last_peered":"2023-11-28T03:05:00.802388+0000","last_clean":"2023-11-28T03:05:00.802388+0000","last_became_active":"2023-11-28T03:05:00.801184+0000","last_became_peered":"2023-11-28T03:05:00.801184+0000","last_unstale":"2023-11-28T03:05:00.802388+0000","last_undegraded":"2023-11-28T03:05:00.802388+0000","last_fullsized":"2023-11-28T03:05:00.802388+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794713+0000","last_change":"2023-11-28T03:05:00.428126+0000","last_active":"2023-11-28T03:05:00.794713+0000","last_peered":"2023-11-28T03:05:00.794713+0000","last_clean":"2023-11-28T03:05:00.794713+0000","last_became_active":"2023-11-28T03:05:00.427535+0000","last_became_peered":"2023-11-28T03:05:00.427535+0000","last_unstale":"2023-11-28T03:05:00.794713+0000","last_undegraded":"2023-11-28T03:05:00.794713+0000","last_fullsized":"2023-11-28T03:05:00.794713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795601+0000","last_change":"2023-11-28T03:05:00.793229+0000","last_active":"2023-11-28T03:05:00.795601+0000","last_peered":"2023-11-28T03:05:00.795601+0000","last_clean":"2023-11-28T03:05:00.795601+0000","last_became_active":"2023-11-28T03:05:00.792929+0000","last_became_peered":"2023-11-28T03:05:00.792929+0000","last_unstale":"2023-11-28T03:05:00.795601+0000","last_undegraded":"2023-11-28T03:05:00.795601+0000","last_fullsized":"2023-11-28T03:05:00.795601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795082+0000","last_change":"2023-11-28T03:05:00.430218+0000","last_active":"2023-11-28T03:05:00.795082+0000","last_peered":"2023-11-28T03:05:00.795082+0000","last_clean":"2023-11-28T03:05:00.795082+0000","last_became_active":"2023-11-28T03:05:00.427406+0000","last_became_peered":"2023-11-28T03:05:00.427406+0000","last_unstale":"2023-11-28T03:05:00.795082+0000","last_undegraded":"2023-11-28T03:05:00.795082+0000","last_fullsized":"2023-11-28T03:05:00.795082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794703+0000","last_change":"2023-11-28T03:05:00.792518+0000","last_active":"2023-11-28T03:05:00.794703+0000","last_peered":"2023-11-28T03:05:00.794703+0000","last_clean":"2023-11-28T03:05:00.794703+0000","last_became_active":"2023-11-28T03:05:00.792351+0000","last_became_peered":"2023-11-28T03:05:00.792351+0000","last_unstale":"2023-11-28T03:05:00.794703+0000","last_undegraded":"2023-11-28T03:05:00.794703+0000","last_fullsized":"2023-11-28T03:05:00.794703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785523+0000","last_change":"2023-11-28T03:05:00.426818+0000","last_active":"2023-11-28T03:05:00.785523+0000","last_peered":"2023-11-28T03:05:00.785523+0000","last_clean":"2023-11-28T03:05:00.785523+0000","last_became_active":"2023-11-28T03:05:00.426491+0000","last_became_peered":"2023-11-28T03:05:00.426491+0000","last_unstale":"2023-11-28T03:05:00.785523+0000","last_undegraded":"2023-11-28T03:05:00.785523+0000","last_fullsized":"2023-11-28T03:05:00.785523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854409+0000","last_change":"2023-11-28T03:05:00.853635+0000","last_active":"2023-11-28T03:05:00.854409+0000","last_peered":"2023-11-28T03:05:00.854409+0000","last_clean":"2023-11-28T03:05:00.854409+0000","last_became_active":"2023-11-28T03:05:00.853345+0000","last_became_peered":"2023-11-28T03:05:00.853345+0000","last_unstale":"2023-11-28T03:05:00.854409+0000","last_undegraded":"2023-11-28T03:05:00.854409+0000","last_fullsized":"2023-11-28T03:05:00.854409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782759+0000","last_change":"2023-11-28T03:05:00.423415+0000","last_active":"2023-11-28T03:05:00.782759+0000","last_peered":"2023-11-28T03:05:00.782759+0000","last_clean":"2023-11-28T03:05:00.782759+0000","last_became_active":"2023-11-28T03:05:00.423212+0000","last_became_peered":"2023-11-28T03:05:00.423212+0000","last_unstale":"2023-11-28T03:05:00.782759+0000","last_undegraded":"2023-11-28T03:05:00.782759+0000","last_fullsized":"2023-11-28T03:05:00.782759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802074+0000","last_change":"2023-11-28T03:05:00.797871+0000","last_active":"2023-11-28T03:05:00.802074+0000","last_peered":"2023-11-28T03:05:00.802074+0000","last_clean":"2023-11-28T03:05:00.802074+0000","last_became_active":"2023-11-28T03:05:00.796189+0000","last_became_peered":"2023-11-28T03:05:00.796189+0000","last_unstale":"2023-11-28T03:05:00.802074+0000","last_undegraded":"2023-11-28T03:05:00.802074+0000","last_fullsized":"2023-11-28T03:05:00.802074+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783586+0000","last_change":"2023-11-28T03:05:00.411962+0000","last_active":"2023-11-28T03:05:00.783586+0000","last_peered":"2023-11-28T03:05:00.783586+0000","last_clean":"2023-11-28T03:05:00.783586+0000","last_became_active":"2023-11-28T03:05:00.411666+0000","last_became_peered":"2023-11-28T03:05:00.411666+0000","last_unstale":"2023-11-28T03:05:00.783586+0000","last_undegraded":"2023-11-28T03:05:00.783586+0000","last_fullsized":"2023-11-28T03:05:00.783586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807208+0000","last_change":"2023-11-28T03:05:00.802294+0000","last_active":"2023-11-28T03:05:00.807208+0000","last_peered":"2023-11-28T03:05:00.807208+0000","last_clean":"2023-11-28T03:05:00.807208+0000","last_became_active":"2023-11-28T03:05:00.802023+0000","last_became_peered":"2023-11-28T03:05:00.802023+0000","last_unstale":"2023-11-28T03:05:00.807208+0000","last_undegraded":"2023-11-28T03:05:00.807208+0000","last_fullsized":"2023-11-28T03:05:00.807208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781451+0000","last_change":"2023-11-28T03:05:00.424948+0000","last_active":"2023-11-28T03:05:00.781451+0000","last_peered":"2023-11-28T03:05:00.781451+0000","last_clean":"2023-11-28T03:05:00.781451+0000","last_became_active":"2023-11-28T03:05:00.424627+0000","last_became_peered":"2023-11-28T03:05:00.424627+0000","last_unstale":"2023-11-28T03:05:00.781451+0000","last_undegraded":"2023-11-28T03:05:00.781451+0000","last_fullsized":"2023-11-28T03:05:00.781451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809387+0000","last_change":"2023-11-28T03:05:00.803552+0000","last_active":"2023-11-28T03:05:00.809387+0000","last_peered":"2023-11-28T03:05:00.809387+0000","last_clean":"2023-11-28T03:05:00.809387+0000","last_became_active":"2023-11-28T03:05:00.803295+0000","last_became_peered":"2023-11-28T03:05:00.803295+0000","last_unstale":"2023-11-28T03:05:00.809387+0000","last_undegraded":"2023-11-28T03:05:00.809387+0000","last_fullsized":"2023-11-28T03:05:00.809387+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785601+0000","last_change":"2023-11-28T03:05:00.420449+0000","last_active":"2023-11-28T03:05:00.785601+0000","last_peered":"2023-11-28T03:05:00.785601+0000","last_clean":"2023-11-28T03:05:00.785601+0000","last_became_active":"2023-11-28T03:05:00.420083+0000","last_became_peered":"2023-11-28T03:05:00.420083+0000","last_unstale":"2023-11-28T03:05:00.785601+0000","last_undegraded":"2023-11-28T03:05:00.785601+0000","last_fullsized":"2023-11-28T03:05:00.785601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854379+0000","last_change":"2023-11-28T03:05:00.853332+0000","last_active":"2023-11-28T03:05:00.854379+0000","last_peered":"2023-11-28T03:05:00.854379+0000","last_clean":"2023-11-28T03:05:00.854379+0000","last_became_active":"2023-11-28T03:05:00.852934+0000","last_became_peered":"2023-11-28T03:05:00.852934+0000","last_unstale":"2023-11-28T03:05:00.854379+0000","last_undegraded":"2023-11-28T03:05:00.854379+0000","last_fullsized":"2023-11-28T03:05:00.854379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787447+0000","last_change":"2023-11-28T03:05:00.425379+0000","last_active":"2023-11-28T03:05:00.787447+0000","last_peered":"2023-11-28T03:05:00.787447+0000","last_clean":"2023-11-28T03:05:00.787447+0000","last_became_active":"2023-11-28T03:05:00.424874+0000","last_became_peered":"2023-11-28T03:05:00.424874+0000","last_unstale":"2023-11-28T03:05:00.787447+0000","last_undegraded":"2023-11-28T03:05:00.787447+0000","last_fullsized":"2023-11-28T03:05:00.787447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797031+0000","last_change":"2023-11-28T03:05:00.793571+0000","last_active":"2023-11-28T03:05:00.797031+0000","last_peered":"2023-11-28T03:05:00.797031+0000","last_clean":"2023-11-28T03:05:00.797031+0000","last_became_active":"2023-11-28T03:05:00.793295+0000","last_became_peered":"2023-11-28T03:05:00.793295+0000","last_unstale":"2023-11-28T03:05:00.797031+0000","last_undegraded":"2023-11-28T03:05:00.797031+0000","last_fullsized":"2023-11-28T03:05:00.797031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784157+0000","last_change":"2023-11-28T03:05:00.436731+0000","last_active":"2023-11-28T03:05:00.784157+0000","last_peered":"2023-11-28T03:05:00.784157+0000","last_clean":"2023-11-28T03:05:00.784157+0000","last_became_active":"2023-11-28T03:05:00.436450+0000","last_became_peered":"2023-11-28T03:05:00.436450+0000","last_unstale":"2023-11-28T03:05:00.784157+0000","last_undegraded":"2023-11-28T03:05:00.784157+0000","last_fullsized":"2023-11-28T03:05:00.784157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806630+0000","last_change":"2023-11-28T03:05:00.800644+0000","last_active":"2023-11-28T03:05:00.806630+0000","last_peered":"2023-11-28T03:05:00.806630+0000","last_clean":"2023-11-28T03:05:00.806630+0000","last_became_active":"2023-11-28T03:05:00.800385+0000","last_became_peered":"2023-11-28T03:05:00.800385+0000","last_unstale":"2023-11-28T03:05:00.806630+0000","last_undegraded":"2023-11-28T03:05:00.806630+0000","last_fullsized":"2023-11-28T03:05:00.806630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800698+0000","last_change":"2023-11-28T03:05:00.431729+0000","last_active":"2023-11-28T03:05:00.800698+0000","last_peered":"2023-11-28T03:05:00.800698+0000","last_clean":"2023-11-28T03:05:00.800698+0000","last_became_active":"2023-11-28T03:05:00.431423+0000","last_became_peered":"2023-11-28T03:05:00.431423+0000","last_unstale":"2023-11-28T03:05:00.800698+0000","last_undegraded":"2023-11-28T03:05:00.800698+0000","last_fullsized":"2023-11-28T03:05:00.800698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794255+0000","last_change":"2023-11-28T03:05:00.792257+0000","last_active":"2023-11-28T03:05:00.794255+0000","last_peered":"2023-11-28T03:05:00.794255+0000","last_clean":"2023-11-28T03:05:00.794255+0000","last_became_active":"2023-11-28T03:05:00.791902+0000","last_became_peered":"2023-11-28T03:05:00.791902+0000","last_unstale":"2023-11-28T03:05:00.794255+0000","last_undegraded":"2023-11-28T03:05:00.794255+0000","last_fullsized":"2023-11-28T03:05:00.794255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803922+0000","last_change":"2023-11-28T03:05:00.424419+0000","last_active":"2023-11-28T03:05:00.803922+0000","last_peered":"2023-11-28T03:05:00.803922+0000","last_clean":"2023-11-28T03:05:00.803922+0000","last_became_active":"2023-11-28T03:05:00.424147+0000","last_became_peered":"2023-11-28T03:05:00.424147+0000","last_unstale":"2023-11-28T03:05:00.803922+0000","last_undegraded":"2023-11-28T03:05:00.803922+0000","last_fullsized":"2023-11-28T03:05:00.803922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801857+0000","last_change":"2023-11-28T03:05:00.799216+0000","last_active":"2023-11-28T03:05:00.801857+0000","last_peered":"2023-11-28T03:05:00.801857+0000","last_clean":"2023-11-28T03:05:00.801857+0000","last_became_active":"2023-11-28T03:05:00.798916+0000","last_became_peered":"2023-11-28T03:05:00.798916+0000","last_unstale":"2023-11-28T03:05:00.801857+0000","last_undegraded":"2023-11-28T03:05:00.801857+0000","last_fullsized":"2023-11-28T03:05:00.801857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801481+0000","last_change":"2023-11-28T03:05:00.426430+0000","last_active":"2023-11-28T03:05:00.801481+0000","last_peered":"2023-11-28T03:05:00.801481+0000","last_clean":"2023-11-28T03:05:00.801481+0000","last_became_active":"2023-11-28T03:05:00.425465+0000","last_became_peered":"2023-11-28T03:05:00.425465+0000","last_unstale":"2023-11-28T03:05:00.801481+0000","last_undegraded":"2023-11-28T03:05:00.801481+0000","last_fullsized":"2023-11-28T03:05:00.801481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806594+0000","last_change":"2023-11-28T03:05:00.800001+0000","last_active":"2023-11-28T03:05:00.806594+0000","last_peered":"2023-11-28T03:05:00.806594+0000","last_clean":"2023-11-28T03:05:00.806594+0000","last_became_active":"2023-11-28T03:05:00.799703+0000","last_became_peered":"2023-11-28T03:05:00.799703+0000","last_unstale":"2023-11-28T03:05:00.806594+0000","last_undegraded":"2023-11-28T03:05:00.806594+0000","last_fullsized":"2023-11-28T03:05:00.806594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781671+0000","last_change":"2023-11-28T03:05:00.436846+0000","last_active":"2023-11-28T03:05:00.781671+0000","last_peered":"2023-11-28T03:05:00.781671+0000","last_clean":"2023-11-28T03:05:00.781671+0000","last_became_active":"2023-11-28T03:05:00.436418+0000","last_became_peered":"2023-11-28T03:05:00.436418+0000","last_unstale":"2023-11-28T03:05:00.781671+0000","last_undegraded":"2023-11-28T03:05:00.781671+0000","last_fullsized":"2023-11-28T03:05:00.781671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795961+0000","last_change":"2023-11-28T03:05:00.793678+0000","last_active":"2023-11-28T03:05:00.795961+0000","last_peered":"2023-11-28T03:05:00.795961+0000","last_clean":"2023-11-28T03:05:00.795961+0000","last_became_active":"2023-11-28T03:05:00.793025+0000","last_became_peered":"2023-11-28T03:05:00.793025+0000","last_unstale":"2023-11-28T03:05:00.795961+0000","last_undegraded":"2023-11-28T03:05:00.795961+0000","last_fullsized":"2023-11-28T03:05:00.795961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787533+0000","last_change":"2023-11-28T03:05:00.428834+0000","last_active":"2023-11-28T03:05:00.787533+0000","last_peered":"2023-11-28T03:05:00.787533+0000","last_clean":"2023-11-28T03:05:00.787533+0000","last_became_active":"2023-11-28T03:05:00.428487+0000","last_became_peered":"2023-11-28T03:05:00.428487+0000","last_unstale":"2023-11-28T03:05:00.787533+0000","last_undegraded":"2023-11-28T03:05:00.787533+0000","last_fullsized":"2023-11-28T03:05:00.787533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806674+0000","last_change":"2023-11-28T03:05:00.800007+0000","last_active":"2023-11-28T03:05:00.806674+0000","last_peered":"2023-11-28T03:05:00.806674+0000","last_clean":"2023-11-28T03:05:00.806674+0000","last_became_active":"2023-11-28T03:05:00.799737+0000","last_became_peered":"2023-11-28T03:05:00.799737+0000","last_unstale":"2023-11-28T03:05:00.806674+0000","last_undegraded":"2023-11-28T03:05:00.806674+0000","last_fullsized":"2023-11-28T03:05:00.806674+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806127+0000","last_change":"2023-11-28T03:05:00.424398+0000","last_active":"2023-11-28T03:05:00.806127+0000","last_peered":"2023-11-28T03:05:00.806127+0000","last_clean":"2023-11-28T03:05:00.806127+0000","last_became_active":"2023-11-28T03:05:00.424133+0000","last_became_peered":"2023-11-28T03:05:00.424133+0000","last_unstale":"2023-11-28T03:05:00.806127+0000","last_undegraded":"2023-11-28T03:05:00.806127+0000","last_fullsized":"2023-11-28T03:05:00.806127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807703+0000","last_change":"2023-11-28T03:05:00.804770+0000","last_active":"2023-11-28T03:05:00.807703+0000","last_peered":"2023-11-28T03:05:00.807703+0000","last_clean":"2023-11-28T03:05:00.807703+0000","last_became_active":"2023-11-28T03:05:00.804141+0000","last_became_peered":"2023-11-28T03:05:00.804141+0000","last_unstale":"2023-11-28T03:05:00.807703+0000","last_undegraded":"2023-11-28T03:05:00.807703+0000","last_fullsized":"2023-11-28T03:05:00.807703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802987+0000","last_change":"2023-11-28T03:05:00.426975+0000","last_active":"2023-11-28T03:05:00.802987+0000","last_peered":"2023-11-28T03:05:00.802987+0000","last_clean":"2023-11-28T03:05:00.802987+0000","last_became_active":"2023-11-28T03:05:00.426697+0000","last_became_peered":"2023-11-28T03:05:00.426697+0000","last_unstale":"2023-11-28T03:05:00.802987+0000","last_undegraded":"2023-11-28T03:05:00.802987+0000","last_fullsized":"2023-11-28T03:05:00.802987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.810119+0000","last_change":"2023-11-28T03:05:00.807548+0000","last_active":"2023-11-28T03:05:00.810119+0000","last_peered":"2023-11-28T03:05:00.810119+0000","last_clean":"2023-11-28T03:05:00.810119+0000","last_became_active":"2023-11-28T03:05:00.807224+0000","last_became_peered":"2023-11-28T03:05:00.807224+0000","last_unstale":"2023-11-28T03:05:00.810119+0000","last_undegraded":"2023-11-28T03:05:00.810119+0000","last_fullsized":"2023-11-28T03:05:00.810119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802720+0000","last_change":"2023-11-28T03:05:00.419344+0000","last_active":"2023-11-28T03:05:00.802720+0000","last_peered":"2023-11-28T03:05:00.802720+0000","last_clean":"2023-11-28T03:05:00.802720+0000","last_became_active":"2023-11-28T03:05:00.419064+0000","last_became_peered":"2023-11-28T03:05:00.419064+0000","last_unstale":"2023-11-28T03:05:00.802720+0000","last_undegraded":"2023-11-28T03:05:00.802720+0000","last_fullsized":"2023-11-28T03:05:00.802720+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.853572+0000","last_change":"2023-11-28T03:05:00.794660+0000","last_active":"2023-11-28T03:05:00.853572+0000","last_peered":"2023-11-28T03:05:00.853572+0000","last_clean":"2023-11-28T03:05:00.853572+0000","last_became_active":"2023-11-28T03:05:00.794429+0000","last_became_peered":"2023-11-28T03:05:00.794429+0000","last_unstale":"2023-11-28T03:05:00.853572+0000","last_undegraded":"2023-11-28T03:05:00.853572+0000","last_fullsized":"2023-11-28T03:05:00.853572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782115+0000","last_change":"2023-11-28T03:05:00.431077+0000","last_active":"2023-11-28T03:05:00.782115+0000","last_peered":"2023-11-28T03:05:00.782115+0000","last_clean":"2023-11-28T03:05:00.782115+0000","last_became_active":"2023-11-28T03:05:00.430764+0000","last_became_peered":"2023-11-28T03:05:00.430764+0000","last_unstale":"2023-11-28T03:05:00.782115+0000","last_undegraded":"2023-11-28T03:05:00.782115+0000","last_fullsized":"2023-11-28T03:05:00.782115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799252+0000","last_change":"2023-11-28T03:05:00.797610+0000","last_active":"2023-11-28T03:05:00.799252+0000","last_peered":"2023-11-28T03:05:00.799252+0000","last_clean":"2023-11-28T03:05:00.799252+0000","last_became_active":"2023-11-28T03:05:00.797306+0000","last_became_peered":"2023-11-28T03:05:00.797306+0000","last_unstale":"2023-11-28T03:05:00.799252+0000","last_undegraded":"2023-11-28T03:05:00.799252+0000","last_fullsized":"2023-11-28T03:05:00.799252+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782946+0000","last_change":"2023-11-28T03:05:00.426981+0000","last_active":"2023-11-28T03:05:00.782946+0000","last_peered":"2023-11-28T03:05:00.782946+0000","last_clean":"2023-11-28T03:05:00.782946+0000","last_became_active":"2023-11-28T03:05:00.426634+0000","last_became_peered":"2023-11-28T03:05:00.426634+0000","last_unstale":"2023-11-28T03:05:00.782946+0000","last_undegraded":"2023-11-28T03:05:00.782946+0000","last_fullsized":"2023-11-28T03:05:00.782946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799386+0000","last_change":"2023-11-28T03:05:00.795865+0000","last_active":"2023-11-28T03:05:00.799386+0000","last_peered":"2023-11-28T03:05:00.799386+0000","last_clean":"2023-11-28T03:05:00.799386+0000","last_became_active":"2023-11-28T03:05:00.795590+0000","last_became_peered":"2023-11-28T03:05:00.795590+0000","last_unstale":"2023-11-28T03:05:00.799386+0000","last_undegraded":"2023-11-28T03:05:00.799386+0000","last_fullsized":"2023-11-28T03:05:00.799386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797332+0000","last_change":"2023-11-28T03:05:00.427931+0000","last_active":"2023-11-28T03:05:00.797332+0000","last_peered":"2023-11-28T03:05:00.797332+0000","last_clean":"2023-11-28T03:05:00.797332+0000","last_became_active":"2023-11-28T03:05:00.427505+0000","last_became_peered":"2023-11-28T03:05:00.427505+0000","last_unstale":"2023-11-28T03:05:00.797332+0000","last_undegraded":"2023-11-28T03:05:00.797332+0000","last_fullsized":"2023-11-28T03:05:00.797332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795645+0000","last_change":"2023-11-28T03:05:00.792894+0000","last_active":"2023-11-28T03:05:00.795645+0000","last_peered":"2023-11-28T03:05:00.795645+0000","last_clean":"2023-11-28T03:05:00.795645+0000","last_became_active":"2023-11-28T03:05:00.792612+0000","last_became_peered":"2023-11-28T03:05:00.792612+0000","last_unstale":"2023-11-28T03:05:00.795645+0000","last_undegraded":"2023-11-28T03:05:00.795645+0000","last_fullsized":"2023-11-28T03:05:00.795645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780367+0000","last_change":"2023-11-28T03:05:00.425301+0000","last_active":"2023-11-28T03:05:00.780367+0000","last_peered":"2023-11-28T03:05:00.780367+0000","last_clean":"2023-11-28T03:05:00.780367+0000","last_became_active":"2023-11-28T03:05:00.424733+0000","last_became_peered":"2023-11-28T03:05:00.424733+0000","last_unstale":"2023-11-28T03:05:00.780367+0000","last_undegraded":"2023-11-28T03:05:00.780367+0000","last_fullsized":"2023-11-28T03:05:00.780367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806624+0000","last_change":"2023-11-28T03:05:00.801628+0000","last_active":"2023-11-28T03:05:00.806624+0000","last_peered":"2023-11-28T03:05:00.806624+0000","last_clean":"2023-11-28T03:05:00.806624+0000","last_became_active":"2023-11-28T03:05:00.801373+0000","last_became_peered":"2023-11-28T03:05:00.801373+0000","last_unstale":"2023-11-28T03:05:00.806624+0000","last_undegraded":"2023-11-28T03:05:00.806624+0000","last_fullsized":"2023-11-28T03:05:00.806624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783391+0000","last_change":"2023-11-28T03:05:00.430297+0000","last_active":"2023-11-28T03:05:00.783391+0000","last_peered":"2023-11-28T03:05:00.783391+0000","last_clean":"2023-11-28T03:05:00.783391+0000","last_became_active":"2023-11-28T03:05:00.429995+0000","last_became_peered":"2023-11-28T03:05:00.429995+0000","last_unstale":"2023-11-28T03:05:00.783391+0000","last_undegraded":"2023-11-28T03:05:00.783391+0000","last_fullsized":"2023-11-28T03:05:00.783391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797007+0000","last_change":"2023-11-28T03:05:00.795438+0000","last_active":"2023-11-28T03:05:00.797007+0000","last_peered":"2023-11-28T03:05:00.797007+0000","last_clean":"2023-11-28T03:05:00.797007+0000","last_became_active":"2023-11-28T03:05:00.795177+0000","last_became_peered":"2023-11-28T03:05:00.795177+0000","last_unstale":"2023-11-28T03:05:00.797007+0000","last_undegraded":"2023-11-28T03:05:00.797007+0000","last_fullsized":"2023-11-28T03:05:00.797007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784584+0000","last_change":"2023-11-28T03:05:00.431854+0000","last_active":"2023-11-28T03:05:00.784584+0000","last_peered":"2023-11-28T03:05:00.784584+0000","last_clean":"2023-11-28T03:05:00.784584+0000","last_became_active":"2023-11-28T03:05:00.431452+0000","last_became_peered":"2023-11-28T03:05:00.431452+0000","last_unstale":"2023-11-28T03:05:00.784584+0000","last_undegraded":"2023-11-28T03:05:00.784584+0000","last_fullsized":"2023-11-28T03:05:00.784584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806507+0000","last_change":"2023-11-28T03:05:00.800134+0000","last_active":"2023-11-28T03:05:00.806507+0000","last_peered":"2023-11-28T03:05:00.806507+0000","last_clean":"2023-11-28T03:05:00.806507+0000","last_became_active":"2023-11-28T03:05:00.799831+0000","last_became_peered":"2023-11-28T03:05:00.799831+0000","last_unstale":"2023-11-28T03:05:00.806507+0000","last_undegraded":"2023-11-28T03:05:00.806507+0000","last_fullsized":"2023-11-28T03:05:00.806507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783530+0000","last_change":"2023-11-28T03:05:00.422074+0000","last_active":"2023-11-28T03:05:00.783530+0000","last_peered":"2023-11-28T03:05:00.783530+0000","last_clean":"2023-11-28T03:05:00.783530+0000","last_became_active":"2023-11-28T03:05:00.421818+0000","last_became_peered":"2023-11-28T03:05:00.421818+0000","last_unstale":"2023-11-28T03:05:00.783530+0000","last_undegraded":"2023-11-28T03:05:00.783530+0000","last_fullsized":"2023-11-28T03:05:00.783530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786429+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:05:00.786429+0000","last_peered":"2023-11-28T03:05:00.786429+0000","last_clean":"2023-11-28T03:05:00.786429+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:05:00.786429+0000","last_undegraded":"2023-11-28T03:05:00.786429+0000","last_fullsized":"2023-11-28T03:05:00.786429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799800+0000","last_change":"2023-11-28T03:05:00.797902+0000","last_active":"2023-11-28T03:05:00.799800+0000","last_peered":"2023-11-28T03:05:00.799800+0000","last_clean":"2023-11-28T03:05:00.799800+0000","last_became_active":"2023-11-28T03:05:00.797319+0000","last_became_peered":"2023-11-28T03:05:00.797319+0000","last_unstale":"2023-11-28T03:05:00.799800+0000","last_undegraded":"2023-11-28T03:05:00.799800+0000","last_fullsized":"2023-11-28T03:05:00.799800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803283+0000","last_change":"2023-11-28T03:05:00.423249+0000","last_active":"2023-11-28T03:05:00.803283+0000","last_peered":"2023-11-28T03:05:00.803283+0000","last_clean":"2023-11-28T03:05:00.803283+0000","last_became_active":"2023-11-28T03:05:00.422926+0000","last_became_peered":"2023-11-28T03:05:00.422926+0000","last_unstale":"2023-11-28T03:05:00.803283+0000","last_undegraded":"2023-11-28T03:05:00.803283+0000","last_fullsized":"2023-11-28T03:05:00.803283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784573+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:05:00.784573+0000","last_peered":"2023-11-28T03:05:00.784573+0000","last_clean":"2023-11-28T03:05:00.784573+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:05:00.784573+0000","last_undegraded":"2023-11-28T03:05:00.784573+0000","last_fullsized":"2023-11-28T03:05:00.784573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794685+0000","last_change":"2023-11-28T03:05:00.792511+0000","last_active":"2023-11-28T03:05:00.794685+0000","last_peered":"2023-11-28T03:05:00.794685+0000","last_clean":"2023-11-28T03:05:00.794685+0000","last_became_active":"2023-11-28T03:05:00.792289+0000","last_became_peered":"2023-11-28T03:05:00.792289+0000","last_unstale":"2023-11-28T03:05:00.794685+0000","last_undegraded":"2023-11-28T03:05:00.794685+0000","last_fullsized":"2023-11-28T03:05:00.794685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783059+0000","last_change":"2023-11-28T03:05:00.436271+0000","last_active":"2023-11-28T03:05:00.783059+0000","last_peered":"2023-11-28T03:05:00.783059+0000","last_clean":"2023-11-28T03:05:00.783059+0000","last_became_active":"2023-11-28T03:05:00.435974+0000","last_became_peered":"2023-11-28T03:05:00.435974+0000","last_unstale":"2023-11-28T03:05:00.783059+0000","last_undegraded":"2023-11-28T03:05:00.783059+0000","last_fullsized":"2023-11-28T03:05:00.783059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789838+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:05:00.789838+0000","last_peered":"2023-11-28T03:05:00.789838+0000","last_clean":"2023-11-28T03:05:00.789838+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:05:00.789838+0000","last_undegraded":"2023-11-28T03:05:00.789838+0000","last_fullsized":"2023-11-28T03:05:00.789838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799223+0000","last_change":"2023-11-28T03:05:00.797875+0000","last_active":"2023-11-28T03:05:00.799223+0000","last_peered":"2023-11-28T03:05:00.799223+0000","last_clean":"2023-11-28T03:05:00.799223+0000","last_became_active":"2023-11-28T03:05:00.797594+0000","last_became_peered":"2023-11-28T03:05:00.797594+0000","last_unstale":"2023-11-28T03:05:00.799223+0000","last_undegraded":"2023-11-28T03:05:00.799223+0000","last_fullsized":"2023-11-28T03:05:00.799223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782843+0000","last_change":"2023-11-28T03:05:00.437608+0000","last_active":"2023-11-28T03:05:00.782843+0000","last_peered":"2023-11-28T03:05:00.782843+0000","last_clean":"2023-11-28T03:05:00.782843+0000","last_became_active":"2023-11-28T03:05:00.437267+0000","last_became_peered":"2023-11-28T03:05:00.437267+0000","last_unstale":"2023-11-28T03:05:00.782843+0000","last_undegraded":"2023-11-28T03:05:00.782843+0000","last_fullsized":"2023-11-28T03:05:00.782843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785039+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:05:00.785039+0000","last_peered":"2023-11-28T03:05:00.785039+0000","last_clean":"2023-11-28T03:05:00.785039+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:05:00.785039+0000","last_undegraded":"2023-11-28T03:05:00.785039+0000","last_fullsized":"2023-11-28T03:05:00.785039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786291+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:05:00.786291+0000","last_peered":"2023-11-28T03:05:00.786291+0000","last_clean":"2023-11-28T03:05:00.786291+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:05:00.786291+0000","last_undegraded":"2023-11-28T03:05:00.786291+0000","last_fullsized":"2023-11-28T03:05:00.786291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796734+0000","last_change":"2023-11-28T03:05:00.794511+0000","last_active":"2023-11-28T03:05:00.796734+0000","last_peered":"2023-11-28T03:05:00.796734+0000","last_clean":"2023-11-28T03:05:00.796734+0000","last_became_active":"2023-11-28T03:05:00.794228+0000","last_became_peered":"2023-11-28T03:05:00.794228+0000","last_unstale":"2023-11-28T03:05:00.796734+0000","last_undegraded":"2023-11-28T03:05:00.796734+0000","last_fullsized":"2023-11-28T03:05:00.796734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789844+0000","last_change":"2023-11-28T03:05:00.425225+0000","last_active":"2023-11-28T03:05:00.789844+0000","last_peered":"2023-11-28T03:05:00.789844+0000","last_clean":"2023-11-28T03:05:00.789844+0000","last_became_active":"2023-11-28T03:05:00.424968+0000","last_became_peered":"2023-11-28T03:05:00.424968+0000","last_unstale":"2023-11-28T03:05:00.789844+0000","last_undegraded":"2023-11-28T03:05:00.789844+0000","last_fullsized":"2023-11-28T03:05:00.789844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786377+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:05:00.786377+0000","last_peered":"2023-11-28T03:05:00.786377+0000","last_clean":"2023-11-28T03:05:00.786377+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:05:00.786377+0000","last_undegraded":"2023-11-28T03:05:00.786377+0000","last_fullsized":"2023-11-28T03:05:00.786377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801882+0000","last_change":"2023-11-28T03:05:00.793949+0000","last_active":"2023-11-28T03:05:00.801882+0000","last_peered":"2023-11-28T03:05:00.801882+0000","last_clean":"2023-11-28T03:05:00.801882+0000","last_became_active":"2023-11-28T03:05:00.793037+0000","last_became_peered":"2023-11-28T03:05:00.793037+0000","last_unstale":"2023-11-28T03:05:00.801882+0000","last_undegraded":"2023-11-28T03:05:00.801882+0000","last_fullsized":"2023-11-28T03:05:00.801882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787693+0000","last_change":"2023-11-28T03:05:00.431872+0000","last_active":"2023-11-28T03:05:00.787693+0000","last_peered":"2023-11-28T03:05:00.787693+0000","last_clean":"2023-11-28T03:05:00.787693+0000","last_became_active":"2023-11-28T03:05:00.431547+0000","last_became_peered":"2023-11-28T03:05:00.431547+0000","last_unstale":"2023-11-28T03:05:00.787693+0000","last_undegraded":"2023-11-28T03:05:00.787693+0000","last_fullsized":"2023-11-28T03:05:00.787693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789916+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:05:00.789916+0000","last_peered":"2023-11-28T03:05:00.789916+0000","last_clean":"2023-11-28T03:05:00.789916+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:05:00.789916+0000","last_undegraded":"2023-11-28T03:05:00.789916+0000","last_fullsized":"2023-11-28T03:05:00.789916+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807908+0000","last_change":"2023-11-28T03:05:00.805547+0000","last_active":"2023-11-28T03:05:00.807908+0000","last_peered":"2023-11-28T03:05:00.807908+0000","last_clean":"2023-11-28T03:05:00.807908+0000","last_became_active":"2023-11-28T03:05:00.805325+0000","last_became_peered":"2023-11-28T03:05:00.805325+0000","last_unstale":"2023-11-28T03:05:00.807908+0000","last_undegraded":"2023-11-28T03:05:00.807908+0000","last_fullsized":"2023-11-28T03:05:00.807908+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781035+0000","last_change":"2023-11-28T03:05:00.426376+0000","last_active":"2023-11-28T03:05:00.781035+0000","last_peered":"2023-11-28T03:05:00.781035+0000","last_clean":"2023-11-28T03:05:00.781035+0000","last_became_active":"2023-11-28T03:05:00.426131+0000","last_became_peered":"2023-11-28T03:05:00.426131+0000","last_unstale":"2023-11-28T03:05:00.781035+0000","last_undegraded":"2023-11-28T03:05:00.781035+0000","last_fullsized":"2023-11-28T03:05:00.781035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.788011+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:05:00.788011+0000","last_peered":"2023-11-28T03:05:00.788011+0000","last_clean":"2023-11-28T03:05:00.788011+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:05:00.788011+0000","last_undegraded":"2023-11-28T03:05:00.788011+0000","last_fullsized":"2023-11-28T03:05:00.788011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802290+0000","last_change":"2023-11-28T03:05:00.800651+0000","last_active":"2023-11-28T03:05:00.802290+0000","last_peered":"2023-11-28T03:05:00.802290+0000","last_clean":"2023-11-28T03:05:00.802290+0000","last_became_active":"2023-11-28T03:05:00.800387+0000","last_became_peered":"2023-11-28T03:05:00.800387+0000","last_unstale":"2023-11-28T03:05:00.802290+0000","last_undegraded":"2023-11-28T03:05:00.802290+0000","last_fullsized":"2023-11-28T03:05:00.802290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780548+0000","last_change":"2023-11-28T03:05:00.431674+0000","last_active":"2023-11-28T03:05:00.780548+0000","last_peered":"2023-11-28T03:05:00.780548+0000","last_clean":"2023-11-28T03:05:00.780548+0000","last_became_active":"2023-11-28T03:05:00.431396+0000","last_became_peered":"2023-11-28T03:05:00.431396+0000","last_unstale":"2023-11-28T03:05:00.780548+0000","last_undegraded":"2023-11-28T03:05:00.780548+0000","last_fullsized":"2023-11-28T03:05:00.780548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782094+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:05:00.782094+0000","last_peered":"2023-11-28T03:05:00.782094+0000","last_clean":"2023-11-28T03:05:00.782094+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:05:00.782094+0000","last_undegraded":"2023-11-28T03:05:00.782094+0000","last_fullsized":"2023-11-28T03:05:00.782094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.85999999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60899999999999999,"15min":0.59599999999999997},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.435},{"interface":"front","average":{"1min":0.501,"5min":0.56299999999999994,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.749,"15min":0.749},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.749,"15min":0.77100000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75800000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68799999999999994,"5min":0.77800000000000002,"15min":0.83099999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.81899999999999995},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.84499999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83199999999999996},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.79800000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.28999999999999998},"max":{"1min":0.749,"5min":0.77800000000000002,"15min":0.84499999999999997},"last":0.58599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.379,"15min":0.32800000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.746},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.81000000000000005,"15min":0.84399999999999997},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.746,"15min":0.82499999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.317},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.38600000000000001,"5min":0.56599999999999995,"15min":0.70699999999999996},"last":0.247},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.52700000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.43099999999999999,"5min":0.56200000000000006,"15min":0.73499999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.27600000000000002,"15min":0.19800000000000001},"max":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.72599999999999998},"last":0.36099999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.52800000000000002,"5min":0.66300000000000003,"15min":0.749},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.44,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.54400000000000004,"5min":0.60899999999999999,"15min":1.6870000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":1.0900000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.038,"15min":1.111},"last":0.72899999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":1.8080000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.073},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.81499999999999995,"5min":1.024,"15min":1.024},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.38700000000000001},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297864,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":786432,"data_stored":379918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.184},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.313},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.436},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.38900000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.24099999999999999},"max":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.41299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.621,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63900000000000001,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88500000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65100000000000002,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.39000000000000001},"max":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42599999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.36299999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97399999999999998,"5min":1.046,"15min":1.046},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":381328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.374,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.20499999999999999},"max":{"1min":0.41099999999999998,"5min":0.57399999999999995,"15min":0.623},"last":0.247},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.42199999999999999,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23599999999999999},"max":{"1min":0.47599999999999998,"5min":0.64700000000000002,"15min":0.65900000000000003},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.377},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51100000000000001,"5min":0.68600000000000005,"15min":0.73099999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.40600000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53600000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.94599999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":1.004,"15min":1.004},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74399999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36899999999999999},"max":{"1min":0.73699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36799999999999999,"15min":0.34699999999999998},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.501,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38900000000000001,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.19900000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.60899999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.44800000000000001,"15min":0.42599999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.68200000000000005},"last":0.375}]},{"osd":4,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6000000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.621,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70699999999999996,"5min":0.94199999999999995,"15min":1.196},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.996,"15min":1.206},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.97199999999999998,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.56499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.372,"15min":0.27300000000000002},"max":{"1min":0.73099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.32800000000000001,"15min":0.314},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.30199999999999999},"max":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.82199999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.38,"15min":0.33500000000000002},"max":{"1min":0.73799999999999999,"5min":0.76300000000000001,"15min":0.78100000000000003},"last":0.441}]},{"osd":3,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.39000000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.878,"15min":0.878},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.69199999999999995,"15min":0.71699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.622,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.82699999999999996},"last":0.501}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69199999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.432},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.377,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.76400000000000001,"15min":0.79800000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78700000000000003},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42599999999999999,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:05:05.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T03:05:05.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.246+0000 7f5b4c5f9700 1 Processor -- start 2023-11-28T03:05:05.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.246+0000 7f5b4c5f9700 1 -- start start 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4c5f9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b44064e20 0x7f5b44155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4c5f9700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b440683d0 con 0x7f5b44095e80 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4c5f9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b44067850 con 0x7f5b441544d0 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4c5f9700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b44067650 con 0x7f5b44064e20 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b49b94700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b44064e20 0x7f5b44155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4ab96700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5b44095e80 0x7f5b4415e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57424/0 (socket says 172.21.15.73:57424) 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.247+0000 7f5b4ab96700 1 -- 172.21.15.73:0/2865291614 learned_addr learned my addr 172.21.15.73:0/2865291614 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49b94700 1 -- 172.21.15.73:0/2865291614 >> v1:172.21.15.150:6789/0 conn(0x7f5b441544d0 legacy=0x7f5b44066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49b94700 1 -- 172.21.15.73:0/2865291614 >> v1:172.21.15.73:6789/0 conn(0x7f5b44095e80 legacy=0x7f5b4415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49b94700 1 -- 172.21.15.73:0/2865291614 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b44067b30 con 0x7f5b44064e20 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49393700 1 -- 172.21.15.73:0/2865291614 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2781877917 0 0) 0x7f5b440683d0 con 0x7f5b44095e80 2023-11-28T03:05:05.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49b94700 1 --2- 172.21.15.73:0/2865291614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44064e20 0x7f5b44155750 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f5b34009760 tx=0x7f5b34009060).ready entity=mon.0 client_cookie=4e86aa529c16ca7e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49393700 1 -- 172.21.15.73:0/2865291614 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400b520 con 0x7f5b44064e20 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49393700 1 -- 172.21.15.73:0/2865291614 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400b680 con 0x7f5b44064e20 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b49393700 1 -- 172.21.15.73:0/2865291614 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400ba00 con 0x7f5b44064e20 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/2865291614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44064e20 msgr2=0x7f5b44155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.248+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/2865291614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44064e20 0x7f5b44155750 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 rx=0x7f5b34009760 tx=0x7f5b34009060).stop 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.249+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/2865291614 shutdown_connections 2023-11-28T03:05:05.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.249+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/2865291614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44064e20 0x7f5b44155750 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.249+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/2865291614 >> 172.21.15.73:0/2865291614 conn(0x7f5b440631d0 msgr2=0x7f5b440c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:05.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.249+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/2865291614 shutdown_connections 2023-11-28T03:05:05.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.249+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/2865291614 wait complete. 2023-11-28T03:05:05.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 Processor -- start 2023-11-28T03:05:05.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 -- start start 2023-11-28T03:05:05.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b44064e20 0x7f5b4414dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b441544d0 0x7f5b440adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b440ad720 con 0x7f5b44095e80 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b4414e1c0 con 0x7f5b441544d0 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.250+0000 7f5b4c5f9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b440c1550 con 0x7f5b44064e20 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54498/0 (socket says 172.21.15.73:54498) 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 -- 172.21.15.73:0/3533823603 learned_addr learned my addr 172.21.15.73:0/3533823603 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b4ab96700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b441544d0 0x7f5b440adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b4a395700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b44064e20 0x7f5b4414dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 -- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b44064e20 msgr2=0x7f5b4414dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b44064e20 0x7f5b4414dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 -- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b441544d0 msgr2=0x7f5b440adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b441544d0 0x7f5b440adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 -- 172.21.15.73:0/3533823603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b340092a0 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b49b94700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5b4406c7f0 tx=0x7f5b34017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3400b7f0 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b3400b950 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b440c1830 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b3401b620 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.251+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b440c1210 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.252+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b4406c860 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.252+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5b3401ba10 con 0x7f5b44095e80 2023-11-28T03:05:05.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.253+0000 7f5b3a7fc700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 0x7f5b28061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.253+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5b3407bd30 con 0x7f5b44095e80 2023-11-28T03:05:05.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.253+0000 7f5b4a395700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 0x7f5b28061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.253+0000 7f5b4a395700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 0x7f5b28061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5b40009b30 tx=0x7f5b40008040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.257+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b3404d000 con 0x7f5b44095e80 2023-11-28T03:05:05.388 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:05:05.387+0000 7f2624ec8700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:05:05.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.486+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f5b440ad860 con 0x7f5b44095e80 2023-11-28T03:05:05.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.486+0000 7f5b3a7fc700 1 -- 172.21.15.73:0/3533823603 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+952 (secure 0 0 0) 0x7f5b34028020 con 0x7f5b44095e80 2023-11-28T03:05:05.487 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:05.487 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":759,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":65,"num_osds":8,"num_up_osds":8,"osd_up_since":1701139955,"num_in_osds":8,"osd_in_since":1701139948,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":137}],"num_pgs":137,"num_pools":4,"num_objects":21,"data_bytes":2306,"bytes_used":2469429248,"bytes_avail":770624684032,"bytes_total":773094113280},"fsmap":{"epoch":20,"id":3,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":3,"rank":0,"name":"a","status":"up:active","gid":5775}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2023-11-28T02:54:45.738944+0000","services":{}},"progress_events":{}} 2023-11-28T03:05:05.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 msgr2=0x7f5b28061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 0x7f5b28061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f5b40009b30 tx=0x7f5b40008040).stop 2023-11-28T03:05:05.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 msgr2=0x7f5b440c0d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 rx=0x7f5b4406c7f0 tx=0x7f5b34017000).stop 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 shutdown_connections 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b2805f150 0x7f5b28061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b441544d0 0x7f5b440adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b44064e20 0x7f5b4414dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 --2- 172.21.15.73:0/3533823603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b44095e80 0x7f5b440c0d20 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 >> 172.21.15.73:0/3533823603 conn(0x7f5b440631d0 msgr2=0x7f5b440c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 shutdown_connections 2023-11-28T03:05:05.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.488+0000 7f5b4c5f9700 1 -- 172.21.15.73:0/3533823603 wait complete. 2023-11-28T03:05:05.501 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T03:05:05.501 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:05:05.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.613+0000 7fe040f49700 1 Processor -- start 2023-11-28T03:05:05.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.613+0000 7fe040f49700 1 -- start start 2023-11-28T03:05:05.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe040f49700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe03c1544e0 0x7fe03c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe040f49700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe03c06a610 0x7fe03c0c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe040f49700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe03c0c69a0 con 0x7fe03c0c6e30 2023-11-28T03:05:05.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe040f49700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0c13a0 con 0x7fe03c06a610 2023-11-28T03:05:05.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe040f49700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03c0c1680 con 0x7fe03c1544e0 2023-11-28T03:05:05.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe031d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe03c06a610 0x7fe03c0c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe031d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe03c06a610 0x7fe03c0c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54500/0 (socket says 172.21.15.73:54500) 2023-11-28T03:05:05.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe031d9b700 1 -- 172.21.15.73:0/224089999 learned_addr learned my addr 172.21.15.73:0/224089999 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:05.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.614+0000 7fe03a59c700 1 --2- 172.21.15.73:0/224089999 >> v2:172.21.15.150:3300/0 conn(0x7fe03c1544e0 0x7fe03c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe031d9b700 1 -- 172.21.15.73:0/224089999 >> v2:172.21.15.150:3300/0 conn(0x7fe03c1544e0 msgr2=0x7fe03c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:05.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe031d9b700 1 --2- 172.21.15.73:0/224089999 >> v2:172.21.15.150:3300/0 conn(0x7fe03c1544e0 0x7fe03c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe031d9b700 1 -- 172.21.15.73:0/224089999 >> v1:172.21.15.150:6789/0 conn(0x7fe03c0c6e30 legacy=0x7fe03c0a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe031d9b700 1 -- 172.21.15.73:0/224089999 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03c0c6630 con 0x7fe03c06a610 2023-11-28T03:05:05.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe031d9b700 1 --2- 172.21.15.73:0/224089999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c06a610 0x7fe03c0c0bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fe03c085760 tx=0x7fe0240090f0).ready entity=mon.0 client_cookie=b0c932718efb6bd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe039d9b700 1 -- 172.21.15.73:0/224089999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02400b520 con 0x7fe03c06a610 2023-11-28T03:05:05.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe039d9b700 1 -- 172.21.15.73:0/224089999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02400b680 con 0x7fe03c06a610 2023-11-28T03:05:05.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe039d9b700 1 -- 172.21.15.73:0/224089999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe02400ba00 con 0x7fe03c06a610 2023-11-28T03:05:05.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe040f49700 1 -- 172.21.15.73:0/224089999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c06a610 msgr2=0x7fe03c0c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.615+0000 7fe040f49700 1 --2- 172.21.15.73:0/224089999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c06a610 0x7fe03c0c0bc0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 rx=0x7fe03c085760 tx=0x7fe0240090f0).stop 2023-11-28T03:05:05.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 -- 172.21.15.73:0/224089999 shutdown_connections 2023-11-28T03:05:05.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 --2- 172.21.15.73:0/224089999 >> v2:172.21.15.150:3300/0 conn(0x7fe03c1544e0 0x7fe03c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 --2- 172.21.15.73:0/224089999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c06a610 0x7fe03c0c0bc0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 -- 172.21.15.73:0/224089999 >> 172.21.15.73:0/224089999 conn(0x7fe03c085020 msgr2=0x7fe03c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:05.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 -- 172.21.15.73:0/224089999 shutdown_connections 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.616+0000 7fe040f49700 1 -- 172.21.15.73:0/224089999 wait complete. 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.617+0000 7fe040f49700 1 Processor -- start 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.617+0000 7fe040f49700 1 -- start start 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c06a610 0x7fe03c0c2d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c1544e0 0x7fe03c151220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c094940 con 0x7fe03c0c6e30 2023-11-28T03:05:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c1442a0 con 0x7fe03c1544e0 2023-11-28T03:05:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03c0c3330 con 0x7fe03c06a610 2023-11-28T03:05:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54506/0 (socket says 172.21.15.73:54506) 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 -- 172.21.15.73:0/251369547 learned_addr learned my addr 172.21.15.73:0/251369547 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe03a59c700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c06a610 0x7fe03c0c2d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe03ad9d700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c1544e0 0x7fe03c151220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 -- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c06a610 msgr2=0x7fe03c0c2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c06a610 0x7fe03c0c2d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 -- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c1544e0 msgr2=0x7fe03c151220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c1544e0 0x7fe03c151220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 -- 172.21.15.73:0/251369547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe024009370 con 0x7fe03c0c6e30 2023-11-28T03:05:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe031d9b700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fe024009130 tx=0x7fe024017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024000bc0 con 0x7fe03c0c6e30 2023-11-28T03:05:05.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe02400b7f0 con 0x7fe03c0c6e30 2023-11-28T03:05:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe03c0c3610 con 0x7fe03c0c6e30 2023-11-28T03:05:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe024000d20 con 0x7fe03c0c6e30 2023-11-28T03:05:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.618+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe03c0c3bd0 con 0x7fe03c0c6e30 2023-11-28T03:05:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.619+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe03c094a80 con 0x7fe03c0c6e30 2023-11-28T03:05:05.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.620+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe02401b8b0 con 0x7fe03c0c6e30 2023-11-28T03:05:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.620+0000 7fe032ffd700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 0x7fe01c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.620+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe02407bb70 con 0x7fe03c0c6e30 2023-11-28T03:05:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.620+0000 7fe03a59c700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 0x7fe01c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:05.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.620+0000 7fe03a59c700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 0x7fe01c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fe02c00f5b0 tx=0x7fe02c00b040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:05.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.624+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe03c094a80 con 0x7fe03c0c6e30 2023-11-28T03:05:05.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.783+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe03c094a80 con 0x7fe01c05f150 2023-11-28T03:05:05.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.792+0000 7fe032ffd700 1 -- 172.21.15.73:0/251369547 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361534 (secure 0 0 0) 0x7fe03c094a80 con 0x7fe01c05f150 2023-11-28T03:05:05.800 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:05.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 msgr2=0x7fe01c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 0x7fe01c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fe02c00f5b0 tx=0x7fe02c00b040).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 msgr2=0x7fe03c143db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7fe024009130 tx=0x7fe024017000).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 shutdown_connections 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe01c05f150 0x7fe01c061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe03c1544e0 0x7fe03c151220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe03c06a610 0x7fe03c0c2d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 --2- 172.21.15.73:0/251369547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe03c0c6e30 0x7fe03c143db0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 >> 172.21.15.73:0/251369547 conn(0x7fe03c085020 msgr2=0x7fe03c0aeba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 shutdown_connections 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:05.796+0000 7fe040f49700 1 -- 172.21.15.73:0/251369547 wait complete. 2023-11-28T03:05:05.805 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:05:05.808 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-11-28T03:05:04.869760+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":195,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411552,"kb_used_data":6112,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752563168,"statfs":{"total":773094113280,"available":770624684032,"internally_reserved":0,"allocated":6258688,"data_stored":3042948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14996258,"num_objects":-26,"num_object_clones":0,"num_object_copies":-52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-26,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-721,"num_write_kb":-15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.498335"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796416+0000","last_change":"2023-11-28T03:05:00.791917+0000","last_active":"2023-11-28T03:05:00.796416+0000","last_peered":"2023-11-28T03:05:00.796416+0000","last_clean":"2023-11-28T03:05:00.796416+0000","last_became_active":"2023-11-28T03:05:00.791612+0000","last_became_peered":"2023-11-28T03:05:00.791612+0000","last_unstale":"2023-11-28T03:05:00.796416+0000","last_undegraded":"2023-11-28T03:05:00.796416+0000","last_fullsized":"2023-11-28T03:05:00.796416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795306+0000","last_change":"2023-11-28T03:05:00.425524+0000","last_active":"2023-11-28T03:05:00.795306+0000","last_peered":"2023-11-28T03:05:00.795306+0000","last_clean":"2023-11-28T03:05:00.795306+0000","last_became_active":"2023-11-28T03:05:00.425163+0000","last_became_peered":"2023-11-28T03:05:00.425163+0000","last_unstale":"2023-11-28T03:05:00.795306+0000","last_undegraded":"2023-11-28T03:05:00.795306+0000","last_fullsized":"2023-11-28T03:05:00.795306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800549+0000","last_change":"2023-11-28T03:05:00.795761+0000","last_active":"2023-11-28T03:05:00.800549+0000","last_peered":"2023-11-28T03:05:00.800549+0000","last_clean":"2023-11-28T03:05:00.800549+0000","last_became_active":"2023-11-28T03:05:00.795500+0000","last_became_peered":"2023-11-28T03:05:00.795500+0000","last_unstale":"2023-11-28T03:05:00.800549+0000","last_undegraded":"2023-11-28T03:05:00.800549+0000","last_fullsized":"2023-11-28T03:05:00.800549+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784809+0000","last_change":"2023-11-28T03:05:00.438659+0000","last_active":"2023-11-28T03:05:00.784809+0000","last_peered":"2023-11-28T03:05:00.784809+0000","last_clean":"2023-11-28T03:05:00.784809+0000","last_became_active":"2023-11-28T03:05:00.438343+0000","last_became_peered":"2023-11-28T03:05:00.438343+0000","last_unstale":"2023-11-28T03:05:00.784809+0000","last_undegraded":"2023-11-28T03:05:00.784809+0000","last_fullsized":"2023-11-28T03:05:00.784809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807835+0000","last_change":"2023-11-28T03:05:00.804536+0000","last_active":"2023-11-28T03:05:00.807835+0000","last_peered":"2023-11-28T03:05:00.807835+0000","last_clean":"2023-11-28T03:05:00.807835+0000","last_became_active":"2023-11-28T03:05:00.804274+0000","last_became_peered":"2023-11-28T03:05:00.804274+0000","last_unstale":"2023-11-28T03:05:00.807835+0000","last_undegraded":"2023-11-28T03:05:00.807835+0000","last_fullsized":"2023-11-28T03:05:00.807835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783457+0000","last_change":"2023-11-28T03:05:00.413262+0000","last_active":"2023-11-28T03:05:00.783457+0000","last_peered":"2023-11-28T03:05:00.783457+0000","last_clean":"2023-11-28T03:05:00.783457+0000","last_became_active":"2023-11-28T03:05:00.412961+0000","last_became_peered":"2023-11-28T03:05:00.412961+0000","last_unstale":"2023-11-28T03:05:00.783457+0000","last_undegraded":"2023-11-28T03:05:00.783457+0000","last_fullsized":"2023-11-28T03:05:00.783457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796118+0000","last_change":"2023-11-28T03:05:00.791909+0000","last_active":"2023-11-28T03:05:00.796118+0000","last_peered":"2023-11-28T03:05:00.796118+0000","last_clean":"2023-11-28T03:05:00.796118+0000","last_became_active":"2023-11-28T03:05:00.791622+0000","last_became_peered":"2023-11-28T03:05:00.791622+0000","last_unstale":"2023-11-28T03:05:00.796118+0000","last_undegraded":"2023-11-28T03:05:00.796118+0000","last_fullsized":"2023-11-28T03:05:00.796118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801601+0000","last_change":"2023-11-28T03:05:00.436611+0000","last_active":"2023-11-28T03:05:00.801601+0000","last_peered":"2023-11-28T03:05:00.801601+0000","last_clean":"2023-11-28T03:05:00.801601+0000","last_became_active":"2023-11-28T03:05:00.436324+0000","last_became_peered":"2023-11-28T03:05:00.436324+0000","last_unstale":"2023-11-28T03:05:00.801601+0000","last_undegraded":"2023-11-28T03:05:00.801601+0000","last_fullsized":"2023-11-28T03:05:00.801601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804685+0000","last_change":"2023-11-28T03:05:00.791992+0000","last_active":"2023-11-28T03:05:00.804685+0000","last_peered":"2023-11-28T03:05:00.804685+0000","last_clean":"2023-11-28T03:05:00.804685+0000","last_became_active":"2023-11-28T03:05:00.791738+0000","last_became_peered":"2023-11-28T03:05:00.791738+0000","last_unstale":"2023-11-28T03:05:00.804685+0000","last_undegraded":"2023-11-28T03:05:00.804685+0000","last_fullsized":"2023-11-28T03:05:00.804685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804714+0000","last_change":"2023-11-28T03:05:00.437137+0000","last_active":"2023-11-28T03:05:00.804714+0000","last_peered":"2023-11-28T03:05:00.804714+0000","last_clean":"2023-11-28T03:05:00.804714+0000","last_became_active":"2023-11-28T03:05:00.436830+0000","last_became_peered":"2023-11-28T03:05:00.436830+0000","last_unstale":"2023-11-28T03:05:00.804714+0000","last_undegraded":"2023-11-28T03:05:00.804714+0000","last_fullsized":"2023-11-28T03:05:00.804714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801683+0000","last_change":"2023-11-28T03:05:00.798043+0000","last_active":"2023-11-28T03:05:00.801683+0000","last_peered":"2023-11-28T03:05:00.801683+0000","last_clean":"2023-11-28T03:05:00.801683+0000","last_became_active":"2023-11-28T03:05:00.797785+0000","last_became_peered":"2023-11-28T03:05:00.797785+0000","last_unstale":"2023-11-28T03:05:00.801683+0000","last_undegraded":"2023-11-28T03:05:00.801683+0000","last_fullsized":"2023-11-28T03:05:00.801683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794948+0000","last_change":"2023-11-28T03:05:00.425489+0000","last_active":"2023-11-28T03:05:00.794948+0000","last_peered":"2023-11-28T03:05:00.794948+0000","last_clean":"2023-11-28T03:05:00.794948+0000","last_became_active":"2023-11-28T03:05:00.425041+0000","last_became_peered":"2023-11-28T03:05:00.425041+0000","last_unstale":"2023-11-28T03:05:00.794948+0000","last_undegraded":"2023-11-28T03:05:00.794948+0000","last_fullsized":"2023-11-28T03:05:00.794948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796138+0000","last_change":"2023-11-28T03:05:00.792581+0000","last_active":"2023-11-28T03:05:00.796138+0000","last_peered":"2023-11-28T03:05:00.796138+0000","last_clean":"2023-11-28T03:05:00.796138+0000","last_became_active":"2023-11-28T03:05:00.792322+0000","last_became_peered":"2023-11-28T03:05:00.792322+0000","last_unstale":"2023-11-28T03:05:00.796138+0000","last_undegraded":"2023-11-28T03:05:00.796138+0000","last_fullsized":"2023-11-28T03:05:00.796138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787429+0000","last_change":"2023-11-28T03:05:00.431726+0000","last_active":"2023-11-28T03:05:00.787429+0000","last_peered":"2023-11-28T03:05:00.787429+0000","last_clean":"2023-11-28T03:05:00.787429+0000","last_became_active":"2023-11-28T03:05:00.431380+0000","last_became_peered":"2023-11-28T03:05:00.431380+0000","last_unstale":"2023-11-28T03:05:00.787429+0000","last_undegraded":"2023-11-28T03:05:00.787429+0000","last_fullsized":"2023-11-28T03:05:00.787429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807353+0000","last_change":"2023-11-28T03:05:00.803609+0000","last_active":"2023-11-28T03:05:00.807353+0000","last_peered":"2023-11-28T03:05:00.807353+0000","last_clean":"2023-11-28T03:05:00.807353+0000","last_became_active":"2023-11-28T03:05:00.803362+0000","last_became_peered":"2023-11-28T03:05:00.803362+0000","last_unstale":"2023-11-28T03:05:00.807353+0000","last_undegraded":"2023-11-28T03:05:00.807353+0000","last_fullsized":"2023-11-28T03:05:00.807353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785509+0000","last_change":"2023-11-28T03:05:00.420800+0000","last_active":"2023-11-28T03:05:00.785509+0000","last_peered":"2023-11-28T03:05:00.785509+0000","last_clean":"2023-11-28T03:05:00.785509+0000","last_became_active":"2023-11-28T03:05:00.420414+0000","last_became_peered":"2023-11-28T03:05:00.420414+0000","last_unstale":"2023-11-28T03:05:00.785509+0000","last_undegraded":"2023-11-28T03:05:00.785509+0000","last_fullsized":"2023-11-28T03:05:00.785509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805747+0000","last_change":"2023-11-28T03:05:00.795247+0000","last_active":"2023-11-28T03:05:00.805747+0000","last_peered":"2023-11-28T03:05:00.805747+0000","last_clean":"2023-11-28T03:05:00.805747+0000","last_became_active":"2023-11-28T03:05:00.795001+0000","last_became_peered":"2023-11-28T03:05:00.795001+0000","last_unstale":"2023-11-28T03:05:00.805747+0000","last_undegraded":"2023-11-28T03:05:00.805747+0000","last_fullsized":"2023-11-28T03:05:00.805747+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784311+0000","last_change":"2023-11-28T03:05:00.429277+0000","last_active":"2023-11-28T03:05:00.784311+0000","last_peered":"2023-11-28T03:05:00.784311+0000","last_clean":"2023-11-28T03:05:00.784311+0000","last_became_active":"2023-11-28T03:05:00.428938+0000","last_became_peered":"2023-11-28T03:05:00.428938+0000","last_unstale":"2023-11-28T03:05:00.784311+0000","last_undegraded":"2023-11-28T03:05:00.784311+0000","last_fullsized":"2023-11-28T03:05:00.784311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806756+0000","last_change":"2023-11-28T03:05:00.803701+0000","last_active":"2023-11-28T03:05:00.806756+0000","last_peered":"2023-11-28T03:05:00.806756+0000","last_clean":"2023-11-28T03:05:00.806756+0000","last_became_active":"2023-11-28T03:05:00.803424+0000","last_became_peered":"2023-11-28T03:05:00.803424+0000","last_unstale":"2023-11-28T03:05:00.806756+0000","last_undegraded":"2023-11-28T03:05:00.806756+0000","last_fullsized":"2023-11-28T03:05:00.806756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783367+0000","last_change":"2023-11-28T03:05:00.437611+0000","last_active":"2023-11-28T03:05:00.783367+0000","last_peered":"2023-11-28T03:05:00.783367+0000","last_clean":"2023-11-28T03:05:00.783367+0000","last_became_active":"2023-11-28T03:05:00.437351+0000","last_became_peered":"2023-11-28T03:05:00.437351+0000","last_unstale":"2023-11-28T03:05:00.783367+0000","last_undegraded":"2023-11-28T03:05:00.783367+0000","last_fullsized":"2023-11-28T03:05:00.783367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806347+0000","last_change":"2023-11-28T03:05:00.801846+0000","last_active":"2023-11-28T03:05:00.806347+0000","last_peered":"2023-11-28T03:05:00.806347+0000","last_clean":"2023-11-28T03:05:00.806347+0000","last_became_active":"2023-11-28T03:05:00.801594+0000","last_became_peered":"2023-11-28T03:05:00.801594+0000","last_unstale":"2023-11-28T03:05:00.806347+0000","last_undegraded":"2023-11-28T03:05:00.806347+0000","last_fullsized":"2023-11-28T03:05:00.806347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783455+0000","last_change":"2023-11-28T03:05:00.437201+0000","last_active":"2023-11-28T03:05:00.783455+0000","last_peered":"2023-11-28T03:05:00.783455+0000","last_clean":"2023-11-28T03:05:00.783455+0000","last_became_active":"2023-11-28T03:05:00.436875+0000","last_became_peered":"2023-11-28T03:05:00.436875+0000","last_unstale":"2023-11-28T03:05:00.783455+0000","last_undegraded":"2023-11-28T03:05:00.783455+0000","last_fullsized":"2023-11-28T03:05:00.783455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854418+0000","last_change":"2023-11-28T03:05:00.853613+0000","last_active":"2023-11-28T03:05:00.854418+0000","last_peered":"2023-11-28T03:05:00.854418+0000","last_clean":"2023-11-28T03:05:00.854418+0000","last_became_active":"2023-11-28T03:05:00.853268+0000","last_became_peered":"2023-11-28T03:05:00.853268+0000","last_unstale":"2023-11-28T03:05:00.854418+0000","last_undegraded":"2023-11-28T03:05:00.854418+0000","last_fullsized":"2023-11-28T03:05:00.854418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809619+0000","last_change":"2023-11-28T03:05:00.424153+0000","last_active":"2023-11-28T03:05:00.809619+0000","last_peered":"2023-11-28T03:05:00.809619+0000","last_clean":"2023-11-28T03:05:00.809619+0000","last_became_active":"2023-11-28T03:05:00.423700+0000","last_became_peered":"2023-11-28T03:05:00.423700+0000","last_unstale":"2023-11-28T03:05:00.809619+0000","last_undegraded":"2023-11-28T03:05:00.809619+0000","last_fullsized":"2023-11-28T03:05:00.809619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809404+0000","last_change":"2023-11-28T03:05:00.805354+0000","last_active":"2023-11-28T03:05:00.809404+0000","last_peered":"2023-11-28T03:05:00.809404+0000","last_clean":"2023-11-28T03:05:00.809404+0000","last_became_active":"2023-11-28T03:05:00.805055+0000","last_became_peered":"2023-11-28T03:05:00.805055+0000","last_unstale":"2023-11-28T03:05:00.809404+0000","last_undegraded":"2023-11-28T03:05:00.809404+0000","last_fullsized":"2023-11-28T03:05:00.809404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807391+0000","last_change":"2023-11-28T03:05:00.425518+0000","last_active":"2023-11-28T03:05:00.807391+0000","last_peered":"2023-11-28T03:05:00.807391+0000","last_clean":"2023-11-28T03:05:00.807391+0000","last_became_active":"2023-11-28T03:05:00.425264+0000","last_became_peered":"2023-11-28T03:05:00.425264+0000","last_unstale":"2023-11-28T03:05:00.807391+0000","last_undegraded":"2023-11-28T03:05:00.807391+0000","last_fullsized":"2023-11-28T03:05:00.807391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806750+0000","last_change":"2023-11-28T03:05:00.802976+0000","last_active":"2023-11-28T03:05:00.806750+0000","last_peered":"2023-11-28T03:05:00.806750+0000","last_clean":"2023-11-28T03:05:00.806750+0000","last_became_active":"2023-11-28T03:05:00.802705+0000","last_became_peered":"2023-11-28T03:05:00.802705+0000","last_unstale":"2023-11-28T03:05:00.806750+0000","last_undegraded":"2023-11-28T03:05:00.806750+0000","last_fullsized":"2023-11-28T03:05:00.806750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787102+0000","last_change":"2023-11-28T03:05:00.429504+0000","last_active":"2023-11-28T03:05:00.787102+0000","last_peered":"2023-11-28T03:05:00.787102+0000","last_clean":"2023-11-28T03:05:00.787102+0000","last_became_active":"2023-11-28T03:05:00.429272+0000","last_became_peered":"2023-11-28T03:05:00.429272+0000","last_unstale":"2023-11-28T03:05:00.787102+0000","last_undegraded":"2023-11-28T03:05:00.787102+0000","last_fullsized":"2023-11-28T03:05:00.787102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802325+0000","last_change":"2023-11-28T03:05:00.800107+0000","last_active":"2023-11-28T03:05:00.802325+0000","last_peered":"2023-11-28T03:05:00.802325+0000","last_clean":"2023-11-28T03:05:00.802325+0000","last_became_active":"2023-11-28T03:05:00.799586+0000","last_became_peered":"2023-11-28T03:05:00.799586+0000","last_unstale":"2023-11-28T03:05:00.802325+0000","last_undegraded":"2023-11-28T03:05:00.802325+0000","last_fullsized":"2023-11-28T03:05:00.802325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780802+0000","last_change":"2023-11-28T03:05:00.423474+0000","last_active":"2023-11-28T03:05:00.780802+0000","last_peered":"2023-11-28T03:05:00.780802+0000","last_clean":"2023-11-28T03:05:00.780802+0000","last_became_active":"2023-11-28T03:05:00.423230+0000","last_became_peered":"2023-11-28T03:05:00.423230+0000","last_unstale":"2023-11-28T03:05:00.780802+0000","last_undegraded":"2023-11-28T03:05:00.780802+0000","last_fullsized":"2023-11-28T03:05:00.780802+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799306+0000","last_change":"2023-11-28T03:05:00.797655+0000","last_active":"2023-11-28T03:05:00.799306+0000","last_peered":"2023-11-28T03:05:00.799306+0000","last_clean":"2023-11-28T03:05:00.799306+0000","last_became_active":"2023-11-28T03:05:00.797293+0000","last_became_peered":"2023-11-28T03:05:00.797293+0000","last_unstale":"2023-11-28T03:05:00.799306+0000","last_undegraded":"2023-11-28T03:05:00.799306+0000","last_fullsized":"2023-11-28T03:05:00.799306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805668+0000","last_change":"2023-11-28T03:05:00.423972+0000","last_active":"2023-11-28T03:05:00.805668+0000","last_peered":"2023-11-28T03:05:00.805668+0000","last_clean":"2023-11-28T03:05:00.805668+0000","last_became_active":"2023-11-28T03:05:00.423595+0000","last_became_peered":"2023-11-28T03:05:00.423595+0000","last_unstale":"2023-11-28T03:05:00.805668+0000","last_undegraded":"2023-11-28T03:05:00.805668+0000","last_fullsized":"2023-11-28T03:05:00.805668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801748+0000","last_change":"2023-11-28T03:05:00.799701+0000","last_active":"2023-11-28T03:05:00.801748+0000","last_peered":"2023-11-28T03:05:00.801748+0000","last_clean":"2023-11-28T03:05:00.801748+0000","last_became_active":"2023-11-28T03:05:00.797686+0000","last_became_peered":"2023-11-28T03:05:00.797686+0000","last_unstale":"2023-11-28T03:05:00.801748+0000","last_undegraded":"2023-11-28T03:05:00.801748+0000","last_fullsized":"2023-11-28T03:05:00.801748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783803+0000","last_change":"2023-11-28T03:05:00.422200+0000","last_active":"2023-11-28T03:05:00.783803+0000","last_peered":"2023-11-28T03:05:00.783803+0000","last_clean":"2023-11-28T03:05:00.783803+0000","last_became_active":"2023-11-28T03:05:00.421929+0000","last_became_peered":"2023-11-28T03:05:00.421929+0000","last_unstale":"2023-11-28T03:05:00.783803+0000","last_undegraded":"2023-11-28T03:05:00.783803+0000","last_fullsized":"2023-11-28T03:05:00.783803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806499+0000","last_change":"2023-11-28T03:05:00.800024+0000","last_active":"2023-11-28T03:05:00.806499+0000","last_peered":"2023-11-28T03:05:00.806499+0000","last_clean":"2023-11-28T03:05:00.806499+0000","last_became_active":"2023-11-28T03:05:00.799789+0000","last_became_peered":"2023-11-28T03:05:00.799789+0000","last_unstale":"2023-11-28T03:05:00.806499+0000","last_undegraded":"2023-11-28T03:05:00.806499+0000","last_fullsized":"2023-11-28T03:05:00.806499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783737+0000","last_change":"2023-11-28T03:05:00.436677+0000","last_active":"2023-11-28T03:05:00.783737+0000","last_peered":"2023-11-28T03:05:00.783737+0000","last_clean":"2023-11-28T03:05:00.783737+0000","last_became_active":"2023-11-28T03:05:00.436405+0000","last_became_peered":"2023-11-28T03:05:00.436405+0000","last_unstale":"2023-11-28T03:05:00.783737+0000","last_undegraded":"2023-11-28T03:05:00.783737+0000","last_fullsized":"2023-11-28T03:05:00.783737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801912+0000","last_change":"2023-11-28T03:05:00.796280+0000","last_active":"2023-11-28T03:05:00.801912+0000","last_peered":"2023-11-28T03:05:00.801912+0000","last_clean":"2023-11-28T03:05:00.801912+0000","last_became_active":"2023-11-28T03:05:00.795931+0000","last_became_peered":"2023-11-28T03:05:00.795931+0000","last_unstale":"2023-11-28T03:05:00.801912+0000","last_undegraded":"2023-11-28T03:05:00.801912+0000","last_fullsized":"2023-11-28T03:05:00.801912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805408+0000","last_change":"2023-11-28T03:05:00.437430+0000","last_active":"2023-11-28T03:05:00.805408+0000","last_peered":"2023-11-28T03:05:00.805408+0000","last_clean":"2023-11-28T03:05:00.805408+0000","last_became_active":"2023-11-28T03:05:00.437155+0000","last_became_peered":"2023-11-28T03:05:00.437155+0000","last_unstale":"2023-11-28T03:05:00.805408+0000","last_undegraded":"2023-11-28T03:05:00.805408+0000","last_fullsized":"2023-11-28T03:05:00.805408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806687+0000","last_change":"2023-11-28T03:05:00.803087+0000","last_active":"2023-11-28T03:05:00.806687+0000","last_peered":"2023-11-28T03:05:00.806687+0000","last_clean":"2023-11-28T03:05:00.806687+0000","last_became_active":"2023-11-28T03:05:00.801329+0000","last_became_peered":"2023-11-28T03:05:00.801329+0000","last_unstale":"2023-11-28T03:05:00.806687+0000","last_undegraded":"2023-11-28T03:05:00.806687+0000","last_fullsized":"2023-11-28T03:05:00.806687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.790507+0000","last_change":"2023-11-28T03:05:00.437255+0000","last_active":"2023-11-28T03:05:00.790507+0000","last_peered":"2023-11-28T03:05:00.790507+0000","last_clean":"2023-11-28T03:05:00.790507+0000","last_became_active":"2023-11-28T03:05:00.436973+0000","last_became_peered":"2023-11-28T03:05:00.436973+0000","last_unstale":"2023-11-28T03:05:00.790507+0000","last_undegraded":"2023-11-28T03:05:00.790507+0000","last_fullsized":"2023-11-28T03:05:00.790507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799336+0000","last_change":"2023-11-28T03:05:00.797320+0000","last_active":"2023-11-28T03:05:00.799336+0000","last_peered":"2023-11-28T03:05:00.799336+0000","last_clean":"2023-11-28T03:05:00.799336+0000","last_became_active":"2023-11-28T03:05:00.796972+0000","last_became_peered":"2023-11-28T03:05:00.796972+0000","last_unstale":"2023-11-28T03:05:00.799336+0000","last_undegraded":"2023-11-28T03:05:00.799336+0000","last_fullsized":"2023-11-28T03:05:00.799336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795672+0000","last_change":"2023-11-28T03:05:00.423051+0000","last_active":"2023-11-28T03:05:00.795672+0000","last_peered":"2023-11-28T03:05:00.795672+0000","last_clean":"2023-11-28T03:05:00.795672+0000","last_became_active":"2023-11-28T03:05:00.422598+0000","last_became_peered":"2023-11-28T03:05:00.422598+0000","last_unstale":"2023-11-28T03:05:00.795672+0000","last_undegraded":"2023-11-28T03:05:00.795672+0000","last_fullsized":"2023-11-28T03:05:00.795672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809795+0000","last_change":"2023-11-28T03:05:00.806914+0000","last_active":"2023-11-28T03:05:00.809795+0000","last_peered":"2023-11-28T03:05:00.809795+0000","last_clean":"2023-11-28T03:05:00.809795+0000","last_became_active":"2023-11-28T03:05:00.806635+0000","last_became_peered":"2023-11-28T03:05:00.806635+0000","last_unstale":"2023-11-28T03:05:00.809795+0000","last_undegraded":"2023-11-28T03:05:00.809795+0000","last_fullsized":"2023-11-28T03:05:00.809795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787095+0000","last_change":"2023-11-28T03:05:00.423259+0000","last_active":"2023-11-28T03:05:00.787095+0000","last_peered":"2023-11-28T03:05:00.787095+0000","last_clean":"2023-11-28T03:05:00.787095+0000","last_became_active":"2023-11-28T03:05:00.422970+0000","last_became_peered":"2023-11-28T03:05:00.422970+0000","last_unstale":"2023-11-28T03:05:00.787095+0000","last_undegraded":"2023-11-28T03:05:00.787095+0000","last_fullsized":"2023-11-28T03:05:00.787095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807191+0000","last_change":"2023-11-28T03:05:00.802199+0000","last_active":"2023-11-28T03:05:00.807191+0000","last_peered":"2023-11-28T03:05:00.807191+0000","last_clean":"2023-11-28T03:05:00.807191+0000","last_became_active":"2023-11-28T03:05:00.801950+0000","last_became_peered":"2023-11-28T03:05:00.801950+0000","last_unstale":"2023-11-28T03:05:00.807191+0000","last_undegraded":"2023-11-28T03:05:00.807191+0000","last_fullsized":"2023-11-28T03:05:00.807191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781349+0000","last_change":"2023-11-28T03:05:00.418282+0000","last_active":"2023-11-28T03:05:00.781349+0000","last_peered":"2023-11-28T03:05:00.781349+0000","last_clean":"2023-11-28T03:05:00.781349+0000","last_became_active":"2023-11-28T03:05:00.417976+0000","last_became_peered":"2023-11-28T03:05:00.417976+0000","last_unstale":"2023-11-28T03:05:00.781349+0000","last_undegraded":"2023-11-28T03:05:00.781349+0000","last_fullsized":"2023-11-28T03:05:00.781349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806696+0000","last_change":"2023-11-28T03:05:00.802071+0000","last_active":"2023-11-28T03:05:00.806696+0000","last_peered":"2023-11-28T03:05:00.806696+0000","last_clean":"2023-11-28T03:05:00.806696+0000","last_became_active":"2023-11-28T03:05:00.801430+0000","last_became_peered":"2023-11-28T03:05:00.801430+0000","last_unstale":"2023-11-28T03:05:00.806696+0000","last_undegraded":"2023-11-28T03:05:00.806696+0000","last_fullsized":"2023-11-28T03:05:00.806696+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781670+0000","last_change":"2023-11-28T03:05:00.418862+0000","last_active":"2023-11-28T03:05:00.781670+0000","last_peered":"2023-11-28T03:05:00.781670+0000","last_clean":"2023-11-28T03:05:00.781670+0000","last_became_active":"2023-11-28T03:05:00.418589+0000","last_became_peered":"2023-11-28T03:05:00.418589+0000","last_unstale":"2023-11-28T03:05:00.781670+0000","last_undegraded":"2023-11-28T03:05:00.781670+0000","last_fullsized":"2023-11-28T03:05:00.781670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806755+0000","last_change":"2023-11-28T03:05:00.803646+0000","last_active":"2023-11-28T03:05:00.806755+0000","last_peered":"2023-11-28T03:05:00.806755+0000","last_clean":"2023-11-28T03:05:00.806755+0000","last_became_active":"2023-11-28T03:05:00.803400+0000","last_became_peered":"2023-11-28T03:05:00.803400+0000","last_unstale":"2023-11-28T03:05:00.806755+0000","last_undegraded":"2023-11-28T03:05:00.806755+0000","last_fullsized":"2023-11-28T03:05:00.806755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781821+0000","last_change":"2023-11-28T03:05:00.428730+0000","last_active":"2023-11-28T03:05:00.781821+0000","last_peered":"2023-11-28T03:05:00.781821+0000","last_clean":"2023-11-28T03:05:00.781821+0000","last_became_active":"2023-11-28T03:05:00.428404+0000","last_became_peered":"2023-11-28T03:05:00.428404+0000","last_unstale":"2023-11-28T03:05:00.781821+0000","last_undegraded":"2023-11-28T03:05:00.781821+0000","last_fullsized":"2023-11-28T03:05:00.781821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803533+0000","last_change":"2023-11-28T03:05:00.802752+0000","last_active":"2023-11-28T03:05:00.803533+0000","last_peered":"2023-11-28T03:05:00.803533+0000","last_clean":"2023-11-28T03:05:00.803533+0000","last_became_active":"2023-11-28T03:05:00.802481+0000","last_became_peered":"2023-11-28T03:05:00.802481+0000","last_unstale":"2023-11-28T03:05:00.803533+0000","last_undegraded":"2023-11-28T03:05:00.803533+0000","last_fullsized":"2023-11-28T03:05:00.803533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.792184+0000","last_change":"2023-11-28T03:05:00.425905+0000","last_active":"2023-11-28T03:05:00.792184+0000","last_peered":"2023-11-28T03:05:00.792184+0000","last_clean":"2023-11-28T03:05:00.792184+0000","last_became_active":"2023-11-28T03:05:00.424944+0000","last_became_peered":"2023-11-28T03:05:00.424944+0000","last_unstale":"2023-11-28T03:05:00.792184+0000","last_undegraded":"2023-11-28T03:05:00.792184+0000","last_fullsized":"2023-11-28T03:05:00.792184+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801767+0000","last_change":"2023-11-28T03:05:00.793768+0000","last_active":"2023-11-28T03:05:00.801767+0000","last_peered":"2023-11-28T03:05:00.801767+0000","last_clean":"2023-11-28T03:05:00.801767+0000","last_became_active":"2023-11-28T03:05:00.793493+0000","last_became_peered":"2023-11-28T03:05:00.793493+0000","last_unstale":"2023-11-28T03:05:00.801767+0000","last_undegraded":"2023-11-28T03:05:00.801767+0000","last_fullsized":"2023-11-28T03:05:00.801767+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783477+0000","last_change":"2023-11-28T03:05:00.418986+0000","last_active":"2023-11-28T03:05:00.783477+0000","last_peered":"2023-11-28T03:05:00.783477+0000","last_clean":"2023-11-28T03:05:00.783477+0000","last_became_active":"2023-11-28T03:05:00.418713+0000","last_became_peered":"2023-11-28T03:05:00.418713+0000","last_unstale":"2023-11-28T03:05:00.783477+0000","last_undegraded":"2023-11-28T03:05:00.783477+0000","last_fullsized":"2023-11-28T03:05:00.783477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799787+0000","last_change":"2023-11-28T03:05:00.798686+0000","last_active":"2023-11-28T03:05:00.799787+0000","last_peered":"2023-11-28T03:05:00.799787+0000","last_clean":"2023-11-28T03:05:00.799787+0000","last_became_active":"2023-11-28T03:05:00.798354+0000","last_became_peered":"2023-11-28T03:05:00.798354+0000","last_unstale":"2023-11-28T03:05:00.799787+0000","last_undegraded":"2023-11-28T03:05:00.799787+0000","last_fullsized":"2023-11-28T03:05:00.799787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785410+0000","last_change":"2023-11-28T03:05:00.423987+0000","last_active":"2023-11-28T03:05:00.785410+0000","last_peered":"2023-11-28T03:05:00.785410+0000","last_clean":"2023-11-28T03:05:00.785410+0000","last_became_active":"2023-11-28T03:05:00.423314+0000","last_became_peered":"2023-11-28T03:05:00.423314+0000","last_unstale":"2023-11-28T03:05:00.785410+0000","last_undegraded":"2023-11-28T03:05:00.785410+0000","last_fullsized":"2023-11-28T03:05:00.785410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795546+0000","last_change":"2023-11-28T03:05:00.792221+0000","last_active":"2023-11-28T03:05:00.795546+0000","last_peered":"2023-11-28T03:05:00.795546+0000","last_clean":"2023-11-28T03:05:00.795546+0000","last_became_active":"2023-11-28T03:05:00.791903+0000","last_became_peered":"2023-11-28T03:05:00.791903+0000","last_unstale":"2023-11-28T03:05:00.795546+0000","last_undegraded":"2023-11-28T03:05:00.795546+0000","last_fullsized":"2023-11-28T03:05:00.795546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782316+0000","last_change":"2023-11-28T03:05:00.428717+0000","last_active":"2023-11-28T03:05:00.782316+0000","last_peered":"2023-11-28T03:05:00.782316+0000","last_clean":"2023-11-28T03:05:00.782316+0000","last_became_active":"2023-11-28T03:05:00.428428+0000","last_became_peered":"2023-11-28T03:05:00.428428+0000","last_unstale":"2023-11-28T03:05:00.782316+0000","last_undegraded":"2023-11-28T03:05:00.782316+0000","last_fullsized":"2023-11-28T03:05:00.782316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854381+0000","last_change":"2023-11-28T03:05:00.853726+0000","last_active":"2023-11-28T03:05:00.854381+0000","last_peered":"2023-11-28T03:05:00.854381+0000","last_clean":"2023-11-28T03:05:00.854381+0000","last_became_active":"2023-11-28T03:05:00.853447+0000","last_became_peered":"2023-11-28T03:05:00.853447+0000","last_unstale":"2023-11-28T03:05:00.854381+0000","last_undegraded":"2023-11-28T03:05:00.854381+0000","last_fullsized":"2023-11-28T03:05:00.854381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783385+0000","last_change":"2023-11-28T03:05:00.420805+0000","last_active":"2023-11-28T03:05:00.783385+0000","last_peered":"2023-11-28T03:05:00.783385+0000","last_clean":"2023-11-28T03:05:00.783385+0000","last_became_active":"2023-11-28T03:05:00.420529+0000","last_became_peered":"2023-11-28T03:05:00.420529+0000","last_unstale":"2023-11-28T03:05:00.783385+0000","last_undegraded":"2023-11-28T03:05:00.783385+0000","last_fullsized":"2023-11-28T03:05:00.783385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802344+0000","last_change":"2023-11-28T03:05:00.800261+0000","last_active":"2023-11-28T03:05:00.802344+0000","last_peered":"2023-11-28T03:05:00.802344+0000","last_clean":"2023-11-28T03:05:00.802344+0000","last_became_active":"2023-11-28T03:05:00.799863+0000","last_became_peered":"2023-11-28T03:05:00.799863+0000","last_unstale":"2023-11-28T03:05:00.802344+0000","last_undegraded":"2023-11-28T03:05:00.802344+0000","last_fullsized":"2023-11-28T03:05:00.802344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.779918+0000","last_change":"2023-11-28T03:05:00.424347+0000","last_active":"2023-11-28T03:05:00.779918+0000","last_peered":"2023-11-28T03:05:00.779918+0000","last_clean":"2023-11-28T03:05:00.779918+0000","last_became_active":"2023-11-28T03:05:00.424002+0000","last_became_peered":"2023-11-28T03:05:00.424002+0000","last_unstale":"2023-11-28T03:05:00.779918+0000","last_undegraded":"2023-11-28T03:05:00.779918+0000","last_fullsized":"2023-11-28T03:05:00.779918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806717+0000","last_change":"2023-11-28T03:05:00.803173+0000","last_active":"2023-11-28T03:05:00.806717+0000","last_peered":"2023-11-28T03:05:00.806717+0000","last_clean":"2023-11-28T03:05:00.806717+0000","last_became_active":"2023-11-28T03:05:00.802493+0000","last_became_peered":"2023-11-28T03:05:00.802493+0000","last_unstale":"2023-11-28T03:05:00.806717+0000","last_undegraded":"2023-11-28T03:05:00.806717+0000","last_fullsized":"2023-11-28T03:05:00.806717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784497+0000","last_change":"2023-11-28T03:05:00.432485+0000","last_active":"2023-11-28T03:05:00.784497+0000","last_peered":"2023-11-28T03:05:00.784497+0000","last_clean":"2023-11-28T03:05:00.784497+0000","last_became_active":"2023-11-28T03:05:00.432202+0000","last_became_peered":"2023-11-28T03:05:00.432202+0000","last_unstale":"2023-11-28T03:05:00.784497+0000","last_undegraded":"2023-11-28T03:05:00.784497+0000","last_fullsized":"2023-11-28T03:05:00.784497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805558+0000","last_change":"2023-11-28T03:05:00.800890+0000","last_active":"2023-11-28T03:05:00.805558+0000","last_peered":"2023-11-28T03:05:00.805558+0000","last_clean":"2023-11-28T03:05:00.805558+0000","last_became_active":"2023-11-28T03:05:00.800607+0000","last_became_peered":"2023-11-28T03:05:00.800607+0000","last_unstale":"2023-11-28T03:05:00.805558+0000","last_undegraded":"2023-11-28T03:05:00.805558+0000","last_fullsized":"2023-11-28T03:05:00.805558+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795186+0000","last_change":"2023-11-28T03:05:00.420535+0000","last_active":"2023-11-28T03:05:00.795186+0000","last_peered":"2023-11-28T03:05:00.795186+0000","last_clean":"2023-11-28T03:05:00.795186+0000","last_became_active":"2023-11-28T03:05:00.419707+0000","last_became_peered":"2023-11-28T03:05:00.419707+0000","last_unstale":"2023-11-28T03:05:00.795186+0000","last_undegraded":"2023-11-28T03:05:00.795186+0000","last_fullsized":"2023-11-28T03:05:00.795186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802388+0000","last_change":"2023-11-28T03:05:00.801453+0000","last_active":"2023-11-28T03:05:00.802388+0000","last_peered":"2023-11-28T03:05:00.802388+0000","last_clean":"2023-11-28T03:05:00.802388+0000","last_became_active":"2023-11-28T03:05:00.801184+0000","last_became_peered":"2023-11-28T03:05:00.801184+0000","last_unstale":"2023-11-28T03:05:00.802388+0000","last_undegraded":"2023-11-28T03:05:00.802388+0000","last_fullsized":"2023-11-28T03:05:00.802388+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794713+0000","last_change":"2023-11-28T03:05:00.428126+0000","last_active":"2023-11-28T03:05:00.794713+0000","last_peered":"2023-11-28T03:05:00.794713+0000","last_clean":"2023-11-28T03:05:00.794713+0000","last_became_active":"2023-11-28T03:05:00.427535+0000","last_became_peered":"2023-11-28T03:05:00.427535+0000","last_unstale":"2023-11-28T03:05:00.794713+0000","last_undegraded":"2023-11-28T03:05:00.794713+0000","last_fullsized":"2023-11-28T03:05:00.794713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795601+0000","last_change":"2023-11-28T03:05:00.793229+0000","last_active":"2023-11-28T03:05:00.795601+0000","last_peered":"2023-11-28T03:05:00.795601+0000","last_clean":"2023-11-28T03:05:00.795601+0000","last_became_active":"2023-11-28T03:05:00.792929+0000","last_became_peered":"2023-11-28T03:05:00.792929+0000","last_unstale":"2023-11-28T03:05:00.795601+0000","last_undegraded":"2023-11-28T03:05:00.795601+0000","last_fullsized":"2023-11-28T03:05:00.795601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795082+0000","last_change":"2023-11-28T03:05:00.430218+0000","last_active":"2023-11-28T03:05:00.795082+0000","last_peered":"2023-11-28T03:05:00.795082+0000","last_clean":"2023-11-28T03:05:00.795082+0000","last_became_active":"2023-11-28T03:05:00.427406+0000","last_became_peered":"2023-11-28T03:05:00.427406+0000","last_unstale":"2023-11-28T03:05:00.795082+0000","last_undegraded":"2023-11-28T03:05:00.795082+0000","last_fullsized":"2023-11-28T03:05:00.795082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794703+0000","last_change":"2023-11-28T03:05:00.792518+0000","last_active":"2023-11-28T03:05:00.794703+0000","last_peered":"2023-11-28T03:05:00.794703+0000","last_clean":"2023-11-28T03:05:00.794703+0000","last_became_active":"2023-11-28T03:05:00.792351+0000","last_became_peered":"2023-11-28T03:05:00.792351+0000","last_unstale":"2023-11-28T03:05:00.794703+0000","last_undegraded":"2023-11-28T03:05:00.794703+0000","last_fullsized":"2023-11-28T03:05:00.794703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785523+0000","last_change":"2023-11-28T03:05:00.426818+0000","last_active":"2023-11-28T03:05:00.785523+0000","last_peered":"2023-11-28T03:05:00.785523+0000","last_clean":"2023-11-28T03:05:00.785523+0000","last_became_active":"2023-11-28T03:05:00.426491+0000","last_became_peered":"2023-11-28T03:05:00.426491+0000","last_unstale":"2023-11-28T03:05:00.785523+0000","last_undegraded":"2023-11-28T03:05:00.785523+0000","last_fullsized":"2023-11-28T03:05:00.785523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854409+0000","last_change":"2023-11-28T03:05:00.853635+0000","last_active":"2023-11-28T03:05:00.854409+0000","last_peered":"2023-11-28T03:05:00.854409+0000","last_clean":"2023-11-28T03:05:00.854409+0000","last_became_active":"2023-11-28T03:05:00.853345+0000","last_became_peered":"2023-11-28T03:05:00.853345+0000","last_unstale":"2023-11-28T03:05:00.854409+0000","last_undegraded":"2023-11-28T03:05:00.854409+0000","last_fullsized":"2023-11-28T03:05:00.854409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782759+0000","last_change":"2023-11-28T03:05:00.423415+0000","last_active":"2023-11-28T03:05:00.782759+0000","last_peered":"2023-11-28T03:05:00.782759+0000","last_clean":"2023-11-28T03:05:00.782759+0000","last_became_active":"2023-11-28T03:05:00.423212+0000","last_became_peered":"2023-11-28T03:05:00.423212+0000","last_unstale":"2023-11-28T03:05:00.782759+0000","last_undegraded":"2023-11-28T03:05:00.782759+0000","last_fullsized":"2023-11-28T03:05:00.782759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802074+0000","last_change":"2023-11-28T03:05:00.797871+0000","last_active":"2023-11-28T03:05:00.802074+0000","last_peered":"2023-11-28T03:05:00.802074+0000","last_clean":"2023-11-28T03:05:00.802074+0000","last_became_active":"2023-11-28T03:05:00.796189+0000","last_became_peered":"2023-11-28T03:05:00.796189+0000","last_unstale":"2023-11-28T03:05:00.802074+0000","last_undegraded":"2023-11-28T03:05:00.802074+0000","last_fullsized":"2023-11-28T03:05:00.802074+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783586+0000","last_change":"2023-11-28T03:05:00.411962+0000","last_active":"2023-11-28T03:05:00.783586+0000","last_peered":"2023-11-28T03:05:00.783586+0000","last_clean":"2023-11-28T03:05:00.783586+0000","last_became_active":"2023-11-28T03:05:00.411666+0000","last_became_peered":"2023-11-28T03:05:00.411666+0000","last_unstale":"2023-11-28T03:05:00.783586+0000","last_undegraded":"2023-11-28T03:05:00.783586+0000","last_fullsized":"2023-11-28T03:05:00.783586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807208+0000","last_change":"2023-11-28T03:05:00.802294+0000","last_active":"2023-11-28T03:05:00.807208+0000","last_peered":"2023-11-28T03:05:00.807208+0000","last_clean":"2023-11-28T03:05:00.807208+0000","last_became_active":"2023-11-28T03:05:00.802023+0000","last_became_peered":"2023-11-28T03:05:00.802023+0000","last_unstale":"2023-11-28T03:05:00.807208+0000","last_undegraded":"2023-11-28T03:05:00.807208+0000","last_fullsized":"2023-11-28T03:05:00.807208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781451+0000","last_change":"2023-11-28T03:05:00.424948+0000","last_active":"2023-11-28T03:05:00.781451+0000","last_peered":"2023-11-28T03:05:00.781451+0000","last_clean":"2023-11-28T03:05:00.781451+0000","last_became_active":"2023-11-28T03:05:00.424627+0000","last_became_peered":"2023-11-28T03:05:00.424627+0000","last_unstale":"2023-11-28T03:05:00.781451+0000","last_undegraded":"2023-11-28T03:05:00.781451+0000","last_fullsized":"2023-11-28T03:05:00.781451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809387+0000","last_change":"2023-11-28T03:05:00.803552+0000","last_active":"2023-11-28T03:05:00.809387+0000","last_peered":"2023-11-28T03:05:00.809387+0000","last_clean":"2023-11-28T03:05:00.809387+0000","last_became_active":"2023-11-28T03:05:00.803295+0000","last_became_peered":"2023-11-28T03:05:00.803295+0000","last_unstale":"2023-11-28T03:05:00.809387+0000","last_undegraded":"2023-11-28T03:05:00.809387+0000","last_fullsized":"2023-11-28T03:05:00.809387+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785601+0000","last_change":"2023-11-28T03:05:00.420449+0000","last_active":"2023-11-28T03:05:00.785601+0000","last_peered":"2023-11-28T03:05:00.785601+0000","last_clean":"2023-11-28T03:05:00.785601+0000","last_became_active":"2023-11-28T03:05:00.420083+0000","last_became_peered":"2023-11-28T03:05:00.420083+0000","last_unstale":"2023-11-28T03:05:00.785601+0000","last_undegraded":"2023-11-28T03:05:00.785601+0000","last_fullsized":"2023-11-28T03:05:00.785601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854379+0000","last_change":"2023-11-28T03:05:00.853332+0000","last_active":"2023-11-28T03:05:00.854379+0000","last_peered":"2023-11-28T03:05:00.854379+0000","last_clean":"2023-11-28T03:05:00.854379+0000","last_became_active":"2023-11-28T03:05:00.852934+0000","last_became_peered":"2023-11-28T03:05:00.852934+0000","last_unstale":"2023-11-28T03:05:00.854379+0000","last_undegraded":"2023-11-28T03:05:00.854379+0000","last_fullsized":"2023-11-28T03:05:00.854379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787447+0000","last_change":"2023-11-28T03:05:00.425379+0000","last_active":"2023-11-28T03:05:00.787447+0000","last_peered":"2023-11-28T03:05:00.787447+0000","last_clean":"2023-11-28T03:05:00.787447+0000","last_became_active":"2023-11-28T03:05:00.424874+0000","last_became_peered":"2023-11-28T03:05:00.424874+0000","last_unstale":"2023-11-28T03:05:00.787447+0000","last_undegraded":"2023-11-28T03:05:00.787447+0000","last_fullsized":"2023-11-28T03:05:00.787447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797031+0000","last_change":"2023-11-28T03:05:00.793571+0000","last_active":"2023-11-28T03:05:00.797031+0000","last_peered":"2023-11-28T03:05:00.797031+0000","last_clean":"2023-11-28T03:05:00.797031+0000","last_became_active":"2023-11-28T03:05:00.793295+0000","last_became_peered":"2023-11-28T03:05:00.793295+0000","last_unstale":"2023-11-28T03:05:00.797031+0000","last_undegraded":"2023-11-28T03:05:00.797031+0000","last_fullsized":"2023-11-28T03:05:00.797031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784157+0000","last_change":"2023-11-28T03:05:00.436731+0000","last_active":"2023-11-28T03:05:00.784157+0000","last_peered":"2023-11-28T03:05:00.784157+0000","last_clean":"2023-11-28T03:05:00.784157+0000","last_became_active":"2023-11-28T03:05:00.436450+0000","last_became_peered":"2023-11-28T03:05:00.436450+0000","last_unstale":"2023-11-28T03:05:00.784157+0000","last_undegraded":"2023-11-28T03:05:00.784157+0000","last_fullsized":"2023-11-28T03:05:00.784157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806630+0000","last_change":"2023-11-28T03:05:00.800644+0000","last_active":"2023-11-28T03:05:00.806630+0000","last_peered":"2023-11-28T03:05:00.806630+0000","last_clean":"2023-11-28T03:05:00.806630+0000","last_became_active":"2023-11-28T03:05:00.800385+0000","last_became_peered":"2023-11-28T03:05:00.800385+0000","last_unstale":"2023-11-28T03:05:00.806630+0000","last_undegraded":"2023-11-28T03:05:00.806630+0000","last_fullsized":"2023-11-28T03:05:00.806630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800698+0000","last_change":"2023-11-28T03:05:00.431729+0000","last_active":"2023-11-28T03:05:00.800698+0000","last_peered":"2023-11-28T03:05:00.800698+0000","last_clean":"2023-11-28T03:05:00.800698+0000","last_became_active":"2023-11-28T03:05:00.431423+0000","last_became_peered":"2023-11-28T03:05:00.431423+0000","last_unstale":"2023-11-28T03:05:00.800698+0000","last_undegraded":"2023-11-28T03:05:00.800698+0000","last_fullsized":"2023-11-28T03:05:00.800698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794255+0000","last_change":"2023-11-28T03:05:00.792257+0000","last_active":"2023-11-28T03:05:00.794255+0000","last_peered":"2023-11-28T03:05:00.794255+0000","last_clean":"2023-11-28T03:05:00.794255+0000","last_became_active":"2023-11-28T03:05:00.791902+0000","last_became_peered":"2023-11-28T03:05:00.791902+0000","last_unstale":"2023-11-28T03:05:00.794255+0000","last_undegraded":"2023-11-28T03:05:00.794255+0000","last_fullsized":"2023-11-28T03:05:00.794255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803922+0000","last_change":"2023-11-28T03:05:00.424419+0000","last_active":"2023-11-28T03:05:00.803922+0000","last_peered":"2023-11-28T03:05:00.803922+0000","last_clean":"2023-11-28T03:05:00.803922+0000","last_became_active":"2023-11-28T03:05:00.424147+0000","last_became_peered":"2023-11-28T03:05:00.424147+0000","last_unstale":"2023-11-28T03:05:00.803922+0000","last_undegraded":"2023-11-28T03:05:00.803922+0000","last_fullsized":"2023-11-28T03:05:00.803922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801857+0000","last_change":"2023-11-28T03:05:00.799216+0000","last_active":"2023-11-28T03:05:00.801857+0000","last_peered":"2023-11-28T03:05:00.801857+0000","last_clean":"2023-11-28T03:05:00.801857+0000","last_became_active":"2023-11-28T03:05:00.798916+0000","last_became_peered":"2023-11-28T03:05:00.798916+0000","last_unstale":"2023-11-28T03:05:00.801857+0000","last_undegraded":"2023-11-28T03:05:00.801857+0000","last_fullsized":"2023-11-28T03:05:00.801857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801481+0000","last_change":"2023-11-28T03:05:00.426430+0000","last_active":"2023-11-28T03:05:00.801481+0000","last_peered":"2023-11-28T03:05:00.801481+0000","last_clean":"2023-11-28T03:05:00.801481+0000","last_became_active":"2023-11-28T03:05:00.425465+0000","last_became_peered":"2023-11-28T03:05:00.425465+0000","last_unstale":"2023-11-28T03:05:00.801481+0000","last_undegraded":"2023-11-28T03:05:00.801481+0000","last_fullsized":"2023-11-28T03:05:00.801481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806594+0000","last_change":"2023-11-28T03:05:00.800001+0000","last_active":"2023-11-28T03:05:00.806594+0000","last_peered":"2023-11-28T03:05:00.806594+0000","last_clean":"2023-11-28T03:05:00.806594+0000","last_became_active":"2023-11-28T03:05:00.799703+0000","last_became_peered":"2023-11-28T03:05:00.799703+0000","last_unstale":"2023-11-28T03:05:00.806594+0000","last_undegraded":"2023-11-28T03:05:00.806594+0000","last_fullsized":"2023-11-28T03:05:00.806594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781671+0000","last_change":"2023-11-28T03:05:00.436846+0000","last_active":"2023-11-28T03:05:00.781671+0000","last_peered":"2023-11-28T03:05:00.781671+0000","last_clean":"2023-11-28T03:05:00.781671+0000","last_became_active":"2023-11-28T03:05:00.436418+0000","last_became_peered":"2023-11-28T03:05:00.436418+0000","last_unstale":"2023-11-28T03:05:00.781671+0000","last_undegraded":"2023-11-28T03:05:00.781671+0000","last_fullsized":"2023-11-28T03:05:00.781671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795961+0000","last_change":"2023-11-28T03:05:00.793678+0000","last_active":"2023-11-28T03:05:00.795961+0000","last_peered":"2023-11-28T03:05:00.795961+0000","last_clean":"2023-11-28T03:05:00.795961+0000","last_became_active":"2023-11-28T03:05:00.793025+0000","last_became_peered":"2023-11-28T03:05:00.793025+0000","last_unstale":"2023-11-28T03:05:00.795961+0000","last_undegraded":"2023-11-28T03:05:00.795961+0000","last_fullsized":"2023-11-28T03:05:00.795961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787533+0000","last_change":"2023-11-28T03:05:00.428834+0000","last_active":"2023-11-28T03:05:00.787533+0000","last_peered":"2023-11-28T03:05:00.787533+0000","last_clean":"2023-11-28T03:05:00.787533+0000","last_became_active":"2023-11-28T03:05:00.428487+0000","last_became_peered":"2023-11-28T03:05:00.428487+0000","last_unstale":"2023-11-28T03:05:00.787533+0000","last_undegraded":"2023-11-28T03:05:00.787533+0000","last_fullsized":"2023-11-28T03:05:00.787533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806674+0000","last_change":"2023-11-28T03:05:00.800007+0000","last_active":"2023-11-28T03:05:00.806674+0000","last_peered":"2023-11-28T03:05:00.806674+0000","last_clean":"2023-11-28T03:05:00.806674+0000","last_became_active":"2023-11-28T03:05:00.799737+0000","last_became_peered":"2023-11-28T03:05:00.799737+0000","last_unstale":"2023-11-28T03:05:00.806674+0000","last_undegraded":"2023-11-28T03:05:00.806674+0000","last_fullsized":"2023-11-28T03:05:00.806674+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806127+0000","last_change":"2023-11-28T03:05:00.424398+0000","last_active":"2023-11-28T03:05:00.806127+0000","last_peered":"2023-11-28T03:05:00.806127+0000","last_clean":"2023-11-28T03:05:00.806127+0000","last_became_active":"2023-11-28T03:05:00.424133+0000","last_became_peered":"2023-11-28T03:05:00.424133+0000","last_unstale":"2023-11-28T03:05:00.806127+0000","last_undegraded":"2023-11-28T03:05:00.806127+0000","last_fullsized":"2023-11-28T03:05:00.806127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807703+0000","last_change":"2023-11-28T03:05:00.804770+0000","last_active":"2023-11-28T03:05:00.807703+0000","last_peered":"2023-11-28T03:05:00.807703+0000","last_clean":"2023-11-28T03:05:00.807703+0000","last_became_active":"2023-11-28T03:05:00.804141+0000","last_became_peered":"2023-11-28T03:05:00.804141+0000","last_unstale":"2023-11-28T03:05:00.807703+0000","last_undegraded":"2023-11-28T03:05:00.807703+0000","last_fullsized":"2023-11-28T03:05:00.807703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802987+0000","last_change":"2023-11-28T03:05:00.426975+0000","last_active":"2023-11-28T03:05:00.802987+0000","last_peered":"2023-11-28T03:05:00.802987+0000","last_clean":"2023-11-28T03:05:00.802987+0000","last_became_active":"2023-11-28T03:05:00.426697+0000","last_became_peered":"2023-11-28T03:05:00.426697+0000","last_unstale":"2023-11-28T03:05:00.802987+0000","last_undegraded":"2023-11-28T03:05:00.802987+0000","last_fullsized":"2023-11-28T03:05:00.802987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.810119+0000","last_change":"2023-11-28T03:05:00.807548+0000","last_active":"2023-11-28T03:05:00.810119+0000","last_peered":"2023-11-28T03:05:00.810119+0000","last_clean":"2023-11-28T03:05:00.810119+0000","last_became_active":"2023-11-28T03:05:00.807224+0000","last_became_peered":"2023-11-28T03:05:00.807224+0000","last_unstale":"2023-11-28T03:05:00.810119+0000","last_undegraded":"2023-11-28T03:05:00.810119+0000","last_fullsized":"2023-11-28T03:05:00.810119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802720+0000","last_change":"2023-11-28T03:05:00.419344+0000","last_active":"2023-11-28T03:05:00.802720+0000","last_peered":"2023-11-28T03:05:00.802720+0000","last_clean":"2023-11-28T03:05:00.802720+0000","last_became_active":"2023-11-28T03:05:00.419064+0000","last_became_peered":"2023-11-28T03:05:00.419064+0000","last_unstale":"2023-11-28T03:05:00.802720+0000","last_undegraded":"2023-11-28T03:05:00.802720+0000","last_fullsized":"2023-11-28T03:05:00.802720+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.853572+0000","last_change":"2023-11-28T03:05:00.794660+0000","last_active":"2023-11-28T03:05:00.853572+0000","last_peered":"2023-11-28T03:05:00.853572+0000","last_clean":"2023-11-28T03:05:00.853572+0000","last_became_active":"2023-11-28T03:05:00.794429+0000","last_became_peered":"2023-11-28T03:05:00.794429+0000","last_unstale":"2023-11-28T03:05:00.853572+0000","last_undegraded":"2023-11-28T03:05:00.853572+0000","last_fullsized":"2023-11-28T03:05:00.853572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782115+0000","last_change":"2023-11-28T03:05:00.431077+0000","last_active":"2023-11-28T03:05:00.782115+0000","last_peered":"2023-11-28T03:05:00.782115+0000","last_clean":"2023-11-28T03:05:00.782115+0000","last_became_active":"2023-11-28T03:05:00.430764+0000","last_became_peered":"2023-11-28T03:05:00.430764+0000","last_unstale":"2023-11-28T03:05:00.782115+0000","last_undegraded":"2023-11-28T03:05:00.782115+0000","last_fullsized":"2023-11-28T03:05:00.782115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799252+0000","last_change":"2023-11-28T03:05:00.797610+0000","last_active":"2023-11-28T03:05:00.799252+0000","last_peered":"2023-11-28T03:05:00.799252+0000","last_clean":"2023-11-28T03:05:00.799252+0000","last_became_active":"2023-11-28T03:05:00.797306+0000","last_became_peered":"2023-11-28T03:05:00.797306+0000","last_unstale":"2023-11-28T03:05:00.799252+0000","last_undegraded":"2023-11-28T03:05:00.799252+0000","last_fullsized":"2023-11-28T03:05:00.799252+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782946+0000","last_change":"2023-11-28T03:05:00.426981+0000","last_active":"2023-11-28T03:05:00.782946+0000","last_peered":"2023-11-28T03:05:00.782946+0000","last_clean":"2023-11-28T03:05:00.782946+0000","last_became_active":"2023-11-28T03:05:00.426634+0000","last_became_peered":"2023-11-28T03:05:00.426634+0000","last_unstale":"2023-11-28T03:05:00.782946+0000","last_undegraded":"2023-11-28T03:05:00.782946+0000","last_fullsized":"2023-11-28T03:05:00.782946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799386+0000","last_change":"2023-11-28T03:05:00.795865+0000","last_active":"2023-11-28T03:05:00.799386+0000","last_peered":"2023-11-28T03:05:00.799386+0000","last_clean":"2023-11-28T03:05:00.799386+0000","last_became_active":"2023-11-28T03:05:00.795590+0000","last_became_peered":"2023-11-28T03:05:00.795590+0000","last_unstale":"2023-11-28T03:05:00.799386+0000","last_undegraded":"2023-11-28T03:05:00.799386+0000","last_fullsized":"2023-11-28T03:05:00.799386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797332+0000","last_change":"2023-11-28T03:05:00.427931+0000","last_active":"2023-11-28T03:05:00.797332+0000","last_peered":"2023-11-28T03:05:00.797332+0000","last_clean":"2023-11-28T03:05:00.797332+0000","last_became_active":"2023-11-28T03:05:00.427505+0000","last_became_peered":"2023-11-28T03:05:00.427505+0000","last_unstale":"2023-11-28T03:05:00.797332+0000","last_undegraded":"2023-11-28T03:05:00.797332+0000","last_fullsized":"2023-11-28T03:05:00.797332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795645+0000","last_change":"2023-11-28T03:05:00.792894+0000","last_active":"2023-11-28T03:05:00.795645+0000","last_peered":"2023-11-28T03:05:00.795645+0000","last_clean":"2023-11-28T03:05:00.795645+0000","last_became_active":"2023-11-28T03:05:00.792612+0000","last_became_peered":"2023-11-28T03:05:00.792612+0000","last_unstale":"2023-11-28T03:05:00.795645+0000","last_undegraded":"2023-11-28T03:05:00.795645+0000","last_fullsized":"2023-11-28T03:05:00.795645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780367+0000","last_change":"2023-11-28T03:05:00.425301+0000","last_active":"2023-11-28T03:05:00.780367+0000","last_peered":"2023-11-28T03:05:00.780367+0000","last_clean":"2023-11-28T03:05:00.780367+0000","last_became_active":"2023-11-28T03:05:00.424733+0000","last_became_peered":"2023-11-28T03:05:00.424733+0000","last_unstale":"2023-11-28T03:05:00.780367+0000","last_undegraded":"2023-11-28T03:05:00.780367+0000","last_fullsized":"2023-11-28T03:05:00.780367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806624+0000","last_change":"2023-11-28T03:05:00.801628+0000","last_active":"2023-11-28T03:05:00.806624+0000","last_peered":"2023-11-28T03:05:00.806624+0000","last_clean":"2023-11-28T03:05:00.806624+0000","last_became_active":"2023-11-28T03:05:00.801373+0000","last_became_peered":"2023-11-28T03:05:00.801373+0000","last_unstale":"2023-11-28T03:05:00.806624+0000","last_undegraded":"2023-11-28T03:05:00.806624+0000","last_fullsized":"2023-11-28T03:05:00.806624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783391+0000","last_change":"2023-11-28T03:05:00.430297+0000","last_active":"2023-11-28T03:05:00.783391+0000","last_peered":"2023-11-28T03:05:00.783391+0000","last_clean":"2023-11-28T03:05:00.783391+0000","last_became_active":"2023-11-28T03:05:00.429995+0000","last_became_peered":"2023-11-28T03:05:00.429995+0000","last_unstale":"2023-11-28T03:05:00.783391+0000","last_undegraded":"2023-11-28T03:05:00.783391+0000","last_fullsized":"2023-11-28T03:05:00.783391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797007+0000","last_change":"2023-11-28T03:05:00.795438+0000","last_active":"2023-11-28T03:05:00.797007+0000","last_peered":"2023-11-28T03:05:00.797007+0000","last_clean":"2023-11-28T03:05:00.797007+0000","last_became_active":"2023-11-28T03:05:00.795177+0000","last_became_peered":"2023-11-28T03:05:00.795177+0000","last_unstale":"2023-11-28T03:05:00.797007+0000","last_undegraded":"2023-11-28T03:05:00.797007+0000","last_fullsized":"2023-11-28T03:05:00.797007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784584+0000","last_change":"2023-11-28T03:05:00.431854+0000","last_active":"2023-11-28T03:05:00.784584+0000","last_peered":"2023-11-28T03:05:00.784584+0000","last_clean":"2023-11-28T03:05:00.784584+0000","last_became_active":"2023-11-28T03:05:00.431452+0000","last_became_peered":"2023-11-28T03:05:00.431452+0000","last_unstale":"2023-11-28T03:05:00.784584+0000","last_undegraded":"2023-11-28T03:05:00.784584+0000","last_fullsized":"2023-11-28T03:05:00.784584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806507+0000","last_change":"2023-11-28T03:05:00.800134+0000","last_active":"2023-11-28T03:05:00.806507+0000","last_peered":"2023-11-28T03:05:00.806507+0000","last_clean":"2023-11-28T03:05:00.806507+0000","last_became_active":"2023-11-28T03:05:00.799831+0000","last_became_peered":"2023-11-28T03:05:00.799831+0000","last_unstale":"2023-11-28T03:05:00.806507+0000","last_undegraded":"2023-11-28T03:05:00.806507+0000","last_fullsized":"2023-11-28T03:05:00.806507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783530+0000","last_change":"2023-11-28T03:05:00.422074+0000","last_active":"2023-11-28T03:05:00.783530+0000","last_peered":"2023-11-28T03:05:00.783530+0000","last_clean":"2023-11-28T03:05:00.783530+0000","last_became_active":"2023-11-28T03:05:00.421818+0000","last_became_peered":"2023-11-28T03:05:00.421818+0000","last_unstale":"2023-11-28T03:05:00.783530+0000","last_undegraded":"2023-11-28T03:05:00.783530+0000","last_fullsized":"2023-11-28T03:05:00.783530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786429+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:05:00.786429+0000","last_peered":"2023-11-28T03:05:00.786429+0000","last_clean":"2023-11-28T03:05:00.786429+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:05:00.786429+0000","last_undegraded":"2023-11-28T03:05:00.786429+0000","last_fullsized":"2023-11-28T03:05:00.786429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799800+0000","last_change":"2023-11-28T03:05:00.797902+0000","last_active":"2023-11-28T03:05:00.799800+0000","last_peered":"2023-11-28T03:05:00.799800+0000","last_clean":"2023-11-28T03:05:00.799800+0000","last_became_active":"2023-11-28T03:05:00.797319+0000","last_became_peered":"2023-11-28T03:05:00.797319+0000","last_unstale":"2023-11-28T03:05:00.799800+0000","last_undegraded":"2023-11-28T03:05:00.799800+0000","last_fullsized":"2023-11-28T03:05:00.799800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803283+0000","last_change":"2023-11-28T03:05:00.423249+0000","last_active":"2023-11-28T03:05:00.803283+0000","last_peered":"2023-11-28T03:05:00.803283+0000","last_clean":"2023-11-28T03:05:00.803283+0000","last_became_active":"2023-11-28T03:05:00.422926+0000","last_became_peered":"2023-11-28T03:05:00.422926+0000","last_unstale":"2023-11-28T03:05:00.803283+0000","last_undegraded":"2023-11-28T03:05:00.803283+0000","last_fullsized":"2023-11-28T03:05:00.803283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784573+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:05:00.784573+0000","last_peered":"2023-11-28T03:05:00.784573+0000","last_clean":"2023-11-28T03:05:00.784573+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:05:00.784573+0000","last_undegraded":"2023-11-28T03:05:00.784573+0000","last_fullsized":"2023-11-28T03:05:00.784573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794685+0000","last_change":"2023-11-28T03:05:00.792511+0000","last_active":"2023-11-28T03:05:00.794685+0000","last_peered":"2023-11-28T03:05:00.794685+0000","last_clean":"2023-11-28T03:05:00.794685+0000","last_became_active":"2023-11-28T03:05:00.792289+0000","last_became_peered":"2023-11-28T03:05:00.792289+0000","last_unstale":"2023-11-28T03:05:00.794685+0000","last_undegraded":"2023-11-28T03:05:00.794685+0000","last_fullsized":"2023-11-28T03:05:00.794685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783059+0000","last_change":"2023-11-28T03:05:00.436271+0000","last_active":"2023-11-28T03:05:00.783059+0000","last_peered":"2023-11-28T03:05:00.783059+0000","last_clean":"2023-11-28T03:05:00.783059+0000","last_became_active":"2023-11-28T03:05:00.435974+0000","last_became_peered":"2023-11-28T03:05:00.435974+0000","last_unstale":"2023-11-28T03:05:00.783059+0000","last_undegraded":"2023-11-28T03:05:00.783059+0000","last_fullsized":"2023-11-28T03:05:00.783059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789838+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:05:00.789838+0000","last_peered":"2023-11-28T03:05:00.789838+0000","last_clean":"2023-11-28T03:05:00.789838+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:05:00.789838+0000","last_undegraded":"2023-11-28T03:05:00.789838+0000","last_fullsized":"2023-11-28T03:05:00.789838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799223+0000","last_change":"2023-11-28T03:05:00.797875+0000","last_active":"2023-11-28T03:05:00.799223+0000","last_peered":"2023-11-28T03:05:00.799223+0000","last_clean":"2023-11-28T03:05:00.799223+0000","last_became_active":"2023-11-28T03:05:00.797594+0000","last_became_peered":"2023-11-28T03:05:00.797594+0000","last_unstale":"2023-11-28T03:05:00.799223+0000","last_undegraded":"2023-11-28T03:05:00.799223+0000","last_fullsized":"2023-11-28T03:05:00.799223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782843+0000","last_change":"2023-11-28T03:05:00.437608+0000","last_active":"2023-11-28T03:05:00.782843+0000","last_peered":"2023-11-28T03:05:00.782843+0000","last_clean":"2023-11-28T03:05:00.782843+0000","last_became_active":"2023-11-28T03:05:00.437267+0000","last_became_peered":"2023-11-28T03:05:00.437267+0000","last_unstale":"2023-11-28T03:05:00.782843+0000","last_undegraded":"2023-11-28T03:05:00.782843+0000","last_fullsized":"2023-11-28T03:05:00.782843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785039+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:05:00.785039+0000","last_peered":"2023-11-28T03:05:00.785039+0000","last_clean":"2023-11-28T03:05:00.785039+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:05:00.785039+0000","last_undegraded":"2023-11-28T03:05:00.785039+0000","last_fullsized":"2023-11-28T03:05:00.785039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786291+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:05:00.786291+0000","last_peered":"2023-11-28T03:05:00.786291+0000","last_clean":"2023-11-28T03:05:00.786291+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:05:00.786291+0000","last_undegraded":"2023-11-28T03:05:00.786291+0000","last_fullsized":"2023-11-28T03:05:00.786291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796734+0000","last_change":"2023-11-28T03:05:00.794511+0000","last_active":"2023-11-28T03:05:00.796734+0000","last_peered":"2023-11-28T03:05:00.796734+0000","last_clean":"2023-11-28T03:05:00.796734+0000","last_became_active":"2023-11-28T03:05:00.794228+0000","last_became_peered":"2023-11-28T03:05:00.794228+0000","last_unstale":"2023-11-28T03:05:00.796734+0000","last_undegraded":"2023-11-28T03:05:00.796734+0000","last_fullsized":"2023-11-28T03:05:00.796734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789844+0000","last_change":"2023-11-28T03:05:00.425225+0000","last_active":"2023-11-28T03:05:00.789844+0000","last_peered":"2023-11-28T03:05:00.789844+0000","last_clean":"2023-11-28T03:05:00.789844+0000","last_became_active":"2023-11-28T03:05:00.424968+0000","last_became_peered":"2023-11-28T03:05:00.424968+0000","last_unstale":"2023-11-28T03:05:00.789844+0000","last_undegraded":"2023-11-28T03:05:00.789844+0000","last_fullsized":"2023-11-28T03:05:00.789844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786377+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:05:00.786377+0000","last_peered":"2023-11-28T03:05:00.786377+0000","last_clean":"2023-11-28T03:05:00.786377+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:05:00.786377+0000","last_undegraded":"2023-11-28T03:05:00.786377+0000","last_fullsized":"2023-11-28T03:05:00.786377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801882+0000","last_change":"2023-11-28T03:05:00.793949+0000","last_active":"2023-11-28T03:05:00.801882+0000","last_peered":"2023-11-28T03:05:00.801882+0000","last_clean":"2023-11-28T03:05:00.801882+0000","last_became_active":"2023-11-28T03:05:00.793037+0000","last_became_peered":"2023-11-28T03:05:00.793037+0000","last_unstale":"2023-11-28T03:05:00.801882+0000","last_undegraded":"2023-11-28T03:05:00.801882+0000","last_fullsized":"2023-11-28T03:05:00.801882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787693+0000","last_change":"2023-11-28T03:05:00.431872+0000","last_active":"2023-11-28T03:05:00.787693+0000","last_peered":"2023-11-28T03:05:00.787693+0000","last_clean":"2023-11-28T03:05:00.787693+0000","last_became_active":"2023-11-28T03:05:00.431547+0000","last_became_peered":"2023-11-28T03:05:00.431547+0000","last_unstale":"2023-11-28T03:05:00.787693+0000","last_undegraded":"2023-11-28T03:05:00.787693+0000","last_fullsized":"2023-11-28T03:05:00.787693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789916+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:05:00.789916+0000","last_peered":"2023-11-28T03:05:00.789916+0000","last_clean":"2023-11-28T03:05:00.789916+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:05:00.789916+0000","last_undegraded":"2023-11-28T03:05:00.789916+0000","last_fullsized":"2023-11-28T03:05:00.789916+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807908+0000","last_change":"2023-11-28T03:05:00.805547+0000","last_active":"2023-11-28T03:05:00.807908+0000","last_peered":"2023-11-28T03:05:00.807908+0000","last_clean":"2023-11-28T03:05:00.807908+0000","last_became_active":"2023-11-28T03:05:00.805325+0000","last_became_peered":"2023-11-28T03:05:00.805325+0000","last_unstale":"2023-11-28T03:05:00.807908+0000","last_undegraded":"2023-11-28T03:05:00.807908+0000","last_fullsized":"2023-11-28T03:05:00.807908+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781035+0000","last_change":"2023-11-28T03:05:00.426376+0000","last_active":"2023-11-28T03:05:00.781035+0000","last_peered":"2023-11-28T03:05:00.781035+0000","last_clean":"2023-11-28T03:05:00.781035+0000","last_became_active":"2023-11-28T03:05:00.426131+0000","last_became_peered":"2023-11-28T03:05:00.426131+0000","last_unstale":"2023-11-28T03:05:00.781035+0000","last_undegraded":"2023-11-28T03:05:00.781035+0000","last_fullsized":"2023-11-28T03:05:00.781035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.788011+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:05:00.788011+0000","last_peered":"2023-11-28T03:05:00.788011+0000","last_clean":"2023-11-28T03:05:00.788011+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:05:00.788011+0000","last_undegraded":"2023-11-28T03:05:00.788011+0000","last_fullsized":"2023-11-28T03:05:00.788011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802290+0000","last_change":"2023-11-28T03:05:00.800651+0000","last_active":"2023-11-28T03:05:00.802290+0000","last_peered":"2023-11-28T03:05:00.802290+0000","last_clean":"2023-11-28T03:05:00.802290+0000","last_became_active":"2023-11-28T03:05:00.800387+0000","last_became_peered":"2023-11-28T03:05:00.800387+0000","last_unstale":"2023-11-28T03:05:00.802290+0000","last_undegraded":"2023-11-28T03:05:00.802290+0000","last_fullsized":"2023-11-28T03:05:00.802290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780548+0000","last_change":"2023-11-28T03:05:00.431674+0000","last_active":"2023-11-28T03:05:00.780548+0000","last_peered":"2023-11-28T03:05:00.780548+0000","last_clean":"2023-11-28T03:05:00.780548+0000","last_became_active":"2023-11-28T03:05:00.431396+0000","last_became_peered":"2023-11-28T03:05:00.431396+0000","last_unstale":"2023-11-28T03:05:00.780548+0000","last_undegraded":"2023-11-28T03:05:00.780548+0000","last_fullsized":"2023-11-28T03:05:00.780548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782094+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:05:00.782094+0000","last_peered":"2023-11-28T03:05:00.782094+0000","last_clean":"2023-11-28T03:05:00.782094+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:05:00.782094+0000","last_undegraded":"2023-11-28T03:05:00.782094+0000","last_fullsized":"2023-11-28T03:05:00.782094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.85999999999999999},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60899999999999999,"15min":0.59599999999999997},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.56799999999999995}]},{"osd":1,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.435},{"interface":"front","average":{"1min":0.501,"5min":0.56299999999999994,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.749,"15min":0.749},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.39500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.749,"15min":0.77100000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75800000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.54200000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68799999999999994,"5min":0.77800000000000002,"15min":0.83099999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.81899999999999995},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.84499999999999997},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83199999999999996},"last":0.59999999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.79800000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.28999999999999998},"max":{"1min":0.749,"5min":0.77800000000000002,"15min":0.84499999999999997},"last":0.58599999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574999,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.63600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.379,"15min":0.32800000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.746},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.66400000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.81000000000000005,"15min":0.84399999999999997},"last":0.81299999999999994}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.746,"15min":0.82499999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.72899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.317},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.71399999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574998,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.38600000000000001,"5min":0.56599999999999995,"15min":0.70699999999999996},"last":0.247},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.52700000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.26500000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.43099999999999999,"5min":0.56200000000000006,"15min":0.73499999999999999},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.27600000000000002,"15min":0.19800000000000001},"max":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.72599999999999998},"last":0.36099999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.52800000000000002,"5min":0.66300000000000003,"15min":0.749},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.376,"5min":0.44,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.54400000000000004,"5min":0.60899999999999999,"15min":1.6870000000000001},"last":0.38700000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":1.0900000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.79900000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.038,"15min":1.111},"last":0.72899999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":1.8080000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.073},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.81499999999999995,"5min":1.024,"15min":1.024},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.38700000000000001},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574999,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297864,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":786432,"data_stored":379918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.184},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.313},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.24099999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.436},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.32100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.38900000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.24099999999999999},"max":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.41299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.621,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.35899999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63900000000000001,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.54200000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88500000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65100000000000002,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.39000000000000001},"max":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42599999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.36299999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97399999999999998,"5min":1.046,"15min":1.046},"last":0.55100000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":381328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.374,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.20499999999999999},"max":{"1min":0.41099999999999998,"5min":0.57399999999999995,"15min":0.623},"last":0.247},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.42199999999999999,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23599999999999999},"max":{"1min":0.47599999999999998,"5min":0.64700000000000002,"15min":0.65900000000000003},"last":0.36499999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.377},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51100000000000001,"5min":0.68600000000000005,"15min":0.73099999999999998},"last":0.29999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.40600000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53600000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.314},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.35099999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.94599999999999995},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.497}]},{"osd":5,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":1.004,"15min":1.004},"last":0.54600000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74399999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36899999999999999},"max":{"1min":0.73699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.48699999999999999}]}]},{"osd":3,"up_from":13,"seq":55834574999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36799999999999999,"15min":0.34699999999999998},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.501,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.30399999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38900000000000001,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.19900000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.60899999999999999},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32000000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.44800000000000001,"15min":0.42599999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.68200000000000005},"last":0.375}]},{"osd":4,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6000000000000001},"last":0.501},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.621,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70699999999999996,"5min":0.94199999999999995,"15min":1.196},"last":0.58899999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.996,"15min":1.206},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.55500000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.97199999999999998,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.56499999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574998,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56899999999999995,"15min":0.53200000000000003},"min":{"1min":0.40000000000000002,"5min":0.371,"15min":0.27800000000000002},"max":{"1min":0.68300000000000005,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.57999999999999996,"15min":0.53600000000000003},"min":{"1min":0.379,"5min":0.372,"15min":0.27300000000000002},"max":{"1min":0.73099999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.58999999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56699999999999995,"15min":0.53800000000000003},"min":{"1min":0.41999999999999998,"5min":0.32800000000000001,"15min":0.314},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.57799999999999996,"15min":0.54800000000000004},"min":{"1min":0.46000000000000002,"5min":0.39000000000000001,"15min":0.372},"max":{"1min":0.80100000000000005,"5min":0.81200000000000006,"15min":0.81200000000000006},"last":0.55100000000000005}]},{"osd":2,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.58299999999999996,"15min":0.55600000000000005},"min":{"1min":0.51100000000000001,"5min":0.435,"15min":0.30199999999999999},"max":{"1min":0.79600000000000004,"5min":0.79700000000000004,"15min":0.82199999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.57599999999999996,"15min":0.53900000000000003},"min":{"1min":0.438,"5min":0.38,"15min":0.33500000000000002},"max":{"1min":0.73799999999999999,"5min":0.76300000000000001,"15min":0.78100000000000003},"last":0.441}]},{"osd":3,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.62,"15min":0.59499999999999997},"min":{"1min":0.47399999999999998,"5min":0.435,"15min":0.39000000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64700000000000002,"15min":0.61299999999999999},"min":{"1min":0.47999999999999998,"5min":0.47999999999999998,"15min":0.38500000000000001},"max":{"1min":0.79200000000000004,"5min":0.878,"15min":0.878},"last":0.64700000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.47199999999999998},"min":{"1min":0.35799999999999998,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.64600000000000002,"5min":0.69199999999999995,"15min":0.71699999999999997},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.51200000000000001,"15min":0.51700000000000002},"min":{"1min":0.35199999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57799999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50700000000000001,"15min":0.50800000000000001},"min":{"1min":0.38200000000000001,"5min":0.315,"15min":0.315},"max":{"1min":0.622,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.52800000000000002,"15min":0.52900000000000003},"min":{"1min":0.377,"5min":0.36399999999999999,"15min":0.33600000000000002},"max":{"1min":0.77500000000000002,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.621}]},{"osd":7,"last update":"Tue Nov 28 03:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.55400000000000005,"15min":0.54100000000000004},"min":{"1min":0.41099999999999998,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.748,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.57899999999999996,"15min":0.57699999999999996},"min":{"1min":0.47099999999999997,"5min":0.40000000000000002,"15min":0.32900000000000001},"max":{"1min":0.71699999999999997,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.69499999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575000,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.82699999999999996},"last":0.501}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69199999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.432},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.377,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.76400000000000001,"15min":0.79800000000000004},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78700000000000003},"last":0.47799999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42599999999999999,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.54100000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.65400000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.622}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:05:08.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:05:08.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 Processor -- start 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 -- start start 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8ec0caab0 0x7fd8ec0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0cae90 0x7fd8ec0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd8ec0aa960 0x7fd8ec0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c1490 con 0x7fd8ec0caab0 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8ec0a9a90 con 0x7fd8ec0aa960 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.940+0000 7fd8f1960700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec154ce0 con 0x7fd8ec0cae90 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8ec0caab0 0x7fd8ec0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd8ec0caab0 0x7fd8ec0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54516/0 (socket says 172.21.15.73:54516) 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 -- 172.21.15.73:0/423785409 learned_addr learned my addr 172.21.15.73:0/423785409 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 -- 172.21.15.73:0/423785409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0cae90 msgr2=0x7fd8ec0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:08.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/423785409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0cae90 0x7fd8ec0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eb7fe700 1 --2- 172.21.15.73:0/423785409 >> v2:172.21.15.150:3300/0 conn(0x7fd8ec0aa960 0x7fd8ec0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 -- 172.21.15.73:0/423785409 >> v2:172.21.15.150:3300/0 conn(0x7fd8ec0aa960 msgr2=0x7fd8ec0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/423785409 >> v2:172.21.15.150:3300/0 conn(0x7fd8ec0aa960 0x7fd8ec0c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 -- 172.21.15.73:0/423785409 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8ec0a8e80 con 0x7fd8ec0caab0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/423785409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0caab0 0x7fd8ec0aa470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fd8ec05e5a0 tx=0x7fd8dc009060).ready entity=mon.0 client_cookie=e981a5f157680c51 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8ea7fc700 1 -- 172.21.15.73:0/423785409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005880 con 0x7fd8ec0caab0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8ea7fc700 1 -- 172.21.15.73:0/423785409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8dc0059e0 con 0x7fd8ec0caab0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.941+0000 7fd8ea7fc700 1 -- 172.21.15.73:0/423785409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8dc005d60 con 0x7fd8ec0caab0 2023-11-28T03:05:08.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 -- 172.21.15.73:0/423785409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0caab0 msgr2=0x7fd8ec0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 --2- 172.21.15.73:0/423785409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0caab0 0x7fd8ec0aa470 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 rx=0x7fd8ec05e5a0 tx=0x7fd8dc009060).stop 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 -- 172.21.15.73:0/423785409 shutdown_connections 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 --2- 172.21.15.73:0/423785409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0caab0 0x7fd8ec0aa470 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 --2- 172.21.15.73:0/423785409 >> v2:172.21.15.150:3300/0 conn(0x7fd8ec0aa960 0x7fd8ec0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 --2- 172.21.15.73:0/423785409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0cae90 0x7fd8ec0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 -- 172.21.15.73:0/423785409 >> 172.21.15.73:0/423785409 conn(0x7fd8ec094920 msgr2=0x7fd8ec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 -- 172.21.15.73:0/423785409 shutdown_connections 2023-11-28T03:05:08.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.942+0000 7fd8f1960700 1 -- 172.21.15.73:0/423785409 wait complete. 2023-11-28T03:05:08.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.943+0000 7fd8f1960700 1 Processor -- start 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.943+0000 7fd8f1960700 1 -- start start 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 0x7fd8ec153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8ec0b0650 0x7fd8ec0b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0b0fc0 con 0x7fd8ec0cae90 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec0c3ac0 con 0x7fd8ec0b0650 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8f1960700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8ec155b60 con 0x7fd8ec0caab0 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54528/0 (socket says 172.21.15.73:54528) 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 -- 172.21.15.73:0/4290217690 learned_addr learned my addr 172.21.15.73:0/4290217690 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8eb7fe700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8ec0b0650 0x7fd8ec0b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 0x7fd8ec153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 -- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 msgr2=0x7fd8ec153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 0x7fd8ec153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 -- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8ec0b0650 msgr2=0x7fd8ec0b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8ec0b0650 0x7fd8ec0b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8e27fc700 1 -- 172.21.15.73:0/4290217690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8dc009300 con 0x7fd8ec0cae90 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.944+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 0x7fd8ec153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8e27fc700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fd8d400b490 tx=0x7fd8d4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4002320 con 0x7fd8ec0cae90 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d4002480 con 0x7fd8ec0cae90 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d4002840 con 0x7fd8ec0cae90 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8ec155e40 con 0x7fd8ec0cae90 2023-11-28T03:05:08.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.945+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8ec153750 con 0x7fd8ec0cae90 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.946+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8ec0c3fa0 con 0x7fd8ec0cae90 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.946+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd8d4002bf0 con 0x7fd8ec0cae90 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.946+0000 7fd8e37fe700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 0x7fd8cc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.947+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd8d4079fd0 con 0x7fd8ec0cae90 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.947+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 0x7fd8cc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.947+0000 7fd8eaffd700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 0x7fd8cc061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd8dc00b3c0 tx=0x7fd8dc016000).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:08.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:08.951+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd8ec0c3fa0 con 0x7fd8ec0cae90 2023-11-28T03:05:09.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.112+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8ec0c3fa0 con 0x7fd8cc05f150 2023-11-28T03:05:09.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.121+0000 7fd8e37fe700 1 -- 172.21.15.73:0/4290217690 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361552 (secure 0 0 0) 0x7fd8ec0c3fa0 con 0x7fd8cc05f150 2023-11-28T03:05:09.127 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:09.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 msgr2=0x7fd8cc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 0x7fd8cc061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fd8dc00b3c0 tx=0x7fd8dc016000).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 msgr2=0x7fd8ec146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 rx=0x7fd8d400b490 tx=0x7fd8d4009060).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 shutdown_connections 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8cc05f150 0x7fd8cc061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd8ec0b0650 0x7fd8ec0b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd8ec0caab0 0x7fd8ec153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 --2- 172.21.15.73:0/4290217690 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd8ec0cae90 0x7fd8ec146e50 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 >> 172.21.15.73:0/4290217690 conn(0x7fd8ec094920 msgr2=0x7fd8ec0c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 shutdown_connections 2023-11-28T03:05:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.125+0000 7fd8f1960700 1 -- 172.21.15.73:0/4290217690 wait complete. 2023-11-28T03:05:09.131 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:05:09.138 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-28T03:05:08.870643+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2411552,"kb_used_data":6112,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752563168,"statfs":{"total":773094113280,"available":770624684032,"internally_reserved":0,"allocated":6258688,"data_stored":3042948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-14996258,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-720,"num_write_kb":-15545,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.499218"},"pg_stats":[{"pgid":"8.30","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796416+0000","last_change":"2023-11-28T03:05:00.791917+0000","last_active":"2023-11-28T03:05:00.796416+0000","last_peered":"2023-11-28T03:05:00.796416+0000","last_clean":"2023-11-28T03:05:00.796416+0000","last_became_active":"2023-11-28T03:05:00.791612+0000","last_became_peered":"2023-11-28T03:05:00.791612+0000","last_unstale":"2023-11-28T03:05:00.796416+0000","last_undegraded":"2023-11-28T03:05:00.796416+0000","last_fullsized":"2023-11-28T03:05:00.796416+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3f","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795306+0000","last_change":"2023-11-28T03:05:00.425524+0000","last_active":"2023-11-28T03:05:00.795306+0000","last_peered":"2023-11-28T03:05:00.795306+0000","last_clean":"2023-11-28T03:05:00.795306+0000","last_became_active":"2023-11-28T03:05:00.425163+0000","last_became_peered":"2023-11-28T03:05:00.425163+0000","last_unstale":"2023-11-28T03:05:00.795306+0000","last_undegraded":"2023-11-28T03:05:00.795306+0000","last_fullsized":"2023-11-28T03:05:00.795306+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.31","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800549+0000","last_change":"2023-11-28T03:05:00.795761+0000","last_active":"2023-11-28T03:05:00.800549+0000","last_peered":"2023-11-28T03:05:00.800549+0000","last_clean":"2023-11-28T03:05:00.800549+0000","last_became_active":"2023-11-28T03:05:00.795500+0000","last_became_peered":"2023-11-28T03:05:00.795500+0000","last_unstale":"2023-11-28T03:05:00.800549+0000","last_undegraded":"2023-11-28T03:05:00.800549+0000","last_fullsized":"2023-11-28T03:05:00.800549+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784809+0000","last_change":"2023-11-28T03:05:00.438659+0000","last_active":"2023-11-28T03:05:00.784809+0000","last_peered":"2023-11-28T03:05:00.784809+0000","last_clean":"2023-11-28T03:05:00.784809+0000","last_became_active":"2023-11-28T03:05:00.438343+0000","last_became_peered":"2023-11-28T03:05:00.438343+0000","last_unstale":"2023-11-28T03:05:00.784809+0000","last_undegraded":"2023-11-28T03:05:00.784809+0000","last_fullsized":"2023-11-28T03:05:00.784809+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.32","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807835+0000","last_change":"2023-11-28T03:05:00.804536+0000","last_active":"2023-11-28T03:05:00.807835+0000","last_peered":"2023-11-28T03:05:00.807835+0000","last_clean":"2023-11-28T03:05:00.807835+0000","last_became_active":"2023-11-28T03:05:00.804274+0000","last_became_peered":"2023-11-28T03:05:00.804274+0000","last_unstale":"2023-11-28T03:05:00.807835+0000","last_undegraded":"2023-11-28T03:05:00.807835+0000","last_fullsized":"2023-11-28T03:05:00.807835+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783457+0000","last_change":"2023-11-28T03:05:00.413262+0000","last_active":"2023-11-28T03:05:00.783457+0000","last_peered":"2023-11-28T03:05:00.783457+0000","last_clean":"2023-11-28T03:05:00.783457+0000","last_became_active":"2023-11-28T03:05:00.412961+0000","last_became_peered":"2023-11-28T03:05:00.412961+0000","last_unstale":"2023-11-28T03:05:00.783457+0000","last_undegraded":"2023-11-28T03:05:00.783457+0000","last_fullsized":"2023-11-28T03:05:00.783457+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.33","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796118+0000","last_change":"2023-11-28T03:05:00.791909+0000","last_active":"2023-11-28T03:05:00.796118+0000","last_peered":"2023-11-28T03:05:00.796118+0000","last_clean":"2023-11-28T03:05:00.796118+0000","last_became_active":"2023-11-28T03:05:00.791622+0000","last_became_peered":"2023-11-28T03:05:00.791622+0000","last_unstale":"2023-11-28T03:05:00.796118+0000","last_undegraded":"2023-11-28T03:05:00.796118+0000","last_fullsized":"2023-11-28T03:05:00.796118+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801601+0000","last_change":"2023-11-28T03:05:00.436611+0000","last_active":"2023-11-28T03:05:00.801601+0000","last_peered":"2023-11-28T03:05:00.801601+0000","last_clean":"2023-11-28T03:05:00.801601+0000","last_became_active":"2023-11-28T03:05:00.436324+0000","last_became_peered":"2023-11-28T03:05:00.436324+0000","last_unstale":"2023-11-28T03:05:00.801601+0000","last_undegraded":"2023-11-28T03:05:00.801601+0000","last_fullsized":"2023-11-28T03:05:00.801601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.34","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804685+0000","last_change":"2023-11-28T03:05:00.791992+0000","last_active":"2023-11-28T03:05:00.804685+0000","last_peered":"2023-11-28T03:05:00.804685+0000","last_clean":"2023-11-28T03:05:00.804685+0000","last_became_active":"2023-11-28T03:05:00.791738+0000","last_became_peered":"2023-11-28T03:05:00.791738+0000","last_unstale":"2023-11-28T03:05:00.804685+0000","last_undegraded":"2023-11-28T03:05:00.804685+0000","last_fullsized":"2023-11-28T03:05:00.804685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.804714+0000","last_change":"2023-11-28T03:05:00.437137+0000","last_active":"2023-11-28T03:05:00.804714+0000","last_peered":"2023-11-28T03:05:00.804714+0000","last_clean":"2023-11-28T03:05:00.804714+0000","last_became_active":"2023-11-28T03:05:00.436830+0000","last_became_peered":"2023-11-28T03:05:00.436830+0000","last_unstale":"2023-11-28T03:05:00.804714+0000","last_undegraded":"2023-11-28T03:05:00.804714+0000","last_fullsized":"2023-11-28T03:05:00.804714+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.35","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801683+0000","last_change":"2023-11-28T03:05:00.798043+0000","last_active":"2023-11-28T03:05:00.801683+0000","last_peered":"2023-11-28T03:05:00.801683+0000","last_clean":"2023-11-28T03:05:00.801683+0000","last_became_active":"2023-11-28T03:05:00.797785+0000","last_became_peered":"2023-11-28T03:05:00.797785+0000","last_unstale":"2023-11-28T03:05:00.801683+0000","last_undegraded":"2023-11-28T03:05:00.801683+0000","last_fullsized":"2023-11-28T03:05:00.801683+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3a","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794948+0000","last_change":"2023-11-28T03:05:00.425489+0000","last_active":"2023-11-28T03:05:00.794948+0000","last_peered":"2023-11-28T03:05:00.794948+0000","last_clean":"2023-11-28T03:05:00.794948+0000","last_became_active":"2023-11-28T03:05:00.425041+0000","last_became_peered":"2023-11-28T03:05:00.425041+0000","last_unstale":"2023-11-28T03:05:00.794948+0000","last_undegraded":"2023-11-28T03:05:00.794948+0000","last_fullsized":"2023-11-28T03:05:00.794948+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.36","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796138+0000","last_change":"2023-11-28T03:05:00.792581+0000","last_active":"2023-11-28T03:05:00.796138+0000","last_peered":"2023-11-28T03:05:00.796138+0000","last_clean":"2023-11-28T03:05:00.796138+0000","last_became_active":"2023-11-28T03:05:00.792322+0000","last_became_peered":"2023-11-28T03:05:00.792322+0000","last_unstale":"2023-11-28T03:05:00.796138+0000","last_undegraded":"2023-11-28T03:05:00.796138+0000","last_fullsized":"2023-11-28T03:05:00.796138+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787429+0000","last_change":"2023-11-28T03:05:00.431726+0000","last_active":"2023-11-28T03:05:00.787429+0000","last_peered":"2023-11-28T03:05:00.787429+0000","last_clean":"2023-11-28T03:05:00.787429+0000","last_became_active":"2023-11-28T03:05:00.431380+0000","last_became_peered":"2023-11-28T03:05:00.431380+0000","last_unstale":"2023-11-28T03:05:00.787429+0000","last_undegraded":"2023-11-28T03:05:00.787429+0000","last_fullsized":"2023-11-28T03:05:00.787429+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.37","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807353+0000","last_change":"2023-11-28T03:05:00.803609+0000","last_active":"2023-11-28T03:05:00.807353+0000","last_peered":"2023-11-28T03:05:00.807353+0000","last_clean":"2023-11-28T03:05:00.807353+0000","last_became_active":"2023-11-28T03:05:00.803362+0000","last_became_peered":"2023-11-28T03:05:00.803362+0000","last_unstale":"2023-11-28T03:05:00.807353+0000","last_undegraded":"2023-11-28T03:05:00.807353+0000","last_fullsized":"2023-11-28T03:05:00.807353+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785509+0000","last_change":"2023-11-28T03:05:00.420800+0000","last_active":"2023-11-28T03:05:00.785509+0000","last_peered":"2023-11-28T03:05:00.785509+0000","last_clean":"2023-11-28T03:05:00.785509+0000","last_became_active":"2023-11-28T03:05:00.420414+0000","last_became_peered":"2023-11-28T03:05:00.420414+0000","last_unstale":"2023-11-28T03:05:00.785509+0000","last_undegraded":"2023-11-28T03:05:00.785509+0000","last_fullsized":"2023-11-28T03:05:00.785509+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.38","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805747+0000","last_change":"2023-11-28T03:05:00.795247+0000","last_active":"2023-11-28T03:05:00.805747+0000","last_peered":"2023-11-28T03:05:00.805747+0000","last_clean":"2023-11-28T03:05:00.805747+0000","last_became_active":"2023-11-28T03:05:00.795001+0000","last_became_peered":"2023-11-28T03:05:00.795001+0000","last_unstale":"2023-11-28T03:05:00.805747+0000","last_undegraded":"2023-11-28T03:05:00.805747+0000","last_fullsized":"2023-11-28T03:05:00.805747+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784311+0000","last_change":"2023-11-28T03:05:00.429277+0000","last_active":"2023-11-28T03:05:00.784311+0000","last_peered":"2023-11-28T03:05:00.784311+0000","last_clean":"2023-11-28T03:05:00.784311+0000","last_became_active":"2023-11-28T03:05:00.428938+0000","last_became_peered":"2023-11-28T03:05:00.428938+0000","last_unstale":"2023-11-28T03:05:00.784311+0000","last_undegraded":"2023-11-28T03:05:00.784311+0000","last_fullsized":"2023-11-28T03:05:00.784311+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.39","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806756+0000","last_change":"2023-11-28T03:05:00.803701+0000","last_active":"2023-11-28T03:05:00.806756+0000","last_peered":"2023-11-28T03:05:00.806756+0000","last_clean":"2023-11-28T03:05:00.806756+0000","last_became_active":"2023-11-28T03:05:00.803424+0000","last_became_peered":"2023-11-28T03:05:00.803424+0000","last_unstale":"2023-11-28T03:05:00.806756+0000","last_undegraded":"2023-11-28T03:05:00.806756+0000","last_fullsized":"2023-11-28T03:05:00.806756+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783367+0000","last_change":"2023-11-28T03:05:00.437611+0000","last_active":"2023-11-28T03:05:00.783367+0000","last_peered":"2023-11-28T03:05:00.783367+0000","last_clean":"2023-11-28T03:05:00.783367+0000","last_became_active":"2023-11-28T03:05:00.437351+0000","last_became_peered":"2023-11-28T03:05:00.437351+0000","last_unstale":"2023-11-28T03:05:00.783367+0000","last_undegraded":"2023-11-28T03:05:00.783367+0000","last_fullsized":"2023-11-28T03:05:00.783367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806347+0000","last_change":"2023-11-28T03:05:00.801846+0000","last_active":"2023-11-28T03:05:00.806347+0000","last_peered":"2023-11-28T03:05:00.806347+0000","last_clean":"2023-11-28T03:05:00.806347+0000","last_became_active":"2023-11-28T03:05:00.801594+0000","last_became_peered":"2023-11-28T03:05:00.801594+0000","last_unstale":"2023-11-28T03:05:00.806347+0000","last_undegraded":"2023-11-28T03:05:00.806347+0000","last_fullsized":"2023-11-28T03:05:00.806347+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783455+0000","last_change":"2023-11-28T03:05:00.437201+0000","last_active":"2023-11-28T03:05:00.783455+0000","last_peered":"2023-11-28T03:05:00.783455+0000","last_clean":"2023-11-28T03:05:00.783455+0000","last_became_active":"2023-11-28T03:05:00.436875+0000","last_became_peered":"2023-11-28T03:05:00.436875+0000","last_unstale":"2023-11-28T03:05:00.783455+0000","last_undegraded":"2023-11-28T03:05:00.783455+0000","last_fullsized":"2023-11-28T03:05:00.783455+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854418+0000","last_change":"2023-11-28T03:05:00.853613+0000","last_active":"2023-11-28T03:05:00.854418+0000","last_peered":"2023-11-28T03:05:00.854418+0000","last_clean":"2023-11-28T03:05:00.854418+0000","last_became_active":"2023-11-28T03:05:00.853268+0000","last_became_peered":"2023-11-28T03:05:00.853268+0000","last_unstale":"2023-11-28T03:05:00.854418+0000","last_undegraded":"2023-11-28T03:05:00.854418+0000","last_fullsized":"2023-11-28T03:05:00.854418+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.34","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809619+0000","last_change":"2023-11-28T03:05:00.424153+0000","last_active":"2023-11-28T03:05:00.809619+0000","last_peered":"2023-11-28T03:05:00.809619+0000","last_clean":"2023-11-28T03:05:00.809619+0000","last_became_active":"2023-11-28T03:05:00.423700+0000","last_became_peered":"2023-11-28T03:05:00.423700+0000","last_unstale":"2023-11-28T03:05:00.809619+0000","last_undegraded":"2023-11-28T03:05:00.809619+0000","last_fullsized":"2023-11-28T03:05:00.809619+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.3c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809404+0000","last_change":"2023-11-28T03:05:00.805354+0000","last_active":"2023-11-28T03:05:00.809404+0000","last_peered":"2023-11-28T03:05:00.809404+0000","last_clean":"2023-11-28T03:05:00.809404+0000","last_became_active":"2023-11-28T03:05:00.805055+0000","last_became_peered":"2023-11-28T03:05:00.805055+0000","last_unstale":"2023-11-28T03:05:00.809404+0000","last_undegraded":"2023-11-28T03:05:00.809404+0000","last_fullsized":"2023-11-28T03:05:00.809404+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.33","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807391+0000","last_change":"2023-11-28T03:05:00.425518+0000","last_active":"2023-11-28T03:05:00.807391+0000","last_peered":"2023-11-28T03:05:00.807391+0000","last_clean":"2023-11-28T03:05:00.807391+0000","last_became_active":"2023-11-28T03:05:00.425264+0000","last_became_peered":"2023-11-28T03:05:00.425264+0000","last_unstale":"2023-11-28T03:05:00.807391+0000","last_undegraded":"2023-11-28T03:05:00.807391+0000","last_fullsized":"2023-11-28T03:05:00.807391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806750+0000","last_change":"2023-11-28T03:05:00.802976+0000","last_active":"2023-11-28T03:05:00.806750+0000","last_peered":"2023-11-28T03:05:00.806750+0000","last_clean":"2023-11-28T03:05:00.806750+0000","last_became_active":"2023-11-28T03:05:00.802705+0000","last_became_peered":"2023-11-28T03:05:00.802705+0000","last_unstale":"2023-11-28T03:05:00.806750+0000","last_undegraded":"2023-11-28T03:05:00.806750+0000","last_fullsized":"2023-11-28T03:05:00.806750+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787102+0000","last_change":"2023-11-28T03:05:00.429504+0000","last_active":"2023-11-28T03:05:00.787102+0000","last_peered":"2023-11-28T03:05:00.787102+0000","last_clean":"2023-11-28T03:05:00.787102+0000","last_became_active":"2023-11-28T03:05:00.429272+0000","last_became_peered":"2023-11-28T03:05:00.429272+0000","last_unstale":"2023-11-28T03:05:00.787102+0000","last_undegraded":"2023-11-28T03:05:00.787102+0000","last_fullsized":"2023-11-28T03:05:00.787102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.3e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802325+0000","last_change":"2023-11-28T03:05:00.800107+0000","last_active":"2023-11-28T03:05:00.802325+0000","last_peered":"2023-11-28T03:05:00.802325+0000","last_clean":"2023-11-28T03:05:00.802325+0000","last_became_active":"2023-11-28T03:05:00.799586+0000","last_became_peered":"2023-11-28T03:05:00.799586+0000","last_unstale":"2023-11-28T03:05:00.802325+0000","last_undegraded":"2023-11-28T03:05:00.802325+0000","last_fullsized":"2023-11-28T03:05:00.802325+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780802+0000","last_change":"2023-11-28T03:05:00.423474+0000","last_active":"2023-11-28T03:05:00.780802+0000","last_peered":"2023-11-28T03:05:00.780802+0000","last_clean":"2023-11-28T03:05:00.780802+0000","last_became_active":"2023-11-28T03:05:00.423230+0000","last_became_peered":"2023-11-28T03:05:00.423230+0000","last_unstale":"2023-11-28T03:05:00.780802+0000","last_undegraded":"2023-11-28T03:05:00.780802+0000","last_fullsized":"2023-11-28T03:05:00.780802+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799306+0000","last_change":"2023-11-28T03:05:00.797655+0000","last_active":"2023-11-28T03:05:00.799306+0000","last_peered":"2023-11-28T03:05:00.799306+0000","last_clean":"2023-11-28T03:05:00.799306+0000","last_became_active":"2023-11-28T03:05:00.797293+0000","last_became_peered":"2023-11-28T03:05:00.797293+0000","last_unstale":"2023-11-28T03:05:00.799306+0000","last_undegraded":"2023-11-28T03:05:00.799306+0000","last_fullsized":"2023-11-28T03:05:00.799306+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.30","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805668+0000","last_change":"2023-11-28T03:05:00.423972+0000","last_active":"2023-11-28T03:05:00.805668+0000","last_peered":"2023-11-28T03:05:00.805668+0000","last_clean":"2023-11-28T03:05:00.805668+0000","last_became_active":"2023-11-28T03:05:00.423595+0000","last_became_peered":"2023-11-28T03:05:00.423595+0000","last_unstale":"2023-11-28T03:05:00.805668+0000","last_undegraded":"2023-11-28T03:05:00.805668+0000","last_fullsized":"2023-11-28T03:05:00.805668+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.20","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801748+0000","last_change":"2023-11-28T03:05:00.799701+0000","last_active":"2023-11-28T03:05:00.801748+0000","last_peered":"2023-11-28T03:05:00.801748+0000","last_clean":"2023-11-28T03:05:00.801748+0000","last_became_active":"2023-11-28T03:05:00.797686+0000","last_became_peered":"2023-11-28T03:05:00.797686+0000","last_unstale":"2023-11-28T03:05:00.801748+0000","last_undegraded":"2023-11-28T03:05:00.801748+0000","last_fullsized":"2023-11-28T03:05:00.801748+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783803+0000","last_change":"2023-11-28T03:05:00.422200+0000","last_active":"2023-11-28T03:05:00.783803+0000","last_peered":"2023-11-28T03:05:00.783803+0000","last_clean":"2023-11-28T03:05:00.783803+0000","last_became_active":"2023-11-28T03:05:00.421929+0000","last_became_peered":"2023-11-28T03:05:00.421929+0000","last_unstale":"2023-11-28T03:05:00.783803+0000","last_undegraded":"2023-11-28T03:05:00.783803+0000","last_fullsized":"2023-11-28T03:05:00.783803+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.21","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806499+0000","last_change":"2023-11-28T03:05:00.800024+0000","last_active":"2023-11-28T03:05:00.806499+0000","last_peered":"2023-11-28T03:05:00.806499+0000","last_clean":"2023-11-28T03:05:00.806499+0000","last_became_active":"2023-11-28T03:05:00.799789+0000","last_became_peered":"2023-11-28T03:05:00.799789+0000","last_unstale":"2023-11-28T03:05:00.806499+0000","last_undegraded":"2023-11-28T03:05:00.806499+0000","last_fullsized":"2023-11-28T03:05:00.806499+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783737+0000","last_change":"2023-11-28T03:05:00.436677+0000","last_active":"2023-11-28T03:05:00.783737+0000","last_peered":"2023-11-28T03:05:00.783737+0000","last_clean":"2023-11-28T03:05:00.783737+0000","last_became_active":"2023-11-28T03:05:00.436405+0000","last_became_peered":"2023-11-28T03:05:00.436405+0000","last_unstale":"2023-11-28T03:05:00.783737+0000","last_undegraded":"2023-11-28T03:05:00.783737+0000","last_fullsized":"2023-11-28T03:05:00.783737+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.22","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801912+0000","last_change":"2023-11-28T03:05:00.796280+0000","last_active":"2023-11-28T03:05:00.801912+0000","last_peered":"2023-11-28T03:05:00.801912+0000","last_clean":"2023-11-28T03:05:00.801912+0000","last_became_active":"2023-11-28T03:05:00.795931+0000","last_became_peered":"2023-11-28T03:05:00.795931+0000","last_unstale":"2023-11-28T03:05:00.801912+0000","last_undegraded":"2023-11-28T03:05:00.801912+0000","last_fullsized":"2023-11-28T03:05:00.801912+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2d","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805408+0000","last_change":"2023-11-28T03:05:00.437430+0000","last_active":"2023-11-28T03:05:00.805408+0000","last_peered":"2023-11-28T03:05:00.805408+0000","last_clean":"2023-11-28T03:05:00.805408+0000","last_became_active":"2023-11-28T03:05:00.437155+0000","last_became_peered":"2023-11-28T03:05:00.437155+0000","last_unstale":"2023-11-28T03:05:00.805408+0000","last_undegraded":"2023-11-28T03:05:00.805408+0000","last_fullsized":"2023-11-28T03:05:00.805408+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.23","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806687+0000","last_change":"2023-11-28T03:05:00.803087+0000","last_active":"2023-11-28T03:05:00.806687+0000","last_peered":"2023-11-28T03:05:00.806687+0000","last_clean":"2023-11-28T03:05:00.806687+0000","last_became_active":"2023-11-28T03:05:00.801329+0000","last_became_peered":"2023-11-28T03:05:00.801329+0000","last_unstale":"2023-11-28T03:05:00.806687+0000","last_undegraded":"2023-11-28T03:05:00.806687+0000","last_fullsized":"2023-11-28T03:05:00.806687+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.790507+0000","last_change":"2023-11-28T03:05:00.437255+0000","last_active":"2023-11-28T03:05:00.790507+0000","last_peered":"2023-11-28T03:05:00.790507+0000","last_clean":"2023-11-28T03:05:00.790507+0000","last_became_active":"2023-11-28T03:05:00.436973+0000","last_became_peered":"2023-11-28T03:05:00.436973+0000","last_unstale":"2023-11-28T03:05:00.790507+0000","last_undegraded":"2023-11-28T03:05:00.790507+0000","last_fullsized":"2023-11-28T03:05:00.790507+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.24","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799336+0000","last_change":"2023-11-28T03:05:00.797320+0000","last_active":"2023-11-28T03:05:00.799336+0000","last_peered":"2023-11-28T03:05:00.799336+0000","last_clean":"2023-11-28T03:05:00.799336+0000","last_became_active":"2023-11-28T03:05:00.796972+0000","last_became_peered":"2023-11-28T03:05:00.796972+0000","last_unstale":"2023-11-28T03:05:00.799336+0000","last_undegraded":"2023-11-28T03:05:00.799336+0000","last_fullsized":"2023-11-28T03:05:00.799336+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795672+0000","last_change":"2023-11-28T03:05:00.423051+0000","last_active":"2023-11-28T03:05:00.795672+0000","last_peered":"2023-11-28T03:05:00.795672+0000","last_clean":"2023-11-28T03:05:00.795672+0000","last_became_active":"2023-11-28T03:05:00.422598+0000","last_became_peered":"2023-11-28T03:05:00.422598+0000","last_unstale":"2023-11-28T03:05:00.795672+0000","last_undegraded":"2023-11-28T03:05:00.795672+0000","last_fullsized":"2023-11-28T03:05:00.795672+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.25","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809795+0000","last_change":"2023-11-28T03:05:00.806914+0000","last_active":"2023-11-28T03:05:00.809795+0000","last_peered":"2023-11-28T03:05:00.809795+0000","last_clean":"2023-11-28T03:05:00.809795+0000","last_became_active":"2023-11-28T03:05:00.806635+0000","last_became_peered":"2023-11-28T03:05:00.806635+0000","last_unstale":"2023-11-28T03:05:00.809795+0000","last_undegraded":"2023-11-28T03:05:00.809795+0000","last_fullsized":"2023-11-28T03:05:00.809795+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787095+0000","last_change":"2023-11-28T03:05:00.423259+0000","last_active":"2023-11-28T03:05:00.787095+0000","last_peered":"2023-11-28T03:05:00.787095+0000","last_clean":"2023-11-28T03:05:00.787095+0000","last_became_active":"2023-11-28T03:05:00.422970+0000","last_became_peered":"2023-11-28T03:05:00.422970+0000","last_unstale":"2023-11-28T03:05:00.787095+0000","last_undegraded":"2023-11-28T03:05:00.787095+0000","last_fullsized":"2023-11-28T03:05:00.787095+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.26","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807191+0000","last_change":"2023-11-28T03:05:00.802199+0000","last_active":"2023-11-28T03:05:00.807191+0000","last_peered":"2023-11-28T03:05:00.807191+0000","last_clean":"2023-11-28T03:05:00.807191+0000","last_became_active":"2023-11-28T03:05:00.801950+0000","last_became_peered":"2023-11-28T03:05:00.801950+0000","last_unstale":"2023-11-28T03:05:00.807191+0000","last_undegraded":"2023-11-28T03:05:00.807191+0000","last_fullsized":"2023-11-28T03:05:00.807191+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781349+0000","last_change":"2023-11-28T03:05:00.418282+0000","last_active":"2023-11-28T03:05:00.781349+0000","last_peered":"2023-11-28T03:05:00.781349+0000","last_clean":"2023-11-28T03:05:00.781349+0000","last_became_active":"2023-11-28T03:05:00.417976+0000","last_became_peered":"2023-11-28T03:05:00.417976+0000","last_unstale":"2023-11-28T03:05:00.781349+0000","last_undegraded":"2023-11-28T03:05:00.781349+0000","last_fullsized":"2023-11-28T03:05:00.781349+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.27","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806696+0000","last_change":"2023-11-28T03:05:00.802071+0000","last_active":"2023-11-28T03:05:00.806696+0000","last_peered":"2023-11-28T03:05:00.806696+0000","last_clean":"2023-11-28T03:05:00.806696+0000","last_became_active":"2023-11-28T03:05:00.801430+0000","last_became_peered":"2023-11-28T03:05:00.801430+0000","last_unstale":"2023-11-28T03:05:00.806696+0000","last_undegraded":"2023-11-28T03:05:00.806696+0000","last_fullsized":"2023-11-28T03:05:00.806696+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781670+0000","last_change":"2023-11-28T03:05:00.418862+0000","last_active":"2023-11-28T03:05:00.781670+0000","last_peered":"2023-11-28T03:05:00.781670+0000","last_clean":"2023-11-28T03:05:00.781670+0000","last_became_active":"2023-11-28T03:05:00.418589+0000","last_became_peered":"2023-11-28T03:05:00.418589+0000","last_unstale":"2023-11-28T03:05:00.781670+0000","last_undegraded":"2023-11-28T03:05:00.781670+0000","last_fullsized":"2023-11-28T03:05:00.781670+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.28","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806755+0000","last_change":"2023-11-28T03:05:00.803646+0000","last_active":"2023-11-28T03:05:00.806755+0000","last_peered":"2023-11-28T03:05:00.806755+0000","last_clean":"2023-11-28T03:05:00.806755+0000","last_became_active":"2023-11-28T03:05:00.803400+0000","last_became_peered":"2023-11-28T03:05:00.803400+0000","last_unstale":"2023-11-28T03:05:00.806755+0000","last_undegraded":"2023-11-28T03:05:00.806755+0000","last_fullsized":"2023-11-28T03:05:00.806755+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781821+0000","last_change":"2023-11-28T03:05:00.428730+0000","last_active":"2023-11-28T03:05:00.781821+0000","last_peered":"2023-11-28T03:05:00.781821+0000","last_clean":"2023-11-28T03:05:00.781821+0000","last_became_active":"2023-11-28T03:05:00.428404+0000","last_became_peered":"2023-11-28T03:05:00.428404+0000","last_unstale":"2023-11-28T03:05:00.781821+0000","last_undegraded":"2023-11-28T03:05:00.781821+0000","last_fullsized":"2023-11-28T03:05:00.781821+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.29","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803533+0000","last_change":"2023-11-28T03:05:00.802752+0000","last_active":"2023-11-28T03:05:00.803533+0000","last_peered":"2023-11-28T03:05:00.803533+0000","last_clean":"2023-11-28T03:05:00.803533+0000","last_became_active":"2023-11-28T03:05:00.802481+0000","last_became_peered":"2023-11-28T03:05:00.802481+0000","last_unstale":"2023-11-28T03:05:00.803533+0000","last_undegraded":"2023-11-28T03:05:00.803533+0000","last_fullsized":"2023-11-28T03:05:00.803533+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.26","version":"65'1","reported_seq":11,"reported_epoch":65,"state":"active+clean","last_fresh":"2023-11-28T03:05:01.819126+0000","last_change":"2023-11-28T03:05:00.425905+0000","last_active":"2023-11-28T03:05:01.819126+0000","last_peered":"2023-11-28T03:05:01.819126+0000","last_clean":"2023-11-28T03:05:01.819126+0000","last_became_active":"2023-11-28T03:05:00.424944+0000","last_became_peered":"2023-11-28T03:05:00.424944+0000","last_unstale":"2023-11-28T03:05:01.819126+0000","last_undegraded":"2023-11-28T03:05:01.819126+0000","last_fullsized":"2023-11-28T03:05:01.819126+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801767+0000","last_change":"2023-11-28T03:05:00.793768+0000","last_active":"2023-11-28T03:05:00.801767+0000","last_peered":"2023-11-28T03:05:00.801767+0000","last_clean":"2023-11-28T03:05:00.801767+0000","last_became_active":"2023-11-28T03:05:00.793493+0000","last_became_peered":"2023-11-28T03:05:00.793493+0000","last_unstale":"2023-11-28T03:05:00.801767+0000","last_undegraded":"2023-11-28T03:05:00.801767+0000","last_fullsized":"2023-11-28T03:05:00.801767+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783477+0000","last_change":"2023-11-28T03:05:00.418986+0000","last_active":"2023-11-28T03:05:00.783477+0000","last_peered":"2023-11-28T03:05:00.783477+0000","last_clean":"2023-11-28T03:05:00.783477+0000","last_became_active":"2023-11-28T03:05:00.418713+0000","last_became_peered":"2023-11-28T03:05:00.418713+0000","last_unstale":"2023-11-28T03:05:00.783477+0000","last_undegraded":"2023-11-28T03:05:00.783477+0000","last_fullsized":"2023-11-28T03:05:00.783477+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799787+0000","last_change":"2023-11-28T03:05:00.798686+0000","last_active":"2023-11-28T03:05:00.799787+0000","last_peered":"2023-11-28T03:05:00.799787+0000","last_clean":"2023-11-28T03:05:00.799787+0000","last_became_active":"2023-11-28T03:05:00.798354+0000","last_became_peered":"2023-11-28T03:05:00.798354+0000","last_unstale":"2023-11-28T03:05:00.799787+0000","last_undegraded":"2023-11-28T03:05:00.799787+0000","last_fullsized":"2023-11-28T03:05:00.799787+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785410+0000","last_change":"2023-11-28T03:05:00.423987+0000","last_active":"2023-11-28T03:05:00.785410+0000","last_peered":"2023-11-28T03:05:00.785410+0000","last_clean":"2023-11-28T03:05:00.785410+0000","last_became_active":"2023-11-28T03:05:00.423314+0000","last_became_peered":"2023-11-28T03:05:00.423314+0000","last_unstale":"2023-11-28T03:05:00.785410+0000","last_undegraded":"2023-11-28T03:05:00.785410+0000","last_fullsized":"2023-11-28T03:05:00.785410+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.2c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795546+0000","last_change":"2023-11-28T03:05:00.792221+0000","last_active":"2023-11-28T03:05:00.795546+0000","last_peered":"2023-11-28T03:05:00.795546+0000","last_clean":"2023-11-28T03:05:00.795546+0000","last_became_active":"2023-11-28T03:05:00.791903+0000","last_became_peered":"2023-11-28T03:05:00.791903+0000","last_unstale":"2023-11-28T03:05:00.795546+0000","last_undegraded":"2023-11-28T03:05:00.795546+0000","last_fullsized":"2023-11-28T03:05:00.795546+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782316+0000","last_change":"2023-11-28T03:05:00.428717+0000","last_active":"2023-11-28T03:05:00.782316+0000","last_peered":"2023-11-28T03:05:00.782316+0000","last_clean":"2023-11-28T03:05:00.782316+0000","last_became_active":"2023-11-28T03:05:00.428428+0000","last_became_peered":"2023-11-28T03:05:00.428428+0000","last_unstale":"2023-11-28T03:05:00.782316+0000","last_undegraded":"2023-11-28T03:05:00.782316+0000","last_fullsized":"2023-11-28T03:05:00.782316+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.2d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854381+0000","last_change":"2023-11-28T03:05:00.853726+0000","last_active":"2023-11-28T03:05:00.854381+0000","last_peered":"2023-11-28T03:05:00.854381+0000","last_clean":"2023-11-28T03:05:00.854381+0000","last_became_active":"2023-11-28T03:05:00.853447+0000","last_became_peered":"2023-11-28T03:05:00.853447+0000","last_unstale":"2023-11-28T03:05:00.854381+0000","last_undegraded":"2023-11-28T03:05:00.854381+0000","last_fullsized":"2023-11-28T03:05:00.854381+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783385+0000","last_change":"2023-11-28T03:05:00.420805+0000","last_active":"2023-11-28T03:05:00.783385+0000","last_peered":"2023-11-28T03:05:00.783385+0000","last_clean":"2023-11-28T03:05:00.783385+0000","last_became_active":"2023-11-28T03:05:00.420529+0000","last_became_peered":"2023-11-28T03:05:00.420529+0000","last_unstale":"2023-11-28T03:05:00.783385+0000","last_undegraded":"2023-11-28T03:05:00.783385+0000","last_fullsized":"2023-11-28T03:05:00.783385+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.2e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802344+0000","last_change":"2023-11-28T03:05:00.800261+0000","last_active":"2023-11-28T03:05:00.802344+0000","last_peered":"2023-11-28T03:05:00.802344+0000","last_clean":"2023-11-28T03:05:00.802344+0000","last_became_active":"2023-11-28T03:05:00.799863+0000","last_became_peered":"2023-11-28T03:05:00.799863+0000","last_unstale":"2023-11-28T03:05:00.802344+0000","last_undegraded":"2023-11-28T03:05:00.802344+0000","last_fullsized":"2023-11-28T03:05:00.802344+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.779918+0000","last_change":"2023-11-28T03:05:00.424347+0000","last_active":"2023-11-28T03:05:00.779918+0000","last_peered":"2023-11-28T03:05:00.779918+0000","last_clean":"2023-11-28T03:05:00.779918+0000","last_became_active":"2023-11-28T03:05:00.424002+0000","last_became_peered":"2023-11-28T03:05:00.424002+0000","last_unstale":"2023-11-28T03:05:00.779918+0000","last_undegraded":"2023-11-28T03:05:00.779918+0000","last_fullsized":"2023-11-28T03:05:00.779918+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.2f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806717+0000","last_change":"2023-11-28T03:05:00.803173+0000","last_active":"2023-11-28T03:05:00.806717+0000","last_peered":"2023-11-28T03:05:00.806717+0000","last_clean":"2023-11-28T03:05:00.806717+0000","last_became_active":"2023-11-28T03:05:00.802493+0000","last_became_peered":"2023-11-28T03:05:00.802493+0000","last_unstale":"2023-11-28T03:05:00.806717+0000","last_undegraded":"2023-11-28T03:05:00.806717+0000","last_fullsized":"2023-11-28T03:05:00.806717+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784497+0000","last_change":"2023-11-28T03:05:00.432485+0000","last_active":"2023-11-28T03:05:00.784497+0000","last_peered":"2023-11-28T03:05:00.784497+0000","last_clean":"2023-11-28T03:05:00.784497+0000","last_became_active":"2023-11-28T03:05:00.432202+0000","last_became_peered":"2023-11-28T03:05:00.432202+0000","last_unstale":"2023-11-28T03:05:00.784497+0000","last_undegraded":"2023-11-28T03:05:00.784497+0000","last_fullsized":"2023-11-28T03:05:00.784497+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.805558+0000","last_change":"2023-11-28T03:05:00.800890+0000","last_active":"2023-11-28T03:05:00.805558+0000","last_peered":"2023-11-28T03:05:00.805558+0000","last_clean":"2023-11-28T03:05:00.805558+0000","last_became_active":"2023-11-28T03:05:00.800607+0000","last_became_peered":"2023-11-28T03:05:00.800607+0000","last_unstale":"2023-11-28T03:05:00.805558+0000","last_undegraded":"2023-11-28T03:05:00.805558+0000","last_fullsized":"2023-11-28T03:05:00.805558+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795186+0000","last_change":"2023-11-28T03:05:00.420535+0000","last_active":"2023-11-28T03:05:00.795186+0000","last_peered":"2023-11-28T03:05:00.795186+0000","last_clean":"2023-11-28T03:05:00.795186+0000","last_became_active":"2023-11-28T03:05:00.419707+0000","last_became_peered":"2023-11-28T03:05:00.419707+0000","last_unstale":"2023-11-28T03:05:00.795186+0000","last_undegraded":"2023-11-28T03:05:00.795186+0000","last_fullsized":"2023-11-28T03:05:00.795186+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802388+0000","last_change":"2023-11-28T03:05:00.801453+0000","last_active":"2023-11-28T03:05:00.802388+0000","last_peered":"2023-11-28T03:05:00.802388+0000","last_clean":"2023-11-28T03:05:00.802388+0000","last_became_active":"2023-11-28T03:05:00.801184+0000","last_became_peered":"2023-11-28T03:05:00.801184+0000","last_unstale":"2023-11-28T03:05:00.802388+0000","last_undegraded":"2023-11-28T03:05:00.802388+0000","last_fullsized":"2023-11-28T03:05:00.802388+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794713+0000","last_change":"2023-11-28T03:05:00.428126+0000","last_active":"2023-11-28T03:05:00.794713+0000","last_peered":"2023-11-28T03:05:00.794713+0000","last_clean":"2023-11-28T03:05:00.794713+0000","last_became_active":"2023-11-28T03:05:00.427535+0000","last_became_peered":"2023-11-28T03:05:00.427535+0000","last_unstale":"2023-11-28T03:05:00.794713+0000","last_undegraded":"2023-11-28T03:05:00.794713+0000","last_fullsized":"2023-11-28T03:05:00.794713+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795601+0000","last_change":"2023-11-28T03:05:00.793229+0000","last_active":"2023-11-28T03:05:00.795601+0000","last_peered":"2023-11-28T03:05:00.795601+0000","last_clean":"2023-11-28T03:05:00.795601+0000","last_became_active":"2023-11-28T03:05:00.792929+0000","last_became_peered":"2023-11-28T03:05:00.792929+0000","last_unstale":"2023-11-28T03:05:00.795601+0000","last_undegraded":"2023-11-28T03:05:00.795601+0000","last_fullsized":"2023-11-28T03:05:00.795601+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795082+0000","last_change":"2023-11-28T03:05:00.430218+0000","last_active":"2023-11-28T03:05:00.795082+0000","last_peered":"2023-11-28T03:05:00.795082+0000","last_clean":"2023-11-28T03:05:00.795082+0000","last_became_active":"2023-11-28T03:05:00.427406+0000","last_became_peered":"2023-11-28T03:05:00.427406+0000","last_unstale":"2023-11-28T03:05:00.795082+0000","last_undegraded":"2023-11-28T03:05:00.795082+0000","last_fullsized":"2023-11-28T03:05:00.795082+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794703+0000","last_change":"2023-11-28T03:05:00.792518+0000","last_active":"2023-11-28T03:05:00.794703+0000","last_peered":"2023-11-28T03:05:00.794703+0000","last_clean":"2023-11-28T03:05:00.794703+0000","last_became_active":"2023-11-28T03:05:00.792351+0000","last_became_peered":"2023-11-28T03:05:00.792351+0000","last_unstale":"2023-11-28T03:05:00.794703+0000","last_undegraded":"2023-11-28T03:05:00.794703+0000","last_fullsized":"2023-11-28T03:05:00.794703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785523+0000","last_change":"2023-11-28T03:05:00.426818+0000","last_active":"2023-11-28T03:05:00.785523+0000","last_peered":"2023-11-28T03:05:00.785523+0000","last_clean":"2023-11-28T03:05:00.785523+0000","last_became_active":"2023-11-28T03:05:00.426491+0000","last_became_peered":"2023-11-28T03:05:00.426491+0000","last_unstale":"2023-11-28T03:05:00.785523+0000","last_undegraded":"2023-11-28T03:05:00.785523+0000","last_fullsized":"2023-11-28T03:05:00.785523+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854409+0000","last_change":"2023-11-28T03:05:00.853635+0000","last_active":"2023-11-28T03:05:00.854409+0000","last_peered":"2023-11-28T03:05:00.854409+0000","last_clean":"2023-11-28T03:05:00.854409+0000","last_became_active":"2023-11-28T03:05:00.853345+0000","last_became_peered":"2023-11-28T03:05:00.853345+0000","last_unstale":"2023-11-28T03:05:00.854409+0000","last_undegraded":"2023-11-28T03:05:00.854409+0000","last_fullsized":"2023-11-28T03:05:00.854409+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782759+0000","last_change":"2023-11-28T03:05:00.423415+0000","last_active":"2023-11-28T03:05:00.782759+0000","last_peered":"2023-11-28T03:05:00.782759+0000","last_clean":"2023-11-28T03:05:00.782759+0000","last_became_active":"2023-11-28T03:05:00.423212+0000","last_became_peered":"2023-11-28T03:05:00.423212+0000","last_unstale":"2023-11-28T03:05:00.782759+0000","last_undegraded":"2023-11-28T03:05:00.782759+0000","last_fullsized":"2023-11-28T03:05:00.782759+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802074+0000","last_change":"2023-11-28T03:05:00.797871+0000","last_active":"2023-11-28T03:05:00.802074+0000","last_peered":"2023-11-28T03:05:00.802074+0000","last_clean":"2023-11-28T03:05:00.802074+0000","last_became_active":"2023-11-28T03:05:00.796189+0000","last_became_peered":"2023-11-28T03:05:00.796189+0000","last_unstale":"2023-11-28T03:05:00.802074+0000","last_undegraded":"2023-11-28T03:05:00.802074+0000","last_fullsized":"2023-11-28T03:05:00.802074+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783586+0000","last_change":"2023-11-28T03:05:00.411962+0000","last_active":"2023-11-28T03:05:00.783586+0000","last_peered":"2023-11-28T03:05:00.783586+0000","last_clean":"2023-11-28T03:05:00.783586+0000","last_became_active":"2023-11-28T03:05:00.411666+0000","last_became_peered":"2023-11-28T03:05:00.411666+0000","last_unstale":"2023-11-28T03:05:00.783586+0000","last_undegraded":"2023-11-28T03:05:00.783586+0000","last_fullsized":"2023-11-28T03:05:00.783586+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807208+0000","last_change":"2023-11-28T03:05:00.802294+0000","last_active":"2023-11-28T03:05:00.807208+0000","last_peered":"2023-11-28T03:05:00.807208+0000","last_clean":"2023-11-28T03:05:00.807208+0000","last_became_active":"2023-11-28T03:05:00.802023+0000","last_became_peered":"2023-11-28T03:05:00.802023+0000","last_unstale":"2023-11-28T03:05:00.807208+0000","last_undegraded":"2023-11-28T03:05:00.807208+0000","last_fullsized":"2023-11-28T03:05:00.807208+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781451+0000","last_change":"2023-11-28T03:05:00.424948+0000","last_active":"2023-11-28T03:05:00.781451+0000","last_peered":"2023-11-28T03:05:00.781451+0000","last_clean":"2023-11-28T03:05:00.781451+0000","last_became_active":"2023-11-28T03:05:00.424627+0000","last_became_peered":"2023-11-28T03:05:00.424627+0000","last_unstale":"2023-11-28T03:05:00.781451+0000","last_undegraded":"2023-11-28T03:05:00.781451+0000","last_fullsized":"2023-11-28T03:05:00.781451+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.809387+0000","last_change":"2023-11-28T03:05:00.803552+0000","last_active":"2023-11-28T03:05:00.809387+0000","last_peered":"2023-11-28T03:05:00.809387+0000","last_clean":"2023-11-28T03:05:00.809387+0000","last_became_active":"2023-11-28T03:05:00.803295+0000","last_became_peered":"2023-11-28T03:05:00.803295+0000","last_unstale":"2023-11-28T03:05:00.809387+0000","last_undegraded":"2023-11-28T03:05:00.809387+0000","last_fullsized":"2023-11-28T03:05:00.809387+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785601+0000","last_change":"2023-11-28T03:05:00.420449+0000","last_active":"2023-11-28T03:05:00.785601+0000","last_peered":"2023-11-28T03:05:00.785601+0000","last_clean":"2023-11-28T03:05:00.785601+0000","last_became_active":"2023-11-28T03:05:00.420083+0000","last_became_peered":"2023-11-28T03:05:00.420083+0000","last_unstale":"2023-11-28T03:05:00.785601+0000","last_undegraded":"2023-11-28T03:05:00.785601+0000","last_fullsized":"2023-11-28T03:05:00.785601+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.854379+0000","last_change":"2023-11-28T03:05:00.853332+0000","last_active":"2023-11-28T03:05:00.854379+0000","last_peered":"2023-11-28T03:05:00.854379+0000","last_clean":"2023-11-28T03:05:00.854379+0000","last_became_active":"2023-11-28T03:05:00.852934+0000","last_became_peered":"2023-11-28T03:05:00.852934+0000","last_unstale":"2023-11-28T03:05:00.854379+0000","last_undegraded":"2023-11-28T03:05:00.854379+0000","last_fullsized":"2023-11-28T03:05:00.854379+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787447+0000","last_change":"2023-11-28T03:05:00.425379+0000","last_active":"2023-11-28T03:05:00.787447+0000","last_peered":"2023-11-28T03:05:00.787447+0000","last_clean":"2023-11-28T03:05:00.787447+0000","last_became_active":"2023-11-28T03:05:00.424874+0000","last_became_peered":"2023-11-28T03:05:00.424874+0000","last_unstale":"2023-11-28T03:05:00.787447+0000","last_undegraded":"2023-11-28T03:05:00.787447+0000","last_fullsized":"2023-11-28T03:05:00.787447+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797031+0000","last_change":"2023-11-28T03:05:00.793571+0000","last_active":"2023-11-28T03:05:00.797031+0000","last_peered":"2023-11-28T03:05:00.797031+0000","last_clean":"2023-11-28T03:05:00.797031+0000","last_became_active":"2023-11-28T03:05:00.793295+0000","last_became_peered":"2023-11-28T03:05:00.793295+0000","last_unstale":"2023-11-28T03:05:00.797031+0000","last_undegraded":"2023-11-28T03:05:00.797031+0000","last_fullsized":"2023-11-28T03:05:00.797031+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784157+0000","last_change":"2023-11-28T03:05:00.436731+0000","last_active":"2023-11-28T03:05:00.784157+0000","last_peered":"2023-11-28T03:05:00.784157+0000","last_clean":"2023-11-28T03:05:00.784157+0000","last_became_active":"2023-11-28T03:05:00.436450+0000","last_became_peered":"2023-11-28T03:05:00.436450+0000","last_unstale":"2023-11-28T03:05:00.784157+0000","last_undegraded":"2023-11-28T03:05:00.784157+0000","last_fullsized":"2023-11-28T03:05:00.784157+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806630+0000","last_change":"2023-11-28T03:05:00.800644+0000","last_active":"2023-11-28T03:05:00.806630+0000","last_peered":"2023-11-28T03:05:00.806630+0000","last_clean":"2023-11-28T03:05:00.806630+0000","last_became_active":"2023-11-28T03:05:00.800385+0000","last_became_peered":"2023-11-28T03:05:00.800385+0000","last_unstale":"2023-11-28T03:05:00.806630+0000","last_undegraded":"2023-11-28T03:05:00.806630+0000","last_fullsized":"2023-11-28T03:05:00.806630+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.15","version":"64'3","reported_seq":13,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.800698+0000","last_change":"2023-11-28T03:05:00.431729+0000","last_active":"2023-11-28T03:05:00.800698+0000","last_peered":"2023-11-28T03:05:00.800698+0000","last_clean":"2023-11-28T03:05:00.800698+0000","last_became_active":"2023-11-28T03:05:00.431423+0000","last_became_peered":"2023-11-28T03:05:00.431423+0000","last_unstale":"2023-11-28T03:05:00.800698+0000","last_undegraded":"2023-11-28T03:05:00.800698+0000","last_fullsized":"2023-11-28T03:05:00.800698+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794255+0000","last_change":"2023-11-28T03:05:00.792257+0000","last_active":"2023-11-28T03:05:00.794255+0000","last_peered":"2023-11-28T03:05:00.794255+0000","last_clean":"2023-11-28T03:05:00.794255+0000","last_became_active":"2023-11-28T03:05:00.791902+0000","last_became_peered":"2023-11-28T03:05:00.791902+0000","last_unstale":"2023-11-28T03:05:00.794255+0000","last_undegraded":"2023-11-28T03:05:00.794255+0000","last_fullsized":"2023-11-28T03:05:00.794255+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803922+0000","last_change":"2023-11-28T03:05:00.424419+0000","last_active":"2023-11-28T03:05:00.803922+0000","last_peered":"2023-11-28T03:05:00.803922+0000","last_clean":"2023-11-28T03:05:00.803922+0000","last_became_active":"2023-11-28T03:05:00.424147+0000","last_became_peered":"2023-11-28T03:05:00.424147+0000","last_unstale":"2023-11-28T03:05:00.803922+0000","last_undegraded":"2023-11-28T03:05:00.803922+0000","last_fullsized":"2023-11-28T03:05:00.803922+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801857+0000","last_change":"2023-11-28T03:05:00.799216+0000","last_active":"2023-11-28T03:05:00.801857+0000","last_peered":"2023-11-28T03:05:00.801857+0000","last_clean":"2023-11-28T03:05:00.801857+0000","last_became_active":"2023-11-28T03:05:00.798916+0000","last_became_peered":"2023-11-28T03:05:00.798916+0000","last_unstale":"2023-11-28T03:05:00.801857+0000","last_undegraded":"2023-11-28T03:05:00.801857+0000","last_fullsized":"2023-11-28T03:05:00.801857+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.13","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801481+0000","last_change":"2023-11-28T03:05:00.426430+0000","last_active":"2023-11-28T03:05:00.801481+0000","last_peered":"2023-11-28T03:05:00.801481+0000","last_clean":"2023-11-28T03:05:00.801481+0000","last_became_active":"2023-11-28T03:05:00.425465+0000","last_became_peered":"2023-11-28T03:05:00.425465+0000","last_unstale":"2023-11-28T03:05:00.801481+0000","last_undegraded":"2023-11-28T03:05:00.801481+0000","last_fullsized":"2023-11-28T03:05:00.801481+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806594+0000","last_change":"2023-11-28T03:05:00.800001+0000","last_active":"2023-11-28T03:05:00.806594+0000","last_peered":"2023-11-28T03:05:00.806594+0000","last_clean":"2023-11-28T03:05:00.806594+0000","last_became_active":"2023-11-28T03:05:00.799703+0000","last_became_peered":"2023-11-28T03:05:00.799703+0000","last_unstale":"2023-11-28T03:05:00.806594+0000","last_undegraded":"2023-11-28T03:05:00.806594+0000","last_fullsized":"2023-11-28T03:05:00.806594+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781671+0000","last_change":"2023-11-28T03:05:00.436846+0000","last_active":"2023-11-28T03:05:00.781671+0000","last_peered":"2023-11-28T03:05:00.781671+0000","last_clean":"2023-11-28T03:05:00.781671+0000","last_became_active":"2023-11-28T03:05:00.436418+0000","last_became_peered":"2023-11-28T03:05:00.436418+0000","last_unstale":"2023-11-28T03:05:00.781671+0000","last_undegraded":"2023-11-28T03:05:00.781671+0000","last_fullsized":"2023-11-28T03:05:00.781671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795961+0000","last_change":"2023-11-28T03:05:00.793678+0000","last_active":"2023-11-28T03:05:00.795961+0000","last_peered":"2023-11-28T03:05:00.795961+0000","last_clean":"2023-11-28T03:05:00.795961+0000","last_became_active":"2023-11-28T03:05:00.793025+0000","last_became_peered":"2023-11-28T03:05:00.793025+0000","last_unstale":"2023-11-28T03:05:00.795961+0000","last_undegraded":"2023-11-28T03:05:00.795961+0000","last_fullsized":"2023-11-28T03:05:00.795961+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787533+0000","last_change":"2023-11-28T03:05:00.428834+0000","last_active":"2023-11-28T03:05:00.787533+0000","last_peered":"2023-11-28T03:05:00.787533+0000","last_clean":"2023-11-28T03:05:00.787533+0000","last_became_active":"2023-11-28T03:05:00.428487+0000","last_became_peered":"2023-11-28T03:05:00.428487+0000","last_unstale":"2023-11-28T03:05:00.787533+0000","last_undegraded":"2023-11-28T03:05:00.787533+0000","last_fullsized":"2023-11-28T03:05:00.787533+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806674+0000","last_change":"2023-11-28T03:05:00.800007+0000","last_active":"2023-11-28T03:05:00.806674+0000","last_peered":"2023-11-28T03:05:00.806674+0000","last_clean":"2023-11-28T03:05:00.806674+0000","last_became_active":"2023-11-28T03:05:00.799737+0000","last_became_peered":"2023-11-28T03:05:00.799737+0000","last_unstale":"2023-11-28T03:05:00.806674+0000","last_undegraded":"2023-11-28T03:05:00.806674+0000","last_fullsized":"2023-11-28T03:05:00.806674+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"64'4","reported_seq":14,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806127+0000","last_change":"2023-11-28T03:05:00.424398+0000","last_active":"2023-11-28T03:05:00.806127+0000","last_peered":"2023-11-28T03:05:00.806127+0000","last_clean":"2023-11-28T03:05:00.806127+0000","last_became_active":"2023-11-28T03:05:00.424133+0000","last_became_peered":"2023-11-28T03:05:00.424133+0000","last_unstale":"2023-11-28T03:05:00.806127+0000","last_undegraded":"2023-11-28T03:05:00.806127+0000","last_fullsized":"2023-11-28T03:05:00.806127+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807703+0000","last_change":"2023-11-28T03:05:00.804770+0000","last_active":"2023-11-28T03:05:00.807703+0000","last_peered":"2023-11-28T03:05:00.807703+0000","last_clean":"2023-11-28T03:05:00.807703+0000","last_became_active":"2023-11-28T03:05:00.804141+0000","last_became_peered":"2023-11-28T03:05:00.804141+0000","last_unstale":"2023-11-28T03:05:00.807703+0000","last_undegraded":"2023-11-28T03:05:00.807703+0000","last_fullsized":"2023-11-28T03:05:00.807703+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.f","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802987+0000","last_change":"2023-11-28T03:05:00.426975+0000","last_active":"2023-11-28T03:05:00.802987+0000","last_peered":"2023-11-28T03:05:00.802987+0000","last_clean":"2023-11-28T03:05:00.802987+0000","last_became_active":"2023-11-28T03:05:00.426697+0000","last_became_peered":"2023-11-28T03:05:00.426697+0000","last_unstale":"2023-11-28T03:05:00.802987+0000","last_undegraded":"2023-11-28T03:05:00.802987+0000","last_fullsized":"2023-11-28T03:05:00.802987+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.810119+0000","last_change":"2023-11-28T03:05:00.807548+0000","last_active":"2023-11-28T03:05:00.810119+0000","last_peered":"2023-11-28T03:05:00.810119+0000","last_clean":"2023-11-28T03:05:00.810119+0000","last_became_active":"2023-11-28T03:05:00.807224+0000","last_became_peered":"2023-11-28T03:05:00.807224+0000","last_unstale":"2023-11-28T03:05:00.810119+0000","last_undegraded":"2023-11-28T03:05:00.810119+0000","last_fullsized":"2023-11-28T03:05:00.810119+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.e","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802720+0000","last_change":"2023-11-28T03:05:00.419344+0000","last_active":"2023-11-28T03:05:00.802720+0000","last_peered":"2023-11-28T03:05:00.802720+0000","last_clean":"2023-11-28T03:05:00.802720+0000","last_became_active":"2023-11-28T03:05:00.419064+0000","last_became_peered":"2023-11-28T03:05:00.419064+0000","last_unstale":"2023-11-28T03:05:00.802720+0000","last_undegraded":"2023-11-28T03:05:00.802720+0000","last_fullsized":"2023-11-28T03:05:00.802720+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.853572+0000","last_change":"2023-11-28T03:05:00.794660+0000","last_active":"2023-11-28T03:05:00.853572+0000","last_peered":"2023-11-28T03:05:00.853572+0000","last_clean":"2023-11-28T03:05:00.853572+0000","last_became_active":"2023-11-28T03:05:00.794429+0000","last_became_peered":"2023-11-28T03:05:00.794429+0000","last_unstale":"2023-11-28T03:05:00.853572+0000","last_undegraded":"2023-11-28T03:05:00.853572+0000","last_fullsized":"2023-11-28T03:05:00.853572+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782115+0000","last_change":"2023-11-28T03:05:00.431077+0000","last_active":"2023-11-28T03:05:00.782115+0000","last_peered":"2023-11-28T03:05:00.782115+0000","last_clean":"2023-11-28T03:05:00.782115+0000","last_became_active":"2023-11-28T03:05:00.430764+0000","last_became_peered":"2023-11-28T03:05:00.430764+0000","last_unstale":"2023-11-28T03:05:00.782115+0000","last_undegraded":"2023-11-28T03:05:00.782115+0000","last_fullsized":"2023-11-28T03:05:00.782115+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799252+0000","last_change":"2023-11-28T03:05:00.797610+0000","last_active":"2023-11-28T03:05:00.799252+0000","last_peered":"2023-11-28T03:05:00.799252+0000","last_clean":"2023-11-28T03:05:00.799252+0000","last_became_active":"2023-11-28T03:05:00.797306+0000","last_became_peered":"2023-11-28T03:05:00.797306+0000","last_unstale":"2023-11-28T03:05:00.799252+0000","last_undegraded":"2023-11-28T03:05:00.799252+0000","last_fullsized":"2023-11-28T03:05:00.799252+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782946+0000","last_change":"2023-11-28T03:05:00.426981+0000","last_active":"2023-11-28T03:05:00.782946+0000","last_peered":"2023-11-28T03:05:00.782946+0000","last_clean":"2023-11-28T03:05:00.782946+0000","last_became_active":"2023-11-28T03:05:00.426634+0000","last_became_peered":"2023-11-28T03:05:00.426634+0000","last_unstale":"2023-11-28T03:05:00.782946+0000","last_undegraded":"2023-11-28T03:05:00.782946+0000","last_fullsized":"2023-11-28T03:05:00.782946+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799386+0000","last_change":"2023-11-28T03:05:00.795865+0000","last_active":"2023-11-28T03:05:00.799386+0000","last_peered":"2023-11-28T03:05:00.799386+0000","last_clean":"2023-11-28T03:05:00.799386+0000","last_became_active":"2023-11-28T03:05:00.795590+0000","last_became_peered":"2023-11-28T03:05:00.795590+0000","last_unstale":"2023-11-28T03:05:00.799386+0000","last_undegraded":"2023-11-28T03:05:00.799386+0000","last_fullsized":"2023-11-28T03:05:00.799386+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.b","version":"64'1","reported_seq":11,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797332+0000","last_change":"2023-11-28T03:05:00.427931+0000","last_active":"2023-11-28T03:05:00.797332+0000","last_peered":"2023-11-28T03:05:00.797332+0000","last_clean":"2023-11-28T03:05:00.797332+0000","last_became_active":"2023-11-28T03:05:00.427505+0000","last_became_peered":"2023-11-28T03:05:00.427505+0000","last_unstale":"2023-11-28T03:05:00.797332+0000","last_undegraded":"2023-11-28T03:05:00.797332+0000","last_fullsized":"2023-11-28T03:05:00.797332+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.795645+0000","last_change":"2023-11-28T03:05:00.792894+0000","last_active":"2023-11-28T03:05:00.795645+0000","last_peered":"2023-11-28T03:05:00.795645+0000","last_clean":"2023-11-28T03:05:00.795645+0000","last_became_active":"2023-11-28T03:05:00.792612+0000","last_became_peered":"2023-11-28T03:05:00.792612+0000","last_unstale":"2023-11-28T03:05:00.795645+0000","last_undegraded":"2023-11-28T03:05:00.795645+0000","last_fullsized":"2023-11-28T03:05:00.795645+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780367+0000","last_change":"2023-11-28T03:05:00.425301+0000","last_active":"2023-11-28T03:05:00.780367+0000","last_peered":"2023-11-28T03:05:00.780367+0000","last_clean":"2023-11-28T03:05:00.780367+0000","last_became_active":"2023-11-28T03:05:00.424733+0000","last_became_peered":"2023-11-28T03:05:00.424733+0000","last_unstale":"2023-11-28T03:05:00.780367+0000","last_undegraded":"2023-11-28T03:05:00.780367+0000","last_fullsized":"2023-11-28T03:05:00.780367+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806624+0000","last_change":"2023-11-28T03:05:00.801628+0000","last_active":"2023-11-28T03:05:00.806624+0000","last_peered":"2023-11-28T03:05:00.806624+0000","last_clean":"2023-11-28T03:05:00.806624+0000","last_became_active":"2023-11-28T03:05:00.801373+0000","last_became_peered":"2023-11-28T03:05:00.801373+0000","last_unstale":"2023-11-28T03:05:00.806624+0000","last_undegraded":"2023-11-28T03:05:00.806624+0000","last_fullsized":"2023-11-28T03:05:00.806624+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783391+0000","last_change":"2023-11-28T03:05:00.430297+0000","last_active":"2023-11-28T03:05:00.783391+0000","last_peered":"2023-11-28T03:05:00.783391+0000","last_clean":"2023-11-28T03:05:00.783391+0000","last_became_active":"2023-11-28T03:05:00.429995+0000","last_became_peered":"2023-11-28T03:05:00.429995+0000","last_unstale":"2023-11-28T03:05:00.783391+0000","last_undegraded":"2023-11-28T03:05:00.783391+0000","last_fullsized":"2023-11-28T03:05:00.783391+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.797007+0000","last_change":"2023-11-28T03:05:00.795438+0000","last_active":"2023-11-28T03:05:00.797007+0000","last_peered":"2023-11-28T03:05:00.797007+0000","last_clean":"2023-11-28T03:05:00.797007+0000","last_became_active":"2023-11-28T03:05:00.795177+0000","last_became_peered":"2023-11-28T03:05:00.795177+0000","last_unstale":"2023-11-28T03:05:00.797007+0000","last_undegraded":"2023-11-28T03:05:00.797007+0000","last_fullsized":"2023-11-28T03:05:00.797007+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784584+0000","last_change":"2023-11-28T03:05:00.431854+0000","last_active":"2023-11-28T03:05:00.784584+0000","last_peered":"2023-11-28T03:05:00.784584+0000","last_clean":"2023-11-28T03:05:00.784584+0000","last_became_active":"2023-11-28T03:05:00.431452+0000","last_became_peered":"2023-11-28T03:05:00.431452+0000","last_unstale":"2023-11-28T03:05:00.784584+0000","last_undegraded":"2023-11-28T03:05:00.784584+0000","last_fullsized":"2023-11-28T03:05:00.784584+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.806507+0000","last_change":"2023-11-28T03:05:00.800134+0000","last_active":"2023-11-28T03:05:00.806507+0000","last_peered":"2023-11-28T03:05:00.806507+0000","last_clean":"2023-11-28T03:05:00.806507+0000","last_became_active":"2023-11-28T03:05:00.799831+0000","last_became_peered":"2023-11-28T03:05:00.799831+0000","last_unstale":"2023-11-28T03:05:00.806507+0000","last_undegraded":"2023-11-28T03:05:00.806507+0000","last_fullsized":"2023-11-28T03:05:00.806507+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783530+0000","last_change":"2023-11-28T03:05:00.422074+0000","last_active":"2023-11-28T03:05:00.783530+0000","last_peered":"2023-11-28T03:05:00.783530+0000","last_clean":"2023-11-28T03:05:00.783530+0000","last_became_active":"2023-11-28T03:05:00.421818+0000","last_became_peered":"2023-11-28T03:05:00.421818+0000","last_unstale":"2023-11-28T03:05:00.783530+0000","last_undegraded":"2023-11-28T03:05:00.783530+0000","last_fullsized":"2023-11-28T03:05:00.783530+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786429+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:05:00.786429+0000","last_peered":"2023-11-28T03:05:00.786429+0000","last_clean":"2023-11-28T03:05:00.786429+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:05:00.786429+0000","last_undegraded":"2023-11-28T03:05:00.786429+0000","last_fullsized":"2023-11-28T03:05:00.786429+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799800+0000","last_change":"2023-11-28T03:05:00.797902+0000","last_active":"2023-11-28T03:05:00.799800+0000","last_peered":"2023-11-28T03:05:00.799800+0000","last_clean":"2023-11-28T03:05:00.799800+0000","last_became_active":"2023-11-28T03:05:00.797319+0000","last_became_peered":"2023-11-28T03:05:00.797319+0000","last_unstale":"2023-11-28T03:05:00.799800+0000","last_undegraded":"2023-11-28T03:05:00.799800+0000","last_fullsized":"2023-11-28T03:05:00.799800+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.7","version":"64'2","reported_seq":12,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.803283+0000","last_change":"2023-11-28T03:05:00.423249+0000","last_active":"2023-11-28T03:05:00.803283+0000","last_peered":"2023-11-28T03:05:00.803283+0000","last_clean":"2023-11-28T03:05:00.803283+0000","last_became_active":"2023-11-28T03:05:00.422926+0000","last_became_peered":"2023-11-28T03:05:00.422926+0000","last_unstale":"2023-11-28T03:05:00.803283+0000","last_undegraded":"2023-11-28T03:05:00.803283+0000","last_fullsized":"2023-11-28T03:05:00.803283+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.784573+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:05:00.784573+0000","last_peered":"2023-11-28T03:05:00.784573+0000","last_clean":"2023-11-28T03:05:00.784573+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:05:00.784573+0000","last_undegraded":"2023-11-28T03:05:00.784573+0000","last_fullsized":"2023-11-28T03:05:00.784573+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.794685+0000","last_change":"2023-11-28T03:05:00.792511+0000","last_active":"2023-11-28T03:05:00.794685+0000","last_peered":"2023-11-28T03:05:00.794685+0000","last_clean":"2023-11-28T03:05:00.794685+0000","last_became_active":"2023-11-28T03:05:00.792289+0000","last_became_peered":"2023-11-28T03:05:00.792289+0000","last_unstale":"2023-11-28T03:05:00.794685+0000","last_undegraded":"2023-11-28T03:05:00.794685+0000","last_fullsized":"2023-11-28T03:05:00.794685+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.783059+0000","last_change":"2023-11-28T03:05:00.436271+0000","last_active":"2023-11-28T03:05:00.783059+0000","last_peered":"2023-11-28T03:05:00.783059+0000","last_clean":"2023-11-28T03:05:00.783059+0000","last_became_active":"2023-11-28T03:05:00.435974+0000","last_became_peered":"2023-11-28T03:05:00.435974+0000","last_unstale":"2023-11-28T03:05:00.783059+0000","last_undegraded":"2023-11-28T03:05:00.783059+0000","last_fullsized":"2023-11-28T03:05:00.783059+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789838+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:05:00.789838+0000","last_peered":"2023-11-28T03:05:00.789838+0000","last_clean":"2023-11-28T03:05:00.789838+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:05:00.789838+0000","last_undegraded":"2023-11-28T03:05:00.789838+0000","last_fullsized":"2023-11-28T03:05:00.789838+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.799223+0000","last_change":"2023-11-28T03:05:00.797875+0000","last_active":"2023-11-28T03:05:00.799223+0000","last_peered":"2023-11-28T03:05:00.799223+0000","last_clean":"2023-11-28T03:05:00.799223+0000","last_became_active":"2023-11-28T03:05:00.797594+0000","last_became_peered":"2023-11-28T03:05:00.797594+0000","last_unstale":"2023-11-28T03:05:00.799223+0000","last_undegraded":"2023-11-28T03:05:00.799223+0000","last_fullsized":"2023-11-28T03:05:00.799223+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782843+0000","last_change":"2023-11-28T03:05:00.437608+0000","last_active":"2023-11-28T03:05:00.782843+0000","last_peered":"2023-11-28T03:05:00.782843+0000","last_clean":"2023-11-28T03:05:00.782843+0000","last_became_active":"2023-11-28T03:05:00.437267+0000","last_became_peered":"2023-11-28T03:05:00.437267+0000","last_unstale":"2023-11-28T03:05:00.782843+0000","last_undegraded":"2023-11-28T03:05:00.782843+0000","last_fullsized":"2023-11-28T03:05:00.782843+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.785039+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:05:00.785039+0000","last_peered":"2023-11-28T03:05:00.785039+0000","last_clean":"2023-11-28T03:05:00.785039+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:05:00.785039+0000","last_undegraded":"2023-11-28T03:05:00.785039+0000","last_fullsized":"2023-11-28T03:05:00.785039+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":53,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786291+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:05:00.786291+0000","last_peered":"2023-11-28T03:05:00.786291+0000","last_clean":"2023-11-28T03:05:00.786291+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:05:00.786291+0000","last_undegraded":"2023-11-28T03:05:00.786291+0000","last_fullsized":"2023-11-28T03:05:00.786291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.796734+0000","last_change":"2023-11-28T03:05:00.794511+0000","last_active":"2023-11-28T03:05:00.796734+0000","last_peered":"2023-11-28T03:05:00.796734+0000","last_clean":"2023-11-28T03:05:00.796734+0000","last_became_active":"2023-11-28T03:05:00.794228+0000","last_became_peered":"2023-11-28T03:05:00.794228+0000","last_unstale":"2023-11-28T03:05:00.796734+0000","last_undegraded":"2023-11-28T03:05:00.796734+0000","last_fullsized":"2023-11-28T03:05:00.796734+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789844+0000","last_change":"2023-11-28T03:05:00.425225+0000","last_active":"2023-11-28T03:05:00.789844+0000","last_peered":"2023-11-28T03:05:00.789844+0000","last_clean":"2023-11-28T03:05:00.789844+0000","last_became_active":"2023-11-28T03:05:00.424968+0000","last_became_peered":"2023-11-28T03:05:00.424968+0000","last_unstale":"2023-11-28T03:05:00.789844+0000","last_undegraded":"2023-11-28T03:05:00.789844+0000","last_fullsized":"2023-11-28T03:05:00.789844+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.786377+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:05:00.786377+0000","last_peered":"2023-11-28T03:05:00.786377+0000","last_clean":"2023-11-28T03:05:00.786377+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:05:00.786377+0000","last_undegraded":"2023-11-28T03:05:00.786377+0000","last_fullsized":"2023-11-28T03:05:00.786377+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.801882+0000","last_change":"2023-11-28T03:05:00.793949+0000","last_active":"2023-11-28T03:05:00.801882+0000","last_peered":"2023-11-28T03:05:00.801882+0000","last_clean":"2023-11-28T03:05:00.801882+0000","last_became_active":"2023-11-28T03:05:00.793037+0000","last_became_peered":"2023-11-28T03:05:00.793037+0000","last_unstale":"2023-11-28T03:05:00.801882+0000","last_undegraded":"2023-11-28T03:05:00.801882+0000","last_fullsized":"2023-11-28T03:05:00.801882+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.787693+0000","last_change":"2023-11-28T03:05:00.431872+0000","last_active":"2023-11-28T03:05:00.787693+0000","last_peered":"2023-11-28T03:05:00.787693+0000","last_clean":"2023-11-28T03:05:00.787693+0000","last_became_active":"2023-11-28T03:05:00.431547+0000","last_became_peered":"2023-11-28T03:05:00.431547+0000","last_unstale":"2023-11-28T03:05:00.787693+0000","last_undegraded":"2023-11-28T03:05:00.787693+0000","last_fullsized":"2023-11-28T03:05:00.787693+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":51,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.789916+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:05:00.789916+0000","last_peered":"2023-11-28T03:05:00.789916+0000","last_clean":"2023-11-28T03:05:00.789916+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:05:00.789916+0000","last_undegraded":"2023-11-28T03:05:00.789916+0000","last_fullsized":"2023-11-28T03:05:00.789916+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.807908+0000","last_change":"2023-11-28T03:05:00.805547+0000","last_active":"2023-11-28T03:05:00.807908+0000","last_peered":"2023-11-28T03:05:00.807908+0000","last_clean":"2023-11-28T03:05:00.807908+0000","last_became_active":"2023-11-28T03:05:00.805325+0000","last_became_peered":"2023-11-28T03:05:00.805325+0000","last_unstale":"2023-11-28T03:05:00.807908+0000","last_undegraded":"2023-11-28T03:05:00.807908+0000","last_fullsized":"2023-11-28T03:05:00.807908+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.781035+0000","last_change":"2023-11-28T03:05:00.426376+0000","last_active":"2023-11-28T03:05:00.781035+0000","last_peered":"2023-11-28T03:05:00.781035+0000","last_clean":"2023-11-28T03:05:00.781035+0000","last_became_active":"2023-11-28T03:05:00.426131+0000","last_became_peered":"2023-11-28T03:05:00.426131+0000","last_unstale":"2023-11-28T03:05:00.781035+0000","last_undegraded":"2023-11-28T03:05:00.781035+0000","last_fullsized":"2023-11-28T03:05:00.781035+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":50,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.788011+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:05:00.788011+0000","last_peered":"2023-11-28T03:05:00.788011+0000","last_clean":"2023-11-28T03:05:00.788011+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:05:00.788011+0000","last_undegraded":"2023-11-28T03:05:00.788011+0000","last_fullsized":"2023-11-28T03:05:00.788011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":9,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.802290+0000","last_change":"2023-11-28T03:05:00.800651+0000","last_active":"2023-11-28T03:05:00.802290+0000","last_peered":"2023-11-28T03:05:00.802290+0000","last_clean":"2023-11-28T03:05:00.802290+0000","last_became_active":"2023-11-28T03:05:00.800387+0000","last_became_peered":"2023-11-28T03:05:00.800387+0000","last_unstale":"2023-11-28T03:05:00.802290+0000","last_undegraded":"2023-11-28T03:05:00.802290+0000","last_fullsized":"2023-11-28T03:05:00.802290+0000","mapping_epoch":63,"log_start":"0'0","ondisk_log_start":"0'0","created":63,"last_epoch_clean":64,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:05:00.400291+0000","last_clean_scrub_stamp":"2023-11-28T03:05:00.400291+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.780548+0000","last_change":"2023-11-28T03:05:00.431674+0000","last_active":"2023-11-28T03:05:00.780548+0000","last_peered":"2023-11-28T03:05:00.780548+0000","last_clean":"2023-11-28T03:05:00.780548+0000","last_became_active":"2023-11-28T03:05:00.431396+0000","last_became_peered":"2023-11-28T03:05:00.431396+0000","last_unstale":"2023-11-28T03:05:00.780548+0000","last_undegraded":"2023-11-28T03:05:00.780548+0000","last_fullsized":"2023-11-28T03:05:00.780548+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:04:59.395780+0000","last_clean_scrub_stamp":"2023-11-28T03:04:59.395780+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":64,"state":"active+clean","last_fresh":"2023-11-28T03:05:00.782094+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:05:00.782094+0000","last_peered":"2023-11-28T03:05:00.782094+0000","last_clean":"2023-11-28T03:05:00.782094+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:05:00.782094+0000","last_undegraded":"2023-11-28T03:05:00.782094+0000","last_fullsized":"2023-11-28T03:05:00.782094+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.33800000000000002,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.71299999999999997,"5min":0.82399999999999995,"15min":0.85999999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.60899999999999999,"15min":0.59599999999999997},"min":{"1min":0.46300000000000002,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.72399999999999998,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.72199999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.54400000000000004,"15min":0.54400000000000004},"min":{"1min":0.41899999999999998,"5min":0.36599999999999999,"15min":0.32000000000000001},"max":{"1min":0.66400000000000003,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.501,"5min":0.56299999999999994,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.73699999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.68300000000000005}]},{"osd":2,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.378,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.73299999999999998,"5min":0.749,"15min":0.749},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56499999999999995,"15min":0.55900000000000005},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.69399999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.36599999999999999},"max":{"1min":0.70299999999999996,"5min":0.749,"15min":0.77100000000000002},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.57699999999999996,"15min":0.57399999999999995},"min":{"1min":0.41199999999999998,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75800000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.77100000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.46400000000000002,"15min":0.46800000000000003},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68799999999999994,"5min":0.77800000000000002,"15min":0.83099999999999996},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.47599999999999998,"15min":0.49099999999999999},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.81899999999999995},"last":0.55400000000000005}]},{"osd":5,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.49099999999999999,"15min":0.50800000000000001},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.313},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.84499999999999997},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.77300000000000002,"5min":0.83099999999999996,"15min":0.83199999999999996},"last":0.58199999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:05:00 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.38200000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.79800000000000004,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55100000000000005,"15min":0.55000000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.28999999999999998},"max":{"1min":0.749,"5min":0.77800000000000002,"15min":0.84499999999999997},"last":0.66500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55300000000000005,"15min":0.53100000000000003},"min":{"1min":0.35699999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":1.4810000000000001,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.39000000000000001,"15min":0.33400000000000002},"max":{"1min":0.69899999999999995,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.443}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.54900000000000004},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.53900000000000003},"min":{"1min":0.38,"5min":0.379,"15min":0.32800000000000001},"max":{"1min":1.506,"5min":1.506,"15min":1.506},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.57099999999999995,"15min":0.55500000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.498},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58799999999999997,"15min":0.55700000000000005},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47599999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.58599999999999997,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.57299999999999995,"15min":0.56699999999999995},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.76600000000000001,"5min":0.81000000000000005,"15min":0.84399999999999997},"last":0.52600000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.46600000000000003,"15min":0.45200000000000001},"min":{"1min":0.32000000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":1.7310000000000001,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.497,"5min":0.48399999999999999,"15min":0.5},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.745,"5min":0.746,"15min":0.82499999999999996},"last":0.33100000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.505,"15min":0.51700000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.55900000000000005,"15min":0.54600000000000004},"min":{"1min":0.35899999999999999,"5min":0.33600000000000002,"15min":0.311},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.39500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54600000000000004},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.317},"max":{"1min":1.3799999999999999,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.57699999999999996,"15min":0.55600000000000005},"min":{"1min":0.40200000000000002,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":1.1040000000000001,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.42399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574999,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":379882,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.28799999999999998,"5min":0.34699999999999998,"15min":0.33500000000000002},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.38600000000000001,"5min":0.56599999999999995,"15min":0.70699999999999996},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.33400000000000002,"5min":0.38500000000000001,"15min":0.36299999999999999},"min":{"1min":0.20899999999999999,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.52700000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.245}]},{"osd":2,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.38600000000000001,"15min":0.373},"min":{"1min":0.21199999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.43099999999999999,"5min":0.56200000000000006,"15min":0.73499999999999999},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.41499999999999998,"15min":0.39100000000000001},"min":{"1min":0.29399999999999998,"5min":0.27600000000000002,"15min":0.19800000000000001},"max":{"1min":0.54900000000000004,"5min":0.63400000000000001,"15min":0.72599999999999998},"last":0.308}]},{"osd":3,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.441},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.52800000000000002,"5min":0.66300000000000003,"15min":0.749},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.376,"5min":0.44,"15min":0.42999999999999999},"min":{"1min":0.29399999999999998,"5min":0.28799999999999998,"15min":0.26000000000000001},"max":{"1min":0.54400000000000004,"5min":0.60899999999999999,"15min":1.6870000000000001},"last":0.68799999999999994}]},{"osd":4,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.60299999999999998,"15min":0.58799999999999997},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":0.76900000000000002,"5min":0.83899999999999997,"15min":1.0900000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.62,"15min":0.59399999999999997},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.84699999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.38700000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65400000000000003,"15min":0.64200000000000002},"min":{"1min":0.38,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81399999999999995,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.373},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.66500000000000004,"15min":0.64700000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.83299999999999996,"5min":1.038,"15min":1.111},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.61299999999999999,"15min":0.60899999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.84199999999999997,"15min":1.8080000000000001},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63600000000000001,"15min":0.61499999999999999},"min":{"1min":0.45500000000000002,"5min":0.42699999999999999,"15min":0.40600000000000003},"max":{"1min":0.83699999999999997,"5min":0.93500000000000005,"15min":1.073},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:05:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.63900000000000001,"15min":0.61799999999999999},"min":{"1min":0.42699999999999999,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.81499999999999995,"5min":1.024,"15min":1.024},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63500000000000001,"15min":0.63200000000000001},"min":{"1min":0.435,"5min":0.435,"15min":0.38700000000000001},"max":{"1min":0.80000000000000004,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.433}]}]},{"osd":0,"up_from":13,"seq":55834575000,"num_pgs":49,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297864,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073976,"statfs":{"total":96636764160,"available":96331751424,"internally_reserved":0,"allocated":786432,"data_stored":379918,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.35999999999999999,"15min":0.34100000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.184},"max":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.38900000000000001,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.27500000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.378,"15min":0.36699999999999999},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.52600000000000002,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.249},{"interface":"front","average":{"1min":0.42899999999999999,"5min":0.45900000000000002,"15min":0.436},"min":{"1min":0.307,"5min":0.27100000000000002,"15min":0.26300000000000001},"max":{"1min":0.64200000000000002,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.40600000000000003,"15min":0.38900000000000001},"min":{"1min":0.29299999999999998,"5min":0.28399999999999997,"15min":0.24099999999999999},"max":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.42799999999999999,"15min":0.41299999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.621,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.311}]},{"osd":4,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.63900000000000001,"15min":0.62},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.90000000000000002,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.66200000000000003,"15min":0.63400000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.69799999999999995}]},{"osd":5,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63200000000000001},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.88500000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.65100000000000002,"15min":0.63400000000000001},"min":{"1min":0.44900000000000001,"5min":0.443,"15min":0.39000000000000001},"max":{"1min":0.90100000000000002,"5min":0.94499999999999995,"15min":0.97299999999999998},"last":0.67000000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.627,"15min":0.60299999999999998},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":1.679,"5min":1.679,"15min":1.679},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.625,"5min":0.63800000000000001,"15min":0.61699999999999999},"min":{"1min":0.439,"5min":0.39000000000000001,"15min":0.35899999999999999},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:04:59 2023","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.65700000000000003,"15min":0.629},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.36299999999999999},"max":{"1min":0.89700000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.69399999999999995,"15min":0.66400000000000003},"min":{"1min":0.48099999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97399999999999998,"5min":1.046,"15min":1.046},"last":0.57699999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575001,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":381328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.32900000000000001,"5min":0.374,"15min":0.36099999999999999},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.20499999999999999},"max":{"1min":0.41099999999999998,"5min":0.57399999999999995,"15min":0.623},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.42199999999999999,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.23599999999999999},"max":{"1min":0.47599999999999998,"5min":0.64700000000000002,"15min":0.65900000000000003},"last":0.35699999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.311,"5min":0.374,"15min":0.35399999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.42899999999999999,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.40400000000000003,"15min":0.377},"min":{"1min":0.218,"5min":0.218,"15min":0.218},"max":{"1min":0.51100000000000001,"5min":0.68600000000000005,"15min":0.73099999999999998},"last":0.308}]},{"osd":3,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.432,"15min":0.40600000000000003},"min":{"1min":0.19800000000000001,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.53600000000000003,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.378},{"interface":"front","average":{"1min":0.35199999999999998,"5min":0.40100000000000002,"15min":0.38900000000000001},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.23999999999999999},"max":{"1min":0.46400000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.59099999999999997,"15min":0.57799999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.90200000000000002,"15min":0.94599999999999995},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.60399999999999998,"15min":0.59199999999999997},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.39800000000000002},"max":{"1min":0.77100000000000002,"5min":0.874,"15min":0.874},"last":0.60399999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63700000000000001,"15min":0.61799999999999999},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.36699999999999999},"max":{"1min":0.75900000000000001,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.81499999999999995,"5min":1.004,"15min":1.004},"last":0.69199999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.628,"15min":0.61799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.74399999999999999,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.66100000000000003,"15min":0.63900000000000001},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.74099999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:04:52 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.63700000000000001,"15min":0.61599999999999999},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.36899999999999999},"max":{"1min":0.73699999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.64000000000000001,"15min":0.61899999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.71199999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575000,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301960,"kb_used_data":768,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069880,"statfs":{"total":96636764160,"available":96327557120,"internally_reserved":0,"allocated":786432,"data_stored":380458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32600000000000001,"5min":0.36799999999999999,"15min":0.34699999999999998},"min":{"1min":0.24099999999999999,"5min":0.22800000000000001,"15min":0.21199999999999999},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.33200000000000002,"5min":0.38900000000000001,"15min":0.36399999999999999},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.501,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35099999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.38900000000000001,"15min":0.36099999999999999},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.19900000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.60899999999999999},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.42599999999999999,"15min":0.40699999999999997},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.24299999999999999},"max":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.40300000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.42899999999999999,"15min":0.41099999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.436},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.44800000000000001,"15min":0.42599999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.65700000000000003,"5min":0.66100000000000003,"15min":0.68200000000000005},"last":0.45300000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.61699999999999999,"15min":0.625},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.35199999999999998},"max":{"1min":1.581,"5min":1.581,"15min":1.6000000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.621,"15min":0.62},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70699999999999996,"5min":0.94199999999999995,"15min":1.196},"last":0.64800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.76100000000000001,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.80100000000000005,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.53500000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.64700000000000002,"15min":0.64100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.35699999999999998},"max":{"1min":0.66200000000000003,"5min":0.996,"15min":1.206},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.65100000000000002,"15min":0.64500000000000002},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.378},"max":{"1min":0.83599999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.61399999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:04:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.65600000000000003,"15min":0.65500000000000003},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.88,"5min":0.97199999999999998,"15min":1.1559999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.65600000000000003,"15min":0.65200000000000002},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.85699999999999998,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.59199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574999,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.55100000000000005,"15min":0.53300000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.27800000000000002},"max":{"1min":0.63700000000000001,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.57099999999999995,"15min":0.54100000000000004},"min":{"1min":0.39400000000000002,"5min":0.372,"15min":0.27300000000000002},"max":{"1min":0.69699999999999995,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.60999999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.66200000000000003,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.56699999999999995,"15min":0.55000000000000004},"min":{"1min":0.374,"5min":0.374,"15min":0.372},"max":{"1min":0.68400000000000005,"5min":0.80100000000000005,"15min":0.81200000000000006},"last":0.66600000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56599999999999995,"15min":0.55500000000000005},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.64900000000000002,"5min":0.79700000000000004,"15min":0.82199999999999995},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56899999999999995,"15min":0.54300000000000004},"min":{"1min":0.41099999999999998,"5min":0.38,"15min":0.33500000000000002},"max":{"1min":0.69799999999999995,"5min":0.76300000000000001,"15min":0.78100000000000003},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.45700000000000002,"5min":0.435,"15min":0.39000000000000001},"max":{"1min":0.70199999999999996,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.63300000000000001,"15min":0.61499999999999999},"min":{"1min":0.48999999999999999,"5min":0.47999999999999998,"15min":0.38500000000000001},"max":{"1min":0.69699999999999995,"5min":0.878,"15min":0.878},"last":0.65500000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.47099999999999997,"15min":0.47299999999999998},"min":{"1min":0.34200000000000003,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.72699999999999998,"5min":0.72699999999999998,"15min":0.72699999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51500000000000001,"15min":0.51600000000000001},"min":{"1min":0.35299999999999998,"5min":0.32700000000000001,"15min":0.31},"max":{"1min":0.69099999999999995,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58099999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.50800000000000001,"15min":0.50700000000000001},"min":{"1min":0.34200000000000003,"5min":0.315,"15min":0.315},"max":{"1min":0.67500000000000004,"5min":0.83899999999999997,"15min":0.83899999999999997},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.53300000000000003,"15min":0.52800000000000002},"min":{"1min":0.40500000000000003,"5min":0.36699999999999999,"15min":0.33600000000000002},"max":{"1min":0.753,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.46800000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:05:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.54300000000000004},"min":{"1min":0.39400000000000002,"5min":0.38600000000000001,"15min":0.32900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.32900000000000001},"max":{"1min":0.72799999999999998,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.49199999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301952,"kb_used_data":760,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069888,"statfs":{"total":96636764160,"available":96327565312,"internally_reserved":0,"allocated":778240,"data_stored":380684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.51800000000000002,"15min":0.51600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63900000000000001,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.56599999999999995,"15min":0.55400000000000005},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.73299999999999998,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.58199999999999996,"15min":0.56999999999999995},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.74199999999999999,"5min":0.81200000000000006,"15min":0.82699999999999996},"last":0.67200000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.59699999999999998,"15min":0.57999999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.35099999999999998},"max":{"1min":0.69199999999999995,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59699999999999998,"15min":0.59999999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.63300000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.379,"5min":0.377,"15min":0.34599999999999997},"max":{"1min":0.70299999999999996,"5min":0.76400000000000001,"15min":0.79800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.56200000000000006,"15min":0.56799999999999995},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.73699999999999999,"5min":0.75700000000000001,"15min":0.78700000000000003},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.42599999999999999,"15min":0.45900000000000002},"min":{"1min":0.28999999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52600000000000002,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.33500000000000002,"15min":0.32400000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.44800000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47899999999999998,"15min":0.51400000000000001},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.53600000000000003,"15min":0.57199999999999995},"min":{"1min":0.38900000000000001,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.434}]},{"osd":7,"last update":"Tue Nov 28 03:04:57 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.49299999999999999,"15min":0.52200000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51200000000000001,"15min":0.55700000000000005},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.41899999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:05:09.144 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:05:09.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:05:09.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.299+0000 7fd516d17700 1 Processor -- start 2023-11-28T03:05:09.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.299+0000 7fd516d17700 1 -- start start 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd516d17700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd510098d10 0x7fd5100c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd516d17700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd5100c6190 0x7fd510063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd516d17700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5100c72d0 con 0x7fd510064350 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd516d17700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd510154bc0 con 0x7fd510098d10 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd516d17700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5100c66c0 con 0x7fd5100c6190 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd510098d10 0x7fd5100c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd5152b4700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd510064350 0x7fd510158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57428/0 (socket says 172.21.15.73:57428) 2023-11-28T03:05:09.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd5152b4700 1 -- 172.21.15.73:0/3567565655 learned_addr learned my addr 172.21.15.73:0/3567565655 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd50ffff700 1 --2- 172.21.15.73:0/3567565655 >> v2:172.21.15.150:3300/0 conn(0x7fd5100c6190 0x7fd510063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 -- 172.21.15.73:0/3567565655 >> v2:172.21.15.150:3300/0 conn(0x7fd5100c6190 msgr2=0x7fd510063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 --2- 172.21.15.73:0/3567565655 >> v2:172.21.15.150:3300/0 conn(0x7fd5100c6190 0x7fd510063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd50f7fe700 1 -- 172.21.15.73:0/3567565655 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3140126409 0 0) 0x7fd5100c72d0 con 0x7fd510064350 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 -- 172.21.15.73:0/3567565655 >> v1:172.21.15.73:6789/0 conn(0x7fd510064350 legacy=0x7fd510158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 -- 172.21.15.73:0/3567565655 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5100c69a0 con 0x7fd510098d10 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd514ab3700 1 --2- 172.21.15.73:0/3567565655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd5100c7760 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fd5100c5b40 tx=0x7fd500009780).ready entity=mon.0 client_cookie=be89ba2a8e16a211 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.300+0000 7fd50f7fe700 1 -- 172.21.15.73:0/3567565655 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50000e070 con 0x7fd510098d10 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd50f7fe700 1 -- 172.21.15.73:0/3567565655 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50000bc30 con 0x7fd510098d10 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd50f7fe700 1 -- 172.21.15.73:0/3567565655 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd500012410 con 0x7fd510098d10 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 -- 172.21.15.73:0/3567565655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 msgr2=0x7fd5100c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 --2- 172.21.15.73:0/3567565655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd5100c7760 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 rx=0x7fd5100c5b40 tx=0x7fd500009780).stop 2023-11-28T03:05:09.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 -- 172.21.15.73:0/3567565655 shutdown_connections 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 --2- 172.21.15.73:0/3567565655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd5100c7760 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 --2- 172.21.15.73:0/3567565655 >> v2:172.21.15.150:3300/0 conn(0x7fd5100c6190 0x7fd510063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 -- 172.21.15.73:0/3567565655 >> 172.21.15.73:0/3567565655 conn(0x7fd5100cb310 msgr2=0x7fd5100cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 -- 172.21.15.73:0/3567565655 shutdown_connections 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.301+0000 7fd516d17700 1 -- 172.21.15.73:0/3567565655 wait complete. 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.302+0000 7fd516d17700 1 Processor -- start 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.302+0000 7fd516d17700 1 -- start start 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd510064350 0x7fd5100a9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5100c6190 0x7fd51014b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100ad310 con 0x7fd510098d10 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100ad030 con 0x7fd510064350 2023-11-28T03:05:09.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd516d17700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5100a9500 con 0x7fd5100c6190 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd50ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54546/0 (socket says 172.21.15.73:54546) 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd50ffff700 1 -- 172.21.15.73:0/3815241337 learned_addr learned my addr 172.21.15.73:0/3815241337 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd5152b4700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5100c6190 0x7fd51014b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd514ab3700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd510064350 0x7fd5100a9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.303+0000 7fd50ffff700 1 -- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5100c6190 msgr2=0x7fd51014b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50ffff700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5100c6190 0x7fd51014b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50ffff700 1 -- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd510064350 msgr2=0x7fd5100a9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50ffff700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd510064350 0x7fd5100a9fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50ffff700 1 -- 172.21.15.73:0/3815241337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd500009a20 con 0x7fd510098d10 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50ffff700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fd50400d890 tx=0x7fd504008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400f520 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd50400f680 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5100a97e0 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd50400fa00 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.304+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5101579f0 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.305+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5100aa4d0 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.305+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd504013480 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.305+0000 7fd50cff9700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 0x7fd4fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.305+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd504079c80 con 0x7fd510098d10 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.305+0000 7fd514ab3700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 0x7fd4fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.306+0000 7fd514ab3700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 0x7fd4fc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd500009fd0 tx=0x7fd5100a8040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.309+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd50404aef0 con 0x7fd510098d10 2023-11-28T03:05:09.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.536+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fd51006c860 con 0x7fd510098d10 2023-11-28T03:05:09.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.540+0000 7fd50cff9700 1 -- 172.21.15.73:0/3815241337 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v12) v1 ==== 190+0+0 (secure 0 0 0) 0x7fd50404e510 con 0x7fd510098d10 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 msgr2=0x7fd4fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 0x7fd4fc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7fd500009fd0 tx=0x7fd5100a8040).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 msgr2=0x7fd510157500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 rx=0x7fd50400d890 tx=0x7fd504008040).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 shutdown_connections 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4fc05f150 0x7fd4fc061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd5100c6190 0x7fd51014b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd510064350 0x7fd5100a9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 --2- 172.21.15.73:0/3815241337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd510098d10 0x7fd510157500 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.541+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 >> 172.21.15.73:0/3815241337 conn(0x7fd5100cb310 msgr2=0x7fd51014c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.542+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 shutdown_connections 2023-11-28T03:05:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.542+0000 7fd516d17700 1 -- 172.21.15.73:0/3815241337 wait complete. 2023-11-28T03:05:09.544 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:05:09.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:05:09.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.666+0000 7faa998b7700 1 Processor -- start 2023-11-28T03:05:09.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.666+0000 7faa998b7700 1 -- start start 2023-11-28T03:05:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa998b7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa941542d0 0x7faa940697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa998b7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa940661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa998b7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa94065760 con 0x7faa94069ce0 2023-11-28T03:05:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa998b7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa940c3250 con 0x7faa941542d0 2023-11-28T03:05:09.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa998b7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406a340 con 0x7faa94062e10 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa927fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faa94069ce0 0x7faa94062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57442/0 (socket says 172.21.15.73:57442) 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.667+0000 7faa927fc700 1 -- 172.21.15.73:0/2746552628 learned_addr learned my addr 172.21.15.73:0/2746552628 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa92ffd700 1 --2- 172.21.15.73:0/2746552628 >> v2:172.21.15.150:3300/0 conn(0x7faa941542d0 0x7faa940697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa937fe700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa940661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213658837 0 0) 0x7faa94065760 con 0x7faa94069ce0 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faa7c0033f0 con 0x7faa94069ce0 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304421607 0 0) 0x7faa7c0033f0 con 0x7faa94069ce0 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 msgr2=0x7faa940661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa940661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 >> v2:172.21.15.150:3300/0 conn(0x7faa941542d0 msgr2=0x7faa940697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 --2- 172.21.15.73:0/2746552628 >> v2:172.21.15.150:3300/0 conn(0x7faa941542d0 0x7faa940697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa9406a620 con 0x7faa94069ce0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7faa88001470 con 0x7faa94069ce0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 >> v1:172.21.15.73:6789/0 conn(0x7faa94069ce0 legacy=0x7faa94062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c0038e0 0x7faa7c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa7c004690 0x7faa7c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa7c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa940c3250 con 0x7faa7c004690 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9406a340 con 0x7faa7c0038e0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa7c00b8e0 con 0x7faa94062e10 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa7c004690 0x7faa7c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa92ffd700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c0038e0 0x7faa7c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa937fe700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa7c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 -- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 msgr2=0x7faa7c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa7c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 -- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c0038e0 msgr2=0x7faa7c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c0038e0 0x7faa7c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa927fc700 1 -- 172.21.15.73:0/2746552628 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa7c0033f0 con 0x7faa7c004690 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.668+0000 7faa937fe700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa94062e10 0x7faa7c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa927fc700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa7c004690 0x7faa7c004b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7faa8800c210 tx=0x7faa88001e90).ready entity=mon.0 client_cookie=c4b395f6c1a9d702 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88007a60 con 0x7faa7c004690 2023-11-28T03:05:09.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa8800cae0 con 0x7faa7c004690 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa91ffb700 1 -- 172.21.15.73:0/2746552628 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa8800ce60 con 0x7faa7c004690 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 -- 172.21.15.73:0/2746552628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa7c004690 msgr2=0x7faa7c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 --2- 172.21.15.73:0/2746552628 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa7c004690 0x7faa7c004b10 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 rx=0x7faa8800c210 tx=0x7faa88001e90).stop 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa937fe700 1 -- 172.21.15.73:0/2746552628 reap_dead start 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 -- 172.21.15.73:0/2746552628 shutdown_connections 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 -- 172.21.15.73:0/2746552628 >> 172.21.15.73:0/2746552628 conn(0x7faa940c7fd0 msgr2=0x7faa940c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 -- 172.21.15.73:0/2746552628 shutdown_connections 2023-11-28T03:05:09.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.669+0000 7faa998b7700 1 -- 172.21.15.73:0/2746552628 wait complete. 2023-11-28T03:05:09.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.670+0000 7faa998b7700 1 Processor -- start 2023-11-28T03:05:09.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.670+0000 7faa998b7700 1 -- start start 2023-11-28T03:05:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c004690 0x7faa9414bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa9414e360 0x7faa9414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa940bf070 con 0x7faa940aa210 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa9414ecd0 con 0x7faa7c004690 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa998b7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa940a8d60 con 0x7faa9414e360 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54556/0 (socket says 172.21.15.73:54556) 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 -- 172.21.15.73:0/1920590394 learned_addr learned my addr 172.21.15.73:0/1920590394 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa937fe700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa9414e360 0x7faa9414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa92ffd700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c004690 0x7faa9414bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 -- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa9414e360 msgr2=0x7faa9414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa9414e360 0x7faa9414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 -- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c004690 msgr2=0x7faa9414bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.671+0000 7faa927fc700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c004690 0x7faa9414bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa927fc700 1 -- 172.21.15.73:0/1920590394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa940bf2c0 con 0x7faa940aa210 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa927fc700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7faa7c0044b0 tx=0x7faa88015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88007640 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa880046b0 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa940bf5a0 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa88004a70 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.672+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa940bfb60 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.673+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa9406c860 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.673+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faa88020450 con 0x7faa940aa210 2023-11-28T03:05:09.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.673+0000 7faa7affd700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 0x7faa80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.674+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faa8807aff0 con 0x7faa940aa210 2023-11-28T03:05:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.674+0000 7faa92ffd700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 0x7faa80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.674+0000 7faa92ffd700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 0x7faa80061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa840094d0 tx=0x7faa8400b040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:09.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.678+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faa880a9af0 con 0x7faa940aa210 2023-11-28T03:05:09.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.881+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7faa940648d0 con 0x7faa940aa210 2023-11-28T03:05:09.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.882+0000 7faa7affd700 1 -- 172.21.15.73:0/1920590394 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7faa88025020 con 0x7faa940aa210 2023-11-28T03:05:09.883 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:09.883 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:05:09.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 msgr2=0x7faa80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 0x7faa80061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa840094d0 tx=0x7faa8400b040).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 msgr2=0x7faa940aa690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 rx=0x7faa7c0044b0 tx=0x7faa88015000).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 shutdown_connections 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa8005f150 0x7faa80061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa9414e360 0x7faa9414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa7c004690 0x7faa9414bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 --2- 172.21.15.73:0/1920590394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa940aa210 0x7faa940aa690 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 >> 172.21.15.73:0/1920590394 conn(0x7faa940c7fd0 msgr2=0x7faa940c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 shutdown_connections 2023-11-28T03:05:09.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:09.884+0000 7faa998b7700 1 -- 172.21.15.73:0/1920590394 wait complete. 2023-11-28T03:05:09.887 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:05:10.299 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:05:00.776599+0000', 'modified': '2023-11-28T03:05:03.427098+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5775}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5775': {'gid': 5775, 'name': 'a', 'rank': 0, 'incarnation': 17, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/3689972589', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6836', 'nonce': 3689972589}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3689972589}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [8], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:05:10.300 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:05:10.300 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T03:05:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.413+0000 7f1eff15b700 1 Processor -- start 2023-11-28T03:05:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.413+0000 7f1eff15b700 1 -- start start 2023-11-28T03:05:10.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1eff15b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ef8098d10 0x7f1ef80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1eff15b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1ef80c6190 0x7f1ef8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1eff15b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ef80c72d0 con 0x7f1ef8064350 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1eff15b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef8154bc0 con 0x7f1ef8098d10 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1eff15b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ef80c66c0 con 0x7f1ef80c6190 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1ef8098d10 0x7f1ef80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efd6f8700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ef8064350 0x7f1ef8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57454/0 (socket says 172.21.15.73:57454) 2023-11-28T03:05:10.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efd6f8700 1 -- 172.21.15.73:0/59074316 learned_addr learned my addr 172.21.15.73:0/59074316 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/59074316 >> v2:172.21.15.150:3300/0 conn(0x7f1ef80c6190 0x7f1ef8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 -- 172.21.15.73:0/59074316 >> v2:172.21.15.150:3300/0 conn(0x7f1ef80c6190 msgr2=0x7f1ef8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 --2- 172.21.15.73:0/59074316 >> v2:172.21.15.150:3300/0 conn(0x7f1ef80c6190 0x7f1ef8063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 -- 172.21.15.73:0/59074316 >> v1:172.21.15.73:6789/0 conn(0x7f1ef8064350 legacy=0x7f1ef8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 -- 172.21.15.73:0/59074316 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ef80c69a0 con 0x7f1ef8098d10 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.414+0000 7f1efcef7700 1 --2- 172.21.15.73:0/59074316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef8098d10 0x7f1ef80c7760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f1ef80c5b40 tx=0x7f1ee8009780).ready entity=mon.0 client_cookie=f6f6fa02c5579d54 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1ef77fe700 1 -- 172.21.15.73:0/59074316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee800e070 con 0x7f1ef8098d10 2023-11-28T03:05:10.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1ef77fe700 1 -- 172.21.15.73:0/59074316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee800bc30 con 0x7f1ef8098d10 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1ef77fe700 1 -- 172.21.15.73:0/59074316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee8012410 con 0x7f1ef8098d10 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 -- 172.21.15.73:0/59074316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef8098d10 msgr2=0x7f1ef80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 --2- 172.21.15.73:0/59074316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef8098d10 0x7f1ef80c7760 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 rx=0x7f1ef80c5b40 tx=0x7f1ee8009780).stop 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 -- 172.21.15.73:0/59074316 shutdown_connections 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 --2- 172.21.15.73:0/59074316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef8098d10 0x7f1ef80c7760 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 --2- 172.21.15.73:0/59074316 >> v2:172.21.15.150:3300/0 conn(0x7f1ef80c6190 0x7f1ef8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 -- 172.21.15.73:0/59074316 >> 172.21.15.73:0/59074316 conn(0x7f1ef80cb310 msgr2=0x7f1ef80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 -- 172.21.15.73:0/59074316 shutdown_connections 2023-11-28T03:05:10.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.415+0000 7f1eff15b700 1 -- 172.21.15.73:0/59074316 wait complete. 2023-11-28T03:05:10.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.416+0000 7f1eff15b700 1 Processor -- start 2023-11-28T03:05:10.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.416+0000 7f1eff15b700 1 -- start start 2023-11-28T03:05:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef8064350 0x7f1ef80ab230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef8098d10 0x7f1ef80aa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef814a7a0 con 0x7f1ef80c6190 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef80aa9f0 con 0x7f1ef8098d10 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1eff15b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ef8155c90 con 0x7f1ef8064350 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1efd6f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1efd6f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54572/0 (socket says 172.21.15.73:54572) 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1efd6f8700 1 -- 172.21.15.73:0/4286713864 learned_addr learned my addr 172.21.15.73:0/4286713864 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef8098d10 0x7f1ef80aa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.417+0000 7f1efcef7700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef8064350 0x7f1ef80ab230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 -- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef8064350 msgr2=0x7f1ef80ab230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef8064350 0x7f1ef80ab230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 -- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef8098d10 msgr2=0x7f1ef80aa010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef8098d10 0x7f1ef80aa010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 -- 172.21.15.73:0/4286713864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee8009a20 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f1ef000c410 tx=0x7f1ef0007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1ef4ff9700 1 -- 172.21.15.73:0/4286713864 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00072e0 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1ef4ff9700 1 -- 172.21.15.73:0/4286713864 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef0007440 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1ef4ff9700 1 -- 172.21.15.73:0/4286713864 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef00057a0 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ef814a0e0 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.418+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ef80ad4c0 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.419+0000 7f1ef4ff9700 1 -- 172.21.15.73:0/4286713864 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1ef0005b50 con 0x7f1ef80c6190 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.419+0000 7f1ef4ff9700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 0x7f1ed8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.419+0000 7f1ef4ff9700 1 -- 172.21.15.73:0/4286713864 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1ef00798e0 con 0x7f1ef80c6190 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.420+0000 7f1efcef7700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 0x7f1ed8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.420+0000 7f1efcef7700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 0x7f1ed8061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1ee80097c0 tx=0x7f1ef80a8040).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 Processor -- start 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 -- start start 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e43d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e16600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10de1060 con 0x55ae10e64ea0 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10d581f0 con 0x55ae10bd7e40 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae10d6d440 con 0x55ae10e1bcb0 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54576/0 (socket says 172.21.15.73:54576) 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1ef7fff700 1 -- 172.21.15.73:0/605129297 learned_addr learned my addr 172.21.15.73:0/605129297 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.422+0000 7f1efcef7700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e43d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e16600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 -- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 msgr2=0x55ae10e16600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e16600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 -- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 msgr2=0x55ae10e43d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e43d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 -- 172.21.15.73:0/605129297 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae10d66620 con 0x55ae10e64ea0 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1efcef7700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e43d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f1ee000d870 tx=0x7f1ee0008040).ready entity=mon.0 client_cookie=7eb5db8b2feba30b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1eedffb700 1 -- 172.21.15.73:0/605129297 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000f460 con 0x55ae10e64ea0 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1eedffb700 1 -- 172.21.15.73:0/605129297 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000f5c0 con 0x55ae10e64ea0 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1eedffb700 1 -- 172.21.15.73:0/605129297 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee000f940 con 0x55ae10e64ea0 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 -- 172.21.15.73:0/605129297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 msgr2=0x55ae10e1b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f1ee000d870 tx=0x7f1ee0008040).stop 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 -- 172.21.15.73:0/605129297 shutdown_connections 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e16600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e43d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/605129297 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e64ea0 0x55ae10e1b7c0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 -- 172.21.15.73:0/605129297 >> 172.21.15.73:0/605129297 conn(0x55ae10c854a0 msgr2=0x55ae10e64790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.423+0000 7f1f10af3b80 1 -- 172.21.15.73:0/605129297 shutdown_connections 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.424+0000 7f1f10af3b80 1 -- 172.21.15.73:0/605129297 wait complete. 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.424+0000 7f1f10af3b80 1 -- start start 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.424+0000 7f1f10af3b80 1 Processor -- start 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 0x55ae10e177a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e41af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10d6d580 con 0x55ae10e1bcb0 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10d6d800 con 0x55ae10e64ea0 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae10d38e10 con 0x55ae10bd7e40 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1f10af3b80 10 client.0 ll_register_callbacks cb 0x55ae10ddb460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54592/0 (socket says 172.21.15.73:54592) 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 -- 172.21.15.73:0/3498016909 learned_addr learned my addr 172.21.15.73:0/3498016909 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e41af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 0x55ae10e177a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 -- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 msgr2=0x55ae10e177a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 0x55ae10e177a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 -- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 msgr2=0x55ae10e41af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e41af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1ef7fff700 1 -- 172.21.15.73:0/3498016909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ee000e040 con 0x55ae10e1bcb0 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.425+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 0x55ae10e177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x55ae10e18e10 tx=0x7f1ee0008040).ready entity=mon.0 client_cookie=cd32bfa0c8bde8dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0006910 con 0x55ae10e1bcb0 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1ee001e620 con 0x55ae10e1bcb0 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ae10e0f720 con 0x55ae10e1bcb0 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee001e7e0 con 0x55ae10e1bcb0 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 10 client.5940 fetch_fsmap learned FSMap version 20 2023-11-28T03:05:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55ae10d619c0 con 0x55ae10e1bcb0 2023-11-28T03:05:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55ae10e0f720 con 0x55ae10e1bcb0 2023-11-28T03:05:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 20) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1ee001e940 con 0x55ae10e1bcb0 2023-11-28T03:05:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 10 client.5940 fetch_fsmap finished waiting for FSMap version 20 2023-11-28T03:05:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 10 client.5940 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 20 client.5940 populate_metadata read hostname 'smithi073' 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10e421e0 0x55ae10e5bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 4 client.5940 mds_command: new command op to 5775 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 --> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ae10bd3ca0 con 0x55ae10e421e0 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.426+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10e421e0 0x55ae10e5bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1ed6ffd700 10 client.5940 ms_handle_connect on v2:172.21.15.73:6836/3689972589 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10e421e0 0x55ae10e5bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=ae06c54f44ef3588 server_cookie=95e849b732e09adb in_seq=0 out_seq=0 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3498016909 <== mds.0 v2:172.21.15.73:6836/3689972589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55ae10bd3ca0 con 0x55ae10e421e0 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1ed6ffd700 10 client.5940 handle_command_reply: tid=0 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 client.5940 shutdown 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 msgr2=0x55ae10e41600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 rx=0x55ae10e18e10 tx=0x7f1ee0008040).stop 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 shutdown_connections 2023-11-28T03:05:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e41af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10e421e0 0x55ae10e5bda0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10bd7e40 0x55ae10e177a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3498016909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10e41600 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1ed6ffd700 0 client.5940 ms_handle_reset on v2:172.21.15.73:6836/3689972589 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 >> 172.21.15.73:0/3498016909 conn(0x55ae10c854a0 msgr2=0x55ae10e1caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 shutdown_connections 2023-11-28T03:05:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.427+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3498016909 wait complete. 2023-11-28T03:05:10.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.428+0000 7f1f10af3b80 20 client.5940 trim_cache size 0 max 16384 2023-11-28T03:05:10.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.444+0000 7f1f10af3b80 1 Processor -- start 2023-11-28T03:05:10.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.444+0000 7f1f10af3b80 1 -- start start 2023-11-28T03:05:10.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e5bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e64ea0 0x55ae10ec13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10e42e90 con 0x55ae10e1bcb0 2023-11-28T03:05:10.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10e19960 con 0x55ae10bd7e40 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae10e1ab10 con 0x55ae10e64ea0 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54608/0 (socket says 172.21.15.73:54608) 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 -- 172.21.15.73:0/1124801798 learned_addr learned my addr 172.21.15.73:0/1124801798 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1efcef7700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e5bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e64ea0 0x55ae10ec13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 -- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e64ea0 msgr2=0x55ae10ec13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e64ea0 0x55ae10ec13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 -- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 msgr2=0x55ae10e5bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e5bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 -- 172.21.15.73:0/1124801798 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ae10e0f340 con 0x55ae10e1bcb0 2023-11-28T03:05:10.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1efcef7700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e5bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f1ee001ef70 tx=0x7f1ee0020070).ready entity=mon.0 client_cookie=ba7bf54356597e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1eee7fc700 1 -- 172.21.15.73:0/1124801798 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee0006910 con 0x55ae10e1bcb0 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1eee7fc700 1 -- 172.21.15.73:0/1124801798 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000fa40 con 0x55ae10e1bcb0 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.445+0000 7f1eee7fc700 1 -- 172.21.15.73:0/1124801798 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee001e620 con 0x55ae10e1bcb0 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- 172.21.15.73:0/1124801798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 msgr2=0x55ae10ec0ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 rx=0x7f1ee001ef70 tx=0x7f1ee0020070).stop 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- 172.21.15.73:0/1124801798 shutdown_connections 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e64ea0 0x55ae10ec13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10bd7e40 0x55ae10e5bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/1124801798 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10e1bcb0 0x55ae10ec0ee0 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- 172.21.15.73:0/1124801798 >> 172.21.15.73:0/1124801798 conn(0x55ae10c854a0 msgr2=0x55ae10e18440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- 172.21.15.73:0/1124801798 shutdown_connections 2023-11-28T03:05:10.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- 172.21.15.73:0/1124801798 wait complete. 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.446+0000 7f1f10af3b80 1 -- start start 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 Processor -- start 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e49630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e49b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10eafc90 con 0x55ae10bd7e40 2023-11-28T03:05:10.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ae10e4a010 con 0x55ae10e64ea0 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.447+0000 7f1f10af3b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55ae10e77eb0 con 0x55ae10e1bcb0 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1f10af3b80 10 client.0 ll_register_callbacks cb 0x55ae10ddb460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54624/0 (socket says 172.21.15.73:54624) 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 -- 172.21.15.73:0/3016931435 learned_addr learned my addr 172.21.15.73:0/3016931435 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e49630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efd6f8700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e49b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 -- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 msgr2=0x55ae10e49630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e49630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 -- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 msgr2=0x55ae10e49b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e49b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 -- 172.21.15.73:0/3016931435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ae10d619c0 con 0x55ae10bd7e40 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1efcef7700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ef80c2360 tx=0x7f1ee8030280).ready entity=mon.0 client_cookie=935fa3c28a74406f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ed6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:05:10.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ee802f070 con 0x55ae10bd7e40 2023-11-28T03:05:10.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1ee8029420 con 0x55ae10bd7e40 2023-11-28T03:05:10.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1ee000f5c0 con 0x55ae10bd7e40 2023-11-28T03:05:10.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1ee80297f0 con 0x55ae10bd7e40 2023-11-28T03:05:10.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1f10af3b80 10 client.5946 fetch_fsmap learned FSMap version 20 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=2+}) v3 -- 0x55ae10d61710 con 0x55ae10bd7e40 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.448+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ee000f5c0 con 0x55ae10bd7e40 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 20) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1ee8029950 con 0x55ae10bd7e40 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 10 client.5946 fetch_fsmap finished waiting for FSMap version 20 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 10 client.5946 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 20 client.5946 populate_metadata read hostname 'smithi073' 2023-11-28T03:05:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10ec9c00 0x55ae10ec9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 4 client.5946 mds_command: new command op to 5775 tid=0[{"prefix": "status"}] 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 --> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55ae10c0aca0 con 0x55ae10ec9c00 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10ec9c00 0x55ae10ec9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ed6ffd700 10 client.5946 ms_handle_connect on v2:172.21.15.73:6836/3689972589 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ef7fff700 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10ec9c00 0x55ae10ec9fe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=84f8c1983afb11e0 server_cookie=e9627750d988704c in_seq=0 out_seq=0 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ed6ffd700 1 -- 172.21.15.73:0/3016931435 <== mds.0 v2:172.21.15.73:6836/3689972589 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+325 (crc 0 0 0) 0x55ae10c0aca0 con 0x55ae10ec9c00 2023-11-28T03:05:10.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ed6ffd700 10 client.5946 handle_command_reply: tid=0 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 client.5946 shutdown 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 msgr2=0x55ae10eaf7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7f1ef80c2360 tx=0x7f1ee8030280).stop 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 shutdown_connections 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55ae10e64ea0 0x55ae10e49b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55ae10bd7e40 0x55ae10eaf7a0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.73:6836/3689972589,v1:172.21.15.73:6837/3689972589] conn(0x55ae10ec9c00 0x55ae10ec9fe0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 --2- 172.21.15.73:0/3016931435 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55ae10e1bcb0 0x55ae10e49630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1ed6ffd700 0 client.5946 ms_handle_reset on v2:172.21.15.73:6836/3689972589 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 >> 172.21.15.73:0/3016931435 conn(0x55ae10c854a0 msgr2=0x55ae10e18440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 shutdown_connections 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.449+0000 7f1f10af3b80 1 -- 172.21.15.73:0/3016931435 wait complete. 2023-11-28T03:05:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1f10af3b80 20 client.5946 trim_cache size 0 max 16384 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 5775, 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:05:10.475 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 9.6723373580000001, 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 20, 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 65, 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 65, 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 15.237969904 2023-11-28T03:05:10.476 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:05:10.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 msgr2=0x7f1ed8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 0x7f1ed8061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f1ee80097c0 tx=0x7f1ef80a8040).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 msgr2=0x7f1ef80aa500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 rx=0x7f1ef000c410 tx=0x7f1ef0007d70).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 shutdown_connections 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ed805f150 0x7f1ed8061610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ef80c6190 0x7f1ef80aa500 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ef8064350 0x7f1ef80ab230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 --2- 172.21.15.73:0/4286713864 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ef8098d10 0x7f1ef80aa010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 >> 172.21.15.73:0/4286713864 conn(0x7f1ef80cb310 msgr2=0x7f1ef8063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 shutdown_connections 2023-11-28T03:05:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.450+0000 7f1eff15b700 1 -- 172.21.15.73:0/4286713864 wait complete. 2023-11-28T03:05:10.479 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:05:10.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.578+0000 7fa960d25700 1 Processor -- start 2023-11-28T03:05:10.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.578+0000 7fa960d25700 1 -- start start 2023-11-28T03:05:10.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa95c0ca740 0x7fa95c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa95c0aa0d0 0x7fa95c0aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0c0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c154ae0 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c0a9700 con 0x7fa95c0ca740 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa960d25700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0c1280 con 0x7fa95c0c0930 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa95c0aa0d0 0x7fa95c0aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa95c0aa0d0 0x7fa95c0aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54632/0 (socket says 172.21.15.73:54632) 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 -- 172.21.15.73:0/2188478103 learned_addr learned my addr 172.21.15.73:0/2188478103 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95b7fe700 1 --2- 172.21.15.73:0/2188478103 >> v2:172.21.15.150:3300/0 conn(0x7fa95c0ca740 0x7fa95c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 -- 172.21.15.73:0/2188478103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 msgr2=0x7fa95c0c0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 --2- 172.21.15.73:0/2188478103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0c0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:10.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 -- 172.21.15.73:0/2188478103 >> v2:172.21.15.150:3300/0 conn(0x7fa95c0ca740 msgr2=0x7fa95c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 --2- 172.21.15.73:0/2188478103 >> v2:172.21.15.150:3300/0 conn(0x7fa95c0ca740 0x7fa95c0cab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.579+0000 7fa95affd700 1 -- 172.21.15.73:0/2188478103 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c0a9b90 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa95affd700 1 --2- 172.21.15.73:0/2188478103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c0aa540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fa95c084520 tx=0x7fa9500090f0).ready entity=mon.0 client_cookie=bdd0829d911f6b91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa95a7fc700 1 -- 172.21.15.73:0/2188478103 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000b690 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa95a7fc700 1 -- 172.21.15.73:0/2188478103 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa95000b7f0 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa95a7fc700 1 -- 172.21.15.73:0/2188478103 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa95000bb70 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 -- 172.21.15.73:0/2188478103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0aa0d0 msgr2=0x7fa95c0aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 --2- 172.21.15.73:0/2188478103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c0aa540 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 rx=0x7fa95c084520 tx=0x7fa9500090f0).stop 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 -- 172.21.15.73:0/2188478103 shutdown_connections 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 --2- 172.21.15.73:0/2188478103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0c0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 --2- 172.21.15.73:0/2188478103 >> v2:172.21.15.150:3300/0 conn(0x7fa95c0ca740 0x7fa95c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 --2- 172.21.15.73:0/2188478103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c0aa540 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 -- 172.21.15.73:0/2188478103 >> 172.21.15.73:0/2188478103 conn(0x7fa95c0946d0 msgr2=0x7fa95c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 -- 172.21.15.73:0/2188478103 shutdown_connections 2023-11-28T03:05:10.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.580+0000 7fa960d25700 1 -- 172.21.15.73:0/2188478103 wait complete. 2023-11-28T03:05:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.581+0000 7fa960d25700 1 Processor -- start 2023-11-28T03:05:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.581+0000 7fa960d25700 1 -- start start 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c164c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0b0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c1650f0 con 0x7fa95c0ca740 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c169110 con 0x7fa95c0aa0d0 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa960d25700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c0b0c90 con 0x7fa95c0c0930 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa95bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa95bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54640/0 (socket says 172.21.15.73:54640) 2023-11-28T03:05:10.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa95bfff700 1 -- 172.21.15.73:0/3388808425 learned_addr learned my addr 172.21.15.73:0/3388808425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa95b7fe700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.582+0000 7fa95affd700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0b0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 -- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 msgr2=0x7fa95c0b0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0b0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 -- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c0aa0d0 msgr2=0x7fa95c164c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c164c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 -- 172.21.15.73:0/3388808425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa950009370 con 0x7fa95c0ca740 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95affd700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa95bfff700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fa95400b5e0 tx=0x7fa954009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954002320 con 0x7fa95c0ca740 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa954002480 con 0x7fa95c0ca740 2023-11-28T03:05:10.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c168500 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa954002840 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.583+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c168ac0 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.584+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c0c3660 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.584+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa954002bf0 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.585+0000 7fa9437fe700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 0x7fa948061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.585+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa95407a060 con 0x7fa95c0ca740 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.585+0000 7fa95b7fe700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 0x7fa948061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.585+0000 7fa95b7fe700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 0x7fa948061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa94c00f570 tx=0x7fa94c007750).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:10.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.589+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa95c0c3660 con 0x7fa95c0ca740 2023-11-28T03:05:10.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.794+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa95c0c3660 con 0x7fa95c0ca740 2023-11-28T03:05:10.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.794+0000 7fa9437fe700 1 -- 172.21.15.73:0/3388808425 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa95c0c3660 con 0x7fa95c0ca740 2023-11-28T03:05:10.796 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:05:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 msgr2=0x7fa948061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 0x7fa948061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7fa94c00f570 tx=0x7fa94c007750).stop 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 msgr2=0x7fa95c152140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 rx=0x7fa95400b5e0 tx=0x7fa954009060).stop 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 shutdown_connections 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa94805f150 0x7fa948061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa95c0ca740 0x7fa95c152140 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa95c0aa0d0 0x7fa95c164c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 --2- 172.21.15.73:0/3388808425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa95c0c0930 0x7fa95c0b0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 >> 172.21.15.73:0/3388808425 conn(0x7fa95c0946d0 msgr2=0x7fa95c1530c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 shutdown_connections 2023-11-28T03:05:10.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:10.796+0000 7fa960d25700 1 -- 172.21.15.73:0/3388808425 wait complete. 2023-11-28T03:05:10.808 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:05:10.808 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:05:10.808 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:05:10.808 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:05:10.808 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:05:10.809 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:05:10.809 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:05:10.809 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:05:10.809 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:05:10.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:10.809 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:05:10.824 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:05:10.824 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:05:10.825 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:05:10.825 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.825 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:05:10.825 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.825 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:05:10.826 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41769sec preferred_lft 41769sec 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:05:10.827 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:05:10.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:10.829 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:05:10.888 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:05:10.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:10.889 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:05:10.946 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:05:10.948 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:05:10.949 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:05:10.949 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:05:10.950 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:05:10.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:10.950 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:05:10.951 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:05:10.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:05:10.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:05:10.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:05:10.951 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:05:11.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:11.048 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:05:11.062 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:05:11.062 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:05:11.062 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:05:11.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:11.063 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:05:11.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:11.127 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:05:11.181 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:05:11.182 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:05:11.261 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:05:11.261 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:05:11.286 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:05:11.286 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:05:11.288 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:05:11.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:05:11.288 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:05:11.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:05:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.506+0000 7f1f108dc700 1 Processor -- start 2023-11-28T03:05:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.506+0000 7f1f108dc700 1 -- start start 2023-11-28T03:05:11.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f108dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f108dc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f0c158320 0x7f1f0c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f108dc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f0c158cc0 con 0x7f1f0c0c6190 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f108dc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c72d0 con 0x7f1f0c158320 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f108dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c064570 con 0x7f1f0c098d10 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f0ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f0c158320 0x7f1f0c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f0ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1f0c158320 0x7f1f0c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54650/0 (socket says 172.21.15.73:54650) 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.507+0000 7f1f0ad9d700 1 -- 172.21.15.73:0/517738419 learned_addr learned my addr 172.21.15.73:0/517738419 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0ad9d700 1 -- 172.21.15.73:0/517738419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 msgr2=0x7f1f0c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:11.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0ad9d700 1 --2- 172.21.15.73:0/517738419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0ad9d700 1 -- 172.21.15.73:0/517738419 >> v1:172.21.15.150:6789/0 conn(0x7f1f0c0c6190 legacy=0x7f1f0c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0ad9d700 1 -- 172.21.15.73:0/517738419 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f0c154bc0 con 0x7f1f0c158320 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0ad9d700 1 --2- 172.21.15.73:0/517738419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c158320 0x7f1f0c1587d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1f04006fd0 tx=0x7f1f040090f0).ready entity=mon.0 client_cookie=eddd90e9a337261f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0959a700 1 -- 172.21.15.73:0/517738419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400b600 con 0x7f1f0c158320 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0959a700 1 -- 172.21.15.73:0/517738419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0400b760 con 0x7f1f0c158320 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f0959a700 1 -- 172.21.15.73:0/517738419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0400bae0 con 0x7f1f0c158320 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 -- 172.21.15.73:0/517738419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c158320 msgr2=0x7f1f0c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 --2- 172.21.15.73:0/517738419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c158320 0x7f1f0c1587d0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7f1f04006fd0 tx=0x7f1f040090f0).stop 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 -- 172.21.15.73:0/517738419 shutdown_connections 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 --2- 172.21.15.73:0/517738419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c158320 0x7f1f0c1587d0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 --2- 172.21.15.73:0/517738419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 -- 172.21.15.73:0/517738419 >> 172.21.15.73:0/517738419 conn(0x7f1f0c0cb310 msgr2=0x7f1f0c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 -- 172.21.15.73:0/517738419 shutdown_connections 2023-11-28T03:05:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.508+0000 7f1f108dc700 1 -- 172.21.15.73:0/517738419 wait complete. 2023-11-28T03:05:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 Processor -- start 2023-11-28T03:05:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 -- start start 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f0c158320 0x7f1f0c1551d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0ae860 con 0x7f1f0c0c6190 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c156cf0 con 0x7f1f0c158320 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f108dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0c0c4de0 con 0x7f1f0c098d10 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f09d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f09d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54664/0 (socket says 172.21.15.73:54664) 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.510+0000 7f1f09d9b700 1 -- 172.21.15.73:0/2936920507 learned_addr learned my addr 172.21.15.73:0/2936920507 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f0ad9d700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f0c158320 0x7f1f0c1551d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f0a59c700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 -- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 msgr2=0x7f1f0c156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 -- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f0c158320 msgr2=0x7f1f0c1551d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f0c158320 0x7f1f0c1551d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 -- 172.21.15.73:0/2936920507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f04009370 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f09d9b700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f1f0000b470 tx=0x7f1f00006f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00012410 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f00012590 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f0c0c50c0 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f00012950 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.511+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f0c0c53a0 con 0x7f1f0c0c6190 2023-11-28T03:05:11.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.512+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f0c06c860 con 0x7f1f0c0c6190 2023-11-28T03:05:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.512+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1f00012d00 con 0x7f1f0c0c6190 2023-11-28T03:05:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.513+0000 7f1efa7fc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 0x7f1eec0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.513+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1f00079e10 con 0x7f1f0c0c6190 2023-11-28T03:05:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.513+0000 7f1f0a59c700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 0x7f1eec0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.513+0000 7f1f0a59c700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 0x7f1eec0615e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1efc001250 tx=0x7f1efc009620).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.517+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1f0004b0e0 con 0x7f1f0c0c6190 2023-11-28T03:05:11.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.706+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f0c0c66c0 con 0x7f1eec05f120 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.707+0000 7f1efa7fc700 1 -- 172.21.15.73:0/2936920507 <== mgr.5574 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1f0c0c66c0 con 0x7f1eec05f120 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:05:11.708 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 msgr2=0x7f1eec0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 0x7f1eec0615e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f1efc001250 tx=0x7f1efc009620).stop 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 msgr2=0x7f1f0c150920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 rx=0x7f1f0000b470 tx=0x7f1f00006f90).stop 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 shutdown_connections 2023-11-28T03:05:11.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1eec05f120 0x7f1eec0615e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1f0c158320 0x7f1f0c1551d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1f0c098d10 0x7f1f0c156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 --2- 172.21.15.73:0/2936920507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1f0c0c6190 0x7f1f0c150920 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 >> 172.21.15.73:0/2936920507 conn(0x7f1f0c0cb310 msgr2=0x7f1f0c0c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 shutdown_connections 2023-11-28T03:05:11.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.709+0000 7f1f108dc700 1 -- 172.21.15.73:0/2936920507 wait complete. 2023-11-28T03:05:11.722 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:05:11.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 Processor -- start 2023-11-28T03:05:11.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 -- start start 2023-11-28T03:05:11.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 0x7f0de80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0de80c72d0 con 0x7f0de8064350 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0de8154bc0 con 0x7f0de80c6190 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.837+0000 7f0def17a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de80c66c0 con 0x7f0de8098d10 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0de7fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 0x7f0de80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0ded717700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0de8064350 0x7f0de8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:46106/0 (socket says 172.21.15.73:46106) 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0ded717700 1 -- 172.21.15.73:0/3303915455 learned_addr learned my addr 172.21.15.73:0/3303915455 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0de7fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:49658/0 (socket says 172.21.15.73:49658) 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 -- 172.21.15.73:0/3303915455 >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 msgr2=0x7f0de8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 --2- 172.21.15.73:0/3303915455 >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 -- 172.21.15.73:0/3303915455 >> v1:172.21.15.150:6789/0 conn(0x7f0de8064350 legacy=0x7f0de8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 -- 172.21.15.73:0/3303915455 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0de80c69a0 con 0x7f0de8098d10 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0de7fff700 1 --2- 172.21.15.73:0/3303915455 >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0decf16700 1 --2- 172.21.15.73:0/3303915455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 0x7f0de80c7760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0dd8009e60 tx=0x7f0dd8009780).ready entity=mon.2 client_cookie=31bf29f733bc3cb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.838+0000 7f0de77fe700 1 -- 172.21.15.73:0/3303915455 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd800e070 con 0x7f0de8098d10 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0de77fe700 1 -- 172.21.15.73:0/3303915455 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dd800bb60 con 0x7f0de8098d10 2023-11-28T03:05:11.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0de77fe700 1 -- 172.21.15.73:0/3303915455 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dd8012410 con 0x7f0de8098d10 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 -- 172.21.15.73:0/3303915455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 msgr2=0x7f0de80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 --2- 172.21.15.73:0/3303915455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 0x7f0de80c7760 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 rx=0x7f0dd8009e60 tx=0x7f0dd8009780).stop 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 -- 172.21.15.73:0/3303915455 shutdown_connections 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 --2- 172.21.15.73:0/3303915455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8098d10 0x7f0de80c7760 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 --2- 172.21.15.73:0/3303915455 >> v2:172.21.15.150:3300/0 conn(0x7f0de80c6190 0x7f0de8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 -- 172.21.15.73:0/3303915455 >> 172.21.15.73:0/3303915455 conn(0x7f0de80cb310 msgr2=0x7f0de80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:11.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 -- 172.21.15.73:0/3303915455 shutdown_connections 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.839+0000 7f0def17a700 1 -- 172.21.15.73:0/3303915455 wait complete. 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.840+0000 7f0def17a700 1 Processor -- start 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.840+0000 7f0def17a700 1 -- start start 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8064350 0x7f0de8156c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de80c6190 0x7f0de80a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de81567a0 con 0x7f0de8098d10 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8155b90 con 0x7f0de80c6190 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0def17a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0de8155e70 con 0x7f0de8064350 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0de7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54676/0 (socket says 172.21.15.73:54676) 2023-11-28T03:05:11.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.841+0000 7f0de7fff700 1 -- 172.21.15.73:0/2892215681 learned_addr learned my addr 172.21.15.73:0/2892215681 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0ded717700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de80c6190 0x7f0de80a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0decf16700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8064350 0x7f0de8156c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 -- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8064350 msgr2=0x7f0de8156c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8064350 0x7f0de8156c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 -- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de80c6190 msgr2=0x7f0de80a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de80c6190 0x7f0de80a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 -- 172.21.15.73:0/2892215681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ddc0092a0 con 0x7f0de8098d10 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de7fff700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f0ddc00e930 tx=0x7f0ddc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc012600 con 0x7f0de8098d10 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc012760 con 0x7f0de8098d10 2023-11-28T03:05:11.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc012ae0 con 0x7f0de8098d10 2023-11-28T03:05:11.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0dd8009a20 con 0x7f0de8098d10 2023-11-28T03:05:11.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.842+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0de8156430 con 0x7f0de8098d10 2023-11-28T03:05:11.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.843+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0de806c860 con 0x7f0de8098d10 2023-11-28T03:05:11.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.843+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 14) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0ddc01b400 con 0x7f0de8098d10 2023-11-28T03:05:11.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.844+0000 7f0de4ff9700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 0x7f0dd4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:11.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.844+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0ddc07a610 con 0x7f0de8098d10 2023-11-28T03:05:11.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.844+0000 7f0decf16700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 0x7f0dd4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:11.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.844+0000 7f0decf16700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 0x7f0dd4061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0dd8009fd0 tx=0x7f0dd8009780).ready entity=mgr.5574 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:11.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:11.848+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0ddc04b880 con 0x7f0de8098d10 2023-11-28T03:05:12.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.056+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f0de81568e0 con 0x7f0de8098d10 2023-11-28T03:05:12.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.216+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 15) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f0ddc0783e0 con 0x7f0de8098d10 2023-11-28T03:05:12.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.217+0000 7f0de4ff9700 1 -- 172.21.15.73:0/2892215681 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v15) v1 ==== 92+0+0 (secure 0 0 0) 0x7f0ddc071910 con 0x7f0de8098d10 2023-11-28T03:05:12.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 msgr2=0x7f0dd4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 0x7f0dd4061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f0dd8009fd0 tx=0x7f0dd8009780).stop 2023-11-28T03:05:12.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 msgr2=0x7f0de80ad330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 rx=0x7f0ddc00e930 tx=0x7f0ddc009060).stop 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 shutdown_connections 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0dd405f150 0x7f0dd4061610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0de80c6190 0x7f0de80a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0de8064350 0x7f0de8156c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 --2- 172.21.15.73:0/2892215681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0de8098d10 0x7f0de80ad330 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 >> 172.21.15.73:0/2892215681 conn(0x7f0de80cb310 msgr2=0x7f0de8148a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 shutdown_connections 2023-11-28T03:05:12.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.220+0000 7f0def17a700 1 -- 172.21.15.73:0/2892215681 wait complete. 2023-11-28T03:05:12.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:05:12.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.352+0000 7f4e72bf0700 1 Processor -- start 2023-11-28T03:05:12.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.352+0000 7f4e72bf0700 1 -- start start 2023-11-28T03:05:12.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e72bf0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4e6c1589d0 0x7f4e6c0c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e72bf0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c154bc0 con 0x7f4e6c0640b0 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e72bf0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e6c064540 con 0x7f4e6c098e50 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e72bf0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e6c099230 con 0x7f4e6c1589d0 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e7118d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4e6c0640b0 0x7f4e6c0a8180 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:57456/0 (socket says 172.21.15.73:57456) 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e7118d700 1 -- 172.21.15.73:0/203255022 learned_addr learned my addr 172.21.15.73:0/203255022 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/203255022 >> v2:172.21.15.150:3300/0 conn(0x7f4e6c1589d0 0x7f4e6c0c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3674963631 0 0) 0x7f4e64001490 con 0x7f4e6c0640b0 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4e5c0033f0 con 0x7f4e6c0640b0 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6bfff700 1 -- 172.21.15.73:0/203255022 >> v1:172.21.15.150:6789/0 conn(0x7f4e6c098e50 legacy=0x7f4e6c1582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045776329 0 0) 0x7f4e6c064540 con 0x7f4e6c098e50 2023-11-28T03:05:12.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6bfff700 1 -- 172.21.15.73:0/203255022 >> v1:172.21.15.73:6789/0 conn(0x7f4e6c0640b0 legacy=0x7f4e6c0a8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6bfff700 1 -- 172.21.15.73:0/203255022 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e6c0c6580 con 0x7f4e6c1589d0 2023-11-28T03:05:12.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.353+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/203255022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c0c7620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4e58009760 tx=0x7f4e58009060).ready entity=mon.1 client_cookie=eba27205bd5189ed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:12.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800f510 con 0x7f4e6c1589d0 2023-11-28T03:05:12.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e5800f670 con 0x7f4e6c1589d0 2023-11-28T03:05:12.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e6b7fe700 1 -- 172.21.15.73:0/203255022 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e5800f9f0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 -- 172.21.15.73:0/203255022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c1589d0 msgr2=0x7f4e6c0c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/203255022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c0c7620 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 rx=0x7f4e58009760 tx=0x7f4e58009060).stop 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 -- 172.21.15.73:0/203255022 shutdown_connections 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/203255022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c0c7620 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 -- 172.21.15.73:0/203255022 >> 172.21.15.73:0/203255022 conn(0x7f4e6c0c7db0 msgr2=0x7f4e6c0c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 -- 172.21.15.73:0/203255022 shutdown_connections 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.354+0000 7f4e72bf0700 1 -- 172.21.15.73:0/203255022 wait complete. 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.355+0000 7f4e72bf0700 1 Processor -- start 2023-11-28T03:05:12.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 -- start start 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c0640b0 0x7f4e6c164d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c098e50 0x7f4e6c1510d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c154d60 con 0x7f4e6c1589d0 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c155040 con 0x7f4e6c0640b0 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e72bf0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e6c157520 con 0x7f4e6c098e50 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e7118d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e7118d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54714/0 (socket says 172.21.15.73:54714) 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e7118d700 1 -- 172.21.15.73:0/2521885880 learned_addr learned my addr 172.21.15.73:0/2521885880 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e7098c700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c0640b0 0x7f4e6c164d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.356+0000 7f4e6bfff700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c098e50 0x7f4e6c1510d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 -- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c098e50 msgr2=0x7f4e6c1510d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c098e50 0x7f4e6c1510d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 -- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c0640b0 msgr2=0x7f4e6c164d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c0640b0 0x7f4e6c164d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 -- 172.21.15.73:0/2521885880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e580092a0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e7118d700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f4e6400c7c0 tx=0x7f4e64007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e64005d30 con 0x7f4e6c1589d0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e64005e90 con 0x7f4e6c1589d0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e640072e0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e6c157800 con 0x7f4e6c1589d0 2023-11-28T03:05:12.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.357+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e6c0c5260 con 0x7f4e6c1589d0 2023-11-28T03:05:12.362 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:12.360+0000 7fbe770b5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:05:12.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.358+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e6c06c860 con 0x7f4e6c1589d0 2023-11-28T03:05:12.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.359+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f4e64007710 con 0x7f4e6c1589d0 2023-11-28T03:05:12.362 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:12.359+0000 7fcab6370680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:05:12.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.359+0000 7f4e68ff9700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 0x7f4e50061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.359+0000 7f4e7098c700 1 -- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 msgr2=0x7f4e50061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:12.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.359+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4e64079900 con 0x7f4e6c1589d0 2023-11-28T03:05:12.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.359+0000 7f4e7098c700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 0x7f4e50061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:05:12.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.363+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4e6404abd0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.425 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:12.424+0000 7fbe770b5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:05:12.426 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:12.425+0000 7fcab6370680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:05:12.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.535+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4e6c154bc0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.540+0000 7f4e68ff9700 1 -- 172.21.15.73:0/2521885880 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v7)=0 v7) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4e6c154bc0 con 0x7f4e6c1589d0 2023-11-28T03:05:12.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 msgr2=0x7f4e50061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:05:12.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 0x7f4e50061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 msgr2=0x7f4e6c156a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 rx=0x7f4e6400c7c0 tx=0x7f4e64007e30).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 shutdown_connections 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4e5005f1a0 0x7f4e50061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4e6c1589d0 0x7f4e6c156a60 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4e6c0640b0 0x7f4e6c164d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 --2- 172.21.15.73:0/2521885880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4e6c098e50 0x7f4e6c1510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 >> 172.21.15.73:0/2521885880 conn(0x7f4e6c0c7db0 msgr2=0x7f4e6c1580f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 shutdown_connections 2023-11-28T03:05:12.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.542+0000 7f4e72bf0700 1 -- 172.21.15.73:0/2521885880 wait complete. 2023-11-28T03:05:12.556 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:12.556 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T03:05:12.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-28T03:05:12.699 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:12.698+0000 7fbe770b5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:05:12.715 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:12.714+0000 7fcab6370680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:05:12.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.736+0000 7f30acf86700 1 Processor -- start 2023-11-28T03:05:12.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.736+0000 7f30acf86700 1 -- start start 2023-11-28T03:05:12.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30acf86700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30acf86700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f30a80aa1c0 0x7f30a80aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30acf86700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30a8159210 con 0x7f30a80ab3b0 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30acf86700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30a80c0c70 con 0x7f30a80aa1c0 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30acf86700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80c2110 con 0x7f30a8154610 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f30a80aa1c0 0x7f30a80aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a659c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f30a80aa1c0 0x7f30a80aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:49712/0 (socket says 172.21.15.73:49712) 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a5d9b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f30a80ab3b0 0x7f30a80c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:46126/0 (socket says 172.21.15.73:46126) 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 -- 172.21.15.73:0/3541338463 learned_addr learned my addr 172.21.15.73:0/3541338463 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a659c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:58564/0 (socket says 172.21.15.73:58564) 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 -- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 msgr2=0x7f30a81582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 -- 172.21.15.73:0/3541338463 >> v1:172.21.15.150:6789/0 conn(0x7f30a80ab3b0 legacy=0x7f30a80c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a6d9d700 1 -- 172.21.15.73:0/3541338463 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a80aaae0 con 0x7f30a80aa1c0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a559a700 1 -- 172.21.15.73:0/3541338463 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3570499142 0 0) 0x7f30a8159210 con 0x7f30a80ab3b0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.737+0000 7f30a659c700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30a6d9d700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80aa1c0 0x7f30a80aa5f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f30a0009f20 tx=0x7f30a0009840).ready entity=mon.1 client_cookie=e636d71c9bf18204 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30a559a700 1 -- 172.21.15.73:0/3541338463 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a000e070 con 0x7f30a80aa1c0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30a559a700 1 -- 172.21.15.73:0/3541338463 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30a000b9d0 con 0x7f30a80aa1c0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30a559a700 1 -- 172.21.15.73:0/3541338463 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30a000bd50 con 0x7f30a80aa1c0 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 -- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80aa1c0 msgr2=0x7f30a80aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80aa1c0 0x7f30a80aa5f0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 rx=0x7f30a0009f20 tx=0x7f30a0009840).stop 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 -- 172.21.15.73:0/3541338463 shutdown_connections 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80aa1c0 0x7f30a80aa5f0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 --2- 172.21.15.73:0/3541338463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a8154610 0x7f30a81582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 -- 172.21.15.73:0/3541338463 >> 172.21.15.73:0/3541338463 conn(0x7f30a80655a0 msgr2=0x7f30a8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:12.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 -- 172.21.15.73:0/3541338463 shutdown_connections 2023-11-28T03:05:12.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.738+0000 7f30acf86700 1 -- 172.21.15.73:0/3541338463 wait complete. 2023-11-28T03:05:12.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.739+0000 7f30acf86700 1 Processor -- start 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.739+0000 7f30acf86700 1 -- start start 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a80aa1c0 0x7f30a80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 0x7f30a80c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80c3a30 con 0x7f30a8154610 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80c42d0 con 0x7f30a80c4760 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30acf86700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a80b1020 con 0x7f30a80aa1c0 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30a5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30a5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54724/0 (socket says 172.21.15.73:54724) 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30a5d9b700 1 -- 172.21.15.73:0/2774411825 learned_addr learned my addr 172.21.15.73:0/2774411825 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:12.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30a6d9d700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 0x7f30a80c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.740+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a80aa1c0 0x7f30a80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a80aa1c0 msgr2=0x7f30a80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a80aa1c0 0x7f30a80b0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 msgr2=0x7f30a80c2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 0x7f30a80c2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30980092a0 con 0x7f30a8154610 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a6d9d700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 0x7f30a80c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30a5d9b700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f309c00c4c0 tx=0x7f309c007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0070d0 con 0x7f30a8154610 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f309c007230 con 0x7f30a8154610 2023-11-28T03:05:12.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30a0009ae0 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f309c0056e0 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.741+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30a80c3f60 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.742+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30a806c860 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.742+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 15) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f309c005ad0 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.743+0000 7f30967fc700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.743+0000 7f30a659c700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.743+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f309c079830 con 0x7f30a8154610 2023-11-28T03:05:12.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.743+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:05:12.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.750+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f309c04aba0 con 0x7f30a8154610 2023-11-28T03:05:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.944+0000 7f30a659c700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:12.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.944+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:05:12.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:12.946+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a80ad7c0 con 0x7f308805f1a0 2023-11-28T03:05:13.313 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:13.312+0000 7fbe770b5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:05:13.341 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:13.340+0000 7fcab6370680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:05:13.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:13.345+0000 7f30a659c700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:13.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:13.345+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:05:13.385 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:13.384+0000 7fbe770b5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:05:13.414 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:13.413+0000 7fcab6370680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:05:13.572 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:13.571+0000 7fbe770b5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:05:13.598 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:13.597+0000 7fcab6370680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:05:13.991 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:13.991+0000 7fbe770b5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:05:14.016 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.016+0000 7fcab6370680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:05:14.095 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.094+0000 7fbe770b5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:05:14.118 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.118+0000 7fcab6370680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:05:14.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:14.147+0000 7f30a659c700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:14.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:14.147+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:05:14.248 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.247+0000 7fbe770b5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:05:14.270 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.270+0000 7fcab6370680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:05:14.324 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.323+0000 7fbe770b5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:05:14.345 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.345+0000 7fcab6370680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:05:14.391 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.390+0000 7fbe770b5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:05:14.412 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.412+0000 7fcab6370680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:05:14.650 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.649+0000 7fbe770b5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:05:14.672 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.672+0000 7fcab6370680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:05:14.837 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:14.836+0000 7fbe770b5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:05:14.853 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:14.854+0000 7fcab6370680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:05:15.489 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.489+0000 7fbe770b5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:05:15.506 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.507+0000 7fcab6370680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:05:15.558 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.558+0000 7fbe770b5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:05:15.574 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.574+0000 7fcab6370680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:05:15.634 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.633+0000 7fbe770b5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:05:15.647 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.647+0000 7fcab6370680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:05:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:15.748+0000 7f30a659c700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:05:15.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:15.748+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:05:15.780 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.779+0000 7fbe770b5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:05:15.792 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.792+0000 7fcab6370680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:05:15.851 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.850+0000 7fbe770b5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:05:15.862 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.862+0000 7fcab6370680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:05:15.966 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:15.966+0000 7fbe770b5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:05:15.977 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:15.977+0000 7fcab6370680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:05:16.075 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:16.074+0000 7fbe770b5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:05:16.083 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.084+0000 7fcab6370680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:05:16.370 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.370+0000 7fcab6370680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:05:16.382 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:16.381+0000 7fbe770b5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:05:16.442 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.442+0000 7fcab6370680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:05:16.455 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:05:16.454+0000 7fbe770b5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:05:16.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:16.462+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 16) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f309c0776a0 con 0x7f30a8154610 2023-11-28T03:05:16.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:16.463+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f3088061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:05:16.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:16.463+0000 7f30967fc700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f3088061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:05:16.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2aaf6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2aaf6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2aaf6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2aaf6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2aaf6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2cafa700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2cafa700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2cafa700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2cafa700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:16.525 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:05:16.524+0000 7fca2cafa700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:05:17.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.467+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f309c077a20 con 0x7f30a8154610 2023-11-28T03:05:17.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.468+0000 7f30967fc700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f30880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.468+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f30a80ad7c0 con 0x7f308805f1a0 2023-11-28T03:05:17.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.469+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f30880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.470+0000 7f30a659c700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f30880615e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3098009740 tx=0x7f3098009060).ready entity=mgr.5970 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.483+0000 7f30967fc700 1 -- 172.21.15.73:0/2774411825 <== mgr.5970 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f30a80ad7c0 con 0x7f308805f1a0 2023-11-28T03:05:17.483 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T03:05:17.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 msgr2=0x7f30880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f30880615e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 rx=0x7f3098009740 tx=0x7f3098009060).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 msgr2=0x7f30a8168e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 rx=0x7f309c00c4c0 tx=0x7f309c007da0).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 shutdown_connections 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f308805f1a0 0x7f30880615e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30a80c4760 0x7f30a80c2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30a80aa1c0 0x7f30a80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 --2- 172.21.15.73:0/2774411825 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f30a8154610 0x7f30a8168e40 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 >> 172.21.15.73:0/2774411825 conn(0x7f30a80655a0 msgr2=0x7f30a80c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 shutdown_connections 2023-11-28T03:05:17.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.485+0000 7f30acf86700 1 -- 172.21.15.73:0/2774411825 wait complete. 2023-11-28T03:05:17.498 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T03:05:17.499 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 2M 2023-11-28T03:05:17.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.618+0000 7fad41102700 1 Processor -- start 2023-11-28T03:05:17.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.618+0000 7fad41102700 1 -- start start 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad41102700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad3c1556f0 0x7fad3c067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad41102700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 0x7fad3c0bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad41102700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3c0bddb0 con 0x7fad3c13ab60 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad41102700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c067c70 con 0x7fad3c1556f0 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad41102700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad3c0c3510 con 0x7fad3c13af40 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad3c1556f0 0x7fad3c067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad3c1556f0 0x7fad3c067780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54042/0 (socket says 172.21.15.73:54042) 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3259c700 1 -- 172.21.15.73:0/4188015289 learned_addr learned my addr 172.21.15.73:0/4188015289 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3b59e700 1 --2- 172.21.15.73:0/4188015289 >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 0x7fad3c0bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3259c700 1 -- 172.21.15.73:0/4188015289 >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 msgr2=0x7fad3c0bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.619+0000 7fad3259c700 1 --2- 172.21.15.73:0/4188015289 >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 0x7fad3c0bd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3259c700 1 -- 172.21.15.73:0/4188015289 >> v1:172.21.15.150:6789/0 conn(0x7fad3c13ab60 legacy=0x7fad3c154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3259c700 1 -- 172.21.15.73:0/4188015289 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad3c0bdfe0 con 0x7fad3c1556f0 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3b59e700 1 --2- 172.21.15.73:0/4188015289 >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 0x7fad3c0bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3259c700 1 --2- 172.21.15.73:0/4188015289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c067780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fad3c06acd0 tx=0x7fad240090f0).ready entity=mon.0 client_cookie=367dd72625795e2d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3a59c700 1 -- 172.21.15.73:0/4188015289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400b5a0 con 0x7fad3c1556f0 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3a59c700 1 -- 172.21.15.73:0/4188015289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad2400b700 con 0x7fad3c1556f0 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad3a59c700 1 -- 172.21.15.73:0/4188015289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad2400ba80 con 0x7fad3c1556f0 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 -- 172.21.15.73:0/4188015289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 msgr2=0x7fad3c067780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 --2- 172.21.15.73:0/4188015289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c067780 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 rx=0x7fad3c06acd0 tx=0x7fad240090f0).stop 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 -- 172.21.15.73:0/4188015289 shutdown_connections 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 --2- 172.21.15.73:0/4188015289 >> v2:172.21.15.150:3300/0 conn(0x7fad3c13af40 0x7fad3c0bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 --2- 172.21.15.73:0/4188015289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c067780 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 -- 172.21.15.73:0/4188015289 >> 172.21.15.73:0/4188015289 conn(0x7fad3c063810 msgr2=0x7fad3c063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 -- 172.21.15.73:0/4188015289 shutdown_connections 2023-11-28T03:05:17.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.620+0000 7fad41102700 1 -- 172.21.15.73:0/4188015289 wait complete. 2023-11-28T03:05:17.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 Processor -- start 2023-11-28T03:05:17.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 -- start start 2023-11-28T03:05:17.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad3c13ab60 0x7fad3c0bf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3c13af40 0x7fad3c150e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0a5c00 con 0x7fad3c1556f0 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0ac140 con 0x7fad3c13ab60 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad41102700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad3c0a5360 con 0x7fad3c13af40 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad3b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54052/0 (socket says 172.21.15.73:54052) 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.622+0000 7fad3b59e700 1 -- 172.21.15.73:0/1006075401 learned_addr learned my addr 172.21.15.73:0/1006075401 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3ad9d700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad3c13ab60 0x7fad3c0bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3259c700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3c13af40 0x7fad3c150e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 -- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3c13af40 msgr2=0x7fad3c150e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3c13af40 0x7fad3c150e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 -- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad3c13ab60 msgr2=0x7fad3c0bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad3c13ab60 0x7fad3c0bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 -- 172.21.15.73:0/1006075401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad24009370 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad3b59e700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fad3400cfb0 tx=0x7fad340078c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad3400aea0 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad34016470 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad340167f0 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad3c0a5640 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.623+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad3c151300 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.624+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3c0abf10 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.624+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fad34013070 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.625+0000 7fad337fe700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 0x7fad1c061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.625+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fad3407a9a0 con 0x7fad3c1556f0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.625+0000 7fad3ad9d700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 0x7fad1c061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.625+0000 7fad3ad9d700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 0x7fad1c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fad2c00b940 tx=0x7fad2c006ee0).ready entity=mgr.5970 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.629+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fad3c0abf10 con 0x7fad3c1556f0 2023-11-28T03:05:17.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.815+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7fad3c0abf10 con 0x7fad1c05f080 2023-11-28T03:05:17.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.820+0000 7fad337fe700 1 -- 172.21.15.73:0/1006075401 <== mgr.5970 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fad3c0abf10 con 0x7fad1c05f080 2023-11-28T03:05:17.820 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.821+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 msgr2=0x7fad1c061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.821+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 0x7fad1c061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fad2c00b940 tx=0x7fad2c006ee0).stop 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.821+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 msgr2=0x7fad3c0a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.821+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 rx=0x7fad3400cfb0 tx=0x7fad340078c0).stop 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 shutdown_connections 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad1c05f080 0x7fad1c061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad3c1556f0 0x7fad3c0a48e0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad3c13ab60 0x7fad3c0bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 --2- 172.21.15.73:0/1006075401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad3c13af40 0x7fad3c150e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 >> 172.21.15.73:0/1006075401 conn(0x7fad3c063810 msgr2=0x7fad3c063bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 shutdown_connections 2023-11-28T03:05:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.822+0000 7fad41102700 1 -- 172.21.15.73:0/1006075401 wait complete. 2023-11-28T03:05:17.833 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T03:05:17.834 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-28-03_06_00 in ~43s... 2023-11-28T03:05:17.834 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-28-03_07_00 in ~103s... 2023-11-28T03:05:17.834 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-11-28T03:05:17.835 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-28T03:05:17.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.951+0000 7faf3f3dc700 1 Processor -- start 2023-11-28T03:05:17.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.951+0000 7faf3f3dc700 1 -- start start 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf380c3c90 0x7faf380c4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faf380be030 0x7faf380bcc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38066f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380bdcd0 con 0x7faf380c3c90 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf380c2520 con 0x7faf380be030 2023-11-28T03:05:17.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.952+0000 7faf3f3dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf380bd430 con 0x7faf38066a90 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf380c3c90 0x7faf380c4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faf380c3c90 0x7faf380c4070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54056/0 (socket says 172.21.15.73:54056) 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 -- 172.21.15.73:0/3345355302 learned_addr learned my addr 172.21.15.73:0/3345355302 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3dbd9700 1 --2- 172.21.15.73:0/3345355302 >> v2:172.21.15.150:3300/0 conn(0x7faf380be030 0x7faf380bcc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3ebdb700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38066f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 -- 172.21.15.73:0/3345355302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 msgr2=0x7faf38066f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38066f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 -- 172.21.15.73:0/3345355302 >> v2:172.21.15.150:3300/0 conn(0x7faf380be030 msgr2=0x7faf380bcc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 --2- 172.21.15.73:0/3345355302 >> v2:172.21.15.150:3300/0 conn(0x7faf380be030 0x7faf380bcc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 -- 172.21.15.73:0/3345355302 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380bd710 con 0x7faf380c3c90 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3ebdb700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:17.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3e3da700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380c3c90 0x7faf380c4070 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7faf380c35a0 tx=0x7faf28009840).ready entity=mon.0 client_cookie=b48beb89c9fc7a47 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3d3d8700 1 -- 172.21.15.73:0/3345355302 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800e070 con 0x7faf380c3c90 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3d3d8700 1 -- 172.21.15.73:0/3345355302 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2800ba50 con 0x7faf380c3c90 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.953+0000 7faf3d3d8700 1 -- 172.21.15.73:0/3345355302 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2800bdd0 con 0x7faf380c3c90 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 -- 172.21.15.73:0/3345355302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380c3c90 msgr2=0x7faf380c4070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380c3c90 0x7faf380c4070 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 rx=0x7faf380c35a0 tx=0x7faf28009840).stop 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 -- 172.21.15.73:0/3345355302 shutdown_connections 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380c3c90 0x7faf380c4070 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/3345355302 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38066f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/3345355302 >> v2:172.21.15.150:3300/0 conn(0x7faf380be030 0x7faf380bcc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 -- 172.21.15.73:0/3345355302 >> 172.21.15.73:0/3345355302 conn(0x7faf380642e0 msgr2=0x7faf380c7cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 -- 172.21.15.73:0/3345355302 shutdown_connections 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.954+0000 7faf3f3dc700 1 -- 172.21.15.73:0/3345355302 wait complete. 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.955+0000 7faf3f3dc700 1 Processor -- start 2023-11-28T03:05:17.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.955+0000 7faf3f3dc700 1 -- start start 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38063180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380c3c90 0x7faf380ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf380a8c70 con 0x7faf380be030 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf38166140 con 0x7faf380c3c90 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf381663f0 con 0x7faf38066a90 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54066/0 (socket says 172.21.15.73:54066) 2023-11-28T03:05:17.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 -- 172.21.15.73:0/2724124808 learned_addr learned my addr 172.21.15.73:0/2724124808 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3e3da700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38063180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3ebdb700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380c3c90 0x7faf380ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 -- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 msgr2=0x7faf38063180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38063180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 -- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380c3c90 msgr2=0x7faf380ad7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380c3c90 0x7faf380ad7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 -- 172.21.15.73:0/2724124808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf28009ae0 con 0x7faf380be030 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3e3da700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38063180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3dbd9700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7faf2c00d890 tx=0x7faf2c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c0125f0 con 0x7faf380be030 2023-11-28T03:05:17.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf2c012750 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2c012ad0 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.956+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf381666d0 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.957+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf380a8ec0 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.958+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf380a9230 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.958+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 17) v1 ==== 75357+0+0 (secure 0 0 0) 0x7faf2c013480 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.958+0000 7faf367fc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 0x7faf24061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.958+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6293+0+0 (secure 0 0 0) 0x7faf2c07a1f0 con 0x7faf380be030 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.958+0000 7faf3e3da700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 0x7faf24061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:05:17.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.959+0000 7faf3e3da700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 0x7faf24061510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7faf2800ba20 tx=0x7faf28009840).ready entity=mgr.5970 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:05:17.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:17.962+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf2c04b580 con 0x7faf380be030 2023-11-28T03:05:18.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.148+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7faf3806c7c0 con 0x7faf2405f050 2023-11-28T03:05:18.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.151+0000 7faf367fc700 1 -- 172.21.15.73:0/2724124808 <== mgr.5970 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7faf3806c7c0 con 0x7faf2405f050 2023-11-28T03:05:18.151 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:05:18.151 INFO:teuthology.orchestra.run.smithi073.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-11-28T03:05:18.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 msgr2=0x7faf24061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:18.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 0x7faf24061510 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7faf2800ba20 tx=0x7faf28009840).stop 2023-11-28T03:05:18.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 msgr2=0x7faf38165c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:05:18.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 rx=0x7faf2c00d890 tx=0x7faf2c008040).stop 2023-11-28T03:05:18.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 shutdown_connections 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2405f050 0x7faf24061510 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380c3c90 0x7faf380ad7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38066a90 0x7faf38063180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 --2- 172.21.15.73:0/2724124808 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf380be030 0x7faf38165c50 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 >> 172.21.15.73:0/2724124808 conn(0x7faf380642e0 msgr2=0x7faf38138930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 shutdown_connections 2023-11-28T03:05:18.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:05:18.153+0000 7faf3f3dc700 1 -- 172.21.15.73:0/2724124808 wait complete. 2023-11-28T03:05:18.165 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-11-28T03:05:18.165 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-11-28T03:05:18.165 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:18.165 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:19.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:19.182 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:20.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:20.199 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:21.219 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:21.219 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:22.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:22.238 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:22.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:05:22.427 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:05:23.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:23.258 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:24.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:24.278 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:25.297 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:25.297 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:26.318 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:26.318 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:27.337 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:27.337 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:28.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:28.357 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:29.378 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:29.378 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:30.397 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:30.397 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:31.417 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:31.417 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:32.437 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:32.437 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:33.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:33.456 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:34.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:34.476 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:35.496 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:35.497 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:36.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:36.517 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:37.537 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:37.537 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:38.557 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:38.557 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:39.577 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:39.578 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:40.597 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:40.597 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:41.617 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:41.617 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:42.637 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:42.637 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:43.657 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:43.657 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:44.677 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:44.677 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:45.698 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:45.698 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:46.717 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:46.717 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:47.737 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:47.737 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:48.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:48.756 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:49.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:49.776 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:50.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:50.796 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:51.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:51.816 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:52.483 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:05:52.486 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:05:52.836 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:52.837 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:53.856 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:53.856 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:54.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:54.877 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:55.896 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:55.897 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:56.916 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:56.916 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:57.935 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:57.935 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:58.954 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:58.954 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:05:59.975 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:05:59.975 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:00.824 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:06:00.823+0000 7fca1cada700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:06:00.824 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:06:00.823+0000 7fca1cada700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:06:00.824 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:06:00.823+0000 7fca1cada700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:06:00.825 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:06:00.823+0000 7fca1cada700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:06:00.825 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:06:00.823+0000 7fca1cada700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:06:00.997 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:00.997 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:01.016 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:01.016 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-28-03_06_00_UTC'] 2023-11-28T03:06:01.016 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701140717.8342059, now = 1701140761.0164807, timo = 43 2023-11-28T03:06:02.017 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:02.017 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:02.036 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:03.037 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:03.038 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:03.056 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:04.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:04.057 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:04.076 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:05.077 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:05.077 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:05.096 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:06.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:06.097 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:06.115 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:07.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:07.117 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:07.134 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:08.136 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:08.136 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:08.156 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:09.157 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:09.157 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:09.176 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:10.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:10.178 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:10.196 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:11.197 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:11.198 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:11.215 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:12.217 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:12.217 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:12.234 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:13.236 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:13.236 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:13.254 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:14.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:14.256 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:14.274 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:15.275 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:15.276 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:15.294 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:16.296 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:16.297 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:17.316 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:17.317 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:17.334 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:18.336 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:18.336 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:18.353 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:19.354 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:19.354 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:19.373 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:20.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:20.375 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:20.393 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:21.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:21.395 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:21.414 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:22.415 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:22.415 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:22.434 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:22.589 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:06:22.592 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:06:23.436 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:23.436 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:23.454 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:24.456 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:24.456 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:24.475 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:25.476 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:25.476 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:25.494 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:26.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:26.495 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:26.514 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:27.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:27.515 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:27.533 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:28.535 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:28.536 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:28.554 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:29.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:29.555 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:29.572 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:30.574 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:30.574 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:30.593 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:31.594 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:31.594 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:31.613 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:32.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:32.614 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:32.633 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:33.634 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:33.634 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:33.652 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:34.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:34.655 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:34.674 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:35.675 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:35.675 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:35.694 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:36.695 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:36.696 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:36.714 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:37.716 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:37.716 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:37.735 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:38.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:38.736 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:38.755 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:39.756 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:39.756 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:39.775 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:40.776 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:40.776 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:40.794 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:41.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:41.796 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:41.816 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:42.816 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:42.817 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:42.838 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:43.838 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:43.838 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:43.857 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:44.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:44.859 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:44.877 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:45.878 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:45.878 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:45.898 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:46.899 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:46.899 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:46.918 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:47.919 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:47.920 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:47.938 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:48.939 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:48.940 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:48.959 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:49.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:49.960 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:49.978 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:50.980 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:50.980 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:50.998 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:51.999 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:51.999 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:52.018 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:52.653 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:06:52.658 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:06:53.019 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:53.019 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:53.037 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:54.038 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:54.038 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:54.057 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:55.059 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:55.059 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:55.077 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:56.079 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:56.079 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:56.097 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:57.099 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:57.099 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:57.120 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:58.122 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:58.122 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:58.141 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:06:59.142 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:06:59.142 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:06:59.161 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:07:00.163 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:00.163 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:07:00.181 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:07:01.182 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:01.182 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:07:01.201 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:07:01.201 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_07_00_UTC 2023-11-28T03:07:01.202 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-28-03_07_00_UTC'] 2023-11-28T03:07:01.202 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701140717.8342059, now = 1701140821.2029219, timo = 103 2023-11-28T03:07:02.204 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-28T03:07:02.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.332+0000 7fdc0f9d3700 1 Processor -- start 2023-11-28T03:07:02.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.332+0000 7fdc0f9d3700 1 -- start start 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08154610 0x7fdc081582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdc080ab3b0 0x7fdc080c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc080aa1c0 0x7fdc080aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc08159210 con 0x7fdc080aa1c0 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080c0c70 con 0x7fdc080ab3b0 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0f9d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080aaae0 con 0x7fdc08154610 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc080aa1c0 0x7fdc080aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdc080aa1c0 0x7fdc080aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43966/0 (socket says 172.21.15.73:43966) 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 -- 172.21.15.73:0/2748059771 learned_addr learned my addr 172.21.15.73:0/2748059771 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0cf6e700 1 --2- 172.21.15.73:0/2748059771 >> v2:172.21.15.150:3300/0 conn(0x7fdc080ab3b0 0x7fdc080c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0d76f700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08154610 0x7fdc081582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 -- 172.21.15.73:0/2748059771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08154610 msgr2=0x7fdc081582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08154610 0x7fdc081582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 -- 172.21.15.73:0/2748059771 >> v2:172.21.15.150:3300/0 conn(0x7fdc080ab3b0 msgr2=0x7fdc080c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 --2- 172.21.15.73:0/2748059771 >> v2:172.21.15.150:3300/0 conn(0x7fdc080ab3b0 0x7fdc080c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.333+0000 7fdc0df70700 1 -- 172.21.15.73:0/2748059771 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08158970 con 0x7fdc080aa1c0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0df70700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc080aa5f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fdc080c5810 tx=0x7fdc00009840).ready entity=mon.0 client_cookie=c3bd2c31624f326e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdbfffff700 1 -- 172.21.15.73:0/2748059771 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000e070 con 0x7fdc080aa1c0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdbfffff700 1 -- 172.21.15.73:0/2748059771 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc0000ba20 con 0x7fdc080aa1c0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdbfffff700 1 -- 172.21.15.73:0/2748059771 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc0000bda0 con 0x7fdc080aa1c0 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/2748059771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 msgr2=0x7fdc080aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc080aa5f0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 rx=0x7fdc080c5810 tx=0x7fdc00009840).stop 2023-11-28T03:07:02.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/2748059771 shutdown_connections 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc080aa5f0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/2748059771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08154610 0x7fdc081582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/2748059771 >> v2:172.21.15.150:3300/0 conn(0x7fdc080ab3b0 0x7fdc080c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/2748059771 >> 172.21.15.73:0/2748059771 conn(0x7fdc080655a0 msgr2=0x7fdc08069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/2748059771 shutdown_connections 2023-11-28T03:07:02.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.334+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/2748059771 wait complete. 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.335+0000 7fdc0f9d3700 1 Processor -- start 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 -- start start 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 0x7fdc080c4670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08164860 0x7fdc080c4b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080af770 con 0x7fdc080aa1c0 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080aee40 con 0x7fdc08154610 2023-11-28T03:07:02.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.336+0000 7fdc0f9d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc080af120 con 0x7fdc08164860 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43970/0 (socket says 172.21.15.73:43970) 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 -- 172.21.15.73:0/1293942658 learned_addr learned my addr 172.21.15.73:0/1293942658 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0cf6e700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 0x7fdc080c4670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0df70700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08164860 0x7fdc080c4b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 -- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08164860 msgr2=0x7fdc080c4b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08164860 0x7fdc080c4b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 -- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 msgr2=0x7fdc080c4670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 0x7fdc080c4670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 -- 172.21.15.73:0/1293942658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc00009ae0 con 0x7fdc080aa1c0 2023-11-28T03:07:02.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0cf6e700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 0x7fdc080c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0d76f700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fdc0400d890 tx=0x7fdc04008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc040125f0 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc04012750 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc04012ad0 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc080af400 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.337+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc08141140 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.338+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc0806c860 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.338+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fdc04013480 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.339+0000 7fdbfd7fa700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 0x7fdbec061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.339+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7fdc0407a290 con 0x7fdc080aa1c0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.339+0000 7fdc0cf6e700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 0x7fdbec061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.339+0000 7fdc0cf6e700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 0x7fdbec061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdc080c5770 tx=0x7fdbf8008040).ready entity=mgr.5970 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.343+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdc0404b260 con 0x7fdc080aa1c0 2023-11-28T03:07:02.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.531+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc080af8b0 con 0x7fdbec05f2c0 2023-11-28T03:07:02.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.535+0000 7fdbfd7fa700 1 -- 172.21.15.73:0/1293942658 <== mgr.5970 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fdbf8009920 con 0x7fdbec05f2c0 2023-11-28T03:07:02.536 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path snap_test_dir1 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 msgr2=0x7fdbec061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 0x7fdbec061780 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fdc080c5770 tx=0x7fdbf8008040).stop 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 msgr2=0x7fdc08140c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 rx=0x7fdc0400d890 tx=0x7fdc04008040).stop 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 shutdown_connections 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdbec05f2c0 0x7fdbec061780 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdc08164860 0x7fdc080c4b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdc080aa1c0 0x7fdc08140c50 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 --2- 172.21.15.73:0/1293942658 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdc08154610 0x7fdc080c4670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.536+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 >> 172.21.15.73:0/1293942658 conn(0x7fdc080655a0 msgr2=0x7fdc0814da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.537+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 shutdown_connections 2023-11-28T03:07:02.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.537+0000 7fdc0f9d3700 1 -- 172.21.15.73:0/1293942658 wait complete. 2023-11-28T03:07:02.549 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-28T03:07:02.549 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:02.549 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:07:02.564 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:07:02.565 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_07_00_UTC 2023-11-28T03:07:02.565 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_06_00_UTC 2023-11-28T03:07:02.565 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:02.565 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_06_00_UTC) 2023-11-28T03:07:02.623 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_07_00_UTC 2023-11-28T03:07:02.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:02.624 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_07_00_UTC) 2023-11-28T03:07:02.682 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:02.683 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T03:07:02.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:07:02.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.893+0000 7fc1028a1700 1 Processor -- start 2023-11-28T03:07:02.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.893+0000 7fc1028a1700 1 -- start start 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc1028a1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0fc1560e0 0x7fc0fc156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc1028a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc1564c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc1028a1700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0fc062810 con 0x7fc0fc0638b0 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc1028a1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0fc0670e0 con 0x7fc0fc1560e0 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc1028a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc062af0 con 0x7fc0fc0650a0 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc0fbfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0fc1560e0 0x7fc0fc156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc0fbfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc0fc1560e0 0x7fc0fc156cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43986/0 (socket says 172.21.15.73:43986) 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.894+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2705997010 learned_addr learned my addr 172.21.15.73:0/2705997010 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:02.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2705997010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 msgr2=0x7fc0fc1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0fbfff700 1 --2- 172.21.15.73:0/2705997010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc1564c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2705997010 >> v1:172.21.15.150:6789/0 conn(0x7fc0fc0638b0 legacy=0x7fc0fc0bcc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2705997010 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0fc062dd0 con 0x7fc0fc1560e0 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0fbfff700 1 --2- 172.21.15.73:0/2705997010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc156cb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fc0ec009810 tx=0x7fc0ec009060).ready entity=mon.0 client_cookie=8656f102527805eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0faffd700 1 -- 172.21.15.73:0/2705997010 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec005880 con 0x7fc0fc1560e0 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0faffd700 1 -- 172.21.15.73:0/2705997010 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec0059e0 con 0x7fc0fc1560e0 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc0faffd700 1 -- 172.21.15.73:0/2705997010 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec005d60 con 0x7fc0fc1560e0 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 -- 172.21.15.73:0/2705997010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc1560e0 msgr2=0x7fc0fc156cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2705997010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc156cb0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 rx=0x7fc0ec009810 tx=0x7fc0ec009060).stop 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 -- 172.21.15.73:0/2705997010 shutdown_connections 2023-11-28T03:07:02.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2705997010 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc156cb0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2705997010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc1564c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:02.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 -- 172.21.15.73:0/2705997010 >> 172.21.15.73:0/2705997010 conn(0x7fc0fc0c4200 msgr2=0x7fc0fc0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:02.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.895+0000 7fc1028a1700 1 -- 172.21.15.73:0/2705997010 shutdown_connections 2023-11-28T03:07:02.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.896+0000 7fc1028a1700 1 -- 172.21.15.73:0/2705997010 wait complete. 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.897+0000 7fc1028a1700 1 Processor -- start 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.897+0000 7fc1028a1700 1 -- start start 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.897+0000 7fc1028a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.897+0000 7fc1028a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc0acc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.897+0000 7fc1028a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc0ad170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc1028a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc0aafd0 con 0x7fc0fc0638b0 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc1028a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc14f650 con 0x7fc0fc1560e0 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc1028a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0fc14f930 con 0x7fc0fc0650a0 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:43990/0 (socket says 172.21.15.73:43990) 2023-11-28T03:07:02.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2289678457 learned_addr learned my addr 172.21.15.73:0/2289678457 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fb7fe700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc0acc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc100e3e700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc0ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 msgr2=0x7fc0fc0acc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc0acc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc1560e0 msgr2=0x7fc0fc0ad170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc0ad170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 -- 172.21.15.73:0/2289678457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0ec009300 con 0x7fc0fc0638b0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0fbfff700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fc0fc064810 tx=0x7fc0ec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec005880 con 0x7fc0fc0638b0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ec0068d0 con 0x7fc0fc0638b0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ec00ba60 con 0x7fc0fc0638b0 2023-11-28T03:07:02.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0fc0aac60 con 0x7fc0fc0638b0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.898+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0fc0a8150 con 0x7fc0fc0638b0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.899+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0fc0ab2b0 con 0x7fc0fc0638b0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.899+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 20) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fc0ec018070 con 0x7fc0fc0638b0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.900+0000 7fc0e3fff700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 0x7fc0e8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.900+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7fc0ec07b4a0 con 0x7fc0fc0638b0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.900+0000 7fc0fb7fe700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 0x7fc0e8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.900+0000 7fc0fb7fe700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 0x7fc0e8061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc0fc0c4b20 tx=0x7fc0f0008040).ready entity=mgr.5970 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:02.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:02.904+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc0ec04c4d0 con 0x7fc0fc0638b0 2023-11-28T03:07:03.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.110+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc0fc06c860 con 0x7fc0fc0638b0 2023-11-28T03:07:03.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.247+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc0ec078fd0 con 0x7fc0fc0638b0 2023-11-28T03:07:03.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.248+0000 7fc0e3fff700 1 -- 172.21.15.73:0/2289678457 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v21) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc0ec041960 con 0x7fc0fc0638b0 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 msgr2=0x7fc0e8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 0x7fc0e8061780 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fc0fc0c4b20 tx=0x7fc0f0008040).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 msgr2=0x7fc0fc14edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 rx=0x7fc0fc064810 tx=0x7fc0ec016000).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 shutdown_connections 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc0e805f2c0 0x7fc0e8061780 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0fc1560e0 0x7fc0fc0ad170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc0fc0638b0 0x7fc0fc14edc0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 --2- 172.21.15.73:0/2289678457 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0fc0650a0 0x7fc0fc0acc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.250+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 >> 172.21.15.73:0/2289678457 conn(0x7fc0fc0c4200 msgr2=0x7fc0fc0c0810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:03.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.251+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 shutdown_connections 2023-11-28T03:07:03.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.251+0000 7fc1028a1700 1 -- 172.21.15.73:0/2289678457 wait complete. 2023-11-28T03:07:03.267 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:07:03.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:03.304 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:03.345 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:07:03.346 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:07:03.346 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:03.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:03.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:07:03.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:03.389 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:07:03.393 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:03.393+0000 7fdf63a05680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:03.396 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:03.395+0000 7f3394da2680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:03.410 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:03.457 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:03.456+0000 7fdf63a05680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:03.461 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:03.461+0000 7f3394da2680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:03.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.568+0000 7f2421944700 1 Processor -- start 2023-11-28T03:07:03.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.568+0000 7f2421944700 1 -- start start 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f241c0bc320 0x7f241c069900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f241c069df0 0x7f241c1544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c0c28c0 con 0x7f241c069df0 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f241c06a240 con 0x7f241c0bc320 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f2421944700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c1552e0 con 0x7f241c150890 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f241b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f241c069df0 0x7f241c1544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f241b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f241c069df0 0x7f241c1544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47424/0 (socket says 172.21.15.73:47424) 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f241b7fe700 1 -- 172.21.15.73:0/4286784885 learned_addr learned my addr 172.21.15.73:0/4286784885 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f241a7fc700 1 --2- 172.21.15.73:0/4286784885 >> v2:172.21.15.150:3300/0 conn(0x7f241c0bc320 0x7f241c069900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.569+0000 7f241affd700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 -- 172.21.15.73:0/4286784885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 msgr2=0x7f241c0bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 -- 172.21.15.73:0/4286784885 >> v2:172.21.15.150:3300/0 conn(0x7f241c0bc320 msgr2=0x7f241c069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 --2- 172.21.15.73:0/4286784885 >> v2:172.21.15.150:3300/0 conn(0x7f241c0bc320 0x7f241c069900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 -- 172.21.15.73:0/4286784885 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f241c06a4b0 con 0x7f241c069df0 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f241b7fe700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1544f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f241c0685e0 tx=0x7f2414009840).ready entity=mon.0 client_cookie=55a364095d6d25b0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f2419ffb700 1 -- 172.21.15.73:0/4286784885 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400e070 con 0x7f241c069df0 2023-11-28T03:07:03.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f2419ffb700 1 -- 172.21.15.73:0/4286784885 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241400b970 con 0x7f241c069df0 2023-11-28T03:07:03.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f2419ffb700 1 -- 172.21.15.73:0/4286784885 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241400bcf0 con 0x7f241c069df0 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f2421944700 1 -- 172.21.15.73:0/4286784885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 msgr2=0x7f241c1544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.570+0000 7f2421944700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1544f0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 rx=0x7f241c0685e0 tx=0x7f2414009840).stop 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 -- 172.21.15.73:0/4286784885 shutdown_connections 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1544f0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 --2- 172.21.15.73:0/4286784885 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 --2- 172.21.15.73:0/4286784885 >> v2:172.21.15.150:3300/0 conn(0x7f241c0bc320 0x7f241c069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 -- 172.21.15.73:0/4286784885 >> 172.21.15.73:0/4286784885 conn(0x7f241c0631d0 msgr2=0x7f241c0c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:03.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 -- 172.21.15.73:0/4286784885 shutdown_connections 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.571+0000 7f2421944700 1 -- 172.21.15.73:0/4286784885 wait complete. 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.572+0000 7f2421944700 1 Processor -- start 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.572+0000 7f2421944700 1 -- start start 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f241c0bc320 0x7f241c152d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c147990 con 0x7f241c069df0 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f241c0a7000 con 0x7f241c0bc320 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f241c153700 con 0x7f241c150890 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47430/0 (socket says 172.21.15.73:47430) 2023-11-28T03:07:03.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 -- 172.21.15.73:0/4175583311 learned_addr learned my addr 172.21.15.73:0/4175583311 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241a7fc700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f241c0bc320 0x7f241c152d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241b7fe700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 msgr2=0x7f241c0c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f241c0bc320 msgr2=0x7f241c152d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f241c0bc320 0x7f241c152d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 -- 172.21.15.73:0/4175583311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2414009ae0 con 0x7f241c069df0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f241affd700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f241000e950 tx=0x7f2410009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410009530 con 0x7f241c069df0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2410012600 con 0x7f241c069df0 2023-11-28T03:07:03.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f241c1539e0 con 0x7f241c069df0 2023-11-28T03:07:03.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2410012980 con 0x7f241c069df0 2023-11-28T03:07:03.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.573+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f241c0a6c90 con 0x7f241c069df0 2023-11-28T03:07:03.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.574+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f241c06c860 con 0x7f241c069df0 2023-11-28T03:07:03.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.575+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2410012d30 con 0x7f241c069df0 2023-11-28T03:07:03.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.575+0000 7f2402ffd700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 0x7f2408061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.575+0000 7f241a7fc700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 msgr2=0x7f2408061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:03.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.575+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f241007aa00 con 0x7f241c069df0 2023-11-28T03:07:03.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.575+0000 7f241a7fc700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 0x7f2408061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:03.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.579+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f241004ba30 con 0x7f241c069df0 2023-11-28T03:07:03.731 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:03.729+0000 7fdf63a05680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:03.750 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:03.749+0000 7f3394da2680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.776+0000 7f241a7fc700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 msgr2=0x7f2408061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:03.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.776+0000 7f241a7fc700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 0x7f2408061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:03.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.790+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f241c0c1f90 con 0x7f241c069df0 2023-11-28T03:07:03.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.791+0000 7f2402ffd700 1 -- 172.21.15.73:0/4175583311 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f241004f050 con 0x7f241c069df0 2023-11-28T03:07:03.792 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:03.792 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:07:03.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 msgr2=0x7f2408061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:03.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 0x7f2408061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 msgr2=0x7f241c1474a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 rx=0x7f241000e950 tx=0x7f2410009060).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 shutdown_connections 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f240805f270 0x7f2408061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f241c150890 0x7f241c0c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f241c069df0 0x7f241c1474a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 --2- 172.21.15.73:0/4175583311 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f241c0bc320 0x7f241c152d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 >> 172.21.15.73:0/4175583311 conn(0x7f241c0631d0 msgr2=0x7f241c0c2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 shutdown_connections 2023-11-28T03:07:03.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.793+0000 7f2421944700 1 -- 172.21.15.73:0/4175583311 wait complete. 2023-11-28T03:07:03.796 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:07:03.807 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:03.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.962+0000 7f8169a1a700 1 Processor -- start 2023-11-28T03:07:03.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.962+0000 7f8169a1a700 1 -- start start 2023-11-28T03:07:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8169a1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f816406b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8169a1a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 0x7f8164154ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8169a1a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640664a0 con 0x7f8164155190 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8169a1a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8164155640 con 0x7f81640c3210 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8169a1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8164065000 con 0x7f816414eb80 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f81637fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8164155190 0x7f81641516a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42542/0 (socket says 172.21.15.73:42542) 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f81637fe700 1 -- 172.21.15.73:0/707233870 learned_addr learned my addr 172.21.15.73:0/707233870 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8162ffd700 1 --2- 172.21.15.73:0/707233870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f816406b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f81627fc700 1 --2- 172.21.15.73:0/707233870 >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 0x7f8164154ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3729259833 0 0) 0x7f81640664a0 con 0x7f8164155190 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814c0033f0 con 0x7f8164155190 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8162ffd700 1 -- 172.21.15.73:0/707233870 >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 msgr2=0x7f8164154ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8162ffd700 1 --2- 172.21.15.73:0/707233870 >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 0x7f8164154ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4235947922 0 0) 0x7f814c0033f0 con 0x7f8164155190 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8162ffd700 1 -- 172.21.15.73:0/707233870 >> v1:172.21.15.73:6789/0 conn(0x7f8164155190 legacy=0x7f81641516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f8162ffd700 1 -- 172.21.15.73:0/707233870 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f816406bbd0 con 0x7f816414eb80 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.963+0000 7f81627fc700 1 --2- 172.21.15.73:0/707233870 >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 0x7f8164154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8162ffd700 1 --2- 172.21.15.73:0/707233870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f816406b6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8158009760 tx=0x7f8158009060).ready entity=mon.2 client_cookie=acf9eb4d63a96b86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0017f0 con 0x7f816414eb80 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8158011400 con 0x7f816414eb80 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8161ffb700 1 -- 172.21.15.73:0/707233870 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158011780 con 0x7f816414eb80 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 -- 172.21.15.73:0/707233870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 msgr2=0x7f816406b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 --2- 172.21.15.73:0/707233870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f816406b6e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 rx=0x7f8158009760 tx=0x7f8158009060).stop 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 -- 172.21.15.73:0/707233870 shutdown_connections 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 --2- 172.21.15.73:0/707233870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f816406b6e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 --2- 172.21.15.73:0/707233870 >> v2:172.21.15.150:3300/0 conn(0x7f81640c3210 0x7f8164154ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 -- 172.21.15.73:0/707233870 >> 172.21.15.73:0/707233870 conn(0x7f81640c3f20 msgr2=0x7f81640c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:03.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 -- 172.21.15.73:0/707233870 shutdown_connections 2023-11-28T03:07:03.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.964+0000 7f8169a1a700 1 -- 172.21.15.73:0/707233870 wait complete. 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 Processor -- start 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 -- start start 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f8164062d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8164155190 0x7f816414ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640a79d0 con 0x7f81640c3210 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640aa780 con 0x7f8164155190 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.966+0000 7f8169a1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81640acba0 con 0x7f816414eb80 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47442/0 (socket says 172.21.15.73:47442) 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 -- 172.21.15.73:0/1974295516 learned_addr learned my addr 172.21.15.73:0/1974295516 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f81627fc700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f8164062d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f81637fe700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8164155190 0x7f816414ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 -- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 msgr2=0x7f8164062d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f8164062d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 -- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8164155190 msgr2=0x7f816414ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8164155190 0x7f816414ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 -- 172.21.15.73:0/1974295516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81540092a0 con 0x7f81640c3210 2023-11-28T03:07:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8162ffd700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f8158009d50 tx=0x7f8158016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158021070 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81580106c0 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81580092a0 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158010a40 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.967+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81640ad160 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.968+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f815801d430 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.969+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816406c7c0 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.969+0000 7f814affd700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 0x7f8150061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.969+0000 7f81627fc700 1 -- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 msgr2=0x7f8150061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.969+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f815807b540 con 0x7f81640c3210 2023-11-28T03:07:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.969+0000 7f81627fc700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 0x7f8150061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:03.973+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f815804c5c0 con 0x7f81640c3210 2023-11-28T03:07:04.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.143+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8164151140 con 0x7f81640c3210 2023-11-28T03:07:04.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.144+0000 7f814affd700 1 -- 172.21.15.73:0/1974295516 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v68) v1 ==== 74+0+17357 (secure 0 0 0) 0x7f815804fbe0 con 0x7f81640c3210 2023-11-28T03:07:04.145 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:04.146 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":68,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:06:18.268123+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"65","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/59040213":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6833/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6832/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:0/1205165186":"2023-11-29T03:05:16.454932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:04.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 msgr2=0x7f8150061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:04.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 0x7f8150061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 msgr2=0x7f81640a6f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 rx=0x7f8158009d50 tx=0x7f8158016000).stop 2023-11-28T03:07:04.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 shutdown_connections 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f815005f2a0 0x7f8150061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8164155190 0x7f816414ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81640c3210 0x7f81640a6f60 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 --2- 172.21.15.73:0/1974295516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f816414eb80 0x7f8164062d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 >> 172.21.15.73:0/1974295516 conn(0x7f81640c3f20 msgr2=0x7f81640c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 shutdown_connections 2023-11-28T03:07:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.147+0000 7f8169a1a700 1 -- 172.21.15.73:0/1974295516 wait complete. 2023-11-28T03:07:04.165 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '68', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 7, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:04:58.509951+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '64', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 8, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:04:59.750391+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '65', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:07:04.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:07:04.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.285+0000 7faffcb91700 1 Processor -- start 2023-11-28T03:07:04.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.285+0000 7faffcb91700 1 -- start start 2023-11-28T03:07:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faffcb91700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faff8155ae0 0x7faff80bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faffcb91700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80bdc30 0x7faff80c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faffcb91700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faff8155650 con 0x7faff80c6b30 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faffcb91700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faff814bff0 con 0x7faff8155ae0 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faffcb91700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff80be0c0 con 0x7faff80bdc30 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faff8155ae0 0x7faff80bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff659c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faff80c6b30 0x7faff81524b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42552/0 (socket says 172.21.15.73:42552) 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faff8155ae0 0x7faff80bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47456/0 (socket says 172.21.15.73:47456) 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff659c700 1 -- 172.21.15.73:0/845992109 learned_addr learned my addr 172.21.15.73:0/845992109 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff6d9d700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80bdc30 0x7faff80c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff5d9b700 1 -- 172.21.15.73:0/845992109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80bdc30 msgr2=0x7faff80c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.286+0000 7faff5d9b700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80bdc30 0x7faff80c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff5d9b700 1 -- 172.21.15.73:0/845992109 >> v1:172.21.15.73:6789/0 conn(0x7faff80c6b30 legacy=0x7faff81524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff5d9b700 1 -- 172.21.15.73:0/845992109 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faff8155f60 con 0x7faff8155ae0 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff559a700 1 -- 172.21.15.73:0/845992109 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 193465854 0 0) 0x7faff8155650 con 0x7faff80c6b30 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff5d9b700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff80bd740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fafe8006fd0 tx=0x7fafe800b270).ready entity=mon.0 client_cookie=282b2969b2ae73b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff559a700 1 -- 172.21.15.73:0/845992109 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe800da70 con 0x7faff8155ae0 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff559a700 1 -- 172.21.15.73:0/845992109 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafe800dbd0 con 0x7faff8155ae0 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faff559a700 1 -- 172.21.15.73:0/845992109 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafe8005880 con 0x7faff8155ae0 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 -- 172.21.15.73:0/845992109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 msgr2=0x7faff80bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff80bd740 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 rx=0x7fafe8006fd0 tx=0x7fafe800b270).stop 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 -- 172.21.15.73:0/845992109 shutdown_connections 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80bdc30 0x7faff80c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 --2- 172.21.15.73:0/845992109 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff80bd740 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.287+0000 7faffcb91700 1 -- 172.21.15.73:0/845992109 >> 172.21.15.73:0/845992109 conn(0x7faff80c8230 msgr2=0x7faff80c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.288+0000 7faffcb91700 1 -- 172.21.15.73:0/845992109 shutdown_connections 2023-11-28T03:07:04.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.288+0000 7faffcb91700 1 -- 172.21.15.73:0/845992109 wait complete. 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 Processor -- start 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 -- start start 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff80bdc30 0x7faff814a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80c6b30 0x7faff8150550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff80ad7f0 con 0x7faff8155ae0 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faff8150f30 con 0x7faff80bdc30 2023-11-28T03:07:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faffcb91700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faff8165b00 con 0x7faff80c6b30 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faff6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faff6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47464/0 (socket says 172.21.15.73:47464) 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.289+0000 7faff6d9d700 1 -- 172.21.15.73:0/3257574121 learned_addr learned my addr 172.21.15.73:0/3257574121 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff659c700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff80bdc30 0x7faff814a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff5d9b700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80c6b30 0x7faff8150550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80c6b30 msgr2=0x7faff8150550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80c6b30 0x7faff8150550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff80bdc30 msgr2=0x7faff814a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff80bdc30 0x7faff814a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 -- 172.21.15.73:0/3257574121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafe800b4f0 con 0x7faff8155ae0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faff6d9d700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7faff0009e90 tx=0x7faff000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000cb50 con 0x7faff8155ae0 2023-11-28T03:07:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff000f040 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faff8164b00 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff000cec0 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.290+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faff8164dc0 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.291+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faff806c860 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.291+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7faff0013610 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.292+0000 7fafe67fc700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 0x7fafd8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.292+0000 7faff659c700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 msgr2=0x7fafd8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.292+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7faff007ac40 con 0x7faff8155ae0 2023-11-28T03:07:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.292+0000 7faff659c700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 0x7fafd8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:04.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.296+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faff0080020 con 0x7faff8155ae0 2023-11-28T03:07:04.343 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:04.342+0000 7fdf63a05680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:04.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:04.398+0000 7f3394da2680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:04.415 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:04.414+0000 7fdf63a05680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:04.472 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:04.471+0000 7f3394da2680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:04.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.493+0000 7faff659c700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 msgr2=0x7fafd8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:04.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.493+0000 7faff659c700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 0x7fafd8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.498+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7faff8164b00 con 0x7faff8155ae0 2023-11-28T03:07:04.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.498+0000 7fafe67fc700 1 -- 172.21.15.73:0/3257574121 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v68) v1 ==== 110+0+10 (secure 0 0 0) 0x7faff8164b00 con 0x7faff8155ae0 2023-11-28T03:07:04.499 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 msgr2=0x7fafd8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 0x7fafd8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 msgr2=0x7faff8150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 rx=0x7faff0009e90 tx=0x7faff000b040).stop 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 shutdown_connections 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafd805f270 0x7fafd8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faff8155ae0 0x7faff8150a40 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faff80bdc30 0x7faff814a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 --2- 172.21.15.73:0/3257574121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faff80c6b30 0x7faff8150550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.500+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 >> 172.21.15.73:0/3257574121 conn(0x7faff80c8230 msgr2=0x7faff8066470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.501+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 shutdown_connections 2023-11-28T03:07:04.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.501+0000 7faffcb91700 1 -- 172.21.15.73:0/3257574121 wait complete. 2023-11-28T03:07:04.514 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:07:04.601 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:04.600+0000 7fdf63a05680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.634+0000 7fccb111a700 1 Processor -- start 2023-11-28T03:07:04.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.634+0000 7fccb111a700 1 -- start start 2023-11-28T03:07:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb111a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccac0bcb40 0x7fccac062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb111a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 0x7fccac15e230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb111a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccac063a10 con 0x7fccac0bc760 2023-11-28T03:07:04.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb111a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccac062bf0 con 0x7fccac0bcb40 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb111a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac06bbe0 con 0x7fccac1587b0 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccac0bcb40 0x7fccac062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fccac0bcb40 0x7fccac062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47472/0 (socket says 172.21.15.73:47472) 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccabfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fccac0bc760 0x7fccac1580a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42554/0 (socket says 172.21.15.73:42554) 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 -- 172.21.15.73:0/3538570899 learned_addr learned my addr 172.21.15.73:0/3538570899 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb0919700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 0x7fccac15e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 -- 172.21.15.73:0/3538570899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 msgr2=0x7fccac15e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 0x7fccac15e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 -- 172.21.15.73:0/3538570899 >> v1:172.21.15.73:6789/0 conn(0x7fccac0bc760 legacy=0x7fccac1580a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 -- 172.21.15.73:0/3538570899 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccac062e00 con 0x7fccac0bcb40 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccb0919700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 0x7fccac15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:04.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccab7fe700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac0bcb40 0x7fccac062700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fcc9c006fd0 tx=0x7fcc9c0090f0).ready entity=mon.0 client_cookie=6a5b4d95f719e4b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccaaffd700 1 -- 172.21.15.73:0/3538570899 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00b600 con 0x7fccac0bcb40 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccaaffd700 1 -- 172.21.15.73:0/3538570899 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc9c00b760 con 0x7fccac0bcb40 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.635+0000 7fccaaffd700 1 -- 172.21.15.73:0/3538570899 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc9c00bb00 con 0x7fccac0bcb40 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 -- 172.21.15.73:0/3538570899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac0bcb40 msgr2=0x7fccac062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac0bcb40 0x7fccac062700 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 rx=0x7fcc9c006fd0 tx=0x7fcc9c0090f0).stop 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 -- 172.21.15.73:0/3538570899 shutdown_connections 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac1587b0 0x7fccac15e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 --2- 172.21.15.73:0/3538570899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac0bcb40 0x7fccac062700 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 -- 172.21.15.73:0/3538570899 >> 172.21.15.73:0/3538570899 conn(0x7fccac0c4010 msgr2=0x7fccac0c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 -- 172.21.15.73:0/3538570899 shutdown_connections 2023-11-28T03:07:04.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.636+0000 7fccb111a700 1 -- 172.21.15.73:0/3538570899 wait complete. 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.637+0000 7fccb111a700 1 Processor -- start 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.637+0000 7fccb111a700 1 -- start start 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac0bc760 0x7fccac0c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccac0bcb40 0x7fccac066090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0c1b00 con 0x7fccac1587b0 2023-11-28T03:07:04.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0c1de0 con 0x7fccac0bcb40 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb111a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccac0c20c0 con 0x7fccac0bc760 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb0919700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb0919700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47476/0 (socket says 172.21.15.73:47476) 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccb0919700 1 -- 172.21.15.73:0/701515814 learned_addr learned my addr 172.21.15.73:0/701515814 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccabfff700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac0bc760 0x7fccac0c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.638+0000 7fccab7fe700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccac0bcb40 0x7fccac066090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 -- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac0bc760 msgr2=0x7fccac0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac0bc760 0x7fccac0c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 -- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccac0bcb40 msgr2=0x7fccac066090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccac0bcb40 0x7fccac066090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 -- 172.21.15.73:0/701515814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc9c009370 con 0x7fccac1587b0 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb0919700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fcca4009fc0 tx=0x7fcca40099b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca400dbe0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcca400f040 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcca40133f0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccac0c23a0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.639+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccac0c2960 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.640+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccac06c7c0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.640+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fcca40137e0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.641+0000 7fcc93fff700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 0x7fcc98061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.641+0000 7fccabfff700 1 -- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 msgr2=0x7fcc98061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.641+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7fcca407aab0 con 0x7fccac1587b0 2023-11-28T03:07:04.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.641+0000 7fccabfff700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 0x7fcc98061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:04.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.648+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcca404bae0 con 0x7fccac1587b0 2023-11-28T03:07:04.671 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:04.671+0000 7f3394da2680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:04.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.817+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fccac0c2b50 con 0x7fccac1587b0 2023-11-28T03:07:04.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.818+0000 7fcc93fff700 1 -- 172.21.15.73:0/701515814 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v68) v1 ==== 92+0+10 (secure 0 0 0) 0x7fcca407ae60 con 0x7fccac1587b0 2023-11-28T03:07:04.819 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 msgr2=0x7fcc98061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 0x7fcc98061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 msgr2=0x7fccac0c0af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 rx=0x7fcca4009fc0 tx=0x7fcca40099b0).stop 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 shutdown_connections 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcc9805f240 0x7fcc98061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccac1587b0 0x7fccac0c0af0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccac0bc760 0x7fccac0c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 --2- 172.21.15.73:0/701515814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccac0bcb40 0x7fccac066090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.819+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 >> 172.21.15.73:0/701515814 conn(0x7fccac0c4010 msgr2=0x7fccac0c5aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.820+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 shutdown_connections 2023-11-28T03:07:04.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.820+0000 7fccb111a700 1 -- 172.21.15.73:0/701515814 wait complete. 2023-11-28T03:07:04.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:07:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 Processor -- start 2023-11-28T03:07:04.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 -- start start 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f62681589d0 0x7f62680c7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f62680c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6268154bc0 con 0x7f6268098e50 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6268099230 con 0x7f62681589d0 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.953+0000 7f626e66e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6268064540 con 0x7f62680640b0 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6267fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6268098e50 0x7f62681582c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42562/0 (socket says 172.21.15.73:42562) 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6267fff700 1 -- 172.21.15.73:0/2148897577 learned_addr learned my addr 172.21.15.73:0/2148897577 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:04.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f626cc0b700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f62680c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 --2- 172.21.15.73:0/2148897577 >> v2:172.21.15.150:3300/0 conn(0x7f62681589d0 0x7f62680c7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6266ffd700 1 -- 172.21.15.73:0/2148897577 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1232167707 0 0) 0x7f6258001290 con 0x7f6268098e50 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6266ffd700 1 -- 172.21.15.73:0/2148897577 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62500033f0 con 0x7f6268098e50 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 -- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 msgr2=0x7f62680c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f62680c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 -- 172.21.15.73:0/2148897577 >> v1:172.21.15.73:6789/0 conn(0x7f6268098e50 legacy=0x7f62681582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 -- 172.21.15.73:0/2148897577 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62680c6320 con 0x7f62681589d0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f626cc0b700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f62680c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f62677fe700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f62680c7620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f6268099370 tx=0x7f625c009060).ready entity=mon.1 client_cookie=9672902c9ce6d83a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6266ffd700 1 -- 172.21.15.73:0/2148897577 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c00d4a0 con 0x7f62681589d0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6266ffd700 1 -- 172.21.15.73:0/2148897577 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c00d600 con 0x7f62681589d0 2023-11-28T03:07:04.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.954+0000 7f6266ffd700 1 -- 172.21.15.73:0/2148897577 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c00d980 con 0x7f62681589d0 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 -- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 msgr2=0x7f62680c7620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f62680c7620 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 rx=0x7f6268099370 tx=0x7f625c009060).stop 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 -- 172.21.15.73:0/2148897577 shutdown_connections 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f62680c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 --2- 172.21.15.73:0/2148897577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f62680c7620 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 -- 172.21.15.73:0/2148897577 >> 172.21.15.73:0/2148897577 conn(0x7f62680c7db0 msgr2=0x7f62680c81c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 -- 172.21.15.73:0/2148897577 shutdown_connections 2023-11-28T03:07:04.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.955+0000 7f626e66e700 1 -- 172.21.15.73:0/2148897577 wait complete. 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.956+0000 7f626e66e700 1 Processor -- start 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.956+0000 7f626e66e700 1 -- start start 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f6268156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f6268144f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62681571a0 con 0x7f6268098e50 2023-11-28T03:07:04.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6268157fc0 con 0x7f62681589d0 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626e66e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62680a9440 con 0x7f62680640b0 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47480/0 (socket says 172.21.15.73:47480) 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 -- 172.21.15.73:0/3745155593 learned_addr learned my addr 172.21.15.73:0/3745155593 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f6267fff700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f6268156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f626cc0b700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f6268144f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 -- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 msgr2=0x7f6268156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f6268156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 -- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 msgr2=0x7f6268144f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:04.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f6268144f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f62677fe700 1 -- 172.21.15.73:0/3745155593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6260009a20 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.957+0000 7f6267fff700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f6268156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f62677fe700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f62680ae040 tx=0x7f625c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c000c90 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f625c00d770 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f625c0092a0 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f625c01d430 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.958+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6268157710 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.959+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626806c860 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.959+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f625c01d7a0 con 0x7f6268098e50 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.960+0000 7f624ffff700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 0x7f6254061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.960+0000 7f6267fff700 1 -- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 msgr2=0x7f6254061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:04.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.960+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6409+0+0 (secure 0 0 0) 0x7f625c07b320 con 0x7f6268098e50 2023-11-28T03:07:04.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.960+0000 7f6267fff700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 0x7f6254061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:04.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:04.964+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f625c04c400 con 0x7f6268098e50 2023-11-28T03:07:05.020 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.019+0000 7fdf63a05680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:05.110 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.109+0000 7f3394da2680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:05.123 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.122+0000 7fdf63a05680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:05.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.135+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62680c6580 con 0x7f6268098e50 2023-11-28T03:07:05.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.136+0000 7f624ffff700 1 -- 172.21.15.73:0/3745155593 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v68) v1 ==== 104+0+11 (secure 0 0 0) 0x7f625c022020 con 0x7f6268098e50 2023-11-28T03:07:05.137 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:07:05.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 msgr2=0x7f6254061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 0x7f6254061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 msgr2=0x7f62680a8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 rx=0x7f62680ae040 tx=0x7f625c016000).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 shutdown_connections 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f625405f270 0x7f6254061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62681589d0 0x7f6268144f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62680640b0 0x7f6268156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 --2- 172.21.15.73:0/3745155593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6268098e50 0x7f62680a8f50 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.137+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 >> 172.21.15.73:0/3745155593 conn(0x7f62680c7db0 msgr2=0x7f62680ad860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.138+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 shutdown_connections 2023-11-28T03:07:05.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.138+0000 7f626e66e700 1 -- 172.21.15.73:0/3745155593 wait complete. 2023-11-28T03:07:05.151 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:07:05.220 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.219+0000 7f3394da2680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:05.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.270+0000 7f7ccc1ac700 1 Processor -- start 2023-11-28T03:07:05.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.270+0000 7f7ccc1ac700 1 -- start start 2023-11-28T03:07:05.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7ccc1ac700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cc40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7ccc1ac700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc40c72d0 con 0x7f7cc40c6190 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7ccc1ac700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cc4064570 con 0x7f7cc4098d10 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7ccc1ac700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4154bc0 con 0x7f7cc4158a30 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7cc9747700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7cc40c6190 0x7f7cc4158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42570/0 (socket says 172.21.15.73:42570) 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7cc9747700 1 -- 172.21.15.73:0/3469108915 learned_addr learned my addr 172.21.15.73:0/3469108915 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.271+0000 7f7cca749700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cc40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3629353145 0 0) 0x7f7cc40c72d0 con 0x7f7cc40c6190 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cb00033f0 con 0x7f7cc40c6190 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3214534935 0 0) 0x7f7cc4064570 con 0x7f7cc4098d10 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7cc40c72d0 con 0x7f7cc4098d10 2023-11-28T03:07:05.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2916857222 0 0) 0x7f7cb00033f0 con 0x7f7cc40c6190 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 msgr2=0x7f7cc40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cc40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 >> v1:172.21.15.150:6789/0 conn(0x7f7cc4098d10 legacy=0x7f7cc4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40c66c0 con 0x7f7cc40c6190 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7cc0001470 con 0x7f7cc40c6190 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cca749700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cc40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 >> v1:172.21.15.73:6789/0 conn(0x7f7cc40c6190 legacy=0x7f7cc4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0003aa0 0x7f7cb0003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0004690 0x7f7cb0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cb0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc4154bc0 con 0x7f7cb0004690 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb000b910 con 0x7f7cb0003aa0 2023-11-28T03:07:05.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb000bad0 con 0x7f7cc4158a30 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9f48700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0003aa0 0x7f7cb0003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0004690 0x7f7cb0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cca749700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cb0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 -- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 msgr2=0x7f7cb0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc4158a30 0x7f7cb0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 -- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0003aa0 msgr2=0x7f7cb0003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0003aa0 0x7f7cb0003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.272+0000 7f7cc9747700 1 -- 172.21.15.73:0/3469108915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cbc009a20 con 0x7f7cb0004690 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7cc9747700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0004690 0x7f7cb0004b10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f7cc000c0e0 tx=0x7f7cc0001df0).ready entity=mon.0 client_cookie=945f86204541caa2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc000ca10 con 0x7f7cb0004690 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cc000cb70 con 0x7f7cb0004690 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7cc8f46700 1 -- 172.21.15.73:0/3469108915 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cc00070d0 con 0x7f7cb0004690 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/3469108915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0004690 msgr2=0x7f7cb0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/3469108915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0004690 0x7f7cb0004b10 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 rx=0x7f7cc000c0e0 tx=0x7f7cc0001df0).stop 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7cca749700 1 -- 172.21.15.73:0/3469108915 reap_dead start 2023-11-28T03:07:05.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/3469108915 shutdown_connections 2023-11-28T03:07:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/3469108915 >> 172.21.15.73:0/3469108915 conn(0x7f7cc40cb310 msgr2=0x7f7cc40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/3469108915 shutdown_connections 2023-11-28T03:07:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.273+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/3469108915 wait complete. 2023-11-28T03:07:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.274+0000 7f7ccc1ac700 1 Processor -- start 2023-11-28T03:07:05.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 -- start start 2023-11-28T03:07:05.280 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.279+0000 7fdf63a05680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0004690 0x7f7cc414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40aaba0 0x7f7cc4167dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc414be10 con 0x7f7cc41682c0 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40abfe0 con 0x7f7cb0004690 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7ccc1ac700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc40aa710 con 0x7f7cc40aaba0 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7cca749700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7cca749700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47494/0 (socket says 172.21.15.73:47494) 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7cca749700 1 -- 172.21.15.73:0/389598998 learned_addr learned my addr 172.21.15.73:0/389598998 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:05.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7cc9f48700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0004690 0x7f7cc414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.275+0000 7f7cc9747700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40aaba0 0x7f7cc4167dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 -- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40aaba0 msgr2=0x7f7cc4167dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40aaba0 0x7f7cc4167dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 -- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0004690 msgr2=0x7f7cc414b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0004690 0x7f7cc414b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 -- 172.21.15.73:0/389598998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc40aa100 con 0x7f7cc41682c0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cca749700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f7cbc009f00 tx=0x7f7cbc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00e040 con 0x7f7cc41682c0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cbc00f040 con 0x7f7cc41682c0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cc4168870 con 0x7f7cc41682c0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc00fe50 con 0x7f7cc41682c0 2023-11-28T03:07:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.276+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc40aa3a0 con 0x7f7cc41682c0 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.277+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc40aa8a0 con 0x7f7cc41682c0 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.278+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7cbc015660 con 0x7f7cc41682c0 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.278+0000 7f7cb9ffb700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 0x7f7ca8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.278+0000 7f7cc9f48700 1 -- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 msgr2=0x7f7ca8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.278+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f7cbc07b120 con 0x7f7cc41682c0 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.278+0000 7f7cc9f48700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 0x7f7ca8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.282+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7cbc04c110 con 0x7f7cc41682c0 2023-11-28T03:07:05.356 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.355+0000 7fdf63a05680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:05.384 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.383+0000 7f3394da2680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:05.425 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.424+0000 7fdf63a05680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:05.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.457+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7cc406c860 con 0x7f7cc41682c0 2023-11-28T03:07:05.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.458+0000 7f7cb9ffb700 1 -- 172.21.15.73:0/389598998 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v69) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7cbc04f730 con 0x7f7cc41682c0 2023-11-28T03:07:05.459 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 msgr2=0x7f7ca8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 0x7f7ca8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 msgr2=0x7f7cc40a91f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 rx=0x7f7cbc009f00 tx=0x7f7cbc00b040).stop 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 shutdown_connections 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7ca805f270 0x7f7ca8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cc41682c0 0x7f7cc40a91f0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0004690 0x7f7cc414b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 --2- 172.21.15.73:0/389598998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cc40aaba0 0x7f7cc4167dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 >> 172.21.15.73:0/389598998 conn(0x7f7cc40cb310 msgr2=0x7f7cc4153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 shutdown_connections 2023-11-28T03:07:05.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f7ccc1ac700 1 -- 172.21.15.73:0/389598998 wait complete. 2023-11-28T03:07:05.462 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.460+0000 7f3394da2680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:05.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:05.533 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.532+0000 7f3394da2680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:05.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.594+0000 7fbb48f4c700 1 Processor -- start 2023-11-28T03:07:05.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.594+0000 7fbb48f4c700 1 -- start start 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb48f4c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb440c45d0 0x7fbb440c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb48f4c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbb44068090 0x7fbb440c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb48f4c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb44154d20 con 0x7fbb44067cb0 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb48f4c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb44151670 con 0x7fbb440c45d0 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb48f4c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb440a7490 con 0x7fbb44068090 2023-11-28T03:07:05.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb41d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb440c45d0 0x7fbb440c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.595+0000 7fbb4259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbb44067cb0 0x7fbb440c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42584/0 (socket says 172.21.15.73:42584) 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb4259c700 1 -- 172.21.15.73:0/3892961492 learned_addr learned my addr 172.21.15.73:0/3892961492 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb42d9d700 1 --2- 172.21.15.73:0/3892961492 >> v2:172.21.15.150:3300/0 conn(0x7fbb44068090 0x7fbb440c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb41d9b700 1 -- 172.21.15.73:0/3892961492 >> v2:172.21.15.150:3300/0 conn(0x7fbb44068090 msgr2=0x7fbb440c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb41d9b700 1 --2- 172.21.15.73:0/3892961492 >> v2:172.21.15.150:3300/0 conn(0x7fbb44068090 0x7fbb440c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb41d9b700 1 -- 172.21.15.73:0/3892961492 >> v1:172.21.15.73:6789/0 conn(0x7fbb44067cb0 legacy=0x7fbb440c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb4159a700 1 -- 172.21.15.73:0/3892961492 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469610432 0 0) 0x7fbb44154d20 con 0x7fbb44067cb0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb41d9b700 1 -- 172.21.15.73:0/3892961492 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb441555c0 con 0x7fbb440c45d0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb41d9b700 1 --2- 172.21.15.73:0/3892961492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb440c45d0 0x7fbb440c2640 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fbb44154c50 tx=0x7fbb34009060).ready entity=mon.0 client_cookie=ff6af4390fe7dd41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb4159a700 1 -- 172.21.15.73:0/3892961492 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400f510 con 0x7fbb440c45d0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb4159a700 1 -- 172.21.15.73:0/3892961492 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3400f670 con 0x7fbb440c45d0 2023-11-28T03:07:05.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.596+0000 7fbb4159a700 1 -- 172.21.15.73:0/3892961492 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400f9f0 con 0x7fbb440c45d0 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 -- 172.21.15.73:0/3892961492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb440c45d0 msgr2=0x7fbb440c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/3892961492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb440c45d0 0x7fbb440c2640 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 rx=0x7fbb44154c50 tx=0x7fbb34009060).stop 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 -- 172.21.15.73:0/3892961492 shutdown_connections 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/3892961492 >> v2:172.21.15.150:3300/0 conn(0x7fbb44068090 0x7fbb440c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/3892961492 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb440c45d0 0x7fbb440c2640 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 -- 172.21.15.73:0/3892961492 >> 172.21.15.73:0/3892961492 conn(0x7fbb440c7fd0 msgr2=0x7fbb440c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 -- 172.21.15.73:0/3892961492 shutdown_connections 2023-11-28T03:07:05.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.597+0000 7fbb48f4c700 1 -- 172.21.15.73:0/3892961492 wait complete. 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.598+0000 7fbb48f4c700 1 Processor -- start 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.598+0000 7fbb48f4c700 1 -- start start 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 0x7fbb4414cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb440c45d0 0x7fbb44065ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb441526d0 con 0x7fbb44068090 2023-11-28T03:07:05.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44164e50 con 0x7fbb44067cb0 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb48f4c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb44165130 con 0x7fbb440c45d0 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47520/0 (socket says 172.21.15.73:47520) 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 -- 172.21.15.73:0/2933102137 learned_addr learned my addr 172.21.15.73:0/2933102137 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb4259c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 0x7fbb4414cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb42d9d700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb440c45d0 0x7fbb44065ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb440c45d0 msgr2=0x7fbb44065ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb440c45d0 0x7fbb44065ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 msgr2=0x7fbb4414cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 0x7fbb4414cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 -- 172.21.15.73:0/2933102137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb340092a0 con 0x7fbb44068090 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb4259c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 0x7fbb4414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:05.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb41d9b700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fbb440c5b90 tx=0x7fbb34017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.599+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3400fe80 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.600+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb3400f7e0 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.600+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb3401b670 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.600+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb44165410 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.600+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb44164a70 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.600+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb44152860 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.601+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fbb3401ba20 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.601+0000 7fbb327fc700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 0x7fbb24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.602+0000 7fbb4259c700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 msgr2=0x7fbb24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.602+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7fbb3407bab0 con 0x7fbb44068090 2023-11-28T03:07:05.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.602+0000 7fbb4259c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 0x7fbb24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:05.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.606+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbb44152860 con 0x7fbb44068090 2023-11-28T03:07:05.690 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.689+0000 7fdf63a05680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:05.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.803+0000 7fbb4259c700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 msgr2=0x7fbb24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:05.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.803+0000 7fbb4259c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 0x7fbb24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:05.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:05.808+0000 7f3394da2680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:05.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.818+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbb44152860 con 0x7fbb44068090 2023-11-28T03:07:05.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.819+0000 7fbb327fc700 1 -- 172.21.15.73:0/2933102137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7fbb44152860 con 0x7fbb44068090 2023-11-28T03:07:05.820 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:05.820 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:07:05.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 msgr2=0x7fbb24061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:05.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 0x7fbb24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 msgr2=0x7fbb44154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 rx=0x7fbb440c5b90 tx=0x7fbb34017000).stop 2023-11-28T03:07:05.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 shutdown_connections 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb2405f270 0x7fbb24061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb440c45d0 0x7fbb44065ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb44067cb0 0x7fbb4414cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 --2- 172.21.15.73:0/2933102137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb44068090 0x7fbb44154330 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 >> 172.21.15.73:0/2933102137 conn(0x7fbb440c7fd0 msgr2=0x7fbb441507c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 shutdown_connections 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.821+0000 7fbb48f4c700 1 -- 172.21.15.73:0/2933102137 wait complete. 2023-11-28T03:07:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:07:05.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:05.872 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:05.872+0000 7fdf63a05680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:05.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.953+0000 7fda48a3e700 1 Processor -- start 2023-11-28T03:07:05.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.953+0000 7fda48a3e700 1 -- start start 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.954+0000 7fda48a3e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda441544d0 0x7fda44155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.954+0000 7fda48a3e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 0x7fda4415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.954+0000 7fda48a3e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda440683d0 con 0x7fda44066f40 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.954+0000 7fda48a3e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda44067850 con 0x7fda441544d0 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.954+0000 7fda48a3e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda44067b30 con 0x7fda440673c0 2023-11-28T03:07:05.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda441544d0 0x7fda44155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda41d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fda44066f40 0x7fda44095e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42592/0 (socket says 172.21.15.73:42592) 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda41d9b700 1 -- 172.21.15.73:0/3744316067 learned_addr learned my addr 172.21.15.73:0/3744316067 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda42d9d700 1 --2- 172.21.15.73:0/3744316067 >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 0x7fda4415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 -- 172.21.15.73:0/3744316067 >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 msgr2=0x7fda4415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 --2- 172.21.15.73:0/3744316067 >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 0x7fda4415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 -- 172.21.15.73:0/3744316067 >> v1:172.21.15.73:6789/0 conn(0x7fda44066f40 legacy=0x7fda44095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4159a700 1 -- 172.21.15.73:0/3744316067 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1475838493 0 0) 0x7fda440683d0 con 0x7fda44066f40 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 -- 172.21.15.73:0/3744316067 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda44067e10 con 0x7fda441544d0 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda42d9d700 1 --2- 172.21.15.73:0/3744316067 >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 0x7fda4415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4259c700 1 --2- 172.21.15.73:0/3744316067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda441544d0 0x7fda44155750 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fda440c6ef0 tx=0x7fda2c006f40).ready entity=mon.0 client_cookie=574e208cb73eefe9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4159a700 1 -- 172.21.15.73:0/3744316067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c005880 con 0x7fda441544d0 2023-11-28T03:07:05.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4159a700 1 -- 172.21.15.73:0/3744316067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0059e0 con 0x7fda441544d0 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.955+0000 7fda4159a700 1 -- 172.21.15.73:0/3744316067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c005d60 con 0x7fda441544d0 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 -- 172.21.15.73:0/3744316067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda441544d0 msgr2=0x7fda44155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 --2- 172.21.15.73:0/3744316067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda441544d0 0x7fda44155750 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 rx=0x7fda440c6ef0 tx=0x7fda2c006f40).stop 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 -- 172.21.15.73:0/3744316067 shutdown_connections 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 --2- 172.21.15.73:0/3744316067 >> v2:172.21.15.150:3300/0 conn(0x7fda440673c0 0x7fda4415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 --2- 172.21.15.73:0/3744316067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda441544d0 0x7fda44155750 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 -- 172.21.15.73:0/3744316067 >> 172.21.15.73:0/3744316067 conn(0x7fda440631d0 msgr2=0x7fda440c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 -- 172.21.15.73:0/3744316067 shutdown_connections 2023-11-28T03:07:05.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.956+0000 7fda48a3e700 1 -- 172.21.15.73:0/3744316067 wait complete. 2023-11-28T03:07:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.957+0000 7fda48a3e700 1 Processor -- start 2023-11-28T03:07:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.957+0000 7fda48a3e700 1 -- start start 2023-11-28T03:07:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda44066f40 0x7fda4414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda441544d0 0x7fda440a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440a77d0 con 0x7fda440ad2d0 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda440a67e0 con 0x7fda441544d0 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda48a3e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda440a6ac0 con 0x7fda44066f40 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47548/0 (socket says 172.21.15.73:47548) 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 -- 172.21.15.73:0/1919670161 learned_addr learned my addr 172.21.15.73:0/1919670161 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda41d9b700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda441544d0 0x7fda440a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda4259c700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda44066f40 0x7fda4414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 -- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda44066f40 msgr2=0x7fda4414efc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda44066f40 0x7fda4414efc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 -- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda441544d0 msgr2=0x7fda440a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda441544d0 0x7fda440a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 -- 172.21.15.73:0/1919670161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009250 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.958+0000 7fda42d9d700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fda3c00d180 tx=0x7fda3c009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.959+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c00dc60 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.959+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda3c00f040 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.959+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda440a6da0 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.959+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3c0133f0 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.959+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda4414f4b0 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.960+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda4406c860 con 0x7fda440ad2d0 2023-11-28T03:07:05.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.961+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fda3c0137e0 con 0x7fda440ad2d0 2023-11-28T03:07:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.961+0000 7fda3a7fc700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 0x7fda24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.961+0000 7fda4259c700 1 -- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 msgr2=0x7fda24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.961+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7fda3c07b910 con 0x7fda440ad2d0 2023-11-28T03:07:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.961+0000 7fda4259c700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 0x7fda24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:05.965+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda3c04c9c0 con 0x7fda440ad2d0 2023-11-28T03:07:06.002 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:06.001+0000 7f3394da2680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:06.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.138+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda440a7ab0 con 0x7fda440ad2d0 2023-11-28T03:07:06.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.139+0000 7fda3a7fc700 1 -- 172.21.15.73:0/1919670161 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17465 (secure 0 0 0) 0x7fda3c04ffe0 con 0x7fda440ad2d0 2023-11-28T03:07:06.140 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:06.141 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":69,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:05.244911+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/59040213":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6833/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6832/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:0/1205165186":"2023-11-29T03:05:16.454932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:06.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 msgr2=0x7fda24061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:06.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 0x7fda24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 msgr2=0x7fda440a7c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 rx=0x7fda3c00d180 tx=0x7fda3c009a50).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 shutdown_connections 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda2405f270 0x7fda24061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda440ad2d0 0x7fda440a7c60 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda44066f40 0x7fda4414efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 --2- 172.21.15.73:0/1919670161 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda441544d0 0x7fda440a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 >> 172.21.15.73:0/1919670161 conn(0x7fda440631d0 msgr2=0x7fda440c7270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 shutdown_connections 2023-11-28T03:07:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.141+0000 7fda48a3e700 1 -- 172.21.15.73:0/1919670161 wait complete. 2023-11-28T03:07:06.155 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:07:06.155 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:06.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.270+0000 7f2cb8afb700 1 Processor -- start 2023-11-28T03:07:06.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.270+0000 7f2cb8afb700 1 -- start start 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.271+0000 7f2cb8afb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.271+0000 7f2cb8afb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.271+0000 7f2cb8afb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2cb4154d20 con 0x7f2cb4068090 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.271+0000 7f2cb8afb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2cb4151670 con 0x7f2cb40c2d50 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.271+0000 7f2cb8afb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb40c48a0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb2d9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb2d9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:48632/0 (socket says 172.21.15.73:48632) 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:58030/0 (socket says 172.21.15.73:58030) 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb2d9d700 1 -- 172.21.15.73:0/3119070282 learned_addr learned my addr 172.21.15.73:0/3119070282 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb1d9b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f2cb4068090 0x7f2cb40c2640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:35818/0 (socket says 172.21.15.73:35818) 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 -- 172.21.15.73:0/3119070282 >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 msgr2=0x7f2cb40c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 --2- 172.21.15.73:0/3119070282 >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 -- 172.21.15.73:0/3119070282 >> v1:172.21.15.150:6789/0 conn(0x7f2cb4068090 legacy=0x7f2cb40c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 -- 172.21.15.73:0/3119070282 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cb40a7490 con 0x7f2cb4067cb0 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb2d9d700 1 --2- 172.21.15.73:0/3119070282 >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb259c700 1 --2- 172.21.15.73:0/3119070282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2ca800ba00 tx=0x7f2ca800b300).ready entity=mon.2 client_cookie=543056b9125b196e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:06.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb159a700 1 -- 172.21.15.73:0/3119070282 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca800da70 con 0x7f2cb4067cb0 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb159a700 1 -- 172.21.15.73:0/3119070282 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca800dbd0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.272+0000 7f2cb159a700 1 -- 172.21.15.73:0/3119070282 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8010470 con 0x7f2cb4067cb0 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3119070282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 msgr2=0x7f2cb40c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3119070282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 rx=0x7f2ca800ba00 tx=0x7f2ca800b300).stop 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3119070282 shutdown_connections 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3119070282 >> v2:172.21.15.150:3300/0 conn(0x7f2cb40c2d50 0x7f2cb40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3119070282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4067cb0 0x7f2cb40c3ec0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3119070282 >> 172.21.15.73:0/3119070282 conn(0x7f2cb40c7fd0 msgr2=0x7f2cb40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3119070282 shutdown_connections 2023-11-28T03:07:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.273+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3119070282 wait complete. 2023-11-28T03:07:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.274+0000 7f2cb8afb700 1 Processor -- start 2023-11-28T03:07:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.274+0000 7f2cb8afb700 1 -- start start 2023-11-28T03:07:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb416c4d0 con 0x7f2cb40c2d50 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb416ca90 con 0x7f2cb4067cb0 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb8afb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cb41699c0 con 0x7f2cb4068090 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb1d9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:48642/0 (socket says 172.21.15.73:48642) 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47550/0 (socket says 172.21.15.73:47550) 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb1d9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:58034/0 (socket says 172.21.15.73:58034) 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 -- 172.21.15.73:0/3867808401 learned_addr learned my addr 172.21.15.73:0/3867808401 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 msgr2=0x7f2cb4168f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 msgr2=0x7f2cb41693f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 -- 172.21.15.73:0/3867808401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac009ae0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb2d9d700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb1d9b700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.275+0000 7f2cb259c700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2ca800d5d0 tx=0x7f2ca800b300).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.276+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8019450 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.276+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ca80195b0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.276+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca800b540 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.276+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ca8019930 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.276+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cb416c720 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.277+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cb406c860 con 0x7f2cb4067cb0 2023-11-28T03:07:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.277+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2ca800f3d0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.278+0000 7f2ca27fc700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 0x7f2c94061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.278+0000 7f2cb1d9b700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 msgr2=0x7f2c94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:06.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.278+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f2ca807bfa0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.278+0000 7f2cb1d9b700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 0x7f2c94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.283+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ca80aaaf0 con 0x7f2cb4067cb0 2023-11-28T03:07:06.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.479+0000 7f2cb1d9b700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 msgr2=0x7f2c94061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:06.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.479+0000 7f2cb1d9b700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 0x7f2c94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:06.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.496+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2cb40a7490 con 0x7f2cb4067cb0 2023-11-28T03:07:06.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.497+0000 7f2ca27fc700 1 -- 172.21.15.73:0/3867808401 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f2cb40a7490 con 0x7f2cb4067cb0 2023-11-28T03:07:06.498 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:06.498 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:07:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 msgr2=0x7f2c94061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 0x7f2c94061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 msgr2=0x7f2cb414e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 rx=0x7f2ca800d5d0 tx=0x7f2ca800b300).stop 2023-11-28T03:07:06.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 shutdown_connections 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f270 0x7f2c94061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2cb40c2d50 0x7f2cb41693f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2cb4067cb0 0x7f2cb414e910 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 --2- 172.21.15.73:0/3867808401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2cb4068090 0x7f2cb4168f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 >> 172.21.15.73:0/3867808401 conn(0x7f2cb40c7fd0 msgr2=0x7f2cb40bd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 shutdown_connections 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.499+0000 7f2cb8afb700 1 -- 172.21.15.73:0/3867808401 wait complete. 2023-11-28T03:07:06.501 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:07:06.512 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:06.524 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.524+0000 7fdf63a05680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:06.591 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.590+0000 7fdf63a05680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:06.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.632+0000 7f7bce34f700 1 Processor -- start 2023-11-28T03:07:06.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.632+0000 7f7bce34f700 1 -- start start 2023-11-28T03:07:06.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bce34f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7bc81542d0 0x7f7bc80697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bce34f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 0x7f7bc8062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bce34f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bc8065760 con 0x7f7bc8062bf0 2023-11-28T03:07:06.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bce34f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bc806a160 con 0x7f7bc81542d0 2023-11-28T03:07:06.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bce34f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc8063080 con 0x7f7bc8069ce0 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bcc8ec700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7bc8062bf0 0x7f7bc8157fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42608/0 (socket says 172.21.15.73:42608) 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bcc8ec700 1 -- 172.21.15.73:0/2109502158 learned_addr learned my addr 172.21.15.73:0/2109502158 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/2109502158 >> v2:172.21.15.150:3300/0 conn(0x7f7bc81542d0 0x7f7bc80697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bc77fe700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 0x7f7bc8062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bc6ffd700 1 -- 172.21.15.73:0/2109502158 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3156308673 0 0) 0x7f7bc8065760 con 0x7f7bc8062bf0 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.633+0000 7f7bc6ffd700 1 -- 172.21.15.73:0/2109502158 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7bb00033f0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc7fff700 1 -- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 msgr2=0x7f7bc8062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:06.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 0x7f7bc8062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc7fff700 1 -- 172.21.15.73:0/2109502158 >> v1:172.21.15.73:6789/0 conn(0x7f7bc8062bf0 legacy=0x7f7bc8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc7fff700 1 -- 172.21.15.73:0/2109502158 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bc80c3250 con 0x7f7bc81542d0 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc77fe700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 0x7f7bc8062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc81542d0 0x7f7bc80697f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f7bbc009760 tx=0x7f7bbc009060).ready entity=mon.1 client_cookie=b2bf1002706eacc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc6ffd700 1 -- 172.21.15.73:0/2109502158 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00d4a0 con 0x7f7bc81542d0 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc6ffd700 1 -- 172.21.15.73:0/2109502158 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00d600 con 0x7f7bc81542d0 2023-11-28T03:07:06.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bc6ffd700 1 -- 172.21.15.73:0/2109502158 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00d980 con 0x7f7bc81542d0 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bce34f700 1 -- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc81542d0 msgr2=0x7f7bc80697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bce34f700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc81542d0 0x7f7bc80697f0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 rx=0x7f7bbc009760 tx=0x7f7bbc009060).stop 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.634+0000 7f7bce34f700 1 -- 172.21.15.73:0/2109502158 shutdown_connections 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.635+0000 7f7bce34f700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc81542d0 0x7f7bc80697f0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.635+0000 7f7bce34f700 1 --2- 172.21.15.73:0/2109502158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc8069ce0 0x7f7bc8062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.635+0000 7f7bce34f700 1 -- 172.21.15.73:0/2109502158 >> 172.21.15.73:0/2109502158 conn(0x7f7bc80c7fd0 msgr2=0x7f7bc80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:06.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.635+0000 7f7bce34f700 1 -- 172.21.15.73:0/2109502158 shutdown_connections 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.635+0000 7f7bce34f700 1 -- 172.21.15.73:0/2109502158 wait complete. 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.636+0000 7f7bce34f700 1 Processor -- start 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.636+0000 7f7bce34f700 1 -- start start 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8069ce0 0x7f7bc814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc81542d0 0x7f7bc80acf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc814bb20 con 0x7f7bc8062bf0 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80aa070 con 0x7f7bc8069ce0 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bc80a8480 con 0x7f7bc81542d0 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47562/0 (socket says 172.21.15.73:47562) 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 -- 172.21.15.73:0/977211035 learned_addr learned my addr 172.21.15.73:0/977211035 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bcc8ec700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc81542d0 0x7f7bc80acf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc77fe700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8069ce0 0x7f7bc814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc81542d0 msgr2=0x7f7bc80acf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc81542d0 0x7f7bc80acf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8069ce0 msgr2=0x7f7bc814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8069ce0 0x7f7bc814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 -- 172.21.15.73:0/977211035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bb80092a0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bc7fff700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7bbc011fb0 tx=0x7f7bbc016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00dd30 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7bbc00de90 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bbc0092a0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7bbc00d980 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.637+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bc80a89f0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f7bbc01d640 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bc80aaf60 con 0x7f7bc8062bf0 2023-11-28T03:07:06.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7baffff700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 0x7f7bb4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7bc77fe700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 msgr2=0x7f7bb4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f7bbc07bbc0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.639+0000 7f7bc77fe700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 0x7f7bb4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:06.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.643+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7bbc04cbb0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.666 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.665+0000 7fdf63a05680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:06.700 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:06.699+0000 7f3394da2680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:06.771 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:06.770+0000 7f3394da2680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:06.811 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.810+0000 7fdf63a05680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:06.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.840+0000 7f7bc77fe700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 msgr2=0x7f7bb4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:06.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.840+0000 7f7bc77fe700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 0x7f7bb4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:06.847 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:06.847+0000 7f3394da2680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:06.880 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.879+0000 7fdf63a05680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:06.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.881+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7bc806c860 con 0x7f7bc8062bf0 2023-11-28T03:07:06.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.881+0000 7f7baffff700 1 -- 172.21.15.73:0/977211035 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) v1 ==== 94+0+4489 (secure 0 0 0) 0x7f7bbc0501d0 con 0x7f7bc8062bf0 2023-11-28T03:07:06.882 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:06.883 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":20,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15}],"filesystems":[{"mdsmap":{"epoch":20,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:05:00.776599+0000","modified":"2023-11-28T03:05:03.427098+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5775},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5775":{"gid":5775,"name":"a","rank":0,"incarnation":17,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3689972589","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":3689972589},{"type":"v1","addr":"172.21.15.73:6837","nonce":3689972589}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[8],"metadata_pool":7,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-11-28T03:07:06.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 msgr2=0x7f7bb4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:06.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 0x7f7bb4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 msgr2=0x7f7bc814c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 rx=0x7f7bbc011fb0 tx=0x7f7bbc016000).stop 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 shutdown_connections 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7bb405f270 0x7f7bb4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7bc81542d0 0x7f7bc80acf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7bc8062bf0 0x7f7bc814c1b0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 --2- 172.21.15.73:0/977211035 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7bc8069ce0 0x7f7bc814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 >> 172.21.15.73:0/977211035 conn(0x7f7bc80c7fd0 msgr2=0x7f7bc80c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 shutdown_connections 2023-11-28T03:07:06.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:06.884+0000 7f7bce34f700 1 -- 172.21.15.73:0/977211035 wait complete. 2023-11-28T03:07:06.887 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 20 2023-11-28T03:07:06.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:06.993 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:06.992+0000 7fdf63a05680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:07.002 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.001+0000 7f3394da2680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.013+0000 7f66fa34c700 1 Processor -- start 2023-11-28T03:07:07.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.013+0000 7f66fa34c700 1 -- start start 2023-11-28T03:07:07.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66fa34c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f40c32b0 0x7f66f415bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66fa34c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4066e60 con 0x7f66f4066880 2023-11-28T03:07:07.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66fa34c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f4067140 con 0x7f66f413a8d0 2023-11-28T03:07:07.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66fa34c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f4067420 con 0x7f66f40c32b0 2023-11-28T03:07:07.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f40c32b0 0x7f66f415bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f40c32b0 0x7f66f415bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47568/0 (socket says 172.21.15.73:47568) 2023-11-28T03:07:07.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f8b49700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f66f4066880 0x7f66f40694e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42616/0 (socket says 172.21.15.73:42616) 2023-11-28T03:07:07.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 -- 172.21.15.73:0/663485163 learned_addr learned my addr 172.21.15.73:0/663485163 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:07.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 -- 172.21.15.73:0/663485163 >> v1:172.21.15.150:6789/0 conn(0x7f66f413a8d0 legacy=0x7f66f4155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 -- 172.21.15.73:0/663485163 >> v1:172.21.15.73:6789/0 conn(0x7f66f4066880 legacy=0x7f66f40694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66f9b4b700 1 -- 172.21.15.73:0/663485163 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f4067700 con 0x7f66f40c32b0 2023-11-28T03:07:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.014+0000 7f66ebfff700 1 -- 172.21.15.73:0/663485163 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742974414 0 0) 0x7f66f4066e60 con 0x7f66f4066880 2023-11-28T03:07:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66f9b4b700 1 --2- 172.21.15.73:0/663485163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f40c32b0 0x7f66f415bea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f66ec009f20 tx=0x7f66ec009840).ready entity=mon.0 client_cookie=9825313582f10192 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66ebfff700 1 -- 172.21.15.73:0/663485163 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00e070 con 0x7f66f40c32b0 2023-11-28T03:07:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66ebfff700 1 -- 172.21.15.73:0/663485163 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ec00ba80 con 0x7f66f40c32b0 2023-11-28T03:07:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66ebfff700 1 -- 172.21.15.73:0/663485163 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ec00be00 con 0x7f66f40c32b0 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 -- 172.21.15.73:0/663485163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f40c32b0 msgr2=0x7f66f415bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 --2- 172.21.15.73:0/663485163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f40c32b0 0x7f66f415bea0 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 rx=0x7f66ec009f20 tx=0x7f66ec009840).stop 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 -- 172.21.15.73:0/663485163 shutdown_connections 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 --2- 172.21.15.73:0/663485163 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f40c32b0 0x7f66f415bea0 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 -- 172.21.15.73:0/663485163 >> 172.21.15.73:0/663485163 conn(0x7f66f40c7ca0 msgr2=0x7f66f40c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 -- 172.21.15.73:0/663485163 shutdown_connections 2023-11-28T03:07:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.015+0000 7f66fa34c700 1 -- 172.21.15.73:0/663485163 wait complete. 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.016+0000 7f66fa34c700 1 Processor -- start 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 -- start start 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f40c32b0 0x7f66f40a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f413a8d0 0x7f66f4167e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40aa080 con 0x7f66f4066880 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f4168dc0 con 0x7f66f40c32b0 2023-11-28T03:07:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.017+0000 7f66fa34c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f40a7bf0 con 0x7f66f413a8d0 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47570/0 (socket says 172.21.15.73:47570) 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 -- 172.21.15.73:0/1775125466 learned_addr learned my addr 172.21.15.73:0/1775125466 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f8b49700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f40c32b0 0x7f66f40a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f9b4b700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f413a8d0 0x7f66f4167e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 -- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f413a8d0 msgr2=0x7f66f4167e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f413a8d0 0x7f66f4167e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 -- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f40c32b0 msgr2=0x7f66f40a48e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f40c32b0 0x7f66f40a48e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 -- 172.21.15.73:0/1775125466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ec009ae0 con 0x7f66f4066880 2023-11-28T03:07:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66f934a700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f66e400c4c0 tx=0x7f66e4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400ce70 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e4001e90 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f4150cb0 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e4007250 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.018+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f4168ae0 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.019+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f66e40066a0 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.019+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f40c5190 con 0x7f66f4066880 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.020+0000 7f66e97fa700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 0x7f66d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.020+0000 7f66f8b49700 1 -- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 msgr2=0x7f66d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:07.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.020+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f66e4079db0 con 0x7f66f4066880 2023-11-28T03:07:07.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.020+0000 7f66f8b49700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 0x7f66d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:07.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.024+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66e404ada0 con 0x7f66f4066880 2023-11-28T03:07:07.077 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.076+0000 7f3394da2680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:07.103 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:07.102+0000 7fdf63a05680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:07.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.193+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66f40c6db0 con 0x7f66f4066880 2023-11-28T03:07:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.194+0000 7f66e97fa700 1 -- 172.21.15.73:0/1775125466 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v69) v1 ==== 74+0+17465 (secure 0 0 0) 0x7f66e404e3c0 con 0x7f66f4066880 2023-11-28T03:07:07.196 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:07.196 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":69,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:05.244911+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":7,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:04:58.509951+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":8,"pool_name":"cephfs_data","create_time":"2023-11-28T03:04:59.750391+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"69","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":69,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.7","mappings":[{"from":0,"to":4}]},{"pgid":"7.3c","mappings":[{"from":0,"to":5}]},{"pgid":"8.6","mappings":[{"from":0,"to":1}]},{"pgid":"8.1d","mappings":[{"from":0,"to":6}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/59040213":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6833/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:6832/90017":"2023-11-29T03:05:16.454932+0000","172.21.15.73:0/1205165186":"2023-11-29T03:05:16.454932+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":8,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:07.196 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.195+0000 7f3394da2680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 msgr2=0x7f66d8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 0x7f66d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 msgr2=0x7f66f41507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 rx=0x7f66e400c4c0 tx=0x7f66e4007e00).stop 2023-11-28T03:07:07.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 shutdown_connections 2023-11-28T03:07:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66d805f270 0x7f66d8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f413a8d0 0x7f66f4167e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f4066880 0x7f66f41507c0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 --2- 172.21.15.73:0/1775125466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f40c32b0 0x7f66f40a48e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 >> 172.21.15.73:0/1775125466 conn(0x7f66f40c7ca0 msgr2=0x7f66f40bcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.196+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 shutdown_connections 2023-11-28T03:07:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.197+0000 7f66fa34c700 1 -- 172.21.15.73:0/1775125466 wait complete. 2023-11-28T03:07:07.210 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:07:07.312 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.311+0000 7f3394da2680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:07.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.326+0000 7f4c9bd83700 1 Processor -- start 2023-11-28T03:07:07.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.326+0000 7f4c9bd83700 1 -- start start 2023-11-28T03:07:07.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9bd83700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 0x7f4c940c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9bd83700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9bd83700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c94158cc0 con 0x7f4c94098d10 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9bd83700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c940c72d0 con 0x7f4c940c6190 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9bd83700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94064570 con 0x7f4c94158320 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9a320700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c9931e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 0x7f4c940c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c99b1f700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f4c94098d10 0x7f4c94063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:35824/0 (socket says 172.21.15.73:35824) 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.327+0000 7f4c99b1f700 1 -- 172.21.15.73:0/217758384 learned_addr learned my addr 172.21.15.73:0/217758384 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9a320700 1 -- 172.21.15.73:0/217758384 >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 msgr2=0x7f4c940c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:07.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9a320700 1 --2- 172.21.15.73:0/217758384 >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 0x7f4c940c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9a320700 1 -- 172.21.15.73:0/217758384 >> v1:172.21.15.150:6789/0 conn(0x7f4c94098d10 legacy=0x7f4c94063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c98b1d700 1 -- 172.21.15.73:0/217758384 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3587017192 0 0) 0x7f4c84001110 con 0x7f4c94098d10 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9a320700 1 -- 172.21.15.73:0/217758384 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c94154bc0 con 0x7f4c94158320 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9931e700 1 --2- 172.21.15.73:0/217758384 >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 0x7f4c940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c9a320700 1 --2- 172.21.15.73:0/217758384 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941587d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4c8c009e60 tx=0x7f4c8c009780).ready entity=mon.2 client_cookie=f07807606e7c4c31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c98b1d700 1 -- 172.21.15.73:0/217758384 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c00e070 con 0x7f4c94158320 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c98b1d700 1 -- 172.21.15.73:0/217758384 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c8c00bb60 con 0x7f4c94158320 2023-11-28T03:07:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.328+0000 7f4c98b1d700 1 -- 172.21.15.73:0/217758384 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8c012410 con 0x7f4c94158320 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 -- 172.21.15.73:0/217758384 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 msgr2=0x7f4c941587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/217758384 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941587d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 rx=0x7f4c8c009e60 tx=0x7f4c8c009780).stop 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 -- 172.21.15.73:0/217758384 shutdown_connections 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/217758384 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941587d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/217758384 >> v2:172.21.15.150:3300/0 conn(0x7f4c940c6190 0x7f4c940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 -- 172.21.15.73:0/217758384 >> 172.21.15.73:0/217758384 conn(0x7f4c940cb310 msgr2=0x7f4c940cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 -- 172.21.15.73:0/217758384 shutdown_connections 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.329+0000 7f4c9bd83700 1 -- 172.21.15.73:0/217758384 wait complete. 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.330+0000 7f4c9bd83700 1 Processor -- start 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.330+0000 7f4c9bd83700 1 -- start start 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c940c6190 0x7f4c9414df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94154fd0 con 0x7f4c94098d10 2023-11-28T03:07:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94156590 con 0x7f4c940c6190 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c94156870 con 0x7f4c94158320 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47578/0 (socket says 172.21.15.73:47578) 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 -- 172.21.15.73:0/2415839999 learned_addr learned my addr 172.21.15.73:0/2415839999 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9931e700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c940c6190 0x7f4c9414df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9a320700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 msgr2=0x7f4c941558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c940c6190 msgr2=0x7f4c9414df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c940c6190 0x7f4c9414df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 -- 172.21.15.73:0/2415839999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c900092a0 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c99b1f700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f4c8400c510 tx=0x7f4c84007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400ce80 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c840072e0 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c8c009a20 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c84007660 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.331+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c94155520 con 0x7f4c94098d10 2023-11-28T03:07:07.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.332+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 21) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f4c840059b0 con 0x7f4c94098d10 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.333+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c94156a30 con 0x7f4c94098d10 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.333+0000 7f4c89ffb700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 0x7f4c78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.333+0000 7f4c9931e700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 msgr2=0x7f4c78061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.333+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6465+0+0 (secure 0 0 0) 0x7f4c84079b80 con 0x7f4c94098d10 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.333+0000 7f4c9931e700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 0x7f4c78061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:07.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.337+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c94156a30 con 0x7f4c94098d10 2023-11-28T03:07:07.394 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:07.393+0000 7fdf63a05680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:07.466 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:07.465+0000 7fdf63a05680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:07.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.534+0000 7f4c9931e700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 msgr2=0x7f4c78061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:07.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.534+0000 7f4c9931e700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 0x7f4c78061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:07.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.549+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f4c94158cc0 con 0x7f4c94098d10 2023-11-28T03:07:07.612 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.611+0000 7f3394da2680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:07.682 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.681+0000 7f3394da2680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.705+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f4c84077670 con 0x7f4c94098d10 2023-11-28T03:07:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.705+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 msgr2=0x7f4c78061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:07.705+0000 7f4c89ffb700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 0x7f4c78061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:07.707 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:07.706+0000 7f26216c1700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:07.707 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:07.706+0000 7f26216c1700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:07.707 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:07.706+0000 7f26216c1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:07:07.723 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T03:07:07.706+0000 7f26216c1700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:07.723 INFO:tasks.ceph.mds.a.smithi073.stderr: -3> 2023-11-28T03:07:07.706+0000 7f26216c1700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:07.724 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T03:07:07.706+0000 7f26216c1700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:07:07.743 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:07:07.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f330e532700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f330e532700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f330e532700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f330e532700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f330e532700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f3311538700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f3311538700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f3311538700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f3311538700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:07.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:07.762+0000 7f3311538700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:08.261 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:07:08.260+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:07:08.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.269+0000 7f4c89ffb700 1 -- 172.21.15.73:0/2415839999 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) v1 ==== 161+0+0 (secure 0 0 0) 0x7f4c8406fdf0 con 0x7f4c94098d10 2023-11-28T03:07:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 msgr2=0x7f4c940c23b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 rx=0x7f4c8400c510 tx=0x7f4c84007ad0).stop 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 shutdown_connections 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4c7805f270 0x7f4c78061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c94158320 0x7f4c941558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c94098d10 0x7f4c940c23b0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 --2- 172.21.15.73:0/2415839999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c940c6190 0x7f4c9414df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 >> 172.21.15.73:0/2415839999 conn(0x7f4c940cb310 msgr2=0x7f4c94153230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 shutdown_connections 2023-11-28T03:07:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.271+0000 7f4c9bd83700 1 -- 172.21.15.73:0/2415839999 wait complete. 2023-11-28T03:07:08.274 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:07:08.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:07:08.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.401+0000 7f621dad8700 1 Processor -- start 2023-11-28T03:07:08.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.401+0000 7f621dad8700 1 -- start start 2023-11-28T03:07:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f621dad8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6218158a30 0x7f62180c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f621dad8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62180c72d0 con 0x7f62180c6190 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f621dad8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6218064570 con 0x7f6218098d10 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f621dad8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6218154bc0 con 0x7f6218158a30 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f6216ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f62180c6190 0x7f6218158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42632/0 (socket says 172.21.15.73:42632) 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f6216ffd700 1 -- 172.21.15.73:0/1346265225 learned_addr learned my addr 172.21.15.73:0/1346265225 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f6217fff700 1 --2- 172.21.15.73:0/1346265225 >> v2:172.21.15.150:3300/0 conn(0x7f6218158a30 0x7f62180c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1880468199 0 0) 0x7f62180c72d0 con 0x7f62180c6190 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.402+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f62000033f0 con 0x7f62180c6190 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1877896091 0 0) 0x7f62000033f0 con 0x7f62180c6190 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 >> v2:172.21.15.150:3300/0 conn(0x7f6218158a30 msgr2=0x7f62180c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 --2- 172.21.15.73:0/1346265225 >> v2:172.21.15.150:3300/0 conn(0x7f6218158a30 0x7f62180c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 >> v1:172.21.15.150:6789/0 conn(0x7f6218098d10 legacy=0x7f6218063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62180c66c0 con 0x7f62180c6190 2023-11-28T03:07:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f620c001470 con 0x7f62180c6190 2023-11-28T03:07:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 >> v1:172.21.15.73:6789/0 conn(0x7f62180c6190 legacy=0x7f6218158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62000038e0 0x7f6200003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200004690 0x7f6200004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 0x7f620000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6218154bc0 con 0x7f6200005000 2023-11-28T03:07:08.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f620000beb0 con 0x7f6200004690 2023-11-28T03:07:08.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f620000c070 con 0x7f62000038e0 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 0x7f620000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6216ffd700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200004690 0x7f6200004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62177fe700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62000038e0 0x7f6200003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 -- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62000038e0 msgr2=0x7f6200003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62000038e0 0x7f6200003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 -- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200004690 msgr2=0x7f6200004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200004690 0x7f6200004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 -- 172.21.15.73:0/1346265225 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62000033f0 con 0x7f6200005000 2023-11-28T03:07:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 -- 172.21.15.73:0/1346265225 reap_dead start 2023-11-28T03:07:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f6217fff700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 0x7f620000b8e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6218062cb0 tx=0x7f6210009060).ready entity=mon.0 client_cookie=8a64ec6b2629716e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210009430 con 0x7f6200005000 2023-11-28T03:07:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6210009e70 con 0x7f6200005000 2023-11-28T03:07:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.403+0000 7f62167fc700 1 -- 172.21.15.73:0/1346265225 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621000e850 con 0x7f6200005000 2023-11-28T03:07:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 -- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 msgr2=0x7f620000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 0x7f620000b8e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 rx=0x7f6218062cb0 tx=0x7f6210009060).stop 2023-11-28T03:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 -- 172.21.15.73:0/1346265225 shutdown_connections 2023-11-28T03:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 --2- 172.21.15.73:0/1346265225 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6200005000 0x7f620000b8e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 -- 172.21.15.73:0/1346265225 >> 172.21.15.73:0/1346265225 conn(0x7f62180cb310 msgr2=0x7f62180cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 -- 172.21.15.73:0/1346265225 shutdown_connections 2023-11-28T03:07:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.404+0000 7f621dad8700 1 -- 172.21.15.73:0/1346265225 wait complete. 2023-11-28T03:07:08.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.405+0000 7f621dad8700 1 Processor -- start 2023-11-28T03:07:08.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.405+0000 7f621dad8700 1 -- start start 2023-11-28T03:07:08.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200005000 0x7f6218159e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62180c2720 0x7f62180c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621815ae80 con 0x7f62180c45a0 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f621815a380 con 0x7f6200005000 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f621dad8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f621815a5e0 con 0x7f62180c2720 2023-11-28T03:07:08.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f6217fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f6217fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47688/0 (socket says 172.21.15.73:47688) 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f6217fff700 1 -- 172.21.15.73:0/3259062834 learned_addr learned my addr 172.21.15.73:0/3259062834 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f62177fe700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200005000 0x7f6218159e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.406+0000 7f6216ffd700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62180c2720 0x7f62180c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 -- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62180c2720 msgr2=0x7f62180c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62180c2720 0x7f62180c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 -- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200005000 msgr2=0x7f6218159e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:08.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200005000 0x7f6218159e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:08.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 -- 172.21.15.73:0/3259062834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f621815a8c0 con 0x7f62180c45a0 2023-11-28T03:07:08.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f6217fff700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6210000c00 tx=0x7f6210019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:08.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f621001cab0 con 0x7f62180c45a0 2023-11-28T03:07:08.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f621001cc10 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f621815aba0 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6210025440 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.407+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62180ca1d0 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.408+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62180ab5e0 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.408+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 22) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f62100123f0 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.408+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f621007b9f0 con 0x7f62180c45a0 2023-11-28T03:07:08.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.413+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f621007de60 con 0x7f62180c45a0 2023-11-28T03:07:08.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.619+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f62180ab0d0 con 0x7f62180c45a0 2023-11-28T03:07:08.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.710+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f6210079750 con 0x7f62180c45a0 2023-11-28T03:07:08.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.710+0000 7f61ff7fe700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 0x7f6204067cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:08.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.712+0000 7f62177fe700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 0x7f6204067cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:08.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:08.712+0000 7f62177fe700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 0x7f6204067cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6208000f80 tx=0x7f6208007f30).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:09.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.269+0000 7f61ff7fe700 1 -- 172.21.15.73:0/3259062834 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v22) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6210053a50 con 0x7f62180c45a0 2023-11-28T03:07:09.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 msgr2=0x7f6204067cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 0x7f6204067cd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7f6208000f80 tx=0x7f6208007f30).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 msgr2=0x7f62180c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 rx=0x7f6210000c00 tx=0x7f6210019000).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 shutdown_connections 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62040302f0 0x7f6204067cd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62180c45a0 0x7f62180c4a20 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6200005000 0x7f6218159e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 --2- 172.21.15.73:0/3259062834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62180c2720 0x7f62180c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 >> 172.21.15.73:0/3259062834 conn(0x7f62180cb310 msgr2=0x7f62180aa190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:09.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 shutdown_connections 2023-11-28T03:07:09.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.271+0000 7f621dad8700 1 -- 172.21.15.73:0/3259062834 wait complete. 2023-11-28T03:07:09.285 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:07:09.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.402+0000 7f49748ba700 1 Processor -- start 2023-11-28T03:07:09.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.402+0000 7f49748ba700 1 -- start start 2023-11-28T03:07:09.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f49748ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f49700c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f49748ba700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4970158d50 con 0x7f49701542d0 2023-11-28T03:07:09.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f49748ba700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4970158140 con 0x7f49701546b0 2023-11-28T03:07:09.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f49748ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4970158420 con 0x7f49700aa680 2023-11-28T03:07:09.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f49701542d0 0x7f49700a9f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42634/0 (socket says 172.21.15.73:42634) 2023-11-28T03:07:09.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496f7fe700 1 -- 172.21.15.73:0/2996997118 learned_addr learned my addr 172.21.15.73:0/2996997118 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:09.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483737139 0 0) 0x7f4970158d50 con 0x7f49701542d0 2023-11-28T03:07:09.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f495c0033f0 con 0x7f49701542d0 2023-11-28T03:07:09.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496effd700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f49700c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3557972923 0 0) 0x7f495c0033f0 con 0x7f49701542d0 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 msgr2=0x7f49700c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f49700c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 >> v1:172.21.15.150:6789/0 conn(0x7f49701546b0 legacy=0x7f49700c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970158700 con 0x7f49701542d0 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f49680017f0 con 0x7f49701542d0 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 >> v1:172.21.15.73:6789/0 conn(0x7f49701542d0 legacy=0x7f49700a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f495c003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c003ef0 0x7f495c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c004b80 0x7f495c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970158420 con 0x7f495c003ef0 2023-11-28T03:07:09.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f495c00ba90 con 0x7f495c004b80 2023-11-28T03:07:09.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f495c00bc50 con 0x7f49700aa680 2023-11-28T03:07:09.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496effd700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c003ef0 0x7f495c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.403+0000 7f496f7fe700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f495c003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496ffff700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c004b80 0x7f495c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 -- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 msgr2=0x7f495c003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:09.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f495c003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 -- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c004b80 msgr2=0x7f495c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c004b80 0x7f495c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 -- 172.21.15.73:0/2996997118 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f495c0033f0 con 0x7f495c003ef0 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496ffff700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c004b80 0x7f495c005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496f7fe700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49700aa680 0x7f495c003910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496effd700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c003ef0 0x7f495c004690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f495800b9d0 tx=0x7f4958008040).ready entity=mon.0 client_cookie=58834719c3563ce8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f495800b340 con 0x7f495c003ef0 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4958013480 con 0x7f495c003ef0 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496e7fc700 1 -- 172.21.15.73:0/2996997118 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958013800 con 0x7f495c003ef0 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f49748ba700 1 -- 172.21.15.73:0/2996997118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c003ef0 msgr2=0x7f495c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f49748ba700 1 --2- 172.21.15.73:0/2996997118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f495c003ef0 0x7f495c004690 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 rx=0x7f495800b9d0 tx=0x7f4958008040).stop 2023-11-28T03:07:09.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f496ffff700 1 -- 172.21.15.73:0/2996997118 reap_dead start 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f49748ba700 1 -- 172.21.15.73:0/2996997118 shutdown_connections 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.404+0000 7f49748ba700 1 -- 172.21.15.73:0/2996997118 >> 172.21.15.73:0/2996997118 conn(0x7f49700cb010 msgr2=0x7f49700cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.405+0000 7f49748ba700 1 -- 172.21.15.73:0/2996997118 shutdown_connections 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.405+0000 7f49748ba700 1 -- 172.21.15.73:0/2996997118 wait complete. 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.406+0000 7f49748ba700 1 Processor -- start 2023-11-28T03:07:09.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.406+0000 7f49748ba700 1 -- start start 2023-11-28T03:07:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 0x7f4970144dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f497014e790 0x7f497014ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49701452b0 con 0x7f49701554a0 2023-11-28T03:07:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4970156e30 con 0x7f495c003ef0 2023-11-28T03:07:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f497014f100 con 0x7f497014e790 2023-11-28T03:07:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47708/0 (socket says 172.21.15.73:47708) 2023-11-28T03:07:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 -- 172.21.15.73:0/3782621851 learned_addr learned my addr 172.21.15.73:0/3782621851 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496f7fe700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 0x7f4970144dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496effd700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f497014e790 0x7f497014ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 -- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f497014e790 msgr2=0x7f497014ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f497014e790 0x7f497014ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 -- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 msgr2=0x7f4970144dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 0x7f4970144dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 -- 172.21.15.73:0/3782621851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4970156220 con 0x7f49701554a0 2023-11-28T03:07:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496f7fe700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 0x7f4970144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f496ffff700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f4970154a90 tx=0x7f4960009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960014430 con 0x7f49701554a0 2023-11-28T03:07:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4960014590 con 0x7f49701554a0 2023-11-28T03:07:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4970156500 con 0x7f49701554a0 2023-11-28T03:07:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4960014910 con 0x7f49701554a0 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.407+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4970156ac0 con 0x7f49701554a0 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.408+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497006c7c0 con 0x7f49701554a0 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.408+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 23) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f4960014d00 con 0x7f49701554a0 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.409+0000 7f49677fe700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 0x7f49500614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.409+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 6695+0+0 (secure 0 0 0) 0x7f4960079010 con 0x7f49701554a0 2023-11-28T03:07:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.409+0000 7f496f7fe700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 0x7f49500614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.409+0000 7f496f7fe700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 0x7f49500614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4968001190 tx=0x7f4968007ec0).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.413+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f496004a270 con 0x7f49701554a0 2023-11-28T03:07:09.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:09.578+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49700c0120 con 0x7f49701554a0 2023-11-28T03:07:10.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.274+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4960076d70 con 0x7f49701554a0 2023-11-28T03:07:10.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.279+0000 7f49677fe700 1 -- 172.21.15.73:0/3782621851 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v72) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4960071c30 con 0x7f49701554a0 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 msgr2=0x7f49500614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 0x7f49500614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f4968001190 tx=0x7f4968007ec0).stop 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 msgr2=0x7f4970155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 rx=0x7f4970154a90 tx=0x7f4960009040).stop 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 shutdown_connections 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f495005f000 0x7f49500614c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49701554a0 0x7f4970155920 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f495c003ef0 0x7f4970144dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 --2- 172.21.15.73:0/3782621851 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f497014e790 0x7f497014ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.284+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 >> 172.21.15.73:0/3782621851 conn(0x7f49700cb010 msgr2=0x7f49700c9e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.285+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 shutdown_connections 2023-11-28T03:07:10.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.285+0000 7f49748ba700 1 -- 172.21.15.73:0/3782621851 wait complete. 2023-11-28T03:07:10.288 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:07:10.302 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:07:10.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.427+0000 7f4968014700 1 Processor -- start 2023-11-28T03:07:10.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.428+0000 7f4968014700 1 -- start start 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.428+0000 7f4968014700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4960154530 0x7f4960154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.428+0000 7f4968014700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600c6630 con 0x7f49600aa220 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.428+0000 7f4968014700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49600a8470 con 0x7f49600a7ff0 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.428+0000 7f4968014700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49600c6390 con 0x7f4960154530 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f49665b1700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f49600aa220 0x7f49600c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42638/0 (socket says 172.21.15.73:42638) 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f49665b1700 1 -- 172.21.15.73:0/2353006572 learned_addr learned my addr 172.21.15.73:0/2353006572 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4965db0700 1 --2- 172.21.15.73:0/2353006572 >> v2:172.21.15.150:3300/0 conn(0x7f4960154530 0x7f4960154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1571723801 0 0) 0x7f49600c6630 con 0x7f49600aa220 2023-11-28T03:07:10.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f494c0033f0 con 0x7f49600aa220 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 377879519 0 0) 0x7f494c0033f0 con 0x7f49600aa220 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 >> v2:172.21.15.150:3300/0 conn(0x7f4960154530 msgr2=0x7f4960154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 --2- 172.21.15.73:0/2353006572 >> v2:172.21.15.150:3300/0 conn(0x7f4960154530 0x7f4960154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 >> v1:172.21.15.150:6789/0 conn(0x7f49600a7ff0 legacy=0x7f49600c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4960064230 con 0x7f49600aa220 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f49580017f0 con 0x7f49600aa220 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 >> v1:172.21.15.73:6789/0 conn(0x7f49600aa220 legacy=0x7f49600c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c0038e0 0x7f494c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494c004690 0x7f494c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 0x7f494c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49600c6390 con 0x7f494c005000 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f494c00bf60 con 0x7f494c004690 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f494c00c0f0 con 0x7f494c0038e0 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f49665b1700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 0x7f494c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f4965db0700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c0038e0 0x7f494c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.429+0000 7f49655af700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494c004690 0x7f494c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 -- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c0038e0 msgr2=0x7f494c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c0038e0 0x7f494c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 -- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494c004690 msgr2=0x7f494c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f494c004690 0x7f494c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 -- 172.21.15.73:0/2353006572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f494c0033f0 con 0x7f494c005000 2023-11-28T03:07:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 -- 172.21.15.73:0/2353006572 reap_dead start 2023-11-28T03:07:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f49665b1700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 0x7f494c00b8e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f495800c480 tx=0x7f495800c790).ready entity=mon.0 client_cookie=74f21361ade68088 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958002030 con 0x7f494c005000 2023-11-28T03:07:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f495800ce20 con 0x7f494c005000 2023-11-28T03:07:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4964dae700 1 -- 172.21.15.73:0/2353006572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4958005d30 con 0x7f494c005000 2023-11-28T03:07:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4968014700 1 -- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 msgr2=0x7f494c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4968014700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 0x7f494c00b8e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 rx=0x7f495800c480 tx=0x7f495800c790).stop 2023-11-28T03:07:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4968014700 1 -- 172.21.15.73:0/2353006572 shutdown_connections 2023-11-28T03:07:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4968014700 1 --2- 172.21.15.73:0/2353006572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f494c005000 0x7f494c00b8e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.430+0000 7f4968014700 1 -- 172.21.15.73:0/2353006572 >> 172.21.15.73:0/2353006572 conn(0x7f4960094930 msgr2=0x7f496006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.431+0000 7f4968014700 1 -- 172.21.15.73:0/2353006572 shutdown_connections 2023-11-28T03:07:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.431+0000 7f4968014700 1 -- 172.21.15.73:0/2353006572 wait complete. 2023-11-28T03:07:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 Processor -- start 2023-11-28T03:07:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 -- start start 2023-11-28T03:07:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c005000 0x7f49601513f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4960164f10 0x7f49600aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f496014f8b0 con 0x7f49601518e0 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4960151dd0 con 0x7f4960164f10 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.432+0000 7f4968014700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49601520b0 con 0x7f494c005000 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47718/0 (socket says 172.21.15.73:47718) 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 -- 172.21.15.73:0/1398939803 learned_addr learned my addr 172.21.15.73:0/1398939803 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4965db0700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c005000 0x7f49601513f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49665b1700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4960164f10 0x7f49600aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 -- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c005000 msgr2=0x7f49601513f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c005000 0x7f49601513f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 -- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4960164f10 msgr2=0x7f49600aeb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4960164f10 0x7f49600aeb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:10.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 -- 172.21.15.73:0/1398939803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49600ae360 con 0x7f49601518e0 2023-11-28T03:07:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f49655af700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f49500028c0 tx=0x7f4950007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950011590 con 0x7f49601518e0 2023-11-28T03:07:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49500116f0 con 0x7f49601518e0 2023-11-28T03:07:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49600ae640 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4950011a70 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.433+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4960153190 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.434+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49600ae9b0 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.435+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f495001c3b0 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.435+0000 7f4955ffb700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 0x7f4944061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.435+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f4950079c50 con 0x7f49601518e0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.435+0000 7f4965db0700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 0x7f4944061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.435+0000 7f4965db0700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 0x7f4944061610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4960153a90 tx=0x7f495c006f40).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.439+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f495007f020 con 0x7f49601518e0 2023-11-28T03:07:10.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:10.603+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f496006c860 con 0x7f49601518e0 2023-11-28T03:07:11.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.234+0000 7f4955ffb700 1 -- 172.21.15.73:0/1398939803 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v73) v1 ==== 169+0+0 (secure 0 0 0) 0x7f495004e8a0 con 0x7f49601518e0 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.236+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 msgr2=0x7f4944061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.236+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 0x7f4944061610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f4960153a90 tx=0x7f495c006f40).stop 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 msgr2=0x7f4960164a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 rx=0x7f49500028c0 tx=0x7f4950007bb0).stop 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 shutdown_connections 2023-11-28T03:07:11.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f494405f150 0x7f4944061610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4960164f10 0x7f49600aeb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f494c005000 0x7f49601513f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 --2- 172.21.15.73:0/1398939803 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49601518e0 0x7f4960164a20 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 >> 172.21.15.73:0/1398939803 conn(0x7f4960094930 msgr2=0x7f49600c0ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 shutdown_connections 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.237+0000 7f4968014700 1 -- 172.21.15.73:0/1398939803 wait complete. 2023-11-28T03:07:11.239 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:07:11.251 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:07:11.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.367+0000 7fcccde09700 1 Processor -- start 2023-11-28T03:07:11.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.367+0000 7fcccde09700 1 -- start start 2023-11-28T03:07:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fcccde09700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fccc8098d10 0x7fccc80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fcccde09700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccc8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fcccde09700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccc80c72d0 con 0x7fccc8064350 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fcccde09700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccc8154bc0 con 0x7fccc8098d10 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fcccde09700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c66c0 con 0x7fccc80c6190 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fccc7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fccc8064350 0x7fccc8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42652/0 (socket says 172.21.15.73:42652) 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.368+0000 7fccc7fff700 1 -- 172.21.15.73:0/1100446957 learned_addr learned my addr 172.21.15.73:0/1100446957 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc6ffd700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccc8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 --2- 172.21.15.73:0/1100446957 >> v2:172.21.15.150:3300/0 conn(0x7fccc8098d10 0x7fccc80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1007748131 0 0) 0x7fccc80c72d0 con 0x7fccc8064350 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fccb00033f0 con 0x7fccc8064350 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 596190224 0 0) 0x7fccb00033f0 con 0x7fccc8064350 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 msgr2=0x7fccc8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccc8063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 >> v2:172.21.15.150:3300/0 conn(0x7fccc8098d10 msgr2=0x7fccc80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 --2- 172.21.15.73:0/1100446957 >> v2:172.21.15.150:3300/0 conn(0x7fccc8098d10 0x7fccc80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc80c69a0 con 0x7fccc8064350 2023-11-28T03:07:11.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fccc00017f0 con 0x7fccc8064350 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 >> v1:172.21.15.73:6789/0 conn(0x7fccc8064350 legacy=0x7fccc8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccb00038e0 0x7fccb0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0004690 0x7fccb0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccb0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc8154bc0 con 0x7fccb00038e0 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80c66c0 con 0x7fccb0004690 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccb000b8e0 con 0x7fccc80c6190 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccb00038e0 0x7fccb0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc6ffd700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0004690 0x7fccb0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc7fff700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccb0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 -- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 msgr2=0x7fccb0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccb0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 -- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0004690 msgr2=0x7fccb0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccb0004690 0x7fccb0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc77fe700 1 -- 172.21.15.73:0/1100446957 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccb00033f0 con 0x7fccb00038e0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.369+0000 7fccc7fff700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccc80c6190 0x7fccb0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fccc77fe700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccb00038e0 0x7fccb0003d00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fccb8009a80 tx=0x7fccb8009060).ready entity=mon.0 client_cookie=d1ad2b32392c7a04 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb8009430 con 0x7fccb00038e0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccb800e640 con 0x7fccb00038e0 2023-11-28T03:07:11.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fccc67fc700 1 -- 172.21.15.73:0/1100446957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccb800e9c0 con 0x7fccb00038e0 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 -- 172.21.15.73:0/1100446957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccb00038e0 msgr2=0x7fccb0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 --2- 172.21.15.73:0/1100446957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccb00038e0 0x7fccb0003d00 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 rx=0x7fccb8009a80 tx=0x7fccb8009060).stop 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fccc7fff700 1 -- 172.21.15.73:0/1100446957 reap_dead start 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 -- 172.21.15.73:0/1100446957 shutdown_connections 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 -- 172.21.15.73:0/1100446957 >> 172.21.15.73:0/1100446957 conn(0x7fccc80cb310 msgr2=0x7fccc80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 -- 172.21.15.73:0/1100446957 shutdown_connections 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.370+0000 7fcccde09700 1 -- 172.21.15.73:0/1100446957 wait complete. 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.371+0000 7fcccde09700 1 Processor -- start 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 -- start start 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 0x7fccc80af670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 0x7fccc8156800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80afb60 con 0x7fccc813ed80 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80b0960 con 0x7fccc80b0df0 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fcccde09700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccc80afd50 con 0x7fccb00038e0 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fccc7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47730/0 (socket says 172.21.15.73:47730) 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fccc7fff700 1 -- 172.21.15.73:0/2435211463 learned_addr learned my addr 172.21.15.73:0/2435211463 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fccc6ffd700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 0x7fccc8156800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.372+0000 7fccc77fe700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 0x7fccc80af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 -- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 msgr2=0x7fccc80af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 0x7fccc80af670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 -- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 msgr2=0x7fccc8156800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 0x7fccc8156800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 -- 172.21.15.73:0/2435211463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc80b0030 con 0x7fccc813ed80 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc77fe700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 0x7fccc80af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc6ffd700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 0x7fccc8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccc7fff700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fccc0000f80 tx=0x7fccc00019c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000c7b0 con 0x7fccc813ed80 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc000c910 con 0x7fccc813ed80 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccc80b0310 con 0x7fccc813ed80 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc000ccd0 con 0x7fccc813ed80 2023-11-28T03:07:11.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.373+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccc80b1270 con 0x7fccc813ed80 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.374+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccc80b0af0 con 0x7fccc813ed80 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.374+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fccc001d450 con 0x7fccc813ed80 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.374+0000 7fccaf7fe700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 0x7fccb4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.375+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7fccc0079d50 con 0x7fccc813ed80 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.375+0000 7fccc77fe700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 0x7fccb4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.375+0000 7fccc77fe700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 0x7fccb4061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fccb8006d50 tx=0x7fccb801a000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.379+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fccc007d030 con 0x7fccc813ed80 2023-11-28T03:07:11.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.551+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fccc806c860 con 0x7fccc813ed80 2023-11-28T03:07:11.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.555+0000 7fccaf7fe700 1 -- 172.21.15.73:0/2435211463 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v8) v1 ==== 120+0+0 (secure 0 0 0) 0x7fccc004ee60 con 0x7fccc813ed80 2023-11-28T03:07:11.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 msgr2=0x7fccb4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 0x7fccb4061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fccb8006d50 tx=0x7fccb801a000).stop 2023-11-28T03:07:11.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 msgr2=0x7fccc81523d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 rx=0x7fccc0000f80 tx=0x7fccc00019c0).stop 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 shutdown_connections 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb405f150 0x7fccb4061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fccc813ed80 0x7fccc81523d0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fccb00038e0 0x7fccc80af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 --2- 172.21.15.73:0/2435211463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fccc80b0df0 0x7fccc8156800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 >> 172.21.15.73:0/2435211463 conn(0x7fccc80cb310 msgr2=0x7fccc80aa3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 shutdown_connections 2023-11-28T03:07:11.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.557+0000 7fcccde09700 1 -- 172.21.15.73:0/2435211463 wait complete. 2023-11-28T03:07:11.570 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:07:11.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.683+0000 7f3ccd8da700 1 Processor -- start 2023-11-28T03:07:11.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.683+0000 7f3ccd8da700 1 -- start start 2023-11-28T03:07:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3cc814e530 0x7f3cc80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 0x7f3cc814e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 0x7f3cc8158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc80c7bc0 con 0x7f3cc814e530 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cc8155920 con 0x7f3cc8154950 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd8da700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc8154e00 con 0x7f3cc8158570 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3cc814e530 0x7f3cc80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3cc814e530 0x7f3cc80c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47744/0 (socket says 172.21.15.73:47744) 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 -- 172.21.15.73:0/3581279486 learned_addr learned my addr 172.21.15.73:0/3581279486 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 -- 172.21.15.73:0/3581279486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 msgr2=0x7f3cc8158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd0d9700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 0x7f3cc8158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3cbffff700 1 --2- 172.21.15.73:0/3581279486 >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 0x7f3cc814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 0x7f3cc8158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 -- 172.21.15.73:0/3581279486 >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 msgr2=0x7f3cc814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3581279486 >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 0x7f3cc814e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccc8d8700 1 -- 172.21.15.73:0/3581279486 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc8155080 con 0x7f3cc814e530 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3ccd0d9700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 0x7f3cc8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.684+0000 7f3cbffff700 1 --2- 172.21.15.73:0/3581279486 >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 0x7f3cc814e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc814e530 0x7f3cc80c66f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f3cc8157f20 tx=0x7f3cc4009840).ready entity=mon.0 client_cookie=422b70435b0206d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3cbf7fe700 1 -- 172.21.15.73:0/3581279486 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400e070 con 0x7f3cc814e530 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3cbf7fe700 1 -- 172.21.15.73:0/3581279486 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc400ba50 con 0x7f3cc814e530 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3cbf7fe700 1 -- 172.21.15.73:0/3581279486 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc400bdd0 con 0x7f3cc814e530 2023-11-28T03:07:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3581279486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc814e530 msgr2=0x7f3cc80c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc814e530 0x7f3cc80c66f0 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 rx=0x7f3cc8157f20 tx=0x7f3cc4009840).stop 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3581279486 shutdown_connections 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc814e530 0x7f3cc80c66f0 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3581279486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8158570 0x7f3cc8158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3581279486 >> v2:172.21.15.150:3300/0 conn(0x7f3cc8154950 0x7f3cc814e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3581279486 >> 172.21.15.73:0/3581279486 conn(0x7f3cc8064990 msgr2=0x7f3cc8064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3581279486 shutdown_connections 2023-11-28T03:07:11.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.685+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3581279486 wait complete. 2023-11-28T03:07:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.686+0000 7f3ccd8da700 1 Processor -- start 2023-11-28T03:07:11.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 -- start start 2023-11-28T03:07:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cc814e530 0x7f3cc8141bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8154950 0x7f3cc80b0490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc814d380 con 0x7f3cc8158570 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80af9c0 con 0x7f3cc814e530 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd8da700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cc80b0220 con 0x7f3cc8154950 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd0d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd0d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47746/0 (socket says 172.21.15.73:47746) 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.687+0000 7f3ccd0d9700 1 -- 172.21.15.73:0/3961356593 learned_addr learned my addr 172.21.15.73:0/3961356593 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cc814e530 0x7f3cc8141bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3cbffff700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8154950 0x7f3cc80b0490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 -- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8154950 msgr2=0x7f3cc80b0490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8154950 0x7f3cc80b0490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 -- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cc814e530 msgr2=0x7f3cc8141bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cc814e530 0x7f3cc8141bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 -- 172.21.15.73:0/3961356593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc4009ae0 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd0d9700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f3cc000c880 tx=0x7f3cc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0010580 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc00106e0 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cc80afc60 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc0010a60 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.688+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cc80c21c0 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.689+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cc8062ee0 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.689+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3cc001b460 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.690+0000 7f3cbcff9700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 0x7f3cb4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.690+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f3cc007a090 con 0x7f3cc8158570 2023-11-28T03:07:11.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.690+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 0x7f3cb4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.690+0000 7f3ccc8d8700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 0x7f3cb4061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3cc400ba20 tx=0x7f3cc4009840).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.694+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3cc004b490 con 0x7f3cc8158570 2023-11-28T03:07:11.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.860+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3cc806c7c0 con 0x7f3cc8158570 2023-11-28T03:07:11.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.863+0000 7f3cbcff9700 1 -- 172.21.15.73:0/3961356593 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v9) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3cc004eab0 con 0x7f3cc8158570 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 msgr2=0x7f3cb4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 0x7f3cb4061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3cc400ba20 tx=0x7f3cc4009840).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 msgr2=0x7f3cc80aa030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 rx=0x7f3cc000c880 tx=0x7f3cc0008040).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 shutdown_connections 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3cb405f150 0x7f3cb4061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3cc8158570 0x7f3cc80aa030 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3cc814e530 0x7f3cc8141bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 --2- 172.21.15.73:0/3961356593 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3cc8154950 0x7f3cc80b0490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 >> 172.21.15.73:0/3961356593 conn(0x7f3cc8064990 msgr2=0x7f3cc814dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 shutdown_connections 2023-11-28T03:07:11.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.866+0000 7f3ccd8da700 1 -- 172.21.15.73:0/3961356593 wait complete. 2023-11-28T03:07:11.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-11-28T03:07:11.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.990+0000 7f90a8ae4700 1 Processor -- start 2023-11-28T03:07:11.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.990+0000 7f90a8ae4700 1 -- start start 2023-11-28T03:07:11.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a8ae4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f90a40caab0 0x7f90a40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a8ae4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90a40cae90 0x7f90a40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a8ae4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f90a40a9a90 con 0x7f90a40aa960 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a8ae4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a4154ce0 con 0x7f90a40cae90 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a8ae4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90a40a8e80 con 0x7f90a40caab0 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a1d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90a40cae90 0x7f90a40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a1d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f90a40cae90 0x7f90a40c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47750/0 (socket says 172.21.15.73:47750) 2023-11-28T03:07:11.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.991+0000 7f90a1d9b700 1 -- 172.21.15.73:0/437356523 learned_addr learned my addr 172.21.15.73:0/437356523 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a259c700 1 --2- 172.21.15.73:0/437356523 >> v2:172.21.15.150:3300/0 conn(0x7f90a40caab0 0x7f90a40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a1d9b700 1 -- 172.21.15.73:0/437356523 >> v2:172.21.15.150:3300/0 conn(0x7f90a40caab0 msgr2=0x7f90a40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a1d9b700 1 --2- 172.21.15.73:0/437356523 >> v2:172.21.15.150:3300/0 conn(0x7f90a40caab0 0x7f90a40aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a1d9b700 1 -- 172.21.15.73:0/437356523 >> v1:172.21.15.73:6789/0 conn(0x7f90a40aa960 legacy=0x7f90a40c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a1d9b700 1 -- 172.21.15.73:0/437356523 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90a40a9160 con 0x7f90a40cae90 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a1d9b700 1 --2- 172.21.15.73:0/437356523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c0ab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f90a40848c0 tx=0x7f90980090f0).ready entity=mon.0 client_cookie=2cdd8dc35796b655 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a159a700 1 -- 172.21.15.73:0/437356523 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800b600 con 0x7f90a40cae90 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a159a700 1 -- 172.21.15.73:0/437356523 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800b760 con 0x7f90a40cae90 2023-11-28T03:07:11.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a159a700 1 -- 172.21.15.73:0/437356523 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800bae0 con 0x7f90a40cae90 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a8ae4700 1 -- 172.21.15.73:0/437356523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 msgr2=0x7f90a40c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.992+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/437356523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c0ab0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 rx=0x7f90a40848c0 tx=0x7f90980090f0).stop 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 -- 172.21.15.73:0/437356523 shutdown_connections 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/437356523 >> v2:172.21.15.150:3300/0 conn(0x7f90a40caab0 0x7f90a40aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/437356523 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c0ab0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 -- 172.21.15.73:0/437356523 >> 172.21.15.73:0/437356523 conn(0x7f90a4094920 msgr2=0x7f90a406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 -- 172.21.15.73:0/437356523 shutdown_connections 2023-11-28T03:07:11.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.993+0000 7f90a8ae4700 1 -- 172.21.15.73:0/437356523 wait complete. 2023-11-28T03:07:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 Processor -- start 2023-11-28T03:07:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 -- start start 2023-11-28T03:07:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a40caab0 0x7f90a4158320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:11.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 0x7f90a40b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a41592c0 con 0x7f90a40cae90 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4158810 con 0x7f90a40afcc0 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a8ae4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90a4164b10 con 0x7f90a40caab0 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a1d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47754/0 (socket says 172.21.15.73:47754) 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.994+0000 7f90a1d9b700 1 -- 172.21.15.73:0/3645327404 learned_addr learned my addr 172.21.15.73:0/3645327404 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a2d9d700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 0x7f90a40b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a259c700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a40caab0 0x7f90a4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 -- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a40caab0 msgr2=0x7f90a4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a40caab0 0x7f90a4158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 -- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 msgr2=0x7f90a40b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 0x7f90a40b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 -- 172.21.15.73:0/3645327404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9098009370 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a2d9d700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 0x7f90a40b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a1d9b700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f9098009130 tx=0x7f9098017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909800b8d0 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f909800ba30 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90a40c5940 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f909801b660 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.995+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90a4158d00 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.996+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90a4164cc0 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.996+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f909801b9d0 con 0x7f90a40cae90 2023-11-28T03:07:12.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.997+0000 7f90927fc700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 0x7f9084061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.997+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f909807b550 con 0x7f90a40cae90 2023-11-28T03:07:12.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.997+0000 7f90a259c700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 0x7f9084061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:11.997+0000 7f90a259c700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 0x7f9084061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9094007460 tx=0x7f909400b040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:12.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.001+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f90a4164cc0 con 0x7f90a40cae90 2023-11-28T03:07:12.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.229+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f90a4164c80 con 0x7f90a40cae90 2023-11-28T03:07:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.276+0000 7f90927fc700 1 -- 172.21.15.73:0/3645327404 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v514) v1 ==== 154+0+0 (secure 0 0 0) 0x7f909801f0e0 con 0x7f90a40cae90 2023-11-28T03:07:12.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 msgr2=0x7f9084061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 0x7f9084061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f9094007460 tx=0x7f909400b040).stop 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 msgr2=0x7f90a40c5450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 rx=0x7f9098009130 tx=0x7f9098017000).stop 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 shutdown_connections 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f908405f150 0x7f9084061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f90a40afcc0 0x7f90a40b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90a40caab0 0x7f90a4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 --2- 172.21.15.73:0/3645327404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f90a40cae90 0x7f90a40c5450 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 >> 172.21.15.73:0/3645327404 conn(0x7f90a4094920 msgr2=0x7f90a40a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 shutdown_connections 2023-11-28T03:07:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.278+0000 7f90a8ae4700 1 -- 172.21.15.73:0/3645327404 wait complete. 2023-11-28T03:07:12.291 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-11-28T03:07:12.292 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-11-28T03:07:12.292 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:07:12.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-11-28T03:07:12.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.403+0000 7ff1dc214700 1 Processor -- start 2023-11-28T03:07:12.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.404+0000 7ff1dc214700 1 -- start start 2023-11-28T03:07:12.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.404+0000 7ff1dc214700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d4155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.404+0000 7ff1dc214700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff1d4066f40 0x7ff1d4095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1dc214700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1d40683d0 con 0x7ff1d40673c0 2023-11-28T03:07:12.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1dc214700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1d4067850 con 0x7ff1d4066f40 2023-11-28T03:07:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1dc214700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4067b30 con 0x7ff1d41544d0 2023-11-28T03:07:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1da7b1700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1d40673c0 0x7ff1d4063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:42662/0 (socket says 172.21.15.73:42662) 2023-11-28T03:07:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1da7b1700 1 -- 172.21.15.73:0/3530596854 learned_addr learned my addr 172.21.15.73:0/3530596854 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/3530596854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d97af700 1 --2- 172.21.15.73:0/3530596854 >> v2:172.21.15.150:3300/0 conn(0x7ff1d4066f40 0x7ff1d4095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 983883728 0 0) 0x7ff1d40683d0 con 0x7ff1d40673c0 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1c00033f0 con 0x7ff1d40673c0 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 -- 172.21.15.73:0/3530596854 >> v2:172.21.15.150:3300/0 conn(0x7ff1d4066f40 msgr2=0x7ff1d4095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/3530596854 >> v2:172.21.15.150:3300/0 conn(0x7ff1d4066f40 0x7ff1d4095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4077354419 0 0) 0x7ff1c00033f0 con 0x7ff1d40673c0 2023-11-28T03:07:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 -- 172.21.15.73:0/3530596854 >> v1:172.21.15.73:6789/0 conn(0x7ff1d40673c0 legacy=0x7ff1d4063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 -- 172.21.15.73:0/3530596854 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d4067e10 con 0x7ff1d41544d0 2023-11-28T03:07:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/3530596854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d4155750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff1d0006fd0 tx=0x7ff1d0006f40).ready entity=mon.2 client_cookie=e533079cdda905f9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.405+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0017f0 con 0x7ff1d41544d0 2023-11-28T03:07:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d0005880 con 0x7ff1d41544d0 2023-11-28T03:07:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1d8fae700 1 -- 172.21.15.73:0/3530596854 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d0005c00 con 0x7ff1d41544d0 2023-11-28T03:07:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 -- 172.21.15.73:0/3530596854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 msgr2=0x7ff1d4155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 --2- 172.21.15.73:0/3530596854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d4155750 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 rx=0x7ff1d0006fd0 tx=0x7ff1d0006f40).stop 2023-11-28T03:07:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 -- 172.21.15.73:0/3530596854 shutdown_connections 2023-11-28T03:07:12.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 --2- 172.21.15.73:0/3530596854 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d4155750 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 --2- 172.21.15.73:0/3530596854 >> v2:172.21.15.150:3300/0 conn(0x7ff1d4066f40 0x7ff1d4095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 -- 172.21.15.73:0/3530596854 >> 172.21.15.73:0/3530596854 conn(0x7ff1d40631d0 msgr2=0x7ff1d40c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:12.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 -- 172.21.15.73:0/3530596854 shutdown_connections 2023-11-28T03:07:12.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.406+0000 7ff1dc214700 1 -- 172.21.15.73:0/3530596854 wait complete. 2023-11-28T03:07:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.407+0000 7ff1dc214700 1 Processor -- start 2023-11-28T03:07:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.407+0000 7ff1dc214700 1 -- start start 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 0x7ff1d4157180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d40c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40bf7a0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d40c15a0 con 0x7ff1d4066f40 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1dc214700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1d4157670 con 0x7ff1d41544d0 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47760/0 (socket says 172.21.15.73:47760) 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 -- 172.21.15.73:0/2926157171 learned_addr learned my addr 172.21.15.73:0/2926157171 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 0x7ff1d4157180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1d97af700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d40c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 -- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 msgr2=0x7ff1d40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d40c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 -- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 msgr2=0x7ff1d4157180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 0x7ff1d4157180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 -- 172.21.15.73:0/2926157171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d0009250 con 0x7ff1d40c2d70 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1d97af700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d40c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 0x7ff1d4157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.408+0000 7ff1da7b1700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7ff1cc00cb40 tx=0x7ff1cc001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.409+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc004c40 con 0x7ff1d40c2d70 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.409+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1cc004da0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.409+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1cc0073a0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.409+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1d40beb20 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.409+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1d40bf0e0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.410+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1d40bf8e0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.410+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff1cc007750 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.410+0000 7ff1c9ffb700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 0x7ff1b8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.410+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7ff1cc079ff0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.410+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 0x7ff1b8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.411+0000 7ff1d9fb0700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 0x7ff1b8061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff1d0006f80 tx=0x7ff1d0015000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.414+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1d40bf8e0 con 0x7ff1d40c2d70 2023-11-28T03:07:12.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:12.644+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7ff1d40bf8e0 con 0x7ff1d40c2d70 2023-11-28T03:07:13.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.278+0000 7ff1c9ffb700 1 -- 172.21.15.73:0/2926157171 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v515) v1 ==== 159+0+0 (secure 0 0 0) 0x7ff1d40bf8e0 con 0x7ff1d40c2d70 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 msgr2=0x7ff1b8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 0x7ff1b8061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7ff1d0006f80 tx=0x7ff1d0015000).stop 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 msgr2=0x7ff1d40c10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 rx=0x7ff1cc00cb40 tx=0x7ff1cc001d90).stop 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 shutdown_connections 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1b805f150 0x7ff1b8061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1d40c2d70 0x7ff1d40c10b0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1d4066f40 0x7ff1d4157180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 --2- 172.21.15.73:0/2926157171 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1d41544d0 0x7ff1d40c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 >> 172.21.15.73:0/2926157171 conn(0x7ff1d40631d0 msgr2=0x7ff1d40641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 shutdown_connections 2023-11-28T03:07:13.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.280+0000 7ff1dc214700 1 -- 172.21.15.73:0/2926157171 wait complete. 2023-11-28T03:07:13.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:07:13.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.411+0000 7f21ee33f700 1 Processor -- start 2023-11-28T03:07:13.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.411+0000 7f21ee33f700 1 -- start start 2023-11-28T03:07:13.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ee33f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21e80c5ac0 0x7f21e80c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ee33f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e8155f20 con 0x7f21e80c5ea0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ee33f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e8156200 con 0x7f21e8153390 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ee33f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e8153990 con 0x7f21e80c5ac0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ec8dc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21e80c5ea0 0x7f21e8066a70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44368/0 (socket says 172.21.15.73:44368) 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ec8dc700 1 -- 172.21.15.73:0/2689285614 learned_addr learned my addr 172.21.15.73:0/2689285614 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 413809028 0 0) 0x7f21e8155f20 con 0x7f21e80c5ea0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21d40033f0 con 0x7f21e80c5ea0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e7fff700 1 --2- 172.21.15.73:0/2689285614 >> v2:172.21.15.150:3300/0 conn(0x7f21e80c5ac0 0x7f21e80c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 962924085 0 0) 0x7f21d40033f0 con 0x7f21e80c5ea0 2023-11-28T03:07:13.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 >> v2:172.21.15.150:3300/0 conn(0x7f21e80c5ac0 msgr2=0x7f21e80c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 --2- 172.21.15.73:0/2689285614 >> v2:172.21.15.150:3300/0 conn(0x7f21e80c5ac0 0x7f21e80c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 >> v1:172.21.15.150:6789/0 conn(0x7f21e8153390 legacy=0x7f21e8156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80bff70 con 0x7f21e80c5ea0 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f21e00017f0 con 0x7f21e80c5ea0 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 >> v1:172.21.15.73:6789/0 conn(0x7f21e80c5ea0 legacy=0x7f21e8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40038e0 0x7f21d4003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4004690 0x7f21d4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21d400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8153990 con 0x7f21d4005120 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d400c010 con 0x7f21d4004690 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d400c1d0 con 0x7f21d40038e0 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.412+0000 7f21ec8dc700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21d400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21e7fff700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40038e0 0x7f21d4003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21dffff700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4004690 0x7f21d4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 -- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40038e0 msgr2=0x7f21d4003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40038e0 0x7f21d4003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 -- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4004690 msgr2=0x7f21d4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4004690 0x7f21d4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 -- 172.21.15.73:0/2689285614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d40033f0 con 0x7f21d4005120 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21e7fff700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40038e0 0x7f21d4003ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21dffff700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4004690 0x7f21d4004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 -- 172.21.15.73:0/2689285614 reap_dead start 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ec8dc700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21d400b8e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f21e000c5c0 tx=0x7f21e8156170).ready entity=mon.0 client_cookie=7f48b73380c61a31 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000ce90 con 0x7f21d4005120 2023-11-28T03:07:13.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e0005b20 con 0x7f21d4005120 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21e77fe700 1 -- 172.21.15.73:0/2689285614 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e00070d0 con 0x7f21d4005120 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ee33f700 1 -- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 msgr2=0x7f21d400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.413+0000 7f21ee33f700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21d400b8e0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 rx=0x7f21e000c5c0 tx=0x7f21e8156170).stop 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.414+0000 7f21ee33f700 1 -- 172.21.15.73:0/2689285614 shutdown_connections 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.414+0000 7f21ee33f700 1 --2- 172.21.15.73:0/2689285614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21d400b8e0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.414+0000 7f21ee33f700 1 -- 172.21.15.73:0/2689285614 >> 172.21.15.73:0/2689285614 conn(0x7f21e8062700 msgr2=0x7f21e8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.414+0000 7f21ee33f700 1 -- 172.21.15.73:0/2689285614 shutdown_connections 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.414+0000 7f21ee33f700 1 -- 172.21.15.73:0/2689285614 wait complete. 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 Processor -- start 2023-11-28T03:07:13.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 -- start start 2023-11-28T03:07:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c0de0 0x7f21e80c1260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e80673e0 0x7f21e8067860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c1950 con 0x7f21d4005120 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c1750 con 0x7f21e80673e0 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.415+0000 7f21ee33f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80c3210 con 0x7f21e80c0de0 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41518/0 (socket says 172.21.15.73:41518) 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 -- 172.21.15.73:0/3562231872 learned_addr learned my addr 172.21.15.73:0/3562231872 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21ec8dc700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e80673e0 0x7f21e8067860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21dffff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c0de0 0x7f21e80c1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 -- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c0de0 msgr2=0x7f21e80c1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c0de0 0x7f21e80c1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 -- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e80673e0 msgr2=0x7f21e8067860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e80673e0 0x7f21e8067860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 -- 172.21.15.73:0/3562231872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80642e0 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e7fff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f21d800f5c0 tx=0x7f21d8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d800fe90 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8013610 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d8013990 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e8067d50 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.416+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e8063c90 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.417+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e806c860 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.417+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f21d8013d00 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.418+0000 7f21e4ff9700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.418+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f21d8079f40 con 0x7f21d4005120 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.418+0000 7f21dffff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.418+0000 7f21dffff700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f21e8067f80 tx=0x7f21d0009040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.422+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21d804b340 con 0x7f21d4005120 2023-11-28T03:07:13.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.587+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f21e80c4f80 con 0x7f21d4005120 2023-11-28T03:07:13.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.590+0000 7f21e4ff9700 1 -- 172.21.15.73:0/3562231872 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f21d804e960 con 0x7f21d4005120 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 msgr2=0x7f21c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f21e8067f80 tx=0x7f21d0009040).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 msgr2=0x7f21e8150230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 rx=0x7f21d800f5c0 tx=0x7f21d8008040).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 shutdown_connections 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e80673e0 0x7f21e8067860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d4005120 0x7f21e8150230 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 --2- 172.21.15.73:0/3562231872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c0de0 0x7f21e80c1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 >> 172.21.15.73:0/3562231872 conn(0x7f21e8062700 msgr2=0x7f21e80c6c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:13.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 shutdown_connections 2023-11-28T03:07:13.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.592+0000 7f21ee33f700 1 -- 172.21.15.73:0/3562231872 wait complete. 2023-11-28T03:07:13.605 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:07:13.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:13.641 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:13.681 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:07:13.682 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:07:13.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:13.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.794+0000 7f66f86dc700 1 Processor -- start 2023-11-28T03:07:13.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.794+0000 7f66f86dc700 1 -- start start 2023-11-28T03:07:13.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f86dc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f01544d0 0x7f66f0155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f86dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00673c0 0x7f66f015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f86dc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66f00683d0 con 0x7f66f0066f40 2023-11-28T03:07:13.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f86dc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66f0067850 con 0x7f66f01544d0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f86dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f0067b30 con 0x7f66f00673c0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f01544d0 0x7f66f0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f66f01544d0 0x7f66f0155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41522/0 (socket says 172.21.15.73:41522) 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 -- 172.21.15.73:0/3073840327 learned_addr learned my addr 172.21.15.73:0/3073840327 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6c79700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00673c0 0x7f66f015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 -- 172.21.15.73:0/3073840327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00673c0 msgr2=0x7f66f015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00673c0 0x7f66f015e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 -- 172.21.15.73:0/3073840327 >> v1:172.21.15.73:6789/0 conn(0x7f66f0066f40 legacy=0x7f66f0095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 -- 172.21.15.73:0/3073840327 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66f0067e10 con 0x7f66f01544d0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f5476700 1 -- 172.21.15.73:0/3073840327 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 207013524 0 0) 0x7f66f00683d0 con 0x7f66f0066f40 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f6478700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f0155750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f66e8009810 tx=0x7f66e8009060).ready entity=mon.0 client_cookie=4df2e1cbd1c1ff86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f5476700 1 -- 172.21.15.73:0/3073840327 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8005880 con 0x7f66f01544d0 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f5476700 1 -- 172.21.15.73:0/3073840327 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e80059e0 con 0x7f66f01544d0 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.796+0000 7f66f5476700 1 -- 172.21.15.73:0/3073840327 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e8005d60 con 0x7f66f01544d0 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 -- 172.21.15.73:0/3073840327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 msgr2=0x7f66f0155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f0155750 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 rx=0x7f66e8009810 tx=0x7f66e8009060).stop 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 -- 172.21.15.73:0/3073840327 shutdown_connections 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00673c0 0x7f66f015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 --2- 172.21.15.73:0/3073840327 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f0155750 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 -- 172.21.15.73:0/3073840327 >> 172.21.15.73:0/3073840327 conn(0x7f66f00631d0 msgr2=0x7f66f00c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 -- 172.21.15.73:0/3073840327 shutdown_connections 2023-11-28T03:07:13.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.797+0000 7f66f86dc700 1 -- 172.21.15.73:0/3073840327 wait complete. 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.798+0000 7f66f86dc700 1 Processor -- start 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.798+0000 7f66f86dc700 1 -- start start 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f0066f40 0x7f66f014e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00a6e30 0x7f66f00a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00a96b0 con 0x7f66f01544d0 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00a7760 con 0x7f66f0066f40 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f86dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66f00a8aa0 con 0x7f66f00a6e30 2023-11-28T03:07:13.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41534/0 (socket says 172.21.15.73:41534) 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 -- 172.21.15.73:0/2893224729 learned_addr learned my addr 172.21.15.73:0/2893224729 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f6478700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f0066f40 0x7f66f014e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f6c79700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00a6e30 0x7f66f00a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 -- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00a6e30 msgr2=0x7f66f00a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00a6e30 0x7f66f00a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 -- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f0066f40 msgr2=0x7f66f014e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f0066f40 0x7f66f014e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 -- 172.21.15.73:0/2893224729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66e8009300 con 0x7f66f01544d0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.799+0000 7f66f5c77700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f66e400c930 tx=0x7f66e4001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.800+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400a780 con 0x7f66f01544d0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.800+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66e400a900 con 0x7f66f01544d0 2023-11-28T03:07:13.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.800+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66f00a8d80 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.800+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66e400acc0 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.800+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66f00a9340 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.801+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66f006c860 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.801+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f66e4012610 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.801+0000 7f66ee7fc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 0x7f66dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.801+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(73..73 src has 1..73) v4 ==== 5727+0+0 (secure 0 0 0) 0x7f66e407a070 con 0x7f66f01544d0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.801+0000 7f66f6478700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 0x7f66dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:13.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.802+0000 7f66f6478700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 0x7f66dc061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f66e8009fd0 tx=0x7f66e8016000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:13.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:13.805+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66e404b4d0 con 0x7f66f01544d0 2023-11-28T03:07:14.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.008+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66f00a97f0 con 0x7f66f01544d0 2023-11-28T03:07:14.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.009+0000 7f66ee7fc700 1 -- 172.21.15.73:0/2893224729 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f66e404eaf0 con 0x7f66f01544d0 2023-11-28T03:07:14.010 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:14.011 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":22,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5081,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/364879791","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":364879791},{"type":"v1","addr":"172.21.15.150:6833","nonce":364879791}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5757,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6835/2238218878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2238218878},{"type":"v1","addr":"172.21.15.73:6835","nonce":2238218878}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":5769,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/326506390","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":326506390},{"type":"v1","addr":"172.21.15.150:6835","nonce":326506390}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":15},{"gid":6153,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/164069555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6836","nonce":164069555},{"type":"v1","addr":"172.21.15.73:6837","nonce":164069555}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":21}],"filesystems":[]} 2023-11-28T03:07:14.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 msgr2=0x7f66dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 0x7f66dc061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f66e8009fd0 tx=0x7f66e8016000).stop 2023-11-28T03:07:14.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 msgr2=0x7f66f00ac0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 rx=0x7f66e400c930 tx=0x7f66e4001630).stop 2023-11-28T03:07:14.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 shutdown_connections 2023-11-28T03:07:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f66dc05f150 0x7f66dc061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f66f00a6e30 0x7f66f00a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66f0066f40 0x7f66f014e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 --2- 172.21.15.73:0/2893224729 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66f01544d0 0x7f66f00ac0c0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 >> 172.21.15.73:0/2893224729 conn(0x7f66f00631d0 msgr2=0x7f66f00bf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:14.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 shutdown_connections 2023-11-28T03:07:14.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.011+0000 7f66f86dc700 1 -- 172.21.15.73:0/2893224729 wait complete. 2023-11-28T03:07:14.015 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 22 2023-11-28T03:07:14.024 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:07:14.025 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:07:14.025 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:07:14.025 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:14.025 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:07:14.026 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:07:14.026 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:07:14.026 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:14.027 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:07:14.027 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:07:14.027 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:07:14.027 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:14.028 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:07:14.028 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:07:14.028 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:07:14.028 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:14.034 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:07:14.034 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:07:14.035 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:07:14.035 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:07:14.038 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:07:14.038 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:07:14.039 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:07:14.039 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:07:14.077 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:07:14.077 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:14.077 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:07:14.078 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:14.081 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:07:14.081 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:14.082 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:07:14.082 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:14.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:07:14.149 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:07:14.149 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:07:14.150 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:07:14.149+0000 7f5f2e9f0700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T03:07:14.150 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:14.149+0000 7f4ff9c51700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-11-28T03:07:14.162 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:07:14.163 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:07:14.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.206+0000 7f3e9610c700 1 Processor -- start 2023-11-28T03:07:14.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.206+0000 7f3e9610c700 1 -- start start 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e9610c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e900bde50 0x7f3e900c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e9610c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e90155650 con 0x7f3e900c6b30 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e9610c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e90155f60 con 0x7f3e90155ae0 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e9610c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9014bff0 con 0x7f3e900bde50 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e900bde50 0x7f3e900c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e900bde50 0x7f3e900c2ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41580/0 (socket says 172.21.15.73:41580) 2023-11-28T03:07:14.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3e900c6b30 0x7f3e901524b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44388/0 (socket says 172.21.15.73:44388) 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 -- 172.21.15.73:0/3606843657 learned_addr learned my addr 172.21.15.73:0/3606843657 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 -- 172.21.15.73:0/3606843657 >> v1:172.21.15.150:6789/0 conn(0x7f3e90155ae0 legacy=0x7f3e900bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 -- 172.21.15.73:0/3606843657 >> v1:172.21.15.73:6789/0 conn(0x7f3e900c6b30 legacy=0x7f3e901524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8e7fc700 1 -- 172.21.15.73:0/3606843657 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 9019694 0 0) 0x7f3e90155650 con 0x7f3e900c6b30 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 -- 172.21.15.73:0/3606843657 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e90063b60 con 0x7f3e900bde50 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8ffff700 1 --2- 172.21.15.73:0/3606843657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e900c2ce0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f3e88006fd0 tx=0x7f3e880090f0).ready entity=mon.0 client_cookie=8f2645f1f6dd6c55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8e7fc700 1 -- 172.21.15.73:0/3606843657 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800b600 con 0x7f3e900bde50 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8e7fc700 1 -- 172.21.15.73:0/3606843657 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e8800b760 con 0x7f3e900bde50 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.207+0000 7f3e8e7fc700 1 -- 172.21.15.73:0/3606843657 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e8800bb00 con 0x7f3e900bde50 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 -- 172.21.15.73:0/3606843657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 msgr2=0x7f3e900c2ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 --2- 172.21.15.73:0/3606843657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e900c2ce0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 rx=0x7f3e88006fd0 tx=0x7f3e880090f0).stop 2023-11-28T03:07:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 -- 172.21.15.73:0/3606843657 shutdown_connections 2023-11-28T03:07:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 --2- 172.21.15.73:0/3606843657 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e900c2ce0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 -- 172.21.15.73:0/3606843657 >> 172.21.15.73:0/3606843657 conn(0x7f3e900c8230 msgr2=0x7f3e900c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 -- 172.21.15.73:0/3606843657 shutdown_connections 2023-11-28T03:07:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.208+0000 7f3e9610c700 1 -- 172.21.15.73:0/3606843657 wait complete. 2023-11-28T03:07:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.209+0000 7f3e9610c700 1 Processor -- start 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.209+0000 7f3e9610c700 1 -- start start 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 0x7f3e90138c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 0x7f3e901620c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90165670 con 0x7f3e900bde50 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90165950 con 0x7f3e90155ae0 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e9610c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90165c30 con 0x7f3e900c6b30 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41596/0 (socket says 172.21.15.73:41596) 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 -- 172.21.15.73:0/1142838459 learned_addr learned my addr 172.21.15.73:0/1142838459 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8ffff700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 0x7f3e901620c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8effd700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 0x7f3e90138c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 -- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 msgr2=0x7f3e90138c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 0x7f3e90138c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.210+0000 7f3e8f7fe700 1 -- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 msgr2=0x7f3e901620c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e8f7fe700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 0x7f3e901620c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e8f7fe700 1 -- 172.21.15.73:0/1142838459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e88009370 con 0x7f3e900bde50 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e8effd700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 0x7f3e90138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e8ffff700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 0x7f3e901620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e8f7fe700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f3e8000c7f0 tx=0x7f3e80007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80005c00 con 0x7f3e900bde50 2023-11-28T03:07:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e80005d60 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e90161ba0 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e80007360 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.211+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e90165310 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.212+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90161a10 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.212+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3e80007750 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.212+0000 7f3e877fe700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 0x7f3e70061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.212+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5699+0+0 (secure 0 0 0) 0x7f3e8007a130 con 0x7f3e900bde50 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.212+0000 7f3e8effd700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 0x7f3e70061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:14.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.213+0000 7f3e8effd700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 0x7f3e70061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3e9013f510 tx=0x7f3e78009040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:14.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.216+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e8004b5a0 con 0x7f3e900bde50 2023-11-28T03:07:14.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:14.395+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f3e90165a90 con 0x7f3e900bde50 2023-11-28T03:07:15.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.281+0000 7f3e877fe700 1 -- 172.21.15.73:0/1142838459 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v78) v1 ==== 97+0+0 (secure 0 0 0) 0x7f3e90165a90 con 0x7f3e900bde50 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 msgr2=0x7f3e70061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 0x7f3e70061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f3e9013f510 tx=0x7f3e78009040).stop 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 msgr2=0x7f3e90164e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 rx=0x7f3e8000c7f0 tx=0x7f3e80007ec0).stop 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 shutdown_connections 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e7005f150 0x7f3e70061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e90155ae0 0x7f3e901620c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e900bde50 0x7f3e90164e20 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 --2- 172.21.15.73:0/1142838459 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e900c6b30 0x7f3e90138c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.284+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 >> 172.21.15.73:0/1142838459 conn(0x7f3e900c8230 msgr2=0x7f3e9014b3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:15.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.285+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 shutdown_connections 2023-11-28T03:07:15.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.285+0000 7f3e9610c700 1 -- 172.21.15.73:0/1142838459 wait complete. 2023-11-28T03:07:15.287 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:07:15.301 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:07:15.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.417+0000 7f31a5a40700 1 Processor -- start 2023-11-28T03:07:15.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.417+0000 7f31a5a40700 1 -- start start 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31a5a40700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31a5a40700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 0x7f31a00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31a5a40700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f31a00a9a90 con 0x7f31a00aa960 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31a5a40700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f31a0154ce0 con 0x7f31a00cae90 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31a5a40700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00a8e80 con 0x7f31a00caab0 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f319f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f31a00aa960 0x7f31a00c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44392/0 (socket says 172.21.15.73:44392) 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f319f7fe700 1 -- 172.21.15.73:0/890405469 learned_addr learned my addr 172.21.15.73:0/890405469 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f319effd700 1 --2- 172.21.15.73:0/890405469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f31967fc700 1 --2- 172.21.15.73:0/890405469 >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 0x7f31a00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f319e7fc700 1 -- 172.21.15.73:0/890405469 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741178808 0 0) 0x7f31a00a9a90 con 0x7f31a00aa960 2023-11-28T03:07:15.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.418+0000 7f319e7fc700 1 -- 172.21.15.73:0/890405469 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f318c0033f0 con 0x7f31a00aa960 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319effd700 1 -- 172.21.15.73:0/890405469 >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 msgr2=0x7f31a00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319effd700 1 --2- 172.21.15.73:0/890405469 >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 0x7f31a00c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319effd700 1 -- 172.21.15.73:0/890405469 >> v1:172.21.15.73:6789/0 conn(0x7f31a00aa960 legacy=0x7f31a00c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319effd700 1 -- 172.21.15.73:0/890405469 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31a00a9160 con 0x7f31a00caab0 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31967fc700 1 --2- 172.21.15.73:0/890405469 >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 0x7f31a00c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319effd700 1 --2- 172.21.15.73:0/890405469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a00aa470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3190006fd0 tx=0x7f3190006f40).ready entity=mon.2 client_cookie=ab39693a5f74173c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319e7fc700 1 -- 172.21.15.73:0/890405469 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190005880 con 0x7f31a00caab0 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319e7fc700 1 -- 172.21.15.73:0/890405469 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f31900059e0 con 0x7f31a00caab0 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f319e7fc700 1 -- 172.21.15.73:0/890405469 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3190005d60 con 0x7f31a00caab0 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 -- 172.21.15.73:0/890405469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 msgr2=0x7f31a00aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 --2- 172.21.15.73:0/890405469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a00aa470 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 rx=0x7f3190006fd0 tx=0x7f3190006f40).stop 2023-11-28T03:07:15.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 -- 172.21.15.73:0/890405469 shutdown_connections 2023-11-28T03:07:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 --2- 172.21.15.73:0/890405469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a00aa470 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 --2- 172.21.15.73:0/890405469 >> v2:172.21.15.150:3300/0 conn(0x7f31a00cae90 0x7f31a00c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.419+0000 7f31a5a40700 1 -- 172.21.15.73:0/890405469 >> 172.21.15.73:0/890405469 conn(0x7f31a0094920 msgr2=0x7f31a006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.420+0000 7f31a5a40700 1 -- 172.21.15.73:0/890405469 shutdown_connections 2023-11-28T03:07:15.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.420+0000 7f31a5a40700 1 -- 172.21.15.73:0/890405469 wait complete. 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.421+0000 7f31a5a40700 1 Processor -- start 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.421+0000 7f31a5a40700 1 -- start start 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.421+0000 7f31a5a40700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a014ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 0x7f31a0158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a014fb50 con 0x7f31a00cae90 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31a014eac0 con 0x7f31a0150900 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f31a00abbb0 con 0x7f31a00caab0 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41600/0 (socket says 172.21.15.73:41600) 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 -- 172.21.15.73:0/3086096265 learned_addr learned my addr 172.21.15.73:0/3086096265 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f319effd700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a014ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f319f7fe700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 0x7f31a0158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 -- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 msgr2=0x7f31a014ec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a014ec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 -- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 msgr2=0x7f31a0158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 0x7f31a0158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 -- 172.21.15.73:0/3086096265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31880092a0 con 0x7f31a00cae90 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f319f7fe700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 0x7f31a0158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31967fc700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f318800e900 tx=0x7f3188009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318800eea0 con 0x7f31a00cae90 2023-11-28T03:07:15.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f318800c850 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3190009250 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f318800cbd0 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.422+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31a014f590 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.423+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31a006c860 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.423+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f318801b400 con 0x7f31a00cae90 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.424+0000 7f31977fe700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 0x7f31800615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.424+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f318807a3b0 con 0x7f31a00cae90 2023-11-28T03:07:15.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.424+0000 7f319effd700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 0x7f31800615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.424+0000 7f319effd700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 0x7f31800615e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3190006f80 tx=0x7f3190015000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.428+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f318804b970 con 0x7f31a00cae90 2023-11-28T03:07:15.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.654+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f31a00a9440 con 0x7f31a00cae90 2023-11-28T03:07:15.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.654+0000 7f31977fe700 1 -- 172.21.15.73:0/3086096265 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3188016020 con 0x7f31a00cae90 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:15.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:15.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:07:15.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:07:15.662 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:07:15.663 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.665 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.667 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:07:15.671 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:15.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:07:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:15.673 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:07:15.673 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 msgr2=0x7f31800615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 0x7f31800615e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f3190006f80 tx=0x7f3190015000).stop 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 msgr2=0x7f31a014e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 rx=0x7f318800e900 tx=0x7f3188009060).stop 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 shutdown_connections 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f318005f120 0x7f31800615e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f31a0150900 0x7f31a0158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f31a00caab0 0x7f31a014ec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 --2- 172.21.15.73:0/3086096265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f31a00cae90 0x7f31a014e5d0 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 >> 172.21.15.73:0/3086096265 conn(0x7f31a0094920 msgr2=0x7f31a00ae220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:15.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 shutdown_connections 2023-11-28T03:07:15.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.656+0000 7f31a5a40700 1 -- 172.21.15.73:0/3086096265 wait complete. 2023-11-28T03:07:15.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.790+0000 7f476cc52700 1 Processor -- start 2023-11-28T03:07:15.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.790+0000 7f476cc52700 1 -- start start 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476cc52700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4768157f50 0x7f47680c59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476cc52700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476cc52700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47680a88b0 con 0x7f4768154270 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476cc52700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47680c63c0 con 0x7f4768157f50 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476cc52700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47680c77b0 con 0x7f47680be910 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f4766d9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f475dd9b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4768157f50 0x7f47680c59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f4766d9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:60102/0 (socket says 172.21.15.73:60102) 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f4766d9d700 1 -- 172.21.15.73:0/3784115408 learned_addr learned my addr 172.21.15.73:0/3784115408 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.791+0000 7f476659c700 1 --1- 172.21.15.73:0/3784115408 >> v1:172.21.15.150:6789/0 conn(0x7f4768154270 0x7f47680a81a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43368/0 (socket says 172.21.15.73:43368) 2023-11-28T03:07:15.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f475dd9b700 1 -- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 msgr2=0x7f47680c5ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f475dd9b700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f475dd9b700 1 -- 172.21.15.73:0/3784115408 >> v1:172.21.15.150:6789/0 conn(0x7f4768154270 legacy=0x7f47680a81a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f475dd9b700 1 -- 172.21.15.73:0/3784115408 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4768158450 con 0x7f4768157f50 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f4765d9b700 1 -- 172.21.15.73:0/3784115408 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948781032 0 0) 0x7f47680a88b0 con 0x7f4768154270 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f4766d9d700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f475dd9b700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768157f50 0x7f47680c59e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f4750009760 tx=0x7f4750009060).ready entity=mon.1 client_cookie=4c8e5fc67c2b9801 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f4765d9b700 1 -- 172.21.15.73:0/3784115408 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000f510 con 0x7f4768157f50 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f4765d9b700 1 -- 172.21.15.73:0/3784115408 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f475000f670 con 0x7f4768157f50 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f4765d9b700 1 -- 172.21.15.73:0/3784115408 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f475000f9f0 con 0x7f4768157f50 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f476cc52700 1 -- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768157f50 msgr2=0x7f47680c59e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.792+0000 7f476cc52700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768157f50 0x7f47680c59e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 rx=0x7f4750009760 tx=0x7f4750009060).stop 2023-11-28T03:07:15.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 -- 172.21.15.73:0/3784115408 shutdown_connections 2023-11-28T03:07:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 --2- 172.21.15.73:0/3784115408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768157f50 0x7f47680c59e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 -- 172.21.15.73:0/3784115408 >> 172.21.15.73:0/3784115408 conn(0x7f4768062700 msgr2=0x7f4768062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 -- 172.21.15.73:0/3784115408 shutdown_connections 2023-11-28T03:07:15.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.793+0000 7f476cc52700 1 -- 172.21.15.73:0/3784115408 wait complete. 2023-11-28T03:07:15.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.794+0000 7f476cc52700 1 Processor -- start 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.794+0000 7f476cc52700 1 -- start start 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768154270 0x7f47680ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4768168d90 con 0x7f4768157f50 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4768154e40 con 0x7f4768154270 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4768168ab0 con 0x7f47680be910 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41614/0 (socket says 172.21.15.73:41614) 2023-11-28T03:07:15.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 -- 172.21.15.73:0/3904584091 learned_addr learned my addr 172.21.15.73:0/3904584091 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f475dd9b700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768154270 0x7f47680ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476659c700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 -- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 msgr2=0x7f47680c5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 -- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768154270 msgr2=0x7f47680ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768154270 0x7f47680ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 -- 172.21.15.73:0/3904584091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760009ae0 con 0x7f4768157f50 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f4766d9d700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f4760009fb0 tx=0x7f476000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000c9c0 con 0x7f4768157f50 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476000f040 con 0x7f4768157f50 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47500092a0 con 0x7f4768157f50 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000cd30 con 0x7f4768157f50 2023-11-28T03:07:15.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.795+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4768154ad0 con 0x7f4768157f50 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.796+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4768154650 con 0x7f4768157f50 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.797+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4760014670 con 0x7f4768157f50 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.797+0000 7f475effd700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 0x7f4748061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.797+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f4760079da0 con 0x7f4768157f50 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.797+0000 7f476659c700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 0x7f4748061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:15.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.798+0000 7f476659c700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 0x7f4748061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f47580011e0 tx=0x7f4758007bd0).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:15.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.801+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f476004aaf0 con 0x7f4768157f50 2023-11-28T03:07:15.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.964+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f476806c860 con 0x7f4768157f50 2023-11-28T03:07:15.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.965+0000 7f475effd700 1 -- 172.21.15.73:0/3904584091 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+13712 (secure 0 0 0) 0x7f4768154650 con 0x7f4768157f50 2023-11-28T03:07:15.966 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:15.966 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":78,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:15.278664+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":67,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 msgr2=0x7f4748061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 0x7f4748061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f47580011e0 tx=0x7f4758007bd0).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 msgr2=0x7f47680ad5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 rx=0x7f4760009fb0 tx=0x7f476000b040).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 shutdown_connections 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f474805f150 0x7f4748061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4768157f50 0x7f47680ad5f0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47680be910 0x7f47680c5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 --2- 172.21.15.73:0/3904584091 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4768154270 0x7f47680ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:15.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 >> 172.21.15.73:0/3904584091 conn(0x7f4768062700 msgr2=0x7f4768153480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.966+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 shutdown_connections 2023-11-28T03:07:15.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:15.967+0000 7f476cc52700 1 -- 172.21.15.73:0/3904584091 wait complete. 2023-11-28T03:07:15.980 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:07:15.980 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:07:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.094+0000 7fe695b73700 1 Processor -- start 2023-11-28T03:07:16.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.094+0000 7fe695b73700 1 -- start start 2023-11-28T03:07:16.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.094+0000 7fe695b73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe695b73700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 0x7fe6901587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe695b73700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe690158cc0 con 0x7fe690098d10 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe695b73700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6900c72d0 con 0x7fe690158320 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe695b73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe690064570 con 0x7fe6900c6190 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68ffff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 0x7fe6901587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:60132/0 (socket says 172.21.15.73:60132) 2023-11-28T03:07:16.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 -- 172.21.15.73:0/515154004 learned_addr learned my addr 172.21.15.73:0/515154004 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68f7fe700 1 --1- 172.21.15.73:0/515154004 >> v1:172.21.15.150:6789/0 conn(0x7fe690098d10 0x7fe690063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43378/0 (socket says 172.21.15.73:43378) 2023-11-28T03:07:16.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 -- 172.21.15.73:0/515154004 >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 msgr2=0x7fe6901587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:16.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 --2- 172.21.15.73:0/515154004 >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 0x7fe6901587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 -- 172.21.15.73:0/515154004 >> v1:172.21.15.150:6789/0 conn(0x7fe690098d10 legacy=0x7fe690063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 -- 172.21.15.73:0/515154004 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe690154bc0 con 0x7fe6900c6190 2023-11-28T03:07:16.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68ffff700 1 --2- 172.21.15.73:0/515154004 >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 0x7fe6901587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:16.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.095+0000 7fe68effd700 1 --2- 172.21.15.73:0/515154004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe684006fd0 tx=0x7fe6840090f0).ready entity=mon.2 client_cookie=f5acb3fe644aab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe68e7fc700 1 -- 172.21.15.73:0/515154004 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400b580 con 0x7fe6900c6190 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe68e7fc700 1 -- 172.21.15.73:0/515154004 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe68400b6e0 con 0x7fe6900c6190 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe68e7fc700 1 -- 172.21.15.73:0/515154004 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe68400ba60 con 0x7fe6900c6190 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 -- 172.21.15.73:0/515154004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 msgr2=0x7fe6900c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 --2- 172.21.15.73:0/515154004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 rx=0x7fe684006fd0 tx=0x7fe6840090f0).stop 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 -- 172.21.15.73:0/515154004 shutdown_connections 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 --2- 172.21.15.73:0/515154004 >> v2:172.21.15.150:3300/0 conn(0x7fe690158320 0x7fe6901587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 --2- 172.21.15.73:0/515154004 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c7760 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 -- 172.21.15.73:0/515154004 >> 172.21.15.73:0/515154004 conn(0x7fe6900cb310 msgr2=0x7fe6900cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 -- 172.21.15.73:0/515154004 shutdown_connections 2023-11-28T03:07:16.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.096+0000 7fe695b73700 1 -- 172.21.15.73:0/515154004 wait complete. 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.097+0000 7fe695b73700 1 Processor -- start 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.097+0000 7fe695b73700 1 -- start start 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe690098d10 0x7fe690157010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe690155d20 con 0x7fe690158320 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe69014ba60 con 0x7fe690098d10 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe695b73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6900c2d90 con 0x7fe6900c6190 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe68ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41622/0 (socket says 172.21.15.73:41622) 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe68ffff700 1 -- 172.21.15.73:0/3471253159 learned_addr learned my addr 172.21.15.73:0/3471253159 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe68f7fe700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe690098d10 0x7fe690157010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.098+0000 7fe68effd700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 -- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 msgr2=0x7fe6900c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 -- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe690098d10 msgr2=0x7fe690157010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe690098d10 0x7fe690157010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 -- 172.21.15.73:0/3471253159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6880092a0 con 0x7fe690158320 2023-11-28T03:07:16.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe68ffff700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe68800e930 tx=0x7fe688009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012600 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe688012760 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe684009370 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe688012b00 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.099+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe690156250 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.100+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe69006c860 con 0x7fe690158320 2023-11-28T03:07:16.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.100+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe68801b400 con 0x7fe690158320 2023-11-28T03:07:16.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.101+0000 7fe6777fe700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 0x7fe67c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.101+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe68807a480 con 0x7fe690158320 2023-11-28T03:07:16.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.101+0000 7fe68f7fe700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 0x7fe67c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.101+0000 7fe68f7fe700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 0x7fe67c061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe680001a10 tx=0x7fe680001560).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.105+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe68804b980 con 0x7fe690158320 2023-11-28T03:07:16.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.271+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fe6900c66c0 con 0x7fe690158320 2023-11-28T03:07:16.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.272+0000 7fe6777fe700 1 -- 172.21.15.73:0/3471253159 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v78) v1 ==== 110+0+10 (secure 0 0 0) 0x7fe688016020 con 0x7fe690158320 2023-11-28T03:07:16.273 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 msgr2=0x7fe67c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 0x7fe67c061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fe680001a10 tx=0x7fe680001560).stop 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 msgr2=0x7fe6900c28a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 rx=0x7fe68800e930 tx=0x7fe688009060).stop 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 shutdown_connections 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe67c05f150 0x7fe67c061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe690158320 0x7fe6900c28a0 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe690098d10 0x7fe690157010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 --2- 172.21.15.73:0/3471253159 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6900c6190 0x7fe6900c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 >> 172.21.15.73:0/3471253159 conn(0x7fe6900cb310 msgr2=0x7fe69014c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 shutdown_connections 2023-11-28T03:07:16.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.274+0000 7fe695b73700 1 -- 172.21.15.73:0/3471253159 wait complete. 2023-11-28T03:07:16.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:07:16.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.402+0000 7fdd52b70700 1 Processor -- start 2023-11-28T03:07:16.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.402+0000 7fdd52b70700 1 -- start start 2023-11-28T03:07:16.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd52b70700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd52b70700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdd4c0bdb30 0x7fdd4c06a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd52b70700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd4c1554d0 con 0x7fdd4c0bd680 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd52b70700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c06ab10 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd52b70700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd4c1548c0 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41630/0 (socket says 172.21.15.73:41630) 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 -- 172.21.15.73:0/2483656889 learned_addr learned my addr 172.21.15.73:0/2483656889 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5110d700 1 --2- 172.21.15.73:0/2483656889 >> v2:172.21.15.150:3300/0 conn(0x7fdd4c0bdb30 0x7fdd4c06a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 -- 172.21.15.73:0/2483656889 >> v2:172.21.15.150:3300/0 conn(0x7fdd4c0bdb30 msgr2=0x7fdd4c06a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 --2- 172.21.15.73:0/2483656889 >> v2:172.21.15.150:3300/0 conn(0x7fdd4c0bdb30 0x7fdd4c06a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.403+0000 7fdd5090c700 1 -- 172.21.15.73:0/2483656889 >> v1:172.21.15.150:6789/0 conn(0x7fdd4c0bd680 legacy=0x7fdd4c069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd5090c700 1 -- 172.21.15.73:0/2483656889 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c154ba0 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd5090c700 1 --2- 172.21.15.73:0/2483656889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fdd4c155440 tx=0x7fdd44009840).ready entity=mon.0 client_cookie=9def53759ee5efa8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd4b7fe700 1 -- 172.21.15.73:0/2483656889 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400e070 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd4b7fe700 1 -- 172.21.15.73:0/2483656889 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400ba10 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd4b7fe700 1 -- 172.21.15.73:0/2483656889 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400bd90 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 -- 172.21.15.73:0/2483656889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0c20c0 msgr2=0x7fdd4c0c24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2483656889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 rx=0x7fdd4c155440 tx=0x7fdd44009840).stop 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 -- 172.21.15.73:0/2483656889 shutdown_connections 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2483656889 >> v2:172.21.15.150:3300/0 conn(0x7fdd4c0bdb30 0x7fdd4c06a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2483656889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0c20c0 0x7fdd4c0c24a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 -- 172.21.15.73:0/2483656889 >> 172.21.15.73:0/2483656889 conn(0x7fdd4c062700 msgr2=0x7fdd4c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 -- 172.21.15.73:0/2483656889 shutdown_connections 2023-11-28T03:07:16.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.404+0000 7fdd52b70700 1 -- 172.21.15.73:0/2483656889 wait complete. 2023-11-28T03:07:16.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.405+0000 7fdd52b70700 1 Processor -- start 2023-11-28T03:07:16.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 -- start start 2023-11-28T03:07:16.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd4c0bd680 0x7fdd4c148d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd4c0c20c0 0x7fdd4c0a4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c1488d0 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c149250 con 0x7fdd4c0bd680 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd52b70700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd4c166360 con 0x7fdd4c0c20c0 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41642/0 (socket says 172.21.15.73:41642) 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.406+0000 7fdd4bfff700 1 -- 172.21.15.73:0/2697850142 learned_addr learned my addr 172.21.15.73:0/2697850142 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd5090c700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd4c0bd680 0x7fdd4c148d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd5110d700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd4c0c20c0 0x7fdd4c0a4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 -- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd4c0c20c0 msgr2=0x7fdd4c0a4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd4c0c20c0 0x7fdd4c0a4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 -- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd4c0bd680 msgr2=0x7fdd4c148d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd4c0bd680 0x7fdd4c148d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 -- 172.21.15.73:0/2697850142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd44009ae0 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd4bfff700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fdd3c00c470 tx=0x7fdd3c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00ce90 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c00f470 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd4c147fa0 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c00f810 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.407+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd4c148560 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.408+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd4c148a80 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.409+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdd3c00fc40 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.409+0000 7fdd48ff9700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 0x7fdd2c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.409+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fdd3c079380 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.410+0000 7fdd5090c700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 0x7fdd2c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.410+0000 7fdd5090c700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 0x7fdd2c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd4c0c5e80 tx=0x7fdd4c0c2c70).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.414+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd3c048af0 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.580+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd4c06c860 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.580+0000 7fdd48ff9700 1 -- 172.21.15.73:0/2697850142 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v78) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdd4c148a80 con 0x7fdd4c0bdb30 2023-11-28T03:07:16.582 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:07:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 msgr2=0x7fdd2c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 0x7fdd2c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fdd4c0c5e80 tx=0x7fdd4c0c2c70).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 msgr2=0x7fdd4c150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 rx=0x7fdd3c00c470 tx=0x7fdd3c007de0).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 shutdown_connections 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd2c05f150 0x7fdd2c061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd4c0c20c0 0x7fdd4c0a4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd4c0bd680 0x7fdd4c148d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 --2- 172.21.15.73:0/2697850142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd4c0bdb30 0x7fdd4c150f10 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 >> 172.21.15.73:0/2697850142 conn(0x7fdd4c062700 msgr2=0x7fdd4c14caa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 shutdown_connections 2023-11-28T03:07:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.582+0000 7fdd52b70700 1 -- 172.21.15.73:0/2697850142 wait complete. 2023-11-28T03:07:16.595 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:07:16.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.707+0000 7f5c7f60c700 1 Processor -- start 2023-11-28T03:07:16.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.707+0000 7f5c7f60c700 1 -- start start 2023-11-28T03:07:16.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7f60c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 0x7f5c781548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7f60c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c7806a610 0x7f5c780c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7f60c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c780c69a0 con 0x7f5c780c6e30 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7f60c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c13a0 con 0x7f5c7806a610 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7f60c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c780c1680 con 0x7f5c781544e0 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7cba7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c7806a610 0x7f5c780c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7cba7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5c7806a610 0x7f5c780c0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41654/0 (socket says 172.21.15.73:41654) 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7cba7700 1 -- 172.21.15.73:0/714749957 learned_addr learned my addr 172.21.15.73:0/714749957 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.708+0000 7f5c7d3a8700 1 --2- 172.21.15.73:0/714749957 >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 0x7f5c781548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7cba7700 1 -- 172.21.15.73:0/714749957 >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 msgr2=0x7f5c781548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7cba7700 1 --2- 172.21.15.73:0/714749957 >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 0x7f5c781548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7cba7700 1 -- 172.21.15.73:0/714749957 >> v1:172.21.15.150:6789/0 conn(0x7f5c780c6e30 legacy=0x7f5c780a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7cba7700 1 -- 172.21.15.73:0/714749957 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c780c6630 con 0x7f5c7806a610 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7d3a8700 1 --2- 172.21.15.73:0/714749957 >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 0x7f5c781548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7cba7700 1 --2- 172.21.15.73:0/714749957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7806a610 0x7f5c780c0bc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5c78085760 tx=0x7f5c680090f0).ready entity=mon.0 client_cookie=5a2ef7b79a58264b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c6ffff700 1 -- 172.21.15.73:0/714749957 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800b5a0 con 0x7f5c7806a610 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c6ffff700 1 -- 172.21.15.73:0/714749957 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c6800b700 con 0x7f5c7806a610 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c6ffff700 1 -- 172.21.15.73:0/714749957 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6800ba80 con 0x7f5c7806a610 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 -- 172.21.15.73:0/714749957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7806a610 msgr2=0x7f5c780c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/714749957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7806a610 0x7f5c780c0bc0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 rx=0x7f5c78085760 tx=0x7f5c680090f0).stop 2023-11-28T03:07:16.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 -- 172.21.15.73:0/714749957 shutdown_connections 2023-11-28T03:07:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/714749957 >> v2:172.21.15.150:3300/0 conn(0x7f5c781544e0 0x7f5c781548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/714749957 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c7806a610 0x7f5c780c0bc0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.709+0000 7f5c7f60c700 1 -- 172.21.15.73:0/714749957 >> 172.21.15.73:0/714749957 conn(0x7f5c78085020 msgr2=0x7f5c78094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.710+0000 7f5c7f60c700 1 -- 172.21.15.73:0/714749957 shutdown_connections 2023-11-28T03:07:16.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.710+0000 7f5c7f60c700 1 -- 172.21.15.73:0/714749957 wait complete. 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.711+0000 7f5c7f60c700 1 Processor -- start 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.711+0000 7f5c7f60c700 1 -- start start 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.711+0000 7f5c7f60c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c7806a610 0x7f5c78150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.711+0000 7f5c7f60c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 0x7f5c78165c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78099150 con 0x7f5c781544e0 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78064550 con 0x7f5c7806a610 2023-11-28T03:07:16.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c78098b90 con 0x7f5c780c6e30 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41668/0 (socket says 172.21.15.73:41668) 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 -- 172.21.15.73:0/3800712206 learned_addr learned my addr 172.21.15.73:0/3800712206 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7d3a8700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c7806a610 0x7f5c78150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7cba7700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 0x7f5c78165c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 -- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 msgr2=0x7f5c78165c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 0x7f5c78165c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 -- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c7806a610 msgr2=0x7f5c78150870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c7806a610 0x7f5c78150870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 -- 172.21.15.73:0/3800712206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009370 con 0x7f5c781544e0 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7cba7700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 0x7f5c78165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7dba9700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f5c7000c840 tx=0x7f5c70002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70010510 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c700106b0 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c78098e70 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c70010a70 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.712+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c78150d60 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.713+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c7806c860 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.713+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5c70012440 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.714+0000 7f5c6d7fa700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 0x7f5c5c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.714+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5c700797d0 con 0x7f5c781544e0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.714+0000 7f5c7d3a8700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 0x7f5c5c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:16.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.714+0000 7f5c7d3a8700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 0x7f5c5c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5c78099350 tx=0x7f5c74008040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:16.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.718+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5c70048af0 con 0x7f5c781544e0 2023-11-28T03:07:16.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.885+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5c780c5890 con 0x7f5c781544e0 2023-11-28T03:07:16.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.885+0000 7f5c6d7fa700 1 -- 172.21.15.73:0/3800712206 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v78) v1 ==== 84+0+143 (secure 0 0 0) 0x7f5c7004ae40 con 0x7f5c781544e0 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:07:16.887 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:07:16.888 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:16.888 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 msgr2=0x7f5c5c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 0x7f5c5c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f5c78099350 tx=0x7f5c74008040).stop 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 msgr2=0x7f5c78095350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 rx=0x7f5c7000c840 tx=0x7f5c70002560).stop 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 shutdown_connections 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5c5c05f150 0x7f5c5c061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5c781544e0 0x7f5c78095350 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5c7806a610 0x7f5c78150870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 --2- 172.21.15.73:0/3800712206 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5c780c6e30 0x7f5c78165c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 >> 172.21.15.73:0/3800712206 conn(0x7f5c78085020 msgr2=0x7f5c780c9070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 shutdown_connections 2023-11-28T03:07:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:16.887+0000 7f5c7f60c700 1 -- 172.21.15.73:0/3800712206 wait complete. 2023-11-28T03:07:16.900 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:07:16.900 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:07:17.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.017+0000 7fda57bf8700 1 Processor -- start 2023-11-28T03:07:17.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.017+0000 7fda57bf8700 1 -- start start 2023-11-28T03:07:17.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda57bf8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda50151990 0x7fda50155e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda57bf8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda500bd1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda57bf8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda500c16b0 con 0x7fda501515b0 2023-11-28T03:07:17.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda57bf8700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda500bc5a0 con 0x7fda50151990 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda57bf8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda500bc880 con 0x7fda500c2c80 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55193700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda50151990 0x7fda50155e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55994700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fda501515b0 0x7fda500bbde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44408/0 (socket says 172.21.15.73:44408) 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55994700 1 -- 172.21.15.73:0/2069905106 learned_addr learned my addr 172.21.15.73:0/2069905106 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda56195700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda500bd1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55193700 1 -- 172.21.15.73:0/2069905106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 msgr2=0x7fda500bd1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55193700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda500bd1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55193700 1 -- 172.21.15.73:0/2069905106 >> v1:172.21.15.73:6789/0 conn(0x7fda501515b0 legacy=0x7fda500bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda55193700 1 -- 172.21.15.73:0/2069905106 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda500bcb60 con 0x7fda50151990 2023-11-28T03:07:17.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.018+0000 7fda54992700 1 -- 172.21.15.73:0/2069905106 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1295020862 0 0) 0x7fda500c16b0 con 0x7fda501515b0 2023-11-28T03:07:17.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda55193700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda50155e80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fda40009760 tx=0x7fda40009060).ready entity=mon.0 client_cookie=e092f8849fab5c4b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda54992700 1 -- 172.21.15.73:0/2069905106 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda400090d0 con 0x7fda50151990 2023-11-28T03:07:17.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda54992700 1 -- 172.21.15.73:0/2069905106 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4000b580 con 0x7fda50151990 2023-11-28T03:07:17.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda54992700 1 -- 172.21.15.73:0/2069905106 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4000b900 con 0x7fda50151990 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 -- 172.21.15.73:0/2069905106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 msgr2=0x7fda50155e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda50155e80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 rx=0x7fda40009760 tx=0x7fda40009060).stop 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 -- 172.21.15.73:0/2069905106 shutdown_connections 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda500bd1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 --2- 172.21.15.73:0/2069905106 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda50155e80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 -- 172.21.15.73:0/2069905106 >> 172.21.15.73:0/2069905106 conn(0x7fda50062700 msgr2=0x7fda50062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:17.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 -- 172.21.15.73:0/2069905106 shutdown_connections 2023-11-28T03:07:17.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.019+0000 7fda57bf8700 1 -- 172.21.15.73:0/2069905106 wait complete. 2023-11-28T03:07:17.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.020+0000 7fda57bf8700 1 Processor -- start 2023-11-28T03:07:17.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.020+0000 7fda57bf8700 1 -- start start 2023-11-28T03:07:17.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda50149cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda501515b0 0x7fda500a7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda500a87f0 con 0x7fda50151990 2023-11-28T03:07:17.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda50152a60 con 0x7fda501515b0 2023-11-28T03:07:17.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda57bf8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda5014a1b0 con 0x7fda500c2c80 2023-11-28T03:07:17.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41676/0 (socket says 172.21.15.73:41676) 2023-11-28T03:07:17.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 -- 172.21.15.73:0/64225925 learned_addr learned my addr 172.21.15.73:0/64225925 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:17.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda55193700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda501515b0 0x7fda500a7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda55994700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda50149cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 -- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 msgr2=0x7fda50149cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda50149cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 -- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda501515b0 msgr2=0x7fda500a7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda501515b0 0x7fda500a7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda56195700 1 -- 172.21.15.73:0/64225925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda400092a0 con 0x7fda50151990 2023-11-28T03:07:17.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.021+0000 7fda55994700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda50149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:17.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda56195700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fda48007f20 tx=0x7fda48009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda4800eca0 con 0x7fda50151990 2023-11-28T03:07:17.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda4800ee00 con 0x7fda50151990 2023-11-28T03:07:17.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda50151e50 con 0x7fda50151990 2023-11-28T03:07:17.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda48014610 con 0x7fda50151990 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.022+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda50152410 con 0x7fda50151990 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.023+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fda48014980 con 0x7fda50151990 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.023+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda50152bf0 con 0x7fda50151990 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.023+0000 7fda45ffb700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 0x7fda34061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.023+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fda48079e40 con 0x7fda50151990 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.024+0000 7fda55994700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 0x7fda34061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.024+0000 7fda55994700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 0x7fda34061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fda4c0011e0 tx=0x7fda4c009040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.028+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda4804aaf0 con 0x7fda50151990 2023-11-28T03:07:17.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.191+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fda500c30f0 con 0x7fda50151990 2023-11-28T03:07:17.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.301+0000 7fda45ffb700 1 -- 172.21.15.73:0/64225925 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v79) v1 ==== 152+0+0 (secure 0 0 0) 0x7fda4804b510 con 0x7fda50151990 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 msgr2=0x7fda34061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 0x7fda34061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fda4c0011e0 tx=0x7fda4c009040).stop 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 msgr2=0x7fda500a8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 rx=0x7fda48007f20 tx=0x7fda48009860).stop 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 shutdown_connections 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fda3405f150 0x7fda34061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda50151990 0x7fda500a8300 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda500c2c80 0x7fda50149cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 --2- 172.21.15.73:0/64225925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda501515b0 0x7fda500a7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.304+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 >> 172.21.15.73:0/64225925 conn(0x7fda50062700 msgr2=0x7fda500a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:17.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.305+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 shutdown_connections 2023-11-28T03:07:17.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.305+0000 7fda57bf8700 1 -- 172.21.15.73:0/64225925 wait complete. 2023-11-28T03:07:17.307 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:07:17.322 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:07:17.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.439+0000 7f7311a5f700 1 Processor -- start 2023-11-28T03:07:17.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.439+0000 7f7311a5f700 1 -- start start 2023-11-28T03:07:17.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f7311a5f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f730c0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f7311a5f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f730c0aa770 0x7f730c157f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f7311a5f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730c0c9730 con 0x7f730c154650 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f7311a5f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730c0c99f0 con 0x7f730c0aa770 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f7311a5f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0c6b40 con 0x7f730c154270 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f730a7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f730c154650 0x7f730c0aa060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44416/0 (socket says 172.21.15.73:44416) 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.441+0000 7f730a7fc700 1 -- 172.21.15.73:0/542888423 learned_addr learned my addr 172.21.15.73:0/542888423 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730b7fe700 1 --2- 172.21.15.73:0/542888423 >> v2:172.21.15.150:3300/0 conn(0x7f730c0aa770 0x7f730c157f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730affd700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f730c0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3157847580 0 0) 0x7f730c0c9730 con 0x7f730c154650 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f72f40033f0 con 0x7f730c154650 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1264465255 0 0) 0x7f72f40033f0 con 0x7f730c154650 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 msgr2=0x7f730c0c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f730c0c6650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 >> v2:172.21.15.150:3300/0 conn(0x7f730c0aa770 msgr2=0x7f730c157f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 --2- 172.21.15.73:0/542888423 >> v2:172.21.15.150:3300/0 conn(0x7f730c0aa770 0x7f730c157f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c158450 con 0x7f730c154650 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7300001470 con 0x7f730c154650 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 >> v1:172.21.15.73:6789/0 conn(0x7f730c154650 legacy=0x7f730c0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f40038e0 0x7f72f4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72f4004690 0x7f72f4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f72f4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0c99f0 con 0x7f72f4004690 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c0c6b40 con 0x7f72f40038e0 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72f400b8e0 con 0x7f730c154270 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72f4004690 0x7f72f4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730affd700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f40038e0 0x7f72f4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730b7fe700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f72f4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 -- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 msgr2=0x7f72f4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f72f4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 -- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f40038e0 msgr2=0x7f72f4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f40038e0 0x7f72f4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730a7fc700 1 -- 172.21.15.73:0/542888423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72f40033f0 con 0x7f72f4004690 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730b7fe700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c154270 0x7f72f4005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.442+0000 7f730affd700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f40038e0 0x7f72f4003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f730a7fc700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72f4004690 0x7f72f4004b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f730c154a50 tx=0x7f73000077c0).ready entity=mon.0 client_cookie=868f4a14b8563cfe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f730000cab0 con 0x7f72f4004690 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f730000cc10 con 0x7f72f4004690 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7309ffb700 1 -- 172.21.15.73:0/542888423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000070d0 con 0x7f72f4004690 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 -- 172.21.15.73:0/542888423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72f4004690 msgr2=0x7f72f4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 --2- 172.21.15.73:0/542888423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f72f4004690 0x7f72f4004b10 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 rx=0x7f730c154a50 tx=0x7f73000077c0).stop 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f730b7fe700 1 -- 172.21.15.73:0/542888423 reap_dead start 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 -- 172.21.15.73:0/542888423 shutdown_connections 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 -- 172.21.15.73:0/542888423 >> 172.21.15.73:0/542888423 conn(0x7f730c0649b0 msgr2=0x7f730c064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 -- 172.21.15.73:0/542888423 shutdown_connections 2023-11-28T03:07:17.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.443+0000 7f7311a5f700 1 -- 172.21.15.73:0/542888423 wait complete. 2023-11-28T03:07:17.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.444+0000 7f7311a5f700 1 Processor -- start 2023-11-28T03:07:17.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 -- start start 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f4004690 0x7f730c0ae040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c0ae530 0x7f730c0b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c157400 con 0x7f730c1563f0 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730c156880 con 0x7f72f4004690 2023-11-28T03:07:17.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f7311a5f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f730c156b60 con 0x7f730c0ae530 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f730a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f730a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41694/0 (socket says 172.21.15.73:41694) 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.445+0000 7f730a7fc700 1 -- 172.21.15.73:0/426047834 learned_addr learned my addr 172.21.15.73:0/426047834 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730affd700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f4004690 0x7f730c0ae040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730b7fe700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c0ae530 0x7f730c0b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 -- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c0ae530 msgr2=0x7f730c0b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c0ae530 0x7f730c0b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 -- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f4004690 msgr2=0x7f730c0ae040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f4004690 0x7f730c0ae040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 -- 172.21.15.73:0/426047834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730c156e40 con 0x7f730c1563f0 2023-11-28T03:07:17.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f730a7fc700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f730c0848d0 tx=0x7f7300015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7300001640 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73000017a0 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f730c157120 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73000048c0 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.446+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730c0b0670 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.447+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f730c062f00 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.447+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7300004cb0 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.448+0000 7f72f2ffd700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 0x7f72f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.448+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f730007adf0 con 0x7f730c1563f0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.448+0000 7f730affd700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 0x7f72f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.448+0000 7f730affd700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 0x7f72f8061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f72fc006be0 tx=0x7f72fc008040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:17.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.452+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f730004c2a0 con 0x7f730c1563f0 2023-11-28T03:07:17.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:17.615+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f730c06c7c0 con 0x7f730c1563f0 2023-11-28T03:07:18.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.309+0000 7f72f2ffd700 1 -- 172.21.15.73:0/426047834 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v80) v1 ==== 170+0+0 (secure 0 0 0) 0x7f730001a070 con 0x7f730c1563f0 2023-11-28T03:07:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.314+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 msgr2=0x7f72f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.314+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 0x7f72f8061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f72fc006be0 tx=0x7f72fc008040).stop 2023-11-28T03:07:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.314+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 msgr2=0x7f730c0afc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.314+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 rx=0x7f730c0848d0 tx=0x7f7300015000).stop 2023-11-28T03:07:18.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.314+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 shutdown_connections 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f72f805f150 0x7f72f8061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f730c0ae530 0x7f730c0b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f72f4004690 0x7f730c0ae040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 --2- 172.21.15.73:0/426047834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f730c1563f0 0x7f730c0afc90 secure :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 rx=0x7f730c0848d0 tx=0x7f7300015000).stop 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 >> 172.21.15.73:0/426047834 conn(0x7f730c0649b0 msgr2=0x7f730c14d9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 shutdown_connections 2023-11-28T03:07:18.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.315+0000 7f7311a5f700 1 -- 172.21.15.73:0/426047834 wait complete. 2023-11-28T03:07:18.324 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:07:18.342 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:07:18.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.459+0000 7feeab326700 1 Processor -- start 2023-11-28T03:07:18.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.459+0000 7feeab326700 1 -- start start 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feeab326700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feea40c6190 0x7feea40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feeab326700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea40c72d0 con 0x7feea4098d10 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feeab326700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feea4064570 con 0x7feea4158320 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feeab326700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feea4154bc0 con 0x7feea40c6190 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feea88c1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feea40c6190 0x7feea40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feea90c2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feea4098d10 0x7feea4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44424/0 (socket says 172.21.15.73:44424) 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feea90c2700 1 -- 172.21.15.73:0/1993456860 learned_addr learned my addr 172.21.15.73:0/1993456860 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feea88c1700 1 -- 172.21.15.73:0/1993456860 >> v1:172.21.15.150:6789/0 conn(0x7feea4158320 legacy=0x7feea40a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.460+0000 7feea88c1700 1 -- 172.21.15.73:0/1993456860 >> v1:172.21.15.73:6789/0 conn(0x7feea4098d10 legacy=0x7feea4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feea88c1700 1 -- 172.21.15.73:0/1993456860 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea40c66c0 con 0x7feea40c6190 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7fee9bfff700 1 -- 172.21.15.73:0/1993456860 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1321318172 0 0) 0x7feea40c72d0 con 0x7feea4098d10 2023-11-28T03:07:18.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feea88c1700 1 --2- 172.21.15.73:0/1993456860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea40c6190 0x7feea40c7760 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fee94009760 tx=0x7fee94009060).ready entity=mon.0 client_cookie=1f1fd332c8468752 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7fee9bfff700 1 -- 172.21.15.73:0/1993456860 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400b580 con 0x7feea40c6190 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7fee9bfff700 1 -- 172.21.15.73:0/1993456860 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee9400b6e0 con 0x7feea40c6190 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7fee9bfff700 1 -- 172.21.15.73:0/1993456860 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee9400ba60 con 0x7feea40c6190 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 -- 172.21.15.73:0/1993456860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea40c6190 msgr2=0x7feea40c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 --2- 172.21.15.73:0/1993456860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea40c6190 0x7feea40c7760 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 rx=0x7fee94009760 tx=0x7fee94009060).stop 2023-11-28T03:07:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 -- 172.21.15.73:0/1993456860 shutdown_connections 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 --2- 172.21.15.73:0/1993456860 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea40c6190 0x7feea40c7760 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 -- 172.21.15.73:0/1993456860 >> 172.21.15.73:0/1993456860 conn(0x7feea40cb310 msgr2=0x7feea40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 -- 172.21.15.73:0/1993456860 shutdown_connections 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.461+0000 7feeab326700 1 -- 172.21.15.73:0/1993456860 wait complete. 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.462+0000 7feeab326700 1 Processor -- start 2023-11-28T03:07:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 -- start start 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea40c6190 0x7feea40b00c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea4158320 0x7feea40b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea4157710 con 0x7feea4098d10 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c2f40 con 0x7feea40c6190 2023-11-28T03:07:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feeab326700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea40c3220 con 0x7feea4158320 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.463+0000 7feea90c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41702/0 (socket says 172.21.15.73:41702) 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 -- 172.21.15.73:0/963290976 learned_addr learned my addr 172.21.15.73:0/963290976 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea88c1700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea40c6190 0x7feea40b00c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea98c3700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea4158320 0x7feea40b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 -- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea4158320 msgr2=0x7feea40b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea4158320 0x7feea40b05b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 -- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea40c6190 msgr2=0x7feea40b00c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea40c6190 0x7feea40b00c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 -- 172.21.15.73:0/963290976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee940092a0 con 0x7feea4098d10 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feea90c2700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7feea000c4c0 tx=0x7feea0007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea000ce30 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feea00072e0 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feea40c3500 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feea00057a0 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.464+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feea40b0aa0 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.465+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feea406c860 con 0x7feea4098d10 2023-11-28T03:07:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.466+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feea0005bd0 con 0x7feea4098d10 2023-11-28T03:07:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.466+0000 7fee997fa700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 0x7fee880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.466+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6021+0+0 (secure 0 0 0) 0x7feea0079950 con 0x7feea4098d10 2023-11-28T03:07:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.466+0000 7feea88c1700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 0x7fee880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.466+0000 7feea88c1700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 0x7fee880615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fee94009760 tx=0x7fee94017000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.470+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feea004ac80 con 0x7feea4098d10 2023-11-28T03:07:18.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.674+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7feea40c69a0 con 0x7feea4098d10 2023-11-28T03:07:18.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.680+0000 7fee997fa700 1 -- 172.21.15.73:0/963290976 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 9 and data pool 10 v24) v1 ==== 173+0+0 (secure 0 0 0) 0x7feea0018030 con 0x7feea4098d10 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 msgr2=0x7fee880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 0x7fee880615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fee94009760 tx=0x7fee94017000).stop 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 msgr2=0x7feea40ae6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 rx=0x7feea000c4c0 tx=0x7feea0007c10).stop 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 shutdown_connections 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee8805f120 0x7fee880615e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea4158320 0x7feea40b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea4098d10 0x7feea40ae6b0 secure :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 rx=0x7feea000c4c0 tx=0x7feea0007c10).stop 2023-11-28T03:07:18.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 --2- 172.21.15.73:0/963290976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea40c6190 0x7feea40b00c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.684+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 >> 172.21.15.73:0/963290976 conn(0x7feea40cb310 msgr2=0x7feea40ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:18.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.689+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 shutdown_connections 2023-11-28T03:07:18.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.691+0000 7feeab326700 1 -- 172.21.15.73:0/963290976 wait complete. 2023-11-28T03:07:18.693 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 9 and data pool 10 2023-11-28T03:07:18.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.824+0000 7f4413259700 1 Processor -- start 2023-11-28T03:07:18.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.824+0000 7f4413259700 1 -- start start 2023-11-28T03:07:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4413259700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f440c0caab0 0x7f440c0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4413259700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0cae90 0x7f440c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4413259700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f440c0a9a90 con 0x7f440c0aa960 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4413259700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f440c154ce0 con 0x7f440c0caab0 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4413259700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0a8e80 con 0x7f440c0cae90 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f440c0caab0 0x7f440c0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f44117f6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f440c0aa960 0x7f440c0c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44440/0 (socket says 172.21.15.73:44440) 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f44117f6700 1 -- 172.21.15.73:0/3619011611 learned_addr learned my addr 172.21.15.73:0/3619011611 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 --2- 172.21.15.73:0/3619011611 >> v2:172.21.15.73:3300/0 conn(0x7f440c0caab0 0x7f440c0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41704/0 (socket says 172.21.15.73:41704) 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 -- 172.21.15.73:0/3619011611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0cae90 msgr2=0x7f440c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 --2- 172.21.15.73:0/3619011611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0cae90 0x7f440c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 -- 172.21.15.73:0/3619011611 >> v1:172.21.15.73:6789/0 conn(0x7f440c0aa960 legacy=0x7f440c0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f4410ff5700 1 -- 172.21.15.73:0/3619011611 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f440c0a9160 con 0x7f440c0caab0 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.825+0000 7f44037fe700 1 -- 172.21.15.73:0/3619011611 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 735635435 0 0) 0x7f440c0a9a90 con 0x7f440c0aa960 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4410ff5700 1 --2- 172.21.15.73:0/3619011611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c0caab0 0x7f440c0aa470 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f4408006fd0 tx=0x7f4408006f40).ready entity=mon.0 client_cookie=e17122e08f90afc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f44037fe700 1 -- 172.21.15.73:0/3619011611 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408005880 con 0x7f440c0caab0 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f44037fe700 1 -- 172.21.15.73:0/3619011611 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44080059e0 con 0x7f440c0caab0 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f44037fe700 1 -- 172.21.15.73:0/3619011611 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4408005d60 con 0x7f440c0caab0 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 -- 172.21.15.73:0/3619011611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c0caab0 msgr2=0x7f440c0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 --2- 172.21.15.73:0/3619011611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c0caab0 0x7f440c0aa470 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 rx=0x7f4408006fd0 tx=0x7f4408006f40).stop 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 -- 172.21.15.73:0/3619011611 shutdown_connections 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 --2- 172.21.15.73:0/3619011611 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c0caab0 0x7f440c0aa470 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 --2- 172.21.15.73:0/3619011611 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0cae90 0x7f440c0c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 -- 172.21.15.73:0/3619011611 >> 172.21.15.73:0/3619011611 conn(0x7f440c094920 msgr2=0x7f440c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 -- 172.21.15.73:0/3619011611 shutdown_connections 2023-11-28T03:07:18.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.826+0000 7f4413259700 1 -- 172.21.15.73:0/3619011611 wait complete. 2023-11-28T03:07:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.827+0000 7f4413259700 1 Processor -- start 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 -- start start 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 0x7f440c1575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 0x7f440c1550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c1555d0 con 0x7f440c1657a0 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0ac330 con 0x7f440c0cae90 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f4413259700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f440c0abd20 con 0x7f440c0caab0 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f44117f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.828+0000 7f44117f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41708/0 (socket says 172.21.15.73:41708) 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 -- 172.21.15.73:0/194143889 learned_addr learned my addr 172.21.15.73:0/194143889 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4410ff5700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 0x7f440c1575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4403fff700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 0x7f440c1550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 -- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 msgr2=0x7f440c1575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 0x7f440c1575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 -- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 msgr2=0x7f440c1550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 0x7f440c1550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 -- 172.21.15.73:0/194143889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4408009250 con 0x7f440c1657a0 2023-11-28T03:07:18.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4410ff5700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 0x7f440c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4403fff700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 0x7f440c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f44117f6700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f440400c810 tx=0x7f4404007ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404005d30 con 0x7f440c1657a0 2023-11-28T03:07:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4404005e90 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f440c157c50 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4404007320 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.829+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f440c0abfc0 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.830+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f440c0ac4e0 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.830+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4404007710 con 0x7f440c1657a0 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.831+0000 7f4400ff9700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 0x7f43f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.831+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f440407a240 con 0x7f440c1657a0 2023-11-28T03:07:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.831+0000 7f4410ff5700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 0x7f43f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.831+0000 7f4410ff5700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 0x7f43f8061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f440c0afe60 tx=0x7f4408016000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.835+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f440c0ac4e0 con 0x7f440c1657a0 2023-11-28T03:07:18.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.997+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f440c0ac4a0 con 0x7f440c1657a0 2023-11-28T03:07:18.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:18.998+0000 7f4400ff9700 1 -- 172.21.15.73:0/194143889 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f440404b550 con 0x7f440c1657a0 2023-11-28T03:07:18.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:18.999 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":81,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:18.675199+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:19.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 msgr2=0x7f43f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 0x7f43f8061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f440c0afe60 tx=0x7f4408016000).stop 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 msgr2=0x7f440c167360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 rx=0x7f440400c810 tx=0x7f4404007ed0).stop 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 shutdown_connections 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f43f805f150 0x7f43f8061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f440c1657a0 0x7f440c167360 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f440c0caab0 0x7f440c1575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 --2- 172.21.15.73:0/194143889 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f440c0cae90 0x7f440c1550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 >> 172.21.15.73:0/194143889 conn(0x7f440c094920 msgr2=0x7f440c0cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 shutdown_connections 2023-11-28T03:07:19.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.000+0000 7f4413259700 1 -- 172.21.15.73:0/194143889 wait complete. 2023-11-28T03:07:19.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:19.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.125+0000 7f92431f8700 1 Processor -- start 2023-11-28T03:07:19.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.125+0000 7f92431f8700 1 -- start start 2023-11-28T03:07:19.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f92431f8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 0x7f923c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f92431f8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 0x7f923c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f92431f8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f923c0c69a0 con 0x7f923c06a610 2023-11-28T03:07:19.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f92431f8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f923c0c13a0 con 0x7f923c1544e0 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f92431f8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0c1680 con 0x7f923c0c6e30 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f923bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f923c06a610 0x7f923c0c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44450/0 (socket says 172.21.15.73:44450) 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f923bfff700 1 -- 172.21.15.73:0/1847729940 learned_addr learned my addr 172.21.15.73:0/1847729940 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f9240f94700 1 --2- 172.21.15.73:0/1847729940 >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 0x7f923c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f9241795700 1 --2- 172.21.15.73:0/1847729940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 0x7f923c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235478781 0 0) 0x7f923c0c69a0 con 0x7f923c06a610 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.126+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92280033f0 con 0x7f923c06a610 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9241795700 1 -- 172.21.15.73:0/1847729940 >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 msgr2=0x7f923c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9241795700 1 --2- 172.21.15.73:0/1847729940 >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 0x7f923c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9241795700 1 -- 172.21.15.73:0/1847729940 >> v1:172.21.15.73:6789/0 conn(0x7f923c06a610 legacy=0x7f923c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2761527654 0 0) 0x7f92280033f0 con 0x7f923c06a610 2023-11-28T03:07:19.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9241795700 1 -- 172.21.15.73:0/1847729940 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c0c6630 con 0x7f923c0c6e30 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9240f94700 1 --2- 172.21.15.73:0/1847729940 >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 0x7f923c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f9241795700 1 --2- 172.21.15.73:0/1847729940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 0x7f923c0c72e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9234009e60 tx=0x7f9234009780).ready entity=mon.2 client_cookie=bd13bbea40e0206c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f923400e070 con 0x7f923c0c6e30 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f923400b7a0 con 0x7f923c0c6e30 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f923b7fe700 1 -- 172.21.15.73:0/1847729940 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9224001470 con 0x7f923c0c6e30 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 -- 172.21.15.73:0/1847729940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 msgr2=0x7f923c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 --2- 172.21.15.73:0/1847729940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 0x7f923c0c72e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 rx=0x7f9234009e60 tx=0x7f9234009780).stop 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 -- 172.21.15.73:0/1847729940 shutdown_connections 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 --2- 172.21.15.73:0/1847729940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c0c6e30 0x7f923c0c72e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 --2- 172.21.15.73:0/1847729940 >> v2:172.21.15.150:3300/0 conn(0x7f923c1544e0 0x7f923c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.127+0000 7f92431f8700 1 -- 172.21.15.73:0/1847729940 >> 172.21.15.73:0/1847729940 conn(0x7f923c085020 msgr2=0x7f923c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.128+0000 7f92431f8700 1 -- 172.21.15.73:0/1847729940 shutdown_connections 2023-11-28T03:07:19.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.128+0000 7f92431f8700 1 -- 172.21.15.73:0/1847729940 wait complete. 2023-11-28T03:07:19.133 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.129+0000 7f92431f8700 1 Processor -- start 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.129+0000 7f92431f8700 1 -- start start 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c06a610 0x7f923c145fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f923c1544e0 0x7f923c098680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0a8350 con 0x7f923c0c6e30 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f923c097390 con 0x7f923c1544e0 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f923c0981f0 con 0x7f923c06a610 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41718/0 (socket says 172.21.15.73:41718) 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 -- 172.21.15.73:0/2334435409 learned_addr learned my addr 172.21.15.73:0/2334435409 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f9240f94700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c06a610 0x7f923c145fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f9241795700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f923c1544e0 0x7f923c098680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 -- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c06a610 msgr2=0x7f923c145fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c06a610 0x7f923c145fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 -- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f923c1544e0 msgr2=0x7f923c098680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f923c1544e0 0x7f923c098680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 -- 172.21.15.73:0/2334435409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f922c009250 con 0x7f923c0c6e30 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f923bfff700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f922400c840 tx=0x7f9224001a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240072e0 con 0x7f923c0c6e30 2023-11-28T03:07:19.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9224007440 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92240057a0 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9234009a20 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.130+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f923c0975e0 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.131+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f923c06c860 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.132+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9224005bd0 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.132+0000 7f9238ff9700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 0x7f921c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.132+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9224079af0 con 0x7f923c0c6e30 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.132+0000 7f9240f94700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 0x7f921c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.136 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.132+0000 7f9240f94700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 0x7f921c061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f922c006fd0 tx=0x7f922c006f40).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.136+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f922404adc0 con 0x7f923c0c6e30 2023-11-28T03:07:19.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.301+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f923c0c6250 con 0x7f923c0c6e30 2023-11-28T03:07:19.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.302+0000 7f9238ff9700 1 -- 172.21.15.73:0/2334435409 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f922404e3e0 con 0x7f923c0c6e30 2023-11-28T03:07:19.303 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:19.303 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":81,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:18.675199+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:19.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 msgr2=0x7f921c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 0x7f921c061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f922c006fd0 tx=0x7f922c006f40).stop 2023-11-28T03:07:19.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 msgr2=0x7f923c1533a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 rx=0x7f922400c840 tx=0x7f9224001a00).stop 2023-11-28T03:07:19.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 shutdown_connections 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f921c05f150 0x7f921c061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f923c1544e0 0x7f923c098680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f923c06a610 0x7f923c145fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 --2- 172.21.15.73:0/2334435409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f923c0c6e30 0x7f923c1533a0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.304+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 >> 172.21.15.73:0/2334435409 conn(0x7f923c085020 msgr2=0x7f923c063fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.305+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 shutdown_connections 2023-11-28T03:07:19.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.305+0000 7f92431f8700 1 -- 172.21.15.73:0/2334435409 wait complete. 2023-11-28T03:07:19.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:07:19.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.434+0000 7f21fc16a700 1 Processor -- start 2023-11-28T03:07:19.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.434+0000 7f21fc16a700 1 -- start start 2023-11-28T03:07:19.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21fc16a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21f40c6190 0x7f21f40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21fc16a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f40c72d0 con 0x7f21f4098d10 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21fc16a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21f4064570 con 0x7f21f4158320 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21fc16a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21f4154bc0 con 0x7f21f40c6190 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21f9f06700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21f4098d10 0x7f21f4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44464/0 (socket says 172.21.15.73:44464) 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.435+0000 7f21f9f06700 1 -- 172.21.15.73:0/2514026538 learned_addr learned my addr 172.21.15.73:0/2514026538 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f9705700 1 --2- 172.21.15.73:0/2514026538 >> v2:172.21.15.150:3300/0 conn(0x7f21f40c6190 0x7f21f40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686341759 0 0) 0x7f21f40c72d0 con 0x7f21f4098d10 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f21e00033f0 con 0x7f21f4098d10 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1257344776 0 0) 0x7f21e00033f0 con 0x7f21f4098d10 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 >> v2:172.21.15.150:3300/0 conn(0x7f21f40c6190 msgr2=0x7f21f40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 --2- 172.21.15.73:0/2514026538 >> v2:172.21.15.150:3300/0 conn(0x7f21f40c6190 0x7f21f40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 >> v1:172.21.15.150:6789/0 conn(0x7f21f4158320 legacy=0x7f21f40a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40c66c0 con 0x7f21f4098d10 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f21f0001470 con 0x7f21f4098d10 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 >> v1:172.21.15.73:6789/0 conn(0x7f21f4098d10 legacy=0x7f21f4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e00038e0 0x7f21e0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0004690 0x7f21e0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 0x7f21e000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f4154bc0 con 0x7f21e0005000 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000beb0 con 0x7f21e00038e0 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e000c070 con 0x7f21e0004690 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 0x7f21e000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f9f06700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e00038e0 0x7f21e0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21f9705700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0004690 0x7f21e0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 -- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0004690 msgr2=0x7f21e0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0004690 0x7f21e0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 -- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e00038e0 msgr2=0x7f21e0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e00038e0 0x7f21e0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 -- 172.21.15.73:0/2514026538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e00033f0 con 0x7f21e0005000 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.436+0000 7f21fa707700 1 -- 172.21.15.73:0/2514026538 reap_dead start 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fa707700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 0x7f21e000b8e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f21ec001e30 tx=0x7f21ec007f30).ready entity=mon.0 client_cookie=de43dbc8f918d794 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec006a20 con 0x7f21e0005000 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21ec006b80 con 0x7f21e0005000 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21f8f04700 1 -- 172.21.15.73:0/2514026538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21ec00c5d0 con 0x7f21e0005000 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 -- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 msgr2=0x7f21e000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 0x7f21e000b8e0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 rx=0x7f21ec001e30 tx=0x7f21ec007f30).stop 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 -- 172.21.15.73:0/2514026538 shutdown_connections 2023-11-28T03:07:19.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 --2- 172.21.15.73:0/2514026538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e0005000 0x7f21e000b8e0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 -- 172.21.15.73:0/2514026538 >> 172.21.15.73:0/2514026538 conn(0x7f21f40cb310 msgr2=0x7f21f40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 -- 172.21.15.73:0/2514026538 shutdown_connections 2023-11-28T03:07:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.437+0000 7f21fc16a700 1 -- 172.21.15.73:0/2514026538 wait complete. 2023-11-28T03:07:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.438+0000 7f21fc16a700 1 Processor -- start 2023-11-28T03:07:19.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.438+0000 7f21fc16a700 1 -- start start 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 0x7f21f40a9590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 0x7f21f40a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40adc50 con 0x7f21f414bf70 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21f414be10 con 0x7f21f40ab8b0 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fc16a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21f40ad1a0 con 0x7f21e0005000 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9705700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9705700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41734/0 (socket says 172.21.15.73:41734) 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9705700 1 -- 172.21.15.73:0/944101050 learned_addr learned my addr 172.21.15.73:0/944101050 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:19.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9f06700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 0x7f21f40a9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21fa707700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 0x7f21f40a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9705700 1 -- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 msgr2=0x7f21f40a9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.439+0000 7f21f9705700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 0x7f21f40a9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21f9705700 1 -- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 msgr2=0x7f21f40a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21f9705700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 0x7f21f40a80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21f9705700 1 -- 172.21.15.73:0/944101050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21f40ad3b0 con 0x7f21f414bf70 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21f9f06700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 0x7f21f40a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:19.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21fa707700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 0x7f21f40a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21f9705700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f21e40119f0 tx=0x7f21e4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e4011e60 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e400f8e0 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21f40ad690 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e400fc60 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.440+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21f40ab390 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.441+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21f40ab5b0 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.441+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f21e4014650 con 0x7f21f414bf70 2023-11-28T03:07:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.442+0000 7f21e9ffb700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 0x7f21d8061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.442+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f21e407aa50 con 0x7f21f414bf70 2023-11-28T03:07:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.442+0000 7f21f9f06700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 0x7f21d8061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.442+0000 7f21f9f06700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 0x7f21d8061640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f21f0001ce0 tx=0x7f21f0009040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.446+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21e404bd20 con 0x7f21f414bf70 2023-11-28T03:07:19.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.651+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f21f406c860 con 0x7f21f414bf70 2023-11-28T03:07:19.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.679+0000 7f21e9ffb700 1 -- 172.21.15.73:0/944101050 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) v1 ==== 118+0+0 (secure 0 0 0) 0x7f21e4022070 con 0x7f21f414bf70 2023-11-28T03:07:19.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 msgr2=0x7f21d8061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 0x7f21d8061640 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f21f0001ce0 tx=0x7f21f0009040).stop 2023-11-28T03:07:19.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 msgr2=0x7f21f414b920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 rx=0x7f21e40119f0 tx=0x7f21e4009060).stop 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 shutdown_connections 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d805f180 0x7f21d8061640 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21f40ab8b0 0x7f21f40a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e0005000 0x7f21f40a9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 --2- 172.21.15.73:0/944101050 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21f414bf70 0x7f21f414b920 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 >> 172.21.15.73:0/944101050 conn(0x7f21f40cb310 msgr2=0x7f21f4153df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 shutdown_connections 2023-11-28T03:07:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.682+0000 7f21fc16a700 1 -- 172.21.15.73:0/944101050 wait complete. 2023-11-28T03:07:19.694 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:07:19.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:07:19.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.810+0000 7f6ff0a07700 1 Processor -- start 2023-11-28T03:07:19.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.810+0000 7f6ff0a07700 1 -- start start 2023-11-28T03:07:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6ff0a07700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fec098d10 0x7f6fec0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6ff0a07700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec158320 0x7f6fec1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6ff0a07700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6fec158cc0 con 0x7f6fec0c6190 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6ff0a07700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6fec0c72d0 con 0x7f6fec098d10 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6ff0a07700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec064570 con 0x7f6fec158320 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6fea59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fec098d10 0x7f6fec0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6fea59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6fec098d10 0x7f6fec0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41748/0 (socket says 172.21.15.73:41748) 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.811+0000 7f6fea59c700 1 -- 172.21.15.73:0/491726801 learned_addr learned my addr 172.21.15.73:0/491726801 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fead9d700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec158320 0x7f6fec1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fea59c700 1 -- 172.21.15.73:0/491726801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec158320 msgr2=0x7f6fec1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fea59c700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec158320 0x7f6fec1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fea59c700 1 -- 172.21.15.73:0/491726801 >> v1:172.21.15.150:6789/0 conn(0x7f6fec0c6190 legacy=0x7f6fec063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fea59c700 1 -- 172.21.15.73:0/491726801 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fec154bc0 con 0x7f6fec098d10 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fea59c700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec098d10 0x7f6fec0c7760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f6fdc009f20 tx=0x7f6fdc009840).ready entity=mon.0 client_cookie=750a21ff845552a7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fe959a700 1 -- 172.21.15.73:0/491726801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00e070 con 0x7f6fec098d10 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fe959a700 1 -- 172.21.15.73:0/491726801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fdc00ba80 con 0x7f6fec098d10 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6fe959a700 1 -- 172.21.15.73:0/491726801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fdc00be00 con 0x7f6fec098d10 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6ff0a07700 1 -- 172.21.15.73:0/491726801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec098d10 msgr2=0x7f6fec0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.812+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec098d10 0x7f6fec0c7760 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 rx=0x7f6fdc009f20 tx=0x7f6fdc009840).stop 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 -- 172.21.15.73:0/491726801 shutdown_connections 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec158320 0x7f6fec1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/491726801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec098d10 0x7f6fec0c7760 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 -- 172.21.15.73:0/491726801 >> 172.21.15.73:0/491726801 conn(0x7f6fec0cb310 msgr2=0x7f6fec0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 -- 172.21.15.73:0/491726801 shutdown_connections 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.813+0000 7f6ff0a07700 1 -- 172.21.15.73:0/491726801 wait complete. 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 Processor -- start 2023-11-28T03:07:19.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 -- start start 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 0x7f6fec0b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 0x7f6fec0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec156080 con 0x7f6fec158320 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec155470 con 0x7f6fec0c6190 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.814+0000 7f6ff0a07700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6fec155750 con 0x7f6fec098d10 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41750/0 (socket says 172.21.15.73:41750) 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 -- 172.21.15.73:0/2282099430 learned_addr learned my addr 172.21.15.73:0/2282099430 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fea59c700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 0x7f6fec0b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fe9d9b700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 0x7f6fec0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 -- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 msgr2=0x7f6fec0b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 0x7f6fec0b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 -- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 msgr2=0x7f6fec0acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 0x7f6fec0acdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 -- 172.21.15.73:0/2282099430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6fdc009ae0 con 0x7f6fec158320 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fea59c700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 0x7f6fec0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fe9d9b700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 0x7f6fec0acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fead9d700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f6fe400bfd0 tx=0x7f6fe4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4011590 con 0x7f6fec158320 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6fe40116f0 con 0x7f6fec158320 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6fec155a30 con 0x7f6fec158320 2023-11-28T03:07:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6fe4011a70 con 0x7f6fec158320 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.815+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6fec0ae2c0 con 0x7f6fec158320 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.816+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6fec0aad70 con 0x7f6fec158320 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.816+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6fe4012420 con 0x7f6fec158320 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.817+0000 7f6fe27fc700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 0x7f6fcc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.817+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6fe4079b80 con 0x7f6fec158320 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.817+0000 7f6fea59c700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 0x7f6fcc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.817+0000 7f6fea59c700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 0x7f6fcc061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fdc009880 tx=0x7f6fec0a8040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:19.821+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6fec0aad70 con 0x7f6fec158320 2023-11-28T03:07:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.027+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6fec0aad70 con 0x7f6fec158320 2023-11-28T03:07:20.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.687+0000 7f6fe27fc700 1 -- 172.21.15.73:0/2282099430 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6fec0aad70 con 0x7f6fec158320 2023-11-28T03:07:20.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 msgr2=0x7f6fcc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 0x7f6fcc061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f6fdc009880 tx=0x7f6fec0a8040).stop 2023-11-28T03:07:20.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 msgr2=0x7f6fec0a9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 rx=0x7f6fe400bfd0 tx=0x7f6fe4008040).stop 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 shutdown_connections 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6fcc05f150 0x7f6fcc061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6fec158320 0x7f6fec0a9660 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6fec098d10 0x7f6fec0b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 --2- 172.21.15.73:0/2282099430 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6fec0c6190 0x7f6fec0acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 >> 172.21.15.73:0/2282099430 conn(0x7f6fec0cb310 msgr2=0x7f6fec0ca980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:20.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 shutdown_connections 2023-11-28T03:07:20.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.689+0000 7f6ff0a07700 1 -- 172.21.15.73:0/2282099430 wait complete. 2023-11-28T03:07:20.702 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:07:20.702 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:20.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.816+0000 7f6b11311700 1 Processor -- start 2023-11-28T03:07:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.816+0000 7f6b11311700 1 -- start start 2023-11-28T03:07:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b11311700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6b0c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b11311700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c0a9a90 con 0x7f6b0c0ca180 2023-11-28T03:07:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b11311700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b0c154ce0 con 0x7f6b0c0caab0 2023-11-28T03:07:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b11311700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0aab80 con 0x7f6b0c0cae90 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b0b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6b0c0ca180 0x7f6b0c15e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44466/0 (socket says 172.21.15.73:44466) 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b0b7fe700 1 -- 172.21.15.73:0/3391412493 learned_addr learned my addr 172.21.15.73:0/3391412493 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.817+0000 7f6b027fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6b0c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3865377597 0 0) 0x7f6b0c0a9a90 con 0x7f6b0c0ca180 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6af80033f0 con 0x7f6b0c0ca180 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2313960597 0 0) 0x7f6af80033f0 con 0x7f6b0c0ca180 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 msgr2=0x7f6b0c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6b0c0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 >> v1:172.21.15.150:6789/0 conn(0x7f6b0c0caab0 legacy=0x7f6b0c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0a8e80 con 0x7f6b0c0ca180 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6b040017f0 con 0x7f6b0c0ca180 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 >> v1:172.21.15.73:6789/0 conn(0x7f6b0c0ca180 legacy=0x7f6b0c15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6af8003910 0x7f6af8003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6af8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6af8004b80 0x7f6af8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0aab80 con 0x7f6af8004b80 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af800b8e0 con 0x7f6af8003910 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af800ba50 con 0x7f6b0c0cae90 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6af8004b80 0x7f6af8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b027fc700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6af8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0affd700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6af8003910 0x7f6af8003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 -- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 msgr2=0x7f6af8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0cae90 0x7f6af8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 -- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6af8003910 msgr2=0x7f6af8003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:20.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6af8003910 0x7f6af8003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 -- 172.21.15.73:0/3391412493 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80033f0 con 0x7f6af8004b80 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.818+0000 7f6b0b7fe700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6af8004b80 0x7f6af8005000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6b0c084890 tx=0x7f6b04001d90).ready entity=mon.0 client_cookie=8e6599cca7147838 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04001fc0 con 0x7f6af8004b80 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b0400ce00 con 0x7f6af8004b80 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b0a7fc700 1 -- 172.21.15.73:0/3391412493 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04005d50 con 0x7f6af8004b80 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 -- 172.21.15.73:0/3391412493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6af8004b80 msgr2=0x7f6af8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 --2- 172.21.15.73:0/3391412493 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6af8004b80 0x7f6af8005000 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 rx=0x7f6b0c084890 tx=0x7f6b04001d90).stop 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b0b7fe700 1 -- 172.21.15.73:0/3391412493 reap_dead start 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 -- 172.21.15.73:0/3391412493 shutdown_connections 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 -- 172.21.15.73:0/3391412493 >> 172.21.15.73:0/3391412493 conn(0x7f6b0c094920 msgr2=0x7f6b0c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 -- 172.21.15.73:0/3391412493 shutdown_connections 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.819+0000 7f6b11311700 1 -- 172.21.15.73:0/3391412493 wait complete. 2023-11-28T03:07:20.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.820+0000 7f6b11311700 1 Processor -- start 2023-11-28T03:07:20.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.820+0000 7f6b11311700 1 -- start start 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b0c14c010 0x7f6b0c0c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0ac060 0x7f6b0c14ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0c2800 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0aff70 con 0x7f6b0c14c010 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b11311700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c0ac4e0 con 0x7f6b0c0ac060 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41768/0 (socket says 172.21.15.73:41768) 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 -- 172.21.15.73:0/3030787896 learned_addr learned my addr 172.21.15.73:0/3030787896 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 -- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0ac060 msgr2=0x7f6b0c14ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0ac060 0x7f6b0c14ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 -- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b0c14c010 msgr2=0x7f6b0c0c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b0c14c010 0x7f6b0c0c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.821+0000 7f6b027fc700 1 -- 172.21.15.73:0/3030787896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b0c0af360 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b027fc700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f6af40119c0 tx=0x7f6af4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400f8e0 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af400fa40 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b0c0af640 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af400fe00 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.822+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c0afc00 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.823+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b0c0b0250 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.823+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6af4014650 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.823+0000 7f6b037fe700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 0x7f6aec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:20.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.824+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6af407a7e0 con 0x7f6b0c14c3f0 2023-11-28T03:07:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.824+0000 7f6b0affd700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 0x7f6aec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.824+0000 7f6b0affd700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 0x7f6aec061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6afc007cf0 tx=0x7f6afc007be0).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:20.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:20.828+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6af404bab0 con 0x7f6b0c14c3f0 2023-11-28T03:07:21.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.028+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b0c06c860 con 0x7f6b0c14c3f0 2023-11-28T03:07:21.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.028+0000 7f6b037fe700 1 -- 172.21.15.73:0/3030787896 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f6af404f0d0 con 0x7f6b0c14c3f0 2023-11-28T03:07:21.029 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:21.029 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:21.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 msgr2=0x7f6aec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 0x7f6aec061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f6afc007cf0 tx=0x7f6afc007be0).stop 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 msgr2=0x7f6b0c0abb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 rx=0x7f6af40119c0 tx=0x7f6af4009060).stop 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 shutdown_connections 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6aec05f150 0x7f6aec061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6b0c0ac060 0x7f6b0c14ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6b0c14c010 0x7f6b0c0c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:21.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 --2- 172.21.15.73:0/3030787896 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6b0c14c3f0 0x7f6b0c0abb70 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.030+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 >> 172.21.15.73:0/3030787896 conn(0x7f6b0c094920 msgr2=0x7f6b0c0ad4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.031+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 shutdown_connections 2023-11-28T03:07:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.031+0000 7f6b11311700 1 -- 172.21.15.73:0/3030787896 wait complete. 2023-11-28T03:07:21.033 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:21.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:21.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.157+0000 7fce824c8700 1 Processor -- start 2023-11-28T03:07:21.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.157+0000 7fce824c8700 1 -- start start 2023-11-28T03:07:21.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce824c8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 0x7fce7c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce824c8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 0x7fce7c0c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce824c8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce7c0c12c0 con 0x7fce7c0c0180 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce824c8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce7c0c06b0 con 0x7fce7c0c1750 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce824c8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c0c0990 con 0x7fce7c157660 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce80cc5700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fce7c0c0180 0x7fce7c1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44474/0 (socket says 172.21.15.73:44474) 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce80cc5700 1 -- 172.21.15.73:0/2265876289 learned_addr learned my addr 172.21.15.73:0/2265876289 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce814c6700 1 --2- 172.21.15.73:0/2265876289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 0x7fce7c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce81cc7700 1 --2- 172.21.15.73:0/2265876289 >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 0x7fce7c0c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce73fff700 1 -- 172.21.15.73:0/2265876289 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3445399696 0 0) 0x7fce7c0c12c0 con 0x7fce7c0c0180 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce73fff700 1 -- 172.21.15.73:0/2265876289 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce680033f0 con 0x7fce7c0c0180 2023-11-28T03:07:21.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce814c6700 1 -- 172.21.15.73:0/2265876289 >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 msgr2=0x7fce7c0c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce814c6700 1 --2- 172.21.15.73:0/2265876289 >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 0x7fce7c0c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce814c6700 1 -- 172.21.15.73:0/2265876289 >> v1:172.21.15.73:6789/0 conn(0x7fce7c0c0180 legacy=0x7fce7c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce814c6700 1 -- 172.21.15.73:0/2265876289 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce7c0c0c70 con 0x7fce7c157660 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.158+0000 7fce81cc7700 1 --2- 172.21.15.73:0/2265876289 >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 0x7fce7c0c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce814c6700 1 --2- 172.21.15.73:0/2265876289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 0x7fce7c157a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fce74009e60 tx=0x7fce74009780).ready entity=mon.2 client_cookie=66166fe8fdc24550 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce73fff700 1 -- 172.21.15.73:0/2265876289 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400e070 con 0x7fce7c157660 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce73fff700 1 -- 172.21.15.73:0/2265876289 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7400bab0 con 0x7fce7c157660 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce73fff700 1 -- 172.21.15.73:0/2265876289 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7400be30 con 0x7fce7c157660 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 -- 172.21.15.73:0/2265876289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 msgr2=0x7fce7c157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 --2- 172.21.15.73:0/2265876289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 0x7fce7c157a40 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7fce74009e60 tx=0x7fce74009780).stop 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 -- 172.21.15.73:0/2265876289 shutdown_connections 2023-11-28T03:07:21.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 --2- 172.21.15.73:0/2265876289 >> v2:172.21.15.150:3300/0 conn(0x7fce7c0c1750 0x7fce7c0c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 --2- 172.21.15.73:0/2265876289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c157660 0x7fce7c157a40 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 -- 172.21.15.73:0/2265876289 >> 172.21.15.73:0/2265876289 conn(0x7fce7c0cb000 msgr2=0x7fce7c0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 -- 172.21.15.73:0/2265876289 shutdown_connections 2023-11-28T03:07:21.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.159+0000 7fce824c8700 1 -- 172.21.15.73:0/2265876289 wait complete. 2023-11-28T03:07:21.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.160+0000 7fce824c8700 1 Processor -- start 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.160+0000 7fce824c8700 1 -- start start 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c0c1750 0x7fce7c143da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 0x7fce7c063ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c144290 con 0x7fce7c0c0180 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c155550 con 0x7fce7c157660 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce824c8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce7c164bf0 con 0x7fce7c0c1750 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce814c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce814c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41770/0 (socket says 172.21.15.73:41770) 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.161+0000 7fce814c6700 1 -- 172.21.15.73:0/198154079 learned_addr learned my addr 172.21.15.73:0/198154079 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce81cc7700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 0x7fce7c063ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce80cc5700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c0c1750 0x7fce7c143da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 -- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c0c1750 msgr2=0x7fce7c143da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c0c1750 0x7fce7c143da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 -- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 msgr2=0x7fce7c063ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 0x7fce7c063ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 -- 172.21.15.73:0/198154079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce780092a0 con 0x7fce7c0c0180 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce81cc7700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 0x7fce7c063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce814c6700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fce74009fd0 tx=0x7fce74009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7401d420 con 0x7fce7c0c0180 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce7401d580 con 0x7fce7c0c0180 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce74009a20 con 0x7fce7c0c0180 2023-11-28T03:07:21.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce7401d940 con 0x7fce7c0c0180 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.162+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce7c1651b0 con 0x7fce7c0c0180 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.163+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce7c155830 con 0x7fce7c0c0180 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.163+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fce74003040 con 0x7fce7c0c0180 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.164+0000 7fce717fa700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 0x7fce60061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.164+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fce7407bf10 con 0x7fce7c0c0180 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.164+0000 7fce80cc5700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 0x7fce60061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.164+0000 7fce80cc5700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 0x7fce60061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fce6c005750 tx=0x7fce6c009040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.168+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce7404d280 con 0x7fce7c0c0180 2023-11-28T03:07:21.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.334+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fce7c06c7c0 con 0x7fce7c0c0180 2023-11-28T03:07:21.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.335+0000 7fce717fa700 1 -- 172.21.15.73:0/198154079 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v82) v1 ==== 74+0+16930 (secure 0 0 0) 0x7fce74056240 con 0x7fce7c0c0180 2023-11-28T03:07:21.336 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:21.336 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":82,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:19.677300+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.337+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 msgr2=0x7fce60061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.337+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 0x7fce60061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fce6c005750 tx=0x7fce6c009040).stop 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.337+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 msgr2=0x7fce7c1645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.337+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 rx=0x7fce74009fd0 tx=0x7fce74009780).stop 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 shutdown_connections 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce6005f150 0x7fce60061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce7c157660 0x7fce7c063ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce7c0c0180 0x7fce7c1645d0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 --2- 172.21.15.73:0/198154079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce7c0c1750 0x7fce7c143da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 >> 172.21.15.73:0/198154079 conn(0x7fce7c0cb000 msgr2=0x7fce7c0aaee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 shutdown_connections 2023-11-28T03:07:21.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.338+0000 7fce824c8700 1 -- 172.21.15.73:0/198154079 wait complete. 2023-11-28T03:07:21.350 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:07:21.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:07:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.464+0000 7f0daa02e700 1 Processor -- start 2023-11-28T03:07:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.464+0000 7f0daa02e700 1 -- start start 2023-11-28T03:07:21.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0daa02e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0da4064e20 0x7f0da4155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0daa02e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 0x7f0da40962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0daa02e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0da40967e0 con 0x7f0da41544d0 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0daa02e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da4067650 con 0x7f0da4064e20 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0daa02e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0da40683d0 con 0x7f0da4095e80 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0da4064e20 0x7f0da4155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0da4064e20 0x7f0da4155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41786/0 (socket says 172.21.15.73:41786) 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 -- 172.21.15.73:0/3557420136 learned_addr learned my addr 172.21.15.73:0/3557420136 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da3fff700 1 --2- 172.21.15.73:0/3557420136 >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 0x7f0da40962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 -- 172.21.15.73:0/3557420136 >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 msgr2=0x7f0da40962f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 --2- 172.21.15.73:0/3557420136 >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 0x7f0da40962f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 -- 172.21.15.73:0/3557420136 >> v1:172.21.15.150:6789/0 conn(0x7f0da41544d0 legacy=0x7f0da4066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 -- 172.21.15.73:0/3557420136 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da4067850 con 0x7f0da4064e20 2023-11-28T03:07:21.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da3fff700 1 --2- 172.21.15.73:0/3557420136 >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 0x7f0da40962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da2ffd700 1 --2- 172.21.15.73:0/3557420136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da4155750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f0da40c9020 tx=0x7f0d980090f0).ready entity=mon.0 client_cookie=ed18d046bbc40d37 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da27fc700 1 -- 172.21.15.73:0/3557420136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800b5a0 con 0x7f0da4064e20 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da27fc700 1 -- 172.21.15.73:0/3557420136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d9800b700 con 0x7f0da4064e20 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.465+0000 7f0da27fc700 1 -- 172.21.15.73:0/3557420136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d9800ba80 con 0x7f0da4064e20 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 -- 172.21.15.73:0/3557420136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 msgr2=0x7f0da4155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 --2- 172.21.15.73:0/3557420136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da4155750 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 rx=0x7f0da40c9020 tx=0x7f0d980090f0).stop 2023-11-28T03:07:21.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 -- 172.21.15.73:0/3557420136 shutdown_connections 2023-11-28T03:07:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 --2- 172.21.15.73:0/3557420136 >> v2:172.21.15.150:3300/0 conn(0x7f0da4095e80 0x7f0da40962f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 --2- 172.21.15.73:0/3557420136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da4155750 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 -- 172.21.15.73:0/3557420136 >> 172.21.15.73:0/3557420136 conn(0x7f0da40631d0 msgr2=0x7f0da40c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 -- 172.21.15.73:0/3557420136 shutdown_connections 2023-11-28T03:07:21.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.466+0000 7f0daa02e700 1 -- 172.21.15.73:0/3557420136 wait complete. 2023-11-28T03:07:21.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.467+0000 7f0daa02e700 1 Processor -- start 2023-11-28T03:07:21.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.467+0000 7f0daa02e700 1 -- start start 2023-11-28T03:07:21.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0da4095e80 0x7f0da414da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 0x7f0da4156470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a6bf0 con 0x7f0da4064e20 2023-11-28T03:07:21.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a5fe0 con 0x7f0da41544d0 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0daa02e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0da40a62c0 con 0x7f0da4095e80 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41798/0 (socket says 172.21.15.73:41798) 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 -- 172.21.15.73:0/49301154 learned_addr learned my addr 172.21.15.73:0/49301154 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da3fff700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 0x7f0da4156470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da2ffd700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0da4095e80 0x7f0da414da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 -- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0da4095e80 msgr2=0x7f0da414da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0da4095e80 0x7f0da414da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 -- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 msgr2=0x7f0da4156470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 0x7f0da4156470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 -- 172.21.15.73:0/49301154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d98009370 con 0x7f0da4064e20 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da3fff700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 0x7f0da4156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.468+0000 7f0da37fe700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f0d9400b490 tx=0x7f0d94006f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.469+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94012410 con 0x7f0da4064e20 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.469+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d940125b0 con 0x7f0da4064e20 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.469+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d94012970 con 0x7f0da4064e20 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.469+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0da40a65a0 con 0x7f0da4064e20 2023-11-28T03:07:21.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.469+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0da414df70 con 0x7f0da4064e20 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.470+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0da40a6ed0 con 0x7f0da4064e20 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.470+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0d94012d20 con 0x7f0da4064e20 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.470+0000 7f0d8b7fe700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 0x7f0d90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.470+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0d94079f80 con 0x7f0da4064e20 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.470+0000 7f0da2ffd700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 0x7f0d90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.471+0000 7f0da2ffd700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 0x7f0d90061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0d98009130 tx=0x7f0d98015000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.474+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d9404b250 con 0x7f0da4064e20 2023-11-28T03:07:21.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.633+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0da414e220 con 0x7f0d9005f150 2023-11-28T03:07:21.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.643+0000 7f0d8b7fe700 1 -- 172.21.15.73:0/49301154 <== mgr.6060 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361391 (secure 0 0 0) 0x7f0d9800b870 con 0x7f0d9005f150 2023-11-28T03:07:21.647 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 msgr2=0x7f0d90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 0x7f0d90061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0d98009130 tx=0x7f0d98015000).stop 2023-11-28T03:07:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 msgr2=0x7f0da40a7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 rx=0x7f0d9400b490 tx=0x7f0d94006f10).stop 2023-11-28T03:07:21.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 shutdown_connections 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f150 0x7f0d90061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0da41544d0 0x7f0da4156470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0da4064e20 0x7f0da40a7080 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.646+0000 7f0daa02e700 1 --2- 172.21.15.73:0/49301154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0da4095e80 0x7f0da414da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.647+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 >> 172.21.15.73:0/49301154 conn(0x7f0da40631d0 msgr2=0x7f0da40c87b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.647+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 shutdown_connections 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.647+0000 7f0daa02e700 1 -- 172.21.15.73:0/49301154 wait complete. 2023-11-28T03:07:21.652 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:07:21.659 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T03:07:19.716672+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2412964,"kb_used_data":7524,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752561756,"statfs":{"total":773094113280,"available":770623238144,"internally_reserved":0,"allocated":7704576,"data_stored":3798401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27602,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-79,"num_write_kb":-64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.598445"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706878+0000","last_change":"2023-11-28T03:07:18.706878+0000","last_active":"2023-11-28T03:07:18.706878+0000","last_peered":"2023-11-28T03:07:18.706878+0000","last_clean":"2023-11-28T03:07:18.706878+0000","last_became_active":"2023-11-28T03:07:18.706607+0000","last_became_peered":"2023-11-28T03:07:18.706607+0000","last_unstale":"2023-11-28T03:07:18.706878+0000","last_undegraded":"2023-11-28T03:07:18.706878+0000","last_fullsized":"2023-11-28T03:07:18.706878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"81'3","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690764+0000","last_change":"2023-11-28T03:07:18.321698+0000","last_active":"2023-11-28T03:07:18.690764+0000","last_peered":"2023-11-28T03:07:18.690764+0000","last_clean":"2023-11-28T03:07:18.690764+0000","last_became_active":"2023-11-28T03:07:18.321444+0000","last_became_peered":"2023-11-28T03:07:18.321444+0000","last_unstale":"2023-11-28T03:07:18.690764+0000","last_undegraded":"2023-11-28T03:07:18.690764+0000","last_fullsized":"2023-11-28T03:07:18.690764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702578+0000","last_change":"2023-11-28T03:07:18.702578+0000","last_active":"2023-11-28T03:07:18.702578+0000","last_peered":"2023-11-28T03:07:18.702578+0000","last_clean":"2023-11-28T03:07:18.702578+0000","last_became_active":"2023-11-28T03:07:18.702292+0000","last_became_peered":"2023-11-28T03:07:18.702292+0000","last_unstale":"2023-11-28T03:07:18.702578+0000","last_undegraded":"2023-11-28T03:07:18.702578+0000","last_fullsized":"2023-11-28T03:07:18.702578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.336556+0000","last_change":"2023-11-28T03:07:18.330522+0000","last_active":"2023-11-28T03:07:18.336556+0000","last_peered":"2023-11-28T03:07:18.336556+0000","last_clean":"2023-11-28T03:07:18.336556+0000","last_became_active":"2023-11-28T03:07:18.330137+0000","last_became_peered":"2023-11-28T03:07:18.330137+0000","last_unstale":"2023-11-28T03:07:18.336556+0000","last_undegraded":"2023-11-28T03:07:18.336556+0000","last_fullsized":"2023-11-28T03:07:18.336556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696512+0000","last_change":"2023-11-28T03:07:18.696512+0000","last_active":"2023-11-28T03:07:18.696512+0000","last_peered":"2023-11-28T03:07:18.696512+0000","last_clean":"2023-11-28T03:07:18.696512+0000","last_became_active":"2023-11-28T03:07:18.696189+0000","last_became_peered":"2023-11-28T03:07:18.696189+0000","last_unstale":"2023-11-28T03:07:18.696512+0000","last_undegraded":"2023-11-28T03:07:18.696512+0000","last_fullsized":"2023-11-28T03:07:18.696512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333645+0000","last_change":"2023-11-28T03:07:18.328975+0000","last_active":"2023-11-28T03:07:18.333645+0000","last_peered":"2023-11-28T03:07:18.333645+0000","last_clean":"2023-11-28T03:07:18.333645+0000","last_became_active":"2023-11-28T03:07:18.328524+0000","last_became_peered":"2023-11-28T03:07:18.328524+0000","last_unstale":"2023-11-28T03:07:18.333645+0000","last_undegraded":"2023-11-28T03:07:18.333645+0000","last_fullsized":"2023-11-28T03:07:18.333645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697781+0000","last_change":"2023-11-28T03:07:18.697781+0000","last_active":"2023-11-28T03:07:18.697781+0000","last_peered":"2023-11-28T03:07:18.697781+0000","last_clean":"2023-11-28T03:07:18.697781+0000","last_became_active":"2023-11-28T03:07:18.697510+0000","last_became_peered":"2023-11-28T03:07:18.697510+0000","last_unstale":"2023-11-28T03:07:18.697781+0000","last_undegraded":"2023-11-28T03:07:18.697781+0000","last_fullsized":"2023-11-28T03:07:18.697781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.691749+0000","last_change":"2023-11-28T03:07:18.327244+0000","last_active":"2023-11-28T03:07:18.691749+0000","last_peered":"2023-11-28T03:07:18.691749+0000","last_clean":"2023-11-28T03:07:18.691749+0000","last_became_active":"2023-11-28T03:07:18.326843+0000","last_became_peered":"2023-11-28T03:07:18.326843+0000","last_unstale":"2023-11-28T03:07:18.691749+0000","last_undegraded":"2023-11-28T03:07:18.691749+0000","last_fullsized":"2023-11-28T03:07:18.691749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699615+0000","last_change":"2023-11-28T03:07:18.699615+0000","last_active":"2023-11-28T03:07:18.699615+0000","last_peered":"2023-11-28T03:07:18.699615+0000","last_clean":"2023-11-28T03:07:18.699615+0000","last_became_active":"2023-11-28T03:07:18.699328+0000","last_became_peered":"2023-11-28T03:07:18.699328+0000","last_unstale":"2023-11-28T03:07:18.699615+0000","last_undegraded":"2023-11-28T03:07:18.699615+0000","last_fullsized":"2023-11-28T03:07:18.699615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.679721+0000","last_change":"2023-11-28T03:07:18.322709+0000","last_active":"2023-11-28T03:07:18.679721+0000","last_peered":"2023-11-28T03:07:18.679721+0000","last_clean":"2023-11-28T03:07:18.679721+0000","last_became_active":"2023-11-28T03:07:18.322454+0000","last_became_peered":"2023-11-28T03:07:18.322454+0000","last_unstale":"2023-11-28T03:07:18.679721+0000","last_undegraded":"2023-11-28T03:07:18.679721+0000","last_fullsized":"2023-11-28T03:07:18.679721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702406+0000","last_change":"2023-11-28T03:07:18.702406+0000","last_active":"2023-11-28T03:07:18.702406+0000","last_peered":"2023-11-28T03:07:18.702406+0000","last_clean":"2023-11-28T03:07:18.702406+0000","last_became_active":"2023-11-28T03:07:18.702134+0000","last_became_peered":"2023-11-28T03:07:18.702134+0000","last_unstale":"2023-11-28T03:07:18.702406+0000","last_undegraded":"2023-11-28T03:07:18.702406+0000","last_fullsized":"2023-11-28T03:07:18.702406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.687789+0000","last_change":"2023-11-28T03:07:18.325296+0000","last_active":"2023-11-28T03:07:18.687789+0000","last_peered":"2023-11-28T03:07:18.687789+0000","last_clean":"2023-11-28T03:07:18.687789+0000","last_became_active":"2023-11-28T03:07:18.325013+0000","last_became_peered":"2023-11-28T03:07:18.325013+0000","last_unstale":"2023-11-28T03:07:18.687789+0000","last_undegraded":"2023-11-28T03:07:18.687789+0000","last_fullsized":"2023-11-28T03:07:18.687789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704977+0000","last_change":"2023-11-28T03:07:18.704977+0000","last_active":"2023-11-28T03:07:18.704977+0000","last_peered":"2023-11-28T03:07:18.704977+0000","last_clean":"2023-11-28T03:07:18.704977+0000","last_became_active":"2023-11-28T03:07:18.704718+0000","last_became_peered":"2023-11-28T03:07:18.704718+0000","last_unstale":"2023-11-28T03:07:18.704977+0000","last_undegraded":"2023-11-28T03:07:18.704977+0000","last_fullsized":"2023-11-28T03:07:18.704977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330820+0000","last_change":"2023-11-28T03:07:18.323993+0000","last_active":"2023-11-28T03:07:18.330820+0000","last_peered":"2023-11-28T03:07:18.330820+0000","last_clean":"2023-11-28T03:07:18.330820+0000","last_became_active":"2023-11-28T03:07:18.323657+0000","last_became_peered":"2023-11-28T03:07:18.323657+0000","last_unstale":"2023-11-28T03:07:18.330820+0000","last_undegraded":"2023-11-28T03:07:18.330820+0000","last_fullsized":"2023-11-28T03:07:18.330820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708719+0000","last_change":"2023-11-28T03:07:18.708719+0000","last_active":"2023-11-28T03:07:18.708719+0000","last_peered":"2023-11-28T03:07:18.708719+0000","last_clean":"2023-11-28T03:07:18.708719+0000","last_became_active":"2023-11-28T03:07:18.708422+0000","last_became_peered":"2023-11-28T03:07:18.708422+0000","last_unstale":"2023-11-28T03:07:18.708719+0000","last_undegraded":"2023-11-28T03:07:18.708719+0000","last_fullsized":"2023-11-28T03:07:18.708719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.321047+0000","last_change":"2023-11-28T03:07:18.318086+0000","last_active":"2023-11-28T03:07:18.321047+0000","last_peered":"2023-11-28T03:07:18.321047+0000","last_clean":"2023-11-28T03:07:18.321047+0000","last_became_active":"2023-11-28T03:07:18.317831+0000","last_became_peered":"2023-11-28T03:07:18.317831+0000","last_unstale":"2023-11-28T03:07:18.321047+0000","last_undegraded":"2023-11-28T03:07:18.321047+0000","last_fullsized":"2023-11-28T03:07:18.321047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.703450+0000","last_change":"2023-11-28T03:07:18.703450+0000","last_active":"2023-11-28T03:07:18.703450+0000","last_peered":"2023-11-28T03:07:18.703450+0000","last_clean":"2023-11-28T03:07:18.703450+0000","last_became_active":"2023-11-28T03:07:18.703111+0000","last_became_peered":"2023-11-28T03:07:18.703111+0000","last_unstale":"2023-11-28T03:07:18.703450+0000","last_undegraded":"2023-11-28T03:07:18.703450+0000","last_fullsized":"2023-11-28T03:07:18.703450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331034+0000","last_change":"2023-11-28T03:07:18.326463+0000","last_active":"2023-11-28T03:07:18.331034+0000","last_peered":"2023-11-28T03:07:18.331034+0000","last_clean":"2023-11-28T03:07:18.331034+0000","last_became_active":"2023-11-28T03:07:18.326206+0000","last_became_peered":"2023-11-28T03:07:18.326206+0000","last_unstale":"2023-11-28T03:07:18.331034+0000","last_undegraded":"2023-11-28T03:07:18.331034+0000","last_fullsized":"2023-11-28T03:07:18.331034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705851+0000","last_change":"2023-11-28T03:07:18.705851+0000","last_active":"2023-11-28T03:07:18.705851+0000","last_peered":"2023-11-28T03:07:18.705851+0000","last_clean":"2023-11-28T03:07:18.705851+0000","last_became_active":"2023-11-28T03:07:18.705134+0000","last_became_peered":"2023-11-28T03:07:18.705134+0000","last_unstale":"2023-11-28T03:07:18.705851+0000","last_undegraded":"2023-11-28T03:07:18.705851+0000","last_fullsized":"2023-11-28T03:07:18.705851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332498+0000","last_change":"2023-11-28T03:07:18.329346+0000","last_active":"2023-11-28T03:07:18.332498+0000","last_peered":"2023-11-28T03:07:18.332498+0000","last_clean":"2023-11-28T03:07:18.332498+0000","last_became_active":"2023-11-28T03:07:18.329075+0000","last_became_peered":"2023-11-28T03:07:18.329075+0000","last_unstale":"2023-11-28T03:07:18.332498+0000","last_undegraded":"2023-11-28T03:07:18.332498+0000","last_fullsized":"2023-11-28T03:07:18.332498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700276+0000","last_change":"2023-11-28T03:07:18.700276+0000","last_active":"2023-11-28T03:07:18.700276+0000","last_peered":"2023-11-28T03:07:18.700276+0000","last_clean":"2023-11-28T03:07:18.700276+0000","last_became_active":"2023-11-28T03:07:18.699951+0000","last_became_peered":"2023-11-28T03:07:18.699951+0000","last_unstale":"2023-11-28T03:07:18.700276+0000","last_undegraded":"2023-11-28T03:07:18.700276+0000","last_fullsized":"2023-11-28T03:07:18.700276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331679+0000","last_change":"2023-11-28T03:07:18.321892+0000","last_active":"2023-11-28T03:07:18.331679+0000","last_peered":"2023-11-28T03:07:18.331679+0000","last_clean":"2023-11-28T03:07:18.331679+0000","last_became_active":"2023-11-28T03:07:18.321640+0000","last_became_peered":"2023-11-28T03:07:18.321640+0000","last_unstale":"2023-11-28T03:07:18.331679+0000","last_undegraded":"2023-11-28T03:07:18.331679+0000","last_fullsized":"2023-11-28T03:07:18.331679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701508+0000","last_change":"2023-11-28T03:07:18.701508+0000","last_active":"2023-11-28T03:07:18.701508+0000","last_peered":"2023-11-28T03:07:18.701508+0000","last_clean":"2023-11-28T03:07:18.701508+0000","last_became_active":"2023-11-28T03:07:18.701220+0000","last_became_peered":"2023-11-28T03:07:18.701220+0000","last_unstale":"2023-11-28T03:07:18.701508+0000","last_undegraded":"2023-11-28T03:07:18.701508+0000","last_fullsized":"2023-11-28T03:07:18.701508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695480+0000","last_change":"2023-11-28T03:07:18.325208+0000","last_active":"2023-11-28T03:07:18.695480+0000","last_peered":"2023-11-28T03:07:18.695480+0000","last_clean":"2023-11-28T03:07:18.695480+0000","last_became_active":"2023-11-28T03:07:18.324734+0000","last_became_peered":"2023-11-28T03:07:18.324734+0000","last_unstale":"2023-11-28T03:07:18.695480+0000","last_undegraded":"2023-11-28T03:07:18.695480+0000","last_fullsized":"2023-11-28T03:07:18.695480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692251+0000","last_change":"2023-11-28T03:07:18.692251+0000","last_active":"2023-11-28T03:07:18.692251+0000","last_peered":"2023-11-28T03:07:18.692251+0000","last_clean":"2023-11-28T03:07:18.692251+0000","last_became_active":"2023-11-28T03:07:18.692031+0000","last_became_peered":"2023-11-28T03:07:18.692031+0000","last_unstale":"2023-11-28T03:07:18.692251+0000","last_undegraded":"2023-11-28T03:07:18.692251+0000","last_fullsized":"2023-11-28T03:07:18.692251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.688653+0000","last_change":"2023-11-28T03:07:18.327445+0000","last_active":"2023-11-28T03:07:18.688653+0000","last_peered":"2023-11-28T03:07:18.688653+0000","last_clean":"2023-11-28T03:07:18.688653+0000","last_became_active":"2023-11-28T03:07:18.327162+0000","last_became_peered":"2023-11-28T03:07:18.327162+0000","last_unstale":"2023-11-28T03:07:18.688653+0000","last_undegraded":"2023-11-28T03:07:18.688653+0000","last_fullsized":"2023-11-28T03:07:18.688653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701652+0000","last_change":"2023-11-28T03:07:18.701652+0000","last_active":"2023-11-28T03:07:18.701652+0000","last_peered":"2023-11-28T03:07:18.701652+0000","last_clean":"2023-11-28T03:07:18.701652+0000","last_became_active":"2023-11-28T03:07:18.701357+0000","last_became_peered":"2023-11-28T03:07:18.701357+0000","last_unstale":"2023-11-28T03:07:18.701652+0000","last_undegraded":"2023-11-28T03:07:18.701652+0000","last_fullsized":"2023-11-28T03:07:18.701652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.316599+0000","last_change":"2023-11-28T03:07:18.314740+0000","last_active":"2023-11-28T03:07:18.316599+0000","last_peered":"2023-11-28T03:07:18.316599+0000","last_clean":"2023-11-28T03:07:18.316599+0000","last_became_active":"2023-11-28T03:07:18.314419+0000","last_became_peered":"2023-11-28T03:07:18.314419+0000","last_unstale":"2023-11-28T03:07:18.316599+0000","last_undegraded":"2023-11-28T03:07:18.316599+0000","last_fullsized":"2023-11-28T03:07:18.316599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702794+0000","last_change":"2023-11-28T03:07:18.702794+0000","last_active":"2023-11-28T03:07:18.702794+0000","last_peered":"2023-11-28T03:07:18.702794+0000","last_clean":"2023-11-28T03:07:18.702794+0000","last_became_active":"2023-11-28T03:07:18.702509+0000","last_became_peered":"2023-11-28T03:07:18.702509+0000","last_unstale":"2023-11-28T03:07:18.702794+0000","last_undegraded":"2023-11-28T03:07:18.702794+0000","last_fullsized":"2023-11-28T03:07:18.702794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330794+0000","last_change":"2023-11-28T03:07:18.321976+0000","last_active":"2023-11-28T03:07:18.330794+0000","last_peered":"2023-11-28T03:07:18.330794+0000","last_clean":"2023-11-28T03:07:18.330794+0000","last_became_active":"2023-11-28T03:07:18.321650+0000","last_became_peered":"2023-11-28T03:07:18.321650+0000","last_unstale":"2023-11-28T03:07:18.330794+0000","last_undegraded":"2023-11-28T03:07:18.330794+0000","last_fullsized":"2023-11-28T03:07:18.330794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707791+0000","last_change":"2023-11-28T03:07:18.708510+0000","last_active":"2023-11-28T03:07:18.707791+0000","last_peered":"2023-11-28T03:07:18.707791+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.707570+0000","last_became_peered":"2023-11-28T03:07:18.707570+0000","last_unstale":"2023-11-28T03:07:18.707791+0000","last_undegraded":"2023-11-28T03:07:18.707791+0000","last_fullsized":"2023-11-28T03:07:18.707791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697252+0000","last_change":"2023-11-28T03:07:18.331676+0000","last_active":"2023-11-28T03:07:18.697252+0000","last_peered":"2023-11-28T03:07:18.697252+0000","last_clean":"2023-11-28T03:07:18.697252+0000","last_became_active":"2023-11-28T03:07:18.331391+0000","last_became_peered":"2023-11-28T03:07:18.331391+0000","last_unstale":"2023-11-28T03:07:18.697252+0000","last_undegraded":"2023-11-28T03:07:18.697252+0000","last_fullsized":"2023-11-28T03:07:18.697252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698612+0000","last_change":"2023-11-28T03:07:18.698612+0000","last_active":"2023-11-28T03:07:18.698612+0000","last_peered":"2023-11-28T03:07:18.698612+0000","last_clean":"2023-11-28T03:07:18.698612+0000","last_became_active":"2023-11-28T03:07:18.697282+0000","last_became_peered":"2023-11-28T03:07:18.697282+0000","last_unstale":"2023-11-28T03:07:18.698612+0000","last_undegraded":"2023-11-28T03:07:18.698612+0000","last_fullsized":"2023-11-28T03:07:18.698612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334386+0000","last_change":"2023-11-28T03:07:18.330147+0000","last_active":"2023-11-28T03:07:18.334386+0000","last_peered":"2023-11-28T03:07:18.334386+0000","last_clean":"2023-11-28T03:07:18.334386+0000","last_became_active":"2023-11-28T03:07:18.329899+0000","last_became_peered":"2023-11-28T03:07:18.329899+0000","last_unstale":"2023-11-28T03:07:18.334386+0000","last_undegraded":"2023-11-28T03:07:18.334386+0000","last_fullsized":"2023-11-28T03:07:18.334386+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697252+0000","last_change":"2023-11-28T03:07:18.697252+0000","last_active":"2023-11-28T03:07:18.697252+0000","last_peered":"2023-11-28T03:07:18.697252+0000","last_clean":"2023-11-28T03:07:18.697252+0000","last_became_active":"2023-11-28T03:07:18.696853+0000","last_became_peered":"2023-11-28T03:07:18.696853+0000","last_unstale":"2023-11-28T03:07:18.697252+0000","last_undegraded":"2023-11-28T03:07:18.697252+0000","last_fullsized":"2023-11-28T03:07:18.697252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333250+0000","last_change":"2023-11-28T03:07:18.327992+0000","last_active":"2023-11-28T03:07:18.333250+0000","last_peered":"2023-11-28T03:07:18.333250+0000","last_clean":"2023-11-28T03:07:18.333250+0000","last_became_active":"2023-11-28T03:07:18.327663+0000","last_became_peered":"2023-11-28T03:07:18.327663+0000","last_unstale":"2023-11-28T03:07:18.333250+0000","last_undegraded":"2023-11-28T03:07:18.333250+0000","last_fullsized":"2023-11-28T03:07:18.333250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705562+0000","last_change":"2023-11-28T03:07:18.705562+0000","last_active":"2023-11-28T03:07:18.705562+0000","last_peered":"2023-11-28T03:07:18.705562+0000","last_clean":"2023-11-28T03:07:18.705562+0000","last_became_active":"2023-11-28T03:07:18.705247+0000","last_became_peered":"2023-11-28T03:07:18.705247+0000","last_unstale":"2023-11-28T03:07:18.705562+0000","last_undegraded":"2023-11-28T03:07:18.705562+0000","last_fullsized":"2023-11-28T03:07:18.705562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.680341+0000","last_change":"2023-11-28T03:07:18.324630+0000","last_active":"2023-11-28T03:07:18.680341+0000","last_peered":"2023-11-28T03:07:18.680341+0000","last_clean":"2023-11-28T03:07:18.680341+0000","last_became_active":"2023-11-28T03:07:18.324353+0000","last_became_peered":"2023-11-28T03:07:18.324353+0000","last_unstale":"2023-11-28T03:07:18.680341+0000","last_undegraded":"2023-11-28T03:07:18.680341+0000","last_fullsized":"2023-11-28T03:07:18.680341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707767+0000","last_change":"2023-11-28T03:07:18.707767+0000","last_active":"2023-11-28T03:07:18.707767+0000","last_peered":"2023-11-28T03:07:18.707767+0000","last_clean":"2023-11-28T03:07:18.707767+0000","last_became_active":"2023-11-28T03:07:18.707486+0000","last_became_peered":"2023-11-28T03:07:18.707486+0000","last_unstale":"2023-11-28T03:07:18.707767+0000","last_undegraded":"2023-11-28T03:07:18.707767+0000","last_fullsized":"2023-11-28T03:07:18.707767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330657+0000","last_change":"2023-11-28T03:07:18.325875+0000","last_active":"2023-11-28T03:07:18.330657+0000","last_peered":"2023-11-28T03:07:18.330657+0000","last_clean":"2023-11-28T03:07:18.330657+0000","last_became_active":"2023-11-28T03:07:18.325614+0000","last_became_peered":"2023-11-28T03:07:18.325614+0000","last_unstale":"2023-11-28T03:07:18.330657+0000","last_undegraded":"2023-11-28T03:07:18.330657+0000","last_fullsized":"2023-11-28T03:07:18.330657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700255+0000","last_change":"2023-11-28T03:07:18.700255+0000","last_active":"2023-11-28T03:07:18.700255+0000","last_peered":"2023-11-28T03:07:18.700255+0000","last_clean":"2023-11-28T03:07:18.700255+0000","last_became_active":"2023-11-28T03:07:18.699890+0000","last_became_peered":"2023-11-28T03:07:18.699890+0000","last_unstale":"2023-11-28T03:07:18.700255+0000","last_undegraded":"2023-11-28T03:07:18.700255+0000","last_fullsized":"2023-11-28T03:07:18.700255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.686378+0000","last_change":"2023-11-28T03:07:18.334497+0000","last_active":"2023-11-28T03:07:18.686378+0000","last_peered":"2023-11-28T03:07:18.686378+0000","last_clean":"2023-11-28T03:07:18.686378+0000","last_became_active":"2023-11-28T03:07:18.334242+0000","last_became_peered":"2023-11-28T03:07:18.334242+0000","last_unstale":"2023-11-28T03:07:18.686378+0000","last_undegraded":"2023-11-28T03:07:18.686378+0000","last_fullsized":"2023-11-28T03:07:18.686378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702339+0000","last_change":"2023-11-28T03:07:18.702339+0000","last_active":"2023-11-28T03:07:18.702339+0000","last_peered":"2023-11-28T03:07:18.702339+0000","last_clean":"2023-11-28T03:07:18.702339+0000","last_became_active":"2023-11-28T03:07:18.702049+0000","last_became_peered":"2023-11-28T03:07:18.702049+0000","last_unstale":"2023-11-28T03:07:18.702339+0000","last_undegraded":"2023-11-28T03:07:18.702339+0000","last_fullsized":"2023-11-28T03:07:18.702339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326549+0000","last_change":"2023-11-28T03:07:18.321229+0000","last_active":"2023-11-28T03:07:18.326549+0000","last_peered":"2023-11-28T03:07:18.326549+0000","last_clean":"2023-11-28T03:07:18.326549+0000","last_became_active":"2023-11-28T03:07:18.320950+0000","last_became_peered":"2023-11-28T03:07:18.320950+0000","last_unstale":"2023-11-28T03:07:18.326549+0000","last_undegraded":"2023-11-28T03:07:18.326549+0000","last_fullsized":"2023-11-28T03:07:18.326549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702984+0000","last_change":"2023-11-28T03:07:18.702984+0000","last_active":"2023-11-28T03:07:18.702984+0000","last_peered":"2023-11-28T03:07:18.702984+0000","last_clean":"2023-11-28T03:07:18.702984+0000","last_became_active":"2023-11-28T03:07:18.702707+0000","last_became_peered":"2023-11-28T03:07:18.702707+0000","last_unstale":"2023-11-28T03:07:18.702984+0000","last_undegraded":"2023-11-28T03:07:18.702984+0000","last_fullsized":"2023-11-28T03:07:18.702984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331259+0000","last_change":"2023-11-28T03:07:18.324721+0000","last_active":"2023-11-28T03:07:18.331259+0000","last_peered":"2023-11-28T03:07:18.331259+0000","last_clean":"2023-11-28T03:07:18.331259+0000","last_became_active":"2023-11-28T03:07:18.324453+0000","last_became_peered":"2023-11-28T03:07:18.324453+0000","last_unstale":"2023-11-28T03:07:18.331259+0000","last_undegraded":"2023-11-28T03:07:18.331259+0000","last_fullsized":"2023-11-28T03:07:18.331259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695305+0000","last_change":"2023-11-28T03:07:18.695305+0000","last_active":"2023-11-28T03:07:18.695305+0000","last_peered":"2023-11-28T03:07:18.695305+0000","last_clean":"2023-11-28T03:07:18.695305+0000","last_became_active":"2023-11-28T03:07:18.694771+0000","last_became_peered":"2023-11-28T03:07:18.694771+0000","last_unstale":"2023-11-28T03:07:18.695305+0000","last_undegraded":"2023-11-28T03:07:18.695305+0000","last_fullsized":"2023-11-28T03:07:18.695305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330604+0000","last_change":"2023-11-28T03:07:18.326763+0000","last_active":"2023-11-28T03:07:18.330604+0000","last_peered":"2023-11-28T03:07:18.330604+0000","last_clean":"2023-11-28T03:07:18.330604+0000","last_became_active":"2023-11-28T03:07:18.326472+0000","last_became_peered":"2023-11-28T03:07:18.326472+0000","last_unstale":"2023-11-28T03:07:18.330604+0000","last_undegraded":"2023-11-28T03:07:18.330604+0000","last_fullsized":"2023-11-28T03:07:18.330604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701528+0000","last_change":"2023-11-28T03:07:18.701528+0000","last_active":"2023-11-28T03:07:18.701528+0000","last_peered":"2023-11-28T03:07:18.701528+0000","last_clean":"2023-11-28T03:07:18.701528+0000","last_became_active":"2023-11-28T03:07:18.701246+0000","last_became_peered":"2023-11-28T03:07:18.701246+0000","last_unstale":"2023-11-28T03:07:18.701528+0000","last_undegraded":"2023-11-28T03:07:18.701528+0000","last_fullsized":"2023-11-28T03:07:18.701528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334356+0000","last_change":"2023-11-28T03:07:18.329642+0000","last_active":"2023-11-28T03:07:18.334356+0000","last_peered":"2023-11-28T03:07:18.334356+0000","last_clean":"2023-11-28T03:07:18.334356+0000","last_became_active":"2023-11-28T03:07:18.329378+0000","last_became_peered":"2023-11-28T03:07:18.329378+0000","last_unstale":"2023-11-28T03:07:18.334356+0000","last_undegraded":"2023-11-28T03:07:18.334356+0000","last_fullsized":"2023-11-28T03:07:18.334356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706031+0000","last_change":"2023-11-28T03:07:18.706031+0000","last_active":"2023-11-28T03:07:18.706031+0000","last_peered":"2023-11-28T03:07:18.706031+0000","last_clean":"2023-11-28T03:07:18.706031+0000","last_became_active":"2023-11-28T03:07:18.705416+0000","last_became_peered":"2023-11-28T03:07:18.705416+0000","last_unstale":"2023-11-28T03:07:18.706031+0000","last_undegraded":"2023-11-28T03:07:18.706031+0000","last_fullsized":"2023-11-28T03:07:18.706031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332531+0000","last_change":"2023-11-28T03:07:18.324147+0000","last_active":"2023-11-28T03:07:18.332531+0000","last_peered":"2023-11-28T03:07:18.332531+0000","last_clean":"2023-11-28T03:07:18.332531+0000","last_became_active":"2023-11-28T03:07:18.323503+0000","last_became_peered":"2023-11-28T03:07:18.323503+0000","last_unstale":"2023-11-28T03:07:18.332531+0000","last_undegraded":"2023-11-28T03:07:18.332531+0000","last_fullsized":"2023-11-28T03:07:18.332531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692668+0000","last_change":"2023-11-28T03:07:18.692668+0000","last_active":"2023-11-28T03:07:18.692668+0000","last_peered":"2023-11-28T03:07:18.692668+0000","last_clean":"2023-11-28T03:07:18.692668+0000","last_became_active":"2023-11-28T03:07:18.692375+0000","last_became_peered":"2023-11-28T03:07:18.692375+0000","last_unstale":"2023-11-28T03:07:18.692668+0000","last_undegraded":"2023-11-28T03:07:18.692668+0000","last_fullsized":"2023-11-28T03:07:18.692668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332785+0000","last_change":"2023-11-28T03:07:18.331573+0000","last_active":"2023-11-28T03:07:18.332785+0000","last_peered":"2023-11-28T03:07:18.332785+0000","last_clean":"2023-11-28T03:07:18.332785+0000","last_became_active":"2023-11-28T03:07:18.331278+0000","last_became_peered":"2023-11-28T03:07:18.331278+0000","last_unstale":"2023-11-28T03:07:18.332785+0000","last_undegraded":"2023-11-28T03:07:18.332785+0000","last_fullsized":"2023-11-28T03:07:18.332785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693495+0000","last_change":"2023-11-28T03:07:18.693495+0000","last_active":"2023-11-28T03:07:18.693495+0000","last_peered":"2023-11-28T03:07:18.693495+0000","last_clean":"2023-11-28T03:07:18.693495+0000","last_became_active":"2023-11-28T03:07:18.691131+0000","last_became_peered":"2023-11-28T03:07:18.691131+0000","last_unstale":"2023-11-28T03:07:18.693495+0000","last_undegraded":"2023-11-28T03:07:18.693495+0000","last_fullsized":"2023-11-28T03:07:18.693495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.339356+0000","last_change":"2023-11-28T03:07:18.335639+0000","last_active":"2023-11-28T03:07:18.339356+0000","last_peered":"2023-11-28T03:07:18.339356+0000","last_clean":"2023-11-28T03:07:18.339356+0000","last_became_active":"2023-11-28T03:07:18.335375+0000","last_became_peered":"2023-11-28T03:07:18.335375+0000","last_unstale":"2023-11-28T03:07:18.339356+0000","last_undegraded":"2023-11-28T03:07:18.339356+0000","last_fullsized":"2023-11-28T03:07:18.339356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696864+0000","last_change":"2023-11-28T03:07:18.696864+0000","last_active":"2023-11-28T03:07:18.696864+0000","last_peered":"2023-11-28T03:07:18.696864+0000","last_clean":"2023-11-28T03:07:18.696864+0000","last_became_active":"2023-11-28T03:07:18.696585+0000","last_became_peered":"2023-11-28T03:07:18.696585+0000","last_unstale":"2023-11-28T03:07:18.696864+0000","last_undegraded":"2023-11-28T03:07:18.696864+0000","last_fullsized":"2023-11-28T03:07:18.696864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331857+0000","last_change":"2023-11-28T03:07:18.327444+0000","last_active":"2023-11-28T03:07:18.331857+0000","last_peered":"2023-11-28T03:07:18.331857+0000","last_clean":"2023-11-28T03:07:18.331857+0000","last_became_active":"2023-11-28T03:07:18.327181+0000","last_became_peered":"2023-11-28T03:07:18.327181+0000","last_unstale":"2023-11-28T03:07:18.331857+0000","last_undegraded":"2023-11-28T03:07:18.331857+0000","last_fullsized":"2023-11-28T03:07:18.331857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704933+0000","last_change":"2023-11-28T03:07:18.704933+0000","last_active":"2023-11-28T03:07:18.704933+0000","last_peered":"2023-11-28T03:07:18.704933+0000","last_clean":"2023-11-28T03:07:18.704933+0000","last_became_active":"2023-11-28T03:07:18.704570+0000","last_became_peered":"2023-11-28T03:07:18.704570+0000","last_unstale":"2023-11-28T03:07:18.704933+0000","last_undegraded":"2023-11-28T03:07:18.704933+0000","last_fullsized":"2023-11-28T03:07:18.704933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333217+0000","last_change":"2023-11-28T03:07:18.327860+0000","last_active":"2023-11-28T03:07:18.333217+0000","last_peered":"2023-11-28T03:07:18.333217+0000","last_clean":"2023-11-28T03:07:18.333217+0000","last_became_active":"2023-11-28T03:07:18.327540+0000","last_became_peered":"2023-11-28T03:07:18.327540+0000","last_unstale":"2023-11-28T03:07:18.333217+0000","last_undegraded":"2023-11-28T03:07:18.333217+0000","last_fullsized":"2023-11-28T03:07:18.333217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705088+0000","last_change":"2023-11-28T03:07:18.705088+0000","last_active":"2023-11-28T03:07:18.705088+0000","last_peered":"2023-11-28T03:07:18.705088+0000","last_clean":"2023-11-28T03:07:18.705088+0000","last_became_active":"2023-11-28T03:07:18.704729+0000","last_became_peered":"2023-11-28T03:07:18.704729+0000","last_unstale":"2023-11-28T03:07:18.705088+0000","last_undegraded":"2023-11-28T03:07:18.705088+0000","last_fullsized":"2023-11-28T03:07:18.705088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334180+0000","last_change":"2023-11-28T03:07:18.329017+0000","last_active":"2023-11-28T03:07:18.334180+0000","last_peered":"2023-11-28T03:07:18.334180+0000","last_clean":"2023-11-28T03:07:18.334180+0000","last_became_active":"2023-11-28T03:07:18.328722+0000","last_became_peered":"2023-11-28T03:07:18.328722+0000","last_unstale":"2023-11-28T03:07:18.334180+0000","last_undegraded":"2023-11-28T03:07:18.334180+0000","last_fullsized":"2023-11-28T03:07:18.334180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708870+0000","last_change":"2023-11-28T03:07:18.708941+0000","last_active":"2023-11-28T03:07:18.708870+0000","last_peered":"2023-11-28T03:07:18.708870+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.708676+0000","last_became_peered":"2023-11-28T03:07:18.708676+0000","last_unstale":"2023-11-28T03:07:18.708870+0000","last_undegraded":"2023-11-28T03:07:18.708870+0000","last_fullsized":"2023-11-28T03:07:18.708870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326433+0000","last_change":"2023-11-28T03:07:18.321575+0000","last_active":"2023-11-28T03:07:18.326433+0000","last_peered":"2023-11-28T03:07:18.326433+0000","last_clean":"2023-11-28T03:07:18.326433+0000","last_became_active":"2023-11-28T03:07:18.321289+0000","last_became_peered":"2023-11-28T03:07:18.321289+0000","last_unstale":"2023-11-28T03:07:18.326433+0000","last_undegraded":"2023-11-28T03:07:18.326433+0000","last_fullsized":"2023-11-28T03:07:18.326433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698169+0000","last_change":"2023-11-28T03:07:18.698169+0000","last_active":"2023-11-28T03:07:18.698169+0000","last_peered":"2023-11-28T03:07:18.698169+0000","last_clean":"2023-11-28T03:07:18.698169+0000","last_became_active":"2023-11-28T03:07:18.697586+0000","last_became_peered":"2023-11-28T03:07:18.697586+0000","last_unstale":"2023-11-28T03:07:18.698169+0000","last_undegraded":"2023-11-28T03:07:18.698169+0000","last_fullsized":"2023-11-28T03:07:18.698169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.688984+0000","last_change":"2023-11-28T03:07:18.318292+0000","last_active":"2023-11-28T03:07:18.688984+0000","last_peered":"2023-11-28T03:07:18.688984+0000","last_clean":"2023-11-28T03:07:18.688984+0000","last_became_active":"2023-11-28T03:07:18.318121+0000","last_became_peered":"2023-11-28T03:07:18.318121+0000","last_unstale":"2023-11-28T03:07:18.688984+0000","last_undegraded":"2023-11-28T03:07:18.688984+0000","last_fullsized":"2023-11-28T03:07:18.688984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705687+0000","last_change":"2023-11-28T03:07:18.705687+0000","last_active":"2023-11-28T03:07:18.705687+0000","last_peered":"2023-11-28T03:07:18.705687+0000","last_clean":"2023-11-28T03:07:18.705687+0000","last_became_active":"2023-11-28T03:07:18.704866+0000","last_became_peered":"2023-11-28T03:07:18.704866+0000","last_unstale":"2023-11-28T03:07:18.705687+0000","last_undegraded":"2023-11-28T03:07:18.705687+0000","last_fullsized":"2023-11-28T03:07:18.705687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690925+0000","last_change":"2023-11-28T03:07:18.332883+0000","last_active":"2023-11-28T03:07:18.690925+0000","last_peered":"2023-11-28T03:07:18.690925+0000","last_clean":"2023-11-28T03:07:18.690925+0000","last_became_active":"2023-11-28T03:07:18.332630+0000","last_became_peered":"2023-11-28T03:07:18.332630+0000","last_unstale":"2023-11-28T03:07:18.690925+0000","last_undegraded":"2023-11-28T03:07:18.690925+0000","last_fullsized":"2023-11-28T03:07:18.690925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695813+0000","last_change":"2023-11-28T03:07:18.695813+0000","last_active":"2023-11-28T03:07:18.695813+0000","last_peered":"2023-11-28T03:07:18.695813+0000","last_clean":"2023-11-28T03:07:18.695813+0000","last_became_active":"2023-11-28T03:07:18.694746+0000","last_became_peered":"2023-11-28T03:07:18.694746+0000","last_unstale":"2023-11-28T03:07:18.695813+0000","last_undegraded":"2023-11-28T03:07:18.695813+0000","last_fullsized":"2023-11-28T03:07:18.695813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706656+0000","last_change":"2023-11-28T03:07:18.326125+0000","last_active":"2023-11-28T03:07:18.706656+0000","last_peered":"2023-11-28T03:07:18.706656+0000","last_clean":"2023-11-28T03:07:18.706656+0000","last_became_active":"2023-11-28T03:07:18.325841+0000","last_became_peered":"2023-11-28T03:07:18.325841+0000","last_unstale":"2023-11-28T03:07:18.706656+0000","last_undegraded":"2023-11-28T03:07:18.706656+0000","last_fullsized":"2023-11-28T03:07:18.706656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699429+0000","last_change":"2023-11-28T03:07:18.699429+0000","last_active":"2023-11-28T03:07:18.699429+0000","last_peered":"2023-11-28T03:07:18.699429+0000","last_clean":"2023-11-28T03:07:18.699429+0000","last_became_active":"2023-11-28T03:07:18.699112+0000","last_became_peered":"2023-11-28T03:07:18.699112+0000","last_unstale":"2023-11-28T03:07:18.699429+0000","last_undegraded":"2023-11-28T03:07:18.699429+0000","last_fullsized":"2023-11-28T03:07:18.699429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332425+0000","last_change":"2023-11-28T03:07:18.329147+0000","last_active":"2023-11-28T03:07:18.332425+0000","last_peered":"2023-11-28T03:07:18.332425+0000","last_clean":"2023-11-28T03:07:18.332425+0000","last_became_active":"2023-11-28T03:07:18.328870+0000","last_became_peered":"2023-11-28T03:07:18.328870+0000","last_unstale":"2023-11-28T03:07:18.332425+0000","last_undegraded":"2023-11-28T03:07:18.332425+0000","last_fullsized":"2023-11-28T03:07:18.332425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692223+0000","last_change":"2023-11-28T03:07:18.692223+0000","last_active":"2023-11-28T03:07:18.692223+0000","last_peered":"2023-11-28T03:07:18.692223+0000","last_clean":"2023-11-28T03:07:18.692223+0000","last_became_active":"2023-11-28T03:07:18.691720+0000","last_became_peered":"2023-11-28T03:07:18.691720+0000","last_unstale":"2023-11-28T03:07:18.692223+0000","last_undegraded":"2023-11-28T03:07:18.692223+0000","last_fullsized":"2023-11-28T03:07:18.692223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331822+0000","last_change":"2023-11-28T03:07:18.327323+0000","last_active":"2023-11-28T03:07:18.331822+0000","last_peered":"2023-11-28T03:07:18.331822+0000","last_clean":"2023-11-28T03:07:18.331822+0000","last_became_active":"2023-11-28T03:07:18.327092+0000","last_became_peered":"2023-11-28T03:07:18.327092+0000","last_unstale":"2023-11-28T03:07:18.331822+0000","last_undegraded":"2023-11-28T03:07:18.331822+0000","last_fullsized":"2023-11-28T03:07:18.331822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707797+0000","last_change":"2023-11-28T03:07:18.707797+0000","last_active":"2023-11-28T03:07:18.707797+0000","last_peered":"2023-11-28T03:07:18.707797+0000","last_clean":"2023-11-28T03:07:18.707797+0000","last_became_active":"2023-11-28T03:07:18.707504+0000","last_became_peered":"2023-11-28T03:07:18.707504+0000","last_unstale":"2023-11-28T03:07:18.707797+0000","last_undegraded":"2023-11-28T03:07:18.707797+0000","last_fullsized":"2023-11-28T03:07:18.707797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.320791+0000","last_change":"2023-11-28T03:07:18.315713+0000","last_active":"2023-11-28T03:07:18.320791+0000","last_peered":"2023-11-28T03:07:18.320791+0000","last_clean":"2023-11-28T03:07:18.320791+0000","last_became_active":"2023-11-28T03:07:18.315399+0000","last_became_peered":"2023-11-28T03:07:18.315399+0000","last_unstale":"2023-11-28T03:07:18.320791+0000","last_undegraded":"2023-11-28T03:07:18.320791+0000","last_fullsized":"2023-11-28T03:07:18.320791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701652+0000","last_change":"2023-11-28T03:07:18.701652+0000","last_active":"2023-11-28T03:07:18.701652+0000","last_peered":"2023-11-28T03:07:18.701652+0000","last_clean":"2023-11-28T03:07:18.701652+0000","last_became_active":"2023-11-28T03:07:18.701377+0000","last_became_peered":"2023-11-28T03:07:18.701377+0000","last_unstale":"2023-11-28T03:07:18.701652+0000","last_undegraded":"2023-11-28T03:07:18.701652+0000","last_fullsized":"2023-11-28T03:07:18.701652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334449+0000","last_change":"2023-11-28T03:07:18.330040+0000","last_active":"2023-11-28T03:07:18.334449+0000","last_peered":"2023-11-28T03:07:18.334449+0000","last_clean":"2023-11-28T03:07:18.334449+0000","last_became_active":"2023-11-28T03:07:18.327794+0000","last_became_peered":"2023-11-28T03:07:18.327794+0000","last_unstale":"2023-11-28T03:07:18.334449+0000","last_undegraded":"2023-11-28T03:07:18.334449+0000","last_fullsized":"2023-11-28T03:07:18.334449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705859+0000","last_change":"2023-11-28T03:07:18.705859+0000","last_active":"2023-11-28T03:07:18.705859+0000","last_peered":"2023-11-28T03:07:18.705859+0000","last_clean":"2023-11-28T03:07:18.705859+0000","last_became_active":"2023-11-28T03:07:18.705610+0000","last_became_peered":"2023-11-28T03:07:18.705610+0000","last_unstale":"2023-11-28T03:07:18.705859+0000","last_undegraded":"2023-11-28T03:07:18.705859+0000","last_fullsized":"2023-11-28T03:07:18.705859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330622+0000","last_change":"2023-11-28T03:07:18.326813+0000","last_active":"2023-11-28T03:07:18.330622+0000","last_peered":"2023-11-28T03:07:18.330622+0000","last_clean":"2023-11-28T03:07:18.330622+0000","last_became_active":"2023-11-28T03:07:18.326507+0000","last_became_peered":"2023-11-28T03:07:18.326507+0000","last_unstale":"2023-11-28T03:07:18.330622+0000","last_undegraded":"2023-11-28T03:07:18.330622+0000","last_fullsized":"2023-11-28T03:07:18.330622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693070+0000","last_change":"2023-11-28T03:07:18.693070+0000","last_active":"2023-11-28T03:07:18.693070+0000","last_peered":"2023-11-28T03:07:18.693070+0000","last_clean":"2023-11-28T03:07:18.693070+0000","last_became_active":"2023-11-28T03:07:18.692797+0000","last_became_peered":"2023-11-28T03:07:18.692797+0000","last_unstale":"2023-11-28T03:07:18.693070+0000","last_undegraded":"2023-11-28T03:07:18.693070+0000","last_fullsized":"2023-11-28T03:07:18.693070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.335099+0000","last_change":"2023-11-28T03:07:18.332734+0000","last_active":"2023-11-28T03:07:18.335099+0000","last_peered":"2023-11-28T03:07:18.335099+0000","last_clean":"2023-11-28T03:07:18.335099+0000","last_became_active":"2023-11-28T03:07:18.332433+0000","last_became_peered":"2023-11-28T03:07:18.332433+0000","last_unstale":"2023-11-28T03:07:18.335099+0000","last_undegraded":"2023-11-28T03:07:18.335099+0000","last_fullsized":"2023-11-28T03:07:18.335099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692820+0000","last_change":"2023-11-28T03:07:18.692820+0000","last_active":"2023-11-28T03:07:18.692820+0000","last_peered":"2023-11-28T03:07:18.692820+0000","last_clean":"2023-11-28T03:07:18.692820+0000","last_became_active":"2023-11-28T03:07:18.692571+0000","last_became_peered":"2023-11-28T03:07:18.692571+0000","last_unstale":"2023-11-28T03:07:18.692820+0000","last_undegraded":"2023-11-28T03:07:18.692820+0000","last_fullsized":"2023-11-28T03:07:18.692820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330778+0000","last_change":"2023-11-28T03:07:18.322012+0000","last_active":"2023-11-28T03:07:18.330778+0000","last_peered":"2023-11-28T03:07:18.330778+0000","last_clean":"2023-11-28T03:07:18.330778+0000","last_became_active":"2023-11-28T03:07:18.321741+0000","last_became_peered":"2023-11-28T03:07:18.321741+0000","last_unstale":"2023-11-28T03:07:18.330778+0000","last_undegraded":"2023-11-28T03:07:18.330778+0000","last_fullsized":"2023-11-28T03:07:18.330778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696598+0000","last_change":"2023-11-28T03:07:18.696598+0000","last_active":"2023-11-28T03:07:18.696598+0000","last_peered":"2023-11-28T03:07:18.696598+0000","last_clean":"2023-11-28T03:07:18.696598+0000","last_became_active":"2023-11-28T03:07:18.696280+0000","last_became_peered":"2023-11-28T03:07:18.696280+0000","last_unstale":"2023-11-28T03:07:18.696598+0000","last_undegraded":"2023-11-28T03:07:18.696598+0000","last_fullsized":"2023-11-28T03:07:18.696598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"81'3","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704761+0000","last_change":"2023-11-28T03:07:18.333084+0000","last_active":"2023-11-28T03:07:18.704761+0000","last_peered":"2023-11-28T03:07:18.704761+0000","last_clean":"2023-11-28T03:07:18.704761+0000","last_became_active":"2023-11-28T03:07:18.332341+0000","last_became_peered":"2023-11-28T03:07:18.332341+0000","last_unstale":"2023-11-28T03:07:18.704761+0000","last_undegraded":"2023-11-28T03:07:18.704761+0000","last_fullsized":"2023-11-28T03:07:18.704761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698702+0000","last_change":"2023-11-28T03:07:18.698702+0000","last_active":"2023-11-28T03:07:18.698702+0000","last_peered":"2023-11-28T03:07:18.698702+0000","last_clean":"2023-11-28T03:07:18.698702+0000","last_became_active":"2023-11-28T03:07:18.698359+0000","last_became_peered":"2023-11-28T03:07:18.698359+0000","last_unstale":"2023-11-28T03:07:18.698702+0000","last_undegraded":"2023-11-28T03:07:18.698702+0000","last_fullsized":"2023-11-28T03:07:18.698702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708516+0000","last_change":"2023-11-28T03:07:18.319397+0000","last_active":"2023-11-28T03:07:18.708516+0000","last_peered":"2023-11-28T03:07:18.708516+0000","last_clean":"2023-11-28T03:07:18.708516+0000","last_became_active":"2023-11-28T03:07:18.319044+0000","last_became_peered":"2023-11-28T03:07:18.319044+0000","last_unstale":"2023-11-28T03:07:18.708516+0000","last_undegraded":"2023-11-28T03:07:18.708516+0000","last_fullsized":"2023-11-28T03:07:18.708516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.703657+0000","last_change":"2023-11-28T03:07:18.703657+0000","last_active":"2023-11-28T03:07:18.703657+0000","last_peered":"2023-11-28T03:07:18.703657+0000","last_clean":"2023-11-28T03:07:18.703657+0000","last_became_active":"2023-11-28T03:07:18.703363+0000","last_became_peered":"2023-11-28T03:07:18.703363+0000","last_unstale":"2023-11-28T03:07:18.703657+0000","last_undegraded":"2023-11-28T03:07:18.703657+0000","last_fullsized":"2023-11-28T03:07:18.703657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.686466+0000","last_change":"2023-11-28T03:07:18.331256+0000","last_active":"2023-11-28T03:07:18.686466+0000","last_peered":"2023-11-28T03:07:18.686466+0000","last_clean":"2023-11-28T03:07:18.686466+0000","last_became_active":"2023-11-28T03:07:18.330974+0000","last_became_peered":"2023-11-28T03:07:18.330974+0000","last_unstale":"2023-11-28T03:07:18.686466+0000","last_undegraded":"2023-11-28T03:07:18.686466+0000","last_fullsized":"2023-11-28T03:07:18.686466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699865+0000","last_change":"2023-11-28T03:07:18.699865+0000","last_active":"2023-11-28T03:07:18.699865+0000","last_peered":"2023-11-28T03:07:18.699865+0000","last_clean":"2023-11-28T03:07:18.699865+0000","last_became_active":"2023-11-28T03:07:18.698850+0000","last_became_peered":"2023-11-28T03:07:18.698850+0000","last_unstale":"2023-11-28T03:07:18.699865+0000","last_undegraded":"2023-11-28T03:07:18.699865+0000","last_fullsized":"2023-11-28T03:07:18.699865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334633+0000","last_change":"2023-11-28T03:07:18.332481+0000","last_active":"2023-11-28T03:07:18.334633+0000","last_peered":"2023-11-28T03:07:18.334633+0000","last_clean":"2023-11-28T03:07:18.334633+0000","last_became_active":"2023-11-28T03:07:18.332182+0000","last_became_peered":"2023-11-28T03:07:18.332182+0000","last_unstale":"2023-11-28T03:07:18.334633+0000","last_undegraded":"2023-11-28T03:07:18.334633+0000","last_fullsized":"2023-11-28T03:07:18.334633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698560+0000","last_change":"2023-11-28T03:07:18.698560+0000","last_active":"2023-11-28T03:07:18.698560+0000","last_peered":"2023-11-28T03:07:18.698560+0000","last_clean":"2023-11-28T03:07:18.698560+0000","last_became_active":"2023-11-28T03:07:18.696810+0000","last_became_peered":"2023-11-28T03:07:18.696810+0000","last_unstale":"2023-11-28T03:07:18.698560+0000","last_undegraded":"2023-11-28T03:07:18.698560+0000","last_fullsized":"2023-11-28T03:07:18.698560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331280+0000","last_change":"2023-11-28T03:07:18.324854+0000","last_active":"2023-11-28T03:07:18.331280+0000","last_peered":"2023-11-28T03:07:18.331280+0000","last_clean":"2023-11-28T03:07:18.331280+0000","last_became_active":"2023-11-28T03:07:18.324573+0000","last_became_peered":"2023-11-28T03:07:18.324573+0000","last_unstale":"2023-11-28T03:07:18.331280+0000","last_undegraded":"2023-11-28T03:07:18.331280+0000","last_fullsized":"2023-11-28T03:07:18.331280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702968+0000","last_change":"2023-11-28T03:07:18.702968+0000","last_active":"2023-11-28T03:07:18.702968+0000","last_peered":"2023-11-28T03:07:18.702968+0000","last_clean":"2023-11-28T03:07:18.702968+0000","last_became_active":"2023-11-28T03:07:18.702691+0000","last_became_peered":"2023-11-28T03:07:18.702691+0000","last_unstale":"2023-11-28T03:07:18.702968+0000","last_undegraded":"2023-11-28T03:07:18.702968+0000","last_fullsized":"2023-11-28T03:07:18.702968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706623+0000","last_change":"2023-11-28T03:07:18.318505+0000","last_active":"2023-11-28T03:07:18.706623+0000","last_peered":"2023-11-28T03:07:18.706623+0000","last_clean":"2023-11-28T03:07:18.706623+0000","last_became_active":"2023-11-28T03:07:18.318173+0000","last_became_peered":"2023-11-28T03:07:18.318173+0000","last_unstale":"2023-11-28T03:07:18.706623+0000","last_undegraded":"2023-11-28T03:07:18.706623+0000","last_fullsized":"2023-11-28T03:07:18.706623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708779+0000","last_change":"2023-11-28T03:07:18.708779+0000","last_active":"2023-11-28T03:07:18.708779+0000","last_peered":"2023-11-28T03:07:18.708779+0000","last_clean":"2023-11-28T03:07:18.708779+0000","last_became_active":"2023-11-28T03:07:18.708503+0000","last_became_peered":"2023-11-28T03:07:18.708503+0000","last_unstale":"2023-11-28T03:07:18.708779+0000","last_undegraded":"2023-11-28T03:07:18.708779+0000","last_fullsized":"2023-11-28T03:07:18.708779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697354+0000","last_change":"2023-11-28T03:07:18.329942+0000","last_active":"2023-11-28T03:07:18.697354+0000","last_peered":"2023-11-28T03:07:18.697354+0000","last_clean":"2023-11-28T03:07:18.697354+0000","last_became_active":"2023-11-28T03:07:18.329664+0000","last_became_peered":"2023-11-28T03:07:18.329664+0000","last_unstale":"2023-11-28T03:07:18.697354+0000","last_undegraded":"2023-11-28T03:07:18.697354+0000","last_fullsized":"2023-11-28T03:07:18.697354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.689393+0000","last_change":"2023-11-28T03:07:18.689393+0000","last_active":"2023-11-28T03:07:18.689393+0000","last_peered":"2023-11-28T03:07:18.689393+0000","last_clean":"2023-11-28T03:07:18.689393+0000","last_became_active":"2023-11-28T03:07:18.688368+0000","last_became_peered":"2023-11-28T03:07:18.688368+0000","last_unstale":"2023-11-28T03:07:18.689393+0000","last_undegraded":"2023-11-28T03:07:18.689393+0000","last_fullsized":"2023-11-28T03:07:18.689393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332923+0000","last_change":"2023-11-28T03:07:18.327819+0000","last_active":"2023-11-28T03:07:18.332923+0000","last_peered":"2023-11-28T03:07:18.332923+0000","last_clean":"2023-11-28T03:07:18.332923+0000","last_became_active":"2023-11-28T03:07:18.326925+0000","last_became_peered":"2023-11-28T03:07:18.326925+0000","last_unstale":"2023-11-28T03:07:18.332923+0000","last_undegraded":"2023-11-28T03:07:18.332923+0000","last_fullsized":"2023-11-28T03:07:18.332923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693734+0000","last_change":"2023-11-28T03:07:18.693734+0000","last_active":"2023-11-28T03:07:18.693734+0000","last_peered":"2023-11-28T03:07:18.693734+0000","last_clean":"2023-11-28T03:07:18.693734+0000","last_became_active":"2023-11-28T03:07:18.691466+0000","last_became_peered":"2023-11-28T03:07:18.691466+0000","last_unstale":"2023-11-28T03:07:18.693734+0000","last_undegraded":"2023-11-28T03:07:18.693734+0000","last_fullsized":"2023-11-28T03:07:18.693734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334813+0000","last_change":"2023-11-28T03:07:18.334046+0000","last_active":"2023-11-28T03:07:18.334813+0000","last_peered":"2023-11-28T03:07:18.334813+0000","last_clean":"2023-11-28T03:07:18.334813+0000","last_became_active":"2023-11-28T03:07:18.333743+0000","last_became_peered":"2023-11-28T03:07:18.333743+0000","last_unstale":"2023-11-28T03:07:18.334813+0000","last_undegraded":"2023-11-28T03:07:18.334813+0000","last_fullsized":"2023-11-28T03:07:18.334813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701396+0000","last_change":"2023-11-28T03:07:18.701396+0000","last_active":"2023-11-28T03:07:18.701396+0000","last_peered":"2023-11-28T03:07:18.701396+0000","last_clean":"2023-11-28T03:07:18.701396+0000","last_became_active":"2023-11-28T03:07:18.701046+0000","last_became_peered":"2023-11-28T03:07:18.701046+0000","last_unstale":"2023-11-28T03:07:18.701396+0000","last_undegraded":"2023-11-28T03:07:18.701396+0000","last_fullsized":"2023-11-28T03:07:18.701396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.328543+0000","last_change":"2023-11-28T03:07:18.317867+0000","last_active":"2023-11-28T03:07:18.328543+0000","last_peered":"2023-11-28T03:07:18.328543+0000","last_clean":"2023-11-28T03:07:18.328543+0000","last_became_active":"2023-11-28T03:07:18.315981+0000","last_became_peered":"2023-11-28T03:07:18.315981+0000","last_unstale":"2023-11-28T03:07:18.328543+0000","last_undegraded":"2023-11-28T03:07:18.328543+0000","last_fullsized":"2023-11-28T03:07:18.328543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696920+0000","last_change":"2023-11-28T03:07:18.696920+0000","last_active":"2023-11-28T03:07:18.696920+0000","last_peered":"2023-11-28T03:07:18.696920+0000","last_clean":"2023-11-28T03:07:18.696920+0000","last_became_active":"2023-11-28T03:07:18.696474+0000","last_became_peered":"2023-11-28T03:07:18.696474+0000","last_unstale":"2023-11-28T03:07:18.696920+0000","last_undegraded":"2023-11-28T03:07:18.696920+0000","last_fullsized":"2023-11-28T03:07:18.696920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333000+0000","last_change":"2023-11-28T03:07:18.326232+0000","last_active":"2023-11-28T03:07:18.333000+0000","last_peered":"2023-11-28T03:07:18.333000+0000","last_clean":"2023-11-28T03:07:18.333000+0000","last_became_active":"2023-11-28T03:07:18.325971+0000","last_became_peered":"2023-11-28T03:07:18.325971+0000","last_unstale":"2023-11-28T03:07:18.333000+0000","last_undegraded":"2023-11-28T03:07:18.333000+0000","last_fullsized":"2023-11-28T03:07:18.333000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706384+0000","last_change":"2023-11-28T03:07:18.706384+0000","last_active":"2023-11-28T03:07:18.706384+0000","last_peered":"2023-11-28T03:07:18.706384+0000","last_clean":"2023-11-28T03:07:18.706384+0000","last_became_active":"2023-11-28T03:07:18.704207+0000","last_became_peered":"2023-11-28T03:07:18.704207+0000","last_unstale":"2023-11-28T03:07:18.706384+0000","last_undegraded":"2023-11-28T03:07:18.706384+0000","last_fullsized":"2023-11-28T03:07:18.706384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.336538+0000","last_change":"2023-11-28T03:07:18.330557+0000","last_active":"2023-11-28T03:07:18.336538+0000","last_peered":"2023-11-28T03:07:18.336538+0000","last_clean":"2023-11-28T03:07:18.336538+0000","last_became_active":"2023-11-28T03:07:18.330299+0000","last_became_peered":"2023-11-28T03:07:18.330299+0000","last_unstale":"2023-11-28T03:07:18.336538+0000","last_undegraded":"2023-11-28T03:07:18.336538+0000","last_fullsized":"2023-11-28T03:07:18.336538+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696929+0000","last_change":"2023-11-28T03:07:18.696929+0000","last_active":"2023-11-28T03:07:18.696929+0000","last_peered":"2023-11-28T03:07:18.696929+0000","last_clean":"2023-11-28T03:07:18.696929+0000","last_became_active":"2023-11-28T03:07:18.696694+0000","last_became_peered":"2023-11-28T03:07:18.696694+0000","last_unstale":"2023-11-28T03:07:18.696929+0000","last_undegraded":"2023-11-28T03:07:18.696929+0000","last_fullsized":"2023-11-28T03:07:18.696929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326233+0000","last_change":"2023-11-28T03:07:18.323531+0000","last_active":"2023-11-28T03:07:18.326233+0000","last_peered":"2023-11-28T03:07:18.326233+0000","last_clean":"2023-11-28T03:07:18.326233+0000","last_became_active":"2023-11-28T03:07:18.323277+0000","last_became_peered":"2023-11-28T03:07:18.323277+0000","last_unstale":"2023-11-28T03:07:18.326233+0000","last_undegraded":"2023-11-28T03:07:18.326233+0000","last_fullsized":"2023-11-28T03:07:18.326233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.691919+0000","last_change":"2023-11-28T03:07:18.691919+0000","last_active":"2023-11-28T03:07:18.691919+0000","last_peered":"2023-11-28T03:07:18.691919+0000","last_clean":"2023-11-28T03:07:18.691919+0000","last_became_active":"2023-11-28T03:07:18.691649+0000","last_became_peered":"2023-11-28T03:07:18.691649+0000","last_unstale":"2023-11-28T03:07:18.691919+0000","last_undegraded":"2023-11-28T03:07:18.691919+0000","last_fullsized":"2023-11-28T03:07:18.691919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332320+0000","last_change":"2023-11-28T03:07:18.327972+0000","last_active":"2023-11-28T03:07:18.332320+0000","last_peered":"2023-11-28T03:07:18.332320+0000","last_clean":"2023-11-28T03:07:18.332320+0000","last_became_active":"2023-11-28T03:07:18.327696+0000","last_became_peered":"2023-11-28T03:07:18.327696+0000","last_unstale":"2023-11-28T03:07:18.332320+0000","last_undegraded":"2023-11-28T03:07:18.332320+0000","last_fullsized":"2023-11-28T03:07:18.332320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.709544+0000","last_change":"2023-11-28T03:07:18.709624+0000","last_active":"2023-11-28T03:07:18.709544+0000","last_peered":"2023-11-28T03:07:18.709544+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.709328+0000","last_became_peered":"2023-11-28T03:07:18.709328+0000","last_unstale":"2023-11-28T03:07:18.709544+0000","last_undegraded":"2023-11-28T03:07:18.709544+0000","last_fullsized":"2023-11-28T03:07:18.709544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.683982+0000","last_change":"2023-11-28T03:07:18.334358+0000","last_active":"2023-11-28T03:07:18.683982+0000","last_peered":"2023-11-28T03:07:18.683982+0000","last_clean":"2023-11-28T03:07:18.683982+0000","last_became_active":"2023-11-28T03:07:18.333966+0000","last_became_peered":"2023-11-28T03:07:18.333966+0000","last_unstale":"2023-11-28T03:07:18.683982+0000","last_undegraded":"2023-11-28T03:07:18.683982+0000","last_fullsized":"2023-11-28T03:07:18.683982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.307970+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:07:18.307970+0000","last_peered":"2023-11-28T03:07:18.307970+0000","last_clean":"2023-11-28T03:07:18.307970+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:07:18.307970+0000","last_undegraded":"2023-11-28T03:07:18.307970+0000","last_fullsized":"2023-11-28T03:07:18.307970+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699939+0000","last_change":"2023-11-28T03:07:18.699939+0000","last_active":"2023-11-28T03:07:18.699939+0000","last_peered":"2023-11-28T03:07:18.699939+0000","last_clean":"2023-11-28T03:07:18.699939+0000","last_became_active":"2023-11-28T03:07:18.699583+0000","last_became_peered":"2023-11-28T03:07:18.699583+0000","last_unstale":"2023-11-28T03:07:18.699939+0000","last_undegraded":"2023-11-28T03:07:18.699939+0000","last_fullsized":"2023-11-28T03:07:18.699939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331175+0000","last_change":"2023-11-28T03:07:18.325091+0000","last_active":"2023-11-28T03:07:18.331175+0000","last_peered":"2023-11-28T03:07:18.331175+0000","last_clean":"2023-11-28T03:07:18.331175+0000","last_became_active":"2023-11-28T03:07:18.324845+0000","last_became_peered":"2023-11-28T03:07:18.324845+0000","last_unstale":"2023-11-28T03:07:18.331175+0000","last_undegraded":"2023-11-28T03:07:18.331175+0000","last_fullsized":"2023-11-28T03:07:18.331175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.325083+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:07:18.325083+0000","last_peered":"2023-11-28T03:07:18.325083+0000","last_clean":"2023-11-28T03:07:18.325083+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:07:18.325083+0000","last_undegraded":"2023-11-28T03:07:18.325083+0000","last_fullsized":"2023-11-28T03:07:18.325083+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690236+0000","last_change":"2023-11-28T03:07:18.690236+0000","last_active":"2023-11-28T03:07:18.690236+0000","last_peered":"2023-11-28T03:07:18.690236+0000","last_clean":"2023-11-28T03:07:18.690236+0000","last_became_active":"2023-11-28T03:07:18.689942+0000","last_became_peered":"2023-11-28T03:07:18.689942+0000","last_unstale":"2023-11-28T03:07:18.690236+0000","last_undegraded":"2023-11-28T03:07:18.690236+0000","last_fullsized":"2023-11-28T03:07:18.690236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333482+0000","last_change":"2023-11-28T03:07:18.331714+0000","last_active":"2023-11-28T03:07:18.333482+0000","last_peered":"2023-11-28T03:07:18.333482+0000","last_clean":"2023-11-28T03:07:18.333482+0000","last_became_active":"2023-11-28T03:07:18.331441+0000","last_became_peered":"2023-11-28T03:07:18.331441+0000","last_unstale":"2023-11-28T03:07:18.333482+0000","last_undegraded":"2023-11-28T03:07:18.333482+0000","last_fullsized":"2023-11-28T03:07:18.333482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.312386+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:07:18.312386+0000","last_peered":"2023-11-28T03:07:18.312386+0000","last_clean":"2023-11-28T03:07:18.312386+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:07:18.312386+0000","last_undegraded":"2023-11-28T03:07:18.312386+0000","last_fullsized":"2023-11-28T03:07:18.312386+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700764+0000","last_change":"2023-11-28T03:07:18.700764+0000","last_active":"2023-11-28T03:07:18.700764+0000","last_peered":"2023-11-28T03:07:18.700764+0000","last_clean":"2023-11-28T03:07:18.700764+0000","last_became_active":"2023-11-28T03:07:18.699307+0000","last_became_peered":"2023-11-28T03:07:18.699307+0000","last_unstale":"2023-11-28T03:07:18.700764+0000","last_undegraded":"2023-11-28T03:07:18.700764+0000","last_fullsized":"2023-11-28T03:07:18.700764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331778+0000","last_change":"2023-11-28T03:07:18.327278+0000","last_active":"2023-11-28T03:07:18.331778+0000","last_peered":"2023-11-28T03:07:18.331778+0000","last_clean":"2023-11-28T03:07:18.331778+0000","last_became_active":"2023-11-28T03:07:18.327009+0000","last_became_peered":"2023-11-28T03:07:18.327009+0000","last_unstale":"2023-11-28T03:07:18.331778+0000","last_undegraded":"2023-11-28T03:07:18.331778+0000","last_fullsized":"2023-11-28T03:07:18.331778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326448+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:07:18.326448+0000","last_peered":"2023-11-28T03:07:18.326448+0000","last_clean":"2023-11-28T03:07:18.326448+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:07:18.326448+0000","last_undegraded":"2023-11-28T03:07:18.326448+0000","last_fullsized":"2023-11-28T03:07:18.326448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.314862+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:07:18.314862+0000","last_peered":"2023-11-28T03:07:18.314862+0000","last_clean":"2023-11-28T03:07:18.314862+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:07:18.314862+0000","last_undegraded":"2023-11-28T03:07:18.314862+0000","last_fullsized":"2023-11-28T03:07:18.314862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697635+0000","last_change":"2023-11-28T03:07:18.697635+0000","last_active":"2023-11-28T03:07:18.697635+0000","last_peered":"2023-11-28T03:07:18.697635+0000","last_clean":"2023-11-28T03:07:18.697635+0000","last_became_active":"2023-11-28T03:07:18.697361+0000","last_became_peered":"2023-11-28T03:07:18.697361+0000","last_unstale":"2023-11-28T03:07:18.697635+0000","last_undegraded":"2023-11-28T03:07:18.697635+0000","last_fullsized":"2023-11-28T03:07:18.697635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690083+0000","last_change":"2023-11-28T03:07:18.331583+0000","last_active":"2023-11-28T03:07:18.690083+0000","last_peered":"2023-11-28T03:07:18.690083+0000","last_clean":"2023-11-28T03:07:18.690083+0000","last_became_active":"2023-11-28T03:07:18.331296+0000","last_became_peered":"2023-11-28T03:07:18.331296+0000","last_unstale":"2023-11-28T03:07:18.690083+0000","last_undegraded":"2023-11-28T03:07:18.690083+0000","last_fullsized":"2023-11-28T03:07:18.690083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.314950+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:07:18.314950+0000","last_peered":"2023-11-28T03:07:18.314950+0000","last_clean":"2023-11-28T03:07:18.314950+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:07:18.314950+0000","last_undegraded":"2023-11-28T03:07:18.314950+0000","last_fullsized":"2023-11-28T03:07:18.314950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708677+0000","last_change":"2023-11-28T03:07:18.708677+0000","last_active":"2023-11-28T03:07:18.708677+0000","last_peered":"2023-11-28T03:07:18.708677+0000","last_clean":"2023-11-28T03:07:18.708677+0000","last_became_active":"2023-11-28T03:07:18.708400+0000","last_became_peered":"2023-11-28T03:07:18.708400+0000","last_unstale":"2023-11-28T03:07:18.708677+0000","last_undegraded":"2023-11-28T03:07:18.708677+0000","last_fullsized":"2023-11-28T03:07:18.708677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.709316+0000","last_change":"2023-11-28T03:07:18.325748+0000","last_active":"2023-11-28T03:07:18.709316+0000","last_peered":"2023-11-28T03:07:18.709316+0000","last_clean":"2023-11-28T03:07:18.709316+0000","last_became_active":"2023-11-28T03:07:18.325468+0000","last_became_peered":"2023-11-28T03:07:18.325468+0000","last_unstale":"2023-11-28T03:07:18.709316+0000","last_undegraded":"2023-11-28T03:07:18.709316+0000","last_fullsized":"2023-11-28T03:07:18.709316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.318033+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:07:18.318033+0000","last_peered":"2023-11-28T03:07:18.318033+0000","last_clean":"2023-11-28T03:07:18.318033+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:07:18.318033+0000","last_undegraded":"2023-11-28T03:07:18.318033+0000","last_fullsized":"2023-11-28T03:07:18.318033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702226+0000","last_change":"2023-11-28T03:07:18.702226+0000","last_active":"2023-11-28T03:07:18.702226+0000","last_peered":"2023-11-28T03:07:18.702226+0000","last_clean":"2023-11-28T03:07:18.702226+0000","last_became_active":"2023-11-28T03:07:18.701911+0000","last_became_peered":"2023-11-28T03:07:18.701911+0000","last_unstale":"2023-11-28T03:07:18.702226+0000","last_undegraded":"2023-11-28T03:07:18.702226+0000","last_fullsized":"2023-11-28T03:07:18.702226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.335179+0000","last_change":"2023-11-28T03:07:18.333307+0000","last_active":"2023-11-28T03:07:18.335179+0000","last_peered":"2023-11-28T03:07:18.335179+0000","last_clean":"2023-11-28T03:07:18.335179+0000","last_became_active":"2023-11-28T03:07:18.332718+0000","last_became_peered":"2023-11-28T03:07:18.332718+0000","last_unstale":"2023-11-28T03:07:18.335179+0000","last_undegraded":"2023-11-28T03:07:18.335179+0000","last_fullsized":"2023-11-28T03:07:18.335179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.308170+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:07:18.308170+0000","last_peered":"2023-11-28T03:07:18.308170+0000","last_clean":"2023-11-28T03:07:18.308170+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:07:18.308170+0000","last_undegraded":"2023-11-28T03:07:18.308170+0000","last_fullsized":"2023-11-28T03:07:18.308170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701240+0000","last_change":"2023-11-28T03:07:18.701240+0000","last_active":"2023-11-28T03:07:18.701240+0000","last_peered":"2023-11-28T03:07:18.701240+0000","last_clean":"2023-11-28T03:07:18.701240+0000","last_became_active":"2023-11-28T03:07:18.700933+0000","last_became_peered":"2023-11-28T03:07:18.700933+0000","last_unstale":"2023-11-28T03:07:18.701240+0000","last_undegraded":"2023-11-28T03:07:18.701240+0000","last_fullsized":"2023-11-28T03:07:18.701240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.323371+0000","last_change":"2023-11-28T03:07:18.320644+0000","last_active":"2023-11-28T03:07:18.323371+0000","last_peered":"2023-11-28T03:07:18.323371+0000","last_clean":"2023-11-28T03:07:18.323371+0000","last_became_active":"2023-11-28T03:07:18.320327+0000","last_became_peered":"2023-11-28T03:07:18.320327+0000","last_unstale":"2023-11-28T03:07:18.323371+0000","last_undegraded":"2023-11-28T03:07:18.323371+0000","last_fullsized":"2023-11-28T03:07:18.323371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.312253+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:07:18.312253+0000","last_peered":"2023-11-28T03:07:18.312253+0000","last_clean":"2023-11-28T03:07:18.312253+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:07:18.312253+0000","last_undegraded":"2023-11-28T03:07:18.312253+0000","last_fullsized":"2023-11-28T03:07:18.312253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575025,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.375}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.73899999999999999,"5min":0.749,"15min":0.749},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.77100000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.66100000000000003,"5min":0.748,"15min":0.81899999999999995},"last":0.34899999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.755,"5min":0.755,"15min":0.84499999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.83199999999999996},"last":0.34100000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.86499999999999999},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834575025,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":1.125,"5min":1.506,"15min":1.506},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.53000000000000003,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.442},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67800000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.82499999999999996},"last":0.45700000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.755,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.434}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.40500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34399999999999997,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.318},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.28000000000000003,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46000000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.19800000000000001},"max":{"1min":0.56799999999999995,"5min":0.628,"15min":0.72599999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.69599999999999995,"15min":0.749},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.66500000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":1.038,"15min":1.111},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.374},"max":{"1min":0.76000000000000001,"5min":1.024,"15min":1.024},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575025,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.221,"15min":0.184},"max":{"1min":0.45100000000000001,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.51700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36499999999999999,"15min":0.36899999999999999},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.505,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.372,"5min":0.307,"15min":0.26300000000000001},"max":{"1min":0.57599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.26200000000000001,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.78000000000000003,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.77600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":1.304,"15min":1.304},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39000000000000001},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":1.679,"15min":1.679},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.36599999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.435},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.046,"15min":1.046},"last":0.45000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.27600000000000002,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.623},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.36299999999999999},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.27900000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.47899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.874},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.439,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.004,"15min":1.004},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.628,"15min":0.628},"min":{"1min":0.55200000000000005,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.28499999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35599999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.373},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.32900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32400000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41899999999999998},"min":{"1min":0.32400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.438,"15min":0.435},"min":{"1min":0.36799999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85799999999999998,"5min":1.581,"15min":1.6000000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.94199999999999995,"15min":1.196},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.63700000000000001},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.95799999999999996,"15min":1.206},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.65100000000000002},"min":{"1min":0.51500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.45100000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.78100000000000003},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.38500000000000001},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.878},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.77900000000000003},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83899999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.91300000000000003},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302156,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069684,"statfs":{"total":96636764160,"available":96327356416,"internally_reserved":0,"allocated":987136,"data_stored":485221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.79800000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42699999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.621,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.62,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:07:21.666 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:07:21.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.812+0000 7f545f305700 1 Processor -- start 2023-11-28T03:07:21.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.813+0000 7f545f305700 1 -- start start 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.813+0000 7f545f305700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54580caab0 0x7f54580aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.813+0000 7f545f305700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54580a9a90 con 0x7f54580cae90 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.813+0000 7f545f305700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5458154ce0 con 0x7f54580ca180 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.813+0000 7f545f305700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54580a8e80 con 0x7f54580caab0 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545c8a0700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f54580cae90 0x7f54580c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44476/0 (socket says 172.21.15.73:44476) 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545c8a0700 1 -- 172.21.15.73:0/693900740 learned_addr learned my addr 172.21.15.73:0/693900740 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330581826 0 0) 0x7f54580a9a90 con 0x7f54580cae90 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54440033f0 con 0x7f54580cae90 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> v2:172.21.15.150:3300/0 conn(0x7f54580caab0 0x7f54580aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 530234042 0 0) 0x7f54440033f0 con 0x7f54580cae90 2023-11-28T03:07:21.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 >> v2:172.21.15.150:3300/0 conn(0x7f54580caab0 msgr2=0x7f54580aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 --2- 172.21.15.73:0/693900740 >> v2:172.21.15.150:3300/0 conn(0x7f54580caab0 0x7f54580aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 >> v1:172.21.15.150:6789/0 conn(0x7f54580ca180 legacy=0x7f545815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54580a9160 con 0x7f54580cae90 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f5448001470 con 0x7f54580cae90 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> v2:172.21.15.150:3300/0 conn(0x7f54580caab0 0x7f54580aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 >> v1:172.21.15.73:6789/0 conn(0x7f54580cae90 legacy=0x7f54580c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5444003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5444004690 0x7f5444004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5444005050 0x7f544400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580a8e80 con 0x7f54440038e0 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f544400bfe0 con 0x7f5444005050 2023-11-28T03:07:21.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f544400c1a0 con 0x7f5444004690 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5444003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545c8a0700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5444004690 0x7f5444004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d8a2700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5444005050 0x7f544400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 -- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5444004690 msgr2=0x7f5444004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5444004690 0x7f5444004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 -- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5444005050 msgr2=0x7f544400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5444005050 0x7f544400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d8a2700 1 -- 172.21.15.73:0/693900740 reap_dead start 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.814+0000 7f545d0a1700 1 -- 172.21.15.73:0/693900740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54440033f0 con 0x7f54440038e0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545d0a1700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5444003d00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f5454007530 tx=0x7f5454009820).ready entity=mon.0 client_cookie=ad8d1d06e0a7b83c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454009b30 con 0x7f54440038e0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5454007d80 con 0x7f54440038e0 2023-11-28T03:07:21.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f544ffff700 1 -- 172.21.15.73:0/693900740 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454015660 con 0x7f54440038e0 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 -- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 msgr2=0x7f5444003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5444003d00 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 rx=0x7f5454007530 tx=0x7f5454009820).stop 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 -- 172.21.15.73:0/693900740 shutdown_connections 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 --2- 172.21.15.73:0/693900740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5444003d00 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 -- 172.21.15.73:0/693900740 >> 172.21.15.73:0/693900740 conn(0x7f5458094920 msgr2=0x7f545806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 -- 172.21.15.73:0/693900740 shutdown_connections 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.815+0000 7f545f305700 1 -- 172.21.15.73:0/693900740 wait complete. 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.816+0000 7f545f305700 1 Processor -- start 2023-11-28T03:07:21.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.816+0000 7f545f305700 1 -- start start 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 0x7f54580acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 0x7f5458153c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54580aa3c0 con 0x7f54440038e0 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5458165480 con 0x7f5458164bc0 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545f305700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5458165760 con 0x7f54580aa850 2023-11-28T03:07:21.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545d0a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545d0a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41830/0 (socket says 172.21.15.73:41830) 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.817+0000 7f545d0a1700 1 -- 172.21.15.73:0/1471971303 learned_addr learned my addr 172.21.15.73:0/1471971303 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545c8a0700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 0x7f54580acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d8a2700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 0x7f5458153c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 -- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 msgr2=0x7f5458153c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 0x7f5458153c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 -- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 msgr2=0x7f54580acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 0x7f54580acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 -- 172.21.15.73:0/1471971303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5458165a40 con 0x7f54440038e0 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545c8a0700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 0x7f54580acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d8a2700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 0x7f5458153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:21.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545d0a1700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5454007b20 tx=0x7f5454018000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454014480 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54540145e0 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54580aa050 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5454014960 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.818+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5458165040 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.819+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54580aa6a0 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.819+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5454003040 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.819+0000 7f544d7fa700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 0x7f543c061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.820+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f545407b9f0 con 0x7f54440038e0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.820+0000 7f545c8a0700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 0x7f543c061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.820+0000 7f545c8a0700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 0x7f543c061640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f54480010e0 tx=0x7f5448000f60).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:21.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.824+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f545404cd70 con 0x7f54440038e0 2023-11-28T03:07:21.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.983+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5458165260 con 0x7f543c05f180 2023-11-28T03:07:21.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.992+0000 7f544d7fa700 1 -- 172.21.15.73:0/1471971303 <== mgr.6060 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361391 (secure 0 0 0) 0x7f5448005e90 con 0x7f543c05f180 2023-11-28T03:07:21.999 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 msgr2=0x7f543c061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 0x7f543c061640 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f54480010e0 tx=0x7f5448000f60).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 msgr2=0x7f5458157be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 rx=0x7f5454007b20 tx=0x7f5454018000).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 shutdown_connections 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f543c05f180 0x7f543c061640 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54580aa850 0x7f5458153c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54440038e0 0x7f5458157be0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 --2- 172.21.15.73:0/1471971303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5458164bc0 0x7f54580acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 >> 172.21.15.73:0/1471971303 conn(0x7f5458094920 msgr2=0x7f54580cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 shutdown_connections 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:21.996+0000 7f545f305700 1 -- 172.21.15.73:0/1471971303 wait complete. 2023-11-28T03:07:22.003 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:07:22.009 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:07:21.717053+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":68,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2412964,"kb_used_data":7524,"kb_used_omap":0,"kb_used_meta":2405376,"kb_avail":752561756,"statfs":{"total":773094113280,"available":770623238144,"internally_reserved":0,"allocated":7704576,"data_stored":3798401,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2463105024},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27602,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-79,"num_write_kb":-64,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.598826"},"pg_stats":[{"pgid":"10.3c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706878+0000","last_change":"2023-11-28T03:07:18.706878+0000","last_active":"2023-11-28T03:07:18.706878+0000","last_peered":"2023-11-28T03:07:18.706878+0000","last_clean":"2023-11-28T03:07:18.706878+0000","last_became_active":"2023-11-28T03:07:18.706607+0000","last_became_peered":"2023-11-28T03:07:18.706607+0000","last_unstale":"2023-11-28T03:07:18.706878+0000","last_undegraded":"2023-11-28T03:07:18.706878+0000","last_fullsized":"2023-11-28T03:07:18.706878+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.3f","version":"81'3","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690764+0000","last_change":"2023-11-28T03:07:18.321698+0000","last_active":"2023-11-28T03:07:18.690764+0000","last_peered":"2023-11-28T03:07:18.690764+0000","last_clean":"2023-11-28T03:07:18.690764+0000","last_became_active":"2023-11-28T03:07:18.321444+0000","last_became_peered":"2023-11-28T03:07:18.321444+0000","last_unstale":"2023-11-28T03:07:18.690764+0000","last_undegraded":"2023-11-28T03:07:18.690764+0000","last_fullsized":"2023-11-28T03:07:18.690764+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702578+0000","last_change":"2023-11-28T03:07:18.702578+0000","last_active":"2023-11-28T03:07:18.702578+0000","last_peered":"2023-11-28T03:07:18.702578+0000","last_clean":"2023-11-28T03:07:18.702578+0000","last_became_active":"2023-11-28T03:07:18.702292+0000","last_became_peered":"2023-11-28T03:07:18.702292+0000","last_unstale":"2023-11-28T03:07:18.702578+0000","last_undegraded":"2023-11-28T03:07:18.702578+0000","last_fullsized":"2023-11-28T03:07:18.702578+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.336556+0000","last_change":"2023-11-28T03:07:18.330522+0000","last_active":"2023-11-28T03:07:18.336556+0000","last_peered":"2023-11-28T03:07:18.336556+0000","last_clean":"2023-11-28T03:07:18.336556+0000","last_became_active":"2023-11-28T03:07:18.330137+0000","last_became_peered":"2023-11-28T03:07:18.330137+0000","last_unstale":"2023-11-28T03:07:18.336556+0000","last_undegraded":"2023-11-28T03:07:18.336556+0000","last_fullsized":"2023-11-28T03:07:18.336556+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696512+0000","last_change":"2023-11-28T03:07:18.696512+0000","last_active":"2023-11-28T03:07:18.696512+0000","last_peered":"2023-11-28T03:07:18.696512+0000","last_clean":"2023-11-28T03:07:18.696512+0000","last_became_active":"2023-11-28T03:07:18.696189+0000","last_became_peered":"2023-11-28T03:07:18.696189+0000","last_unstale":"2023-11-28T03:07:18.696512+0000","last_undegraded":"2023-11-28T03:07:18.696512+0000","last_fullsized":"2023-11-28T03:07:18.696512+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333645+0000","last_change":"2023-11-28T03:07:18.328975+0000","last_active":"2023-11-28T03:07:18.333645+0000","last_peered":"2023-11-28T03:07:18.333645+0000","last_clean":"2023-11-28T03:07:18.333645+0000","last_became_active":"2023-11-28T03:07:18.328524+0000","last_became_peered":"2023-11-28T03:07:18.328524+0000","last_unstale":"2023-11-28T03:07:18.333645+0000","last_undegraded":"2023-11-28T03:07:18.333645+0000","last_fullsized":"2023-11-28T03:07:18.333645+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.3f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697781+0000","last_change":"2023-11-28T03:07:18.697781+0000","last_active":"2023-11-28T03:07:18.697781+0000","last_peered":"2023-11-28T03:07:18.697781+0000","last_clean":"2023-11-28T03:07:18.697781+0000","last_became_active":"2023-11-28T03:07:18.697510+0000","last_became_peered":"2023-11-28T03:07:18.697510+0000","last_unstale":"2023-11-28T03:07:18.697781+0000","last_undegraded":"2023-11-28T03:07:18.697781+0000","last_fullsized":"2023-11-28T03:07:18.697781+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3c","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.691749+0000","last_change":"2023-11-28T03:07:18.327244+0000","last_active":"2023-11-28T03:07:18.691749+0000","last_peered":"2023-11-28T03:07:18.691749+0000","last_clean":"2023-11-28T03:07:18.691749+0000","last_became_active":"2023-11-28T03:07:18.326843+0000","last_became_peered":"2023-11-28T03:07:18.326843+0000","last_unstale":"2023-11-28T03:07:18.691749+0000","last_undegraded":"2023-11-28T03:07:18.691749+0000","last_fullsized":"2023-11-28T03:07:18.691749+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.38","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699615+0000","last_change":"2023-11-28T03:07:18.699615+0000","last_active":"2023-11-28T03:07:18.699615+0000","last_peered":"2023-11-28T03:07:18.699615+0000","last_clean":"2023-11-28T03:07:18.699615+0000","last_became_active":"2023-11-28T03:07:18.699328+0000","last_became_peered":"2023-11-28T03:07:18.699328+0000","last_unstale":"2023-11-28T03:07:18.699615+0000","last_undegraded":"2023-11-28T03:07:18.699615+0000","last_fullsized":"2023-11-28T03:07:18.699615+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.679721+0000","last_change":"2023-11-28T03:07:18.322709+0000","last_active":"2023-11-28T03:07:18.679721+0000","last_peered":"2023-11-28T03:07:18.679721+0000","last_clean":"2023-11-28T03:07:18.679721+0000","last_became_active":"2023-11-28T03:07:18.322454+0000","last_became_peered":"2023-11-28T03:07:18.322454+0000","last_unstale":"2023-11-28T03:07:18.679721+0000","last_undegraded":"2023-11-28T03:07:18.679721+0000","last_fullsized":"2023-11-28T03:07:18.679721+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.39","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702406+0000","last_change":"2023-11-28T03:07:18.702406+0000","last_active":"2023-11-28T03:07:18.702406+0000","last_peered":"2023-11-28T03:07:18.702406+0000","last_clean":"2023-11-28T03:07:18.702406+0000","last_became_active":"2023-11-28T03:07:18.702134+0000","last_became_peered":"2023-11-28T03:07:18.702134+0000","last_unstale":"2023-11-28T03:07:18.702406+0000","last_undegraded":"2023-11-28T03:07:18.702406+0000","last_fullsized":"2023-11-28T03:07:18.702406+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.687789+0000","last_change":"2023-11-28T03:07:18.325296+0000","last_active":"2023-11-28T03:07:18.687789+0000","last_peered":"2023-11-28T03:07:18.687789+0000","last_clean":"2023-11-28T03:07:18.687789+0000","last_became_active":"2023-11-28T03:07:18.325013+0000","last_became_peered":"2023-11-28T03:07:18.325013+0000","last_unstale":"2023-11-28T03:07:18.687789+0000","last_undegraded":"2023-11-28T03:07:18.687789+0000","last_fullsized":"2023-11-28T03:07:18.687789+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.3a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704977+0000","last_change":"2023-11-28T03:07:18.704977+0000","last_active":"2023-11-28T03:07:18.704977+0000","last_peered":"2023-11-28T03:07:18.704977+0000","last_clean":"2023-11-28T03:07:18.704977+0000","last_became_active":"2023-11-28T03:07:18.704718+0000","last_became_peered":"2023-11-28T03:07:18.704718+0000","last_unstale":"2023-11-28T03:07:18.704977+0000","last_undegraded":"2023-11-28T03:07:18.704977+0000","last_fullsized":"2023-11-28T03:07:18.704977+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330820+0000","last_change":"2023-11-28T03:07:18.323993+0000","last_active":"2023-11-28T03:07:18.330820+0000","last_peered":"2023-11-28T03:07:18.330820+0000","last_clean":"2023-11-28T03:07:18.330820+0000","last_became_active":"2023-11-28T03:07:18.323657+0000","last_became_peered":"2023-11-28T03:07:18.323657+0000","last_unstale":"2023-11-28T03:07:18.330820+0000","last_undegraded":"2023-11-28T03:07:18.330820+0000","last_fullsized":"2023-11-28T03:07:18.330820+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.3b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708719+0000","last_change":"2023-11-28T03:07:18.708719+0000","last_active":"2023-11-28T03:07:18.708719+0000","last_peered":"2023-11-28T03:07:18.708719+0000","last_clean":"2023-11-28T03:07:18.708719+0000","last_became_active":"2023-11-28T03:07:18.708422+0000","last_became_peered":"2023-11-28T03:07:18.708422+0000","last_unstale":"2023-11-28T03:07:18.708719+0000","last_undegraded":"2023-11-28T03:07:18.708719+0000","last_fullsized":"2023-11-28T03:07:18.708719+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.321047+0000","last_change":"2023-11-28T03:07:18.318086+0000","last_active":"2023-11-28T03:07:18.321047+0000","last_peered":"2023-11-28T03:07:18.321047+0000","last_clean":"2023-11-28T03:07:18.321047+0000","last_became_active":"2023-11-28T03:07:18.317831+0000","last_became_peered":"2023-11-28T03:07:18.317831+0000","last_unstale":"2023-11-28T03:07:18.321047+0000","last_undegraded":"2023-11-28T03:07:18.321047+0000","last_fullsized":"2023-11-28T03:07:18.321047+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.34","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.703450+0000","last_change":"2023-11-28T03:07:18.703450+0000","last_active":"2023-11-28T03:07:18.703450+0000","last_peered":"2023-11-28T03:07:18.703450+0000","last_clean":"2023-11-28T03:07:18.703450+0000","last_became_active":"2023-11-28T03:07:18.703111+0000","last_became_peered":"2023-11-28T03:07:18.703111+0000","last_unstale":"2023-11-28T03:07:18.703450+0000","last_undegraded":"2023-11-28T03:07:18.703450+0000","last_fullsized":"2023-11-28T03:07:18.703450+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331034+0000","last_change":"2023-11-28T03:07:18.326463+0000","last_active":"2023-11-28T03:07:18.331034+0000","last_peered":"2023-11-28T03:07:18.331034+0000","last_clean":"2023-11-28T03:07:18.331034+0000","last_became_active":"2023-11-28T03:07:18.326206+0000","last_became_peered":"2023-11-28T03:07:18.326206+0000","last_unstale":"2023-11-28T03:07:18.331034+0000","last_undegraded":"2023-11-28T03:07:18.331034+0000","last_fullsized":"2023-11-28T03:07:18.331034+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.35","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705851+0000","last_change":"2023-11-28T03:07:18.705851+0000","last_active":"2023-11-28T03:07:18.705851+0000","last_peered":"2023-11-28T03:07:18.705851+0000","last_clean":"2023-11-28T03:07:18.705851+0000","last_became_active":"2023-11-28T03:07:18.705134+0000","last_became_peered":"2023-11-28T03:07:18.705134+0000","last_unstale":"2023-11-28T03:07:18.705851+0000","last_undegraded":"2023-11-28T03:07:18.705851+0000","last_fullsized":"2023-11-28T03:07:18.705851+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332498+0000","last_change":"2023-11-28T03:07:18.329346+0000","last_active":"2023-11-28T03:07:18.332498+0000","last_peered":"2023-11-28T03:07:18.332498+0000","last_clean":"2023-11-28T03:07:18.332498+0000","last_became_active":"2023-11-28T03:07:18.329075+0000","last_became_peered":"2023-11-28T03:07:18.329075+0000","last_unstale":"2023-11-28T03:07:18.332498+0000","last_undegraded":"2023-11-28T03:07:18.332498+0000","last_fullsized":"2023-11-28T03:07:18.332498+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.36","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700276+0000","last_change":"2023-11-28T03:07:18.700276+0000","last_active":"2023-11-28T03:07:18.700276+0000","last_peered":"2023-11-28T03:07:18.700276+0000","last_clean":"2023-11-28T03:07:18.700276+0000","last_became_active":"2023-11-28T03:07:18.699951+0000","last_became_peered":"2023-11-28T03:07:18.699951+0000","last_unstale":"2023-11-28T03:07:18.700276+0000","last_undegraded":"2023-11-28T03:07:18.700276+0000","last_fullsized":"2023-11-28T03:07:18.700276+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331679+0000","last_change":"2023-11-28T03:07:18.321892+0000","last_active":"2023-11-28T03:07:18.331679+0000","last_peered":"2023-11-28T03:07:18.331679+0000","last_clean":"2023-11-28T03:07:18.331679+0000","last_became_active":"2023-11-28T03:07:18.321640+0000","last_became_peered":"2023-11-28T03:07:18.321640+0000","last_unstale":"2023-11-28T03:07:18.331679+0000","last_undegraded":"2023-11-28T03:07:18.331679+0000","last_fullsized":"2023-11-28T03:07:18.331679+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.37","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701508+0000","last_change":"2023-11-28T03:07:18.701508+0000","last_active":"2023-11-28T03:07:18.701508+0000","last_peered":"2023-11-28T03:07:18.701508+0000","last_clean":"2023-11-28T03:07:18.701508+0000","last_became_active":"2023-11-28T03:07:18.701220+0000","last_became_peered":"2023-11-28T03:07:18.701220+0000","last_unstale":"2023-11-28T03:07:18.701508+0000","last_undegraded":"2023-11-28T03:07:18.701508+0000","last_fullsized":"2023-11-28T03:07:18.701508+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.34","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695480+0000","last_change":"2023-11-28T03:07:18.325208+0000","last_active":"2023-11-28T03:07:18.695480+0000","last_peered":"2023-11-28T03:07:18.695480+0000","last_clean":"2023-11-28T03:07:18.695480+0000","last_became_active":"2023-11-28T03:07:18.324734+0000","last_became_peered":"2023-11-28T03:07:18.324734+0000","last_unstale":"2023-11-28T03:07:18.695480+0000","last_undegraded":"2023-11-28T03:07:18.695480+0000","last_fullsized":"2023-11-28T03:07:18.695480+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.30","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692251+0000","last_change":"2023-11-28T03:07:18.692251+0000","last_active":"2023-11-28T03:07:18.692251+0000","last_peered":"2023-11-28T03:07:18.692251+0000","last_clean":"2023-11-28T03:07:18.692251+0000","last_became_active":"2023-11-28T03:07:18.692031+0000","last_became_peered":"2023-11-28T03:07:18.692031+0000","last_unstale":"2023-11-28T03:07:18.692251+0000","last_undegraded":"2023-11-28T03:07:18.692251+0000","last_fullsized":"2023-11-28T03:07:18.692251+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.33","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.688653+0000","last_change":"2023-11-28T03:07:18.327445+0000","last_active":"2023-11-28T03:07:18.688653+0000","last_peered":"2023-11-28T03:07:18.688653+0000","last_clean":"2023-11-28T03:07:18.688653+0000","last_became_active":"2023-11-28T03:07:18.327162+0000","last_became_peered":"2023-11-28T03:07:18.327162+0000","last_unstale":"2023-11-28T03:07:18.688653+0000","last_undegraded":"2023-11-28T03:07:18.688653+0000","last_fullsized":"2023-11-28T03:07:18.688653+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.31","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701652+0000","last_change":"2023-11-28T03:07:18.701652+0000","last_active":"2023-11-28T03:07:18.701652+0000","last_peered":"2023-11-28T03:07:18.701652+0000","last_clean":"2023-11-28T03:07:18.701652+0000","last_became_active":"2023-11-28T03:07:18.701357+0000","last_became_peered":"2023-11-28T03:07:18.701357+0000","last_unstale":"2023-11-28T03:07:18.701652+0000","last_undegraded":"2023-11-28T03:07:18.701652+0000","last_fullsized":"2023-11-28T03:07:18.701652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.316599+0000","last_change":"2023-11-28T03:07:18.314740+0000","last_active":"2023-11-28T03:07:18.316599+0000","last_peered":"2023-11-28T03:07:18.316599+0000","last_clean":"2023-11-28T03:07:18.316599+0000","last_became_active":"2023-11-28T03:07:18.314419+0000","last_became_peered":"2023-11-28T03:07:18.314419+0000","last_unstale":"2023-11-28T03:07:18.316599+0000","last_undegraded":"2023-11-28T03:07:18.316599+0000","last_fullsized":"2023-11-28T03:07:18.316599+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.32","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702794+0000","last_change":"2023-11-28T03:07:18.702794+0000","last_active":"2023-11-28T03:07:18.702794+0000","last_peered":"2023-11-28T03:07:18.702794+0000","last_clean":"2023-11-28T03:07:18.702794+0000","last_became_active":"2023-11-28T03:07:18.702509+0000","last_became_peered":"2023-11-28T03:07:18.702509+0000","last_unstale":"2023-11-28T03:07:18.702794+0000","last_undegraded":"2023-11-28T03:07:18.702794+0000","last_fullsized":"2023-11-28T03:07:18.702794+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330794+0000","last_change":"2023-11-28T03:07:18.321976+0000","last_active":"2023-11-28T03:07:18.330794+0000","last_peered":"2023-11-28T03:07:18.330794+0000","last_clean":"2023-11-28T03:07:18.330794+0000","last_became_active":"2023-11-28T03:07:18.321650+0000","last_became_peered":"2023-11-28T03:07:18.321650+0000","last_unstale":"2023-11-28T03:07:18.330794+0000","last_undegraded":"2023-11-28T03:07:18.330794+0000","last_fullsized":"2023-11-28T03:07:18.330794+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.33","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707791+0000","last_change":"2023-11-28T03:07:18.708510+0000","last_active":"2023-11-28T03:07:18.707791+0000","last_peered":"2023-11-28T03:07:18.707791+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.707570+0000","last_became_peered":"2023-11-28T03:07:18.707570+0000","last_unstale":"2023-11-28T03:07:18.707791+0000","last_undegraded":"2023-11-28T03:07:18.707791+0000","last_fullsized":"2023-11-28T03:07:18.707791+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.30","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697252+0000","last_change":"2023-11-28T03:07:18.331676+0000","last_active":"2023-11-28T03:07:18.697252+0000","last_peered":"2023-11-28T03:07:18.697252+0000","last_clean":"2023-11-28T03:07:18.697252+0000","last_became_active":"2023-11-28T03:07:18.331391+0000","last_became_peered":"2023-11-28T03:07:18.331391+0000","last_unstale":"2023-11-28T03:07:18.697252+0000","last_undegraded":"2023-11-28T03:07:18.697252+0000","last_fullsized":"2023-11-28T03:07:18.697252+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.2c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698612+0000","last_change":"2023-11-28T03:07:18.698612+0000","last_active":"2023-11-28T03:07:18.698612+0000","last_peered":"2023-11-28T03:07:18.698612+0000","last_clean":"2023-11-28T03:07:18.698612+0000","last_became_active":"2023-11-28T03:07:18.697282+0000","last_became_peered":"2023-11-28T03:07:18.697282+0000","last_unstale":"2023-11-28T03:07:18.698612+0000","last_undegraded":"2023-11-28T03:07:18.698612+0000","last_fullsized":"2023-11-28T03:07:18.698612+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334386+0000","last_change":"2023-11-28T03:07:18.330147+0000","last_active":"2023-11-28T03:07:18.334386+0000","last_peered":"2023-11-28T03:07:18.334386+0000","last_clean":"2023-11-28T03:07:18.334386+0000","last_became_active":"2023-11-28T03:07:18.329899+0000","last_became_peered":"2023-11-28T03:07:18.329899+0000","last_unstale":"2023-11-28T03:07:18.334386+0000","last_undegraded":"2023-11-28T03:07:18.334386+0000","last_fullsized":"2023-11-28T03:07:18.334386+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697252+0000","last_change":"2023-11-28T03:07:18.697252+0000","last_active":"2023-11-28T03:07:18.697252+0000","last_peered":"2023-11-28T03:07:18.697252+0000","last_clean":"2023-11-28T03:07:18.697252+0000","last_became_active":"2023-11-28T03:07:18.696853+0000","last_became_peered":"2023-11-28T03:07:18.696853+0000","last_unstale":"2023-11-28T03:07:18.697252+0000","last_undegraded":"2023-11-28T03:07:18.697252+0000","last_fullsized":"2023-11-28T03:07:18.697252+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333250+0000","last_change":"2023-11-28T03:07:18.327992+0000","last_active":"2023-11-28T03:07:18.333250+0000","last_peered":"2023-11-28T03:07:18.333250+0000","last_clean":"2023-11-28T03:07:18.333250+0000","last_became_active":"2023-11-28T03:07:18.327663+0000","last_became_peered":"2023-11-28T03:07:18.327663+0000","last_unstale":"2023-11-28T03:07:18.333250+0000","last_undegraded":"2023-11-28T03:07:18.333250+0000","last_fullsized":"2023-11-28T03:07:18.333250+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705562+0000","last_change":"2023-11-28T03:07:18.705562+0000","last_active":"2023-11-28T03:07:18.705562+0000","last_peered":"2023-11-28T03:07:18.705562+0000","last_clean":"2023-11-28T03:07:18.705562+0000","last_became_active":"2023-11-28T03:07:18.705247+0000","last_became_peered":"2023-11-28T03:07:18.705247+0000","last_unstale":"2023-11-28T03:07:18.705562+0000","last_undegraded":"2023-11-28T03:07:18.705562+0000","last_fullsized":"2023-11-28T03:07:18.705562+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2d","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.680341+0000","last_change":"2023-11-28T03:07:18.324630+0000","last_active":"2023-11-28T03:07:18.680341+0000","last_peered":"2023-11-28T03:07:18.680341+0000","last_clean":"2023-11-28T03:07:18.680341+0000","last_became_active":"2023-11-28T03:07:18.324353+0000","last_became_peered":"2023-11-28T03:07:18.324353+0000","last_unstale":"2023-11-28T03:07:18.680341+0000","last_undegraded":"2023-11-28T03:07:18.680341+0000","last_fullsized":"2023-11-28T03:07:18.680341+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.2f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707767+0000","last_change":"2023-11-28T03:07:18.707767+0000","last_active":"2023-11-28T03:07:18.707767+0000","last_peered":"2023-11-28T03:07:18.707767+0000","last_clean":"2023-11-28T03:07:18.707767+0000","last_became_active":"2023-11-28T03:07:18.707486+0000","last_became_peered":"2023-11-28T03:07:18.707486+0000","last_unstale":"2023-11-28T03:07:18.707767+0000","last_undegraded":"2023-11-28T03:07:18.707767+0000","last_fullsized":"2023-11-28T03:07:18.707767+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330657+0000","last_change":"2023-11-28T03:07:18.325875+0000","last_active":"2023-11-28T03:07:18.330657+0000","last_peered":"2023-11-28T03:07:18.330657+0000","last_clean":"2023-11-28T03:07:18.330657+0000","last_became_active":"2023-11-28T03:07:18.325614+0000","last_became_peered":"2023-11-28T03:07:18.325614+0000","last_unstale":"2023-11-28T03:07:18.330657+0000","last_undegraded":"2023-11-28T03:07:18.330657+0000","last_fullsized":"2023-11-28T03:07:18.330657+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.28","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700255+0000","last_change":"2023-11-28T03:07:18.700255+0000","last_active":"2023-11-28T03:07:18.700255+0000","last_peered":"2023-11-28T03:07:18.700255+0000","last_clean":"2023-11-28T03:07:18.700255+0000","last_became_active":"2023-11-28T03:07:18.699890+0000","last_became_peered":"2023-11-28T03:07:18.699890+0000","last_unstale":"2023-11-28T03:07:18.700255+0000","last_undegraded":"2023-11-28T03:07:18.700255+0000","last_fullsized":"2023-11-28T03:07:18.700255+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.686378+0000","last_change":"2023-11-28T03:07:18.334497+0000","last_active":"2023-11-28T03:07:18.686378+0000","last_peered":"2023-11-28T03:07:18.686378+0000","last_clean":"2023-11-28T03:07:18.686378+0000","last_became_active":"2023-11-28T03:07:18.334242+0000","last_became_peered":"2023-11-28T03:07:18.334242+0000","last_unstale":"2023-11-28T03:07:18.686378+0000","last_undegraded":"2023-11-28T03:07:18.686378+0000","last_fullsized":"2023-11-28T03:07:18.686378+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.29","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702339+0000","last_change":"2023-11-28T03:07:18.702339+0000","last_active":"2023-11-28T03:07:18.702339+0000","last_peered":"2023-11-28T03:07:18.702339+0000","last_clean":"2023-11-28T03:07:18.702339+0000","last_became_active":"2023-11-28T03:07:18.702049+0000","last_became_peered":"2023-11-28T03:07:18.702049+0000","last_unstale":"2023-11-28T03:07:18.702339+0000","last_undegraded":"2023-11-28T03:07:18.702339+0000","last_fullsized":"2023-11-28T03:07:18.702339+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326549+0000","last_change":"2023-11-28T03:07:18.321229+0000","last_active":"2023-11-28T03:07:18.326549+0000","last_peered":"2023-11-28T03:07:18.326549+0000","last_clean":"2023-11-28T03:07:18.326549+0000","last_became_active":"2023-11-28T03:07:18.320950+0000","last_became_peered":"2023-11-28T03:07:18.320950+0000","last_unstale":"2023-11-28T03:07:18.326549+0000","last_undegraded":"2023-11-28T03:07:18.326549+0000","last_fullsized":"2023-11-28T03:07:18.326549+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702984+0000","last_change":"2023-11-28T03:07:18.702984+0000","last_active":"2023-11-28T03:07:18.702984+0000","last_peered":"2023-11-28T03:07:18.702984+0000","last_clean":"2023-11-28T03:07:18.702984+0000","last_became_active":"2023-11-28T03:07:18.702707+0000","last_became_peered":"2023-11-28T03:07:18.702707+0000","last_unstale":"2023-11-28T03:07:18.702984+0000","last_undegraded":"2023-11-28T03:07:18.702984+0000","last_fullsized":"2023-11-28T03:07:18.702984+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331259+0000","last_change":"2023-11-28T03:07:18.324721+0000","last_active":"2023-11-28T03:07:18.331259+0000","last_peered":"2023-11-28T03:07:18.331259+0000","last_clean":"2023-11-28T03:07:18.331259+0000","last_became_active":"2023-11-28T03:07:18.324453+0000","last_became_peered":"2023-11-28T03:07:18.324453+0000","last_unstale":"2023-11-28T03:07:18.331259+0000","last_undegraded":"2023-11-28T03:07:18.331259+0000","last_fullsized":"2023-11-28T03:07:18.331259+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.2b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695305+0000","last_change":"2023-11-28T03:07:18.695305+0000","last_active":"2023-11-28T03:07:18.695305+0000","last_peered":"2023-11-28T03:07:18.695305+0000","last_clean":"2023-11-28T03:07:18.695305+0000","last_became_active":"2023-11-28T03:07:18.694771+0000","last_became_peered":"2023-11-28T03:07:18.694771+0000","last_unstale":"2023-11-28T03:07:18.695305+0000","last_undegraded":"2023-11-28T03:07:18.695305+0000","last_fullsized":"2023-11-28T03:07:18.695305+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330604+0000","last_change":"2023-11-28T03:07:18.326763+0000","last_active":"2023-11-28T03:07:18.330604+0000","last_peered":"2023-11-28T03:07:18.330604+0000","last_clean":"2023-11-28T03:07:18.330604+0000","last_became_active":"2023-11-28T03:07:18.326472+0000","last_became_peered":"2023-11-28T03:07:18.326472+0000","last_unstale":"2023-11-28T03:07:18.330604+0000","last_undegraded":"2023-11-28T03:07:18.330604+0000","last_fullsized":"2023-11-28T03:07:18.330604+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.24","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701528+0000","last_change":"2023-11-28T03:07:18.701528+0000","last_active":"2023-11-28T03:07:18.701528+0000","last_peered":"2023-11-28T03:07:18.701528+0000","last_clean":"2023-11-28T03:07:18.701528+0000","last_became_active":"2023-11-28T03:07:18.701246+0000","last_became_peered":"2023-11-28T03:07:18.701246+0000","last_unstale":"2023-11-28T03:07:18.701528+0000","last_undegraded":"2023-11-28T03:07:18.701528+0000","last_fullsized":"2023-11-28T03:07:18.701528+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334356+0000","last_change":"2023-11-28T03:07:18.329642+0000","last_active":"2023-11-28T03:07:18.334356+0000","last_peered":"2023-11-28T03:07:18.334356+0000","last_clean":"2023-11-28T03:07:18.334356+0000","last_became_active":"2023-11-28T03:07:18.329378+0000","last_became_peered":"2023-11-28T03:07:18.329378+0000","last_unstale":"2023-11-28T03:07:18.334356+0000","last_undegraded":"2023-11-28T03:07:18.334356+0000","last_fullsized":"2023-11-28T03:07:18.334356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.25","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706031+0000","last_change":"2023-11-28T03:07:18.706031+0000","last_active":"2023-11-28T03:07:18.706031+0000","last_peered":"2023-11-28T03:07:18.706031+0000","last_clean":"2023-11-28T03:07:18.706031+0000","last_became_active":"2023-11-28T03:07:18.705416+0000","last_became_peered":"2023-11-28T03:07:18.705416+0000","last_unstale":"2023-11-28T03:07:18.706031+0000","last_undegraded":"2023-11-28T03:07:18.706031+0000","last_fullsized":"2023-11-28T03:07:18.706031+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332531+0000","last_change":"2023-11-28T03:07:18.324147+0000","last_active":"2023-11-28T03:07:18.332531+0000","last_peered":"2023-11-28T03:07:18.332531+0000","last_clean":"2023-11-28T03:07:18.332531+0000","last_became_active":"2023-11-28T03:07:18.323503+0000","last_became_peered":"2023-11-28T03:07:18.323503+0000","last_unstale":"2023-11-28T03:07:18.332531+0000","last_undegraded":"2023-11-28T03:07:18.332531+0000","last_fullsized":"2023-11-28T03:07:18.332531+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.26","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692668+0000","last_change":"2023-11-28T03:07:18.692668+0000","last_active":"2023-11-28T03:07:18.692668+0000","last_peered":"2023-11-28T03:07:18.692668+0000","last_clean":"2023-11-28T03:07:18.692668+0000","last_became_active":"2023-11-28T03:07:18.692375+0000","last_became_peered":"2023-11-28T03:07:18.692375+0000","last_unstale":"2023-11-28T03:07:18.692668+0000","last_undegraded":"2023-11-28T03:07:18.692668+0000","last_fullsized":"2023-11-28T03:07:18.692668+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332785+0000","last_change":"2023-11-28T03:07:18.331573+0000","last_active":"2023-11-28T03:07:18.332785+0000","last_peered":"2023-11-28T03:07:18.332785+0000","last_clean":"2023-11-28T03:07:18.332785+0000","last_became_active":"2023-11-28T03:07:18.331278+0000","last_became_peered":"2023-11-28T03:07:18.331278+0000","last_unstale":"2023-11-28T03:07:18.332785+0000","last_undegraded":"2023-11-28T03:07:18.332785+0000","last_fullsized":"2023-11-28T03:07:18.332785+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.27","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693495+0000","last_change":"2023-11-28T03:07:18.693495+0000","last_active":"2023-11-28T03:07:18.693495+0000","last_peered":"2023-11-28T03:07:18.693495+0000","last_clean":"2023-11-28T03:07:18.693495+0000","last_became_active":"2023-11-28T03:07:18.691131+0000","last_became_peered":"2023-11-28T03:07:18.691131+0000","last_unstale":"2023-11-28T03:07:18.693495+0000","last_undegraded":"2023-11-28T03:07:18.693495+0000","last_fullsized":"2023-11-28T03:07:18.693495+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.339356+0000","last_change":"2023-11-28T03:07:18.335639+0000","last_active":"2023-11-28T03:07:18.339356+0000","last_peered":"2023-11-28T03:07:18.339356+0000","last_clean":"2023-11-28T03:07:18.339356+0000","last_became_active":"2023-11-28T03:07:18.335375+0000","last_became_peered":"2023-11-28T03:07:18.335375+0000","last_unstale":"2023-11-28T03:07:18.339356+0000","last_undegraded":"2023-11-28T03:07:18.339356+0000","last_fullsized":"2023-11-28T03:07:18.339356+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.20","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696864+0000","last_change":"2023-11-28T03:07:18.696864+0000","last_active":"2023-11-28T03:07:18.696864+0000","last_peered":"2023-11-28T03:07:18.696864+0000","last_clean":"2023-11-28T03:07:18.696864+0000","last_became_active":"2023-11-28T03:07:18.696585+0000","last_became_peered":"2023-11-28T03:07:18.696585+0000","last_unstale":"2023-11-28T03:07:18.696864+0000","last_undegraded":"2023-11-28T03:07:18.696864+0000","last_fullsized":"2023-11-28T03:07:18.696864+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331857+0000","last_change":"2023-11-28T03:07:18.327444+0000","last_active":"2023-11-28T03:07:18.331857+0000","last_peered":"2023-11-28T03:07:18.331857+0000","last_clean":"2023-11-28T03:07:18.331857+0000","last_became_active":"2023-11-28T03:07:18.327181+0000","last_became_peered":"2023-11-28T03:07:18.327181+0000","last_unstale":"2023-11-28T03:07:18.331857+0000","last_undegraded":"2023-11-28T03:07:18.331857+0000","last_fullsized":"2023-11-28T03:07:18.331857+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.21","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704933+0000","last_change":"2023-11-28T03:07:18.704933+0000","last_active":"2023-11-28T03:07:18.704933+0000","last_peered":"2023-11-28T03:07:18.704933+0000","last_clean":"2023-11-28T03:07:18.704933+0000","last_became_active":"2023-11-28T03:07:18.704570+0000","last_became_peered":"2023-11-28T03:07:18.704570+0000","last_unstale":"2023-11-28T03:07:18.704933+0000","last_undegraded":"2023-11-28T03:07:18.704933+0000","last_fullsized":"2023-11-28T03:07:18.704933+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333217+0000","last_change":"2023-11-28T03:07:18.327860+0000","last_active":"2023-11-28T03:07:18.333217+0000","last_peered":"2023-11-28T03:07:18.333217+0000","last_clean":"2023-11-28T03:07:18.333217+0000","last_became_active":"2023-11-28T03:07:18.327540+0000","last_became_peered":"2023-11-28T03:07:18.327540+0000","last_unstale":"2023-11-28T03:07:18.333217+0000","last_undegraded":"2023-11-28T03:07:18.333217+0000","last_fullsized":"2023-11-28T03:07:18.333217+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.22","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705088+0000","last_change":"2023-11-28T03:07:18.705088+0000","last_active":"2023-11-28T03:07:18.705088+0000","last_peered":"2023-11-28T03:07:18.705088+0000","last_clean":"2023-11-28T03:07:18.705088+0000","last_became_active":"2023-11-28T03:07:18.704729+0000","last_became_peered":"2023-11-28T03:07:18.704729+0000","last_unstale":"2023-11-28T03:07:18.705088+0000","last_undegraded":"2023-11-28T03:07:18.705088+0000","last_fullsized":"2023-11-28T03:07:18.705088+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334180+0000","last_change":"2023-11-28T03:07:18.329017+0000","last_active":"2023-11-28T03:07:18.334180+0000","last_peered":"2023-11-28T03:07:18.334180+0000","last_clean":"2023-11-28T03:07:18.334180+0000","last_became_active":"2023-11-28T03:07:18.328722+0000","last_became_peered":"2023-11-28T03:07:18.328722+0000","last_unstale":"2023-11-28T03:07:18.334180+0000","last_undegraded":"2023-11-28T03:07:18.334180+0000","last_fullsized":"2023-11-28T03:07:18.334180+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.23","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708870+0000","last_change":"2023-11-28T03:07:18.708941+0000","last_active":"2023-11-28T03:07:18.708870+0000","last_peered":"2023-11-28T03:07:18.708870+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.708676+0000","last_became_peered":"2023-11-28T03:07:18.708676+0000","last_unstale":"2023-11-28T03:07:18.708870+0000","last_undegraded":"2023-11-28T03:07:18.708870+0000","last_fullsized":"2023-11-28T03:07:18.708870+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326433+0000","last_change":"2023-11-28T03:07:18.321575+0000","last_active":"2023-11-28T03:07:18.326433+0000","last_peered":"2023-11-28T03:07:18.326433+0000","last_clean":"2023-11-28T03:07:18.326433+0000","last_became_active":"2023-11-28T03:07:18.321289+0000","last_became_peered":"2023-11-28T03:07:18.321289+0000","last_unstale":"2023-11-28T03:07:18.326433+0000","last_undegraded":"2023-11-28T03:07:18.326433+0000","last_fullsized":"2023-11-28T03:07:18.326433+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1c","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698169+0000","last_change":"2023-11-28T03:07:18.698169+0000","last_active":"2023-11-28T03:07:18.698169+0000","last_peered":"2023-11-28T03:07:18.698169+0000","last_clean":"2023-11-28T03:07:18.698169+0000","last_became_active":"2023-11-28T03:07:18.697586+0000","last_became_peered":"2023-11-28T03:07:18.697586+0000","last_unstale":"2023-11-28T03:07:18.698169+0000","last_undegraded":"2023-11-28T03:07:18.698169+0000","last_fullsized":"2023-11-28T03:07:18.698169+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1f","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.688984+0000","last_change":"2023-11-28T03:07:18.318292+0000","last_active":"2023-11-28T03:07:18.688984+0000","last_peered":"2023-11-28T03:07:18.688984+0000","last_clean":"2023-11-28T03:07:18.688984+0000","last_became_active":"2023-11-28T03:07:18.318121+0000","last_became_peered":"2023-11-28T03:07:18.318121+0000","last_unstale":"2023-11-28T03:07:18.688984+0000","last_undegraded":"2023-11-28T03:07:18.688984+0000","last_fullsized":"2023-11-28T03:07:18.688984+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705687+0000","last_change":"2023-11-28T03:07:18.705687+0000","last_active":"2023-11-28T03:07:18.705687+0000","last_peered":"2023-11-28T03:07:18.705687+0000","last_clean":"2023-11-28T03:07:18.705687+0000","last_became_active":"2023-11-28T03:07:18.704866+0000","last_became_peered":"2023-11-28T03:07:18.704866+0000","last_unstale":"2023-11-28T03:07:18.705687+0000","last_undegraded":"2023-11-28T03:07:18.705687+0000","last_fullsized":"2023-11-28T03:07:18.705687+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690925+0000","last_change":"2023-11-28T03:07:18.332883+0000","last_active":"2023-11-28T03:07:18.690925+0000","last_peered":"2023-11-28T03:07:18.690925+0000","last_clean":"2023-11-28T03:07:18.690925+0000","last_became_active":"2023-11-28T03:07:18.332630+0000","last_became_peered":"2023-11-28T03:07:18.332630+0000","last_unstale":"2023-11-28T03:07:18.690925+0000","last_undegraded":"2023-11-28T03:07:18.690925+0000","last_fullsized":"2023-11-28T03:07:18.690925+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.695813+0000","last_change":"2023-11-28T03:07:18.695813+0000","last_active":"2023-11-28T03:07:18.695813+0000","last_peered":"2023-11-28T03:07:18.695813+0000","last_clean":"2023-11-28T03:07:18.695813+0000","last_became_active":"2023-11-28T03:07:18.694746+0000","last_became_peered":"2023-11-28T03:07:18.694746+0000","last_unstale":"2023-11-28T03:07:18.695813+0000","last_undegraded":"2023-11-28T03:07:18.695813+0000","last_fullsized":"2023-11-28T03:07:18.695813+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706656+0000","last_change":"2023-11-28T03:07:18.326125+0000","last_active":"2023-11-28T03:07:18.706656+0000","last_peered":"2023-11-28T03:07:18.706656+0000","last_clean":"2023-11-28T03:07:18.706656+0000","last_became_active":"2023-11-28T03:07:18.325841+0000","last_became_peered":"2023-11-28T03:07:18.325841+0000","last_unstale":"2023-11-28T03:07:18.706656+0000","last_undegraded":"2023-11-28T03:07:18.706656+0000","last_fullsized":"2023-11-28T03:07:18.706656+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.1f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699429+0000","last_change":"2023-11-28T03:07:18.699429+0000","last_active":"2023-11-28T03:07:18.699429+0000","last_peered":"2023-11-28T03:07:18.699429+0000","last_clean":"2023-11-28T03:07:18.699429+0000","last_became_active":"2023-11-28T03:07:18.699112+0000","last_became_peered":"2023-11-28T03:07:18.699112+0000","last_unstale":"2023-11-28T03:07:18.699429+0000","last_undegraded":"2023-11-28T03:07:18.699429+0000","last_fullsized":"2023-11-28T03:07:18.699429+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332425+0000","last_change":"2023-11-28T03:07:18.329147+0000","last_active":"2023-11-28T03:07:18.332425+0000","last_peered":"2023-11-28T03:07:18.332425+0000","last_clean":"2023-11-28T03:07:18.332425+0000","last_became_active":"2023-11-28T03:07:18.328870+0000","last_became_peered":"2023-11-28T03:07:18.328870+0000","last_unstale":"2023-11-28T03:07:18.332425+0000","last_undegraded":"2023-11-28T03:07:18.332425+0000","last_fullsized":"2023-11-28T03:07:18.332425+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692223+0000","last_change":"2023-11-28T03:07:18.692223+0000","last_active":"2023-11-28T03:07:18.692223+0000","last_peered":"2023-11-28T03:07:18.692223+0000","last_clean":"2023-11-28T03:07:18.692223+0000","last_became_active":"2023-11-28T03:07:18.691720+0000","last_became_peered":"2023-11-28T03:07:18.691720+0000","last_unstale":"2023-11-28T03:07:18.692223+0000","last_undegraded":"2023-11-28T03:07:18.692223+0000","last_fullsized":"2023-11-28T03:07:18.692223+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331822+0000","last_change":"2023-11-28T03:07:18.327323+0000","last_active":"2023-11-28T03:07:18.331822+0000","last_peered":"2023-11-28T03:07:18.331822+0000","last_clean":"2023-11-28T03:07:18.331822+0000","last_became_active":"2023-11-28T03:07:18.327092+0000","last_became_peered":"2023-11-28T03:07:18.327092+0000","last_unstale":"2023-11-28T03:07:18.331822+0000","last_undegraded":"2023-11-28T03:07:18.331822+0000","last_fullsized":"2023-11-28T03:07:18.331822+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.707797+0000","last_change":"2023-11-28T03:07:18.707797+0000","last_active":"2023-11-28T03:07:18.707797+0000","last_peered":"2023-11-28T03:07:18.707797+0000","last_clean":"2023-11-28T03:07:18.707797+0000","last_became_active":"2023-11-28T03:07:18.707504+0000","last_became_peered":"2023-11-28T03:07:18.707504+0000","last_unstale":"2023-11-28T03:07:18.707797+0000","last_undegraded":"2023-11-28T03:07:18.707797+0000","last_fullsized":"2023-11-28T03:07:18.707797+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.320791+0000","last_change":"2023-11-28T03:07:18.315713+0000","last_active":"2023-11-28T03:07:18.320791+0000","last_peered":"2023-11-28T03:07:18.320791+0000","last_clean":"2023-11-28T03:07:18.320791+0000","last_became_active":"2023-11-28T03:07:18.315399+0000","last_became_peered":"2023-11-28T03:07:18.315399+0000","last_unstale":"2023-11-28T03:07:18.320791+0000","last_undegraded":"2023-11-28T03:07:18.320791+0000","last_fullsized":"2023-11-28T03:07:18.320791+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701652+0000","last_change":"2023-11-28T03:07:18.701652+0000","last_active":"2023-11-28T03:07:18.701652+0000","last_peered":"2023-11-28T03:07:18.701652+0000","last_clean":"2023-11-28T03:07:18.701652+0000","last_became_active":"2023-11-28T03:07:18.701377+0000","last_became_peered":"2023-11-28T03:07:18.701377+0000","last_unstale":"2023-11-28T03:07:18.701652+0000","last_undegraded":"2023-11-28T03:07:18.701652+0000","last_fullsized":"2023-11-28T03:07:18.701652+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334449+0000","last_change":"2023-11-28T03:07:18.330040+0000","last_active":"2023-11-28T03:07:18.334449+0000","last_peered":"2023-11-28T03:07:18.334449+0000","last_clean":"2023-11-28T03:07:18.334449+0000","last_became_active":"2023-11-28T03:07:18.327794+0000","last_became_peered":"2023-11-28T03:07:18.327794+0000","last_unstale":"2023-11-28T03:07:18.334449+0000","last_undegraded":"2023-11-28T03:07:18.334449+0000","last_fullsized":"2023-11-28T03:07:18.334449+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.705859+0000","last_change":"2023-11-28T03:07:18.705859+0000","last_active":"2023-11-28T03:07:18.705859+0000","last_peered":"2023-11-28T03:07:18.705859+0000","last_clean":"2023-11-28T03:07:18.705859+0000","last_became_active":"2023-11-28T03:07:18.705610+0000","last_became_peered":"2023-11-28T03:07:18.705610+0000","last_unstale":"2023-11-28T03:07:18.705859+0000","last_undegraded":"2023-11-28T03:07:18.705859+0000","last_fullsized":"2023-11-28T03:07:18.705859+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330622+0000","last_change":"2023-11-28T03:07:18.326813+0000","last_active":"2023-11-28T03:07:18.330622+0000","last_peered":"2023-11-28T03:07:18.330622+0000","last_clean":"2023-11-28T03:07:18.330622+0000","last_became_active":"2023-11-28T03:07:18.326507+0000","last_became_peered":"2023-11-28T03:07:18.326507+0000","last_unstale":"2023-11-28T03:07:18.330622+0000","last_undegraded":"2023-11-28T03:07:18.330622+0000","last_fullsized":"2023-11-28T03:07:18.330622+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.14","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693070+0000","last_change":"2023-11-28T03:07:18.693070+0000","last_active":"2023-11-28T03:07:18.693070+0000","last_peered":"2023-11-28T03:07:18.693070+0000","last_clean":"2023-11-28T03:07:18.693070+0000","last_became_active":"2023-11-28T03:07:18.692797+0000","last_became_peered":"2023-11-28T03:07:18.692797+0000","last_unstale":"2023-11-28T03:07:18.693070+0000","last_undegraded":"2023-11-28T03:07:18.693070+0000","last_fullsized":"2023-11-28T03:07:18.693070+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.335099+0000","last_change":"2023-11-28T03:07:18.332734+0000","last_active":"2023-11-28T03:07:18.335099+0000","last_peered":"2023-11-28T03:07:18.335099+0000","last_clean":"2023-11-28T03:07:18.335099+0000","last_became_active":"2023-11-28T03:07:18.332433+0000","last_became_peered":"2023-11-28T03:07:18.332433+0000","last_unstale":"2023-11-28T03:07:18.335099+0000","last_undegraded":"2023-11-28T03:07:18.335099+0000","last_fullsized":"2023-11-28T03:07:18.335099+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.15","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.692820+0000","last_change":"2023-11-28T03:07:18.692820+0000","last_active":"2023-11-28T03:07:18.692820+0000","last_peered":"2023-11-28T03:07:18.692820+0000","last_clean":"2023-11-28T03:07:18.692820+0000","last_became_active":"2023-11-28T03:07:18.692571+0000","last_became_peered":"2023-11-28T03:07:18.692571+0000","last_unstale":"2023-11-28T03:07:18.692820+0000","last_undegraded":"2023-11-28T03:07:18.692820+0000","last_fullsized":"2023-11-28T03:07:18.692820+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.330778+0000","last_change":"2023-11-28T03:07:18.322012+0000","last_active":"2023-11-28T03:07:18.330778+0000","last_peered":"2023-11-28T03:07:18.330778+0000","last_clean":"2023-11-28T03:07:18.330778+0000","last_became_active":"2023-11-28T03:07:18.321741+0000","last_became_peered":"2023-11-28T03:07:18.321741+0000","last_unstale":"2023-11-28T03:07:18.330778+0000","last_undegraded":"2023-11-28T03:07:18.330778+0000","last_fullsized":"2023-11-28T03:07:18.330778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696598+0000","last_change":"2023-11-28T03:07:18.696598+0000","last_active":"2023-11-28T03:07:18.696598+0000","last_peered":"2023-11-28T03:07:18.696598+0000","last_clean":"2023-11-28T03:07:18.696598+0000","last_became_active":"2023-11-28T03:07:18.696280+0000","last_became_peered":"2023-11-28T03:07:18.696280+0000","last_unstale":"2023-11-28T03:07:18.696598+0000","last_undegraded":"2023-11-28T03:07:18.696598+0000","last_fullsized":"2023-11-28T03:07:18.696598+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.15","version":"81'3","reported_seq":12,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.704761+0000","last_change":"2023-11-28T03:07:18.333084+0000","last_active":"2023-11-28T03:07:18.704761+0000","last_peered":"2023-11-28T03:07:18.704761+0000","last_clean":"2023-11-28T03:07:18.704761+0000","last_became_active":"2023-11-28T03:07:18.332341+0000","last_became_peered":"2023-11-28T03:07:18.332341+0000","last_unstale":"2023-11-28T03:07:18.704761+0000","last_undegraded":"2023-11-28T03:07:18.704761+0000","last_fullsized":"2023-11-28T03:07:18.704761+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698702+0000","last_change":"2023-11-28T03:07:18.698702+0000","last_active":"2023-11-28T03:07:18.698702+0000","last_peered":"2023-11-28T03:07:18.698702+0000","last_clean":"2023-11-28T03:07:18.698702+0000","last_became_active":"2023-11-28T03:07:18.698359+0000","last_became_peered":"2023-11-28T03:07:18.698359+0000","last_unstale":"2023-11-28T03:07:18.698702+0000","last_undegraded":"2023-11-28T03:07:18.698702+0000","last_fullsized":"2023-11-28T03:07:18.698702+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.14","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708516+0000","last_change":"2023-11-28T03:07:18.319397+0000","last_active":"2023-11-28T03:07:18.708516+0000","last_peered":"2023-11-28T03:07:18.708516+0000","last_clean":"2023-11-28T03:07:18.708516+0000","last_became_active":"2023-11-28T03:07:18.319044+0000","last_became_peered":"2023-11-28T03:07:18.319044+0000","last_unstale":"2023-11-28T03:07:18.708516+0000","last_undegraded":"2023-11-28T03:07:18.708516+0000","last_fullsized":"2023-11-28T03:07:18.708516+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.10","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.703657+0000","last_change":"2023-11-28T03:07:18.703657+0000","last_active":"2023-11-28T03:07:18.703657+0000","last_peered":"2023-11-28T03:07:18.703657+0000","last_clean":"2023-11-28T03:07:18.703657+0000","last_became_active":"2023-11-28T03:07:18.703363+0000","last_became_peered":"2023-11-28T03:07:18.703363+0000","last_unstale":"2023-11-28T03:07:18.703657+0000","last_undegraded":"2023-11-28T03:07:18.703657+0000","last_fullsized":"2023-11-28T03:07:18.703657+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.13","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.686466+0000","last_change":"2023-11-28T03:07:18.331256+0000","last_active":"2023-11-28T03:07:18.686466+0000","last_peered":"2023-11-28T03:07:18.686466+0000","last_clean":"2023-11-28T03:07:18.686466+0000","last_became_active":"2023-11-28T03:07:18.330974+0000","last_became_peered":"2023-11-28T03:07:18.330974+0000","last_unstale":"2023-11-28T03:07:18.686466+0000","last_undegraded":"2023-11-28T03:07:18.686466+0000","last_fullsized":"2023-11-28T03:07:18.686466+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699865+0000","last_change":"2023-11-28T03:07:18.699865+0000","last_active":"2023-11-28T03:07:18.699865+0000","last_peered":"2023-11-28T03:07:18.699865+0000","last_clean":"2023-11-28T03:07:18.699865+0000","last_became_active":"2023-11-28T03:07:18.698850+0000","last_became_peered":"2023-11-28T03:07:18.698850+0000","last_unstale":"2023-11-28T03:07:18.699865+0000","last_undegraded":"2023-11-28T03:07:18.699865+0000","last_fullsized":"2023-11-28T03:07:18.699865+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334633+0000","last_change":"2023-11-28T03:07:18.332481+0000","last_active":"2023-11-28T03:07:18.334633+0000","last_peered":"2023-11-28T03:07:18.334633+0000","last_clean":"2023-11-28T03:07:18.334633+0000","last_became_active":"2023-11-28T03:07:18.332182+0000","last_became_peered":"2023-11-28T03:07:18.332182+0000","last_unstale":"2023-11-28T03:07:18.334633+0000","last_undegraded":"2023-11-28T03:07:18.334633+0000","last_fullsized":"2023-11-28T03:07:18.334633+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.698560+0000","last_change":"2023-11-28T03:07:18.698560+0000","last_active":"2023-11-28T03:07:18.698560+0000","last_peered":"2023-11-28T03:07:18.698560+0000","last_clean":"2023-11-28T03:07:18.698560+0000","last_became_active":"2023-11-28T03:07:18.696810+0000","last_became_peered":"2023-11-28T03:07:18.696810+0000","last_unstale":"2023-11-28T03:07:18.698560+0000","last_undegraded":"2023-11-28T03:07:18.698560+0000","last_fullsized":"2023-11-28T03:07:18.698560+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331280+0000","last_change":"2023-11-28T03:07:18.324854+0000","last_active":"2023-11-28T03:07:18.331280+0000","last_peered":"2023-11-28T03:07:18.331280+0000","last_clean":"2023-11-28T03:07:18.331280+0000","last_became_active":"2023-11-28T03:07:18.324573+0000","last_became_peered":"2023-11-28T03:07:18.324573+0000","last_unstale":"2023-11-28T03:07:18.331280+0000","last_undegraded":"2023-11-28T03:07:18.331280+0000","last_fullsized":"2023-11-28T03:07:18.331280+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702968+0000","last_change":"2023-11-28T03:07:18.702968+0000","last_active":"2023-11-28T03:07:18.702968+0000","last_peered":"2023-11-28T03:07:18.702968+0000","last_clean":"2023-11-28T03:07:18.702968+0000","last_became_active":"2023-11-28T03:07:18.702691+0000","last_became_peered":"2023-11-28T03:07:18.702691+0000","last_unstale":"2023-11-28T03:07:18.702968+0000","last_undegraded":"2023-11-28T03:07:18.702968+0000","last_fullsized":"2023-11-28T03:07:18.702968+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.10","version":"81'4","reported_seq":13,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706623+0000","last_change":"2023-11-28T03:07:18.318505+0000","last_active":"2023-11-28T03:07:18.706623+0000","last_peered":"2023-11-28T03:07:18.706623+0000","last_clean":"2023-11-28T03:07:18.706623+0000","last_became_active":"2023-11-28T03:07:18.318173+0000","last_became_peered":"2023-11-28T03:07:18.318173+0000","last_unstale":"2023-11-28T03:07:18.706623+0000","last_undegraded":"2023-11-28T03:07:18.706623+0000","last_fullsized":"2023-11-28T03:07:18.706623+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708779+0000","last_change":"2023-11-28T03:07:18.708779+0000","last_active":"2023-11-28T03:07:18.708779+0000","last_peered":"2023-11-28T03:07:18.708779+0000","last_clean":"2023-11-28T03:07:18.708779+0000","last_became_active":"2023-11-28T03:07:18.708503+0000","last_became_peered":"2023-11-28T03:07:18.708503+0000","last_unstale":"2023-11-28T03:07:18.708779+0000","last_undegraded":"2023-11-28T03:07:18.708779+0000","last_fullsized":"2023-11-28T03:07:18.708779+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.7","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697354+0000","last_change":"2023-11-28T03:07:18.329942+0000","last_active":"2023-11-28T03:07:18.697354+0000","last_peered":"2023-11-28T03:07:18.697354+0000","last_clean":"2023-11-28T03:07:18.697354+0000","last_became_active":"2023-11-28T03:07:18.329664+0000","last_became_peered":"2023-11-28T03:07:18.329664+0000","last_unstale":"2023-11-28T03:07:18.697354+0000","last_undegraded":"2023-11-28T03:07:18.697354+0000","last_fullsized":"2023-11-28T03:07:18.697354+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.689393+0000","last_change":"2023-11-28T03:07:18.689393+0000","last_active":"2023-11-28T03:07:18.689393+0000","last_peered":"2023-11-28T03:07:18.689393+0000","last_clean":"2023-11-28T03:07:18.689393+0000","last_became_active":"2023-11-28T03:07:18.688368+0000","last_became_peered":"2023-11-28T03:07:18.688368+0000","last_unstale":"2023-11-28T03:07:18.689393+0000","last_undegraded":"2023-11-28T03:07:18.689393+0000","last_fullsized":"2023-11-28T03:07:18.689393+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332923+0000","last_change":"2023-11-28T03:07:18.327819+0000","last_active":"2023-11-28T03:07:18.332923+0000","last_peered":"2023-11-28T03:07:18.332923+0000","last_clean":"2023-11-28T03:07:18.332923+0000","last_became_active":"2023-11-28T03:07:18.326925+0000","last_became_peered":"2023-11-28T03:07:18.326925+0000","last_unstale":"2023-11-28T03:07:18.332923+0000","last_undegraded":"2023-11-28T03:07:18.332923+0000","last_fullsized":"2023-11-28T03:07:18.332923+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.693734+0000","last_change":"2023-11-28T03:07:18.693734+0000","last_active":"2023-11-28T03:07:18.693734+0000","last_peered":"2023-11-28T03:07:18.693734+0000","last_clean":"2023-11-28T03:07:18.693734+0000","last_became_active":"2023-11-28T03:07:18.691466+0000","last_became_peered":"2023-11-28T03:07:18.691466+0000","last_unstale":"2023-11-28T03:07:18.693734+0000","last_undegraded":"2023-11-28T03:07:18.693734+0000","last_fullsized":"2023-11-28T03:07:18.693734+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.334813+0000","last_change":"2023-11-28T03:07:18.334046+0000","last_active":"2023-11-28T03:07:18.334813+0000","last_peered":"2023-11-28T03:07:18.334813+0000","last_clean":"2023-11-28T03:07:18.334813+0000","last_became_active":"2023-11-28T03:07:18.333743+0000","last_became_peered":"2023-11-28T03:07:18.333743+0000","last_unstale":"2023-11-28T03:07:18.334813+0000","last_undegraded":"2023-11-28T03:07:18.334813+0000","last_fullsized":"2023-11-28T03:07:18.334813+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701396+0000","last_change":"2023-11-28T03:07:18.701396+0000","last_active":"2023-11-28T03:07:18.701396+0000","last_peered":"2023-11-28T03:07:18.701396+0000","last_clean":"2023-11-28T03:07:18.701396+0000","last_became_active":"2023-11-28T03:07:18.701046+0000","last_became_peered":"2023-11-28T03:07:18.701046+0000","last_unstale":"2023-11-28T03:07:18.701396+0000","last_undegraded":"2023-11-28T03:07:18.701396+0000","last_fullsized":"2023-11-28T03:07:18.701396+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.328543+0000","last_change":"2023-11-28T03:07:18.317867+0000","last_active":"2023-11-28T03:07:18.328543+0000","last_peered":"2023-11-28T03:07:18.328543+0000","last_clean":"2023-11-28T03:07:18.328543+0000","last_became_active":"2023-11-28T03:07:18.315981+0000","last_became_peered":"2023-11-28T03:07:18.315981+0000","last_unstale":"2023-11-28T03:07:18.328543+0000","last_undegraded":"2023-11-28T03:07:18.328543+0000","last_fullsized":"2023-11-28T03:07:18.328543+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696920+0000","last_change":"2023-11-28T03:07:18.696920+0000","last_active":"2023-11-28T03:07:18.696920+0000","last_peered":"2023-11-28T03:07:18.696920+0000","last_clean":"2023-11-28T03:07:18.696920+0000","last_became_active":"2023-11-28T03:07:18.696474+0000","last_became_peered":"2023-11-28T03:07:18.696474+0000","last_unstale":"2023-11-28T03:07:18.696920+0000","last_undegraded":"2023-11-28T03:07:18.696920+0000","last_fullsized":"2023-11-28T03:07:18.696920+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333000+0000","last_change":"2023-11-28T03:07:18.326232+0000","last_active":"2023-11-28T03:07:18.333000+0000","last_peered":"2023-11-28T03:07:18.333000+0000","last_clean":"2023-11-28T03:07:18.333000+0000","last_became_active":"2023-11-28T03:07:18.325971+0000","last_became_peered":"2023-11-28T03:07:18.325971+0000","last_unstale":"2023-11-28T03:07:18.333000+0000","last_undegraded":"2023-11-28T03:07:18.333000+0000","last_fullsized":"2023-11-28T03:07:18.333000+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.706384+0000","last_change":"2023-11-28T03:07:18.706384+0000","last_active":"2023-11-28T03:07:18.706384+0000","last_peered":"2023-11-28T03:07:18.706384+0000","last_clean":"2023-11-28T03:07:18.706384+0000","last_became_active":"2023-11-28T03:07:18.704207+0000","last_became_peered":"2023-11-28T03:07:18.704207+0000","last_unstale":"2023-11-28T03:07:18.706384+0000","last_undegraded":"2023-11-28T03:07:18.706384+0000","last_fullsized":"2023-11-28T03:07:18.706384+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.336538+0000","last_change":"2023-11-28T03:07:18.330557+0000","last_active":"2023-11-28T03:07:18.336538+0000","last_peered":"2023-11-28T03:07:18.336538+0000","last_clean":"2023-11-28T03:07:18.336538+0000","last_became_active":"2023-11-28T03:07:18.330299+0000","last_became_peered":"2023-11-28T03:07:18.330299+0000","last_unstale":"2023-11-28T03:07:18.336538+0000","last_undegraded":"2023-11-28T03:07:18.336538+0000","last_fullsized":"2023-11-28T03:07:18.336538+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.696929+0000","last_change":"2023-11-28T03:07:18.696929+0000","last_active":"2023-11-28T03:07:18.696929+0000","last_peered":"2023-11-28T03:07:18.696929+0000","last_clean":"2023-11-28T03:07:18.696929+0000","last_became_active":"2023-11-28T03:07:18.696694+0000","last_became_peered":"2023-11-28T03:07:18.696694+0000","last_unstale":"2023-11-28T03:07:18.696929+0000","last_undegraded":"2023-11-28T03:07:18.696929+0000","last_fullsized":"2023-11-28T03:07:18.696929+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326233+0000","last_change":"2023-11-28T03:07:18.323531+0000","last_active":"2023-11-28T03:07:18.326233+0000","last_peered":"2023-11-28T03:07:18.326233+0000","last_clean":"2023-11-28T03:07:18.326233+0000","last_became_active":"2023-11-28T03:07:18.323277+0000","last_became_peered":"2023-11-28T03:07:18.323277+0000","last_unstale":"2023-11-28T03:07:18.326233+0000","last_undegraded":"2023-11-28T03:07:18.326233+0000","last_fullsized":"2023-11-28T03:07:18.326233+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.691919+0000","last_change":"2023-11-28T03:07:18.691919+0000","last_active":"2023-11-28T03:07:18.691919+0000","last_peered":"2023-11-28T03:07:18.691919+0000","last_clean":"2023-11-28T03:07:18.691919+0000","last_became_active":"2023-11-28T03:07:18.691649+0000","last_became_peered":"2023-11-28T03:07:18.691649+0000","last_unstale":"2023-11-28T03:07:18.691919+0000","last_undegraded":"2023-11-28T03:07:18.691919+0000","last_fullsized":"2023-11-28T03:07:18.691919+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.332320+0000","last_change":"2023-11-28T03:07:18.327972+0000","last_active":"2023-11-28T03:07:18.332320+0000","last_peered":"2023-11-28T03:07:18.332320+0000","last_clean":"2023-11-28T03:07:18.332320+0000","last_became_active":"2023-11-28T03:07:18.327696+0000","last_became_peered":"2023-11-28T03:07:18.327696+0000","last_unstale":"2023-11-28T03:07:18.332320+0000","last_undegraded":"2023-11-28T03:07:18.332320+0000","last_fullsized":"2023-11-28T03:07:18.332320+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":7,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.709544+0000","last_change":"2023-11-28T03:07:18.709624+0000","last_active":"2023-11-28T03:07:18.709544+0000","last_peered":"2023-11-28T03:07:18.709544+0000","last_clean":"2023-11-28T03:07:18.300627+0000","last_became_active":"2023-11-28T03:07:18.709328+0000","last_became_peered":"2023-11-28T03:07:18.709328+0000","last_unstale":"2023-11-28T03:07:18.709544+0000","last_undegraded":"2023-11-28T03:07:18.709544+0000","last_fullsized":"2023-11-28T03:07:18.709544+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.683982+0000","last_change":"2023-11-28T03:07:18.334358+0000","last_active":"2023-11-28T03:07:18.683982+0000","last_peered":"2023-11-28T03:07:18.683982+0000","last_clean":"2023-11-28T03:07:18.683982+0000","last_became_active":"2023-11-28T03:07:18.333966+0000","last_became_peered":"2023-11-28T03:07:18.333966+0000","last_unstale":"2023-11-28T03:07:18.683982+0000","last_undegraded":"2023-11-28T03:07:18.683982+0000","last_fullsized":"2023-11-28T03:07:18.683982+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.307970+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:07:18.307970+0000","last_peered":"2023-11-28T03:07:18.307970+0000","last_clean":"2023-11-28T03:07:18.307970+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:07:18.307970+0000","last_undegraded":"2023-11-28T03:07:18.307970+0000","last_fullsized":"2023-11-28T03:07:18.307970+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.699939+0000","last_change":"2023-11-28T03:07:18.699939+0000","last_active":"2023-11-28T03:07:18.699939+0000","last_peered":"2023-11-28T03:07:18.699939+0000","last_clean":"2023-11-28T03:07:18.699939+0000","last_became_active":"2023-11-28T03:07:18.699583+0000","last_became_peered":"2023-11-28T03:07:18.699583+0000","last_unstale":"2023-11-28T03:07:18.699939+0000","last_undegraded":"2023-11-28T03:07:18.699939+0000","last_fullsized":"2023-11-28T03:07:18.699939+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331175+0000","last_change":"2023-11-28T03:07:18.325091+0000","last_active":"2023-11-28T03:07:18.331175+0000","last_peered":"2023-11-28T03:07:18.331175+0000","last_clean":"2023-11-28T03:07:18.331175+0000","last_became_active":"2023-11-28T03:07:18.324845+0000","last_became_peered":"2023-11-28T03:07:18.324845+0000","last_unstale":"2023-11-28T03:07:18.331175+0000","last_undegraded":"2023-11-28T03:07:18.331175+0000","last_fullsized":"2023-11-28T03:07:18.331175+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.325083+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:07:18.325083+0000","last_peered":"2023-11-28T03:07:18.325083+0000","last_clean":"2023-11-28T03:07:18.325083+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:07:18.325083+0000","last_undegraded":"2023-11-28T03:07:18.325083+0000","last_fullsized":"2023-11-28T03:07:18.325083+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690236+0000","last_change":"2023-11-28T03:07:18.690236+0000","last_active":"2023-11-28T03:07:18.690236+0000","last_peered":"2023-11-28T03:07:18.690236+0000","last_clean":"2023-11-28T03:07:18.690236+0000","last_became_active":"2023-11-28T03:07:18.689942+0000","last_became_peered":"2023-11-28T03:07:18.689942+0000","last_unstale":"2023-11-28T03:07:18.690236+0000","last_undegraded":"2023-11-28T03:07:18.690236+0000","last_fullsized":"2023-11-28T03:07:18.690236+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.333482+0000","last_change":"2023-11-28T03:07:18.331714+0000","last_active":"2023-11-28T03:07:18.333482+0000","last_peered":"2023-11-28T03:07:18.333482+0000","last_clean":"2023-11-28T03:07:18.333482+0000","last_became_active":"2023-11-28T03:07:18.331441+0000","last_became_peered":"2023-11-28T03:07:18.331441+0000","last_unstale":"2023-11-28T03:07:18.333482+0000","last_undegraded":"2023-11-28T03:07:18.333482+0000","last_fullsized":"2023-11-28T03:07:18.333482+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":65,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.312386+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:07:18.312386+0000","last_peered":"2023-11-28T03:07:18.312386+0000","last_clean":"2023-11-28T03:07:18.312386+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:07:18.312386+0000","last_undegraded":"2023-11-28T03:07:18.312386+0000","last_fullsized":"2023-11-28T03:07:18.312386+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.b","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.700764+0000","last_change":"2023-11-28T03:07:18.700764+0000","last_active":"2023-11-28T03:07:18.700764+0000","last_peered":"2023-11-28T03:07:18.700764+0000","last_clean":"2023-11-28T03:07:18.700764+0000","last_became_active":"2023-11-28T03:07:18.699307+0000","last_became_peered":"2023-11-28T03:07:18.699307+0000","last_unstale":"2023-11-28T03:07:18.700764+0000","last_undegraded":"2023-11-28T03:07:18.700764+0000","last_fullsized":"2023-11-28T03:07:18.700764+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.331778+0000","last_change":"2023-11-28T03:07:18.327278+0000","last_active":"2023-11-28T03:07:18.331778+0000","last_peered":"2023-11-28T03:07:18.331778+0000","last_clean":"2023-11-28T03:07:18.331778+0000","last_became_active":"2023-11-28T03:07:18.327009+0000","last_became_peered":"2023-11-28T03:07:18.327009+0000","last_unstale":"2023-11-28T03:07:18.331778+0000","last_undegraded":"2023-11-28T03:07:18.331778+0000","last_fullsized":"2023-11-28T03:07:18.331778+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.326448+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:07:18.326448+0000","last_peered":"2023-11-28T03:07:18.326448+0000","last_clean":"2023-11-28T03:07:18.326448+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:07:18.326448+0000","last_undegraded":"2023-11-28T03:07:18.326448+0000","last_fullsized":"2023-11-28T03:07:18.326448+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":66,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.314862+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:07:18.314862+0000","last_peered":"2023-11-28T03:07:18.314862+0000","last_clean":"2023-11-28T03:07:18.314862+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:07:18.314862+0000","last_undegraded":"2023-11-28T03:07:18.314862+0000","last_fullsized":"2023-11-28T03:07:18.314862+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.697635+0000","last_change":"2023-11-28T03:07:18.697635+0000","last_active":"2023-11-28T03:07:18.697635+0000","last_peered":"2023-11-28T03:07:18.697635+0000","last_clean":"2023-11-28T03:07:18.697635+0000","last_became_active":"2023-11-28T03:07:18.697361+0000","last_became_peered":"2023-11-28T03:07:18.697361+0000","last_unstale":"2023-11-28T03:07:18.697635+0000","last_undegraded":"2023-11-28T03:07:18.697635+0000","last_fullsized":"2023-11-28T03:07:18.697635+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.b","version":"81'1","reported_seq":10,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.690083+0000","last_change":"2023-11-28T03:07:18.331583+0000","last_active":"2023-11-28T03:07:18.690083+0000","last_peered":"2023-11-28T03:07:18.690083+0000","last_clean":"2023-11-28T03:07:18.690083+0000","last_became_active":"2023-11-28T03:07:18.331296+0000","last_became_peered":"2023-11-28T03:07:18.331296+0000","last_unstale":"2023-11-28T03:07:18.690083+0000","last_undegraded":"2023-11-28T03:07:18.690083+0000","last_fullsized":"2023-11-28T03:07:18.690083+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.314950+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:07:18.314950+0000","last_peered":"2023-11-28T03:07:18.314950+0000","last_clean":"2023-11-28T03:07:18.314950+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:07:18.314950+0000","last_undegraded":"2023-11-28T03:07:18.314950+0000","last_fullsized":"2023-11-28T03:07:18.314950+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.708677+0000","last_change":"2023-11-28T03:07:18.708677+0000","last_active":"2023-11-28T03:07:18.708677+0000","last_peered":"2023-11-28T03:07:18.708677+0000","last_clean":"2023-11-28T03:07:18.708677+0000","last_became_active":"2023-11-28T03:07:18.708400+0000","last_became_peered":"2023-11-28T03:07:18.708400+0000","last_unstale":"2023-11-28T03:07:18.708677+0000","last_undegraded":"2023-11-28T03:07:18.708677+0000","last_fullsized":"2023-11-28T03:07:18.708677+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.e","version":"81'2","reported_seq":11,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.709316+0000","last_change":"2023-11-28T03:07:18.325748+0000","last_active":"2023-11-28T03:07:18.709316+0000","last_peered":"2023-11-28T03:07:18.709316+0000","last_clean":"2023-11-28T03:07:18.709316+0000","last_became_active":"2023-11-28T03:07:18.325468+0000","last_became_peered":"2023-11-28T03:07:18.325468+0000","last_unstale":"2023-11-28T03:07:18.709316+0000","last_undegraded":"2023-11-28T03:07:18.709316+0000","last_fullsized":"2023-11-28T03:07:18.709316+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":64,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.318033+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:07:18.318033+0000","last_peered":"2023-11-28T03:07:18.318033+0000","last_clean":"2023-11-28T03:07:18.318033+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:07:18.318033+0000","last_undegraded":"2023-11-28T03:07:18.318033+0000","last_fullsized":"2023-11-28T03:07:18.318033+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.702226+0000","last_change":"2023-11-28T03:07:18.702226+0000","last_active":"2023-11-28T03:07:18.702226+0000","last_peered":"2023-11-28T03:07:18.702226+0000","last_clean":"2023-11-28T03:07:18.702226+0000","last_became_active":"2023-11-28T03:07:18.701911+0000","last_became_peered":"2023-11-28T03:07:18.701911+0000","last_unstale":"2023-11-28T03:07:18.702226+0000","last_undegraded":"2023-11-28T03:07:18.702226+0000","last_fullsized":"2023-11-28T03:07:18.702226+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.335179+0000","last_change":"2023-11-28T03:07:18.333307+0000","last_active":"2023-11-28T03:07:18.335179+0000","last_peered":"2023-11-28T03:07:18.335179+0000","last_clean":"2023-11-28T03:07:18.335179+0000","last_became_active":"2023-11-28T03:07:18.332718+0000","last_became_peered":"2023-11-28T03:07:18.332718+0000","last_unstale":"2023-11-28T03:07:18.335179+0000","last_undegraded":"2023-11-28T03:07:18.335179+0000","last_fullsized":"2023-11-28T03:07:18.335179+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":63,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.308170+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:07:18.308170+0000","last_peered":"2023-11-28T03:07:18.308170+0000","last_clean":"2023-11-28T03:07:18.308170+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:07:18.308170+0000","last_undegraded":"2023-11-28T03:07:18.308170+0000","last_fullsized":"2023-11-28T03:07:18.308170+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.f","version":"0'0","reported_seq":8,"reported_epoch":81,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.701240+0000","last_change":"2023-11-28T03:07:18.701240+0000","last_active":"2023-11-28T03:07:18.701240+0000","last_peered":"2023-11-28T03:07:18.701240+0000","last_clean":"2023-11-28T03:07:18.701240+0000","last_became_active":"2023-11-28T03:07:18.700933+0000","last_became_peered":"2023-11-28T03:07:18.700933+0000","last_unstale":"2023-11-28T03:07:18.701240+0000","last_undegraded":"2023-11-28T03:07:18.701240+0000","last_fullsized":"2023-11-28T03:07:18.701240+0000","mapping_epoch":80,"log_start":"0'0","ondisk_log_start":"0'0","created":80,"last_epoch_clean":81,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:18.300627+0000","last_clean_scrub_stamp":"2023-11-28T03:07:18.300627+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":9,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.323371+0000","last_change":"2023-11-28T03:07:18.320644+0000","last_active":"2023-11-28T03:07:18.323371+0000","last_peered":"2023-11-28T03:07:18.323371+0000","last_clean":"2023-11-28T03:07:18.323371+0000","last_became_active":"2023-11-28T03:07:18.320327+0000","last_became_peered":"2023-11-28T03:07:18.320327+0000","last_unstale":"2023-11-28T03:07:18.323371+0000","last_undegraded":"2023-11-28T03:07:18.323371+0000","last_fullsized":"2023-11-28T03:07:18.323371+0000","mapping_epoch":79,"log_start":"0'0","ondisk_log_start":"0'0","created":79,"last_epoch_clean":80,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:17.296055+0000","last_clean_scrub_stamp":"2023-11-28T03:07:17.296055+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":69,"reported_epoch":80,"state":"active+clean","last_fresh":"2023-11-28T03:07:18.312253+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:07:18.312253+0000","last_peered":"2023-11-28T03:07:18.312253+0000","last_clean":"2023-11-28T03:07:18.312253+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:07:18.312253+0000","last_undegraded":"2023-11-28T03:07:18.312253+0000","last_fullsized":"2023-11-28T03:07:18.312253+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":10,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575025,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.375}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.40799999999999997}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.73899999999999999,"5min":0.749,"15min":0.749},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.39600000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.77100000000000002},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.66100000000000003,"5min":0.748,"15min":0.81899999999999995},"last":0.34899999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.755,"5min":0.755,"15min":0.84499999999999997},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.83199999999999996},"last":0.34100000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.86499999999999999},"last":0.433},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.44}]}]},{"osd":6,"up_from":13,"seq":55834575025,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":1.125,"5min":1.506,"15min":1.506},"last":0.41299999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.53000000000000003,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.442},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.38400000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67800000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.82499999999999996},"last":0.45700000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.755,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.434}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.40500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575024,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34399999999999997,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.318},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.28000000000000003,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46000000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.378},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.19800000000000001},"max":{"1min":0.56799999999999995,"5min":0.628,"15min":0.72599999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.69599999999999995,"15min":0.749},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.66500000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":1.038,"15min":1.111},"last":0.77400000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.374},"max":{"1min":0.76000000000000001,"5min":1.024,"15min":1.024},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575025,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298040,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":297088,"kb_avail":94073800,"statfs":{"total":96636764160,"available":96331571200,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304218112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.221,"15min":0.184},"max":{"1min":0.45100000000000001,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.51700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.25}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36499999999999999,"15min":0.36899999999999999},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.505,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.372,"5min":0.307,"15min":0.26300000000000001},"max":{"1min":0.57599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.35999999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.26200000000000001,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.30599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.78000000000000003,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.77600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.41599999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":1.304,"15min":1.304},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39000000000000001},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.44400000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":1.679,"15min":1.679},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.36599999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.435},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.046,"15min":1.046},"last":0.45000000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575026,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.27600000000000002,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.623},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.36299999999999999},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.27900000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.47899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.874},"last":0.57499999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.439,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.004,"15min":1.004},"last":0.58599999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.628,"15min":0.628},"min":{"1min":0.55200000000000005,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.55600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575025,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302136,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069704,"statfs":{"total":96636764160,"available":96327376896,"internally_reserved":0,"allocated":966656,"data_stored":477128,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.28499999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35599999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.373},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.432},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.32900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32400000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41899999999999998},"min":{"1min":0.32400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.438,"15min":0.435},"min":{"1min":0.36799999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.29699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85799999999999998,"5min":1.581,"15min":1.6000000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.94199999999999995,"15min":1.196},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.63700000000000001},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.95799999999999996,"15min":1.206},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.67800000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.65100000000000002},"min":{"1min":0.51500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.78900000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575024,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302124,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069716,"statfs":{"total":96636764160,"available":96327389184,"internally_reserved":0,"allocated":954368,"data_stored":470449,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.505},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.621}]},{"osd":1,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.45100000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.55800000000000005}]},{"osd":2,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.78100000000000003},"last":0.63500000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.38500000000000001},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.878},"last":0.66300000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.77900000000000003},"last":0.64900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83899999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.91300000000000003},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575026,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302156,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069684,"statfs":{"total":96636764160,"available":96327356416,"internally_reserved":0,"allocated":987136,"data_stored":485221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.79800000000000004},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42699999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.57699999999999996}]},{"osd":3,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.56299999999999994}]},{"osd":4,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.58799999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.621,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.60599999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.62,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.61899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":756,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:07:22.015 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:07:22.015 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:07:22.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.178+0000 7f34d9b36700 1 Processor -- start 2023-11-28T03:07:22.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.178+0000 7f34d9b36700 1 -- start start 2023-11-28T03:07:22.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 0x7f34d40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d40c6190 0x7f34d4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34d4064350 0x7f34d4158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d40c72d0 con 0x7f34d4064350 2023-11-28T03:07:22.182 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34d4154bc0 con 0x7f34d4098d10 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d9b36700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40c66c0 con 0x7f34d40c6190 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34d4064350 0x7f34d4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f34d4064350 0x7f34d4158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41838/0 (socket says 172.21.15.73:41838) 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.179+0000 7f34d3fff700 1 -- 172.21.15.73:0/2859286477 learned_addr learned my addr 172.21.15.73:0/2859286477 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d2ffd700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d40c6190 0x7f34d4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d37fe700 1 --2- 172.21.15.73:0/2859286477 >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 0x7f34d40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 -- 172.21.15.73:0/2859286477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d40c6190 msgr2=0x7f34d4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d40c6190 0x7f34d4063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 -- 172.21.15.73:0/2859286477 >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 msgr2=0x7f34d40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 --2- 172.21.15.73:0/2859286477 >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 0x7f34d40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 -- 172.21.15.73:0/2859286477 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34d40c69a0 con 0x7f34d4064350 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d37fe700 1 --2- 172.21.15.73:0/2859286477 >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 0x7f34d40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d3fff700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d4064350 0x7f34d4158320 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f34d4083f80 tx=0x7f34cc009840).ready entity=mon.0 client_cookie=227fdaae65dce6fb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d27fc700 1 -- 172.21.15.73:0/2859286477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00e070 con 0x7f34d4064350 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d27fc700 1 -- 172.21.15.73:0/2859286477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc00ba50 con 0x7f34d4064350 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d27fc700 1 -- 172.21.15.73:0/2859286477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc00bdd0 con 0x7f34d4064350 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 -- 172.21.15.73:0/2859286477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d4064350 msgr2=0x7f34d4158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d4064350 0x7f34d4158320 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 rx=0x7f34d4083f80 tx=0x7f34cc009840).stop 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 -- 172.21.15.73:0/2859286477 shutdown_connections 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d4064350 0x7f34d4158320 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 --2- 172.21.15.73:0/2859286477 >> v2:172.21.15.150:3300/0 conn(0x7f34d4098d10 0x7f34d40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 --2- 172.21.15.73:0/2859286477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d40c6190 0x7f34d4063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 -- 172.21.15.73:0/2859286477 >> 172.21.15.73:0/2859286477 conn(0x7f34d40cb310 msgr2=0x7f34d40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.180+0000 7f34d9b36700 1 -- 172.21.15.73:0/2859286477 shutdown_connections 2023-11-28T03:07:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.181+0000 7f34d9b36700 1 -- 172.21.15.73:0/2859286477 wait complete. 2023-11-28T03:07:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 Processor -- start 2023-11-28T03:07:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 -- start start 2023-11-28T03:07:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d4064350 0x7f34d4139ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 0x7f34d40ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40c2830 con 0x7f34d40c6190 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34d40aeca0 con 0x7f34d4098d10 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.182+0000 7f34d9b36700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34d4156410 con 0x7f34d4064350 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41850/0 (socket says 172.21.15.73:41850) 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 -- 172.21.15.73:0/1313833307 learned_addr learned my addr 172.21.15.73:0/1313833307 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d37fe700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d4064350 0x7f34d4139ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d2ffd700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 0x7f34d40ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 -- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d4064350 msgr2=0x7f34d4139ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d4064350 0x7f34d4139ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 -- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 msgr2=0x7f34d40ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 0x7f34d40ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 -- 172.21.15.73:0/1313833307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34cc009ae0 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d2ffd700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 0x7f34d40ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d3fff700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f34cc009fd0 tx=0x7f34cc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc013e60 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34cc01e470 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34d41566f0 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34cc01e830 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.183+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34d4156cb0 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.184+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34d406c860 con 0x7f34d40c6190 2023-11-28T03:07:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.184+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f34cc003040 con 0x7f34d40c6190 2023-11-28T03:07:22.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.185+0000 7f34bb7fe700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 0x7f34c00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.185+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f34cc07b930 con 0x7f34d40c6190 2023-11-28T03:07:22.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.185+0000 7f34d37fe700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 0x7f34c00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.185+0000 7f34d37fe700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 0x7f34c00615e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f34c400abb0 tx=0x7f34c4008040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.189+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f34cc04cc00 con 0x7f34d40c6190 2023-11-28T03:07:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.418+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f34d40c6c80 con 0x7f34d40c6190 2023-11-28T03:07:22.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.421+0000 7f34bb7fe700 1 -- 172.21.15.73:0/1313833307 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7f34cc023030 con 0x7f34d40c6190 2023-11-28T03:07:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 msgr2=0x7f34c00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 0x7f34c00615e0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f34c400abb0 tx=0x7f34c4008040).stop 2023-11-28T03:07:22.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 msgr2=0x7f34d40ae7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 rx=0x7f34cc009fd0 tx=0x7f34cc009840).stop 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 shutdown_connections 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34c005f120 0x7f34c00615e0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34d40c6190 0x7f34d40ae7b0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34d4064350 0x7f34d4139ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 --2- 172.21.15.73:0/1313833307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34d4098d10 0x7f34d40ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 >> 172.21.15.73:0/1313833307 conn(0x7f34d40cb310 msgr2=0x7f34d41487d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 shutdown_connections 2023-11-28T03:07:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.423+0000 7f34d9b36700 1 -- 172.21.15.73:0/1313833307 wait complete. 2023-11-28T03:07:22.427 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:07:22.436 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:22.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.548+0000 7f21ecc09700 1 Processor -- start 2023-11-28T03:07:22.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 -- start start 2023-11-28T03:07:22.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21e81544e0 0x7f21e81548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21e806a610 0x7f21e80c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e80c69a0 con 0x7f21e80c6e30 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80c13a0 con 0x7f21e81544e0 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.549+0000 7f21ecc09700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e80c1680 con 0x7f21e806a610 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21e81544e0 0x7f21e81548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e6d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21e80c6e30 0x7f21e80a8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44492/0 (socket says 172.21.15.73:44492) 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21e81544e0 0x7f21e81548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41852/0 (socket says 172.21.15.73:41852) 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e6d9d700 1 -- 172.21.15.73:0/759004037 learned_addr learned my addr 172.21.15.73:0/759004037 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e5d9b700 1 --2- 172.21.15.73:0/759004037 >> v2:172.21.15.150:3300/0 conn(0x7f21e806a610 0x7f21e80c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 -- 172.21.15.73:0/759004037 >> v2:172.21.15.150:3300/0 conn(0x7f21e806a610 msgr2=0x7f21e80c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 --2- 172.21.15.73:0/759004037 >> v2:172.21.15.150:3300/0 conn(0x7f21e806a610 0x7f21e80c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 -- 172.21.15.73:0/759004037 >> v1:172.21.15.73:6789/0 conn(0x7f21e80c6e30 legacy=0x7f21e80a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e559a700 1 -- 172.21.15.73:0/759004037 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390470988 0 0) 0x7f21e80c69a0 con 0x7f21e80c6e30 2023-11-28T03:07:22.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 -- 172.21.15.73:0/759004037 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e80c6630 con 0x7f21e81544e0 2023-11-28T03:07:22.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e659c700 1 --2- 172.21.15.73:0/759004037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e81548c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f21e8084fb0 tx=0x7f21e0009780).ready entity=mon.0 client_cookie=f57e6719254f662d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e559a700 1 -- 172.21.15.73:0/759004037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e000e070 con 0x7f21e81544e0 2023-11-28T03:07:22.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e559a700 1 -- 172.21.15.73:0/759004037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21e000bc30 con 0x7f21e81544e0 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.550+0000 7f21e559a700 1 -- 172.21.15.73:0/759004037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21e0012410 con 0x7f21e81544e0 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 -- 172.21.15.73:0/759004037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 msgr2=0x7f21e81548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 --2- 172.21.15.73:0/759004037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e81548c0 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 rx=0x7f21e8084fb0 tx=0x7f21e0009780).stop 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 -- 172.21.15.73:0/759004037 shutdown_connections 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 --2- 172.21.15.73:0/759004037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e81548c0 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 --2- 172.21.15.73:0/759004037 >> v2:172.21.15.150:3300/0 conn(0x7f21e806a610 0x7f21e80c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 -- 172.21.15.73:0/759004037 >> 172.21.15.73:0/759004037 conn(0x7f21e8085020 msgr2=0x7f21e8094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 -- 172.21.15.73:0/759004037 shutdown_connections 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.551+0000 7f21ecc09700 1 -- 172.21.15.73:0/759004037 wait complete. 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.552+0000 7f21ecc09700 1 Processor -- start 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.552+0000 7f21ecc09700 1 -- start start 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 0x7f21e8152c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c6e30 0x7f21e814fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80964c0 con 0x7f21e81544e0 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e8150280 con 0x7f21e806a610 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e80958b0 con 0x7f21e80c6e30 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41856/0 (socket says 172.21.15.73:41856) 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 -- 172.21.15.73:0/839846473 learned_addr learned my addr 172.21.15.73:0/839846473 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e659c700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 0x7f21e8152c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e5d9b700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c6e30 0x7f21e814fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 -- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c6e30 msgr2=0x7f21e814fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c6e30 0x7f21e814fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 -- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 msgr2=0x7f21e8152c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 0x7f21e8152c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 -- 172.21.15.73:0/839846473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e0009a20 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e659c700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 0x7f21e8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21e6d9d700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f21d800c490 tx=0x7f21d8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80072e0 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d8007440 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e8095b90 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d80057a0 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.553+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e8096150 con 0x7f21e81544e0 2023-11-28T03:07:22.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.554+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e8096670 con 0x7f21e81544e0 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.555+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f21d8005b90 con 0x7f21e81544e0 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.555+0000 7f21de7fc700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.555+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f21d8079ac0 con 0x7f21e81544e0 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.555+0000 7f21e659c700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.555+0000 7f21e659c700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f21e0003380 tx=0x7f21e0009780).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:22.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.559+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21e8096670 con 0x7f21e81544e0 2023-11-28T03:07:22.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:07:22.721 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:07:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.769+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f21e8096600 con 0x7f21e81544e0 2023-11-28T03:07:22.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.770+0000 7f21de7fc700 1 -- 172.21.15.73:0/839846473 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f21d8079e60 con 0x7f21e81544e0 2023-11-28T03:07:22.771 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:22.771 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 msgr2=0x7f21c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f21e0003380 tx=0x7f21e0009780).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 msgr2=0x7f21e80950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 rx=0x7f21d800c490 tx=0x7f21d8007c10).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 shutdown_connections 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21c805f150 0x7f21c8061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e81544e0 0x7f21e80950a0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e806a610 0x7f21e8152c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 --2- 172.21.15.73:0/839846473 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e80c6e30 0x7f21e814fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 >> 172.21.15.73:0/839846473 conn(0x7f21e8085020 msgr2=0x7f21e8094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:22.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 shutdown_connections 2023-11-28T03:07:22.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:22.772+0000 7f21ecc09700 1 -- 172.21.15.73:0/839846473 wait complete. 2023-11-28T03:07:22.776 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:23.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:07:18.675550+0000', 'modified': '2023-11-28T03:07:20.681373+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6246}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6246': {'gid': 6246, 'name': 'a', 'rank': 0, 'incarnation': 24, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/2389605358', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2389605358}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 2389605358}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [10], 'metadata_pool': 9, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:07:23.187 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:07:23.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T03:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.307+0000 7feac91b5700 1 Processor -- start 2023-11-28T03:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.307+0000 7feac91b5700 1 -- start start 2023-11-28T03:07:23.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 0x7feac40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feac40c6190 0x7feac4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 0x7feac4158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac40c72d0 con 0x7feac40c6190 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feac4154bc0 con 0x7feac4098d10 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac91b5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40c66c0 con 0x7feac4064350 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feac40c6190 0x7feac4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feac40c6190 0x7feac4063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41858/0 (socket says 172.21.15.73:41858) 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 -- 172.21.15.73:0/2037771951 learned_addr learned my addr 172.21.15.73:0/2037771951 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac359e700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 0x7feac4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 -- 172.21.15.73:0/2037771951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 msgr2=0x7feac4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2037771951 >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 0x7feac40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 0x7feac4158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 -- 172.21.15.73:0/2037771951 >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 msgr2=0x7feac40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 --2- 172.21.15.73:0/2037771951 >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 0x7feac40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 -- 172.21.15.73:0/2037771951 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac40c69a0 con 0x7feac40c6190 2023-11-28T03:07:23.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2037771951 >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 0x7feac40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac359e700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 0x7feac4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac259c700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac40c6190 0x7feac4063e60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7feac4083f80 tx=0x7feab80090f0).ready entity=mon.0 client_cookie=d2e258e6c8f15dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac1d9b700 1 -- 172.21.15.73:0/2037771951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800b600 con 0x7feac40c6190 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac1d9b700 1 -- 172.21.15.73:0/2037771951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab800b760 con 0x7feac40c6190 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.308+0000 7feac1d9b700 1 -- 172.21.15.73:0/2037771951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab800bae0 con 0x7feac40c6190 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 -- 172.21.15.73:0/2037771951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac40c6190 msgr2=0x7feac4063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac40c6190 0x7feac4063e60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 rx=0x7feac4083f80 tx=0x7feab80090f0).stop 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 -- 172.21.15.73:0/2037771951 shutdown_connections 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4064350 0x7feac4158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 --2- 172.21.15.73:0/2037771951 >> v2:172.21.15.150:3300/0 conn(0x7feac4098d10 0x7feac40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 --2- 172.21.15.73:0/2037771951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac40c6190 0x7feac4063e60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 -- 172.21.15.73:0/2037771951 >> 172.21.15.73:0/2037771951 conn(0x7feac40cb310 msgr2=0x7feac40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 -- 172.21.15.73:0/2037771951 shutdown_connections 2023-11-28T03:07:23.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.309+0000 7feac91b5700 1 -- 172.21.15.73:0/2037771951 wait complete. 2023-11-28T03:07:23.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.310+0000 7feac91b5700 1 Processor -- start 2023-11-28T03:07:23.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.310+0000 7feac91b5700 1 -- start start 2023-11-28T03:07:23.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4098d10 0x7feac40ad530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 0x7feac4150820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40aa1c0 con 0x7feac4064350 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feac40a8a20 con 0x7feac40c6190 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac91b5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feac40a8d00 con 0x7feac4098d10 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41872/0 (socket says 172.21.15.73:41872) 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 -- 172.21.15.73:0/1658903794 learned_addr learned my addr 172.21.15.73:0/1658903794 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 -- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4098d10 msgr2=0x7feac40ad530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4098d10 0x7feac40ad530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac359e700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 0x7feac4150820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 -- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 msgr2=0x7feac4150820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 0x7feac4150820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 -- 172.21.15.73:0/1658903794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feab8009370 con 0x7feac4064350 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac359e700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 0x7feac4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.311+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7feab400c880 tx=0x7feab4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.312+0000 7feaaaffd700 1 -- 172.21.15.73:0/1658903794 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4010580 con 0x7feac4064350 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.312+0000 7feaaaffd700 1 -- 172.21.15.73:0/1658903794 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab40106e0 con 0x7feac4064350 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.312+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feac40a8fe0 con 0x7feac4064350 2023-11-28T03:07:23.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.312+0000 7feaaaffd700 1 -- 172.21.15.73:0/1658903794 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4010a60 con 0x7feac4064350 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.312+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40ada20 con 0x7feac4064350 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.313+0000 7feaaaffd700 1 -- 172.21.15.73:0/1658903794 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feab401b460 con 0x7feac4064350 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.313+0000 7feaaaffd700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 0x7feab00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.313+0000 7feaaaffd700 1 -- 172.21.15.73:0/1658903794 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feab407a260 con 0x7feac4064350 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.313+0000 7feac259c700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 0x7feab00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.314+0000 7feac259c700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 0x7feab00615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7feac4083f80 tx=0x7feab8017000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.315+0000 7feadab4db80 1 Processor -- start 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 -- start start 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a258d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a230cb0 0x55b86a22b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a1f6060 con 0x55b86a279ea0 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a16d1f0 con 0x55b86a230cb0 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a182440 con 0x55b869fece40 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.316+0000 7feac259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41884/0 (socket says 172.21.15.73:41884) 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 -- 172.21.15.73:0/1260640158 learned_addr learned my addr 172.21.15.73:0/1260640158 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a258d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac359e700 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a230cb0 0x55b86a22b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 -- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 msgr2=0x55b86a258d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a258d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 -- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a230cb0 msgr2=0x55b86a22b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a230cb0 0x55b86a22b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 -- 172.21.15.73:0/1260640158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86a17b620 con 0x55b86a279ea0 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feac259c700 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7feac40c2360 tx=0x7feab801d000).ready entity=mon.0 client_cookie=5ef8e1b1cf70bb78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feaa3fff700 1 -- 172.21.15.73:0/1260640158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab801c880 con 0x55b86a279ea0 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feaa3fff700 1 -- 172.21.15.73:0/1260640158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab801c9e0 con 0x55b86a279ea0 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.317+0000 7feaa3fff700 1 -- 172.21.15.73:0/1260640158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab801cd60 con 0x55b86a279ea0 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- 172.21.15.73:0/1260640158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 msgr2=0x55b86a2307c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 rx=0x7feac40c2360 tx=0x7feab801d000).stop 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- 172.21.15.73:0/1260640158 shutdown_connections 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a230cb0 0x55b86a22b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a258d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 --2- 172.21.15.73:0/1260640158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a279ea0 0x55b86a2307c0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- 172.21.15.73:0/1260640158 >> 172.21.15.73:0/1260640158 conn(0x55b86a09a4a0 msgr2=0x55b86a279790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- 172.21.15.73:0/1260640158 shutdown_connections 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- 172.21.15.73:0/1260640158 wait complete. 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 -- start start 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.318+0000 7feadab4db80 1 Processor -- start 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a288070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a288560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a182580 con 0x55b869fece40 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a182800 con 0x55b86a279ea0 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a14de10 con 0x55b86a230cb0 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feadab4db80 10 client.0 ll_register_callbacks cb 0x55b86a1f0460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.319+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41894/0 (socket says 172.21.15.73:41894) 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 -- 172.21.15.73:0/2532126073 learned_addr learned my addr 172.21.15.73:0/2532126073 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac259c700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a288070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac359e700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a288560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 -- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 msgr2=0x55b86a288070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a288070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 -- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 msgr2=0x55b86a288560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a288560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 -- 172.21.15.73:0/2532126073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feac40c69a0 con 0x55b869fece40 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7feab4078910 tx=0x7feab401d040).ready entity=mon.0 client_cookie=257dbe90aeddb106 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40548d0 con 0x55b869fece40 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feab404e680 con 0x55b869fece40 2023-11-28T03:07:23.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b86a224720 con 0x55b869fece40 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab4018b80 con 0x55b869fece40 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feadab4db80 10 client.6396 fetch_fsmap learned FSMap version 26 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55b86a1769c0 con 0x55b869fece40 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b86a224720 con 0x55b869fece40 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.320+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feab4018ce0 con 0x55b869fece40 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 10 client.6396 fetch_fsmap finished waiting for FSMap version 26 2023-11-28T03:07:23.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 10 client.6396 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 20 client.6396 populate_metadata read hostname 'smithi073' 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2879c0 0x55b86a25e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 4 client.6396 mds_command: new command op to 6246 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 --> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b869fe8ca0 con 0x55b86a2879c0 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feac259c700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2879c0 0x55b86a25e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feaa17fa700 10 client.6396 ms_handle_connect on v2:172.21.15.73:6834/2389605358 2023-11-28T03:07:23.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feac259c700 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2879c0 0x55b86a25e2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b1a723f8e1bb9465 server_cookie=a2fa60703bbbe4bd in_seq=0 out_seq=0 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feaa17fa700 1 -- 172.21.15.73:0/2532126073 <== mds.0 v2:172.21.15.73:6834/2389605358 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55b869fe8ca0 con 0x55b86a2879c0 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.321+0000 7feaa17fa700 10 client.6396 handle_command_reply: tid=0 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 client.6396 shutdown 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 msgr2=0x55b86a25ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 rx=0x7feab4078910 tx=0x7feab401d040).stop 2023-11-28T03:07:23.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 shutdown_connections 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2879c0 0x55b86a25e2c0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a288560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a25ae90 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 --2- 172.21.15.73:0/2532126073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a288070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feaa17fa700 0 client.6396 ms_handle_reset on v2:172.21.15.73:6834/2389605358 2023-11-28T03:07:23.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 >> 172.21.15.73:0/2532126073 conn(0x55b86a09a4a0 msgr2=0x55b86a231a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 shutdown_connections 2023-11-28T03:07:23.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 1 -- 172.21.15.73:0/2532126073 wait complete. 2023-11-28T03:07:23.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.322+0000 7feadab4db80 20 client.6396 trim_cache size 0 max 16384 2023-11-28T03:07:23.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.338+0000 7feadab4db80 1 Processor -- start 2023-11-28T03:07:23.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.338+0000 7feadab4db80 1 -- start start 2023-11-28T03:07:23.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a272e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a275090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a273370 con 0x55b869fece40 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a22ffd0 con 0x55b86a279ea0 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a25c610 con 0x55b86a230cb0 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49244/0 (socket says 172.21.15.73:49244) 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 -- 172.21.15.73:0/2539972042 learned_addr learned my addr 172.21.15.73:0/2539972042 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac359e700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a275090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac259c700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a272e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 -- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 msgr2=0x55b86a272e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a272e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 -- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 msgr2=0x55b86a275090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a275090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac2d9d700 1 -- 172.21.15.73:0/2539972042 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b86a224340 con 0x55b869fece40 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.339+0000 7feac259c700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a272e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feac2d9d700 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7feab400c7a0 tx=0x7feab4056000).ready entity=mon.0 client_cookie=88234c50ea43d99a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feaa8bcf700 1 -- 172.21.15.73:0/2539972042 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab40548d0 con 0x55b869fece40 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feaa8bcf700 1 -- 172.21.15.73:0/2539972042 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab404eaf0 con 0x55b869fece40 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feaa8bcf700 1 -- 172.21.15.73:0/2539972042 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab4018d90 con 0x55b869fece40 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 -- 172.21.15.73:0/2539972042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 msgr2=0x55b86a272990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 rx=0x7feab400c7a0 tx=0x7feab4056000).stop 2023-11-28T03:07:23.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 -- 172.21.15.73:0/2539972042 shutdown_connections 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a275090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b869fece40 0x55b86a272990 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 --2- 172.21.15.73:0/2539972042 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b86a230cb0 0x55b86a272e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 -- 172.21.15.73:0/2539972042 >> 172.21.15.73:0/2539972042 conn(0x55b86a09a4a0 msgr2=0x55b86a270230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 -- 172.21.15.73:0/2539972042 shutdown_connections 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.340+0000 7feadab4db80 1 -- 172.21.15.73:0/2539972042 wait complete. 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.341+0000 7feadab4db80 1 -- start start 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.341+0000 7feadab4db80 1 Processor -- start 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a22c690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a272e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a273370 con 0x55b86a230cb0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b86a2de550 con 0x55b86a279ea0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b86a25d630 con 0x55b869fece40 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feadab4db80 10 client.0 ll_register_callbacks cb 0x55b86a1f0460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49252/0 (socket says 172.21.15.73:49252) 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 -- 172.21.15.73:0/1873785181 learned_addr learned my addr 172.21.15.73:0/1873785181 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac359e700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a272e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a22c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 -- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 msgr2=0x55b86a22c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a22c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 -- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 msgr2=0x55b86a272e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a272e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 -- 172.21.15.73:0/1873785181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b86a1769c0 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feac259c700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x55b86a22de10 tx=0x7feab801e040).ready entity=mon.0 client_cookie=ca0da2fce25bf3e1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feaa17fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.342+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feab8006d90 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feab800bcc0 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7feab801c9e0 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7feab80368a0 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 10 client.6402 fetch_fsmap learned FSMap version 26 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55b86a230b20 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feab801c9e0 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 26) v1 ==== 3100+0+0 (secure 0 0 0) 0x7feab8024070 con 0x55b86a230cb0 2023-11-28T03:07:23.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 10 client.6402 fetch_fsmap finished waiting for FSMap version 26 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 10 client.6402 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 20 client.6402 populate_metadata read hostname 'smithi073' 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2dd9d0 0x55b86a2dddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 4 client.6402 mds_command: new command op to 6246 tid=0[{"prefix": "status"}] 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 --> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b86a01fca0 con 0x55b86a2dd9d0 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2dd9d0 0x55b86a2dddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feaa17fa700 10 client.6402 ms_handle_connect on v2:172.21.15.73:6834/2389605358 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.343+0000 7feac2d9d700 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2dd9d0 0x55b86a2dddb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4479cd2b9f33dde9 server_cookie=8d1b58b2484d15bc in_seq=0 out_seq=0 2023-11-28T03:07:23.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feaa17fa700 1 -- 172.21.15.73:0/1873785181 <== mds.0 v2:172.21.15.73:6834/2389605358 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55b86a01fca0 con 0x55b86a2dd9d0 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feaa17fa700 10 client.6402 handle_command_reply: tid=0 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 client.6402 shutdown 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 msgr2=0x55b86a272990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 rx=0x55b86a22de10 tx=0x7feab801e040).stop 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 shutdown_connections 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:6834/2389605358,v1:172.21.15.73:6837/2389605358] conn(0x55b86a2dd9d0 0x55b86a2dddb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b86a279ea0 0x55b86a272e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b869fece40 0x55b86a22c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feaa17fa700 0 client.6402 ms_handle_reset on v2:172.21.15.73:6834/2389605358 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 --2- 172.21.15.73:0/1873785181 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b86a230cb0 0x55b86a272990 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 >> 172.21.15.73:0/1873785181 conn(0x55b86a09a4a0 msgr2=0x55b86a2594a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 shutdown_connections 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 1 -- 172.21.15.73:0/1873785181 wait complete. 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feadab4db80 20 client.6402 trim_cache size 0 max 16384 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:07:23.358 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 6246, 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 4.6676783510000002, 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 26, 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 82, 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 82, 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 9.1938660700000003 2023-11-28T03:07:23.359 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:07:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 msgr2=0x7feab00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 0x7feab00615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7feac4083f80 tx=0x7feab8017000).stop 2023-11-28T03:07:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 msgr2=0x7feac4155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 rx=0x7feab400c880 tx=0x7feab4008040).stop 2023-11-28T03:07:23.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 shutdown_connections 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feab005f120 0x7feab00615e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feac40c6190 0x7feac4150820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feac4064350 0x7feac4155fe0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 --2- 172.21.15.73:0/1658903794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feac4098d10 0x7feac40ad530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 >> 172.21.15.73:0/1658903794 conn(0x7feac40cb310 msgr2=0x7feac40ca780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.344+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 shutdown_connections 2023-11-28T03:07:23.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.345+0000 7feac91b5700 1 -- 172.21.15.73:0/1658903794 wait complete. 2023-11-28T03:07:23.363 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:07:23.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.462+0000 7f06cfd67700 1 Processor -- start 2023-11-28T03:07:23.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.462+0000 7f06cfd67700 1 -- start start 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cfd67700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f06c81542f0 0x7f06c80c8860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cfd67700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 0x7f06c80c0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cfd67700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c80c6910 con 0x7f06c80a7fc0 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cfd67700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c80c18f0 con 0x7f06c81542f0 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cfd67700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c80c1330 con 0x7f06c81546d0 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f06c81542f0 0x7f06c80c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f06c81542f0 0x7f06c80c8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49258/0 (socket says 172.21.15.73:49258) 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 -- 172.21.15.73:0/2083235656 learned_addr learned my addr 172.21.15.73:0/2083235656 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cd302700 1 --2- 172.21.15.73:0/2083235656 >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 0x7f06c80c0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 -- 172.21.15.73:0/2083235656 >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 msgr2=0x7f06c80c0ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 --2- 172.21.15.73:0/2083235656 >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 0x7f06c80c0ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 -- 172.21.15.73:0/2083235656 >> v1:172.21.15.150:6789/0 conn(0x7f06c80a7fc0 legacy=0x7f06c80c6da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 -- 172.21.15.73:0/2083235656 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c80c1610 con 0x7f06c81542f0 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cd302700 1 --2- 172.21.15.73:0/2083235656 >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 0x7f06c80c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06cdb03700 1 --2- 172.21.15.73:0/2083235656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c81542f0 0x7f06c80c8860 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f06c8159160 tx=0x7f06c4009060).ready entity=mon.0 client_cookie=34588f5672f36b7f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06ccb01700 1 -- 172.21.15.73:0/2083235656 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4005880 con 0x7f06c81542f0 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06ccb01700 1 -- 172.21.15.73:0/2083235656 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c40059e0 con 0x7f06c81542f0 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.463+0000 7f06ccb01700 1 -- 172.21.15.73:0/2083235656 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c4005d60 con 0x7f06c81542f0 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 -- 172.21.15.73:0/2083235656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c81542f0 msgr2=0x7f06c80c8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 --2- 172.21.15.73:0/2083235656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c81542f0 0x7f06c80c8860 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 rx=0x7f06c8159160 tx=0x7f06c4009060).stop 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 -- 172.21.15.73:0/2083235656 shutdown_connections 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 --2- 172.21.15.73:0/2083235656 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c81542f0 0x7f06c80c8860 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 --2- 172.21.15.73:0/2083235656 >> v2:172.21.15.150:3300/0 conn(0x7f06c81546d0 0x7f06c80c0ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 -- 172.21.15.73:0/2083235656 >> 172.21.15.73:0/2083235656 conn(0x7f06c8094900 msgr2=0x7f06c806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 -- 172.21.15.73:0/2083235656 shutdown_connections 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.464+0000 7f06cfd67700 1 -- 172.21.15.73:0/2083235656 wait complete. 2023-11-28T03:07:23.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.465+0000 7f06cfd67700 1 Processor -- start 2023-11-28T03:07:23.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.465+0000 7f06cfd67700 1 -- start start 2023-11-28T03:07:23.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c81542f0 0x7f06c8150f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06c81546d0 0x7f06c80b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8098900 con 0x7f06c80a7fc0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8097cf0 con 0x7f06c81542f0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cfd67700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c8097fd0 con 0x7f06c81546d0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49268/0 (socket says 172.21.15.73:49268) 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 -- 172.21.15.73:0/557515759 learned_addr learned my addr 172.21.15.73:0/557515759 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cd302700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c81542f0 0x7f06c8150f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06ce304700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06c81546d0 0x7f06c80b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 -- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06c81546d0 msgr2=0x7f06c80b0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06c81546d0 0x7f06c80b0230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 -- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c81542f0 msgr2=0x7f06c8150f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c81542f0 0x7f06c8150f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 -- 172.21.15.73:0/557515759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c4009300 con 0x7f06c80a7fc0 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.466+0000 7f06cdb03700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f06c8060a30 tx=0x7f06c4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.467+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c40067e0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.467+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06c400b810 con 0x7f06c80a7fc0 2023-11-28T03:07:23.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.467+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c80982b0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.467+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06c400bbb0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.467+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c8164530 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.468+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06c80475e0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.468+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f06c4018030 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.468+0000 7f06bdffb700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 0x7f06ac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.469+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f06c407b2f0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.469+0000 7f06cd302700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 0x7f06ac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:23.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.469+0000 7f06cd302700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 0x7f06ac061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f06c8152cc0 tx=0x7f06b8008040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:23.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.473+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f06c404c4a0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.679+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f06c80c65a0 con 0x7f06c80a7fc0 2023-11-28T03:07:23.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.680+0000 7f06bdffb700 1 -- 172.21.15.73:0/557515759 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+73 (secure 0 0 0) 0x7f06c4028020 con 0x7f06c80a7fc0 2023-11-28T03:07:23.681 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:07:23.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.681+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 msgr2=0x7f06ac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 0x7f06ac061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f06c8152cc0 tx=0x7f06b8008040).stop 2023-11-28T03:07:23.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 msgr2=0x7f06c80c9a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:23.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 rx=0x7f06c8060a30 tx=0x7f06c4016000).stop 2023-11-28T03:07:23.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 shutdown_connections 2023-11-28T03:07:23.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06ac05f150 0x7f06ac061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f06c81546d0 0x7f06c80b0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f06c80a7fc0 0x7f06c80c9a50 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 --2- 172.21.15.73:0/557515759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f06c81542f0 0x7f06c8150f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:23.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 >> 172.21.15.73:0/557515759 conn(0x7f06c8094900 msgr2=0x7f06c80640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:23.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 shutdown_connections 2023-11-28T03:07:23.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:23.682+0000 7f06cfd67700 1 -- 172.21.15.73:0/557515759 wait complete. 2023-11-28T03:07:23.694 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:07:23.694 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:07:23.694 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:07:23.695 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:07:23.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.695 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.711 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:07:23.712 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41636sec preferred_lft 41636sec 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.713 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.714 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:07:23.715 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:23.716 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.716 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:07:23.774 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:07:23.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.776 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:07:23.831 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:07:23.831 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:07:23.832 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:07:23.832 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:07:23.832 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:07:23.832 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.832 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:07:23.832 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:07:23.832 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:07:23.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:07:23.833 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:07:23.833 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:07:23.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.932 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:07:23.947 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:07:23.947 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:07:23.947 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:07:23.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:23.948 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:07:24.012 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:24.013 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:24.067 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:07:24.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:24.146 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:24.147 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:07:24.171 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:24.171 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:07:24.173 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:07:24.173 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:24.173 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:24.237 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:07:24.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.390+0000 7f21dc3ee700 1 Processor -- start 2023-11-28T03:07:24.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.390+0000 7f21dc3ee700 1 -- start start 2023-11-28T03:07:24.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21dc3ee700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21d40c5ac0 0x7f21d40c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21dc3ee700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21d40c5ea0 0x7f21d4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21dc3ee700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21d4155f20 con 0x7f21d4153390 2023-11-28T03:07:24.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21dc3ee700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d4153990 con 0x7f21d40c5ea0 2023-11-28T03:07:24.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21dc3ee700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21d40c62d0 con 0x7f21d40c5ac0 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21da98b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21d40c5ea0 0x7f21d4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21da98b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21d40c5ea0 0x7f21d4066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49284/0 (socket says 172.21.15.73:49284) 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21da98b700 1 -- 172.21.15.73:0/2724909732 learned_addr learned my addr 172.21.15.73:0/2724909732 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21d9989700 1 --1- 172.21.15.73:0/2724909732 >> v1:172.21.15.73:6789/0 conn(0x7f21d4153390 0x7f21d4156690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46272/0 (socket says 172.21.15.73:46272) 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.391+0000 7f21da18a700 1 --2- 172.21.15.73:0/2724909732 >> v2:172.21.15.150:3300/0 conn(0x7f21d40c5ac0 0x7f21d40c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21da98b700 1 -- 172.21.15.73:0/2724909732 >> v2:172.21.15.150:3300/0 conn(0x7f21d40c5ac0 msgr2=0x7f21d40c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:24.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21da98b700 1 --2- 172.21.15.73:0/2724909732 >> v2:172.21.15.150:3300/0 conn(0x7f21d40c5ac0 0x7f21d40c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21da98b700 1 -- 172.21.15.73:0/2724909732 >> v1:172.21.15.73:6789/0 conn(0x7f21d4153390 legacy=0x7f21d4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21d9188700 1 -- 172.21.15.73:0/2724909732 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1163569715 0 0) 0x7f21d4155f20 con 0x7f21d4153390 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21da98b700 1 -- 172.21.15.73:0/2724909732 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4156200 con 0x7f21d40c5ea0 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21da98b700 1 --2- 172.21.15.73:0/2724909732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d4066a70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f21d40c9050 tx=0x7f21d00090f0).ready entity=mon.0 client_cookie=863e6b143ed88506 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21d9188700 1 -- 172.21.15.73:0/2724909732 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000b520 con 0x7f21d40c5ea0 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21d9188700 1 -- 172.21.15.73:0/2724909732 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d000b680 con 0x7f21d40c5ea0 2023-11-28T03:07:24.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21d9188700 1 -- 172.21.15.73:0/2724909732 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d000ba20 con 0x7f21d40c5ea0 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 -- 172.21.15.73:0/2724909732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 msgr2=0x7f21d4066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/2724909732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d4066a70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 rx=0x7f21d40c9050 tx=0x7f21d00090f0).stop 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 -- 172.21.15.73:0/2724909732 shutdown_connections 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/2724909732 >> v2:172.21.15.150:3300/0 conn(0x7f21d40c5ac0 0x7f21d40c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/2724909732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d4066a70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 -- 172.21.15.73:0/2724909732 >> 172.21.15.73:0/2724909732 conn(0x7f21d4062700 msgr2=0x7f21d4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:24.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.392+0000 7f21dc3ee700 1 -- 172.21.15.73:0/2724909732 shutdown_connections 2023-11-28T03:07:24.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.393+0000 7f21dc3ee700 1 -- 172.21.15.73:0/2724909732 wait complete. 2023-11-28T03:07:24.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 Processor -- start 2023-11-28T03:07:24.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 -- start start 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 0x7f21d40ae8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4153390 0x7f21d40c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d414e940 con 0x7f21d40c5ea0 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4162320 con 0x7f21d4153390 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21dc3ee700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21d4162600 con 0x7f21d40c5ac0 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21d9989700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21d9989700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49288/0 (socket says 172.21.15.73:49288) 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.394+0000 7f21d9989700 1 -- 172.21.15.73:0/1972616164 learned_addr learned my addr 172.21.15.73:0/1972616164 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21da98b700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4153390 0x7f21d40c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21da18a700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 0x7f21d40ae8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 -- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 msgr2=0x7f21d40ae8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 0x7f21d40ae8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 -- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4153390 msgr2=0x7f21d40c1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4153390 0x7f21d40c1a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 -- 172.21.15.73:0/1972616164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d0009370 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21da18a700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 0x7f21d40ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21d9989700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f21c400c490 tx=0x7f21c4007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c40072e0 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21c4007440 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21d41628e0 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21c4005930 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.395+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21d4162020 con 0x7f21d40c5ea0 2023-11-28T03:07:24.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.396+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21d4165630 con 0x7f21d40c5ea0 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.396+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f21c4005ce0 con 0x7f21d40c5ea0 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.397+0000 7f21ca7fc700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 0x7f21b8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.397+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f21c40799e0 con 0x7f21d40c5ea0 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.397+0000 7f21da18a700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 0x7f21b8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.397+0000 7f21da18a700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 0x7f21b8061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f21cc007780 tx=0x7f21cc00b040).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.401+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21d4165630 con 0x7f21d40c5ea0 2023-11-28T03:07:24.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.588+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f21d4165630 con 0x7f21b805f150 2023-11-28T03:07:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.590+0000 7f21ca7fc700 1 -- 172.21.15.73:0/1972616164 <== mgr.6060 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f21d4165630 con 0x7f21b805f150 2023-11-28T03:07:24.591 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:07:24.591 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:24.591 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:07:24.591 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:24.591 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 msgr2=0x7f21b8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 0x7f21b8061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f21cc007780 tx=0x7f21cc00b040).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 msgr2=0x7f21d40ad0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 rx=0x7f21c400c490 tx=0x7f21c4007de0).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 shutdown_connections 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21b805f150 0x7f21b8061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21d4153390 0x7f21d40c1a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21d40c5ac0 0x7f21d40ae8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 --2- 172.21.15.73:0/1972616164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21d40c5ea0 0x7f21d40ad0a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 >> 172.21.15.73:0/1972616164 conn(0x7f21d4062700 msgr2=0x7f21d40c78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:24.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.591+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 shutdown_connections 2023-11-28T03:07:24.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.592+0000 7f21dc3ee700 1 -- 172.21.15.73:0/1972616164 wait complete. 2023-11-28T03:07:24.598 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:24.597+0000 7f4ff3c45700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:07:24.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:07:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.720+0000 7faa47fff700 1 Processor -- start 2023-11-28T03:07:24.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.720+0000 7faa47fff700 1 -- start start 2023-11-28T03:07:24.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa47fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa4814e530 0x7faa480c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa47fff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa480c7bc0 con 0x7faa48154950 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa47fff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa48155920 con 0x7faa4814e530 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa47fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa48154e00 con 0x7faa48158570 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa477fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa46ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faa4814e530 0x7faa480c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa477fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:37044/0 (socket says 172.21.15.73:37044) 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa477fe700 1 -- 172.21.15.73:0/3290078182 learned_addr learned my addr 172.21.15.73:0/3290078182 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.721+0000 7faa3e7fc700 1 --1- 172.21.15.73:0/3290078182 >> v1:172.21.15.150:6789/0 conn(0x7faa48154950 0x7faa4814e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:52930/0 (socket says 172.21.15.73:52930) 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa46ffd700 1 -- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 msgr2=0x7faa48158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa46ffd700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa46ffd700 1 -- 172.21.15.73:0/3290078182 >> v1:172.21.15.150:6789/0 conn(0x7faa48154950 legacy=0x7faa4814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa46ffd700 1 -- 172.21.15.73:0/3290078182 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa48155080 con 0x7faa4814e530 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa467fc700 1 -- 172.21.15.73:0/3290078182 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2843801345 0 0) 0x7faa480c7bc0 con 0x7faa48154950 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa477fe700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa46ffd700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa480c66f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7faa38009760 tx=0x7faa38009060).ready entity=mon.1 client_cookie=d4f87229323fa771 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa467fc700 1 -- 172.21.15.73:0/3290078182 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800f510 con 0x7faa4814e530 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa467fc700 1 -- 172.21.15.73:0/3290078182 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3800f670 con 0x7faa4814e530 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa467fc700 1 -- 172.21.15.73:0/3290078182 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800f9f0 con 0x7faa4814e530 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa47fff700 1 -- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 msgr2=0x7faa480c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.722+0000 7faa47fff700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa480c66f0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 rx=0x7faa38009760 tx=0x7faa38009060).stop 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 -- 172.21.15.73:0/3290078182 shutdown_connections 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 --2- 172.21.15.73:0/3290078182 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa480c66f0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 -- 172.21.15.73:0/3290078182 >> 172.21.15.73:0/3290078182 conn(0x7faa48064990 msgr2=0x7faa48064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:24.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 -- 172.21.15.73:0/3290078182 shutdown_connections 2023-11-28T03:07:24.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.723+0000 7faa47fff700 1 -- 172.21.15.73:0/3290078182 wait complete. 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.724+0000 7faa47fff700 1 Processor -- start 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.724+0000 7faa47fff700 1 -- start start 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa4814dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa4814e180 con 0x7faa48154950 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa481656a0 con 0x7faa4814e530 2023-11-28T03:07:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa480c2490 con 0x7faa48158570 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49294/0 (socket says 172.21.15.73:49294) 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 -- 172.21.15.73:0/1052574516 learned_addr learned my addr 172.21.15.73:0/1052574516 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa46ffd700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa4814dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa477fe700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 -- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 msgr2=0x7faa48165940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48165940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 -- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 msgr2=0x7faa4814dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa4814dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 -- 172.21.15.73:0/1052574516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa40009ae0 con 0x7faa48154950 2023-11-28T03:07:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3e7fc700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7faa3000c560 tx=0x7faa30007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300072e0 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa30007440 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.725+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa380092a0 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.726+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa300058f0 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.726+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa480c2a50 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.726+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa48062ee0 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.727+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 24) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faa30005ce0 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.727+0000 7faa3f7fe700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 0x7faa28061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.727+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7faa30079a20 con 0x7faa48154950 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.727+0000 7faa46ffd700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 0x7faa28061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.728+0000 7faa46ffd700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 0x7faa28061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa480c4420 tx=0x7faa38017000).ready entity=mgr.6060 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.731+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faa3004acf0 con 0x7faa48154950 2023-11-28T03:07:24.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:24.939+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7faa4806c7c0 con 0x7faa48154950 2023-11-28T03:07:25.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.715+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 25) v1 ==== 75463+0+0 (secure 0 0 0) 0x7faa300777f0 con 0x7faa48154950 2023-11-28T03:07:25.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.716+0000 7faa3f7fe700 1 -- 172.21.15.73:0/1052574516 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v25) v1 ==== 92+0+0 (secure 0 0 0) 0x7faa3001e040 con 0x7faa48154950 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.718+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 msgr2=0x7faa28061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 0x7faa28061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7faa480c4420 tx=0x7faa38017000).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 msgr2=0x7faa481651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 rx=0x7faa3000c560 tx=0x7faa30007c90).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 shutdown_connections 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faa2805f150 0x7faa28061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faa48158570 0x7faa48165940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faa4814e530 0x7faa4814dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 --2- 172.21.15.73:0/1052574516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faa48154950 0x7faa481651b0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 >> 172.21.15.73:0/1052574516 conn(0x7faa48064990 msgr2=0x7faa48153e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:25.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 shutdown_connections 2023-11-28T03:07:25.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.719+0000 7faa47fff700 1 -- 172.21.15.73:0/1052574516 wait complete. 2023-11-28T03:07:25.734 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:07:25.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.854+0000 7fb07c56c700 1 Processor -- start 2023-11-28T03:07:25.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.854+0000 7fb07c56c700 1 -- start start 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb074098d10 0x7fb0740c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0740c6190 0x7fb074063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb074158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0740c72d0 con 0x7fb0740c6190 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb074154bc0 con 0x7fb074098d10 2023-11-28T03:07:25.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.855+0000 7fb07c56c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740c66c0 con 0x7fb074064350 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0740c6190 0x7fb074063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0740c6190 0x7fb074063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49330/0 (socket says 172.21.15.73:49330) 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 -- 172.21.15.73:0/376796918 learned_addr learned my addr 172.21.15.73:0/376796918 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb07ab09700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb074158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb07a308700 1 --2- 172.21.15.73:0/376796918 >> v2:172.21.15.150:3300/0 conn(0x7fb074098d10 0x7fb0740c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 -- 172.21.15.73:0/376796918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 msgr2=0x7fb074158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb074158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 -- 172.21.15.73:0/376796918 >> v2:172.21.15.150:3300/0 conn(0x7fb074098d10 msgr2=0x7fb0740c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 --2- 172.21.15.73:0/376796918 >> v2:172.21.15.150:3300/0 conn(0x7fb074098d10 0x7fb0740c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 -- 172.21.15.73:0/376796918 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0740c69a0 con 0x7fb0740c6190 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079b07700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0740c6190 0x7fb074063e60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fb074083f80 tx=0x7fb0700090f0).ready entity=mon.0 client_cookie=1579e4b415448fb6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:25.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079306700 1 -- 172.21.15.73:0/376796918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07000f510 con 0x7fb0740c6190 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079306700 1 -- 172.21.15.73:0/376796918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07000f670 con 0x7fb0740c6190 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.856+0000 7fb079306700 1 -- 172.21.15.73:0/376796918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07000f9f0 con 0x7fb0740c6190 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 -- 172.21.15.73:0/376796918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0740c6190 msgr2=0x7fb074063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0740c6190 0x7fb074063e60 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 rx=0x7fb074083f80 tx=0x7fb0700090f0).stop 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 -- 172.21.15.73:0/376796918 shutdown_connections 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb074158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 --2- 172.21.15.73:0/376796918 >> v2:172.21.15.150:3300/0 conn(0x7fb074098d10 0x7fb0740c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 --2- 172.21.15.73:0/376796918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0740c6190 0x7fb074063e60 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 -- 172.21.15.73:0/376796918 >> 172.21.15.73:0/376796918 conn(0x7fb0740cb310 msgr2=0x7fb0740cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 -- 172.21.15.73:0/376796918 shutdown_connections 2023-11-28T03:07:25.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.857+0000 7fb07c56c700 1 -- 172.21.15.73:0/376796918 wait complete. 2023-11-28T03:07:25.861 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:25.857+0000 7fae8a8dc680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.858+0000 7fb07c56c700 1 Processor -- start 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.858+0000 7fb07c56c700 1 -- start start 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb0740b05c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0740c6190 0x7fb07414aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740b1440 con 0x7fb074098d10 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb074155980 con 0x7fb0740c6190 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07c56c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0740b0ba0 con 0x7fb074064350 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49332/0 (socket says 172.21.15.73:49332) 2023-11-28T03:07:25.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 -- 172.21.15.73:0/1824199115 learned_addr learned my addr 172.21.15.73:0/1824199115 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07a308700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb0740b05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb07ab09700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0740c6190 0x7fb07414aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 -- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 msgr2=0x7fb0740b05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb0740b05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 -- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0740c6190 msgr2=0x7fb07414aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0740c6190 0x7fb07414aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 -- 172.21.15.73:0/1824199115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb070009370 con 0x7fb074098d10 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.859+0000 7fb079b07700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fb070011000 tx=0x7fb070017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.860+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07000f7e0 con 0x7fb074098d10 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.860+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb07000f940 con 0x7fb074098d10 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.860+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb07001c670 con 0x7fb074098d10 2023-11-28T03:07:25.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.860+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0740b0e80 con 0x7fb074098d10 2023-11-28T03:07:25.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.860+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0740c54e0 con 0x7fb074098d10 2023-11-28T03:07:25.865 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:25.860+0000 7f9897e68680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0741556e0 con 0x7fb074098d10 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fb07001ca20 con 0x7fb074098d10 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb06a7fc700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 0x7fb058061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb07a308700 1 -- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 msgr2=0x7fb058061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb07007b1a0 con 0x7fb074098d10 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.861+0000 7fb07a308700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 0x7fb058061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:25.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:25.865+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0741556e0 con 0x7fb074098d10 2023-11-28T03:07:25.921 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:25.921+0000 7fae8a8dc680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:25.927 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:25.926+0000 7f9897e68680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:26.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.038+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb0741556e0 con 0x7fb074098d10 2023-11-28T03:07:26.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.042+0000 7fb06a7fc700 1 -- 172.21.15.73:0/1824199115 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb0741556e0 con 0x7fb074098d10 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 msgr2=0x7fb058061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 0x7fb058061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 msgr2=0x7fb0740c4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 rx=0x7fb070011000 tx=0x7fb070017000).stop 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 shutdown_connections 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb05805f1a0 0x7fb058061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0740c6190 0x7fb07414aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb074064350 0x7fb0740b05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 --2- 172.21.15.73:0/1824199115 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb074098d10 0x7fb0740c4ff0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 >> 172.21.15.73:0/1824199115 conn(0x7fb0740cb310 msgr2=0x7fb074153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:26.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 shutdown_connections 2023-11-28T03:07:26.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.044+0000 7fb07c56c700 1 -- 172.21.15.73:0/1824199115 wait complete. 2023-11-28T03:07:26.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:26.057 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T03:07:26.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-28T03:07:26.198 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:26.198+0000 7fae8a8dc680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:26.213 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:26.212+0000 7f9897e68680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:26.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.237+0000 7faceb2c5700 1 Processor -- start 2023-11-28T03:07:26.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 -- start start 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face4067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7face413af40 0x7face40bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 0x7face4155490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face4067c70 con 0x7face413af40 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face40bddb0 con 0x7face4154fe0 2023-11-28T03:07:26.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.238+0000 7faceb2c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face4155980 con 0x7face413ab60 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7face413af40 0x7face40bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7face413af40 0x7face40bd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49340/0 (socket says 172.21.15.73:49340) 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 -- 172.21.15.73:0/271938475 learned_addr learned my addr 172.21.15.73:0/271938475 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face9061700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face4067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face9862700 1 --2- 172.21.15.73:0/271938475 >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 0x7face4155490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 -- 172.21.15.73:0/271938475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 msgr2=0x7face4067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face4067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 -- 172.21.15.73:0/271938475 >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 msgr2=0x7face4155490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 --2- 172.21.15.73:0/271938475 >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 0x7face4155490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 -- 172.21.15.73:0/271938475 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face40c3510 con 0x7face413af40 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face9862700 1 --2- 172.21.15.73:0/271938475 >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 0x7face4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:26.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face8860700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face413af40 0x7face40bd8c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7face406adf0 tx=0x7facd8009840).ready entity=mon.0 client_cookie=b539e5e58363abec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face3fff700 1 -- 172.21.15.73:0/271938475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800e070 con 0x7face413af40 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face3fff700 1 -- 172.21.15.73:0/271938475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd800b930 con 0x7face413af40 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.239+0000 7face3fff700 1 -- 172.21.15.73:0/271938475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd800bcb0 con 0x7face413af40 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 -- 172.21.15.73:0/271938475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face413af40 msgr2=0x7face40bd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face413af40 0x7face40bd8c0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 rx=0x7face406adf0 tx=0x7facd8009840).stop 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 -- 172.21.15.73:0/271938475 shutdown_connections 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 --2- 172.21.15.73:0/271938475 >> v2:172.21.15.150:3300/0 conn(0x7face4154fe0 0x7face4155490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face4067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 --2- 172.21.15.73:0/271938475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face413af40 0x7face40bd8c0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 -- 172.21.15.73:0/271938475 >> 172.21.15.73:0/271938475 conn(0x7face4063810 msgr2=0x7face4063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 -- 172.21.15.73:0/271938475 shutdown_connections 2023-11-28T03:07:26.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.240+0000 7faceb2c5700 1 -- 172.21.15.73:0/271938475 wait complete. 2023-11-28T03:07:26.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.241+0000 7faceb2c5700 1 Processor -- start 2023-11-28T03:07:26.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.241+0000 7faceb2c5700 1 -- start start 2023-11-28T03:07:26.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face40aa740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7face413af40 0x7face40bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face4154fe0 0x7face40bff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40a8dc0 con 0x7face4154fe0 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face40c04e0 con 0x7face413af40 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7faceb2c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face4162050 con 0x7face413ab60 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face4154fe0 0x7face40bff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face4154fe0 0x7face40bff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49352/0 (socket says 172.21.15.73:49352) 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 -- 172.21.15.73:0/1472145477 learned_addr learned my addr 172.21.15.73:0/1472145477 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face40aa740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face8860700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7face413af40 0x7face40bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 msgr2=0x7face40aa740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7face413ab60 0x7face40aa740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7face413af40 msgr2=0x7face40bfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:26.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7face413af40 0x7face40bfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.242+0000 7face9862700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7facd40092a0 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7face9862700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7face4154fe0 0x7face40bff30 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 rx=0x7facd4010b10 tx=0x7facd4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd400ea00 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7facd400eb60 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7faceb2c5700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7facd8009ae0 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7facd401a470 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.243+0000 7faceb2c5700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face4162610 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.244+0000 7faceb2c5700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face406c860 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.244+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 25) v1 ==== 75463+0+0 (secure 0 0 0) 0x7facd4017070 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.244+0000 7face17fa700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.245+0000 7face9061700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.245+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6121+0+0 (secure 0 0 0) 0x7facd407c3b0 con 0x7face4154fe0 2023-11-28T03:07:26.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.245+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:26.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.249+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7facd404d680 con 0x7face4154fe0 2023-11-28T03:07:26.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.446+0000 7face9061700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:26.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.446+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:26.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.448+0000 7faceb2c5700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7face413a3b0 con 0x7facd005f1a0 2023-11-28T03:07:26.821 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:26.820+0000 7fae8a8dc680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:26.841 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:26.840+0000 7f9897e68680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:26.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.847+0000 7face9061700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:26.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:26.847+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:07:26.894 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:26.893+0000 7fae8a8dc680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:26.913 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:26.912+0000 7f9897e68680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:27.081 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.080+0000 7fae8a8dc680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:27.100 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.099+0000 7f9897e68680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:27.504 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.503+0000 7fae8a8dc680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:27.521 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.519+0000 7f9897e68680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:27.608 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.607+0000 7fae8a8dc680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:27.623 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.622+0000 7f9897e68680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:27.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:27.648+0000 7face9061700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:27.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:27.648+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:07:27.762 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.761+0000 7fae8a8dc680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:30.443 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.775+0000 7f9897e68680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.849+0000 7f9897e68680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:27.916+0000 7f9897e68680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:28.175+0000 7f9897e68680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:28.356+0000 7f9897e68680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.003+0000 7f9897e68680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.072+0000 7f9897e68680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.147+0000 7f9897e68680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:30.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:29.249+0000 7face9061700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:30.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:29.249+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:07:30.445 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.836+0000 7fae8a8dc680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:30.445 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:27.903+0000 7fae8a8dc680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:30.445 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:28.162+0000 7fae8a8dc680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:28.346+0000 7fae8a8dc680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.000+0000 7fae8a8dc680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.068+0000 7fae8a8dc680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.143+0000 7fae8a8dc680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.291+0000 7fae8a8dc680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.362+0000 7fae8a8dc680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.478+0000 7fae8a8dc680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.587+0000 7fae8a8dc680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.879+0000 7fae8a8dc680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:30.446 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:29.949+0000 7fae8a8dc680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.296+0000 7f9897e68680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.366+0000 7f9897e68680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.482+0000 7f9897e68680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.590+0000 7f9897e68680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.876+0000 7f9897e68680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:29.947+0000 7f9897e68680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:30.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:29.966+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 26) v1 ==== 75287+0+0 (secure 0 0 0) 0x7facd407a180 con 0x7face4154fe0 2023-11-28T03:07:30.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:29.966+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 msgr2=0x7facd0061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:30.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:29.966+0000 7face17fa700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd0061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:30.448 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980c5ee700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.448 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980c5ee700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.448 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980c5ee700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.448 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980c5ee700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.448 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980fdf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.449 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980c5ee700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.449 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980fdf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.449 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980fdf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.449 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980fdf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.449 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.029+0000 7f980fdf5700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:30.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.968+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7facd407a500 con 0x7face4154fe0 2023-11-28T03:07:30.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.969+0000 7face17fa700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:30.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.969+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7face413a3b0 con 0x7facd005f1a0 2023-11-28T03:07:30.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.970+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:30.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.971+0000 7face9061700 1 --2- 172.21.15.73:0/1472145477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7facd005f1a0 0x7facd00615e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7facdc009670 tx=0x7facdc009060).ready entity=mgr.6441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:30.983 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:30.982+0000 7f9817604700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-28T03:07:30.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:30.982+0000 7face17fa700 1 -- 172.21.15.73:0/1472145477 <== mgr.6441 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7face413a3b0 con 0x7facd005f1a0 2023-11-28T03:07:30.985 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-28T03:07:30.988 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-28T03:07:30.989 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:30.989 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T03:07:31.007 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:07:31.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.162+0000 7faece62d700 1 Processor -- start 2023-11-28T03:07:31.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.162+0000 7faece62d700 1 -- start start 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faece62d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faec8098d10 0x7faec80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faece62d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec80c72d0 con 0x7faec80c6190 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faece62d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faec8064570 con 0x7faec8158320 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faece62d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faec8154bc0 con 0x7faec8098d10 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faec7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faec8098d10 0x7faec80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faec77fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faec80c6190 0x7faec8063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46282/0 (socket says 172.21.15.73:46282) 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.163+0000 7faec77fe700 1 -- 172.21.15.73:0/3104297964 learned_addr learned my addr 172.21.15.73:0/3104297964 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec7fff700 1 -- 172.21.15.73:0/3104297964 >> v1:172.21.15.150:6789/0 conn(0x7faec8158320 legacy=0x7faec80a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec7fff700 1 -- 172.21.15.73:0/3104297964 >> v1:172.21.15.73:6789/0 conn(0x7faec80c6190 legacy=0x7faec8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec7fff700 1 -- 172.21.15.73:0/3104297964 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faec80c66c0 con 0x7faec8098d10 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec6ffd700 1 -- 172.21.15.73:0/3104297964 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 134976433 0 0) 0x7faec80c72d0 con 0x7faec80c6190 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec7fff700 1 --2- 172.21.15.73:0/3104297964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8098d10 0x7faec80c7760 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7faeb8009760 tx=0x7faeb8009060).ready entity=mon.0 client_cookie=42149ccba632bfb4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:31.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec6ffd700 1 -- 172.21.15.73:0/3104297964 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800b580 con 0x7faec8098d10 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec6ffd700 1 -- 172.21.15.73:0/3104297964 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faeb800b6e0 con 0x7faec8098d10 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faec6ffd700 1 -- 172.21.15.73:0/3104297964 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faeb800ba60 con 0x7faec8098d10 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 -- 172.21.15.73:0/3104297964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8098d10 msgr2=0x7faec80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 --2- 172.21.15.73:0/3104297964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8098d10 0x7faec80c7760 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 rx=0x7faeb8009760 tx=0x7faeb8009060).stop 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 -- 172.21.15.73:0/3104297964 shutdown_connections 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 --2- 172.21.15.73:0/3104297964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec8098d10 0x7faec80c7760 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 -- 172.21.15.73:0/3104297964 >> 172.21.15.73:0/3104297964 conn(0x7faec80cb310 msgr2=0x7faec80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 -- 172.21.15.73:0/3104297964 shutdown_connections 2023-11-28T03:07:31.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.164+0000 7faece62d700 1 -- 172.21.15.73:0/3104297964 wait complete. 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.165+0000 7faece62d700 1 Processor -- start 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 -- start start 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faec8098d10 0x7faec814ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 0x7faec80b0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1030 con 0x7faec80c6190 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.166+0000 7faece62d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faec80b1310 con 0x7faec8158320 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faece62d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faec80af010 con 0x7faec8098d10 2023-11-28T03:07:31.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49426/0 (socket says 172.21.15.73:49426) 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 -- 172.21.15.73:0/1597671415 learned_addr learned my addr 172.21.15.73:0/1597671415 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec7fff700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faec8098d10 0x7faec814ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faeccbca700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 0x7faec80b0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 -- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faec8098d10 msgr2=0x7faec814ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faec8098d10 0x7faec814ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 -- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 msgr2=0x7faec80b0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 0x7faec80b0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 -- 172.21.15.73:0/1597671415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faeb80092a0 con 0x7faec80c6190 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faeccbca700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 0x7faec80b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faec77fe700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7faebc001eb0 tx=0x7faebc007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0072e0 con 0x7faec80c6190 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faebc007460 con 0x7faec80c6190 2023-11-28T03:07:31.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faebc0059f0 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faec80af2f0 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.167+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faec80afaf0 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.168+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faec80afd10 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.169+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 27) v1 ==== 75357+0+0 (secure 0 0 0) 0x7faebc013450 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.169+0000 7faeaffff700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 0x7faeb4061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.169+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7faebc079850 con 0x7faec80c6190 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.169+0000 7faec7fff700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 0x7faeb4061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:31.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.169+0000 7faec7fff700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 0x7faeb4061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faeb800b340 tx=0x7faeb8017000).ready entity=mgr.6441 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:31.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.173+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faebc04ab90 con 0x7faec80c6190 2023-11-28T03:07:31.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:31.380+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7faec806c860 con 0x7faec80c6190 2023-11-28T03:07:32.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.247+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7faebc0776f0 con 0x7faec80c6190 2023-11-28T03:07:32.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.248+0000 7faeaffff700 1 -- 172.21.15.73:0/1597671415 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v28) v1 ==== 93+0+0 (secure 0 0 0) 0x7faebc01fb90 con 0x7faec80c6190 2023-11-28T03:07:32.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 msgr2=0x7faeb4061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 0x7faeb4061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7faeb800b340 tx=0x7faeb8017000).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 msgr2=0x7faec80b0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 rx=0x7faebc001eb0 tx=0x7faebc007bf0).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 shutdown_connections 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faeb405f050 0x7faeb4061510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faec8158320 0x7faec80b0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faec8098d10 0x7faec814ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 --2- 172.21.15.73:0/1597671415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faec80c6190 0x7faec80b0150 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 >> 172.21.15.73:0/1597671415 conn(0x7faec80cb310 msgr2=0x7faec814c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 shutdown_connections 2023-11-28T03:07:32.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.250+0000 7faece62d700 1 -- 172.21.15.73:0/1597671415 wait complete. 2023-11-28T03:07:32.265 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:07:32.265 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:32.303 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:32.339 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:07:32.339 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:07:32.340 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:07:32.340 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:32.388 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:07:32.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:32.388 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:07:32.392 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:32.391+0000 7f00f7d8e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:32.392 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:32.392+0000 7fdfc50fa680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:32.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:32.455 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:32.454+0000 7f00f7d8e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:32.458 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:32.457+0000 7fdfc50fa680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:32.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.560+0000 7fcb64db7700 1 Processor -- start 2023-11-28T03:07:32.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.560+0000 7fcb64db7700 1 -- start start 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb64db7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb601548c0 0x7fcb6006a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb64db7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb600c6e30 0x7fcb600c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb64db7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb600c69a0 con 0x7fcb601544e0 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb64db7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb600c13a0 con 0x7fcb601548c0 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb64db7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb600c1680 con 0x7fcb600c6e30 2023-11-28T03:07:32.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb5dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcb601548c0 0x7fcb6006a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb5e59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcb601544e0 0x7fcb600c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:46284/0 (socket says 172.21.15.73:46284) 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb5e59c700 1 -- 172.21.15.73:0/3772756750 learned_addr learned my addr 172.21.15.73:0/3772756750 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.561+0000 7fcb5ed9d700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb600c6e30 0x7fcb600c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5dd9b700 1 -- 172.21.15.73:0/3772756750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb600c6e30 msgr2=0x7fcb600c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5dd9b700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb600c6e30 0x7fcb600c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5dd9b700 1 -- 172.21.15.73:0/3772756750 >> v1:172.21.15.73:6789/0 conn(0x7fcb601544e0 legacy=0x7fcb600c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5d59a700 1 -- 172.21.15.73:0/3772756750 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 119573002 0 0) 0x7fcb600c69a0 con 0x7fcb601544e0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5dd9b700 1 -- 172.21.15.73:0/3772756750 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb600c6630 con 0x7fcb601548c0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5dd9b700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb601548c0 0x7fcb6006a610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fcb50009760 tx=0x7fcb50009060).ready entity=mon.0 client_cookie=b857716988528f46 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5d59a700 1 -- 172.21.15.73:0/3772756750 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000f510 con 0x7fcb601548c0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5d59a700 1 -- 172.21.15.73:0/3772756750 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb5000f670 con 0x7fcb601548c0 2023-11-28T03:07:32.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb5d59a700 1 -- 172.21.15.73:0/3772756750 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb5000f9f0 con 0x7fcb601548c0 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb64db7700 1 -- 172.21.15.73:0/3772756750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb601548c0 msgr2=0x7fcb6006a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.562+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb601548c0 0x7fcb6006a610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 rx=0x7fcb50009760 tx=0x7fcb50009060).stop 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 -- 172.21.15.73:0/3772756750 shutdown_connections 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb600c6e30 0x7fcb600c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3772756750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb601548c0 0x7fcb6006a610 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 -- 172.21.15.73:0/3772756750 >> 172.21.15.73:0/3772756750 conn(0x7fcb60085020 msgr2=0x7fcb60094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 -- 172.21.15.73:0/3772756750 shutdown_connections 2023-11-28T03:07:32.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.563+0000 7fcb64db7700 1 -- 172.21.15.73:0/3772756750 wait complete. 2023-11-28T03:07:32.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.564+0000 7fcb64db7700 1 Processor -- start 2023-11-28T03:07:32.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.564+0000 7fcb64db7700 1 -- start start 2023-11-28T03:07:32.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.564+0000 7fcb64db7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.564+0000 7fcb64db7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb601544e0 0x7fcb600950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb601548c0 0x7fcb60095590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60095b40 con 0x7fcb600c6e30 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60095e20 con 0x7fcb601548c0 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb60096100 con 0x7fcb601544e0 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49464/0 (socket says 172.21.15.73:49464) 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 -- 172.21.15.73:0/3489263945 learned_addr learned my addr 172.21.15.73:0/3489263945 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5dd9b700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb601544e0 0x7fcb600950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5ed9d700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb601548c0 0x7fcb60095590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb601544e0 msgr2=0x7fcb600950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb601544e0 0x7fcb600950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb601548c0 msgr2=0x7fcb60095590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb601548c0 0x7fcb60095590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 -- 172.21.15.73:0/3489263945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb500092a0 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb5e59c700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fcb5400c490 tx=0x7fcb54007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb540070d0 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb54007230 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb54005670 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb600963e0 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.565+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb600c5430 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.566+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb600c3300 con 0x7fcb600c6e30 2023-11-28T03:07:32.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.567+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fcb54005c70 con 0x7fcb600c6e30 2023-11-28T03:07:32.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.567+0000 7fcb4e7fc700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 0x7fcb40061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.567+0000 7fcb5dd9b700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 msgr2=0x7fcb40061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:32.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.567+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fcb54079c90 con 0x7fcb600c6e30 2023-11-28T03:07:32.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.567+0000 7fcb5dd9b700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 0x7fcb40061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:32.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.572+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb5404ac50 con 0x7fcb600c6e30 2023-11-28T03:07:32.731 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:32.730+0000 7f00f7d8e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:32.747 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:32.746+0000 7fdfc50fa680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:32.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.768+0000 7fcb5dd9b700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 msgr2=0x7fcb40061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:32.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.768+0000 7fcb5dd9b700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 0x7fcb40061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:32.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.787+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb6006c860 con 0x7fcb600c6e30 2023-11-28T03:07:32.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.787+0000 7fcb4e7fc700 1 -- 172.21.15.73:0/3489263945 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fcb54018030 con 0x7fcb600c6e30 2023-11-28T03:07:32.788 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:32.789 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:32.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 msgr2=0x7fcb40061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 0x7fcb40061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 msgr2=0x7fcb60151280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 rx=0x7fcb5400c490 tx=0x7fcb54007d00).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 shutdown_connections 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb4005f2d0 0x7fcb40061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcb601548c0 0x7fcb60095590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcb600c6e30 0x7fcb60151280 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 --2- 172.21.15.73:0/3489263945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcb601544e0 0x7fcb600950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 >> 172.21.15.73:0/3489263945 conn(0x7fcb60085020 msgr2=0x7fcb600c0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 shutdown_connections 2023-11-28T03:07:32.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.789+0000 7fcb64db7700 1 -- 172.21.15.73:0/3489263945 wait complete. 2023-11-28T03:07:32.792 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:32.803 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.960+0000 7f717599e700 1 Processor -- start 2023-11-28T03:07:32.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.960+0000 7f717599e700 1 -- start start 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f717599e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71701544d0 0x7f7170155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f717599e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7170066f40 0x7f7170095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f717599e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71700683d0 con 0x7f71700673c0 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f717599e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7170067850 con 0x7f71701544d0 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f717599e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7170067b30 con 0x7f7170066f40 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f716effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71701544d0 0x7f7170155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f716effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f71701544d0 0x7f7170155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49472/0 (socket says 172.21.15.73:49472) 2023-11-28T03:07:32.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.961+0000 7f716effd700 1 -- 172.21.15.73:0/1568806875 learned_addr learned my addr 172.21.15.73:0/1568806875 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716e7fc700 1 --2- 172.21.15.73:0/1568806875 >> v2:172.21.15.150:3300/0 conn(0x7f7170066f40 0x7f7170095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716effd700 1 -- 172.21.15.73:0/1568806875 >> v2:172.21.15.150:3300/0 conn(0x7f7170066f40 msgr2=0x7f7170095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716effd700 1 --2- 172.21.15.73:0/1568806875 >> v2:172.21.15.150:3300/0 conn(0x7f7170066f40 0x7f7170095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716effd700 1 -- 172.21.15.73:0/1568806875 >> v1:172.21.15.150:6789/0 conn(0x7f71700673c0 legacy=0x7f7170063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716effd700 1 -- 172.21.15.73:0/1568806875 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7170067e10 con 0x7f71701544d0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716effd700 1 --2- 172.21.15.73:0/1568806875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71701544d0 0x7f7170155750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f717006a7d0 tx=0x7f7160009060).ready entity=mon.0 client_cookie=a6546ed2c9207826 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716dffb700 1 -- 172.21.15.73:0/1568806875 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160005880 con 0x7f71701544d0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716dffb700 1 -- 172.21.15.73:0/1568806875 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71600059e0 con 0x7f71701544d0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f716dffb700 1 -- 172.21.15.73:0/1568806875 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7160005d60 con 0x7f71701544d0 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f717599e700 1 -- 172.21.15.73:0/1568806875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71701544d0 msgr2=0x7f7170155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.962+0000 7f717599e700 1 --2- 172.21.15.73:0/1568806875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71701544d0 0x7f7170155750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 rx=0x7f717006a7d0 tx=0x7f7160009060).stop 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 -- 172.21.15.73:0/1568806875 shutdown_connections 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 --2- 172.21.15.73:0/1568806875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71701544d0 0x7f7170155750 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 --2- 172.21.15.73:0/1568806875 >> v2:172.21.15.150:3300/0 conn(0x7f7170066f40 0x7f7170095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 -- 172.21.15.73:0/1568806875 >> 172.21.15.73:0/1568806875 conn(0x7f71700631d0 msgr2=0x7f71700c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 -- 172.21.15.73:0/1568806875 shutdown_connections 2023-11-28T03:07:32.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.963+0000 7f717599e700 1 -- 172.21.15.73:0/1568806875 wait complete. 2023-11-28T03:07:32.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.964+0000 7f717599e700 1 Processor -- start 2023-11-28T03:07:32.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.964+0000 7f717599e700 1 -- start start 2023-11-28T03:07:32.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 0x7f7170168310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 0x7f71700a8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f717014d190 con 0x7f71700a92e0 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71700ab590 con 0x7f7170066f40 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71700ab870 con 0x7f71701544d0 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49484/0 (socket says 172.21.15.73:49484) 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 0x7f7170168310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 -- 172.21.15.73:0/1353698233 learned_addr learned my addr 172.21.15.73:0/1353698233 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716e7fc700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 0x7f71700a8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 -- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 msgr2=0x7f71700a8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 0x7f71700a8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 -- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 msgr2=0x7f7170168310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 0x7f7170168310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 -- 172.21.15.73:0/1353698233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7160009300 con 0x7f71700a92e0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716effd700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 0x7f7170168310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716e7fc700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 0x7f71700a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f716f7fe700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f716800c7c0 tx=0x7f7168001560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800e4a0 con 0x7f71700a92e0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f716800e600 con 0x7f71700a92e0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7170167830 con 0x7f71700a92e0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f716800e980 con 0x7f71700a92e0 2023-11-28T03:07:32.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.965+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7170167df0 con 0x7f71700a92e0 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.966+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f717006c860 con 0x7f71700a92e0 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.967+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f716801b3b0 con 0x7f71700a92e0 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.967+0000 7f7156ffd700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 0x7f715c0617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.967+0000 7f716effd700 1 -- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 msgr2=0x7f715c0617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.967+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7168079bd0 con 0x7f71700a92e0 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.967+0000 7f716effd700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 0x7f715c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:32.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:32.971+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7168048af0 con 0x7f71700a92e0 2023-11-28T03:07:33.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.143+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71700680f0 con 0x7f71700a92e0 2023-11-28T03:07:33.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.144+0000 7f7156ffd700 1 -- 172.21.15.73:0/1353698233 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17172 (secure 0 0 0) 0x7f716804ac40 con 0x7f71700a92e0 2023-11-28T03:07:33.145 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:33.145 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:29.961136+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/4214520789":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6833/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6832/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:0/954922268":"2023-11-29T03:07:29.961096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:33.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 msgr2=0x7f715c0617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 0x7f715c0617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 msgr2=0x7f71700c80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 rx=0x7f716800c7c0 tx=0x7f7168001560).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 shutdown_connections 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f715c05f330 0x7f715c0617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71700a92e0 0x7f71700c80a0 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7170066f40 0x7f7170168310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 --2- 172.21.15.73:0/1353698233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71701544d0 0x7f71700a8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 >> 172.21.15.73:0/1353698233 conn(0x7f71700631d0 msgr2=0x7f7170063d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 shutdown_connections 2023-11-28T03:07:33.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.146+0000 7f717599e700 1 -- 172.21.15.73:0/1353698233 wait complete. 2023-11-28T03:07:33.160 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '73', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 9, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:07:17.214635+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '81', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 10, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:07:17.638322+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '82', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:07:33.160 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:07:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.280+0000 7f5e8de7a700 1 Processor -- start 2023-11-28T03:07:33.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.280+0000 7f5e8de7a700 1 -- start start 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e8de7a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e88155830 0x7f5e88155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e8de7a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e8de7a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e881529c0 con 0x7f5e88155030 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e8de7a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e881554b0 con 0x7f5e88155830 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e8de7a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88151db0 con 0x7f5e880c3090 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e877fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e88155830 0x7f5e88155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e877fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e88155830 0x7f5e88155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49492/0 (socket says 172.21.15.73:49492) 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e877fe700 1 -- 172.21.15.73:0/638476962 learned_addr learned my addr 172.21.15.73:0/638476962 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.281+0000 7f5e87fff700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e877fe700 1 -- 172.21.15.73:0/638476962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 msgr2=0x7f5e880c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e877fe700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880c5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e877fe700 1 -- 172.21.15.73:0/638476962 >> v1:172.21.15.150:6789/0 conn(0x7f5e88155030 legacy=0x7f5e880c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e877fe700 1 -- 172.21.15.73:0/638476962 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e88152090 con 0x7f5e88155830 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e877fe700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e88155c10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f5e78006fd0 tx=0x7f5e780090f0).ready entity=mon.0 client_cookie=65a361e77c5573eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e86ffd700 1 -- 172.21.15.73:0/638476962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800b580 con 0x7f5e88155830 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e86ffd700 1 -- 172.21.15.73:0/638476962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e7800b6e0 con 0x7f5e88155830 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e86ffd700 1 -- 172.21.15.73:0/638476962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ba80 con 0x7f5e88155830 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 -- 172.21.15.73:0/638476962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 msgr2=0x7f5e88155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e88155c10 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 rx=0x7f5e78006fd0 tx=0x7f5e780090f0).stop 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 -- 172.21.15.73:0/638476962 shutdown_connections 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880c5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/638476962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e88155c10 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.282+0000 7f5e8de7a700 1 -- 172.21.15.73:0/638476962 >> 172.21.15.73:0/638476962 conn(0x7f5e88062700 msgr2=0x7f5e88062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.283+0000 7f5e8de7a700 1 -- 172.21.15.73:0/638476962 shutdown_connections 2023-11-28T03:07:33.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.283+0000 7f5e8de7a700 1 -- 172.21.15.73:0/638476962 wait complete. 2023-11-28T03:07:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.284+0000 7f5e8de7a700 1 Processor -- start 2023-11-28T03:07:33.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.284+0000 7f5e8de7a700 1 -- start start 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e88155030 0x7f5e880bce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e880a5540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88092690 con 0x7f5e88155830 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88093ad0 con 0x7f5e88155030 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e88093db0 con 0x7f5e880c3090 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e880a5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e7effd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e88155030 0x7f5e880bce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:37248/0 (socket says 172.21.15.73:37248) 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 -- 172.21.15.73:0/3930991725 learned_addr learned my addr 172.21.15.73:0/3930991725 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e88155030 msgr2=0x7f5e880bce90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e88155030 0x7f5e880bce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 msgr2=0x7f5e880a5540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e880a5540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 -- 172.21.15.73:0/3930991725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e78009370 con 0x7f5e880c3090 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e87fff700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e880a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e877fe700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5e880677f0 tx=0x7f5e78017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800b580 con 0x7f5e880c3090 2023-11-28T03:07:33.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e78006990 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e7800ba80 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e88094090 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.285+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e88094650 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.286+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e8806c860 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.287+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f5e7801d640 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.287+0000 7f5e7ffff700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 0x7f5e680617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.287+0000 7f5e7effd700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 msgr2=0x7f5e680617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.287+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f5e7807bea0 con 0x7f5e880c3090 2023-11-28T03:07:33.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.287+0000 7f5e7effd700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 0x7f5e680617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:33.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.291+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5e7804cf70 con 0x7f5e880c3090 2023-11-28T03:07:33.349 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:33.347+0000 7f00f7d8e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:33.392 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:33.391+0000 7fdfc50fa680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:33.420 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:33.419+0000 7f00f7d8e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:33.464 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:33.464+0000 7fdfc50fa680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:33.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.488+0000 7f5e7effd700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 msgr2=0x7f5e680617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:33.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.488+0000 7f5e7effd700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 0x7f5e680617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:33.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.493+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f5e881681a0 con 0x7f5e880c3090 2023-11-28T03:07:33.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.494+0000 7f5e7ffff700 1 -- 172.21.15.73:0/3930991725 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v83) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5e78021070 con 0x7f5e880c3090 2023-11-28T03:07:33.495 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:07:33.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 msgr2=0x7f5e680617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 0x7f5e680617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 msgr2=0x7f5e880a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 rx=0x7f5e880677f0 tx=0x7f5e78017000).stop 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 shutdown_connections 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e6805f300 0x7f5e680617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e88155830 0x7f5e880a5540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e880c3090 0x7f5e880a3da0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 --2- 172.21.15.73:0/3930991725 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e88155030 0x7f5e880bce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 >> 172.21.15.73:0/3930991725 conn(0x7f5e88062700 msgr2=0x7f5e8814cc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 shutdown_connections 2023-11-28T03:07:33.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.496+0000 7f5e8de7a700 1 -- 172.21.15.73:0/3930991725 wait complete. 2023-11-28T03:07:33.510 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:07:33.608 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:33.607+0000 7f00f7d8e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:33.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.629+0000 7efc03788700 1 Processor -- start 2023-11-28T03:07:33.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.629+0000 7efc03788700 1 -- start start 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc03788700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc03788700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efbfc0bfbe0 0x7efbfc067ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc03788700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efbfc154c00 con 0x7efbfc096370 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc03788700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efbfc14f190 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc03788700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc155770 con 0x7efbfc14edb0 2023-11-28T03:07:33.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc01d25700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efbfc0bfbe0 0x7efbfc067ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc00d23700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efbfc096370 0x7efbfc0673d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45080/0 (socket says 172.21.15.73:45080) 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc01d25700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efbfc0bfbe0 0x7efbfc067ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53936/0 (socket says 172.21.15.73:53936) 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc00d23700 1 -- 172.21.15.73:0/2625230269 learned_addr learned my addr 172.21.15.73:0/2625230269 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.630+0000 7efc01524700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efc01d25700 1 -- 172.21.15.73:0/2625230269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 msgr2=0x7efbfc095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efc01d25700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efc01d25700 1 -- 172.21.15.73:0/2625230269 >> v1:172.21.15.73:6789/0 conn(0x7efbfc096370 legacy=0x7efbfc0673d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efc01d25700 1 -- 172.21.15.73:0/2625230269 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbfc0967f0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efbf3fff700 1 -- 172.21.15.73:0/2625230269 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589678563 0 0) 0x7efbf8001110 con 0x7efbfc096370 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efc01d25700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc067ae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7efbf4009e60 tx=0x7efbf4009780).ready entity=mon.0 client_cookie=2b8b2d5cce736a8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efbf3fff700 1 -- 172.21.15.73:0/2625230269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf400e070 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efbf3fff700 1 -- 172.21.15.73:0/2625230269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf400bb60 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.631+0000 7efbf3fff700 1 -- 172.21.15.73:0/2625230269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf4012410 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 -- 172.21.15.73:0/2625230269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 msgr2=0x7efbfc067ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc067ae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 rx=0x7efbf4009e60 tx=0x7efbf4009780).stop 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 -- 172.21.15.73:0/2625230269 shutdown_connections 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 --2- 172.21.15.73:0/2625230269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc067ae0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 -- 172.21.15.73:0/2625230269 >> 172.21.15.73:0/2625230269 conn(0x7efbfc0631d0 msgr2=0x7efbfc0c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 -- 172.21.15.73:0/2625230269 shutdown_connections 2023-11-28T03:07:33.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.632+0000 7efc03788700 1 -- 172.21.15.73:0/2625230269 wait complete. 2023-11-28T03:07:33.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.634+0000 7efc03788700 1 Processor -- start 2023-11-28T03:07:33.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.634+0000 7efc03788700 1 -- start start 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efbfc096370 0x7efbfc1615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc0c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0c8ef0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0aee30 con 0x7efbfc096370 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc03788700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efbfc0ae500 con 0x7efbfc14edb0 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53942/0 (socket says 172.21.15.73:53942) 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 -- 172.21.15.73:0/2571589080 learned_addr learned my addr 172.21.15.73:0/2571589080 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc01d25700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc0c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc01524700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efbfc096370 0x7efbfc1615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 -- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 msgr2=0x7efbfc0c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc0c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.635+0000 7efc00d23700 1 -- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efbfc096370 msgr2=0x7efbfc1615e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efc00d23700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efbfc096370 0x7efbfc1615e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efc00d23700 1 -- 172.21.15.73:0/2571589080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efbf4009a20 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efc00d23700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7efbf800c490 tx=0x7efbf8007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80070d0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efbf8007230 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efbf80075d0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efbfc0ae7e0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.636+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efbfc161ad0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.637+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efbfc0aefe0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.637+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7efbf8005bc0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.638+0000 7efbf17fa700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 0x7efbe00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.638+0000 7efc01524700 1 -- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 msgr2=0x7efbe00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:33.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.638+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7efbf8079b60 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.638+0000 7efc01524700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 0x7efbe00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:33.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.642+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efbfc0aefe0 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.672 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:33.671+0000 7fdfc50fa680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:33.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.832+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7efbfc154c00 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.832+0000 7efbf17fa700 1 -- 172.21.15.73:0/2571589080 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v83) v1 ==== 92+0+10 (secure 0 0 0) 0x7efbfc154c00 con 0x7efbfc0bfbe0 2023-11-28T03:07:33.834 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:07:33.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 msgr2=0x7efbe00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:33.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 0x7efbe00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 msgr2=0x7efbfc0c8510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 rx=0x7efbf800c490 tx=0x7efbf8007e60).stop 2023-11-28T03:07:33.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 shutdown_connections 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efbe005f300 0x7efbe00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efbfc14edb0 0x7efbfc0c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efbfc096370 0x7efbfc1615e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 --2- 172.21.15.73:0/2571589080 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efbfc0bfbe0 0x7efbfc0c8510 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 >> 172.21.15.73:0/2571589080 conn(0x7efbfc0631d0 msgr2=0x7efbfc152880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 shutdown_connections 2023-11-28T03:07:33.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.835+0000 7efc03788700 1 -- 172.21.15.73:0/2571589080 wait complete. 2023-11-28T03:07:33.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:07:33.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.969+0000 7f00fb18b700 1 Processor -- start 2023-11-28T03:07:33.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.969+0000 7f00fb18b700 1 -- start start 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00fb18b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00fb18b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f40c6fc0 con 0x7f00f40c82b0 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00fb18b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00f40c7e20 con 0x7f00f40a84d0 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00fb18b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40c7210 con 0x7f00f40bf030 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00f9728700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f00f40c82b0 0x7f00f4064320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45096/0 (socket says 172.21.15.73:45096) 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00f9728700 1 -- 172.21.15.73:0/3711189665 learned_addr learned my addr 172.21.15.73:0/3711189665 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:33.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00f3fff700 1 --2- 172.21.15.73:0/3711189665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40bf4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 393835536 0 0) 0x7f00f40c6fc0 con 0x7f00f40c82b0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.970+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00e40033f0 con 0x7f00f40c82b0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914142324 0 0) 0x7f00f40c7e20 con 0x7f00f40a84d0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f00f40c6fc0 con 0x7f00f40a84d0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f3fff700 1 -- 172.21.15.73:0/3711189665 >> v1:172.21.15.150:6789/0 conn(0x7f00f40a84d0 legacy=0x7f00f40c36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f3fff700 1 -- 172.21.15.73:0/3711189665 >> v1:172.21.15.73:6789/0 conn(0x7f00f40c82b0 legacy=0x7f00f4064320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f3fff700 1 -- 172.21.15.73:0/3711189665 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f40c74f0 con 0x7f00f40bf030 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f3fff700 1 --2- 172.21.15.73:0/3711189665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40bf4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f00e0009760 tx=0x7f00e0006f40).ready entity=mon.2 client_cookie=35912ef046fd5d44 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0005880 con 0x7f00f40bf030 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00e00059e0 con 0x7f00f40bf030 2023-11-28T03:07:33.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.971+0000 7f00f37fe700 1 -- 172.21.15.73:0/3711189665 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00e0005d60 con 0x7f00f40bf030 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 -- 172.21.15.73:0/3711189665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 msgr2=0x7f00f40bf4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 --2- 172.21.15.73:0/3711189665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40bf4b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 rx=0x7f00e0009760 tx=0x7f00e0006f40).stop 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 -- 172.21.15.73:0/3711189665 shutdown_connections 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 --2- 172.21.15.73:0/3711189665 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40bf4b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 -- 172.21.15.73:0/3711189665 >> 172.21.15.73:0/3711189665 conn(0x7f00f4062ee0 msgr2=0x7f00f40c90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:33.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 -- 172.21.15.73:0/3711189665 shutdown_connections 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.972+0000 7f00fb18b700 1 -- 172.21.15.73:0/3711189665 wait complete. 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.973+0000 7f00fb18b700 1 Processor -- start 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.973+0000 7f00fb18b700 1 -- start start 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 0x7f00f40c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40aa830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f4155e90 con 0x7f00f40c82b0 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40abb40 con 0x7f00f40a84d0 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00fb18b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00f40ab2a0 con 0x7f00f40bf030 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f3fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40aa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f8f27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 0x7f00f40c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53952/0 (socket says 172.21.15.73:53952) 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 -- 172.21.15.73:0/2753903956 learned_addr learned my addr 172.21.15.73:0/2753903956 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 -- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 msgr2=0x7f00f40aa830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40aa830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 -- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 msgr2=0x7f00f40c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 0x7f00f40c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 -- 172.21.15.73:0/2753903956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00e0009250 con 0x7f00f40c82b0 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f3fff700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f8f27700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 0x7f00f40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f9728700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f00ec001560 tx=0x7f00ec0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec005d30 con 0x7f00f40c82b0 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.974+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec005e90 con 0x7f00f40c82b0 2023-11-28T03:07:33.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.975+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec0072e0 con 0x7f00f40c82b0 2023-11-28T03:07:33.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.975+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00f40ab580 con 0x7f00f40c82b0 2023-11-28T03:07:33.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.975+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00f40c9220 con 0x7f00f40c82b0 2023-11-28T03:07:33.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00f4156000 con 0x7f00f40c82b0 2023-11-28T03:07:33.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f00ec007650 con 0x7f00f40c82b0 2023-11-28T03:07:33.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00f0ff9700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 0x7f00d80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:33.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00f8f27700 1 -- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 msgr2=0x7f00d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:33.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f00ec079ef0 con 0x7f00f40c82b0 2023-11-28T03:07:33.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.976+0000 7f00f8f27700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 0x7f00d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:33.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:33.980+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f00ec04af10 con 0x7f00f40c82b0 2023-11-28T03:07:34.032 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.031+0000 7f00f7d8e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:34.114 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.113+0000 7fdfc50fa680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:34.135 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.134+0000 7f00f7d8e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:34.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.151+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f00f406c830 con 0x7f00f40c82b0 2023-11-28T03:07:34.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.152+0000 7f00f0ff9700 1 -- 172.21.15.73:0/2753903956 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v83) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00ec04e530 con 0x7f00f40c82b0 2023-11-28T03:07:34.153 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:07:34.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 msgr2=0x7f00d80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:34.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 0x7f00d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 msgr2=0x7f00f40a88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 rx=0x7f00ec001560 tx=0x7f00ec0015b0).stop 2023-11-28T03:07:34.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 shutdown_connections 2023-11-28T03:07:34.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00d805f300 0x7f00d80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00f40c82b0 0x7f00f40a88b0 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00f40a84d0 0x7f00f40c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 --2- 172.21.15.73:0/2753903956 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00f40bf030 0x7f00f40aa830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 >> 172.21.15.73:0/2753903956 conn(0x7f00f4062ee0 msgr2=0x7f00f40a6850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 shutdown_connections 2023-11-28T03:07:34.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.154+0000 7f00fb18b700 1 -- 172.21.15.73:0/2753903956 wait complete. 2023-11-28T03:07:34.167 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:07:34.227 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.226+0000 7fdfc50fa680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:34.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.288+0000 7f7d3b440700 1 Processor -- start 2023-11-28T03:07:34.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.288+0000 7f7d3b440700 1 -- start start 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d3b440700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d340c1780 0x7f7d340c1bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d3b440700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340c12f0 con 0x7f7d340c0250 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d3b440700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d340a8790 con 0x7f7d34157c10 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d3b440700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d340c06e0 con 0x7f7d340c1780 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d340c1780 0x7f7d340c1bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7d340c1780 0x7f7d340c1bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53968/0 (socket says 172.21.15.73:53968) 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d389db700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7d340c0250 0x7f7d340c9ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45108/0 (socket says 172.21.15.73:45108) 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 -- 172.21.15.73:0/2459165954 learned_addr learned my addr 172.21.15.73:0/2459165954 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 -- 172.21.15.73:0/2459165954 >> v1:172.21.15.150:6789/0 conn(0x7f7d34157c10 legacy=0x7f7d340a8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 -- 172.21.15.73:0/2459165954 >> v1:172.21.15.73:6789/0 conn(0x7f7d340c0250 legacy=0x7f7d340c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d2bfff700 1 -- 172.21.15.73:0/2459165954 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2031838203 0 0) 0x7f7d340c12f0 con 0x7f7d340c0250 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.289+0000 7f7d399dd700 1 -- 172.21.15.73:0/2459165954 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d340c09c0 con 0x7f7d340c1780 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d399dd700 1 --2- 172.21.15.73:0/2459165954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340c1bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f7d2c009f20 tx=0x7f7d2c009840).ready entity=mon.0 client_cookie=69c33e3c7b4aa7b8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d2bfff700 1 -- 172.21.15.73:0/2459165954 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00e070 con 0x7f7d340c1780 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d2bfff700 1 -- 172.21.15.73:0/2459165954 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d2c00ba80 con 0x7f7d340c1780 2023-11-28T03:07:34.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d2bfff700 1 -- 172.21.15.73:0/2459165954 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d2c00be00 con 0x7f7d340c1780 2023-11-28T03:07:34.293 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.290+0000 7f00f7d8e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d3b440700 1 -- 172.21.15.73:0/2459165954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 msgr2=0x7f7d340c1bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d3b440700 1 --2- 172.21.15.73:0/2459165954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340c1bb0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 rx=0x7f7d2c009f20 tx=0x7f7d2c009840).stop 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d3b440700 1 -- 172.21.15.73:0/2459165954 shutdown_connections 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d3b440700 1 --2- 172.21.15.73:0/2459165954 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340c1bb0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.290+0000 7f7d3b440700 1 -- 172.21.15.73:0/2459165954 >> 172.21.15.73:0/2459165954 conn(0x7f7d3406a610 msgr2=0x7f7d340cb2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.291+0000 7f7d3b440700 1 -- 172.21.15.73:0/2459165954 shutdown_connections 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.291+0000 7f7d3b440700 1 -- 172.21.15.73:0/2459165954 wait complete. 2023-11-28T03:07:34.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.292+0000 7f7d3b440700 1 Processor -- start 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.292+0000 7f7d3b440700 1 -- start start 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d34157c10 0x7f7d340c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 0x7f7d34152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340abb10 con 0x7f7d340c1780 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340aead0 con 0x7f7d340ca2b0 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d340aedb0 con 0x7f7d34157c10 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53982/0 (socket says 172.21.15.73:53982) 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 -- 172.21.15.73:0/1581429064 learned_addr learned my addr 172.21.15.73:0/1581429064 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d399dd700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 0x7f7d34152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d389db700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d34157c10 0x7f7d340c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 -- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d34157c10 msgr2=0x7f7d340c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d34157c10 0x7f7d340c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 -- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 msgr2=0x7f7d34152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 0x7f7d34152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 -- 172.21.15.73:0/1581429064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d2c009ae0 con 0x7f7d340c1780 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d399dd700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 0x7f7d34152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d391dc700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f7d2400c440 tx=0x7f7d24007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24007020 con 0x7f7d340c1780 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d24007180 con 0x7f7d340c1780 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d340af090 con 0x7f7d340c1780 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d24007520 con 0x7f7d340c1780 2023-11-28T03:07:34.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.293+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d340ae770 con 0x7f7d340c1780 2023-11-28T03:07:34.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.294+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d340ab600 con 0x7f7d340c1780 2023-11-28T03:07:34.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.295+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f7d24006900 con 0x7f7d340c1780 2023-11-28T03:07:34.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.295+0000 7f7d297fa700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 0x7f7d18061810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.295+0000 7f7d389db700 1 -- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 msgr2=0x7f7d18061810 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:34.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.295+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7d240796c0 con 0x7f7d340c1780 2023-11-28T03:07:34.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.295+0000 7f7d389db700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 0x7f7d18061810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:34.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.300+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d24047af0 con 0x7f7d340c1780 2023-11-28T03:07:34.370 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.369+0000 7f00f7d8e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:34.390 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.390+0000 7fdfc50fa680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:34.438 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.437+0000 7f00f7d8e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:34.468 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.467+0000 7fdfc50fa680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:34.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.473+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7d3406c860 con 0x7f7d340c1780 2023-11-28T03:07:34.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.474+0000 7f7d297fa700 1 -- 172.21.15.73:0/1581429064 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v83) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7d340ab600 con 0x7f7d340c1780 2023-11-28T03:07:34.475 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:07:34.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 msgr2=0x7f7d18061810 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:34.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 0x7f7d18061810 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 msgr2=0x7f7d340ae280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 rx=0x7f7d2400c440 tx=0x7f7d24007e50).stop 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 shutdown_connections 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d1805f350 0x7f7d18061810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d340ca2b0 0x7f7d34152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d340c1780 0x7f7d340ae280 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 --2- 172.21.15.73:0/1581429064 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d34157c10 0x7f7d340c9dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 >> 172.21.15.73:0/1581429064 conn(0x7f7d3406a610 msgr2=0x7f7d34156550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 shutdown_connections 2023-11-28T03:07:34.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.476+0000 7f7d3b440700 1 -- 172.21.15.73:0/1581429064 wait complete. 2023-11-28T03:07:34.489 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:34.541 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.540+0000 7fdfc50fa680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.604+0000 7f8a91ca5700 1 Processor -- start 2023-11-28T03:07:34.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.604+0000 7f8a91ca5700 1 -- start start 2023-11-28T03:07:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a91ca5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a91ca5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c0a6720 con 0x7f8a8c062700 2023-11-28T03:07:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a91ca5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8c0c2540 con 0x7f8a8c154230 2023-11-28T03:07:34.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a91ca5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8c0a4f70 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53990/0 (socket says 172.21.15.73:53990) 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2578542659 learned_addr learned my addr 172.21.15.73:0/2578542659 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a914a4700 1 --1- 172.21.15.73:0/2578542659 >> v1:172.21.15.73:6789/0 conn(0x7f8a8c062700 0x7f8a8c062b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45120/0 (socket says 172.21.15.73:45120) 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2578542659 >> v1:172.21.15.150:6789/0 conn(0x7f8a8c154230 legacy=0x7f8a8c0a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2578542659 >> v1:172.21.15.73:6789/0 conn(0x7f8a8c062700 legacy=0x7f8a8c062b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8b7fe700 1 -- 172.21.15.73:0/2578542659 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842341780 0 0) 0x7f8a8c0a6720 con 0x7f8a8c062700 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2578542659 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8c0a5250 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.605+0000 7f8a8bfff700 1 --2- 172.21.15.73:0/2578542659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f8a78006fd0 tx=0x7f8a780090f0).ready entity=mon.0 client_cookie=d3eb361c1eb161f5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a8b7fe700 1 -- 172.21.15.73:0/2578542659 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800b600 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a8b7fe700 1 -- 172.21.15.73:0/2578542659 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800b760 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a8b7fe700 1 -- 172.21.15.73:0/2578542659 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800bae0 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2578542659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c0c2cb0 msgr2=0x7f8a8c0a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2578542659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 rx=0x7f8a78006fd0 tx=0x7f8a780090f0).stop 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2578542659 shutdown_connections 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2578542659 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0a48e0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2578542659 >> 172.21.15.73:0/2578542659 conn(0x7f8a8c0c7ca0 msgr2=0x7f8a8c0c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2578542659 shutdown_connections 2023-11-28T03:07:34.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.606+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2578542659 wait complete. 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.607+0000 7f8a91ca5700 1 Processor -- start 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 -- start start 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0aac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a8c154230 0x7f8a8c0ab140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0ab630 con 0x7f8a8c062700 2023-11-28T03:07:34.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0bed40 con 0x7f8a8c154230 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a91ca5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8c0bf5e0 con 0x7f8a8c0c2cb0 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a90ca3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a90ca3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53998/0 (socket says 172.21.15.73:53998) 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a90ca3700 1 -- 172.21.15.73:0/2964637113 learned_addr learned my addr 172.21.15.73:0/2964637113 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a914a4700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a8c154230 0x7f8a8c0ab140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.608+0000 7f8a8bfff700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0aac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a8c0c2cb0 msgr2=0x7f8a8c0aac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0aac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a8c154230 msgr2=0x7f8a8c0ab140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a8c154230 0x7f8a8c0ab140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 -- 172.21.15.73:0/2964637113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a78009370 con 0x7f8a8c062700 2023-11-28T03:07:34.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a90ca3700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f8a8000c440 tx=0x7f8a80007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80007020 con 0x7f8a8c062700 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a80007180 con 0x7f8a8c062700 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8c0adae0 con 0x7f8a8c062700 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a80007520 con 0x7f8a8c062700 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.609+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8c0bf270 con 0x7f8a8c062700 2023-11-28T03:07:34.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.610+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8c06c7c0 con 0x7f8a8c062700 2023-11-28T03:07:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.611+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f8a800068b0 con 0x7f8a8c062700 2023-11-28T03:07:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.611+0000 7f8a88ff9700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 0x7f8a700617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.611+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 msgr2=0x7f8a700617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.611+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f8a80079fa0 con 0x7f8a8c062700 2023-11-28T03:07:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.611+0000 7f8a8bfff700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 0x7f8a700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:34.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.615+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8a8004b040 con 0x7f8a8c062700 2023-11-28T03:07:34.699 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.698+0000 7f00f7d8e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.812+0000 7f8a8bfff700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 msgr2=0x7f8a700617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:34.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.812+0000 7f8a8bfff700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 0x7f8a700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:34.815 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:34.814+0000 7fdfc50fa680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:34.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.831+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a8c0a5530 con 0x7f8a8c062700 2023-11-28T03:07:34.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.831+0000 7f8a88ff9700 1 -- 172.21.15.73:0/2964637113 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f8a80020030 con 0x7f8a8c062700 2023-11-28T03:07:34.832 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:34.832 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 msgr2=0x7f8a700617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 0x7f8a700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 msgr2=0x7f8a8c141480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 rx=0x7f8a8000c440 tx=0x7f8a80007e00).stop 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 shutdown_connections 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8a7005f300 0x7f8a700617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8a8c154230 0x7f8a8c0ab140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8a8c062700 0x7f8a8c141480 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 --2- 172.21.15.73:0/2964637113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8a8c0c2cb0 0x7f8a8c0aac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 >> 172.21.15.73:0/2964637113 conn(0x7f8a8c0c7ca0 msgr2=0x7f8a8c0bccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 shutdown_connections 2023-11-28T03:07:34.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.833+0000 7f8a91ca5700 1 -- 172.21.15.73:0/2964637113 wait complete. 2023-11-28T03:07:34.836 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:34.846 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:34.885 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:34.882+0000 7f00f7d8e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:34.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.965+0000 7f853d792700 1 Processor -- start 2023-11-28T03:07:34.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 -- start start 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8538150890 0x7f85380bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f85380bc320 0x7f8538069900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85380c28c0 con 0x7f8538069df0 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f853806a260 con 0x7f8538150890 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f853d792700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85381552e0 con 0x7f85380bc320 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.966+0000 7f8536ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8538150890 0x7f85380bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f85377fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8538069df0 0x7f85381544f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45122/0 (socket says 172.21.15.73:45122) 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8538150890 0x7f85380bbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54000/0 (socket says 172.21.15.73:54000) 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f85377fe700 1 -- 172.21.15.73:0/797098222 learned_addr learned my addr 172.21.15.73:0/797098222 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f85367fc700 1 --2- 172.21.15.73:0/797098222 >> v2:172.21.15.150:3300/0 conn(0x7f85380bc320 0x7f8538069900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 -- 172.21.15.73:0/797098222 >> v2:172.21.15.150:3300/0 conn(0x7f85380bc320 msgr2=0x7f8538069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 --2- 172.21.15.73:0/797098222 >> v2:172.21.15.150:3300/0 conn(0x7f85380bc320 0x7f8538069900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 -- 172.21.15.73:0/797098222 >> v1:172.21.15.73:6789/0 conn(0x7f8538069df0 legacy=0x7f85381544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8535ffb700 1 -- 172.21.15.73:0/797098222 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4056202394 0 0) 0x7f85380c28c0 con 0x7f8538069df0 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 -- 172.21.15.73:0/797098222 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f853806a4b0 con 0x7f8538150890 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8536ffd700 1 --2- 172.21.15.73:0/797098222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380bbe30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f85380a6980 tx=0x7f852c009060).ready entity=mon.0 client_cookie=fbb9a959c557d65c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8535ffb700 1 -- 172.21.15.73:0/797098222 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00f510 con 0x7f8538150890 2023-11-28T03:07:34.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8535ffb700 1 -- 172.21.15.73:0/797098222 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f852c00f670 con 0x7f8538150890 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.967+0000 7f8535ffb700 1 -- 172.21.15.73:0/797098222 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f852c00f9f0 con 0x7f8538150890 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 -- 172.21.15.73:0/797098222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 msgr2=0x7f85380bbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 --2- 172.21.15.73:0/797098222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380bbe30 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 rx=0x7f85380a6980 tx=0x7f852c009060).stop 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 -- 172.21.15.73:0/797098222 shutdown_connections 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 --2- 172.21.15.73:0/797098222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380bbe30 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 --2- 172.21.15.73:0/797098222 >> v2:172.21.15.150:3300/0 conn(0x7f85380bc320 0x7f8538069900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 -- 172.21.15.73:0/797098222 >> 172.21.15.73:0/797098222 conn(0x7f85380631d0 msgr2=0x7f85380c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 -- 172.21.15.73:0/797098222 shutdown_connections 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.968+0000 7f853d792700 1 -- 172.21.15.73:0/797098222 wait complete. 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.969+0000 7f853d792700 1 Processor -- start 2023-11-28T03:07:34.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.969+0000 7f853d792700 1 -- start start 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8538069df0 0x7f8538144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85380bc320 0x7f85380a8c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85380a76a0 con 0x7f8538150890 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85380aa1f0 con 0x7f8538069df0 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f853d792700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85380a8640 con 0x7f85380bc320 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54014/0 (socket says 172.21.15.73:54014) 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 -- 172.21.15.73:0/1167671868 learned_addr learned my addr 172.21.15.73:0/1167671868 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85367fc700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85380bc320 0x7f85380a8c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f8536ffd700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8538069df0 0x7f8538144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 -- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85380bc320 msgr2=0x7f85380a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85380bc320 0x7f85380a8c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 -- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8538069df0 msgr2=0x7f8538144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8538069df0 0x7f8538144870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 -- 172.21.15.73:0/1167671868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f852c0092a0 con 0x7f8538150890 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.970+0000 7f85377fe700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f853000c7e0 tx=0x7f8530007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.971+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530005d30 con 0x7f8538150890 2023-11-28T03:07:34.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.971+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8530005e90 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.971+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8530007320 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.971+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85380a81b0 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.971+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85380a7e40 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85380a8360 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f8530007710 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f851effd700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 0x7f85240617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f8536ffd700 1 -- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 msgr2=0x7f85240617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f853007a4c0 con 0x7f8538150890 2023-11-28T03:07:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.972+0000 7f8536ffd700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 0x7f85240617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:34.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:34.983+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85380a8360 con 0x7f8538150890 2023-11-28T03:07:35.008 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:35.007+0000 7fdfc50fa680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:35.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.154+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f853806c860 con 0x7f8538150890 2023-11-28T03:07:35.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.155+0000 7f851effd700 1 -- 172.21.15.73:0/1167671868 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17172 (secure 0 0 0) 0x7f853004b500 con 0x7f8538150890 2023-11-28T03:07:35.156 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:35.157 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:29.961136+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/4214520789":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6833/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6832/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:0/954922268":"2023-11-29T03:07:29.961096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:35.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 msgr2=0x7f85240617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:35.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 0x7f85240617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 msgr2=0x7f85380a9110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 rx=0x7f853000c7e0 tx=0x7f8530007f90).stop 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 shutdown_connections 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f852405f300 0x7f85240617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8538150890 0x7f85380a9110 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8538069df0 0x7f8538144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 --2- 172.21.15.73:0/1167671868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85380bc320 0x7f85380a8c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 >> 172.21.15.73:0/1167671868 conn(0x7f85380631d0 msgr2=0x7f85380c4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 shutdown_connections 2023-11-28T03:07:35.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.157+0000 7f853d792700 1 -- 172.21.15.73:0/1167671868 wait complete. 2023-11-28T03:07:35.171 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:07:35.171 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:35.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.287+0000 7fef99ba2700 1 Processor -- start 2023-11-28T03:07:35.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.287+0000 7fef99ba2700 1 -- start start 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef99ba2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fef940c0180 0x7fef94157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef99ba2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940c12c0 con 0x7fef94157660 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef99ba2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef940c06b0 con 0x7fef940c1750 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef99ba2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef940c0990 con 0x7fef940c0180 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef98ba0700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fef94157660 0x7fef941589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45130/0 (socket says 172.21.15.73:45130) 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef98ba0700 1 -- 172.21.15.73:0/1111952398 learned_addr learned my addr 172.21.15.73:0/1111952398 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:35.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.288+0000 7fef93fff700 1 --2- 172.21.15.73:0/1111952398 >> v2:172.21.15.150:3300/0 conn(0x7fef940c0180 0x7fef94157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 789344103 0 0) 0x7fef940c12c0 con 0x7fef94157660 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef840033f0 con 0x7fef94157660 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203899129 0 0) 0x7fef940c06b0 con 0x7fef940c1750 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fef940c12c0 con 0x7fef940c1750 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef93fff700 1 -- 172.21.15.73:0/1111952398 >> v1:172.21.15.150:6789/0 conn(0x7fef940c1750 legacy=0x7fef940648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef93fff700 1 -- 172.21.15.73:0/1111952398 >> v1:172.21.15.73:6789/0 conn(0x7fef94157660 legacy=0x7fef941589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef93fff700 1 -- 172.21.15.73:0/1111952398 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef940c0c70 con 0x7fef940c0180 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef93fff700 1 --2- 172.21.15.73:0/1111952398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940c0180 0x7fef94157a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fef80009760 tx=0x7fef80009060).ready entity=mon.1 client_cookie=3dd0e718c4a55aef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000d4a0 con 0x7fef940c0180 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef8000d600 con 0x7fef940c0180 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.289+0000 7fef937fe700 1 -- 172.21.15.73:0/1111952398 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef8000d980 con 0x7fef940c0180 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 -- 172.21.15.73:0/1111952398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940c0180 msgr2=0x7fef94157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1111952398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940c0180 0x7fef94157a40 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 rx=0x7fef80009760 tx=0x7fef80009060).stop 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 -- 172.21.15.73:0/1111952398 shutdown_connections 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1111952398 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef940c0180 0x7fef94157a40 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 -- 172.21.15.73:0/1111952398 >> 172.21.15.73:0/1111952398 conn(0x7fef940cb000 msgr2=0x7fef940cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:35.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 -- 172.21.15.73:0/1111952398 shutdown_connections 2023-11-28T03:07:35.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.290+0000 7fef99ba2700 1 -- 172.21.15.73:0/1111952398 wait complete. 2023-11-28T03:07:35.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.291+0000 7fef99ba2700 1 Processor -- start 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.291+0000 7fef99ba2700 1 -- start start 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940c1750 0x7fef940adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef94157660 0x7fef94143ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef94153fa0 con 0x7fef940c0180 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef940afea0 con 0x7fef94157660 2023-11-28T03:07:35.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef940aeec0 con 0x7fef940c1750 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54028/0 (socket says 172.21.15.73:54028) 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 -- 172.21.15.73:0/1480719736 learned_addr learned my addr 172.21.15.73:0/1480719736 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef93fff700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940c1750 0x7fef940adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef993a1700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef94157660 0x7fef94143ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940c1750 msgr2=0x7fef940adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940c1750 0x7fef940adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef94157660 msgr2=0x7fef94143ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef94157660 0x7fef94143ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 -- 172.21.15.73:0/1480719736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef800092a0 con 0x7fef940c0180 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef98ba0700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fef8800c4c0 tx=0x7fef88007d20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880070d0 con 0x7fef940c0180 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef88007230 con 0x7fef940c0180 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef880075b0 con 0x7fef940c0180 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef940ae900 con 0x7fef940c0180 2023-11-28T03:07:35.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.292+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef940afb30 con 0x7fef940c0180 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef9406c7c0 con 0x7fef940c0180 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fef880059b0 con 0x7fef940c0180 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef90ff9700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 0x7fef780617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef93fff700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 msgr2=0x7fef780617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fef88079c00 con 0x7fef940c0180 2023-11-28T03:07:35.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.294+0000 7fef93fff700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 0x7fef780617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:35.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.298+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fef8807f020 con 0x7fef940c0180 2023-11-28T03:07:35.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.495+0000 7fef93fff700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 msgr2=0x7fef780617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:35.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.495+0000 7fef93fff700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 0x7fef780617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:35.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.511+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef940c0f50 con 0x7fef940c0180 2023-11-28T03:07:35.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.512+0000 7fef90ff9700 1 -- 172.21.15.73:0/1480719736 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fef8804ac20 con 0x7fef940c0180 2023-11-28T03:07:35.513 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:35.513 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:35.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 msgr2=0x7fef780617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:35.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 0x7fef780617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 msgr2=0x7fef94153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 rx=0x7fef8800c4c0 tx=0x7fef88007d20).stop 2023-11-28T03:07:35.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 shutdown_connections 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fef7805f300 0x7fef780617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef94157660 0x7fef94143ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef940c0180 0x7fef94153ab0 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 --2- 172.21.15.73:0/1480719736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef940c1750 0x7fef940adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 >> 172.21.15.73:0/1480719736 conn(0x7fef940cb000 msgr2=0x7fef940ac690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 shutdown_connections 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.514+0000 7fef99ba2700 1 -- 172.21.15.73:0/1480719736 wait complete. 2023-11-28T03:07:35.516 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:35.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:35.541 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:35.540+0000 7f00f7d8e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:35.608 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:35.608+0000 7f00f7d8e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:35.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.646+0000 7feb4bd82700 1 Processor -- start 2023-11-28T03:07:35.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.646+0000 7feb4bd82700 1 -- start start 2023-11-28T03:07:35.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4bd82700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 0x7feb440697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4bd82700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb44062e10 0x7feb440661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4bd82700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44065760 con 0x7feb44069ce0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4bd82700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb4406a160 con 0x7feb44062e10 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4bd82700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb440c3250 con 0x7feb441542d0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4931d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feb44069ce0 0x7feb44062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45142/0 (socket says 172.21.15.73:45142) 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4931d700 1 -- 172.21.15.73:0/1560988045 learned_addr learned my addr 172.21.15.73:0/1560988045 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4a31f700 1 --2- 172.21.15.73:0/1560988045 >> v2:172.21.15.150:3300/0 conn(0x7feb44062e10 0x7feb440661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb49b1e700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 0x7feb440697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb48b1c700 1 -- 172.21.15.73:0/1560988045 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428875173 0 0) 0x7feb44065760 con 0x7feb44069ce0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb48b1c700 1 -- 172.21.15.73:0/1560988045 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb300033f0 con 0x7feb44069ce0 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4a31f700 1 -- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 msgr2=0x7feb440697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4a31f700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 0x7feb440697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4a31f700 1 -- 172.21.15.73:0/1560988045 >> v1:172.21.15.73:6789/0 conn(0x7feb44069ce0 legacy=0x7feb44062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb4a31f700 1 -- 172.21.15.73:0/1560988045 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb4406a340 con 0x7feb44062e10 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.647+0000 7feb49b1e700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 0x7feb440697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4a31f700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb44062e10 0x7feb440661b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7feb3c009e60 tx=0x7feb3c009780).ready entity=mon.1 client_cookie=d2f064c77d12a107 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb48b1c700 1 -- 172.21.15.73:0/1560988045 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00e070 con 0x7feb44062e10 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb48b1c700 1 -- 172.21.15.73:0/1560988045 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb3c00bab0 con 0x7feb44062e10 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb48b1c700 1 -- 172.21.15.73:0/1560988045 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3c00be30 con 0x7feb44062e10 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 -- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb44062e10 msgr2=0x7feb440661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb44062e10 0x7feb440661b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 rx=0x7feb3c009e60 tx=0x7feb3c009780).stop 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 -- 172.21.15.73:0/1560988045 shutdown_connections 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb44062e10 0x7feb440661b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 --2- 172.21.15.73:0/1560988045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb441542d0 0x7feb440697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 -- 172.21.15.73:0/1560988045 >> 172.21.15.73:0/1560988045 conn(0x7feb440c7fd0 msgr2=0x7feb440c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 -- 172.21.15.73:0/1560988045 shutdown_connections 2023-11-28T03:07:35.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.648+0000 7feb4bd82700 1 -- 172.21.15.73:0/1560988045 wait complete. 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.649+0000 7feb4bd82700 1 Processor -- start 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 -- start start 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb44062e10 0x7feb440ad270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb441542d0 0x7feb44164270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb440a8050 con 0x7feb44069ce0 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb440a7a90 con 0x7feb441542d0 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4bd82700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb441648f0 con 0x7feb44062e10 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4931d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4931d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54034/0 (socket says 172.21.15.73:54034) 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4931d700 1 -- 172.21.15.73:0/324673958 learned_addr learned my addr 172.21.15.73:0/324673958 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb49b1e700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb44062e10 0x7feb440ad270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.650+0000 7feb4a31f700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb441542d0 0x7feb44164270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb44062e10 msgr2=0x7feb440ad270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb44062e10 0x7feb440ad270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb441542d0 msgr2=0x7feb44164270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb441542d0 0x7feb44164270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 -- 172.21.15.73:0/324673958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb400092a0 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4931d700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7feb3400c4c0 tx=0x7feb34007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400ce70 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb340072e0 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c009a20 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340057a0 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.651+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb440a7ce0 con 0x7feb44069ce0 2023-11-28T03:07:35.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.652+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb4406c860 con 0x7feb44069ce0 2023-11-28T03:07:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.653+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7feb34005bd0 con 0x7feb44069ce0 2023-11-28T03:07:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.653+0000 7feb39ffb700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 0x7feb280617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.653+0000 7feb49b1e700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 msgr2=0x7feb280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:35.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.653+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7feb34079be0 con 0x7feb44069ce0 2023-11-28T03:07:35.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.653+0000 7feb49b1e700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 0x7feb280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:35.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.658+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feb3404ac00 con 0x7feb44069ce0 2023-11-28T03:07:35.684 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:35.682+0000 7f00f7d8e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:35.693 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:35.692+0000 7fdfc50fa680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:35.764 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:35.763+0000 7fdfc50fa680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:35.829 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:35.828+0000 7f00f7d8e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:35.840 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:35.839+0000 7fdfc50fa680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.854+0000 7feb49b1e700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 msgr2=0x7feb280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:35.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.854+0000 7feb49b1e700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 0x7feb280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:35.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.868+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb4405f660 con 0x7feb44069ce0 2023-11-28T03:07:35.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.868+0000 7feb39ffb700 1 -- 172.21.15.73:0/324673958 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+4496 (secure 0 0 0) 0x7feb3404e220 con 0x7feb44069ce0 2023-11-28T03:07:35.869 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:35.869 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":26,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:18.675550+0000","modified":"2023-11-28T03:07:20.681373+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6246},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6246":{"gid":6246,"name":"a","rank":0,"incarnation":24,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/2389605358","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2389605358},{"type":"v1","addr":"172.21.15.73:6837","nonce":2389605358}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[10],"metadata_pool":9,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":4}]} 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 msgr2=0x7feb280617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 0x7feb280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 msgr2=0x7feb44163d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 rx=0x7feb3400c4c0 tx=0x7feb34007c50).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 shutdown_connections 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb2805f300 0x7feb280617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb441542d0 0x7feb44164270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb44062e10 0x7feb440ad270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 --2- 172.21.15.73:0/324673958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb44069ce0 0x7feb44163d80 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:35.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 >> 172.21.15.73:0/324673958 conn(0x7feb440c7fd0 msgr2=0x7feb440c5fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:35.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.870+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 shutdown_connections 2023-11-28T03:07:35.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:35.871+0000 7feb4bd82700 1 -- 172.21.15.73:0/324673958 wait complete. 2023-11-28T03:07:35.873 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 26 2023-11-28T03:07:35.884 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:35.898 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:35.897+0000 7f00f7d8e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:35.998 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:35.997+0000 7fdfc50fa680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:36.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.001+0000 7ff8d211e700 1 Processor -- start 2023-11-28T03:07:36.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.001+0000 7ff8d211e700 1 -- start start 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8d211e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8d211e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0683d0 con 0x7ff8cc064e20 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8d211e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc067850 con 0x7ff8cc1544d0 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8d211e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc067650 con 0x7ff8cc096590 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8caffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff8cc064e20 0x7ff8cc095e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45144/0 (socket says 172.21.15.73:45144) 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.002+0000 7ff8caffd700 1 -- 172.21.15.73:0/1650277066 learned_addr learned my addr 172.21.15.73:0/1650277066 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8cbfff700 1 --2- 172.21.15.73:0/1650277066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1224835107 0 0) 0x7ff8cc0683d0 con 0x7ff8cc064e20 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b80033f0 con 0x7ff8cc064e20 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 73984691 0 0) 0x7ff8cc067850 con 0x7ff8cc1544d0 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8cc0683d0 con 0x7ff8cc1544d0 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8cbfff700 1 -- 172.21.15.73:0/1650277066 >> v1:172.21.15.150:6789/0 conn(0x7ff8cc1544d0 legacy=0x7ff8cc066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2556627367 0 0) 0x7ff8b80033f0 con 0x7ff8cc064e20 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8cbfff700 1 -- 172.21.15.73:0/1650277066 >> v1:172.21.15.73:6789/0 conn(0x7ff8cc064e20 legacy=0x7ff8cc095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8cbfff700 1 -- 172.21.15.73:0/1650277066 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc067b30 con 0x7ff8cc096590 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8cbfff700 1 --2- 172.21.15.73:0/1650277066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc155750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff8bc009790 tx=0x7ff8bc009060).ready entity=mon.2 client_cookie=a660ff3fbaa9feb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8b4001470 con 0x7ff8cc096590 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc011400 con 0x7ff8cc096590 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.003+0000 7ff8ca7fc700 1 -- 172.21.15.73:0/1650277066 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc011780 con 0x7ff8cc096590 2023-11-28T03:07:36.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 -- 172.21.15.73:0/1650277066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 msgr2=0x7ff8cc155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1650277066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc155750 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 rx=0x7ff8bc009790 tx=0x7ff8bc009060).stop 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 -- 172.21.15.73:0/1650277066 shutdown_connections 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1650277066 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc155750 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 -- 172.21.15.73:0/1650277066 >> 172.21.15.73:0/1650277066 conn(0x7ff8cc0631d0 msgr2=0x7ff8cc0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 -- 172.21.15.73:0/1650277066 shutdown_connections 2023-11-28T03:07:36.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.004+0000 7ff8d211e700 1 -- 172.21.15.73:0/1650277066 wait complete. 2023-11-28T03:07:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.005+0000 7ff8d211e700 1 Processor -- start 2023-11-28T03:07:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.005+0000 7ff8d211e700 1 -- start start 2023-11-28T03:07:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc14cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 0x7ff8cc0c8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc168750 con 0x7ff8cc064e20 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0c0e20 con 0x7ff8cc1544d0 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8d211e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0c1100 con 0x7ff8cc096590 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54042/0 (socket says 172.21.15.73:54042) 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 -- 172.21.15.73:0/1196235255 learned_addr learned my addr 172.21.15.73:0/1196235255 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8caffd700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc14cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cbfff700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 0x7ff8cc0c8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 -- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 msgr2=0x7ff8cc14cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc14cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 -- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 msgr2=0x7ff8cc0c8320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 0x7ff8cc0c8320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cb7fe700 1 -- 172.21.15.73:0/1196235255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc0092a0 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.006+0000 7ff8cbfff700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 0x7ff8cc0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8cb7fe700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7ff8c400c7c0 tx=0x7ff8c4002650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400e4a0 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8c400e600 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8c400e9a0 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc0c13e0 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.007+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc0c0ad0 con 0x7ff8cc064e20 2023-11-28T03:07:36.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.008+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc06c860 con 0x7ff8cc064e20 2023-11-28T03:07:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.009+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff8c400ed10 con 0x7ff8cc064e20 2023-11-28T03:07:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.009+0000 7ff8c37fe700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 0x7ff8ac0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.009+0000 7ff8caffd700 1 -- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 msgr2=0x7ff8ac0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:36.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.009+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff8c407a0c0 con 0x7ff8cc064e20 2023-11-28T03:07:36.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.009+0000 7ff8caffd700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 0x7ff8ac0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:36.014 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:36.013+0000 7f00f7d8e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:36.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.013+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff8c404b160 con 0x7ff8cc064e20 2023-11-28T03:07:36.075 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.074+0000 7fdfc50fa680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:36.123 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:36.122+0000 7f00f7d8e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:36.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.183+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff8cc067e10 con 0x7ff8cc064e20 2023-11-28T03:07:36.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.184+0000 7ff8c37fe700 1 -- 172.21.15.73:0/1196235255 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v83) v1 ==== 74+0+17172 (secure 0 0 0) 0x7ff8c404e780 con 0x7ff8cc064e20 2023-11-28T03:07:36.186 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:36.186 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":83,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:29.961136+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":9,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:17.214635+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"81","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":10,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:17.638322+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"82","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/4214520789":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6833/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:6832/90017":"2023-11-29T03:07:29.961096+0000","172.21.15.73:0/954922268":"2023-11-29T03:07:29.961096+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.186+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 msgr2=0x7ff8ac0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 0x7ff8ac0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 msgr2=0x7ff8cc168260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 rx=0x7ff8c400c7c0 tx=0x7ff8c4002650).stop 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 shutdown_connections 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff8ac05f300 0x7ff8ac0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8cc1544d0 0x7ff8cc0c8320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8cc064e20 0x7ff8cc168260 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 --2- 172.21.15.73:0/1196235255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8cc096590 0x7ff8cc14cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 >> 172.21.15.73:0/1196235255 conn(0x7ff8cc0631d0 msgr2=0x7ff8cc0c4f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 shutdown_connections 2023-11-28T03:07:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.187+0000 7ff8d211e700 1 -- 172.21.15.73:0/1196235255 wait complete. 2023-11-28T03:07:36.196 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.194+0000 7fdfc50fa680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:36.201 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:07:36.312 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.311+0000 7fdfc50fa680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.318+0000 7f5f4a131700 1 Processor -- start 2023-11-28T03:07:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.318+0000 7f5f4a131700 1 -- start start 2023-11-28T03:07:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f4a131700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5f44151a20 0x7f5f4406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f4a131700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5f440bd1f0 0x7f5f440bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f4a131700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f440c3470 con 0x7f5f44069d10 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f4a131700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440c48f0 con 0x7f5f440bd1f0 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f4a131700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f440bdb70 con 0x7f5f44151a20 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5f440bd1f0 0x7f5f440bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f437fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5f44069d10 0x7f5f440be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45150/0 (socket says 172.21.15.73:45150) 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f437fe700 1 -- 172.21.15.73:0/2579512565 learned_addr learned my addr 172.21.15.73:0/2579512565 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f42ffd700 1 --2- 172.21.15.73:0/2579512565 >> v2:172.21.15.150:3300/0 conn(0x7f5f44151a20 0x7f5f4406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 -- 172.21.15.73:0/2579512565 >> v2:172.21.15.150:3300/0 conn(0x7f5f44151a20 msgr2=0x7f5f4406a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 --2- 172.21.15.73:0/2579512565 >> v2:172.21.15.150:3300/0 conn(0x7f5f44151a20 0x7f5f4406a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 -- 172.21.15.73:0/2579512565 >> v1:172.21.15.73:6789/0 conn(0x7f5f44069d10 legacy=0x7f5f440be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 -- 172.21.15.73:0/2579512565 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f44151ea0 con 0x7f5f440bd1f0 2023-11-28T03:07:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f427fc700 1 -- 172.21.15.73:0/2579512565 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4015485892 0 0) 0x7f5f440c3470 con 0x7f5f44069d10 2023-11-28T03:07:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f43fff700 1 --2- 172.21.15.73:0/2579512565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f440bd680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f5f44069370 tx=0x7f5f3c009780).ready entity=mon.0 client_cookie=cbe72448b864f819 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f427fc700 1 -- 172.21.15.73:0/2579512565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c00e070 con 0x7f5f440bd1f0 2023-11-28T03:07:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f427fc700 1 -- 172.21.15.73:0/2579512565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f3c00bb60 con 0x7f5f440bd1f0 2023-11-28T03:07:36.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.319+0000 7f5f427fc700 1 -- 172.21.15.73:0/2579512565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f3c012410 con 0x7f5f440bd1f0 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 -- 172.21.15.73:0/2579512565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 msgr2=0x7f5f440bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 --2- 172.21.15.73:0/2579512565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f440bd680 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 rx=0x7f5f44069370 tx=0x7f5f3c009780).stop 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 -- 172.21.15.73:0/2579512565 shutdown_connections 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 --2- 172.21.15.73:0/2579512565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f440bd680 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 --2- 172.21.15.73:0/2579512565 >> v2:172.21.15.150:3300/0 conn(0x7f5f44151a20 0x7f5f4406a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 -- 172.21.15.73:0/2579512565 >> 172.21.15.73:0/2579512565 conn(0x7f5f440642d0 msgr2=0x7f5f440c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 -- 172.21.15.73:0/2579512565 shutdown_connections 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.320+0000 7f5f4a131700 1 -- 172.21.15.73:0/2579512565 wait complete. 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.321+0000 7f5f4a131700 1 Processor -- start 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.321+0000 7f5f4a131700 1 -- start start 2023-11-28T03:07:36.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f44069d10 0x7f5f440c15f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f44151a20 0x7f5f44150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414fbf0 con 0x7f5f440bd1f0 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4414fed0 con 0x7f5f44069d10 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f4a131700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f441501b0 con 0x7f5f44151a20 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54056/0 (socket says 172.21.15.73:54056) 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f42ffd700 1 -- 172.21.15.73:0/3255897947 learned_addr learned my addr 172.21.15.73:0/3255897947 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f437fe700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f44069d10 0x7f5f440c15f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.322+0000 7f5f43fff700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f44151a20 0x7f5f44150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:36.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f44151a20 msgr2=0x7f5f44150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f44151a20 0x7f5f44150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f44069d10 msgr2=0x7f5f440c15f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f44069d10 0x7f5f440c15f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 -- 172.21.15.73:0/3255897947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f3c009a20 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f42ffd700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f5f38010ae0 tx=0x7f5f38009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38009a40 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f38009ba0 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f38011450 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f44150490 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.323+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f440c1ae0 con 0x7f5f440bd1f0 2023-11-28T03:07:36.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.324+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f4406c860 con 0x7f5f440bd1f0 2023-11-28T03:07:36.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.324+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 28) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f5f38017070 con 0x7f5f440bd1f0 2023-11-28T03:07:36.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.325+0000 7f5f2b7fe700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 0x7f5f300617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:36.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.325+0000 7f5f437fe700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 msgr2=0x7f5f300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:36.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.325+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f5f3807c440 con 0x7f5f440bd1f0 2023-11-28T03:07:36.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.325+0000 7f5f437fe700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 0x7f5f300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:36.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.332+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5f3804d460 con 0x7f5f440bd1f0 2023-11-28T03:07:36.415 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:36.414+0000 7f00f7d8e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:36.487 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:36.486+0000 7f00f7d8e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.526+0000 7f5f437fe700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 msgr2=0x7f5f300617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.526+0000 7f5f437fe700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 0x7f5f300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:36.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.546+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f5f4406a620 con 0x7f5f440bd1f0 2023-11-28T03:07:36.614 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.613+0000 7fdfc50fa680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:36.686 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.685+0000 7fdfc50fa680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:07:36.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.704+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f5f38079f60 con 0x7f5f440bd1f0 2023-11-28T03:07:36.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.705+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 msgr2=0x7f5f300617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:36.705+0000 7f5f2b7fe700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 0x7f5f300617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:36.710 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:36.710 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:36.710 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:07:36.712 INFO:tasks.ceph.mds.a.smithi073.stderr: -3503> 2023-11-28T03:07:14.149+0000 7f4ff9c51700 -1 PosixStack listen unable to listen on v1:0.0.0.0:6836/0: (98) Address already in use 2023-11-28T03:07:36.715 INFO:tasks.ceph.mds.a.smithi073.stderr: -2132> 2023-11-28T03:07:24.597+0000 7f4ff3c45700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:07:36.718 INFO:tasks.ceph.mds.a.smithi073.stderr: -5> 2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:36.718 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:07:36.718 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T03:07:36.709+0000 7f4ff043e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:07:36.737 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:07:36.762 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf42091700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf42091700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf42091700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf42091700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf42091700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf3e88a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf3e88a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf3e88a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf3e88a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:36.763 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:36.761+0000 7fdf3e88a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:07:37.251 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:07:37.250+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:07:37.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.256+0000 7f5f2b7fe700 1 -- 172.21.15.73:0/3255897947 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7f5f38072040 con 0x7f5f440bd1f0 2023-11-28T03:07:37.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 msgr2=0x7f5f4413ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 rx=0x7f5f38010ae0 tx=0x7f5f38009060).stop 2023-11-28T03:07:37.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 shutdown_connections 2023-11-28T03:07:37.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5f3005f300 0x7f5f300617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:37.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5f44151a20 0x7f5f44150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5f44069d10 0x7f5f440c15f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 --2- 172.21.15.73:0/3255897947 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5f440bd1f0 0x7f5f4413ca60 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 >> 172.21.15.73:0/3255897947 conn(0x7f5f440642d0 msgr2=0x7f5f440c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:37.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 shutdown_connections 2023-11-28T03:07:37.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.259+0000 7f5f4a131700 1 -- 172.21.15.73:0/3255897947 wait complete. 2023-11-28T03:07:37.262 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:07:37.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:07:37.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.386+0000 7f94c9e71700 1 Processor -- start 2023-11-28T03:07:37.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.386+0000 7f94c9e71700 1 -- start start 2023-11-28T03:07:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c9e71700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94c40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c9e71700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c40c72d0 con 0x7f94c4098d10 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c9e71700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94c4064570 con 0x7f94c4158320 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c9e71700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4154bc0 con 0x7f94c40c6190 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c37fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f94c4098d10 0x7f94c4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45156/0 (socket says 172.21.15.73:45156) 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.387+0000 7f94c37fe700 1 -- 172.21.15.73:0/4218416566 learned_addr learned my addr 172.21.15.73:0/4218416566 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817603960 0 0) 0x7f94c40c72d0 con 0x7f94c4098d10 2023-11-28T03:07:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94ac0033f0 con 0x7f94c4098d10 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c2ffd700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94c40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1465883400 0 0) 0x7f94ac0033f0 con 0x7f94c4098d10 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 msgr2=0x7f94c40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94c40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 >> v1:172.21.15.150:6789/0 conn(0x7f94c4158320 legacy=0x7f94c40a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40c66c0 con 0x7f94c4098d10 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f94b40017f0 con 0x7f94c4098d10 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 >> v1:172.21.15.73:6789/0 conn(0x7f94c4098d10 legacy=0x7f94c4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94ac003910 0x7f94ac003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94ac004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac004b80 0x7f94ac005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4154bc0 con 0x7f94ac003910 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac00b910 con 0x7f94ac004b80 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94ac00bad0 con 0x7f94c40c6190 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94ac003910 0x7f94ac003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c2ffd700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94ac004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c3fff700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac004b80 0x7f94ac005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 -- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 msgr2=0x7f94ac004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94c40c6190 0x7f94ac004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 -- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac004b80 msgr2=0x7f94ac005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94ac004b80 0x7f94ac005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 -- 172.21.15.73:0/4218416566 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94ac0033f0 con 0x7f94ac003910 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.388+0000 7f94c37fe700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94ac003910 0x7f94ac003d30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f94b400c480 tx=0x7f94b400c790).ready entity=mon.0 client_cookie=b8c98c558d151bf3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4002030 con 0x7f94ac003910 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b400ce20 con 0x7f94ac003910 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c27fc700 1 -- 172.21.15.73:0/4218416566 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b4005d30 con 0x7f94ac003910 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 -- 172.21.15.73:0/4218416566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94ac003910 msgr2=0x7f94ac003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 --2- 172.21.15.73:0/4218416566 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94ac003910 0x7f94ac003d30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 rx=0x7f94b400c480 tx=0x7f94b400c790).stop 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c3fff700 1 -- 172.21.15.73:0/4218416566 reap_dead start 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 -- 172.21.15.73:0/4218416566 shutdown_connections 2023-11-28T03:07:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 -- 172.21.15.73:0/4218416566 >> 172.21.15.73:0/4218416566 conn(0x7f94c40cb310 msgr2=0x7f94c40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 -- 172.21.15.73:0/4218416566 shutdown_connections 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.389+0000 7f94c9e71700 1 -- 172.21.15.73:0/4218416566 wait complete. 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.390+0000 7f94c9e71700 1 Processor -- start 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.390+0000 7f94c9e71700 1 -- start start 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94ac003910 0x7f94c40cab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94c4164680 0x7f94c4164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c41657a0 con 0x7f94c40b0170 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94c4164ff0 con 0x7f94c4164680 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c9e71700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94c40b0690 con 0x7f94ac003910 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54186/0 (socket says 172.21.15.73:54186) 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c2ffd700 1 -- 172.21.15.73:0/367683167 learned_addr learned my addr 172.21.15.73:0/367683167 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c37fe700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94ac003910 0x7f94c40cab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c3fff700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94c4164680 0x7f94c4164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c2ffd700 1 -- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94ac003910 msgr2=0x7f94c40cab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.391+0000 7f94c2ffd700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94ac003910 0x7f94c40cab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c2ffd700 1 -- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94c4164680 msgr2=0x7f94c4164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c2ffd700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94c4164680 0x7f94c4164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c2ffd700 1 -- 172.21.15.73:0/367683167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94c40b0970 con 0x7f94c40b0170 2023-11-28T03:07:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c2ffd700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f94b8011a20 tx=0x7f94b8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b8011e60 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94b800f940 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94b800fcc0 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94c40b0c50 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.392+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94c40cb040 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.393+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94c4165950 con 0x7f94c40b0170 2023-11-28T03:07:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.393+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 29) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f94b8014650 con 0x7f94c40b0170 2023-11-28T03:07:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.393+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f94b807a860 con 0x7f94c40b0170 2023-11-28T03:07:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.397+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f94c4165950 con 0x7f94c40b0170 2023-11-28T03:07:37.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.602+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f94c406c860 con 0x7f94c40b0170 2023-11-28T03:07:37.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.707+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f94b8078620 con 0x7f94c40b0170 2023-11-28T03:07:37.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.707+0000 7f94ab7fe700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 0x7f94b0067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:37.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.708+0000 7f94c37fe700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 0x7f94b0067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:37.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:37.709+0000 7f94c37fe700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 0x7f94b0067aa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f94b400c7d0 tx=0x7f94b4015000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:38.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.257+0000 7f94ab7fe700 1 -- 172.21.15.73:0/367683167 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f94b806d310 con 0x7f94c40b0170 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.259+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 msgr2=0x7f94b0067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.259+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 0x7f94b0067aa0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f94b400c7d0 tx=0x7f94b4015000).stop 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.259+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 msgr2=0x7f94c4093640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.259+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 rx=0x7f94b8011a20 tx=0x7f94b8009060).stop 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.259+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 shutdown_connections 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f94b00302d0 0x7f94b0067aa0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f94c4164680 0x7f94c4164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f94ac003910 0x7f94c40cab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 --2- 172.21.15.73:0/367683167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f94c40b0170 0x7f94c4093640 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 >> 172.21.15.73:0/367683167 conn(0x7f94c40cb310 msgr2=0x7f94c4153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:38.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 shutdown_connections 2023-11-28T03:07:38.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.260+0000 7f94c9e71700 1 -- 172.21.15.73:0/367683167 wait complete. 2023-11-28T03:07:38.273 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:07:38.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.389+0000 7f4911000700 1 Processor -- start 2023-11-28T03:07:38.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.389+0000 7f4911000700 1 -- start start 2023-11-28T03:07:38.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4911000700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f490c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4911000700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c158d50 con 0x7f490c1546b0 2023-11-28T03:07:38.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4911000700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f490c158140 con 0x7f490c0c66d0 2023-11-28T03:07:38.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4911000700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c158420 con 0x7f490c1542d0 2023-11-28T03:07:38.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4902ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f490c1546b0 0x7f490c0c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45160/0 (socket says 172.21.15.73:45160) 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f4902ffd700 1 -- 172.21.15.73:0/467189852 learned_addr learned my addr 172.21.15.73:0/467189852 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490b7fe700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f490c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1335848896 0 0) 0x7f490c158d50 con 0x7f490c1546b0 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f48f80033f0 con 0x7f490c1546b0 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916757308 0 0) 0x7f490c158140 con 0x7f490c0c66d0 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c158d50 con 0x7f490c0c66d0 2023-11-28T03:07:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1987834739 0 0) 0x7f48f80033f0 con 0x7f490c1546b0 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 msgr2=0x7f490c0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f490c0a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 >> v1:172.21.15.150:6789/0 conn(0x7f490c0c66d0 legacy=0x7f490c0c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c158700 con 0x7f490c1546b0 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f48f4001470 con 0x7f490c1546b0 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 >> v1:172.21.15.73:6789/0 conn(0x7f490c1546b0 legacy=0x7f490c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f8003aa0 0x7f48f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f48f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8004b80 0x7f48f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c158420 con 0x7f48f8003aa0 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48f800b990 con 0x7f48f8004b80 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48f800bb00 con 0x7f490c1542d0 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.390+0000 7f490b7fe700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f8003aa0 0x7f48f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4902ffd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f48f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490bfff700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8004b80 0x7f48f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 -- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 msgr2=0x7f48f8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:38.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f48f8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 -- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8004b80 msgr2=0x7f48f8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8004b80 0x7f48f8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 -- 172.21.15.73:0/467189852 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48f80033f0 con 0x7f48f8003aa0 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4902ffd700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c1542d0 0x7f48f8004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490b7fe700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f8003aa0 0x7f48f8003ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f48fc009a80 tx=0x7f48fc009060).ready entity=mon.0 client_cookie=d7979a0ae5a995d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc009430 con 0x7f48f8003aa0 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48fc00e640 con 0x7f48f8003aa0 2023-11-28T03:07:38.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490affd700 1 -- 172.21.15.73:0/467189852 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48fc00e9c0 con 0x7f48f8003aa0 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 -- 172.21.15.73:0/467189852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f8003aa0 msgr2=0x7f48f8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 --2- 172.21.15.73:0/467189852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f48f8003aa0 0x7f48f8003ea0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 rx=0x7f48fc009a80 tx=0x7f48fc009060).stop 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f490bfff700 1 -- 172.21.15.73:0/467189852 reap_dead start 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 -- 172.21.15.73:0/467189852 shutdown_connections 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 -- 172.21.15.73:0/467189852 >> 172.21.15.73:0/467189852 conn(0x7f490c0cb010 msgr2=0x7f490c0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:38.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 -- 172.21.15.73:0/467189852 shutdown_connections 2023-11-28T03:07:38.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.391+0000 7f4911000700 1 -- 172.21.15.73:0/467189852 wait complete. 2023-11-28T03:07:38.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.393+0000 7f4911000700 1 Processor -- start 2023-11-28T03:07:38.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.393+0000 7f4911000700 1 -- start start 2023-11-28T03:07:38.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.393+0000 7f4911000700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 0x7f490c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.393+0000 7f4911000700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c0c2760 0x7f490c0c2bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c168960 con 0x7f490c167ff0 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0c34d0 con 0x7f48f8003aa0 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f490c0c37b0 con 0x7f490c0c2760 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54214/0 (socket says 172.21.15.73:54214) 2023-11-28T03:07:38.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 -- 172.21.15.73:0/290982425 learned_addr learned my addr 172.21.15.73:0/290982425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f490bfff700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c0c2760 0x7f490c0c2bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f490b7fe700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 0x7f490c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 -- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c0c2760 msgr2=0x7f490c0c2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c0c2760 0x7f490c0c2bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 -- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 msgr2=0x7f490c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 0x7f490c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 -- 172.21.15.73:0/290982425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f490c0c3a90 con 0x7f490c167ff0 2023-11-28T03:07:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f490b7fe700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 0x7f490c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4902ffd700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f48f400e810 tx=0x7f48f4000cc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4010520 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48f4010680 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f490c0c3d70 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48f4010a00 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.394+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f490c1537b0 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.395+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f490c1539a0 con 0x7f490c167ff0 2023-11-28T03:07:38.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.396+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f48f4014430 con 0x7f490c167ff0 2023-11-28T03:07:38.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.396+0000 7f4903fff700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 0x7f48ec0614c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:38.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.396+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(85..85 src has 1..85) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f48f407a230 con 0x7f490c167ff0 2023-11-28T03:07:38.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.396+0000 7f490b7fe700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 0x7f48ec0614c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:38.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.396+0000 7f490b7fe700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 0x7f48ec0614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f48fc006d50 tx=0x7f48fc01a000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:38.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.400+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f48f404b490 con 0x7f490c167ff0 2023-11-28T03:07:38.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:38.564+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f490c06c7c0 con 0x7f490c167ff0 2023-11-28T03:07:39.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.259+0000 7f4903fff700 1 -- 172.21.15.73:0/290982425 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v86) v1 ==== 181+0+0 (secure 0 0 0) 0x7f48f4010db0 con 0x7f490c167ff0 2023-11-28T03:07:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 msgr2=0x7f48ec0614c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 0x7f48ec0614c0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f48fc006d50 tx=0x7f48fc01a000).stop 2023-11-28T03:07:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 msgr2=0x7f490c168470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 rx=0x7f48f400e810 tx=0x7f48f4000cc0).stop 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 shutdown_connections 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f48ec05f000 0x7f48ec0614c0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f490c0c2760 0x7f490c0c2bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f48f8003aa0 0x7f490c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 --2- 172.21.15.73:0/290982425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f490c167ff0 0x7f490c168470 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 >> 172.21.15.73:0/290982425 conn(0x7f490c0cb010 msgr2=0x7f490c0ab100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 shutdown_connections 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.262+0000 7f4911000700 1 -- 172.21.15.73:0/290982425 wait complete. 2023-11-28T03:07:39.264 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:07:39.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:07:39.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.393+0000 7f0db5164700 1 Processor -- start 2023-11-28T03:07:39.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.393+0000 7f0db5164700 1 -- start start 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0db5164700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0db01373a0 0x7f0db00c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0db5164700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0db0064b00 0x7f0db00c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0db5164700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db00c68d0 con 0x7f0db00c5f30 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0db5164700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db0158e20 con 0x7f0db01373a0 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0db5164700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db01582a0 con 0x7f0db0064b00 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0daed9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0db01373a0 0x7f0db00c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0da659c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0db00c5f30 0x7f0db00669a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45166/0 (socket says 172.21.15.73:45166) 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.394+0000 7f0da659c700 1 -- 172.21.15.73:0/3535073292 learned_addr learned my addr 172.21.15.73:0/3535073292 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0daed9d700 1 -- 172.21.15.73:0/3535073292 >> v2:172.21.15.150:3300/0 conn(0x7f0db0064b00 msgr2=0x7f0db00c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0daed9d700 1 --2- 172.21.15.73:0/3535073292 >> v2:172.21.15.150:3300/0 conn(0x7f0db0064b00 0x7f0db00c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:39.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0daed9d700 1 -- 172.21.15.73:0/3535073292 >> v1:172.21.15.73:6789/0 conn(0x7f0db00c5f30 legacy=0x7f0db00669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0dae59c700 1 -- 172.21.15.73:0/3535073292 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 844530479 0 0) 0x7f0d98001110 con 0x7f0db00c5f30 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0daed9d700 1 -- 172.21.15.73:0/3535073292 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db0158580 con 0x7f0db01373a0 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0daed9d700 1 --2- 172.21.15.73:0/3535073292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db01373a0 0x7f0db00c72e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f0db00c5b00 tx=0x7f0da0009060).ready entity=mon.0 client_cookie=547188c181cde9a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0dae59c700 1 -- 172.21.15.73:0/3535073292 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da000b580 con 0x7f0db01373a0 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0dae59c700 1 -- 172.21.15.73:0/3535073292 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da000b6e0 con 0x7f0db01373a0 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0dae59c700 1 -- 172.21.15.73:0/3535073292 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da000ba60 con 0x7f0db01373a0 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 -- 172.21.15.73:0/3535073292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db01373a0 msgr2=0x7f0db00c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 --2- 172.21.15.73:0/3535073292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db01373a0 0x7f0db00c72e0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 rx=0x7f0db00c5b00 tx=0x7f0da0009060).stop 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 -- 172.21.15.73:0/3535073292 shutdown_connections 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 --2- 172.21.15.73:0/3535073292 >> v2:172.21.15.150:3300/0 conn(0x7f0db0064b00 0x7f0db00c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:39.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 --2- 172.21.15.73:0/3535073292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db01373a0 0x7f0db00c72e0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.395+0000 7f0db5164700 1 -- 172.21.15.73:0/3535073292 >> 172.21.15.73:0/3535073292 conn(0x7f0db00628f0 msgr2=0x7f0db0062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:39.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.396+0000 7f0db5164700 1 -- 172.21.15.73:0/3535073292 shutdown_connections 2023-11-28T03:07:39.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.396+0000 7f0db5164700 1 -- 172.21.15.73:0/3535073292 wait complete. 2023-11-28T03:07:39.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 Processor -- start 2023-11-28T03:07:39.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 -- start start 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db0064b00 0x7f0db014e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db01373a0 0x7f0db00ab1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db014e630 con 0x7f0db00c5f30 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db014fbd0 con 0x7f0db01373a0 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.397+0000 7f0db5164700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db014efc0 con 0x7f0db0064b00 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54234/0 (socket says 172.21.15.73:54234) 2023-11-28T03:07:39.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 -- 172.21.15.73:0/2590385552 learned_addr learned my addr 172.21.15.73:0/2590385552 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0daf59e700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db01373a0 0x7f0db00ab1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 -- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db0064b00 msgr2=0x7f0db014e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db0064b00 0x7f0db014e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 -- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db01373a0 msgr2=0x7f0db00ab1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db01373a0 0x7f0db00ab1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 -- 172.21.15.73:0/2590385552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da00092a0 con 0x7f0db00c5f30 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da659c700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f0d9800c540 tx=0x7f0d98007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980072e0 con 0x7f0db00c5f30 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d98007440 con 0x7f0db00c5f30 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db014f2a0 con 0x7f0db00c5f30 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d980057a0 con 0x7f0db00c5f30 2023-11-28T03:07:39.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.398+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db014f860 con 0x7f0db00c5f30 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.399+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db006c860 con 0x7f0db00c5f30 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.399+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 30) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f0d98005bd0 con 0x7f0db00c5f30 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.400+0000 7f0da77fe700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 0x7f0d900614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.400+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f0d98079650 con 0x7f0db00c5f30 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.400+0000 7f0daed9d700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 0x7f0d900614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.400+0000 7f0daed9d700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 0x7f0d900614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0db00c95d0 tx=0x7f0da0017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:39.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.404+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d9804aa70 con 0x7f0db00c5f30 2023-11-28T03:07:39.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.566+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0db00c68d0 con 0x7f0db00c5f30 2023-11-28T03:07:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:39.710+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0d980775d0 con 0x7f0db00c5f30 2023-11-28T03:07:40.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.261+0000 7f0da77fe700 1 -- 172.21.15.73:0/2590385552 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v87) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0d98076650 con 0x7f0db00c5f30 2023-11-28T03:07:40.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 msgr2=0x7f0d900614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 0x7f0d900614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f0db00c95d0 tx=0x7f0da0017000).stop 2023-11-28T03:07:40.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 msgr2=0x7f0db00aacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 rx=0x7f0d9800c540 tx=0x7f0d98007ad0).stop 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 shutdown_connections 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d9005f030 0x7f0d900614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0db01373a0 0x7f0db00ab1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0db0064b00 0x7f0db014e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 --2- 172.21.15.73:0/2590385552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0db00c5f30 0x7f0db00aacb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 >> 172.21.15.73:0/2590385552 conn(0x7f0db00628f0 msgr2=0x7f0db0154200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 shutdown_connections 2023-11-28T03:07:40.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.264+0000 7f0db5164700 1 -- 172.21.15.73:0/2590385552 wait complete. 2023-11-28T03:07:40.267 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:07:40.279 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:07:40.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.391+0000 7fe16b59e700 1 Processor -- start 2023-11-28T03:07:40.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.391+0000 7fe16b59e700 1 -- start start 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe16c098cc0 0x7fe16c1580e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 0x7fe16c1585d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c063eb0 0x7fe16c064320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c064ec0 con 0x7fe16c098cc0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe16c158ac0 con 0x7fe16c155ce0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16b59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0c0c80 con 0x7fe16c063eb0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe16c098cc0 0x7fe16c1580e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe16c098cc0 0x7fe16c1580e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54246/0 (socket says 172.21.15.73:54246) 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 -- 172.21.15.73:0/3744047322 learned_addr learned my addr 172.21.15.73:0/3744047322 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe161d9b700 1 --2- 172.21.15.73:0/3744047322 >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 0x7fe16c1585d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16ad9d700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c063eb0 0x7fe16c064320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 -- 172.21.15.73:0/3744047322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c063eb0 msgr2=0x7fe16c064320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c063eb0 0x7fe16c064320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 -- 172.21.15.73:0/3744047322 >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 msgr2=0x7fe16c1585d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 --2- 172.21.15.73:0/3744047322 >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 0x7fe16c1585d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe16a59c700 1 -- 172.21.15.73:0/3744047322 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe16c064be0 con 0x7fe16c098cc0 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.392+0000 7fe161d9b700 1 --2- 172.21.15.73:0/3744047322 >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 0x7fe16c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16a59c700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c098cc0 0x7fe16c1580e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe16c060080 tx=0x7fe15c0090f0).ready entity=mon.0 client_cookie=cbbe7e9cdfde51dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe169d9b700 1 -- 172.21.15.73:0/3744047322 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00b600 con 0x7fe16c098cc0 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe169d9b700 1 -- 172.21.15.73:0/3744047322 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00b760 con 0x7fe16c098cc0 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe169d9b700 1 -- 172.21.15.73:0/3744047322 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00bae0 con 0x7fe16c098cc0 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 -- 172.21.15.73:0/3744047322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c098cc0 msgr2=0x7fe16c1580e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c098cc0 0x7fe16c1580e0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 rx=0x7fe16c060080 tx=0x7fe15c0090f0).stop 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 -- 172.21.15.73:0/3744047322 shutdown_connections 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c098cc0 0x7fe16c1580e0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 --2- 172.21.15.73:0/3744047322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c063eb0 0x7fe16c064320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 --2- 172.21.15.73:0/3744047322 >> v2:172.21.15.150:3300/0 conn(0x7fe16c155ce0 0x7fe16c1585d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 -- 172.21.15.73:0/3744047322 >> 172.21.15.73:0/3744047322 conn(0x7fe16c0cb0f0 msgr2=0x7fe16c0cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 -- 172.21.15.73:0/3744047322 shutdown_connections 2023-11-28T03:07:40.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.393+0000 7fe16b59e700 1 -- 172.21.15.73:0/3744047322 wait complete. 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.394+0000 7fe16b59e700 1 Processor -- start 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 -- start start 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 0x7fe16c0ad580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c155ce0 0x7fe16c1572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0abbf0 con 0x7fe16c063eb0 2023-11-28T03:07:40.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c1516a0 con 0x7fe16c098cc0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16b59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe16c0aafe0 con 0x7fe16c155ce0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54252/0 (socket says 172.21.15.73:54252) 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.395+0000 7fe16a59c700 1 -- 172.21.15.73:0/1592579826 learned_addr learned my addr 172.21.15.73:0/1592579826 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe161d9b700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 0x7fe16c0ad580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16ad9d700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c155ce0 0x7fe16c1572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 -- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c155ce0 msgr2=0x7fe16c1572b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c155ce0 0x7fe16c1572b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 -- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 msgr2=0x7fe16c0ad580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 0x7fe16c0ad580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 -- 172.21.15.73:0/1592579826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c009370 con 0x7fe16c063eb0 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe161d9b700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 0x7fe16c0ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:40.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16a59c700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fe16c05e790 tx=0x7fe15c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c00b8d0 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15c00ba30 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe16c0ab2c0 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15c01b690 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.396+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe16c0ab880 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.397+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe16c06c7c0 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.397+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe15c01bac0 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.397+0000 7fe162ffd700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 0x7fe14c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.398+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fe15c07b870 con 0x7fe16c063eb0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.398+0000 7fe161d9b700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 0x7fe14c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.398+0000 7fe161d9b700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 0x7fe14c061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe16c0c9c90 tx=0x7fe154008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.401+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe15c04cd70 con 0x7fe16c063eb0 2023-11-28T03:07:40.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.576+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe16c0c9060 con 0x7fe16c063eb0 2023-11-28T03:07:40.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.580+0000 7fe162ffd700 1 -- 172.21.15.73:0/1592579826 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v12) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe15c050390 con 0x7fe16c063eb0 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 msgr2=0x7fe14c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 0x7fe14c061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fe16c0c9c90 tx=0x7fe154008040).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 msgr2=0x7fe16c1511b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 rx=0x7fe16c05e790 tx=0x7fe15c017000).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 shutdown_connections 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe14c05f150 0x7fe14c061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe16c155ce0 0x7fe16c1572b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe16c063eb0 0x7fe16c1511b0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 --2- 172.21.15.73:0/1592579826 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe16c098cc0 0x7fe16c0ad580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 >> 172.21.15.73:0/1592579826 conn(0x7fe16c0cb0f0 msgr2=0x7fe16c0a9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:40.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 shutdown_connections 2023-11-28T03:07:40.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.582+0000 7fe16b59e700 1 -- 172.21.15.73:0/1592579826 wait complete. 2023-11-28T03:07:40.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:07:40.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.709+0000 7f174e9e0700 1 Processor -- start 2023-11-28T03:07:40.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.709+0000 7f174e9e0700 1 -- start start 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174e9e0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1748066e10 0x7f174813d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174e9e0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1748063810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174e9e0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17481586a0 con 0x7f1748066a30 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174e9e0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1748157de0 con 0x7f1748066e10 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174e9e0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17480c7140 con 0x7f174813d940 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174d9de700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1748066a30 0x7f17480c6a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45168/0 (socket says 172.21.15.73:45168) 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.710+0000 7f174d9de700 1 -- 172.21.15.73:0/3768779183 learned_addr learned my addr 172.21.15.73:0/3768779183 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d1dd700 1 --2- 172.21.15.73:0/3768779183 >> v2:172.21.15.150:3300/0 conn(0x7f1748066e10 0x7f174813d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174e1df700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1748063810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3761253473 0 0) 0x7f17481586a0 con 0x7f1748066a30 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f17340033f0 con 0x7f1748066a30 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 710388168 0 0) 0x7f17340033f0 con 0x7f1748066a30 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 msgr2=0x7f1748063810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1748063810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 >> v2:172.21.15.150:3300/0 conn(0x7f1748066e10 msgr2=0x7f174813d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 --2- 172.21.15.73:0/3768779183 >> v2:172.21.15.150:3300/0 conn(0x7f1748066e10 0x7f174813d450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748158050 con 0x7f1748066a30 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f17440017f0 con 0x7f1748066a30 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 >> v1:172.21.15.73:6789/0 conn(0x7f1748066a30 legacy=0x7f17480c6a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f1734003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1734004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1734004b80 0x7f1734005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748157de0 con 0x7f17340038e0 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17480c7140 con 0x7f1734004b80 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17340140f0 con 0x7f174813d940 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f1734003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d1dd700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1734004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174e1df700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1734004b80 0x7f1734005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 -- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 msgr2=0x7f1734004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1734004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 -- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1734004b80 msgr2=0x7f1734005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1734004b80 0x7f1734005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 -- 172.21.15.73:0/3768779183 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17340033f0 con 0x7f17340038e0 2023-11-28T03:07:40.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174e1df700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1734004b80 0x7f1734005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d1dd700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f174813d940 0x7f1734004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.711+0000 7f174d9de700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f1734003d00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f174400c530 tx=0x7f1744002210).ready entity=mon.0 client_cookie=e9034c9b8b2b7899 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744007de0 con 0x7f17340038e0 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f174400ce60 con 0x7f17340038e0 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174c9dc700 1 -- 172.21.15.73:0/3768779183 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744005d50 con 0x7f17340038e0 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 -- 172.21.15.73:0/3768779183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 msgr2=0x7f1734003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3768779183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f1734003d00 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 rx=0x7f174400c530 tx=0x7f1744002210).stop 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e1df700 1 -- 172.21.15.73:0/3768779183 reap_dead start 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 -- 172.21.15.73:0/3768779183 shutdown_connections 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 -- 172.21.15.73:0/3768779183 >> 172.21.15.73:0/3768779183 conn(0x7f17480c8ed0 msgr2=0x7f17480c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 -- 172.21.15.73:0/3768779183 shutdown_connections 2023-11-28T03:07:40.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.712+0000 7f174e9e0700 1 -- 172.21.15.73:0/3768779183 wait complete. 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.713+0000 7f174e9e0700 1 Processor -- start 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.713+0000 7f174e9e0700 1 -- start start 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748138950 0x7f1748138dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f17480ace50 0x7f17480ad2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17481381b0 con 0x7f17340038e0 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1748139fe0 con 0x7f1748138950 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174e9e0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17481393d0 con 0x7f17480ace50 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174d9de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174d9de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54272/0 (socket says 172.21.15.73:54272) 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.714+0000 7f174d9de700 1 -- 172.21.15.73:0/3405550708 learned_addr learned my addr 172.21.15.73:0/3405550708 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 -- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f17480ace50 msgr2=0x7f17480ad2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f17480ace50 0x7f17480ad2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 -- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748138950 msgr2=0x7f1748138dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748138950 0x7f1748138dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 -- 172.21.15.73:0/3405550708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17481396b0 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174d9de700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f1744002250 tx=0x7f1744015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1744007de0 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17440074f0 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17440078b0 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17481381b0 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.715+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1748139c70 con 0x7f17340038e0 2023-11-28T03:07:40.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.716+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f174813a2c0 con 0x7f17340038e0 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.716+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1744021670 con 0x7f17340038e0 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.717+0000 7f173dffb700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 0x7f172c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.717+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f1744019070 con 0x7f17340038e0 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.717+0000 7f174d1dd700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 0x7f172c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.717+0000 7f174d1dd700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 0x7f172c061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f17481386c0 tx=0x7f1738008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:40.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.721+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f174404ca20 con 0x7f17340038e0 2023-11-28T03:07:40.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.885+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f174806c7c0 con 0x7f17340038e0 2023-11-28T03:07:40.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.889+0000 7f173dffb700 1 -- 172.21.15.73:0/3405550708 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1744026070 con 0x7f17340038e0 2023-11-28T03:07:40.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 msgr2=0x7f172c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 0x7f172c061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f17481386c0 tx=0x7f1738008040).stop 2023-11-28T03:07:40.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 msgr2=0x7f17480c1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 rx=0x7f1744002250 tx=0x7f1744015000).stop 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 shutdown_connections 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f172c05f150 0x7f172c061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f17480ace50 0x7f17480ad2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f17340038e0 0x7f17480c1c50 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 --2- 172.21.15.73:0/3405550708 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1748138950 0x7f1748138dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 >> 172.21.15.73:0/3405550708 conn(0x7f17480c8ed0 msgr2=0x7f174813c800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 shutdown_connections 2023-11-28T03:07:40.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:40.891+0000 7f174e9e0700 1 -- 172.21.15.73:0/3405550708 wait complete. 2023-11-28T03:07:40.904 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-11-28T03:07:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.017+0000 7fe3564df700 1 Processor -- start 2023-11-28T03:07:41.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.017+0000 7fe3564df700 1 -- start start 2023-11-28T03:07:41.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe3564df700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3501544e0 0x7fe3501548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe3564df700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 0x7fe3500c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe3564df700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3500c69a0 con 0x7fe3500c6e30 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe3564df700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3500c13a0 con 0x7fe3501544e0 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe3564df700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3500c1680 con 0x7fe35006a610 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3501544e0 0x7fe3501548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe354a7c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe3500c6e30 0x7fe3500a8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45180/0 (socket says 172.21.15.73:45180) 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe354a7c700 1 -- 172.21.15.73:0/3168877994 learned_addr learned my addr 172.21.15.73:0/3168877994 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 0x7fe3500c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 -- 172.21.15.73:0/3168877994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 msgr2=0x7fe3500c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 0x7fe3500c0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 -- 172.21.15.73:0/3168877994 >> v1:172.21.15.73:6789/0 conn(0x7fe3500c6e30 legacy=0x7fe3500a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34effd700 1 -- 172.21.15.73:0/3168877994 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1180153366 0 0) 0x7fe3500c69a0 con 0x7fe3500c6e30 2023-11-28T03:07:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 -- 172.21.15.73:0/3168877994 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3500c6630 con 0x7fe3501544e0 2023-11-28T03:07:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 0x7fe3500c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34ffff700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3501544e0 0x7fe3501548c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fe340006fd0 tx=0x7fe340006f40).ready entity=mon.0 client_cookie=391ce53b4ba87395 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34effd700 1 -- 172.21.15.73:0/3168877994 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340005880 con 0x7fe3501544e0 2023-11-28T03:07:41.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34effd700 1 -- 172.21.15.73:0/3168877994 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3400059e0 con 0x7fe3501544e0 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.018+0000 7fe34effd700 1 -- 172.21.15.73:0/3168877994 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe340005d60 con 0x7fe3501544e0 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 -- 172.21.15.73:0/3168877994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3501544e0 msgr2=0x7fe3501548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3501544e0 0x7fe3501548c0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 rx=0x7fe340006fd0 tx=0x7fe340006f40).stop 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 -- 172.21.15.73:0/3168877994 shutdown_connections 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3501544e0 0x7fe3501548c0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 --2- 172.21.15.73:0/3168877994 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe35006a610 0x7fe3500c0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 -- 172.21.15.73:0/3168877994 >> 172.21.15.73:0/3168877994 conn(0x7fe350085020 msgr2=0x7fe350094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 -- 172.21.15.73:0/3168877994 shutdown_connections 2023-11-28T03:07:41.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.019+0000 7fe3564df700 1 -- 172.21.15.73:0/3168877994 wait complete. 2023-11-28T03:07:41.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.020+0000 7fe3564df700 1 Processor -- start 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.020+0000 7fe3564df700 1 -- start start 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 0x7fe35014ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3501544e0 0x7fe3500950a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe350096730 con 0x7fe35006a610 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3501504c0 con 0x7fe3500c6e30 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe3564df700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe350095b20 con 0x7fe3501544e0 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54298/0 (socket says 172.21.15.73:54298) 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 -- 172.21.15.73:0/2977066469 learned_addr learned my addr 172.21.15.73:0/2977066469 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:41.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 0x7fe35014ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe354a7c700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3501544e0 0x7fe3500950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 -- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3501544e0 msgr2=0x7fe3500950a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3501544e0 0x7fe3500950a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 -- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 msgr2=0x7fe35014ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 0x7fe35014ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 -- 172.21.15.73:0/2977066469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe340009250 con 0x7fe35006a610 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 0x7fe35014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.021+0000 7fe34ffff700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fe340006fa0 tx=0x7fe340016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.022+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34000b860 con 0x7fe35006a610 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.022+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34000b9c0 con 0x7fe35006a610 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.022+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34000bd80 con 0x7fe35006a610 2023-11-28T03:07:41.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.022+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe350095e00 con 0x7fe35006a610 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.022+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3500963c0 con 0x7fe35006a610 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.023+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe350096a10 con 0x7fe35006a610 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.023+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe340011650 con 0x7fe35006a610 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.023+0000 7fe337fff700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 0x7fe33c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.023+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fe34007ada0 con 0x7fe35006a610 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.023+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 0x7fe33c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.024+0000 7fe34f7fe700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 0x7fe33c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe344006fd0 tx=0x7fe344008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.027+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe34004c1f0 con 0x7fe35006a610 2023-11-28T03:07:41.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.253+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fe35006c860 con 0x7fe35006a610 2023-11-28T03:07:41.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.272+0000 7fe337fff700 1 -- 172.21.15.73:0/2977066469 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v540) v1 ==== 156+0+0 (secure 0 0 0) 0x7fe340021070 con 0x7fe35006a610 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.273+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 msgr2=0x7fe33c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.273+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 0x7fe33c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fe344006fd0 tx=0x7fe344008040).stop 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 msgr2=0x7fe350096bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 rx=0x7fe340006fa0 tx=0x7fe340016000).stop 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 shutdown_connections 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe33c05f150 0x7fe33c061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3501544e0 0x7fe3500950a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe35006a610 0x7fe350096bc0 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 --2- 172.21.15.73:0/2977066469 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3500c6e30 0x7fe35014ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 >> 172.21.15.73:0/2977066469 conn(0x7fe350085020 msgr2=0x7fe3500caac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:41.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 shutdown_connections 2023-11-28T03:07:41.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.274+0000 7fe3564df700 1 -- 172.21.15.73:0/2977066469 wait complete. 2023-11-28T03:07:41.287 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-11-28T03:07:41.287 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-11-28T03:07:41.287 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:07:41.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-11-28T03:07:41.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.400+0000 7f528eae2700 1 Processor -- start 2023-11-28T03:07:41.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.400+0000 7f528eae2700 1 -- start start 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.400+0000 7f528eae2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52881548c0 0x7f528806a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528eae2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52880c69a0 con 0x7f52881544e0 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528eae2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52880c13a0 con 0x7f52880c6e30 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528eae2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52880c1680 con 0x7f52881548c0 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528c87e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f52881544e0 0x7f52880c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45186/0 (socket says 172.21.15.73:45186) 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528c87e700 1 -- 172.21.15.73:0/2562044831 learned_addr learned my addr 172.21.15.73:0/2562044831 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527ffff700 1 --2- 172.21.15.73:0/2562044831 >> v2:172.21.15.150:3300/0 conn(0x7f52881548c0 0x7f528806a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 431811117 0 0) 0x7f52880c69a0 con 0x7f52881544e0 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52700033f0 con 0x7f52881544e0 2023-11-28T03:07:41.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192197926 0 0) 0x7f52700033f0 con 0x7f52881544e0 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 >> v2:172.21.15.150:3300/0 conn(0x7f52881548c0 msgr2=0x7f528806a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 --2- 172.21.15.73:0/2562044831 >> v2:172.21.15.150:3300/0 conn(0x7f52881548c0 0x7f528806a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 >> v1:172.21.15.150:6789/0 conn(0x7f52880c6e30 legacy=0x7f52880a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52880c6630 con 0x7f52881544e0 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f5284001970 con 0x7f52881544e0 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 >> v1:172.21.15.73:6789/0 conn(0x7f52881544e0 legacy=0x7f52880c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 0x7f5270003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5270004690 0x7f5270004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5270005000 0x7f527000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52880c1680 con 0x7f52700038e0 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f527000beb0 con 0x7f5270004690 2023-11-28T03:07:41.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f527000c070 con 0x7f5270005000 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.401+0000 7f528c87e700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 0x7f5270003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f527ffff700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5270004690 0x7f5270004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528d07f700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5270005000 0x7f527000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 -- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5270005000 msgr2=0x7f527000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5270005000 0x7f527000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 -- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5270004690 msgr2=0x7f5270004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5270004690 0x7f5270004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 -- 172.21.15.73:0/2562044831 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52700033f0 con 0x7f52700038e0 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528d07f700 1 -- 172.21.15.73:0/2562044831 reap_dead start 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f527ffff700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5270004690 0x7f5270004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:41.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528c87e700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 0x7f5270003d00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f528400c5f0 tx=0x7f5284007b20).ready entity=mon.0 client_cookie=bdc0ea4a11aaa8de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284007d50 con 0x7f52700038e0 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528400cdf0 con 0x7f52700038e0 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f527f7fe700 1 -- 172.21.15.73:0/2562044831 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284005c00 con 0x7f52700038e0 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 -- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 msgr2=0x7f5270003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 0x7f5270003d00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 rx=0x7f528400c5f0 tx=0x7f5284007b20).stop 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 -- 172.21.15.73:0/2562044831 shutdown_connections 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 --2- 172.21.15.73:0/2562044831 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52700038e0 0x7f5270003d00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 -- 172.21.15.73:0/2562044831 >> 172.21.15.73:0/2562044831 conn(0x7f5288085020 msgr2=0x7f5288094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.402+0000 7f528eae2700 1 -- 172.21.15.73:0/2562044831 shutdown_connections 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.403+0000 7f528eae2700 1 -- 172.21.15.73:0/2562044831 wait complete. 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 Processor -- start 2023-11-28T03:07:41.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 -- start start 2023-11-28T03:07:41.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 0x7f5288096fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 0x7f5288153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288097550 con 0x7f52880985f0 2023-11-28T03:07:41.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5288097830 con 0x7f52700038e0 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.404+0000 7f528eae2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5288097b10 con 0x7f5288168c70 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54316/0 (socket says 172.21.15.73:54316) 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 -- 172.21.15.73:0/1587116021 learned_addr learned my addr 172.21.15.73:0/1587116021 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528c87e700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 0x7f5288096fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f527ffff700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 0x7f5288153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 -- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 msgr2=0x7f5288153370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 0x7f5288153370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 -- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 msgr2=0x7f5288096fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 0x7f5288096fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:41.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 -- 172.21.15.73:0/1587116021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5288097df0 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f527ffff700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 0x7f5288153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528c87e700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 0x7f5288096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528d07f700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f5288062ec0 tx=0x7f5280009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280014430 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5280014590 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52880980d0 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5280014910 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.405+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5288153860 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.406+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f528806c860 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.406+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5280014d40 con 0x7f52880985f0 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.407+0000 7f527cff9700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 0x7f5274061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:41.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.407+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f5280079840 con 0x7f52880985f0 2023-11-28T03:07:41.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.407+0000 7f528c87e700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 0x7f5274061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:41.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.407+0000 7f528c87e700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 0x7f5274061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f5284001270 tx=0x7f5284015000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:41.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.411+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5280049af0 con 0x7f52880985f0 2023-11-28T03:07:41.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:41.638+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f52880c6250 con 0x7f52880985f0 2023-11-28T03:07:42.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.274+0000 7f527cff9700 1 -- 172.21.15.73:0/1587116021 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v541) v1 ==== 169+0+0 (secure 0 0 0) 0x7f528004ac90 con 0x7f52880985f0 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 msgr2=0x7f5274061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 0x7f5274061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f5284001270 tx=0x7f5284015000).stop 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 msgr2=0x7f5288098a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 rx=0x7f5288062ec0 tx=0x7f5280009040).stop 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 shutdown_connections 2023-11-28T03:07:42.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f527405f150 0x7f5274061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52880985f0 0x7f5288098a70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52700038e0 0x7f5288096fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 --2- 172.21.15.73:0/1587116021 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5288168c70 0x7f5288153370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 >> 172.21.15.73:0/1587116021 conn(0x7f5288085020 msgr2=0x7f52880ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 shutdown_connections 2023-11-28T03:07:42.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.276+0000 7f528eae2700 1 -- 172.21.15.73:0/1587116021 wait complete. 2023-11-28T03:07:42.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:07:42.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.404+0000 7f74d092b700 1 Processor -- start 2023-11-28T03:07:42.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.404+0000 7f74d092b700 1 -- start start 2023-11-28T03:07:42.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74d092b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74cc158a30 0x7f74cc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74d092b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc0c72d0 con 0x7f74cc0c6190 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74d092b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74cc064570 con 0x7f74cc098d10 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74d092b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74cc154bc0 con 0x7f74cc158a30 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74cad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74cc158a30 0x7f74cc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74c1d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f74cc0c6190 0x7f74cc158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45198/0 (socket says 172.21.15.73:45198) 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74c1d9b700 1 -- 172.21.15.73:0/3334631234 learned_addr learned my addr 172.21.15.73:0/3334631234 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74cad9d700 1 -- 172.21.15.73:0/3334631234 >> v1:172.21.15.150:6789/0 conn(0x7f74cc098d10 legacy=0x7f74cc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74cad9d700 1 -- 172.21.15.73:0/3334631234 >> v1:172.21.15.73:6789/0 conn(0x7f74cc0c6190 legacy=0x7f74cc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.405+0000 7f74cad9d700 1 -- 172.21.15.73:0/3334631234 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc0c66c0 con 0x7f74cc158a30 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74cad9d700 1 --2- 172.21.15.73:0/3334631234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc158a30 0x7f74cc0c7760 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f74c4009e60 tx=0x7f74c4009780).ready entity=mon.0 client_cookie=6aa967b20509c51b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74c9d9b700 1 -- 172.21.15.73:0/3334631234 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c400e070 con 0x7f74cc158a30 2023-11-28T03:07:42.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74c9d9b700 1 -- 172.21.15.73:0/3334631234 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c400bc30 con 0x7f74cc158a30 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74c9d9b700 1 -- 172.21.15.73:0/3334631234 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c4012410 con 0x7f74cc158a30 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 -- 172.21.15.73:0/3334631234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc158a30 msgr2=0x7f74cc0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 --2- 172.21.15.73:0/3334631234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc158a30 0x7f74cc0c7760 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 rx=0x7f74c4009e60 tx=0x7f74c4009780).stop 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 -- 172.21.15.73:0/3334631234 shutdown_connections 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 --2- 172.21.15.73:0/3334631234 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc158a30 0x7f74cc0c7760 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 -- 172.21.15.73:0/3334631234 >> 172.21.15.73:0/3334631234 conn(0x7f74cc0cb310 msgr2=0x7f74cc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 -- 172.21.15.73:0/3334631234 shutdown_connections 2023-11-28T03:07:42.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.406+0000 7f74d092b700 1 -- 172.21.15.73:0/3334631234 wait complete. 2023-11-28T03:07:42.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.407+0000 7f74d092b700 1 Processor -- start 2023-11-28T03:07:42.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.407+0000 7f74d092b700 1 -- start start 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74cc0c6190 0x7f74cc0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74cc158a30 0x7f74cc0b06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc156f10 con 0x7f74cc098d10 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc156300 con 0x7f74cc0c6190 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74d092b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74cc1565e0 con 0x7f74cc158a30 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74ca59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54336/0 (socket says 172.21.15.73:54336) 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.408+0000 7f74ca59c700 1 -- 172.21.15.73:0/3148368240 learned_addr learned my addr 172.21.15.73:0/3148368240 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:42.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 -- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74cc158a30 msgr2=0x7f74cc0b06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74cc158a30 0x7f74cc0b06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 -- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74cc0c6190 msgr2=0x7f74cc0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74c1d9b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74cc0c6190 0x7f74cc0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74cc0c6190 0x7f74cc0acdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 -- 172.21.15.73:0/3148368240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c4009a20 con 0x7f74cc098d10 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74ca59c700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f74bc00c4c0 tx=0x7f74bc007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc007020 con 0x7f74cc098d10 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74bc001e90 con 0x7f74cc098d10 2023-11-28T03:07:42.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74cc1568c0 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74bc0073d0 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.409+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74cc0ad2e0 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.410+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74cc1571f0 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.410+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f74bc0068b0 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.411+0000 7f74c2ffd700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 0x7f74ac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.411+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f74bc079970 con 0x7f74cc098d10 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.411+0000 7f74c1d9b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 0x7f74ac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.411+0000 7f74c1d9b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 0x7f74ac061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f74cc0c5080 tx=0x7f74b4009040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.415+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f74bc04adc0 con 0x7f74cc098d10 2023-11-28T03:07:42.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.580+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f74cc06c860 con 0x7f74cc098d10 2023-11-28T03:07:42.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.586+0000 7f74c2ffd700 1 -- 172.21.15.73:0/3148368240 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v14)=0 v14) v1 ==== 122+0+0 (secure 0 0 0) 0x7f74bc04e3e0 con 0x7f74cc098d10 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.587+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 msgr2=0x7f74ac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.587+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 0x7f74ac061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f74cc0c5080 tx=0x7f74b4009040).stop 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.587+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 msgr2=0x7f74cc1573a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.587+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 rx=0x7f74bc00c4c0 tx=0x7f74bc007e00).stop 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 shutdown_connections 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f74ac05f150 0x7f74ac061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74cc158a30 0x7f74cc0b06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74cc098d10 0x7f74cc1573a0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 --2- 172.21.15.73:0/3148368240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74cc0c6190 0x7f74cc0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 >> 172.21.15.73:0/3148368240 conn(0x7f74cc0cb310 msgr2=0x7f74cc0ca020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 shutdown_connections 2023-11-28T03:07:42.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.588+0000 7f74d092b700 1 -- 172.21.15.73:0/3148368240 wait complete. 2023-11-28T03:07:42.600 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:07:42.601 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:42.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:07:42.683 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:07:42.684 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:07:42.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:42.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.796+0000 7f4b6a0a4700 1 Processor -- start 2023-11-28T03:07:42.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.796+0000 7f4b6a0a4700 1 -- start start 2023-11-28T03:07:42.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b6a0a4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4b640c0180 0x7f4b64157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b6a0a4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b640c1750 0x7f4b640c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b6a0a4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b640c12c0 con 0x7f4b64157660 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b6a0a4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b640c06b0 con 0x7f4b640c1750 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b6a0a4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b640c0990 con 0x7f4b640c0180 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b698a3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b640c1750 0x7f4b640c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b690a2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4b64157660 0x7f4b641589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45202/0 (socket says 172.21.15.73:45202) 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b690a2700 1 -- 172.21.15.73:0/2474628120 learned_addr learned my addr 172.21.15.73:0/2474628120 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b688a1700 1 --2- 172.21.15.73:0/2474628120 >> v2:172.21.15.150:3300/0 conn(0x7f4b640c0180 0x7f4b64157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b698a3700 1 -- 172.21.15.73:0/2474628120 >> v2:172.21.15.150:3300/0 conn(0x7f4b640c0180 msgr2=0x7f4b64157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b698a3700 1 --2- 172.21.15.73:0/2474628120 >> v2:172.21.15.150:3300/0 conn(0x7f4b640c0180 0x7f4b64157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b698a3700 1 -- 172.21.15.73:0/2474628120 >> v1:172.21.15.73:6789/0 conn(0x7f4b64157660 legacy=0x7f4b641589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b698a3700 1 -- 172.21.15.73:0/2474628120 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b640c0c70 con 0x7f4b640c1750 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.797+0000 7f4b5bfff700 1 -- 172.21.15.73:0/2474628120 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1638621880 0 0) 0x7f4b640c12c0 con 0x7f4b64157660 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b698a3700 1 --2- 172.21.15.73:0/2474628120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c1750 0x7f4b640c1b70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f4b640c5820 tx=0x7f4b60009780).ready entity=mon.0 client_cookie=d23d1cfe8b840535 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b5bfff700 1 -- 172.21.15.73:0/2474628120 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b6000e070 con 0x7f4b640c1750 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b5bfff700 1 -- 172.21.15.73:0/2474628120 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b6000bc30 con 0x7f4b640c1750 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b5bfff700 1 -- 172.21.15.73:0/2474628120 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b60012410 con 0x7f4b640c1750 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/2474628120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c1750 msgr2=0x7f4b640c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/2474628120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c1750 0x7f4b640c1b70 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 rx=0x7f4b640c5820 tx=0x7f4b60009780).stop 2023-11-28T03:07:42.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/2474628120 shutdown_connections 2023-11-28T03:07:42.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/2474628120 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c1750 0x7f4b640c1b70 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/2474628120 >> v2:172.21.15.150:3300/0 conn(0x7f4b640c0180 0x7f4b64157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/2474628120 >> 172.21.15.73:0/2474628120 conn(0x7f4b640cb000 msgr2=0x7f4b640cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:42.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/2474628120 shutdown_connections 2023-11-28T03:07:42.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.798+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/2474628120 wait complete. 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.799+0000 7f4b6a0a4700 1 Processor -- start 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.799+0000 7f4b6a0a4700 1 -- start start 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b640c1750 0x7f4b640c3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b64157660 0x7f4b640ae380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640c37f0 con 0x7f4b640c0180 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640c3ad0 con 0x7f4b640c1750 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b6a0a4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b640c3db0 con 0x7f4b64157660 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b690a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b690a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54350/0 (socket says 172.21.15.73:54350) 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b690a2700 1 -- 172.21.15.73:0/793358046 learned_addr learned my addr 172.21.15.73:0/793358046 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.800+0000 7f4b688a1700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b640c1750 0x7f4b640c3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b698a3700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b64157660 0x7f4b640ae380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 -- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b64157660 msgr2=0x7f4b640ae380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b64157660 0x7f4b640ae380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 -- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b640c1750 msgr2=0x7f4b640c3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b640c1750 0x7f4b640c3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 -- 172.21.15.73:0/793358046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b60009a20 con 0x7f4b640c0180 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b690a2700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f4b5c001eb0 tx=0x7f4b5c007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c00ce90 con 0x7f4b640c0180 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5c0072e0 con 0x7f4b640c0180 2023-11-28T03:07:42.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5c007680 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b640c4090 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.801+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b641480f0 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.802+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b640ab7d0 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.802+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4b5c0059b0 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.803+0000 7f4b597fa700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 0x7f4b480615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.803+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f4b5c079730 con 0x7f4b640c0180 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.803+0000 7f4b688a1700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 0x7f4b480615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.803+0000 7f4b688a1700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 0x7f4b480615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4b640cbf00 tx=0x7f4b54008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:42.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:42.807+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b5c04ab80 con 0x7f4b640c0180 2023-11-28T03:07:43.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.011+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b6406c7c0 con 0x7f4b640c0180 2023-11-28T03:07:43.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.012+0000 7f4b597fa700 1 -- 172.21.15.73:0/793358046 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+3535 (secure 0 0 0) 0x7f4b5c04e1a0 con 0x7f4b640c0180 2023-11-28T03:07:43.013 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:43.013 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5905,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2650321712","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2650321712},{"type":"v1","addr":"172.21.15.150:6834","nonce":2650321712}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":5908,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3855248005","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3855248005},{"type":"v1","addr":"172.21.15.150:6835","nonce":3855248005}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6243,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/3707616705","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3707616705},{"type":"v1","addr":"172.21.15.73:6836","nonce":3707616705}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":23},{"gid":6597,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3758518996","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":3758518996},{"type":"v1","addr":"172.21.15.73:6837","nonce":3758518996}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[]} 2023-11-28T03:07:43.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 msgr2=0x7f4b480615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 0x7f4b480615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f4b640cbf00 tx=0x7f4b54008040).stop 2023-11-28T03:07:43.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 msgr2=0x7f4b640c2d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 rx=0x7f4b5c001eb0 tx=0x7f4b5c007e90).stop 2023-11-28T03:07:43.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 shutdown_connections 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4b4805f120 0x7f4b480615e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b64157660 0x7f4b640ae380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b640c0180 0x7f4b640c2d50 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 --2- 172.21.15.73:0/793358046 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b640c1750 0x7f4b640c3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 >> 172.21.15.73:0/793358046 conn(0x7f4b640cb000 msgr2=0x7f4b640ca6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 shutdown_connections 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.014+0000 7f4b6a0a4700 1 -- 172.21.15.73:0/793358046 wait complete. 2023-11-28T03:07:43.016 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 28 2023-11-28T03:07:43.027 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:07:43.027 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:07:43.027 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:07:43.028 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:43.028 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:07:43.028 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:07:43.028 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:07:43.028 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:43.028 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:07:43.028 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:07:43.029 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:07:43.029 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:43.029 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:07:43.029 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:07:43.029 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:07:43.029 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:07:43.036 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:07:43.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:07:43.037 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:07:43.037 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:07:43.040 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:07:43.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:07:43.041 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:07:43.041 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:07:43.078 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:07:43.078 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:43.079 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:07:43.079 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:43.083 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:07:43.083 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:43.084 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:07:43.084 DEBUG:teuthology.parallel:result is None 2023-11-28T03:07:43.084 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:07:43.150 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:07:43.150 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:07:43.167 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:07:43.168 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:07:43.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.206+0000 7fd786335700 1 Processor -- start 2023-11-28T03:07:43.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.206+0000 7fd786335700 1 -- start start 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd786335700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd780069d10 0x7fd78006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd786335700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780151a20 0x7fd7800be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd786335700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7800c3470 con 0x7fd7800be6a0 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd786335700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7800c48f0 con 0x7fd780069d10 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd786335700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd780151ea0 con 0x7fd780151a20 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd77ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd780069d10 0x7fd78006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd7848d2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd7800be6a0 0x7fd7800bd1f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45230/0 (socket says 172.21.15.73:45230) 2023-11-28T03:07:43.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.207+0000 7fd7848d2700 1 -- 172.21.15.73:0/4220756970 learned_addr learned my addr 172.21.15.73:0/4220756970 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77f7fe700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780151a20 0x7fd7800be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77ffff700 1 -- 172.21.15.73:0/4220756970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780151a20 msgr2=0x7fd7800be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77ffff700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780151a20 0x7fd7800be1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77ffff700 1 -- 172.21.15.73:0/4220756970 >> v1:172.21.15.73:6789/0 conn(0x7fd7800be6a0 legacy=0x7fd7800bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77ffff700 1 -- 172.21.15.73:0/4220756970 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780152180 con 0x7fd780069d10 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77effd700 1 -- 172.21.15.73:0/4220756970 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540417043 0 0) 0x7fd7800c3470 con 0x7fd7800be6a0 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77ffff700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd780069d10 0x7fd78006a0f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fd770009760 tx=0x7fd770009060).ready entity=mon.0 client_cookie=76979186d749e7ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77effd700 1 -- 172.21.15.73:0/4220756970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000b580 con 0x7fd780069d10 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77effd700 1 -- 172.21.15.73:0/4220756970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd77000b6e0 con 0x7fd780069d10 2023-11-28T03:07:43.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd77effd700 1 -- 172.21.15.73:0/4220756970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd77000ba60 con 0x7fd780069d10 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 -- 172.21.15.73:0/4220756970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd780069d10 msgr2=0x7fd78006a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd780069d10 0x7fd78006a0f0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 rx=0x7fd770009760 tx=0x7fd770009060).stop 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 -- 172.21.15.73:0/4220756970 shutdown_connections 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd780069d10 0x7fd78006a0f0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 --2- 172.21.15.73:0/4220756970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780151a20 0x7fd7800be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.208+0000 7fd786335700 1 -- 172.21.15.73:0/4220756970 >> 172.21.15.73:0/4220756970 conn(0x7fd7800642d0 msgr2=0x7fd7800c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.209+0000 7fd786335700 1 -- 172.21.15.73:0/4220756970 shutdown_connections 2023-11-28T03:07:43.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.209+0000 7fd786335700 1 -- 172.21.15.73:0/4220756970 wait complete. 2023-11-28T03:07:43.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.210+0000 7fd786335700 1 Processor -- start 2023-11-28T03:07:43.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.210+0000 7fd786335700 1 -- start start 2023-11-28T03:07:43.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.210+0000 7fd786335700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780069d10 0x7fd78014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.210+0000 7fd786335700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd780151a20 0x7fd78014e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd780162dc0 con 0x7fd7800be6a0 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78014e3c0 con 0x7fd780151a20 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd780162490 con 0x7fd780069d10 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54392/0 (socket says 172.21.15.73:54392) 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 -- 172.21.15.73:0/4058104919 learned_addr learned my addr 172.21.15.73:0/4058104919 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77ffff700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780069d10 0x7fd78014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd7848d2700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd780151a20 0x7fd78014e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 -- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780069d10 msgr2=0x7fd78014ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780069d10 0x7fd78014ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 -- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd780151a20 msgr2=0x7fd78014e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd780151a20 0x7fd78014e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 -- 172.21.15.73:0/4058104919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7700092a0 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd77f7fe700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fd77400d890 tx=0x7fd774008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7740125f0 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd774012750 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd780162770 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd774012ad0 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.211+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd780162140 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.212+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7801630a0 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.212+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd774013480 con 0x7fd7800be6a0 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.213+0000 7fd767fff700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 0x7fd76c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:43.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.213+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fd774079dd0 con 0x7fd7800be6a0 2023-11-28T03:07:43.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.213+0000 7fd77ffff700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 0x7fd76c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:43.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.213+0000 7fd77ffff700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 0x7fd76c061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd77000b340 tx=0x7fd770017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:43.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.217+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd77404b1c0 con 0x7fd7800be6a0 2023-11-28T03:07:43.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:43.396+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd78006c860 con 0x7fd7800be6a0 2023-11-28T03:07:44.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.277+0000 7fd767fff700 1 -- 172.21.15.73:0/4058104919 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v92) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd77407b080 con 0x7fd7800be6a0 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 msgr2=0x7fd76c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 0x7fd76c061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd77000b340 tx=0x7fd770017000).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 msgr2=0x7fd7800a6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 rx=0x7fd77400d890 tx=0x7fd774008040).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 shutdown_connections 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd76c05f150 0x7fd76c061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd780151a20 0x7fd78014e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd780069d10 0x7fd78014ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 --2- 172.21.15.73:0/4058104919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7800be6a0 0x7fd7800a6440 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 >> 172.21.15.73:0/4058104919 conn(0x7fd7800642d0 msgr2=0x7fd7800bd690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:44.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 shutdown_connections 2023-11-28T03:07:44.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.279+0000 7fd786335700 1 -- 172.21.15.73:0/4058104919 wait complete. 2023-11-28T03:07:44.282 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:07:44.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:07:44.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.405+0000 7fc581546700 1 Processor -- start 2023-11-28T03:07:44.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.405+0000 7fc581546700 1 -- start start 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc581546700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 0x7fc57c0697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc581546700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc57c069ce0 0x7fc57c062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc581546700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc57c0630a0 con 0x7fc57c062bf0 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc581546700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc57c06a160 con 0x7fc57c069ce0 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc581546700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c065760 con 0x7fc57c1542d0 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.406+0000 7fc5727fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc57c069ce0 0x7fc57c062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc57c069ce0 0x7fc57c062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49758/0 (socket says 172.21.15.73:49758) 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 -- 172.21.15.73:0/4286377422 learned_addr learned my addr 172.21.15.73:0/4286377422 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 -- 172.21.15.73:0/4286377422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 msgr2=0x7fc57c0697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc57affd700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 0x7fc57c0697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 0x7fc57c0697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 -- 172.21.15.73:0/4286377422 >> v1:172.21.15.150:6789/0 conn(0x7fc57c062bf0 legacy=0x7fc57c157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 -- 172.21.15.73:0/4286377422 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc57c0c3250 con 0x7fc57c069ce0 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc57affd700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 0x7fc57c0697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc5727fc700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c069ce0 0x7fc57c062700 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fc564006fd0 tx=0x7fc5640090f0).ready entity=mon.0 client_cookie=620b0a5b7d7f7dc3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc57a7fc700 1 -- 172.21.15.73:0/4286377422 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400b600 con 0x7fc57c069ce0 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc57a7fc700 1 -- 172.21.15.73:0/4286377422 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc56400b760 con 0x7fc57c069ce0 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.407+0000 7fc57a7fc700 1 -- 172.21.15.73:0/4286377422 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc56400bae0 con 0x7fc57c069ce0 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 -- 172.21.15.73:0/4286377422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c069ce0 msgr2=0x7fc57c062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c069ce0 0x7fc57c062700 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 rx=0x7fc564006fd0 tx=0x7fc5640090f0).stop 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 -- 172.21.15.73:0/4286377422 shutdown_connections 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c1542d0 0x7fc57c0697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 --2- 172.21.15.73:0/4286377422 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c069ce0 0x7fc57c062700 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 -- 172.21.15.73:0/4286377422 >> 172.21.15.73:0/4286377422 conn(0x7fc57c0c7fd0 msgr2=0x7fc57c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:44.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 -- 172.21.15.73:0/4286377422 shutdown_connections 2023-11-28T03:07:44.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.408+0000 7fc581546700 1 -- 172.21.15.73:0/4286377422 wait complete. 2023-11-28T03:07:44.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.409+0000 7fc581546700 1 Processor -- start 2023-11-28T03:07:44.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.409+0000 7fc581546700 1 -- start start 2023-11-28T03:07:44.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc57c062bf0 0x7fc57c0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c069ce0 0x7fc57c0a66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0a9050 con 0x7fc57c1542d0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0a7fb0 con 0x7fc57c062bf0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc57c0a8290 con 0x7fc57c069ce0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49774/0 (socket says 172.21.15.73:49774) 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 -- 172.21.15.73:0/1525977513 learned_addr learned my addr 172.21.15.73:0/1525977513 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57affd700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc57c062bf0 0x7fc57c0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc5727fc700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c069ce0 0x7fc57c0a66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 -- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c069ce0 msgr2=0x7fc57c0a66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c069ce0 0x7fc57c0a66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 -- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc57c062bf0 msgr2=0x7fc57c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc57c062bf0 0x7fc57c0aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 -- 172.21.15.73:0/1525977513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc564009370 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc57b7fe700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fc57400c810 tx=0x7fc574002560).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574010510 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5740106b0 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc574010a70 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc57c0a8570 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.410+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc57c0a8b30 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.411+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc57c0a8d20 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.412+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc574012440 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.412+0000 7fc5737fe700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 0x7fc55c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.412+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc5740797a0 con 0x7fc57c1542d0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.412+0000 7fc57affd700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 0x7fc55c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.412+0000 7fc57affd700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 0x7fc55c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc57c0ae070 tx=0x7fc56c008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.416+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc574048af0 con 0x7fc57c1542d0 2023-11-28T03:07:44.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.644+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fc57c06c860 con 0x7fc57c1542d0 2023-11-28T03:07:44.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.645+0000 7fc5737fe700 1 -- 172.21.15.73:0/1525977513 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v13) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fc57404ae10 con 0x7fc57c1542d0 2023-11-28T03:07:44.646 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:44.646 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:07:44.646 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:07:44.646 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:07:44.647 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:07:44.648 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:44.649 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.650 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:07:44.651 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:07:44.652 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.653 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.654 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.655 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:07:44.656 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:44.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:07:44.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:07:44.661 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:07:44.662 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 msgr2=0x7fc55c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 0x7fc55c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7fc57c0ae070 tx=0x7fc56c008040).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 msgr2=0x7fc57c1677b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 rx=0x7fc57400c810 tx=0x7fc574002560).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 shutdown_connections 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc55c05f150 0x7fc55c061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc57c1542d0 0x7fc57c1677b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc57c062bf0 0x7fc57c0aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 --2- 172.21.15.73:0/1525977513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc57c069ce0 0x7fc57c0a66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 >> 172.21.15.73:0/1525977513 conn(0x7fc57c0c7fd0 msgr2=0x7fc57c066fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:44.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 shutdown_connections 2023-11-28T03:07:44.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.647+0000 7fc581546700 1 -- 172.21.15.73:0/1525977513 wait complete. 2023-11-28T03:07:44.665 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:44.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.777+0000 7f7e37b16700 1 Processor -- start 2023-11-28T03:07:44.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.777+0000 7f7e37b16700 1 -- start start 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301520c0 0x7f7e30066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 0x7f7e30066a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e30067a40 0x7f7e30067ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300683e0 con 0x7f7e30067a40 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e300be110 con 0x7f7e301524a0 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e37b16700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e30066fc0 con 0x7f7e301520c0 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e30067a40 0x7f7e30067ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7e30067a40 0x7f7e30067ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49782/0 (socket says 172.21.15.73:49782) 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 -- 172.21.15.73:0/1544914518 learned_addr learned my addr 172.21.15.73:0/1544914518 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e358b2700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301520c0 0x7f7e30066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e350b1700 1 --2- 172.21.15.73:0/1544914518 >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 0x7f7e30066a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 -- 172.21.15.73:0/1544914518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301520c0 msgr2=0x7f7e30066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301520c0 0x7f7e30066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 -- 172.21.15.73:0/1544914518 >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 msgr2=0x7f7e30066a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 --2- 172.21.15.73:0/1544914518 >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 0x7f7e30066a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e360b3700 1 -- 172.21.15.73:0/1544914518 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e300672a0 con 0x7f7e30067a40 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.778+0000 7f7e350b1700 1 --2- 172.21.15.73:0/1544914518 >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 0x7f7e30066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e360b3700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e30067a40 0x7f7e30067ef0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f7e300af3a0 tx=0x7f7e28009840).ready entity=mon.0 client_cookie=88c76b359583b5cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e348b0700 1 -- 172.21.15.73:0/1544914518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800e070 con 0x7f7e30067a40 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e348b0700 1 -- 172.21.15.73:0/1544914518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800ba50 con 0x7f7e30067a40 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e348b0700 1 -- 172.21.15.73:0/1544914518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800bdd0 con 0x7f7e30067a40 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 -- 172.21.15.73:0/1544914518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e30067a40 msgr2=0x7f7e30067ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e30067a40 0x7f7e30067ef0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 rx=0x7f7e300af3a0 tx=0x7f7e28009840).stop 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 -- 172.21.15.73:0/1544914518 shutdown_connections 2023-11-28T03:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e30067a40 0x7f7e30067ef0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1544914518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301520c0 0x7f7e30066520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1544914518 >> v2:172.21.15.150:3300/0 conn(0x7f7e301524a0 0x7f7e30066a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 -- 172.21.15.73:0/1544914518 >> 172.21.15.73:0/1544914518 conn(0x7f7e300631d0 msgr2=0x7f7e300c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:44.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 -- 172.21.15.73:0/1544914518 shutdown_connections 2023-11-28T03:07:44.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.779+0000 7f7e37b16700 1 -- 172.21.15.73:0/1544914518 wait complete. 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.780+0000 7f7e37b16700 1 Processor -- start 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.780+0000 7f7e37b16700 1 -- start start 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e30067a40 0x7f7e3014edd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301524a0 0x7f7e300c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300c8a70 con 0x7f7e301520c0 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300a9460 con 0x7f7e30067a40 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e37b16700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e300a8850 con 0x7f7e301524a0 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e350b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e350b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49790/0 (socket says 172.21.15.73:49790) 2023-11-28T03:07:44.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e350b1700 1 -- 172.21.15.73:0/1296097713 learned_addr learned my addr 172.21.15.73:0/1296097713 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e358b2700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e30067a40 0x7f7e3014edd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.781+0000 7f7e360b3700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301524a0 0x7f7e300c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 -- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301524a0 msgr2=0x7f7e300c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301524a0 0x7f7e300c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 -- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e30067a40 msgr2=0x7f7e3014edd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e30067a40 0x7f7e3014edd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 -- 172.21.15.73:0/1296097713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e28009ae0 con 0x7f7e301520c0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e350b1700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f7e2000c880 tx=0x7f7e20008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20010580 con 0x7f7e301520c0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e200106e0 con 0x7f7e301520c0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e300a8b30 con 0x7f7e301520c0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e20010a60 con 0x7f7e301520c0 2023-11-28T03:07:44.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.782+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e300a90f0 con 0x7f7e301520c0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.783+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e30066d40 con 0x7f7e301520c0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.783+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7e2001b460 con 0x7f7e301520c0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.783+0000 7f7e25ffb700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 0x7f7e14061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.784+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7e20079cb0 con 0x7f7e301520c0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.784+0000 7f7e358b2700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 0x7f7e14061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.784+0000 7f7e358b2700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 0x7f7e14061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f7e2c006fd0 tx=0x7f7e2c008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:44.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.788+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7e2004b270 con 0x7f7e301520c0 2023-11-28T03:07:44.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.950+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7e300a95d0 con 0x7f7e301520c0 2023-11-28T03:07:44.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.951+0000 7f7e25ffb700 1 -- 172.21.15.73:0/1296097713 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+13713 (secure 0 0 0) 0x7f7e2004e890 con 0x7f7e301520c0 2023-11-28T03:07:44.952 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:44.952 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":92,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:44.274512+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":10,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":80,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 msgr2=0x7f7e14061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 0x7f7e14061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f7e2c006fd0 tx=0x7f7e2c008040).stop 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 msgr2=0x7f7e300c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 rx=0x7f7e2000c880 tx=0x7f7e20008040).stop 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 shutdown_connections 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e1405f150 0x7f7e14061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e301524a0 0x7f7e300c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e30067a40 0x7f7e3014edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 --2- 172.21.15.73:0/1296097713 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e301520c0 0x7f7e300c8090 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:44.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 >> 172.21.15.73:0/1296097713 conn(0x7f7e300631d0 msgr2=0x7f7e300c54b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:44.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 shutdown_connections 2023-11-28T03:07:44.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:44.953+0000 7f7e37b16700 1 -- 172.21.15.73:0/1296097713 wait complete. 2023-11-28T03:07:44.966 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:07:44.967 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:07:45.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.081+0000 7f55208ba700 1 Processor -- start 2023-11-28T03:07:45.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.081+0000 7f55208ba700 1 -- start start 2023-11-28T03:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f55208ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f551c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f55208ba700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f551c158570 0x7f551c158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f55208ba700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551c0c7bc0 con 0x7f551c154950 2023-11-28T03:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f55208ba700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f551c155920 con 0x7f551c158570 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f55208ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c154dd0 con 0x7f551c14e530 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f551a59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f551c154950 0x7f551c14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51074/0 (socket says 172.21.15.73:51074) 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f551a59c700 1 -- 172.21.15.73:0/96741767 learned_addr learned my addr 172.21.15.73:0/96741767 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f551ad9d700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f551c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f551b59e700 1 --2- 172.21.15.73:0/96741767 >> v2:172.21.15.150:3300/0 conn(0x7f551c158570 0x7f551c158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2810507385 0 0) 0x7f551c0c7bc0 con 0x7f551c154950 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55040033f0 con 0x7f551c154950 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 14752823 0 0) 0x7f55040033f0 con 0x7f551c154950 2023-11-28T03:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 msgr2=0x7f551c0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f551c0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 >> v2:172.21.15.150:3300/0 conn(0x7f551c158570 msgr2=0x7f551c158a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 --2- 172.21.15.73:0/96741767 >> v2:172.21.15.150:3300/0 conn(0x7f551c158570 0x7f551c158a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c155080 con 0x7f551c154950 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f551ad9d700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f551c0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f5510001470 con 0x7f551c154950 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 >> v1:172.21.15.73:6789/0 conn(0x7f551c154950 legacy=0x7f551c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55040038e0 0x7f5504003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5504004690 0x7f5504004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f5504005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c155920 con 0x7f5504004690 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c154dd0 con 0x7f55040038e0 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.082+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f550400b8e0 con 0x7f551c14e530 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551b59e700 1 --2- 172.21.15.73:0/96741767 >> v2:172.21.15.150:3300/0 conn(0x7f551c158570 0x7f551c158a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5504004690 0x7f5504004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551ad9d700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55040038e0 0x7f5504003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551b59e700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f5504005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 -- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 msgr2=0x7f5504005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c14e530 0x7f5504005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 -- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55040038e0 msgr2=0x7f5504003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55040038e0 0x7f5504003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 -- 172.21.15.73:0/96741767 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f550c0092a0 con 0x7f5504004690 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551a59c700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5504004690 0x7f5504004b10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f551000c1e0 tx=0x7f5510001e90).ready entity=mon.0 client_cookie=7d9023cd0dfe4af2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510007a60 con 0x7f5504004690 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f551000cab0 con 0x7f5504004690 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f5519d9b700 1 -- 172.21.15.73:0/96741767 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f551000ce50 con 0x7f5504004690 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f55208ba700 1 -- 172.21.15.73:0/96741767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5504004690 msgr2=0x7f5504004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f55208ba700 1 --2- 172.21.15.73:0/96741767 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5504004690 0x7f5504004b10 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 rx=0x7f551000c1e0 tx=0x7f5510001e90).stop 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.083+0000 7f551b59e700 1 -- 172.21.15.73:0/96741767 reap_dead start 2023-11-28T03:07:45.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.084+0000 7f55208ba700 1 -- 172.21.15.73:0/96741767 shutdown_connections 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.084+0000 7f55208ba700 1 -- 172.21.15.73:0/96741767 >> 172.21.15.73:0/96741767 conn(0x7f551c064990 msgr2=0x7f551c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.084+0000 7f55208ba700 1 -- 172.21.15.73:0/96741767 shutdown_connections 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.084+0000 7f55208ba700 1 -- 172.21.15.73:0/96741767 wait complete. 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.085+0000 7f55208ba700 1 Processor -- start 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.085+0000 7f55208ba700 1 -- start start 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.085+0000 7f55208ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5504004690 0x7f551c14cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c159330 0x7f551c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c168190 con 0x7f551c0c1160 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f551c0c1ad0 con 0x7f5504004690 2023-11-28T03:07:45.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f551c1675c0 con 0x7f551c159330 2023-11-28T03:07:45.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49814/0 (socket says 172.21.15.73:49814) 2023-11-28T03:07:45.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 -- 172.21.15.73:0/1004375132 learned_addr learned my addr 172.21.15.73:0/1004375132 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551ad9d700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5504004690 0x7f551c14cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551b59e700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c159330 0x7f551c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 -- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c159330 msgr2=0x7f551c159790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c159330 0x7f551c159790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 -- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5504004690 msgr2=0x7f551c14cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5504004690 0x7f551c14cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 -- 172.21.15.73:0/1004375132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f551c167860 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f551a59c700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f55040055c0 tx=0x7f5510015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5510007a60 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55100074f0 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55100046b0 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f551c168190 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.086+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f551c167e20 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.087+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f551c062ee0 con 0x7f551c0c1160 2023-11-28T03:07:45.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.087+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5510004a20 con 0x7f551c0c1160 2023-11-28T03:07:45.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.088+0000 7f5502ffd700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 0x7f5508061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.088+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f551001c040 con 0x7f551c0c1160 2023-11-28T03:07:45.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.088+0000 7f551ad9d700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 0x7f5508061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.088+0000 7f551ad9d700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 0x7f5508061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f550c006d80 tx=0x7f550c009060).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.092+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f551007ce60 con 0x7f551c0c1160 2023-11-28T03:07:45.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.255+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f551c167fb0 con 0x7f551c0c1160 2023-11-28T03:07:45.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.255+0000 7f5502ffd700 1 -- 172.21.15.73:0/1004375132 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v92) v1 ==== 110+0+10 (secure 0 0 0) 0x7f5510026030 con 0x7f551c0c1160 2023-11-28T03:07:45.256 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 msgr2=0x7f5508061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 0x7f5508061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f550c006d80 tx=0x7f550c009060).stop 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 msgr2=0x7f551c0c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 rx=0x7f55040055c0 tx=0x7f5510015000).stop 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 shutdown_connections 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f550805f150 0x7f5508061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f551c159330 0x7f551c159790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5504004690 0x7f551c14cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 --2- 172.21.15.73:0/1004375132 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f551c0c1160 0x7f551c0c15e0 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 >> 172.21.15.73:0/1004375132 conn(0x7f551c064990 msgr2=0x7f551c153c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 shutdown_connections 2023-11-28T03:07:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.257+0000 7f55208ba700 1 -- 172.21.15.73:0/1004375132 wait complete. 2023-11-28T03:07:45.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:07:45.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.385+0000 7f7eb7ac3700 1 Processor -- start 2023-11-28T03:07:45.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.385+0000 7f7eb7ac3700 1 -- start start 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7eb0098d10 0x7f7eb00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb00c6190 0x7f7eb0063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb0158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb00c72d0 con 0x7f7eb00c6190 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb0154bc0 con 0x7f7eb0098d10 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb7ac3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00c66c0 con 0x7f7eb0064350 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb505e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb00c6190 0x7f7eb0063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb505e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb00c6190 0x7f7eb0063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49826/0 (socket says 172.21.15.73:49826) 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb505e700 1 -- 172.21.15.73:0/2845522274 learned_addr learned my addr 172.21.15.73:0/2845522274 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb6060700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb0158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb585f700 1 --2- 172.21.15.73:0/2845522274 >> v2:172.21.15.150:3300/0 conn(0x7f7eb0098d10 0x7f7eb00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb505e700 1 -- 172.21.15.73:0/2845522274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 msgr2=0x7f7eb0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.386+0000 7f7eb505e700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb0158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb505e700 1 -- 172.21.15.73:0/2845522274 >> v2:172.21.15.150:3300/0 conn(0x7f7eb0098d10 msgr2=0x7f7eb00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb505e700 1 --2- 172.21.15.73:0/2845522274 >> v2:172.21.15.150:3300/0 conn(0x7f7eb0098d10 0x7f7eb00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb505e700 1 -- 172.21.15.73:0/2845522274 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb00c69a0 con 0x7f7eb00c6190 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb505e700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb0063e60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f7eb0083f80 tx=0x7f7ea00090f0).ready entity=mon.0 client_cookie=a92c48da90185e6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb485d700 1 -- 172.21.15.73:0/2845522274 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000b600 con 0x7f7eb00c6190 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb485d700 1 -- 172.21.15.73:0/2845522274 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea000b760 con 0x7f7eb00c6190 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb485d700 1 -- 172.21.15.73:0/2845522274 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea000bae0 con 0x7f7eb00c6190 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/2845522274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 msgr2=0x7f7eb0063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb0063e60 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 rx=0x7f7eb0083f80 tx=0x7f7ea00090f0).stop 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/2845522274 shutdown_connections 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb0158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/2845522274 >> v2:172.21.15.150:3300/0 conn(0x7f7eb0098d10 0x7f7eb00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/2845522274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb0063e60 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/2845522274 >> 172.21.15.73:0/2845522274 conn(0x7f7eb00cb310 msgr2=0x7f7eb00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/2845522274 shutdown_connections 2023-11-28T03:07:45.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.387+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/2845522274 wait complete. 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.388+0000 7f7eb7ac3700 1 Processor -- start 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 -- start start 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb00af490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 0x7f7eb0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb01552b0 con 0x7f7eb00c6190 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00b0260 con 0x7f7eb0098d10 2023-11-28T03:07:45.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb7ac3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb00c4aa0 con 0x7f7eb0064350 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb6060700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb6060700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49832/0 (socket says 172.21.15.73:49832) 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.389+0000 7f7eb6060700 1 -- 172.21.15.73:0/793873950 learned_addr learned my addr 172.21.15.73:0/793873950 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb585f700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb00af490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb505e700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 0x7f7eb0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 -- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 msgr2=0x7f7eb00af490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb00af490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 -- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 msgr2=0x7f7eb0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 0x7f7eb0154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 -- 172.21.15.73:0/793873950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009370 con 0x7f7eb00c6190 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb585f700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb00af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:45.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb505e700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 0x7f7eb0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb6060700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f7ea8009fa0 tx=0x7f7ea8009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea800ed20 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea800ee80 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb00c4d80 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea8014610 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.390+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb00afef0 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.391+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb00b03f0 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.391+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7ea801d3b0 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.392+0000 7f7ea5ffb700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.392+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7ea807a470 con 0x7f7eb00c6190 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.392+0000 7f7eb585f700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.392+0000 7f7eb585f700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7eac0098c0 tx=0x7f7eac008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.396+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7ea804ba30 con 0x7f7eb00c6190 2023-11-28T03:07:45.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.558+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f7eb006c860 con 0x7f7eb00c6190 2023-11-28T03:07:45.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.558+0000 7f7ea5ffb700 1 -- 172.21.15.73:0/793873950 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v92) v1 ==== 92+0+10 (secure 0 0 0) 0x7f7ea804f050 con 0x7f7eb00c6190 2023-11-28T03:07:45.559 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:07:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 msgr2=0x7f7e94061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f7eac0098c0 tx=0x7f7eac008040).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 msgr2=0x7f7eb00c42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 rx=0x7f7ea8009fa0 tx=0x7f7ea8009900).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 shutdown_connections 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb00c6190 0x7f7eb00c42a0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb0064350 0x7f7eb00af490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 --2- 172.21.15.73:0/793873950 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb0098d10 0x7f7eb0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 >> 172.21.15.73:0/793873950 conn(0x7f7eb00cb310 msgr2=0x7f7eb014c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 shutdown_connections 2023-11-28T03:07:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.560+0000 7f7eb7ac3700 1 -- 172.21.15.73:0/793873950 wait complete. 2023-11-28T03:07:45.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:07:45.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.685+0000 7f76416c2700 1 Processor -- start 2023-11-28T03:07:45.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.685+0000 7f76416c2700 1 -- start start 2023-11-28T03:07:45.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76416c2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f763c0cae90 0x7f763c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76416c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0ca180 0x7f763c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76416c2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f763c0a9a90 con 0x7f763c0caab0 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76416c2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f763c0c1490 con 0x7f763c0cae90 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76416c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c154ce0 con 0x7f763c0ca180 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76327fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f763c0cae90 0x7f763c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76327fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f763c0cae90 0x7f763c0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49840/0 (socket says 172.21.15.73:49840) 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f76327fc700 1 -- 172.21.15.73:0/2189331084 learned_addr learned my addr 172.21.15.73:0/2189331084 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.686+0000 7f763b7fe700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0ca180 0x7f763c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76327fc700 1 -- 172.21.15.73:0/2189331084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0ca180 msgr2=0x7f763c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76327fc700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0ca180 0x7f763c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76327fc700 1 -- 172.21.15.73:0/2189331084 >> v1:172.21.15.150:6789/0 conn(0x7f763c0caab0 legacy=0x7f763c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76327fc700 1 -- 172.21.15.73:0/2189331084 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f763c0aab80 con 0x7f763c0cae90 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76327fc700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0cae90 0x7f763c0c0ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f7624006fd0 tx=0x7f76240090f0).ready entity=mon.0 client_cookie=bb5dc92ca710fb64 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f763a7fc700 1 -- 172.21.15.73:0/2189331084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400b580 con 0x7f763c0cae90 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f763a7fc700 1 -- 172.21.15.73:0/2189331084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762400b6e0 con 0x7f763c0cae90 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f763a7fc700 1 -- 172.21.15.73:0/2189331084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762400ba60 con 0x7f763c0cae90 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 -- 172.21.15.73:0/2189331084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0cae90 msgr2=0x7f763c0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0cae90 0x7f763c0c0ab0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 rx=0x7f7624006fd0 tx=0x7f76240090f0).stop 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 -- 172.21.15.73:0/2189331084 shutdown_connections 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0ca180 0x7f763c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 --2- 172.21.15.73:0/2189331084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0cae90 0x7f763c0c0ab0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 -- 172.21.15.73:0/2189331084 >> 172.21.15.73:0/2189331084 conn(0x7f763c094920 msgr2=0x7f763c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 -- 172.21.15.73:0/2189331084 shutdown_connections 2023-11-28T03:07:45.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.687+0000 7f76416c2700 1 -- 172.21.15.73:0/2189331084 wait complete. 2023-11-28T03:07:45.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 Processor -- start 2023-11-28T03:07:45.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 -- start start 2023-11-28T03:07:45.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c0caab0 0x7f763c0affe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0cae90 0x7f763c1518e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c168e90 con 0x7f763c0ca180 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f763c0b04d0 con 0x7f763c0caab0 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f76416c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f763c168280 con 0x7f763c0cae90 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f763affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.689+0000 7f763affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49844/0 (socket says 172.21.15.73:49844) 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 -- 172.21.15.73:0/3449311246 learned_addr learned my addr 172.21.15.73:0/3449311246 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:45.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76327fc700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c0caab0 0x7f763c0affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763b7fe700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0cae90 0x7f763c1518e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 -- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0cae90 msgr2=0x7f763c1518e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0cae90 0x7f763c1518e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 -- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c0caab0 msgr2=0x7f763c0affe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c0caab0 0x7f763c0affe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 -- 172.21.15.73:0/3449311246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7624009370 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f763affd700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f762c00c4c0 tx=0x7f762c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c00ce70 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f762c007020 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f762c0073c0 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f763c168560 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.690+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f763c168b20 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.691+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f763c158120 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.691+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f762c006900 con 0x7f763c0ca180 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.692+0000 7f76337fe700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 0x7f761c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.692+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f762c0797f0 con 0x7f763c0ca180 2023-11-28T03:07:45.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.692+0000 7f76327fc700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 0x7f761c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.692+0000 7f76327fc700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 0x7f761c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7624009fd0 tx=0x7f7624017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.696+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f762c04ae10 con 0x7f763c0ca180 2023-11-28T03:07:45.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.861+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f763c06c860 con 0x7f763c0ca180 2023-11-28T03:07:45.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.862+0000 7f76337fe700 1 -- 172.21.15.73:0/3449311246 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v92) v1 ==== 84+0+143 (secure 0 0 0) 0x7f762c04e430 con 0x7f763c0ca180 2023-11-28T03:07:45.863 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:45.863 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:07:45.863 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:45.863 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:07:45.864 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:45.865 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:07:45.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.863+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 msgr2=0x7f761c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.863+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 0x7f761c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f7624009fd0 tx=0x7f7624017000).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.863+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 msgr2=0x7f763c165740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.863+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 rx=0x7f762c00c4c0 tx=0x7f762c007df0).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 shutdown_connections 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f761c05f150 0x7f761c061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f763c0cae90 0x7f763c1518e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f763c0ca180 0x7f763c165740 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 --2- 172.21.15.73:0/3449311246 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f763c0caab0 0x7f763c0affe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 >> 172.21.15.73:0/3449311246 conn(0x7f763c094920 msgr2=0x7f763c153ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 shutdown_connections 2023-11-28T03:07:45.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.864+0000 7f76416c2700 1 -- 172.21.15.73:0/3449311246 wait complete. 2023-11-28T03:07:45.877 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:07:45.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:07:45.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.993+0000 7fe4993c6700 1 Processor -- start 2023-11-28T03:07:45.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.993+0000 7fe4993c6700 1 -- start start 2023-11-28T03:07:45.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4993c6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 0x7fe494154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4993c6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 0x7fe4940aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:45.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4993c6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4940aabc0 con 0x7fe4940a7ff0 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4993c6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4940c6390 con 0x7fe4940aa220 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4993c6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940c6630 con 0x7fe494154530 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4937fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 0x7fe4940aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe48a7fc700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fe4940a7ff0 0x7fe4940c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:51664/0 (socket says 172.21.15.73:51664) 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe492ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 0x7fe494154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe48a7fc700 1 -- 172.21.15.73:0/346717993 learned_addr learned my addr 172.21.15.73:0/346717993 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4927fc700 1 -- 172.21.15.73:0/346717993 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1568870048 0 0) 0x7fe47c001110 con 0x7fe4940a7ff0 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4927fc700 1 -- 172.21.15.73:0/346717993 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe4800033f0 con 0x7fe4940a7ff0 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe492ffd700 1 -- 172.21.15.73:0/346717993 >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 msgr2=0x7fe4940aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe492ffd700 1 --2- 172.21.15.73:0/346717993 >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 0x7fe4940aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe492ffd700 1 -- 172.21.15.73:0/346717993 >> v1:172.21.15.150:6789/0 conn(0x7fe4940a7ff0 legacy=0x7fe4940c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe492ffd700 1 -- 172.21.15.73:0/346717993 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4940a8470 con 0x7fe494154530 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.994+0000 7fe4937fe700 1 --2- 172.21.15.73:0/346717993 >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 0x7fe4940aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe492ffd700 1 --2- 172.21.15.73:0/346717993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 0x7fe494154910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe484006fd0 tx=0x7fe484006f40).ready entity=mon.2 client_cookie=b6620370e01c23e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4927fc700 1 -- 172.21.15.73:0/346717993 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484005880 con 0x7fe494154530 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4927fc700 1 -- 172.21.15.73:0/346717993 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4840059e0 con 0x7fe494154530 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4927fc700 1 -- 172.21.15.73:0/346717993 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe484005d60 con 0x7fe494154530 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 -- 172.21.15.73:0/346717993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 msgr2=0x7fe494154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 --2- 172.21.15.73:0/346717993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 0x7fe494154910 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 rx=0x7fe484006fd0 tx=0x7fe484006f40).stop 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 -- 172.21.15.73:0/346717993 shutdown_connections 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 --2- 172.21.15.73:0/346717993 >> v2:172.21.15.150:3300/0 conn(0x7fe4940aa220 0x7fe4940aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 --2- 172.21.15.73:0/346717993 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe494154530 0x7fe494154910 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 -- 172.21.15.73:0/346717993 >> 172.21.15.73:0/346717993 conn(0x7fe494094930 msgr2=0x7fe49406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:45.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 -- 172.21.15.73:0/346717993 shutdown_connections 2023-11-28T03:07:45.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.995+0000 7fe4993c6700 1 -- 172.21.15.73:0/346717993 wait complete. 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.996+0000 7fe4993c6700 1 Processor -- start 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 -- start start 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4940a7ff0 0x7fe4940ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe494154530 0x7fe494150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940aad60 con 0x7fe4940aa220 2023-11-28T03:07:46.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940aeb10 con 0x7fe494154530 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4993c6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4940aedf0 con 0x7fe4940a7ff0 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe48a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe48a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49856/0 (socket says 172.21.15.73:49856) 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe48a7fc700 1 -- 172.21.15.73:0/3777283269 learned_addr learned my addr 172.21.15.73:0/3777283269 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe492ffd700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4940a7ff0 0x7fe4940ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.997+0000 7fe4937fe700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe494154530 0x7fe494150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 -- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4940a7ff0 msgr2=0x7fe4940ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4940a7ff0 0x7fe4940ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 -- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe494154530 msgr2=0x7fe494150c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe494154530 0x7fe494150c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 -- 172.21.15.73:0/3777283269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe48c009a20 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48a7fc700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fe47c00c470 tx=0x7fe47c007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0072e0 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47c007440 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47c0057a0 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe484009250 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.998+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4940aafb0 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.999+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4940ab170 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:45.999+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe47c005b50 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.000+0000 7fe48b7fe700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 0x7fe474061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.000+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe47c0793f0 con 0x7fe4940aa220 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.000+0000 7fe492ffd700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 0x7fe474061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.000+0000 7fe492ffd700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 0x7fe474061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe484009fd0 tx=0x7fe484015000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:46.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.004+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe47c04aa10 con 0x7fe4940aa220 2023-11-28T03:07:46.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.166+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fe4940ab320 con 0x7fe4940aa220 2023-11-28T03:07:46.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.294+0000 7fe48b7fe700 1 -- 172.21.15.73:0/3777283269 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v93) v1 ==== 152+0+0 (secure 0 0 0) 0x7fe47c04e030 con 0x7fe4940aa220 2023-11-28T03:07:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 msgr2=0x7fe474061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 0x7fe474061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fe484009fd0 tx=0x7fe484015000).stop 2023-11-28T03:07:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 msgr2=0x7fe4940c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 rx=0x7fe47c00c470 tx=0x7fe47c007ad0).stop 2023-11-28T03:07:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 shutdown_connections 2023-11-28T03:07:46.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe47405f150 0x7fe474061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe494154530 0x7fe494150c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe4940a7ff0 0x7fe4940ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 --2- 172.21.15.73:0/3777283269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe4940aa220 0x7fe4940c4630 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 >> 172.21.15.73:0/3777283269 conn(0x7fe494094930 msgr2=0x7fe4940c0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 shutdown_connections 2023-11-28T03:07:46.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.298+0000 7fe4993c6700 1 -- 172.21.15.73:0/3777283269 wait complete. 2023-11-28T03:07:46.304 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:07:46.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:07:46.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.432+0000 7f154fac9700 1 Processor -- start 2023-11-28T03:07:46.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.432+0000 7f154fac9700 1 -- start start 2023-11-28T03:07:46.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154fac9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15480c86f0 0x7f15480a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154fac9700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480c7e30 con 0x7f1548062700 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154fac9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480bcf40 con 0x7f15481547f0 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154fac9700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480c6b20 con 0x7f15480c86f0 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154e066700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1548062700 0x7f15480a4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51080/0 (socket says 172.21.15.73:51080) 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.433+0000 7f154e066700 1 -- 172.21.15.73:0/638077287 learned_addr learned my addr 172.21.15.73:0/638077287 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154d064700 1 --2- 172.21.15.73:0/638077287 >> v2:172.21.15.150:3300/0 conn(0x7f15480c86f0 0x7f15480a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 221707101 0 0) 0x7f15480c7e30 con 0x7f1548062700 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15340033f0 con 0x7f1548062700 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 558265843 0 0) 0x7f15340033f0 con 0x7f1548062700 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 >> v2:172.21.15.150:3300/0 conn(0x7f15480c86f0 msgr2=0x7f15480a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 --2- 172.21.15.73:0/638077287 >> v2:172.21.15.150:3300/0 conn(0x7f15480c86f0 0x7f15480a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 >> v1:172.21.15.150:6789/0 conn(0x7f15481547f0 legacy=0x7f15480c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480a77d0 con 0x7f1548062700 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f1540001470 con 0x7f1548062700 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 >> v1:172.21.15.73:6789/0 conn(0x7f1548062700 legacy=0x7f15480a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340038e0 0x7f1534003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004690 0x7f1534004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f153400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480c6b20 con 0x7f1534005050 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f153400bfe0 con 0x7f15340038e0 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f153400c1a0 con 0x7f1534004690 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f153400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154d865700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340038e0 0x7f1534003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154d064700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004690 0x7f1534004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 -- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004690 msgr2=0x7f1534004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004690 0x7f1534004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 -- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340038e0 msgr2=0x7f1534003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15340038e0 0x7f1534003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154e066700 1 -- 172.21.15.73:0/638077287 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340033f0 con 0x7f1534005050 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.434+0000 7f154d064700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1534004690 0x7f1534004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154e066700 1 -- 172.21.15.73:0/638077287 reap_dead start 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154e066700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f153400b8e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f154000c160 tx=0x7f1540001e50).ready entity=mon.0 client_cookie=4a41f1ecb5dd46a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540007a20 con 0x7f1534005050 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f154000ca90 con 0x7f1534005050 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154c863700 1 -- 172.21.15.73:0/638077287 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f154000ce10 con 0x7f1534005050 2023-11-28T03:07:46.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 -- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 msgr2=0x7f153400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f153400b8e0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 rx=0x7f154000c160 tx=0x7f1540001e50).stop 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 -- 172.21.15.73:0/638077287 shutdown_connections 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 --2- 172.21.15.73:0/638077287 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f153400b8e0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 -- 172.21.15.73:0/638077287 >> 172.21.15.73:0/638077287 conn(0x7f15480642d0 msgr2=0x7f15480c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 -- 172.21.15.73:0/638077287 shutdown_connections 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.435+0000 7f154fac9700 1 -- 172.21.15.73:0/638077287 wait complete. 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.436+0000 7f154fac9700 1 Processor -- start 2023-11-28T03:07:46.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.436+0000 7f154fac9700 1 -- start start 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15480a9330 0x7f1548066da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 0x7f1548066360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15480a8ea0 con 0x7f1534005050 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1548066850 con 0x7f1548065ee0 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154fac9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1548153380 con 0x7f15480a9330 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49866/0 (socket says 172.21.15.73:49866) 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 -- 172.21.15.73:0/2590076804 learned_addr learned my addr 172.21.15.73:0/2590076804 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d064700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15480a9330 0x7f1548066da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154e066700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 0x7f1548066360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 -- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15480a9330 msgr2=0x7f1548066da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15480a9330 0x7f1548066da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 -- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 msgr2=0x7f1548066360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 0x7f1548066360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154d865700 1 -- 172.21.15.73:0/2590076804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480a8570 con 0x7f1534005050 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.437+0000 7f154e066700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 0x7f1548066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:46.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f154d865700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f154400e990 tx=0x7f1544007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544017070 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1544001800 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1544001b80 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480a8850 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.438+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15480aaf90 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.439+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15480a9180 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.439+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f154400c7e0 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.440+0000 7f153dffb700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 0x7f152c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.440+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f1544079e10 con 0x7f1534005050 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.440+0000 7f154d064700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 0x7f152c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:46.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.440+0000 7f154d064700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 0x7f152c061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f15380030c0 tx=0x7f1538008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:46.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.444+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f154404b2c0 con 0x7f1534005050 2023-11-28T03:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:46.607+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f15480a8cf0 con 0x7f1534005050 2023-11-28T03:07:47.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.303+0000 7f153dffb700 1 -- 172.21.15.73:0/2590076804 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v94) v1 ==== 170+0+0 (secure 0 0 0) 0x7f154404e8e0 con 0x7f1534005050 2023-11-28T03:07:47.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.308+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 msgr2=0x7f152c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.308+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 0x7f152c061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f15380030c0 tx=0x7f1538008040).stop 2023-11-28T03:07:47.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.308+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 msgr2=0x7f154814e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.308+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 rx=0x7f154400e990 tx=0x7f1544007f30).stop 2023-11-28T03:07:47.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 shutdown_connections 2023-11-28T03:07:47.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f152c05f150 0x7f152c061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1548065ee0 0x7f1548066360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1534005050 0x7f154814e030 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 --2- 172.21.15.73:0/2590076804 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15480a9330 0x7f1548066da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.309+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 >> 172.21.15.73:0/2590076804 conn(0x7f15480642d0 msgr2=0x7f15480c5e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:47.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.311+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 shutdown_connections 2023-11-28T03:07:47.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.314+0000 7f154fac9700 1 -- 172.21.15.73:0/2590076804 wait complete. 2023-11-28T03:07:47.320 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:07:47.335 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:07:47.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.450+0000 7ff5530d4700 1 Processor -- start 2023-11-28T03:07:47.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.450+0000 7ff5530d4700 1 -- start start 2023-11-28T03:07:47.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff5530d4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff5530d4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff54c158320 0x7ff54c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff5530d4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c158cc0 con 0x7ff54c098d10 2023-11-28T03:07:47.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff5530d4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c0c72d0 con 0x7ff54c158320 2023-11-28T03:07:47.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff5530d4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c064570 con 0x7ff54c0c6190 2023-11-28T03:07:47.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff551671700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff54c158320 0x7ff54c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff551671700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff54c158320 0x7ff54c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49882/0 (socket says 172.21.15.73:49882) 2023-11-28T03:07:47.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.451+0000 7ff551671700 1 -- 172.21.15.73:0/1290463964 learned_addr learned my addr 172.21.15.73:0/1290463964 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:47.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff551671700 1 -- 172.21.15.73:0/1290463964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 msgr2=0x7ff54c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:47.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff551671700 1 --2- 172.21.15.73:0/1290463964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:47.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff551671700 1 -- 172.21.15.73:0/1290463964 >> v1:172.21.15.150:6789/0 conn(0x7ff54c098d10 legacy=0x7ff54c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:47.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff551671700 1 -- 172.21.15.73:0/1290463964 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c154bc0 con 0x7ff54c158320 2023-11-28T03:07:47.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff551671700 1 --2- 172.21.15.73:0/1290463964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c158320 0x7ff54c1587d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7ff544006fd0 tx=0x7ff5440090f0).ready entity=mon.0 client_cookie=486eee1633e76541 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5437fe700 1 -- 172.21.15.73:0/1290463964 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400b600 con 0x7ff54c158320 2023-11-28T03:07:47.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5437fe700 1 -- 172.21.15.73:0/1290463964 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54400b760 con 0x7ff54c158320 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5437fe700 1 -- 172.21.15.73:0/1290463964 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54400bae0 con 0x7ff54c158320 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 -- 172.21.15.73:0/1290463964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c158320 msgr2=0x7ff54c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1290463964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c158320 0x7ff54c1587d0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 rx=0x7ff544006fd0 tx=0x7ff5440090f0).stop 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 -- 172.21.15.73:0/1290463964 shutdown_connections 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1290463964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c158320 0x7ff54c1587d0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1290463964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 -- 172.21.15.73:0/1290463964 >> 172.21.15.73:0/1290463964 conn(0x7ff54c0cb310 msgr2=0x7ff54c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 -- 172.21.15.73:0/1290463964 shutdown_connections 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.452+0000 7ff5530d4700 1 -- 172.21.15.73:0/1290463964 wait complete. 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.453+0000 7ff5530d4700 1 Processor -- start 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 -- start start 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 0x7ff54c154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c157bc0 con 0x7ff54c098d10 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0b0150 con 0x7ff54c158320 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.454+0000 7ff5530d4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c1578e0 con 0x7ff54c0c6190 2023-11-28T03:07:47.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49888/0 (socket says 172.21.15.73:49888) 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 -- 172.21.15.73:0/1175682364 learned_addr learned my addr 172.21.15.73:0/1175682364 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff551671700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 0x7ff54c154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff543fff700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 -- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 msgr2=0x7ff54c0c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c23b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 -- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 msgr2=0x7ff54c154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 0x7ff54c154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 -- 172.21.15.73:0/1175682364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff544009370 con 0x7ff54c098d10 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff551671700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 0x7ff54c154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:47.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff550e70700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7ff54800b790 tx=0x7ff5480092f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800d600 con 0x7ff54c098d10 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff54800d7a0 con 0x7ff54c098d10 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c0afb00 con 0x7ff54c098d10 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff54800db60 con 0x7ff54c098d10 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.455+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c150520 con 0x7ff54c098d10 2023-11-28T03:07:47.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.456+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff54c06c860 con 0x7ff54c098d10 2023-11-28T03:07:47.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.457+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff548019460 con 0x7ff54c098d10 2023-11-28T03:07:47.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.457+0000 7ff540ff9700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 0x7ff538061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.457+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6021+0+0 (secure 0 0 0) 0x7ff54807a350 con 0x7ff54c098d10 2023-11-28T03:07:47.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.457+0000 7ff543fff700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 0x7ff538061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.458+0000 7ff543fff700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 0x7ff538061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff54c0ad9b0 tx=0x7ff53c008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.461+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff54804b680 con 0x7ff54c098d10 2023-11-28T03:07:47.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.667+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7ff54c150b60 con 0x7ff54c098d10 2023-11-28T03:07:47.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.679+0000 7ff540ff9700 1 -- 172.21.15.73:0/1175682364 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 11 and data pool 12 v30) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff54804eca0 con 0x7ff54c098d10 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.685+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 msgr2=0x7ff538061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.685+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 0x7ff538061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7ff54c0ad9b0 tx=0x7ff53c008040).stop 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.685+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 msgr2=0x7ff54c157300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.685+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 rx=0x7ff54800b790 tx=0x7ff5480092f0).stop 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 shutdown_connections 2023-11-28T03:07:47.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff53805f150 0x7ff538061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff54c158320 0x7ff54c154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff54c098d10 0x7ff54c157300 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 --2- 172.21.15.73:0/1175682364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff54c0c6190 0x7ff54c0c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.687+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 >> 172.21.15.73:0/1175682364 conn(0x7ff54c0cb310 msgr2=0x7ff54c0c9d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:47.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.688+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 shutdown_connections 2023-11-28T03:07:47.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.689+0000 7ff5530d4700 1 -- 172.21.15.73:0/1175682364 wait complete. 2023-11-28T03:07:47.697 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 11 and data pool 12 2023-11-28T03:07:47.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:47.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.821+0000 7f7086fc6700 1 Processor -- start 2023-11-28T03:07:47.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.821+0000 7f7086fc6700 1 -- start start 2023-11-28T03:07:47.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7086fc6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70801544d0 0x7f7080155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7086fc6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7080066f40 0x7f7080095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7086fc6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70800683d0 con 0x7f70800673c0 2023-11-28T03:07:47.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7086fc6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7080067850 con 0x7f7080066f40 2023-11-28T03:07:47.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7086fc6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7080067b30 con 0x7f70801544d0 2023-11-28T03:07:47.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7085563700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f70800673c0 0x7f7080063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51094/0 (socket says 172.21.15.73:51094) 2023-11-28T03:07:47.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f7085563700 1 -- 172.21.15.73:0/1630514411 learned_addr learned my addr 172.21.15.73:0/1630514411 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:47.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.822+0000 7f707ffff700 1 --2- 172.21.15.73:0/1630514411 >> v2:172.21.15.73:3300/0 conn(0x7f7080066f40 0x7f7080095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7084d62700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70801544d0 0x7f7080155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707ffff700 1 -- 172.21.15.73:0/1630514411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70801544d0 msgr2=0x7f7080155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:47.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707ffff700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70801544d0 0x7f7080155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707ffff700 1 -- 172.21.15.73:0/1630514411 >> v1:172.21.15.73:6789/0 conn(0x7f70800673c0 legacy=0x7f7080063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707f7fe700 1 -- 172.21.15.73:0/1630514411 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1933482460 0 0) 0x7f70800683d0 con 0x7f70800673c0 2023-11-28T03:07:47.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707ffff700 1 -- 172.21.15.73:0/1630514411 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7080067e10 con 0x7f7080066f40 2023-11-28T03:07:47.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707ffff700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7080066f40 0x7f7080095e80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f7074009760 tx=0x7f7074009060).ready entity=mon.0 client_cookie=6bef6a1ab6837c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707f7fe700 1 -- 172.21.15.73:0/1630514411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400f510 con 0x7f7080066f40 2023-11-28T03:07:47.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707f7fe700 1 -- 172.21.15.73:0/1630514411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707400f670 con 0x7f7080066f40 2023-11-28T03:07:47.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f707f7fe700 1 -- 172.21.15.73:0/1630514411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707400f9f0 con 0x7f7080066f40 2023-11-28T03:07:47.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 -- 172.21.15.73:0/1630514411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7080066f40 msgr2=0x7f7080095e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7080066f40 0x7f7080095e80 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 rx=0x7f7074009760 tx=0x7f7074009060).stop 2023-11-28T03:07:47.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 -- 172.21.15.73:0/1630514411 shutdown_connections 2023-11-28T03:07:47.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70801544d0 0x7f7080155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 --2- 172.21.15.73:0/1630514411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7080066f40 0x7f7080095e80 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.823+0000 7f7086fc6700 1 -- 172.21.15.73:0/1630514411 >> 172.21.15.73:0/1630514411 conn(0x7f70800631d0 msgr2=0x7f70800c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:47.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.824+0000 7f7086fc6700 1 -- 172.21.15.73:0/1630514411 shutdown_connections 2023-11-28T03:07:47.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.824+0000 7f7086fc6700 1 -- 172.21.15.73:0/1630514411 wait complete. 2023-11-28T03:07:47.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.825+0000 7f7086fc6700 1 Processor -- start 2023-11-28T03:07:47.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.825+0000 7f7086fc6700 1 -- start start 2023-11-28T03:07:47.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.825+0000 7f7086fc6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 0x7f70801583d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.825+0000 7f7086fc6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70800a6cc0 0x7f70801566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800a6830 con 0x7f70801544d0 2023-11-28T03:07:47.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70800a5f00 con 0x7f70800a6cc0 2023-11-28T03:07:47.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70800a61e0 con 0x7f7080066f40 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49896/0 (socket says 172.21.15.73:49896) 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 -- 172.21.15.73:0/2767731307 learned_addr learned my addr 172.21.15.73:0/2767731307 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7084d62700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 0x7f70801583d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7085563700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70800a6cc0 0x7f70801566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 -- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 msgr2=0x7f70801583d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 0x7f70801583d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 -- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70800a6cc0 msgr2=0x7f70801566d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70800a6cc0 0x7f70801566d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 -- 172.21.15.73:0/2767731307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70740092a0 con 0x7f70801544d0 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7084d62700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 0x7f70801583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:47.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707ffff700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f708006c7f0 tx=0x7f7074017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011550 con 0x7f70801544d0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707401e070 con 0x7f70801544d0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7074011940 con 0x7f70801544d0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70800a64c0 con 0x7f70801544d0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.826+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7080159870 con 0x7f70801544d0 2023-11-28T03:07:47.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.827+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70800a69a0 con 0x7f70801544d0 2023-11-28T03:07:47.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.828+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7074020030 con 0x7f70801544d0 2023-11-28T03:07:47.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.828+0000 7f707cff9700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 0x7f706c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:47.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.828+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f707407b290 con 0x7f70801544d0 2023-11-28T03:07:47.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.828+0000 7f7084d62700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 0x7f706c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:47.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.828+0000 7f7084d62700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 0x7f706c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7070009fd0 tx=0x7f707000b040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:47.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.832+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f707404c560 con 0x7f70801544d0 2023-11-28T03:07:47.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.997+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f708006c860 con 0x7f70801544d0 2023-11-28T03:07:48.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:47.998+0000 7f707cff9700 1 -- 172.21.15.73:0/2767731307 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16944 (secure 0 0 0) 0x7f7074011cb0 con 0x7f70801544d0 2023-11-28T03:07:48.012 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:48.012 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:47.668910+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:48.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 msgr2=0x7f706c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 0x7f706c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f7070009fd0 tx=0x7f707000b040).stop 2023-11-28T03:07:48.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 msgr2=0x7f7080159380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 rx=0x7f708006c7f0 tx=0x7f7074017000).stop 2023-11-28T03:07:48.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 shutdown_connections 2023-11-28T03:07:48.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f706c05f150 0x7f706c061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70800a6cc0 0x7f70801566d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7080066f40 0x7f70801583d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 --2- 172.21.15.73:0/2767731307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70801544d0 0x7f7080159380 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 >> 172.21.15.73:0/2767731307 conn(0x7f70800631d0 msgr2=0x7f70800c77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:48.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 shutdown_connections 2023-11-28T03:07:48.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.000+0000 7f7086fc6700 1 -- 172.21.15.73:0/2767731307 wait complete. 2023-11-28T03:07:48.019 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:48.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.133+0000 7f86d731a700 1 Processor -- start 2023-11-28T03:07:48.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.133+0000 7f86d731a700 1 -- start start 2023-11-28T03:07:48.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d731a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86d01548c0 0x7f86d006a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d731a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f86d00c6e30 0x7f86d00c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d731a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86d00c69a0 con 0x7f86d01544e0 2023-11-28T03:07:48.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d731a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00c13a0 con 0x7f86d01548c0 2023-11-28T03:07:48.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d731a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86d00c1680 con 0x7f86d00c6e30 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d48b5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86d01548c0 0x7f86d006a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d48b5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f86d01548c0 0x7f86d006a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49904/0 (socket says 172.21.15.73:49904) 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.134+0000 7f86d48b5700 1 -- 172.21.15.73:0/2567669505 learned_addr learned my addr 172.21.15.73:0/2567669505 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d58b7700 1 --2- 172.21.15.73:0/2567669505 >> v2:172.21.15.150:3300/0 conn(0x7f86d00c6e30 0x7f86d00c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d48b5700 1 -- 172.21.15.73:0/2567669505 >> v2:172.21.15.150:3300/0 conn(0x7f86d00c6e30 msgr2=0x7f86d00c72a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:48.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d48b5700 1 --2- 172.21.15.73:0/2567669505 >> v2:172.21.15.150:3300/0 conn(0x7f86d00c6e30 0x7f86d00c72a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d48b5700 1 -- 172.21.15.73:0/2567669505 >> v1:172.21.15.150:6789/0 conn(0x7f86d01544e0 legacy=0x7f86d00c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d48b5700 1 -- 172.21.15.73:0/2567669505 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86d00c6630 con 0x7f86d01548c0 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d48b5700 1 --2- 172.21.15.73:0/2567669505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01548c0 0x7f86d006a610 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f86d00cb240 tx=0x7f86c00090f0).ready entity=mon.0 client_cookie=c197cc86238d07cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86c7fff700 1 -- 172.21.15.73:0/2567669505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000b600 con 0x7f86d01548c0 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86c7fff700 1 -- 172.21.15.73:0/2567669505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86c000b760 con 0x7f86d01548c0 2023-11-28T03:07:48.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86c7fff700 1 -- 172.21.15.73:0/2567669505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86c000bae0 con 0x7f86d01548c0 2023-11-28T03:07:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 -- 172.21.15.73:0/2567669505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01548c0 msgr2=0x7f86d006a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 --2- 172.21.15.73:0/2567669505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01548c0 0x7f86d006a610 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 rx=0x7f86d00cb240 tx=0x7f86c00090f0).stop 2023-11-28T03:07:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 -- 172.21.15.73:0/2567669505 shutdown_connections 2023-11-28T03:07:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 --2- 172.21.15.73:0/2567669505 >> v2:172.21.15.150:3300/0 conn(0x7f86d00c6e30 0x7f86d00c72a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 --2- 172.21.15.73:0/2567669505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d01548c0 0x7f86d006a610 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.135+0000 7f86d731a700 1 -- 172.21.15.73:0/2567669505 >> 172.21.15.73:0/2567669505 conn(0x7f86d0085020 msgr2=0x7f86d0094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.136+0000 7f86d731a700 1 -- 172.21.15.73:0/2567669505 shutdown_connections 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.136+0000 7f86d731a700 1 -- 172.21.15.73:0/2567669505 wait complete. 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.137+0000 7f86d731a700 1 Processor -- start 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.137+0000 7f86d731a700 1 -- start start 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.137+0000 7f86d731a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.137+0000 7f86d731a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86d01544e0 0x7f86d0095350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.137+0000 7f86d731a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86d01548c0 0x7f86d00a8a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d731a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0096fd0 con 0x7f86d00c6e30 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d731a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86d00972b0 con 0x7f86d01548c0 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d731a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86d0095d30 con 0x7f86d01544e0 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49908/0 (socket says 172.21.15.73:49908) 2023-11-28T03:07:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 -- 172.21.15.73:0/749623233 learned_addr learned my addr 172.21.15.73:0/749623233 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d58b7700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86d01548c0 0x7f86d00a8a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d48b5700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86d01544e0 0x7f86d0095350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 -- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86d01544e0 msgr2=0x7f86d0095350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86d01544e0 0x7f86d0095350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 -- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86d01548c0 msgr2=0x7f86d00a8a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86d01548c0 0x7f86d00a8a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 -- 172.21.15.73:0/749623233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c0009370 con 0x7f86d00c6e30 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d50b6700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f86cc00c740 tx=0x7f86cc0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc006a70 con 0x7f86d00c6e30 2023-11-28T03:07:48.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86cc006bd0 con 0x7f86d00c6e30 2023-11-28T03:07:48.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86cc007020 con 0x7f86d00c6e30 2023-11-28T03:07:48.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86d0096010 con 0x7f86d00c6e30 2023-11-28T03:07:48.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.138+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86d0152d10 con 0x7f86d00c6e30 2023-11-28T03:07:48.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.139+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86d006c860 con 0x7f86d00c6e30 2023-11-28T03:07:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.139+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f86cc0073d0 con 0x7f86d00c6e30 2023-11-28T03:07:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.140+0000 7f86c57fa700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 0x7f86b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.140+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f86cc0797b0 con 0x7f86d00c6e30 2023-11-28T03:07:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.140+0000 7f86d48b5700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 0x7f86b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.140+0000 7f86d48b5700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 0x7f86b4061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f86c0009130 tx=0x7f86c0017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.144+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f86cc047af0 con 0x7f86d00c6e30 2023-11-28T03:07:48.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.309+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86d0097590 con 0x7f86d00c6e30 2023-11-28T03:07:48.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.310+0000 7f86c57fa700 1 -- 172.21.15.73:0/749623233 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16944 (secure 0 0 0) 0x7f86cc04aa80 con 0x7f86d00c6e30 2023-11-28T03:07:48.311 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:48.311 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":95,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:47.668910+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 msgr2=0x7f86b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 0x7f86b4061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f86c0009130 tx=0x7f86c0017000).stop 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 msgr2=0x7f86d0152820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 rx=0x7f86cc00c740 tx=0x7f86cc0015e0).stop 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 shutdown_connections 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f86b405f150 0x7f86b4061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f86d01548c0 0x7f86d00a8a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f86d00c6e30 0x7f86d0152820 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 --2- 172.21.15.73:0/749623233 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f86d01544e0 0x7f86d0095350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 >> 172.21.15.73:0/749623233 conn(0x7f86d0085020 msgr2=0x7f86d00c4340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:48.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 shutdown_connections 2023-11-28T03:07:48.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.312+0000 7f86d731a700 1 -- 172.21.15.73:0/749623233 wait complete. 2023-11-28T03:07:48.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:07:48.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.479+0000 7f3f29c6b700 1 Processor -- start 2023-11-28T03:07:48.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 -- start start 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f240c6190 0x7f3f24063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 0x7f3f24158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f240c72d0 con 0x7f3f240c6190 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f24154bc0 con 0x7f3f24064350 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.480+0000 7f3f29c6b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f240c66c0 con 0x7f3f24098d10 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f240c6190 0x7f3f24063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3f240c6190 0x7f3f24063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49918/0 (socket says 172.21.15.73:49918) 2023-11-28T03:07:48.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 -- 172.21.15.73:0/1549817421 learned_addr learned my addr 172.21.15.73:0/1549817421 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f237fe700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f23fff700 1 --2- 172.21.15.73:0/1549817421 >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 0x7f3f24158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 -- 172.21.15.73:0/1549817421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 msgr2=0x7f3f240c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 -- 172.21.15.73:0/1549817421 >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 msgr2=0x7f3f24158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/1549817421 >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 0x7f3f24158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 -- 172.21.15.73:0/1549817421 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f240c69a0 con 0x7f3f240c6190 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f23fff700 1 --2- 172.21.15.73:0/1549817421 >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 0x7f3f24158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f240c6190 0x7f3f24063e60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f3f24084570 tx=0x7f3f0c0090f0).ready entity=mon.0 client_cookie=12d2816c2914022d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f22ffd700 1 -- 172.21.15.73:0/1549817421 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00b600 con 0x7f3f240c6190 2023-11-28T03:07:48.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f22ffd700 1 -- 172.21.15.73:0/1549817421 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c00b760 con 0x7f3f240c6190 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.481+0000 7f3f22ffd700 1 -- 172.21.15.73:0/1549817421 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c00bae0 con 0x7f3f240c6190 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 -- 172.21.15.73:0/1549817421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f240c6190 msgr2=0x7f3f24063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f240c6190 0x7f3f24063e60 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 rx=0x7f3f24084570 tx=0x7f3f0c0090f0).stop 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 -- 172.21.15.73:0/1549817421 shutdown_connections 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/1549817421 >> v2:172.21.15.150:3300/0 conn(0x7f3f24064350 0x7f3f24158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/1549817421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f240c6190 0x7f3f24063e60 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 -- 172.21.15.73:0/1549817421 >> 172.21.15.73:0/1549817421 conn(0x7f3f240cb310 msgr2=0x7f3f240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 -- 172.21.15.73:0/1549817421 shutdown_connections 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.482+0000 7f3f29c6b700 1 -- 172.21.15.73:0/1549817421 wait complete. 2023-11-28T03:07:48.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.483+0000 7f3f29c6b700 1 Processor -- start 2023-11-28T03:07:48.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.483+0000 7f3f29c6b700 1 -- start start 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240af050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f240c6190 0x7f3f240af540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24155890 con 0x7f3f24064350 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24155b70 con 0x7f3f240c6190 2023-11-28T03:07:48.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f24155e50 con 0x7f3f24098d10 2023-11-28T03:07:48.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49928/0 (socket says 172.21.15.73:49928) 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 -- 172.21.15.73:0/2469718196 learned_addr learned my addr 172.21.15.73:0/2469718196 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 -- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 msgr2=0x7f3f240af050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240af050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240af050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f23fff700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f240c6190 0x7f3f240af540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 -- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f240c6190 msgr2=0x7f3f240af540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f240c6190 0x7f3f240af540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 -- 172.21.15.73:0/2469718196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c009370 con 0x7f3f24064350 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f237fe700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f3f1400d890 tx=0x7f3f14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f140125f0 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14012750 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f24156130 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14012ad0 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.484+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f24156e20 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.485+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f24157100 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.486+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3f14013480 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.486+0000 7f3f1b7fe700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 0x7f3f04061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.486+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3f1407a390 con 0x7f3f24064350 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.486+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 0x7f3f04061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:48.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.486+0000 7f3f1bfff700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 0x7f3f04061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3f24084570 tx=0x7f3f0c017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:48.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.490+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f1404b600 con 0x7f3f24064350 2023-11-28T03:07:48.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:48.699+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f3f2406c860 con 0x7f3f24064350 2023-11-28T03:07:49.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.680+0000 7f3f1b7fe700 1 -- 172.21.15.73:0/2469718196 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v33) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3f1404ec20 con 0x7f3f24064350 2023-11-28T03:07:49.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 msgr2=0x7f3f04061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 0x7f3f04061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f3f24084570 tx=0x7f3f0c017000).stop 2023-11-28T03:07:49.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 msgr2=0x7f3f24156930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 rx=0x7f3f1400d890 tx=0x7f3f14008040).stop 2023-11-28T03:07:49.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 shutdown_connections 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3f0405f150 0x7f3f04061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f240c6190 0x7f3f240af540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f24064350 0x7f3f24156930 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 --2- 172.21.15.73:0/2469718196 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f24098d10 0x7f3f240af050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 >> 172.21.15.73:0/2469718196 conn(0x7f3f240cb310 msgr2=0x7f3f24158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:49.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 shutdown_connections 2023-11-28T03:07:49.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.682+0000 7f3f29c6b700 1 -- 172.21.15.73:0/2469718196 wait complete. 2023-11-28T03:07:49.698 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:07:49.698 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:07:49.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.818+0000 7f1975c8f700 1 Processor -- start 2023-11-28T03:07:49.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.818+0000 7f1975c8f700 1 -- start start 2023-11-28T03:07:49.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f1975c8f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1970158a30 0x7f19700c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:49.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f1975c8f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19700c72d0 con 0x7f1970098d10 2023-11-28T03:07:49.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f1975c8f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1970064570 con 0x7f19700c6190 2023-11-28T03:07:49.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f1975c8f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1970154bc0 con 0x7f1970158a30 2023-11-28T03:07:49.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1970098d10 0x7f1970063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51098/0 (socket says 172.21.15.73:51098) 2023-11-28T03:07:49.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196f7fe700 1 -- 172.21.15.73:0/641118605 learned_addr learned my addr 172.21.15.73:0/641118605 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:49.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196ffff700 1 --2- 172.21.15.73:0/641118605 >> v2:172.21.15.73:3300/0 conn(0x7f1970158a30 0x7f19700c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:49.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196e7fc700 1 -- 172.21.15.73:0/641118605 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 188441929 0 0) 0x7f19700c72d0 con 0x7f1970098d10 2023-11-28T03:07:49.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196e7fc700 1 -- 172.21.15.73:0/641118605 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19580033f0 con 0x7f1970098d10 2023-11-28T03:07:49.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196ffff700 1 -- 172.21.15.73:0/641118605 >> v1:172.21.15.150:6789/0 conn(0x7f19700c6190 legacy=0x7f1970158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196ffff700 1 -- 172.21.15.73:0/641118605 >> v1:172.21.15.73:6789/0 conn(0x7f1970098d10 legacy=0x7f1970063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.819+0000 7f196ffff700 1 -- 172.21.15.73:0/641118605 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19700c66c0 con 0x7f1970158a30 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f196ffff700 1 --2- 172.21.15.73:0/641118605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700c7760 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f1968009760 tx=0x7f1968009060).ready entity=mon.0 client_cookie=cc3049d73bc1bf15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f196e7fc700 1 -- 172.21.15.73:0/641118605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800f510 con 0x7f1970158a30 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f196e7fc700 1 -- 172.21.15.73:0/641118605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196800f670 con 0x7f1970158a30 2023-11-28T03:07:49.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f196e7fc700 1 -- 172.21.15.73:0/641118605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800f9f0 con 0x7f1970158a30 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 -- 172.21.15.73:0/641118605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 msgr2=0x7f19700c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 --2- 172.21.15.73:0/641118605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700c7760 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 rx=0x7f1968009760 tx=0x7f1968009060).stop 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 -- 172.21.15.73:0/641118605 shutdown_connections 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 --2- 172.21.15.73:0/641118605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700c7760 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 -- 172.21.15.73:0/641118605 >> 172.21.15.73:0/641118605 conn(0x7f19700cb310 msgr2=0x7f19700cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 -- 172.21.15.73:0/641118605 shutdown_connections 2023-11-28T03:07:49.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.820+0000 7f1975c8f700 1 -- 172.21.15.73:0/641118605 wait complete. 2023-11-28T03:07:49.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.821+0000 7f1975c8f700 1 Processor -- start 2023-11-28T03:07:49.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 -- start start 2023-11-28T03:07:49.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970098d10 0x7f197014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:49.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19700c6190 0x7f1970155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:49.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1970156ca0 con 0x7f1970158a30 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19701566e0 con 0x7f1970098d10 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f1975c8f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19701569c0 con 0x7f19700c6190 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f196ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f196ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49940/0 (socket says 172.21.15.73:49940) 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.822+0000 7f196ffff700 1 -- 172.21.15.73:0/635241436 learned_addr learned my addr 172.21.15.73:0/635241436 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:49.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196f7fe700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970098d10 0x7f197014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196effd700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19700c6190 0x7f1970155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 -- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19700c6190 msgr2=0x7f1970155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19700c6190 0x7f1970155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 -- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970098d10 msgr2=0x7f197014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970098d10 0x7f197014bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 -- 172.21.15.73:0/635241436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680092a0 con 0x7f1970158a30 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f196ffff700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f19700c2360 tx=0x7f1968017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:49.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800f510 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f196801b410 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19700afd20 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f196800f9f0 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.823+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f197014c4a0 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.824+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19700c99a0 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.824+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f196801ba60 con 0x7f1970158a30 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.825+0000 7f19577fe700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 0x7f195c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:49.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.825+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f196807bc50 con 0x7f1970158a30 2023-11-28T03:07:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.825+0000 7f196f7fe700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 0x7f195c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.825+0000 7f196f7fe700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 0x7f195c061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1960005ad0 tx=0x7f1960009040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:49.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:49.829+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f19700c99a0 con 0x7f1970158a30 2023-11-28T03:07:50.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.035+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f19700c99a0 con 0x7f1970158a30 2023-11-28T03:07:50.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.686+0000 7f19577fe700 1 -- 172.21.15.73:0/635241436 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f19700c99a0 con 0x7f1970158a30 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 msgr2=0x7f195c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 0x7f195c061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f1960005ad0 tx=0x7f1960009040).stop 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 msgr2=0x7f19700b0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 rx=0x7f19700c2360 tx=0x7f1968017000).stop 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 shutdown_connections 2023-11-28T03:07:50.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f195c05f150 0x7f195c061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1970158a30 0x7f19700b0520 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1970098d10 0x7f197014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 --2- 172.21.15.73:0/635241436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f19700c6190 0x7f1970155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 >> 172.21.15.73:0/635241436 conn(0x7f19700cb310 msgr2=0x7f1970153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 shutdown_connections 2023-11-28T03:07:50.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.689+0000 7f1975c8f700 1 -- 172.21.15.73:0/635241436 wait complete. 2023-11-28T03:07:50.703 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:07:50.703 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:50.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.818+0000 7effa6b0a700 1 Processor -- start 2023-11-28T03:07:50.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.818+0000 7effa6b0a700 1 -- start start 2023-11-28T03:07:50.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa6b0a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7effa013b790 0x7effa0065840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa6b0a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7effa00a7890 0x7effa0065d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa6b0a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa00c2a90 con 0x7effa013bb70 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa6b0a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa0156bf0 con 0x7effa013b790 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa6b0a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa0066220 con 0x7effa00a7890 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7effa013b790 0x7effa0065840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7effa013b790 0x7effa0065840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49952/0 (socket says 172.21.15.73:49952) 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 -- 172.21.15.73:0/3870158513 learned_addr learned my addr 172.21.15.73:0/3870158513 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5307700 1 --2- 172.21.15.73:0/3870158513 >> v2:172.21.15.150:3300/0 conn(0x7effa00a7890 0x7effa0065d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 -- 172.21.15.73:0/3870158513 >> v2:172.21.15.150:3300/0 conn(0x7effa00a7890 msgr2=0x7effa0065d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 --2- 172.21.15.73:0/3870158513 >> v2:172.21.15.150:3300/0 conn(0x7effa00a7890 0x7effa0065d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 -- 172.21.15.73:0/3870158513 >> v1:172.21.15.150:6789/0 conn(0x7effa013bb70 legacy=0x7effa0155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.819+0000 7effa5b08700 1 -- 172.21.15.73:0/3870158513 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa0156910 con 0x7effa013b790 2023-11-28T03:07:50.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa5b08700 1 --2- 172.21.15.73:0/3870158513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013b790 0x7effa0065840 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7effa00c4150 tx=0x7eff9c0090f0).ready entity=mon.0 client_cookie=ff74e2f427f09059 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa4b06700 1 -- 172.21.15.73:0/3870158513 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00b580 con 0x7effa013b790 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa4b06700 1 -- 172.21.15.73:0/3870158513 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00b6e0 con 0x7effa013b790 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa4b06700 1 -- 172.21.15.73:0/3870158513 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00ba60 con 0x7effa013b790 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 -- 172.21.15.73:0/3870158513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013b790 msgr2=0x7effa0065840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 --2- 172.21.15.73:0/3870158513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013b790 0x7effa0065840 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 rx=0x7effa00c4150 tx=0x7eff9c0090f0).stop 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 -- 172.21.15.73:0/3870158513 shutdown_connections 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 --2- 172.21.15.73:0/3870158513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013b790 0x7effa0065840 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 --2- 172.21.15.73:0/3870158513 >> v2:172.21.15.150:3300/0 conn(0x7effa00a7890 0x7effa0065d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 -- 172.21.15.73:0/3870158513 >> 172.21.15.73:0/3870158513 conn(0x7effa00c7ca0 msgr2=0x7effa00c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 -- 172.21.15.73:0/3870158513 shutdown_connections 2023-11-28T03:07:50.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.820+0000 7effa6b0a700 1 -- 172.21.15.73:0/3870158513 wait complete. 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.821+0000 7effa6b0a700 1 Processor -- start 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.821+0000 7effa6b0a700 1 -- start start 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa00a7890 0x7effa0151300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effa013b790 0x7effa00aae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00bf230 con 0x7effa013bb70 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c20d0 con 0x7effa00a7890 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6b0a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa01668c0 con 0x7effa013b790 2023-11-28T03:07:50.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6309700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6309700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49964/0 (socket says 172.21.15.73:49964) 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.822+0000 7effa6309700 1 -- 172.21.15.73:0/1872559433 learned_addr learned my addr 172.21.15.73:0/1872559433 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 -- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effa013b790 msgr2=0x7effa00aae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effa013b790 0x7effa00aae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 -- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa00a7890 msgr2=0x7effa0151300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa00a7890 0x7effa0151300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 -- 172.21.15.73:0/1872559433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c009370 con 0x7effa013bb70 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6309700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7eff9800b840 tx=0x7eff98009290).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800d580 con 0x7effa013bb70 2023-11-28T03:07:50.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9800d700 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa00aba00 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9800dac0 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.823+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa01665e0 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.824+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa006c7c0 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.824+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff98010440 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.825+0000 7eff95ffb700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 0x7eff84061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.825+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7eff9807a1e0 con 0x7effa013bb70 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.825+0000 7effa5b08700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 0x7eff84061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:50.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.825+0000 7effa5b08700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 0x7eff84061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7eff9c009130 tx=0x7eff9c017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:50.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:50.829+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff9804b450 con 0x7effa013bb70 2023-11-28T03:07:51.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.033+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effa00c2440 con 0x7effa013bb70 2023-11-28T03:07:51.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.034+0000 7eff95ffb700 1 -- 172.21.15.73:0/1872559433 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7eff980093b0 con 0x7effa013bb70 2023-11-28T03:07:51.035 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:51.035 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 msgr2=0x7eff84061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 0x7eff84061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7eff9c009130 tx=0x7eff9c017000).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 msgr2=0x7effa00bed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 rx=0x7eff9800b840 tx=0x7eff98009290).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 shutdown_connections 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff8405f150 0x7eff84061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effa013bb70 0x7effa00bed40 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effa00a7890 0x7effa0151300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 --2- 172.21.15.73:0/1872559433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7effa013b790 0x7effa00aae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:51.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 >> 172.21.15.73:0/1872559433 conn(0x7effa00c7ca0 msgr2=0x7effa00bcf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.035+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 shutdown_connections 2023-11-28T03:07:51.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.036+0000 7effa6b0a700 1 -- 172.21.15.73:0/1872559433 wait complete. 2023-11-28T03:07:51.038 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:07:51.048 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:07:51.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.159+0000 7f0d1ad3c700 1 Processor -- start 2023-11-28T03:07:51.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d1ad3c700 1 -- start start 2023-11-28T03:07:51.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d1ad3c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0d140c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d1ad3c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140a7490 con 0x7f0d14067cb0 2023-11-28T03:07:51.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d1ad3c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d14154d20 con 0x7f0d14068090 2023-11-28T03:07:51.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d1ad3c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14151670 con 0x7f0d140c45d0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.160+0000 7f0d18ad8700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d14067cb0 0x7f0d140c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51112/0 (socket says 172.21.15.73:51112) 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 -- 172.21.15.73:0/2709666846 learned_addr learned my addr 172.21.15.73:0/2709666846 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0bfff700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0d140c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2449763319 0 0) 0x7f0d10001490 con 0x7f0d14067cb0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0cfc0033f0 con 0x7f0d14067cb0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2490422570 0 0) 0x7f0cfc0033f0 con 0x7f0d14067cb0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 msgr2=0x7f0d140c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0d140c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 >> v1:172.21.15.150:6789/0 conn(0x7f0d14068090 legacy=0x7f0d140c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d141555c0 con 0x7f0d14067cb0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f0d10001980 con 0x7f0d14067cb0 2023-11-28T03:07:51.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 >> v1:172.21.15.73:6789/0 conn(0x7f0d14067cb0 legacy=0x7f0d140c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0cfc003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cfc004690 0x7f0cfc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0cfc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00b8e0 con 0x7f0cfc003910 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00ba50 con 0x7f0cfc004690 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cfc00bc10 con 0x7f0d140c45d0 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0cfc003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d0bfff700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cfc004690 0x7f0cfc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d192d9700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0cfc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 -- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 msgr2=0x7f0cfc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:51.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140c45d0 0x7f0cfc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 -- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cfc004690 msgr2=0x7f0cfc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0cfc004690 0x7f0cfc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.161+0000 7f0d18ad8700 1 -- 172.21.15.73:0/2709666846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cfc0033f0 con 0x7f0cfc003910 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0cfc003d30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f0d1000c670 tx=0x7f0d14155530).ready entity=mon.0 client_cookie=fdd37c9217e4d059 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10005b20 con 0x7f0cfc003910 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10005c80 con 0x7f0cfc003910 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d0b7fe700 1 -- 172.21.15.73:0/2709666846 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10007110 con 0x7f0cfc003910 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/2709666846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 msgr2=0x7f0cfc003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/2709666846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0cfc003d30 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 rx=0x7f0d1000c670 tx=0x7f0d14155530).stop 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d192d9700 1 -- 172.21.15.73:0/2709666846 reap_dead start 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/2709666846 shutdown_connections 2023-11-28T03:07:51.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/2709666846 >> 172.21.15.73:0/2709666846 conn(0x7f0d140c7fd0 msgr2=0x7f0d140c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/2709666846 shutdown_connections 2023-11-28T03:07:51.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.162+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/2709666846 wait complete. 2023-11-28T03:07:51.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.163+0000 7f0d1ad3c700 1 Processor -- start 2023-11-28T03:07:51.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 -- start start 2023-11-28T03:07:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140a81e0 0x7f0d140697f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140a8660 0x7f0d14069ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1406a1d0 con 0x7f0cfc003910 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140ab9d0 con 0x7f0d140a81e0 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d1ad3c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140abcb0 con 0x7f0d140a8660 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d18ad8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d18ad8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:49984/0 (socket says 172.21.15.73:49984) 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d18ad8700 1 -- 172.21.15.73:0/993920186 learned_addr learned my addr 172.21.15.73:0/993920186 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d192d9700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140a8660 0x7f0d14069ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.164+0000 7f0d0bfff700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140a81e0 0x7f0d140697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 -- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140a8660 msgr2=0x7f0d14069ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140a8660 0x7f0d14069ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 -- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140a81e0 msgr2=0x7f0d140697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140a81e0 0x7f0d140697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 -- 172.21.15.73:0/993920186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140abf90 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d18ad8700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f0cfc00ba20 tx=0x7f0d10015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d1000ced0 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d10001940 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d140ac270 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d10007070 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.165+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d140aa070 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.166+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d140ac790 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.166+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0d10014410 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.167+0000 7f0d08ff9700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.167+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0d1007b280 con 0x7f0cfc003910 2023-11-28T03:07:51.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.167+0000 7f0d0bfff700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.167+0000 7f0d0bfff700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0d1414c550 tx=0x7f0d04009060).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.171+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d140ac790 con 0x7f0cfc003910 2023-11-28T03:07:51.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.334+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d140a7490 con 0x7f0cfc003910 2023-11-28T03:07:51.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.335+0000 7f0d08ff9700 1 -- 172.21.15.73:0/993920186 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+16931 (secure 0 0 0) 0x7f0d140a7490 con 0x7f0cfc003910 2023-11-28T03:07:51.336 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:51.336 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":96,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:07:48.670976+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 msgr2=0x7f0d00061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f0d1414c550 tx=0x7f0d04009060).stop 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 msgr2=0x7f0d14160440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 rx=0x7f0cfc00ba20 tx=0x7f0d10015000).stop 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 shutdown_connections 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140a8660 0x7f0d14069ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0cfc003910 0x7f0d14160440 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 --2- 172.21.15.73:0/993920186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140a81e0 0x7f0d140697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 >> 172.21.15.73:0/993920186 conn(0x7f0d140c7fd0 msgr2=0x7f0d140bd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 shutdown_connections 2023-11-28T03:07:51.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.337+0000 7f0d1ad3c700 1 -- 172.21.15.73:0/993920186 wait complete. 2023-11-28T03:07:51.350 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:07:51.350 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:07:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.462+0000 7f2122ef0700 1 Processor -- start 2023-11-28T03:07:51.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 -- start start 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f211c1544d0 0x7f211c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f211c0683d0 con 0x7f211c0673c0 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f211c067850 con 0x7f211c1544d0 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.463+0000 7f2122ef0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c067b30 con 0x7f211c066f40 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f211c1544d0 0x7f211c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f212148d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f211c0673c0 0x7f211c063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51114/0 (socket says 172.21.15.73:51114) 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f212148d700 1 -- 172.21.15.73:0/858517639 learned_addr learned my addr 172.21.15.73:0/858517639 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f211bfff700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 -- 172.21.15.73:0/858517639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 msgr2=0x7f211c095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 -- 172.21.15.73:0/858517639 >> v1:172.21.15.73:6789/0 conn(0x7f211c0673c0 legacy=0x7f211c063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f211b7fe700 1 -- 172.21.15.73:0/858517639 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1976827968 0 0) 0x7f211c0683d0 con 0x7f211c0673c0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 -- 172.21.15.73:0/858517639 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f211c067e10 con 0x7f211c1544d0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f2120c8c700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c1544d0 0x7f211c155750 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f2110006fd0 tx=0x7f2110006f40).ready entity=mon.0 client_cookie=cc4a51481516b8da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f211b7fe700 1 -- 172.21.15.73:0/858517639 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005880 con 0x7f211c1544d0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f211b7fe700 1 -- 172.21.15.73:0/858517639 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21100059e0 con 0x7f211c1544d0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.464+0000 7f211b7fe700 1 -- 172.21.15.73:0/858517639 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2110005d60 con 0x7f211c1544d0 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 -- 172.21.15.73:0/858517639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c1544d0 msgr2=0x7f211c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c1544d0 0x7f211c155750 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 rx=0x7f2110006fd0 tx=0x7f2110006f40).stop 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 -- 172.21.15.73:0/858517639 shutdown_connections 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c1544d0 0x7f211c155750 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 --2- 172.21.15.73:0/858517639 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 -- 172.21.15.73:0/858517639 >> 172.21.15.73:0/858517639 conn(0x7f211c0631d0 msgr2=0x7f211c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 -- 172.21.15.73:0/858517639 shutdown_connections 2023-11-28T03:07:51.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.465+0000 7f2122ef0700 1 -- 172.21.15.73:0/858517639 wait complete. 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.466+0000 7f2122ef0700 1 Processor -- start 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.466+0000 7f2122ef0700 1 -- start start 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c0a5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 0x7f211c1689f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0a7100 con 0x7f211c0a7590 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f211c168ee0 con 0x7f211c1544d0 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f211c0a64f0 con 0x7f211c066f40 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50002/0 (socket says 172.21.15.73:50002) 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 -- 172.21.15.73:0/4160700117 learned_addr learned my addr 172.21.15.73:0/4160700117 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f211bfff700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 0x7f211c1689f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2120c8c700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c0a5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 -- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 msgr2=0x7f211c0a5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c0a5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 -- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 msgr2=0x7f211c1689f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 0x7f211c1689f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 -- 172.21.15.73:0/4160700117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2110009250 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f211bfff700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 0x7f211c1689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2120c8c700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c0a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f212148d700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f211400c7e0 tx=0x7f2114007f10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114005d30 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2114005e90 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f211c0a67d0 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2114007340 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.467+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f211c0a6d90 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.468+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f211c0a7270 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.469+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f21140076f0 con 0x7f211c0a7590 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.469+0000 7f2118ff9700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 0x7f2100061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.469+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f211407a200 con 0x7f211c0a7590 2023-11-28T03:07:51.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.469+0000 7f2120c8c700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 0x7f2100061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.469+0000 7f2120c8c700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 0x7f2100061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2110009fd0 tx=0x7f2110016000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.473+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f211404b4d0 con 0x7f211c0a7590 2023-11-28T03:07:51.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.633+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f211c06c860 con 0x7f210005f150 2023-11-28T03:07:51.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.644+0000 7f2118ff9700 1 -- 172.21.15.73:0/4160700117 <== mgr.6082 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361392 (secure 0 0 0) 0x7f211c06c860 con 0x7f210005f150 2023-11-28T03:07:51.649 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:51.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 msgr2=0x7f2100061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 0x7f2100061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f2110009fd0 tx=0x7f2110016000).stop 2023-11-28T03:07:51.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 msgr2=0x7f211c0ae570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 rx=0x7f211400c7e0 tx=0x7f2114007f10).stop 2023-11-28T03:07:51.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 shutdown_connections 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f210005f150 0x7f2100061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f211c0a7590 0x7f211c0ae570 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f211c066f40 0x7f211c0a5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 --2- 172.21.15.73:0/4160700117 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f211c1544d0 0x7f211c1689f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 >> 172.21.15.73:0/4160700117 conn(0x7f211c0631d0 msgr2=0x7f211c0c72e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 shutdown_connections 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.647+0000 7f2122ef0700 1 -- 172.21.15.73:0/4160700117 wait complete. 2023-11-28T03:07:51.653 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:07:51.660 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:07:50.714846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2418424,"kb_used_data":8888,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752556296,"statfs":{"total":773094113280,"available":770617647104,"internally_reserved":0,"allocated":9101312,"data_stored":4499342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10151,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-59,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.583405"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692249+0000","last_change":"2023-11-28T03:07:47.692249+0000","last_active":"2023-11-28T03:07:47.692249+0000","last_peered":"2023-11-28T03:07:47.692249+0000","last_clean":"2023-11-28T03:07:47.692249+0000","last_became_active":"2023-11-28T03:07:47.691962+0000","last_became_peered":"2023-11-28T03:07:47.691962+0000","last_unstale":"2023-11-28T03:07:47.692249+0000","last_undegraded":"2023-11-28T03:07:47.692249+0000","last_fullsized":"2023-11-28T03:07:47.692249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"95'3","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696498+0000","last_change":"2023-11-28T03:07:47.334144+0000","last_active":"2023-11-28T03:07:47.696498+0000","last_peered":"2023-11-28T03:07:47.696498+0000","last_clean":"2023-11-28T03:07:47.696498+0000","last_became_active":"2023-11-28T03:07:47.333881+0000","last_became_peered":"2023-11-28T03:07:47.333881+0000","last_unstale":"2023-11-28T03:07:47.696498+0000","last_undegraded":"2023-11-28T03:07:47.696498+0000","last_fullsized":"2023-11-28T03:07:47.696498+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688167+0000","last_change":"2023-11-28T03:07:47.688167+0000","last_active":"2023-11-28T03:07:47.688167+0000","last_peered":"2023-11-28T03:07:47.688167+0000","last_clean":"2023-11-28T03:07:47.688167+0000","last_became_active":"2023-11-28T03:07:47.687766+0000","last_became_peered":"2023-11-28T03:07:47.687766+0000","last_unstale":"2023-11-28T03:07:47.688167+0000","last_undegraded":"2023-11-28T03:07:47.688167+0000","last_fullsized":"2023-11-28T03:07:47.688167+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325881+0000","last_change":"2023-11-28T03:07:47.323580+0000","last_active":"2023-11-28T03:07:47.325881+0000","last_peered":"2023-11-28T03:07:47.325881+0000","last_clean":"2023-11-28T03:07:47.325881+0000","last_became_active":"2023-11-28T03:07:47.323305+0000","last_became_peered":"2023-11-28T03:07:47.323305+0000","last_unstale":"2023-11-28T03:07:47.325881+0000","last_undegraded":"2023-11-28T03:07:47.325881+0000","last_fullsized":"2023-11-28T03:07:47.325881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696755+0000","last_change":"2023-11-28T03:07:47.696755+0000","last_active":"2023-11-28T03:07:47.696755+0000","last_peered":"2023-11-28T03:07:47.696755+0000","last_clean":"2023-11-28T03:07:47.696755+0000","last_became_active":"2023-11-28T03:07:47.696487+0000","last_became_peered":"2023-11-28T03:07:47.696487+0000","last_unstale":"2023-11-28T03:07:47.696755+0000","last_undegraded":"2023-11-28T03:07:47.696755+0000","last_fullsized":"2023-11-28T03:07:47.696755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333584+0000","last_change":"2023-11-28T03:07:47.323854+0000","last_active":"2023-11-28T03:07:47.333584+0000","last_peered":"2023-11-28T03:07:47.333584+0000","last_clean":"2023-11-28T03:07:47.333584+0000","last_became_active":"2023-11-28T03:07:47.320163+0000","last_became_peered":"2023-11-28T03:07:47.320163+0000","last_unstale":"2023-11-28T03:07:47.333584+0000","last_undegraded":"2023-11-28T03:07:47.333584+0000","last_fullsized":"2023-11-28T03:07:47.333584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695481+0000","last_change":"2023-11-28T03:07:47.695481+0000","last_active":"2023-11-28T03:07:47.695481+0000","last_peered":"2023-11-28T03:07:47.695481+0000","last_clean":"2023-11-28T03:07:47.695481+0000","last_became_active":"2023-11-28T03:07:47.695213+0000","last_became_peered":"2023-11-28T03:07:47.695213+0000","last_unstale":"2023-11-28T03:07:47.695481+0000","last_undegraded":"2023-11-28T03:07:47.695481+0000","last_fullsized":"2023-11-28T03:07:47.695481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696607+0000","last_change":"2023-11-28T03:07:47.323582+0000","last_active":"2023-11-28T03:07:47.696607+0000","last_peered":"2023-11-28T03:07:47.696607+0000","last_clean":"2023-11-28T03:07:47.696607+0000","last_became_active":"2023-11-28T03:07:47.323303+0000","last_became_peered":"2023-11-28T03:07:47.323303+0000","last_unstale":"2023-11-28T03:07:47.696607+0000","last_undegraded":"2023-11-28T03:07:47.696607+0000","last_fullsized":"2023-11-28T03:07:47.696607+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698597+0000","last_change":"2023-11-28T03:07:47.698597+0000","last_active":"2023-11-28T03:07:47.698597+0000","last_peered":"2023-11-28T03:07:47.698597+0000","last_clean":"2023-11-28T03:07:47.698597+0000","last_became_active":"2023-11-28T03:07:47.698082+0000","last_became_peered":"2023-11-28T03:07:47.698082+0000","last_unstale":"2023-11-28T03:07:47.698597+0000","last_undegraded":"2023-11-28T03:07:47.698597+0000","last_fullsized":"2023-11-28T03:07:47.698597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.675151+0000","last_change":"2023-11-28T03:07:47.326018+0000","last_active":"2023-11-28T03:07:47.675151+0000","last_peered":"2023-11-28T03:07:47.675151+0000","last_clean":"2023-11-28T03:07:47.675151+0000","last_became_active":"2023-11-28T03:07:47.325774+0000","last_became_peered":"2023-11-28T03:07:47.325774+0000","last_unstale":"2023-11-28T03:07:47.675151+0000","last_undegraded":"2023-11-28T03:07:47.675151+0000","last_fullsized":"2023-11-28T03:07:47.675151+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691619+0000","last_change":"2023-11-28T03:07:47.691619+0000","last_active":"2023-11-28T03:07:47.691619+0000","last_peered":"2023-11-28T03:07:47.691619+0000","last_clean":"2023-11-28T03:07:47.691619+0000","last_became_active":"2023-11-28T03:07:47.691006+0000","last_became_peered":"2023-11-28T03:07:47.691006+0000","last_unstale":"2023-11-28T03:07:47.691619+0000","last_undegraded":"2023-11-28T03:07:47.691619+0000","last_fullsized":"2023-11-28T03:07:47.691619+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.674766+0000","last_change":"2023-11-28T03:07:47.327576+0000","last_active":"2023-11-28T03:07:47.674766+0000","last_peered":"2023-11-28T03:07:47.674766+0000","last_clean":"2023-11-28T03:07:47.674766+0000","last_became_active":"2023-11-28T03:07:47.327255+0000","last_became_peered":"2023-11-28T03:07:47.327255+0000","last_unstale":"2023-11-28T03:07:47.674766+0000","last_undegraded":"2023-11-28T03:07:47.674766+0000","last_fullsized":"2023-11-28T03:07:47.674766+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690774+0000","last_change":"2023-11-28T03:07:47.690774+0000","last_active":"2023-11-28T03:07:47.690774+0000","last_peered":"2023-11-28T03:07:47.690774+0000","last_clean":"2023-11-28T03:07:47.690774+0000","last_became_active":"2023-11-28T03:07:47.690521+0000","last_became_peered":"2023-11-28T03:07:47.690521+0000","last_unstale":"2023-11-28T03:07:47.690774+0000","last_undegraded":"2023-11-28T03:07:47.690774+0000","last_fullsized":"2023-11-28T03:07:47.690774+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329800+0000","last_change":"2023-11-28T03:07:47.325059+0000","last_active":"2023-11-28T03:07:47.329800+0000","last_peered":"2023-11-28T03:07:47.329800+0000","last_clean":"2023-11-28T03:07:47.329800+0000","last_became_active":"2023-11-28T03:07:47.324393+0000","last_became_peered":"2023-11-28T03:07:47.324393+0000","last_unstale":"2023-11-28T03:07:47.329800+0000","last_undegraded":"2023-11-28T03:07:47.329800+0000","last_fullsized":"2023-11-28T03:07:47.329800+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695215+0000","last_change":"2023-11-28T03:07:47.695215+0000","last_active":"2023-11-28T03:07:47.695215+0000","last_peered":"2023-11-28T03:07:47.695215+0000","last_clean":"2023-11-28T03:07:47.695215+0000","last_became_active":"2023-11-28T03:07:47.694898+0000","last_became_peered":"2023-11-28T03:07:47.694898+0000","last_unstale":"2023-11-28T03:07:47.695215+0000","last_undegraded":"2023-11-28T03:07:47.695215+0000","last_fullsized":"2023-11-28T03:07:47.695215+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324435+0000","last_change":"2023-11-28T03:07:47.317037+0000","last_active":"2023-11-28T03:07:47.324435+0000","last_peered":"2023-11-28T03:07:47.324435+0000","last_clean":"2023-11-28T03:07:47.324435+0000","last_became_active":"2023-11-28T03:07:47.316745+0000","last_became_peered":"2023-11-28T03:07:47.316745+0000","last_unstale":"2023-11-28T03:07:47.324435+0000","last_undegraded":"2023-11-28T03:07:47.324435+0000","last_fullsized":"2023-11-28T03:07:47.324435+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699729+0000","last_change":"2023-11-28T03:07:47.699729+0000","last_active":"2023-11-28T03:07:47.699729+0000","last_peered":"2023-11-28T03:07:47.699729+0000","last_clean":"2023-11-28T03:07:47.699729+0000","last_became_active":"2023-11-28T03:07:47.699467+0000","last_became_peered":"2023-11-28T03:07:47.699467+0000","last_unstale":"2023-11-28T03:07:47.699729+0000","last_undegraded":"2023-11-28T03:07:47.699729+0000","last_fullsized":"2023-11-28T03:07:47.699729+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326748+0000","last_change":"2023-11-28T03:07:47.319703+0000","last_active":"2023-11-28T03:07:47.326748+0000","last_peered":"2023-11-28T03:07:47.326748+0000","last_clean":"2023-11-28T03:07:47.326748+0000","last_became_active":"2023-11-28T03:07:47.319204+0000","last_became_peered":"2023-11-28T03:07:47.319204+0000","last_unstale":"2023-11-28T03:07:47.326748+0000","last_undegraded":"2023-11-28T03:07:47.326748+0000","last_fullsized":"2023-11-28T03:07:47.326748+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689521+0000","last_change":"2023-11-28T03:07:47.689521+0000","last_active":"2023-11-28T03:07:47.689521+0000","last_peered":"2023-11-28T03:07:47.689521+0000","last_clean":"2023-11-28T03:07:47.689521+0000","last_became_active":"2023-11-28T03:07:47.689253+0000","last_became_peered":"2023-11-28T03:07:47.689253+0000","last_unstale":"2023-11-28T03:07:47.689521+0000","last_undegraded":"2023-11-28T03:07:47.689521+0000","last_fullsized":"2023-11-28T03:07:47.689521+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329988+0000","last_change":"2023-11-28T03:07:47.325325+0000","last_active":"2023-11-28T03:07:47.329988+0000","last_peered":"2023-11-28T03:07:47.329988+0000","last_clean":"2023-11-28T03:07:47.329988+0000","last_became_active":"2023-11-28T03:07:47.325017+0000","last_became_peered":"2023-11-28T03:07:47.325017+0000","last_unstale":"2023-11-28T03:07:47.329988+0000","last_undegraded":"2023-11-28T03:07:47.329988+0000","last_fullsized":"2023-11-28T03:07:47.329988+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696431+0000","last_change":"2023-11-28T03:07:47.696431+0000","last_active":"2023-11-28T03:07:47.696431+0000","last_peered":"2023-11-28T03:07:47.696431+0000","last_clean":"2023-11-28T03:07:47.696431+0000","last_became_active":"2023-11-28T03:07:47.696167+0000","last_became_peered":"2023-11-28T03:07:47.696167+0000","last_unstale":"2023-11-28T03:07:47.696431+0000","last_undegraded":"2023-11-28T03:07:47.696431+0000","last_fullsized":"2023-11-28T03:07:47.696431+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336261+0000","last_change":"2023-11-28T03:07:47.334251+0000","last_active":"2023-11-28T03:07:47.336261+0000","last_peered":"2023-11-28T03:07:47.336261+0000","last_clean":"2023-11-28T03:07:47.336261+0000","last_became_active":"2023-11-28T03:07:47.333977+0000","last_became_peered":"2023-11-28T03:07:47.333977+0000","last_unstale":"2023-11-28T03:07:47.336261+0000","last_undegraded":"2023-11-28T03:07:47.336261+0000","last_fullsized":"2023-11-28T03:07:47.336261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687411+0000","last_change":"2023-11-28T03:07:47.687411+0000","last_active":"2023-11-28T03:07:47.687411+0000","last_peered":"2023-11-28T03:07:47.687411+0000","last_clean":"2023-11-28T03:07:47.687411+0000","last_became_active":"2023-11-28T03:07:47.687104+0000","last_became_peered":"2023-11-28T03:07:47.687104+0000","last_unstale":"2023-11-28T03:07:47.687411+0000","last_undegraded":"2023-11-28T03:07:47.687411+0000","last_fullsized":"2023-11-28T03:07:47.687411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695635+0000","last_change":"2023-11-28T03:07:47.327306+0000","last_active":"2023-11-28T03:07:47.695635+0000","last_peered":"2023-11-28T03:07:47.695635+0000","last_clean":"2023-11-28T03:07:47.695635+0000","last_became_active":"2023-11-28T03:07:47.327024+0000","last_became_peered":"2023-11-28T03:07:47.327024+0000","last_unstale":"2023-11-28T03:07:47.695635+0000","last_undegraded":"2023-11-28T03:07:47.695635+0000","last_fullsized":"2023-11-28T03:07:47.695635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.697211+0000","last_change":"2023-11-28T03:07:47.697211+0000","last_active":"2023-11-28T03:07:47.697211+0000","last_peered":"2023-11-28T03:07:47.697211+0000","last_clean":"2023-11-28T03:07:47.697211+0000","last_became_active":"2023-11-28T03:07:47.696875+0000","last_became_peered":"2023-11-28T03:07:47.696875+0000","last_unstale":"2023-11-28T03:07:47.697211+0000","last_undegraded":"2023-11-28T03:07:47.697211+0000","last_fullsized":"2023-11-28T03:07:47.697211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681879+0000","last_change":"2023-11-28T03:07:47.321297+0000","last_active":"2023-11-28T03:07:47.681879+0000","last_peered":"2023-11-28T03:07:47.681879+0000","last_clean":"2023-11-28T03:07:47.681879+0000","last_became_active":"2023-11-28T03:07:47.321002+0000","last_became_peered":"2023-11-28T03:07:47.321002+0000","last_unstale":"2023-11-28T03:07:47.681879+0000","last_undegraded":"2023-11-28T03:07:47.681879+0000","last_fullsized":"2023-11-28T03:07:47.681879+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693002+0000","last_change":"2023-11-28T03:07:47.693002+0000","last_active":"2023-11-28T03:07:47.693002+0000","last_peered":"2023-11-28T03:07:47.693002+0000","last_clean":"2023-11-28T03:07:47.693002+0000","last_became_active":"2023-11-28T03:07:47.692534+0000","last_became_peered":"2023-11-28T03:07:47.692534+0000","last_unstale":"2023-11-28T03:07:47.693002+0000","last_undegraded":"2023-11-28T03:07:47.693002+0000","last_fullsized":"2023-11-28T03:07:47.693002+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325355+0000","last_change":"2023-11-28T03:07:47.321867+0000","last_active":"2023-11-28T03:07:47.325355+0000","last_peered":"2023-11-28T03:07:47.325355+0000","last_clean":"2023-11-28T03:07:47.325355+0000","last_became_active":"2023-11-28T03:07:47.321585+0000","last_became_peered":"2023-11-28T03:07:47.321585+0000","last_unstale":"2023-11-28T03:07:47.325355+0000","last_undegraded":"2023-11-28T03:07:47.325355+0000","last_fullsized":"2023-11-28T03:07:47.325355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699168+0000","last_change":"2023-11-28T03:07:47.699168+0000","last_active":"2023-11-28T03:07:47.699168+0000","last_peered":"2023-11-28T03:07:47.699168+0000","last_clean":"2023-11-28T03:07:47.699168+0000","last_became_active":"2023-11-28T03:07:47.698861+0000","last_became_peered":"2023-11-28T03:07:47.698861+0000","last_unstale":"2023-11-28T03:07:47.699168+0000","last_undegraded":"2023-11-28T03:07:47.699168+0000","last_fullsized":"2023-11-28T03:07:47.699168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320293+0000","last_change":"2023-11-28T03:07:47.315532+0000","last_active":"2023-11-28T03:07:47.320293+0000","last_peered":"2023-11-28T03:07:47.320293+0000","last_clean":"2023-11-28T03:07:47.320293+0000","last_became_active":"2023-11-28T03:07:47.315209+0000","last_became_peered":"2023-11-28T03:07:47.315209+0000","last_unstale":"2023-11-28T03:07:47.320293+0000","last_undegraded":"2023-11-28T03:07:47.320293+0000","last_fullsized":"2023-11-28T03:07:47.320293+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.684777+0000","last_change":"2023-11-28T03:07:47.684777+0000","last_active":"2023-11-28T03:07:47.684777+0000","last_peered":"2023-11-28T03:07:47.684777+0000","last_clean":"2023-11-28T03:07:47.684777+0000","last_became_active":"2023-11-28T03:07:47.684484+0000","last_became_peered":"2023-11-28T03:07:47.684484+0000","last_unstale":"2023-11-28T03:07:47.684777+0000","last_undegraded":"2023-11-28T03:07:47.684777+0000","last_fullsized":"2023-11-28T03:07:47.684777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696909+0000","last_change":"2023-11-28T03:07:47.323752+0000","last_active":"2023-11-28T03:07:47.696909+0000","last_peered":"2023-11-28T03:07:47.696909+0000","last_clean":"2023-11-28T03:07:47.696909+0000","last_became_active":"2023-11-28T03:07:47.323479+0000","last_became_peered":"2023-11-28T03:07:47.323479+0000","last_unstale":"2023-11-28T03:07:47.696909+0000","last_undegraded":"2023-11-28T03:07:47.696909+0000","last_fullsized":"2023-11-28T03:07:47.696909+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688698+0000","last_change":"2023-11-28T03:07:47.688698+0000","last_active":"2023-11-28T03:07:47.688698+0000","last_peered":"2023-11-28T03:07:47.688698+0000","last_clean":"2023-11-28T03:07:47.688698+0000","last_became_active":"2023-11-28T03:07:47.686107+0000","last_became_peered":"2023-11-28T03:07:47.686107+0000","last_unstale":"2023-11-28T03:07:47.688698+0000","last_undegraded":"2023-11-28T03:07:47.688698+0000","last_fullsized":"2023-11-28T03:07:47.688698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326782+0000","last_change":"2023-11-28T03:07:47.319867+0000","last_active":"2023-11-28T03:07:47.326782+0000","last_peered":"2023-11-28T03:07:47.326782+0000","last_clean":"2023-11-28T03:07:47.326782+0000","last_became_active":"2023-11-28T03:07:47.319456+0000","last_became_peered":"2023-11-28T03:07:47.319456+0000","last_unstale":"2023-11-28T03:07:47.326782+0000","last_undegraded":"2023-11-28T03:07:47.326782+0000","last_fullsized":"2023-11-28T03:07:47.326782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.700610+0000","last_change":"2023-11-28T03:07:47.700610+0000","last_active":"2023-11-28T03:07:47.700610+0000","last_peered":"2023-11-28T03:07:47.700610+0000","last_clean":"2023-11-28T03:07:47.700610+0000","last_became_active":"2023-11-28T03:07:47.700299+0000","last_became_peered":"2023-11-28T03:07:47.700299+0000","last_unstale":"2023-11-28T03:07:47.700610+0000","last_undegraded":"2023-11-28T03:07:47.700610+0000","last_fullsized":"2023-11-28T03:07:47.700610+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.328009+0000","last_change":"2023-11-28T03:07:47.326183+0000","last_active":"2023-11-28T03:07:47.328009+0000","last_peered":"2023-11-28T03:07:47.328009+0000","last_clean":"2023-11-28T03:07:47.328009+0000","last_became_active":"2023-11-28T03:07:47.325943+0000","last_became_peered":"2023-11-28T03:07:47.325943+0000","last_unstale":"2023-11-28T03:07:47.328009+0000","last_undegraded":"2023-11-28T03:07:47.328009+0000","last_fullsized":"2023-11-28T03:07:47.328009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686823+0000","last_change":"2023-11-28T03:07:47.686823+0000","last_active":"2023-11-28T03:07:47.686823+0000","last_peered":"2023-11-28T03:07:47.686823+0000","last_clean":"2023-11-28T03:07:47.686823+0000","last_became_active":"2023-11-28T03:07:47.686557+0000","last_became_peered":"2023-11-28T03:07:47.686557+0000","last_unstale":"2023-11-28T03:07:47.686823+0000","last_undegraded":"2023-11-28T03:07:47.686823+0000","last_fullsized":"2023-11-28T03:07:47.686823+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.676175+0000","last_change":"2023-11-28T03:07:47.323256+0000","last_active":"2023-11-28T03:07:47.676175+0000","last_peered":"2023-11-28T03:07:47.676175+0000","last_clean":"2023-11-28T03:07:47.676175+0000","last_became_active":"2023-11-28T03:07:47.322919+0000","last_became_peered":"2023-11-28T03:07:47.322919+0000","last_unstale":"2023-11-28T03:07:47.676175+0000","last_undegraded":"2023-11-28T03:07:47.676175+0000","last_fullsized":"2023-11-28T03:07:47.676175+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693868+0000","last_change":"2023-11-28T03:07:47.693868+0000","last_active":"2023-11-28T03:07:47.693868+0000","last_peered":"2023-11-28T03:07:47.693868+0000","last_clean":"2023-11-28T03:07:47.693868+0000","last_became_active":"2023-11-28T03:07:47.693582+0000","last_became_peered":"2023-11-28T03:07:47.693582+0000","last_unstale":"2023-11-28T03:07:47.693868+0000","last_undegraded":"2023-11-28T03:07:47.693868+0000","last_fullsized":"2023-11-28T03:07:47.693868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.323264+0000","last_change":"2023-11-28T03:07:47.321871+0000","last_active":"2023-11-28T03:07:47.323264+0000","last_peered":"2023-11-28T03:07:47.323264+0000","last_clean":"2023-11-28T03:07:47.323264+0000","last_became_active":"2023-11-28T03:07:47.321625+0000","last_became_peered":"2023-11-28T03:07:47.321625+0000","last_unstale":"2023-11-28T03:07:47.323264+0000","last_undegraded":"2023-11-28T03:07:47.323264+0000","last_fullsized":"2023-11-28T03:07:47.323264+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690246+0000","last_change":"2023-11-28T03:07:47.690246+0000","last_active":"2023-11-28T03:07:47.690246+0000","last_peered":"2023-11-28T03:07:47.690246+0000","last_clean":"2023-11-28T03:07:47.690246+0000","last_became_active":"2023-11-28T03:07:47.690002+0000","last_became_peered":"2023-11-28T03:07:47.690002+0000","last_unstale":"2023-11-28T03:07:47.690246+0000","last_undegraded":"2023-11-28T03:07:47.690246+0000","last_fullsized":"2023-11-28T03:07:47.690246+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.678118+0000","last_change":"2023-11-28T03:07:47.320539+0000","last_active":"2023-11-28T03:07:47.678118+0000","last_peered":"2023-11-28T03:07:47.678118+0000","last_clean":"2023-11-28T03:07:47.678118+0000","last_became_active":"2023-11-28T03:07:47.320311+0000","last_became_peered":"2023-11-28T03:07:47.320311+0000","last_unstale":"2023-11-28T03:07:47.678118+0000","last_undegraded":"2023-11-28T03:07:47.678118+0000","last_fullsized":"2023-11-28T03:07:47.678118+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696543+0000","last_change":"2023-11-28T03:07:47.696543+0000","last_active":"2023-11-28T03:07:47.696543+0000","last_peered":"2023-11-28T03:07:47.696543+0000","last_clean":"2023-11-28T03:07:47.696543+0000","last_became_active":"2023-11-28T03:07:47.696374+0000","last_became_peered":"2023-11-28T03:07:47.696374+0000","last_unstale":"2023-11-28T03:07:47.696543+0000","last_undegraded":"2023-11-28T03:07:47.696543+0000","last_fullsized":"2023-11-28T03:07:47.696543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330010+0000","last_change":"2023-11-28T03:07:47.325913+0000","last_active":"2023-11-28T03:07:47.330010+0000","last_peered":"2023-11-28T03:07:47.330010+0000","last_clean":"2023-11-28T03:07:47.330010+0000","last_became_active":"2023-11-28T03:07:47.325639+0000","last_became_peered":"2023-11-28T03:07:47.325639+0000","last_unstale":"2023-11-28T03:07:47.330010+0000","last_undegraded":"2023-11-28T03:07:47.330010+0000","last_fullsized":"2023-11-28T03:07:47.330010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698335+0000","last_change":"2023-11-28T03:07:47.698335+0000","last_active":"2023-11-28T03:07:47.698335+0000","last_peered":"2023-11-28T03:07:47.698335+0000","last_clean":"2023-11-28T03:07:47.698335+0000","last_became_active":"2023-11-28T03:07:47.698053+0000","last_became_peered":"2023-11-28T03:07:47.698053+0000","last_unstale":"2023-11-28T03:07:47.698335+0000","last_undegraded":"2023-11-28T03:07:47.698335+0000","last_fullsized":"2023-11-28T03:07:47.698335+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330600+0000","last_change":"2023-11-28T03:07:47.328284+0000","last_active":"2023-11-28T03:07:47.330600+0000","last_peered":"2023-11-28T03:07:47.330600+0000","last_clean":"2023-11-28T03:07:47.330600+0000","last_became_active":"2023-11-28T03:07:47.327815+0000","last_became_peered":"2023-11-28T03:07:47.327815+0000","last_unstale":"2023-11-28T03:07:47.330600+0000","last_undegraded":"2023-11-28T03:07:47.330600+0000","last_fullsized":"2023-11-28T03:07:47.330600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.683118+0000","last_change":"2023-11-28T03:07:47.683118+0000","last_active":"2023-11-28T03:07:47.683118+0000","last_peered":"2023-11-28T03:07:47.683118+0000","last_clean":"2023-11-28T03:07:47.683118+0000","last_became_active":"2023-11-28T03:07:47.682285+0000","last_became_peered":"2023-11-28T03:07:47.682285+0000","last_unstale":"2023-11-28T03:07:47.683118+0000","last_undegraded":"2023-11-28T03:07:47.683118+0000","last_fullsized":"2023-11-28T03:07:47.683118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330534+0000","last_change":"2023-11-28T03:07:47.328131+0000","last_active":"2023-11-28T03:07:47.330534+0000","last_peered":"2023-11-28T03:07:47.330534+0000","last_clean":"2023-11-28T03:07:47.330534+0000","last_became_active":"2023-11-28T03:07:47.327258+0000","last_became_peered":"2023-11-28T03:07:47.327258+0000","last_unstale":"2023-11-28T03:07:47.330534+0000","last_undegraded":"2023-11-28T03:07:47.330534+0000","last_fullsized":"2023-11-28T03:07:47.330534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696069+0000","last_change":"2023-11-28T03:07:47.696069+0000","last_active":"2023-11-28T03:07:47.696069+0000","last_peered":"2023-11-28T03:07:47.696069+0000","last_clean":"2023-11-28T03:07:47.696069+0000","last_became_active":"2023-11-28T03:07:47.695756+0000","last_became_peered":"2023-11-28T03:07:47.695756+0000","last_unstale":"2023-11-28T03:07:47.696069+0000","last_undegraded":"2023-11-28T03:07:47.696069+0000","last_fullsized":"2023-11-28T03:07:47.696069+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330901+0000","last_change":"2023-11-28T03:07:47.328938+0000","last_active":"2023-11-28T03:07:47.330901+0000","last_peered":"2023-11-28T03:07:47.330901+0000","last_clean":"2023-11-28T03:07:47.330901+0000","last_became_active":"2023-11-28T03:07:47.328635+0000","last_became_peered":"2023-11-28T03:07:47.328635+0000","last_unstale":"2023-11-28T03:07:47.330901+0000","last_undegraded":"2023-11-28T03:07:47.330901+0000","last_fullsized":"2023-11-28T03:07:47.330901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695203+0000","last_change":"2023-11-28T03:07:47.695203+0000","last_active":"2023-11-28T03:07:47.695203+0000","last_peered":"2023-11-28T03:07:47.695203+0000","last_clean":"2023-11-28T03:07:47.695203+0000","last_became_active":"2023-11-28T03:07:47.694968+0000","last_became_peered":"2023-11-28T03:07:47.694968+0000","last_unstale":"2023-11-28T03:07:47.695203+0000","last_undegraded":"2023-11-28T03:07:47.695203+0000","last_fullsized":"2023-11-28T03:07:47.695203+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329513+0000","last_change":"2023-11-28T03:07:47.321952+0000","last_active":"2023-11-28T03:07:47.329513+0000","last_peered":"2023-11-28T03:07:47.329513+0000","last_clean":"2023-11-28T03:07:47.329513+0000","last_became_active":"2023-11-28T03:07:47.321674+0000","last_became_peered":"2023-11-28T03:07:47.321674+0000","last_unstale":"2023-11-28T03:07:47.329513+0000","last_undegraded":"2023-11-28T03:07:47.329513+0000","last_fullsized":"2023-11-28T03:07:47.329513+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696444+0000","last_change":"2023-11-28T03:07:47.696444+0000","last_active":"2023-11-28T03:07:47.696444+0000","last_peered":"2023-11-28T03:07:47.696444+0000","last_clean":"2023-11-28T03:07:47.696444+0000","last_became_active":"2023-11-28T03:07:47.696103+0000","last_became_peered":"2023-11-28T03:07:47.696103+0000","last_unstale":"2023-11-28T03:07:47.696444+0000","last_undegraded":"2023-11-28T03:07:47.696444+0000","last_fullsized":"2023-11-28T03:07:47.696444+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333760+0000","last_change":"2023-11-28T03:07:47.325631+0000","last_active":"2023-11-28T03:07:47.333760+0000","last_peered":"2023-11-28T03:07:47.333760+0000","last_clean":"2023-11-28T03:07:47.333760+0000","last_became_active":"2023-11-28T03:07:47.325357+0000","last_became_peered":"2023-11-28T03:07:47.325357+0000","last_unstale":"2023-11-28T03:07:47.333760+0000","last_undegraded":"2023-11-28T03:07:47.333760+0000","last_fullsized":"2023-11-28T03:07:47.333760+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696290+0000","last_change":"2023-11-28T03:07:47.696290+0000","last_active":"2023-11-28T03:07:47.696290+0000","last_peered":"2023-11-28T03:07:47.696290+0000","last_clean":"2023-11-28T03:07:47.696290+0000","last_became_active":"2023-11-28T03:07:47.696044+0000","last_became_peered":"2023-11-28T03:07:47.696044+0000","last_unstale":"2023-11-28T03:07:47.696290+0000","last_undegraded":"2023-11-28T03:07:47.696290+0000","last_fullsized":"2023-11-28T03:07:47.696290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329899+0000","last_change":"2023-11-28T03:07:47.325277+0000","last_active":"2023-11-28T03:07:47.329899+0000","last_peered":"2023-11-28T03:07:47.329899+0000","last_clean":"2023-11-28T03:07:47.329899+0000","last_became_active":"2023-11-28T03:07:47.324796+0000","last_became_peered":"2023-11-28T03:07:47.324796+0000","last_unstale":"2023-11-28T03:07:47.329899+0000","last_undegraded":"2023-11-28T03:07:47.329899+0000","last_fullsized":"2023-11-28T03:07:47.329899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701331+0000","last_change":"2023-11-28T03:07:47.701331+0000","last_active":"2023-11-28T03:07:47.701331+0000","last_peered":"2023-11-28T03:07:47.701331+0000","last_clean":"2023-11-28T03:07:47.701331+0000","last_became_active":"2023-11-28T03:07:47.701080+0000","last_became_peered":"2023-11-28T03:07:47.701080+0000","last_unstale":"2023-11-28T03:07:47.701331+0000","last_undegraded":"2023-11-28T03:07:47.701331+0000","last_fullsized":"2023-11-28T03:07:47.701331+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333579+0000","last_change":"2023-11-28T03:07:47.324380+0000","last_active":"2023-11-28T03:07:47.333579+0000","last_peered":"2023-11-28T03:07:47.333579+0000","last_clean":"2023-11-28T03:07:47.333579+0000","last_became_active":"2023-11-28T03:07:47.324124+0000","last_became_peered":"2023-11-28T03:07:47.324124+0000","last_unstale":"2023-11-28T03:07:47.333579+0000","last_undegraded":"2023-11-28T03:07:47.333579+0000","last_fullsized":"2023-11-28T03:07:47.333579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.700290+0000","last_change":"2023-11-28T03:07:47.700290+0000","last_active":"2023-11-28T03:07:47.700290+0000","last_peered":"2023-11-28T03:07:47.700290+0000","last_clean":"2023-11-28T03:07:47.700290+0000","last_became_active":"2023-11-28T03:07:47.699102+0000","last_became_peered":"2023-11-28T03:07:47.699102+0000","last_unstale":"2023-11-28T03:07:47.700290+0000","last_undegraded":"2023-11-28T03:07:47.700290+0000","last_fullsized":"2023-11-28T03:07:47.700290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329930+0000","last_change":"2023-11-28T03:07:47.325628+0000","last_active":"2023-11-28T03:07:47.329930+0000","last_peered":"2023-11-28T03:07:47.329930+0000","last_clean":"2023-11-28T03:07:47.329930+0000","last_became_active":"2023-11-28T03:07:47.325364+0000","last_became_peered":"2023-11-28T03:07:47.325364+0000","last_unstale":"2023-11-28T03:07:47.329930+0000","last_undegraded":"2023-11-28T03:07:47.329930+0000","last_fullsized":"2023-11-28T03:07:47.329930+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698153+0000","last_change":"2023-11-28T03:07:47.698153+0000","last_active":"2023-11-28T03:07:47.698153+0000","last_peered":"2023-11-28T03:07:47.698153+0000","last_clean":"2023-11-28T03:07:47.698153+0000","last_became_active":"2023-11-28T03:07:47.697726+0000","last_became_peered":"2023-11-28T03:07:47.697726+0000","last_unstale":"2023-11-28T03:07:47.698153+0000","last_undegraded":"2023-11-28T03:07:47.698153+0000","last_fullsized":"2023-11-28T03:07:47.698153+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330758+0000","last_change":"2023-11-28T03:07:47.326407+0000","last_active":"2023-11-28T03:07:47.330758+0000","last_peered":"2023-11-28T03:07:47.330758+0000","last_clean":"2023-11-28T03:07:47.330758+0000","last_became_active":"2023-11-28T03:07:47.326117+0000","last_became_peered":"2023-11-28T03:07:47.326117+0000","last_unstale":"2023-11-28T03:07:47.330758+0000","last_undegraded":"2023-11-28T03:07:47.330758+0000","last_fullsized":"2023-11-28T03:07:47.330758+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688412+0000","last_change":"2023-11-28T03:07:47.688412+0000","last_active":"2023-11-28T03:07:47.688412+0000","last_peered":"2023-11-28T03:07:47.688412+0000","last_clean":"2023-11-28T03:07:47.688412+0000","last_became_active":"2023-11-28T03:07:47.686169+0000","last_became_peered":"2023-11-28T03:07:47.686169+0000","last_unstale":"2023-11-28T03:07:47.688412+0000","last_undegraded":"2023-11-28T03:07:47.688412+0000","last_fullsized":"2023-11-28T03:07:47.688412+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326010+0000","last_change":"2023-11-28T03:07:47.322989+0000","last_active":"2023-11-28T03:07:47.326010+0000","last_peered":"2023-11-28T03:07:47.326010+0000","last_clean":"2023-11-28T03:07:47.326010+0000","last_became_active":"2023-11-28T03:07:47.322709+0000","last_became_peered":"2023-11-28T03:07:47.322709+0000","last_unstale":"2023-11-28T03:07:47.326010+0000","last_undegraded":"2023-11-28T03:07:47.326010+0000","last_fullsized":"2023-11-28T03:07:47.326010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.694142+0000","last_change":"2023-11-28T03:07:47.694142+0000","last_active":"2023-11-28T03:07:47.694142+0000","last_peered":"2023-11-28T03:07:47.694142+0000","last_clean":"2023-11-28T03:07:47.694142+0000","last_became_active":"2023-11-28T03:07:47.693837+0000","last_became_peered":"2023-11-28T03:07:47.693837+0000","last_unstale":"2023-11-28T03:07:47.694142+0000","last_undegraded":"2023-11-28T03:07:47.694142+0000","last_fullsized":"2023-11-28T03:07:47.694142+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.675403+0000","last_change":"2023-11-28T03:07:47.319952+0000","last_active":"2023-11-28T03:07:47.675403+0000","last_peered":"2023-11-28T03:07:47.675403+0000","last_clean":"2023-11-28T03:07:47.675403+0000","last_became_active":"2023-11-28T03:07:47.319744+0000","last_became_peered":"2023-11-28T03:07:47.319744+0000","last_unstale":"2023-11-28T03:07:47.675403+0000","last_undegraded":"2023-11-28T03:07:47.675403+0000","last_fullsized":"2023-11-28T03:07:47.675403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701394+0000","last_change":"2023-11-28T03:07:47.701394+0000","last_active":"2023-11-28T03:07:47.701394+0000","last_peered":"2023-11-28T03:07:47.701394+0000","last_clean":"2023-11-28T03:07:47.701394+0000","last_became_active":"2023-11-28T03:07:47.701128+0000","last_became_peered":"2023-11-28T03:07:47.701128+0000","last_unstale":"2023-11-28T03:07:47.701394+0000","last_undegraded":"2023-11-28T03:07:47.701394+0000","last_fullsized":"2023-11-28T03:07:47.701394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685152+0000","last_change":"2023-11-28T03:07:47.326855+0000","last_active":"2023-11-28T03:07:47.685152+0000","last_peered":"2023-11-28T03:07:47.685152+0000","last_clean":"2023-11-28T03:07:47.685152+0000","last_became_active":"2023-11-28T03:07:47.326562+0000","last_became_peered":"2023-11-28T03:07:47.326562+0000","last_unstale":"2023-11-28T03:07:47.685152+0000","last_undegraded":"2023-11-28T03:07:47.685152+0000","last_fullsized":"2023-11-28T03:07:47.685152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690148+0000","last_change":"2023-11-28T03:07:47.690148+0000","last_active":"2023-11-28T03:07:47.690148+0000","last_peered":"2023-11-28T03:07:47.690148+0000","last_clean":"2023-11-28T03:07:47.690148+0000","last_became_active":"2023-11-28T03:07:47.689761+0000","last_became_peered":"2023-11-28T03:07:47.689761+0000","last_unstale":"2023-11-28T03:07:47.690148+0000","last_undegraded":"2023-11-28T03:07:47.690148+0000","last_fullsized":"2023-11-28T03:07:47.690148+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696623+0000","last_change":"2023-11-28T03:07:47.328540+0000","last_active":"2023-11-28T03:07:47.696623+0000","last_peered":"2023-11-28T03:07:47.696623+0000","last_clean":"2023-11-28T03:07:47.696623+0000","last_became_active":"2023-11-28T03:07:47.328229+0000","last_became_peered":"2023-11-28T03:07:47.328229+0000","last_unstale":"2023-11-28T03:07:47.696623+0000","last_undegraded":"2023-11-28T03:07:47.696623+0000","last_fullsized":"2023-11-28T03:07:47.696623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696146+0000","last_change":"2023-11-28T03:07:47.696146+0000","last_active":"2023-11-28T03:07:47.696146+0000","last_peered":"2023-11-28T03:07:47.696146+0000","last_clean":"2023-11-28T03:07:47.696146+0000","last_became_active":"2023-11-28T03:07:47.695865+0000","last_became_peered":"2023-11-28T03:07:47.695865+0000","last_unstale":"2023-11-28T03:07:47.696146+0000","last_undegraded":"2023-11-28T03:07:47.696146+0000","last_fullsized":"2023-11-28T03:07:47.696146+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326108+0000","last_change":"2023-11-28T03:07:47.322817+0000","last_active":"2023-11-28T03:07:47.326108+0000","last_peered":"2023-11-28T03:07:47.326108+0000","last_clean":"2023-11-28T03:07:47.326108+0000","last_became_active":"2023-11-28T03:07:47.319973+0000","last_became_peered":"2023-11-28T03:07:47.319973+0000","last_unstale":"2023-11-28T03:07:47.326108+0000","last_undegraded":"2023-11-28T03:07:47.326108+0000","last_fullsized":"2023-11-28T03:07:47.326108+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690285+0000","last_change":"2023-11-28T03:07:47.690285+0000","last_active":"2023-11-28T03:07:47.690285+0000","last_peered":"2023-11-28T03:07:47.690285+0000","last_clean":"2023-11-28T03:07:47.690285+0000","last_became_active":"2023-11-28T03:07:47.690005+0000","last_became_peered":"2023-11-28T03:07:47.690005+0000","last_unstale":"2023-11-28T03:07:47.690285+0000","last_undegraded":"2023-11-28T03:07:47.690285+0000","last_fullsized":"2023-11-28T03:07:47.690285+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330880+0000","last_change":"2023-11-28T03:07:47.328937+0000","last_active":"2023-11-28T03:07:47.330880+0000","last_peered":"2023-11-28T03:07:47.330880+0000","last_clean":"2023-11-28T03:07:47.330880+0000","last_became_active":"2023-11-28T03:07:47.328627+0000","last_became_peered":"2023-11-28T03:07:47.328627+0000","last_unstale":"2023-11-28T03:07:47.330880+0000","last_undegraded":"2023-11-28T03:07:47.330880+0000","last_fullsized":"2023-11-28T03:07:47.330880+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692839+0000","last_change":"2023-11-28T03:07:47.692839+0000","last_active":"2023-11-28T03:07:47.692839+0000","last_peered":"2023-11-28T03:07:47.692839+0000","last_clean":"2023-11-28T03:07:47.692839+0000","last_became_active":"2023-11-28T03:07:47.692456+0000","last_became_peered":"2023-11-28T03:07:47.692456+0000","last_unstale":"2023-11-28T03:07:47.692839+0000","last_undegraded":"2023-11-28T03:07:47.692839+0000","last_fullsized":"2023-11-28T03:07:47.692839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336075+0000","last_change":"2023-11-28T03:07:47.331231+0000","last_active":"2023-11-28T03:07:47.336075+0000","last_peered":"2023-11-28T03:07:47.336075+0000","last_clean":"2023-11-28T03:07:47.336075+0000","last_became_active":"2023-11-28T03:07:47.330926+0000","last_became_peered":"2023-11-28T03:07:47.330926+0000","last_unstale":"2023-11-28T03:07:47.336075+0000","last_undegraded":"2023-11-28T03:07:47.336075+0000","last_fullsized":"2023-11-28T03:07:47.336075+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.684747+0000","last_change":"2023-11-28T03:07:47.684747+0000","last_active":"2023-11-28T03:07:47.684747+0000","last_peered":"2023-11-28T03:07:47.684747+0000","last_clean":"2023-11-28T03:07:47.684747+0000","last_became_active":"2023-11-28T03:07:47.684471+0000","last_became_peered":"2023-11-28T03:07:47.684471+0000","last_unstale":"2023-11-28T03:07:47.684747+0000","last_undegraded":"2023-11-28T03:07:47.684747+0000","last_fullsized":"2023-11-28T03:07:47.684747+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327014+0000","last_change":"2023-11-28T03:07:47.324387+0000","last_active":"2023-11-28T03:07:47.327014+0000","last_peered":"2023-11-28T03:07:47.327014+0000","last_clean":"2023-11-28T03:07:47.327014+0000","last_became_active":"2023-11-28T03:07:47.324039+0000","last_became_peered":"2023-11-28T03:07:47.324039+0000","last_unstale":"2023-11-28T03:07:47.327014+0000","last_undegraded":"2023-11-28T03:07:47.327014+0000","last_fullsized":"2023-11-28T03:07:47.327014+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687116+0000","last_change":"2023-11-28T03:07:47.687116+0000","last_active":"2023-11-28T03:07:47.687116+0000","last_peered":"2023-11-28T03:07:47.687116+0000","last_clean":"2023-11-28T03:07:47.687116+0000","last_became_active":"2023-11-28T03:07:47.686736+0000","last_became_peered":"2023-11-28T03:07:47.686736+0000","last_unstale":"2023-11-28T03:07:47.687116+0000","last_undegraded":"2023-11-28T03:07:47.687116+0000","last_fullsized":"2023-11-28T03:07:47.687116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324620+0000","last_change":"2023-11-28T03:07:47.316401+0000","last_active":"2023-11-28T03:07:47.324620+0000","last_peered":"2023-11-28T03:07:47.324620+0000","last_clean":"2023-11-28T03:07:47.324620+0000","last_became_active":"2023-11-28T03:07:47.315681+0000","last_became_peered":"2023-11-28T03:07:47.315681+0000","last_unstale":"2023-11-28T03:07:47.324620+0000","last_undegraded":"2023-11-28T03:07:47.324620+0000","last_fullsized":"2023-11-28T03:07:47.324620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696655+0000","last_change":"2023-11-28T03:07:47.696655+0000","last_active":"2023-11-28T03:07:47.696655+0000","last_peered":"2023-11-28T03:07:47.696655+0000","last_clean":"2023-11-28T03:07:47.696655+0000","last_became_active":"2023-11-28T03:07:47.695841+0000","last_became_peered":"2023-11-28T03:07:47.695841+0000","last_unstale":"2023-11-28T03:07:47.696655+0000","last_undegraded":"2023-11-28T03:07:47.696655+0000","last_fullsized":"2023-11-28T03:07:47.696655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330320+0000","last_change":"2023-11-28T03:07:47.326305+0000","last_active":"2023-11-28T03:07:47.330320+0000","last_peered":"2023-11-28T03:07:47.330320+0000","last_clean":"2023-11-28T03:07:47.330320+0000","last_became_active":"2023-11-28T03:07:47.326021+0000","last_became_peered":"2023-11-28T03:07:47.326021+0000","last_unstale":"2023-11-28T03:07:47.330320+0000","last_undegraded":"2023-11-28T03:07:47.330320+0000","last_fullsized":"2023-11-28T03:07:47.330320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699740+0000","last_change":"2023-11-28T03:07:47.699740+0000","last_active":"2023-11-28T03:07:47.699740+0000","last_peered":"2023-11-28T03:07:47.699740+0000","last_clean":"2023-11-28T03:07:47.699740+0000","last_became_active":"2023-11-28T03:07:47.699374+0000","last_became_peered":"2023-11-28T03:07:47.699374+0000","last_unstale":"2023-11-28T03:07:47.699740+0000","last_undegraded":"2023-11-28T03:07:47.699740+0000","last_fullsized":"2023-11-28T03:07:47.699740+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329535+0000","last_change":"2023-11-28T03:07:47.322477+0000","last_active":"2023-11-28T03:07:47.329535+0000","last_peered":"2023-11-28T03:07:47.329535+0000","last_clean":"2023-11-28T03:07:47.329535+0000","last_became_active":"2023-11-28T03:07:47.322220+0000","last_became_peered":"2023-11-28T03:07:47.322220+0000","last_unstale":"2023-11-28T03:07:47.329535+0000","last_undegraded":"2023-11-28T03:07:47.329535+0000","last_fullsized":"2023-11-28T03:07:47.329535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693094+0000","last_change":"2023-11-28T03:07:47.693094+0000","last_active":"2023-11-28T03:07:47.693094+0000","last_peered":"2023-11-28T03:07:47.693094+0000","last_clean":"2023-11-28T03:07:47.693094+0000","last_became_active":"2023-11-28T03:07:47.692826+0000","last_became_peered":"2023-11-28T03:07:47.692826+0000","last_unstale":"2023-11-28T03:07:47.693094+0000","last_undegraded":"2023-11-28T03:07:47.693094+0000","last_fullsized":"2023-11-28T03:07:47.693094+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"95'3","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695227+0000","last_change":"2023-11-28T03:07:47.322767+0000","last_active":"2023-11-28T03:07:47.695227+0000","last_peered":"2023-11-28T03:07:47.695227+0000","last_clean":"2023-11-28T03:07:47.695227+0000","last_became_active":"2023-11-28T03:07:47.322282+0000","last_became_peered":"2023-11-28T03:07:47.322282+0000","last_unstale":"2023-11-28T03:07:47.695227+0000","last_undegraded":"2023-11-28T03:07:47.695227+0000","last_fullsized":"2023-11-28T03:07:47.695227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698607+0000","last_change":"2023-11-28T03:07:47.698607+0000","last_active":"2023-11-28T03:07:47.698607+0000","last_peered":"2023-11-28T03:07:47.698607+0000","last_clean":"2023-11-28T03:07:47.698607+0000","last_became_active":"2023-11-28T03:07:47.697698+0000","last_became_peered":"2023-11-28T03:07:47.697698+0000","last_unstale":"2023-11-28T03:07:47.698607+0000","last_undegraded":"2023-11-28T03:07:47.698607+0000","last_fullsized":"2023-11-28T03:07:47.698607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.694731+0000","last_change":"2023-11-28T03:07:47.327445+0000","last_active":"2023-11-28T03:07:47.694731+0000","last_peered":"2023-11-28T03:07:47.694731+0000","last_clean":"2023-11-28T03:07:47.694731+0000","last_became_active":"2023-11-28T03:07:47.327157+0000","last_became_peered":"2023-11-28T03:07:47.327157+0000","last_unstale":"2023-11-28T03:07:47.694731+0000","last_undegraded":"2023-11-28T03:07:47.694731+0000","last_fullsized":"2023-11-28T03:07:47.694731+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698416+0000","last_change":"2023-11-28T03:07:47.698416+0000","last_active":"2023-11-28T03:07:47.698416+0000","last_peered":"2023-11-28T03:07:47.698416+0000","last_clean":"2023-11-28T03:07:47.698416+0000","last_became_active":"2023-11-28T03:07:47.697773+0000","last_became_peered":"2023-11-28T03:07:47.697773+0000","last_unstale":"2023-11-28T03:07:47.698416+0000","last_undegraded":"2023-11-28T03:07:47.698416+0000","last_fullsized":"2023-11-28T03:07:47.698416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685350+0000","last_change":"2023-11-28T03:07:47.320276+0000","last_active":"2023-11-28T03:07:47.685350+0000","last_peered":"2023-11-28T03:07:47.685350+0000","last_clean":"2023-11-28T03:07:47.685350+0000","last_became_active":"2023-11-28T03:07:47.320004+0000","last_became_peered":"2023-11-28T03:07:47.320004+0000","last_unstale":"2023-11-28T03:07:47.685350+0000","last_undegraded":"2023-11-28T03:07:47.685350+0000","last_fullsized":"2023-11-28T03:07:47.685350+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692503+0000","last_change":"2023-11-28T03:07:47.692503+0000","last_active":"2023-11-28T03:07:47.692503+0000","last_peered":"2023-11-28T03:07:47.692503+0000","last_clean":"2023-11-28T03:07:47.692503+0000","last_became_active":"2023-11-28T03:07:47.692205+0000","last_became_peered":"2023-11-28T03:07:47.692205+0000","last_unstale":"2023-11-28T03:07:47.692503+0000","last_undegraded":"2023-11-28T03:07:47.692503+0000","last_fullsized":"2023-11-28T03:07:47.692503+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336212+0000","last_change":"2023-11-28T03:07:47.331587+0000","last_active":"2023-11-28T03:07:47.336212+0000","last_peered":"2023-11-28T03:07:47.336212+0000","last_clean":"2023-11-28T03:07:47.336212+0000","last_became_active":"2023-11-28T03:07:47.331293+0000","last_became_peered":"2023-11-28T03:07:47.331293+0000","last_unstale":"2023-11-28T03:07:47.336212+0000","last_undegraded":"2023-11-28T03:07:47.336212+0000","last_fullsized":"2023-11-28T03:07:47.336212+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681629+0000","last_change":"2023-11-28T03:07:47.681629+0000","last_active":"2023-11-28T03:07:47.681629+0000","last_peered":"2023-11-28T03:07:47.681629+0000","last_clean":"2023-11-28T03:07:47.681629+0000","last_became_active":"2023-11-28T03:07:47.681148+0000","last_became_peered":"2023-11-28T03:07:47.681148+0000","last_unstale":"2023-11-28T03:07:47.681629+0000","last_undegraded":"2023-11-28T03:07:47.681629+0000","last_fullsized":"2023-11-28T03:07:47.681629+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324743+0000","last_change":"2023-11-28T03:07:47.319585+0000","last_active":"2023-11-28T03:07:47.324743+0000","last_peered":"2023-11-28T03:07:47.324743+0000","last_clean":"2023-11-28T03:07:47.324743+0000","last_became_active":"2023-11-28T03:07:47.319302+0000","last_became_peered":"2023-11-28T03:07:47.319302+0000","last_unstale":"2023-11-28T03:07:47.324743+0000","last_undegraded":"2023-11-28T03:07:47.324743+0000","last_fullsized":"2023-11-28T03:07:47.324743+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686244+0000","last_change":"2023-11-28T03:07:47.686244+0000","last_active":"2023-11-28T03:07:47.686244+0000","last_peered":"2023-11-28T03:07:47.686244+0000","last_clean":"2023-11-28T03:07:47.686244+0000","last_became_active":"2023-11-28T03:07:47.685946+0000","last_became_peered":"2023-11-28T03:07:47.685946+0000","last_unstale":"2023-11-28T03:07:47.686244+0000","last_undegraded":"2023-11-28T03:07:47.686244+0000","last_fullsized":"2023-11-28T03:07:47.686244+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695767+0000","last_change":"2023-11-28T03:07:47.321334+0000","last_active":"2023-11-28T03:07:47.695767+0000","last_peered":"2023-11-28T03:07:47.695767+0000","last_clean":"2023-11-28T03:07:47.695767+0000","last_became_active":"2023-11-28T03:07:47.321043+0000","last_became_peered":"2023-11-28T03:07:47.321043+0000","last_unstale":"2023-11-28T03:07:47.695767+0000","last_undegraded":"2023-11-28T03:07:47.695767+0000","last_fullsized":"2023-11-28T03:07:47.695767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.682764+0000","last_change":"2023-11-28T03:07:47.682764+0000","last_active":"2023-11-28T03:07:47.682764+0000","last_peered":"2023-11-28T03:07:47.682764+0000","last_clean":"2023-11-28T03:07:47.682764+0000","last_became_active":"2023-11-28T03:07:47.682093+0000","last_became_peered":"2023-11-28T03:07:47.682093+0000","last_unstale":"2023-11-28T03:07:47.682764+0000","last_undegraded":"2023-11-28T03:07:47.682764+0000","last_fullsized":"2023-11-28T03:07:47.682764+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.697796+0000","last_change":"2023-11-28T03:07:47.324776+0000","last_active":"2023-11-28T03:07:47.697796+0000","last_peered":"2023-11-28T03:07:47.697796+0000","last_clean":"2023-11-28T03:07:47.697796+0000","last_became_active":"2023-11-28T03:07:47.324494+0000","last_became_peered":"2023-11-28T03:07:47.324494+0000","last_unstale":"2023-11-28T03:07:47.697796+0000","last_undegraded":"2023-11-28T03:07:47.697796+0000","last_fullsized":"2023-11-28T03:07:47.697796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686340+0000","last_change":"2023-11-28T03:07:47.686340+0000","last_active":"2023-11-28T03:07:47.686340+0000","last_peered":"2023-11-28T03:07:47.686340+0000","last_clean":"2023-11-28T03:07:47.686340+0000","last_became_active":"2023-11-28T03:07:47.684496+0000","last_became_peered":"2023-11-28T03:07:47.684496+0000","last_unstale":"2023-11-28T03:07:47.686340+0000","last_undegraded":"2023-11-28T03:07:47.686340+0000","last_fullsized":"2023-11-28T03:07:47.686340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320392+0000","last_change":"2023-11-28T03:07:47.317497+0000","last_active":"2023-11-28T03:07:47.320392+0000","last_peered":"2023-11-28T03:07:47.320392+0000","last_clean":"2023-11-28T03:07:47.320392+0000","last_became_active":"2023-11-28T03:07:47.317147+0000","last_became_peered":"2023-11-28T03:07:47.317147+0000","last_unstale":"2023-11-28T03:07:47.320392+0000","last_undegraded":"2023-11-28T03:07:47.320392+0000","last_fullsized":"2023-11-28T03:07:47.320392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686190+0000","last_change":"2023-11-28T03:07:47.686190+0000","last_active":"2023-11-28T03:07:47.686190+0000","last_peered":"2023-11-28T03:07:47.686190+0000","last_clean":"2023-11-28T03:07:47.686190+0000","last_became_active":"2023-11-28T03:07:47.685887+0000","last_became_peered":"2023-11-28T03:07:47.685887+0000","last_unstale":"2023-11-28T03:07:47.686190+0000","last_undegraded":"2023-11-28T03:07:47.686190+0000","last_fullsized":"2023-11-28T03:07:47.686190+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330435+0000","last_change":"2023-11-28T03:07:47.327833+0000","last_active":"2023-11-28T03:07:47.330435+0000","last_peered":"2023-11-28T03:07:47.330435+0000","last_clean":"2023-11-28T03:07:47.330435+0000","last_became_active":"2023-11-28T03:07:47.326683+0000","last_became_peered":"2023-11-28T03:07:47.326683+0000","last_unstale":"2023-11-28T03:07:47.330435+0000","last_undegraded":"2023-11-28T03:07:47.330435+0000","last_fullsized":"2023-11-28T03:07:47.330435+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693950+0000","last_change":"2023-11-28T03:07:47.693950+0000","last_active":"2023-11-28T03:07:47.693950+0000","last_peered":"2023-11-28T03:07:47.693950+0000","last_clean":"2023-11-28T03:07:47.693950+0000","last_became_active":"2023-11-28T03:07:47.693766+0000","last_became_peered":"2023-11-28T03:07:47.693766+0000","last_unstale":"2023-11-28T03:07:47.693950+0000","last_undegraded":"2023-11-28T03:07:47.693950+0000","last_fullsized":"2023-11-28T03:07:47.693950+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330407+0000","last_change":"2023-11-28T03:07:47.327903+0000","last_active":"2023-11-28T03:07:47.330407+0000","last_peered":"2023-11-28T03:07:47.330407+0000","last_clean":"2023-11-28T03:07:47.330407+0000","last_became_active":"2023-11-28T03:07:47.327616+0000","last_became_peered":"2023-11-28T03:07:47.327616+0000","last_unstale":"2023-11-28T03:07:47.330407+0000","last_undegraded":"2023-11-28T03:07:47.330407+0000","last_fullsized":"2023-11-28T03:07:47.330407+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687101+0000","last_change":"2023-11-28T03:07:47.687101+0000","last_active":"2023-11-28T03:07:47.687101+0000","last_peered":"2023-11-28T03:07:47.687101+0000","last_clean":"2023-11-28T03:07:47.687101+0000","last_became_active":"2023-11-28T03:07:47.683927+0000","last_became_peered":"2023-11-28T03:07:47.683927+0000","last_unstale":"2023-11-28T03:07:47.687101+0000","last_undegraded":"2023-11-28T03:07:47.687101+0000","last_fullsized":"2023-11-28T03:07:47.687101+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325839+0000","last_change":"2023-11-28T03:07:47.321990+0000","last_active":"2023-11-28T03:07:47.325839+0000","last_peered":"2023-11-28T03:07:47.325839+0000","last_clean":"2023-11-28T03:07:47.325839+0000","last_became_active":"2023-11-28T03:07:47.321622+0000","last_became_peered":"2023-11-28T03:07:47.321622+0000","last_unstale":"2023-11-28T03:07:47.325839+0000","last_undegraded":"2023-11-28T03:07:47.325839+0000","last_fullsized":"2023-11-28T03:07:47.325839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681098+0000","last_change":"2023-11-28T03:07:47.681098+0000","last_active":"2023-11-28T03:07:47.681098+0000","last_peered":"2023-11-28T03:07:47.681098+0000","last_clean":"2023-11-28T03:07:47.681098+0000","last_became_active":"2023-11-28T03:07:47.680736+0000","last_became_peered":"2023-11-28T03:07:47.680736+0000","last_unstale":"2023-11-28T03:07:47.681098+0000","last_undegraded":"2023-11-28T03:07:47.681098+0000","last_fullsized":"2023-11-28T03:07:47.681098+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325337+0000","last_change":"2023-11-28T03:07:47.319993+0000","last_active":"2023-11-28T03:07:47.325337+0000","last_peered":"2023-11-28T03:07:47.325337+0000","last_clean":"2023-11-28T03:07:47.325337+0000","last_became_active":"2023-11-28T03:07:47.319678+0000","last_became_peered":"2023-11-28T03:07:47.319678+0000","last_unstale":"2023-11-28T03:07:47.325337+0000","last_undegraded":"2023-11-28T03:07:47.325337+0000","last_fullsized":"2023-11-28T03:07:47.325337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693342+0000","last_change":"2023-11-28T03:07:47.693342+0000","last_active":"2023-11-28T03:07:47.693342+0000","last_peered":"2023-11-28T03:07:47.693342+0000","last_clean":"2023-11-28T03:07:47.693342+0000","last_became_active":"2023-11-28T03:07:47.693057+0000","last_became_peered":"2023-11-28T03:07:47.693057+0000","last_unstale":"2023-11-28T03:07:47.693342+0000","last_undegraded":"2023-11-28T03:07:47.693342+0000","last_fullsized":"2023-11-28T03:07:47.693342+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326639+0000","last_change":"2023-11-28T03:07:47.324185+0000","last_active":"2023-11-28T03:07:47.326639+0000","last_peered":"2023-11-28T03:07:47.326639+0000","last_clean":"2023-11-28T03:07:47.326639+0000","last_became_active":"2023-11-28T03:07:47.323376+0000","last_became_peered":"2023-11-28T03:07:47.323376+0000","last_unstale":"2023-11-28T03:07:47.326639+0000","last_undegraded":"2023-11-28T03:07:47.326639+0000","last_fullsized":"2023-11-28T03:07:47.326639+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692545+0000","last_change":"2023-11-28T03:07:47.692545+0000","last_active":"2023-11-28T03:07:47.692545+0000","last_peered":"2023-11-28T03:07:47.692545+0000","last_clean":"2023-11-28T03:07:47.692545+0000","last_became_active":"2023-11-28T03:07:47.692281+0000","last_became_peered":"2023-11-28T03:07:47.692281+0000","last_unstale":"2023-11-28T03:07:47.692545+0000","last_undegraded":"2023-11-28T03:07:47.692545+0000","last_fullsized":"2023-11-28T03:07:47.692545+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.323367+0000","last_change":"2023-11-28T03:07:47.321117+0000","last_active":"2023-11-28T03:07:47.323367+0000","last_peered":"2023-11-28T03:07:47.323367+0000","last_clean":"2023-11-28T03:07:47.323367+0000","last_became_active":"2023-11-28T03:07:47.320851+0000","last_became_peered":"2023-11-28T03:07:47.320851+0000","last_unstale":"2023-11-28T03:07:47.323367+0000","last_undegraded":"2023-11-28T03:07:47.323367+0000","last_fullsized":"2023-11-28T03:07:47.323367+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688168+0000","last_change":"2023-11-28T03:07:47.688168+0000","last_active":"2023-11-28T03:07:47.688168+0000","last_peered":"2023-11-28T03:07:47.688168+0000","last_clean":"2023-11-28T03:07:47.688168+0000","last_became_active":"2023-11-28T03:07:47.687905+0000","last_became_peered":"2023-11-28T03:07:47.687905+0000","last_unstale":"2023-11-28T03:07:47.688168+0000","last_undegraded":"2023-11-28T03:07:47.688168+0000","last_fullsized":"2023-11-28T03:07:47.688168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327396+0000","last_change":"2023-11-28T03:07:47.316377+0000","last_active":"2023-11-28T03:07:47.327396+0000","last_peered":"2023-11-28T03:07:47.327396+0000","last_clean":"2023-11-28T03:07:47.327396+0000","last_became_active":"2023-11-28T03:07:47.316098+0000","last_became_peered":"2023-11-28T03:07:47.316098+0000","last_unstale":"2023-11-28T03:07:47.327396+0000","last_undegraded":"2023-11-28T03:07:47.327396+0000","last_fullsized":"2023-11-28T03:07:47.327396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.317733+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:07:47.317733+0000","last_peered":"2023-11-28T03:07:47.317733+0000","last_clean":"2023-11-28T03:07:47.317733+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:07:47.317733+0000","last_undegraded":"2023-11-28T03:07:47.317733+0000","last_fullsized":"2023-11-28T03:07:47.317733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701326+0000","last_change":"2023-11-28T03:07:47.701326+0000","last_active":"2023-11-28T03:07:47.701326+0000","last_peered":"2023-11-28T03:07:47.701326+0000","last_clean":"2023-11-28T03:07:47.701326+0000","last_became_active":"2023-11-28T03:07:47.701065+0000","last_became_peered":"2023-11-28T03:07:47.701065+0000","last_unstale":"2023-11-28T03:07:47.701326+0000","last_undegraded":"2023-11-28T03:07:47.701326+0000","last_fullsized":"2023-11-28T03:07:47.701326+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.680884+0000","last_change":"2023-11-28T03:07:47.319788+0000","last_active":"2023-11-28T03:07:47.680884+0000","last_peered":"2023-11-28T03:07:47.680884+0000","last_clean":"2023-11-28T03:07:47.680884+0000","last_became_active":"2023-11-28T03:07:47.318830+0000","last_became_peered":"2023-11-28T03:07:47.318830+0000","last_unstale":"2023-11-28T03:07:47.680884+0000","last_undegraded":"2023-11-28T03:07:47.680884+0000","last_fullsized":"2023-11-28T03:07:47.680884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.315933+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:07:47.315933+0000","last_peered":"2023-11-28T03:07:47.315933+0000","last_clean":"2023-11-28T03:07:47.315933+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:07:47.315933+0000","last_undegraded":"2023-11-28T03:07:47.315933+0000","last_fullsized":"2023-11-28T03:07:47.315933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698646+0000","last_change":"2023-11-28T03:07:47.698646+0000","last_active":"2023-11-28T03:07:47.698646+0000","last_peered":"2023-11-28T03:07:47.698646+0000","last_clean":"2023-11-28T03:07:47.698646+0000","last_became_active":"2023-11-28T03:07:47.698365+0000","last_became_peered":"2023-11-28T03:07:47.698365+0000","last_unstale":"2023-11-28T03:07:47.698646+0000","last_undegraded":"2023-11-28T03:07:47.698646+0000","last_fullsized":"2023-11-28T03:07:47.698646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333781+0000","last_change":"2023-11-28T03:07:47.325927+0000","last_active":"2023-11-28T03:07:47.333781+0000","last_peered":"2023-11-28T03:07:47.333781+0000","last_clean":"2023-11-28T03:07:47.333781+0000","last_became_active":"2023-11-28T03:07:47.325640+0000","last_became_peered":"2023-11-28T03:07:47.325640+0000","last_unstale":"2023-11-28T03:07:47.333781+0000","last_undegraded":"2023-11-28T03:07:47.333781+0000","last_fullsized":"2023-11-28T03:07:47.333781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.309649+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:07:47.309649+0000","last_peered":"2023-11-28T03:07:47.309649+0000","last_clean":"2023-11-28T03:07:47.309649+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:07:47.309649+0000","last_undegraded":"2023-11-28T03:07:47.309649+0000","last_fullsized":"2023-11-28T03:07:47.309649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695982+0000","last_change":"2023-11-28T03:07:47.695982+0000","last_active":"2023-11-28T03:07:47.695982+0000","last_peered":"2023-11-28T03:07:47.695982+0000","last_clean":"2023-11-28T03:07:47.695982+0000","last_became_active":"2023-11-28T03:07:47.695674+0000","last_became_peered":"2023-11-28T03:07:47.695674+0000","last_unstale":"2023-11-28T03:07:47.695982+0000","last_undegraded":"2023-11-28T03:07:47.695982+0000","last_fullsized":"2023-11-28T03:07:47.695982+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333790+0000","last_change":"2023-11-28T03:07:47.325980+0000","last_active":"2023-11-28T03:07:47.333790+0000","last_peered":"2023-11-28T03:07:47.333790+0000","last_clean":"2023-11-28T03:07:47.333790+0000","last_became_active":"2023-11-28T03:07:47.325630+0000","last_became_peered":"2023-11-28T03:07:47.325630+0000","last_unstale":"2023-11-28T03:07:47.333790+0000","last_undegraded":"2023-11-28T03:07:47.333790+0000","last_fullsized":"2023-11-28T03:07:47.333790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.312218+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:07:47.312218+0000","last_peered":"2023-11-28T03:07:47.312218+0000","last_clean":"2023-11-28T03:07:47.312218+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:07:47.312218+0000","last_undegraded":"2023-11-28T03:07:47.312218+0000","last_fullsized":"2023-11-28T03:07:47.312218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320245+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:07:47.320245+0000","last_peered":"2023-11-28T03:07:47.320245+0000","last_clean":"2023-11-28T03:07:47.320245+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:07:47.320245+0000","last_undegraded":"2023-11-28T03:07:47.320245+0000","last_fullsized":"2023-11-28T03:07:47.320245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689443+0000","last_change":"2023-11-28T03:07:47.689443+0000","last_active":"2023-11-28T03:07:47.689443+0000","last_peered":"2023-11-28T03:07:47.689443+0000","last_clean":"2023-11-28T03:07:47.689443+0000","last_became_active":"2023-11-28T03:07:47.689185+0000","last_became_peered":"2023-11-28T03:07:47.689185+0000","last_unstale":"2023-11-28T03:07:47.689443+0000","last_undegraded":"2023-11-28T03:07:47.689443+0000","last_fullsized":"2023-11-28T03:07:47.689443+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333823+0000","last_change":"2023-11-28T03:07:47.327511+0000","last_active":"2023-11-28T03:07:47.333823+0000","last_peered":"2023-11-28T03:07:47.333823+0000","last_clean":"2023-11-28T03:07:47.333823+0000","last_became_active":"2023-11-28T03:07:47.325928+0000","last_became_peered":"2023-11-28T03:07:47.325928+0000","last_unstale":"2023-11-28T03:07:47.333823+0000","last_undegraded":"2023-11-28T03:07:47.333823+0000","last_fullsized":"2023-11-28T03:07:47.333823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.318995+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:07:47.318995+0000","last_peered":"2023-11-28T03:07:47.318995+0000","last_clean":"2023-11-28T03:07:47.318995+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:07:47.318995+0000","last_undegraded":"2023-11-28T03:07:47.318995+0000","last_fullsized":"2023-11-28T03:07:47.318995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691626+0000","last_change":"2023-11-28T03:07:47.691626+0000","last_active":"2023-11-28T03:07:47.691626+0000","last_peered":"2023-11-28T03:07:47.691626+0000","last_clean":"2023-11-28T03:07:47.691626+0000","last_became_active":"2023-11-28T03:07:47.691354+0000","last_became_peered":"2023-11-28T03:07:47.691354+0000","last_unstale":"2023-11-28T03:07:47.691626+0000","last_undegraded":"2023-11-28T03:07:47.691626+0000","last_fullsized":"2023-11-28T03:07:47.691626+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327923+0000","last_change":"2023-11-28T03:07:47.325183+0000","last_active":"2023-11-28T03:07:47.327923+0000","last_peered":"2023-11-28T03:07:47.327923+0000","last_clean":"2023-11-28T03:07:47.327923+0000","last_became_active":"2023-11-28T03:07:47.324891+0000","last_became_peered":"2023-11-28T03:07:47.324891+0000","last_unstale":"2023-11-28T03:07:47.327923+0000","last_undegraded":"2023-11-28T03:07:47.327923+0000","last_fullsized":"2023-11-28T03:07:47.327923+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.319402+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:07:47.319402+0000","last_peered":"2023-11-28T03:07:47.319402+0000","last_clean":"2023-11-28T03:07:47.319402+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:07:47.319402+0000","last_undegraded":"2023-11-28T03:07:47.319402+0000","last_fullsized":"2023-11-28T03:07:47.319402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691585+0000","last_change":"2023-11-28T03:07:47.691585+0000","last_active":"2023-11-28T03:07:47.691585+0000","last_peered":"2023-11-28T03:07:47.691585+0000","last_clean":"2023-11-28T03:07:47.691585+0000","last_became_active":"2023-11-28T03:07:47.691314+0000","last_became_peered":"2023-11-28T03:07:47.691314+0000","last_unstale":"2023-11-28T03:07:47.691585+0000","last_undegraded":"2023-11-28T03:07:47.691585+0000","last_fullsized":"2023-11-28T03:07:47.691585+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689186+0000","last_change":"2023-11-28T03:07:47.324365+0000","last_active":"2023-11-28T03:07:47.689186+0000","last_peered":"2023-11-28T03:07:47.689186+0000","last_clean":"2023-11-28T03:07:47.689186+0000","last_became_active":"2023-11-28T03:07:47.324073+0000","last_became_peered":"2023-11-28T03:07:47.324073+0000","last_unstale":"2023-11-28T03:07:47.689186+0000","last_undegraded":"2023-11-28T03:07:47.689186+0000","last_fullsized":"2023-11-28T03:07:47.689186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.317635+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:07:47.317635+0000","last_peered":"2023-11-28T03:07:47.317635+0000","last_clean":"2023-11-28T03:07:47.317635+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:07:47.317635+0000","last_undegraded":"2023-11-28T03:07:47.317635+0000","last_fullsized":"2023-11-28T03:07:47.317635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685725+0000","last_change":"2023-11-28T03:07:47.685725+0000","last_active":"2023-11-28T03:07:47.685725+0000","last_peered":"2023-11-28T03:07:47.685725+0000","last_clean":"2023-11-28T03:07:47.685725+0000","last_became_active":"2023-11-28T03:07:47.685393+0000","last_became_peered":"2023-11-28T03:07:47.685393+0000","last_unstale":"2023-11-28T03:07:47.685725+0000","last_undegraded":"2023-11-28T03:07:47.685725+0000","last_fullsized":"2023-11-28T03:07:47.685725+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693197+0000","last_change":"2023-11-28T03:07:47.324769+0000","last_active":"2023-11-28T03:07:47.693197+0000","last_peered":"2023-11-28T03:07:47.693197+0000","last_clean":"2023-11-28T03:07:47.693197+0000","last_became_active":"2023-11-28T03:07:47.324249+0000","last_became_peered":"2023-11-28T03:07:47.324249+0000","last_unstale":"2023-11-28T03:07:47.693197+0000","last_undegraded":"2023-11-28T03:07:47.693197+0000","last_fullsized":"2023-11-28T03:07:47.693197+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.311570+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:07:47.311570+0000","last_peered":"2023-11-28T03:07:47.311570+0000","last_clean":"2023-11-28T03:07:47.311570+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:07:47.311570+0000","last_undegraded":"2023-11-28T03:07:47.311570+0000","last_fullsized":"2023-11-28T03:07:47.311570+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":1146880,"data_stored":567427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.628}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.73899999999999999,"5min":0.749,"15min":0.749},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.70399999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.66100000000000003,"5min":0.748,"15min":0.81899999999999995},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.755,"5min":0.755,"15min":0.84499999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.83199999999999996},"last":0.77100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.86499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.378},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.441},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":1.125,"5min":1.506,"15min":1.506},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.53000000000000003,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67800000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.82499999999999996},"last":0.38800000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.755,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34399999999999997,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.28000000000000003,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46000000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.372}]},{"osd":2,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.19800000000000001},"max":{"1min":0.56799999999999995,"5min":0.628,"15min":0.72599999999999998},"last":0.34499999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.69599999999999995,"15min":0.749},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.434},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.66500000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":1.038,"15min":1.111},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.374},"max":{"1min":0.76000000000000001,"5min":1.024,"15min":1.024},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.41399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":1146880,"data_stored":567427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.221,"15min":0.184},"max":{"1min":0.45100000000000001,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.51700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.25600000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36499999999999999,"15min":0.36899999999999999},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.505,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.372,"5min":0.307,"15min":0.26300000000000001},"max":{"1min":0.57599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.26200000000000001,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.78000000000000003,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.77600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":1.304,"15min":1.304},"last":0.438},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39000000000000001},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":1.679,"15min":1.679},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.495},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.046,"15min":1.046},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.27600000000000002,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.623},"last":0.308},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.29299999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.36299999999999999},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.252},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.378},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.44},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.47899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.874},"last":0.501}]},{"osd":5,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.439,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.004,"15min":1.004},"last":0.39100000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.45300000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.628,"15min":0.628},"min":{"1min":0.55200000000000005,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.28499999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.251},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27400000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.373},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.32900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.28499999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41899999999999998},"min":{"1min":0.32400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.438,"15min":0.435},"min":{"1min":0.36799999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85799999999999998,"5min":1.581,"15min":1.6000000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.94199999999999995,"15min":1.196},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.63700000000000001},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.95799999999999996,"15min":1.206},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.65100000000000002},"min":{"1min":0.51500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.51800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42699999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.45100000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.78100000000000003},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.38500000000000001},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.878},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.77900000000000003},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83899999999999997},"last":0.5},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.91300000000000003},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.79800000000000004},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.442}]},{"osd":2,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.752}]},{"osd":3,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.621,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.62,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:07:51.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:07:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.813+0000 7fb9d9576700 1 Processor -- start 2023-11-28T03:07:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.813+0000 7fb9d9576700 1 -- start start 2023-11-28T03:07:51.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9d9576700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9d40caab0 0x7fb9d40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9d9576700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d40a9a90 con 0x7fb9d40cae90 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9d9576700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9d4154ce0 con 0x7fb9d40ca180 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9d9576700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9d40a8e80 con 0x7fb9d40caab0 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9d2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9d40caab0 0x7fb9d40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9ca7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb9d40cae90 0x7fb9d40c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51130/0 (socket says 172.21.15.73:51130) 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.814+0000 7fb9ca7fc700 1 -- 172.21.15.73:0/849210543 learned_addr learned my addr 172.21.15.73:0/849210543 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/849210543 >> v1:172.21.15.150:6789/0 conn(0x7fb9d40ca180 legacy=0x7fb9d415e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/849210543 >> v1:172.21.15.73:6789/0 conn(0x7fb9d40cae90 legacy=0x7fb9d40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d27fc700 1 -- 172.21.15.73:0/849210543 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 786400452 0 0) 0x7fb9d40a9a90 con 0x7fb9d40cae90 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/849210543 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d40a9160 con 0x7fb9d40caab0 2023-11-28T03:07:51.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d2ffd700 1 --2- 172.21.15.73:0/849210543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40caab0 0x7fb9d40aa470 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fb9c4006fd0 tx=0x7fb9c4006f40).ready entity=mon.0 client_cookie=c459e924444e2a4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d27fc700 1 -- 172.21.15.73:0/849210543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4005880 con 0x7fb9d40caab0 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d27fc700 1 -- 172.21.15.73:0/849210543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c40059e0 con 0x7fb9d40caab0 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d27fc700 1 -- 172.21.15.73:0/849210543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c4005d60 con 0x7fb9d40caab0 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 -- 172.21.15.73:0/849210543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40caab0 msgr2=0x7fb9d40aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 --2- 172.21.15.73:0/849210543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40caab0 0x7fb9d40aa470 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 rx=0x7fb9c4006fd0 tx=0x7fb9c4006f40).stop 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 -- 172.21.15.73:0/849210543 shutdown_connections 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 --2- 172.21.15.73:0/849210543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40caab0 0x7fb9d40aa470 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 -- 172.21.15.73:0/849210543 >> 172.21.15.73:0/849210543 conn(0x7fb9d4094920 msgr2=0x7fb9d406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 -- 172.21.15.73:0/849210543 shutdown_connections 2023-11-28T03:07:51.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.815+0000 7fb9d9576700 1 -- 172.21.15.73:0/849210543 wait complete. 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 Processor -- start 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 -- start start 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9d40caab0 0x7fb9d4165090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 0x7fb9d40c43a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c3520 con 0x7fb9d40ca180 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c9ad0 con 0x7fb9d40cae90 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d9576700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9d40c2910 con 0x7fb9d40caab0 2023-11-28T03:07:51.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50008/0 (socket says 172.21.15.73:50008) 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/1381423788 learned_addr learned my addr 172.21.15.73:0/1381423788 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9ca7fc700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9d40caab0 0x7fb9d4165090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.817+0000 7fb9d37fe700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 0x7fb9d40c43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9d40caab0 msgr2=0x7fb9d4165090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9d40caab0 0x7fb9d4165090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 msgr2=0x7fb9d40c43a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 0x7fb9d40c43a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 -- 172.21.15.73:0/1381423788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9c4009250 con 0x7fb9d40ca180 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d37fe700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 0x7fb9d40c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d2ffd700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fb9d4083c30 tx=0x7fb9c4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400f720 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9c400f880 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9d40c2bf0 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9c400fc40 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.818+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9d40c31b0 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.819+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9d40c3800 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.819+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb9c401c650 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.819+0000 7fb9cb7fe700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 0x7fb9b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.820+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb9c407bac0 con 0x7fb9d40ca180 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.820+0000 7fb9ca7fc700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 0x7fb9b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:51.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.820+0000 7fb9ca7fc700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 0x7fb9b4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb9d41573f0 tx=0x7fb9bc007dd0).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:51.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.824+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb9c404cd90 con 0x7fb9d40ca180 2023-11-28T03:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.982+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9d40c3660 con 0x7fb9b405f150 2023-11-28T03:07:51.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.992+0000 7fb9cb7fe700 1 -- 172.21.15.73:0/1381423788 <== mgr.6082 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361392 (secure 0 0 0) 0x7fb9bc0074f0 con 0x7fb9b405f150 2023-11-28T03:07:51.997 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:52.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.995+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 msgr2=0x7fb9b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.995+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 0x7fb9b4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fb9d41573f0 tx=0x7fb9bc007dd0).stop 2023-11-28T03:07:52.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.995+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 msgr2=0x7fb9d40c95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.000 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.995+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 rx=0x7fb9d4083c30 tx=0x7fb9c4015000).stop 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 shutdown_connections 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb9b405f150 0x7fb9b4061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9d40cae90 0x7fb9d40c43a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9d40ca180 0x7fb9d40c95e0 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 --2- 172.21.15.73:0/1381423788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9d40caab0 0x7fb9d4165090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 >> 172.21.15.73:0/1381423788 conn(0x7fb9d4094920 msgr2=0x7fb9d4153f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 shutdown_connections 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:51.996+0000 7fb9d9576700 1 -- 172.21.15.73:0/1381423788 wait complete. 2023-11-28T03:07:52.001 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:07:52.008 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:07:50.714846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":60,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2418424,"kb_used_data":8888,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752556296,"statfs":{"total":773094113280,"available":770617647104,"internally_reserved":0,"allocated":9101312,"data_stored":4499342,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10151,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-59,"num_write_kb":-28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.583405"},"pg_stats":[{"pgid":"12.38","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692249+0000","last_change":"2023-11-28T03:07:47.692249+0000","last_active":"2023-11-28T03:07:47.692249+0000","last_peered":"2023-11-28T03:07:47.692249+0000","last_clean":"2023-11-28T03:07:47.692249+0000","last_became_active":"2023-11-28T03:07:47.691962+0000","last_became_peered":"2023-11-28T03:07:47.691962+0000","last_unstale":"2023-11-28T03:07:47.692249+0000","last_undegraded":"2023-11-28T03:07:47.692249+0000","last_fullsized":"2023-11-28T03:07:47.692249+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3f","version":"95'3","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696498+0000","last_change":"2023-11-28T03:07:47.334144+0000","last_active":"2023-11-28T03:07:47.696498+0000","last_peered":"2023-11-28T03:07:47.696498+0000","last_clean":"2023-11-28T03:07:47.696498+0000","last_became_active":"2023-11-28T03:07:47.333881+0000","last_became_peered":"2023-11-28T03:07:47.333881+0000","last_unstale":"2023-11-28T03:07:47.696498+0000","last_undegraded":"2023-11-28T03:07:47.696498+0000","last_fullsized":"2023-11-28T03:07:47.696498+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.39","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688167+0000","last_change":"2023-11-28T03:07:47.688167+0000","last_active":"2023-11-28T03:07:47.688167+0000","last_peered":"2023-11-28T03:07:47.688167+0000","last_clean":"2023-11-28T03:07:47.688167+0000","last_became_active":"2023-11-28T03:07:47.687766+0000","last_became_peered":"2023-11-28T03:07:47.687766+0000","last_unstale":"2023-11-28T03:07:47.688167+0000","last_undegraded":"2023-11-28T03:07:47.688167+0000","last_fullsized":"2023-11-28T03:07:47.688167+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325881+0000","last_change":"2023-11-28T03:07:47.323580+0000","last_active":"2023-11-28T03:07:47.325881+0000","last_peered":"2023-11-28T03:07:47.325881+0000","last_clean":"2023-11-28T03:07:47.325881+0000","last_became_active":"2023-11-28T03:07:47.323305+0000","last_became_peered":"2023-11-28T03:07:47.323305+0000","last_unstale":"2023-11-28T03:07:47.325881+0000","last_undegraded":"2023-11-28T03:07:47.325881+0000","last_fullsized":"2023-11-28T03:07:47.325881+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696755+0000","last_change":"2023-11-28T03:07:47.696755+0000","last_active":"2023-11-28T03:07:47.696755+0000","last_peered":"2023-11-28T03:07:47.696755+0000","last_clean":"2023-11-28T03:07:47.696755+0000","last_became_active":"2023-11-28T03:07:47.696487+0000","last_became_peered":"2023-11-28T03:07:47.696487+0000","last_unstale":"2023-11-28T03:07:47.696755+0000","last_undegraded":"2023-11-28T03:07:47.696755+0000","last_fullsized":"2023-11-28T03:07:47.696755+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333584+0000","last_change":"2023-11-28T03:07:47.323854+0000","last_active":"2023-11-28T03:07:47.333584+0000","last_peered":"2023-11-28T03:07:47.333584+0000","last_clean":"2023-11-28T03:07:47.333584+0000","last_became_active":"2023-11-28T03:07:47.320163+0000","last_became_peered":"2023-11-28T03:07:47.320163+0000","last_unstale":"2023-11-28T03:07:47.333584+0000","last_undegraded":"2023-11-28T03:07:47.333584+0000","last_fullsized":"2023-11-28T03:07:47.333584+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.3b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695481+0000","last_change":"2023-11-28T03:07:47.695481+0000","last_active":"2023-11-28T03:07:47.695481+0000","last_peered":"2023-11-28T03:07:47.695481+0000","last_clean":"2023-11-28T03:07:47.695481+0000","last_became_active":"2023-11-28T03:07:47.695213+0000","last_became_peered":"2023-11-28T03:07:47.695213+0000","last_unstale":"2023-11-28T03:07:47.695481+0000","last_undegraded":"2023-11-28T03:07:47.695481+0000","last_fullsized":"2023-11-28T03:07:47.695481+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3c","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696607+0000","last_change":"2023-11-28T03:07:47.323582+0000","last_active":"2023-11-28T03:07:47.696607+0000","last_peered":"2023-11-28T03:07:47.696607+0000","last_clean":"2023-11-28T03:07:47.696607+0000","last_became_active":"2023-11-28T03:07:47.323303+0000","last_became_peered":"2023-11-28T03:07:47.323303+0000","last_unstale":"2023-11-28T03:07:47.696607+0000","last_undegraded":"2023-11-28T03:07:47.696607+0000","last_fullsized":"2023-11-28T03:07:47.696607+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698597+0000","last_change":"2023-11-28T03:07:47.698597+0000","last_active":"2023-11-28T03:07:47.698597+0000","last_peered":"2023-11-28T03:07:47.698597+0000","last_clean":"2023-11-28T03:07:47.698597+0000","last_became_active":"2023-11-28T03:07:47.698082+0000","last_became_peered":"2023-11-28T03:07:47.698082+0000","last_unstale":"2023-11-28T03:07:47.698597+0000","last_undegraded":"2023-11-28T03:07:47.698597+0000","last_fullsized":"2023-11-28T03:07:47.698597+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.675151+0000","last_change":"2023-11-28T03:07:47.326018+0000","last_active":"2023-11-28T03:07:47.675151+0000","last_peered":"2023-11-28T03:07:47.675151+0000","last_clean":"2023-11-28T03:07:47.675151+0000","last_became_active":"2023-11-28T03:07:47.325774+0000","last_became_peered":"2023-11-28T03:07:47.325774+0000","last_unstale":"2023-11-28T03:07:47.675151+0000","last_undegraded":"2023-11-28T03:07:47.675151+0000","last_fullsized":"2023-11-28T03:07:47.675151+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.3d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691619+0000","last_change":"2023-11-28T03:07:47.691619+0000","last_active":"2023-11-28T03:07:47.691619+0000","last_peered":"2023-11-28T03:07:47.691619+0000","last_clean":"2023-11-28T03:07:47.691619+0000","last_became_active":"2023-11-28T03:07:47.691006+0000","last_became_peered":"2023-11-28T03:07:47.691006+0000","last_unstale":"2023-11-28T03:07:47.691619+0000","last_undegraded":"2023-11-28T03:07:47.691619+0000","last_fullsized":"2023-11-28T03:07:47.691619+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3a","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.674766+0000","last_change":"2023-11-28T03:07:47.327576+0000","last_active":"2023-11-28T03:07:47.674766+0000","last_peered":"2023-11-28T03:07:47.674766+0000","last_clean":"2023-11-28T03:07:47.674766+0000","last_became_active":"2023-11-28T03:07:47.327255+0000","last_became_peered":"2023-11-28T03:07:47.327255+0000","last_unstale":"2023-11-28T03:07:47.674766+0000","last_undegraded":"2023-11-28T03:07:47.674766+0000","last_fullsized":"2023-11-28T03:07:47.674766+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.3e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690774+0000","last_change":"2023-11-28T03:07:47.690774+0000","last_active":"2023-11-28T03:07:47.690774+0000","last_peered":"2023-11-28T03:07:47.690774+0000","last_clean":"2023-11-28T03:07:47.690774+0000","last_became_active":"2023-11-28T03:07:47.690521+0000","last_became_peered":"2023-11-28T03:07:47.690521+0000","last_unstale":"2023-11-28T03:07:47.690774+0000","last_undegraded":"2023-11-28T03:07:47.690774+0000","last_fullsized":"2023-11-28T03:07:47.690774+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329800+0000","last_change":"2023-11-28T03:07:47.325059+0000","last_active":"2023-11-28T03:07:47.329800+0000","last_peered":"2023-11-28T03:07:47.329800+0000","last_clean":"2023-11-28T03:07:47.329800+0000","last_became_active":"2023-11-28T03:07:47.324393+0000","last_became_peered":"2023-11-28T03:07:47.324393+0000","last_unstale":"2023-11-28T03:07:47.329800+0000","last_undegraded":"2023-11-28T03:07:47.329800+0000","last_fullsized":"2023-11-28T03:07:47.329800+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695215+0000","last_change":"2023-11-28T03:07:47.695215+0000","last_active":"2023-11-28T03:07:47.695215+0000","last_peered":"2023-11-28T03:07:47.695215+0000","last_clean":"2023-11-28T03:07:47.695215+0000","last_became_active":"2023-11-28T03:07:47.694898+0000","last_became_peered":"2023-11-28T03:07:47.694898+0000","last_unstale":"2023-11-28T03:07:47.695215+0000","last_undegraded":"2023-11-28T03:07:47.695215+0000","last_fullsized":"2023-11-28T03:07:47.695215+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324435+0000","last_change":"2023-11-28T03:07:47.317037+0000","last_active":"2023-11-28T03:07:47.324435+0000","last_peered":"2023-11-28T03:07:47.324435+0000","last_clean":"2023-11-28T03:07:47.324435+0000","last_became_active":"2023-11-28T03:07:47.316745+0000","last_became_peered":"2023-11-28T03:07:47.316745+0000","last_unstale":"2023-11-28T03:07:47.324435+0000","last_undegraded":"2023-11-28T03:07:47.324435+0000","last_fullsized":"2023-11-28T03:07:47.324435+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.30","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699729+0000","last_change":"2023-11-28T03:07:47.699729+0000","last_active":"2023-11-28T03:07:47.699729+0000","last_peered":"2023-11-28T03:07:47.699729+0000","last_clean":"2023-11-28T03:07:47.699729+0000","last_became_active":"2023-11-28T03:07:47.699467+0000","last_became_peered":"2023-11-28T03:07:47.699467+0000","last_unstale":"2023-11-28T03:07:47.699729+0000","last_undegraded":"2023-11-28T03:07:47.699729+0000","last_fullsized":"2023-11-28T03:07:47.699729+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326748+0000","last_change":"2023-11-28T03:07:47.319703+0000","last_active":"2023-11-28T03:07:47.326748+0000","last_peered":"2023-11-28T03:07:47.326748+0000","last_clean":"2023-11-28T03:07:47.326748+0000","last_became_active":"2023-11-28T03:07:47.319204+0000","last_became_peered":"2023-11-28T03:07:47.319204+0000","last_unstale":"2023-11-28T03:07:47.326748+0000","last_undegraded":"2023-11-28T03:07:47.326748+0000","last_fullsized":"2023-11-28T03:07:47.326748+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.31","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689521+0000","last_change":"2023-11-28T03:07:47.689521+0000","last_active":"2023-11-28T03:07:47.689521+0000","last_peered":"2023-11-28T03:07:47.689521+0000","last_clean":"2023-11-28T03:07:47.689521+0000","last_became_active":"2023-11-28T03:07:47.689253+0000","last_became_peered":"2023-11-28T03:07:47.689253+0000","last_unstale":"2023-11-28T03:07:47.689521+0000","last_undegraded":"2023-11-28T03:07:47.689521+0000","last_fullsized":"2023-11-28T03:07:47.689521+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329988+0000","last_change":"2023-11-28T03:07:47.325325+0000","last_active":"2023-11-28T03:07:47.329988+0000","last_peered":"2023-11-28T03:07:47.329988+0000","last_clean":"2023-11-28T03:07:47.329988+0000","last_became_active":"2023-11-28T03:07:47.325017+0000","last_became_peered":"2023-11-28T03:07:47.325017+0000","last_unstale":"2023-11-28T03:07:47.329988+0000","last_undegraded":"2023-11-28T03:07:47.329988+0000","last_fullsized":"2023-11-28T03:07:47.329988+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.32","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696431+0000","last_change":"2023-11-28T03:07:47.696431+0000","last_active":"2023-11-28T03:07:47.696431+0000","last_peered":"2023-11-28T03:07:47.696431+0000","last_clean":"2023-11-28T03:07:47.696431+0000","last_became_active":"2023-11-28T03:07:47.696167+0000","last_became_peered":"2023-11-28T03:07:47.696167+0000","last_unstale":"2023-11-28T03:07:47.696431+0000","last_undegraded":"2023-11-28T03:07:47.696431+0000","last_fullsized":"2023-11-28T03:07:47.696431+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336261+0000","last_change":"2023-11-28T03:07:47.334251+0000","last_active":"2023-11-28T03:07:47.336261+0000","last_peered":"2023-11-28T03:07:47.336261+0000","last_clean":"2023-11-28T03:07:47.336261+0000","last_became_active":"2023-11-28T03:07:47.333977+0000","last_became_peered":"2023-11-28T03:07:47.333977+0000","last_unstale":"2023-11-28T03:07:47.336261+0000","last_undegraded":"2023-11-28T03:07:47.336261+0000","last_fullsized":"2023-11-28T03:07:47.336261+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.33","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687411+0000","last_change":"2023-11-28T03:07:47.687411+0000","last_active":"2023-11-28T03:07:47.687411+0000","last_peered":"2023-11-28T03:07:47.687411+0000","last_clean":"2023-11-28T03:07:47.687411+0000","last_became_active":"2023-11-28T03:07:47.687104+0000","last_became_peered":"2023-11-28T03:07:47.687104+0000","last_unstale":"2023-11-28T03:07:47.687411+0000","last_undegraded":"2023-11-28T03:07:47.687411+0000","last_fullsized":"2023-11-28T03:07:47.687411+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.34","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695635+0000","last_change":"2023-11-28T03:07:47.327306+0000","last_active":"2023-11-28T03:07:47.695635+0000","last_peered":"2023-11-28T03:07:47.695635+0000","last_clean":"2023-11-28T03:07:47.695635+0000","last_became_active":"2023-11-28T03:07:47.327024+0000","last_became_peered":"2023-11-28T03:07:47.327024+0000","last_unstale":"2023-11-28T03:07:47.695635+0000","last_undegraded":"2023-11-28T03:07:47.695635+0000","last_fullsized":"2023-11-28T03:07:47.695635+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.34","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.697211+0000","last_change":"2023-11-28T03:07:47.697211+0000","last_active":"2023-11-28T03:07:47.697211+0000","last_peered":"2023-11-28T03:07:47.697211+0000","last_clean":"2023-11-28T03:07:47.697211+0000","last_became_active":"2023-11-28T03:07:47.696875+0000","last_became_peered":"2023-11-28T03:07:47.696875+0000","last_unstale":"2023-11-28T03:07:47.697211+0000","last_undegraded":"2023-11-28T03:07:47.697211+0000","last_fullsized":"2023-11-28T03:07:47.697211+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.33","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681879+0000","last_change":"2023-11-28T03:07:47.321297+0000","last_active":"2023-11-28T03:07:47.681879+0000","last_peered":"2023-11-28T03:07:47.681879+0000","last_clean":"2023-11-28T03:07:47.681879+0000","last_became_active":"2023-11-28T03:07:47.321002+0000","last_became_peered":"2023-11-28T03:07:47.321002+0000","last_unstale":"2023-11-28T03:07:47.681879+0000","last_undegraded":"2023-11-28T03:07:47.681879+0000","last_fullsized":"2023-11-28T03:07:47.681879+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.35","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693002+0000","last_change":"2023-11-28T03:07:47.693002+0000","last_active":"2023-11-28T03:07:47.693002+0000","last_peered":"2023-11-28T03:07:47.693002+0000","last_clean":"2023-11-28T03:07:47.693002+0000","last_became_active":"2023-11-28T03:07:47.692534+0000","last_became_peered":"2023-11-28T03:07:47.692534+0000","last_unstale":"2023-11-28T03:07:47.693002+0000","last_undegraded":"2023-11-28T03:07:47.693002+0000","last_fullsized":"2023-11-28T03:07:47.693002+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325355+0000","last_change":"2023-11-28T03:07:47.321867+0000","last_active":"2023-11-28T03:07:47.325355+0000","last_peered":"2023-11-28T03:07:47.325355+0000","last_clean":"2023-11-28T03:07:47.325355+0000","last_became_active":"2023-11-28T03:07:47.321585+0000","last_became_peered":"2023-11-28T03:07:47.321585+0000","last_unstale":"2023-11-28T03:07:47.325355+0000","last_undegraded":"2023-11-28T03:07:47.325355+0000","last_fullsized":"2023-11-28T03:07:47.325355+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.36","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699168+0000","last_change":"2023-11-28T03:07:47.699168+0000","last_active":"2023-11-28T03:07:47.699168+0000","last_peered":"2023-11-28T03:07:47.699168+0000","last_clean":"2023-11-28T03:07:47.699168+0000","last_became_active":"2023-11-28T03:07:47.698861+0000","last_became_peered":"2023-11-28T03:07:47.698861+0000","last_unstale":"2023-11-28T03:07:47.699168+0000","last_undegraded":"2023-11-28T03:07:47.699168+0000","last_fullsized":"2023-11-28T03:07:47.699168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320293+0000","last_change":"2023-11-28T03:07:47.315532+0000","last_active":"2023-11-28T03:07:47.320293+0000","last_peered":"2023-11-28T03:07:47.320293+0000","last_clean":"2023-11-28T03:07:47.320293+0000","last_became_active":"2023-11-28T03:07:47.315209+0000","last_became_peered":"2023-11-28T03:07:47.315209+0000","last_unstale":"2023-11-28T03:07:47.320293+0000","last_undegraded":"2023-11-28T03:07:47.320293+0000","last_fullsized":"2023-11-28T03:07:47.320293+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.37","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.684777+0000","last_change":"2023-11-28T03:07:47.684777+0000","last_active":"2023-11-28T03:07:47.684777+0000","last_peered":"2023-11-28T03:07:47.684777+0000","last_clean":"2023-11-28T03:07:47.684777+0000","last_became_active":"2023-11-28T03:07:47.684484+0000","last_became_peered":"2023-11-28T03:07:47.684484+0000","last_unstale":"2023-11-28T03:07:47.684777+0000","last_undegraded":"2023-11-28T03:07:47.684777+0000","last_fullsized":"2023-11-28T03:07:47.684777+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696909+0000","last_change":"2023-11-28T03:07:47.323752+0000","last_active":"2023-11-28T03:07:47.696909+0000","last_peered":"2023-11-28T03:07:47.696909+0000","last_clean":"2023-11-28T03:07:47.696909+0000","last_became_active":"2023-11-28T03:07:47.323479+0000","last_became_peered":"2023-11-28T03:07:47.323479+0000","last_unstale":"2023-11-28T03:07:47.696909+0000","last_undegraded":"2023-11-28T03:07:47.696909+0000","last_fullsized":"2023-11-28T03:07:47.696909+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.28","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688698+0000","last_change":"2023-11-28T03:07:47.688698+0000","last_active":"2023-11-28T03:07:47.688698+0000","last_peered":"2023-11-28T03:07:47.688698+0000","last_clean":"2023-11-28T03:07:47.688698+0000","last_became_active":"2023-11-28T03:07:47.686107+0000","last_became_peered":"2023-11-28T03:07:47.686107+0000","last_unstale":"2023-11-28T03:07:47.688698+0000","last_undegraded":"2023-11-28T03:07:47.688698+0000","last_fullsized":"2023-11-28T03:07:47.688698+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326782+0000","last_change":"2023-11-28T03:07:47.319867+0000","last_active":"2023-11-28T03:07:47.326782+0000","last_peered":"2023-11-28T03:07:47.326782+0000","last_clean":"2023-11-28T03:07:47.326782+0000","last_became_active":"2023-11-28T03:07:47.319456+0000","last_became_peered":"2023-11-28T03:07:47.319456+0000","last_unstale":"2023-11-28T03:07:47.326782+0000","last_undegraded":"2023-11-28T03:07:47.326782+0000","last_fullsized":"2023-11-28T03:07:47.326782+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.29","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.700610+0000","last_change":"2023-11-28T03:07:47.700610+0000","last_active":"2023-11-28T03:07:47.700610+0000","last_peered":"2023-11-28T03:07:47.700610+0000","last_clean":"2023-11-28T03:07:47.700610+0000","last_became_active":"2023-11-28T03:07:47.700299+0000","last_became_peered":"2023-11-28T03:07:47.700299+0000","last_unstale":"2023-11-28T03:07:47.700610+0000","last_undegraded":"2023-11-28T03:07:47.700610+0000","last_fullsized":"2023-11-28T03:07:47.700610+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.328009+0000","last_change":"2023-11-28T03:07:47.326183+0000","last_active":"2023-11-28T03:07:47.328009+0000","last_peered":"2023-11-28T03:07:47.328009+0000","last_clean":"2023-11-28T03:07:47.328009+0000","last_became_active":"2023-11-28T03:07:47.325943+0000","last_became_peered":"2023-11-28T03:07:47.325943+0000","last_unstale":"2023-11-28T03:07:47.328009+0000","last_undegraded":"2023-11-28T03:07:47.328009+0000","last_fullsized":"2023-11-28T03:07:47.328009+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.2a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686823+0000","last_change":"2023-11-28T03:07:47.686823+0000","last_active":"2023-11-28T03:07:47.686823+0000","last_peered":"2023-11-28T03:07:47.686823+0000","last_clean":"2023-11-28T03:07:47.686823+0000","last_became_active":"2023-11-28T03:07:47.686557+0000","last_became_peered":"2023-11-28T03:07:47.686557+0000","last_unstale":"2023-11-28T03:07:47.686823+0000","last_undegraded":"2023-11-28T03:07:47.686823+0000","last_fullsized":"2023-11-28T03:07:47.686823+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2d","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.676175+0000","last_change":"2023-11-28T03:07:47.323256+0000","last_active":"2023-11-28T03:07:47.676175+0000","last_peered":"2023-11-28T03:07:47.676175+0000","last_clean":"2023-11-28T03:07:47.676175+0000","last_became_active":"2023-11-28T03:07:47.322919+0000","last_became_peered":"2023-11-28T03:07:47.322919+0000","last_unstale":"2023-11-28T03:07:47.676175+0000","last_undegraded":"2023-11-28T03:07:47.676175+0000","last_fullsized":"2023-11-28T03:07:47.676175+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693868+0000","last_change":"2023-11-28T03:07:47.693868+0000","last_active":"2023-11-28T03:07:47.693868+0000","last_peered":"2023-11-28T03:07:47.693868+0000","last_clean":"2023-11-28T03:07:47.693868+0000","last_became_active":"2023-11-28T03:07:47.693582+0000","last_became_peered":"2023-11-28T03:07:47.693582+0000","last_unstale":"2023-11-28T03:07:47.693868+0000","last_undegraded":"2023-11-28T03:07:47.693868+0000","last_fullsized":"2023-11-28T03:07:47.693868+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.323264+0000","last_change":"2023-11-28T03:07:47.321871+0000","last_active":"2023-11-28T03:07:47.323264+0000","last_peered":"2023-11-28T03:07:47.323264+0000","last_clean":"2023-11-28T03:07:47.323264+0000","last_became_active":"2023-11-28T03:07:47.321625+0000","last_became_peered":"2023-11-28T03:07:47.321625+0000","last_unstale":"2023-11-28T03:07:47.323264+0000","last_undegraded":"2023-11-28T03:07:47.323264+0000","last_fullsized":"2023-11-28T03:07:47.323264+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690246+0000","last_change":"2023-11-28T03:07:47.690246+0000","last_active":"2023-11-28T03:07:47.690246+0000","last_peered":"2023-11-28T03:07:47.690246+0000","last_clean":"2023-11-28T03:07:47.690246+0000","last_became_active":"2023-11-28T03:07:47.690002+0000","last_became_peered":"2023-11-28T03:07:47.690002+0000","last_unstale":"2023-11-28T03:07:47.690246+0000","last_undegraded":"2023-11-28T03:07:47.690246+0000","last_fullsized":"2023-11-28T03:07:47.690246+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.678118+0000","last_change":"2023-11-28T03:07:47.320539+0000","last_active":"2023-11-28T03:07:47.678118+0000","last_peered":"2023-11-28T03:07:47.678118+0000","last_clean":"2023-11-28T03:07:47.678118+0000","last_became_active":"2023-11-28T03:07:47.320311+0000","last_became_peered":"2023-11-28T03:07:47.320311+0000","last_unstale":"2023-11-28T03:07:47.678118+0000","last_undegraded":"2023-11-28T03:07:47.678118+0000","last_fullsized":"2023-11-28T03:07:47.678118+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696543+0000","last_change":"2023-11-28T03:07:47.696543+0000","last_active":"2023-11-28T03:07:47.696543+0000","last_peered":"2023-11-28T03:07:47.696543+0000","last_clean":"2023-11-28T03:07:47.696543+0000","last_became_active":"2023-11-28T03:07:47.696374+0000","last_became_peered":"2023-11-28T03:07:47.696374+0000","last_unstale":"2023-11-28T03:07:47.696543+0000","last_undegraded":"2023-11-28T03:07:47.696543+0000","last_fullsized":"2023-11-28T03:07:47.696543+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330010+0000","last_change":"2023-11-28T03:07:47.325913+0000","last_active":"2023-11-28T03:07:47.330010+0000","last_peered":"2023-11-28T03:07:47.330010+0000","last_clean":"2023-11-28T03:07:47.330010+0000","last_became_active":"2023-11-28T03:07:47.325639+0000","last_became_peered":"2023-11-28T03:07:47.325639+0000","last_unstale":"2023-11-28T03:07:47.330010+0000","last_undegraded":"2023-11-28T03:07:47.330010+0000","last_fullsized":"2023-11-28T03:07:47.330010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.2e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698335+0000","last_change":"2023-11-28T03:07:47.698335+0000","last_active":"2023-11-28T03:07:47.698335+0000","last_peered":"2023-11-28T03:07:47.698335+0000","last_clean":"2023-11-28T03:07:47.698335+0000","last_became_active":"2023-11-28T03:07:47.698053+0000","last_became_peered":"2023-11-28T03:07:47.698053+0000","last_unstale":"2023-11-28T03:07:47.698335+0000","last_undegraded":"2023-11-28T03:07:47.698335+0000","last_fullsized":"2023-11-28T03:07:47.698335+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330600+0000","last_change":"2023-11-28T03:07:47.328284+0000","last_active":"2023-11-28T03:07:47.330600+0000","last_peered":"2023-11-28T03:07:47.330600+0000","last_clean":"2023-11-28T03:07:47.330600+0000","last_became_active":"2023-11-28T03:07:47.327815+0000","last_became_peered":"2023-11-28T03:07:47.327815+0000","last_unstale":"2023-11-28T03:07:47.330600+0000","last_undegraded":"2023-11-28T03:07:47.330600+0000","last_fullsized":"2023-11-28T03:07:47.330600+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.2f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.683118+0000","last_change":"2023-11-28T03:07:47.683118+0000","last_active":"2023-11-28T03:07:47.683118+0000","last_peered":"2023-11-28T03:07:47.683118+0000","last_clean":"2023-11-28T03:07:47.683118+0000","last_became_active":"2023-11-28T03:07:47.682285+0000","last_became_peered":"2023-11-28T03:07:47.682285+0000","last_unstale":"2023-11-28T03:07:47.683118+0000","last_undegraded":"2023-11-28T03:07:47.683118+0000","last_fullsized":"2023-11-28T03:07:47.683118+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330534+0000","last_change":"2023-11-28T03:07:47.328131+0000","last_active":"2023-11-28T03:07:47.330534+0000","last_peered":"2023-11-28T03:07:47.330534+0000","last_clean":"2023-11-28T03:07:47.330534+0000","last_became_active":"2023-11-28T03:07:47.327258+0000","last_became_peered":"2023-11-28T03:07:47.327258+0000","last_unstale":"2023-11-28T03:07:47.330534+0000","last_undegraded":"2023-11-28T03:07:47.330534+0000","last_fullsized":"2023-11-28T03:07:47.330534+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.20","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696069+0000","last_change":"2023-11-28T03:07:47.696069+0000","last_active":"2023-11-28T03:07:47.696069+0000","last_peered":"2023-11-28T03:07:47.696069+0000","last_clean":"2023-11-28T03:07:47.696069+0000","last_became_active":"2023-11-28T03:07:47.695756+0000","last_became_peered":"2023-11-28T03:07:47.695756+0000","last_unstale":"2023-11-28T03:07:47.696069+0000","last_undegraded":"2023-11-28T03:07:47.696069+0000","last_fullsized":"2023-11-28T03:07:47.696069+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330901+0000","last_change":"2023-11-28T03:07:47.328938+0000","last_active":"2023-11-28T03:07:47.330901+0000","last_peered":"2023-11-28T03:07:47.330901+0000","last_clean":"2023-11-28T03:07:47.330901+0000","last_became_active":"2023-11-28T03:07:47.328635+0000","last_became_peered":"2023-11-28T03:07:47.328635+0000","last_unstale":"2023-11-28T03:07:47.330901+0000","last_undegraded":"2023-11-28T03:07:47.330901+0000","last_fullsized":"2023-11-28T03:07:47.330901+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.21","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695203+0000","last_change":"2023-11-28T03:07:47.695203+0000","last_active":"2023-11-28T03:07:47.695203+0000","last_peered":"2023-11-28T03:07:47.695203+0000","last_clean":"2023-11-28T03:07:47.695203+0000","last_became_active":"2023-11-28T03:07:47.694968+0000","last_became_peered":"2023-11-28T03:07:47.694968+0000","last_unstale":"2023-11-28T03:07:47.695203+0000","last_undegraded":"2023-11-28T03:07:47.695203+0000","last_fullsized":"2023-11-28T03:07:47.695203+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329513+0000","last_change":"2023-11-28T03:07:47.321952+0000","last_active":"2023-11-28T03:07:47.329513+0000","last_peered":"2023-11-28T03:07:47.329513+0000","last_clean":"2023-11-28T03:07:47.329513+0000","last_became_active":"2023-11-28T03:07:47.321674+0000","last_became_peered":"2023-11-28T03:07:47.321674+0000","last_unstale":"2023-11-28T03:07:47.329513+0000","last_undegraded":"2023-11-28T03:07:47.329513+0000","last_fullsized":"2023-11-28T03:07:47.329513+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.22","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696444+0000","last_change":"2023-11-28T03:07:47.696444+0000","last_active":"2023-11-28T03:07:47.696444+0000","last_peered":"2023-11-28T03:07:47.696444+0000","last_clean":"2023-11-28T03:07:47.696444+0000","last_became_active":"2023-11-28T03:07:47.696103+0000","last_became_peered":"2023-11-28T03:07:47.696103+0000","last_unstale":"2023-11-28T03:07:47.696444+0000","last_undegraded":"2023-11-28T03:07:47.696444+0000","last_fullsized":"2023-11-28T03:07:47.696444+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333760+0000","last_change":"2023-11-28T03:07:47.325631+0000","last_active":"2023-11-28T03:07:47.333760+0000","last_peered":"2023-11-28T03:07:47.333760+0000","last_clean":"2023-11-28T03:07:47.333760+0000","last_became_active":"2023-11-28T03:07:47.325357+0000","last_became_peered":"2023-11-28T03:07:47.325357+0000","last_unstale":"2023-11-28T03:07:47.333760+0000","last_undegraded":"2023-11-28T03:07:47.333760+0000","last_fullsized":"2023-11-28T03:07:47.333760+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.23","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696290+0000","last_change":"2023-11-28T03:07:47.696290+0000","last_active":"2023-11-28T03:07:47.696290+0000","last_peered":"2023-11-28T03:07:47.696290+0000","last_clean":"2023-11-28T03:07:47.696290+0000","last_became_active":"2023-11-28T03:07:47.696044+0000","last_became_peered":"2023-11-28T03:07:47.696044+0000","last_unstale":"2023-11-28T03:07:47.696290+0000","last_undegraded":"2023-11-28T03:07:47.696290+0000","last_fullsized":"2023-11-28T03:07:47.696290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329899+0000","last_change":"2023-11-28T03:07:47.325277+0000","last_active":"2023-11-28T03:07:47.329899+0000","last_peered":"2023-11-28T03:07:47.329899+0000","last_clean":"2023-11-28T03:07:47.329899+0000","last_became_active":"2023-11-28T03:07:47.324796+0000","last_became_peered":"2023-11-28T03:07:47.324796+0000","last_unstale":"2023-11-28T03:07:47.329899+0000","last_undegraded":"2023-11-28T03:07:47.329899+0000","last_fullsized":"2023-11-28T03:07:47.329899+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.24","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701331+0000","last_change":"2023-11-28T03:07:47.701331+0000","last_active":"2023-11-28T03:07:47.701331+0000","last_peered":"2023-11-28T03:07:47.701331+0000","last_clean":"2023-11-28T03:07:47.701331+0000","last_became_active":"2023-11-28T03:07:47.701080+0000","last_became_peered":"2023-11-28T03:07:47.701080+0000","last_unstale":"2023-11-28T03:07:47.701331+0000","last_undegraded":"2023-11-28T03:07:47.701331+0000","last_fullsized":"2023-11-28T03:07:47.701331+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333579+0000","last_change":"2023-11-28T03:07:47.324380+0000","last_active":"2023-11-28T03:07:47.333579+0000","last_peered":"2023-11-28T03:07:47.333579+0000","last_clean":"2023-11-28T03:07:47.333579+0000","last_became_active":"2023-11-28T03:07:47.324124+0000","last_became_peered":"2023-11-28T03:07:47.324124+0000","last_unstale":"2023-11-28T03:07:47.333579+0000","last_undegraded":"2023-11-28T03:07:47.333579+0000","last_fullsized":"2023-11-28T03:07:47.333579+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.25","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.700290+0000","last_change":"2023-11-28T03:07:47.700290+0000","last_active":"2023-11-28T03:07:47.700290+0000","last_peered":"2023-11-28T03:07:47.700290+0000","last_clean":"2023-11-28T03:07:47.700290+0000","last_became_active":"2023-11-28T03:07:47.699102+0000","last_became_peered":"2023-11-28T03:07:47.699102+0000","last_unstale":"2023-11-28T03:07:47.700290+0000","last_undegraded":"2023-11-28T03:07:47.700290+0000","last_fullsized":"2023-11-28T03:07:47.700290+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329930+0000","last_change":"2023-11-28T03:07:47.325628+0000","last_active":"2023-11-28T03:07:47.329930+0000","last_peered":"2023-11-28T03:07:47.329930+0000","last_clean":"2023-11-28T03:07:47.329930+0000","last_became_active":"2023-11-28T03:07:47.325364+0000","last_became_peered":"2023-11-28T03:07:47.325364+0000","last_unstale":"2023-11-28T03:07:47.329930+0000","last_undegraded":"2023-11-28T03:07:47.329930+0000","last_fullsized":"2023-11-28T03:07:47.329930+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.26","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698153+0000","last_change":"2023-11-28T03:07:47.698153+0000","last_active":"2023-11-28T03:07:47.698153+0000","last_peered":"2023-11-28T03:07:47.698153+0000","last_clean":"2023-11-28T03:07:47.698153+0000","last_became_active":"2023-11-28T03:07:47.697726+0000","last_became_peered":"2023-11-28T03:07:47.697726+0000","last_unstale":"2023-11-28T03:07:47.698153+0000","last_undegraded":"2023-11-28T03:07:47.698153+0000","last_fullsized":"2023-11-28T03:07:47.698153+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330758+0000","last_change":"2023-11-28T03:07:47.326407+0000","last_active":"2023-11-28T03:07:47.330758+0000","last_peered":"2023-11-28T03:07:47.330758+0000","last_clean":"2023-11-28T03:07:47.330758+0000","last_became_active":"2023-11-28T03:07:47.326117+0000","last_became_peered":"2023-11-28T03:07:47.326117+0000","last_unstale":"2023-11-28T03:07:47.330758+0000","last_undegraded":"2023-11-28T03:07:47.330758+0000","last_fullsized":"2023-11-28T03:07:47.330758+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.27","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688412+0000","last_change":"2023-11-28T03:07:47.688412+0000","last_active":"2023-11-28T03:07:47.688412+0000","last_peered":"2023-11-28T03:07:47.688412+0000","last_clean":"2023-11-28T03:07:47.688412+0000","last_became_active":"2023-11-28T03:07:47.686169+0000","last_became_peered":"2023-11-28T03:07:47.686169+0000","last_unstale":"2023-11-28T03:07:47.688412+0000","last_undegraded":"2023-11-28T03:07:47.688412+0000","last_fullsized":"2023-11-28T03:07:47.688412+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326010+0000","last_change":"2023-11-28T03:07:47.322989+0000","last_active":"2023-11-28T03:07:47.326010+0000","last_peered":"2023-11-28T03:07:47.326010+0000","last_clean":"2023-11-28T03:07:47.326010+0000","last_became_active":"2023-11-28T03:07:47.322709+0000","last_became_peered":"2023-11-28T03:07:47.322709+0000","last_unstale":"2023-11-28T03:07:47.326010+0000","last_undegraded":"2023-11-28T03:07:47.326010+0000","last_fullsized":"2023-11-28T03:07:47.326010+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.694142+0000","last_change":"2023-11-28T03:07:47.694142+0000","last_active":"2023-11-28T03:07:47.694142+0000","last_peered":"2023-11-28T03:07:47.694142+0000","last_clean":"2023-11-28T03:07:47.694142+0000","last_became_active":"2023-11-28T03:07:47.693837+0000","last_became_peered":"2023-11-28T03:07:47.693837+0000","last_unstale":"2023-11-28T03:07:47.694142+0000","last_undegraded":"2023-11-28T03:07:47.694142+0000","last_fullsized":"2023-11-28T03:07:47.694142+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1f","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.675403+0000","last_change":"2023-11-28T03:07:47.319952+0000","last_active":"2023-11-28T03:07:47.675403+0000","last_peered":"2023-11-28T03:07:47.675403+0000","last_clean":"2023-11-28T03:07:47.675403+0000","last_became_active":"2023-11-28T03:07:47.319744+0000","last_became_peered":"2023-11-28T03:07:47.319744+0000","last_unstale":"2023-11-28T03:07:47.675403+0000","last_undegraded":"2023-11-28T03:07:47.675403+0000","last_fullsized":"2023-11-28T03:07:47.675403+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701394+0000","last_change":"2023-11-28T03:07:47.701394+0000","last_active":"2023-11-28T03:07:47.701394+0000","last_peered":"2023-11-28T03:07:47.701394+0000","last_clean":"2023-11-28T03:07:47.701394+0000","last_became_active":"2023-11-28T03:07:47.701128+0000","last_became_peered":"2023-11-28T03:07:47.701128+0000","last_unstale":"2023-11-28T03:07:47.701394+0000","last_undegraded":"2023-11-28T03:07:47.701394+0000","last_fullsized":"2023-11-28T03:07:47.701394+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685152+0000","last_change":"2023-11-28T03:07:47.326855+0000","last_active":"2023-11-28T03:07:47.685152+0000","last_peered":"2023-11-28T03:07:47.685152+0000","last_clean":"2023-11-28T03:07:47.685152+0000","last_became_active":"2023-11-28T03:07:47.326562+0000","last_became_peered":"2023-11-28T03:07:47.326562+0000","last_unstale":"2023-11-28T03:07:47.685152+0000","last_undegraded":"2023-11-28T03:07:47.685152+0000","last_fullsized":"2023-11-28T03:07:47.685152+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690148+0000","last_change":"2023-11-28T03:07:47.690148+0000","last_active":"2023-11-28T03:07:47.690148+0000","last_peered":"2023-11-28T03:07:47.690148+0000","last_clean":"2023-11-28T03:07:47.690148+0000","last_became_active":"2023-11-28T03:07:47.689761+0000","last_became_peered":"2023-11-28T03:07:47.689761+0000","last_unstale":"2023-11-28T03:07:47.690148+0000","last_undegraded":"2023-11-28T03:07:47.690148+0000","last_fullsized":"2023-11-28T03:07:47.690148+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696623+0000","last_change":"2023-11-28T03:07:47.328540+0000","last_active":"2023-11-28T03:07:47.696623+0000","last_peered":"2023-11-28T03:07:47.696623+0000","last_clean":"2023-11-28T03:07:47.696623+0000","last_became_active":"2023-11-28T03:07:47.328229+0000","last_became_peered":"2023-11-28T03:07:47.328229+0000","last_unstale":"2023-11-28T03:07:47.696623+0000","last_undegraded":"2023-11-28T03:07:47.696623+0000","last_fullsized":"2023-11-28T03:07:47.696623+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696146+0000","last_change":"2023-11-28T03:07:47.696146+0000","last_active":"2023-11-28T03:07:47.696146+0000","last_peered":"2023-11-28T03:07:47.696146+0000","last_clean":"2023-11-28T03:07:47.696146+0000","last_became_active":"2023-11-28T03:07:47.695865+0000","last_became_peered":"2023-11-28T03:07:47.695865+0000","last_unstale":"2023-11-28T03:07:47.696146+0000","last_undegraded":"2023-11-28T03:07:47.696146+0000","last_fullsized":"2023-11-28T03:07:47.696146+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326108+0000","last_change":"2023-11-28T03:07:47.322817+0000","last_active":"2023-11-28T03:07:47.326108+0000","last_peered":"2023-11-28T03:07:47.326108+0000","last_clean":"2023-11-28T03:07:47.326108+0000","last_became_active":"2023-11-28T03:07:47.319973+0000","last_became_peered":"2023-11-28T03:07:47.319973+0000","last_unstale":"2023-11-28T03:07:47.326108+0000","last_undegraded":"2023-11-28T03:07:47.326108+0000","last_fullsized":"2023-11-28T03:07:47.326108+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.690285+0000","last_change":"2023-11-28T03:07:47.690285+0000","last_active":"2023-11-28T03:07:47.690285+0000","last_peered":"2023-11-28T03:07:47.690285+0000","last_clean":"2023-11-28T03:07:47.690285+0000","last_became_active":"2023-11-28T03:07:47.690005+0000","last_became_peered":"2023-11-28T03:07:47.690005+0000","last_unstale":"2023-11-28T03:07:47.690285+0000","last_undegraded":"2023-11-28T03:07:47.690285+0000","last_fullsized":"2023-11-28T03:07:47.690285+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330880+0000","last_change":"2023-11-28T03:07:47.328937+0000","last_active":"2023-11-28T03:07:47.330880+0000","last_peered":"2023-11-28T03:07:47.330880+0000","last_clean":"2023-11-28T03:07:47.330880+0000","last_became_active":"2023-11-28T03:07:47.328627+0000","last_became_peered":"2023-11-28T03:07:47.328627+0000","last_unstale":"2023-11-28T03:07:47.330880+0000","last_undegraded":"2023-11-28T03:07:47.330880+0000","last_fullsized":"2023-11-28T03:07:47.330880+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692839+0000","last_change":"2023-11-28T03:07:47.692839+0000","last_active":"2023-11-28T03:07:47.692839+0000","last_peered":"2023-11-28T03:07:47.692839+0000","last_clean":"2023-11-28T03:07:47.692839+0000","last_became_active":"2023-11-28T03:07:47.692456+0000","last_became_peered":"2023-11-28T03:07:47.692456+0000","last_unstale":"2023-11-28T03:07:47.692839+0000","last_undegraded":"2023-11-28T03:07:47.692839+0000","last_fullsized":"2023-11-28T03:07:47.692839+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336075+0000","last_change":"2023-11-28T03:07:47.331231+0000","last_active":"2023-11-28T03:07:47.336075+0000","last_peered":"2023-11-28T03:07:47.336075+0000","last_clean":"2023-11-28T03:07:47.336075+0000","last_became_active":"2023-11-28T03:07:47.330926+0000","last_became_peered":"2023-11-28T03:07:47.330926+0000","last_unstale":"2023-11-28T03:07:47.336075+0000","last_undegraded":"2023-11-28T03:07:47.336075+0000","last_fullsized":"2023-11-28T03:07:47.336075+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.684747+0000","last_change":"2023-11-28T03:07:47.684747+0000","last_active":"2023-11-28T03:07:47.684747+0000","last_peered":"2023-11-28T03:07:47.684747+0000","last_clean":"2023-11-28T03:07:47.684747+0000","last_became_active":"2023-11-28T03:07:47.684471+0000","last_became_peered":"2023-11-28T03:07:47.684471+0000","last_unstale":"2023-11-28T03:07:47.684747+0000","last_undegraded":"2023-11-28T03:07:47.684747+0000","last_fullsized":"2023-11-28T03:07:47.684747+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327014+0000","last_change":"2023-11-28T03:07:47.324387+0000","last_active":"2023-11-28T03:07:47.327014+0000","last_peered":"2023-11-28T03:07:47.327014+0000","last_clean":"2023-11-28T03:07:47.327014+0000","last_became_active":"2023-11-28T03:07:47.324039+0000","last_became_peered":"2023-11-28T03:07:47.324039+0000","last_unstale":"2023-11-28T03:07:47.327014+0000","last_undegraded":"2023-11-28T03:07:47.327014+0000","last_fullsized":"2023-11-28T03:07:47.327014+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687116+0000","last_change":"2023-11-28T03:07:47.687116+0000","last_active":"2023-11-28T03:07:47.687116+0000","last_peered":"2023-11-28T03:07:47.687116+0000","last_clean":"2023-11-28T03:07:47.687116+0000","last_became_active":"2023-11-28T03:07:47.686736+0000","last_became_peered":"2023-11-28T03:07:47.686736+0000","last_unstale":"2023-11-28T03:07:47.687116+0000","last_undegraded":"2023-11-28T03:07:47.687116+0000","last_fullsized":"2023-11-28T03:07:47.687116+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324620+0000","last_change":"2023-11-28T03:07:47.316401+0000","last_active":"2023-11-28T03:07:47.324620+0000","last_peered":"2023-11-28T03:07:47.324620+0000","last_clean":"2023-11-28T03:07:47.324620+0000","last_became_active":"2023-11-28T03:07:47.315681+0000","last_became_peered":"2023-11-28T03:07:47.315681+0000","last_unstale":"2023-11-28T03:07:47.324620+0000","last_undegraded":"2023-11-28T03:07:47.324620+0000","last_fullsized":"2023-11-28T03:07:47.324620+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.696655+0000","last_change":"2023-11-28T03:07:47.696655+0000","last_active":"2023-11-28T03:07:47.696655+0000","last_peered":"2023-11-28T03:07:47.696655+0000","last_clean":"2023-11-28T03:07:47.696655+0000","last_became_active":"2023-11-28T03:07:47.695841+0000","last_became_peered":"2023-11-28T03:07:47.695841+0000","last_unstale":"2023-11-28T03:07:47.696655+0000","last_undegraded":"2023-11-28T03:07:47.696655+0000","last_fullsized":"2023-11-28T03:07:47.696655+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330320+0000","last_change":"2023-11-28T03:07:47.326305+0000","last_active":"2023-11-28T03:07:47.330320+0000","last_peered":"2023-11-28T03:07:47.330320+0000","last_clean":"2023-11-28T03:07:47.330320+0000","last_became_active":"2023-11-28T03:07:47.326021+0000","last_became_peered":"2023-11-28T03:07:47.326021+0000","last_unstale":"2023-11-28T03:07:47.330320+0000","last_undegraded":"2023-11-28T03:07:47.330320+0000","last_fullsized":"2023-11-28T03:07:47.330320+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.699740+0000","last_change":"2023-11-28T03:07:47.699740+0000","last_active":"2023-11-28T03:07:47.699740+0000","last_peered":"2023-11-28T03:07:47.699740+0000","last_clean":"2023-11-28T03:07:47.699740+0000","last_became_active":"2023-11-28T03:07:47.699374+0000","last_became_peered":"2023-11-28T03:07:47.699374+0000","last_unstale":"2023-11-28T03:07:47.699740+0000","last_undegraded":"2023-11-28T03:07:47.699740+0000","last_fullsized":"2023-11-28T03:07:47.699740+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.329535+0000","last_change":"2023-11-28T03:07:47.322477+0000","last_active":"2023-11-28T03:07:47.329535+0000","last_peered":"2023-11-28T03:07:47.329535+0000","last_clean":"2023-11-28T03:07:47.329535+0000","last_became_active":"2023-11-28T03:07:47.322220+0000","last_became_peered":"2023-11-28T03:07:47.322220+0000","last_unstale":"2023-11-28T03:07:47.329535+0000","last_undegraded":"2023-11-28T03:07:47.329535+0000","last_fullsized":"2023-11-28T03:07:47.329535+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693094+0000","last_change":"2023-11-28T03:07:47.693094+0000","last_active":"2023-11-28T03:07:47.693094+0000","last_peered":"2023-11-28T03:07:47.693094+0000","last_clean":"2023-11-28T03:07:47.693094+0000","last_became_active":"2023-11-28T03:07:47.692826+0000","last_became_peered":"2023-11-28T03:07:47.692826+0000","last_unstale":"2023-11-28T03:07:47.693094+0000","last_undegraded":"2023-11-28T03:07:47.693094+0000","last_fullsized":"2023-11-28T03:07:47.693094+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"95'3","reported_seq":12,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695227+0000","last_change":"2023-11-28T03:07:47.322767+0000","last_active":"2023-11-28T03:07:47.695227+0000","last_peered":"2023-11-28T03:07:47.695227+0000","last_clean":"2023-11-28T03:07:47.695227+0000","last_became_active":"2023-11-28T03:07:47.322282+0000","last_became_peered":"2023-11-28T03:07:47.322282+0000","last_unstale":"2023-11-28T03:07:47.695227+0000","last_undegraded":"2023-11-28T03:07:47.695227+0000","last_fullsized":"2023-11-28T03:07:47.695227+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698607+0000","last_change":"2023-11-28T03:07:47.698607+0000","last_active":"2023-11-28T03:07:47.698607+0000","last_peered":"2023-11-28T03:07:47.698607+0000","last_clean":"2023-11-28T03:07:47.698607+0000","last_became_active":"2023-11-28T03:07:47.697698+0000","last_became_peered":"2023-11-28T03:07:47.697698+0000","last_unstale":"2023-11-28T03:07:47.698607+0000","last_undegraded":"2023-11-28T03:07:47.698607+0000","last_fullsized":"2023-11-28T03:07:47.698607+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.14","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.694731+0000","last_change":"2023-11-28T03:07:47.327445+0000","last_active":"2023-11-28T03:07:47.694731+0000","last_peered":"2023-11-28T03:07:47.694731+0000","last_clean":"2023-11-28T03:07:47.694731+0000","last_became_active":"2023-11-28T03:07:47.327157+0000","last_became_peered":"2023-11-28T03:07:47.327157+0000","last_unstale":"2023-11-28T03:07:47.694731+0000","last_undegraded":"2023-11-28T03:07:47.694731+0000","last_fullsized":"2023-11-28T03:07:47.694731+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698416+0000","last_change":"2023-11-28T03:07:47.698416+0000","last_active":"2023-11-28T03:07:47.698416+0000","last_peered":"2023-11-28T03:07:47.698416+0000","last_clean":"2023-11-28T03:07:47.698416+0000","last_became_active":"2023-11-28T03:07:47.697773+0000","last_became_peered":"2023-11-28T03:07:47.697773+0000","last_unstale":"2023-11-28T03:07:47.698416+0000","last_undegraded":"2023-11-28T03:07:47.698416+0000","last_fullsized":"2023-11-28T03:07:47.698416+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.13","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685350+0000","last_change":"2023-11-28T03:07:47.320276+0000","last_active":"2023-11-28T03:07:47.685350+0000","last_peered":"2023-11-28T03:07:47.685350+0000","last_clean":"2023-11-28T03:07:47.685350+0000","last_became_active":"2023-11-28T03:07:47.320004+0000","last_became_peered":"2023-11-28T03:07:47.320004+0000","last_unstale":"2023-11-28T03:07:47.685350+0000","last_undegraded":"2023-11-28T03:07:47.685350+0000","last_fullsized":"2023-11-28T03:07:47.685350+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692503+0000","last_change":"2023-11-28T03:07:47.692503+0000","last_active":"2023-11-28T03:07:47.692503+0000","last_peered":"2023-11-28T03:07:47.692503+0000","last_clean":"2023-11-28T03:07:47.692503+0000","last_became_active":"2023-11-28T03:07:47.692205+0000","last_became_peered":"2023-11-28T03:07:47.692205+0000","last_unstale":"2023-11-28T03:07:47.692503+0000","last_undegraded":"2023-11-28T03:07:47.692503+0000","last_fullsized":"2023-11-28T03:07:47.692503+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.336212+0000","last_change":"2023-11-28T03:07:47.331587+0000","last_active":"2023-11-28T03:07:47.336212+0000","last_peered":"2023-11-28T03:07:47.336212+0000","last_clean":"2023-11-28T03:07:47.336212+0000","last_became_active":"2023-11-28T03:07:47.331293+0000","last_became_peered":"2023-11-28T03:07:47.331293+0000","last_unstale":"2023-11-28T03:07:47.336212+0000","last_undegraded":"2023-11-28T03:07:47.336212+0000","last_fullsized":"2023-11-28T03:07:47.336212+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681629+0000","last_change":"2023-11-28T03:07:47.681629+0000","last_active":"2023-11-28T03:07:47.681629+0000","last_peered":"2023-11-28T03:07:47.681629+0000","last_clean":"2023-11-28T03:07:47.681629+0000","last_became_active":"2023-11-28T03:07:47.681148+0000","last_became_peered":"2023-11-28T03:07:47.681148+0000","last_unstale":"2023-11-28T03:07:47.681629+0000","last_undegraded":"2023-11-28T03:07:47.681629+0000","last_fullsized":"2023-11-28T03:07:47.681629+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.324743+0000","last_change":"2023-11-28T03:07:47.319585+0000","last_active":"2023-11-28T03:07:47.324743+0000","last_peered":"2023-11-28T03:07:47.324743+0000","last_clean":"2023-11-28T03:07:47.324743+0000","last_became_active":"2023-11-28T03:07:47.319302+0000","last_became_peered":"2023-11-28T03:07:47.319302+0000","last_unstale":"2023-11-28T03:07:47.324743+0000","last_undegraded":"2023-11-28T03:07:47.324743+0000","last_fullsized":"2023-11-28T03:07:47.324743+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686244+0000","last_change":"2023-11-28T03:07:47.686244+0000","last_active":"2023-11-28T03:07:47.686244+0000","last_peered":"2023-11-28T03:07:47.686244+0000","last_clean":"2023-11-28T03:07:47.686244+0000","last_became_active":"2023-11-28T03:07:47.685946+0000","last_became_peered":"2023-11-28T03:07:47.685946+0000","last_unstale":"2023-11-28T03:07:47.686244+0000","last_undegraded":"2023-11-28T03:07:47.686244+0000","last_fullsized":"2023-11-28T03:07:47.686244+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"95'4","reported_seq":13,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695767+0000","last_change":"2023-11-28T03:07:47.321334+0000","last_active":"2023-11-28T03:07:47.695767+0000","last_peered":"2023-11-28T03:07:47.695767+0000","last_clean":"2023-11-28T03:07:47.695767+0000","last_became_active":"2023-11-28T03:07:47.321043+0000","last_became_peered":"2023-11-28T03:07:47.321043+0000","last_unstale":"2023-11-28T03:07:47.695767+0000","last_undegraded":"2023-11-28T03:07:47.695767+0000","last_fullsized":"2023-11-28T03:07:47.695767+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.682764+0000","last_change":"2023-11-28T03:07:47.682764+0000","last_active":"2023-11-28T03:07:47.682764+0000","last_peered":"2023-11-28T03:07:47.682764+0000","last_clean":"2023-11-28T03:07:47.682764+0000","last_became_active":"2023-11-28T03:07:47.682093+0000","last_became_peered":"2023-11-28T03:07:47.682093+0000","last_unstale":"2023-11-28T03:07:47.682764+0000","last_undegraded":"2023-11-28T03:07:47.682764+0000","last_fullsized":"2023-11-28T03:07:47.682764+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.7","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.697796+0000","last_change":"2023-11-28T03:07:47.324776+0000","last_active":"2023-11-28T03:07:47.697796+0000","last_peered":"2023-11-28T03:07:47.697796+0000","last_clean":"2023-11-28T03:07:47.697796+0000","last_became_active":"2023-11-28T03:07:47.324494+0000","last_became_peered":"2023-11-28T03:07:47.324494+0000","last_unstale":"2023-11-28T03:07:47.697796+0000","last_undegraded":"2023-11-28T03:07:47.697796+0000","last_fullsized":"2023-11-28T03:07:47.697796+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686340+0000","last_change":"2023-11-28T03:07:47.686340+0000","last_active":"2023-11-28T03:07:47.686340+0000","last_peered":"2023-11-28T03:07:47.686340+0000","last_clean":"2023-11-28T03:07:47.686340+0000","last_became_active":"2023-11-28T03:07:47.684496+0000","last_became_peered":"2023-11-28T03:07:47.684496+0000","last_unstale":"2023-11-28T03:07:47.686340+0000","last_undegraded":"2023-11-28T03:07:47.686340+0000","last_fullsized":"2023-11-28T03:07:47.686340+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320392+0000","last_change":"2023-11-28T03:07:47.317497+0000","last_active":"2023-11-28T03:07:47.320392+0000","last_peered":"2023-11-28T03:07:47.320392+0000","last_clean":"2023-11-28T03:07:47.320392+0000","last_became_active":"2023-11-28T03:07:47.317147+0000","last_became_peered":"2023-11-28T03:07:47.317147+0000","last_unstale":"2023-11-28T03:07:47.320392+0000","last_undegraded":"2023-11-28T03:07:47.320392+0000","last_fullsized":"2023-11-28T03:07:47.320392+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.686190+0000","last_change":"2023-11-28T03:07:47.686190+0000","last_active":"2023-11-28T03:07:47.686190+0000","last_peered":"2023-11-28T03:07:47.686190+0000","last_clean":"2023-11-28T03:07:47.686190+0000","last_became_active":"2023-11-28T03:07:47.685887+0000","last_became_peered":"2023-11-28T03:07:47.685887+0000","last_unstale":"2023-11-28T03:07:47.686190+0000","last_undegraded":"2023-11-28T03:07:47.686190+0000","last_fullsized":"2023-11-28T03:07:47.686190+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330435+0000","last_change":"2023-11-28T03:07:47.327833+0000","last_active":"2023-11-28T03:07:47.330435+0000","last_peered":"2023-11-28T03:07:47.330435+0000","last_clean":"2023-11-28T03:07:47.330435+0000","last_became_active":"2023-11-28T03:07:47.326683+0000","last_became_peered":"2023-11-28T03:07:47.326683+0000","last_unstale":"2023-11-28T03:07:47.330435+0000","last_undegraded":"2023-11-28T03:07:47.330435+0000","last_fullsized":"2023-11-28T03:07:47.330435+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693950+0000","last_change":"2023-11-28T03:07:47.693950+0000","last_active":"2023-11-28T03:07:47.693950+0000","last_peered":"2023-11-28T03:07:47.693950+0000","last_clean":"2023-11-28T03:07:47.693950+0000","last_became_active":"2023-11-28T03:07:47.693766+0000","last_became_peered":"2023-11-28T03:07:47.693766+0000","last_unstale":"2023-11-28T03:07:47.693950+0000","last_undegraded":"2023-11-28T03:07:47.693950+0000","last_fullsized":"2023-11-28T03:07:47.693950+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.330407+0000","last_change":"2023-11-28T03:07:47.327903+0000","last_active":"2023-11-28T03:07:47.330407+0000","last_peered":"2023-11-28T03:07:47.330407+0000","last_clean":"2023-11-28T03:07:47.330407+0000","last_became_active":"2023-11-28T03:07:47.327616+0000","last_became_peered":"2023-11-28T03:07:47.327616+0000","last_unstale":"2023-11-28T03:07:47.330407+0000","last_undegraded":"2023-11-28T03:07:47.330407+0000","last_fullsized":"2023-11-28T03:07:47.330407+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.687101+0000","last_change":"2023-11-28T03:07:47.687101+0000","last_active":"2023-11-28T03:07:47.687101+0000","last_peered":"2023-11-28T03:07:47.687101+0000","last_clean":"2023-11-28T03:07:47.687101+0000","last_became_active":"2023-11-28T03:07:47.683927+0000","last_became_peered":"2023-11-28T03:07:47.683927+0000","last_unstale":"2023-11-28T03:07:47.687101+0000","last_undegraded":"2023-11-28T03:07:47.687101+0000","last_fullsized":"2023-11-28T03:07:47.687101+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325839+0000","last_change":"2023-11-28T03:07:47.321990+0000","last_active":"2023-11-28T03:07:47.325839+0000","last_peered":"2023-11-28T03:07:47.325839+0000","last_clean":"2023-11-28T03:07:47.325839+0000","last_became_active":"2023-11-28T03:07:47.321622+0000","last_became_peered":"2023-11-28T03:07:47.321622+0000","last_unstale":"2023-11-28T03:07:47.325839+0000","last_undegraded":"2023-11-28T03:07:47.325839+0000","last_fullsized":"2023-11-28T03:07:47.325839+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.681098+0000","last_change":"2023-11-28T03:07:47.681098+0000","last_active":"2023-11-28T03:07:47.681098+0000","last_peered":"2023-11-28T03:07:47.681098+0000","last_clean":"2023-11-28T03:07:47.681098+0000","last_became_active":"2023-11-28T03:07:47.680736+0000","last_became_peered":"2023-11-28T03:07:47.680736+0000","last_unstale":"2023-11-28T03:07:47.681098+0000","last_undegraded":"2023-11-28T03:07:47.681098+0000","last_fullsized":"2023-11-28T03:07:47.681098+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.325337+0000","last_change":"2023-11-28T03:07:47.319993+0000","last_active":"2023-11-28T03:07:47.325337+0000","last_peered":"2023-11-28T03:07:47.325337+0000","last_clean":"2023-11-28T03:07:47.325337+0000","last_became_active":"2023-11-28T03:07:47.319678+0000","last_became_peered":"2023-11-28T03:07:47.319678+0000","last_unstale":"2023-11-28T03:07:47.325337+0000","last_undegraded":"2023-11-28T03:07:47.325337+0000","last_fullsized":"2023-11-28T03:07:47.325337+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693342+0000","last_change":"2023-11-28T03:07:47.693342+0000","last_active":"2023-11-28T03:07:47.693342+0000","last_peered":"2023-11-28T03:07:47.693342+0000","last_clean":"2023-11-28T03:07:47.693342+0000","last_became_active":"2023-11-28T03:07:47.693057+0000","last_became_peered":"2023-11-28T03:07:47.693057+0000","last_unstale":"2023-11-28T03:07:47.693342+0000","last_undegraded":"2023-11-28T03:07:47.693342+0000","last_fullsized":"2023-11-28T03:07:47.693342+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.326639+0000","last_change":"2023-11-28T03:07:47.324185+0000","last_active":"2023-11-28T03:07:47.326639+0000","last_peered":"2023-11-28T03:07:47.326639+0000","last_clean":"2023-11-28T03:07:47.326639+0000","last_became_active":"2023-11-28T03:07:47.323376+0000","last_became_peered":"2023-11-28T03:07:47.323376+0000","last_unstale":"2023-11-28T03:07:47.326639+0000","last_undegraded":"2023-11-28T03:07:47.326639+0000","last_fullsized":"2023-11-28T03:07:47.326639+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.692545+0000","last_change":"2023-11-28T03:07:47.692545+0000","last_active":"2023-11-28T03:07:47.692545+0000","last_peered":"2023-11-28T03:07:47.692545+0000","last_clean":"2023-11-28T03:07:47.692545+0000","last_became_active":"2023-11-28T03:07:47.692281+0000","last_became_peered":"2023-11-28T03:07:47.692281+0000","last_unstale":"2023-11-28T03:07:47.692545+0000","last_undegraded":"2023-11-28T03:07:47.692545+0000","last_fullsized":"2023-11-28T03:07:47.692545+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.323367+0000","last_change":"2023-11-28T03:07:47.321117+0000","last_active":"2023-11-28T03:07:47.323367+0000","last_peered":"2023-11-28T03:07:47.323367+0000","last_clean":"2023-11-28T03:07:47.323367+0000","last_became_active":"2023-11-28T03:07:47.320851+0000","last_became_peered":"2023-11-28T03:07:47.320851+0000","last_unstale":"2023-11-28T03:07:47.323367+0000","last_undegraded":"2023-11-28T03:07:47.323367+0000","last_fullsized":"2023-11-28T03:07:47.323367+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.688168+0000","last_change":"2023-11-28T03:07:47.688168+0000","last_active":"2023-11-28T03:07:47.688168+0000","last_peered":"2023-11-28T03:07:47.688168+0000","last_clean":"2023-11-28T03:07:47.688168+0000","last_became_active":"2023-11-28T03:07:47.687905+0000","last_became_peered":"2023-11-28T03:07:47.687905+0000","last_unstale":"2023-11-28T03:07:47.688168+0000","last_undegraded":"2023-11-28T03:07:47.688168+0000","last_fullsized":"2023-11-28T03:07:47.688168+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327396+0000","last_change":"2023-11-28T03:07:47.316377+0000","last_active":"2023-11-28T03:07:47.327396+0000","last_peered":"2023-11-28T03:07:47.327396+0000","last_clean":"2023-11-28T03:07:47.327396+0000","last_became_active":"2023-11-28T03:07:47.316098+0000","last_became_peered":"2023-11-28T03:07:47.316098+0000","last_unstale":"2023-11-28T03:07:47.327396+0000","last_undegraded":"2023-11-28T03:07:47.327396+0000","last_fullsized":"2023-11-28T03:07:47.327396+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.317733+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:07:47.317733+0000","last_peered":"2023-11-28T03:07:47.317733+0000","last_clean":"2023-11-28T03:07:47.317733+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:07:47.317733+0000","last_undegraded":"2023-11-28T03:07:47.317733+0000","last_fullsized":"2023-11-28T03:07:47.317733+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.701326+0000","last_change":"2023-11-28T03:07:47.701326+0000","last_active":"2023-11-28T03:07:47.701326+0000","last_peered":"2023-11-28T03:07:47.701326+0000","last_clean":"2023-11-28T03:07:47.701326+0000","last_became_active":"2023-11-28T03:07:47.701065+0000","last_became_peered":"2023-11-28T03:07:47.701065+0000","last_unstale":"2023-11-28T03:07:47.701326+0000","last_undegraded":"2023-11-28T03:07:47.701326+0000","last_fullsized":"2023-11-28T03:07:47.701326+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.b","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.680884+0000","last_change":"2023-11-28T03:07:47.319788+0000","last_active":"2023-11-28T03:07:47.680884+0000","last_peered":"2023-11-28T03:07:47.680884+0000","last_clean":"2023-11-28T03:07:47.680884+0000","last_became_active":"2023-11-28T03:07:47.318830+0000","last_became_peered":"2023-11-28T03:07:47.318830+0000","last_unstale":"2023-11-28T03:07:47.680884+0000","last_undegraded":"2023-11-28T03:07:47.680884+0000","last_fullsized":"2023-11-28T03:07:47.680884+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.315933+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:07:47.315933+0000","last_peered":"2023-11-28T03:07:47.315933+0000","last_clean":"2023-11-28T03:07:47.315933+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:07:47.315933+0000","last_undegraded":"2023-11-28T03:07:47.315933+0000","last_fullsized":"2023-11-28T03:07:47.315933+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.698646+0000","last_change":"2023-11-28T03:07:47.698646+0000","last_active":"2023-11-28T03:07:47.698646+0000","last_peered":"2023-11-28T03:07:47.698646+0000","last_clean":"2023-11-28T03:07:47.698646+0000","last_became_active":"2023-11-28T03:07:47.698365+0000","last_became_peered":"2023-11-28T03:07:47.698365+0000","last_unstale":"2023-11-28T03:07:47.698646+0000","last_undegraded":"2023-11-28T03:07:47.698646+0000","last_fullsized":"2023-11-28T03:07:47.698646+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333781+0000","last_change":"2023-11-28T03:07:47.325927+0000","last_active":"2023-11-28T03:07:47.333781+0000","last_peered":"2023-11-28T03:07:47.333781+0000","last_clean":"2023-11-28T03:07:47.333781+0000","last_became_active":"2023-11-28T03:07:47.325640+0000","last_became_peered":"2023-11-28T03:07:47.325640+0000","last_unstale":"2023-11-28T03:07:47.333781+0000","last_undegraded":"2023-11-28T03:07:47.333781+0000","last_fullsized":"2023-11-28T03:07:47.333781+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":77,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.309649+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:07:47.309649+0000","last_peered":"2023-11-28T03:07:47.309649+0000","last_clean":"2023-11-28T03:07:47.309649+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:07:47.309649+0000","last_undegraded":"2023-11-28T03:07:47.309649+0000","last_fullsized":"2023-11-28T03:07:47.309649+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.695982+0000","last_change":"2023-11-28T03:07:47.695982+0000","last_active":"2023-11-28T03:07:47.695982+0000","last_peered":"2023-11-28T03:07:47.695982+0000","last_clean":"2023-11-28T03:07:47.695982+0000","last_became_active":"2023-11-28T03:07:47.695674+0000","last_became_peered":"2023-11-28T03:07:47.695674+0000","last_unstale":"2023-11-28T03:07:47.695982+0000","last_undegraded":"2023-11-28T03:07:47.695982+0000","last_fullsized":"2023-11-28T03:07:47.695982+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333790+0000","last_change":"2023-11-28T03:07:47.325980+0000","last_active":"2023-11-28T03:07:47.333790+0000","last_peered":"2023-11-28T03:07:47.333790+0000","last_clean":"2023-11-28T03:07:47.333790+0000","last_became_active":"2023-11-28T03:07:47.325630+0000","last_became_peered":"2023-11-28T03:07:47.325630+0000","last_unstale":"2023-11-28T03:07:47.333790+0000","last_undegraded":"2023-11-28T03:07:47.333790+0000","last_fullsized":"2023-11-28T03:07:47.333790+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.312218+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:07:47.312218+0000","last_peered":"2023-11-28T03:07:47.312218+0000","last_clean":"2023-11-28T03:07:47.312218+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:07:47.312218+0000","last_undegraded":"2023-11-28T03:07:47.312218+0000","last_fullsized":"2023-11-28T03:07:47.312218+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.320245+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:07:47.320245+0000","last_peered":"2023-11-28T03:07:47.320245+0000","last_clean":"2023-11-28T03:07:47.320245+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:07:47.320245+0000","last_undegraded":"2023-11-28T03:07:47.320245+0000","last_fullsized":"2023-11-28T03:07:47.320245+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689443+0000","last_change":"2023-11-28T03:07:47.689443+0000","last_active":"2023-11-28T03:07:47.689443+0000","last_peered":"2023-11-28T03:07:47.689443+0000","last_clean":"2023-11-28T03:07:47.689443+0000","last_became_active":"2023-11-28T03:07:47.689185+0000","last_became_peered":"2023-11-28T03:07:47.689185+0000","last_unstale":"2023-11-28T03:07:47.689443+0000","last_undegraded":"2023-11-28T03:07:47.689443+0000","last_fullsized":"2023-11-28T03:07:47.689443+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.333823+0000","last_change":"2023-11-28T03:07:47.327511+0000","last_active":"2023-11-28T03:07:47.333823+0000","last_peered":"2023-11-28T03:07:47.333823+0000","last_clean":"2023-11-28T03:07:47.333823+0000","last_became_active":"2023-11-28T03:07:47.325928+0000","last_became_peered":"2023-11-28T03:07:47.325928+0000","last_unstale":"2023-11-28T03:07:47.333823+0000","last_undegraded":"2023-11-28T03:07:47.333823+0000","last_fullsized":"2023-11-28T03:07:47.333823+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.318995+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:07:47.318995+0000","last_peered":"2023-11-28T03:07:47.318995+0000","last_clean":"2023-11-28T03:07:47.318995+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:07:47.318995+0000","last_undegraded":"2023-11-28T03:07:47.318995+0000","last_fullsized":"2023-11-28T03:07:47.318995+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691626+0000","last_change":"2023-11-28T03:07:47.691626+0000","last_active":"2023-11-28T03:07:47.691626+0000","last_peered":"2023-11-28T03:07:47.691626+0000","last_clean":"2023-11-28T03:07:47.691626+0000","last_became_active":"2023-11-28T03:07:47.691354+0000","last_became_peered":"2023-11-28T03:07:47.691354+0000","last_unstale":"2023-11-28T03:07:47.691626+0000","last_undegraded":"2023-11-28T03:07:47.691626+0000","last_fullsized":"2023-11-28T03:07:47.691626+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":9,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.327923+0000","last_change":"2023-11-28T03:07:47.325183+0000","last_active":"2023-11-28T03:07:47.327923+0000","last_peered":"2023-11-28T03:07:47.327923+0000","last_clean":"2023-11-28T03:07:47.327923+0000","last_became_active":"2023-11-28T03:07:47.324891+0000","last_became_peered":"2023-11-28T03:07:47.324891+0000","last_unstale":"2023-11-28T03:07:47.327923+0000","last_undegraded":"2023-11-28T03:07:47.327923+0000","last_fullsized":"2023-11-28T03:07:47.327923+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":74,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.319402+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:07:47.319402+0000","last_peered":"2023-11-28T03:07:47.319402+0000","last_clean":"2023-11-28T03:07:47.319402+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:07:47.319402+0000","last_undegraded":"2023-11-28T03:07:47.319402+0000","last_fullsized":"2023-11-28T03:07:47.319402+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.691585+0000","last_change":"2023-11-28T03:07:47.691585+0000","last_active":"2023-11-28T03:07:47.691585+0000","last_peered":"2023-11-28T03:07:47.691585+0000","last_clean":"2023-11-28T03:07:47.691585+0000","last_became_active":"2023-11-28T03:07:47.691314+0000","last_became_peered":"2023-11-28T03:07:47.691314+0000","last_unstale":"2023-11-28T03:07:47.691585+0000","last_undegraded":"2023-11-28T03:07:47.691585+0000","last_fullsized":"2023-11-28T03:07:47.691585+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"95'1","reported_seq":10,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.689186+0000","last_change":"2023-11-28T03:07:47.324365+0000","last_active":"2023-11-28T03:07:47.689186+0000","last_peered":"2023-11-28T03:07:47.689186+0000","last_clean":"2023-11-28T03:07:47.689186+0000","last_became_active":"2023-11-28T03:07:47.324073+0000","last_became_peered":"2023-11-28T03:07:47.324073+0000","last_unstale":"2023-11-28T03:07:47.689186+0000","last_undegraded":"2023-11-28T03:07:47.689186+0000","last_fullsized":"2023-11-28T03:07:47.689186+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":73,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.317635+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:07:47.317635+0000","last_peered":"2023-11-28T03:07:47.317635+0000","last_clean":"2023-11-28T03:07:47.317635+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:07:47.317635+0000","last_undegraded":"2023-11-28T03:07:47.317635+0000","last_fullsized":"2023-11-28T03:07:47.317635+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":8,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.685725+0000","last_change":"2023-11-28T03:07:47.685725+0000","last_active":"2023-11-28T03:07:47.685725+0000","last_peered":"2023-11-28T03:07:47.685725+0000","last_clean":"2023-11-28T03:07:47.685725+0000","last_became_active":"2023-11-28T03:07:47.685393+0000","last_became_peered":"2023-11-28T03:07:47.685393+0000","last_unstale":"2023-11-28T03:07:47.685725+0000","last_undegraded":"2023-11-28T03:07:47.685725+0000","last_fullsized":"2023-11-28T03:07:47.685725+0000","mapping_epoch":94,"log_start":"0'0","ondisk_log_start":"0'0","created":94,"last_epoch_clean":95,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:47.294269+0000","last_clean_scrub_stamp":"2023-11-28T03:07:47.294269+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.e","version":"95'2","reported_seq":11,"reported_epoch":95,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.693197+0000","last_change":"2023-11-28T03:07:47.324769+0000","last_active":"2023-11-28T03:07:47.693197+0000","last_peered":"2023-11-28T03:07:47.693197+0000","last_clean":"2023-11-28T03:07:47.693197+0000","last_became_active":"2023-11-28T03:07:47.324249+0000","last_became_peered":"2023-11-28T03:07:47.324249+0000","last_unstale":"2023-11-28T03:07:47.693197+0000","last_undegraded":"2023-11-28T03:07:47.693197+0000","last_fullsized":"2023-11-28T03:07:47.693197+0000","mapping_epoch":93,"log_start":"0'0","ondisk_log_start":"0'0","created":93,"last_epoch_clean":94,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:07:46.287425+0000","last_clean_scrub_stamp":"2023-11-28T03:07:46.287425+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":83,"reported_epoch":94,"state":"active+clean","last_fresh":"2023-11-28T03:07:47.311570+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:07:47.311570+0000","last_peered":"2023-11-28T03:07:47.311570+0000","last_clean":"2023-11-28T03:07:47.311570+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:07:47.311570+0000","last_undegraded":"2023-11-28T03:07:47.311570+0000","last_fullsized":"2023-11-28T03:07:47.311570+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":12,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575030,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":1146880,"data_stored":567427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.59599999999999997,"15min":0.59299999999999997},"min":{"1min":0.40600000000000003,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77900000000000003,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.45100000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.628}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.36699999999999999,"5min":0.33600000000000002,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.55400000000000005},"min":{"1min":0.40300000000000002,"5min":0.36399999999999999,"15min":0.34999999999999998},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54000000000000004,"15min":0.54600000000000004},"min":{"1min":0.39300000000000002,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.73899999999999999,"5min":0.749,"15min":0.749},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56499999999999995,"15min":0.56399999999999995},"min":{"1min":0.36499999999999999,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.73999999999999999,"5min":0.79000000000000004,"15min":0.81899999999999995},"last":0.48599999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56399999999999995,"15min":0.56200000000000006},"min":{"1min":0.373,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.77100000000000002},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.441,"5min":0.39200000000000002,"15min":0.34499999999999997},"max":{"1min":0.75600000000000001,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.70399999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.46400000000000002,"15min":0.46500000000000002},"min":{"1min":0.33700000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.68700000000000006,"5min":0.77100000000000002,"15min":0.83099999999999996},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.48099999999999998,"15min":0.48799999999999999},"min":{"1min":0.33300000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.66100000000000003,"5min":0.748,"15min":0.81899999999999995},"last":0.64900000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51200000000000001,"5min":0.502,"15min":0.504},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.755,"5min":0.755,"15min":0.84499999999999997},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53000000000000003,"15min":0.53900000000000003},"min":{"1min":0.38300000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.73699999999999999,"5min":0.77300000000000002,"15min":0.83199999999999996},"last":0.77100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.55000000000000004,"15min":0.56000000000000005},"min":{"1min":0.41399999999999998,"5min":0.375,"15min":0.375},"max":{"1min":0.78400000000000003,"5min":0.79800000000000004,"15min":0.86499999999999999},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55400000000000005,"15min":0.55000000000000004},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.75800000000000001,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575030,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.54300000000000004,"15min":0.54100000000000004},"min":{"1min":0.35399999999999998,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.378},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.56100000000000005},"min":{"1min":0.375,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.40699999999999997,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.75600000000000001,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.441},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.56899999999999995,"15min":0.55300000000000005},"min":{"1min":0.46100000000000002,"5min":0.38,"15min":0.32800000000000001},"max":{"1min":1.125,"5min":1.506,"15min":1.506},"last":0.41799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.56299999999999994,"15min":0.56499999999999995},"min":{"1min":0.39100000000000001,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.78400000000000003,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.59499999999999997,"15min":0.57499999999999996},"min":{"1min":0.53000000000000003,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.77600000000000002,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47199999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.47099999999999997,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.58599999999999997,"15min":0.57799999999999996},"min":{"1min":0.46000000000000002,"5min":0.38800000000000001,"15min":0.36299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.47799999999999998,"15min":0.45500000000000002},"min":{"1min":0.31900000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.67800000000000005,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.49399999999999999,"15min":0.497},"min":{"1min":0.33600000000000002,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.76400000000000001,"5min":0.76400000000000001,"15min":0.82499999999999996},"last":0.38800000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.53100000000000003,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.35999999999999999,"15min":0.309},"max":{"1min":0.755,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.56799999999999995,"15min":0.55100000000000005},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73799999999999999,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.46500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.91100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.58099999999999996,"15min":0.55900000000000005},"min":{"1min":0.35899999999999999,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.78700000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51100000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575029,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.34399999999999997,"15min":0.34100000000000003},"min":{"1min":0.25600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.502,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.38100000000000001,"15min":0.36799999999999999},"min":{"1min":0.28000000000000003,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.46000000000000002,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.372}]},{"osd":2,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.38800000000000001,"15min":0.38},"min":{"1min":0.29599999999999999,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.40999999999999998,"15min":0.39900000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.19800000000000001},"max":{"1min":0.56799999999999995,"5min":0.628,"15min":0.72599999999999998},"last":0.34499999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46100000000000002,"15min":0.45100000000000001},"min":{"1min":0.30499999999999999,"5min":0.26000000000000001,"15min":0.25800000000000001},"max":{"1min":0.53000000000000003,"5min":0.69599999999999995,"15min":0.749},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.434,"15min":0.42899999999999999},"min":{"1min":0.29799999999999999,"5min":0.26400000000000001,"15min":0.26000000000000001},"max":{"1min":0.56299999999999994,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.36199999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.59499999999999997,"15min":0.58899999999999997},"min":{"1min":0.42299999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.751,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.434},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.60299999999999998,"15min":0.59499999999999997},"min":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.66500000000000004,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.39800000000000002,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.70399999999999996,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.65200000000000002,"15min":0.64800000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.77300000000000002,"5min":1.038,"15min":1.111},"last":0.48799999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.59599999999999997,"15min":0.60299999999999998},"min":{"1min":0.40600000000000003,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.63200000000000001,"15min":0.61899999999999999},"min":{"1min":0.436,"5min":0.436,"15min":0.40600000000000003},"max":{"1min":0.90700000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.46899999999999997,"5min":0.42699999999999999,"15min":0.374},"max":{"1min":0.76000000000000001,"5min":1.024,"15min":1.024},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.62,"15min":0.626},"min":{"1min":0.45100000000000001,"5min":0.433,"15min":0.38700000000000001},"max":{"1min":0.73799999999999999,"5min":0.88300000000000001,"15min":1.7929999999999999},"last":0.41399999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575030,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302312,"kb_used_data":1120,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069528,"statfs":{"total":96636764160,"available":96327196672,"internally_reserved":0,"allocated":1146880,"data_stored":567427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35499999999999998,"15min":0.34499999999999997},"min":{"1min":0.245,"5min":0.221,"15min":0.184},"max":{"1min":0.45100000000000001,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.22800000000000001},{"interface":"front","average":{"1min":0.35799999999999998,"5min":0.372,"15min":0.36799999999999999},"min":{"1min":0.251,"5min":0.23799999999999999,"15min":0.23200000000000001},"max":{"1min":0.51700000000000002,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.25600000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.36499999999999999,"15min":0.36899999999999999},"min":{"1min":0.28100000000000003,"5min":0.23100000000000001,"15min":0.20100000000000001},"max":{"1min":0.505,"5min":0.55800000000000005,"15min":0.69699999999999995},"last":0.26900000000000002},{"interface":"front","average":{"1min":0.44,"5min":0.44400000000000001,"15min":0.441},"min":{"1min":0.372,"5min":0.307,"15min":0.26300000000000001},"max":{"1min":0.57599999999999996,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.379,"5min":0.39600000000000002,"15min":0.39200000000000002},"min":{"1min":0.29199999999999998,"5min":0.26200000000000001,"15min":0.24099999999999999},"max":{"1min":0.53200000000000003,"5min":0.60699999999999998,"15min":0.73099999999999998},"last":0.28499999999999998},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.41799999999999998,"15min":0.41699999999999998},"min":{"1min":0.32800000000000001,"5min":0.28100000000000003,"15min":0.251},"max":{"1min":0.55600000000000005,"5min":0.65400000000000003,"15min":0.66500000000000004},"last":0.29499999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.625,"15min":0.61699999999999999},"min":{"1min":0.376,"5min":0.35599999999999998,"15min":0.35399999999999998},"max":{"1min":0.78000000000000003,"5min":0.95899999999999996,"15min":1.0489999999999999},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.65100000000000002,"15min":0.63600000000000001},"min":{"1min":0.433,"5min":0.41799999999999998,"15min":0.39900000000000002},"max":{"1min":0.77600000000000002,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.47799999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.64400000000000002,"15min":0.63400000000000001},"min":{"1min":0.39700000000000002,"5min":0.374,"15min":0.374},"max":{"1min":0.80600000000000005,"5min":1.304,"15min":1.304},"last":0.438},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63500000000000001,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.39000000000000001},"max":{"1min":0.755,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.52300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.60999999999999999,"15min":0.59999999999999998},"min":{"1min":0.443,"5min":0.40000000000000002,"15min":0.38700000000000001},"max":{"1min":0.75600000000000001,"5min":1.679,"15min":1.679},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.435,"5min":0.42599999999999999,"15min":0.35899999999999999},"max":{"1min":0.83499999999999996,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.63600000000000001,"15min":0.626},"min":{"1min":0.44600000000000001,"5min":0.377,"15min":0.36299999999999999},"max":{"1min":0.79200000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.495},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.67900000000000005,"15min":0.66800000000000004},"min":{"1min":0.51300000000000001,"5min":0.44,"15min":0.41699999999999998},"max":{"1min":0.85499999999999998,"5min":1.046,"15min":1.046},"last":0.48999999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.36699999999999999,"15min":0.36599999999999999},"min":{"1min":0.27600000000000002,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.623},"last":0.308},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.40799999999999997,"15min":0.41299999999999998},"min":{"1min":0.32300000000000001,"5min":0.28899999999999998,"15min":0.23599999999999999},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.29299999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.37,"15min":0.36299999999999999},"min":{"1min":0.29299999999999998,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.56100000000000005,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.252},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.39500000000000002,"15min":0.38800000000000001},"min":{"1min":0.28399999999999997,"5min":0.218,"15min":0.218},"max":{"1min":0.70399999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.32700000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.41899999999999998,"15min":0.41399999999999998},"min":{"1min":0.30099999999999999,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.58199999999999996,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.378},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"min":{"1min":0.35599999999999998,"5min":0.26100000000000001,"15min":0.23999999999999999},"max":{"1min":0.64200000000000002,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.58999999999999997,"15min":0.58799999999999997},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.91900000000000004,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.44},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.60399999999999998,"15min":0.60299999999999998},"min":{"1min":0.47899999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.874},"last":0.501}]},{"osd":5,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.64000000000000001,"15min":0.63},"min":{"1min":0.439,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.64300000000000002,"15min":0.64000000000000001},"min":{"1min":0.5,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.94699999999999995,"5min":1.004,"15min":1.004},"last":0.39100000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.54000000000000004,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.91000000000000003,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.65400000000000003,"15min":0.65100000000000002},"min":{"1min":0.47699999999999998,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.90200000000000002,"5min":0.91400000000000003,"15min":0.93999999999999995},"last":0.45300000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:06:54 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.628,"15min":0.628},"min":{"1min":0.55200000000000005,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.85499999999999998,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.628},"min":{"1min":0.52300000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.93200000000000005,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.53100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575030,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.35999999999999999,"15min":0.35599999999999998},"min":{"1min":0.28499999999999998,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.53600000000000003,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.251},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.376,"15min":0.372},"min":{"1min":0.29899999999999999,"5min":0.23300000000000001,"15min":0.218},"max":{"1min":0.59799999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.27400000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.38100000000000001,"15min":0.373},"min":{"1min":0.29899999999999999,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.41499999999999998},"min":{"1min":0.32900000000000001,"5min":0.246,"15min":0.24299999999999999},"max":{"1min":0.622,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.28499999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.41999999999999998,"15min":0.41899999999999998},"min":{"1min":0.32400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.60699999999999998,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.438,"15min":0.435},"min":{"1min":0.36799999999999999,"5min":0.27500000000000002,"15min":0.27300000000000002},"max":{"1min":0.58899999999999997,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.34100000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.59899999999999998,"15min":0.61299999999999999},"min":{"1min":0.432,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.85799999999999998,"5min":1.581,"15min":1.6000000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.62,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.45800000000000002,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.80900000000000005,"5min":0.94199999999999995,"15min":1.196},"last":0.56599999999999995}]},{"osd":5,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.59799999999999998,"15min":0.60999999999999999},"min":{"1min":0.42299999999999999,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.97899999999999998,"15min":1.1439999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82899999999999996,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.51000000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.623,"15min":0.63700000000000001},"min":{"1min":0.48899999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.81699999999999995,"5min":0.95799999999999996,"15min":1.206},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66000000000000003,"5min":0.63500000000000001,"15min":0.64200000000000002},"min":{"1min":0.53400000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.84199999999999997,"5min":1.0409999999999999,"15min":1.1759999999999999},"last":0.61099999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:06:56 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.53200000000000003,"5min":0.37,"15min":0.37},"max":{"1min":0.89900000000000002,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.63300000000000001,"15min":0.65100000000000002},"min":{"1min":0.51500000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.88600000000000001,"5min":0.98899999999999999,"15min":1.2290000000000001},"last":0.51800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.53900000000000003,"15min":0.54200000000000004},"min":{"1min":0.40799999999999997,"5min":0.36599999999999999,"15min":0.27800000000000002},"max":{"1min":0.75800000000000001,"5min":0.77900000000000003,"15min":0.78300000000000003},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.57399999999999995,"15min":0.55600000000000005},"min":{"1min":0.438,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.76000000000000001,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.42699999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.55400000000000005,"15min":0.54900000000000004},"min":{"1min":0.45100000000000001,"5min":0.36799999999999999,"15min":0.314},"max":{"1min":0.73399999999999999,"5min":0.81100000000000005,"15min":0.81499999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56799999999999995,"15min":0.55900000000000005},"min":{"1min":0.35799999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80100000000000005,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56699999999999995,"15min":0.56299999999999994},"min":{"1min":0.41599999999999998,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.68700000000000006,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.56200000000000006,"15min":0.55500000000000005},"min":{"1min":0.44800000000000001,"5min":0.38600000000000001,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.78100000000000003},"last":0.436}]},{"osd":3,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.60499999999999998,"15min":0.60099999999999998},"min":{"1min":0.45700000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.627,"15min":0.625},"min":{"1min":0.47599999999999998,"5min":0.47199999999999998,"15min":0.38500000000000001},"max":{"1min":0.83999999999999997,"5min":0.84599999999999997,"15min":0.878},"last":0.57999999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.48099999999999998,"15min":0.47199999999999998},"min":{"1min":0.33500000000000002,"5min":0.30099999999999999,"15min":0.28699999999999998},"max":{"1min":0.74099999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.502,"5min":0.504,"15min":0.50700000000000001},"min":{"1min":0.33300000000000002,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.74299999999999999,"5min":0.748,"15min":0.77900000000000003},"last":0.38700000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.505,"15min":0.505},"min":{"1min":0.36299999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.82799999999999996,"5min":0.82799999999999996,"15min":0.83899999999999997},"last":0.5},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.53100000000000003,"15min":0.52500000000000002},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.33600000000000002},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.81599999999999995},"last":0.47899999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:13 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.55200000000000005,"15min":0.54100000000000004},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.32900000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.82599999999999996},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57199999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.35499999999999998,"15min":0.32900000000000001},"max":{"1min":0.80700000000000005,"5min":0.83399999999999996,"15min":0.91300000000000003},"last":0.46800000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575031,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302300,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069540,"statfs":{"total":96636764160,"available":96327208960,"internally_reserved":0,"allocated":1134592,"data_stored":560748,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52100000000000002,"15min":0.52100000000000002},"min":{"1min":0.42499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.746,"5min":0.746,"15min":0.79800000000000004},"last":0.621},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.54800000000000004,"15min":0.54300000000000004},"min":{"1min":0.40300000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.71999999999999997,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.372,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.78300000000000003,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58599999999999997,"15min":0.57999999999999996},"min":{"1min":0.46700000000000003,"5min":0.35799999999999998,"15min":0.31900000000000001},"max":{"1min":0.77300000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.442}]},{"osd":2,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58799999999999997,"15min":0.58499999999999996},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.68400000000000005,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60399999999999998},"min":{"1min":0.46999999999999997,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.82299999999999995,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.752}]},{"osd":3,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.54100000000000004,"15min":0.54300000000000004},"min":{"1min":0.38200000000000001,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.70699999999999996,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56200000000000006,"15min":0.57199999999999995},"min":{"1min":0.46700000000000003,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.71199999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.441,"15min":0.45300000000000001},"min":{"1min":0.32600000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.51900000000000002,"15min":0.53900000000000003},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.745,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.65300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.47299999999999998,"15min":0.503},"min":{"1min":0.34200000000000003,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.621,"5min":0.77700000000000002,"15min":0.82899999999999996},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.56100000000000005},"min":{"1min":0.40100000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.69699999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.69199999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:07:02 2023","interfaces":[{"interface":"back","average":{"1min":0.49099999999999999,"5min":0.49299999999999999,"15min":0.51300000000000001},"min":{"1min":0.36599999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.62,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52100000000000002,"15min":0.54700000000000004},"min":{"1min":0.38400000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.752,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.76500000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1074,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:07:52.014 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:07:52.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:07:52.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.168+0000 7fafcccd4700 1 Processor -- start 2023-11-28T03:07:52.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.168+0000 7fafcccd4700 1 -- start start 2023-11-28T03:07:52.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafcccd4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafc8098d10 0x7fafc80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafcccd4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 0x7fafc8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafcccd4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc80c72d0 con 0x7fafc8064350 2023-11-28T03:07:52.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafcccd4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc8154bc0 con 0x7fafc8098d10 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafcccd4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc80c66c0 con 0x7fafc80c6190 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafc8098d10 0x7fafc80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafc8098d10 0x7fafc80c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50018/0 (socket says 172.21.15.73:50018) 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 -- 172.21.15.73:0/711128128 learned_addr learned my addr 172.21.15.73:0/711128128 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc5d9b700 1 --2- 172.21.15.73:0/711128128 >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 0x7fafc8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 -- 172.21.15.73:0/711128128 >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 msgr2=0x7fafc8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 --2- 172.21.15.73:0/711128128 >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 0x7fafc8063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 -- 172.21.15.73:0/711128128 >> v1:172.21.15.150:6789/0 conn(0x7fafc8064350 legacy=0x7fafc8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 -- 172.21.15.73:0/711128128 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc80c69a0 con 0x7fafc8098d10 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc5d9b700 1 --2- 172.21.15.73:0/711128128 >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 0x7fafc8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc659c700 1 --2- 172.21.15.73:0/711128128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8098d10 0x7fafc80c7760 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fafc80c5b40 tx=0x7fafb8009840).ready entity=mon.0 client_cookie=a30afd7840e4611e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc559a700 1 -- 172.21.15.73:0/711128128 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800e070 con 0x7fafc8098d10 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc559a700 1 -- 172.21.15.73:0/711128128 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb800ba10 con 0x7fafc8098d10 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.169+0000 7fafc559a700 1 -- 172.21.15.73:0/711128128 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb800bd90 con 0x7fafc8098d10 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 -- 172.21.15.73:0/711128128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8098d10 msgr2=0x7fafc80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 --2- 172.21.15.73:0/711128128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8098d10 0x7fafc80c7760 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 rx=0x7fafc80c5b40 tx=0x7fafb8009840).stop 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 -- 172.21.15.73:0/711128128 shutdown_connections 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 --2- 172.21.15.73:0/711128128 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8098d10 0x7fafc80c7760 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 --2- 172.21.15.73:0/711128128 >> v2:172.21.15.150:3300/0 conn(0x7fafc80c6190 0x7fafc8063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 -- 172.21.15.73:0/711128128 >> 172.21.15.73:0/711128128 conn(0x7fafc80cb310 msgr2=0x7fafc80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 -- 172.21.15.73:0/711128128 shutdown_connections 2023-11-28T03:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.170+0000 7fafcccd4700 1 -- 172.21.15.73:0/711128128 wait complete. 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.171+0000 7fafcccd4700 1 Processor -- start 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.171+0000 7fafcccd4700 1 -- start start 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc8098d10 0x7fafc80a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc80c6190 0x7fafc80a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc8150af0 con 0x7fafc8064350 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80a9b40 con 0x7fafc8098d10 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc80a9e20 con 0x7fafc80c6190 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50032/0 (socket says 172.21.15.73:50032) 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 -- 172.21.15.73:0/333754880 learned_addr learned my addr 172.21.15.73:0/333754880 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc6d9d700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc80c6190 0x7fafc80a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc5d9b700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc8098d10 0x7fafc80a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 -- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc80c6190 msgr2=0x7fafc80a9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc80c6190 0x7fafc80a9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 -- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc8098d10 msgr2=0x7fafc80a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc8098d10 0x7fafc80a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 -- 172.21.15.73:0/333754880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafb8009ae0 con 0x7fafc8064350 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafc659c700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fafb8009880 tx=0x7fafc80a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb8013d50 con 0x7fafc8064350 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafb801d410 con 0x7fafc8064350 2023-11-28T03:07:52.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafb801d7d0 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.172+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc80ab420 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.173+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc8155410 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.173+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc80ab8c0 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.174+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fafb801dbc0 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.174+0000 7fafbe7fc700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.174+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fafb807be20 con 0x7fafc8064350 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.174+0000 7fafc5d9b700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.174+0000 7fafc5d9b700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 0x7fafa8061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fafc8154660 tx=0x7fafb0008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.178+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fafb804d1a0 con 0x7fafc8064350 2023-11-28T03:07:52.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.406+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fafc80ab8c0 con 0x7fafc8064350 2023-11-28T03:07:52.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.410+0000 7fafbe7fc700 1 -- 172.21.15.73:0/333754880 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) v1 ==== 190+0+0 (secure 0 0 0) 0x7fafb80507c0 con 0x7fafc8064350 2023-11-28T03:07:52.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 msgr2=0x7fafa8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 0x7fafa8061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7fafc8154660 tx=0x7fafb0008040).stop 2023-11-28T03:07:52.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 msgr2=0x7fafc8154f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 rx=0x7fafb8009880 tx=0x7fafc80a8040).stop 2023-11-28T03:07:52.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 shutdown_connections 2023-11-28T03:07:52.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc80c6190 0x7fafc80a9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc8064350 0x7fafc8154f20 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 --2- 172.21.15.73:0/333754880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc8098d10 0x7fafc80a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 >> 172.21.15.73:0/333754880 conn(0x7fafc80cb310 msgr2=0x7fafc81529d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:52.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 shutdown_connections 2023-11-28T03:07:52.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.412+0000 7fafcccd4700 1 -- 172.21.15.73:0/333754880 wait complete. 2023-11-28T03:07:52.416 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:07:52.424 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:07:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.536+0000 7fa7b9fd8700 1 Processor -- start 2023-11-28T03:07:52.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.536+0000 7fa7b9fd8700 1 -- start start 2023-11-28T03:07:52.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b9fd8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 0x7fa7b4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b9fd8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b40c12c0 con 0x7fa7b40c1750 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b9fd8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b40c06b0 con 0x7fa7b40c0180 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b9fd8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40c0990 con 0x7fa7b4157660 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b97d7700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa7b40c1750 0x7fa7b40648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51142/0 (socket says 172.21.15.73:51142) 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b97d7700 1 -- 172.21.15.73:0/337498083 learned_addr learned my addr 172.21.15.73:0/337498083 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b8fd6700 1 --2- 172.21.15.73:0/337498083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 0x7fa7b4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2879295451 0 0) 0x7fa7b40c12c0 con 0x7fa7b40c1750 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7a40033f0 con 0x7fa7b40c1750 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2571203939 0 0) 0x7fa7b40c06b0 con 0x7fa7b40c0180 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7b40c12c0 con 0x7fa7b40c0180 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b8fd6700 1 -- 172.21.15.73:0/337498083 >> v1:172.21.15.150:6789/0 conn(0x7fa7b40c0180 legacy=0x7fa7b41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b8fd6700 1 -- 172.21.15.73:0/337498083 >> v1:172.21.15.73:6789/0 conn(0x7fa7b40c1750 legacy=0x7fa7b40648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.537+0000 7fa7b8fd6700 1 -- 172.21.15.73:0/337498083 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b40c0c70 con 0x7fa7b4157660 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b8fd6700 1 --2- 172.21.15.73:0/337498083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 0x7fa7b4157a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fa7a8009790 tx=0x7fa7a8009060).ready entity=mon.2 client_cookie=36c23ab4b83052ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f510 con 0x7fa7b4157660 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a800f670 con 0x7fa7b4157660 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b37fe700 1 -- 172.21.15.73:0/337498083 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a800f9f0 con 0x7fa7b4157660 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/337498083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 msgr2=0x7fa7b4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/337498083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 0x7fa7b4157a40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 rx=0x7fa7a8009790 tx=0x7fa7a8009060).stop 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/337498083 shutdown_connections 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/337498083 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b4157660 0x7fa7b4157a40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/337498083 >> 172.21.15.73:0/337498083 conn(0x7fa7b40cb000 msgr2=0x7fa7b40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/337498083 shutdown_connections 2023-11-28T03:07:52.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.538+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/337498083 wait complete. 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.539+0000 7fa7b9fd8700 1 Processor -- start 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 -- start start 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b40c0180 0x7fa7b4155650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b40c1750 0x7fa7b4153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4154b30 con 0x7fa7b4157660 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4154570 con 0x7fa7b40c1750 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b9fd8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4154850 con 0x7fa7b40c0180 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b97d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.540+0000 7fa7b97d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50042/0 (socket says 172.21.15.73:50042) 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 -- 172.21.15.73:0/406664350 learned_addr learned my addr 172.21.15.73:0/406664350 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b3fff700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b40c1750 0x7fa7b4153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b8fd6700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b40c0180 0x7fa7b4155650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 -- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b40c0180 msgr2=0x7fa7b4155650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b40c0180 0x7fa7b4155650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 -- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b40c1750 msgr2=0x7fa7b4153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b40c1750 0x7fa7b4153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 -- 172.21.15.73:0/406664350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a80092a0 con 0x7fa7b4157660 2023-11-28T03:07:52.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b97d7700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fa7ac001560 tx=0x7fa7ac001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac005d30 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac005e90 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b40c1cd0 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac007360 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.541+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b41522e0 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.542+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b406c7c0 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.542+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa7ac0076d0 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.543+0000 7fa7b0ff9700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 0x7fa798061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.543+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa7ac07a070 con 0x7fa7b4157660 2023-11-28T03:07:52.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.543+0000 7fa7b8fd6700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 0x7fa798061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:52.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.543+0000 7fa7b8fd6700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 0x7fa798061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa7b4152aa0 tx=0x7fa7a8017000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:52.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.547+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa7ac04b340 con 0x7fa7b4157660 2023-11-28T03:07:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.751+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa7b40c0f50 con 0x7fa7b4157660 2023-11-28T03:07:52.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.752+0000 7fa7b0ff9700 1 -- 172.21.15.73:0/406664350 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fa7ac04e960 con 0x7fa7b4157660 2023-11-28T03:07:52.753 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:07:52.753 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 msgr2=0x7fa798061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 0x7fa798061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fa7b4152aa0 tx=0x7fa7a8017000).stop 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 msgr2=0x7fa7b4153fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 rx=0x7fa7ac001560 tx=0x7fa7ac001650).stop 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 shutdown_connections 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa79805f150 0x7fa798061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa7b4157660 0x7fa7b4153fa0 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa7b40c0180 0x7fa7b4155650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 --2- 172.21.15.73:0/406664350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa7b40c1750 0x7fa7b4153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 >> 172.21.15.73:0/406664350 conn(0x7fa7b40cb000 msgr2=0x7fa7b4152c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 shutdown_connections 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:52.754+0000 7fa7b9fd8700 1 -- 172.21.15.73:0/406664350 wait complete. 2023-11-28T03:07:52.756 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:07:52.799 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:07:52.802 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:07:53.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 34, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:07:47.669246+0000', 'modified': '2023-11-28T03:07:50.680150+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6705}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6705': {'gid': 6705, 'name': 'b', 'rank': 0, 'incarnation': 30, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.150:6835/3882811189', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.150:6832', 'nonce': 3882811189}, {'type': 'v1', 'addr': '172.21.15.150:6835', 'nonce': 3882811189}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [12], 'metadata_pool': 11, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:07:53.170 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:07:53.170 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-11-28T03:07:53.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.286+0000 7f4dc59a2700 1 Processor -- start 2023-11-28T03:07:53.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.286+0000 7f4dc59a2700 1 -- start start 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dc59a2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4dc00a9f80 0x7f4dc00aa360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dc59a2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 0x7f4dc0062fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dc59a2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc00be010 con 0x7f4dc0155780 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dc59a2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc0069c90 con 0x7f4dc00a9f80 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dc59a2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc00be280 con 0x7f4dc00bea80 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dbeffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4dc00a9f80 0x7f4dc00aa360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dbf7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4dc0155780 0x7f4dc00c38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51156/0 (socket says 172.21.15.73:51156) 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.287+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/1948723982 learned_addr learned my addr 172.21.15.73:0/1948723982 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbeffd700 1 -- 172.21.15.73:0/1948723982 >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 msgr2=0x7f4dc0062fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4db67fc700 1 --2- 172.21.15.73:0/1948723982 >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 0x7f4dc0062fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbe7fc700 1 -- 172.21.15.73:0/1948723982 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2377916752 0 0) 0x7f4dc00be010 con 0x7f4dc0155780 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/1948723982 >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 0x7f4dc0062fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbeffd700 1 -- 172.21.15.73:0/1948723982 >> v1:172.21.15.73:6789/0 conn(0x7f4dc0155780 legacy=0x7f4dc00c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbeffd700 1 -- 172.21.15.73:0/1948723982 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc00be560 con 0x7f4dc00a9f80 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4db67fc700 1 --2- 172.21.15.73:0/1948723982 >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 0x7f4dc0062fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/1948723982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc00aa360 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f4dc00692b0 tx=0x7f4db0009060).ready entity=mon.0 client_cookie=3d301937cccfb440 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbe7fc700 1 -- 172.21.15.73:0/1948723982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db000b580 con 0x7f4dc00a9f80 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbe7fc700 1 -- 172.21.15.73:0/1948723982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db000b6e0 con 0x7f4dc00a9f80 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dbe7fc700 1 -- 172.21.15.73:0/1948723982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db000ba60 con 0x7f4dc00a9f80 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 -- 172.21.15.73:0/1948723982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 msgr2=0x7f4dc00aa360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/1948723982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc00aa360 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 rx=0x7f4dc00692b0 tx=0x7f4db0009060).stop 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 -- 172.21.15.73:0/1948723982 shutdown_connections 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/1948723982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc00aa360 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/1948723982 >> v2:172.21.15.150:3300/0 conn(0x7f4dc00bea80 0x7f4dc0062fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 -- 172.21.15.73:0/1948723982 >> 172.21.15.73:0/1948723982 conn(0x7f4dc0063810 msgr2=0x7f4dc0063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 -- 172.21.15.73:0/1948723982 shutdown_connections 2023-11-28T03:07:53.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.288+0000 7f4dc59a2700 1 -- 172.21.15.73:0/1948723982 wait complete. 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 Processor -- start 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 -- start start 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dc00bea80 0x7f4dc00c5e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dc0155780 0x7f4dc00c6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00c6840 con 0x7f4dc00a9f80 2023-11-28T03:07:53.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00a2a10 con 0x7f4dc0155780 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.290+0000 7f4dc59a2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc00a1e00 con 0x7f4dc00bea80 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50068/0 (socket says 172.21.15.73:50068) 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 -- 172.21.15.73:0/370179702 learned_addr learned my addr 172.21.15.73:0/370179702 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4db67fc700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dc00bea80 0x7f4dc00c5e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dc0155780 0x7f4dc00c6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 -- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dc00bea80 msgr2=0x7f4dc00c5e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dc00bea80 0x7f4dc00c5e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 -- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dc0155780 msgr2=0x7f4dc00c6350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dc0155780 0x7f4dc00c6350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 -- 172.21.15.73:0/370179702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4db00092a0 con 0x7f4dc00a9f80 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f4dc0064ff0 tx=0x7f4db0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4db77fe700 1 -- 172.21.15.73:0/370179702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db000ba60 con 0x7f4dc00a9f80 2023-11-28T03:07:53.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4db77fe700 1 -- 172.21.15.73:0/370179702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0000c90 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc00a20e0 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4db77fe700 1 -- 172.21.15.73:0/370179702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db001b660 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.291+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc00a26a0 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.292+0000 7f4db77fe700 1 -- 172.21.15.73:0/370179702 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4db001b9d0 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.293+0000 7f4db77fe700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 0x7f4da0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.293+0000 7f4db77fe700 1 -- 172.21.15.73:0/370179702 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4db007bd50 con 0x7f4dc00a9f80 2023-11-28T03:07:53.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.293+0000 7f4db67fc700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 0x7f4da0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.293+0000 7f4db67fc700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 0x7f4da0061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4dc013f840 tx=0x7f4da8008040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.295+0000 7f4dd733ab80 1 Processor -- start 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.295+0000 7f4dd733ab80 1 -- start start 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 0x56094bc9a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bc9bd70 0x56094bc9acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bc60060 con 0x56094bce3ea0 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bbd71f0 con 0x56094ba56e40 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094bbec440 con 0x56094bc9bd70 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50078/0 (socket says 172.21.15.73:50078) 2023-11-28T03:07:53.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/3626583951 learned_addr learned my addr 172.21.15.73:0/3626583951 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4db67fc700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bc9bd70 0x56094bc9acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 0x56094bc9a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bc9bd70 msgr2=0x56094bc9acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bc9bd70 0x56094bc9acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 msgr2=0x56094bc9a7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 0x56094bc9a7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/3626583951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56094bbe5620 con 0x56094bce3ea0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 0x56094bc9a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f4db8001c10 tx=0x7f4db8009040).ready entity=mon.0 client_cookie=4a8a080cae060c2b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4d9ffff700 1 -- 172.21.15.73:0/3626583951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8005d10 con 0x56094bce3ea0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.296+0000 7f4d9ffff700 1 -- 172.21.15.73:0/3626583951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8005e70 con 0x56094bce3ea0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4d9ffff700 1 -- 172.21.15.73:0/3626583951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db800e800 con 0x56094bce3ea0 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3626583951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 msgr2=0x56094bcc24f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 rx=0x7f4db8001c10 tx=0x7f4db8009040).stop 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3626583951 shutdown_connections 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcc24f0 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094ba56e40 0x56094bc9a7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3626583951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bc9bd70 0x56094bc9acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3626583951 >> 172.21.15.73:0/3626583951 conn(0x56094bb044a0 msgr2=0x56094bce3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3626583951 shutdown_connections 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.297+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3626583951 wait complete. 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.298+0000 7f4dd733ab80 1 -- start start 2023-11-28T03:07:53.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.298+0000 7f4dd733ab80 1 Processor -- start 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcb9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcc1850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bbec580 con 0x56094ba56e40 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bbec800 con 0x56094bc9bd70 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094bbb7e10 con 0x56094bce3ea0 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 10 client.0 ll_register_callbacks cb 0x56094bc5a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50080/0 (socket says 172.21.15.73:50080) 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 -- 172.21.15.73:0/3212368454 learned_addr learned my addr 172.21.15.73:0/3212368454 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcc1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4db67fc700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcb9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 -- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 msgr2=0x56094bcc1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcc1850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 -- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 msgr2=0x56094bcb9b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcb9b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 -- 172.21.15.73:0/3212368454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4db800f040 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f4db00509e0 tx=0x7f4db0003040).ready entity=mon.0 client_cookie=18e3a1c42d94ab7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4d9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0056290 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4db00563f0 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.299+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56094bc8e720 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4db001bc80 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 10 client.6849 fetch_fsmap learned FSMap version 34 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x56094bbe09c0 con 0x56094ba56e40 2023-11-28T03:07:53.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56094bc8e720 con 0x56094ba56e40 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4db00501d0 con 0x56094ba56e40 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 10 client.6849 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 10 client.6849 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 20 client.6849 populate_metadata read hostname 'smithi073' 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bce0230 0x56094bce0610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 4 client.6849 mds_command: new command op to 6705 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 --> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56094ba52ca0 con 0x56094bce0230 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4db67fc700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bce0230 0x56094bce0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4d9d7fa700 10 client.6849 ms_handle_connect on v2:172.21.15.150:6832/3882811189 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.300+0000 7f4db67fc700 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bce0230 0x56094bce0610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=728f1d37d953d7f1 server_cookie=1988d3e665fc8ff9 in_seq=0 out_seq=0 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/3212368454 <== mds.0 v2:172.21.15.150:6832/3882811189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x56094ba52ca0 con 0x56094bce0230 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4d9d7fa700 10 client.6849 handle_command_reply: tid=0 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 client.6849 shutdown 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 msgr2=0x56094bcb8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 rx=0x7f4db00509e0 tx=0x7f4db0003040).stop 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 shutdown_connections 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bce0230 0x56094bce0610 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcc1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4d9d7fa700 0 client.6849 ms_handle_reset on v2:172.21.15.150:6832/3882811189 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcb8e30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/3212368454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcb9b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 >> 172.21.15.73:0/3212368454 conn(0x56094bb044a0 msgr2=0x56094bc9ab30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 shutdown_connections 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.301+0000 7f4dd733ab80 1 -- 172.21.15.73:0/3212368454 wait complete. 2023-11-28T03:07:53.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.302+0000 7f4dd733ab80 20 client.6849 trim_cache size 0 max 16384 2023-11-28T03:07:53.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.318+0000 7f4dd733ab80 1 Processor -- start 2023-11-28T03:07:53.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.318+0000 7f4dd733ab80 1 -- start start 2023-11-28T03:07:53.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bc9aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcb8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bcc0400 con 0x56094ba56e40 2023-11-28T03:07:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bc9af90 con 0x56094bc9bd70 2023-11-28T03:07:53.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094bce4380 con 0x56094bce3ea0 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50088/0 (socket says 172.21.15.73:50088) 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 -- 172.21.15.73:0/125005759 learned_addr learned my addr 172.21.15.73:0/125005759 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcb8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4db67fc700 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bc9aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 -- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 msgr2=0x56094bcb8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcb8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 -- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 msgr2=0x56094bc9aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bc9aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 -- 172.21.15.73:0/125005759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56094bbe0710 con 0x56094ba56e40 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f4db0009e80 tx=0x7f4db002e000).ready entity=mon.0 client_cookie=26a8e8561bb7d5d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4db4ff9700 1 -- 172.21.15.73:0/125005759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db0050640 con 0x56094ba56e40 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4db4ff9700 1 -- 172.21.15.73:0/125005759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db0054070 con 0x56094ba56e40 2023-11-28T03:07:53.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.319+0000 7f4db4ff9700 1 -- 172.21.15.73:0/125005759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db00501d0 con 0x56094ba56e40 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- 172.21.15.73:0/125005759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 msgr2=0x56094bcc9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 rx=0x7f4db0009e80 tx=0x7f4db002e000).stop 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- 172.21.15.73:0/125005759 shutdown_connections 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094bce3ea0 0x56094bcb8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094ba56e40 0x56094bcc9580 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/125005759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bc9aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- 172.21.15.73:0/125005759 >> 172.21.15.73:0/125005759 conn(0x56094bb044a0 msgr2=0x56094bc98930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- 172.21.15.73:0/125005759 shutdown_connections 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- 172.21.15.73:0/125005759 wait complete. 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 -- start start 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.320+0000 7f4dd733ab80 1 Processor -- start 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094ba56e40 0x56094bd48140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcf5070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bc96150 con 0x56094bce3ea0 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x56094bcf5a50 con 0x56094bc9bd70 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x56094bd49df0 con 0x56094ba56e40 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dd733ab80 10 client.0 ll_register_callbacks cb 0x56094bc5a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.321+0000 7f4dbf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:50090/0 (socket says 172.21.15.73:50090) 2023-11-28T03:07:53.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/2683015515 learned_addr learned my addr 172.21.15.73:0/2683015515 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094ba56e40 0x56094bd48140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4db67fc700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcf5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094ba56e40 msgr2=0x56094bd48140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094ba56e40 0x56094bd48140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 msgr2=0x56094bcf5070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcf5070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 -- 172.21.15.73:0/2683015515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56094bbe09c0 con 0x56094bce3ea0 2023-11-28T03:07:53.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dbf7fe700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f4db8001be0 tx=0x56094bc9a2a0).ready entity=mon.0 client_cookie=247597d136ae7e1c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4db8001080 con 0x56094bce3ea0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4db800ebb0 con 0x56094bce3ea0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f4db8005e70 con 0x56094bce3ea0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4db800e800 con 0x56094bce3ea0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dd733ab80 10 client.6855 fetch_fsmap learned FSMap version 34 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=2+}) v3 -- 0x56094ba57f80 con 0x56094bce3ea0 2023-11-28T03:07:53.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4db8005e70 con 0x56094bce3ea0 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.322+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 34) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f4db8004a30 con 0x56094bce3ea0 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 10 client.6855 fetch_fsmap finished waiting for FSMap version 34 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 10 client.6855 resolve_mds: resolved name 'b' to daemon mds.b 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 20 client.6855 populate_metadata read hostname 'smithi073' 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bcb7ad0 0x56094bcb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 4 client.6855 mds_command: new command op to 6705 tid=0[{"prefix": "status"}] 2023-11-28T03:07:53.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 --> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56094ba89ca0 con 0x56094bcb7ad0 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bcb7ad0 0x56094bcb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4d9d7fa700 10 client.6855 ms_handle_connect on v2:172.21.15.150:6832/3882811189 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dbeffd700 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bcb7ad0 0x56094bcb7eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=8509ec9a88f4ced server_cookie=80c715d219b91457 in_seq=0 out_seq=0 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4d9d7fa700 1 -- 172.21.15.73:0/2683015515 <== mds.0 v2:172.21.15.150:6832/3882811189 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56094ba89ca0 con 0x56094bcb7ad0 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4d9d7fa700 10 client.6855 handle_command_reply: tid=0 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 1 client.6855 shutdown 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 msgr2=0x56094bcf5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.323+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 rx=0x7f4db8001be0 tx=0x56094bc9a2a0).stop 2023-11-28T03:07:53.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 shutdown_connections 2023-11-28T03:07:53.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:6832/3882811189,v1:172.21.15.150:6835/3882811189] conn(0x56094bcb7ad0 0x56094bcb7eb0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x56094bce3ea0 0x56094bcf5560 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x56094ba56e40 0x56094bd48140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 --2- 172.21.15.73:0/2683015515 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x56094bc9bd70 0x56094bcf5070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4d9d7fa700 0 client.6855 ms_handle_reset on v2:172.21.15.150:6832/3882811189 2023-11-28T03:07:53.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 >> 172.21.15.73:0/2683015515 conn(0x56094bb044a0 msgr2=0x56094bcc07b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 shutdown_connections 2023-11-28T03:07:53.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 1 -- 172.21.15.73:0/2683015515 wait complete. 2023-11-28T03:07:53.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dd733ab80 20 client.6855 trim_cache size 0 max 16384 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 6705, 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:07:53.339 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 5.653573905, 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 34, 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 96, 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 96, 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 10.155383024000001 2023-11-28T03:07:53.340 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 msgr2=0x7f4da0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 0x7f4da0061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f4dc013f840 tx=0x7f4da8008040).stop 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 msgr2=0x7f4dc01622d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 rx=0x7f4dc0064ff0 tx=0x7f4db0017000).stop 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 shutdown_connections 2023-11-28T03:07:53.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4da005f150 0x7f4da0061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4dc0155780 0x7f4dc00c6350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4dc00a9f80 0x7f4dc01622d0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 --2- 172.21.15.73:0/370179702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4dc00bea80 0x7f4dc00c5e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 >> 172.21.15.73:0/370179702 conn(0x7f4dc0063810 msgr2=0x7f4dc00c4f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 shutdown_connections 2023-11-28T03:07:53.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.324+0000 7f4dc59a2700 1 -- 172.21.15.73:0/370179702 wait complete. 2023-11-28T03:07:53.343 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:07:53.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.443+0000 7ff26c9da700 1 Processor -- start 2023-11-28T03:07:53.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.443+0000 7ff26c9da700 1 -- start start 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff26c9da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff268062780 0x7ff268062b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff26c9da700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff268154490 0x7ff2680c2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff26c9da700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2680bdde0 con 0x7ff2680be270 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff26c9da700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2680bd1d0 con 0x7ff268062780 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff26c9da700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2680bd4b0 con 0x7ff268154490 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff2677fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff268062780 0x7ff268062b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff267fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff2680be270 0x7ff268069610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50010/0 (socket says 172.21.15.73:50010) 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff267fff700 1 -- 172.21.15.73:0/197059437 learned_addr learned my addr 172.21.15.73:0/197059437 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:53.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.444+0000 7ff266ffd700 1 --2- 172.21.15.73:0/197059437 >> v2:172.21.15.150:3300/0 conn(0x7ff268154490 0x7ff2680c2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2677fe700 1 -- 172.21.15.73:0/197059437 >> v2:172.21.15.150:3300/0 conn(0x7ff268154490 msgr2=0x7ff2680c2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2677fe700 1 --2- 172.21.15.73:0/197059437 >> v2:172.21.15.150:3300/0 conn(0x7ff268154490 0x7ff2680c2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2677fe700 1 -- 172.21.15.73:0/197059437 >> v1:172.21.15.73:6789/0 conn(0x7ff2680be270 legacy=0x7ff268069610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2667fc700 1 -- 172.21.15.73:0/197059437 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4203050705 0 0) 0x7ff2680bdde0 con 0x7ff2680be270 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2677fe700 1 -- 172.21.15.73:0/197059437 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680bd790 con 0x7ff268062780 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2677fe700 1 --2- 172.21.15.73:0/197059437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff268062780 0x7ff268062b60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7ff26806acb0 tx=0x7ff258009780).ready entity=mon.0 client_cookie=93bd9278b6c2a511 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2667fc700 1 -- 172.21.15.73:0/197059437 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25800e070 con 0x7ff268062780 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2667fc700 1 -- 172.21.15.73:0/197059437 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25800bc30 con 0x7ff268062780 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff2667fc700 1 -- 172.21.15.73:0/197059437 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff258012410 con 0x7ff268062780 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 -- 172.21.15.73:0/197059437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff268062780 msgr2=0x7ff268062b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 --2- 172.21.15.73:0/197059437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff268062780 0x7ff268062b60 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 rx=0x7ff26806acb0 tx=0x7ff258009780).stop 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 -- 172.21.15.73:0/197059437 shutdown_connections 2023-11-28T03:07:53.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 --2- 172.21.15.73:0/197059437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff268062780 0x7ff268062b60 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 --2- 172.21.15.73:0/197059437 >> v2:172.21.15.150:3300/0 conn(0x7ff268154490 0x7ff2680c2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 -- 172.21.15.73:0/197059437 >> 172.21.15.73:0/197059437 conn(0x7ff2680c7d00 msgr2=0x7ff2680c8110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 -- 172.21.15.73:0/197059437 shutdown_connections 2023-11-28T03:07:53.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.445+0000 7ff26c9da700 1 -- 172.21.15.73:0/197059437 wait complete. 2023-11-28T03:07:53.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.447+0000 7ff26c9da700 1 Processor -- start 2023-11-28T03:07:53.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.447+0000 7ff26c9da700 1 -- start start 2023-11-28T03:07:53.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268062780 0x7ff268152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff268154490 0x7ff2681533e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680ab0c0 con 0x7ff2680be270 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff268153020 con 0x7ff268062780 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2680adb00 con 0x7ff268154490 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34580/0 (socket says 172.21.15.73:34580) 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 -- 172.21.15.73:0/1353669607 learned_addr learned my addr 172.21.15.73:0/1353669607 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:53.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff2677fe700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268062780 0x7ff268152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff267fff700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff268154490 0x7ff2681533e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 -- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff268154490 msgr2=0x7ff2681533e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff268154490 0x7ff2681533e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 -- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268062780 msgr2=0x7ff268152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268062780 0x7ff268152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 -- 172.21.15.73:0/1353669607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff258009a20 con 0x7ff2680be270 2023-11-28T03:07:53.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff266ffd700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ff25c00d890 tx=0x7ff25c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00f520 con 0x7ff2680be270 2023-11-28T03:07:53.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff25c00f680 con 0x7ff2680be270 2023-11-28T03:07:53.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2680ad540 con 0x7ff2680be270 2023-11-28T03:07:53.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff25c00fa00 con 0x7ff2680be270 2023-11-28T03:07:53.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.448+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2680aad50 con 0x7ff2680be270 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.449+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2680ab270 con 0x7ff2680be270 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.450+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff25c013480 con 0x7ff2680be270 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.450+0000 7ff24f7fe700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 0x7ff254061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.450+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff25c079c20 con 0x7ff2680be270 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.450+0000 7ff2677fe700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 0x7ff254061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:53.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.450+0000 7ff2677fe700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 0x7ff254061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff258009fd0 tx=0x7ff2680bdd50).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:53.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.454+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff25c04ae90 con 0x7ff2680be270 2023-11-28T03:07:53.592 INFO:tasks.ceph.mds.b.smithi150.stderr:2023-11-28T03:07:53.591+0000 7f5a0a268700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:07:53.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.657+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff2680bda70 con 0x7ff2680be270 2023-11-28T03:07:53.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.657+0000 7ff24f7fe700 1 -- 172.21.15.73:0/1353669607 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v34) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff25c04e4b0 con 0x7ff2680be270 2023-11-28T03:07:53.658 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 msgr2=0x7ff254061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 0x7ff254061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff258009fd0 tx=0x7ff2680bdd50).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 msgr2=0x7ff2680654f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 rx=0x7ff25c00d890 tx=0x7ff25c008040).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 shutdown_connections 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff25405f150 0x7ff254061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff268154490 0x7ff2681533e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff268062780 0x7ff268152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 --2- 172.21.15.73:0/1353669607 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2680be270 0x7ff2680654f0 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 >> 172.21.15.73:0/1353669607 conn(0x7ff2680c7d00 msgr2=0x7ff268067b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:53.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 shutdown_connections 2023-11-28T03:07:53.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:53.659+0000 7ff26c9da700 1 -- 172.21.15.73:0/1353669607 wait complete. 2023-11-28T03:07:53.671 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:07:53.671 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:07:53.671 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:07:53.672 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:07:53.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.673 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:07:53.688 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:07:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:07:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:07:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.688 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:53.689 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41606sec preferred_lft 41606sec 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:07:53.690 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:07:53.691 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:07:53.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.692 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:07:53.751 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:07:53.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.753 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:07:53.808 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:07:53.808 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:07:53.809 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:07:53.809 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:07:53.809 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:07:53.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.809 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:07:53.809 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:07:53.809 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:07:53.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:07:53.810 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:07:53.810 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:07:53.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.910 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:07:53.924 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:07:53.924 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:07:53.925 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:07:53.925 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:07:53.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:53.989 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:54.044 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:07:54.044 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:54.123 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:54.124 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:07:54.148 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:07:54.148 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:07:54.150 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:07:54.153 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:07:54.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:07:54.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:07:54.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 Processor -- start 2023-11-28T03:07:54.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 -- start start 2023-11-28T03:07:54.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff1340c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1340c72d0 con 0x7ff134158320 2023-11-28T03:07:54.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff134064570 con 0x7ff134098d10 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.368+0000 7ff138c37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff134154bc0 con 0x7ff1340c6190 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff132d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff134158320 0x7ff1340a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50020/0 (socket says 172.21.15.73:50020) 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff132d9d700 1 -- 172.21.15.73:0/1015919394 learned_addr learned my addr 172.21.15.73:0/1015919394 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff129d9b700 1 --2- 172.21.15.73:0/1015919394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff1340c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1352216459 0 0) 0x7ff1340c72d0 con 0x7ff134158320 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1200033f0 con 0x7ff134158320 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3577265274 0 0) 0x7ff134064570 con 0x7ff134098d10 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff129d9b700 1 -- 172.21.15.73:0/1015919394 >> v1:172.21.15.150:6789/0 conn(0x7ff134098d10 legacy=0x7ff134063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff129d9b700 1 -- 172.21.15.73:0/1015919394 >> v1:172.21.15.73:6789/0 conn(0x7ff134158320 legacy=0x7ff1340a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff129d9b700 1 -- 172.21.15.73:0/1015919394 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1340c66c0 con 0x7ff1340c6190 2023-11-28T03:07:54.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff129d9b700 1 --2- 172.21.15.73:0/1015919394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff1340c7760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff11c009790 tx=0x7ff11c009060).ready entity=mon.2 client_cookie=17b03809859d6932 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00f510 con 0x7ff1340c6190 2023-11-28T03:07:54.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff11c00f670 con 0x7ff1340c6190 2023-11-28T03:07:54.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.369+0000 7ff131d9b700 1 -- 172.21.15.73:0/1015919394 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff11c00f9f0 con 0x7ff1340c6190 2023-11-28T03:07:54.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 -- 172.21.15.73:0/1015919394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 msgr2=0x7ff1340c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 --2- 172.21.15.73:0/1015919394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff1340c7760 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 rx=0x7ff11c009790 tx=0x7ff11c009060).stop 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 -- 172.21.15.73:0/1015919394 shutdown_connections 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 --2- 172.21.15.73:0/1015919394 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff1340c7760 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 -- 172.21.15.73:0/1015919394 >> 172.21.15.73:0/1015919394 conn(0x7ff1340cb310 msgr2=0x7ff1340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 -- 172.21.15.73:0/1015919394 shutdown_connections 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.370+0000 7ff138c37700 1 -- 172.21.15.73:0/1015919394 wait complete. 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.371+0000 7ff138c37700 1 Processor -- start 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.371+0000 7ff138c37700 1 -- start start 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff134098d10 0x7ff1341437f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff134161a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1340aef50 con 0x7ff134158320 2023-11-28T03:07:54.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff134161ef0 con 0x7ff134098d10 2023-11-28T03:07:54.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff138c37700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1341682d0 con 0x7ff1340c6190 2023-11-28T03:07:54.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34590/0 (socket says 172.21.15.73:34590) 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 -- 172.21.15.73:0/1585520788 learned_addr learned my addr 172.21.15.73:0/1585520788 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff129d9b700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff134161a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff13259c700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff134098d10 0x7ff1341437f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 -- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 msgr2=0x7ff134161a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff134161a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 -- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff134098d10 msgr2=0x7ff1341437f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff134098d10 0x7ff1341437f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.372+0000 7ff132d9d700 1 -- 172.21.15.73:0/1585520788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff11c0092a0 con 0x7ff134158320 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff132d9d700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7ff12c002250 tx=0x7ff12c007f50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c005d30 con 0x7ff134158320 2023-11-28T03:07:54.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff12c005e90 con 0x7ff134158320 2023-11-28T03:07:54.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1341685b0 con 0x7ff134158320 2023-11-28T03:07:54.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c007360 con 0x7ff134158320 2023-11-28T03:07:54.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.373+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff134164800 con 0x7ff134158320 2023-11-28T03:07:54.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.374+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13406c860 con 0x7ff134158320 2023-11-28T03:07:54.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.374+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff12c007710 con 0x7ff134158320 2023-11-28T03:07:54.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.374+0000 7ff12affd700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 0x7ff114061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.375+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff12c07a2f0 con 0x7ff134158320 2023-11-28T03:07:54.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.375+0000 7ff13259c700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 0x7ff114061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.375+0000 7ff13259c700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 0x7ff114061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff1340b14e0 tx=0x7ff124009040).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.379+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff12c04b500 con 0x7ff134158320 2023-11-28T03:07:54.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.568+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1340c69a0 con 0x7ff11405f150 2023-11-28T03:07:54.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.569+0000 7ff12affd700 1 -- 172.21.15.73:0/1585520788 <== mgr.6082 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff1340c69a0 con 0x7ff11405f150 2023-11-28T03:07:54.571 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:07:54.571 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:07:54.571 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:07:54.571 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:07:54.571 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:07:54.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 msgr2=0x7ff114061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 0x7ff114061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff1340b14e0 tx=0x7ff124009040).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 msgr2=0x7ff134167a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 rx=0x7ff12c002250 tx=0x7ff12c007f50).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 shutdown_connections 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff11405f150 0x7ff114061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff134158320 0x7ff134167a80 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff134098d10 0x7ff1341437f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 --2- 172.21.15.73:0/1585520788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1340c6190 0x7ff134161a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 >> 172.21.15.73:0/1585520788 conn(0x7ff1340cb310 msgr2=0x7ff134153a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 shutdown_connections 2023-11-28T03:07:54.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.571+0000 7ff138c37700 1 -- 172.21.15.73:0/1585520788 wait complete. 2023-11-28T03:07:54.584 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:07:54.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.698+0000 7f094a7b4700 1 Processor -- start 2023-11-28T03:07:54.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.698+0000 7f094a7b4700 1 -- start start 2023-11-28T03:07:54.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f094a7b4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f0944154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f094a7b4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0944154e40 con 0x7f0944158570 2023-11-28T03:07:54.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f094a7b4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09440c7bc0 con 0x7f094414e530 2023-11-28T03:07:54.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f094a7b4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09440c6b70 con 0x7f09440c66f0 2023-11-28T03:07:54.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f0949fb3700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0944158570 0x7f09440aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50032/0 (socket says 172.21.15.73:50032) 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.699+0000 7f0949fb3700 1 -- 172.21.15.73:0/1973009556 learned_addr learned my addr 172.21.15.73:0/1973009556 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f0948fb1700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f0944154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2488491127 0 0) 0x7f093c001490 con 0x7f0944158570 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09300033f0 con 0x7f0944158570 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2088862309 0 0) 0x7f09300033f0 con 0x7f0944158570 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 msgr2=0x7f0944154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f0944154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 >> v1:172.21.15.150:6789/0 conn(0x7f094414e530 legacy=0x7f094414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0944155920 con 0x7f0944158570 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f093c001980 con 0x7f0944158570 2023-11-28T03:07:54.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 >> v1:172.21.15.73:6789/0 conn(0x7f0944158570 legacy=0x7f09440aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0930003910 0x7f0930003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0930004690 0x7f0930004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f09300050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09440c6b70 con 0x7f0930003910 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f093000b9f0 con 0x7f0930004690 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f093000bbb0 con 0x7f09440c66f0 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0930003910 0x7f0930003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f0949fb3700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f09300050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f0948fb1700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0930004690 0x7f0930004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 -- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 msgr2=0x7f09300050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09440c66f0 0x7f09300050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 -- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0930004690 msgr2=0x7f0930004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0930004690 0x7f0930004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.700+0000 7f09497b2700 1 -- 172.21.15.73:0/1973009556 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09300033f0 con 0x7f0930003910 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f09497b2700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0930003910 0x7f0930003d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f0940001730 tx=0x7f0940007bb0).ready entity=mon.0 client_cookie=2da924a340a4b240 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940007020 con 0x7f0930003910 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0940007180 con 0x7f0930003910 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f093bfff700 1 -- 172.21.15.73:0/1973009556 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0940007500 con 0x7f0930003910 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 -- 172.21.15.73:0/1973009556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0930003910 msgr2=0x7f0930003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 --2- 172.21.15.73:0/1973009556 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0930003910 0x7f0930003d30 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 rx=0x7f0940001730 tx=0x7f0940007bb0).stop 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f0949fb3700 1 -- 172.21.15.73:0/1973009556 reap_dead start 2023-11-28T03:07:54.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 -- 172.21.15.73:0/1973009556 shutdown_connections 2023-11-28T03:07:54.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 -- 172.21.15.73:0/1973009556 >> 172.21.15.73:0/1973009556 conn(0x7f0944064990 msgr2=0x7f0944064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:54.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 -- 172.21.15.73:0/1973009556 shutdown_connections 2023-11-28T03:07:54.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.701+0000 7f094a7b4700 1 -- 172.21.15.73:0/1973009556 wait complete. 2023-11-28T03:07:54.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.702+0000 7f094a7b4700 1 Processor -- start 2023-11-28T03:07:54.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.702+0000 7f094a7b4700 1 -- start start 2023-11-28T03:07:54.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 0x7f09441642b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 0x7f0944167d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0944168240 con 0x7f0944166ea0 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09440af000 con 0x7f0944167320 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f094a7b4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09440afb80 con 0x7f0930003910 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f0948fb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f0948fb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34602/0 (socket says 172.21.15.73:34602) 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.703+0000 7f0948fb1700 1 -- 172.21.15.73:0/170057644 learned_addr learned my addr 172.21.15.73:0/170057644 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f09497b2700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 0x7f09441642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0949fb3700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 0x7f0944167d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 -- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 msgr2=0x7f09441642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 0x7f09441642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 -- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 msgr2=0x7f0944167d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:54.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 0x7f0944167d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 -- 172.21.15.73:0/170057644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09441647a0 con 0x7f0944166ea0 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0949fb3700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 0x7f0944167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f09497b2700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 0x7f09441642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f0948fb1700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f0934011710 tx=0x7f0934008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934015690 con 0x7f0944166ea0 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09340157f0 con 0x7f0944166ea0 2023-11-28T03:07:54.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09440af250 con 0x7f0944166ea0 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0934015b70 con 0x7f0944166ea0 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.704+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09440af810 con 0x7f0944166ea0 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.705+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0944062ee0 con 0x7f0944166ea0 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.705+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 31) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0934013410 con 0x7f0944166ea0 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.706+0000 7f09397fa700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 0x7f0928061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:54.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.706+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f093407aa30 con 0x7f0944166ea0 2023-11-28T03:07:54.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.706+0000 7f09497b2700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 0x7f0928061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:54.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.706+0000 7f09497b2700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 0x7f0928061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0940001010 tx=0x7f0940015000).ready entity=mgr.6082 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:54.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.710+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f093404bca0 con 0x7f0944166ea0 2023-11-28T03:07:54.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:54.920+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f094406c7c0 con 0x7f0944166ea0 2023-11-28T03:07:55.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.777+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 32) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f0934078800 con 0x7f0944166ea0 2023-11-28T03:07:55.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.779+0000 7f09397fa700 1 -- 172.21.15.73:0/170057644 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v32) v1 ==== 92+0+0 (secure 0 0 0) 0x7f09340708f0 con 0x7f0944166ea0 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 msgr2=0x7f0928061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 0x7f0928061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f0940001010 tx=0x7f0940015000).stop 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 msgr2=0x7f0944167860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 rx=0x7f0934011710 tx=0x7f0934008040).stop 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 shutdown_connections 2023-11-28T03:07:55.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f092805f150 0x7f0928061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0944167320 0x7f0944167d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0930003910 0x7f09441642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 --2- 172.21.15.73:0/170057644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0944166ea0 0x7f0944167860 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 >> 172.21.15.73:0/170057644 conn(0x7f0944064990 msgr2=0x7f09441567d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 shutdown_connections 2023-11-28T03:07:55.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.781+0000 7f094a7b4700 1 -- 172.21.15.73:0/170057644 wait complete. 2023-11-28T03:07:55.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:07:55.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.913+0000 7f9768d00700 1 Processor -- start 2023-11-28T03:07:55.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.913+0000 7f9768d00700 1 -- start start 2023-11-28T03:07:55.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9768d00700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97640bd100 0x7f9764158f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9768d00700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 0x7f97640c25e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9768d00700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97640c2df0 con 0x7f9764157580 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9768d00700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97640be700 con 0x7f97640bd100 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9768d00700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9764069790 con 0x7f97640bd4e0 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97640bd100 0x7f9764158f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f97640bd100 0x7f9764158f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34622/0 (socket says 172.21.15.73:34622) 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 -- 172.21.15.73:0/1102408967 learned_addr learned my addr 172.21.15.73:0/1102408967 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9761d9b700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 0x7f97640c25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 -- 172.21.15.73:0/1102408967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 msgr2=0x7f97640c25e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 0x7f97640c25e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 -- 172.21.15.73:0/1102408967 >> v1:172.21.15.150:6789/0 conn(0x7f9764157580 legacy=0x7f9764157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 -- 172.21.15.73:0/1102408967 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f976406abf0 con 0x7f97640bd100 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f9761d9b700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 0x7f97640c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:07:55.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.914+0000 7f976259c700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd100 0x7f9764158f40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f9754006fd0 tx=0x7f97540090f0).ready entity=mon.0 client_cookie=44934d5f332d4766 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f976159a700 1 -- 172.21.15.73:0/1102408967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400b600 con 0x7f97640bd100 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f976159a700 1 -- 172.21.15.73:0/1102408967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975400b760 con 0x7f97640bd100 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f976159a700 1 -- 172.21.15.73:0/1102408967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975400bae0 con 0x7f97640bd100 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 -- 172.21.15.73:0/1102408967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd100 msgr2=0x7f9764158f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd100 0x7f9764158f40 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 rx=0x7f9754006fd0 tx=0x7f97540090f0).stop 2023-11-28T03:07:55.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 -- 172.21.15.73:0/1102408967 shutdown_connections 2023-11-28T03:07:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd100 0x7f9764158f40 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 --2- 172.21.15.73:0/1102408967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97640bd4e0 0x7f97640c25e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 -- 172.21.15.73:0/1102408967 >> 172.21.15.73:0/1102408967 conn(0x7f9764063810 msgr2=0x7f9764063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 -- 172.21.15.73:0/1102408967 shutdown_connections 2023-11-28T03:07:55.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.915+0000 7f9768d00700 1 -- 172.21.15.73:0/1102408967 wait complete. 2023-11-28T03:07:55.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.916+0000 7f9768d00700 1 Processor -- start 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.916+0000 7f9768d00700 1 -- start start 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97640bd100 0x7f9764148210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9764157580 0x7f97640c11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9764156c60 con 0x7f97640bd4e0 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9764156050 con 0x7f97640bd100 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9768d00700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9764156330 con 0x7f9764157580 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9761d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34632/0 (socket says 172.21.15.73:34632) 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9761d9b700 1 -- 172.21.15.73:0/1885189068 learned_addr learned my addr 172.21.15.73:0/1885189068 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f9762d9d700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9764157580 0x7f97640c11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:55.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.917+0000 7f976259c700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97640bd100 0x7f9764148210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 -- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9764157580 msgr2=0x7f97640c11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9764157580 0x7f97640c11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 -- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97640bd100 msgr2=0x7f9764148210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97640bd100 0x7f9764148210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 -- 172.21.15.73:0/1885189068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9754009370 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9761d9b700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f975800d860 tx=0x7f9758008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800f520 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f975800f680 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f975800fa00 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9764156610 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.918+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9764148700 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.919+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9764156e10 con 0x7f97640bd4e0 2023-11-28T03:07:55.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.919+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 32) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f97580123e0 con 0x7f97640bd4e0 2023-11-28T03:07:55.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.920+0000 7f97527fc700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 0x7f9744061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:55.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.920+0000 7f976259c700 1 -- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 msgr2=0x7f9744061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:55.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.920+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9758079e00 con 0x7f97640bd4e0 2023-11-28T03:07:55.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.920+0000 7f976259c700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 0x7f9744061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:55.924 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:55.920+0000 7f78795ab680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:55.925 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:55.919+0000 7fdc0c4f0680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:07:55.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:55.924+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9764156e10 con 0x7f97640bd4e0 2023-11-28T03:07:55.984 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:55.983+0000 7fdc0c4f0680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:55.986 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:55.986+0000 7f78795ab680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:07:56.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.097+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f9764156fd0 con 0x7f97640bd4e0 2023-11-28T03:07:56.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.102+0000 7f97527fc700 1 -- 172.21.15.73:0/1885189068 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v15)=0 v15) v1 ==== 138+0+0 (secure 0 0 0) 0x7f975804b070 con 0x7f97640bd4e0 2023-11-28T03:07:56.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 msgr2=0x7f9744061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 0x7f9744061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 msgr2=0x7f97641556e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 rx=0x7f975800d860 tx=0x7f9758008040).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 shutdown_connections 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f974405f1a0 0x7f9744061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9764157580 0x7f97640c11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97640bd100 0x7f9764148210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 --2- 172.21.15.73:0/1885189068 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97640bd4e0 0x7f97641556e0 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 >> 172.21.15.73:0/1885189068 conn(0x7f9764063810 msgr2=0x7f97640c74d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 shutdown_connections 2023-11-28T03:07:56.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.104+0000 7f9768d00700 1 -- 172.21.15.73:0/1885189068 wait complete. 2023-11-28T03:07:56.117 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-28T03:07:56.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.237+0000 7f71319a0700 1 Processor -- start 2023-11-28T03:07:56.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.237+0000 7f71319a0700 1 -- start start 2023-11-28T03:07:56.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f71319a0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f712c1546a0 0x7f712c063cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f71319a0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f71319a0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f712c069a60 con 0x7f712c1542c0 2023-11-28T03:07:56.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f71319a0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f712c1584d0 con 0x7f712c1546a0 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f71319a0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c06a280 con 0x7f712c0695d0 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712a7fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f712c1546a0 0x7f712c063cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712a7fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f712c1546a0 0x7f712c063cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:36600/0 (socket says 172.21.15.73:36600) 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35482/0 (socket says 172.21.15.73:35482) 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712a7fc700 1 -- 172.21.15.73:0/3656545315 learned_addr learned my addr 172.21.15.73:0/3656545315 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.238+0000 7f712affd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f712c1542c0 0x7f712c158960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:56884/0 (socket says 172.21.15.73:56884) 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712a7fc700 1 -- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 msgr2=0x7f712c0c4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712a7fc700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712a7fc700 1 -- 172.21.15.73:0/3656545315 >> v1:172.21.15.150:6789/0 conn(0x7f712c1542c0 legacy=0x7f712c158960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712a7fc700 1 -- 172.21.15.73:0/3656545315 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f712c069ce0 con 0x7f712c1546a0 2023-11-28T03:07:56.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712b7fe700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f712a7fc700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c1546a0 0x7f712c063cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f711c006fd0 tx=0x7f711c0090f0).ready entity=mon.1 client_cookie=481d014db00e5978 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f7129ffb700 1 -- 172.21.15.73:0/3656545315 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c00b580 con 0x7f712c1546a0 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f7129ffb700 1 -- 172.21.15.73:0/3656545315 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711c00b6e0 con 0x7f712c1546a0 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f7129ffb700 1 -- 172.21.15.73:0/3656545315 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711c00ba80 con 0x7f712c1546a0 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 -- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c1546a0 msgr2=0x7f712c063cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c1546a0 0x7f712c063cf0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 rx=0x7f711c006fd0 tx=0x7f711c0090f0).stop 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 -- 172.21.15.73:0/3656545315 shutdown_connections 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c0695d0 0x7f712c0c4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 --2- 172.21.15.73:0/3656545315 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c1546a0 0x7f712c063cf0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 -- 172.21.15.73:0/3656545315 >> 172.21.15.73:0/3656545315 conn(0x7f712c0c7f10 msgr2=0x7f712c0c3f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 -- 172.21.15.73:0/3656545315 shutdown_connections 2023-11-28T03:07:56.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.239+0000 7f71319a0700 1 -- 172.21.15.73:0/3656545315 wait complete. 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 Processor -- start 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 -- start start 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c0695d0 0x7f712c0c0a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c1542c0 0x7f712c1441c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712c1546a0 0x7f712c0c0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0c1d20 con 0x7f712c1546a0 2023-11-28T03:07:56.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f712c1446b0 con 0x7f712c0695d0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.241+0000 7f71319a0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f712c0c0820 con 0x7f712c1542c0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712c1546a0 0x7f712c0c0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712c1546a0 0x7f712c0c0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34640/0 (socket says 172.21.15.73:34640) 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 -- 172.21.15.73:0/3279710967 learned_addr learned my addr 172.21.15.73:0/3279710967 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c0695d0 0x7f712c0c0a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712a7fc700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c1542c0 0x7f712c1441c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c1542c0 msgr2=0x7f712c1441c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f712c1542c0 0x7f712c1441c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c0695d0 msgr2=0x7f712c0c0a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f712c0695d0 0x7f712c0c0a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7124009a20 con 0x7f712c1546a0 2023-11-28T03:07:56.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.242+0000 7f712b7fe700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f712c1546a0 0x7f712c0c0330 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 rx=0x7f7124009f20 tx=0x7f712400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.244+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f712400cb70 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.244+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f712400f040 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.244+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7124013400 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.244+0000 7f71319a0700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f711c009370 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.244+0000 7f71319a0700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f712c0c13f0 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f71319a0700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712c0c1e90 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 32) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f71240137f0 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f7112ffd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f712affd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f712407aa80 con 0x7f712c1546a0 2023-11-28T03:07:56.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.245+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:07:56.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.250+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f712404bd50 con 0x7f712c1546a0 2023-11-28T03:07:56.259 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:56.258+0000 7fdc0c4f0680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:56.276 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:56.275+0000 7f78795ab680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:07:56.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.445+0000 7f71319a0700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f712c06c7c0 con 0x7f711805f1d0 2023-11-28T03:07:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.446+0000 7f712affd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:56.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.446+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:07:56.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.847+0000 7f712affd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:56.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:56.847+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:07:56.872 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:56.871+0000 7fdc0c4f0680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:56.893 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:56.892+0000 7f78795ab680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:07:56.944 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:56.943+0000 7fdc0c4f0680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:56.965 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:56.964+0000 7f78795ab680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:07:57.129 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.128+0000 7fdc0c4f0680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:57.150 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.149+0000 7f78795ab680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:07:57.547 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.546+0000 7fdc0c4f0680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:57.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.568+0000 7f78795ab680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:07:57.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:57.648+0000 7f712affd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:57.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:57.648+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:07:57.652 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.651+0000 7fdc0c4f0680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:57.673 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.672+0000 7f78795ab680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:57.805 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.804+0000 7fdc0c4f0680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:57.826 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.825+0000 7f78795ab680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:57.880 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.879+0000 7fdc0c4f0680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:57.900 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.899+0000 7f78795ab680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:07:57.947 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:57.946+0000 7fdc0c4f0680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:57.967 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:57.966+0000 7f78795ab680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:07:58.206 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:58.206+0000 7fdc0c4f0680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:58.224 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:58.223+0000 7f78795ab680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:07:58.389 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:58.388+0000 7fdc0c4f0680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:58.406 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:58.406+0000 7f78795ab680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:07:59.039 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.038+0000 7fdc0c4f0680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:59.066 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.064+0000 7f78795ab680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:07:59.106 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.105+0000 7fdc0c4f0680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:59.133 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.133+0000 7f78795ab680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:07:59.180 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.179+0000 7fdc0c4f0680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:59.208 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.208+0000 7f78795ab680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:07:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:59.250+0000 7f712affd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:07:59.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:07:59.250+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:07:59.326 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.325+0000 7fdc0c4f0680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:59.355 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.355+0000 7f78795ab680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:07:59.396 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.395+0000 7fdc0c4f0680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:59.427 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.426+0000 7f78795ab680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:07:59.512 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.511+0000 7fdc0c4f0680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:59.541 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.541+0000 7f78795ab680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:07:59.620 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.618+0000 7fdc0c4f0680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:59.649 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.649+0000 7f78795ab680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:07:59.920 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.919+0000 7fdc0c4f0680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:59.938 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:07:59.938+0000 7f78795ab680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:07:59.991 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:07:59.990+0000 7fdc0c4f0680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:00.009 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.010+0000 7f78795ab680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:00.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:00.029+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 33) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f7124078850 con 0x7f712c1546a0 2023-11-28T03:08:00.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:00.029+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 msgr2=0x7f7118061690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:00.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:00.029+0000 7f7112ffd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:00.093 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77edd31700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77edd31700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77edd31700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77edd31700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77f0536700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77f0536700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77f0536700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77f0536700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77f0536700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:00.094 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:00.092+0000 7f77edd31700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.032+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f7124078bd0 con 0x7f712c1546a0 2023-11-28T03:08:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.032+0000 7f7112ffd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.032+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f712c06c7c0 con 0x7f711805f1d0 2023-11-28T03:08:01.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.033+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.035+0000 7f712affd700 1 --2- 172.21.15.73:0/3279710967 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f711805f1d0 0x7f7118061610 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 rx=0x7f7120007fd0 tx=0x7f7120007d30).ready entity=mgr.6885 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:01.036 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:01.037+0000 7f77f8546700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-28T03:08:01.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.037+0000 7f7112ffd700 1 -- 172.21.15.73:0/3279710967 <== mgr.6885 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f712c06c7c0 con 0x7f711805f1d0 2023-11-28T03:08:01.038 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-28T03:08:01.042 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-28T03:08:01.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:08:01.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.157+0000 7fc42db2a700 1 Processor -- start 2023-11-28T03:08:01.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 -- start start 2023-11-28T03:08:01.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc42806a400 0x7fc428150390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4280bdd90 con 0x7fc4280c20c0 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4281554d0 con 0x7fc42806a400 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.158+0000 7fc42db2a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4281548c0 con 0x7fc4280c24a0 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc42806a400 0x7fc428150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc426ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4277fe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc4280c20c0 0x7fc4280bd680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:56888/0 (socket says 172.21.15.73:56888) 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc426ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35612/0 (socket says 172.21.15.73:35612) 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc42806a400 0x7fc428150390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:36736/0 (socket says 172.21.15.73:36736) 2023-11-28T03:08:01.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4277fe700 1 -- 172.21.15.73:0/1618673718 learned_addr learned my addr 172.21.15.73:0/1618673718 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:01.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4267fc700 1 -- 172.21.15.73:0/1618673718 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1331524513 0 0) 0x7fc418001110 con 0x7fc4280c20c0 2023-11-28T03:08:01.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4267fc700 1 -- 172.21.15.73:0/1618673718 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4100033f0 con 0x7fc4280c20c0 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 -- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 msgr2=0x7fc428069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 -- 172.21.15.73:0/1618673718 >> v1:172.21.15.150:6789/0 conn(0x7fc4280c20c0 legacy=0x7fc4280bd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 -- 172.21.15.73:0/1618673718 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428154ba0 con 0x7fc42806a400 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc426ffd700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc427fff700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc42806a400 0x7fc428150390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc42000af70 tx=0x7fc42000a890).ready entity=mon.1 client_cookie=e58c4efc26fd8670 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4267fc700 1 -- 172.21.15.73:0/1618673718 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000f070 con 0x7fc42806a400 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4267fc700 1 -- 172.21.15.73:0/1618673718 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc42000a2e0 con 0x7fc42806a400 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.159+0000 7fc4267fc700 1 -- 172.21.15.73:0/1618673718 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc42000cb80 con 0x7fc42806a400 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 -- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc42806a400 msgr2=0x7fc428150390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc42806a400 0x7fc428150390 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 rx=0x7fc42000af70 tx=0x7fc42000a890).stop 2023-11-28T03:08:01.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 -- 172.21.15.73:0/1618673718 shutdown_connections 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc42806a400 0x7fc428150390 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1618673718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc428069f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 -- 172.21.15.73:0/1618673718 >> 172.21.15.73:0/1618673718 conn(0x7fc428062700 msgr2=0x7fc428062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 -- 172.21.15.73:0/1618673718 shutdown_connections 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.160+0000 7fc42db2a700 1 -- 172.21.15.73:0/1618673718 wait complete. 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.161+0000 7fc42db2a700 1 Processor -- start 2023-11-28T03:08:01.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.161+0000 7fc42db2a700 1 -- start start 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4280c20c0 0x7fc4280aa120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc4280ac5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280bede0 con 0x7fc42806a400 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280acad0 con 0x7fc4280c20c0 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc42db2a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280be4d0 con 0x7fc4280c24a0 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34720/0 (socket says 172.21.15.73:34720) 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 -- 172.21.15.73:0/1994550828 learned_addr learned my addr 172.21.15.73:0/1994550828 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc427fff700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc4280ac5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc426ffd700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4280c20c0 0x7fc4280aa120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 -- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 msgr2=0x7fc4280ac5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc4280ac5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 -- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4280c20c0 msgr2=0x7fc4280aa120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4280c20c0 0x7fc4280aa120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.162+0000 7fc4277fe700 1 -- 172.21.15.73:0/1994550828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc41c0092a0 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc4277fe700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fc41800c440 tx=0x7fc418007ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4180070d0 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc418007250 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc42000ab30 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418007610 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.163+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4280be720 con 0x7fc42806a400 2023-11-28T03:08:01.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.164+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc428094f60 con 0x7fc42806a400 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.164+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 34) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fc4180059b0 con 0x7fc42806a400 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.164+0000 7fc40f7fe700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 0x7fc414061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.164+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fc4180793e0 con 0x7fc42806a400 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.164+0000 7fc426ffd700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 0x7fc414061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.165+0000 7fc426ffd700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 0x7fc414061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc4280c03b0 tx=0x7fc41c009060).ready entity=mgr.6885 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:01.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.168+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc41804a7d0 con 0x7fc42806a400 2023-11-28T03:08:01.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:01.376+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fc4280bf050 con 0x7fc42806a400 2023-11-28T03:08:02.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.255+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fc4180772d0 con 0x7fc42806a400 2023-11-28T03:08:02.256 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.256+0000 7fc40f7fe700 1 -- 172.21.15.73:0/1994550828 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v35) v1 ==== 93+0+0 (secure 0 0 0) 0x7fc418075bc0 con 0x7fc42806a400 2023-11-28T03:08:02.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 msgr2=0x7fc414061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 0x7fc414061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc4280c03b0 tx=0x7fc41c009060).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 msgr2=0x7fc428162000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 rx=0x7fc41800c440 tx=0x7fc418007ab0).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 shutdown_connections 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc41405f080 0x7fc414061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c24a0 0x7fc4280ac5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc42806a400 0x7fc428162000 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 --2- 172.21.15.73:0/1994550828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4280c20c0 0x7fc4280aa120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 >> 172.21.15.73:0/1994550828 conn(0x7fc428062700 msgr2=0x7fc4280c6900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.258+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 shutdown_connections 2023-11-28T03:08:02.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.259+0000 7fc42db2a700 1 -- 172.21.15.73:0/1994550828 wait complete. 2023-11-28T03:08:02.272 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:08:02.272 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:08:02.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:08:02.348 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:08:02.349 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:08:02.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:08:02.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:08:02.389 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:08:02.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:02.389 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:08:02.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:02.399+0000 7ff02c8f5680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:08:02.401 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:02.401+0000 7fd95f643680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:08:02.413 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:02.464 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:02.464+0000 7ff02c8f5680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:08:02.464 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:02.463+0000 7fd95f643680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:08:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.571+0000 7f78756f3700 1 Processor -- start 2023-11-28T03:08:02.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.571+0000 7f78756f3700 1 -- start start 2023-11-28T03:08:02.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.571+0000 7f78756f3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f78701544e0 0x7f78701548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.571+0000 7f78756f3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78700c6e30 0x7f78700c72a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f78756f3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78700c69a0 con 0x7f787006a610 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f78756f3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700c13a0 con 0x7f78700c6e30 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f78756f3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78700c1680 con 0x7f78701544e0 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78700c6e30 0x7f78700c72a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786e7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f787006a610 0x7f78700c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50052/0 (socket says 172.21.15.73:50052) 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78700c6e30 0x7f78700c72a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34746/0 (socket says 172.21.15.73:34746) 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786e7fc700 1 -- 172.21.15.73:0/3348895116 learned_addr learned my addr 172.21.15.73:0/3348895116 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786effd700 1 --2- 172.21.15.73:0/3348895116 >> v2:172.21.15.150:3300/0 conn(0x7f78701544e0 0x7f78701548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 -- 172.21.15.73:0/3348895116 >> v2:172.21.15.150:3300/0 conn(0x7f78701544e0 msgr2=0x7f78701548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 --2- 172.21.15.73:0/3348895116 >> v2:172.21.15.150:3300/0 conn(0x7f78701544e0 0x7f78701548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 -- 172.21.15.73:0/3348895116 >> v1:172.21.15.73:6789/0 conn(0x7f787006a610 legacy=0x7f78700c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 -- 172.21.15.73:0/3348895116 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78700c6630 con 0x7f78700c6e30 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786dffb700 1 -- 172.21.15.73:0/3348895116 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3188864730 0 0) 0x7f78700c69a0 con 0x7f787006a610 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786f7fe700 1 --2- 172.21.15.73:0/3348895116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78700c72a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f7870085760 tx=0x7f7868009780).ready entity=mon.0 client_cookie=5426ff97c81206db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786dffb700 1 -- 172.21.15.73:0/3348895116 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786800e070 con 0x7f78700c6e30 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786dffb700 1 -- 172.21.15.73:0/3348895116 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786800bb60 con 0x7f78700c6e30 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.572+0000 7f786dffb700 1 -- 172.21.15.73:0/3348895116 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7868012410 con 0x7f78700c6e30 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 -- 172.21.15.73:0/3348895116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 msgr2=0x7f78700c72a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 --2- 172.21.15.73:0/3348895116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78700c72a0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 rx=0x7f7870085760 tx=0x7f7868009780).stop 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 -- 172.21.15.73:0/3348895116 shutdown_connections 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 --2- 172.21.15.73:0/3348895116 >> v2:172.21.15.150:3300/0 conn(0x7f78701544e0 0x7f78701548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 --2- 172.21.15.73:0/3348895116 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78700c72a0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 -- 172.21.15.73:0/3348895116 >> 172.21.15.73:0/3348895116 conn(0x7f7870085020 msgr2=0x7f7870094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 -- 172.21.15.73:0/3348895116 shutdown_connections 2023-11-28T03:08:02.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.573+0000 7f78756f3700 1 -- 172.21.15.73:0/3348895116 wait complete. 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.574+0000 7f78756f3700 1 Processor -- start 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.574+0000 7f78756f3700 1 -- start start 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787006a610 0x7f7870150090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78701544e0 0x7f78701508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787014e230 con 0x7f78700c6e30 2023-11-28T03:08:02.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7870098d00 con 0x7f787006a610 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f78756f3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7870098fe0 con 0x7f78701544e0 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34752/0 (socket says 172.21.15.73:34752) 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 -- 172.21.15.73:0/1693233620 learned_addr learned my addr 172.21.15.73:0/1693233620 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786effd700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787006a610 0x7f7870150090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786f7fe700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78701544e0 0x7f78701508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78701544e0 msgr2=0x7f78701508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78701544e0 0x7f78701508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787006a610 msgr2=0x7f7870150090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787006a610 0x7f7870150090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.575+0000 7f786e7fc700 1 -- 172.21.15.73:0/1693233620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7868009a20 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f786e7fc700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f786000c490 tx=0x7f7860007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600070d0 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7860007230 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787014dbe0 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78600075d0 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.576+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78701647c0 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787006c860 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f7860005bc0 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f7856ffd700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 0x7f785c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f786effd700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 msgr2=0x7f785c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7860079d70 con 0x7f78700c6e30 2023-11-28T03:08:02.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.577+0000 7f786effd700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 0x7f785c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:02.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.581+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f786004ad90 con 0x7f78700c6e30 2023-11-28T03:08:02.739 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:02.738+0000 7fd95f643680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:08:02.751 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:02.751+0000 7ff02c8f5680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:08:02.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.778+0000 7f786effd700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 msgr2=0x7f785c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:02.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.778+0000 7f786effd700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 0x7f785c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:02.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.796+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78700c5890 con 0x7f78700c6e30 2023-11-28T03:08:02.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.797+0000 7f7856ffd700 1 -- 172.21.15.73:0/1693233620 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f786004e3b0 con 0x7f78700c6e30 2023-11-28T03:08:02.797 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:02.797 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 msgr2=0x7f785c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 0x7f785c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 msgr2=0x7f78701642d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 rx=0x7f786000c490 tx=0x7f7860007e60).stop 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 shutdown_connections 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f785c05f300 0x7f785c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:02.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78701544e0 0x7f78701508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787006a610 0x7f7870150090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 --2- 172.21.15.73:0/1693233620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78700c6e30 0x7f78701642d0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 >> 172.21.15.73:0/1693233620 conn(0x7f7870085020 msgr2=0x7f78700cbc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 shutdown_connections 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.799+0000 7f78756f3700 1 -- 172.21.15.73:0/1693233620 wait complete. 2023-11-28T03:08:02.800 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:08:02.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:02.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.967+0000 7fadda9ca700 1 Processor -- start 2023-11-28T03:08:02.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.967+0000 7fadda9ca700 1 -- start start 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadda9ca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fadd4150890 0x7fadd40bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadda9ca700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd40c28c0 con 0x7fadd406a010 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadda9ca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fadd41552e0 con 0x7fadd40bc320 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadda9ca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fadd406a4b0 con 0x7fadd4150890 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fadd4150890 0x7fadd40bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd8f67700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fadd406a010 0x7fadd41544f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50060/0 (socket says 172.21.15.73:50060) 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd8f67700 1 -- 172.21.15.73:0/2980698875 learned_addr learned my addr 172.21.15.73:0/2980698875 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fadd4150890 0x7fadd40bbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34768/0 (socket says 172.21.15.73:34768) 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 -- 172.21.15.73:0/2980698875 >> v1:172.21.15.150:6789/0 conn(0x7fadd40bc320 legacy=0x7fadd4069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 -- 172.21.15.73:0/2980698875 >> v1:172.21.15.73:6789/0 conn(0x7fadd406a010 legacy=0x7fadd41544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd2ffd700 1 -- 172.21.15.73:0/2980698875 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 397517885 0 0) 0x7fadd40c28c0 con 0x7fadd406a010 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 -- 172.21.15.73:0/2980698875 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd40c1f90 con 0x7fadd4150890 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd3fff700 1 --2- 172.21.15.73:0/2980698875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40bbe30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fadc8009760 tx=0x7fadc8009060).ready entity=mon.0 client_cookie=3e368e74e6abcfef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.968+0000 7fadd2ffd700 1 -- 172.21.15.73:0/2980698875 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800b580 con 0x7fadd4150890 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadd2ffd700 1 -- 172.21.15.73:0/2980698875 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadc800b6e0 con 0x7fadd4150890 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadd2ffd700 1 -- 172.21.15.73:0/2980698875 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadc800ba60 con 0x7fadd4150890 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 -- 172.21.15.73:0/2980698875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 msgr2=0x7fadd40bbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2980698875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40bbe30 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 rx=0x7fadc8009760 tx=0x7fadc8009060).stop 2023-11-28T03:08:02.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 -- 172.21.15.73:0/2980698875 shutdown_connections 2023-11-28T03:08:02.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2980698875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40bbe30 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 -- 172.21.15.73:0/2980698875 >> 172.21.15.73:0/2980698875 conn(0x7fadd40631d0 msgr2=0x7fadd40c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:02.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 -- 172.21.15.73:0/2980698875 shutdown_connections 2023-11-28T03:08:02.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.969+0000 7fadda9ca700 1 -- 172.21.15.73:0/2980698875 wait complete. 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.970+0000 7fadda9ca700 1 Processor -- start 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.970+0000 7fadda9ca700 1 -- start start 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd406a010 0x7fadd4093ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadd40bc320 0x7fadd4143aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4092c90 con 0x7fadd4150890 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fadd4092ee0 con 0x7fadd406a010 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadda9ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fadd40931c0 con 0x7fadd40bc320 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadd8f67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadd8f67700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34772/0 (socket says 172.21.15.73:34772) 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadd8f67700 1 -- 172.21.15.73:0/2576555491 learned_addr learned my addr 172.21.15.73:0/2576555491 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadd3fff700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd406a010 0x7fadd4093ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.971+0000 7fadd37fe700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadd40bc320 0x7fadd4143aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 -- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadd40bc320 msgr2=0x7fadd4143aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadd40bc320 0x7fadd4143aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 -- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd406a010 msgr2=0x7fadd4093ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd406a010 0x7fadd4093ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:02.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 -- 172.21.15.73:0/2576555491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadc80092a0 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadd8f67700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fadcc00c7e0 tx=0x7fadcc007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc005d30 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadcc005e90 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fadd40934a0 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadcc007320 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.972+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fadd4143f90 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.973+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fadd406c860 con 0x7fadd4150890 2023-11-28T03:08:02.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.973+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fadcc007710 con 0x7fadd4150890 2023-11-28T03:08:02.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.974+0000 7fadbbfff700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 0x7fadc00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:02.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.974+0000 7fadd3fff700 1 -- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 msgr2=0x7fadc00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:02.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.974+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fadcc07a490 con 0x7fadd4150890 2023-11-28T03:08:02.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.974+0000 7fadd3fff700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 0x7fadc00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:02.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:02.978+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fadcc04b450 con 0x7fadd4150890 2023-11-28T03:08:03.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.146+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fadd40c2270 con 0x7fadd4150890 2023-11-28T03:08:03.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.147+0000 7fadbbfff700 1 -- 172.21.15.73:0/2576555491 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17174 (secure 0 0 0) 0x7fadcc04ea70 con 0x7fadd4150890 2023-11-28T03:08:03.148 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:03.148 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":97,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:00.022966+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2262264527":"2023-11-29T03:08:00.022924+0000","172.21.15.73:0/3167061175":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:00.022924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 msgr2=0x7fadc00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 0x7fadc00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 msgr2=0x7fadd40ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 rx=0x7fadcc00c7e0 tx=0x7fadcc007f90).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 shutdown_connections 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fadc005f300 0x7fadc00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fadd4150890 0x7fadd40ac090 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fadd406a010 0x7fadd4093ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 --2- 172.21.15.73:0/2576555491 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fadd40bc320 0x7fadd4143aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 >> 172.21.15.73:0/2576555491 conn(0x7fadd40631d0 msgr2=0x7fadd40c6060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 shutdown_connections 2023-11-28T03:08:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.149+0000 7fadda9ca700 1 -- 172.21.15.73:0/2576555491 wait complete. 2023-11-28T03:08:03.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '87', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 11, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:07:46.190668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '95', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 12, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:07:46.631129+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '96', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:08:03.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:08:03.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.288+0000 7f434cc48700 1 Processor -- start 2023-11-28T03:08:03.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.288+0000 7f434cc48700 1 -- start start 2023-11-28T03:08:03.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f434cc48700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4348158160 0x7f43480a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f434cc48700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4348067770 con 0x7f434814e600 2023-11-28T03:08:03.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f434cc48700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43480be8d0 con 0x7f43480a8280 2023-11-28T03:08:03.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f434cc48700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4348159180 con 0x7f4348158160 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f4346d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f434814e600 0x7f434814ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:50066/0 (socket says 172.21.15.73:50066) 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f4346d9d700 1 -- 172.21.15.73:0/3346344040 learned_addr learned my addr 172.21.15.73:0/3346344040 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.289+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> v2:172.21.15.150:3300/0 conn(0x7f4348158160 0x7f43480a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1456296748 0 0) 0x7f4348067770 con 0x7f434814e600 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f43300033f0 con 0x7f434814e600 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 369437305 0 0) 0x7f43300033f0 con 0x7f434814e600 2023-11-28T03:08:03.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 >> v2:172.21.15.150:3300/0 conn(0x7f4348158160 msgr2=0x7f43480a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 --2- 172.21.15.73:0/3346344040 >> v2:172.21.15.150:3300/0 conn(0x7f4348158160 0x7f43480a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 >> v1:172.21.15.150:6789/0 conn(0x7f43480a8280 legacy=0x7f4348155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348158600 con 0x7f434814e600 2023-11-28T03:08:03.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f43400017f0 con 0x7f434814e600 2023-11-28T03:08:03.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> v2:172.21.15.150:3300/0 conn(0x7f4348158160 0x7f43480a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 >> v1:172.21.15.73:6789/0 conn(0x7f434814e600 legacy=0x7f434814ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43300038e0 0x7f4330003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 0x7f4330004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330005000 0x7f433000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348159180 con 0x7f4330004690 2023-11-28T03:08:03.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f433000be00 con 0x7f4330005000 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f433000bfc0 con 0x7f43300038e0 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 0x7f4330004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434659c700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43300038e0 0x7f4330003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4346d9d700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330005000 0x7f433000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 -- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43300038e0 msgr2=0x7f4330003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43300038e0 0x7f4330003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 -- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330005000 msgr2=0x7f433000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330005000 0x7f433000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 -- 172.21.15.73:0/3346344040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43300033f0 con 0x7f4330004690 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4346d9d700 1 -- 172.21.15.73:0/3346344040 reap_dead start 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f4345d9b700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 0x7f4330004b10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f4338009c70 tx=0x7f4338009060).ready entity=mon.0 client_cookie=60a808d6ed19fce4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.290+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43380095c0 con 0x7f4330004690 2023-11-28T03:08:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433800e690 con 0x7f4330004690 2023-11-28T03:08:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434559a700 1 -- 172.21.15.73:0/3346344040 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433800ea10 con 0x7f4330004690 2023-11-28T03:08:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 -- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 msgr2=0x7f4330004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 0x7f4330004b10 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 rx=0x7f4338009c70 tx=0x7f4338009060).stop 2023-11-28T03:08:03.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 -- 172.21.15.73:0/3346344040 shutdown_connections 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 --2- 172.21.15.73:0/3346344040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4330004690 0x7f4330004b10 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 -- 172.21.15.73:0/3346344040 >> 172.21.15.73:0/3346344040 conn(0x7f4348062700 msgr2=0x7f4348062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 -- 172.21.15.73:0/3346344040 shutdown_connections 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.291+0000 7f434cc48700 1 -- 172.21.15.73:0/3346344040 wait complete. 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.292+0000 7f434cc48700 1 Processor -- start 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.292+0000 7f434cc48700 1 -- start start 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 0x7f4348155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348156ca0 con 0x7f434814de80 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4348155e10 con 0x7f4330004690 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434cc48700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4348156090 con 0x7f4348157130 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f4346d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434659c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:36814/0 (socket says 172.21.15.73:36814) 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f434659c700 1 -- 172.21.15.73:0/398464149 learned_addr learned my addr 172.21.15.73:0/398464149 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:03.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f4346d9d700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34786/0 (socket says 172.21.15.73:34786) 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.293+0000 7f4345d9b700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 0x7f4348155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 msgr2=0x7f4348155920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 0x7f4348155920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 msgr2=0x7f43480ab060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 -- 172.21.15.73:0/398464149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4348156370 con 0x7f434814de80 2023-11-28T03:08:03.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f434659c700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:03.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4345d9b700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 0x7f4348155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:03.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f4346d9d700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f4348154bb0 tx=0x7f43480bea60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000e580 con 0x7f434814de80 2023-11-28T03:08:03.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434000e6e0 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434000ea80 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4348156650 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.294+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43481575b0 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.295+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4348156e50 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.296+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f4340014440 con 0x7f434814de80 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.296+0000 7f43367fc700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 0x7f43280617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.296+0000 7f434659c700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 msgr2=0x7f43280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:03.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.296+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f434007a610 con 0x7f434814de80 2023-11-28T03:08:03.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.296+0000 7f434659c700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 0x7f43280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:03.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.300+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4348156e50 con 0x7f434814de80 2023-11-28T03:08:03.355 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:03.354+0000 7fd95f643680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:08:03.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:03.398+0000 7ff02c8f5680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:08:03.426 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:03.425+0000 7fd95f643680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:08:03.472 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:03.471+0000 7ff02c8f5680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:08:03.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.497+0000 7f434659c700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 msgr2=0x7f43280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:03.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.497+0000 7f434659c700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 0x7f43280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:03.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.506+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4348156e50 con 0x7f434814de80 2023-11-28T03:08:03.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.506+0000 7f43367fc700 1 -- 172.21.15.73:0/398464149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v97) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4348156e50 con 0x7f434814de80 2023-11-28T03:08:03.507 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:08:03.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 msgr2=0x7f43280617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:03.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 0x7f43280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 msgr2=0x7f43480a8c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 rx=0x7f4348154bb0 tx=0x7f43480bea60).stop 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 shutdown_connections 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f432805f300 0x7f43280617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f434814de80 0x7f43480a8c60 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4330004690 0x7f43480ab060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 --2- 172.21.15.73:0/398464149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4348157130 0x7f4348155920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 >> 172.21.15.73:0/398464149 conn(0x7f4348062700 msgr2=0x7f43480becb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 shutdown_connections 2023-11-28T03:08:03.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.508+0000 7f434cc48700 1 -- 172.21.15.73:0/398464149 wait complete. 2023-11-28T03:08:03.522 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:08:03.613 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:03.612+0000 7fd95f643680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:08:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.640+0000 7f351fdbe700 1 Processor -- start 2023-11-28T03:08:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.640+0000 7f351fdbe700 1 -- start start 2023-11-28T03:08:03.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351fdbe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f351814ed10 0x7f35180a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351fdbe700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518067770 con 0x7f35180a8280 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351fdbe700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35180be8d0 con 0x7f3518158160 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351fdbe700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518159180 con 0x7f351814ed10 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351db5a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f35180a8280 0x7f3518155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33110/0 (socket says 172.21.15.73:33110) 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351db5a700 1 -- 172.21.15.73:0/1635705475 learned_addr learned my addr 172.21.15.73:0/1635705475 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351e35b700 1 --2- 172.21.15.73:0/1635705475 >> v2:172.21.15.150:3300/0 conn(0x7f351814ed10 0x7f35180a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637648614 0 0) 0x7f3518067770 con 0x7f35180a8280 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.641+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f35040033f0 con 0x7f35180a8280 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1637340772 0 0) 0x7f35180be8d0 con 0x7f3518158160 2023-11-28T03:08:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3518067770 con 0x7f3518158160 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1584282617 0 0) 0x7f35040033f0 con 0x7f35180a8280 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 >> v2:172.21.15.150:3300/0 conn(0x7f351814ed10 msgr2=0x7f35180a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 --2- 172.21.15.73:0/1635705475 >> v2:172.21.15.150:3300/0 conn(0x7f351814ed10 0x7f35180a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 >> v1:172.21.15.150:6789/0 conn(0x7f3518158160 legacy=0x7f351814e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518158600 con 0x7f35180a8280 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f3508001470 con 0x7f35180a8280 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 >> v1:172.21.15.73:6789/0 conn(0x7f35180a8280 legacy=0x7f3518155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 0x7f3504003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3504004690 0x7f3504004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504005000 0x7f350400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518159180 con 0x7f3504003a70 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f350400be80 con 0x7f3504004690 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f350400c040 con 0x7f3504005000 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 0x7f3504003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351e35b700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504005000 0x7f350400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351d359700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3504004690 0x7f3504004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 -- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504005000 msgr2=0x7f350400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504005000 0x7f350400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 -- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3504004690 msgr2=0x7f3504004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3504004690 0x7f3504004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 -- 172.21.15.73:0/1635705475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35040033f0 con 0x7f3504003a70 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351e35b700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504005000 0x7f350400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351e35b700 1 -- 172.21.15.73:0/1635705475 reap_dead start 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351db5a700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 0x7f3504003e70 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f350800c160 tx=0x7f35180bea60).ready entity=mon.0 client_cookie=a33724636591e239 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350800ca30 con 0x7f3504003a70 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f350800cb90 con 0x7f3504003a70 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.642+0000 7f351cb58700 1 -- 172.21.15.73:0/1635705475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35080070d0 con 0x7f3504003a70 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 -- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 msgr2=0x7f3504003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 0x7f3504003e70 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 rx=0x7f350800c160 tx=0x7f35180bea60).stop 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 -- 172.21.15.73:0/1635705475 shutdown_connections 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 --2- 172.21.15.73:0/1635705475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3504003a70 0x7f3504003e70 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 -- 172.21.15.73:0/1635705475 >> 172.21.15.73:0/1635705475 conn(0x7f3518062700 msgr2=0x7f3518062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 -- 172.21.15.73:0/1635705475 shutdown_connections 2023-11-28T03:08:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.643+0000 7f351fdbe700 1 -- 172.21.15.73:0/1635705475 wait complete. 2023-11-28T03:08:03.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.644+0000 7f351fdbe700 1 Processor -- start 2023-11-28T03:08:03.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.644+0000 7f351fdbe700 1 -- start start 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 0x7f35180ab090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 0x7f35181593e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35180aac70 con 0x7f3518065330 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518156e20 con 0x7f35181569c0 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351fdbe700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f351815a180 con 0x7f3504003a70 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59642/0 (socket says 172.21.15.73:59642) 2023-11-28T03:08:03.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 -- 172.21.15.73:0/304720036 learned_addr learned my addr 172.21.15.73:0/304720036 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351db5a700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 0x7f35180ab090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351e35b700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 0x7f35181593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 -- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 msgr2=0x7f35180ab090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 0x7f35180ab090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 -- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 msgr2=0x7f35181593e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.645+0000 7f351d359700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 0x7f35181593e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351d359700 1 -- 172.21.15.73:0/304720036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35180aa5b0 con 0x7f3518065330 2023-11-28T03:08:03.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351db5a700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 0x7f35180ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351e35b700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 0x7f35181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351d359700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f351400e810 tx=0x7f3514009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514001570 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35140016d0 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35180aa890 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3514001a50 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.646+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35180aa260 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.647+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f351806c860 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.647+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f3514014430 con 0x7f3518065330 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.648+0000 7f350dffb700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 0x7f34fc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.648+0000 7f351db5a700 1 -- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 msgr2=0x7f34fc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:03.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.648+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f351407a340 con 0x7f3518065330 2023-11-28T03:08:03.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.648+0000 7f351db5a700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 0x7f34fc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:03.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.652+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3514049af0 con 0x7f3518065330 2023-11-28T03:08:03.668 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:03.667+0000 7ff02c8f5680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:08:03.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.825+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f35181588e0 con 0x7f3518065330 2023-11-28T03:08:03.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.825+0000 7f350dffb700 1 -- 172.21.15.73:0/304720036 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v97) v1 ==== 92+0+10 (secure 0 0 0) 0x7f351404b300 con 0x7f3518065330 2023-11-28T03:08:03.826 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:08:03.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 msgr2=0x7f34fc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:03.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 0x7f34fc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 msgr2=0x7f35181564d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 rx=0x7f351400e810 tx=0x7f3514009040).stop 2023-11-28T03:08:03.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 shutdown_connections 2023-11-28T03:08:03.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34fc05f300 0x7f34fc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:03.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f35181569c0 0x7f35181593e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3504003a70 0x7f35180ab090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 --2- 172.21.15.73:0/304720036 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3518065330 0x7f35181564d0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 >> 172.21.15.73:0/304720036 conn(0x7f3518062700 msgr2=0x7f35181576f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 shutdown_connections 2023-11-28T03:08:03.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.827+0000 7f351fdbe700 1 -- 172.21.15.73:0/304720036 wait complete. 2023-11-28T03:08:03.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:08:03.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.962+0000 7fcbe88ec700 1 Processor -- start 2023-11-28T03:08:03.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.962+0000 7fcbe88ec700 1 -- start start 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe88ec700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcbe40c51a0 0x7fcbe4153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe88ec700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbe40c5ea0 0x7fcbe4066a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe88ec700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcbe4155f20 con 0x7fcbe40c5ac0 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe88ec700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe4153990 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe88ec700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcbe40c62d0 con 0x7fcbe40c51a0 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe2d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbe40c5ea0 0x7fcbe4066a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe2d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcbe40c5ea0 0x7fcbe4066a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59648/0 (socket says 172.21.15.73:59648) 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.963+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/159643910 learned_addr learned my addr 172.21.15.73:0/159643910 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe1d9b700 1 --2- 172.21.15.73:0/159643910 >> v2:172.21.15.150:3300/0 conn(0x7fcbe40c51a0 0x7fcbe4153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/159643910 >> v2:172.21.15.150:3300/0 conn(0x7fcbe40c51a0 msgr2=0x7fcbe4153390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe2d9d700 1 --2- 172.21.15.73:0/159643910 >> v2:172.21.15.150:3300/0 conn(0x7fcbe40c51a0 0x7fcbe4153390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/159643910 >> v1:172.21.15.150:6789/0 conn(0x7fcbe40c5ac0 legacy=0x7fcbe4156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/159643910 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbe4156200 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe2d9d700 1 --2- 172.21.15.73:0/159643910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe4066a70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fcbe4068200 tx=0x7fcbdc009840).ready entity=mon.0 client_cookie=80bda57e4286f58 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe159a700 1 -- 172.21.15.73:0/159643910 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00e070 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe159a700 1 -- 172.21.15.73:0/159643910 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc00ba50 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe159a700 1 -- 172.21.15.73:0/159643910 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc00bdd0 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe88ec700 1 -- 172.21.15.73:0/159643910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 msgr2=0x7fcbe4066a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.964+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/159643910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe4066a70 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 rx=0x7fcbe4068200 tx=0x7fcbdc009840).stop 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 -- 172.21.15.73:0/159643910 shutdown_connections 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/159643910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe4066a70 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/159643910 >> v2:172.21.15.150:3300/0 conn(0x7fcbe40c51a0 0x7fcbe4153390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 -- 172.21.15.73:0/159643910 >> 172.21.15.73:0/159643910 conn(0x7fcbe4062700 msgr2=0x7fcbe4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 -- 172.21.15.73:0/159643910 shutdown_connections 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.965+0000 7fcbe88ec700 1 -- 172.21.15.73:0/159643910 wait complete. 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 Processor -- start 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 -- start start 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbe40c51a0 0x7fcbe4164350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe40c5ac0 0x7fcbe4065310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe41652d0 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe4164840 con 0x7fcbe40c5ac0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.966+0000 7fcbe88ec700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcbe4167b30 con 0x7fcbe40c51a0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59656/0 (socket says 172.21.15.73:59656) 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/3737408272 learned_addr learned my addr 172.21.15.73:0/3737408272 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe1d9b700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe40c5ac0 0x7fcbe4065310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe259c700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbe40c51a0 0x7fcbe4164350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbe40c51a0 msgr2=0x7fcbe4164350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbe40c51a0 0x7fcbe4164350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe40c5ac0 msgr2=0x7fcbe4065310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe40c5ac0 0x7fcbe4065310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 -- 172.21.15.73:0/3737408272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbdc009ae0 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe2d9d700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fcbdc009fd0 tx=0x7fcbe4156170).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc0092e0 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbdc01e470 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcbe4065800 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbdc01e810 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.967+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcbe4164d10 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.968+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fcbdc003040 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.969+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcbe4167e10 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.969+0000 7fcbd27fc700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 0x7fcbc40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.969+0000 7fcbe259c700 1 -- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 msgr2=0x7fcbc40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.969+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fcbdc07ba90 con 0x7fcbe40c5ea0 2023-11-28T03:08:03.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.969+0000 7fcbe259c700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 0x7fcbc40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:03.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:03.973+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcbdc04ca10 con 0x7fcbe40c5ea0 2023-11-28T03:08:04.036 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.035+0000 7fd95f643680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:08:04.108 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.107+0000 7ff02c8f5680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:08:04.138 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.138+0000 7fd95f643680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:04.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.145+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcbe406c860 con 0x7fcbe40c5ea0 2023-11-28T03:08:04.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.145+0000 7fcbd27fc700 1 -- 172.21.15.73:0/3737408272 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v97) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcbdc023020 con 0x7fcbe40c5ea0 2023-11-28T03:08:04.146 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:08:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 msgr2=0x7fcbc40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 0x7fcbc40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 msgr2=0x7fcbe40c25f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 rx=0x7fcbdc009fd0 tx=0x7fcbe4156170).stop 2023-11-28T03:08:04.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 shutdown_connections 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbc405f300 0x7fcbc40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcbe40c5ea0 0x7fcbe40c25f0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcbe40c51a0 0x7fcbe4164350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 --2- 172.21.15.73:0/3737408272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcbe40c5ac0 0x7fcbe4065310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 >> 172.21.15.73:0/3737408272 conn(0x7fcbe4062700 msgr2=0x7fcbe40c6d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 shutdown_connections 2023-11-28T03:08:04.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.148+0000 7fcbe88ec700 1 -- 172.21.15.73:0/3737408272 wait complete. 2023-11-28T03:08:04.165 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:08:04.224 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.223+0000 7ff02c8f5680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:04.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.283+0000 7f7441012700 1 Processor -- start 2023-11-28T03:08:04.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.283+0000 7f7441012700 1 -- start start 2023-11-28T03:08:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f7441012700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f743c158160 0x7f743c0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f7441012700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c14e600 0x7f743c14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f7441012700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f743c067770 con 0x7f743c0a8280 2023-11-28T03:08:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f7441012700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f743c159180 con 0x7f743c158160 2023-11-28T03:08:04.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f7441012700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0be8d0 con 0x7f743c14e600 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f743a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f743c158160 0x7f743c0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f743a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f743c158160 0x7f743c0a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59660/0 (socket says 172.21.15.73:59660) 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f743a59c700 1 -- 172.21.15.73:0/2207089229 learned_addr learned my addr 172.21.15.73:0/2207089229 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.284+0000 7f743ad9d700 1 --1- 172.21.15.73:0/2207089229 >> v1:172.21.15.73:6789/0 conn(0x7f743c0a8280 0x7f743c155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33126/0 (socket says 172.21.15.73:33126) 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743b59e700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c14e600 0x7f743c14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743a59c700 1 -- 172.21.15.73:0/2207089229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c14e600 msgr2=0x7f743c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743a59c700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c14e600 0x7f743c14ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f7439d9b700 1 -- 172.21.15.73:0/2207089229 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 988443019 0 0) 0x7f743c067770 con 0x7f743c0a8280 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743a59c700 1 -- 172.21.15.73:0/2207089229 >> v1:172.21.15.73:6789/0 conn(0x7f743c0a8280 legacy=0x7f743c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743a59c700 1 -- 172.21.15.73:0/2207089229 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f743c158600 con 0x7f743c158160 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f743a59c700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c158160 0x7f743c0a8660 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f742c006fd0 tx=0x7f742c0090f0).ready entity=mon.0 client_cookie=151081c9092fb65c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f7439d9b700 1 -- 172.21.15.73:0/2207089229 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00b580 con 0x7f743c158160 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f7439d9b700 1 -- 172.21.15.73:0/2207089229 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00b6e0 con 0x7f743c158160 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.285+0000 7f7439d9b700 1 -- 172.21.15.73:0/2207089229 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c00ba60 con 0x7f743c158160 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 -- 172.21.15.73:0/2207089229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c158160 msgr2=0x7f743c0a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c158160 0x7f743c0a8660 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 rx=0x7f742c006fd0 tx=0x7f742c0090f0).stop 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 -- 172.21.15.73:0/2207089229 shutdown_connections 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c14e600 0x7f743c14ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 --2- 172.21.15.73:0/2207089229 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c158160 0x7f743c0a8660 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 -- 172.21.15.73:0/2207089229 >> 172.21.15.73:0/2207089229 conn(0x7f743c062700 msgr2=0x7f743c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 -- 172.21.15.73:0/2207089229 shutdown_connections 2023-11-28T03:08:04.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.286+0000 7f7441012700 1 -- 172.21.15.73:0/2207089229 wait complete. 2023-11-28T03:08:04.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.287+0000 7f7441012700 1 Processor -- start 2023-11-28T03:08:04.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.287+0000 7f7441012700 1 -- start start 2023-11-28T03:08:04.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c0a8280 0x7f743c14cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c158160 0x7f743c064cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0651c0 con 0x7f743c14e600 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c0ab5b0 con 0x7f743c158160 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f7441012700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c1567b0 con 0x7f743c0a8280 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59670/0 (socket says 172.21.15.73:59670) 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 -- 172.21.15.73:0/1556746350 learned_addr learned my addr 172.21.15.73:0/1556746350 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743ad9d700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c0a8280 0x7f743c14cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743b59e700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c158160 0x7f743c064cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 -- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c0a8280 msgr2=0x7f743c14cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c0a8280 0x7f743c14cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 -- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c158160 msgr2=0x7f743c064cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c158160 0x7f743c064cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 -- 172.21.15.73:0/1556746350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c009370 con 0x7f743c14e600 2023-11-28T03:08:04.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.288+0000 7f743a59c700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f742c009ea0 tx=0x7f742c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.289+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c000bc0 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.289+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f742c00b850 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.289+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c15a680 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.289+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f742c000d20 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.289+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c156160 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f742c01c880 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0a8870 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f7422ffd700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 0x7f74280617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f743ad9d700 1 -- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 msgr2=0x7f74280617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f742c07bbd0 con 0x7f743c14e600 2023-11-28T03:08:04.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.290+0000 7f743ad9d700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 0x7f74280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:04.297 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.292+0000 7fd95f643680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:04.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.295+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f742c04cbf0 con 0x7f743c14e600 2023-11-28T03:08:04.367 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.366+0000 7fd95f643680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:08:04.385 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.384+0000 7ff02c8f5680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:04.434 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.433+0000 7fd95f643680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:08:04.461 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.460+0000 7ff02c8f5680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:08:04.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.464+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f743c155c10 con 0x7f743c14e600 2023-11-28T03:08:04.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.465+0000 7f7422ffd700 1 -- 172.21.15.73:0/1556746350 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v97) v1 ==== 100+0+11 (secure 0 0 0) 0x7f742c020020 con 0x7f743c14e600 2023-11-28T03:08:04.466 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 msgr2=0x7f74280617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 0x7f74280617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 msgr2=0x7f743c0647e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 rx=0x7f742c009ea0 tx=0x7f742c017000).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 shutdown_connections 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f742805f300 0x7f74280617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f743c158160 0x7f743c064cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f743c0a8280 0x7f743c14cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 --2- 172.21.15.73:0/1556746350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f743c14e600 0x7f743c0647e0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 >> 172.21.15.73:0/1556746350 conn(0x7f743c062700 msgr2=0x7f743c062ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 shutdown_connections 2023-11-28T03:08:04.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.468+0000 7f7441012700 1 -- 172.21.15.73:0/1556746350 wait complete. 2023-11-28T03:08:04.486 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:04.541 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.540+0000 7ff02c8f5680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:08:04.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.602+0000 7fe1bcbf8700 1 Processor -- start 2023-11-28T03:08:04.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.602+0000 7fe1bcbf8700 1 -- start start 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1bcbf8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1b80c0180 0x7fe1b8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1bcbf8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80c12c0 con 0x7fe1b80c1750 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1bcbf8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1b80c06b0 con 0x7fe1b8157660 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1bcbf8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1b80c0990 con 0x7fe1b80c0180 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1b7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe1b80c1750 0x7fe1b80648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33140/0 (socket says 172.21.15.73:33140) 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1b7fff700 1 -- 172.21.15.73:0/3304911726 learned_addr learned my addr 172.21.15.73:0/3304911726 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1aeffd700 1 --2- 172.21.15.73:0/3304911726 >> v2:172.21.15.150:3300/0 conn(0x7fe1b80c0180 0x7fe1b8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2142113786 0 0) 0x7fe1b80c12c0 con 0x7fe1b80c1750 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1a40033f0 con 0x7fe1b80c1750 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.603+0000 7fe1aeffd700 1 -- 172.21.15.73:0/3304911726 >> v1:172.21.15.150:6789/0 conn(0x7fe1b8157660 legacy=0x7fe1b81589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2105160899 0 0) 0x7fe1b80c06b0 con 0x7fe1b8157660 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1aeffd700 1 -- 172.21.15.73:0/3304911726 >> v1:172.21.15.73:6789/0 conn(0x7fe1b80c1750 legacy=0x7fe1b80648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1aeffd700 1 -- 172.21.15.73:0/3304911726 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1b80c0c70 con 0x7fe1b80c0180 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1aeffd700 1 --2- 172.21.15.73:0/3304911726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c0180 0x7fe1b8157a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fe1a0009760 tx=0x7fe1a0009060).ready entity=mon.1 client_cookie=8a775f799c5f1f32 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1b00017f0 con 0x7fe1b80c0180 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a0012410 con 0x7fe1b80c0180 2023-11-28T03:08:04.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1b6ffd700 1 -- 172.21.15.73:0/3304911726 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a0012790 con 0x7fe1b80c0180 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/3304911726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c0180 msgr2=0x7fe1b8157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/3304911726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c0180 0x7fe1b8157a40 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 rx=0x7fe1a0009760 tx=0x7fe1a0009060).stop 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/3304911726 shutdown_connections 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.604+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/3304911726 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c0180 0x7fe1b8157a40 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.605+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/3304911726 >> 172.21.15.73:0/3304911726 conn(0x7fe1b80cb000 msgr2=0x7fe1b80cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.605+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/3304911726 shutdown_connections 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.605+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/3304911726 wait complete. 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.606+0000 7fe1bcbf8700 1 Processor -- start 2023-11-28T03:08:04.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.606+0000 7fe1bcbf8700 1 -- start start 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.606+0000 7fe1bcbf8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.606+0000 7fe1bcbf8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c1750 0x7fe1b81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b8157660 0x7fe1b80c8e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8141470 con 0x7fe1b80c0180 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8140860 con 0x7fe1b80c1750 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1b8140b40 con 0x7fe1b8157660 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59676/0 (socket says 172.21.15.73:59676) 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 -- 172.21.15.73:0/626344404 learned_addr learned my addr 172.21.15.73:0/626344404 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1aeffd700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c1750 0x7fe1b81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b7fff700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b8157660 0x7fe1b80c8e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b8157660 msgr2=0x7fe1b80c8e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b8157660 0x7fe1b80c8e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c1750 msgr2=0x7fe1b81518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c1750 0x7fe1b81518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 -- 172.21.15.73:0/626344404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1a00092a0 con 0x7fe1b80c0180 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1b77fe700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fe1a800c3c0 tx=0x7fe1a8007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8007020 con 0x7fe1b80c0180 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1a80071a0 con 0x7fe1b80c0180 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1b8140e20 con 0x7fe1b80c0180 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1a8007560 con 0x7fe1b80c0180 2023-11-28T03:08:04.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.607+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1b81400e0 con 0x7fe1b80c0180 2023-11-28T03:08:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.608+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1b806c7c0 con 0x7fe1b80c0180 2023-11-28T03:08:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.609+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fe1a80068b0 con 0x7fe1b80c0180 2023-11-28T03:08:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.609+0000 7fe1affff700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 0x7fe1980617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.609+0000 7fe1aeffd700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 msgr2=0x7fe1980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:04.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.609+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe1a8079820 con 0x7fe1b80c0180 2023-11-28T03:08:04.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.609+0000 7fe1aeffd700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 0x7fe1980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:04.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.613+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1a8047af0 con 0x7fe1b80c0180 2023-11-28T03:08:04.693 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.692+0000 7fd95f643680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:08:04.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:04.808+0000 7ff02c8f5680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:08:04.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.810+0000 7fe1aeffd700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 msgr2=0x7fe1980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:04.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.810+0000 7fe1aeffd700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 0x7fe1980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:04.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.826+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe1b80c0f50 con 0x7fe1b80c0180 2023-11-28T03:08:04.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.827+0000 7fe1affff700 1 -- 172.21.15.73:0/626344404 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fe1a804a840 con 0x7fe1b80c0180 2023-11-28T03:08:04.829 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:04.829 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 msgr2=0x7fe1980617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 0x7fe1980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 msgr2=0x7fe1b813fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 rx=0x7fe1a800c3c0 tx=0x7fe1a8007e00).stop 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 shutdown_connections 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe19805f300 0x7fe1980617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1b8157660 0x7fe1b80c8e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1b80c0180 0x7fe1b813fbf0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 --2- 172.21.15.73:0/626344404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1b80c1750 0x7fe1b81518c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 >> 172.21.15.73:0/626344404 conn(0x7fe1b80cb000 msgr2=0x7fe1b8157f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 shutdown_connections 2023-11-28T03:08:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.829+0000 7fe1bcbf8700 1 -- 172.21.15.73:0/626344404 wait complete. 2023-11-28T03:08:04.832 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:08:04.842 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:04.877 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:04.876+0000 7fd95f643680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.962+0000 7fb685a9b700 1 Processor -- start 2023-11-28T03:08:04.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.962+0000 7fb685a9b700 1 -- start start 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb685a9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb685a9b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb680062910 con 0x7fb6801545c0 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb685a9b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb680154ac0 con 0x7fb6800c2b10 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb685a9b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6800ab1d0 con 0x7fb680150720 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb67f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb677fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb6801545c0 0x7fb6800a9c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33154/0 (socket says 172.21.15.73:33154) 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb677fff700 1 -- 172.21.15.73:0/1708512788 learned_addr learned my addr 172.21.15.73:0/1708512788 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:04.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb67f7fe700 1 -- 172.21.15.73:0/1708512788 >> v1:172.21.15.150:6789/0 conn(0x7fb6800c2b10 legacy=0x7fb680062da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.963+0000 7fb67f7fe700 1 -- 172.21.15.73:0/1708512788 >> v1:172.21.15.73:6789/0 conn(0x7fb6801545c0 legacy=0x7fb6800a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67f7fe700 1 -- 172.21.15.73:0/1708512788 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb680154da0 con 0x7fb680150720 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67effd700 1 -- 172.21.15.73:0/1708512788 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3864760645 0 0) 0x7fb680062910 con 0x7fb6801545c0 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67f7fe700 1 --2- 172.21.15.73:0/1708512788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb680150720 0x7fb680150b00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fb670009760 tx=0x7fb670009060).ready entity=mon.0 client_cookie=1860101433ff7e15 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67effd700 1 -- 172.21.15.73:0/1708512788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000b580 con 0x7fb680150720 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67effd700 1 -- 172.21.15.73:0/1708512788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67000b6e0 con 0x7fb680150720 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb67effd700 1 -- 172.21.15.73:0/1708512788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67000ba60 con 0x7fb680150720 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb685a9b700 1 -- 172.21.15.73:0/1708512788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb680150720 msgr2=0x7fb680150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.964+0000 7fb685a9b700 1 --2- 172.21.15.73:0/1708512788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb680150720 0x7fb680150b00 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 rx=0x7fb670009760 tx=0x7fb670009060).stop 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.965+0000 7fb685a9b700 1 -- 172.21.15.73:0/1708512788 shutdown_connections 2023-11-28T03:08:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.965+0000 7fb685a9b700 1 --2- 172.21.15.73:0/1708512788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.965+0000 7fb685a9b700 1 -- 172.21.15.73:0/1708512788 >> 172.21.15.73:0/1708512788 conn(0x7fb6800c7760 msgr2=0x7fb6800c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:04.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.965+0000 7fb685a9b700 1 -- 172.21.15.73:0/1708512788 shutdown_connections 2023-11-28T03:08:04.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.965+0000 7fb685a9b700 1 -- 172.21.15.73:0/1708512788 wait complete. 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 Processor -- start 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 -- start start 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6800c2b10 0x7fb680140300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680164f80 con 0x7fb6801545c0 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.966+0000 7fb685a9b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb680151550 con 0x7fb680150720 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb685a9b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb680151830 con 0x7fb6800c2b10 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59694/0 (socket says 172.21.15.73:59694) 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 -- 172.21.15.73:0/173062846 learned_addr learned my addr 172.21.15.73:0/173062846 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb677fff700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67f7fe700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6800c2b10 0x7fb680140300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 -- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6800c2b10 msgr2=0x7fb680140300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6800c2b10 0x7fb680140300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 -- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb680150720 msgr2=0x7fb680150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 -- 172.21.15.73:0/173062846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6700092a0 con 0x7fb6801545c0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb67ffff700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fb67800c7c0 tx=0x7fb6780015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb678010510 con 0x7fb6801545c0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678010670 con 0x7fb6801545c0 2023-11-28T03:08:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb680151b10 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780109f0 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.967+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6801407f0 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb67801b400 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6800c65b0 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb6777fe700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 0x7fb6600617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb67f7fe700 1 -- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 msgr2=0x7fb6600617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb67807a240 con 0x7fb6801545c0 2023-11-28T03:08:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.969+0000 7fb67f7fe700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 0x7fb6600617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:04.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:04.973+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb67804b200 con 0x7fb6801545c0 2023-11-28T03:08:05.005 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:05.004+0000 7ff02c8f5680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:05.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.145+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb68006c860 con 0x7fb6801545c0 2023-11-28T03:08:05.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.146+0000 7fb6777fe700 1 -- 172.21.15.73:0/173062846 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17174 (secure 0 0 0) 0x7fb67804e820 con 0x7fb6801545c0 2023-11-28T03:08:05.147 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:05.147 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":97,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:00.022966+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2262264527":"2023-11-29T03:08:00.022924+0000","172.21.15.73:0/3167061175":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:00.022924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:05.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 msgr2=0x7fb6600617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:05.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 0x7fb6600617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 msgr2=0x7fb6800c0b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 rx=0x7fb67800c7c0 tx=0x7fb6780015e0).stop 2023-11-28T03:08:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 shutdown_connections 2023-11-28T03:08:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb66005f300 0x7fb6600617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb6801545c0 0x7fb6800c0b50 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb6800c2b10 0x7fb680140300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 --2- 172.21.15.73:0/173062846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb680150720 0x7fb680150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 >> 172.21.15.73:0/173062846 conn(0x7fb6800c7760 msgr2=0x7fb6800ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:05.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 shutdown_connections 2023-11-28T03:08:05.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.148+0000 7fb685a9b700 1 -- 172.21.15.73:0/173062846 wait complete. 2023-11-28T03:08:05.166 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:08:05.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:05.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.283+0000 7fbf62374700 1 Processor -- start 2023-11-28T03:08:05.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.283+0000 7fbf62374700 1 -- start start 2023-11-28T03:08:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf62374700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbf5c1544d0 0x7fbf5c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf62374700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf5c095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf62374700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf5c0683d0 con 0x7fbf5c0673c0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf62374700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf5c067850 con 0x7fbf5c1544d0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf62374700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c067b30 con 0x7fbf5c066f40 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf60911700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fbf5c0673c0 0x7fbf5c063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33162/0 (socket says 172.21.15.73:33162) 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf60911700 1 -- 172.21.15.73:0/1778517673 learned_addr learned my addr 172.21.15.73:0/1778517673 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> v2:172.21.15.150:3300/0 conn(0x7fbf5c1544d0 0x7fbf5c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.284+0000 7fbf5b7fe700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf5c095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1308393846 0 0) 0x7fbf5c0683d0 con 0x7fbf5c0673c0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf440033f0 con 0x7fbf5c0673c0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2187478495 0 0) 0x7fbf440033f0 con 0x7fbf5c0673c0 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 msgr2=0x7fbf5c095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf5c095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 >> v2:172.21.15.150:3300/0 conn(0x7fbf5c1544d0 msgr2=0x7fbf5c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 --2- 172.21.15.73:0/1778517673 >> v2:172.21.15.150:3300/0 conn(0x7fbf5c1544d0 0x7fbf5c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c067e10 con 0x7fbf5c0673c0 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fbf540017f0 con 0x7fbf5c0673c0 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> v2:172.21.15.150:3300/0 conn(0x7fbf5c1544d0 0x7fbf5c155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5b7fe700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf5c095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 >> v1:172.21.15.73:6789/0 conn(0x7fbf5c0673c0 legacy=0x7fbf5c063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440038e0 0x7fbf44003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf44004690 0x7fbf44004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf44005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c067850 con 0x7fbf440038e0 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c067b30 con 0x7fbf44004690 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf440044f0 con 0x7fbf5c066f40 2023-11-28T03:08:05.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440038e0 0x7fbf44003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5b7fe700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf44004690 0x7fbf44004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf60911700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf44005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 -- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 msgr2=0x7fbf44005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c066f40 0x7fbf44005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 -- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf44004690 msgr2=0x7fbf44004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf44004690 0x7fbf44004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.285+0000 7fbf5bfff700 1 -- 172.21.15.73:0/1778517673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf4c0092a0 con 0x7fbf440038e0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440038e0 0x7fbf44003d00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fbf50009d00 tx=0x7fbf50006f40).ready entity=mon.0 client_cookie=dc64eddf5aeb5f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf50009e60 con 0x7fbf440038e0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf50009670 con 0x7fbf440038e0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf5affd700 1 -- 172.21.15.73:0/1778517673 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5000eb30 con 0x7fbf440038e0 2023-11-28T03:08:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 -- 172.21.15.73:0/1778517673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440038e0 msgr2=0x7fbf44003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 --2- 172.21.15.73:0/1778517673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf440038e0 0x7fbf44003d00 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 rx=0x7fbf50009d00 tx=0x7fbf50006f40).stop 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf60911700 1 -- 172.21.15.73:0/1778517673 reap_dead start 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 -- 172.21.15.73:0/1778517673 shutdown_connections 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 -- 172.21.15.73:0/1778517673 >> 172.21.15.73:0/1778517673 conn(0x7fbf5c0631d0 msgr2=0x7fbf5c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 -- 172.21.15.73:0/1778517673 shutdown_connections 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.286+0000 7fbf62374700 1 -- 172.21.15.73:0/1778517673 wait complete. 2023-11-28T03:08:05.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.287+0000 7fbf62374700 1 Processor -- start 2023-11-28T03:08:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 -- start start 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf440038e0 0x7fbf5c14d990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c0ae5a0 0x7fbf5c0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c158600 con 0x7fbf5c158a90 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0ab4a0 con 0x7fbf440038e0 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf62374700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf5c0ab780 con 0x7fbf5c0ae5a0 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf60911700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf60911700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59714/0 (socket says 172.21.15.73:59714) 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf60911700 1 -- 172.21.15.73:0/1724630847 learned_addr learned my addr 172.21.15.73:0/1724630847 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf5b7fe700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c0ae5a0 0x7fbf5c0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.288+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf440038e0 0x7fbf5c14d990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c0ae5a0 msgr2=0x7fbf5c0aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c0ae5a0 0x7fbf5c0aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf440038e0 msgr2=0x7fbf5c14d990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf440038e0 0x7fbf5c14d990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 -- 172.21.15.73:0/1724630847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf5c158290 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf60911700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fbf54001ab0 tx=0x7fbf54002210).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400f8a0 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf5400fa00 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf5400fd80 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf5c14de80 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.289+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf5c0aef10 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.290+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fbf54012560 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.290+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf5c1580f0 con 0x7fbf5c158a90 2023-11-28T03:08:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.291+0000 7fbf43fff700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 0x7fbf480617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.291+0000 7fbf5bfff700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 msgr2=0x7fbf480617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:05.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.291+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fbf54079b50 con 0x7fbf5c158a90 2023-11-28T03:08:05.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.291+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 0x7fbf480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:05.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.295+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbf5c1580f0 con 0x7fbf5c158a90 2023-11-28T03:08:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.492+0000 7fbf5bfff700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 msgr2=0x7fbf480617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:05.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.492+0000 7fbf5bfff700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 0x7fbf480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.508+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf5c1580f0 con 0x7fbf5c158a90 2023-11-28T03:08:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.509+0000 7fbf43fff700 1 -- 172.21.15.73:0/1724630847 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fbf5c1580f0 con 0x7fbf5c158a90 2023-11-28T03:08:05.510 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:05.510 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:08:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 msgr2=0x7fbf480617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 0x7fbf480617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 msgr2=0x7fbf5c158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 rx=0x7fbf54001ab0 tx=0x7fbf54002210).stop 2023-11-28T03:08:05.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 shutdown_connections 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbf4805f300 0x7fbf480617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbf5c158a90 0x7fbf5c158f10 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbf440038e0 0x7fbf5c14d990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 --2- 172.21.15.73:0/1724630847 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbf5c0ae5a0 0x7fbf5c0aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 >> 172.21.15.73:0/1724630847 conn(0x7fbf5c0631d0 msgr2=0x7fbf5c0bf050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 shutdown_connections 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.511+0000 7fbf62374700 1 -- 172.21.15.73:0/1724630847 wait complete. 2023-11-28T03:08:05.513 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:08:05.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:05.531 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:05.530+0000 7fd95f643680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:08:05.599 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:05.598+0000 7fd95f643680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:08:05.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 Processor -- start 2023-11-28T03:08:05.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 -- start start 2023-11-28T03:08:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd194064e20 0x7fd194155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1940683d0 con 0x7fd1941544d0 2023-11-28T03:08:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd194067850 con 0x7fd194095e80 2023-11-28T03:08:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.648+0000 7fd19909e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd194067650 con 0x7fd194064e20 2023-11-28T03:08:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd192d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd1941544d0 0x7fd194066f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33166/0 (socket says 172.21.15.73:33166) 2023-11-28T03:08:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd192d9d700 1 -- 172.21.15.73:0/4018229434 learned_addr learned my addr 172.21.15.73:0/4018229434 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd19259c700 1 --2- 172.21.15.73:0/4018229434 >> v2:172.21.15.150:3300/0 conn(0x7fd194064e20 0x7fd194155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 443818346 0 0) 0x7fd1940683d0 con 0x7fd1941544d0 2023-11-28T03:08:05.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd17c0033f0 con 0x7fd1941544d0 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2515153994 0 0) 0x7fd194067850 con 0x7fd194095e80 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1940683d0 con 0x7fd194095e80 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd19259c700 1 -- 172.21.15.73:0/4018229434 >> v1:172.21.15.150:6789/0 conn(0x7fd194095e80 legacy=0x7fd19415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd19259c700 1 -- 172.21.15.73:0/4018229434 >> v1:172.21.15.73:6789/0 conn(0x7fd1941544d0 legacy=0x7fd194066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd19259c700 1 -- 172.21.15.73:0/4018229434 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd194067b30 con 0x7fd194064e20 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd19259c700 1 --2- 172.21.15.73:0/4018229434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194064e20 0x7fd194155750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fd188009760 tx=0x7fd188009060).ready entity=mon.1 client_cookie=351bf1da26d730ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800d4a0 con 0x7fd194064e20 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800d600 con 0x7fd194064e20 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.649+0000 7fd191d9b700 1 -- 172.21.15.73:0/4018229434 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800d980 con 0x7fd194064e20 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 -- 172.21.15.73:0/4018229434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194064e20 msgr2=0x7fd194155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 --2- 172.21.15.73:0/4018229434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194064e20 0x7fd194155750 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 rx=0x7fd188009760 tx=0x7fd188009060).stop 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 -- 172.21.15.73:0/4018229434 shutdown_connections 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 --2- 172.21.15.73:0/4018229434 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194064e20 0x7fd194155750 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 -- 172.21.15.73:0/4018229434 >> 172.21.15.73:0/4018229434 conn(0x7fd1940631d0 msgr2=0x7fd1940c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 -- 172.21.15.73:0/4018229434 shutdown_connections 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.650+0000 7fd19909e700 1 -- 172.21.15.73:0/4018229434 wait complete. 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.651+0000 7fd19909e700 1 Processor -- start 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.651+0000 7fd19909e700 1 -- start start 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 0x7fd194156990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 0x7fd19414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 0x7fd1940aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1940c0b40 con 0x7fd194064e20 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd194156500 con 0x7fd194095e80 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19909e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd194156e80 con 0x7fd1941544d0 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 0x7fd1940aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19259c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 0x7fd19414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 0x7fd1940aabe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:36812/0 (socket says 172.21.15.73:36812) 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd19359e700 1 -- 172.21.15.73:0/2332797379 learned_addr learned my addr 172.21.15.73:0/2332797379 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.652+0000 7fd192d9d700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 0x7fd194156990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 msgr2=0x7fd1940aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 0x7fd1940aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 msgr2=0x7fd194156990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 0x7fd194156990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 -- 172.21.15.73:0/2332797379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1880092a0 con 0x7fd194095e80 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd192d9d700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 0x7fd194156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19259c700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 0x7fd19414cfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd19406c7c0 tx=0x7fd188016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800dcf0 con 0x7fd194095e80 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd18800de50 con 0x7fd194095e80 2023-11-28T03:08:05.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1941558f0 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd18800d980 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.653+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd194155eb0 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.654+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1941566b0 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.654+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fd18801d640 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.655+0000 7fd17affd700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 0x7fd1800617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.655+0000 7fd192d9d700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 msgr2=0x7fd1800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.655+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd18807bba0 con 0x7fd194095e80 2023-11-28T03:08:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.655+0000 7fd192d9d700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 0x7fd1800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:05.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.659+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd1941566b0 con 0x7fd194095e80 2023-11-28T03:08:05.676 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:05.675+0000 7fd95f643680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:08:05.698 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:05.697+0000 7ff02c8f5680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:08:05.769 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:05.768+0000 7ff02c8f5680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:08:05.822 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:05.821+0000 7fd95f643680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:08:05.846 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:05.845+0000 7ff02c8f5680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:08:05.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.856+0000 7fd192d9d700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 msgr2=0x7fd1800617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:05.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.856+0000 7fd192d9d700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 0x7fd1800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:05.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.870+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd1941566b0 con 0x7fd194095e80 2023-11-28T03:08:05.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.871+0000 7fd17affd700 1 -- 172.21.15.73:0/2332797379 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 34 v34) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fd1941566b0 con 0x7fd194095e80 2023-11-28T03:08:05.874 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:05.874 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":34,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[{"mdsmap":{"epoch":34,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:07:47.669246+0000","modified":"2023-11-28T03:07:50.680150+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6705},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6705":{"gid":6705,"name":"b","rank":0,"incarnation":30,"state":"up:active","state_seq":3,"addr":"172.21.15.150:6835/3882811189","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3882811189},{"type":"v1","addr":"172.21.15.150:6835","nonce":3882811189}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[12],"metadata_pool":11,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":5}]} 2023-11-28T03:08:05.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 msgr2=0x7fd1800617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:05.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 0x7fd1800617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 msgr2=0x7fd19414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:05.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 0x7fd19414cfd0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 rx=0x7fd19406c7c0 tx=0x7fd188016000).stop 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 shutdown_connections 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd18005f300 0x7fd1800617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd1941544d0 0x7fd1940aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd194064e20 0x7fd194156990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 --2- 172.21.15.73:0/2332797379 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd194095e80 0x7fd19414cfd0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:05.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 >> 172.21.15.73:0/2332797379 conn(0x7fd1940631d0 msgr2=0x7fd1940c3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 shutdown_connections 2023-11-28T03:08:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:05.873+0000 7fd19909e700 1 -- 172.21.15.73:0/2332797379 wait complete. 2023-11-28T03:08:05.878 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 34 2023-11-28T03:08:05.886 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:05.893 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:05.892+0000 7fd95f643680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:08:06.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.002+0000 7f25ccf31700 1 Processor -- start 2023-11-28T03:08:06.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.002+0000 7f25ccf31700 1 -- start start 2023-11-28T03:08:06.003 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.003+0000 7ff02c8f5680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25ccf31700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f25c8064e20 0x7f25c8155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25ccf31700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25c8096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25ccf31700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25c8096800 con 0x7f25c81544d0 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25ccf31700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25c8067650 con 0x7f25c8064e20 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25ccf31700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80683d0 con 0x7f25c8095e80 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c659c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f25c81544d0 0x7f25c8066f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33172/0 (socket says 172.21.15.73:33172) 2023-11-28T03:08:06.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c659c700 1 -- 172.21.15.73:0/726726113 learned_addr learned my addr 172.21.15.73:0/726726113 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/726726113 >> v2:172.21.15.150:3300/0 conn(0x7f25c8064e20 0x7f25c8155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c6d9d700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25c8096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3132012314 0 0) 0x7f25c8096800 con 0x7f25c81544d0 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25b00033f0 con 0x7f25c81544d0 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1253979954 0 0) 0x7f25b00033f0 con 0x7f25c81544d0 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 msgr2=0x7f25c8096310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25c8096310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 >> v2:172.21.15.150:3300/0 conn(0x7f25c8064e20 msgr2=0x7f25c8155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 --2- 172.21.15.73:0/726726113 >> v2:172.21.15.150:3300/0 conn(0x7f25c8064e20 0x7f25c8155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c8067850 con 0x7f25c81544d0 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.003+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f25b80015f0 con 0x7f25c81544d0 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c6d9d700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25c8096310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 >> v1:172.21.15.73:6789/0 conn(0x7f25c81544d0 legacy=0x7f25c8066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b00038e0 0x7f25b0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b0004690 0x7f25b0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25b0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c8067650 con 0x7f25b00038e0 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80683d0 con 0x7f25b0004690 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b00044f0 con 0x7f25c8095e80 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/726726113 >> v2:172.21.15.150:3300/0 conn(0x7f25c8064e20 0x7f25c8155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:06.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b00038e0 0x7f25b0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c6d9d700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25b0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b0004690 0x7f25b0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 -- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 msgr2=0x7f25b0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8095e80 0x7f25b0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 -- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b0004690 msgr2=0x7f25b0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b0004690 0x7f25b0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 -- 172.21.15.73:0/726726113 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c0009a20 con 0x7f25b00038e0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b0004690 0x7f25b0004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c659c700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b00038e0 0x7f25b0003d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f25b800c210 tx=0x7f25b80077a0).ready entity=mon.0 client_cookie=caf391f7ce1d38b5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80079d0 con 0x7f25b00038e0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b800cae0 con 0x7f25b00038e0 2023-11-28T03:08:06.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.004+0000 7f25c559a700 1 -- 172.21.15.73:0/726726113 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b800ce80 con 0x7f25b00038e0 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 -- 172.21.15.73:0/726726113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b00038e0 msgr2=0x7f25b0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 --2- 172.21.15.73:0/726726113 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25b00038e0 0x7f25b0003d00 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 rx=0x7f25b800c210 tx=0x7f25b80077a0).stop 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25c6d9d700 1 -- 172.21.15.73:0/726726113 reap_dead start 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 -- 172.21.15.73:0/726726113 shutdown_connections 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 -- 172.21.15.73:0/726726113 >> 172.21.15.73:0/726726113 conn(0x7f25c80631d0 msgr2=0x7f25c80c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 -- 172.21.15.73:0/726726113 shutdown_connections 2023-11-28T03:08:06.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.005+0000 7f25ccf31700 1 -- 172.21.15.73:0/726726113 wait complete. 2023-11-28T03:08:06.012 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:06.008+0000 7fd95f643680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:08:06.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.007+0000 7f25ccf31700 1 Processor -- start 2023-11-28T03:08:06.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.007+0000 7f25ccf31700 1 -- start start 2023-11-28T03:08:06.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 0x7f25c814ece0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8156fe0 0x7f25c80adbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80ae0c0 con 0x7f25c8156b60 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80ad140 con 0x7f25b00038e0 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25ccf31700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25c80ac530 con 0x7f25c8156fe0 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25c5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59746/0 (socket says 172.21.15.73:59746) 2023-11-28T03:08:06.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25c5d9b700 1 -- 172.21.15.73:0/2539259009 learned_addr learned my addr 172.21.15.73:0/2539259009 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:06.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.008+0000 7f25c659c700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 0x7f25c814ece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c6d9d700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8156fe0 0x7f25c80adbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 -- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8156fe0 msgr2=0x7f25c80adbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8156fe0 0x7f25c80adbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 -- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 msgr2=0x7f25c80ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 -- 172.21.15.73:0/2539259009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25c80ac810 con 0x7f25b00038e0 2023-11-28T03:08:06.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:06.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25c659c700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 0x7f25c814ece0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f25b800c1e0 tx=0x7f25b8013000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b801a070 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25b8018020 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c80acaf0 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b80014f0 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.009+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c814f1d0 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.010+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c806c860 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.011+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f25b8005670 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.011+0000 7f25b67fc700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 0x7f25a80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.011+0000 7f25c5d9b700 1 -- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 msgr2=0x7f25a80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.011+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f25b807ae60 con 0x7f25b00038e0 2023-11-28T03:08:06.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.011+0000 7f25c5d9b700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 0x7f25a80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:06.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.019+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f25b804be40 con 0x7f25b00038e0 2023-11-28T03:08:06.080 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.079+0000 7ff02c8f5680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:08:06.120 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:06.120+0000 7fd95f643680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:06.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.189+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25c80acf90 con 0x7f25b00038e0 2023-11-28T03:08:06.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.191+0000 7f25b67fc700 1 -- 172.21.15.73:0/2539259009 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v97) v1 ==== 74+0+17174 (secure 0 0 0) 0x7f25b8025070 con 0x7f25b00038e0 2023-11-28T03:08:06.192 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:06.193 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":97,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:00.022966+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"87","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":11,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:07:46.190668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"95","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":12,"pool_name":"cephfs_data","create_time":"2023-11-28T03:07:46.631129+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"96","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2262264527":"2023-11-29T03:08:00.022924+0000","172.21.15.73:0/3167061175":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:00.022924+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:00.022924+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:06.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 msgr2=0x7f25a80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 0x7f25a80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 msgr2=0x7f25c814ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 0x7f25c814ece0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 rx=0x7f25b800c1e0 tx=0x7f25b8013000).stop 2023-11-28T03:08:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 shutdown_connections 2023-11-28T03:08:06.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f25a805f300 0x7f25a80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f25c8156fe0 0x7f25c80adbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f25b00038e0 0x7f25c814ece0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 --2- 172.21.15.73:0/2539259009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f25c8156b60 0x7f25c80ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 >> 172.21.15.73:0/2539259009 conn(0x7f25c80631d0 msgr2=0x7f25c80c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 shutdown_connections 2023-11-28T03:08:06.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.193+0000 7f25ccf31700 1 -- 172.21.15.73:0/2539259009 wait complete. 2023-11-28T03:08:06.200 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.199+0000 7ff02c8f5680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:08:06.207 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:08:06.316 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.315+0000 7ff02c8f5680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:06.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.325+0000 7fb8f2a4a700 1 Processor -- start 2023-11-28T03:08:06.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.325+0000 7fb8f2a4a700 1 -- start start 2023-11-28T03:08:06.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.325+0000 7fb8f2a4a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.325+0000 7fb8f2a4a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec0634a0 con 0x7fb8ec062770 2023-11-28T03:08:06.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8f2a4a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8ec063780 con 0x7fb8ec14eb00 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8f2a4a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec137d90 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8f0fe7700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb8ec062770 0x7fb8ec062c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33178/0 (socket says 172.21.15.73:33178) 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8f0fe7700 1 -- 172.21.15.73:0/2635323177 learned_addr learned my addr 172.21.15.73:0/2635323177 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eb7fe700 1 --2- 172.21.15.73:0/2635323177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1374166444 0 0) 0x7fb8ec0634a0 con 0x7fb8ec062770 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8d40033f0 con 0x7fb8ec062770 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 367341425 0 0) 0x7fb8ec063780 con 0x7fb8ec14eb00 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8ec0634a0 con 0x7fb8ec14eb00 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eb7fe700 1 -- 172.21.15.73:0/2635323177 >> v1:172.21.15.150:6789/0 conn(0x7fb8ec14eb00 legacy=0x7fb8ec0bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eb7fe700 1 -- 172.21.15.73:0/2635323177 >> v1:172.21.15.73:6789/0 conn(0x7fb8ec062770 legacy=0x7fb8ec062c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eb7fe700 1 -- 172.21.15.73:0/2635323177 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8ec063a60 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eb7fe700 1 --2- 172.21.15.73:0/2635323177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec14eee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb8e0009760 tx=0x7fb8e0006f40).ready entity=mon.2 client_cookie=8231c7641b6e1fff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0005880 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8e00059e0 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.326+0000 7fb8eaffd700 1 -- 172.21.15.73:0/2635323177 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8e0005d60 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2635323177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 msgr2=0x7fb8ec14eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2635323177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec14eee0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 rx=0x7fb8e0009760 tx=0x7fb8e0006f40).stop 2023-11-28T03:08:06.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2635323177 shutdown_connections 2023-11-28T03:08:06.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2635323177 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec14eee0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2635323177 >> 172.21.15.73:0/2635323177 conn(0x7fb8ec0c8230 msgr2=0x7fb8ec0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:06.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2635323177 shutdown_connections 2023-11-28T03:08:06.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.327+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2635323177 wait complete. 2023-11-28T03:08:06.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.328+0000 7fb8f2a4a700 1 Processor -- start 2023-11-28T03:08:06.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.328+0000 7fb8f2a4a700 1 -- start start 2023-11-28T03:08:06.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec0be840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec14eb00 0x7fb8ec156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec13aef0 con 0x7fb8ec062770 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec156b20 con 0x7fb8ec14eb00 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f2a4a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8ec156510 con 0x7fb8ec0bcdd0 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8ebfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8ebfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59762/0 (socket says 172.21.15.73:59762) 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8ebfff700 1 -- 172.21.15.73:0/2362767461 learned_addr learned my addr 172.21.15.73:0/2362767461 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8f0fe7700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec14eb00 0x7fb8ec156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.329+0000 7fb8eb7fe700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec0be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 msgr2=0x7fb8ec0be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec0be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec14eb00 msgr2=0x7fb8ec156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec14eb00 0x7fb8ec156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 -- 172.21.15.73:0/2362767461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8e0009250 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8ebfff700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7fb8dc00c4c0 tx=0x7fb8dc007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00ce30 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc015b70 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc015d00 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8ec13a930 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.330+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8ec1567b0 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.331+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 35) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb8dc01b020 con 0x7fb8ec062770 2023-11-28T03:08:06.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.331+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8ec06c860 con 0x7fb8ec062770 2023-11-28T03:08:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.331+0000 7fb8d3fff700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 0x7fb8d80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.332+0000 7fb8eb7fe700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 msgr2=0x7fb8d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.332+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb8dc081c30 con 0x7fb8ec062770 2023-11-28T03:08:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.332+0000 7fb8eb7fe700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 0x7fb8d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.336+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb8dc052c50 con 0x7fb8ec062770 2023-11-28T03:08:06.416 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:06.415+0000 7fd95f643680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:08:06.487 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:06.486+0000 7fd95f643680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:06.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.533+0000 7fb8eb7fe700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 msgr2=0x7fb8d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:06.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.533+0000 7fb8eb7fe700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 0x7fb8d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:06.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.559+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb8ec0c4ac0 con 0x7fb8ec062770 2023-11-28T03:08:06.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.617+0000 7ff02c8f5680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:08:06.688 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.687+0000 7ff02c8f5680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:06.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.708+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 36) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fb8dc07f750 con 0x7fb8ec062770 2023-11-28T03:08:06.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.708+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 msgr2=0x7fb8d80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:06.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:06.708+0000 7fb8d3fff700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 0x7fb8d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:06.713 INFO:tasks.ceph.mds.b.smithi150.stderr:2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 mds.0.journaler.mdlog(rw) _finish_write_head got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:08:06.713 INFO:tasks.ceph.mds.b.smithi150.stderr:2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:08:06.713 INFO:tasks.ceph.mds.b.smithi150.stderr:2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:08:06.716 INFO:tasks.ceph.mds.b.smithi150.stderr: -1054> 2023-11-28T03:07:53.591+0000 7f5a0a268700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:08:06.718 INFO:tasks.ceph.mds.b.smithi150.stderr: -4> 2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 mds.0.journaler.mdlog(rw) _finish_write_head got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:08:06.718 INFO:tasks.ceph.mds.b.smithi150.stderr: -3> 2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:08:06.718 INFO:tasks.ceph.mds.b.smithi150.stderr: -1> 2023-11-28T03:08:06.712+0000 7f5a06a61700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:08:06.736 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:08:06.767 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa6085700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa6085700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa6085700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa6085700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa6085700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa908b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa908b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa908b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa908b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:06.768 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:06.766+0000 7fefa908b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:07.258 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:08:07.257+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:08:07.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.265+0000 7fb8d3fff700 1 -- 172.21.15.73:0/2362767461 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v35) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb8dc079920 con 0x7fb8ec062770 2023-11-28T03:08:07.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 msgr2=0x7fb8ec1363c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 rx=0x7fb8dc00c4c0 tx=0x7fb8dc007c80).stop 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 shutdown_connections 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb8d805f300 0x7fb8d80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb8ec14eb00 0x7fb8ec156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8ec062770 0x7fb8ec1363c0 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 --2- 172.21.15.73:0/2362767461 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8ec0bcdd0 0x7fb8ec0be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.267+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 >> 172.21.15.73:0/2362767461 conn(0x7fb8ec0c8230 msgr2=0x7fb8ec0c4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.268+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 shutdown_connections 2023-11-28T03:08:07.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.268+0000 7fb8f2a4a700 1 -- 172.21.15.73:0/2362767461 wait complete. 2023-11-28T03:08:07.270 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:08:07.282 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:08:07.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.398+0000 7f1997e80700 1 Processor -- start 2023-11-28T03:08:07.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.399+0000 7f1997e80700 1 -- start start 2023-11-28T03:08:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.399+0000 7f1997e80700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f19900bd900 0x7f199006a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.399+0000 7f1997e80700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19900c3470 con 0x7f1990069d10 2023-11-28T03:08:07.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.399+0000 7f1997e80700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19900c48f0 con 0x7f1990151a20 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.399+0000 7f1997e80700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1990151ea0 con 0x7f19900bd900 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1995c1c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1990069d10 0x7f19900be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33190/0 (socket says 172.21.15.73:33190) 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1995c1c700 1 -- 172.21.15.73:0/3692490683 learned_addr learned my addr 172.21.15.73:0/3692490683 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 87201793 0 0) 0x7f19900c3470 con 0x7f1990069d10 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f197c0033f0 con 0x7f1990069d10 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f199641d700 1 --2- 172.21.15.73:0/3692490683 >> v2:172.21.15.150:3300/0 conn(0x7f19900bd900 0x7f199006a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 243833429 0 0) 0x7f197c0033f0 con 0x7f1990069d10 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 >> v2:172.21.15.150:3300/0 conn(0x7f19900bd900 msgr2=0x7f199006a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 --2- 172.21.15.73:0/3692490683 >> v2:172.21.15.150:3300/0 conn(0x7f19900bd900 0x7f199006a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 >> v1:172.21.15.150:6789/0 conn(0x7f1990151a20 legacy=0x7f19900bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1990152180 con 0x7f1990069d10 2023-11-28T03:08:07.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f198c0017f0 con 0x7f1990069d10 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 >> v1:172.21.15.73:6789/0 conn(0x7f1990069d10 legacy=0x7f19900be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 0x7f197c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c004690 0x7f197c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f197c005000 0x7f197c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1990151ea0 con 0x7f197c0038e0 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f197c00bf60 con 0x7f197c005000 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f197c00c0f0 con 0x7f197c004690 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f1995c1c700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 0x7f197c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f199641d700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f197c005000 0x7f197c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.400+0000 7f199541b700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c004690 0x7f197c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 -- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c004690 msgr2=0x7f197c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c004690 0x7f197c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 -- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f197c005000 msgr2=0x7f197c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f197c005000 0x7f197c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 -- 172.21.15.73:0/3692490683 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f197c0033f0 con 0x7f197c0038e0 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f199641d700 1 -- 172.21.15.73:0/3692490683 reap_dead start 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1995c1c700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 0x7f197c003d00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f198c00c480 tx=0x7f198c00c790).ready entity=mon.0 client_cookie=68ce3eaa55b0cee3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c002030 con 0x7f197c0038e0 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f198c00cda0 con 0x7f197c0038e0 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1994c1a700 1 -- 172.21.15.73:0/3692490683 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c005d30 con 0x7f197c0038e0 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 -- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 msgr2=0x7f197c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 0x7f197c003d00 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 rx=0x7f198c00c480 tx=0x7f198c00c790).stop 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 -- 172.21.15.73:0/3692490683 shutdown_connections 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 --2- 172.21.15.73:0/3692490683 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f197c0038e0 0x7f197c003d00 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 -- 172.21.15.73:0/3692490683 >> 172.21.15.73:0/3692490683 conn(0x7f19900642d0 msgr2=0x7f19900c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:07.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 -- 172.21.15.73:0/3692490683 shutdown_connections 2023-11-28T03:08:07.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.401+0000 7f1997e80700 1 -- 172.21.15.73:0/3692490683 wait complete. 2023-11-28T03:08:07.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.402+0000 7f1997e80700 1 Processor -- start 2023-11-28T03:08:07.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 -- start start 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c0038e0 0x7f19900a9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1990166b40 0x7f19900a8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19900aa130 con 0x7f1990062700 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1990163b10 con 0x7f1990166b40 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1997e80700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1990063070 con 0x7f197c0038e0 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f199641d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f199641d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59842/0 (socket says 172.21.15.73:59842) 2023-11-28T03:08:07.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f199641d700 1 -- 172.21.15.73:0/1618734881 learned_addr learned my addr 172.21.15.73:0/1618734881 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f199541b700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1990166b40 0x7f19900a8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.403+0000 7f1995c1c700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c0038e0 0x7f19900a9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 -- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c0038e0 msgr2=0x7f19900a9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c0038e0 0x7f19900a9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 -- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1990166b40 msgr2=0x7f19900a8fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1990166b40 0x7f19900a8fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 -- 172.21.15.73:0/1618734881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19900a94d0 con 0x7f1990062700 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f199641d700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f19900bdd00 tx=0x7f1988009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988013680 con 0x7f1990062700 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19880137e0 con 0x7f1990062700 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1990162f90 con 0x7f1990062700 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1988013b80 con 0x7f1990062700 2023-11-28T03:08:07.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.404+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1990163550 con 0x7f1990062700 2023-11-28T03:08:07.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.405+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f199006c860 con 0x7f1990062700 2023-11-28T03:08:07.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.405+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 36) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f198801f460 con 0x7f1990062700 2023-11-28T03:08:07.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.406+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f198807a450 con 0x7f1990062700 2023-11-28T03:08:07.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.410+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f198804b6b0 con 0x7f1990062700 2023-11-28T03:08:07.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.614+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f199006a620 con 0x7f1990062700 2023-11-28T03:08:07.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.713+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 37) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f1988078210 con 0x7f1990062700 2023-11-28T03:08:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.714+0000 7f1985ffb700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 0x7f1974067aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:07.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.714+0000 7f1995c1c700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 0x7f1974067aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:07.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:07.715+0000 7f1995c1c700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 0x7f1974067aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f198c00c7d0 tx=0x7f198c015000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:08.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.269+0000 7f1985ffb700 1 -- 172.21.15.73:0/1618734881 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v36) v1 ==== 104+0+0 (secure 0 0 0) 0x7f198806c440 con 0x7f1990062700 2023-11-28T03:08:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 msgr2=0x7f1974067aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 0x7f1974067aa0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 rx=0x7f198c00c7d0 tx=0x7f198c015000).stop 2023-11-28T03:08:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 msgr2=0x7f1990062b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 rx=0x7f19900bdd00 tx=0x7f1988009060).stop 2023-11-28T03:08:08.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 shutdown_connections 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f19740302f0 0x7f1974067aa0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1990062700 0x7f1990062b80 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f197c0038e0 0x7f19900a9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 --2- 172.21.15.73:0/1618734881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1990166b40 0x7f19900a8fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 >> 172.21.15.73:0/1618734881 conn(0x7f19900642d0 msgr2=0x7f199014ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 shutdown_connections 2023-11-28T03:08:08.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.271+0000 7f1997e80700 1 -- 172.21.15.73:0/1618734881 wait complete. 2023-11-28T03:08:08.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:08:08.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.400+0000 7ff94d1ca700 1 Processor -- start 2023-11-28T03:08:08.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.400+0000 7ff94d1ca700 1 -- start start 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff94d1ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff9480640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff94d1ca700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9481559b0 con 0x7ff9480c57e0 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff94d1ca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff948064a00 con 0x7ff9480bef90 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff94d1ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff948065680 con 0x7ff9480c6530 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff9477fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff9480c57e0 0x7ff9480be880 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33196/0 (socket says 172.21.15.73:33196) 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff9477fe700 1 -- 172.21.15.73:0/3201446872 learned_addr learned my addr 172.21.15.73:0/3201446872 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff947fff700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff9480640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454080165 0 0) 0x7ff9481559b0 con 0x7ff9480c57e0 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9300033f0 con 0x7ff9480c57e0 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245241444 0 0) 0x7ff948064a00 con 0x7ff9480bef90 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9481559b0 con 0x7ff9480bef90 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3174281511 0 0) 0x7ff9300033f0 con 0x7ff9480c57e0 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 msgr2=0x7ff9480640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff9480640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 >> v1:172.21.15.150:6789/0 conn(0x7ff9480bef90 legacy=0x7ff9480c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff948064ce0 con 0x7ff9480c57e0 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff93c001470 con 0x7ff9480c57e0 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 >> v1:172.21.15.73:6789/0 conn(0x7ff9480c57e0 legacy=0x7ff9480be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff930003aa0 0x7ff930003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.401+0000 7ff946ffd700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff930004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff930004b80 0x7ff930005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff948065680 con 0x7ff930004b80 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff93000b990 con 0x7ff930003aa0 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff93000bb00 con 0x7ff9480c6530 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff930004b80 0x7ff930005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff9477fe700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff930004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff947fff700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff930003aa0 0x7ff930003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 msgr2=0x7ff930004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff9480c6530 0x7ff930004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff930003aa0 msgr2=0x7ff930003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff930003aa0 0x7ff930003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3201446872 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9300033f0 con 0x7ff930004b80 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff930004b80 0x7ff930005000 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7ff940001f90 tx=0x7ff940007f30).ready entity=mon.0 client_cookie=94e818174f6d6194 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400020f0 con 0x7ff930004b80 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940006a20 con 0x7ff930004b80 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.402+0000 7ff946ffd700 1 -- 172.21.15.73:0/3201446872 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940006dc0 con 0x7ff930004b80 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3201446872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff930004b80 msgr2=0x7ff930005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3201446872 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff930004b80 0x7ff930005000 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 rx=0x7ff940001f90 tx=0x7ff940007f30).stop 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3201446872 reap_dead start 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3201446872 shutdown_connections 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3201446872 >> 172.21.15.73:0/3201446872 conn(0x7ff948092520 msgr2=0x7ff948062ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3201446872 shutdown_connections 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.403+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3201446872 wait complete. 2023-11-28T03:08:08.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.404+0000 7ff94d1ca700 1 Processor -- start 2023-11-28T03:08:08.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.404+0000 7ff94d1ca700 1 -- start start 2023-11-28T03:08:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff948162150 0x7ff948162530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 0x7ff9480c6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480c7340 con 0x7ff9481658d0 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480c7620 con 0x7ff9480c83e0 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94d1ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9480c7900 con 0x7ff948162150 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59866/0 (socket says 172.21.15.73:59866) 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3643486594 learned_addr learned my addr 172.21.15.73:0/3643486594 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff947fff700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff948162150 0x7ff948162530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff9477fe700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 0x7ff9480c6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff948162150 msgr2=0x7ff948162530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff948162150 0x7ff948162530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 msgr2=0x7ff9480c6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 0x7ff9480c6cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:08.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 -- 172.21.15.73:0/3643486594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9480c7be0 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff9477fe700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 0x7ff9480c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff94c9c9700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7ff940011000 tx=0x7ff940016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.405+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940012610 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.406+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff940012770 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.406+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9480c7ec0 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.406+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff940012b10 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.406+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9480aabc0 con 0x7ff9481658d0 2023-11-28T03:08:08.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.407+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9480beab0 con 0x7ff9481658d0 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.407+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 37) v1 ==== 75340+0+0 (secure 0 0 0) 0x7ff94001e480 con 0x7ff9481658d0 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.407+0000 7ff92ffff700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 0x7ff9340614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.407+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 6594+0+0 (secure 0 0 0) 0x7ff94007b180 con 0x7ff9481658d0 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.407+0000 7ff947fff700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 0x7ff9340614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.408+0000 7ff947fff700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 0x7ff9340614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff938006d20 tx=0x7ff938008040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:08.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.411+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff94004c490 con 0x7ff9481658d0 2023-11-28T03:08:08.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:08.574+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff94806c7c0 con 0x7ff9481658d0 2023-11-28T03:08:09.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.272+0000 7ff92ffff700 1 -- 172.21.15.73:0/3643486594 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v100) v1 ==== 181+0+0 (secure 0 0 0) 0x7ff940023020 con 0x7ff9481658d0 2023-11-28T03:08:09.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 msgr2=0x7ff9340614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 0x7ff9340614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7ff938006d20 tx=0x7ff938008040).stop 2023-11-28T03:08:09.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 msgr2=0x7ff9480aa6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 rx=0x7ff940011000 tx=0x7ff940016000).stop 2023-11-28T03:08:09.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 shutdown_connections 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff93405f030 0x7ff9340614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff9481658d0 0x7ff9480aa6d0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff948162150 0x7ff948162530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 --2- 172.21.15.73:0/3643486594 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff9480c83e0 0x7ff9480c6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 >> 172.21.15.73:0/3643486594 conn(0x7ff948092520 msgr2=0x7ff9480a5c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 shutdown_connections 2023-11-28T03:08:09.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.275+0000 7ff94d1ca700 1 -- 172.21.15.73:0/3643486594 wait complete. 2023-11-28T03:08:09.278 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:08:09.289 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:08:09.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.405+0000 7fd4f8b19700 1 Processor -- start 2023-11-28T03:08:09.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.405+0000 7fd4f8b19700 1 -- start start 2023-11-28T03:08:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f8b19700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4f40aa930 0x7fd4f40a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f8b19700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f40c6630 con 0x7fd4f4154530 2023-11-28T03:08:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f8b19700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4f4064230 con 0x7fd4f4154910 2023-11-28T03:08:09.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f8b19700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4f40c6390 con 0x7fd4f40aa930 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd4f4154530 0x7fd4f40c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33200/0 (socket says 172.21.15.73:33200) 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f259c700 1 -- 172.21.15.73:0/1321476878 learned_addr learned my addr 172.21.15.73:0/1321476878 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f2d9d700 1 --2- 172.21.15.73:0/1321476878 >> v2:172.21.15.150:3300/0 conn(0x7fd4f40aa930 0x7fd4f40a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702239984 0 0) 0x7fd4f40c6630 con 0x7fd4f4154530 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.406+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd4e00033f0 con 0x7fd4f4154530 2023-11-28T03:08:09.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 56385953 0 0) 0x7fd4e00033f0 con 0x7fd4f4154530 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 >> v2:172.21.15.150:3300/0 conn(0x7fd4f40aa930 msgr2=0x7fd4f40a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 --2- 172.21.15.73:0/1321476878 >> v2:172.21.15.150:3300/0 conn(0x7fd4f40aa930 0x7fd4f40a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 >> v1:172.21.15.150:6789/0 conn(0x7fd4f4154910 legacy=0x7fd4f40aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f4064510 con 0x7fd4f4154530 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fd4e40015f0 con 0x7fd4f4154530 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 >> v1:172.21.15.73:6789/0 conn(0x7fd4f4154530 legacy=0x7fd4f40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e00038e0 0x7fd4e0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 0x7fd4e0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0005000 0x7fd4e000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f40c6390 con 0x7fd4e0004690 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e000bf60 con 0x7fd4e0005000 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e000c0f0 con 0x7fd4e00038e0 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 0x7fd4e0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f2d9d700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0005000 0x7fd4e000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f259c700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e00038e0 0x7fd4e0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e00038e0 msgr2=0x7fd4e0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4e00038e0 0x7fd4e0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0005000 msgr2=0x7fd4e000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0005000 0x7fd4e000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1321476878 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e00033f0 con 0x7fd4e0004690 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f2d9d700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0005000 0x7fd4e000b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f2d9d700 1 -- 172.21.15.73:0/1321476878 reap_dead start 2023-11-28T03:08:09.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 0x7fd4e0004b10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7fd4dc001730 tx=0x7fd4dc007bb0).ready entity=mon.0 client_cookie=1ca2e26820f14abc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007020 con 0x7fd4e0004690 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc007180 con 0x7fd4e0004690 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.407+0000 7fd4f1d9b700 1 -- 172.21.15.73:0/1321476878 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc007520 con 0x7fd4e0004690 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 msgr2=0x7fd4e0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 0x7fd4e0004b10 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 rx=0x7fd4dc001730 tx=0x7fd4dc007bb0).stop 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1321476878 shutdown_connections 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1321476878 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4e0004690 0x7fd4e0004b10 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1321476878 >> 172.21.15.73:0/1321476878 conn(0x7fd4f4094930 msgr2=0x7fd4f406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1321476878 shutdown_connections 2023-11-28T03:08:09.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.408+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1321476878 wait complete. 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.409+0000 7fd4f8b19700 1 Processor -- start 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.409+0000 7fd4f8b19700 1 -- start start 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0004690 0x7fd4f4141e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4f4151c80 0x7fd4f4152100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4150c70 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f41517f0 con 0x7fd4e0004690 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4f4150ec0 con 0x7fd4f4151c80 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59888/0 (socket says 172.21.15.73:59888) 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1328054135 learned_addr learned my addr 172.21.15.73:0/1328054135 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f259c700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0004690 0x7fd4f4141e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f2d9d700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4f4151c80 0x7fd4f4152100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4f4151c80 msgr2=0x7fd4f4152100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4f4151c80 0x7fd4f4152100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0004690 msgr2=0x7fd4f4141e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0004690 0x7fd4f4141e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 -- 172.21.15.73:0/1328054135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4f41511a0 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4e9d9b700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7fd4e0005640 tx=0x7fd4dc015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:09.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc013740 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4dc0138a0 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4f4151480 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4dc013c40 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.410+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4f40aaf60 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.411+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4f406c860 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.412+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 37) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fd4dc0123c0 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.412+0000 7fd4eaffd700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 0x7fd4d40614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.412+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fd4dc07b0d0 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.412+0000 7fd4f259c700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 0x7fd4d40614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.412+0000 7fd4f259c700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 0x7fd4d40614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd4e4001380 tx=0x7fd4e4009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.416+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd4dc04c4f0 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.580+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd4f4150c70 con 0x7fd4f40ac8c0 2023-11-28T03:08:09.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:09.717+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd4dc079050 con 0x7fd4f40ac8c0 2023-11-28T03:08:10.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.275+0000 7fd4eaffd700 1 -- 172.21.15.73:0/1328054135 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v101) v1 ==== 169+0+0 (secure 0 0 0) 0x7fd4dc040fa0 con 0x7fd4f40ac8c0 2023-11-28T03:08:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 msgr2=0x7fd4d40614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 0x7fd4d40614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fd4e4001380 tx=0x7fd4e4009040).stop 2023-11-28T03:08:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 msgr2=0x7fd4f40acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 rx=0x7fd4e0005640 tx=0x7fd4dc015000).stop 2023-11-28T03:08:10.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 shutdown_connections 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd4d405f030 0x7fd4d40614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4f4151c80 0x7fd4f4152100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.279+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd4e0004690 0x7fd4f4141e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.280+0000 7fd4f8b19700 1 --2- 172.21.15.73:0/1328054135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4f40ac8c0 0x7fd4f40acd40 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.280+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 >> 172.21.15.73:0/1328054135 conn(0x7fd4f4094930 msgr2=0x7fd4f4150030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:10.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.280+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 shutdown_connections 2023-11-28T03:08:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.280+0000 7fd4f8b19700 1 -- 172.21.15.73:0/1328054135 wait complete. 2023-11-28T03:08:10.283 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:08:10.294 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:08:10.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.406+0000 7f7113acd700 1 Processor -- start 2023-11-28T03:08:10.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.406+0000 7f7113acd700 1 -- start start 2023-11-28T03:08:10.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7113acd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f710c098d10 0x7f710c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7113acd700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c0c72d0 con 0x7f710c0c6190 2023-11-28T03:08:10.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7113acd700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f710c064570 con 0x7f710c158320 2023-11-28T03:08:10.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7113acd700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f710c154bc0 con 0x7f710c098d10 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7111869700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f710c098d10 0x7f710c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7111068700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f710c0c6190 0x7f710c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33206/0 (socket says 172.21.15.73:33206) 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.407+0000 7f7111068700 1 -- 172.21.15.73:0/980431349 learned_addr learned my addr 172.21.15.73:0/980431349 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7111869700 1 -- 172.21.15.73:0/980431349 >> v1:172.21.15.150:6789/0 conn(0x7f710c158320 legacy=0x7f710c0a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7111869700 1 -- 172.21.15.73:0/980431349 >> v1:172.21.15.73:6789/0 conn(0x7f710c0c6190 legacy=0x7f710c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7110867700 1 -- 172.21.15.73:0/980431349 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 540969898 0 0) 0x7f710c0c72d0 con 0x7f710c0c6190 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7111869700 1 -- 172.21.15.73:0/980431349 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f710c0c66c0 con 0x7f710c098d10 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7111869700 1 --2- 172.21.15.73:0/980431349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c098d10 0x7f710c0c7760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f7108009e60 tx=0x7f7108009780).ready entity=mon.0 client_cookie=75716e0efe1d3e0c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7110867700 1 -- 172.21.15.73:0/980431349 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f710800e070 con 0x7f710c098d10 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7110867700 1 -- 172.21.15.73:0/980431349 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f710800bc30 con 0x7f710c098d10 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7110867700 1 -- 172.21.15.73:0/980431349 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7108012410 con 0x7f710c098d10 2023-11-28T03:08:10.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 -- 172.21.15.73:0/980431349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c098d10 msgr2=0x7f710c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 --2- 172.21.15.73:0/980431349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c098d10 0x7f710c0c7760 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 rx=0x7f7108009e60 tx=0x7f7108009780).stop 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 -- 172.21.15.73:0/980431349 shutdown_connections 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 --2- 172.21.15.73:0/980431349 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c098d10 0x7f710c0c7760 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 -- 172.21.15.73:0/980431349 >> 172.21.15.73:0/980431349 conn(0x7f710c0cb310 msgr2=0x7f710c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 -- 172.21.15.73:0/980431349 shutdown_connections 2023-11-28T03:08:10.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.408+0000 7f7113acd700 1 -- 172.21.15.73:0/980431349 wait complete. 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.409+0000 7f7113acd700 1 Processor -- start 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.409+0000 7f7113acd700 1 -- start start 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 0x7f710c14e000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c158320 0x7f710c14c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c0aec20 con 0x7f710c0c6190 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710c1511f0 con 0x7f710c158320 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7113acd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f710c151450 con 0x7f710c098d10 2023-11-28T03:08:10.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7111068700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7111068700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59904/0 (socket says 172.21.15.73:59904) 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7111068700 1 -- 172.21.15.73:0/1182851540 learned_addr learned my addr 172.21.15.73:0/1182851540 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f7111869700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 0x7f710c14e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.410+0000 7f711206a700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c158320 0x7f710c14c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 -- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 msgr2=0x7f710c14e000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 0x7f710c14e000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 -- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c158320 msgr2=0x7f710c14c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c158320 0x7f710c14c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 -- 172.21.15.73:0/1182851540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7108009a20 con 0x7f710c0c6190 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111869700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 0x7f710c14e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:10.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7111068700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f70fc001eb0 tx=0x7f70fc007bf0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0072e0 con 0x7f710c0c6190 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70fc007460 con 0x7f710c0c6190 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710c151730 con 0x7f710c0c6190 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70fc0059f0 con 0x7f710c0c6190 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.411+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710c151cf0 con 0x7f710c0c6190 2023-11-28T03:08:10.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.412+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710c0ae9f0 con 0x7f710c0c6190 2023-11-28T03:08:10.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.412+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f70fc013450 con 0x7f710c0c6190 2023-11-28T03:08:10.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.413+0000 7f7101ffb700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 0x7f70f00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.413+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f70fc079830 con 0x7f710c0c6190 2023-11-28T03:08:10.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.413+0000 7f7111869700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 0x7f70f00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.413+0000 7f7111869700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 0x7f70f00615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f71080099f0 tx=0x7f7108009780).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.417+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f710c0ae9f0 con 0x7f710c0c6190 2023-11-28T03:08:10.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.585+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f710c0ae9f0 con 0x7f710c0c6190 2023-11-28T03:08:10.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.589+0000 7f7101ffb700 1 -- 172.21.15.73:0/1182851540 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v16) v1 ==== 120+0+0 (secure 0 0 0) 0x7f710c0ae9f0 con 0x7f710c0c6190 2023-11-28T03:08:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 msgr2=0x7f70f00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 0x7f70f00615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f71080099f0 tx=0x7f7108009780).stop 2023-11-28T03:08:10.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 msgr2=0x7f710c150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 rx=0x7f70fc001eb0 tx=0x7f70fc007bf0).stop 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 shutdown_connections 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f70f005f120 0x7f70f00615e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f710c158320 0x7f710c14c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f710c098d10 0x7f710c14e000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 --2- 172.21.15.73:0/1182851540 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f710c0c6190 0x7f710c150d00 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 >> 172.21.15.73:0/1182851540 conn(0x7f710c0cb310 msgr2=0x7f710c0c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 shutdown_connections 2023-11-28T03:08:10.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.591+0000 7f7113acd700 1 -- 172.21.15.73:0/1182851540 wait complete. 2023-11-28T03:08:10.604 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:08:10.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.718+0000 7f365d17e700 1 Processor -- start 2023-11-28T03:08:10.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365d17e700 1 -- start start 2023-11-28T03:08:10.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365d17e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f365814eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365d17e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36580634a0 con 0x7f3658062770 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365d17e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3658063780 con 0x7f365814eb00 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365d17e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3658137d90 con 0x7f36580bcdd0 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.719+0000 7f365759e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3658062770 0x7f3658062c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33214/0 (socket says 172.21.15.73:33214) 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f365759e700 1 -- 172.21.15.73:0/1497604843 learned_addr learned my addr 172.21.15.73:0/1497604843 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1890025400 0 0) 0x7f36580634a0 con 0x7f3658062770 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f365659c700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f365814eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f36400033f0 con 0x7f3658062770 2023-11-28T03:08:10.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3190702001 0 0) 0x7f36400033f0 con 0x7f3658062770 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 msgr2=0x7f365814eee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f365814eee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 >> v1:172.21.15.150:6789/0 conn(0x7f365814eb00 legacy=0x7f36580bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658063a60 con 0x7f3658062770 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f36500017f0 con 0x7f3658062770 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 >> v1:172.21.15.73:6789/0 conn(0x7f3658062770 legacy=0x7f3658062c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640003910 0x7f3640003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f3640004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3640004b80 0x7f3640005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3658137d90 con 0x7f3640004b80 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f364000b990 con 0x7f3640003910 2023-11-28T03:08:10.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f364000bb20 con 0x7f36580bcdd0 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f365759e700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3640004b80 0x7f3640005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f365659c700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f3640004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.720+0000 7f3656d9d700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640003910 0x7f3640003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 -- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 msgr2=0x7f3640004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580bcdd0 0x7f3640004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 -- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640003910 msgr2=0x7f3640003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3640003910 0x7f3640003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 -- 172.21.15.73:0/1497604843 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36400033f0 con 0x7f3640004b80 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3640004b80 0x7f3640005000 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f365000c3c0 tx=0x7f365000c6d0).ready entity=mon.0 client_cookie=be8ab98f1110e29f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650001f60 con 0x7f3640004b80 2023-11-28T03:08:10.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365000cd60 con 0x7f3640004b80 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f3655d9b700 1 -- 172.21.15.73:0/1497604843 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3650005d30 con 0x7f3640004b80 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 -- 172.21.15.73:0/1497604843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3640004b80 msgr2=0x7f3640005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 --2- 172.21.15.73:0/1497604843 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3640004b80 0x7f3640005000 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 rx=0x7f365000c3c0 tx=0x7f365000c6d0).stop 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365759e700 1 -- 172.21.15.73:0/1497604843 reap_dead start 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 -- 172.21.15.73:0/1497604843 shutdown_connections 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 -- 172.21.15.73:0/1497604843 >> 172.21.15.73:0/1497604843 conn(0x7f36580c8230 msgr2=0x7f36580c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:10.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 -- 172.21.15.73:0/1497604843 shutdown_connections 2023-11-28T03:08:10.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.721+0000 7f365d17e700 1 -- 172.21.15.73:0/1497604843 wait complete. 2023-11-28T03:08:10.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 Processor -- start 2023-11-28T03:08:10.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 -- start start 2023-11-28T03:08:10.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36581344d0 0x7f36580bf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580c25a0 0x7f36580c0360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3658138d50 con 0x7f36581348b0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3658137ef0 con 0x7f36581344d0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365d17e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36580c29e0 con 0x7f36580c25a0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.723+0000 7f365659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59928/0 (socket says 172.21.15.73:59928) 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 -- 172.21.15.73:0/3016455407 learned_addr learned my addr 172.21.15.73:0/3016455407 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365759e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580c25a0 0x7f36580c0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f3656d9d700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36581344d0 0x7f36580bf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 -- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580c25a0 msgr2=0x7f36580c0360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580c25a0 0x7f36580c0360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 -- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36581344d0 msgr2=0x7f36580bf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36581344d0 0x7f36580bf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 -- 172.21.15.73:0/3016455407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3658138140 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365659c700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f364c006cf0 tx=0x7f364c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00f6e0 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f364c00f840 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3658138420 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f364c00fbc0 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.724+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36581389e0 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.725+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3658138f00 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.725+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f364c014440 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.726+0000 7f363effd700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 0x7f3644061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.726+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f364c07a7f0 con 0x7f36581348b0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.726+0000 7f3656d9d700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 0x7f3644061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:10.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.726+0000 7f3656d9d700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 0x7f3644061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3648007fd0 tx=0x7f3648007c00).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:10.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.730+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3658138f00 con 0x7f36581348b0 2023-11-28T03:08:10.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.900+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f3658138f00 con 0x7f36581348b0 2023-11-28T03:08:10.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.903+0000 7f363effd700 1 -- 172.21.15.73:0/3016455407 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v17) v1 ==== 104+0+0 (secure 0 0 0) 0x7f3658138f00 con 0x7f36581348b0 2023-11-28T03:08:10.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 msgr2=0x7f3644061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 0x7f3644061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3648007fd0 tx=0x7f3648007c00).stop 2023-11-28T03:08:10.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 msgr2=0x7f36580c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 rx=0x7f364c006cf0 tx=0x7f364c006f40).stop 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 shutdown_connections 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f364405f150 0x7f3644061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36580c25a0 0x7f36580c0360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36581344d0 0x7f36580bf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 --2- 172.21.15.73:0/3016455407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36581348b0 0x7f36580c20b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 >> 172.21.15.73:0/3016455407 conn(0x7f36580c8230 msgr2=0x7f36580a68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 shutdown_connections 2023-11-28T03:08:10.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:10.905+0000 7f365d17e700 1 -- 172.21.15.73:0/3016455407 wait complete. 2023-11-28T03:08:10.918 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-11-28T03:08:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 Processor -- start 2023-11-28T03:08:11.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 -- start start 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f787c068090 0x7f787c0671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0bd590 con 0x7f787c155e40 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f787c0a6090 con 0x7f787c066e00 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.030+0000 7f78824c4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f787c156370 con 0x7f787c068090 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f7880a61700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f787c068090 0x7f787c0671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f787c155e40 0x7f787c067980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33216/0 (socket says 172.21.15.73:33216) 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787b7fe700 1 -- 172.21.15.73:0/3754723625 learned_addr learned my addr 172.21.15.73:0/3754723625 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f7880a61700 1 -- 172.21.15.73:0/3754723625 >> v1:172.21.15.150:6789/0 conn(0x7f787c066e00 legacy=0x7f787c0a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787affd700 1 -- 172.21.15.73:0/3754723625 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 716590654 0 0) 0x7f787c0bd590 con 0x7f787c155e40 2023-11-28T03:08:11.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f7880a61700 1 -- 172.21.15.73:0/3754723625 >> v1:172.21.15.73:6789/0 conn(0x7f787c155e40 legacy=0x7f787c067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f7880a61700 1 -- 172.21.15.73:0/3754723625 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c156650 con 0x7f787c068090 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f7880a61700 1 --2- 172.21.15.73:0/3754723625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c068090 0x7f787c0671e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f7874009e60 tx=0x7f7874009780).ready entity=mon.0 client_cookie=b25aae06f546a729 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787affd700 1 -- 172.21.15.73:0/3754723625 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400e070 con 0x7f787c068090 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787affd700 1 -- 172.21.15.73:0/3754723625 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787400bc30 con 0x7f787c068090 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.031+0000 7f787affd700 1 -- 172.21.15.73:0/3754723625 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7874012410 con 0x7f787c068090 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 -- 172.21.15.73:0/3754723625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c068090 msgr2=0x7f787c0671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 --2- 172.21.15.73:0/3754723625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c068090 0x7f787c0671e0 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 rx=0x7f7874009e60 tx=0x7f7874009780).stop 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 -- 172.21.15.73:0/3754723625 shutdown_connections 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 --2- 172.21.15.73:0/3754723625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c068090 0x7f787c0671e0 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 -- 172.21.15.73:0/3754723625 >> 172.21.15.73:0/3754723625 conn(0x7f787c0c7fd0 msgr2=0x7f787c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:11.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 -- 172.21.15.73:0/3754723625 shutdown_connections 2023-11-28T03:08:11.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.032+0000 7f78824c4700 1 -- 172.21.15.73:0/3754723625 wait complete. 2023-11-28T03:08:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.033+0000 7f78824c4700 1 Processor -- start 2023-11-28T03:08:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.033+0000 7f78824c4700 1 -- start start 2023-11-28T03:08:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c068090 0x7f787c1653e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f787c155e40 0x7f787c0c6a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c166ce0 con 0x7f787c066e00 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f787c0a7d10 con 0x7f787c068090 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f787c1660d0 con 0x7f787c155e40 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59944/0 (socket says 172.21.15.73:59944) 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 -- 172.21.15.73:0/856860650 learned_addr learned my addr 172.21.15.73:0/856860650 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787b7fe700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c068090 0x7f787c1653e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f7880a61700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f787c155e40 0x7f787c0c6a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 -- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f787c155e40 msgr2=0x7f787c0c6a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f787c155e40 0x7f787c0c6a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 -- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c068090 msgr2=0x7f787c1653e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c068090 0x7f787c1653e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 -- 172.21.15.73:0/856860650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7874009a20 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f787bfff700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f786c00c4c0 tx=0x7f786c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c007020 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f786c001e90 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f787c1663b0 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f786c0073b0 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.034+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f787c166970 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f787c06c860 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f786c0068b0 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f7863fff700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 0x7f7868061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f786c079a70 con 0x7f787c066e00 2023-11-28T03:08:11.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f787b7fe700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 0x7f7868061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.036+0000 7f787b7fe700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 0x7f7868061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f787c14db10 tx=0x7f7870009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.040+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f786c04aec0 con 0x7f787c066e00 2023-11-28T03:08:11.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.269+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f787c156930 con 0x7f787c066e00 2023-11-28T03:08:11.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.276+0000 7f7863fff700 1 -- 172.21.15.73:0/856860650 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v566) v1 ==== 166+0+0 (secure 0 0 0) 0x7f786c04e4e0 con 0x7f787c066e00 2023-11-28T03:08:11.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 msgr2=0x7f7868061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 0x7f7868061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f787c14db10 tx=0x7f7870009040).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 msgr2=0x7f787c0a7820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 rx=0x7f786c00c4c0 tx=0x7f786c007e00).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 shutdown_connections 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f786805f150 0x7f7868061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f787c155e40 0x7f787c0c6a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f787c066e00 0x7f787c0a7820 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 --2- 172.21.15.73:0/856860650 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f787c068090 0x7f787c1653e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 >> 172.21.15.73:0/856860650 conn(0x7f787c0c7fd0 msgr2=0x7f787c149c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 shutdown_connections 2023-11-28T03:08:11.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.278+0000 7f78824c4700 1 -- 172.21.15.73:0/856860650 wait complete. 2023-11-28T03:08:11.291 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-11-28T03:08:11.292 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-11-28T03:08:11.292 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:08:11.292 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-11-28T03:08:11.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.404+0000 7f4d4db36700 1 Processor -- start 2023-11-28T03:08:11.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.404+0000 7f4d4db36700 1 -- start start 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d4db36700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d480a8000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d4db36700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480c69a0 con 0x7f4d481544e0 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d4db36700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d480c13a0 con 0x7f4d481548c0 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d4db36700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480c1680 con 0x7f4d4806a610 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d477fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4d481544e0 0x7f4d480c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33230/0 (socket says 172.21.15.73:33230) 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d477fe700 1 -- 172.21.15.73:0/4110177952 learned_addr learned my addr 172.21.15.73:0/4110177952 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:11.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d47fff700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d480a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1879040725 0 0) 0x7f4d480c69a0 con 0x7f4d481544e0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4d300033f0 con 0x7f4d481544e0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3599783633 0 0) 0x7f4d300033f0 con 0x7f4d481544e0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.405+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 msgr2=0x7f4d480a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d480a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 >> v1:172.21.15.150:6789/0 conn(0x7f4d481548c0 legacy=0x7f4d480c6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d480c6630 con 0x7f4d481544e0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f4d38001970 con 0x7f4d481544e0 2023-11-28T03:08:11.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 >> v1:172.21.15.73:6789/0 conn(0x7f4d481544e0 legacy=0x7f4d480c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30003910 0x7f4d30003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d30004690 0x7f4d30004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d30005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480c1680 con 0x7f4d30004690 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3000b8e0 con 0x7f4d30003910 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d3000ba50 con 0x7f4d4806a610 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d30004690 0x7f4d30004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d477fe700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30003910 0x7f4d30003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d47fff700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d30005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 -- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 msgr2=0x7f4d30005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:11.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4806a610 0x7f4d30005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 -- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30003910 msgr2=0x7f4d30003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30003910 0x7f4d30003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 -- 172.21.15.73:0/4110177952 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d300033f0 con 0x7f4d30004690 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d46ffd700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d30004690 0x7f4d30004b10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f4d3c0018e0 tx=0x7f4d3c007bb0).ready entity=mon.0 client_cookie=f8a87cba988b4e9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c012400 con 0x7f4d30004690 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d3c012560 con 0x7f4d30004690 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.406+0000 7f4d467fc700 1 -- 172.21.15.73:0/4110177952 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d3c012920 con 0x7f4d30004690 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 -- 172.21.15.73:0/4110177952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d30004690 msgr2=0x7f4d30004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 --2- 172.21.15.73:0/4110177952 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d30004690 0x7f4d30004b10 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 rx=0x7f4d3c0018e0 tx=0x7f4d3c007bb0).stop 2023-11-28T03:08:11.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d47fff700 1 -- 172.21.15.73:0/4110177952 reap_dead start 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 -- 172.21.15.73:0/4110177952 shutdown_connections 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 -- 172.21.15.73:0/4110177952 >> 172.21.15.73:0/4110177952 conn(0x7f4d48085020 msgr2=0x7f4d48094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 -- 172.21.15.73:0/4110177952 shutdown_connections 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.407+0000 7f4d4db36700 1 -- 172.21.15.73:0/4110177952 wait complete. 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.408+0000 7f4d4db36700 1 Processor -- start 2023-11-28T03:08:11.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.408+0000 7f4d4db36700 1 -- start start 2023-11-28T03:08:11.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30004690 0x7f4d4814be70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4814c360 0x7f4d480a8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d48096fc0 con 0x7f4d480a84f0 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480969b0 con 0x7f4d30004690 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d4db36700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d480a8910 con 0x7f4d4814c360 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59958/0 (socket says 172.21.15.73:59958) 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 -- 172.21.15.73:0/2933037423 learned_addr learned my addr 172.21.15.73:0/2933037423 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d477fe700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30004690 0x7f4d4814be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d46ffd700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4814c360 0x7f4d480a8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 -- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4814c360 msgr2=0x7f4d480a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4814c360 0x7f4d480a8000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 -- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30004690 msgr2=0x7f4d4814be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30004690 0x7f4d4814be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 -- 172.21.15.73:0/2933037423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d48095f30 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d47fff700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f4d48062ec0 tx=0x7f4d40009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000f700 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.409+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d4000f860 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.410+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d48096c50 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.410+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d4000fc00 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.410+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d48095750 con 0x7f4d480a84f0 2023-11-28T03:08:11.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.410+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d480c83f0 con 0x7f4d480a84f0 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.411+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4d4001f460 con 0x7f4d480a84f0 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.411+0000 7f4d2f7fe700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 0x7f4d34061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.411+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f4d4007a6d0 con 0x7f4d480a84f0 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.411+0000 7f4d477fe700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 0x7f4d34061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.411+0000 7f4d477fe700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 0x7f4d34061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f4d38001460 tx=0x7f4d38009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:11.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.415+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d4004bac0 con 0x7f4d480a84f0 2023-11-28T03:08:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:11.644+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f4d4806c860 con 0x7f4d480a84f0 2023-11-28T03:08:12.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.278+0000 7f4d2f7fe700 1 -- 172.21.15.73:0/2933037423 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v567) v1 ==== 186+0+0 (secure 0 0 0) 0x7f4d4004f0e0 con 0x7f4d480a84f0 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 msgr2=0x7f4d34061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 0x7f4d34061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f4d38001460 tx=0x7f4d38009040).stop 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 msgr2=0x7f4d48095260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 rx=0x7f4d48062ec0 tx=0x7f4d40009060).stop 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 shutdown_connections 2023-11-28T03:08:12.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f4d3405f150 0x7f4d34061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d480a84f0 0x7f4d48095260 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d30004690 0x7f4d4814be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 --2- 172.21.15.73:0/2933037423 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d4814c360 0x7f4d480a8000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 >> 172.21.15.73:0/2933037423 conn(0x7f4d48085020 msgr2=0x7f4d480b04e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 shutdown_connections 2023-11-28T03:08:12.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.280+0000 7f4d4db36700 1 -- 172.21.15.73:0/2933037423 wait complete. 2023-11-28T03:08:12.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:08:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.408+0000 7f2e48e6a700 1 Processor -- start 2023-11-28T03:08:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.408+0000 7f2e48e6a700 1 -- start start 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.408+0000 7f2e48e6a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2e4414e530 0x7f2e440c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e48e6a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e440c7bc0 con 0x7f2e44154950 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e48e6a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e44155920 con 0x7f2e44158570 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e48e6a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e44154e00 con 0x7f2e4414e530 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e42ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2e44154950 0x7f2e4414e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33232/0 (socket says 172.21.15.73:33232) 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e42ffd700 1 -- 172.21.15.73:0/1449858996 learned_addr learned my addr 172.21.15.73:0/1449858996 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e437fe700 1 --2- 172.21.15.73:0/1449858996 >> v2:172.21.15.150:3300/0 conn(0x7f2e4414e530 0x7f2e440c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921522649 0 0) 0x7f2e440c7bc0 con 0x7f2e44154950 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2e300033f0 con 0x7f2e44154950 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2435056219 0 0) 0x7f2e300033f0 con 0x7f2e44154950 2023-11-28T03:08:12.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 >> v2:172.21.15.150:3300/0 conn(0x7f2e4414e530 msgr2=0x7f2e440c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 --2- 172.21.15.73:0/1449858996 >> v2:172.21.15.150:3300/0 conn(0x7f2e4414e530 0x7f2e440c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 >> v1:172.21.15.150:6789/0 conn(0x7f2e44158570 legacy=0x7f2e440aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e44155080 con 0x7f2e44154950 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2e2c001470 con 0x7f2e44154950 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 >> v1:172.21.15.73:6789/0 conn(0x7f2e44154950 legacy=0x7f2e4414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 0x7f2e30003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30004690 0x7f2e30004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e30005000 0x7f2e3000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e44154e00 con 0x7f2e300038e0 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3000bf60 con 0x7f2e30004690 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e3000c0f0 con 0x7f2e30005000 2023-11-28T03:08:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.409+0000 7f2e437fe700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 0x7f2e30003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e42ffd700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30004690 0x7f2e30004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e43fff700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e30005000 0x7f2e3000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 -- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e30005000 msgr2=0x7f2e3000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e30005000 0x7f2e3000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 -- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30004690 msgr2=0x7f2e30004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30004690 0x7f2e30004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e43fff700 1 -- 172.21.15.73:0/1449858996 reap_dead start 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 -- 172.21.15.73:0/1449858996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e300033f0 con 0x7f2e300038e0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e42ffd700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e30004690 0x7f2e30004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e437fe700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 0x7f2e30003d00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f2e3c00e210 tx=0x7f2e3c00b040).ready entity=mon.0 client_cookie=7eb9b0e62734e05d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c009ac0 con 0x7f2e300038e0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3c00f040 con 0x7f2e300038e0 2023-11-28T03:08:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e427fc700 1 -- 172.21.15.73:0/1449858996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3c00ecd0 con 0x7f2e300038e0 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e48e6a700 1 -- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 msgr2=0x7f2e30003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 0x7f2e30003d00 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 rx=0x7f2e3c00e210 tx=0x7f2e3c00b040).stop 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e48e6a700 1 -- 172.21.15.73:0/1449858996 shutdown_connections 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/1449858996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e300038e0 0x7f2e30003d00 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.410+0000 7f2e48e6a700 1 -- 172.21.15.73:0/1449858996 >> 172.21.15.73:0/1449858996 conn(0x7f2e44064990 msgr2=0x7f2e44064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.411+0000 7f2e48e6a700 1 -- 172.21.15.73:0/1449858996 shutdown_connections 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.411+0000 7f2e48e6a700 1 -- 172.21.15.73:0/1449858996 wait complete. 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.412+0000 7f2e48e6a700 1 Processor -- start 2023-11-28T03:08:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.412+0000 7f2e48e6a700 1 -- start start 2023-11-28T03:08:12.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.412+0000 7f2e48e6a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e300038e0 0x7f2e4414db00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e440af660 0x7f2e440afae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4415b880 con 0x7f2e440c3590 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e440affd0 con 0x7f2e300038e0 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4415af50 con 0x7f2e440af660 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59976/0 (socket says 172.21.15.73:59976) 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 -- 172.21.15.73:0/3169401454 learned_addr learned my addr 172.21.15.73:0/3169401454 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e437fe700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e300038e0 0x7f2e4414db00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e43fff700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e440af660 0x7f2e440afae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 -- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e440af660 msgr2=0x7f2e440afae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e440af660 0x7f2e440afae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 -- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e300038e0 msgr2=0x7f2e4414db00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e300038e0 0x7f2e4414db00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 -- 172.21.15.73:0/3169401454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4415b230 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e42ffd700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f2e2c00e790 tx=0x7f2e2c001e20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c010520 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e2c010680 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4415b510 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e2c010a20 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.413+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4414dff0 con 0x7f2e440c3590 2023-11-28T03:08:12.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.414+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e44062ee0 con 0x7f2e440c3590 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.414+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2e2c01d440 con 0x7f2e440c3590 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.415+0000 7f2e3b7fe700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 0x7f2e24061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.415+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f2e2c079ba0 con 0x7f2e440c3590 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.415+0000 7f2e437fe700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 0x7f2e24061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.415+0000 7f2e437fe700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 0x7f2e24061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2e3c00afd0 tx=0x7f2e3c01a000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.419+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2e2c04af90 con 0x7f2e440c3590 2023-11-28T03:08:12.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.583+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f2e4406c7c0 con 0x7f2e440c3590 2023-11-28T03:08:12.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.587+0000 7f2e3b7fe700 1 -- 172.21.15.73:0/3169401454 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v18)=0 v18) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2e2c04e5b0 con 0x7f2e440c3590 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 msgr2=0x7f2e24061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 0x7f2e24061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f2e3c00afd0 tx=0x7f2e3c01a000).stop 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 msgr2=0x7f2e440c3a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 rx=0x7f2e2c00e790 tx=0x7f2e2c001e20).stop 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 shutdown_connections 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2e2405f150 0x7f2e24061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2e440af660 0x7f2e440afae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2e300038e0 0x7f2e4414db00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 --2- 172.21.15.73:0/3169401454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2e440c3590 0x7f2e440c3a10 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 >> 172.21.15.73:0/3169401454 conn(0x7f2e44064990 msgr2=0x7f2e440caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.588+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 shutdown_connections 2023-11-28T03:08:12.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.589+0000 7f2e48e6a700 1 -- 172.21.15.73:0/3169401454 wait complete. 2023-11-28T03:08:12.602 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:08:12.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:08:12.638 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:08:12.684 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:08:12.684 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:08:12.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.797+0000 7f1f02eb2700 1 Processor -- start 2023-11-28T03:08:12.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.797+0000 7f1f02eb2700 1 -- start start 2023-11-28T03:08:12.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f02eb2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1efc0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f02eb2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc0a9a90 con 0x7f1efc0caab0 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f02eb2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1efc154ce0 con 0x7f1efc0ca180 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f02eb2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0aab80 con 0x7f1efc0cae90 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f00c4e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1efc0caab0 0x7f1efc0aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33234/0 (socket says 172.21.15.73:33234) 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f00c4e700 1 -- 172.21.15.73:0/1106653795 learned_addr learned my addr 172.21.15.73:0/1106653795 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1efc0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4234443933 0 0) 0x7f1efc0a9a90 con 0x7f1efc0caab0 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ee40033f0 con 0x7f1efc0caab0 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4081279063 0 0) 0x7f1ee40033f0 con 0x7f1efc0caab0 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 msgr2=0x7f1efc0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1efc0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 >> v1:172.21.15.150:6789/0 conn(0x7f1efc0ca180 legacy=0x7f1efc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0a8e80 con 0x7f1efc0caab0 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f1ef8001970 con 0x7f1efc0caab0 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 >> v1:172.21.15.73:6789/0 conn(0x7f1efc0caab0 legacy=0x7f1efc0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1ee4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4004b80 0x7f1ee4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0aab80 con 0x7f1ee4003910 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee400b910 con 0x7f1ee4004b80 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ee400bad0 con 0x7f1efc0cae90 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.798+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f0144f700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4004b80 0x7f1ee4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1ee4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 -- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 msgr2=0x7f1ee4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1ee4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 -- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4004b80 msgr2=0x7f1ee4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4004b80 0x7f1ee4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 -- 172.21.15.73:0/1106653795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ee40033f0 con 0x7f1ee4003910 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc0cae90 0x7f1ee4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f1ef800c5f0 tx=0x7f1ef8007b20).ready entity=mon.0 client_cookie=6bafa4e2822747ad server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8007d50 con 0x7f1ee4003910 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ef800cdf0 con 0x7f1ee4003910 2023-11-28T03:08:12.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1ef37fe700 1 -- 172.21.15.73:0/1106653795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8005c00 con 0x7f1ee4003910 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f02eb2700 1 -- 172.21.15.73:0/1106653795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee4003910 msgr2=0x7f1ee4003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/1106653795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ee4003910 0x7f1ee4003d30 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 rx=0x7f1ef800c5f0 tx=0x7f1ef8007b20).stop 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f0144f700 1 -- 172.21.15.73:0/1106653795 reap_dead start 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f02eb2700 1 -- 172.21.15.73:0/1106653795 shutdown_connections 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f02eb2700 1 -- 172.21.15.73:0/1106653795 >> 172.21.15.73:0/1106653795 conn(0x7f1efc094920 msgr2=0x7f1efc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.799+0000 7f1f02eb2700 1 -- 172.21.15.73:0/1106653795 shutdown_connections 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.800+0000 7f1f02eb2700 1 -- 172.21.15.73:0/1106653795 wait complete. 2023-11-28T03:08:12.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.801+0000 7f1f02eb2700 1 Processor -- start 2023-11-28T03:08:12.806 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.801+0000 7f1f02eb2700 1 -- start start 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4003910 0x7f1efc145960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc158800 0x7f1efc157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc0c6ad0 con 0x7f1efc144f00 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc158c80 con 0x7f1ee4003910 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1efc157ec0 con 0x7f1efc158800 2023-11-28T03:08:12.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:59998/0 (socket says 172.21.15.73:59998) 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 -- 172.21.15.73:0/3283252005 learned_addr learned my addr 172.21.15.73:0/3283252005 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4003910 0x7f1efc145960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f0144f700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc158800 0x7f1efc157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 -- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc158800 msgr2=0x7f1efc157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc158800 0x7f1efc157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 -- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4003910 msgr2=0x7f1efc145960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4003910 0x7f1efc145960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 -- 172.21.15.73:0/3283252005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1efc0c6480 con 0x7f1efc144f00 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef3fff700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f1eec011990 tx=0x7f1eec008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec013620 con 0x7f1efc144f00 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eec013780 con 0x7f1efc144f00 2023-11-28T03:08:12.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eec013b00 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1efc0c6760 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.802+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1efc157640 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.803+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1efc0c62e0 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.804+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1eec015480 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.804+0000 7f1ef0ff9700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 0x7f1ee80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.804+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(101..101 src has 1..101) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f1eec07a680 con 0x7f1efc144f00 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.804+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 0x7f1ee80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.804+0000 7f1f00c4e700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 0x7f1ee80615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1ef8001270 tx=0x7f1ef8015000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:12.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:12.808+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1efc0c62e0 con 0x7f1efc144f00 2023-11-28T03:08:13.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.013+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1efc0c62e0 con 0x7f1efc144f00 2023-11-28T03:08:13.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.013+0000 7f1ef0ff9700 1 -- 172.21.15.73:0/3283252005 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f1efc0c62e0 con 0x7f1efc144f00 2023-11-28T03:08:13.015 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:13.015 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":36,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5666,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1954051330","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":1954051330},{"type":"v1","addr":"172.21.15.150:6834","nonce":1954051330}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6523,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/939836819","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":939836819},{"type":"v1","addr":"172.21.15.150:6835","nonce":939836819}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":35},{"gid":6696,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/926733739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":926733739},{"type":"v1","addr":"172.21.15.73:6836","nonce":926733739}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29},{"gid":6699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/1463959408","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1463959408},{"type":"v1","addr":"172.21.15.73:6837","nonce":1463959408}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":29}],"filesystems":[]} 2023-11-28T03:08:13.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 msgr2=0x7f1ee80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 0x7f1ee80615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f1ef8001270 tx=0x7f1ef8015000).stop 2023-11-28T03:08:13.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 msgr2=0x7f1efc158310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 rx=0x7f1eec011990 tx=0x7f1eec008040).stop 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 shutdown_connections 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1ee805f120 0x7f1ee80615e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1efc158800 0x7f1efc157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ee4003910 0x7f1efc145960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 --2- 172.21.15.73:0/3283252005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1efc144f00 0x7f1efc158310 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 >> 172.21.15.73:0/3283252005 conn(0x7f1efc094920 msgr2=0x7f1efc0aedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 shutdown_connections 2023-11-28T03:08:13.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.015+0000 7f1f02eb2700 1 -- 172.21.15.73:0/3283252005 wait complete. 2023-11-28T03:08:13.018 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 36 2023-11-28T03:08:13.029 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:08:13.029 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:08:13.030 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:08:13.030 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:08:13.030 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:08:13.030 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:08:13.031 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:08:13.031 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:08:13.031 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:08:13.031 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:08:13.032 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:08:13.032 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:08:13.032 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:08:13.032 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:08:13.033 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:08:13.033 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:08:13.041 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:08:13.041 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:08:13.042 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:08:13.043 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:08:13.045 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:08:13.045 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:08:13.046 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:08:13.046 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:08:13.083 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:08:13.084 DEBUG:teuthology.parallel:result is None 2023-11-28T03:08:13.084 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:08:13.085 DEBUG:teuthology.parallel:result is None 2023-11-28T03:08:13.087 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:08:13.087 DEBUG:teuthology.parallel:result is None 2023-11-28T03:08:13.088 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:08:13.089 DEBUG:teuthology.parallel:result is None 2023-11-28T03:08:13.089 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:08:13.156 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:08:13.156 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:08:13.171 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:08:13.175 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:08:13.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.212+0000 7f3641693700 1 Processor -- start 2023-11-28T03:08:13.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.212+0000 7f3641693700 1 -- start start 2023-11-28T03:08:13.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f3641693700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c151a20 0x7f363c06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f3641693700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f363c0bd1f0 0x7f363c0bd640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f3641693700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f363c0c3470 con 0x7f363c069d10 2023-11-28T03:08:13.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f3641693700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f363c0c48f0 con 0x7f363c0bd1f0 2023-11-28T03:08:13.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f3641693700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c0bdb30 con 0x7f363c151a20 2023-11-28T03:08:13.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f363c069d10 0x7f363c0be1b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33240/0 (socket says 172.21.15.73:33240) 2023-11-28T03:08:13.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f363c0bd1f0 0x7f363c0bd640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363affd700 1 -- 172.21.15.73:0/3893103069 learned_addr learned my addr 172.21.15.73:0/3893103069 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363a7fc700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c151a20 0x7f363c06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363b7fe700 1 -- 172.21.15.73:0/3893103069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c151a20 msgr2=0x7f363c06a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363b7fe700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c151a20 0x7f363c06a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363b7fe700 1 -- 172.21.15.73:0/3893103069 >> v1:172.21.15.73:6789/0 conn(0x7f363c069d10 legacy=0x7f363c0be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.213+0000 7f363b7fe700 1 -- 172.21.15.73:0/3893103069 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f363c151ea0 con 0x7f363c0bd1f0 2023-11-28T03:08:13.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.214+0000 7f363b7fe700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c0bd1f0 0x7f363c0bd640 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f362c009760 tx=0x7f362c009060).ready entity=mon.0 client_cookie=723f89fe89fcdc80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:13.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.215+0000 7f3639ffb700 1 -- 172.21.15.73:0/3893103069 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00f510 con 0x7f363c0bd1f0 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.215+0000 7f3639ffb700 1 -- 172.21.15.73:0/3893103069 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f362c00f670 con 0x7f363c0bd1f0 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.215+0000 7f3639ffb700 1 -- 172.21.15.73:0/3893103069 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f362c00f9f0 con 0x7f363c0bd1f0 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 -- 172.21.15.73:0/3893103069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c0bd1f0 msgr2=0x7f363c0bd640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c0bd1f0 0x7f363c0bd640 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 rx=0x7f362c009760 tx=0x7f362c009060).stop 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 -- 172.21.15.73:0/3893103069 shutdown_connections 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c0bd1f0 0x7f363c0bd640 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 --2- 172.21.15.73:0/3893103069 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c151a20 0x7f363c06a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 -- 172.21.15.73:0/3893103069 >> 172.21.15.73:0/3893103069 conn(0x7f363c0642d0 msgr2=0x7f363c0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 -- 172.21.15.73:0/3893103069 shutdown_connections 2023-11-28T03:08:13.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.216+0000 7f3641693700 1 -- 172.21.15.73:0/3893103069 wait complete. 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.217+0000 7f3641693700 1 Processor -- start 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.217+0000 7f3641693700 1 -- start start 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c0bd1f0 0x7f363c0a9b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c151a20 0x7f363c0a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c1560e0 con 0x7f363c069d10 2023-11-28T03:08:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f363c153200 con 0x7f363c151a20 2023-11-28T03:08:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f3641693700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f363c152bd0 con 0x7f363c0bd1f0 2023-11-28T03:08:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60048/0 (socket says 172.21.15.73:60048) 2023-11-28T03:08:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 -- 172.21.15.73:0/1027697911 learned_addr learned my addr 172.21.15.73:0/1027697911 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363a7fc700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c0bd1f0 0x7f363c0a9b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363b7fe700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c151a20 0x7f363c0a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 -- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c0bd1f0 msgr2=0x7f363c0a9b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c0bd1f0 0x7f363c0a9b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 -- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c151a20 msgr2=0x7f363c0a79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:13.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c151a20 0x7f363c0a79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:13.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 -- 172.21.15.73:0/1027697911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f362c0092a0 con 0x7f363c069d10 2023-11-28T03:08:13.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.218+0000 7f363affd700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f363400c840 tx=0x7f3634001610).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:13.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.219+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634005d30 con 0x7f363c069d10 2023-11-28T03:08:13.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.219+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3634005e90 con 0x7f363c069d10 2023-11-28T03:08:13.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.219+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f363c0aa480 con 0x7f363c069d10 2023-11-28T03:08:13.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.219+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3634007320 con 0x7f363c069d10 2023-11-28T03:08:13.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.219+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f363c152e90 con 0x7f363c069d10 2023-11-28T03:08:13.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.220+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f363c06c860 con 0x7f363c069d10 2023-11-28T03:08:13.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.220+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3634007750 con 0x7f363c069d10 2023-11-28T03:08:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.220+0000 7f3622ffd700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 0x7f3628061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.220+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f3634079fb0 con 0x7f363c069d10 2023-11-28T03:08:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.220+0000 7f363a7fc700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 0x7f3628061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.221+0000 7f363a7fc700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 0x7f3628061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f363c154b90 tx=0x7f3630009060).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:13.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.224+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f363404b3a0 con 0x7f363c069d10 2023-11-28T03:08:13.247 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a is failed for ~0s 2023-11-28T03:08:13.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2023-11-28T03:08:13.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T03:08:13.248 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-28T03:08:13.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:13.403+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f363c06a620 con 0x7f363c069d10 2023-11-28T03:08:14.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.281+0000 7f3622ffd700 1 -- 172.21.15.73:0/1027697911 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v106) v1 ==== 97+0+0 (secure 0 0 0) 0x7f363404e9c0 con 0x7f363c069d10 2023-11-28T03:08:14.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 msgr2=0x7f3628061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 0x7f3628061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f363c154b90 tx=0x7f3630009060).stop 2023-11-28T03:08:14.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 msgr2=0x7f363c0c4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 rx=0x7f363400c840 tx=0x7f3634001610).stop 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 shutdown_connections 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f362805f150 0x7f3628061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f363c151a20 0x7f363c0a79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f363c069d10 0x7f363c0c4110 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 --2- 172.21.15.73:0/1027697911 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f363c0bd1f0 0x7f363c0a9b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 >> 172.21.15.73:0/1027697911 conn(0x7f363c0642d0 msgr2=0x7f363c154f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 shutdown_connections 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.283+0000 7f3641693700 1 -- 172.21.15.73:0/1027697911 wait complete. 2023-11-28T03:08:14.285 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:08:14.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:08:14.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.412+0000 7efe30f62700 1 Processor -- start 2023-11-28T03:08:14.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.412+0000 7efe30f62700 1 -- start start 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe30f62700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe30f62700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe2c0c6e30 0x7efe2c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe30f62700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe2c0c69a0 con 0x7efe2c06a610 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe30f62700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe2c0c13a0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe30f62700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c0c1680 con 0x7efe2c1544e0 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe2c0c6e30 0x7efe2c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efe2c0c6e30 0x7efe2c0c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54762/0 (socket says 172.21.15.73:54762) 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 -- 172.21.15.73:0/714185543 learned_addr learned my addr 172.21.15.73:0/714185543 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:14.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2a59c700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 -- 172.21.15.73:0/714185543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 msgr2=0x7efe2c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 -- 172.21.15.73:0/714185543 >> v1:172.21.15.150:6789/0 conn(0x7efe2c06a610 legacy=0x7efe2c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.413+0000 7efe2ad9d700 1 -- 172.21.15.73:0/714185543 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2c0c6630 con 0x7efe2c0c6e30 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe2ad9d700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c0c72e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7efe24009f20 tx=0x7efe24009840).ready entity=mon.0 client_cookie=89db607b1bcbf28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe29d9b700 1 -- 172.21.15.73:0/714185543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400e070 con 0x7efe2c0c6e30 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe29d9b700 1 -- 172.21.15.73:0/714185543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe2400b9d0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe29d9b700 1 -- 172.21.15.73:0/714185543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe2400bd50 con 0x7efe2c0c6e30 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 -- 172.21.15.73:0/714185543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 msgr2=0x7efe2c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c0c72e0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 rx=0x7efe24009f20 tx=0x7efe24009840).stop 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 -- 172.21.15.73:0/714185543 shutdown_connections 2023-11-28T03:08:14.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c0c72e0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 --2- 172.21.15.73:0/714185543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 -- 172.21.15.73:0/714185543 >> 172.21.15.73:0/714185543 conn(0x7efe2c085020 msgr2=0x7efe2c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 -- 172.21.15.73:0/714185543 shutdown_connections 2023-11-28T03:08:14.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.414+0000 7efe30f62700 1 -- 172.21.15.73:0/714185543 wait complete. 2023-11-28T03:08:14.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 Processor -- start 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 -- start start 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe2c06a610 0x7efe2c0981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c0cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c0986e0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c0c3bc0 con 0x7efe2c06a610 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe30f62700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe2c099160 con 0x7efe2c1544e0 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe21d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe21d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54774/0 (socket says 172.21.15.73:54774) 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe21d9b700 1 -- 172.21.15.73:0/4044528577 learned_addr learned my addr 172.21.15.73:0/4044528577 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:14.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.416+0000 7efe2ad9d700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c0cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 -- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 msgr2=0x7efe2c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c0cafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 -- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe2c06a610 msgr2=0x7efe2c0981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe2c06a610 0x7efe2c0981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 -- 172.21.15.73:0/4044528577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe24009ae0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe21d9b700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7efe1400c440 tx=0x7efe14007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1400cea0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe14007020 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2c0988c0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe140073c0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.417+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2c0961c0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.418+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe2c098cb0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.418+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efe140068b0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.419+0000 7efe22ffd700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 0x7efe0c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.419+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7efe140791d0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.419+0000 7efe2a59c700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 0x7efe0c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.419+0000 7efe2a59c700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 0x7efe0c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7efe1c006fd0 tx=0x7efe1c006f40).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.423+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efe14047af0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.654+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7efe2c0c6250 con 0x7efe2c0c6e30 2023-11-28T03:08:14.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.654+0000 7efe22ffd700 1 -- 172.21.15.73:0/4044528577 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) v1 ==== 80+0+5304 (secure 0 0 0) 0x7efe1404a7f0 con 0x7efe2c0c6e30 2023-11-28T03:08:14.655 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:14.655 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.656 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:08:14.657 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.658 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:08:14.659 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.660 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:08:14.661 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:08:14.662 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.663 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.664 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:08:14.665 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:08:14.666 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.667 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.668 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:08:14.669 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.670 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.671 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:08:14.672 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:08:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 msgr2=0x7efe0c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 0x7efe0c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7efe1c006fd0 tx=0x7efe1c006f40).stop 2023-11-28T03:08:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 msgr2=0x7efe2c095cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 rx=0x7efe1400c440 tx=0x7efe14007dd0).stop 2023-11-28T03:08:14.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 shutdown_connections 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe0c05f150 0x7efe0c061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe2c1544e0 0x7efe2c0cafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe2c06a610 0x7efe2c0981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 --2- 172.21.15.73:0/4044528577 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe2c0c6e30 0x7efe2c095cd0 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 >> 172.21.15.73:0/4044528577 conn(0x7efe2c085020 msgr2=0x7efe2c0b0c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 shutdown_connections 2023-11-28T03:08:14.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.656+0000 7efe30f62700 1 -- 172.21.15.73:0/4044528577 wait complete. 2023-11-28T03:08:14.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:14.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.787+0000 7f071edf7700 1 Processor -- start 2023-11-28T03:08:14.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.787+0000 7f071edf7700 1 -- start start 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071edf7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 0x7f0718155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071edf7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07180683d0 con 0x7f07181544d0 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071edf7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0718067850 con 0x7f0718064e20 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071edf7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0718067650 con 0x7f0718096590 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071cb93700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f07181544d0 0x7f0718066f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35470/0 (socket says 172.21.15.73:35470) 2023-11-28T03:08:14.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.788+0000 7f071cb93700 1 -- 172.21.15.73:0/473079372 learned_addr learned my addr 172.21.15.73:0/473079372 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f071d394700 1 --2- 172.21.15.73:0/473079372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 0x7f0718155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227523290 0 0) 0x7f07180683d0 con 0x7f07181544d0 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07040033f0 con 0x7f07181544d0 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053320067 0 0) 0x7f0718067850 con 0x7f0718064e20 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07180683d0 con 0x7f0718064e20 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3095625598 0 0) 0x7f07040033f0 con 0x7f07181544d0 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f071d394700 1 -- 172.21.15.73:0/473079372 >> v1:172.21.15.150:6789/0 conn(0x7f0718064e20 legacy=0x7f0718095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f071d394700 1 -- 172.21.15.73:0/473079372 >> v1:172.21.15.73:6789/0 conn(0x7f07181544d0 legacy=0x7f0718066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f071d394700 1 -- 172.21.15.73:0/473079372 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718067b30 con 0x7f0718096590 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f071d394700 1 --2- 172.21.15.73:0/473079372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 0x7f0718155750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0710009790 tx=0x7f0710009060).ready entity=mon.2 client_cookie=7fb3b1725dc0add5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0708001970 con 0x7f0718096590 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0710011400 con 0x7f0718096590 2023-11-28T03:08:14.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.789+0000 7f07177fe700 1 -- 172.21.15.73:0/473079372 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0710011780 con 0x7f0718096590 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 -- 172.21.15.73:0/473079372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 msgr2=0x7f0718155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 --2- 172.21.15.73:0/473079372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 0x7f0718155750 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 rx=0x7f0710009790 tx=0x7f0710009060).stop 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 -- 172.21.15.73:0/473079372 shutdown_connections 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 --2- 172.21.15.73:0/473079372 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718096590 0x7f0718155750 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 -- 172.21.15.73:0/473079372 >> 172.21.15.73:0/473079372 conn(0x7f07180631d0 msgr2=0x7f07180c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 -- 172.21.15.73:0/473079372 shutdown_connections 2023-11-28T03:08:14.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.790+0000 7f071edf7700 1 -- 172.21.15.73:0/473079372 wait complete. 2023-11-28T03:08:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.791+0000 7f071edf7700 1 Processor -- start 2023-11-28T03:08:14.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.791+0000 7f071edf7700 1 -- start start 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718064e20 0x7f0718159520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07181544d0 0x7f0718166b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07180c7060 con 0x7f0718096590 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0718159b00 con 0x7f07181544d0 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071edf7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0718159de0 con 0x7f0718064e20 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54780/0 (socket says 172.21.15.73:54780) 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 -- 172.21.15.73:0/3753437982 learned_addr learned my addr 172.21.15.73:0/3753437982 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071cb93700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718064e20 0x7f0718159520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f071d394700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07181544d0 0x7f0718166b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 -- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718064e20 msgr2=0x7f0718159520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718064e20 0x7f0718159520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 -- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07181544d0 msgr2=0x7f0718166b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07181544d0 0x7f0718166b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 -- 172.21.15.73:0/3753437982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07100092a0 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.792+0000 7f0717fff700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f070000c4a0 tx=0x7f0700007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.793+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000f470 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.793+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f070000f5d0 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.793+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f070000f970 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.793+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f071815a0c0 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.793+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07180c8d90 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.794+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f071806c860 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.794+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f070000fd20 con 0x7f0718096590 2023-11-28T03:08:14.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.794+0000 7f0714ff9700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 0x7f06f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:14.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.795+0000 7f071cb93700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 0x7f06f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:14.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.795+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f07000793a0 con 0x7f0718096590 2023-11-28T03:08:14.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.795+0000 7f071cb93700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 0x7f06f8061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0708003260 tx=0x7f0708001a60).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:14.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.799+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f070004a9c0 con 0x7f0718096590 2023-11-28T03:08:14.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.962+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0718067e10 con 0x7f0718096590 2023-11-28T03:08:14.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.963+0000 7f0714ff9700 1 -- 172.21.15.73:0/3753437982 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+13715 (secure 0 0 0) 0x7f070004dfe0 con 0x7f0718096590 2023-11-28T03:08:14.964 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:14.964 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":106,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:14.278637+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":12,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":94,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:14.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 msgr2=0x7f06f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 0x7f06f8061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f0708003260 tx=0x7f0708001a60).stop 2023-11-28T03:08:14.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 msgr2=0x7f07180c88a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:14.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 rx=0x7f070000c4a0 tx=0x7f0700007c50).stop 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 shutdown_connections 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f06f805f150 0x7f06f8061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f07181544d0 0x7f0718166b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0718064e20 0x7f0718159520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 --2- 172.21.15.73:0/3753437982 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0718096590 0x7f07180c88a0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 >> 172.21.15.73:0/3753437982 conn(0x7f07180631d0 msgr2=0x7f07180c4bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 shutdown_connections 2023-11-28T03:08:14.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:14.965+0000 7f071edf7700 1 -- 172.21.15.73:0/3753437982 wait complete. 2023-11-28T03:08:14.978 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '101', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:08:14.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:08:15.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.093+0000 7f4a0619e700 1 Processor -- start 2023-11-28T03:08:15.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.094+0000 7f4a0619e700 1 -- start start 2023-11-28T03:08:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0619e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 0x7f4a00154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0619e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a00158570 0x7f4a00158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0619e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a00154e40 con 0x7f4a0014e530 2023-11-28T03:08:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0619e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000c6ba0 con 0x7f4a00158570 2023-11-28T03:08:15.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0619e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a000c7bc0 con 0x7f4a000c66f0 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a00158570 0x7f4a00158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4a00158570 0x7f4a00158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54782/0 (socket says 172.21.15.73:54782) 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0519c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4a0014e530 0x7f4a0014e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35472/0 (socket says 172.21.15.73:35472) 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 -- 172.21.15.73:0/2979479031 learned_addr learned my addr 172.21.15.73:0/2979479031 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0499b700 1 --2- 172.21.15.73:0/2979479031 >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 0x7f4a00154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 -- 172.21.15.73:0/2979479031 >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 msgr2=0x7f4a00154950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 --2- 172.21.15.73:0/2979479031 >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 0x7f4a00154950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 -- 172.21.15.73:0/2979479031 >> v1:172.21.15.73:6789/0 conn(0x7f4a0014e530 legacy=0x7f4a0014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f49f7fff700 1 -- 172.21.15.73:0/2979479031 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79599629 0 0) 0x7f49fc001110 con 0x7f4a0014e530 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 -- 172.21.15.73:0/2979479031 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a00155920 con 0x7f4a00158570 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0499b700 1 --2- 172.21.15.73:0/2979479031 >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 0x7f4a00154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:15.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f4a0599d700 1 --2- 172.21.15.73:0/2979479031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a00158570 0x7f4a00158a00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f4a000ab0a0 tx=0x7f49f8009840).ready entity=mon.0 client_cookie=80a3f70c96d31e9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f49f7fff700 1 -- 172.21.15.73:0/2979479031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800e070 con 0x7f4a00158570 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f49f7fff700 1 -- 172.21.15.73:0/2979479031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f800ba50 con 0x7f4a00158570 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.095+0000 7f49f7fff700 1 -- 172.21.15.73:0/2979479031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f800bdd0 con 0x7f4a00158570 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 -- 172.21.15.73:0/2979479031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a00158570 msgr2=0x7f4a00158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 --2- 172.21.15.73:0/2979479031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a00158570 0x7f4a00158a00 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 rx=0x7f4a000ab0a0 tx=0x7f49f8009840).stop 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 -- 172.21.15.73:0/2979479031 shutdown_connections 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 --2- 172.21.15.73:0/2979479031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a00158570 0x7f4a00158a00 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 --2- 172.21.15.73:0/2979479031 >> v2:172.21.15.150:3300/0 conn(0x7f4a000c66f0 0x7f4a00154950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 -- 172.21.15.73:0/2979479031 >> 172.21.15.73:0/2979479031 conn(0x7f4a00064990 msgr2=0x7f4a00064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 -- 172.21.15.73:0/2979479031 shutdown_connections 2023-11-28T03:08:15.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.096+0000 7f4a0619e700 1 -- 172.21.15.73:0/2979479031 wait complete. 2023-11-28T03:08:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.097+0000 7f4a0619e700 1 Processor -- start 2023-11-28T03:08:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.097+0000 7f4a0619e700 1 -- start start 2023-11-28T03:08:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 0x7f4a000c4af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a00158570 0x7f4a00157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c38f0 con 0x7f4a0014e530 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a000c4fe0 con 0x7f4a000c66f0 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0619e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a00157880 con 0x7f4a00158570 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54784/0 (socket says 172.21.15.73:54784) 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 -- 172.21.15.73:0/3034216768 learned_addr learned my addr 172.21.15.73:0/3034216768 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 -- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a00158570 msgr2=0x7f4a00157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0519c700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 0x7f4a000c4af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a00158570 0x7f4a00157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 -- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 msgr2=0x7f4a000c4af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 0x7f4a000c4af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0499b700 1 -- 172.21.15.73:0/3034216768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49f8009ae0 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.098+0000 7f4a0519c700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 0x7f4a000c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f4a0499b700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f49f000c880 tx=0x7f49f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0010580 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49f00106e0 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a000c3050 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49f0010a60 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.099+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a000c3610 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.100+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a00164a00 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.100+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f49f001b460 con 0x7f4a0014e530 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.100+0000 7f49f57fa700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 0x7f49e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.100+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f49f0079e40 con 0x7f4a0014e530 2023-11-28T03:08:15.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.100+0000 7f4a0519c700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 0x7f49e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.101+0000 7f4a0519c700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 0x7f49e4061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f49fc001740 tx=0x7f49fc009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.104+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f49f004b400 con 0x7f4a0014e530 2023-11-28T03:08:15.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.267+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4a00062ee0 con 0x7f4a0014e530 2023-11-28T03:08:15.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.267+0000 7f49f57fa700 1 -- 172.21.15.73:0/3034216768 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v106) v1 ==== 110+0+10 (secure 0 0 0) 0x7f49f004ea20 con 0x7f4a0014e530 2023-11-28T03:08:15.268 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 msgr2=0x7f49e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 0x7f49e4061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f49fc001740 tx=0x7f49fc009040).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 msgr2=0x7f4a00156ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 rx=0x7f49f000c880 tx=0x7f49f0008040).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 shutdown_connections 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f49e405f150 0x7f49e4061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4a00158570 0x7f4a00157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4a000c66f0 0x7f4a000c4af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 --2- 172.21.15.73:0/3034216768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4a0014e530 0x7f4a00156ea0 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 >> 172.21.15.73:0/3034216768 conn(0x7f4a00064990 msgr2=0x7f4a00153bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 shutdown_connections 2023-11-28T03:08:15.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.269+0000 7f4a0619e700 1 -- 172.21.15.73:0/3034216768 wait complete. 2023-11-28T03:08:15.282 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:08:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.397+0000 7fad80740700 1 Processor -- start 2023-11-28T03:08:15.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.397+0000 7fad80740700 1 -- start start 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fad78155830 0x7fad78155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78155030 0x7fad780c2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad780c2e70 0x7fad780c5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad781554b0 con 0x7fad780c2e70 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad780c3300 con 0x7fad78155830 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad80740700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad781529c0 con 0x7fad78155030 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad7ecdd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad780c2e70 0x7fad780c5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad7ecdd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fad780c2e70 0x7fad780c5ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54792/0 (socket says 172.21.15.73:54792) 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.398+0000 7fad7ecdd700 1 -- 172.21.15.73:0/2520799232 learned_addr learned my addr 172.21.15.73:0/2520799232 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7dcdb700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78155030 0x7fad780c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7e4dc700 1 --2- 172.21.15.73:0/2520799232 >> v2:172.21.15.150:3300/0 conn(0x7fad78155830 0x7fad78155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 -- 172.21.15.73:0/2520799232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78155030 msgr2=0x7fad780c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78155030 0x7fad780c2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 -- 172.21.15.73:0/2520799232 >> v2:172.21.15.150:3300/0 conn(0x7fad78155830 msgr2=0x7fad78155c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/2520799232 >> v2:172.21.15.150:3300/0 conn(0x7fad78155830 0x7fad78155c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 -- 172.21.15.73:0/2520799232 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad78151db0 con 0x7fad780c2e70 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad780c2e70 0x7fad780c5ec0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fad780bc3a0 tx=0x7fad700090f0).ready entity=mon.0 client_cookie=d716a397c2457f0e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7d4da700 1 -- 172.21.15.73:0/2520799232 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000b600 con 0x7fad780c2e70 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7d4da700 1 -- 172.21.15.73:0/2520799232 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000b760 con 0x7fad780c2e70 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad7d4da700 1 -- 172.21.15.73:0/2520799232 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000bb00 con 0x7fad780c2e70 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad80740700 1 -- 172.21.15.73:0/2520799232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad780c2e70 msgr2=0x7fad780c5ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.399+0000 7fad80740700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad780c2e70 0x7fad780c5ec0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 rx=0x7fad780bc3a0 tx=0x7fad700090f0).stop 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 -- 172.21.15.73:0/2520799232 shutdown_connections 2023-11-28T03:08:15.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad780c2e70 0x7fad780c5ec0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 --2- 172.21.15.73:0/2520799232 >> v2:172.21.15.150:3300/0 conn(0x7fad78155830 0x7fad78155c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 --2- 172.21.15.73:0/2520799232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad78155030 0x7fad780c2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 -- 172.21.15.73:0/2520799232 >> 172.21.15.73:0/2520799232 conn(0x7fad78062700 msgr2=0x7fad78062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 -- 172.21.15.73:0/2520799232 shutdown_connections 2023-11-28T03:08:15.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.400+0000 7fad80740700 1 -- 172.21.15.73:0/2520799232 wait complete. 2023-11-28T03:08:15.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 Processor -- start 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 -- start start 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad780c2e70 0x7fad7814bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad78155030 0x7fad780be230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad78152b70 con 0x7fad78155830 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad781533d0 con 0x7fad78155030 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.401+0000 7fad80740700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad780aac70 con 0x7fad780c2e70 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54798/0 (socket says 172.21.15.73:54798) 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 -- 172.21.15.73:0/753166283 learned_addr learned my addr 172.21.15.73:0/753166283 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7e4dc700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad780c2e70 0x7fad7814bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7dcdb700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad78155030 0x7fad780be230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 -- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad780c2e70 msgr2=0x7fad7814bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad780c2e70 0x7fad7814bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 -- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad78155030 msgr2=0x7fad780be230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad78155030 0x7fad780be230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 -- 172.21.15.73:0/753166283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad70009370 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad7ecdd700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fad70009330 tx=0x7fad70017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad7000b600 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad7000bec0 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad78152d80 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad70006d90 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.402+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad7814c150 con 0x7fad78155830 2023-11-28T03:08:15.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.403+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad78153560 con 0x7fad78155830 2023-11-28T03:08:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.403+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fad70021070 con 0x7fad78155830 2023-11-28T03:08:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.404+0000 7fad6e7fc700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 0x7fad5c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.404+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fad7007ba20 con 0x7fad78155830 2023-11-28T03:08:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.404+0000 7fad7e4dc700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 0x7fad5c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.404+0000 7fad7e4dc700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 0x7fad5c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fad740075a0 tx=0x7fad740097e0).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.408+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fad7004d040 con 0x7fad78155830 2023-11-28T03:08:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.574+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fad7814c340 con 0x7fad78155830 2023-11-28T03:08:15.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.574+0000 7fad6e7fc700 1 -- 172.21.15.73:0/753166283 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v106) v1 ==== 92+0+10 (secure 0 0 0) 0x7fad70029070 con 0x7fad78155830 2023-11-28T03:08:15.575 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:08:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 msgr2=0x7fad5c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 0x7fad5c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fad740075a0 tx=0x7fad740097e0).stop 2023-11-28T03:08:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 msgr2=0x7fad780be720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 rx=0x7fad70009330 tx=0x7fad70017000).stop 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 shutdown_connections 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fad5c05f150 0x7fad5c061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fad78155830 0x7fad780be720 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fad780c2e70 0x7fad7814bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 --2- 172.21.15.73:0/753166283 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fad78155030 0x7fad780be230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 >> 172.21.15.73:0/753166283 conn(0x7fad78062700 msgr2=0x7fad7814ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 shutdown_connections 2023-11-28T03:08:15.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.576+0000 7fad80740700 1 -- 172.21.15.73:0/753166283 wait complete. 2023-11-28T03:08:15.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:08:15.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.701+0000 7fc38efbf700 1 Processor -- start 2023-11-28T03:08:15.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.701+0000 7fc38efbf700 1 -- start start 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38efbf700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc388062700 0x7fc388062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38efbf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3880661b0 0x7fc388066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38efbf700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3880630a0 con 0x7fc3881542d0 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38efbf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc388065760 con 0x7fc3880661b0 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38efbf700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3880c3250 con 0x7fc388062700 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.702+0000 7fc38d55c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3880661b0 0x7fc388066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3880661b0 0x7fc388066640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54802/0 (socket says 172.21.15.73:54802) 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 -- 172.21.15.73:0/3070255794 learned_addr learned my addr 172.21.15.73:0/3070255794 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc37ffff700 1 --2- 172.21.15.73:0/3070255794 >> v2:172.21.15.150:3300/0 conn(0x7fc388062700 0x7fc388062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 -- 172.21.15.73:0/3070255794 >> v2:172.21.15.150:3300/0 conn(0x7fc388062700 msgr2=0x7fc388062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 --2- 172.21.15.73:0/3070255794 >> v2:172.21.15.150:3300/0 conn(0x7fc388062700 0x7fc388062bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 -- 172.21.15.73:0/3070255794 >> v1:172.21.15.150:6789/0 conn(0x7fc3881542d0 legacy=0x7fc3880697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 -- 172.21.15.73:0/3070255794 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc38806a340 con 0x7fc3880661b0 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc38d55c700 1 --2- 172.21.15.73:0/3070255794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3880661b0 0x7fc388066640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fc388069020 tx=0x7fc380009840).ready entity=mon.0 client_cookie=b1df29aa1cd6ccc8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc37f7fe700 1 -- 172.21.15.73:0/3070255794 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000e070 con 0x7fc3880661b0 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc37f7fe700 1 -- 172.21.15.73:0/3070255794 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38000ba10 con 0x7fc3880661b0 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.703+0000 7fc37f7fe700 1 -- 172.21.15.73:0/3070255794 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38000bd90 con 0x7fc3880661b0 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 -- 172.21.15.73:0/3070255794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3880661b0 msgr2=0x7fc388066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 --2- 172.21.15.73:0/3070255794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3880661b0 0x7fc388066640 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 rx=0x7fc388069020 tx=0x7fc380009840).stop 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 -- 172.21.15.73:0/3070255794 shutdown_connections 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 --2- 172.21.15.73:0/3070255794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3880661b0 0x7fc388066640 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 --2- 172.21.15.73:0/3070255794 >> v2:172.21.15.150:3300/0 conn(0x7fc388062700 0x7fc388062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 -- 172.21.15.73:0/3070255794 >> 172.21.15.73:0/3070255794 conn(0x7fc3880c7fd0 msgr2=0x7fc3880c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 -- 172.21.15.73:0/3070255794 shutdown_connections 2023-11-28T03:08:15.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.704+0000 7fc38efbf700 1 -- 172.21.15.73:0/3070255794 wait complete. 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.705+0000 7fc38efbf700 1 Processor -- start 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.705+0000 7fc38efbf700 1 -- start start 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 0x7fc388156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 0x7fc3880a9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880a9b40 con 0x7fc3881542d0 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880a8cd0 con 0x7fc3880661b0 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3880a83a0 con 0x7fc388062700 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54808/0 (socket says 172.21.15.73:54808) 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 -- 172.21.15.73:0/2961967714 learned_addr learned my addr 172.21.15.73:0/2961967714 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc37ffff700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 0x7fc3880a9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38cd5b700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 0x7fc388156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 -- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 msgr2=0x7fc388156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 0x7fc388156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 -- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 msgr2=0x7fc3880a9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 0x7fc3880a9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 -- 172.21.15.73:0/2961967714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc380009ae0 con 0x7fc3881542d0 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc37ffff700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 0x7fc3880a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38cd5b700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 0x7fc388156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38d55c700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fc380009df0 tx=0x7fc380009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38001d410 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc38001d570 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3880a8680 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc38001d8f0 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.706+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3880abc90 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.707+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3880a8e60 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.708+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc38001dd20 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.708+0000 7fc37cff9700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 0x7fc374061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.708+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc38007bb00 con 0x7fc3881542d0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.708+0000 7fc38cd5b700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 0x7fc374061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:15.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.708+0000 7fc38cd5b700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 0x7fc374061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc38400a800 tx=0x7fc384008040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:15.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.712+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc38004d120 con 0x7fc3881542d0 2023-11-28T03:08:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.878+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc3880630a0 con 0x7fc3881542d0 2023-11-28T03:08:15.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.878+0000 7fc37cff9700 1 -- 172.21.15.73:0/2961967714 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v106) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc3880630a0 con 0x7fc3881542d0 2023-11-28T03:08:15.879 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:15.879 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:15.880 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:08:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 msgr2=0x7fc374061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 0x7fc374061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc38400a800 tx=0x7fc384008040).stop 2023-11-28T03:08:15.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 msgr2=0x7fc3880a9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 rx=0x7fc380009df0 tx=0x7fc380009840).stop 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 shutdown_connections 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc37405f150 0x7fc374061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3881542d0 0x7fc3880a9650 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc388062700 0x7fc388156b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 --2- 172.21.15.73:0/2961967714 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3880661b0 0x7fc3880a9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 >> 172.21.15.73:0/2961967714 conn(0x7fc3880c7fd0 msgr2=0x7fc3880be6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 shutdown_connections 2023-11-28T03:08:15.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:15.880+0000 7fc38efbf700 1 -- 172.21.15.73:0/2961967714 wait complete. 2023-11-28T03:08:15.894 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:08:15.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:08:16.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.011+0000 7fa5b9cbc700 1 Processor -- start 2023-11-28T03:08:16.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.011+0000 7fa5b9cbc700 1 -- start start 2023-11-28T03:08:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b9cbc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5b4154910 0x7fa5b40a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b9cbc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b40c6630 con 0x7fa5b40aa220 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b9cbc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5b4064230 con 0x7fa5b4154530 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b9cbc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5b40c6390 con 0x7fa5b4154910 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5b4154910 0x7fa5b40a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa5b4154910 0x7fa5b40a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54820/0 (socket says 172.21.15.73:54820) 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/289061606 learned_addr learned my addr 172.21.15.73:0/289061606 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa5b40aa220 0x7fa5b40c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35488/0 (socket says 172.21.15.73:35488) 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/289061606 >> v1:172.21.15.150:6789/0 conn(0x7fa5b4154530 legacy=0x7fa5b40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/289061606 >> v1:172.21.15.73:6789/0 conn(0x7fa5b40aa220 legacy=0x7fa5b40c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b27fc700 1 -- 172.21.15.73:0/289061606 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505536164 0 0) 0x7fa5b40c6630 con 0x7fa5b40aa220 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/289061606 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5b4064510 con 0x7fa5b4154910 2023-11-28T03:08:16.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.012+0000 7fa5b2ffd700 1 --2- 172.21.15.73:0/289061606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154910 0x7fa5b40a7ff0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fa5a8006fd0 tx=0x7fa5a80090f0).ready entity=mon.0 client_cookie=42fa67acff70ca4f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b27fc700 1 -- 172.21.15.73:0/289061606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800b600 con 0x7fa5b4154910 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b27fc700 1 -- 172.21.15.73:0/289061606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a800b760 con 0x7fa5b4154910 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b27fc700 1 -- 172.21.15.73:0/289061606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800bae0 con 0x7fa5b4154910 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/289061606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154910 msgr2=0x7fa5b40a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/289061606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154910 0x7fa5b40a7ff0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 rx=0x7fa5a8006fd0 tx=0x7fa5a80090f0).stop 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/289061606 shutdown_connections 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/289061606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154910 0x7fa5b40a7ff0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/289061606 >> 172.21.15.73:0/289061606 conn(0x7fa5b4094930 msgr2=0x7fa5b406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/289061606 shutdown_connections 2023-11-28T03:08:16.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.013+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/289061606 wait complete. 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.014+0000 7fa5b9cbc700 1 Processor -- start 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.014+0000 7fa5b9cbc700 1 -- start start 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b40aa220 0x7fa5b4164f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b4154910 0x7fa5b4168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41536b0 con 0x7fa5b4154530 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b414f8b0 con 0x7fa5b4154910 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b9cbc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5b41693c0 con 0x7fa5b40aa220 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54828/0 (socket says 172.21.15.73:54828) 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/815593962 learned_addr learned my addr 172.21.15.73:0/815593962 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b37fe700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b40aa220 0x7fa5b4164f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b3fff700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b4154910 0x7fa5b4168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b40aa220 msgr2=0x7fa5b4164f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b40aa220 0x7fa5b4164f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b4154910 msgr2=0x7fa5b4168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.015+0000 7fa5b2ffd700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b4154910 0x7fa5b4168ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa5b2ffd700 1 -- 172.21.15.73:0/815593962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a8009370 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa5b2ffd700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fa5b406ae10 tx=0x7fa5a8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a800bae0 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5a8000c90 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5b4165560 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5a801c670 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.016+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5b4165b20 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.017+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5b406c860 con 0x7fa5b4154530 2023-11-28T03:08:16.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.017+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa5a801c9e0 con 0x7fa5b4154530 2023-11-28T03:08:16.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.018+0000 7fa59b7fe700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 0x7fa5a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.018+0000 7fa5b37fe700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 0x7fa5a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.018+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa5a807bee0 con 0x7fa5b4154530 2023-11-28T03:08:16.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.018+0000 7fa5b37fe700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 0x7fa5a0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa5a4007fd0 tx=0x7fa5a4009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.022+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa5a804d500 con 0x7fa5b4154530 2023-11-28T03:08:16.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.186+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7fa5b4063050 con 0x7fa5b4154530 2023-11-28T03:08:16.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.301+0000 7fa59b7fe700 1 -- 172.21.15.73:0/815593962 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v107) v1 ==== 152+0+0 (secure 0 0 0) 0x7fa5a8020070 con 0x7fa5b4154530 2023-11-28T03:08:16.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 msgr2=0x7fa5a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 0x7fa5a0061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7fa5a4007fd0 tx=0x7fa5a4009040).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 msgr2=0x7fa5b41531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 rx=0x7fa5b406ae10 tx=0x7fa5a8017000).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 shutdown_connections 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa5a005f150 0x7fa5a0061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5b4154910 0x7fa5b4168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5b40aa220 0x7fa5b4164f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 --2- 172.21.15.73:0/815593962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5b4154530 0x7fa5b41531c0 secure :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 rx=0x7fa5b406ae10 tx=0x7fa5a8017000).stop 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 >> 172.21.15.73:0/815593962 conn(0x7fa5b4094930 msgr2=0x7fa5b40c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 shutdown_connections 2023-11-28T03:08:16.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.303+0000 7fa5b9cbc700 1 -- 172.21.15.73:0/815593962 wait complete. 2023-11-28T03:08:16.306 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:08:16.319 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:08:16.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.439+0000 7f7883fff700 1 Processor -- start 2023-11-28T03:08:16.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.439+0000 7f7883fff700 1 -- start start 2023-11-28T03:08:16.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.439+0000 7f7883fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f788414e530 0x7f78840c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.439+0000 7f7883fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78840c7bc0 con 0x7f7884158570 2023-11-28T03:08:16.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7883fff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7884155920 con 0x7f7884154950 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7883fff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7884154e00 con 0x7f788414e530 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f788414e530 0x7f78840c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f78837fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7884158570 0x7f78840aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35504/0 (socket says 172.21.15.73:35504) 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f788414e530 0x7f78840c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54844/0 (socket says 172.21.15.73:54844) 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f78837fe700 1 -- 172.21.15.73:0/575021471 learned_addr learned my addr 172.21.15.73:0/575021471 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 -- 172.21.15.73:0/575021471 >> v1:172.21.15.150:6789/0 conn(0x7f7884154950 legacy=0x7f788414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7881ffb700 1 -- 172.21.15.73:0/575021471 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2284857475 0 0) 0x7f78840c7bc0 con 0x7f7884158570 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 -- 172.21.15.73:0/575021471 >> v1:172.21.15.73:6789/0 conn(0x7f7884158570 legacy=0x7f78840aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 -- 172.21.15.73:0/575021471 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7884155080 con 0x7f788414e530 2023-11-28T03:08:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7882ffd700 1 --2- 172.21.15.73:0/575021471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788414e530 0x7f78840c66f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7874009760 tx=0x7f7874009060).ready entity=mon.0 client_cookie=b8291b6bc4ecfb50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7881ffb700 1 -- 172.21.15.73:0/575021471 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400b520 con 0x7f788414e530 2023-11-28T03:08:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7881ffb700 1 -- 172.21.15.73:0/575021471 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787400b680 con 0x7f788414e530 2023-11-28T03:08:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.440+0000 7f7881ffb700 1 -- 172.21.15.73:0/575021471 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787400ba00 con 0x7f788414e530 2023-11-28T03:08:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 -- 172.21.15.73:0/575021471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788414e530 msgr2=0x7f78840c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 --2- 172.21.15.73:0/575021471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788414e530 0x7f78840c66f0 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 rx=0x7f7874009760 tx=0x7f7874009060).stop 2023-11-28T03:08:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 -- 172.21.15.73:0/575021471 shutdown_connections 2023-11-28T03:08:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 --2- 172.21.15.73:0/575021471 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f788414e530 0x7f78840c66f0 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 -- 172.21.15.73:0/575021471 >> 172.21.15.73:0/575021471 conn(0x7f7884064990 msgr2=0x7f7884064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 -- 172.21.15.73:0/575021471 shutdown_connections 2023-11-28T03:08:16.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.441+0000 7f7883fff700 1 -- 172.21.15.73:0/575021471 wait complete. 2023-11-28T03:08:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.442+0000 7f7883fff700 1 Processor -- start 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.442+0000 7f7883fff700 1 -- start start 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 0x7f788414dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 0x7f78840c2d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788414e100 con 0x7f7884158570 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c3670 con 0x7f7884154950 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7883fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78840c3950 con 0x7f788414e530 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54858/0 (socket says 172.21.15.73:54858) 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 -- 172.21.15.73:0/2905129440 learned_addr learned my addr 172.21.15.73:0/2905129440 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7882ffd700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 0x7f788414dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78827fc700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 0x7f78840c2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 -- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 msgr2=0x7f788414dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 0x7f788414dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 -- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 msgr2=0x7f78840c2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 0x7f78840c2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 -- 172.21.15.73:0/2905129440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78740092a0 con 0x7f7884158570 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f7882ffd700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 0x7f788414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78827fc700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 0x7f78840c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f78837fe700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f787c00c730 tx=0x7f787c001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.443+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c005d30 con 0x7f7884158570 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.444+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f787c005e90 con 0x7f7884158570 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.444+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78840c3c30 con 0x7f7884158570 2023-11-28T03:08:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.444+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787c007340 con 0x7f7884158570 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.444+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78840c41f0 con 0x7f7884158570 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.445+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78840c7980 con 0x7f7884158570 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.445+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f787c0076f0 con 0x7f7884158570 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.445+0000 7f786affd700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 0x7f7870061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.445+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f787c079fb0 con 0x7f7884158570 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.445+0000 7f7882ffd700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 0x7f7870061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.446+0000 7f7882ffd700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 0x7f7870061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f787400b2e0 tx=0x7f7874017000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.450+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f787c04b460 con 0x7f7884158570 2023-11-28T03:08:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:16.614+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7884062ee0 con 0x7f7884158570 2023-11-28T03:08:17.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.307+0000 7f786affd700 1 -- 172.21.15.73:0/2905129440 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v108) v1 ==== 170+0+0 (secure 0 0 0) 0x7f787c0079a0 con 0x7f7884158570 2023-11-28T03:08:17.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 msgr2=0x7f7870061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 0x7f7870061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f787400b2e0 tx=0x7f7874017000).stop 2023-11-28T03:08:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 msgr2=0x7f78840c5120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 rx=0x7f787c00c730 tx=0x7f787c001520).stop 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 shutdown_connections 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f787005f150 0x7f7870061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7884158570 0x7f78840c5120 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f788414e530 0x7f788414dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 --2- 172.21.15.73:0/2905129440 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7884154950 0x7f78840c2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 >> 172.21.15.73:0/2905129440 conn(0x7f7884064990 msgr2=0x7f7884155eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.311+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 shutdown_connections 2023-11-28T03:08:17.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.312+0000 7f7883fff700 1 -- 172.21.15.73:0/2905129440 wait complete. 2023-11-28T03:08:17.319 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:08:17.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:08:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.452+0000 7fc8eb59e700 1 Processor -- start 2023-11-28T03:08:17.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.452+0000 7fc8eb59e700 1 -- start start 2023-11-28T03:08:17.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8eb59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec14e530 0x7fc8ec0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8eb59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8ec158570 0x7fc8ec158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8eb59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8ec0c7bc0 con 0x7fc8ec154950 2023-11-28T03:08:17.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8eb59e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8ec155920 con 0x7fc8ec158570 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8eb59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec154dd0 con 0x7fc8ec14e530 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8ec158570 0x7fc8ec158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8ec158570 0x7fc8ec158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54874/0 (socket says 172.21.15.73:54874) 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 -- 172.21.15.73:0/4220972023 learned_addr learned my addr 172.21.15.73:0/4220972023 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ea59c700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec14e530 0x7fc8ec0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 -- 172.21.15.73:0/4220972023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec14e530 msgr2=0x7fc8ec0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec14e530 0x7fc8ec0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 -- 172.21.15.73:0/4220972023 >> v1:172.21.15.150:6789/0 conn(0x7fc8ec154950 legacy=0x7fc8ec14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 -- 172.21.15.73:0/4220972023 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8ec155080 con 0x7fc8ec158570 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.453+0000 7fc8ead9d700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec158570 0x7fc8ec158a20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fc8e4009f20 tx=0x7fc8e4009840).ready entity=mon.0 client_cookie=6ef598774004b175 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8e9d9b700 1 -- 172.21.15.73:0/4220972023 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400e070 con 0x7fc8ec158570 2023-11-28T03:08:17.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8e9d9b700 1 -- 172.21.15.73:0/4220972023 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8e400ba80 con 0x7fc8ec158570 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8e9d9b700 1 -- 172.21.15.73:0/4220972023 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e400be00 con 0x7fc8ec158570 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 -- 172.21.15.73:0/4220972023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec158570 msgr2=0x7fc8ec158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec158570 0x7fc8ec158a20 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 rx=0x7fc8e4009f20 tx=0x7fc8e4009840).stop 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 -- 172.21.15.73:0/4220972023 shutdown_connections 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec158570 0x7fc8ec158a20 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/4220972023 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec14e530 0x7fc8ec0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 -- 172.21.15.73:0/4220972023 >> 172.21.15.73:0/4220972023 conn(0x7fc8ec064990 msgr2=0x7fc8ec064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 -- 172.21.15.73:0/4220972023 shutdown_connections 2023-11-28T03:08:17.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.454+0000 7fc8eb59e700 1 -- 172.21.15.73:0/4220972023 wait complete. 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.455+0000 7fc8eb59e700 1 Processor -- start 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.455+0000 7fc8eb59e700 1 -- start start 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec154950 0x7fc8ec0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 0x7fc8ec164420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0aff00 con 0x7fc8ec14e530 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0c5130 con 0x7fc8ec154950 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8eb59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8ec0c2ab0 con 0x7fc8ec158570 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8ea59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8ea59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54878/0 (socket says 172.21.15.73:54878) 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8ea59c700 1 -- 172.21.15.73:0/2990832508 learned_addr learned my addr 172.21.15.73:0/2990832508 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8ead9d700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 0x7fc8ec164420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.456+0000 7fc8e1d9b700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec154950 0x7fc8ec0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 -- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 msgr2=0x7fc8ec164420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 0x7fc8ec164420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 -- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec154950 msgr2=0x7fc8ec0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec154950 0x7fc8ec0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 -- 172.21.15.73:0/2990832508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e4009ae0 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ead9d700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 0x7fc8ec164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8ea59c700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fc8dc00d8c0 tx=0x7fc8dc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0095b0 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc009710 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8ec0c21c0 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc009a90 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.457+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8ec0c2740 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.458+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8ec0c2d90 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.458+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc8dc013480 con 0x7fc8ec14e530 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.459+0000 7fc8e2ffd700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 0x7fc8cc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.459+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fc8dc079fd0 con 0x7fc8ec14e530 2023-11-28T03:08:17.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.459+0000 7fc8e1d9b700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 0x7fc8cc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.459+0000 7fc8e1d9b700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 0x7fc8cc061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc8ec0c3cf0 tx=0x7fc8d4009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.463+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc8dc04b2a0 con 0x7fc8ec14e530 2023-11-28T03:08:17.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.667+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc8ec062ee0 con 0x7fc8ec14e530 2023-11-28T03:08:17.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.675+0000 7fc8e2ffd700 1 -- 172.21.15.73:0/2990832508 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 13 and data pool 14 v38) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc8dc04e8c0 con 0x7fc8ec14e530 2023-11-28T03:08:17.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.679+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 msgr2=0x7fc8cc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.679+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 0x7fc8cc061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fc8ec0c3cf0 tx=0x7fc8d4009040).stop 2023-11-28T03:08:17.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.679+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 msgr2=0x7fc8ec0afa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.679+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 rx=0x7fc8dc00d8c0 tx=0x7fc8dc008040).stop 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.683+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 shutdown_connections 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.683+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc8cc05f150 0x7fc8cc061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.683+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8ec158570 0x7fc8ec164420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.683+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8ec14e530 0x7fc8ec0afa10 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.683+0000 7fc8eb59e700 1 --2- 172.21.15.73:0/2990832508 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8ec154950 0x7fc8ec0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.684+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 >> 172.21.15.73:0/2990832508 conn(0x7fc8ec064990 msgr2=0x7fc8ec156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:17.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.684+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 shutdown_connections 2023-11-28T03:08:17.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.684+0000 7fc8eb59e700 1 -- 172.21.15.73:0/2990832508 wait complete. 2023-11-28T03:08:17.696 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 13 and data pool 14 2023-11-28T03:08:17.710 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.822+0000 7ff7effff700 1 Processor -- start 2023-11-28T03:08:17.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.822+0000 7ff7effff700 1 -- start start 2023-11-28T03:08:17.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7effff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7effff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f00c0180 0x7ff7f01589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7effff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7f00c12c0 con 0x7ff7f00c1750 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7effff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7f00c06b0 con 0x7ff7f00c0180 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7effff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c0990 con 0x7ff7f0157660 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7e67fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f00c0180 0x7ff7f01589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7eeffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7eeffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:40980/0 (socket says 172.21.15.73:40980) 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7e67fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff7f00c0180 0x7ff7f01589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:45934/0 (socket says 172.21.15.73:45934) 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7eeffd700 1 -- 172.21.15.73:0/919927003 learned_addr learned my addr 172.21.15.73:0/919927003 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.823+0000 7ff7ef7fe700 1 --1- 172.21.15.73:0/919927003 >> v1:172.21.15.150:6789/0 conn(0x7ff7f00c1750 0x7ff7f00648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:58824/0 (socket says 172.21.15.73:58824) 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7e67fc700 1 -- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 msgr2=0x7ff7f0157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:17.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7e67fc700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7e67fc700 1 -- 172.21.15.73:0/919927003 >> v1:172.21.15.150:6789/0 conn(0x7ff7f00c1750 legacy=0x7ff7f00648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7ee7fc700 1 -- 172.21.15.73:0/919927003 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 219106575 0 0) 0x7ff7f00c12c0 con 0x7ff7f00c1750 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7e67fc700 1 -- 172.21.15.73:0/919927003 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7f00c0c70 con 0x7ff7f00c0180 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7eeffd700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7e67fc700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00c0180 0x7ff7f01589c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7ff7d8009760 tx=0x7ff7d8009060).ready entity=mon.1 client_cookie=6c2c287c5d7d20b1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7ee7fc700 1 -- 172.21.15.73:0/919927003 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f510 con 0x7ff7f00c0180 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7ee7fc700 1 -- 172.21.15.73:0/919927003 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d800f670 con 0x7ff7f00c0180 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7ee7fc700 1 -- 172.21.15.73:0/919927003 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f9f0 con 0x7ff7f00c0180 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 -- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00c0180 msgr2=0x7ff7f01589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00c0180 0x7ff7f01589c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 rx=0x7ff7d8009760 tx=0x7ff7d8009060).stop 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 -- 172.21.15.73:0/919927003 shutdown_connections 2023-11-28T03:08:17.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f0157660 0x7ff7f0157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 --2- 172.21.15.73:0/919927003 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f00c0180 0x7ff7f01589c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 -- 172.21.15.73:0/919927003 >> 172.21.15.73:0/919927003 conn(0x7ff7f00cb000 msgr2=0x7ff7f00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:17.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 -- 172.21.15.73:0/919927003 shutdown_connections 2023-11-28T03:08:17.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.824+0000 7ff7effff700 1 -- 172.21.15.73:0/919927003 wait complete. 2023-11-28T03:08:17.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 Processor -- start 2023-11-28T03:08:17.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 -- start start 2023-11-28T03:08:17.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00c0180 0x7ff7f00c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0157660 0x7ff7f00ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c35d0 con 0x7ff7f00c1750 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c3870 con 0x7ff7f0157660 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.826+0000 7ff7effff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7f00c3b50 con 0x7ff7f00c0180 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54890/0 (socket says 172.21.15.73:54890) 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 -- 172.21.15.73:0/1497393964 learned_addr learned my addr 172.21.15.73:0/1497393964 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7eeffd700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00c0180 0x7ff7f00c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7ef7fe700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0157660 0x7ff7f00ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 -- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00c0180 msgr2=0x7ff7f00c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00c0180 0x7ff7f00c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 -- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0157660 msgr2=0x7ff7f00ae4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0157660 0x7ff7f00ae4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 -- 172.21.15.73:0/1497393964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e0009370 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e67fc700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7ff7d8011000 tx=0x7ff7d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d800f7e0 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d800f940 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d80092a0 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d801b660 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.827+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7f00c4110 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.828+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7f00c4300 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.829+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff7d801ba50 con 0x7ff7f00c1750 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.829+0000 7ff7e77fe700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 0x7ff7d0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:17.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.829+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff7d807bb90 con 0x7ff7f00c1750 2023-11-28T03:08:17.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.829+0000 7ff7eeffd700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 0x7ff7d0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:17.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.829+0000 7ff7eeffd700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 0x7ff7d0061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff7e0006fd0 tx=0x7ff7e00090f0).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:17.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.833+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff7d804ce60 con 0x7ff7f00c1750 2023-11-28T03:08:17.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.997+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7f006c7c0 con 0x7ff7f00c1750 2023-11-28T03:08:17.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:17.998+0000 7ff7e77fe700 1 -- 172.21.15.73:0/1497393964 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16956 (secure 0 0 0) 0x7ff7d8028070 con 0x7ff7f00c1750 2023-11-28T03:08:18.000 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:18.000 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:17.668653+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:18.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 msgr2=0x7ff7d0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 0x7ff7d0061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7ff7e0006fd0 tx=0x7ff7e00090f0).stop 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 msgr2=0x7ff7f00adfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 rx=0x7ff7d8011000 tx=0x7ff7d8017000).stop 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 shutdown_connections 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff7d005f150 0x7ff7d0061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff7f0157660 0x7ff7f00ae4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff7f00c0180 0x7ff7f00c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 --2- 172.21.15.73:0/1497393964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff7f00c1750 0x7ff7f00adfb0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 >> 172.21.15.73:0/1497393964 conn(0x7ff7f00cb000 msgr2=0x7ff7f00ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 shutdown_connections 2023-11-28T03:08:18.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.000+0000 7ff7effff700 1 -- 172.21.15.73:0/1497393964 wait complete. 2023-11-28T03:08:18.014 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:18.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.126+0000 7f7f59036700 1 Processor -- start 2023-11-28T03:08:18.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.126+0000 7f7f59036700 1 -- start start 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f59036700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7f54157660 0x7f7f54157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f59036700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540c12c0 con 0x7f7f540c0180 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f59036700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f540c06b0 con 0x7f7f540c1750 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f59036700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f540c0990 con 0x7f7f54157660 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f537fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7f540c0180 0x7f7f541589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35512/0 (socket says 172.21.15.73:35512) 2023-11-28T03:08:18.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f537fe700 1 -- 172.21.15.73:0/3797242589 learned_addr learned my addr 172.21.15.73:0/3797242589 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f53fff700 1 --2- 172.21.15.73:0/3797242589 >> v2:172.21.15.150:3300/0 conn(0x7f7f54157660 0x7f7f54157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1678124666 0 0) 0x7f7f540c12c0 con 0x7f7f540c0180 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f3c0033f0 con 0x7f7f540c0180 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f53fff700 1 -- 172.21.15.73:0/3797242589 >> v1:172.21.15.150:6789/0 conn(0x7f7f540c1750 legacy=0x7f7f540648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3870303811 0 0) 0x7f7f540c06b0 con 0x7f7f540c1750 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f53fff700 1 -- 172.21.15.73:0/3797242589 >> v1:172.21.15.73:6789/0 conn(0x7f7f540c0180 legacy=0x7f7f541589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.127+0000 7f7f53fff700 1 -- 172.21.15.73:0/3797242589 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f540c0c70 con 0x7f7f54157660 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f53fff700 1 --2- 172.21.15.73:0/3797242589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54157660 0x7f7f54157a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7f44009760 tx=0x7f7f44009060).ready entity=mon.1 client_cookie=8bf5e92bcb69042e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400f510 con 0x7f7f54157660 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4400f670 con 0x7f7f54157660 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f52ffd700 1 -- 172.21.15.73:0/3797242589 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4400f9f0 con 0x7f7f54157660 2023-11-28T03:08:18.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 -- 172.21.15.73:0/3797242589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54157660 msgr2=0x7f7f54157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 --2- 172.21.15.73:0/3797242589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54157660 0x7f7f54157a40 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 rx=0x7f7f44009760 tx=0x7f7f44009060).stop 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 -- 172.21.15.73:0/3797242589 shutdown_connections 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 --2- 172.21.15.73:0/3797242589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f54157660 0x7f7f54157a40 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 -- 172.21.15.73:0/3797242589 >> 172.21.15.73:0/3797242589 conn(0x7f7f540cb000 msgr2=0x7f7f540cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 -- 172.21.15.73:0/3797242589 shutdown_connections 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.128+0000 7f7f59036700 1 -- 172.21.15.73:0/3797242589 wait complete. 2023-11-28T03:08:18.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.129+0000 7f7f59036700 1 Processor -- start 2023-11-28T03:08:18.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 -- start start 2023-11-28T03:08:18.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f540c0180 0x7f7f540b05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f540c1750 0x7f7f540ac490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540c4930 con 0x7f7f54157660 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540af810 con 0x7f7f540c1750 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f59036700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f540afaf0 con 0x7f7f540c0180 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f58835700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f58835700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54892/0 (socket says 172.21.15.73:54892) 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f58835700 1 -- 172.21.15.73:0/1389918350 learned_addr learned my addr 172.21.15.73:0/1389918350 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f53fff700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f540c0180 0x7f7f540b05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.130+0000 7f7f537fe700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f540c1750 0x7f7f540ac490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 -- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f540c0180 msgr2=0x7f7f540b05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f540c0180 0x7f7f540b05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 -- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f540c1750 msgr2=0x7f7f540ac490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f540c1750 0x7f7f540ac490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 -- 172.21.15.73:0/1389918350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f440092a0 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f58835700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f7f4c00c7c0 tx=0x7f7f4c007ee0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00e4a0 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f4c00e620 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f540afdd0 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f4c00e9e0 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.131+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f540b0ac0 con 0x7f7f54157660 2023-11-28T03:08:18.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.132+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f540b02f0 con 0x7f7f54157660 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.132+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7f4c00ed50 con 0x7f7f54157660 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.133+0000 7f7f3bfff700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 0x7f7f40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.133+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7f4c079b30 con 0x7f7f54157660 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.133+0000 7f7f53fff700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 0x7f7f40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.133+0000 7f7f53fff700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 0x7f7f40061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7f44009760 tx=0x7f7f44017000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.137+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7f4c047af0 con 0x7f7f54157660 2023-11-28T03:08:18.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.301+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7f5406c7c0 con 0x7f7f54157660 2023-11-28T03:08:18.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.302+0000 7f7f3bfff700 1 -- 172.21.15.73:0/1389918350 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v109) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f7f4c04ae00 con 0x7f7f54157660 2023-11-28T03:08:18.303 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:18.304 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":109,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:17.668653+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 msgr2=0x7f7f40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 0x7f7f40061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f7f44009760 tx=0x7f7f44017000).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 msgr2=0x7f7f540c3ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 rx=0x7f7f4c00c7c0 tx=0x7f7f4c007ee0).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 shutdown_connections 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f4005f150 0x7f7f40061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7f54157660 0x7f7f540c3ed0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7f540c0180 0x7f7f540b05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 --2- 172.21.15.73:0/1389918350 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7f540c1750 0x7f7f540ac490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 >> 172.21.15.73:0/1389918350 conn(0x7f7f540cb000 msgr2=0x7f7f540aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 shutdown_connections 2023-11-28T03:08:18.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.304+0000 7f7f59036700 1 -- 172.21.15.73:0/1389918350 wait complete. 2023-11-28T03:08:18.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:08:18.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.433+0000 7f6a460fa700 1 Processor -- start 2023-11-28T03:08:18.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.433+0000 7f6a460fa700 1 -- start start 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a460fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6a40098d10 0x7f6a400c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a460fa700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6a40158320 0x7f6a401587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a460fa700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a40158cc0 con 0x7f6a400c6190 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a460fa700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a400c72d0 con 0x7f6a40098d10 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a460fa700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a40064570 con 0x7f6a40158320 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a3f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6a40098d10 0x7f6a400c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a3f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6a40098d10 0x7f6a400c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54896/0 (socket says 172.21.15.73:54896) 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a3f7fe700 1 -- 172.21.15.73:0/877807059 learned_addr learned my addr 172.21.15.73:0/877807059 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a3effd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6a400c6190 0x7f6a40063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35528/0 (socket says 172.21.15.73:35528) 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.434+0000 7f6a3ffff700 1 --2- 172.21.15.73:0/877807059 >> v2:172.21.15.150:3300/0 conn(0x7f6a40158320 0x7f6a401587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3f7fe700 1 -- 172.21.15.73:0/877807059 >> v2:172.21.15.150:3300/0 conn(0x7f6a40158320 msgr2=0x7f6a401587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3f7fe700 1 --2- 172.21.15.73:0/877807059 >> v2:172.21.15.150:3300/0 conn(0x7f6a40158320 0x7f6a401587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3f7fe700 1 -- 172.21.15.73:0/877807059 >> v1:172.21.15.73:6789/0 conn(0x7f6a400c6190 legacy=0x7f6a40063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3f7fe700 1 -- 172.21.15.73:0/877807059 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a40154bc0 con 0x7f6a40098d10 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3e7fc700 1 -- 172.21.15.73:0/877807059 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3473309809 0 0) 0x7f6a34001110 con 0x7f6a400c6190 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3f7fe700 1 --2- 172.21.15.73:0/877807059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40098d10 0x7f6a400c7760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f6a400657a0 tx=0x7f6a30009840).ready entity=mon.0 client_cookie=d7a379937c7b0cd1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3e7fc700 1 -- 172.21.15.73:0/877807059 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000e070 con 0x7f6a40098d10 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3e7fc700 1 -- 172.21.15.73:0/877807059 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a3000ba20 con 0x7f6a40098d10 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a3e7fc700 1 -- 172.21.15.73:0/877807059 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a3000bda0 con 0x7f6a40098d10 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 -- 172.21.15.73:0/877807059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40098d10 msgr2=0x7f6a400c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 --2- 172.21.15.73:0/877807059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40098d10 0x7f6a400c7760 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 rx=0x7f6a400657a0 tx=0x7f6a30009840).stop 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 -- 172.21.15.73:0/877807059 shutdown_connections 2023-11-28T03:08:18.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 --2- 172.21.15.73:0/877807059 >> v2:172.21.15.150:3300/0 conn(0x7f6a40158320 0x7f6a401587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 --2- 172.21.15.73:0/877807059 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40098d10 0x7f6a400c7760 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 -- 172.21.15.73:0/877807059 >> 172.21.15.73:0/877807059 conn(0x7f6a400cb310 msgr2=0x7f6a400cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 -- 172.21.15.73:0/877807059 shutdown_connections 2023-11-28T03:08:18.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.435+0000 7f6a460fa700 1 -- 172.21.15.73:0/877807059 wait complete. 2023-11-28T03:08:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 Processor -- start 2023-11-28T03:08:18.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 -- start start 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a400c6190 0x7f6a400ab220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a40158320 0x7f6a40154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a400ab710 con 0x7f6a40143f90 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a40155240 con 0x7f6a400c6190 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.437+0000 7f6a460fa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a401563b0 con 0x7f6a40158320 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54902/0 (socket says 172.21.15.73:54902) 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 -- 172.21.15.73:0/230076269 learned_addr learned my addr 172.21.15.73:0/230076269 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3f7fe700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a400c6190 0x7f6a400ab220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3effd700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a40158320 0x7f6a40154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 -- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a40158320 msgr2=0x7f6a40154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a40158320 0x7f6a40154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 -- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a400c6190 msgr2=0x7f6a400ab220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a400c6190 0x7f6a400ab220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 -- 172.21.15.73:0/230076269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a30009ae0 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a3ffff700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f6a3800e930 tx=0x7f6a38009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38012600 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a38012760 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a40155450 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a38012ae0 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.438+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a40155a10 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.439+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a4006c860 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.440+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6a3801b400 con 0x7f6a40143f90 2023-11-28T03:08:18.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.440+0000 7f6a277fe700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 0x7f6a2c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.440+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(109..109 src has 1..109) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6a3807a380 con 0x7f6a40143f90 2023-11-28T03:08:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.440+0000 7f6a3f7fe700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 0x7f6a2c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.440+0000 7f6a3f7fe700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 0x7f6a2c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6a400c7d10 tx=0x7f6a400a8040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.444+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6a3804b5f0 con 0x7f6a40143f90 2023-11-28T03:08:18.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.650+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6a401564f0 con 0x7f6a40143f90 2023-11-28T03:08:18.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.674+0000 7f6a277fe700 1 -- 172.21.15.73:0/230076269 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v39) v1 ==== 118+0+0 (secure 0 0 0) 0x7f6a38016020 con 0x7f6a40143f90 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 msgr2=0x7f6a2c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 0x7f6a2c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f6a400c7d10 tx=0x7f6a400a8040).stop 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 msgr2=0x7f6a400a87b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 rx=0x7f6a3800e930 tx=0x7f6a38009060).stop 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 shutdown_connections 2023-11-28T03:08:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6a2c05f150 0x7f6a2c061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6a40143f90 0x7f6a400a87b0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6a400c6190 0x7f6a400ab220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 --2- 172.21.15.73:0/230076269 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6a40158320 0x7f6a40154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 >> 172.21.15.73:0/230076269 conn(0x7f6a400cb310 msgr2=0x7f6a40153360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 shutdown_connections 2023-11-28T03:08:18.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.676+0000 7f6a460fa700 1 -- 172.21.15.73:0/230076269 wait complete. 2023-11-28T03:08:18.740 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:08:18.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:08:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.856+0000 7ff93e424700 1 Processor -- start 2023-11-28T03:08:18.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.856+0000 7ff93e424700 1 -- start start 2023-11-28T03:08:18.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93e424700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff9380c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93e424700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9380c7bc0 con 0x7ff938154950 2023-11-28T03:08:18.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93e424700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff938155920 con 0x7ff938158570 2023-11-28T03:08:18.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93e424700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff938154dd0 con 0x7ff93814e530 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93cc21700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff938154950 0x7ff93814e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35540/0 (socket says 172.21.15.73:35540) 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93cc21700 1 -- 172.21.15.73:0/3500637160 learned_addr learned my addr 172.21.15.73:0/3500637160 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93d422700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff9380c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883447613 0 0) 0x7ff9380c7bc0 con 0x7ff938154950 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9240033f0 con 0x7ff938154950 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 904826307 0 0) 0x7ff938155920 con 0x7ff938158570 2023-11-28T03:08:18.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9380c7bc0 con 0x7ff938158570 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1350099421 0 0) 0x7ff9240033f0 con 0x7ff938154950 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 msgr2=0x7ff9380c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff9380c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 >> v1:172.21.15.150:6789/0 conn(0x7ff938158570 legacy=0x7ff9380aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938155080 con 0x7ff938154950 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff93d422700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff9380c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff928001470 con 0x7ff938154950 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.857+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 >> v1:172.21.15.73:6789/0 conn(0x7ff938154950 legacy=0x7ff93814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff924003aa0 0x7ff924003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff924004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff924005020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff938154dd0 con 0x7ff924004690 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff92400b990 con 0x7ff924003aa0 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff92400bb20 con 0x7ff93814e530 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff924004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93d422700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff924003aa0 0x7ff924003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93dc23700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff924005020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 -- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 msgr2=0x7ff924005020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814e530 0x7ff924005020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 -- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff924003aa0 msgr2=0x7ff924003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff924003aa0 0x7ff924003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 -- 172.21.15.73:0/3500637160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff930009a20 con 0x7ff924004690 2023-11-28T03:08:18.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff924004b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7ff92800c190 tx=0x7ff928001e10).ready entity=mon.0 client_cookie=80451bfbf14e1850 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff92800c9c0 con 0x7ff924004690 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff92800cb20 con 0x7ff924004690 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.858+0000 7ff92ffff700 1 -- 172.21.15.73:0/3500637160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9280070d0 con 0x7ff924004690 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 -- 172.21.15.73:0/3500637160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 msgr2=0x7ff924004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 --2- 172.21.15.73:0/3500637160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff924004b10 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 rx=0x7ff92800c190 tx=0x7ff928001e10).stop 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93dc23700 1 -- 172.21.15.73:0/3500637160 reap_dead start 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 -- 172.21.15.73:0/3500637160 shutdown_connections 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 -- 172.21.15.73:0/3500637160 >> 172.21.15.73:0/3500637160 conn(0x7ff938064990 msgr2=0x7ff938064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 -- 172.21.15.73:0/3500637160 shutdown_connections 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.859+0000 7ff93e424700 1 -- 172.21.15.73:0/3500637160 wait complete. 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.860+0000 7ff93e424700 1 Processor -- start 2023-11-28T03:08:18.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.860+0000 7ff93e424700 1 -- start start 2023-11-28T03:08:18.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 0x7ff9380c4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814caa0 0x7ff9380c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380c3760 con 0x7ff924004690 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380c3f90 con 0x7ff938153fc0 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9380c4910 con 0x7ff93814caa0 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54912/0 (socket says 172.21.15.73:54912) 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 -- 172.21.15.73:0/3941977364 learned_addr learned my addr 172.21.15.73:0/3941977364 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 0x7ff9380c4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93dc23700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814caa0 0x7ff9380c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 -- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814caa0 msgr2=0x7ff9380c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814caa0 0x7ff9380c2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 -- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 msgr2=0x7ff9380c4420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:18.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 0x7ff9380c4420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 -- 172.21.15.73:0/3941977364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff938165060 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 0x7ff9380c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93d422700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7ff93000f2e0 tx=0x7ff93000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93000e040 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93000f040 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9380c3940 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff930015660 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.861+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff938164ce0 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.862+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9380c1e90 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.863+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff930015a10 con 0x7ff924004690 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.863+0000 7ff92d7fa700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 0x7ff91c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:18.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.863+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff93007b170 con 0x7ff924004690 2023-11-28T03:08:18.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.863+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 0x7ff91c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:18.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.863+0000 7ff93cc21700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 0x7ff91c061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff928001e70 tx=0x7ff928015000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:18.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:18.867+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff93004c3e0 con 0x7ff924004690 2023-11-28T03:08:19.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.077+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff938062ee0 con 0x7ff924004690 2023-11-28T03:08:19.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.681+0000 7ff92d7fa700 1 -- 172.21.15.73:0/3941977364 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff93004fa00 con 0x7ff924004690 2023-11-28T03:08:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 msgr2=0x7ff91c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 0x7ff91c061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7ff928001e70 tx=0x7ff928015000).stop 2023-11-28T03:08:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 msgr2=0x7ff93814d310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 rx=0x7ff93000f2e0 tx=0x7ff93000b040).stop 2023-11-28T03:08:19.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 shutdown_connections 2023-11-28T03:08:19.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff91c05f150 0x7ff91c061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff93814caa0 0x7ff9380c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff924004690 0x7ff93814d310 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 --2- 172.21.15.73:0/3941977364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff938153fc0 0x7ff9380c4420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 >> 172.21.15.73:0/3941977364 conn(0x7ff938064990 msgr2=0x7ff9380ca440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:19.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 shutdown_connections 2023-11-28T03:08:19.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.683+0000 7ff93e424700 1 -- 172.21.15.73:0/3941977364 wait complete. 2023-11-28T03:08:19.698 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:08:19.698 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:19.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.814+0000 7f76cc635700 1 Processor -- start 2023-11-28T03:08:19.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.814+0000 7f76cc635700 1 -- start start 2023-11-28T03:08:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76cc635700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76c41548c0 0x7f76c406a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76cc635700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 0x7f76c40c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76cc635700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c40c69a0 con 0x7f76c41544e0 2023-11-28T03:08:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76cc635700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c40c13a0 con 0x7f76c41548c0 2023-11-28T03:08:19.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76cc635700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40c1680 con 0x7f76c40c6e30 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76c9bd0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76c41548c0 0x7f76c406a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76c9bd0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76c41548c0 0x7f76c406a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54926/0 (socket says 172.21.15.73:54926) 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76c9bd0700 1 -- 172.21.15.73:0/3642349341 learned_addr learned my addr 172.21.15.73:0/3642349341 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.815+0000 7f76ca3d1700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f76c41544e0 0x7f76c40c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35544/0 (socket says 172.21.15.73:35544) 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c9bd0700 1 -- 172.21.15.73:0/3642349341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 msgr2=0x7f76c40c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cabd2700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 0x7f76c40c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c93cf700 1 -- 172.21.15.73:0/3642349341 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821779135 0 0) 0x7f76c40c69a0 con 0x7f76c41544e0 2023-11-28T03:08:19.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c9bd0700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 0x7f76c40c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c9bd0700 1 -- 172.21.15.73:0/3642349341 >> v1:172.21.15.73:6789/0 conn(0x7f76c41544e0 legacy=0x7f76c40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c9bd0700 1 -- 172.21.15.73:0/3642349341 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c40c6630 con 0x7f76c41548c0 2023-11-28T03:08:19.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cabd2700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 0x7f76c40c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c9bd0700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c406a610 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f76b4006fd0 tx=0x7f76b40090f0).ready entity=mon.0 client_cookie=36df853dbdcbf982 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c93cf700 1 -- 172.21.15.73:0/3642349341 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400b600 con 0x7f76c41548c0 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c93cf700 1 -- 172.21.15.73:0/3642349341 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76b400b760 con 0x7f76c41548c0 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76c93cf700 1 -- 172.21.15.73:0/3642349341 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76b400bae0 con 0x7f76c41548c0 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 -- 172.21.15.73:0/3642349341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 msgr2=0x7f76c406a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c406a610 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 rx=0x7f76b4006fd0 tx=0x7f76b40090f0).stop 2023-11-28T03:08:19.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 -- 172.21.15.73:0/3642349341 shutdown_connections 2023-11-28T03:08:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c40c6e30 0x7f76c40c72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 --2- 172.21.15.73:0/3642349341 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c406a610 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 -- 172.21.15.73:0/3642349341 >> 172.21.15.73:0/3642349341 conn(0x7f76c4085020 msgr2=0x7f76c4094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 -- 172.21.15.73:0/3642349341 shutdown_connections 2023-11-28T03:08:19.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.816+0000 7f76cc635700 1 -- 172.21.15.73:0/3642349341 wait complete. 2023-11-28T03:08:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 Processor -- start 2023-11-28T03:08:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 -- start start 2023-11-28T03:08:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c40c6e30 0x7f76c4168ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c41544e0 0x7f76c4152be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c40953a0 con 0x7f76c41548c0 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4165c40 con 0x7f76c40c6e30 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cc635700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4165960 con 0x7f76c41544e0 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cabd2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cabd2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54938/0 (socket says 172.21.15.73:54938) 2023-11-28T03:08:19.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76cabd2700 1 -- 172.21.15.73:0/1928443940 learned_addr learned my addr 172.21.15.73:0/1928443940 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.818+0000 7f76ca3d1700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c40c6e30 0x7f76c4168ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76c9bd0700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c41544e0 0x7f76c4152be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 -- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c41544e0 msgr2=0x7f76c4152be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c41544e0 0x7f76c4152be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 -- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c40c6e30 msgr2=0x7f76c4168ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c40c6e30 0x7f76c4168ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 -- 172.21.15.73:0/1928443940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b4009370 con 0x7f76c41548c0 2023-11-28T03:08:19.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cabd2700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f76bc00cef0 tx=0x7f76bc007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc00ad20 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc00ae80 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c41683f0 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc016610 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.819+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c41689b0 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.820+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c4168ba0 con 0x7f76c41548c0 2023-11-28T03:08:19.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.820+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f76bc01f3f0 con 0x7f76c41548c0 2023-11-28T03:08:19.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.821+0000 7f76ba7fc700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 0x7f76a8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:19.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.821+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f76bc07a860 con 0x7f76c41548c0 2023-11-28T03:08:19.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.821+0000 7f76ca3d1700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 0x7f76a8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:19.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.821+0000 7f76ca3d1700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 0x7f76a8061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f76c00027f0 tx=0x7f76c0009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:19.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:19.825+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76bc04bad0 con 0x7f76c41548c0 2023-11-28T03:08:19.958 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.a has been restored 2023-11-28T03:08:19.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2023-11-28T03:08:19.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-28T03:08:19.959 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-28T03:08:20.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.026+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f76c40c6250 con 0x7f76c41548c0 2023-11-28T03:08:20.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.027+0000 7f76ba7fc700 1 -- 172.21.15.73:0/1928443940 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f76bc04f0f0 con 0x7f76c41548c0 2023-11-28T03:08:20.028 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:20.028 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 msgr2=0x7f76a8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 0x7f76a8061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f76c00027f0 tx=0x7f76c0009040).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 msgr2=0x7f76c40960c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 rx=0x7f76bc00cef0 tx=0x7f76bc007860).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 shutdown_connections 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76a805f150 0x7f76a8061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76c41548c0 0x7f76c40960c0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76c40c6e30 0x7f76c4168ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 --2- 172.21.15.73:0/1928443940 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76c41544e0 0x7f76c4152be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 >> 172.21.15.73:0/1928443940 conn(0x7f76c4085020 msgr2=0x7f76c40c0bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.028+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 shutdown_connections 2023-11-28T03:08:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.029+0000 7f76cc635700 1 -- 172.21.15.73:0/1928443940 wait complete. 2023-11-28T03:08:20.031 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:08:20.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:08:20.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.154+0000 7fe6fcb3b700 1 Processor -- start 2023-11-28T03:08:20.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.154+0000 7fe6fcb3b700 1 -- start start 2023-11-28T03:08:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6fcb3b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe6f80c0180 0x7fe6f8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6fcb3b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f80c1750 0x7fe6f80c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6fcb3b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f80c12c0 con 0x7fe6f8157660 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6fcb3b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80c06b0 con 0x7fe6f80c1750 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6fcb3b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f80c0990 con 0x7fe6f80c0180 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f80c1750 0x7fe6f80c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe6f80c1750 0x7fe6f80c1b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54948/0 (socket says 172.21.15.73:54948) 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 -- 172.21.15.73:0/2115253981 learned_addr learned my addr 172.21.15.73:0/2115253981 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f77fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe6f8157660 0x7fe6f81589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35560/0 (socket says 172.21.15.73:35560) 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f6ffd700 1 --2- 172.21.15.73:0/2115253981 >> v2:172.21.15.150:3300/0 conn(0x7fe6f80c0180 0x7fe6f8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 -- 172.21.15.73:0/2115253981 >> v2:172.21.15.150:3300/0 conn(0x7fe6f80c0180 msgr2=0x7fe6f8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 --2- 172.21.15.73:0/2115253981 >> v2:172.21.15.150:3300/0 conn(0x7fe6f80c0180 0x7fe6f8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 -- 172.21.15.73:0/2115253981 >> v1:172.21.15.73:6789/0 conn(0x7fe6f8157660 legacy=0x7fe6f81589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f67fc700 1 -- 172.21.15.73:0/2115253981 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440914131 0 0) 0x7fe6f80c12c0 con 0x7fe6f8157660 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.155+0000 7fe6f7fff700 1 -- 172.21.15.73:0/2115253981 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f80c0c70 con 0x7fe6f80c1750 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6f7fff700 1 --2- 172.21.15.73:0/2115253981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f80c1750 0x7fe6f80c1b70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fe6f80c5820 tx=0x7fe6f00090f0).ready entity=mon.0 client_cookie=91d4b50700214a27 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6f67fc700 1 -- 172.21.15.73:0/2115253981 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000b600 con 0x7fe6f80c1750 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6f67fc700 1 -- 172.21.15.73:0/2115253981 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000b760 con 0x7fe6f80c1750 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6f67fc700 1 -- 172.21.15.73:0/2115253981 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000bae0 con 0x7fe6f80c1750 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/2115253981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f80c1750 msgr2=0x7fe6f80c1b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/2115253981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f80c1750 0x7fe6f80c1b70 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 rx=0x7fe6f80c5820 tx=0x7fe6f00090f0).stop 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/2115253981 shutdown_connections 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/2115253981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f80c1750 0x7fe6f80c1b70 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/2115253981 >> v2:172.21.15.150:3300/0 conn(0x7fe6f80c0180 0x7fe6f8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/2115253981 >> 172.21.15.73:0/2115253981 conn(0x7fe6f80cb000 msgr2=0x7fe6f80cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/2115253981 shutdown_connections 2023-11-28T03:08:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.156+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/2115253981 wait complete. 2023-11-28T03:08:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.157+0000 7fe6fcb3b700 1 Processor -- start 2023-11-28T03:08:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.157+0000 7fe6fcb3b700 1 -- start start 2023-11-28T03:08:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f80c0180 0x7fe6f80ae430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f80c1750 0x7fe6f80c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c4f20 con 0x7fe6f8157660 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c3ac0 con 0x7fe6f80c0180 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6fcb3b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f80c34e0 con 0x7fe6f80c1750 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6f7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54956/0 (socket says 172.21.15.73:54956) 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6f7fff700 1 -- 172.21.15.73:0/1178097596 learned_addr learned my addr 172.21.15.73:0/1178097596 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6f77fe700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f80c0180 0x7fe6f80ae430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.158+0000 7fe6f6ffd700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f80c1750 0x7fe6f80c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 -- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f80c1750 msgr2=0x7fe6f80c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f80c1750 0x7fe6f80c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 -- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f80c0180 msgr2=0x7fe6f80ae430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f80c0180 0x7fe6f80ae430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 -- 172.21.15.73:0/1178097596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f0009370 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6f7fff700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fe6f8084570 tx=0x7fe6f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f0006d90 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f000bea0 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f80aea60 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f000bae0 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.159+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f80c3750 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.160+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f80c5380 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.160+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe6f001c880 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.161+0000 7fe6df7fe700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 0x7fe6e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.161+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe6f007ba50 con 0x7fe6f8157660 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.161+0000 7fe6f77fe700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 0x7fe6e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.161+0000 7fe6f77fe700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 0x7fe6e4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe6e8007fd0 tx=0x7fe6e8001500).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.165+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe6f004cd20 con 0x7fe6f8157660 2023-11-28T03:08:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.327+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe6f80aef80 con 0x7fe6f8157660 2023-11-28T03:08:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.328+0000 7fe6df7fe700 1 -- 172.21.15.73:0/1178097596 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fe6f0020020 con 0x7fe6f8157660 2023-11-28T03:08:20.329 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:20.329 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":110,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:08:18.670789+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.329+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 msgr2=0x7fe6e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.329+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 0x7fe6e4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fe6e8007fd0 tx=0x7fe6e8001500).stop 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.329+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 msgr2=0x7fe6f80c3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.329+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 rx=0x7fe6f8084570 tx=0x7fe6f0017000).stop 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 shutdown_connections 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6e405f150 0x7fe6e4061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe6f8157660 0x7fe6f80c3f50 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe6f80c0180 0x7fe6f80ae430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 --2- 172.21.15.73:0/1178097596 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe6f80c1750 0x7fe6f80c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 >> 172.21.15.73:0/1178097596 conn(0x7fe6f80cb000 msgr2=0x7fe6f80c9e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 shutdown_connections 2023-11-28T03:08:20.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.330+0000 7fe6fcb3b700 1 -- 172.21.15.73:0/1178097596 wait complete. 2023-11-28T03:08:20.343 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:08:20.343 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.454+0000 7fa4ce610700 1 Processor -- start 2023-11-28T03:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.454+0000 7fa4ce610700 1 -- start start 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ce610700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 0x7fa4c80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ce610700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ce610700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4c80a9a90 con 0x7fa4c80cae90 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ce610700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4c8154ce0 con 0x7fa4c80ca180 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ce610700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80a8e80 con 0x7fa4c80caab0 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ccbad700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4c7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 0x7fa4c80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ccbad700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:46058/0 (socket says 172.21.15.73:46058) 2023-11-28T03:08:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4ccbad700 1 -- 172.21.15.73:0/2020172732 learned_addr learned my addr 172.21.15.73:0/2020172732 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.455+0000 7fa4c77fe700 1 --1- 172.21.15.73:0/2020172732 >> v1:172.21.15.150:6789/0 conn(0x7fa4c80cae90 0x7fa4c80c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:58842/0 (socket says 172.21.15.73:58842) 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c7fff700 1 -- 172.21.15.73:0/2020172732 >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 msgr2=0x7fa4c815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c7fff700 1 --2- 172.21.15.73:0/2020172732 >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c7fff700 1 -- 172.21.15.73:0/2020172732 >> v1:172.21.15.150:6789/0 conn(0x7fa4c80cae90 legacy=0x7fa4c80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c6ffd700 1 -- 172.21.15.73:0/2020172732 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2914466425 0 0) 0x7fa4c80a9a90 con 0x7fa4c80cae90 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c7fff700 1 -- 172.21.15.73:0/2020172732 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c80a9160 con 0x7fa4c80caab0 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4ccbad700 1 --2- 172.21.15.73:0/2020172732 >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c7fff700 1 --2- 172.21.15.73:0/2020172732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 0x7fa4c80aa470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fa4b8006fd0 tx=0x7fa4b8006f40).ready entity=mon.2 client_cookie=de6fe4960e548ba9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c6ffd700 1 -- 172.21.15.73:0/2020172732 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8005880 con 0x7fa4c80caab0 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c6ffd700 1 -- 172.21.15.73:0/2020172732 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4b80059e0 con 0x7fa4c80caab0 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.456+0000 7fa4c6ffd700 1 -- 172.21.15.73:0/2020172732 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4b8005d60 con 0x7fa4c80caab0 2023-11-28T03:08:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 -- 172.21.15.73:0/2020172732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 msgr2=0x7fa4c80aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 --2- 172.21.15.73:0/2020172732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 0x7fa4c80aa470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 rx=0x7fa4b8006fd0 tx=0x7fa4b8006f40).stop 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 -- 172.21.15.73:0/2020172732 shutdown_connections 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 --2- 172.21.15.73:0/2020172732 >> v2:172.21.15.150:3300/0 conn(0x7fa4c80ca180 0x7fa4c815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 --2- 172.21.15.73:0/2020172732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80caab0 0x7fa4c80aa470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 -- 172.21.15.73:0/2020172732 >> 172.21.15.73:0/2020172732 conn(0x7fa4c8094920 msgr2=0x7fa4c806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 -- 172.21.15.73:0/2020172732 shutdown_connections 2023-11-28T03:08:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.457+0000 7fa4ce610700 1 -- 172.21.15.73:0/2020172732 wait complete. 2023-11-28T03:08:20.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.458+0000 7fa4ce610700 1 Processor -- start 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.458+0000 7fa4ce610700 1 -- start start 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80ca180 0x7fa4c8153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4c80cae90 0x7fa4c8156870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c2de0 con 0x7fa4c80caab0 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c24b0 con 0x7fa4c80cae90 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c80c2790 con 0x7fa4c80ca180 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54970/0 (socket says 172.21.15.73:54970) 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 -- 172.21.15.73:0/3165403789 learned_addr learned my addr 172.21.15.73:0/3165403789 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c7fff700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80ca180 0x7fa4c8153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ccbad700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4c80cae90 0x7fa4c8156870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 -- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80ca180 msgr2=0x7fa4c8153cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80ca180 0x7fa4c8153cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 -- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4c80cae90 msgr2=0x7fa4c8156870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4c80cae90 0x7fa4c8156870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 -- 172.21.15.73:0/3165403789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c0009ae0 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4c77fe700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fa4bc00c490 tx=0x7fa4bc007da0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0070d0 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4bc007230 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4b8009250 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0056e0 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.459+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c81541c0 con 0x7fa4c80caab0 2023-11-28T03:08:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c81543e0 con 0x7fa4c80caab0 2023-11-28T03:08:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa4bc005ce0 con 0x7fa4c80caab0 2023-11-28T03:08:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4affff700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 0x7fa4b4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fa4bc079ae0 con 0x7fa4c80caab0 2023-11-28T03:08:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4c7fff700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 0x7fa4b4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.461+0000 7fa4c7fff700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 0x7fa4b4061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa4b8006cd0 tx=0x7fa4b8016000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.465+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa4bc04adb0 con 0x7fa4c80caab0 2023-11-28T03:08:20.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.628+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa4c806c860 con 0x7fa4b405f150 2023-11-28T03:08:20.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.638+0000 7fa4affff700 1 -- 172.21.15.73:0/3165403789 <== mgr.5837 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361843 (secure 0 0 0) 0x7fa4c806c860 con 0x7fa4b405f150 2023-11-28T03:08:20.644 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 msgr2=0x7fa4b4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 0x7fa4b4061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fa4b8006cd0 tx=0x7fa4b8016000).stop 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 msgr2=0x7fa4c8156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 rx=0x7fa4bc00c490 tx=0x7fa4bc007da0).stop 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 shutdown_connections 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa4b405f150 0x7fa4b4061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa4c80cae90 0x7fa4c8156870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa4c80ca180 0x7fa4c8153cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 --2- 172.21.15.73:0/3165403789 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa4c80caab0 0x7fa4c8156380 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 >> 172.21.15.73:0/3165403789 conn(0x7fa4c8094920 msgr2=0x7fa4c80637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 shutdown_connections 2023-11-28T03:08:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.642+0000 7fa4ce610700 1 -- 172.21.15.73:0/3165403789 wait complete. 2023-11-28T03:08:20.648 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:08:20.655 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T03:08:18.720550+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419884,"kb_used_data":10348,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752554836,"statfs":{"total":773094113280,"available":770616152064,"internally_reserved":0,"allocated":10596352,"data_stored":5249040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4466,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.589306"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696428+0000","last_change":"2023-11-28T03:08:17.696428+0000","last_active":"2023-11-28T03:08:17.696428+0000","last_peered":"2023-11-28T03:08:17.696428+0000","last_clean":"2023-11-28T03:08:17.696428+0000","last_became_active":"2023-11-28T03:08:17.691520+0000","last_became_peered":"2023-11-28T03:08:17.691520+0000","last_unstale":"2023-11-28T03:08:17.696428+0000","last_undegraded":"2023-11-28T03:08:17.696428+0000","last_fullsized":"2023-11-28T03:08:17.696428+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"109'3","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692670+0000","last_change":"2023-11-28T03:08:17.328984+0000","last_active":"2023-11-28T03:08:17.692670+0000","last_peered":"2023-11-28T03:08:17.692670+0000","last_clean":"2023-11-28T03:08:17.692670+0000","last_became_active":"2023-11-28T03:08:17.328636+0000","last_became_peered":"2023-11-28T03:08:17.328636+0000","last_unstale":"2023-11-28T03:08:17.692670+0000","last_undegraded":"2023-11-28T03:08:17.692670+0000","last_fullsized":"2023-11-28T03:08:17.692670+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688004+0000","last_change":"2023-11-28T03:08:17.688004+0000","last_active":"2023-11-28T03:08:17.688004+0000","last_peered":"2023-11-28T03:08:17.688004+0000","last_clean":"2023-11-28T03:08:17.688004+0000","last_became_active":"2023-11-28T03:08:17.687729+0000","last_became_peered":"2023-11-28T03:08:17.687729+0000","last_unstale":"2023-11-28T03:08:17.688004+0000","last_undegraded":"2023-11-28T03:08:17.688004+0000","last_fullsized":"2023-11-28T03:08:17.688004+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333227+0000","last_change":"2023-11-28T03:08:17.331233+0000","last_active":"2023-11-28T03:08:17.333227+0000","last_peered":"2023-11-28T03:08:17.333227+0000","last_clean":"2023-11-28T03:08:17.333227+0000","last_became_active":"2023-11-28T03:08:17.330982+0000","last_became_peered":"2023-11-28T03:08:17.330982+0000","last_unstale":"2023-11-28T03:08:17.333227+0000","last_undegraded":"2023-11-28T03:08:17.333227+0000","last_fullsized":"2023-11-28T03:08:17.333227+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.697365+0000","last_change":"2023-11-28T03:08:17.697365+0000","last_active":"2023-11-28T03:08:17.697365+0000","last_peered":"2023-11-28T03:08:17.697365+0000","last_clean":"2023-11-28T03:08:17.697365+0000","last_became_active":"2023-11-28T03:08:17.697044+0000","last_became_peered":"2023-11-28T03:08:17.697044+0000","last_unstale":"2023-11-28T03:08:17.697365+0000","last_undegraded":"2023-11-28T03:08:17.697365+0000","last_fullsized":"2023-11-28T03:08:17.697365+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331741+0000","last_change":"2023-11-28T03:08:17.329025+0000","last_active":"2023-11-28T03:08:17.331741+0000","last_peered":"2023-11-28T03:08:17.331741+0000","last_clean":"2023-11-28T03:08:17.331741+0000","last_became_active":"2023-11-28T03:08:17.328715+0000","last_became_peered":"2023-11-28T03:08:17.328715+0000","last_unstale":"2023-11-28T03:08:17.331741+0000","last_undegraded":"2023-11-28T03:08:17.331741+0000","last_fullsized":"2023-11-28T03:08:17.331741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691639+0000","last_change":"2023-11-28T03:08:17.691639+0000","last_active":"2023-11-28T03:08:17.691639+0000","last_peered":"2023-11-28T03:08:17.691639+0000","last_clean":"2023-11-28T03:08:17.691639+0000","last_became_active":"2023-11-28T03:08:17.691381+0000","last_became_peered":"2023-11-28T03:08:17.691381+0000","last_unstale":"2023-11-28T03:08:17.691639+0000","last_undegraded":"2023-11-28T03:08:17.691639+0000","last_fullsized":"2023-11-28T03:08:17.691639+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3c","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.685220+0000","last_change":"2023-11-28T03:08:17.330867+0000","last_active":"2023-11-28T03:08:17.685220+0000","last_peered":"2023-11-28T03:08:17.685220+0000","last_clean":"2023-11-28T03:08:17.685220+0000","last_became_active":"2023-11-28T03:08:17.329994+0000","last_became_peered":"2023-11-28T03:08:17.329994+0000","last_unstale":"2023-11-28T03:08:17.685220+0000","last_undegraded":"2023-11-28T03:08:17.685220+0000","last_fullsized":"2023-11-28T03:08:17.685220+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699910+0000","last_change":"2023-11-28T03:08:17.699910+0000","last_active":"2023-11-28T03:08:17.699910+0000","last_peered":"2023-11-28T03:08:17.699910+0000","last_clean":"2023-11-28T03:08:17.699910+0000","last_became_active":"2023-11-28T03:08:17.698276+0000","last_became_peered":"2023-11-28T03:08:17.698276+0000","last_unstale":"2023-11-28T03:08:17.699910+0000","last_undegraded":"2023-11-28T03:08:17.699910+0000","last_fullsized":"2023-11-28T03:08:17.699910+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.677193+0000","last_change":"2023-11-28T03:08:17.327535+0000","last_active":"2023-11-28T03:08:17.677193+0000","last_peered":"2023-11-28T03:08:17.677193+0000","last_clean":"2023-11-28T03:08:17.677193+0000","last_became_active":"2023-11-28T03:08:17.326680+0000","last_became_peered":"2023-11-28T03:08:17.326680+0000","last_unstale":"2023-11-28T03:08:17.677193+0000","last_undegraded":"2023-11-28T03:08:17.677193+0000","last_fullsized":"2023-11-28T03:08:17.677193+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689179+0000","last_change":"2023-11-28T03:08:17.689179+0000","last_active":"2023-11-28T03:08:17.689179+0000","last_peered":"2023-11-28T03:08:17.689179+0000","last_clean":"2023-11-28T03:08:17.689179+0000","last_became_active":"2023-11-28T03:08:17.688863+0000","last_became_peered":"2023-11-28T03:08:17.688863+0000","last_unstale":"2023-11-28T03:08:17.689179+0000","last_undegraded":"2023-11-28T03:08:17.689179+0000","last_fullsized":"2023-11-28T03:08:17.689179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3a","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.685686+0000","last_change":"2023-11-28T03:08:17.335278+0000","last_active":"2023-11-28T03:08:17.685686+0000","last_peered":"2023-11-28T03:08:17.685686+0000","last_clean":"2023-11-28T03:08:17.685686+0000","last_became_active":"2023-11-28T03:08:17.334949+0000","last_became_peered":"2023-11-28T03:08:17.334949+0000","last_unstale":"2023-11-28T03:08:17.685686+0000","last_undegraded":"2023-11-28T03:08:17.685686+0000","last_fullsized":"2023-11-28T03:08:17.685686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699242+0000","last_change":"2023-11-28T03:08:17.699242+0000","last_active":"2023-11-28T03:08:17.699242+0000","last_peered":"2023-11-28T03:08:17.699242+0000","last_clean":"2023-11-28T03:08:17.699242+0000","last_became_active":"2023-11-28T03:08:17.698989+0000","last_became_peered":"2023-11-28T03:08:17.698989+0000","last_unstale":"2023-11-28T03:08:17.699242+0000","last_undegraded":"2023-11-28T03:08:17.699242+0000","last_fullsized":"2023-11-28T03:08:17.699242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334507+0000","last_change":"2023-11-28T03:08:17.326672+0000","last_active":"2023-11-28T03:08:17.334507+0000","last_peered":"2023-11-28T03:08:17.334507+0000","last_clean":"2023-11-28T03:08:17.334507+0000","last_became_active":"2023-11-28T03:08:17.326378+0000","last_became_peered":"2023-11-28T03:08:17.326378+0000","last_unstale":"2023-11-28T03:08:17.334507+0000","last_undegraded":"2023-11-28T03:08:17.334507+0000","last_fullsized":"2023-11-28T03:08:17.334507+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.682586+0000","last_change":"2023-11-28T03:08:17.682586+0000","last_active":"2023-11-28T03:08:17.682586+0000","last_peered":"2023-11-28T03:08:17.682586+0000","last_clean":"2023-11-28T03:08:17.682586+0000","last_became_active":"2023-11-28T03:08:17.682180+0000","last_became_peered":"2023-11-28T03:08:17.682180+0000","last_unstale":"2023-11-28T03:08:17.682586+0000","last_undegraded":"2023-11-28T03:08:17.682586+0000","last_fullsized":"2023-11-28T03:08:17.682586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333692+0000","last_change":"2023-11-28T03:08:17.330662+0000","last_active":"2023-11-28T03:08:17.333692+0000","last_peered":"2023-11-28T03:08:17.333692+0000","last_clean":"2023-11-28T03:08:17.333692+0000","last_became_active":"2023-11-28T03:08:17.330127+0000","last_became_peered":"2023-11-28T03:08:17.330127+0000","last_unstale":"2023-11-28T03:08:17.333692+0000","last_undegraded":"2023-11-28T03:08:17.333692+0000","last_fullsized":"2023-11-28T03:08:17.333692+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.677889+0000","last_change":"2023-11-28T03:08:17.677889+0000","last_active":"2023-11-28T03:08:17.677889+0000","last_peered":"2023-11-28T03:08:17.677889+0000","last_clean":"2023-11-28T03:08:17.677889+0000","last_became_active":"2023-11-28T03:08:17.677603+0000","last_became_peered":"2023-11-28T03:08:17.677603+0000","last_unstale":"2023-11-28T03:08:17.677889+0000","last_undegraded":"2023-11-28T03:08:17.677889+0000","last_fullsized":"2023-11-28T03:08:17.677889+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331539+0000","last_change":"2023-11-28T03:08:17.326579+0000","last_active":"2023-11-28T03:08:17.331539+0000","last_peered":"2023-11-28T03:08:17.331539+0000","last_clean":"2023-11-28T03:08:17.331539+0000","last_became_active":"2023-11-28T03:08:17.326259+0000","last_became_peered":"2023-11-28T03:08:17.326259+0000","last_unstale":"2023-11-28T03:08:17.331539+0000","last_undegraded":"2023-11-28T03:08:17.331539+0000","last_fullsized":"2023-11-28T03:08:17.331539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691613+0000","last_change":"2023-11-28T03:08:17.691613+0000","last_active":"2023-11-28T03:08:17.691613+0000","last_peered":"2023-11-28T03:08:17.691613+0000","last_clean":"2023-11-28T03:08:17.691613+0000","last_became_active":"2023-11-28T03:08:17.691254+0000","last_became_peered":"2023-11-28T03:08:17.691254+0000","last_unstale":"2023-11-28T03:08:17.691613+0000","last_undegraded":"2023-11-28T03:08:17.691613+0000","last_fullsized":"2023-11-28T03:08:17.691613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.329137+0000","last_change":"2023-11-28T03:08:17.324430+0000","last_active":"2023-11-28T03:08:17.329137+0000","last_peered":"2023-11-28T03:08:17.329137+0000","last_clean":"2023-11-28T03:08:17.329137+0000","last_became_active":"2023-11-28T03:08:17.324194+0000","last_became_peered":"2023-11-28T03:08:17.324194+0000","last_unstale":"2023-11-28T03:08:17.329137+0000","last_undegraded":"2023-11-28T03:08:17.329137+0000","last_fullsized":"2023-11-28T03:08:17.329137+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699389+0000","last_change":"2023-11-28T03:08:17.699389+0000","last_active":"2023-11-28T03:08:17.699389+0000","last_peered":"2023-11-28T03:08:17.699389+0000","last_clean":"2023-11-28T03:08:17.699389+0000","last_became_active":"2023-11-28T03:08:17.699132+0000","last_became_peered":"2023-11-28T03:08:17.699132+0000","last_unstale":"2023-11-28T03:08:17.699389+0000","last_undegraded":"2023-11-28T03:08:17.699389+0000","last_fullsized":"2023-11-28T03:08:17.699389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332895+0000","last_change":"2023-11-28T03:08:17.329005+0000","last_active":"2023-11-28T03:08:17.332895+0000","last_peered":"2023-11-28T03:08:17.332895+0000","last_clean":"2023-11-28T03:08:17.332895+0000","last_became_active":"2023-11-28T03:08:17.328739+0000","last_became_peered":"2023-11-28T03:08:17.328739+0000","last_unstale":"2023-11-28T03:08:17.332895+0000","last_undegraded":"2023-11-28T03:08:17.332895+0000","last_fullsized":"2023-11-28T03:08:17.332895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692140+0000","last_change":"2023-11-28T03:08:17.692140+0000","last_active":"2023-11-28T03:08:17.692140+0000","last_peered":"2023-11-28T03:08:17.692140+0000","last_clean":"2023-11-28T03:08:17.692140+0000","last_became_active":"2023-11-28T03:08:17.691779+0000","last_became_peered":"2023-11-28T03:08:17.691779+0000","last_unstale":"2023-11-28T03:08:17.692140+0000","last_undegraded":"2023-11-28T03:08:17.692140+0000","last_fullsized":"2023-11-28T03:08:17.692140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.34","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698325+0000","last_change":"2023-11-28T03:08:17.324426+0000","last_active":"2023-11-28T03:08:17.698325+0000","last_peered":"2023-11-28T03:08:17.698325+0000","last_clean":"2023-11-28T03:08:17.698325+0000","last_became_active":"2023-11-28T03:08:17.324079+0000","last_became_peered":"2023-11-28T03:08:17.324079+0000","last_unstale":"2023-11-28T03:08:17.698325+0000","last_undegraded":"2023-11-28T03:08:17.698325+0000","last_fullsized":"2023-11-28T03:08:17.698325+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689350+0000","last_change":"2023-11-28T03:08:17.689350+0000","last_active":"2023-11-28T03:08:17.689350+0000","last_peered":"2023-11-28T03:08:17.689350+0000","last_clean":"2023-11-28T03:08:17.689350+0000","last_became_active":"2023-11-28T03:08:17.689067+0000","last_became_peered":"2023-11-28T03:08:17.689067+0000","last_unstale":"2023-11-28T03:08:17.689350+0000","last_undegraded":"2023-11-28T03:08:17.689350+0000","last_fullsized":"2023-11-28T03:08:17.689350+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.680660+0000","last_change":"2023-11-28T03:08:17.327879+0000","last_active":"2023-11-28T03:08:17.680660+0000","last_peered":"2023-11-28T03:08:17.680660+0000","last_clean":"2023-11-28T03:08:17.680660+0000","last_became_active":"2023-11-28T03:08:17.327219+0000","last_became_peered":"2023-11-28T03:08:17.327219+0000","last_unstale":"2023-11-28T03:08:17.680660+0000","last_undegraded":"2023-11-28T03:08:17.680660+0000","last_fullsized":"2023-11-28T03:08:17.680660+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693594+0000","last_change":"2023-11-28T03:08:17.693594+0000","last_active":"2023-11-28T03:08:17.693594+0000","last_peered":"2023-11-28T03:08:17.693594+0000","last_clean":"2023-11-28T03:08:17.693594+0000","last_became_active":"2023-11-28T03:08:17.691337+0000","last_became_peered":"2023-11-28T03:08:17.691337+0000","last_unstale":"2023-11-28T03:08:17.693594+0000","last_undegraded":"2023-11-28T03:08:17.693594+0000","last_fullsized":"2023-11-28T03:08:17.693594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334600+0000","last_change":"2023-11-28T03:08:17.329820+0000","last_active":"2023-11-28T03:08:17.334600+0000","last_peered":"2023-11-28T03:08:17.334600+0000","last_clean":"2023-11-28T03:08:17.334600+0000","last_became_active":"2023-11-28T03:08:17.329538+0000","last_became_peered":"2023-11-28T03:08:17.329538+0000","last_unstale":"2023-11-28T03:08:17.334600+0000","last_undegraded":"2023-11-28T03:08:17.334600+0000","last_fullsized":"2023-11-28T03:08:17.334600+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700412+0000","last_change":"2023-11-28T03:08:17.700412+0000","last_active":"2023-11-28T03:08:17.700412+0000","last_peered":"2023-11-28T03:08:17.700412+0000","last_clean":"2023-11-28T03:08:17.700412+0000","last_became_active":"2023-11-28T03:08:17.700054+0000","last_became_peered":"2023-11-28T03:08:17.700054+0000","last_unstale":"2023-11-28T03:08:17.700412+0000","last_undegraded":"2023-11-28T03:08:17.700412+0000","last_fullsized":"2023-11-28T03:08:17.700412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.327834+0000","last_change":"2023-11-28T03:08:17.326770+0000","last_active":"2023-11-28T03:08:17.327834+0000","last_peered":"2023-11-28T03:08:17.327834+0000","last_clean":"2023-11-28T03:08:17.327834+0000","last_became_active":"2023-11-28T03:08:17.326521+0000","last_became_peered":"2023-11-28T03:08:17.326521+0000","last_unstale":"2023-11-28T03:08:17.327834+0000","last_undegraded":"2023-11-28T03:08:17.327834+0000","last_fullsized":"2023-11-28T03:08:17.327834+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698686+0000","last_change":"2023-11-28T03:08:17.698686+0000","last_active":"2023-11-28T03:08:17.698686+0000","last_peered":"2023-11-28T03:08:17.698686+0000","last_clean":"2023-11-28T03:08:17.698686+0000","last_became_active":"2023-11-28T03:08:17.698339+0000","last_became_peered":"2023-11-28T03:08:17.698339+0000","last_unstale":"2023-11-28T03:08:17.698686+0000","last_undegraded":"2023-11-28T03:08:17.698686+0000","last_fullsized":"2023-11-28T03:08:17.698686+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688996+0000","last_change":"2023-11-28T03:08:17.330988+0000","last_active":"2023-11-28T03:08:17.688996+0000","last_peered":"2023-11-28T03:08:17.688996+0000","last_clean":"2023-11-28T03:08:17.688996+0000","last_became_active":"2023-11-28T03:08:17.330687+0000","last_became_peered":"2023-11-28T03:08:17.330687+0000","last_unstale":"2023-11-28T03:08:17.688996+0000","last_undegraded":"2023-11-28T03:08:17.688996+0000","last_fullsized":"2023-11-28T03:08:17.688996+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696541+0000","last_change":"2023-11-28T03:08:17.696541+0000","last_active":"2023-11-28T03:08:17.696541+0000","last_peered":"2023-11-28T03:08:17.696541+0000","last_clean":"2023-11-28T03:08:17.696541+0000","last_became_active":"2023-11-28T03:08:17.696208+0000","last_became_peered":"2023-11-28T03:08:17.696208+0000","last_unstale":"2023-11-28T03:08:17.696541+0000","last_undegraded":"2023-11-28T03:08:17.696541+0000","last_fullsized":"2023-11-28T03:08:17.696541+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.325946+0000","last_change":"2023-11-28T03:08:17.320522+0000","last_active":"2023-11-28T03:08:17.325946+0000","last_peered":"2023-11-28T03:08:17.325946+0000","last_clean":"2023-11-28T03:08:17.325946+0000","last_became_active":"2023-11-28T03:08:17.320226+0000","last_became_peered":"2023-11-28T03:08:17.320226+0000","last_unstale":"2023-11-28T03:08:17.325946+0000","last_undegraded":"2023-11-28T03:08:17.325946+0000","last_fullsized":"2023-11-28T03:08:17.325946+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692518+0000","last_change":"2023-11-28T03:08:17.692518+0000","last_active":"2023-11-28T03:08:17.692518+0000","last_peered":"2023-11-28T03:08:17.692518+0000","last_clean":"2023-11-28T03:08:17.692518+0000","last_became_active":"2023-11-28T03:08:17.692234+0000","last_became_peered":"2023-11-28T03:08:17.692234+0000","last_unstale":"2023-11-28T03:08:17.692518+0000","last_undegraded":"2023-11-28T03:08:17.692518+0000","last_fullsized":"2023-11-28T03:08:17.692518+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334457+0000","last_change":"2023-11-28T03:08:17.320821+0000","last_active":"2023-11-28T03:08:17.334457+0000","last_peered":"2023-11-28T03:08:17.334457+0000","last_clean":"2023-11-28T03:08:17.334457+0000","last_became_active":"2023-11-28T03:08:17.320551+0000","last_became_peered":"2023-11-28T03:08:17.320551+0000","last_unstale":"2023-11-28T03:08:17.334457+0000","last_undegraded":"2023-11-28T03:08:17.334457+0000","last_fullsized":"2023-11-28T03:08:17.334457+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692045+0000","last_change":"2023-11-28T03:08:17.692045+0000","last_active":"2023-11-28T03:08:17.692045+0000","last_peered":"2023-11-28T03:08:17.692045+0000","last_clean":"2023-11-28T03:08:17.692045+0000","last_became_active":"2023-11-28T03:08:17.691771+0000","last_became_peered":"2023-11-28T03:08:17.691771+0000","last_unstale":"2023-11-28T03:08:17.692045+0000","last_undegraded":"2023-11-28T03:08:17.692045+0000","last_fullsized":"2023-11-28T03:08:17.692045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.687002+0000","last_change":"2023-11-28T03:08:17.325949+0000","last_active":"2023-11-28T03:08:17.687002+0000","last_peered":"2023-11-28T03:08:17.687002+0000","last_clean":"2023-11-28T03:08:17.687002+0000","last_became_active":"2023-11-28T03:08:17.325673+0000","last_became_peered":"2023-11-28T03:08:17.325673+0000","last_unstale":"2023-11-28T03:08:17.687002+0000","last_undegraded":"2023-11-28T03:08:17.687002+0000","last_fullsized":"2023-11-28T03:08:17.687002+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689904+0000","last_change":"2023-11-28T03:08:17.689904+0000","last_active":"2023-11-28T03:08:17.689904+0000","last_peered":"2023-11-28T03:08:17.689904+0000","last_clean":"2023-11-28T03:08:17.689904+0000","last_became_active":"2023-11-28T03:08:17.689592+0000","last_became_peered":"2023-11-28T03:08:17.689592+0000","last_unstale":"2023-11-28T03:08:17.689904+0000","last_undegraded":"2023-11-28T03:08:17.689904+0000","last_fullsized":"2023-11-28T03:08:17.689904+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.330963+0000","last_change":"2023-11-28T03:08:17.318976+0000","last_active":"2023-11-28T03:08:17.330963+0000","last_peered":"2023-11-28T03:08:17.330963+0000","last_clean":"2023-11-28T03:08:17.330963+0000","last_became_active":"2023-11-28T03:08:17.318717+0000","last_became_peered":"2023-11-28T03:08:17.318717+0000","last_unstale":"2023-11-28T03:08:17.330963+0000","last_undegraded":"2023-11-28T03:08:17.330963+0000","last_fullsized":"2023-11-28T03:08:17.330963+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688044+0000","last_change":"2023-11-28T03:08:17.688044+0000","last_active":"2023-11-28T03:08:17.688044+0000","last_peered":"2023-11-28T03:08:17.688044+0000","last_clean":"2023-11-28T03:08:17.688044+0000","last_became_active":"2023-11-28T03:08:17.687797+0000","last_became_peered":"2023-11-28T03:08:17.687797+0000","last_unstale":"2023-11-28T03:08:17.688044+0000","last_undegraded":"2023-11-28T03:08:17.688044+0000","last_fullsized":"2023-11-28T03:08:17.688044+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.672731+0000","last_change":"2023-11-28T03:08:17.317207+0000","last_active":"2023-11-28T03:08:17.672731+0000","last_peered":"2023-11-28T03:08:17.672731+0000","last_clean":"2023-11-28T03:08:17.672731+0000","last_became_active":"2023-11-28T03:08:17.316926+0000","last_became_peered":"2023-11-28T03:08:17.316926+0000","last_unstale":"2023-11-28T03:08:17.672731+0000","last_undegraded":"2023-11-28T03:08:17.672731+0000","last_fullsized":"2023-11-28T03:08:17.672731+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696078+0000","last_change":"2023-11-28T03:08:17.696078+0000","last_active":"2023-11-28T03:08:17.696078+0000","last_peered":"2023-11-28T03:08:17.696078+0000","last_clean":"2023-11-28T03:08:17.696078+0000","last_became_active":"2023-11-28T03:08:17.695791+0000","last_became_peered":"2023-11-28T03:08:17.695791+0000","last_unstale":"2023-11-28T03:08:17.696078+0000","last_undegraded":"2023-11-28T03:08:17.696078+0000","last_fullsized":"2023-11-28T03:08:17.696078+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332843+0000","last_change":"2023-11-28T03:08:17.327494+0000","last_active":"2023-11-28T03:08:17.332843+0000","last_peered":"2023-11-28T03:08:17.332843+0000","last_clean":"2023-11-28T03:08:17.332843+0000","last_became_active":"2023-11-28T03:08:17.325301+0000","last_became_peered":"2023-11-28T03:08:17.325301+0000","last_unstale":"2023-11-28T03:08:17.332843+0000","last_undegraded":"2023-11-28T03:08:17.332843+0000","last_fullsized":"2023-11-28T03:08:17.332843+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699821+0000","last_change":"2023-11-28T03:08:17.699821+0000","last_active":"2023-11-28T03:08:17.699821+0000","last_peered":"2023-11-28T03:08:17.699821+0000","last_clean":"2023-11-28T03:08:17.699821+0000","last_became_active":"2023-11-28T03:08:17.699518+0000","last_became_peered":"2023-11-28T03:08:17.699518+0000","last_unstale":"2023-11-28T03:08:17.699821+0000","last_undegraded":"2023-11-28T03:08:17.699821+0000","last_fullsized":"2023-11-28T03:08:17.699821+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334202+0000","last_change":"2023-11-28T03:08:17.329714+0000","last_active":"2023-11-28T03:08:17.334202+0000","last_peered":"2023-11-28T03:08:17.334202+0000","last_clean":"2023-11-28T03:08:17.334202+0000","last_became_active":"2023-11-28T03:08:17.328666+0000","last_became_peered":"2023-11-28T03:08:17.328666+0000","last_unstale":"2023-11-28T03:08:17.334202+0000","last_undegraded":"2023-11-28T03:08:17.334202+0000","last_fullsized":"2023-11-28T03:08:17.334202+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691371+0000","last_change":"2023-11-28T03:08:17.691371+0000","last_active":"2023-11-28T03:08:17.691371+0000","last_peered":"2023-11-28T03:08:17.691371+0000","last_clean":"2023-11-28T03:08:17.691371+0000","last_became_active":"2023-11-28T03:08:17.690837+0000","last_became_peered":"2023-11-28T03:08:17.690837+0000","last_unstale":"2023-11-28T03:08:17.691371+0000","last_undegraded":"2023-11-28T03:08:17.691371+0000","last_fullsized":"2023-11-28T03:08:17.691371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.319610+0000","last_change":"2023-11-28T03:08:17.317280+0000","last_active":"2023-11-28T03:08:17.319610+0000","last_peered":"2023-11-28T03:08:17.319610+0000","last_clean":"2023-11-28T03:08:17.319610+0000","last_became_active":"2023-11-28T03:08:17.317000+0000","last_became_peered":"2023-11-28T03:08:17.317000+0000","last_unstale":"2023-11-28T03:08:17.319610+0000","last_undegraded":"2023-11-28T03:08:17.319610+0000","last_fullsized":"2023-11-28T03:08:17.319610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696491+0000","last_change":"2023-11-28T03:08:17.696491+0000","last_active":"2023-11-28T03:08:17.696491+0000","last_peered":"2023-11-28T03:08:17.696491+0000","last_clean":"2023-11-28T03:08:17.696491+0000","last_became_active":"2023-11-28T03:08:17.696176+0000","last_became_peered":"2023-11-28T03:08:17.696176+0000","last_unstale":"2023-11-28T03:08:17.696491+0000","last_undegraded":"2023-11-28T03:08:17.696491+0000","last_fullsized":"2023-11-28T03:08:17.696491+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332950+0000","last_change":"2023-11-28T03:08:17.329586+0000","last_active":"2023-11-28T03:08:17.332950+0000","last_peered":"2023-11-28T03:08:17.332950+0000","last_clean":"2023-11-28T03:08:17.332950+0000","last_became_active":"2023-11-28T03:08:17.329312+0000","last_became_peered":"2023-11-28T03:08:17.329312+0000","last_unstale":"2023-11-28T03:08:17.332950+0000","last_undegraded":"2023-11-28T03:08:17.332950+0000","last_fullsized":"2023-11-28T03:08:17.332950+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.678354+0000","last_change":"2023-11-28T03:08:17.678354+0000","last_active":"2023-11-28T03:08:17.678354+0000","last_peered":"2023-11-28T03:08:17.678354+0000","last_clean":"2023-11-28T03:08:17.678354+0000","last_became_active":"2023-11-28T03:08:17.678105+0000","last_became_peered":"2023-11-28T03:08:17.678105+0000","last_unstale":"2023-11-28T03:08:17.678354+0000","last_undegraded":"2023-11-28T03:08:17.678354+0000","last_fullsized":"2023-11-28T03:08:17.678354+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331726+0000","last_change":"2023-11-28T03:08:17.329723+0000","last_active":"2023-11-28T03:08:17.331726+0000","last_peered":"2023-11-28T03:08:17.331726+0000","last_clean":"2023-11-28T03:08:17.331726+0000","last_became_active":"2023-11-28T03:08:17.329435+0000","last_became_peered":"2023-11-28T03:08:17.329435+0000","last_unstale":"2023-11-28T03:08:17.331726+0000","last_undegraded":"2023-11-28T03:08:17.331726+0000","last_fullsized":"2023-11-28T03:08:17.331726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699179+0000","last_change":"2023-11-28T03:08:17.699179+0000","last_active":"2023-11-28T03:08:17.699179+0000","last_peered":"2023-11-28T03:08:17.699179+0000","last_clean":"2023-11-28T03:08:17.699179+0000","last_became_active":"2023-11-28T03:08:17.698858+0000","last_became_peered":"2023-11-28T03:08:17.698858+0000","last_unstale":"2023-11-28T03:08:17.699179+0000","last_undegraded":"2023-11-28T03:08:17.699179+0000","last_fullsized":"2023-11-28T03:08:17.699179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326280+0000","last_change":"2023-11-28T03:08:17.322898+0000","last_active":"2023-11-28T03:08:17.326280+0000","last_peered":"2023-11-28T03:08:17.326280+0000","last_clean":"2023-11-28T03:08:17.326280+0000","last_became_active":"2023-11-28T03:08:17.322234+0000","last_became_peered":"2023-11-28T03:08:17.322234+0000","last_unstale":"2023-11-28T03:08:17.326280+0000","last_undegraded":"2023-11-28T03:08:17.326280+0000","last_fullsized":"2023-11-28T03:08:17.326280+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694550+0000","last_change":"2023-11-28T03:08:17.694550+0000","last_active":"2023-11-28T03:08:17.694550+0000","last_peered":"2023-11-28T03:08:17.694550+0000","last_clean":"2023-11-28T03:08:17.694550+0000","last_became_active":"2023-11-28T03:08:17.694140+0000","last_became_peered":"2023-11-28T03:08:17.694140+0000","last_unstale":"2023-11-28T03:08:17.694550+0000","last_undegraded":"2023-11-28T03:08:17.694550+0000","last_fullsized":"2023-11-28T03:08:17.694550+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334187+0000","last_change":"2023-11-28T03:08:17.329333+0000","last_active":"2023-11-28T03:08:17.334187+0000","last_peered":"2023-11-28T03:08:17.334187+0000","last_clean":"2023-11-28T03:08:17.334187+0000","last_became_active":"2023-11-28T03:08:17.328737+0000","last_became_peered":"2023-11-28T03:08:17.328737+0000","last_unstale":"2023-11-28T03:08:17.334187+0000","last_undegraded":"2023-11-28T03:08:17.334187+0000","last_fullsized":"2023-11-28T03:08:17.334187+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.703616+0000","last_change":"2023-11-28T03:08:17.703616+0000","last_active":"2023-11-28T03:08:17.703616+0000","last_peered":"2023-11-28T03:08:17.703616+0000","last_clean":"2023-11-28T03:08:17.703616+0000","last_became_active":"2023-11-28T03:08:17.703339+0000","last_became_peered":"2023-11-28T03:08:17.703339+0000","last_unstale":"2023-11-28T03:08:17.703616+0000","last_undegraded":"2023-11-28T03:08:17.703616+0000","last_fullsized":"2023-11-28T03:08:17.703616+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332677+0000","last_change":"2023-11-28T03:08:17.326880+0000","last_active":"2023-11-28T03:08:17.332677+0000","last_peered":"2023-11-28T03:08:17.332677+0000","last_clean":"2023-11-28T03:08:17.332677+0000","last_became_active":"2023-11-28T03:08:17.326606+0000","last_became_peered":"2023-11-28T03:08:17.326606+0000","last_unstale":"2023-11-28T03:08:17.332677+0000","last_undegraded":"2023-11-28T03:08:17.332677+0000","last_fullsized":"2023-11-28T03:08:17.332677+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699944+0000","last_change":"2023-11-28T03:08:17.699944+0000","last_active":"2023-11-28T03:08:17.699944+0000","last_peered":"2023-11-28T03:08:17.699944+0000","last_clean":"2023-11-28T03:08:17.699944+0000","last_became_active":"2023-11-28T03:08:17.699649+0000","last_became_peered":"2023-11-28T03:08:17.699649+0000","last_unstale":"2023-11-28T03:08:17.699944+0000","last_undegraded":"2023-11-28T03:08:17.699944+0000","last_fullsized":"2023-11-28T03:08:17.699944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333321+0000","last_change":"2023-11-28T03:08:17.332384+0000","last_active":"2023-11-28T03:08:17.333321+0000","last_peered":"2023-11-28T03:08:17.333321+0000","last_clean":"2023-11-28T03:08:17.333321+0000","last_became_active":"2023-11-28T03:08:17.332169+0000","last_became_peered":"2023-11-28T03:08:17.332169+0000","last_unstale":"2023-11-28T03:08:17.333321+0000","last_undegraded":"2023-11-28T03:08:17.333321+0000","last_fullsized":"2023-11-28T03:08:17.333321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692147+0000","last_change":"2023-11-28T03:08:17.692147+0000","last_active":"2023-11-28T03:08:17.692147+0000","last_peered":"2023-11-28T03:08:17.692147+0000","last_clean":"2023-11-28T03:08:17.692147+0000","last_became_active":"2023-11-28T03:08:17.691847+0000","last_became_peered":"2023-11-28T03:08:17.691847+0000","last_unstale":"2023-11-28T03:08:17.692147+0000","last_undegraded":"2023-11-28T03:08:17.692147+0000","last_fullsized":"2023-11-28T03:08:17.692147+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334372+0000","last_change":"2023-11-28T03:08:17.327365+0000","last_active":"2023-11-28T03:08:17.334372+0000","last_peered":"2023-11-28T03:08:17.334372+0000","last_clean":"2023-11-28T03:08:17.334372+0000","last_became_active":"2023-11-28T03:08:17.327038+0000","last_became_peered":"2023-11-28T03:08:17.327038+0000","last_unstale":"2023-11-28T03:08:17.334372+0000","last_undegraded":"2023-11-28T03:08:17.334372+0000","last_fullsized":"2023-11-28T03:08:17.334372+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689215+0000","last_change":"2023-11-28T03:08:17.689215+0000","last_active":"2023-11-28T03:08:17.689215+0000","last_peered":"2023-11-28T03:08:17.689215+0000","last_clean":"2023-11-28T03:08:17.689215+0000","last_became_active":"2023-11-28T03:08:17.688901+0000","last_became_peered":"2023-11-28T03:08:17.688901+0000","last_unstale":"2023-11-28T03:08:17.689215+0000","last_undegraded":"2023-11-28T03:08:17.689215+0000","last_fullsized":"2023-11-28T03:08:17.689215+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.327827+0000","last_change":"2023-11-28T03:08:17.321360+0000","last_active":"2023-11-28T03:08:17.327827+0000","last_peered":"2023-11-28T03:08:17.327827+0000","last_clean":"2023-11-28T03:08:17.327827+0000","last_became_active":"2023-11-28T03:08:17.321046+0000","last_became_peered":"2023-11-28T03:08:17.321046+0000","last_unstale":"2023-11-28T03:08:17.327827+0000","last_undegraded":"2023-11-28T03:08:17.327827+0000","last_fullsized":"2023-11-28T03:08:17.327827+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691186+0000","last_change":"2023-11-28T03:08:17.691186+0000","last_active":"2023-11-28T03:08:17.691186+0000","last_peered":"2023-11-28T03:08:17.691186+0000","last_clean":"2023-11-28T03:08:17.691186+0000","last_became_active":"2023-11-28T03:08:17.690911+0000","last_became_peered":"2023-11-28T03:08:17.690911+0000","last_unstale":"2023-11-28T03:08:17.691186+0000","last_undegraded":"2023-11-28T03:08:17.691186+0000","last_fullsized":"2023-11-28T03:08:17.691186+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676720+0000","last_change":"2023-11-28T03:08:17.325001+0000","last_active":"2023-11-28T03:08:17.676720+0000","last_peered":"2023-11-28T03:08:17.676720+0000","last_clean":"2023-11-28T03:08:17.676720+0000","last_became_active":"2023-11-28T03:08:17.324713+0000","last_became_peered":"2023-11-28T03:08:17.324713+0000","last_unstale":"2023-11-28T03:08:17.676720+0000","last_undegraded":"2023-11-28T03:08:17.676720+0000","last_fullsized":"2023-11-28T03:08:17.676720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692984+0000","last_change":"2023-11-28T03:08:17.692984+0000","last_active":"2023-11-28T03:08:17.692984+0000","last_peered":"2023-11-28T03:08:17.692984+0000","last_clean":"2023-11-28T03:08:17.692984+0000","last_became_active":"2023-11-28T03:08:17.692350+0000","last_became_peered":"2023-11-28T03:08:17.692350+0000","last_unstale":"2023-11-28T03:08:17.692984+0000","last_undegraded":"2023-11-28T03:08:17.692984+0000","last_fullsized":"2023-11-28T03:08:17.692984+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676916+0000","last_change":"2023-11-28T03:08:17.331853+0000","last_active":"2023-11-28T03:08:17.676916+0000","last_peered":"2023-11-28T03:08:17.676916+0000","last_clean":"2023-11-28T03:08:17.676916+0000","last_became_active":"2023-11-28T03:08:17.331353+0000","last_became_peered":"2023-11-28T03:08:17.331353+0000","last_unstale":"2023-11-28T03:08:17.676916+0000","last_undegraded":"2023-11-28T03:08:17.676916+0000","last_fullsized":"2023-11-28T03:08:17.676916+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699336+0000","last_change":"2023-11-28T03:08:17.699336+0000","last_active":"2023-11-28T03:08:17.699336+0000","last_peered":"2023-11-28T03:08:17.699336+0000","last_clean":"2023-11-28T03:08:17.699336+0000","last_became_active":"2023-11-28T03:08:17.699037+0000","last_became_peered":"2023-11-28T03:08:17.699037+0000","last_unstale":"2023-11-28T03:08:17.699336+0000","last_undegraded":"2023-11-28T03:08:17.699336+0000","last_fullsized":"2023-11-28T03:08:17.699336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688939+0000","last_change":"2023-11-28T03:08:17.335439+0000","last_active":"2023-11-28T03:08:17.688939+0000","last_peered":"2023-11-28T03:08:17.688939+0000","last_clean":"2023-11-28T03:08:17.688939+0000","last_became_active":"2023-11-28T03:08:17.335090+0000","last_became_peered":"2023-11-28T03:08:17.335090+0000","last_unstale":"2023-11-28T03:08:17.688939+0000","last_undegraded":"2023-11-28T03:08:17.688939+0000","last_fullsized":"2023-11-28T03:08:17.688939+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694015+0000","last_change":"2023-11-28T03:08:17.694015+0000","last_active":"2023-11-28T03:08:17.694015+0000","last_peered":"2023-11-28T03:08:17.694015+0000","last_clean":"2023-11-28T03:08:17.694015+0000","last_became_active":"2023-11-28T03:08:17.693750+0000","last_became_peered":"2023-11-28T03:08:17.693750+0000","last_unstale":"2023-11-28T03:08:17.694015+0000","last_undegraded":"2023-11-28T03:08:17.694015+0000","last_fullsized":"2023-11-28T03:08:17.694015+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334318+0000","last_change":"2023-11-28T03:08:17.330606+0000","last_active":"2023-11-28T03:08:17.334318+0000","last_peered":"2023-11-28T03:08:17.334318+0000","last_clean":"2023-11-28T03:08:17.334318+0000","last_became_active":"2023-11-28T03:08:17.329165+0000","last_became_peered":"2023-11-28T03:08:17.329165+0000","last_unstale":"2023-11-28T03:08:17.334318+0000","last_undegraded":"2023-11-28T03:08:17.334318+0000","last_fullsized":"2023-11-28T03:08:17.334318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700623+0000","last_change":"2023-11-28T03:08:17.700623+0000","last_active":"2023-11-28T03:08:17.700623+0000","last_peered":"2023-11-28T03:08:17.700623+0000","last_clean":"2023-11-28T03:08:17.700623+0000","last_became_active":"2023-11-28T03:08:17.700089+0000","last_became_peered":"2023-11-28T03:08:17.700089+0000","last_unstale":"2023-11-28T03:08:17.700623+0000","last_undegraded":"2023-11-28T03:08:17.700623+0000","last_fullsized":"2023-11-28T03:08:17.700623+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334551+0000","last_change":"2023-11-28T03:08:17.330392+0000","last_active":"2023-11-28T03:08:17.334551+0000","last_peered":"2023-11-28T03:08:17.334551+0000","last_clean":"2023-11-28T03:08:17.334551+0000","last_became_active":"2023-11-28T03:08:17.330090+0000","last_became_peered":"2023-11-28T03:08:17.330090+0000","last_unstale":"2023-11-28T03:08:17.334551+0000","last_undegraded":"2023-11-28T03:08:17.334551+0000","last_fullsized":"2023-11-28T03:08:17.334551+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.686378+0000","last_change":"2023-11-28T03:08:17.686378+0000","last_active":"2023-11-28T03:08:17.686378+0000","last_peered":"2023-11-28T03:08:17.686378+0000","last_clean":"2023-11-28T03:08:17.686378+0000","last_became_active":"2023-11-28T03:08:17.686142+0000","last_became_peered":"2023-11-28T03:08:17.686142+0000","last_unstale":"2023-11-28T03:08:17.686378+0000","last_undegraded":"2023-11-28T03:08:17.686378+0000","last_fullsized":"2023-11-28T03:08:17.686378+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333091+0000","last_change":"2023-11-28T03:08:17.330388+0000","last_active":"2023-11-28T03:08:17.333091+0000","last_peered":"2023-11-28T03:08:17.333091+0000","last_clean":"2023-11-28T03:08:17.333091+0000","last_became_active":"2023-11-28T03:08:17.330136+0000","last_became_peered":"2023-11-28T03:08:17.330136+0000","last_unstale":"2023-11-28T03:08:17.333091+0000","last_undegraded":"2023-11-28T03:08:17.333091+0000","last_fullsized":"2023-11-28T03:08:17.333091+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699196+0000","last_change":"2023-11-28T03:08:17.699196+0000","last_active":"2023-11-28T03:08:17.699196+0000","last_peered":"2023-11-28T03:08:17.699196+0000","last_clean":"2023-11-28T03:08:17.699196+0000","last_became_active":"2023-11-28T03:08:17.698927+0000","last_became_peered":"2023-11-28T03:08:17.698927+0000","last_unstale":"2023-11-28T03:08:17.699196+0000","last_undegraded":"2023-11-28T03:08:17.699196+0000","last_fullsized":"2023-11-28T03:08:17.699196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326994+0000","last_change":"2023-11-28T03:08:17.324775+0000","last_active":"2023-11-28T03:08:17.326994+0000","last_peered":"2023-11-28T03:08:17.326994+0000","last_clean":"2023-11-28T03:08:17.326994+0000","last_became_active":"2023-11-28T03:08:17.324509+0000","last_became_peered":"2023-11-28T03:08:17.324509+0000","last_unstale":"2023-11-28T03:08:17.326994+0000","last_undegraded":"2023-11-28T03:08:17.326994+0000","last_fullsized":"2023-11-28T03:08:17.326994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698888+0000","last_change":"2023-11-28T03:08:17.698888+0000","last_active":"2023-11-28T03:08:17.698888+0000","last_peered":"2023-11-28T03:08:17.698888+0000","last_clean":"2023-11-28T03:08:17.698888+0000","last_became_active":"2023-11-28T03:08:17.698263+0000","last_became_peered":"2023-11-28T03:08:17.698263+0000","last_unstale":"2023-11-28T03:08:17.698888+0000","last_undegraded":"2023-11-28T03:08:17.698888+0000","last_fullsized":"2023-11-28T03:08:17.698888+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.329381+0000","last_change":"2023-11-28T03:08:17.328221+0000","last_active":"2023-11-28T03:08:17.329381+0000","last_peered":"2023-11-28T03:08:17.329381+0000","last_clean":"2023-11-28T03:08:17.329381+0000","last_became_active":"2023-11-28T03:08:17.327911+0000","last_became_peered":"2023-11-28T03:08:17.327911+0000","last_unstale":"2023-11-28T03:08:17.329381+0000","last_undegraded":"2023-11-28T03:08:17.329381+0000","last_fullsized":"2023-11-28T03:08:17.329381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.684968+0000","last_change":"2023-11-28T03:08:17.684968+0000","last_active":"2023-11-28T03:08:17.684968+0000","last_peered":"2023-11-28T03:08:17.684968+0000","last_clean":"2023-11-28T03:08:17.684968+0000","last_became_active":"2023-11-28T03:08:17.682305+0000","last_became_peered":"2023-11-28T03:08:17.682305+0000","last_unstale":"2023-11-28T03:08:17.684968+0000","last_undegraded":"2023-11-28T03:08:17.684968+0000","last_fullsized":"2023-11-28T03:08:17.684968+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.336356+0000","last_change":"2023-11-28T03:08:17.335193+0000","last_active":"2023-11-28T03:08:17.336356+0000","last_peered":"2023-11-28T03:08:17.336356+0000","last_clean":"2023-11-28T03:08:17.336356+0000","last_became_active":"2023-11-28T03:08:17.334897+0000","last_became_peered":"2023-11-28T03:08:17.334897+0000","last_unstale":"2023-11-28T03:08:17.336356+0000","last_undegraded":"2023-11-28T03:08:17.336356+0000","last_fullsized":"2023-11-28T03:08:17.336356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693595+0000","last_change":"2023-11-28T03:08:17.693595+0000","last_active":"2023-11-28T03:08:17.693595+0000","last_peered":"2023-11-28T03:08:17.693595+0000","last_clean":"2023-11-28T03:08:17.693595+0000","last_became_active":"2023-11-28T03:08:17.693278+0000","last_became_peered":"2023-11-28T03:08:17.693278+0000","last_unstale":"2023-11-28T03:08:17.693595+0000","last_undegraded":"2023-11-28T03:08:17.693595+0000","last_fullsized":"2023-11-28T03:08:17.693595+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331248+0000","last_change":"2023-11-28T03:08:17.328443+0000","last_active":"2023-11-28T03:08:17.331248+0000","last_peered":"2023-11-28T03:08:17.331248+0000","last_clean":"2023-11-28T03:08:17.331248+0000","last_became_active":"2023-11-28T03:08:17.328158+0000","last_became_peered":"2023-11-28T03:08:17.328158+0000","last_unstale":"2023-11-28T03:08:17.331248+0000","last_undegraded":"2023-11-28T03:08:17.331248+0000","last_fullsized":"2023-11-28T03:08:17.331248+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688900+0000","last_change":"2023-11-28T03:08:17.688900+0000","last_active":"2023-11-28T03:08:17.688900+0000","last_peered":"2023-11-28T03:08:17.688900+0000","last_clean":"2023-11-28T03:08:17.688900+0000","last_became_active":"2023-11-28T03:08:17.688616+0000","last_became_peered":"2023-11-28T03:08:17.688616+0000","last_unstale":"2023-11-28T03:08:17.688900+0000","last_undegraded":"2023-11-28T03:08:17.688900+0000","last_fullsized":"2023-11-28T03:08:17.688900+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"109'3","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691726+0000","last_change":"2023-11-28T03:08:17.328759+0000","last_active":"2023-11-28T03:08:17.691726+0000","last_peered":"2023-11-28T03:08:17.691726+0000","last_clean":"2023-11-28T03:08:17.691726+0000","last_became_active":"2023-11-28T03:08:17.328106+0000","last_became_peered":"2023-11-28T03:08:17.328106+0000","last_unstale":"2023-11-28T03:08:17.691726+0000","last_undegraded":"2023-11-28T03:08:17.691726+0000","last_fullsized":"2023-11-28T03:08:17.691726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698724+0000","last_change":"2023-11-28T03:08:17.698724+0000","last_active":"2023-11-28T03:08:17.698724+0000","last_peered":"2023-11-28T03:08:17.698724+0000","last_clean":"2023-11-28T03:08:17.698724+0000","last_became_active":"2023-11-28T03:08:17.698241+0000","last_became_peered":"2023-11-28T03:08:17.698241+0000","last_unstale":"2023-11-28T03:08:17.698724+0000","last_undegraded":"2023-11-28T03:08:17.698724+0000","last_fullsized":"2023-11-28T03:08:17.698724+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692871+0000","last_change":"2023-11-28T03:08:17.324551+0000","last_active":"2023-11-28T03:08:17.692871+0000","last_peered":"2023-11-28T03:08:17.692871+0000","last_clean":"2023-11-28T03:08:17.692871+0000","last_became_active":"2023-11-28T03:08:17.324289+0000","last_became_peered":"2023-11-28T03:08:17.324289+0000","last_unstale":"2023-11-28T03:08:17.692871+0000","last_undegraded":"2023-11-28T03:08:17.692871+0000","last_fullsized":"2023-11-28T03:08:17.692871+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700778+0000","last_change":"2023-11-28T03:08:17.700778+0000","last_active":"2023-11-28T03:08:17.700778+0000","last_peered":"2023-11-28T03:08:17.700778+0000","last_clean":"2023-11-28T03:08:17.700778+0000","last_became_active":"2023-11-28T03:08:17.700375+0000","last_became_peered":"2023-11-28T03:08:17.700375+0000","last_unstale":"2023-11-28T03:08:17.700778+0000","last_undegraded":"2023-11-28T03:08:17.700778+0000","last_fullsized":"2023-11-28T03:08:17.700778+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.680984+0000","last_change":"2023-11-28T03:08:17.328067+0000","last_active":"2023-11-28T03:08:17.680984+0000","last_peered":"2023-11-28T03:08:17.680984+0000","last_clean":"2023-11-28T03:08:17.680984+0000","last_became_active":"2023-11-28T03:08:17.327612+0000","last_became_peered":"2023-11-28T03:08:17.327612+0000","last_unstale":"2023-11-28T03:08:17.680984+0000","last_undegraded":"2023-11-28T03:08:17.680984+0000","last_fullsized":"2023-11-28T03:08:17.680984+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.682593+0000","last_change":"2023-11-28T03:08:17.682593+0000","last_active":"2023-11-28T03:08:17.682593+0000","last_peered":"2023-11-28T03:08:17.682593+0000","last_clean":"2023-11-28T03:08:17.682593+0000","last_became_active":"2023-11-28T03:08:17.682030+0000","last_became_peered":"2023-11-28T03:08:17.682030+0000","last_unstale":"2023-11-28T03:08:17.682593+0000","last_undegraded":"2023-11-28T03:08:17.682593+0000","last_fullsized":"2023-11-28T03:08:17.682593+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.325683+0000","last_change":"2023-11-28T03:08:17.322614+0000","last_active":"2023-11-28T03:08:17.325683+0000","last_peered":"2023-11-28T03:08:17.325683+0000","last_clean":"2023-11-28T03:08:17.325683+0000","last_became_active":"2023-11-28T03:08:17.322309+0000","last_became_peered":"2023-11-28T03:08:17.322309+0000","last_unstale":"2023-11-28T03:08:17.325683+0000","last_undegraded":"2023-11-28T03:08:17.325683+0000","last_fullsized":"2023-11-28T03:08:17.325683+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699412+0000","last_change":"2023-11-28T03:08:17.699412+0000","last_active":"2023-11-28T03:08:17.699412+0000","last_peered":"2023-11-28T03:08:17.699412+0000","last_clean":"2023-11-28T03:08:17.699412+0000","last_became_active":"2023-11-28T03:08:17.699140+0000","last_became_peered":"2023-11-28T03:08:17.699140+0000","last_unstale":"2023-11-28T03:08:17.699412+0000","last_undegraded":"2023-11-28T03:08:17.699412+0000","last_fullsized":"2023-11-28T03:08:17.699412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332927+0000","last_change":"2023-11-28T03:08:17.329283+0000","last_active":"2023-11-28T03:08:17.332927+0000","last_peered":"2023-11-28T03:08:17.332927+0000","last_clean":"2023-11-28T03:08:17.332927+0000","last_became_active":"2023-11-28T03:08:17.328888+0000","last_became_peered":"2023-11-28T03:08:17.328888+0000","last_unstale":"2023-11-28T03:08:17.332927+0000","last_undegraded":"2023-11-28T03:08:17.332927+0000","last_fullsized":"2023-11-28T03:08:17.332927+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.687995+0000","last_change":"2023-11-28T03:08:17.687995+0000","last_active":"2023-11-28T03:08:17.687995+0000","last_peered":"2023-11-28T03:08:17.687995+0000","last_clean":"2023-11-28T03:08:17.687995+0000","last_became_active":"2023-11-28T03:08:17.687767+0000","last_became_peered":"2023-11-28T03:08:17.687767+0000","last_unstale":"2023-11-28T03:08:17.687995+0000","last_undegraded":"2023-11-28T03:08:17.687995+0000","last_fullsized":"2023-11-28T03:08:17.687995+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.701953+0000","last_change":"2023-11-28T03:08:17.329865+0000","last_active":"2023-11-28T03:08:17.701953+0000","last_peered":"2023-11-28T03:08:17.701953+0000","last_clean":"2023-11-28T03:08:17.701953+0000","last_became_active":"2023-11-28T03:08:17.329617+0000","last_became_peered":"2023-11-28T03:08:17.329617+0000","last_unstale":"2023-11-28T03:08:17.701953+0000","last_undegraded":"2023-11-28T03:08:17.701953+0000","last_fullsized":"2023-11-28T03:08:17.701953+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.690105+0000","last_change":"2023-11-28T03:08:17.690105+0000","last_active":"2023-11-28T03:08:17.690105+0000","last_peered":"2023-11-28T03:08:17.690105+0000","last_clean":"2023-11-28T03:08:17.690105+0000","last_became_active":"2023-11-28T03:08:17.689845+0000","last_became_peered":"2023-11-28T03:08:17.689845+0000","last_unstale":"2023-11-28T03:08:17.690105+0000","last_undegraded":"2023-11-28T03:08:17.690105+0000","last_fullsized":"2023-11-28T03:08:17.690105+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694930+0000","last_change":"2023-11-28T03:08:17.330562+0000","last_active":"2023-11-28T03:08:17.694930+0000","last_peered":"2023-11-28T03:08:17.694930+0000","last_clean":"2023-11-28T03:08:17.694930+0000","last_became_active":"2023-11-28T03:08:17.330307+0000","last_became_peered":"2023-11-28T03:08:17.330307+0000","last_unstale":"2023-11-28T03:08:17.694930+0000","last_undegraded":"2023-11-28T03:08:17.694930+0000","last_fullsized":"2023-11-28T03:08:17.694930+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693174+0000","last_change":"2023-11-28T03:08:17.693174+0000","last_active":"2023-11-28T03:08:17.693174+0000","last_peered":"2023-11-28T03:08:17.693174+0000","last_clean":"2023-11-28T03:08:17.693174+0000","last_became_active":"2023-11-28T03:08:17.692744+0000","last_became_peered":"2023-11-28T03:08:17.692744+0000","last_unstale":"2023-11-28T03:08:17.693174+0000","last_undegraded":"2023-11-28T03:08:17.693174+0000","last_fullsized":"2023-11-28T03:08:17.693174+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334682+0000","last_change":"2023-11-28T03:08:17.332842+0000","last_active":"2023-11-28T03:08:17.334682+0000","last_peered":"2023-11-28T03:08:17.334682+0000","last_clean":"2023-11-28T03:08:17.334682+0000","last_became_active":"2023-11-28T03:08:17.332571+0000","last_became_peered":"2023-11-28T03:08:17.332571+0000","last_unstale":"2023-11-28T03:08:17.334682+0000","last_undegraded":"2023-11-28T03:08:17.334682+0000","last_fullsized":"2023-11-28T03:08:17.334682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.690902+0000","last_change":"2023-11-28T03:08:17.690902+0000","last_active":"2023-11-28T03:08:17.690902+0000","last_peered":"2023-11-28T03:08:17.690902+0000","last_clean":"2023-11-28T03:08:17.690902+0000","last_became_active":"2023-11-28T03:08:17.690510+0000","last_became_peered":"2023-11-28T03:08:17.690510+0000","last_unstale":"2023-11-28T03:08:17.690902+0000","last_undegraded":"2023-11-28T03:08:17.690902+0000","last_fullsized":"2023-11-28T03:08:17.690902+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334502+0000","last_change":"2023-11-28T03:08:17.328915+0000","last_active":"2023-11-28T03:08:17.334502+0000","last_peered":"2023-11-28T03:08:17.334502+0000","last_clean":"2023-11-28T03:08:17.334502+0000","last_became_active":"2023-11-28T03:08:17.328467+0000","last_became_peered":"2023-11-28T03:08:17.328467+0000","last_unstale":"2023-11-28T03:08:17.334502+0000","last_undegraded":"2023-11-28T03:08:17.334502+0000","last_fullsized":"2023-11-28T03:08:17.334502+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693022+0000","last_change":"2023-11-28T03:08:17.693022+0000","last_active":"2023-11-28T03:08:17.693022+0000","last_peered":"2023-11-28T03:08:17.693022+0000","last_clean":"2023-11-28T03:08:17.693022+0000","last_became_active":"2023-11-28T03:08:17.692715+0000","last_became_peered":"2023-11-28T03:08:17.692715+0000","last_unstale":"2023-11-28T03:08:17.693022+0000","last_undegraded":"2023-11-28T03:08:17.693022+0000","last_fullsized":"2023-11-28T03:08:17.693022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334337+0000","last_change":"2023-11-28T03:08:17.331112+0000","last_active":"2023-11-28T03:08:17.334337+0000","last_peered":"2023-11-28T03:08:17.334337+0000","last_clean":"2023-11-28T03:08:17.334337+0000","last_became_active":"2023-11-28T03:08:17.330856+0000","last_became_peered":"2023-11-28T03:08:17.330856+0000","last_unstale":"2023-11-28T03:08:17.334337+0000","last_undegraded":"2023-11-28T03:08:17.334337+0000","last_fullsized":"2023-11-28T03:08:17.334337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689494+0000","last_change":"2023-11-28T03:08:17.689494+0000","last_active":"2023-11-28T03:08:17.689494+0000","last_peered":"2023-11-28T03:08:17.689494+0000","last_clean":"2023-11-28T03:08:17.689494+0000","last_became_active":"2023-11-28T03:08:17.689240+0000","last_became_peered":"2023-11-28T03:08:17.689240+0000","last_unstale":"2023-11-28T03:08:17.689494+0000","last_undegraded":"2023-11-28T03:08:17.689494+0000","last_fullsized":"2023-11-28T03:08:17.689494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.328312+0000","last_change":"2023-11-28T03:08:17.325554+0000","last_active":"2023-11-28T03:08:17.328312+0000","last_peered":"2023-11-28T03:08:17.328312+0000","last_clean":"2023-11-28T03:08:17.328312+0000","last_became_active":"2023-11-28T03:08:17.325266+0000","last_became_peered":"2023-11-28T03:08:17.325266+0000","last_unstale":"2023-11-28T03:08:17.328312+0000","last_undegraded":"2023-11-28T03:08:17.328312+0000","last_fullsized":"2023-11-28T03:08:17.328312+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699350+0000","last_change":"2023-11-28T03:08:17.699350+0000","last_active":"2023-11-28T03:08:17.699350+0000","last_peered":"2023-11-28T03:08:17.699350+0000","last_clean":"2023-11-28T03:08:17.699350+0000","last_became_active":"2023-11-28T03:08:17.698905+0000","last_became_peered":"2023-11-28T03:08:17.698905+0000","last_unstale":"2023-11-28T03:08:17.699350+0000","last_undegraded":"2023-11-28T03:08:17.699350+0000","last_fullsized":"2023-11-28T03:08:17.699350+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331508+0000","last_change":"2023-11-28T03:08:17.327462+0000","last_active":"2023-11-28T03:08:17.331508+0000","last_peered":"2023-11-28T03:08:17.331508+0000","last_clean":"2023-11-28T03:08:17.331508+0000","last_became_active":"2023-11-28T03:08:17.327191+0000","last_became_peered":"2023-11-28T03:08:17.327191+0000","last_unstale":"2023-11-28T03:08:17.331508+0000","last_undegraded":"2023-11-28T03:08:17.331508+0000","last_fullsized":"2023-11-28T03:08:17.331508+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699212+0000","last_change":"2023-11-28T03:08:17.699212+0000","last_active":"2023-11-28T03:08:17.699212+0000","last_peered":"2023-11-28T03:08:17.699212+0000","last_clean":"2023-11-28T03:08:17.699212+0000","last_became_active":"2023-11-28T03:08:17.698911+0000","last_became_peered":"2023-11-28T03:08:17.698911+0000","last_unstale":"2023-11-28T03:08:17.699212+0000","last_undegraded":"2023-11-28T03:08:17.699212+0000","last_fullsized":"2023-11-28T03:08:17.699212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334248+0000","last_change":"2023-11-28T03:08:17.330478+0000","last_active":"2023-11-28T03:08:17.334248+0000","last_peered":"2023-11-28T03:08:17.334248+0000","last_clean":"2023-11-28T03:08:17.334248+0000","last_became_active":"2023-11-28T03:08:17.330198+0000","last_became_peered":"2023-11-28T03:08:17.330198+0000","last_unstale":"2023-11-28T03:08:17.334248+0000","last_undegraded":"2023-11-28T03:08:17.334248+0000","last_fullsized":"2023-11-28T03:08:17.334248+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.686138+0000","last_change":"2023-11-28T03:08:17.686138+0000","last_active":"2023-11-28T03:08:17.686138+0000","last_peered":"2023-11-28T03:08:17.686138+0000","last_clean":"2023-11-28T03:08:17.686138+0000","last_became_active":"2023-11-28T03:08:17.685860+0000","last_became_peered":"2023-11-28T03:08:17.685860+0000","last_unstale":"2023-11-28T03:08:17.686138+0000","last_undegraded":"2023-11-28T03:08:17.686138+0000","last_fullsized":"2023-11-28T03:08:17.686138+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334460+0000","last_change":"2023-11-28T03:08:17.333013+0000","last_active":"2023-11-28T03:08:17.334460+0000","last_peered":"2023-11-28T03:08:17.334460+0000","last_clean":"2023-11-28T03:08:17.334460+0000","last_became_active":"2023-11-28T03:08:17.332678+0000","last_became_peered":"2023-11-28T03:08:17.332678+0000","last_unstale":"2023-11-28T03:08:17.334460+0000","last_undegraded":"2023-11-28T03:08:17.334460+0000","last_fullsized":"2023-11-28T03:08:17.334460+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.703755+0000","last_change":"2023-11-28T03:08:17.703755+0000","last_active":"2023-11-28T03:08:17.703755+0000","last_peered":"2023-11-28T03:08:17.703755+0000","last_clean":"2023-11-28T03:08:17.703755+0000","last_became_active":"2023-11-28T03:08:17.703466+0000","last_became_peered":"2023-11-28T03:08:17.703466+0000","last_unstale":"2023-11-28T03:08:17.703755+0000","last_undegraded":"2023-11-28T03:08:17.703755+0000","last_fullsized":"2023-11-28T03:08:17.703755+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.684192+0000","last_change":"2023-11-28T03:08:17.325894+0000","last_active":"2023-11-28T03:08:17.684192+0000","last_peered":"2023-11-28T03:08:17.684192+0000","last_clean":"2023-11-28T03:08:17.684192+0000","last_became_active":"2023-11-28T03:08:17.325568+0000","last_became_peered":"2023-11-28T03:08:17.325568+0000","last_unstale":"2023-11-28T03:08:17.684192+0000","last_undegraded":"2023-11-28T03:08:17.684192+0000","last_fullsized":"2023-11-28T03:08:17.684192+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.316079+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:08:17.316079+0000","last_peered":"2023-11-28T03:08:17.316079+0000","last_clean":"2023-11-28T03:08:17.316079+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:08:17.316079+0000","last_undegraded":"2023-11-28T03:08:17.316079+0000","last_fullsized":"2023-11-28T03:08:17.316079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.679184+0000","last_change":"2023-11-28T03:08:17.679184+0000","last_active":"2023-11-28T03:08:17.679184+0000","last_peered":"2023-11-28T03:08:17.679184+0000","last_clean":"2023-11-28T03:08:17.679184+0000","last_became_active":"2023-11-28T03:08:17.678903+0000","last_became_peered":"2023-11-28T03:08:17.678903+0000","last_unstale":"2023-11-28T03:08:17.679184+0000","last_undegraded":"2023-11-28T03:08:17.679184+0000","last_fullsized":"2023-11-28T03:08:17.679184+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326301+0000","last_change":"2023-11-28T03:08:17.323102+0000","last_active":"2023-11-28T03:08:17.326301+0000","last_peered":"2023-11-28T03:08:17.326301+0000","last_clean":"2023-11-28T03:08:17.326301+0000","last_became_active":"2023-11-28T03:08:17.322617+0000","last_became_peered":"2023-11-28T03:08:17.322617+0000","last_unstale":"2023-11-28T03:08:17.326301+0000","last_undegraded":"2023-11-28T03:08:17.326301+0000","last_fullsized":"2023-11-28T03:08:17.326301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.322465+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:08:17.322465+0000","last_peered":"2023-11-28T03:08:17.322465+0000","last_clean":"2023-11-28T03:08:17.322465+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:08:17.322465+0000","last_undegraded":"2023-11-28T03:08:17.322465+0000","last_fullsized":"2023-11-28T03:08:17.322465+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692414+0000","last_change":"2023-11-28T03:08:17.692414+0000","last_active":"2023-11-28T03:08:17.692414+0000","last_peered":"2023-11-28T03:08:17.692414+0000","last_clean":"2023-11-28T03:08:17.692414+0000","last_became_active":"2023-11-28T03:08:17.692142+0000","last_became_peered":"2023-11-28T03:08:17.692142+0000","last_unstale":"2023-11-28T03:08:17.692414+0000","last_undegraded":"2023-11-28T03:08:17.692414+0000","last_fullsized":"2023-11-28T03:08:17.692414+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689089+0000","last_change":"2023-11-28T03:08:17.331448+0000","last_active":"2023-11-28T03:08:17.689089+0000","last_peered":"2023-11-28T03:08:17.689089+0000","last_clean":"2023-11-28T03:08:17.689089+0000","last_became_active":"2023-11-28T03:08:17.331172+0000","last_became_peered":"2023-11-28T03:08:17.331172+0000","last_unstale":"2023-11-28T03:08:17.689089+0000","last_undegraded":"2023-11-28T03:08:17.689089+0000","last_fullsized":"2023-11-28T03:08:17.689089+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.307150+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:08:17.307150+0000","last_peered":"2023-11-28T03:08:17.307150+0000","last_clean":"2023-11-28T03:08:17.307150+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:08:17.307150+0000","last_undegraded":"2023-11-28T03:08:17.307150+0000","last_fullsized":"2023-11-28T03:08:17.307150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698279+0000","last_change":"2023-11-28T03:08:17.698279+0000","last_active":"2023-11-28T03:08:17.698279+0000","last_peered":"2023-11-28T03:08:17.698279+0000","last_clean":"2023-11-28T03:08:17.698279+0000","last_became_active":"2023-11-28T03:08:17.697922+0000","last_became_peered":"2023-11-28T03:08:17.697922+0000","last_unstale":"2023-11-28T03:08:17.698279+0000","last_undegraded":"2023-11-28T03:08:17.698279+0000","last_fullsized":"2023-11-28T03:08:17.698279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332864+0000","last_change":"2023-11-28T03:08:17.325384+0000","last_active":"2023-11-28T03:08:17.332864+0000","last_peered":"2023-11-28T03:08:17.332864+0000","last_clean":"2023-11-28T03:08:17.332864+0000","last_became_active":"2023-11-28T03:08:17.325172+0000","last_became_peered":"2023-11-28T03:08:17.325172+0000","last_unstale":"2023-11-28T03:08:17.332864+0000","last_undegraded":"2023-11-28T03:08:17.332864+0000","last_fullsized":"2023-11-28T03:08:17.332864+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.320493+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:08:17.320493+0000","last_peered":"2023-11-28T03:08:17.320493+0000","last_clean":"2023-11-28T03:08:17.320493+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:08:17.320493+0000","last_undegraded":"2023-11-28T03:08:17.320493+0000","last_fullsized":"2023-11-28T03:08:17.320493+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.320643+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:08:17.320643+0000","last_peered":"2023-11-28T03:08:17.320643+0000","last_clean":"2023-11-28T03:08:17.320643+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:08:17.320643+0000","last_undegraded":"2023-11-28T03:08:17.320643+0000","last_fullsized":"2023-11-28T03:08:17.320643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692575+0000","last_change":"2023-11-28T03:08:17.692575+0000","last_active":"2023-11-28T03:08:17.692575+0000","last_peered":"2023-11-28T03:08:17.692575+0000","last_clean":"2023-11-28T03:08:17.692575+0000","last_became_active":"2023-11-28T03:08:17.692302+0000","last_became_peered":"2023-11-28T03:08:17.692302+0000","last_unstale":"2023-11-28T03:08:17.692575+0000","last_undegraded":"2023-11-28T03:08:17.692575+0000","last_fullsized":"2023-11-28T03:08:17.692575+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676617+0000","last_change":"2023-11-28T03:08:17.321111+0000","last_active":"2023-11-28T03:08:17.676617+0000","last_peered":"2023-11-28T03:08:17.676617+0000","last_clean":"2023-11-28T03:08:17.676617+0000","last_became_active":"2023-11-28T03:08:17.318963+0000","last_became_peered":"2023-11-28T03:08:17.318963+0000","last_unstale":"2023-11-28T03:08:17.676617+0000","last_undegraded":"2023-11-28T03:08:17.676617+0000","last_fullsized":"2023-11-28T03:08:17.676617+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.323173+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:08:17.323173+0000","last_peered":"2023-11-28T03:08:17.323173+0000","last_clean":"2023-11-28T03:08:17.323173+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:08:17.323173+0000","last_undegraded":"2023-11-28T03:08:17.323173+0000","last_fullsized":"2023-11-28T03:08:17.323173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689902+0000","last_change":"2023-11-28T03:08:17.689902+0000","last_active":"2023-11-28T03:08:17.689902+0000","last_peered":"2023-11-28T03:08:17.689902+0000","last_clean":"2023-11-28T03:08:17.689902+0000","last_became_active":"2023-11-28T03:08:17.689615+0000","last_became_peered":"2023-11-28T03:08:17.689615+0000","last_unstale":"2023-11-28T03:08:17.689902+0000","last_undegraded":"2023-11-28T03:08:17.689902+0000","last_fullsized":"2023-11-28T03:08:17.689902+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334319+0000","last_change":"2023-11-28T03:08:17.320728+0000","last_active":"2023-11-28T03:08:17.334319+0000","last_peered":"2023-11-28T03:08:17.334319+0000","last_clean":"2023-11-28T03:08:17.334319+0000","last_became_active":"2023-11-28T03:08:17.320449+0000","last_became_peered":"2023-11-28T03:08:17.320449+0000","last_unstale":"2023-11-28T03:08:17.334319+0000","last_undegraded":"2023-11-28T03:08:17.334319+0000","last_fullsized":"2023-11-28T03:08:17.334319+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.322041+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:08:17.322041+0000","last_peered":"2023-11-28T03:08:17.322041+0000","last_clean":"2023-11-28T03:08:17.322041+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:08:17.322041+0000","last_undegraded":"2023-11-28T03:08:17.322041+0000","last_fullsized":"2023-11-28T03:08:17.322041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.701783+0000","last_change":"2023-11-28T03:08:17.701783+0000","last_active":"2023-11-28T03:08:17.701783+0000","last_peered":"2023-11-28T03:08:17.701783+0000","last_clean":"2023-11-28T03:08:17.701783+0000","last_became_active":"2023-11-28T03:08:17.701503+0000","last_became_peered":"2023-11-28T03:08:17.701503+0000","last_unstale":"2023-11-28T03:08:17.701783+0000","last_undegraded":"2023-11-28T03:08:17.701783+0000","last_fullsized":"2023-11-28T03:08:17.701783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333415+0000","last_change":"2023-11-28T03:08:17.329966+0000","last_active":"2023-11-28T03:08:17.333415+0000","last_peered":"2023-11-28T03:08:17.333415+0000","last_clean":"2023-11-28T03:08:17.333415+0000","last_became_active":"2023-11-28T03:08:17.327865+0000","last_became_peered":"2023-11-28T03:08:17.327865+0000","last_unstale":"2023-11-28T03:08:17.333415+0000","last_undegraded":"2023-11-28T03:08:17.333415+0000","last_fullsized":"2023-11-28T03:08:17.333415+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.314487+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:08:17.314487+0000","last_peered":"2023-11-28T03:08:17.314487+0000","last_clean":"2023-11-28T03:08:17.314487+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:08:17.314487+0000","last_undegraded":"2023-11-28T03:08:17.314487+0000","last_fullsized":"2023-11-28T03:08:17.314487+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693459+0000","last_change":"2023-11-28T03:08:17.693459+0000","last_active":"2023-11-28T03:08:17.693459+0000","last_peered":"2023-11-28T03:08:17.693459+0000","last_clean":"2023-11-28T03:08:17.693459+0000","last_became_active":"2023-11-28T03:08:17.693204+0000","last_became_peered":"2023-11-28T03:08:17.693204+0000","last_unstale":"2023-11-28T03:08:17.693459+0000","last_undegraded":"2023-11-28T03:08:17.693459+0000","last_fullsized":"2023-11-28T03:08:17.693459+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334630+0000","last_change":"2023-11-28T03:08:17.329148+0000","last_active":"2023-11-28T03:08:17.334630+0000","last_peered":"2023-11-28T03:08:17.334630+0000","last_clean":"2023-11-28T03:08:17.334630+0000","last_became_active":"2023-11-28T03:08:17.328878+0000","last_became_peered":"2023-11-28T03:08:17.328878+0000","last_unstale":"2023-11-28T03:08:17.334630+0000","last_undegraded":"2023-11-28T03:08:17.334630+0000","last_fullsized":"2023-11-28T03:08:17.334630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.313517+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:08:17.313517+0000","last_peered":"2023-11-28T03:08:17.313517+0000","last_clean":"2023-11-28T03:08:17.313517+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:08:17.313517+0000","last_undegraded":"2023-11-28T03:08:17.313517+0000","last_fullsized":"2023-11-28T03:08:17.313517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.63100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.75,"15min":0.77100000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.376},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.433}]},{"osd":5,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.313},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.49299999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.58499999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.32800000000000001},"max":{"1min":0.66700000000000004,"5min":1.506,"15min":1.506},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.5},{"interface":"front","average":{"1min":0.503,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.65700000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68700000000000006,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47299999999999998,"15min":0.45500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78100000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82499999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.874,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.82099999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.84399999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.90400000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33200000000000002,"15min":0.34100000000000003},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.41799999999999998,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.32900000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.371,"15min":0.379},"min":{"1min":0.22700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45200000000000001,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.39200000000000002,"15min":0.39900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.41199999999999998,"5min":0.628,"15min":0.72599999999999998},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.434,"15min":0.44900000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.44700000000000001,"5min":0.69599999999999995,"15min":0.749},"last":0.441},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58099999999999996,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60399999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.84699999999999998,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.038,"15min":1.111},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57499999999999996,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":1.024,"15min":1.024},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59499999999999997,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.38700000000000001},"max":{"1min":0.72499999999999998,"5min":0.87,"15min":1.7929999999999999},"last":0.437}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33700000000000002,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.184},"max":{"1min":0.40400000000000003,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.309},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.35099999999999998,"15min":0.36499999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.34200000000000003,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.377,"5min":0.52600000000000002,"15min":0.69699999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42099999999999999,"15min":0.438},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.44,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.375,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.73099999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46100000000000002,"5min":0.621,"15min":0.66500000000000004},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64200000000000002,"5min":0.95699999999999996,"15min":1.0489999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67200000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":1.304,"15min":1.304},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.435}]},{"osd":6,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57999999999999996,"15min":0.59499999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":1.679,"15min":1.679},"last":0.441},{"interface":"front","average":{"1min":0.496,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35899999999999999},"max":{"1min":0.61699999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40400000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.63900000000000001,"15min":0.66400000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":1.046,"15min":1.046},"last":0.47599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36499999999999999},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.45400000000000001,"5min":0.59499999999999997,"15min":0.623},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.39300000000000002,"15min":0.41299999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.23599999999999999},"max":{"1min":0.49199999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27300000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35799999999999998,"15min":0.36499999999999999},"min":{"1min":0.249,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.47799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27400000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52300000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23999999999999999},"max":{"1min":0.54000000000000004,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.312}]},{"osd":4,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.64500000000000002,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72899999999999998,"5min":0.84299999999999997,"15min":0.874},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82699999999999996,"5min":1.004,"15min":1.004},"last":0.433}]},{"osd":6,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.441},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77500000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.34899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.23799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.442,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.44800000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.372,"15min":0.376},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.312},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48899999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":1.581,"15min":1.6000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70299999999999996,"5min":0.90800000000000003,"15min":1.196},"last":0.45600000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83799999999999997,"15min":1.1439999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61199999999999999,"15min":0.629},"min":{"1min":0.40000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82299999999999995,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59999999999999998,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.91900000000000004,"15min":1.206},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.99099999999999999,"15min":1.1759999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.68500000000000005,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61699999999999999,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.90000000000000002,"15min":1.2290000000000001},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302504,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069336,"statfs":{"total":96636764160,"available":96327000064,"internally_reserved":0,"allocated":1343488,"data_stored":664995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.72599999999999998,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.65700000000000003,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71899999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.69999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.437,"5min":0.437,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.84599999999999997,"15min":0.878},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.504}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.27900000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.41299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59799999999999998,"5min":0.71999999999999997,"15min":0.77900000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.64700000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67100000000000004,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74299999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.60999999999999999,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80600000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.65300000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.317,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"max":{"1min":0.69199999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52600000000000002,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:08:20.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:08:20.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.808+0000 7fd70e990700 1 Processor -- start 2023-11-28T03:08:20.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 -- start start 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd708069070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7081514a0 0x7fd708069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080a9ea0 con 0x7fd7081510c0 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd708069a50 con 0x7fd7081514a0 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70e990700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080ac4d0 con 0x7fd708063e50 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.809+0000 7fd70cf2d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7081514a0 0x7fd708069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7081514a0 0x7fd708069560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54982/0 (socket says 172.21.15.73:54982) 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 -- 172.21.15.73:0/1471238317 learned_addr learned my addr 172.21.15.73:0/1471238317 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd7077fe700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd708069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 -- 172.21.15.73:0/1471238317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 msgr2=0x7fd708069070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd708069070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 -- 172.21.15.73:0/1471238317 >> v1:172.21.15.150:6789/0 conn(0x7fd7081510c0 legacy=0x7fd7080a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 -- 172.21.15.73:0/1471238317 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080ad4a0 con 0x7fd7081514a0 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd70cf2d700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd708069560 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fd700009f50 tx=0x7fd700009840).ready entity=mon.0 client_cookie=d2b874e2cd255b41 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd706ffd700 1 -- 172.21.15.73:0/1471238317 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000e070 con 0x7fd7081514a0 2023-11-28T03:08:20.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd706ffd700 1 -- 172.21.15.73:0/1471238317 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd70000ba70 con 0x7fd7081514a0 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.810+0000 7fd706ffd700 1 -- 172.21.15.73:0/1471238317 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd70000bdf0 con 0x7fd7081514a0 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 -- 172.21.15.73:0/1471238317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 msgr2=0x7fd708069560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd708069560 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 rx=0x7fd700009f50 tx=0x7fd700009840).stop 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 -- 172.21.15.73:0/1471238317 shutdown_connections 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd708069560 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 --2- 172.21.15.73:0/1471238317 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd708069070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 -- 172.21.15.73:0/1471238317 >> 172.21.15.73:0/1471238317 conn(0x7fd7080c7760 msgr2=0x7fd7080c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 -- 172.21.15.73:0/1471238317 shutdown_connections 2023-11-28T03:08:20.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.811+0000 7fd70e990700 1 -- 172.21.15.73:0/1471238317 wait complete. 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.812+0000 7fd70e990700 1 Processor -- start 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.812+0000 7fd70e990700 1 -- start start 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd70813f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7081510c0 0x7fd70813fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7081497a0 con 0x7fd7081514a0 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708140340 con 0x7fd7081510c0 2023-11-28T03:08:20.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd708148b90 con 0x7fd708063e50 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54988/0 (socket says 172.21.15.73:54988) 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 -- 172.21.15.73:0/591948425 learned_addr learned my addr 172.21.15.73:0/591948425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd707fff700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd70813f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd7077fe700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7081510c0 0x7fd70813fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 -- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 msgr2=0x7fd70813f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd70813f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 -- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7081510c0 msgr2=0x7fd70813fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7081510c0 0x7fd70813fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 -- 172.21.15.73:0/591948425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd700009ae0 con 0x7fd7081514a0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70cf2d700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fd700009520 tx=0x7fd700009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000092e0 con 0x7fd7081514a0 2023-11-28T03:08:20.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd700007610 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7000079d0 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd708148e70 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.813+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd708149430 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.814+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70806c860 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.815+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd700003040 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.815+0000 7fd6effff700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 0x7fd6f4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.815+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd70007b690 con 0x7fd7081514a0 2023-11-28T03:08:20.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.815+0000 7fd707fff700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 0x7fd6f4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.816+0000 7fd707fff700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 0x7fd6f4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd6f8003900 tx=0x7fd6f8007d40).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:20.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.819+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd70004ca10 con 0x7fd7081514a0 2023-11-28T03:08:20.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.978+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7080aaa40 con 0x7fd6f405f150 2023-11-28T03:08:20.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.988+0000 7fd6effff700 1 -- 172.21.15.73:0/591948425 <== mgr.5837 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361843 (secure 0 0 0) 0x7fd7080aaa40 con 0x7fd6f405f150 2023-11-28T03:08:20.993 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:20.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 msgr2=0x7fd6f4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 0x7fd6f4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fd6f8003900 tx=0x7fd6f8007d40).stop 2023-11-28T03:08:20.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 msgr2=0x7fd70813da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:20.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 rx=0x7fd700009520 tx=0x7fd700009840).stop 2023-11-28T03:08:20.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 shutdown_connections 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6f405f150 0x7fd6f4061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7081514a0 0x7fd70813da10 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd708063e50 0x7fd70813f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 --2- 172.21.15.73:0/591948425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7081510c0 0x7fd70813fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.991+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 >> 172.21.15.73:0/591948425 conn(0x7fd7080c7760 msgr2=0x7fd7080c6b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.992+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 shutdown_connections 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:20.992+0000 7fd70e990700 1 -- 172.21.15.73:0/591948425 wait complete. 2023-11-28T03:08:20.997 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:08:21.005 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:08:20.721066+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":89,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2419884,"kb_used_data":10348,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752554836,"statfs":{"total":773094113280,"available":770616152064,"internally_reserved":0,"allocated":10596352,"data_stored":5249040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4466,"num_objects":-22,"num_object_clones":0,"num_object_copies":-44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-50,"num_write_kb":-18,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.589821"},"pg_stats":[{"pgid":"14.3c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696428+0000","last_change":"2023-11-28T03:08:17.696428+0000","last_active":"2023-11-28T03:08:17.696428+0000","last_peered":"2023-11-28T03:08:17.696428+0000","last_clean":"2023-11-28T03:08:17.696428+0000","last_became_active":"2023-11-28T03:08:17.691520+0000","last_became_peered":"2023-11-28T03:08:17.691520+0000","last_unstale":"2023-11-28T03:08:17.696428+0000","last_undegraded":"2023-11-28T03:08:17.696428+0000","last_fullsized":"2023-11-28T03:08:17.696428+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3f","version":"109'3","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692670+0000","last_change":"2023-11-28T03:08:17.328984+0000","last_active":"2023-11-28T03:08:17.692670+0000","last_peered":"2023-11-28T03:08:17.692670+0000","last_clean":"2023-11-28T03:08:17.692670+0000","last_became_active":"2023-11-28T03:08:17.328636+0000","last_became_peered":"2023-11-28T03:08:17.328636+0000","last_unstale":"2023-11-28T03:08:17.692670+0000","last_undegraded":"2023-11-28T03:08:17.692670+0000","last_fullsized":"2023-11-28T03:08:17.692670+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688004+0000","last_change":"2023-11-28T03:08:17.688004+0000","last_active":"2023-11-28T03:08:17.688004+0000","last_peered":"2023-11-28T03:08:17.688004+0000","last_clean":"2023-11-28T03:08:17.688004+0000","last_became_active":"2023-11-28T03:08:17.687729+0000","last_became_peered":"2023-11-28T03:08:17.687729+0000","last_unstale":"2023-11-28T03:08:17.688004+0000","last_undegraded":"2023-11-28T03:08:17.688004+0000","last_fullsized":"2023-11-28T03:08:17.688004+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333227+0000","last_change":"2023-11-28T03:08:17.331233+0000","last_active":"2023-11-28T03:08:17.333227+0000","last_peered":"2023-11-28T03:08:17.333227+0000","last_clean":"2023-11-28T03:08:17.333227+0000","last_became_active":"2023-11-28T03:08:17.330982+0000","last_became_peered":"2023-11-28T03:08:17.330982+0000","last_unstale":"2023-11-28T03:08:17.333227+0000","last_undegraded":"2023-11-28T03:08:17.333227+0000","last_fullsized":"2023-11-28T03:08:17.333227+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.3e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.697365+0000","last_change":"2023-11-28T03:08:17.697365+0000","last_active":"2023-11-28T03:08:17.697365+0000","last_peered":"2023-11-28T03:08:17.697365+0000","last_clean":"2023-11-28T03:08:17.697365+0000","last_became_active":"2023-11-28T03:08:17.697044+0000","last_became_peered":"2023-11-28T03:08:17.697044+0000","last_unstale":"2023-11-28T03:08:17.697365+0000","last_undegraded":"2023-11-28T03:08:17.697365+0000","last_fullsized":"2023-11-28T03:08:17.697365+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331741+0000","last_change":"2023-11-28T03:08:17.329025+0000","last_active":"2023-11-28T03:08:17.331741+0000","last_peered":"2023-11-28T03:08:17.331741+0000","last_clean":"2023-11-28T03:08:17.331741+0000","last_became_active":"2023-11-28T03:08:17.328715+0000","last_became_peered":"2023-11-28T03:08:17.328715+0000","last_unstale":"2023-11-28T03:08:17.331741+0000","last_undegraded":"2023-11-28T03:08:17.331741+0000","last_fullsized":"2023-11-28T03:08:17.331741+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.3f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691639+0000","last_change":"2023-11-28T03:08:17.691639+0000","last_active":"2023-11-28T03:08:17.691639+0000","last_peered":"2023-11-28T03:08:17.691639+0000","last_clean":"2023-11-28T03:08:17.691639+0000","last_became_active":"2023-11-28T03:08:17.691381+0000","last_became_peered":"2023-11-28T03:08:17.691381+0000","last_unstale":"2023-11-28T03:08:17.691639+0000","last_undegraded":"2023-11-28T03:08:17.691639+0000","last_fullsized":"2023-11-28T03:08:17.691639+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3c","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.685220+0000","last_change":"2023-11-28T03:08:17.330867+0000","last_active":"2023-11-28T03:08:17.685220+0000","last_peered":"2023-11-28T03:08:17.685220+0000","last_clean":"2023-11-28T03:08:17.685220+0000","last_became_active":"2023-11-28T03:08:17.329994+0000","last_became_peered":"2023-11-28T03:08:17.329994+0000","last_unstale":"2023-11-28T03:08:17.685220+0000","last_undegraded":"2023-11-28T03:08:17.685220+0000","last_fullsized":"2023-11-28T03:08:17.685220+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.38","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699910+0000","last_change":"2023-11-28T03:08:17.699910+0000","last_active":"2023-11-28T03:08:17.699910+0000","last_peered":"2023-11-28T03:08:17.699910+0000","last_clean":"2023-11-28T03:08:17.699910+0000","last_became_active":"2023-11-28T03:08:17.698276+0000","last_became_peered":"2023-11-28T03:08:17.698276+0000","last_unstale":"2023-11-28T03:08:17.699910+0000","last_undegraded":"2023-11-28T03:08:17.699910+0000","last_fullsized":"2023-11-28T03:08:17.699910+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.677193+0000","last_change":"2023-11-28T03:08:17.327535+0000","last_active":"2023-11-28T03:08:17.677193+0000","last_peered":"2023-11-28T03:08:17.677193+0000","last_clean":"2023-11-28T03:08:17.677193+0000","last_became_active":"2023-11-28T03:08:17.326680+0000","last_became_peered":"2023-11-28T03:08:17.326680+0000","last_unstale":"2023-11-28T03:08:17.677193+0000","last_undegraded":"2023-11-28T03:08:17.677193+0000","last_fullsized":"2023-11-28T03:08:17.677193+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.39","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689179+0000","last_change":"2023-11-28T03:08:17.689179+0000","last_active":"2023-11-28T03:08:17.689179+0000","last_peered":"2023-11-28T03:08:17.689179+0000","last_clean":"2023-11-28T03:08:17.689179+0000","last_became_active":"2023-11-28T03:08:17.688863+0000","last_became_peered":"2023-11-28T03:08:17.688863+0000","last_unstale":"2023-11-28T03:08:17.689179+0000","last_undegraded":"2023-11-28T03:08:17.689179+0000","last_fullsized":"2023-11-28T03:08:17.689179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3a","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.685686+0000","last_change":"2023-11-28T03:08:17.335278+0000","last_active":"2023-11-28T03:08:17.685686+0000","last_peered":"2023-11-28T03:08:17.685686+0000","last_clean":"2023-11-28T03:08:17.685686+0000","last_became_active":"2023-11-28T03:08:17.334949+0000","last_became_peered":"2023-11-28T03:08:17.334949+0000","last_unstale":"2023-11-28T03:08:17.685686+0000","last_undegraded":"2023-11-28T03:08:17.685686+0000","last_fullsized":"2023-11-28T03:08:17.685686+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.3a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699242+0000","last_change":"2023-11-28T03:08:17.699242+0000","last_active":"2023-11-28T03:08:17.699242+0000","last_peered":"2023-11-28T03:08:17.699242+0000","last_clean":"2023-11-28T03:08:17.699242+0000","last_became_active":"2023-11-28T03:08:17.698989+0000","last_became_peered":"2023-11-28T03:08:17.698989+0000","last_unstale":"2023-11-28T03:08:17.699242+0000","last_undegraded":"2023-11-28T03:08:17.699242+0000","last_fullsized":"2023-11-28T03:08:17.699242+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334507+0000","last_change":"2023-11-28T03:08:17.326672+0000","last_active":"2023-11-28T03:08:17.334507+0000","last_peered":"2023-11-28T03:08:17.334507+0000","last_clean":"2023-11-28T03:08:17.334507+0000","last_became_active":"2023-11-28T03:08:17.326378+0000","last_became_peered":"2023-11-28T03:08:17.326378+0000","last_unstale":"2023-11-28T03:08:17.334507+0000","last_undegraded":"2023-11-28T03:08:17.334507+0000","last_fullsized":"2023-11-28T03:08:17.334507+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.682586+0000","last_change":"2023-11-28T03:08:17.682586+0000","last_active":"2023-11-28T03:08:17.682586+0000","last_peered":"2023-11-28T03:08:17.682586+0000","last_clean":"2023-11-28T03:08:17.682586+0000","last_became_active":"2023-11-28T03:08:17.682180+0000","last_became_peered":"2023-11-28T03:08:17.682180+0000","last_unstale":"2023-11-28T03:08:17.682586+0000","last_undegraded":"2023-11-28T03:08:17.682586+0000","last_fullsized":"2023-11-28T03:08:17.682586+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333692+0000","last_change":"2023-11-28T03:08:17.330662+0000","last_active":"2023-11-28T03:08:17.333692+0000","last_peered":"2023-11-28T03:08:17.333692+0000","last_clean":"2023-11-28T03:08:17.333692+0000","last_became_active":"2023-11-28T03:08:17.330127+0000","last_became_peered":"2023-11-28T03:08:17.330127+0000","last_unstale":"2023-11-28T03:08:17.333692+0000","last_undegraded":"2023-11-28T03:08:17.333692+0000","last_fullsized":"2023-11-28T03:08:17.333692+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.34","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.677889+0000","last_change":"2023-11-28T03:08:17.677889+0000","last_active":"2023-11-28T03:08:17.677889+0000","last_peered":"2023-11-28T03:08:17.677889+0000","last_clean":"2023-11-28T03:08:17.677889+0000","last_became_active":"2023-11-28T03:08:17.677603+0000","last_became_peered":"2023-11-28T03:08:17.677603+0000","last_unstale":"2023-11-28T03:08:17.677889+0000","last_undegraded":"2023-11-28T03:08:17.677889+0000","last_fullsized":"2023-11-28T03:08:17.677889+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331539+0000","last_change":"2023-11-28T03:08:17.326579+0000","last_active":"2023-11-28T03:08:17.331539+0000","last_peered":"2023-11-28T03:08:17.331539+0000","last_clean":"2023-11-28T03:08:17.331539+0000","last_became_active":"2023-11-28T03:08:17.326259+0000","last_became_peered":"2023-11-28T03:08:17.326259+0000","last_unstale":"2023-11-28T03:08:17.331539+0000","last_undegraded":"2023-11-28T03:08:17.331539+0000","last_fullsized":"2023-11-28T03:08:17.331539+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.35","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691613+0000","last_change":"2023-11-28T03:08:17.691613+0000","last_active":"2023-11-28T03:08:17.691613+0000","last_peered":"2023-11-28T03:08:17.691613+0000","last_clean":"2023-11-28T03:08:17.691613+0000","last_became_active":"2023-11-28T03:08:17.691254+0000","last_became_peered":"2023-11-28T03:08:17.691254+0000","last_unstale":"2023-11-28T03:08:17.691613+0000","last_undegraded":"2023-11-28T03:08:17.691613+0000","last_fullsized":"2023-11-28T03:08:17.691613+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.329137+0000","last_change":"2023-11-28T03:08:17.324430+0000","last_active":"2023-11-28T03:08:17.329137+0000","last_peered":"2023-11-28T03:08:17.329137+0000","last_clean":"2023-11-28T03:08:17.329137+0000","last_became_active":"2023-11-28T03:08:17.324194+0000","last_became_peered":"2023-11-28T03:08:17.324194+0000","last_unstale":"2023-11-28T03:08:17.329137+0000","last_undegraded":"2023-11-28T03:08:17.329137+0000","last_fullsized":"2023-11-28T03:08:17.329137+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.36","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699389+0000","last_change":"2023-11-28T03:08:17.699389+0000","last_active":"2023-11-28T03:08:17.699389+0000","last_peered":"2023-11-28T03:08:17.699389+0000","last_clean":"2023-11-28T03:08:17.699389+0000","last_became_active":"2023-11-28T03:08:17.699132+0000","last_became_peered":"2023-11-28T03:08:17.699132+0000","last_unstale":"2023-11-28T03:08:17.699389+0000","last_undegraded":"2023-11-28T03:08:17.699389+0000","last_fullsized":"2023-11-28T03:08:17.699389+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332895+0000","last_change":"2023-11-28T03:08:17.329005+0000","last_active":"2023-11-28T03:08:17.332895+0000","last_peered":"2023-11-28T03:08:17.332895+0000","last_clean":"2023-11-28T03:08:17.332895+0000","last_became_active":"2023-11-28T03:08:17.328739+0000","last_became_peered":"2023-11-28T03:08:17.328739+0000","last_unstale":"2023-11-28T03:08:17.332895+0000","last_undegraded":"2023-11-28T03:08:17.332895+0000","last_fullsized":"2023-11-28T03:08:17.332895+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.37","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692140+0000","last_change":"2023-11-28T03:08:17.692140+0000","last_active":"2023-11-28T03:08:17.692140+0000","last_peered":"2023-11-28T03:08:17.692140+0000","last_clean":"2023-11-28T03:08:17.692140+0000","last_became_active":"2023-11-28T03:08:17.691779+0000","last_became_peered":"2023-11-28T03:08:17.691779+0000","last_unstale":"2023-11-28T03:08:17.692140+0000","last_undegraded":"2023-11-28T03:08:17.692140+0000","last_fullsized":"2023-11-28T03:08:17.692140+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.34","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698325+0000","last_change":"2023-11-28T03:08:17.324426+0000","last_active":"2023-11-28T03:08:17.698325+0000","last_peered":"2023-11-28T03:08:17.698325+0000","last_clean":"2023-11-28T03:08:17.698325+0000","last_became_active":"2023-11-28T03:08:17.324079+0000","last_became_peered":"2023-11-28T03:08:17.324079+0000","last_unstale":"2023-11-28T03:08:17.698325+0000","last_undegraded":"2023-11-28T03:08:17.698325+0000","last_fullsized":"2023-11-28T03:08:17.698325+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.30","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689350+0000","last_change":"2023-11-28T03:08:17.689350+0000","last_active":"2023-11-28T03:08:17.689350+0000","last_peered":"2023-11-28T03:08:17.689350+0000","last_clean":"2023-11-28T03:08:17.689350+0000","last_became_active":"2023-11-28T03:08:17.689067+0000","last_became_peered":"2023-11-28T03:08:17.689067+0000","last_unstale":"2023-11-28T03:08:17.689350+0000","last_undegraded":"2023-11-28T03:08:17.689350+0000","last_fullsized":"2023-11-28T03:08:17.689350+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.680660+0000","last_change":"2023-11-28T03:08:17.327879+0000","last_active":"2023-11-28T03:08:17.680660+0000","last_peered":"2023-11-28T03:08:17.680660+0000","last_clean":"2023-11-28T03:08:17.680660+0000","last_became_active":"2023-11-28T03:08:17.327219+0000","last_became_peered":"2023-11-28T03:08:17.327219+0000","last_unstale":"2023-11-28T03:08:17.680660+0000","last_undegraded":"2023-11-28T03:08:17.680660+0000","last_fullsized":"2023-11-28T03:08:17.680660+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.31","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693594+0000","last_change":"2023-11-28T03:08:17.693594+0000","last_active":"2023-11-28T03:08:17.693594+0000","last_peered":"2023-11-28T03:08:17.693594+0000","last_clean":"2023-11-28T03:08:17.693594+0000","last_became_active":"2023-11-28T03:08:17.691337+0000","last_became_peered":"2023-11-28T03:08:17.691337+0000","last_unstale":"2023-11-28T03:08:17.693594+0000","last_undegraded":"2023-11-28T03:08:17.693594+0000","last_fullsized":"2023-11-28T03:08:17.693594+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334600+0000","last_change":"2023-11-28T03:08:17.329820+0000","last_active":"2023-11-28T03:08:17.334600+0000","last_peered":"2023-11-28T03:08:17.334600+0000","last_clean":"2023-11-28T03:08:17.334600+0000","last_became_active":"2023-11-28T03:08:17.329538+0000","last_became_peered":"2023-11-28T03:08:17.329538+0000","last_unstale":"2023-11-28T03:08:17.334600+0000","last_undegraded":"2023-11-28T03:08:17.334600+0000","last_fullsized":"2023-11-28T03:08:17.334600+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.32","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700412+0000","last_change":"2023-11-28T03:08:17.700412+0000","last_active":"2023-11-28T03:08:17.700412+0000","last_peered":"2023-11-28T03:08:17.700412+0000","last_clean":"2023-11-28T03:08:17.700412+0000","last_became_active":"2023-11-28T03:08:17.700054+0000","last_became_peered":"2023-11-28T03:08:17.700054+0000","last_unstale":"2023-11-28T03:08:17.700412+0000","last_undegraded":"2023-11-28T03:08:17.700412+0000","last_fullsized":"2023-11-28T03:08:17.700412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.327834+0000","last_change":"2023-11-28T03:08:17.326770+0000","last_active":"2023-11-28T03:08:17.327834+0000","last_peered":"2023-11-28T03:08:17.327834+0000","last_clean":"2023-11-28T03:08:17.327834+0000","last_became_active":"2023-11-28T03:08:17.326521+0000","last_became_peered":"2023-11-28T03:08:17.326521+0000","last_unstale":"2023-11-28T03:08:17.327834+0000","last_undegraded":"2023-11-28T03:08:17.327834+0000","last_fullsized":"2023-11-28T03:08:17.327834+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.33","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698686+0000","last_change":"2023-11-28T03:08:17.698686+0000","last_active":"2023-11-28T03:08:17.698686+0000","last_peered":"2023-11-28T03:08:17.698686+0000","last_clean":"2023-11-28T03:08:17.698686+0000","last_became_active":"2023-11-28T03:08:17.698339+0000","last_became_peered":"2023-11-28T03:08:17.698339+0000","last_unstale":"2023-11-28T03:08:17.698686+0000","last_undegraded":"2023-11-28T03:08:17.698686+0000","last_fullsized":"2023-11-28T03:08:17.698686+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.30","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688996+0000","last_change":"2023-11-28T03:08:17.330988+0000","last_active":"2023-11-28T03:08:17.688996+0000","last_peered":"2023-11-28T03:08:17.688996+0000","last_clean":"2023-11-28T03:08:17.688996+0000","last_became_active":"2023-11-28T03:08:17.330687+0000","last_became_peered":"2023-11-28T03:08:17.330687+0000","last_unstale":"2023-11-28T03:08:17.688996+0000","last_undegraded":"2023-11-28T03:08:17.688996+0000","last_fullsized":"2023-11-28T03:08:17.688996+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696541+0000","last_change":"2023-11-28T03:08:17.696541+0000","last_active":"2023-11-28T03:08:17.696541+0000","last_peered":"2023-11-28T03:08:17.696541+0000","last_clean":"2023-11-28T03:08:17.696541+0000","last_became_active":"2023-11-28T03:08:17.696208+0000","last_became_peered":"2023-11-28T03:08:17.696208+0000","last_unstale":"2023-11-28T03:08:17.696541+0000","last_undegraded":"2023-11-28T03:08:17.696541+0000","last_fullsized":"2023-11-28T03:08:17.696541+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.325946+0000","last_change":"2023-11-28T03:08:17.320522+0000","last_active":"2023-11-28T03:08:17.325946+0000","last_peered":"2023-11-28T03:08:17.325946+0000","last_clean":"2023-11-28T03:08:17.325946+0000","last_became_active":"2023-11-28T03:08:17.320226+0000","last_became_peered":"2023-11-28T03:08:17.320226+0000","last_unstale":"2023-11-28T03:08:17.325946+0000","last_undegraded":"2023-11-28T03:08:17.325946+0000","last_fullsized":"2023-11-28T03:08:17.325946+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692518+0000","last_change":"2023-11-28T03:08:17.692518+0000","last_active":"2023-11-28T03:08:17.692518+0000","last_peered":"2023-11-28T03:08:17.692518+0000","last_clean":"2023-11-28T03:08:17.692518+0000","last_became_active":"2023-11-28T03:08:17.692234+0000","last_became_peered":"2023-11-28T03:08:17.692234+0000","last_unstale":"2023-11-28T03:08:17.692518+0000","last_undegraded":"2023-11-28T03:08:17.692518+0000","last_fullsized":"2023-11-28T03:08:17.692518+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334457+0000","last_change":"2023-11-28T03:08:17.320821+0000","last_active":"2023-11-28T03:08:17.334457+0000","last_peered":"2023-11-28T03:08:17.334457+0000","last_clean":"2023-11-28T03:08:17.334457+0000","last_became_active":"2023-11-28T03:08:17.320551+0000","last_became_peered":"2023-11-28T03:08:17.320551+0000","last_unstale":"2023-11-28T03:08:17.334457+0000","last_undegraded":"2023-11-28T03:08:17.334457+0000","last_fullsized":"2023-11-28T03:08:17.334457+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.2e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692045+0000","last_change":"2023-11-28T03:08:17.692045+0000","last_active":"2023-11-28T03:08:17.692045+0000","last_peered":"2023-11-28T03:08:17.692045+0000","last_clean":"2023-11-28T03:08:17.692045+0000","last_became_active":"2023-11-28T03:08:17.691771+0000","last_became_peered":"2023-11-28T03:08:17.691771+0000","last_unstale":"2023-11-28T03:08:17.692045+0000","last_undegraded":"2023-11-28T03:08:17.692045+0000","last_fullsized":"2023-11-28T03:08:17.692045+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2d","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.687002+0000","last_change":"2023-11-28T03:08:17.325949+0000","last_active":"2023-11-28T03:08:17.687002+0000","last_peered":"2023-11-28T03:08:17.687002+0000","last_clean":"2023-11-28T03:08:17.687002+0000","last_became_active":"2023-11-28T03:08:17.325673+0000","last_became_peered":"2023-11-28T03:08:17.325673+0000","last_unstale":"2023-11-28T03:08:17.687002+0000","last_undegraded":"2023-11-28T03:08:17.687002+0000","last_fullsized":"2023-11-28T03:08:17.687002+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.2f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689904+0000","last_change":"2023-11-28T03:08:17.689904+0000","last_active":"2023-11-28T03:08:17.689904+0000","last_peered":"2023-11-28T03:08:17.689904+0000","last_clean":"2023-11-28T03:08:17.689904+0000","last_became_active":"2023-11-28T03:08:17.689592+0000","last_became_peered":"2023-11-28T03:08:17.689592+0000","last_unstale":"2023-11-28T03:08:17.689904+0000","last_undegraded":"2023-11-28T03:08:17.689904+0000","last_fullsized":"2023-11-28T03:08:17.689904+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.330963+0000","last_change":"2023-11-28T03:08:17.318976+0000","last_active":"2023-11-28T03:08:17.330963+0000","last_peered":"2023-11-28T03:08:17.330963+0000","last_clean":"2023-11-28T03:08:17.330963+0000","last_became_active":"2023-11-28T03:08:17.318717+0000","last_became_peered":"2023-11-28T03:08:17.318717+0000","last_unstale":"2023-11-28T03:08:17.330963+0000","last_undegraded":"2023-11-28T03:08:17.330963+0000","last_fullsized":"2023-11-28T03:08:17.330963+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.28","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688044+0000","last_change":"2023-11-28T03:08:17.688044+0000","last_active":"2023-11-28T03:08:17.688044+0000","last_peered":"2023-11-28T03:08:17.688044+0000","last_clean":"2023-11-28T03:08:17.688044+0000","last_became_active":"2023-11-28T03:08:17.687797+0000","last_became_peered":"2023-11-28T03:08:17.687797+0000","last_unstale":"2023-11-28T03:08:17.688044+0000","last_undegraded":"2023-11-28T03:08:17.688044+0000","last_fullsized":"2023-11-28T03:08:17.688044+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.672731+0000","last_change":"2023-11-28T03:08:17.317207+0000","last_active":"2023-11-28T03:08:17.672731+0000","last_peered":"2023-11-28T03:08:17.672731+0000","last_clean":"2023-11-28T03:08:17.672731+0000","last_became_active":"2023-11-28T03:08:17.316926+0000","last_became_peered":"2023-11-28T03:08:17.316926+0000","last_unstale":"2023-11-28T03:08:17.672731+0000","last_undegraded":"2023-11-28T03:08:17.672731+0000","last_fullsized":"2023-11-28T03:08:17.672731+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.29","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696078+0000","last_change":"2023-11-28T03:08:17.696078+0000","last_active":"2023-11-28T03:08:17.696078+0000","last_peered":"2023-11-28T03:08:17.696078+0000","last_clean":"2023-11-28T03:08:17.696078+0000","last_became_active":"2023-11-28T03:08:17.695791+0000","last_became_peered":"2023-11-28T03:08:17.695791+0000","last_unstale":"2023-11-28T03:08:17.696078+0000","last_undegraded":"2023-11-28T03:08:17.696078+0000","last_fullsized":"2023-11-28T03:08:17.696078+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332843+0000","last_change":"2023-11-28T03:08:17.327494+0000","last_active":"2023-11-28T03:08:17.332843+0000","last_peered":"2023-11-28T03:08:17.332843+0000","last_clean":"2023-11-28T03:08:17.332843+0000","last_became_active":"2023-11-28T03:08:17.325301+0000","last_became_peered":"2023-11-28T03:08:17.325301+0000","last_unstale":"2023-11-28T03:08:17.332843+0000","last_undegraded":"2023-11-28T03:08:17.332843+0000","last_fullsized":"2023-11-28T03:08:17.332843+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.2a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699821+0000","last_change":"2023-11-28T03:08:17.699821+0000","last_active":"2023-11-28T03:08:17.699821+0000","last_peered":"2023-11-28T03:08:17.699821+0000","last_clean":"2023-11-28T03:08:17.699821+0000","last_became_active":"2023-11-28T03:08:17.699518+0000","last_became_peered":"2023-11-28T03:08:17.699518+0000","last_unstale":"2023-11-28T03:08:17.699821+0000","last_undegraded":"2023-11-28T03:08:17.699821+0000","last_fullsized":"2023-11-28T03:08:17.699821+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334202+0000","last_change":"2023-11-28T03:08:17.329714+0000","last_active":"2023-11-28T03:08:17.334202+0000","last_peered":"2023-11-28T03:08:17.334202+0000","last_clean":"2023-11-28T03:08:17.334202+0000","last_became_active":"2023-11-28T03:08:17.328666+0000","last_became_peered":"2023-11-28T03:08:17.328666+0000","last_unstale":"2023-11-28T03:08:17.334202+0000","last_undegraded":"2023-11-28T03:08:17.334202+0000","last_fullsized":"2023-11-28T03:08:17.334202+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.2b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691371+0000","last_change":"2023-11-28T03:08:17.691371+0000","last_active":"2023-11-28T03:08:17.691371+0000","last_peered":"2023-11-28T03:08:17.691371+0000","last_clean":"2023-11-28T03:08:17.691371+0000","last_became_active":"2023-11-28T03:08:17.690837+0000","last_became_peered":"2023-11-28T03:08:17.690837+0000","last_unstale":"2023-11-28T03:08:17.691371+0000","last_undegraded":"2023-11-28T03:08:17.691371+0000","last_fullsized":"2023-11-28T03:08:17.691371+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.319610+0000","last_change":"2023-11-28T03:08:17.317280+0000","last_active":"2023-11-28T03:08:17.319610+0000","last_peered":"2023-11-28T03:08:17.319610+0000","last_clean":"2023-11-28T03:08:17.319610+0000","last_became_active":"2023-11-28T03:08:17.317000+0000","last_became_peered":"2023-11-28T03:08:17.317000+0000","last_unstale":"2023-11-28T03:08:17.319610+0000","last_undegraded":"2023-11-28T03:08:17.319610+0000","last_fullsized":"2023-11-28T03:08:17.319610+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.24","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.696491+0000","last_change":"2023-11-28T03:08:17.696491+0000","last_active":"2023-11-28T03:08:17.696491+0000","last_peered":"2023-11-28T03:08:17.696491+0000","last_clean":"2023-11-28T03:08:17.696491+0000","last_became_active":"2023-11-28T03:08:17.696176+0000","last_became_peered":"2023-11-28T03:08:17.696176+0000","last_unstale":"2023-11-28T03:08:17.696491+0000","last_undegraded":"2023-11-28T03:08:17.696491+0000","last_fullsized":"2023-11-28T03:08:17.696491+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332950+0000","last_change":"2023-11-28T03:08:17.329586+0000","last_active":"2023-11-28T03:08:17.332950+0000","last_peered":"2023-11-28T03:08:17.332950+0000","last_clean":"2023-11-28T03:08:17.332950+0000","last_became_active":"2023-11-28T03:08:17.329312+0000","last_became_peered":"2023-11-28T03:08:17.329312+0000","last_unstale":"2023-11-28T03:08:17.332950+0000","last_undegraded":"2023-11-28T03:08:17.332950+0000","last_fullsized":"2023-11-28T03:08:17.332950+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.25","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.678354+0000","last_change":"2023-11-28T03:08:17.678354+0000","last_active":"2023-11-28T03:08:17.678354+0000","last_peered":"2023-11-28T03:08:17.678354+0000","last_clean":"2023-11-28T03:08:17.678354+0000","last_became_active":"2023-11-28T03:08:17.678105+0000","last_became_peered":"2023-11-28T03:08:17.678105+0000","last_unstale":"2023-11-28T03:08:17.678354+0000","last_undegraded":"2023-11-28T03:08:17.678354+0000","last_fullsized":"2023-11-28T03:08:17.678354+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331726+0000","last_change":"2023-11-28T03:08:17.329723+0000","last_active":"2023-11-28T03:08:17.331726+0000","last_peered":"2023-11-28T03:08:17.331726+0000","last_clean":"2023-11-28T03:08:17.331726+0000","last_became_active":"2023-11-28T03:08:17.329435+0000","last_became_peered":"2023-11-28T03:08:17.329435+0000","last_unstale":"2023-11-28T03:08:17.331726+0000","last_undegraded":"2023-11-28T03:08:17.331726+0000","last_fullsized":"2023-11-28T03:08:17.331726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.26","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699179+0000","last_change":"2023-11-28T03:08:17.699179+0000","last_active":"2023-11-28T03:08:17.699179+0000","last_peered":"2023-11-28T03:08:17.699179+0000","last_clean":"2023-11-28T03:08:17.699179+0000","last_became_active":"2023-11-28T03:08:17.698858+0000","last_became_peered":"2023-11-28T03:08:17.698858+0000","last_unstale":"2023-11-28T03:08:17.699179+0000","last_undegraded":"2023-11-28T03:08:17.699179+0000","last_fullsized":"2023-11-28T03:08:17.699179+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326280+0000","last_change":"2023-11-28T03:08:17.322898+0000","last_active":"2023-11-28T03:08:17.326280+0000","last_peered":"2023-11-28T03:08:17.326280+0000","last_clean":"2023-11-28T03:08:17.326280+0000","last_became_active":"2023-11-28T03:08:17.322234+0000","last_became_peered":"2023-11-28T03:08:17.322234+0000","last_unstale":"2023-11-28T03:08:17.326280+0000","last_undegraded":"2023-11-28T03:08:17.326280+0000","last_fullsized":"2023-11-28T03:08:17.326280+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.27","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694550+0000","last_change":"2023-11-28T03:08:17.694550+0000","last_active":"2023-11-28T03:08:17.694550+0000","last_peered":"2023-11-28T03:08:17.694550+0000","last_clean":"2023-11-28T03:08:17.694550+0000","last_became_active":"2023-11-28T03:08:17.694140+0000","last_became_peered":"2023-11-28T03:08:17.694140+0000","last_unstale":"2023-11-28T03:08:17.694550+0000","last_undegraded":"2023-11-28T03:08:17.694550+0000","last_fullsized":"2023-11-28T03:08:17.694550+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334187+0000","last_change":"2023-11-28T03:08:17.329333+0000","last_active":"2023-11-28T03:08:17.334187+0000","last_peered":"2023-11-28T03:08:17.334187+0000","last_clean":"2023-11-28T03:08:17.334187+0000","last_became_active":"2023-11-28T03:08:17.328737+0000","last_became_peered":"2023-11-28T03:08:17.328737+0000","last_unstale":"2023-11-28T03:08:17.334187+0000","last_undegraded":"2023-11-28T03:08:17.334187+0000","last_fullsized":"2023-11-28T03:08:17.334187+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.20","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.703616+0000","last_change":"2023-11-28T03:08:17.703616+0000","last_active":"2023-11-28T03:08:17.703616+0000","last_peered":"2023-11-28T03:08:17.703616+0000","last_clean":"2023-11-28T03:08:17.703616+0000","last_became_active":"2023-11-28T03:08:17.703339+0000","last_became_peered":"2023-11-28T03:08:17.703339+0000","last_unstale":"2023-11-28T03:08:17.703616+0000","last_undegraded":"2023-11-28T03:08:17.703616+0000","last_fullsized":"2023-11-28T03:08:17.703616+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332677+0000","last_change":"2023-11-28T03:08:17.326880+0000","last_active":"2023-11-28T03:08:17.332677+0000","last_peered":"2023-11-28T03:08:17.332677+0000","last_clean":"2023-11-28T03:08:17.332677+0000","last_became_active":"2023-11-28T03:08:17.326606+0000","last_became_peered":"2023-11-28T03:08:17.326606+0000","last_unstale":"2023-11-28T03:08:17.332677+0000","last_undegraded":"2023-11-28T03:08:17.332677+0000","last_fullsized":"2023-11-28T03:08:17.332677+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.21","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699944+0000","last_change":"2023-11-28T03:08:17.699944+0000","last_active":"2023-11-28T03:08:17.699944+0000","last_peered":"2023-11-28T03:08:17.699944+0000","last_clean":"2023-11-28T03:08:17.699944+0000","last_became_active":"2023-11-28T03:08:17.699649+0000","last_became_peered":"2023-11-28T03:08:17.699649+0000","last_unstale":"2023-11-28T03:08:17.699944+0000","last_undegraded":"2023-11-28T03:08:17.699944+0000","last_fullsized":"2023-11-28T03:08:17.699944+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333321+0000","last_change":"2023-11-28T03:08:17.332384+0000","last_active":"2023-11-28T03:08:17.333321+0000","last_peered":"2023-11-28T03:08:17.333321+0000","last_clean":"2023-11-28T03:08:17.333321+0000","last_became_active":"2023-11-28T03:08:17.332169+0000","last_became_peered":"2023-11-28T03:08:17.332169+0000","last_unstale":"2023-11-28T03:08:17.333321+0000","last_undegraded":"2023-11-28T03:08:17.333321+0000","last_fullsized":"2023-11-28T03:08:17.333321+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.22","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692147+0000","last_change":"2023-11-28T03:08:17.692147+0000","last_active":"2023-11-28T03:08:17.692147+0000","last_peered":"2023-11-28T03:08:17.692147+0000","last_clean":"2023-11-28T03:08:17.692147+0000","last_became_active":"2023-11-28T03:08:17.691847+0000","last_became_peered":"2023-11-28T03:08:17.691847+0000","last_unstale":"2023-11-28T03:08:17.692147+0000","last_undegraded":"2023-11-28T03:08:17.692147+0000","last_fullsized":"2023-11-28T03:08:17.692147+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334372+0000","last_change":"2023-11-28T03:08:17.327365+0000","last_active":"2023-11-28T03:08:17.334372+0000","last_peered":"2023-11-28T03:08:17.334372+0000","last_clean":"2023-11-28T03:08:17.334372+0000","last_became_active":"2023-11-28T03:08:17.327038+0000","last_became_peered":"2023-11-28T03:08:17.327038+0000","last_unstale":"2023-11-28T03:08:17.334372+0000","last_undegraded":"2023-11-28T03:08:17.334372+0000","last_fullsized":"2023-11-28T03:08:17.334372+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.23","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689215+0000","last_change":"2023-11-28T03:08:17.689215+0000","last_active":"2023-11-28T03:08:17.689215+0000","last_peered":"2023-11-28T03:08:17.689215+0000","last_clean":"2023-11-28T03:08:17.689215+0000","last_became_active":"2023-11-28T03:08:17.688901+0000","last_became_peered":"2023-11-28T03:08:17.688901+0000","last_unstale":"2023-11-28T03:08:17.689215+0000","last_undegraded":"2023-11-28T03:08:17.689215+0000","last_fullsized":"2023-11-28T03:08:17.689215+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.327827+0000","last_change":"2023-11-28T03:08:17.321360+0000","last_active":"2023-11-28T03:08:17.327827+0000","last_peered":"2023-11-28T03:08:17.327827+0000","last_clean":"2023-11-28T03:08:17.327827+0000","last_became_active":"2023-11-28T03:08:17.321046+0000","last_became_peered":"2023-11-28T03:08:17.321046+0000","last_unstale":"2023-11-28T03:08:17.327827+0000","last_undegraded":"2023-11-28T03:08:17.327827+0000","last_fullsized":"2023-11-28T03:08:17.327827+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691186+0000","last_change":"2023-11-28T03:08:17.691186+0000","last_active":"2023-11-28T03:08:17.691186+0000","last_peered":"2023-11-28T03:08:17.691186+0000","last_clean":"2023-11-28T03:08:17.691186+0000","last_became_active":"2023-11-28T03:08:17.690911+0000","last_became_peered":"2023-11-28T03:08:17.690911+0000","last_unstale":"2023-11-28T03:08:17.691186+0000","last_undegraded":"2023-11-28T03:08:17.691186+0000","last_fullsized":"2023-11-28T03:08:17.691186+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1f","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676720+0000","last_change":"2023-11-28T03:08:17.325001+0000","last_active":"2023-11-28T03:08:17.676720+0000","last_peered":"2023-11-28T03:08:17.676720+0000","last_clean":"2023-11-28T03:08:17.676720+0000","last_became_active":"2023-11-28T03:08:17.324713+0000","last_became_peered":"2023-11-28T03:08:17.324713+0000","last_unstale":"2023-11-28T03:08:17.676720+0000","last_undegraded":"2023-11-28T03:08:17.676720+0000","last_fullsized":"2023-11-28T03:08:17.676720+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.1d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692984+0000","last_change":"2023-11-28T03:08:17.692984+0000","last_active":"2023-11-28T03:08:17.692984+0000","last_peered":"2023-11-28T03:08:17.692984+0000","last_clean":"2023-11-28T03:08:17.692984+0000","last_became_active":"2023-11-28T03:08:17.692350+0000","last_became_peered":"2023-11-28T03:08:17.692350+0000","last_unstale":"2023-11-28T03:08:17.692984+0000","last_undegraded":"2023-11-28T03:08:17.692984+0000","last_fullsized":"2023-11-28T03:08:17.692984+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1e","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676916+0000","last_change":"2023-11-28T03:08:17.331853+0000","last_active":"2023-11-28T03:08:17.676916+0000","last_peered":"2023-11-28T03:08:17.676916+0000","last_clean":"2023-11-28T03:08:17.676916+0000","last_became_active":"2023-11-28T03:08:17.331353+0000","last_became_peered":"2023-11-28T03:08:17.331353+0000","last_unstale":"2023-11-28T03:08:17.676916+0000","last_undegraded":"2023-11-28T03:08:17.676916+0000","last_fullsized":"2023-11-28T03:08:17.676916+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699336+0000","last_change":"2023-11-28T03:08:17.699336+0000","last_active":"2023-11-28T03:08:17.699336+0000","last_peered":"2023-11-28T03:08:17.699336+0000","last_clean":"2023-11-28T03:08:17.699336+0000","last_became_active":"2023-11-28T03:08:17.699037+0000","last_became_peered":"2023-11-28T03:08:17.699037+0000","last_unstale":"2023-11-28T03:08:17.699336+0000","last_undegraded":"2023-11-28T03:08:17.699336+0000","last_fullsized":"2023-11-28T03:08:17.699336+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.1d","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688939+0000","last_change":"2023-11-28T03:08:17.335439+0000","last_active":"2023-11-28T03:08:17.688939+0000","last_peered":"2023-11-28T03:08:17.688939+0000","last_clean":"2023-11-28T03:08:17.688939+0000","last_became_active":"2023-11-28T03:08:17.335090+0000","last_became_peered":"2023-11-28T03:08:17.335090+0000","last_unstale":"2023-11-28T03:08:17.688939+0000","last_undegraded":"2023-11-28T03:08:17.688939+0000","last_fullsized":"2023-11-28T03:08:17.688939+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694015+0000","last_change":"2023-11-28T03:08:17.694015+0000","last_active":"2023-11-28T03:08:17.694015+0000","last_peered":"2023-11-28T03:08:17.694015+0000","last_clean":"2023-11-28T03:08:17.694015+0000","last_became_active":"2023-11-28T03:08:17.693750+0000","last_became_peered":"2023-11-28T03:08:17.693750+0000","last_unstale":"2023-11-28T03:08:17.694015+0000","last_undegraded":"2023-11-28T03:08:17.694015+0000","last_fullsized":"2023-11-28T03:08:17.694015+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334318+0000","last_change":"2023-11-28T03:08:17.330606+0000","last_active":"2023-11-28T03:08:17.334318+0000","last_peered":"2023-11-28T03:08:17.334318+0000","last_clean":"2023-11-28T03:08:17.334318+0000","last_became_active":"2023-11-28T03:08:17.329165+0000","last_became_peered":"2023-11-28T03:08:17.329165+0000","last_unstale":"2023-11-28T03:08:17.334318+0000","last_undegraded":"2023-11-28T03:08:17.334318+0000","last_fullsized":"2023-11-28T03:08:17.334318+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700623+0000","last_change":"2023-11-28T03:08:17.700623+0000","last_active":"2023-11-28T03:08:17.700623+0000","last_peered":"2023-11-28T03:08:17.700623+0000","last_clean":"2023-11-28T03:08:17.700623+0000","last_became_active":"2023-11-28T03:08:17.700089+0000","last_became_peered":"2023-11-28T03:08:17.700089+0000","last_unstale":"2023-11-28T03:08:17.700623+0000","last_undegraded":"2023-11-28T03:08:17.700623+0000","last_fullsized":"2023-11-28T03:08:17.700623+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334551+0000","last_change":"2023-11-28T03:08:17.330392+0000","last_active":"2023-11-28T03:08:17.334551+0000","last_peered":"2023-11-28T03:08:17.334551+0000","last_clean":"2023-11-28T03:08:17.334551+0000","last_became_active":"2023-11-28T03:08:17.330090+0000","last_became_peered":"2023-11-28T03:08:17.330090+0000","last_unstale":"2023-11-28T03:08:17.334551+0000","last_undegraded":"2023-11-28T03:08:17.334551+0000","last_fullsized":"2023-11-28T03:08:17.334551+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.686378+0000","last_change":"2023-11-28T03:08:17.686378+0000","last_active":"2023-11-28T03:08:17.686378+0000","last_peered":"2023-11-28T03:08:17.686378+0000","last_clean":"2023-11-28T03:08:17.686378+0000","last_became_active":"2023-11-28T03:08:17.686142+0000","last_became_peered":"2023-11-28T03:08:17.686142+0000","last_unstale":"2023-11-28T03:08:17.686378+0000","last_undegraded":"2023-11-28T03:08:17.686378+0000","last_fullsized":"2023-11-28T03:08:17.686378+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333091+0000","last_change":"2023-11-28T03:08:17.330388+0000","last_active":"2023-11-28T03:08:17.333091+0000","last_peered":"2023-11-28T03:08:17.333091+0000","last_clean":"2023-11-28T03:08:17.333091+0000","last_became_active":"2023-11-28T03:08:17.330136+0000","last_became_peered":"2023-11-28T03:08:17.330136+0000","last_unstale":"2023-11-28T03:08:17.333091+0000","last_undegraded":"2023-11-28T03:08:17.333091+0000","last_fullsized":"2023-11-28T03:08:17.333091+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699196+0000","last_change":"2023-11-28T03:08:17.699196+0000","last_active":"2023-11-28T03:08:17.699196+0000","last_peered":"2023-11-28T03:08:17.699196+0000","last_clean":"2023-11-28T03:08:17.699196+0000","last_became_active":"2023-11-28T03:08:17.698927+0000","last_became_peered":"2023-11-28T03:08:17.698927+0000","last_unstale":"2023-11-28T03:08:17.699196+0000","last_undegraded":"2023-11-28T03:08:17.699196+0000","last_fullsized":"2023-11-28T03:08:17.699196+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326994+0000","last_change":"2023-11-28T03:08:17.324775+0000","last_active":"2023-11-28T03:08:17.326994+0000","last_peered":"2023-11-28T03:08:17.326994+0000","last_clean":"2023-11-28T03:08:17.326994+0000","last_became_active":"2023-11-28T03:08:17.324509+0000","last_became_peered":"2023-11-28T03:08:17.324509+0000","last_unstale":"2023-11-28T03:08:17.326994+0000","last_undegraded":"2023-11-28T03:08:17.326994+0000","last_fullsized":"2023-11-28T03:08:17.326994+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698888+0000","last_change":"2023-11-28T03:08:17.698888+0000","last_active":"2023-11-28T03:08:17.698888+0000","last_peered":"2023-11-28T03:08:17.698888+0000","last_clean":"2023-11-28T03:08:17.698888+0000","last_became_active":"2023-11-28T03:08:17.698263+0000","last_became_peered":"2023-11-28T03:08:17.698263+0000","last_unstale":"2023-11-28T03:08:17.698888+0000","last_undegraded":"2023-11-28T03:08:17.698888+0000","last_fullsized":"2023-11-28T03:08:17.698888+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.329381+0000","last_change":"2023-11-28T03:08:17.328221+0000","last_active":"2023-11-28T03:08:17.329381+0000","last_peered":"2023-11-28T03:08:17.329381+0000","last_clean":"2023-11-28T03:08:17.329381+0000","last_became_active":"2023-11-28T03:08:17.327911+0000","last_became_peered":"2023-11-28T03:08:17.327911+0000","last_unstale":"2023-11-28T03:08:17.329381+0000","last_undegraded":"2023-11-28T03:08:17.329381+0000","last_fullsized":"2023-11-28T03:08:17.329381+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.684968+0000","last_change":"2023-11-28T03:08:17.684968+0000","last_active":"2023-11-28T03:08:17.684968+0000","last_peered":"2023-11-28T03:08:17.684968+0000","last_clean":"2023-11-28T03:08:17.684968+0000","last_became_active":"2023-11-28T03:08:17.682305+0000","last_became_peered":"2023-11-28T03:08:17.682305+0000","last_unstale":"2023-11-28T03:08:17.684968+0000","last_undegraded":"2023-11-28T03:08:17.684968+0000","last_fullsized":"2023-11-28T03:08:17.684968+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.336356+0000","last_change":"2023-11-28T03:08:17.335193+0000","last_active":"2023-11-28T03:08:17.336356+0000","last_peered":"2023-11-28T03:08:17.336356+0000","last_clean":"2023-11-28T03:08:17.336356+0000","last_became_active":"2023-11-28T03:08:17.334897+0000","last_became_peered":"2023-11-28T03:08:17.334897+0000","last_unstale":"2023-11-28T03:08:17.336356+0000","last_undegraded":"2023-11-28T03:08:17.336356+0000","last_fullsized":"2023-11-28T03:08:17.336356+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693595+0000","last_change":"2023-11-28T03:08:17.693595+0000","last_active":"2023-11-28T03:08:17.693595+0000","last_peered":"2023-11-28T03:08:17.693595+0000","last_clean":"2023-11-28T03:08:17.693595+0000","last_became_active":"2023-11-28T03:08:17.693278+0000","last_became_peered":"2023-11-28T03:08:17.693278+0000","last_unstale":"2023-11-28T03:08:17.693595+0000","last_undegraded":"2023-11-28T03:08:17.693595+0000","last_fullsized":"2023-11-28T03:08:17.693595+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331248+0000","last_change":"2023-11-28T03:08:17.328443+0000","last_active":"2023-11-28T03:08:17.331248+0000","last_peered":"2023-11-28T03:08:17.331248+0000","last_clean":"2023-11-28T03:08:17.331248+0000","last_became_active":"2023-11-28T03:08:17.328158+0000","last_became_peered":"2023-11-28T03:08:17.328158+0000","last_unstale":"2023-11-28T03:08:17.331248+0000","last_undegraded":"2023-11-28T03:08:17.331248+0000","last_fullsized":"2023-11-28T03:08:17.331248+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.688900+0000","last_change":"2023-11-28T03:08:17.688900+0000","last_active":"2023-11-28T03:08:17.688900+0000","last_peered":"2023-11-28T03:08:17.688900+0000","last_clean":"2023-11-28T03:08:17.688900+0000","last_became_active":"2023-11-28T03:08:17.688616+0000","last_became_peered":"2023-11-28T03:08:17.688616+0000","last_unstale":"2023-11-28T03:08:17.688900+0000","last_undegraded":"2023-11-28T03:08:17.688900+0000","last_fullsized":"2023-11-28T03:08:17.688900+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.15","version":"109'3","reported_seq":12,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.691726+0000","last_change":"2023-11-28T03:08:17.328759+0000","last_active":"2023-11-28T03:08:17.691726+0000","last_peered":"2023-11-28T03:08:17.691726+0000","last_clean":"2023-11-28T03:08:17.691726+0000","last_became_active":"2023-11-28T03:08:17.328106+0000","last_became_peered":"2023-11-28T03:08:17.328106+0000","last_unstale":"2023-11-28T03:08:17.691726+0000","last_undegraded":"2023-11-28T03:08:17.691726+0000","last_fullsized":"2023-11-28T03:08:17.691726+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698724+0000","last_change":"2023-11-28T03:08:17.698724+0000","last_active":"2023-11-28T03:08:17.698724+0000","last_peered":"2023-11-28T03:08:17.698724+0000","last_clean":"2023-11-28T03:08:17.698724+0000","last_became_active":"2023-11-28T03:08:17.698241+0000","last_became_peered":"2023-11-28T03:08:17.698241+0000","last_unstale":"2023-11-28T03:08:17.698724+0000","last_undegraded":"2023-11-28T03:08:17.698724+0000","last_fullsized":"2023-11-28T03:08:17.698724+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.14","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692871+0000","last_change":"2023-11-28T03:08:17.324551+0000","last_active":"2023-11-28T03:08:17.692871+0000","last_peered":"2023-11-28T03:08:17.692871+0000","last_clean":"2023-11-28T03:08:17.692871+0000","last_became_active":"2023-11-28T03:08:17.324289+0000","last_became_peered":"2023-11-28T03:08:17.324289+0000","last_unstale":"2023-11-28T03:08:17.692871+0000","last_undegraded":"2023-11-28T03:08:17.692871+0000","last_fullsized":"2023-11-28T03:08:17.692871+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.10","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.700778+0000","last_change":"2023-11-28T03:08:17.700778+0000","last_active":"2023-11-28T03:08:17.700778+0000","last_peered":"2023-11-28T03:08:17.700778+0000","last_clean":"2023-11-28T03:08:17.700778+0000","last_became_active":"2023-11-28T03:08:17.700375+0000","last_became_peered":"2023-11-28T03:08:17.700375+0000","last_unstale":"2023-11-28T03:08:17.700778+0000","last_undegraded":"2023-11-28T03:08:17.700778+0000","last_fullsized":"2023-11-28T03:08:17.700778+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.680984+0000","last_change":"2023-11-28T03:08:17.328067+0000","last_active":"2023-11-28T03:08:17.680984+0000","last_peered":"2023-11-28T03:08:17.680984+0000","last_clean":"2023-11-28T03:08:17.680984+0000","last_became_active":"2023-11-28T03:08:17.327612+0000","last_became_peered":"2023-11-28T03:08:17.327612+0000","last_unstale":"2023-11-28T03:08:17.680984+0000","last_undegraded":"2023-11-28T03:08:17.680984+0000","last_fullsized":"2023-11-28T03:08:17.680984+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.682593+0000","last_change":"2023-11-28T03:08:17.682593+0000","last_active":"2023-11-28T03:08:17.682593+0000","last_peered":"2023-11-28T03:08:17.682593+0000","last_clean":"2023-11-28T03:08:17.682593+0000","last_became_active":"2023-11-28T03:08:17.682030+0000","last_became_peered":"2023-11-28T03:08:17.682030+0000","last_unstale":"2023-11-28T03:08:17.682593+0000","last_undegraded":"2023-11-28T03:08:17.682593+0000","last_fullsized":"2023-11-28T03:08:17.682593+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.325683+0000","last_change":"2023-11-28T03:08:17.322614+0000","last_active":"2023-11-28T03:08:17.325683+0000","last_peered":"2023-11-28T03:08:17.325683+0000","last_clean":"2023-11-28T03:08:17.325683+0000","last_became_active":"2023-11-28T03:08:17.322309+0000","last_became_peered":"2023-11-28T03:08:17.322309+0000","last_unstale":"2023-11-28T03:08:17.325683+0000","last_undegraded":"2023-11-28T03:08:17.325683+0000","last_fullsized":"2023-11-28T03:08:17.325683+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699412+0000","last_change":"2023-11-28T03:08:17.699412+0000","last_active":"2023-11-28T03:08:17.699412+0000","last_peered":"2023-11-28T03:08:17.699412+0000","last_clean":"2023-11-28T03:08:17.699412+0000","last_became_active":"2023-11-28T03:08:17.699140+0000","last_became_peered":"2023-11-28T03:08:17.699140+0000","last_unstale":"2023-11-28T03:08:17.699412+0000","last_undegraded":"2023-11-28T03:08:17.699412+0000","last_fullsized":"2023-11-28T03:08:17.699412+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332927+0000","last_change":"2023-11-28T03:08:17.329283+0000","last_active":"2023-11-28T03:08:17.332927+0000","last_peered":"2023-11-28T03:08:17.332927+0000","last_clean":"2023-11-28T03:08:17.332927+0000","last_became_active":"2023-11-28T03:08:17.328888+0000","last_became_peered":"2023-11-28T03:08:17.328888+0000","last_unstale":"2023-11-28T03:08:17.332927+0000","last_undegraded":"2023-11-28T03:08:17.332927+0000","last_fullsized":"2023-11-28T03:08:17.332927+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.13","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.687995+0000","last_change":"2023-11-28T03:08:17.687995+0000","last_active":"2023-11-28T03:08:17.687995+0000","last_peered":"2023-11-28T03:08:17.687995+0000","last_clean":"2023-11-28T03:08:17.687995+0000","last_became_active":"2023-11-28T03:08:17.687767+0000","last_became_peered":"2023-11-28T03:08:17.687767+0000","last_unstale":"2023-11-28T03:08:17.687995+0000","last_undegraded":"2023-11-28T03:08:17.687995+0000","last_fullsized":"2023-11-28T03:08:17.687995+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.10","version":"109'4","reported_seq":13,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.701953+0000","last_change":"2023-11-28T03:08:17.329865+0000","last_active":"2023-11-28T03:08:17.701953+0000","last_peered":"2023-11-28T03:08:17.701953+0000","last_clean":"2023-11-28T03:08:17.701953+0000","last_became_active":"2023-11-28T03:08:17.329617+0000","last_became_peered":"2023-11-28T03:08:17.329617+0000","last_unstale":"2023-11-28T03:08:17.701953+0000","last_undegraded":"2023-11-28T03:08:17.701953+0000","last_fullsized":"2023-11-28T03:08:17.701953+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.690105+0000","last_change":"2023-11-28T03:08:17.690105+0000","last_active":"2023-11-28T03:08:17.690105+0000","last_peered":"2023-11-28T03:08:17.690105+0000","last_clean":"2023-11-28T03:08:17.690105+0000","last_became_active":"2023-11-28T03:08:17.689845+0000","last_became_peered":"2023-11-28T03:08:17.689845+0000","last_unstale":"2023-11-28T03:08:17.690105+0000","last_undegraded":"2023-11-28T03:08:17.690105+0000","last_fullsized":"2023-11-28T03:08:17.690105+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.7","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.694930+0000","last_change":"2023-11-28T03:08:17.330562+0000","last_active":"2023-11-28T03:08:17.694930+0000","last_peered":"2023-11-28T03:08:17.694930+0000","last_clean":"2023-11-28T03:08:17.694930+0000","last_became_active":"2023-11-28T03:08:17.330307+0000","last_became_peered":"2023-11-28T03:08:17.330307+0000","last_unstale":"2023-11-28T03:08:17.694930+0000","last_undegraded":"2023-11-28T03:08:17.694930+0000","last_fullsized":"2023-11-28T03:08:17.694930+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693174+0000","last_change":"2023-11-28T03:08:17.693174+0000","last_active":"2023-11-28T03:08:17.693174+0000","last_peered":"2023-11-28T03:08:17.693174+0000","last_clean":"2023-11-28T03:08:17.693174+0000","last_became_active":"2023-11-28T03:08:17.692744+0000","last_became_peered":"2023-11-28T03:08:17.692744+0000","last_unstale":"2023-11-28T03:08:17.693174+0000","last_undegraded":"2023-11-28T03:08:17.693174+0000","last_fullsized":"2023-11-28T03:08:17.693174+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334682+0000","last_change":"2023-11-28T03:08:17.332842+0000","last_active":"2023-11-28T03:08:17.334682+0000","last_peered":"2023-11-28T03:08:17.334682+0000","last_clean":"2023-11-28T03:08:17.334682+0000","last_became_active":"2023-11-28T03:08:17.332571+0000","last_became_peered":"2023-11-28T03:08:17.332571+0000","last_unstale":"2023-11-28T03:08:17.334682+0000","last_undegraded":"2023-11-28T03:08:17.334682+0000","last_fullsized":"2023-11-28T03:08:17.334682+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.690902+0000","last_change":"2023-11-28T03:08:17.690902+0000","last_active":"2023-11-28T03:08:17.690902+0000","last_peered":"2023-11-28T03:08:17.690902+0000","last_clean":"2023-11-28T03:08:17.690902+0000","last_became_active":"2023-11-28T03:08:17.690510+0000","last_became_peered":"2023-11-28T03:08:17.690510+0000","last_unstale":"2023-11-28T03:08:17.690902+0000","last_undegraded":"2023-11-28T03:08:17.690902+0000","last_fullsized":"2023-11-28T03:08:17.690902+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334502+0000","last_change":"2023-11-28T03:08:17.328915+0000","last_active":"2023-11-28T03:08:17.334502+0000","last_peered":"2023-11-28T03:08:17.334502+0000","last_clean":"2023-11-28T03:08:17.334502+0000","last_became_active":"2023-11-28T03:08:17.328467+0000","last_became_peered":"2023-11-28T03:08:17.328467+0000","last_unstale":"2023-11-28T03:08:17.334502+0000","last_undegraded":"2023-11-28T03:08:17.334502+0000","last_fullsized":"2023-11-28T03:08:17.334502+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.7","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693022+0000","last_change":"2023-11-28T03:08:17.693022+0000","last_active":"2023-11-28T03:08:17.693022+0000","last_peered":"2023-11-28T03:08:17.693022+0000","last_clean":"2023-11-28T03:08:17.693022+0000","last_became_active":"2023-11-28T03:08:17.692715+0000","last_became_peered":"2023-11-28T03:08:17.692715+0000","last_unstale":"2023-11-28T03:08:17.693022+0000","last_undegraded":"2023-11-28T03:08:17.693022+0000","last_fullsized":"2023-11-28T03:08:17.693022+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334337+0000","last_change":"2023-11-28T03:08:17.331112+0000","last_active":"2023-11-28T03:08:17.334337+0000","last_peered":"2023-11-28T03:08:17.334337+0000","last_clean":"2023-11-28T03:08:17.334337+0000","last_became_active":"2023-11-28T03:08:17.330856+0000","last_became_peered":"2023-11-28T03:08:17.330856+0000","last_unstale":"2023-11-28T03:08:17.334337+0000","last_undegraded":"2023-11-28T03:08:17.334337+0000","last_fullsized":"2023-11-28T03:08:17.334337+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689494+0000","last_change":"2023-11-28T03:08:17.689494+0000","last_active":"2023-11-28T03:08:17.689494+0000","last_peered":"2023-11-28T03:08:17.689494+0000","last_clean":"2023-11-28T03:08:17.689494+0000","last_became_active":"2023-11-28T03:08:17.689240+0000","last_became_peered":"2023-11-28T03:08:17.689240+0000","last_unstale":"2023-11-28T03:08:17.689494+0000","last_undegraded":"2023-11-28T03:08:17.689494+0000","last_fullsized":"2023-11-28T03:08:17.689494+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.328312+0000","last_change":"2023-11-28T03:08:17.325554+0000","last_active":"2023-11-28T03:08:17.328312+0000","last_peered":"2023-11-28T03:08:17.328312+0000","last_clean":"2023-11-28T03:08:17.328312+0000","last_became_active":"2023-11-28T03:08:17.325266+0000","last_became_peered":"2023-11-28T03:08:17.325266+0000","last_unstale":"2023-11-28T03:08:17.328312+0000","last_undegraded":"2023-11-28T03:08:17.328312+0000","last_fullsized":"2023-11-28T03:08:17.328312+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699350+0000","last_change":"2023-11-28T03:08:17.699350+0000","last_active":"2023-11-28T03:08:17.699350+0000","last_peered":"2023-11-28T03:08:17.699350+0000","last_clean":"2023-11-28T03:08:17.699350+0000","last_became_active":"2023-11-28T03:08:17.698905+0000","last_became_peered":"2023-11-28T03:08:17.698905+0000","last_unstale":"2023-11-28T03:08:17.699350+0000","last_undegraded":"2023-11-28T03:08:17.699350+0000","last_fullsized":"2023-11-28T03:08:17.699350+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.331508+0000","last_change":"2023-11-28T03:08:17.327462+0000","last_active":"2023-11-28T03:08:17.331508+0000","last_peered":"2023-11-28T03:08:17.331508+0000","last_clean":"2023-11-28T03:08:17.331508+0000","last_became_active":"2023-11-28T03:08:17.327191+0000","last_became_peered":"2023-11-28T03:08:17.327191+0000","last_unstale":"2023-11-28T03:08:17.331508+0000","last_undegraded":"2023-11-28T03:08:17.331508+0000","last_fullsized":"2023-11-28T03:08:17.331508+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.699212+0000","last_change":"2023-11-28T03:08:17.699212+0000","last_active":"2023-11-28T03:08:17.699212+0000","last_peered":"2023-11-28T03:08:17.699212+0000","last_clean":"2023-11-28T03:08:17.699212+0000","last_became_active":"2023-11-28T03:08:17.698911+0000","last_became_peered":"2023-11-28T03:08:17.698911+0000","last_unstale":"2023-11-28T03:08:17.699212+0000","last_undegraded":"2023-11-28T03:08:17.699212+0000","last_fullsized":"2023-11-28T03:08:17.699212+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334248+0000","last_change":"2023-11-28T03:08:17.330478+0000","last_active":"2023-11-28T03:08:17.334248+0000","last_peered":"2023-11-28T03:08:17.334248+0000","last_clean":"2023-11-28T03:08:17.334248+0000","last_became_active":"2023-11-28T03:08:17.330198+0000","last_became_peered":"2023-11-28T03:08:17.330198+0000","last_unstale":"2023-11-28T03:08:17.334248+0000","last_undegraded":"2023-11-28T03:08:17.334248+0000","last_fullsized":"2023-11-28T03:08:17.334248+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.686138+0000","last_change":"2023-11-28T03:08:17.686138+0000","last_active":"2023-11-28T03:08:17.686138+0000","last_peered":"2023-11-28T03:08:17.686138+0000","last_clean":"2023-11-28T03:08:17.686138+0000","last_became_active":"2023-11-28T03:08:17.685860+0000","last_became_peered":"2023-11-28T03:08:17.685860+0000","last_unstale":"2023-11-28T03:08:17.686138+0000","last_undegraded":"2023-11-28T03:08:17.686138+0000","last_fullsized":"2023-11-28T03:08:17.686138+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334460+0000","last_change":"2023-11-28T03:08:17.333013+0000","last_active":"2023-11-28T03:08:17.334460+0000","last_peered":"2023-11-28T03:08:17.334460+0000","last_clean":"2023-11-28T03:08:17.334460+0000","last_became_active":"2023-11-28T03:08:17.332678+0000","last_became_peered":"2023-11-28T03:08:17.332678+0000","last_unstale":"2023-11-28T03:08:17.334460+0000","last_undegraded":"2023-11-28T03:08:17.334460+0000","last_fullsized":"2023-11-28T03:08:17.334460+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.703755+0000","last_change":"2023-11-28T03:08:17.703755+0000","last_active":"2023-11-28T03:08:17.703755+0000","last_peered":"2023-11-28T03:08:17.703755+0000","last_clean":"2023-11-28T03:08:17.703755+0000","last_became_active":"2023-11-28T03:08:17.703466+0000","last_became_peered":"2023-11-28T03:08:17.703466+0000","last_unstale":"2023-11-28T03:08:17.703755+0000","last_undegraded":"2023-11-28T03:08:17.703755+0000","last_fullsized":"2023-11-28T03:08:17.703755+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.b","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.684192+0000","last_change":"2023-11-28T03:08:17.325894+0000","last_active":"2023-11-28T03:08:17.684192+0000","last_peered":"2023-11-28T03:08:17.684192+0000","last_clean":"2023-11-28T03:08:17.684192+0000","last_became_active":"2023-11-28T03:08:17.325568+0000","last_became_peered":"2023-11-28T03:08:17.325568+0000","last_unstale":"2023-11-28T03:08:17.684192+0000","last_undegraded":"2023-11-28T03:08:17.684192+0000","last_fullsized":"2023-11-28T03:08:17.684192+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.316079+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:08:17.316079+0000","last_peered":"2023-11-28T03:08:17.316079+0000","last_clean":"2023-11-28T03:08:17.316079+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:08:17.316079+0000","last_undegraded":"2023-11-28T03:08:17.316079+0000","last_fullsized":"2023-11-28T03:08:17.316079+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.e","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.679184+0000","last_change":"2023-11-28T03:08:17.679184+0000","last_active":"2023-11-28T03:08:17.679184+0000","last_peered":"2023-11-28T03:08:17.679184+0000","last_clean":"2023-11-28T03:08:17.679184+0000","last_became_active":"2023-11-28T03:08:17.678903+0000","last_became_peered":"2023-11-28T03:08:17.678903+0000","last_unstale":"2023-11-28T03:08:17.679184+0000","last_undegraded":"2023-11-28T03:08:17.679184+0000","last_fullsized":"2023-11-28T03:08:17.679184+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.326301+0000","last_change":"2023-11-28T03:08:17.323102+0000","last_active":"2023-11-28T03:08:17.326301+0000","last_peered":"2023-11-28T03:08:17.326301+0000","last_clean":"2023-11-28T03:08:17.326301+0000","last_became_active":"2023-11-28T03:08:17.322617+0000","last_became_peered":"2023-11-28T03:08:17.322617+0000","last_unstale":"2023-11-28T03:08:17.326301+0000","last_undegraded":"2023-11-28T03:08:17.326301+0000","last_fullsized":"2023-11-28T03:08:17.326301+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.322465+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:08:17.322465+0000","last_peered":"2023-11-28T03:08:17.322465+0000","last_clean":"2023-11-28T03:08:17.322465+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:08:17.322465+0000","last_undegraded":"2023-11-28T03:08:17.322465+0000","last_fullsized":"2023-11-28T03:08:17.322465+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692414+0000","last_change":"2023-11-28T03:08:17.692414+0000","last_active":"2023-11-28T03:08:17.692414+0000","last_peered":"2023-11-28T03:08:17.692414+0000","last_clean":"2023-11-28T03:08:17.692414+0000","last_became_active":"2023-11-28T03:08:17.692142+0000","last_became_peered":"2023-11-28T03:08:17.692142+0000","last_unstale":"2023-11-28T03:08:17.692414+0000","last_undegraded":"2023-11-28T03:08:17.692414+0000","last_fullsized":"2023-11-28T03:08:17.692414+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"109'2","reported_seq":11,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689089+0000","last_change":"2023-11-28T03:08:17.331448+0000","last_active":"2023-11-28T03:08:17.689089+0000","last_peered":"2023-11-28T03:08:17.689089+0000","last_clean":"2023-11-28T03:08:17.689089+0000","last_became_active":"2023-11-28T03:08:17.331172+0000","last_became_peered":"2023-11-28T03:08:17.331172+0000","last_unstale":"2023-11-28T03:08:17.689089+0000","last_undegraded":"2023-11-28T03:08:17.689089+0000","last_fullsized":"2023-11-28T03:08:17.689089+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":87,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.307150+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:08:17.307150+0000","last_peered":"2023-11-28T03:08:17.307150+0000","last_clean":"2023-11-28T03:08:17.307150+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:08:17.307150+0000","last_undegraded":"2023-11-28T03:08:17.307150+0000","last_fullsized":"2023-11-28T03:08:17.307150+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.698279+0000","last_change":"2023-11-28T03:08:17.698279+0000","last_active":"2023-11-28T03:08:17.698279+0000","last_peered":"2023-11-28T03:08:17.698279+0000","last_clean":"2023-11-28T03:08:17.698279+0000","last_became_active":"2023-11-28T03:08:17.697922+0000","last_became_peered":"2023-11-28T03:08:17.697922+0000","last_unstale":"2023-11-28T03:08:17.698279+0000","last_undegraded":"2023-11-28T03:08:17.698279+0000","last_fullsized":"2023-11-28T03:08:17.698279+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.332864+0000","last_change":"2023-11-28T03:08:17.325384+0000","last_active":"2023-11-28T03:08:17.332864+0000","last_peered":"2023-11-28T03:08:17.332864+0000","last_clean":"2023-11-28T03:08:17.332864+0000","last_became_active":"2023-11-28T03:08:17.325172+0000","last_became_peered":"2023-11-28T03:08:17.325172+0000","last_unstale":"2023-11-28T03:08:17.332864+0000","last_undegraded":"2023-11-28T03:08:17.332864+0000","last_fullsized":"2023-11-28T03:08:17.332864+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.320493+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:08:17.320493+0000","last_peered":"2023-11-28T03:08:17.320493+0000","last_clean":"2023-11-28T03:08:17.320493+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:08:17.320493+0000","last_undegraded":"2023-11-28T03:08:17.320493+0000","last_fullsized":"2023-11-28T03:08:17.320493+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":85,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.320643+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:08:17.320643+0000","last_peered":"2023-11-28T03:08:17.320643+0000","last_clean":"2023-11-28T03:08:17.320643+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:08:17.320643+0000","last_undegraded":"2023-11-28T03:08:17.320643+0000","last_fullsized":"2023-11-28T03:08:17.320643+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.692575+0000","last_change":"2023-11-28T03:08:17.692575+0000","last_active":"2023-11-28T03:08:17.692575+0000","last_peered":"2023-11-28T03:08:17.692575+0000","last_clean":"2023-11-28T03:08:17.692575+0000","last_became_active":"2023-11-28T03:08:17.692302+0000","last_became_peered":"2023-11-28T03:08:17.692302+0000","last_unstale":"2023-11-28T03:08:17.692575+0000","last_undegraded":"2023-11-28T03:08:17.692575+0000","last_fullsized":"2023-11-28T03:08:17.692575+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.f","version":"109'1","reported_seq":10,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.676617+0000","last_change":"2023-11-28T03:08:17.321111+0000","last_active":"2023-11-28T03:08:17.676617+0000","last_peered":"2023-11-28T03:08:17.676617+0000","last_clean":"2023-11-28T03:08:17.676617+0000","last_became_active":"2023-11-28T03:08:17.318963+0000","last_became_peered":"2023-11-28T03:08:17.318963+0000","last_unstale":"2023-11-28T03:08:17.676617+0000","last_undegraded":"2023-11-28T03:08:17.676617+0000","last_fullsized":"2023-11-28T03:08:17.676617+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":83,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.323173+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:08:17.323173+0000","last_peered":"2023-11-28T03:08:17.323173+0000","last_clean":"2023-11-28T03:08:17.323173+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:08:17.323173+0000","last_undegraded":"2023-11-28T03:08:17.323173+0000","last_fullsized":"2023-11-28T03:08:17.323173+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.689902+0000","last_change":"2023-11-28T03:08:17.689902+0000","last_active":"2023-11-28T03:08:17.689902+0000","last_peered":"2023-11-28T03:08:17.689902+0000","last_clean":"2023-11-28T03:08:17.689902+0000","last_became_active":"2023-11-28T03:08:17.689615+0000","last_became_peered":"2023-11-28T03:08:17.689615+0000","last_unstale":"2023-11-28T03:08:17.689902+0000","last_undegraded":"2023-11-28T03:08:17.689902+0000","last_fullsized":"2023-11-28T03:08:17.689902+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334319+0000","last_change":"2023-11-28T03:08:17.320728+0000","last_active":"2023-11-28T03:08:17.334319+0000","last_peered":"2023-11-28T03:08:17.334319+0000","last_clean":"2023-11-28T03:08:17.334319+0000","last_became_active":"2023-11-28T03:08:17.320449+0000","last_became_peered":"2023-11-28T03:08:17.320449+0000","last_unstale":"2023-11-28T03:08:17.334319+0000","last_undegraded":"2023-11-28T03:08:17.334319+0000","last_fullsized":"2023-11-28T03:08:17.334319+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":83,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.322041+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:08:17.322041+0000","last_peered":"2023-11-28T03:08:17.322041+0000","last_clean":"2023-11-28T03:08:17.322041+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:08:17.322041+0000","last_undegraded":"2023-11-28T03:08:17.322041+0000","last_fullsized":"2023-11-28T03:08:17.322041+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.701783+0000","last_change":"2023-11-28T03:08:17.701783+0000","last_active":"2023-11-28T03:08:17.701783+0000","last_peered":"2023-11-28T03:08:17.701783+0000","last_clean":"2023-11-28T03:08:17.701783+0000","last_became_active":"2023-11-28T03:08:17.701503+0000","last_became_peered":"2023-11-28T03:08:17.701503+0000","last_unstale":"2023-11-28T03:08:17.701783+0000","last_undegraded":"2023-11-28T03:08:17.701783+0000","last_fullsized":"2023-11-28T03:08:17.701783+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.333415+0000","last_change":"2023-11-28T03:08:17.329966+0000","last_active":"2023-11-28T03:08:17.333415+0000","last_peered":"2023-11-28T03:08:17.333415+0000","last_clean":"2023-11-28T03:08:17.333415+0000","last_became_active":"2023-11-28T03:08:17.327865+0000","last_became_peered":"2023-11-28T03:08:17.327865+0000","last_unstale":"2023-11-28T03:08:17.333415+0000","last_undegraded":"2023-11-28T03:08:17.333415+0000","last_fullsized":"2023-11-28T03:08:17.333415+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.314487+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:08:17.314487+0000","last_peered":"2023-11-28T03:08:17.314487+0000","last_clean":"2023-11-28T03:08:17.314487+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:08:17.314487+0000","last_undegraded":"2023-11-28T03:08:17.314487+0000","last_fullsized":"2023-11-28T03:08:17.314487+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.b","version":"0'0","reported_seq":8,"reported_epoch":109,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.693459+0000","last_change":"2023-11-28T03:08:17.693459+0000","last_active":"2023-11-28T03:08:17.693459+0000","last_peered":"2023-11-28T03:08:17.693459+0000","last_clean":"2023-11-28T03:08:17.693459+0000","last_became_active":"2023-11-28T03:08:17.693204+0000","last_became_peered":"2023-11-28T03:08:17.693204+0000","last_unstale":"2023-11-28T03:08:17.693459+0000","last_undegraded":"2023-11-28T03:08:17.693459+0000","last_fullsized":"2023-11-28T03:08:17.693459+0000","mapping_epoch":108,"log_start":"0'0","ondisk_log_start":"0'0","created":108,"last_epoch_clean":109,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:17.300840+0000","last_clean_scrub_stamp":"2023-11-28T03:08:17.300840+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":9,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.334630+0000","last_change":"2023-11-28T03:08:17.329148+0000","last_active":"2023-11-28T03:08:17.334630+0000","last_peered":"2023-11-28T03:08:17.334630+0000","last_clean":"2023-11-28T03:08:17.334630+0000","last_became_active":"2023-11-28T03:08:17.328878+0000","last_became_peered":"2023-11-28T03:08:17.328878+0000","last_unstale":"2023-11-28T03:08:17.334630+0000","last_undegraded":"2023-11-28T03:08:17.334630+0000","last_fullsized":"2023-11-28T03:08:17.334630+0000","mapping_epoch":107,"log_start":"0'0","ondisk_log_start":"0'0","created":107,"last_epoch_clean":108,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:08:16.292327+0000","last_clean_scrub_stamp":"2023-11-28T03:08:16.292327+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":96,"reported_epoch":108,"state":"active+clean","last_fresh":"2023-11-28T03:08:17.313517+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:08:17.313517+0000","last_peered":"2023-11-28T03:08:17.313517+0000","last_clean":"2023-11-28T03:08:17.313517+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:08:17.313517+0000","last_undegraded":"2023-11-28T03:08:17.313517+0000","last_fullsized":"2023-11-28T03:08:17.313517+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":14,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575035,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.57799999999999996,"15min":0.59199999999999997},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.73399999999999999,"5min":0.79300000000000004,"15min":0.85999999999999999},"last":0.621},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.58699999999999997,"15min":0.60099999999999998},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.751,"5min":0.80300000000000005,"15min":0.83199999999999996},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.54500000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.69599999999999995,"5min":0.76100000000000001,"15min":0.80800000000000005},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.53200000000000003,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.63100000000000001,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38900000000000001,"5min":0.377,"15min":0.34499999999999997},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.55100000000000005,"15min":0.56000000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.69599999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.495,"5min":0.55100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38500000000000001,"5min":0.373,"15min":0.36599999999999999},"max":{"1min":0.70799999999999996,"5min":0.75,"15min":0.77100000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.56399999999999995,"15min":0.57399999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":0.74099999999999999,"5min":0.81999999999999995,"15min":0.82399999999999995},"last":0.57999999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.376},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.48899999999999999,"15min":0.48499999999999999},"min":{"1min":0.32500000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.433}]},{"osd":5,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.50800000000000001,"15min":0.499},"min":{"1min":0.315,"5min":0.315,"15min":0.313},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.49299999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:08:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.55900000000000005,"15min":0.54900000000000004},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.76900000000000002,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575035,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.53500000000000003,"15min":0.54300000000000004},"min":{"1min":0.378,"5min":0.34100000000000003,"15min":0.28799999999999998},"max":{"1min":0.73699999999999999,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.54900000000000004,"15min":0.56100000000000005},"min":{"1min":0.39400000000000002,"5min":0.33900000000000002,"15min":0.33400000000000002},"max":{"1min":0.58499999999999996,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.61299999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.38900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.79000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.497,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.375,"5min":0.375,"15min":0.32800000000000001},"max":{"1min":0.66700000000000004,"5min":1.506,"15min":1.506},"last":0.48999999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.55400000000000005,"15min":0.56499999999999995},"min":{"1min":0.35399999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76500000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.5},{"interface":"front","average":{"1min":0.503,"5min":0.58199999999999996,"15min":0.57499999999999996},"min":{"1min":0.36199999999999999,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.65700000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.59199999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.58199999999999996,"15min":0.58399999999999996},"min":{"1min":0.40200000000000002,"5min":0.38200000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57699999999999996,"15min":0.57699999999999996},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.68700000000000006,"5min":0.81999999999999995,"15min":0.84399999999999997},"last":0.58299999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.47299999999999998,"15min":0.45500000000000002},"min":{"1min":0.309,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.78100000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.34399999999999997,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.82499999999999996},"last":0.54500000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.53300000000000003,"15min":0.51600000000000001},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.309},"max":{"1min":0.874,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.44},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.56399999999999995,"15min":0.55000000000000004},"min":{"1min":0.42699999999999999,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.82099999999999995,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.53100000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:05 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.54200000000000004},"min":{"1min":0.36399999999999999,"5min":0.33300000000000002,"15min":0.317},"max":{"1min":0.84399999999999997,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57699999999999996,"15min":0.55800000000000005},"min":{"1min":0.37,"5min":0.35799999999999998,"15min":0.35299999999999998},"max":{"1min":0.90400000000000003,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.51700000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575034,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.30499999999999999,"5min":0.33200000000000002,"15min":0.34100000000000003},"min":{"1min":0.22600000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.48599999999999999,"5min":0.67800000000000005,"15min":0.70699999999999996},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.309,"5min":0.35799999999999998,"15min":0.36699999999999999},"min":{"1min":0.218,"5min":0.20899999999999999,"15min":0.20300000000000001},"max":{"1min":0.41799999999999998,"5min":0.67100000000000004,"15min":0.69899999999999995},"last":0.32900000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.32500000000000001,"5min":0.371,"15min":0.379},"min":{"1min":0.22700000000000001,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.45200000000000001,"5min":0.57499999999999996,"15min":0.73499999999999999},"last":0.307},{"interface":"front","average":{"1min":0.33100000000000002,"5min":0.39200000000000002,"15min":0.39900000000000002},"min":{"1min":0.253,"5min":0.253,"15min":0.19800000000000001},"max":{"1min":0.41199999999999998,"5min":0.628,"15min":0.72599999999999998},"last":0.27800000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.434,"15min":0.44900000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.44700000000000001,"5min":0.69599999999999995,"15min":0.749},"last":0.441},{"interface":"front","average":{"1min":0.34799999999999998,"5min":0.40899999999999997,"15min":0.42399999999999999},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.42299999999999999,"5min":0.68799999999999994,"15min":1.6870000000000001},"last":0.40899999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.57299999999999995,"15min":0.58599999999999997},"min":{"1min":0.33300000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.86799999999999999,"5min":0.91100000000000003,"15min":1.0900000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.58099999999999996,"15min":0.59199999999999997},"min":{"1min":0.42999999999999999,"5min":0.36699999999999999,"15min":0.28699999999999998},"max":{"1min":0.70999999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.57599999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.60399999999999998,"15min":0.623},"min":{"1min":0.41199999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.84699999999999998,"5min":0.99299999999999999,"15min":1.7789999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.629,"15min":0.64400000000000002},"min":{"1min":0.39400000000000002,"5min":0.38,"15min":0.38},"max":{"1min":0.94199999999999995,"5min":1.038,"15min":1.111},"last":0.61399999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.57499999999999996,"15min":0.59499999999999997},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78500000000000003,"5min":0.89400000000000002,"15min":1.8080000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.90900000000000003,"5min":0.93500000000000005,"15min":1.073},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:08:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.374},"max":{"1min":0.79800000000000004,"5min":1.024,"15min":1.024},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.59499999999999997,"15min":0.61799999999999999},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.38700000000000001},"max":{"1min":0.72499999999999998,"5min":0.87,"15min":1.7929999999999999},"last":0.437}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.29999999999999999,"5min":0.33700000000000002,"15min":0.34399999999999997},"min":{"1min":0.217,"5min":0.217,"15min":0.184},"max":{"1min":0.40400000000000003,"5min":0.56599999999999995,"15min":0.75700000000000001},"last":0.309},{"interface":"front","average":{"1min":0.30599999999999999,"5min":0.35099999999999998,"15min":0.36499999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47199999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.28799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.29499999999999998,"5min":0.34200000000000003,"15min":0.36599999999999999},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.20100000000000001},"max":{"1min":0.377,"5min":0.52600000000000002,"15min":0.69699999999999995},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.42099999999999999,"15min":0.438},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.44,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.31900000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.375,"15min":0.38900000000000001},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.41699999999999998,"5min":0.56799999999999995,"15min":0.73099999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.33500000000000002,"5min":0.39500000000000002,"15min":0.41499999999999998},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":0.46100000000000002,"5min":0.621,"15min":0.66500000000000004},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.58699999999999997,"15min":0.60899999999999999},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.64200000000000002,"5min":0.95699999999999996,"15min":1.0489999999999999},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.61399999999999999,"15min":0.63100000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.67200000000000004,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.41099999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.60699999999999998,"15min":0.629},"min":{"1min":0.42099999999999999,"5min":0.374,"15min":0.374},"max":{"1min":0.71099999999999997,"5min":1.304,"15min":1.304},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.60499999999999998,"15min":0.63},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.72899999999999998,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.435}]},{"osd":6,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.57999999999999996,"15min":0.59499999999999997},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.66400000000000003,"5min":1.679,"15min":1.679},"last":0.441},{"interface":"front","average":{"1min":0.496,"5min":0.59499999999999997,"15min":0.61299999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.35899999999999999},"max":{"1min":0.61699999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.40400000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.59999999999999998,"15min":0.621},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.66500000000000004,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.63900000000000001,"15min":0.66400000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.73699999999999999,"5min":1.046,"15min":1.046},"last":0.47599999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575036,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35699999999999998,"15min":0.36499999999999999},"min":{"1min":0.22,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.45400000000000001,"5min":0.59499999999999997,"15min":0.623},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.39300000000000002,"15min":0.41299999999999998},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.23599999999999999},"max":{"1min":0.49199999999999999,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.27300000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.33000000000000002,"5min":0.35799999999999998,"15min":0.36499999999999999},"min":{"1min":0.249,"5min":0.22700000000000001,"15min":0.19800000000000001},"max":{"1min":0.47799999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.32200000000000001,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.252,"5min":0.218,"15min":0.218},"max":{"1min":0.55500000000000005,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.373,"5min":0.40300000000000002,"15min":0.41499999999999998},"min":{"1min":0.27400000000000002,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.52300000000000002,"5min":0.67400000000000004,"15min":0.67400000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.34499999999999997,"5min":0.38300000000000001,"15min":0.39700000000000002},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23999999999999999},"max":{"1min":0.54000000000000004,"5min":0.64600000000000002,"15min":0.65500000000000003},"last":0.312}]},{"osd":4,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.41899999999999998,"5min":0.39100000000000001,"15min":0.36299999999999999},"max":{"1min":0.64500000000000002,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59899999999999998,"15min":0.60599999999999998},"min":{"1min":0.433,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.72899999999999998,"5min":0.84299999999999997,"15min":0.874},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.47799999999999998,"5min":0.378,"15min":0.36699999999999999},"max":{"1min":0.71699999999999997,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.629,"15min":0.63700000000000001},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82699999999999996,"5min":1.004,"15min":1.004},"last":0.433}]},{"osd":6,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.61799999999999999,"15min":0.63},"min":{"1min":0.40000000000000002,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.68899999999999995,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.441},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.64300000000000002,"15min":0.65100000000000002},"min":{"1min":0.45100000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.84499999999999997,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:07:55 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.61499999999999999,"15min":0.629},"min":{"1min":0.40300000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.77500000000000002,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61299999999999999,"15min":0.627},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.81699999999999995,"5min":0.94499999999999995,"15min":0.94499999999999995},"last":0.34899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302476,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069364,"statfs":{"total":96636764160,"available":96327028736,"internally_reserved":0,"allocated":1314816,"data_stored":651047,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.316,"5min":0.35399999999999998,"15min":0.35799999999999998},"min":{"1min":0.23799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.442,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.32500000000000001,"5min":0.36499999999999999,"15min":0.373},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.44800000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.32400000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.372,"15min":0.376},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.19900000000000001},"max":{"1min":0.44700000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.312},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.40400000000000003,"15min":0.41399999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.48899999999999999,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.33600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.40899999999999997,"15min":0.41899999999999998},"min":{"1min":0.29499999999999998,"5min":0.223,"15min":0.223},"max":{"1min":0.51300000000000001,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.42299999999999999,"15min":0.435},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.5,"5min":0.65700000000000003,"15min":0.68200000000000005},"last":0.36599999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.58699999999999997,"15min":0.60599999999999998},"min":{"1min":0.36699999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.878,"5min":1.581,"15min":1.6000000000000001},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.58699999999999997,"15min":0.61199999999999999},"min":{"1min":0.39100000000000001,"5min":0.38100000000000001,"15min":0.29899999999999999},"max":{"1min":0.70299999999999996,"5min":0.90800000000000003,"15min":1.196},"last":0.45600000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57999999999999996,"15min":0.60399999999999998},"min":{"1min":0.373,"5min":0.35699999999999998,"15min":0.33000000000000002},"max":{"1min":0.83199999999999996,"5min":0.83799999999999997,"15min":1.1439999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.61199999999999999,"15min":0.629},"min":{"1min":0.40000000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.82299999999999995,"5min":0.93600000000000005,"15min":1.1859999999999999},"last":0.46899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.59999999999999998,"15min":0.63100000000000001},"min":{"1min":0.42099999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.83699999999999997,"5min":0.91900000000000004,"15min":1.206},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.61399999999999999,"15min":0.63600000000000001},"min":{"1min":0.38400000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.81000000000000005,"5min":0.99099999999999999,"15min":1.1759999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:07:58 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.63100000000000001,"15min":0.64900000000000002},"min":{"1min":0.39500000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.68500000000000005,"5min":0.96199999999999997,"15min":1.1559999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61699999999999999,"15min":0.64600000000000002},"min":{"1min":0.38,"5min":0.376,"15min":0.376},"max":{"1min":0.84999999999999998,"5min":0.90000000000000002,"15min":1.2290000000000001},"last":0.60899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575034,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302504,"kb_used_data":1312,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069336,"statfs":{"total":96636764160,"available":96327000064,"internally_reserved":0,"allocated":1343488,"data_stored":664995,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.52900000000000003,"15min":0.54400000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.27800000000000002},"max":{"1min":0.70099999999999996,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.48499999999999999,"5min":0.55100000000000005,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.27300000000000002},"max":{"1min":0.72599999999999998,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.58199999999999996}]},{"osd":1,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.54400000000000004,"15min":0.55100000000000005},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.314},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.55100000000000005,"15min":0.55800000000000005},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.77100000000000002,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.51900000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.55200000000000005,"15min":0.56100000000000005},"min":{"1min":0.34200000000000003,"5min":0.33500000000000002,"15min":0.30199999999999999},"max":{"1min":0.65700000000000003,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.54800000000000004,"15min":0.55700000000000005},"min":{"1min":0.377,"5min":0.377,"15min":0.33500000000000002},"max":{"1min":0.80900000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.71899999999999997}]},{"osd":3,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.58099999999999996,"15min":0.59699999999999998},"min":{"1min":0.38700000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.69999999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.437,"5min":0.437,"15min":0.38500000000000001},"max":{"1min":0.79600000000000004,"5min":0.84599999999999997,"15min":0.878},"last":0.68799999999999994}]},{"osd":5,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.48699999999999999,"5min":0.48899999999999999,"15min":0.47199999999999998},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.74199999999999999,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.50700000000000001,"15min":0.504},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.31},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.39300000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.371,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53400000000000003,"15min":0.52500000000000002},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55400000000000005,"15min":0.54200000000000004},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.32900000000000001},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.504}]}]},{"osd":5,"up_from":13,"seq":55834575036,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302488,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069352,"statfs":{"total":96636764160,"available":96327016448,"internally_reserved":0,"allocated":1327104,"data_stored":657726,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.53400000000000003,"15min":0.52700000000000002},"min":{"1min":0.27900000000000003,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":2.8959999999999999,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.48699999999999999,"5min":0.53200000000000003,"15min":0.54200000000000004},"min":{"1min":0.41299999999999998,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.59799999999999998,"5min":0.71999999999999997,"15min":0.77900000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.54700000000000004,"15min":0.55900000000000005},"min":{"1min":0.38100000000000001,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.61299999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.48899999999999999,"5min":0.56499999999999995,"15min":0.57699999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.31900000000000001},"max":{"1min":0.64700000000000002,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.57299999999999995}]},{"osd":2,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.57099999999999995,"15min":0.58299999999999996},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.67100000000000004,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.58799999999999997,"15min":0.59899999999999998},"min":{"1min":0.41399999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.752,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.74299999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.52800000000000002,"15min":0.54200000000000004},"min":{"1min":0.35999999999999999,"5min":0.35899999999999999,"15min":0.34599999999999997},"max":{"1min":0.60999999999999999,"5min":0.73899999999999999,"15min":0.79800000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.57999999999999996,"15min":0.57899999999999996},"min":{"1min":0.375,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":2.9079999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.68999999999999995}]},{"osd":4,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.45800000000000002,"15min":0.45300000000000001},"min":{"1min":0.311,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.80600000000000005,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52100000000000002,"15min":0.53600000000000003},"min":{"1min":0.36599999999999999,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.65300000000000002,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.71999999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.317,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.35599999999999998,"15min":0.33700000000000002},"max":{"1min":0.69199999999999995,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.77300000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.49199999999999999,"15min":0.50600000000000001},"min":{"1min":0.32700000000000001,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.77800000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52600000000000002,"15min":0.54100000000000004},"min":{"1min":0.35099999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76500000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.628}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:08:21.013 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:08:21.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:08:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.164+0000 7f3a8630d700 1 Processor -- start 2023-11-28T03:08:21.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.164+0000 7f3a8630d700 1 -- start start 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a8630d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a80098d10 0x7f3a800c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a8630d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 0x7f3a80063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a8630d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a800c72d0 con 0x7f3a80064350 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a8630d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a80154bc0 con 0x7f3a80098d10 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a8630d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a800c66c0 con 0x7f3a800c6190 2023-11-28T03:08:21.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a7ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3a80098d10 0x7f3a800c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a848aa700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3a80064350 0x7f3a80158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35576/0 (socket says 172.21.15.73:35576) 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a848aa700 1 -- 172.21.15.73:0/2028237620 learned_addr learned my addr 172.21.15.73:0/2028237620 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a7f7fe700 1 --2- 172.21.15.73:0/2028237620 >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 0x7f3a80063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.165+0000 7f3a7ffff700 1 -- 172.21.15.73:0/2028237620 >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 msgr2=0x7f3a80063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7ffff700 1 --2- 172.21.15.73:0/2028237620 >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 0x7f3a80063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7ffff700 1 -- 172.21.15.73:0/2028237620 >> v1:172.21.15.73:6789/0 conn(0x7f3a80064350 legacy=0x7f3a80158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7effd700 1 -- 172.21.15.73:0/2028237620 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3228740764 0 0) 0x7f3a800c72d0 con 0x7f3a80064350 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7ffff700 1 -- 172.21.15.73:0/2028237620 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a800c69a0 con 0x7f3a80098d10 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7f7fe700 1 --2- 172.21.15.73:0/2028237620 >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 0x7f3a80063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7ffff700 1 --2- 172.21.15.73:0/2028237620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a800c7760 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f3a800c5b40 tx=0x7f3a70009060).ready entity=mon.0 client_cookie=3ada1ce01fd3fa10 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7effd700 1 -- 172.21.15.73:0/2028237620 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000b580 con 0x7f3a80098d10 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7effd700 1 -- 172.21.15.73:0/2028237620 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a7000b6e0 con 0x7f3a80098d10 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a7effd700 1 -- 172.21.15.73:0/2028237620 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a7000ba60 con 0x7f3a80098d10 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 -- 172.21.15.73:0/2028237620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 msgr2=0x7f3a800c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 --2- 172.21.15.73:0/2028237620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a800c7760 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 rx=0x7f3a800c5b40 tx=0x7f3a70009060).stop 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 -- 172.21.15.73:0/2028237620 shutdown_connections 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 --2- 172.21.15.73:0/2028237620 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a800c7760 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 --2- 172.21.15.73:0/2028237620 >> v2:172.21.15.150:3300/0 conn(0x7f3a800c6190 0x7f3a80063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 -- 172.21.15.73:0/2028237620 >> 172.21.15.73:0/2028237620 conn(0x7f3a800cb310 msgr2=0x7f3a800cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 -- 172.21.15.73:0/2028237620 shutdown_connections 2023-11-28T03:08:21.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.166+0000 7f3a8630d700 1 -- 172.21.15.73:0/2028237620 wait complete. 2023-11-28T03:08:21.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.167+0000 7f3a8630d700 1 Processor -- start 2023-11-28T03:08:21.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 -- start start 2023-11-28T03:08:21.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a80064350 0x7f3a80154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 0x7f3a80158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a80155250 con 0x7f3a80098d10 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a80157c10 con 0x7f3a80064350 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a8630d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a800ac430 con 0x7f3a800c6190 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a7f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55004/0 (socket says 172.21.15.73:55004) 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.168+0000 7f3a7f7fe700 1 -- 172.21.15.73:0/3896728136 learned_addr learned my addr 172.21.15.73:0/3896728136 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:21.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7ffff700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a80064350 0x7f3a80154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a848aa700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 0x7f3a80158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 -- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 msgr2=0x7f3a80158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 0x7f3a80158000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 -- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a80064350 msgr2=0x7f3a80154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a80064350 0x7f3a80154d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 -- 172.21.15.73:0/3896728136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a700092a0 con 0x7f3a80098d10 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a848aa700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 0x7f3a80158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a7f7fe700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f3a7400c8b0 tx=0x7f3a74008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74010580 con 0x7f3a80098d10 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a740106e0 con 0x7f3a80098d10 2023-11-28T03:08:21.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a74010a60 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a800ab820 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.169+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a800abde0 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.170+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a8006c860 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.170+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3a7401b460 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.171+0000 7f3a67fff700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 0x7f3a6c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.171+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3a7407a0c0 con 0x7f3a80098d10 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.171+0000 7f3a7ffff700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 0x7f3a6c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.171+0000 7f3a7ffff700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 0x7f3a6c061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3a70009fd0 tx=0x7f3a70017000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.175+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a7404b330 con 0x7f3a80098d10 2023-11-28T03:08:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.403+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f3a800c6c80 con 0x7f3a80098d10 2023-11-28T03:08:21.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.407+0000 7f3a67fff700 1 -- 172.21.15.73:0/3896728136 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) v1 ==== 190+0+0 (secure 0 0 0) 0x7f3a7404e950 con 0x7f3a80098d10 2023-11-28T03:08:21.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 msgr2=0x7f3a6c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 0x7f3a6c061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f3a70009fd0 tx=0x7f3a70017000).stop 2023-11-28T03:08:21.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 msgr2=0x7f3a8014c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 rx=0x7f3a7400c8b0 tx=0x7f3a74008040).stop 2023-11-28T03:08:21.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 shutdown_connections 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a6c05f150 0x7f3a6c061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a800c6190 0x7f3a80158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a80064350 0x7f3a80154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 --2- 172.21.15.73:0/3896728136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a80098d10 0x7f3a8014c120 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.408+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 >> 172.21.15.73:0/3896728136 conn(0x7f3a800cb310 msgr2=0x7f3a80152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.409+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 shutdown_connections 2023-11-28T03:08:21.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.409+0000 7f3a8630d700 1 -- 172.21.15.73:0/3896728136 wait complete. 2023-11-28T03:08:21.415 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:08:21.422 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:08:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.533+0000 7f919a155700 1 Processor -- start 2023-11-28T03:08:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.533+0000 7f919a155700 1 -- start start 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f919a155700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91941524a0 0x7f9194067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f919a155700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940683e0 con 0x7f9194067f30 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f919a155700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91940be110 con 0x7f91941520c0 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f919a155700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9194066fc0 con 0x7f91941524a0 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f9192ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91941524a0 0x7f9194067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f9193fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9194067f30 0x7f91940be8a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35586/0 (socket says 172.21.15.73:35586) 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f9192ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f91941524a0 0x7f9194067a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55010/0 (socket says 172.21.15.73:55010) 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.534+0000 7f9193fff700 1 -- 172.21.15.73:0/706293304 learned_addr learned my addr 172.21.15.73:0/706293304 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f9192ffd700 1 -- 172.21.15.73:0/706293304 >> v1:172.21.15.150:6789/0 conn(0x7f91941520c0 legacy=0x7f9194066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f9192ffd700 1 -- 172.21.15.73:0/706293304 >> v1:172.21.15.73:6789/0 conn(0x7f9194067f30 legacy=0x7f91940be8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f91927fc700 1 -- 172.21.15.73:0/706293304 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3288118067 0 0) 0x7f918c001110 con 0x7f9194067f30 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f9192ffd700 1 -- 172.21.15.73:0/706293304 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91940672a0 con 0x7f91941524a0 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f9192ffd700 1 --2- 172.21.15.73:0/706293304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91941524a0 0x7f9194067a40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f9188009760 tx=0x7f9188009060).ready entity=mon.0 client_cookie=f1540f4a462354b7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f91927fc700 1 -- 172.21.15.73:0/706293304 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800b520 con 0x7f91941524a0 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f91927fc700 1 -- 172.21.15.73:0/706293304 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f918800b680 con 0x7f91941524a0 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f91927fc700 1 -- 172.21.15.73:0/706293304 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918800ba00 con 0x7f91941524a0 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 -- 172.21.15.73:0/706293304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91941524a0 msgr2=0x7f9194067a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 --2- 172.21.15.73:0/706293304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91941524a0 0x7f9194067a40 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 rx=0x7f9188009760 tx=0x7f9188009060).stop 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 -- 172.21.15.73:0/706293304 shutdown_connections 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 --2- 172.21.15.73:0/706293304 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f91941524a0 0x7f9194067a40 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 -- 172.21.15.73:0/706293304 >> 172.21.15.73:0/706293304 conn(0x7f91940631d0 msgr2=0x7f91940c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 -- 172.21.15.73:0/706293304 shutdown_connections 2023-11-28T03:08:21.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.535+0000 7f919a155700 1 -- 172.21.15.73:0/706293304 wait complete. 2023-11-28T03:08:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.537+0000 7f919a155700 1 Processor -- start 2023-11-28T03:08:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.537+0000 7f919a155700 1 -- start start 2023-11-28T03:08:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.537+0000 7f919a155700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.537+0000 7f919a155700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91941520c0 0x7f91940aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.537+0000 7f919a155700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91941524a0 0x7f91940a9180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f919a155700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940ad230 con 0x7f9194067f30 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f919a155700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91940c09a0 con 0x7f91941520c0 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f919a155700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91940a9da0 con 0x7f91941524a0 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55024/0 (socket says 172.21.15.73:55024) 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 -- 172.21.15.73:0/475573157 learned_addr learned my addr 172.21.15.73:0/475573157 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 -- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91941524a0 msgr2=0x7f91940a9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91941524a0 0x7f91940a9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 -- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91941520c0 msgr2=0x7f91940aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91941520c0 0x7f91940aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 -- 172.21.15.73:0/475573157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880092a0 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f91937fe700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f918400c810 tx=0x7f9184002680).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9184006a70 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9184006c10 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91940c0350 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918400a5c0 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.538+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91940aabc0 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.539+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91940c0c80 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.540+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f918400a9b0 con 0x7f9194067f30 2023-11-28T03:08:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.540+0000 7f917b7fe700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 0x7f9180061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.540+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9184079f70 con 0x7f9194067f30 2023-11-28T03:08:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.540+0000 7f9192ffd700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 0x7f9180061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:21.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.540+0000 7f9192ffd700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 0x7f9180061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9188009760 tx=0x7f9188017000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:21.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.544+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f918404b240 con 0x7f9194067f30 2023-11-28T03:08:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.746+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f91940c0190 con 0x7f9194067f30 2023-11-28T03:08:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.747+0000 7f917b7fe700 1 -- 172.21.15.73:0/475573157 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f918404e860 con 0x7f9194067f30 2023-11-28T03:08:21.748 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:21.748 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:08:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.748+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 msgr2=0x7f9180061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.748+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 0x7f9180061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f9188009760 tx=0x7f9188017000).stop 2023-11-28T03:08:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.748+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 msgr2=0x7f91940acd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.748+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 rx=0x7f918400c810 tx=0x7f9184002680).stop 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 shutdown_connections 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f918005f150 0x7f9180061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f91941524a0 0x7f91940a9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9194067f30 0x7f91940acd40 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 --2- 172.21.15.73:0/475573157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f91941520c0 0x7f91940aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 >> 172.21.15.73:0/475573157 conn(0x7f91940631d0 msgr2=0x7f91940c6340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 shutdown_connections 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:21.749+0000 7f919a155700 1 -- 172.21.15.73:0/475573157 wait complete. 2023-11-28T03:08:21.751 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:08:22.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 40, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:08:17.668973+0000', 'modified': '2023-11-28T03:08:19.679287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7128}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7128': {'gid': 7128, 'name': 'c', 'rank': 0, 'incarnation': 38, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6836/2863774554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 2863774554}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 2863774554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [14], 'metadata_pool': 13, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:08:22.164 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:08:22.164 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T03:08:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.279+0000 7f66602ce700 1 Processor -- start 2023-11-28T03:08:22.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.279+0000 7f66602ce700 1 -- start start 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f66602ce700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f66602ce700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f66602ce700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6658158cc0 con 0x7f66580c6190 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f66602ce700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66580c72d0 con 0x7f6658098d10 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f66602ce700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6658064570 con 0x7f6658158320 2023-11-28T03:08:22.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665e06a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665e86b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665e86b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:41136/0 (socket says 172.21.15.73:41136) 2023-11-28T03:08:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665e06a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:46116/0 (socket says 172.21.15.73:46116) 2023-11-28T03:08:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665d869700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f66580c6190 0x7f6658063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:58870/0 (socket says 172.21.15.73:58870) 2023-11-28T03:08:22.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.280+0000 7f665e86b700 1 -- 172.21.15.73:0/750360281 learned_addr learned my addr 172.21.15.73:0/750360281 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e86b700 1 -- 172.21.15.73:0/750360281 >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 msgr2=0x7f66580c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e86b700 1 --2- 172.21.15.73:0/750360281 >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e86b700 1 -- 172.21.15.73:0/750360281 >> v1:172.21.15.150:6789/0 conn(0x7f66580c6190 legacy=0x7f6658063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e86b700 1 -- 172.21.15.73:0/750360281 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6658154bc0 con 0x7f6658158320 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e06a700 1 --2- 172.21.15.73:0/750360281 >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665e86b700 1 --2- 172.21.15.73:0/750360281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f6650009f20 tx=0x7f6650009840).ready entity=mon.2 client_cookie=a9378be356a65361 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665d068700 1 -- 172.21.15.73:0/750360281 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000e070 con 0x7f6658158320 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665d068700 1 -- 172.21.15.73:0/750360281 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665000b9d0 con 0x7f6658158320 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f665d068700 1 -- 172.21.15.73:0/750360281 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665000bd50 con 0x7f6658158320 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f66602ce700 1 -- 172.21.15.73:0/750360281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 msgr2=0x7f66581587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.281+0000 7f66602ce700 1 --2- 172.21.15.73:0/750360281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 rx=0x7f6650009f20 tx=0x7f6650009840).stop 2023-11-28T03:08:22.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 -- 172.21.15.73:0/750360281 shutdown_connections 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 --2- 172.21.15.73:0/750360281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f66581587d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 --2- 172.21.15.73:0/750360281 >> v2:172.21.15.150:3300/0 conn(0x7f6658098d10 0x7f66580c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 -- 172.21.15.73:0/750360281 >> 172.21.15.73:0/750360281 conn(0x7f66580cb310 msgr2=0x7f66580cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 -- 172.21.15.73:0/750360281 shutdown_connections 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.282+0000 7f66602ce700 1 -- 172.21.15.73:0/750360281 wait complete. 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 Processor -- start 2023-11-28T03:08:22.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 -- start start 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66580c6190 0x7f6658150ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f6658151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6658151680 con 0x7f6658098d10 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f665814d4f0 con 0x7f66580c6190 2023-11-28T03:08:22.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f66602ce700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f665814c8e0 con 0x7f6658158320 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f665e06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.283+0000 7f665e06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55038/0 (socket says 172.21.15.73:55038) 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 -- 172.21.15.73:0/2430426103 learned_addr learned my addr 172.21.15.73:0/2430426103 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e86b700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f6658151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665d869700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66580c6190 0x7f6658150ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 -- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 msgr2=0x7f6658151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f6658151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 -- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66580c6190 msgr2=0x7f6658150ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66580c6190 0x7f6658150ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 -- 172.21.15.73:0/2430426103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66540092a0 con 0x7f6658098d10 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f665e06a700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f665400ea20 tx=0x7f6654009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f664e7fc700 1 -- 172.21.15.73:0/2430426103 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400c850 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f664e7fc700 1 -- 172.21.15.73:0/2430426103 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f665400c9b0 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6650009ae0 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f664e7fc700 1 -- 172.21.15.73:0/2430426103 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665400cd30 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.284+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f665814cea0 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.285+0000 7f664e7fc700 1 -- 172.21.15.73:0/2430426103 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6654012600 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.286+0000 7f664e7fc700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 0x7f663c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.286+0000 7f664e7fc700 1 -- 172.21.15.73:0/2430426103 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f665407a890 con 0x7f6658098d10 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.286+0000 7f665d869700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 0x7f663c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.286+0000 7f665d869700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 0x7f663c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f66580989f0 tx=0x7f6648007bf0).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.288+0000 7f6671c66b80 1 Processor -- start 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.288+0000 7f6671c66b80 1 -- start start 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555775736e40 0x5557759a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x555775975600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555775940060 con 0x5557759c3ea0 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557758b71f0 con 0x555775736e40 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557758cc440 con 0x55577597acb0 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55046/0 (socket says 172.21.15.73:55046) 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 -- 172.21.15.73:0/70872296 learned_addr learned my addr 172.21.15.73:0/70872296 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665e86b700 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x555775975600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665e06a700 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555775736e40 0x5557759a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 -- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 msgr2=0x555775975600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x555775975600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 -- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555775736e40 msgr2=0x5557759a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555775736e40 0x5557759a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 -- 172.21.15.73:0/70872296 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557758c5620 con 0x5557759c3ea0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f665d869700 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f6648018690 tx=0x7f664800a000).ready entity=mon.0 client_cookie=cecb56ae5425be8d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f663b7fe700 1 -- 172.21.15.73:0/70872296 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6648018240 con 0x5557759c3ea0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f663b7fe700 1 -- 172.21.15.73:0/70872296 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648015410 con 0x5557759c3ea0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.289+0000 7f663b7fe700 1 -- 172.21.15.73:0/70872296 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66480157d0 con 0x5557759c3ea0 2023-11-28T03:08:22.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 -- 172.21.15.73:0/70872296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 msgr2=0x55577597a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 rx=0x7f6648018690 tx=0x7f664800a000).stop 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 -- 172.21.15.73:0/70872296 shutdown_connections 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x555775975600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x555775736e40 0x5557759a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 --2- 172.21.15.73:0/70872296 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5557759c3ea0 0x55577597a7c0 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 -- 172.21.15.73:0/70872296 >> 172.21.15.73:0/70872296 conn(0x5557757e44a0 msgr2=0x5557759c3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 -- 172.21.15.73:0/70872296 shutdown_connections 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.290+0000 7f6671c66b80 1 -- 172.21.15.73:0/70872296 wait complete. 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.291+0000 7f6671c66b80 1 -- start start 2023-11-28T03:08:22.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.291+0000 7f6671c66b80 1 Processor -- start 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x5557759bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759bf3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557758cc580 con 0x555775736e40 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557758cc800 con 0x5557759c3ea0 2023-11-28T03:08:22.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x555775897e10 con 0x55577597acb0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 10 client.0 ll_register_callbacks cb 0x55577593a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55062/0 (socket says 172.21.15.73:55062) 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 -- 172.21.15.73:0/908392986 learned_addr learned my addr 172.21.15.73:0/908392986 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665d869700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x5557759bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e86b700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759bf3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 -- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 msgr2=0x5557759bef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x5557759bef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 -- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 msgr2=0x5557759bf3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759bf3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 -- 172.21.15.73:0/908392986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f664800f040 con 0x555775736e40 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f665e06a700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f6654009fd0 tx=0x7f6654051040).ready entity=mon.0 client_cookie=a79bd9d9b389fe7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6638ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f665404f220 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6654019df0 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55577596e720 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f665404efc0 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 10 client.7284 fetch_fsmap learned FSMap version 40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.292+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x5557758c09c0 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55577596e720 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f665404faf0 con 0x555775736e40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 10 client.7284 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 10 client.7284 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T03:08:22.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 20 client.7284 populate_metadata read hostname 'smithi073' 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d1750 0x5557759d1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 4 client.7284 mds_command: new command op to 7128 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 --> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555775732ca0 con 0x5557759d1750 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f665d869700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d1750 0x5557759d1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f6638ff9700 10 client.7284 ms_handle_connect on v2:172.21.15.73:6834/2863774554 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.293+0000 7f665d869700 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d1750 0x5557759d1b30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=4e2d252eb314e1ba server_cookie=42e4f57bffceb50f in_seq=0 out_seq=0 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6638ff9700 1 -- 172.21.15.73:0/908392986 <== mds.0 v2:172.21.15.73:6834/2863774554 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x555775732ca0 con 0x5557759d1750 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6638ff9700 10 client.7284 handle_command_reply: tid=0 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 client.7284 shutdown 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 msgr2=0x5557759a90d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 rx=0x7f6654009fd0 tx=0x7f6654051040).stop 2023-11-28T03:08:22.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 shutdown_connections 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d1750 0x5557759d1b30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759bf3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6638ff9700 0 client.7284 ms_handle_reset on v2:172.21.15.73:6834/2863774554 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x555775736e40 0x5557759a90d0 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 --2- 172.21.15.73:0/908392986 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55577597acb0 0x5557759bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 >> 172.21.15.73:0/908392986 conn(0x5557757e44a0 msgr2=0x55577597b110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 shutdown_connections 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 1 -- 172.21.15.73:0/908392986 wait complete. 2023-11-28T03:08:22.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.294+0000 7f6671c66b80 20 client.7284 trim_cache size 0 max 16384 2023-11-28T03:08:22.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.310+0000 7f6671c66b80 1 Processor -- start 2023-11-28T03:08:22.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.310+0000 7f6671c66b80 1 -- start start 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x5557759c0230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5557759cf3c0 con 0x55577597acb0 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55577597b1c0 con 0x5557759c3ea0 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557759a1ef0 con 0x555775736e40 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55078/0 (socket says 172.21.15.73:55078) 2023-11-28T03:08:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f665d869700 1 -- 172.21.15.73:0/2861750407 learned_addr learned my addr 172.21.15.73:0/2861750407 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.311+0000 7f665d869700 1 -- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 msgr2=0x5557759c0230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f665d869700 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x5557759c0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f665d869700 1 -- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 msgr2=0x5557759ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f665d869700 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f665d869700 1 -- 172.21.15.73:0/2861750407 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5557758c0710 con 0x55577597acb0 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f665d869700 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f6648018c80 tx=0x7f6648015000).ready entity=mon.0 client_cookie=ab8e52756b58b3a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f663bfff700 1 -- 172.21.15.73:0/2861750407 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664802b040 con 0x55577597acb0 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f663bfff700 1 -- 172.21.15.73:0/2861750407 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648019e10 con 0x55577597acb0 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f663bfff700 1 -- 172.21.15.73:0/2861750407 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664802f280 con 0x55577597acb0 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 -- 172.21.15.73:0/2861750407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 msgr2=0x5557759ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 rx=0x7f6648018c80 tx=0x7f6648015000).stop 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 -- 172.21.15.73:0/2861750407 shutdown_connections 2023-11-28T03:08:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x5557759c0230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2861750407 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 -- 172.21.15.73:0/2861750407 >> 172.21.15.73:0/2861750407 conn(0x5557757e44a0 msgr2=0x5557759b9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 -- 172.21.15.73:0/2861750407 shutdown_connections 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.312+0000 7f6671c66b80 1 -- 172.21.15.73:0/2861750407 wait complete. 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.313+0000 7f6671c66b80 1 -- start start 2023-11-28T03:08:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.313+0000 7f6671c66b80 1 Processor -- start 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x555775a0d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x555775a0dae0 con 0x55577597acb0 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x555775a244a0 con 0x5557759c3ea0 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5557759a7930 con 0x555775736e40 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6671c66b80 10 client.0 ll_register_callbacks cb 0x55577593a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55082/0 (socket says 172.21.15.73:55082) 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 -- 172.21.15.73:0/2518715976 learned_addr learned my addr 172.21.15.73:0/2518715976 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665e06a700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x555775a0d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665e86b700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 -- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 msgr2=0x555775a0d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x555775a0d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 -- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 msgr2=0x5557759ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 -- 172.21.15.73:0/2518715976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5557758c5620 con 0x55577597acb0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f665d869700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x5557759cf500 tx=0x7f6648027070).ready entity=mon.0 client_cookie=17d537dc7f385556 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6638ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f664802b030 con 0x55577597acb0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.314+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6648030660 con 0x55577597acb0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6648015410 con 0x55577597acb0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=40) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6648030a30 con 0x55577597acb0 2023-11-28T03:08:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 10 client.7290 fetch_fsmap learned FSMap version 40 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=40,monmap=2+}) v3 -- 0x555775737f80 con 0x55577597acb0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6648015410 con 0x55577597acb0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 40) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6648037070 con 0x55577597acb0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 10 client.7290 fetch_fsmap finished waiting for FSMap version 40 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 10 client.7290 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 20 client.7290 populate_metadata read hostname 'smithi073' 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d7da0 0x5557759d8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 4 client.7290 mds_command: new command op to 7128 tid=0[{"prefix": "status"}] 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 --> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] -- command(tid 0: {"prefix": "status"}) v1 -- 0x555775769ca0 con 0x5557759d7da0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f665e06a700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d7da0 0x5557759d8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 10 client.7290 ms_handle_connect on v2:172.21.15.73:6834/2863774554 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f665e06a700 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d7da0 0x5557759d8180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e32ce5772469f48f server_cookie=fc8359ecb5001aa9 in_seq=0 out_seq=0 2023-11-28T03:08:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 1 -- 172.21.15.73:0/2518715976 <== mds.0 v2:172.21.15.73:6834/2863774554 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x555775769ca0 con 0x5557759d7da0 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6638ff9700 10 client.7290 handle_command_reply: tid=0 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.315+0000 7f6671c66b80 1 client.7290 shutdown 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 msgr2=0x5557759ce310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 rx=0x5557759cf500 tx=0x7f6648027070).stop 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 shutdown_connections 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:6834/2863774554,v1:172.21.15.73:6836/2863774554] conn(0x5557759d7da0 0x5557759d8180 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5557759c3ea0 0x5557759ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6638ff9700 0 client.7290 ms_handle_reset on v2:172.21.15.73:6834/2863774554 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x555775736e40 0x555775a0d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 --2- 172.21.15.73:0/2518715976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55577597acb0 0x5557759ce310 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 >> 172.21.15.73:0/2518715976 conn(0x5557757e44a0 msgr2=0x555775978680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 shutdown_connections 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 1 -- 172.21.15.73:0/2518715976 wait complete. 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f6671c66b80 20 client.7290 trim_cache size 0 max 16384 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 7128, 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:08:22.324 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 4.6462355630000003, 2023-11-28T03:08:22.325 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 40, 2023-11-28T03:08:22.325 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 110, 2023-11-28T03:08:22.325 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 110, 2023-11-28T03:08:22.325 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 9.1591537990000003 2023-11-28T03:08:22.325 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 msgr2=0x7f663c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 0x7f663c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f66580989f0 tx=0x7f6648007bf0).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 msgr2=0x7f665814b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 rx=0x7f665400ea20 tx=0x7f6654009060).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 shutdown_connections 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f663c05f150 0x7f663c061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6658158320 0x7f6658151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6658098d10 0x7f665814b530 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 --2- 172.21.15.73:0/2430426103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66580c6190 0x7f6658150ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 >> 172.21.15.73:0/2430426103 conn(0x7f66580cb310 msgr2=0x7f66580ca3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 shutdown_connections 2023-11-28T03:08:22.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.316+0000 7f66602ce700 1 -- 172.21.15.73:0/2430426103 wait complete. 2023-11-28T03:08:22.329 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:08:22.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.429+0000 7f6eaf349700 1 Processor -- start 2023-11-28T03:08:22.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.429+0000 7f6eaf349700 1 -- start start 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eaf349700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea80bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eaf349700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea80bed20 con 0x7f6ea80c3670 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eaf349700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ea80be430 con 0x7f6ea80671a0 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eaf349700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80be6d0 con 0x7f6ea8152190 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eac8e4700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6ea80c3670 0x7f6ea8066a90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35588/0 (socket says 172.21.15.73:35588) 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6eac8e4700 1 -- 172.21.15.73:0/1590230153 learned_addr learned my addr 172.21.15.73:0/1590230153 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6ead0e5700 1 --2- 172.21.15.73:0/1590230153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea80bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099938028 0 0) 0x7f6ea80bed20 con 0x7f6ea80c3670 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e940033f0 con 0x7f6ea80c3670 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1426434768 0 0) 0x7f6ea80be430 con 0x7f6ea80671a0 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ea80bed20 con 0x7f6ea80671a0 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6ead0e5700 1 -- 172.21.15.73:0/1590230153 >> v1:172.21.15.150:6789/0 conn(0x7f6ea80671a0 legacy=0x7f6ea80a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2257799119 0 0) 0x7f6e940033f0 con 0x7f6ea80c3670 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6ead0e5700 1 -- 172.21.15.73:0/1590230153 >> v1:172.21.15.73:6789/0 conn(0x7f6ea80c3670 legacy=0x7f6ea8066a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6ead0e5700 1 -- 172.21.15.73:0/1590230153 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea80be9b0 con 0x7f6ea8152190 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.430+0000 7f6ead0e5700 1 --2- 172.21.15.73:0/1590230153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea80bf1b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6ea4009790 tx=0x7f6ea4009060).ready entity=mon.2 client_cookie=7abac3d38c339bc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98001470 con 0x7f6ea8152190 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ea4011400 con 0x7f6ea8152190 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6e9ffff700 1 -- 172.21.15.73:0/1590230153 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ea4011780 con 0x7f6ea8152190 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 -- 172.21.15.73:0/1590230153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 msgr2=0x7f6ea80bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 --2- 172.21.15.73:0/1590230153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea80bf1b0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 rx=0x7f6ea4009790 tx=0x7f6ea4009060).stop 2023-11-28T03:08:22.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 -- 172.21.15.73:0/1590230153 shutdown_connections 2023-11-28T03:08:22.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 --2- 172.21.15.73:0/1590230153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea80bf1b0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 -- 172.21.15.73:0/1590230153 >> 172.21.15.73:0/1590230153 conn(0x7f6ea80c9070 msgr2=0x7f6ea80c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 -- 172.21.15.73:0/1590230153 shutdown_connections 2023-11-28T03:08:22.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.431+0000 7f6eaf349700 1 -- 172.21.15.73:0/1590230153 wait complete. 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.432+0000 7f6eaf349700 1 Processor -- start 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.432+0000 7f6eaf349700 1 -- start start 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80671a0 0x7f6ea814ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea81531a0 con 0x7f6ea80c3670 2023-11-28T03:08:22.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80ad1b0 con 0x7f6ea80671a0 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eaf349700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ea80ace80 con 0x7f6ea8152190 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eac8e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eac8e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55084/0 (socket says 172.21.15.73:55084) 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.433+0000 7f6eac8e4700 1 -- 172.21.15.73:0/2933927951 learned_addr learned my addr 172.21.15.73:0/2933927951 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6ead0e5700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80671a0 0x7f6ea814ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6ead8e6700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 -- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 msgr2=0x7f6ea8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 -- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80671a0 msgr2=0x7f6ea814ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80671a0 0x7f6ea814ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 -- 172.21.15.73:0/2933927951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ea40092a0 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eac8e4700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f6e9800c790 tx=0x7f6e98001a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e98007320 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e98007480 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ea8156ff0 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e980059f0 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.434+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ea81575b0 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.435+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ea80ad340 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.435+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6e98013410 con 0x7f6ea80c3670 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.436+0000 7f6e9d7fa700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 0x7f6e8c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:22.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.436+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6e9807a340 con 0x7f6ea80c3670 2023-11-28T03:08:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.436+0000 7f6ead0e5700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 0x7f6e8c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.436+0000 7f6ead0e5700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 0x7f6e8c061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6ea40096e0 tx=0x7f6ea4016000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:22.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.440+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6e9804b4f0 con 0x7f6ea80c3670 2023-11-28T03:08:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.643+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6ea806c380 con 0x7f6ea80c3670 2023-11-28T03:08:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.644+0000 7f6e9d7fa700 1 -- 172.21.15.73:0/2933927951 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v40) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6e9804eb10 con 0x7f6ea80c3670 2023-11-28T03:08:22.645 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:08:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 msgr2=0x7f6e8c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 0x7f6e8c061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f6ea40096e0 tx=0x7f6ea4016000).stop 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 msgr2=0x7f6ea81527c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 rx=0x7f6e9800c790 tx=0x7f6e98001a50).stop 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 shutdown_connections 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6e8c05f150 0x7f6e8c061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ea8152190 0x7f6ea8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ea80671a0 0x7f6ea814ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 --2- 172.21.15.73:0/2933927951 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ea80c3670 0x7f6ea81527c0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 >> 172.21.15.73:0/2933927951 conn(0x7f6ea80c9070 msgr2=0x7f6ea80c6e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 shutdown_connections 2023-11-28T03:08:22.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:22.646+0000 7f6eaf349700 1 -- 172.21.15.73:0/2933927951 wait complete. 2023-11-28T03:08:22.658 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:08:22.658 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:08:22.658 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:08:22.658 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:08:22.658 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:08:22.659 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:08:22.659 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:08:22.659 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:08:22.659 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:08:22.659 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.659 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:08:22.675 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:08:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:08:22.675 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:08:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:08:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.676 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:08:22.676 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:08:22.677 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:08:22.678 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:08:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:08:22.678 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:08:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41577sec preferred_lft 41577sec 2023-11-28T03:08:22.679 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.680 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:08:22.681 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:08:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:08:22.682 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:08:22.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.682 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:08:22.738 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:08:22.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.740 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:08:22.796 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:08:22.796 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:08:22.796 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:08:22.796 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:08:22.797 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:08:22.797 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.797 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:08:22.797 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:08:22.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:08:22.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:08:22.798 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:08:22.798 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:08:22.863 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:08:22.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:08:22.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.896 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:08:22.950 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:08:22.951 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:08:22.951 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:08:22.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:22.951 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:08:23.025 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:23.025 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:08:23.080 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:08:23.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:08:23.158 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:08:23.158 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:08:23.183 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:08:23.183 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:08:23.185 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:08:23.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:08:23.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:08:23.249 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:08:23.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.403+0000 7f3e6d5bf700 1 Processor -- start 2023-11-28T03:08:23.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.403+0000 7f3e6d5bf700 1 -- start start 2023-11-28T03:08:23.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e6d5bf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e68098d10 0x7f3e680c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e6d5bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e68158320 0x7f3e681587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e6d5bf700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e68158cc0 con 0x7f3e680c6190 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e6d5bf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e680c72d0 con 0x7f3e68098d10 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e6d5bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e68064570 con 0x7f3e68158320 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e68098d10 0x7f3e680c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e5e7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3e680c6190 0x7f3e68063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38712/0 (socket says 172.21.15.73:38712) 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e5e7fc700 1 -- 172.21.15.73:0/1861268089 learned_addr learned my addr 172.21.15.73:0/1861268089 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e68098d10 0x7f3e680c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45382/0 (socket says 172.21.15.73:45382) 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1861268089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e68158320 msgr2=0x7f3e681587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 --2- 172.21.15.73:0/1861268089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e68158320 0x7f3e681587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1861268089 >> v1:172.21.15.73:6789/0 conn(0x7f3e680c6190 legacy=0x7f3e68063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1861268089 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e68154bc0 con 0x7f3e68098d10 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.404+0000 7f3e667fc700 1 -- 172.21.15.73:0/1861268089 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 975816703 0 0) 0x7f3e50001110 con 0x7f3e680c6190 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e66ffd700 1 --2- 172.21.15.73:0/1861268089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e680c7760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3e58009760 tx=0x7f3e58009060).ready entity=mon.0 client_cookie=84ce68bf834699a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e667fc700 1 -- 172.21.15.73:0/1861268089 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800b580 con 0x7f3e68098d10 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e667fc700 1 -- 172.21.15.73:0/1861268089 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5800b6e0 con 0x7f3e68098d10 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e667fc700 1 -- 172.21.15.73:0/1861268089 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800ba60 con 0x7f3e68098d10 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1861268089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 msgr2=0x7f3e680c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1861268089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e680c7760 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 rx=0x7f3e58009760 tx=0x7f3e58009060).stop 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1861268089 shutdown_connections 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1861268089 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e68158320 0x7f3e681587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1861268089 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e680c7760 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1861268089 >> 172.21.15.73:0/1861268089 conn(0x7f3e680cb310 msgr2=0x7f3e680cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1861268089 shutdown_connections 2023-11-28T03:08:23.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.405+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1861268089 wait complete. 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.406+0000 7f3e6d5bf700 1 Processor -- start 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.406+0000 7f3e6d5bf700 1 -- start start 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 0x7f3e68155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 0x7f3e68156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e680aed00 con 0x7f3e68098d10 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e68157460 con 0x7f3e68158320 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e6d5bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e681567e0 con 0x7f3e680c6190 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e66ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45396/0 (socket says 172.21.15.73:45396) 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1125551075 learned_addr learned my addr 172.21.15.73:0/1125551075 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.407+0000 7f3e5e7fc700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 0x7f3e68155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e677fe700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 0x7f3e68156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 msgr2=0x7f3e68155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 0x7f3e68155d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 msgr2=0x7f3e68156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 0x7f3e68156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 -- 172.21.15.73:0/1125551075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e580092a0 con 0x7f3e68098d10 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e5e7fc700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 0x7f3e68155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e677fe700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 0x7f3e68156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e66ffd700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f3e680c2360 tx=0x7f3e58017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5800b850 con 0x7f3e68098d10 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e5800b9b0 con 0x7f3e68098d10 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e68156ac0 con 0x7f3e68098d10 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e5801b660 con 0x7f3e68098d10 2023-11-28T03:08:23.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.408+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e68157080 con 0x7f3e68098d10 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.409+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e68083380 con 0x7f3e68098d10 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.409+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3e5801b9d0 con 0x7f3e68098d10 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.410+0000 7f3e5f7fe700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 0x7f3e48061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.410+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3e5807be90 con 0x7f3e68098d10 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.410+0000 7f3e5e7fc700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 0x7f3e48061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.410+0000 7f3e5e7fc700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 0x7f3e48061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3e681463b0 tx=0x7f3e50009040).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.414+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e5804d160 con 0x7f3e68098d10 2023-11-28T03:08:23.600 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:08:23.599+0000 7f5fb6828700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:08:23.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.604+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3e68158120 con 0x7f3e4805f150 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.605+0000 7f3e5f7fe700 1 -- 172.21.15.73:0/1125551075 <== mgr.5837 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3e68158120 con 0x7f3e4805f150 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:08:23.606 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:08:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 msgr2=0x7f3e48061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 0x7f3e48061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3e681463b0 tx=0x7f3e50009040).stop 2023-11-28T03:08:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 msgr2=0x7f3e6813f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 rx=0x7f3e680c2360 tx=0x7f3e58017000).stop 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 shutdown_connections 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e4805f150 0x7f3e48061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e68158320 0x7f3e68156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e68098d10 0x7f3e6813f6a0 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 --2- 172.21.15.73:0/1125551075 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e680c6190 0x7f3e68155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 >> 172.21.15.73:0/1125551075 conn(0x7f3e680cb310 msgr2=0x7f3e680aa590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 shutdown_connections 2023-11-28T03:08:23.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.607+0000 7f3e6d5bf700 1 -- 172.21.15.73:0/1125551075 wait complete. 2023-11-28T03:08:23.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:08:23.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.734+0000 7f660c578700 1 Processor -- start 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.734+0000 7f660c578700 1 -- start start 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660c578700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6604098d10 0x7f66040c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660c578700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 0x7f6604063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660c578700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66040c72d0 con 0x7f6604064350 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660c578700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6604154bc0 con 0x7f6604098d10 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660c578700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040c66c0 con 0x7f66040c6190 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660a314700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6604098d10 0x7f66040c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660ab15700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6604064350 0x7f6604158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38726/0 (socket says 172.21.15.73:38726) 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660ab15700 1 -- 172.21.15.73:0/2673450472 learned_addr learned my addr 172.21.15.73:0/2673450472 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.735+0000 7f660a314700 1 --2- 172.21.15.73:0/2673450472 >> v2:172.21.15.73:3300/0 conn(0x7f6604098d10 0x7f66040c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45408/0 (socket says 172.21.15.73:45408) 2023-11-28T03:08:23.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660a314700 1 -- 172.21.15.73:0/2673450472 >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 msgr2=0x7f6604063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609b13700 1 --2- 172.21.15.73:0/2673450472 >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 0x7f6604063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660a314700 1 --2- 172.21.15.73:0/2673450472 >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 0x7f6604063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609312700 1 -- 172.21.15.73:0/2673450472 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 777220541 0 0) 0x7f66040c72d0 con 0x7f6604064350 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660a314700 1 -- 172.21.15.73:0/2673450472 >> v1:172.21.15.73:6789/0 conn(0x7f6604064350 legacy=0x7f6604158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660a314700 1 -- 172.21.15.73:0/2673450472 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040c69a0 con 0x7f6604098d10 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609b13700 1 --2- 172.21.15.73:0/2673450472 >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 0x7f6604063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660a314700 1 --2- 172.21.15.73:0/2673450472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6604098d10 0x7f66040c7760 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f66040c5b40 tx=0x7f6600009060).ready entity=mon.0 client_cookie=5e7c2376d6098cea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609312700 1 -- 172.21.15.73:0/2673450472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000b580 con 0x7f6604098d10 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609312700 1 -- 172.21.15.73:0/2673450472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f660000b6e0 con 0x7f6604098d10 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f6609312700 1 -- 172.21.15.73:0/2673450472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f660000ba60 con 0x7f6604098d10 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 -- 172.21.15.73:0/2673450472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6604098d10 msgr2=0x7f66040c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 --2- 172.21.15.73:0/2673450472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6604098d10 0x7f66040c7760 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 rx=0x7f66040c5b40 tx=0x7f6600009060).stop 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 -- 172.21.15.73:0/2673450472 shutdown_connections 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 --2- 172.21.15.73:0/2673450472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6604098d10 0x7f66040c7760 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 --2- 172.21.15.73:0/2673450472 >> v2:172.21.15.150:3300/0 conn(0x7f66040c6190 0x7f6604063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.736+0000 7f660c578700 1 -- 172.21.15.73:0/2673450472 >> 172.21.15.73:0/2673450472 conn(0x7f66040cb310 msgr2=0x7f66040cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.737+0000 7f660c578700 1 -- 172.21.15.73:0/2673450472 shutdown_connections 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.737+0000 7f660c578700 1 -- 172.21.15.73:0/2673450472 wait complete. 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 Processor -- start 2023-11-28T03:08:23.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 -- start start 2023-11-28T03:08:23.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6604064350 0x7f6604154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604098d10 0x7f660414c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6604155250 con 0x7f66040c6190 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6604157c10 con 0x7f6604098d10 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660c578700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040ac430 con 0x7f6604064350 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660ab15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660ab15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45412/0 (socket says 172.21.15.73:45412) 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.738+0000 7f660ab15700 1 -- 172.21.15.73:0/2150077158 learned_addr learned my addr 172.21.15.73:0/2150077158 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660a314700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6604064350 0x7f6604154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f6609b13700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604098d10 0x7f660414c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 -- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6604064350 msgr2=0x7f6604154d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6604064350 0x7f6604154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 -- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604098d10 msgr2=0x7f660414c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604098d10 0x7f660414c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 -- 172.21.15.73:0/2150077158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66000092a0 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660ab15700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f65fc00c810 tx=0x7f65fc007f90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc005d30 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65fc005e90 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040ab820 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65fc007320 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.739+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040abde0 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.740+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660406c860 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.740+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 38) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f65fc007710 con 0x7f66040c6190 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.741+0000 7f65fa7fc700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 0x7f65e8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:23.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.741+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f65fc07a270 con 0x7f66040c6190 2023-11-28T03:08:23.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.741+0000 7f660a314700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 0x7f65e8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:23.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.741+0000 7f660a314700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 0x7f65e8061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f660000b340 tx=0x7f6600017000).ready entity=mgr.5837 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:23.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.745+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f65fc04b4e0 con 0x7f66040c6190 2023-11-28T03:08:23.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:23.954+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f66040ac570 con 0x7f66040c6190 2023-11-28T03:08:24.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.261+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 39) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f65fc078040 con 0x7f66040c6190 2023-11-28T03:08:24.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.262+0000 7f65fa7fc700 1 -- 172.21.15.73:0/2150077158 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7f65fc072870 con 0x7f66040c6190 2023-11-28T03:08:24.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 msgr2=0x7f65e8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 0x7f65e8061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f660000b340 tx=0x7f6600017000).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 msgr2=0x7f6604158000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 rx=0x7f65fc00c810 tx=0x7f65fc007f90).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 shutdown_connections 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f65e805f150 0x7f65e8061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66040c6190 0x7f6604158000 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6604064350 0x7f6604154d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 --2- 172.21.15.73:0/2150077158 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6604098d10 0x7f660414c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 >> 172.21.15.73:0/2150077158 conn(0x7f66040cb310 msgr2=0x7f6604152c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 shutdown_connections 2023-11-28T03:08:24.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.264+0000 7f660c578700 1 -- 172.21.15.73:0/2150077158 wait complete. 2023-11-28T03:08:24.281 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:08:24.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.400+0000 7eff25444700 1 Processor -- start 2023-11-28T03:08:24.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.400+0000 7eff25444700 1 -- start start 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 0x7eff200a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff20158160 0x7eff20155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff2014e600 0x7eff2014eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff20159180 con 0x7eff20158160 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff20067770 con 0x7eff2014e600 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff25444700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff200be8d0 con 0x7eff200a8280 2023-11-28T03:08:24.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff20158160 0x7eff20155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff20158160 0x7eff20155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45422/0 (socket says 172.21.15.73:45422) 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 -- 172.21.15.73:0/4094217608 learned_addr learned my addr 172.21.15.73:0/4094217608 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1f7fe700 1 --2- 172.21.15.73:0/4094217608 >> v2:172.21.15.150:3300/0 conn(0x7eff2014e600 0x7eff2014eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1effd700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 0x7eff200a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 -- 172.21.15.73:0/4094217608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 msgr2=0x7eff200a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 0x7eff200a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 -- 172.21.15.73:0/4094217608 >> v2:172.21.15.150:3300/0 conn(0x7eff2014e600 msgr2=0x7eff2014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 --2- 172.21.15.73:0/4094217608 >> v2:172.21.15.150:3300/0 conn(0x7eff2014e600 0x7eff2014eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.401+0000 7eff1e7fc700 1 -- 172.21.15.73:0/4094217608 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff20158600 con 0x7eff20158160 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff1effd700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 0x7eff200a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff1e7fc700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff20158160 0x7eff20155500 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7eff200be410 tx=0x7eff100090f0).ready entity=mon.0 client_cookie=48f85a4c2e004f6a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff1dffb700 1 -- 172.21.15.73:0/4094217608 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000b520 con 0x7eff20158160 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff1dffb700 1 -- 172.21.15.73:0/4094217608 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1000b680 con 0x7eff20158160 2023-11-28T03:08:24.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff1dffb700 1 -- 172.21.15.73:0/4094217608 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1000ba00 con 0x7eff20158160 2023-11-28T03:08:24.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 -- 172.21.15.73:0/4094217608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff20158160 msgr2=0x7eff20155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff20158160 0x7eff20155500 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 rx=0x7eff200be410 tx=0x7eff100090f0).stop 2023-11-28T03:08:24.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 -- 172.21.15.73:0/4094217608 shutdown_connections 2023-11-28T03:08:24.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 --2- 172.21.15.73:0/4094217608 >> v2:172.21.15.150:3300/0 conn(0x7eff2014e600 0x7eff2014eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff200a8280 0x7eff200a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 --2- 172.21.15.73:0/4094217608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff20158160 0x7eff20155500 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 -- 172.21.15.73:0/4094217608 >> 172.21.15.73:0/4094217608 conn(0x7eff20062700 msgr2=0x7eff20062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.402+0000 7eff25444700 1 -- 172.21.15.73:0/4094217608 shutdown_connections 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.403+0000 7eff25444700 1 -- 172.21.15.73:0/4094217608 wait complete. 2023-11-28T03:08:24.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.404+0000 7eff25444700 1 Processor -- start 2023-11-28T03:08:24.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.404+0000 7eff25444700 1 -- start start 2023-11-28T03:08:24.411 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:24.404+0000 7f6a001f6680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:08:24.413 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:24.406+0000 7fd4d70d4680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.404+0000 7eff25444700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff2014e600 0x7eff200c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff20158160 0x7eff20148e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff200c8270 con 0x7eff200a8280 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff20064910 con 0x7eff2014e600 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff200c8480 con 0x7eff20158160 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45432/0 (socket says 172.21.15.73:45432) 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 -- 172.21.15.73:0/3784197727 learned_addr learned my addr 172.21.15.73:0/3784197727 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:24.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1e7fc700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff2014e600 0x7eff200c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1f7fe700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff20158160 0x7eff20148e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 -- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff20158160 msgr2=0x7eff20148e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff20158160 0x7eff20148e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 -- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff2014e600 msgr2=0x7eff200c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff2014e600 0x7eff200c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 -- 172.21.15.73:0/3784197727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff10009370 con 0x7eff200a8280 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff1effd700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7eff1400d860 tx=0x7eff14008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:24.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400f520 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1400f680 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff200c8760 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1400fa00 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.405+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff200645a0 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.406+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff200c6880 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.407+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 39) v1 ==== 75463+0+0 (secure 0 0 0) 0x7eff140123e0 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.407+0000 7eff06ffd700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 0x7eff0c061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.407+0000 7eff1e7fc700 1 -- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 msgr2=0x7eff0c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.407+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7eff14079d40 con 0x7eff200a8280 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.407+0000 7eff1e7fc700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 0x7eff0c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:24.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.411+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff14049af0 con 0x7eff200a8280 2023-11-28T03:08:24.470 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:24.469+0000 7fd4d70d4680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:08:24.471 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:24.470+0000 7f6a001f6680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:08:24.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.584+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7eff200a7f00 con 0x7eff200a8280 2023-11-28T03:08:24.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.587+0000 7eff06ffd700 1 -- 172.21.15.73:0/3784197727 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v19)=0 v19) v1 ==== 138+0+0 (secure 0 0 0) 0x7eff1404b060 con 0x7eff200a8280 2023-11-28T03:08:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 msgr2=0x7eff0c061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 0x7eff0c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 msgr2=0x7eff20141ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 rx=0x7eff1400d860 tx=0x7eff14008040).stop 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 shutdown_connections 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff0c05f1a0 0x7eff0c061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff20158160 0x7eff20148e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff200a8280 0x7eff20141ae0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 --2- 172.21.15.73:0/3784197727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff2014e600 0x7eff200c3eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 >> 172.21.15.73:0/3784197727 conn(0x7eff20062700 msgr2=0x7eff20159ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 shutdown_connections 2023-11-28T03:08:24.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.589+0000 7eff25444700 1 -- 172.21.15.73:0/3784197727 wait complete. 2023-11-28T03:08:24.602 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /bad-path 1M 2023-11-28T03:08:24.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.722+0000 7f5b98cf1700 1 Processor -- start 2023-11-28T03:08:24.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.722+0000 7f5b98cf1700 1 -- start start 2023-11-28T03:08:24.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b941373a0 0x7f5b940c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b940c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 0x7f5b94066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b94158e20 con 0x7f5b941373a0 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b940c68a0 con 0x7f5b940669a0 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b98cf1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b941582a0 con 0x7f5b940c5f30 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b9259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b941373a0 0x7f5b940c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b9259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b941373a0 0x7f5b940c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45440/0 (socket says 172.21.15.73:45440) 2023-11-28T03:08:24.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b9259c700 1 -- 172.21.15.73:0/3998225454 learned_addr learned my addr 172.21.15.73:0/3998225454 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b92d9d700 1 --2- 172.21.15.73:0/3998225454 >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 0x7f5b94066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.723+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b940c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 -- 172.21.15.73:0/3998225454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 msgr2=0x7f5b940c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b940c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 -- 172.21.15.73:0/3998225454 >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 msgr2=0x7f5b94066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 --2- 172.21.15.73:0/3998225454 >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 0x7f5b94066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 -- 172.21.15.73:0/3998225454 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b94158580 con 0x7f5b941373a0 2023-11-28T03:08:24.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b92d9d700 1 --2- 172.21.15.73:0/3998225454 >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 0x7f5b94066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9259c700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b941373a0 0x7f5b940c72e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f5b94068500 tx=0x7f5b880090f0).ready entity=mon.0 client_cookie=86774e5deb9916a5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9159a700 1 -- 172.21.15.73:0/3998225454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800b600 con 0x7f5b941373a0 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9159a700 1 -- 172.21.15.73:0/3998225454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800b760 con 0x7f5b941373a0 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b9159a700 1 -- 172.21.15.73:0/3998225454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800bae0 con 0x7f5b941373a0 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b98cf1700 1 -- 172.21.15.73:0/3998225454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b941373a0 msgr2=0x7f5b940c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.724+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b941373a0 0x7f5b940c72e0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 rx=0x7f5b94068500 tx=0x7f5b880090f0).stop 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 -- 172.21.15.73:0/3998225454 shutdown_connections 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b941373a0 0x7f5b940c72e0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/3998225454 >> v2:172.21.15.150:3300/0 conn(0x7f5b940669a0 0x7f5b94066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/3998225454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b940c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 -- 172.21.15.73:0/3998225454 >> 172.21.15.73:0/3998225454 conn(0x7f5b940628f0 msgr2=0x7f5b94062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 -- 172.21.15.73:0/3998225454 shutdown_connections 2023-11-28T03:08:24.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.725+0000 7f5b98cf1700 1 -- 172.21.15.73:0/3998225454 wait complete. 2023-11-28T03:08:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.726+0000 7f5b98cf1700 1 Processor -- start 2023-11-28T03:08:24.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.726+0000 7f5b98cf1700 1 -- start start 2023-11-28T03:08:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.726+0000 7f5b98cf1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.726+0000 7f5b98cf1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b941506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b941373a0 0x7f5b940adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94137b50 con 0x7f5b940669a0 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94165850 con 0x7f5b941373a0 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b94164690 con 0x7f5b940c5f30 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45446/0 (socket says 172.21.15.73:45446) 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 -- 172.21.15.73:0/172430816 learned_addr learned my addr 172.21.15.73:0/172430816 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b941506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b92d9d700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b941373a0 0x7f5b940adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:24.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 msgr2=0x7f5b941506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b941506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b941373a0 msgr2=0x7f5b940adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b941373a0 0x7f5b940adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b88009370 con 0x7f5b940669a0 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b9259c700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f5b88006fd0 tx=0x7f5b88017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8800bea0 con 0x7f5b940669a0 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b8800b8d0 con 0x7f5b940669a0 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b94164970 con 0x7f5b940669a0 2023-11-28T03:08:24.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b8801b690 con 0x7f5b940669a0 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.727+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b94164f30 con 0x7f5b940669a0 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.728+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b94165120 con 0x7f5b940669a0 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.729+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 39) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f5b8801ba00 con 0x7f5b940669a0 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.729+0000 7f5b827fc700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.729+0000 7f5b91d9b700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.729+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5b8807b960 con 0x7f5b940669a0 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.730+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:08:24.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.734+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5b8804cc30 con 0x7f5b940669a0 2023-11-28T03:08:24.747 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:24.746+0000 7fd4d70d4680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:08:24.757 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:24.757+0000 7f6a001f6680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:08:24.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.930+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b9406c860 con 0x7f5b7405f1a0 2023-11-28T03:08:24.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.931+0000 7f5b91d9b700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:24.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:24.931+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:08:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:25.332+0000 7f5b91d9b700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:25.332+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:08:25.368 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:25.367+0000 7fd4d70d4680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:08:25.375 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:25.374+0000 7f6a001f6680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:08:25.440 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:25.439+0000 7fd4d70d4680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:08:25.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:25.446+0000 7f6a001f6680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:08:25.630 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:25.629+0000 7fd4d70d4680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:08:25.633 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:25.632+0000 7f6a001f6680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:08:26.054 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.053+0000 7fd4d70d4680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:08:26.055 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.054+0000 7f6a001f6680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:08:26.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:26.133+0000 7f5b91d9b700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:26.134 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:26.133+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:08:26.157 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.156+0000 7fd4d70d4680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:26.158 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.157+0000 7f6a001f6680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:26.310 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.309+0000 7f6a001f6680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:26.312 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.312+0000 7fd4d70d4680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:26.384 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.383+0000 7f6a001f6680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:08:26.387 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.386+0000 7fd4d70d4680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:08:26.451 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.450+0000 7f6a001f6680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:08:26.455 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.454+0000 7fd4d70d4680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:08:26.708 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.708+0000 7f6a001f6680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:08:26.717 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.716+0000 7fd4d70d4680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:08:26.893 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:26.892+0000 7f6a001f6680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:26.904 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:26.903+0000 7fd4d70d4680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:08:27.541 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:27.540+0000 7f6a001f6680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:08:27.561 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:27.559+0000 7fd4d70d4680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:08:27.609 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:27.608+0000 7f6a001f6680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:08:27.628 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:27.627+0000 7fd4d70d4680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:08:27.685 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:27.684+0000 7f6a001f6680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:08:27.704 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:27.703+0000 7fd4d70d4680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:08:27.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:27.734+0000 7f5b91d9b700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:08:27.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:27.734+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:08:27.832 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:27.831+0000 7f6a001f6680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:08:27.852 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:27.851+0000 7fd4d70d4680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:08:27.902 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:27.901+0000 7f6a001f6680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:08:27.923 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:27.922+0000 7fd4d70d4680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:08:28.018 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.017+0000 7f6a001f6680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:08:28.039 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:28.038+0000 7fd4d70d4680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:08:28.125 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.124+0000 7f6a001f6680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:28.148 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:28.147+0000 7fd4d70d4680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:08:28.413 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.412+0000 7f6a001f6680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:08:28.440 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:28.439+0000 7fd4d70d4680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:08:28.484 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.483+0000 7f6a001f6680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:28.503+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 40) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f5b88079730 con 0x7f5b940669a0 2023-11-28T03:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:28.504+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b74061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:28.504+0000 7f5b827fc700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b74061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:08:28.515 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:08:28.514+0000 7fd4d70d4680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f6977982700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f6977982700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f6977982700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f6977982700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f6977982700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f697417b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f697417b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f697417b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f697417b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:28.569 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:08:28.567+0000 7f697417b700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:08:29.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.505+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 41) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5b88079ab0 con 0x7f5b940669a0 2023-11-28T03:08:29.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.505+0000 7f5b827fc700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b740615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.505+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b9406c860 con 0x7f5b7405f1a0 2023-11-28T03:08:29.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.506+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b740615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.507+0000 7f5b91d9b700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b740615e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5b84008010 tx=0x7f5b84008040).ready entity=mgr.7323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:29.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.523+0000 7f5b827fc700 1 -- 172.21.15.73:0/172430816 <== mgr.7323 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f5b9406c860 con 0x7f5b7405f1a0 2023-11-28T03:08:29.524 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /bad-path 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 msgr2=0x7f5b740615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b740615e0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5b84008010 tx=0x7f5b84008040).stop 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 msgr2=0x7f5b94165360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 rx=0x7f5b88006fd0 tx=0x7f5b88017000).stop 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 shutdown_connections 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5b7405f1a0 0x7f5b740615e0 secure :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0x7f5b84008010 tx=0x7f5b84008040).stop 2023-11-28T03:08:29.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b941373a0 0x7f5b940adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b940669a0 0x7f5b94165360 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 --2- 172.21.15.73:0/172430816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b940c5f30 0x7f5b941506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 >> 172.21.15.73:0/172430816 conn(0x7f5b940628f0 msgr2=0x7f5b94064270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:29.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 shutdown_connections 2023-11-28T03:08:29.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.525+0000 7f5b98cf1700 1 -- 172.21.15.73:0/172430816 wait complete. 2023-11-28T03:08:29.541 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-11-28T03:08:29.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-11-28T03:08:29.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.660+0000 7fd9eee43700 1 Processor -- start 2023-11-28T03:08:29.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.660+0000 7fd9eee43700 1 -- start start 2023-11-28T03:08:29.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9eee43700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9eee43700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9e80aa1c0 0x7fd9e80aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9eee43700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9e8159210 con 0x7fd9e80ab3b0 2023-11-28T03:08:29.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9eee43700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9e80c0c70 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9eee43700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c2110 con 0x7fd9e8154610 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9ed3e0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9e80aa1c0 0x7fd9e80aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9dffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd9e80ab3b0 0x7fd9e80c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38742/0 (socket says 172.21.15.73:38742) 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.661+0000 7fd9dffff700 1 -- 172.21.15.73:0/3651560859 learned_addr learned my addr 172.21.15.73:0/3651560859 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ecbdf700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/3651560859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 msgr2=0x7fd9e81582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ed3e0700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/3651560859 >> v1:172.21.15.73:6789/0 conn(0x7fd9e80ab3b0 legacy=0x7fd9e80c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/3651560859 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e80aaae0 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9df7fe700 1 -- 172.21.15.73:0/3651560859 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1613193334 0 0) 0x7fd9e8159210 con 0x7fd9e80ab3b0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9ed3e0700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80aa5f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd9e0009760 tx=0x7fd9e0009060).ready entity=mon.0 client_cookie=d3d730f4da44ef9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9df7fe700 1 -- 172.21.15.73:0/3651560859 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000b580 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9df7fe700 1 -- 172.21.15.73:0/3651560859 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e000b6e0 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9df7fe700 1 -- 172.21.15.73:0/3651560859 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000ba60 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 -- 172.21.15.73:0/3651560859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80aa1c0 msgr2=0x7fd9e80aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80aa5f0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 rx=0x7fd9e0009760 tx=0x7fd9e0009060).stop 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 -- 172.21.15.73:0/3651560859 shutdown_connections 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e81582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 --2- 172.21.15.73:0/3651560859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80aa5f0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 -- 172.21.15.73:0/3651560859 >> 172.21.15.73:0/3651560859 conn(0x7fd9e80655a0 msgr2=0x7fd9e8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 -- 172.21.15.73:0/3651560859 shutdown_connections 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.662+0000 7fd9eee43700 1 -- 172.21.15.73:0/3651560859 wait complete. 2023-11-28T03:08:29.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 Processor -- start 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 -- start start 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80b0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e8168e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c3a30 con 0x7fd9e80c4760 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80c42d0 con 0x7fd9e80aa1c0 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.664+0000 7fd9eee43700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9e80b1020 con 0x7fd9e8154610 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45568/0 (socket says 172.21.15.73:45568) 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/2866084766 learned_addr learned my addr 172.21.15.73:0/2866084766 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 msgr2=0x7fd9e8168e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9dffff700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e8168e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ecbdf700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80b0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e8168e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e80aa1c0 msgr2=0x7fd9e80b0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80b0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 -- 172.21.15.73:0/2866084766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e00092a0 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9ed3e0700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fd9e0009eb0 tx=0x7fd9e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e000ba60 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e001b440 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9e80c3c80 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e001b7c0 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.665+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9e8169330 con 0x7fd9e80c4760 2023-11-28T03:08:29.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.666+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9e80c4460 con 0x7fd9e80c4760 2023-11-28T03:08:29.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.666+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 41) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fd9e001bb70 con 0x7fd9e80c4760 2023-11-28T03:08:29.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.667+0000 7fd9dcff9700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 0x7fd9d4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:08:29.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.667+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd9e007bb20 con 0x7fd9e80c4760 2023-11-28T03:08:29.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.667+0000 7fd9ecbdf700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 0x7fd9d4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:08:29.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.667+0000 7fd9ecbdf700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 0x7fd9d4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd9e400d390 tx=0x7fd9e4007510).ready entity=mgr.7323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:08:29.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.671+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd9e004ceb0 con 0x7fd9e80c4760 2023-11-28T03:08:29.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.857+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd9e8158970 con 0x7fd9d405f080 2023-11-28T03:08:29.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.859+0000 7fd9dcff9700 1 -- 172.21.15.73:0/2866084766 <== mgr.7323 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7fd9e8158970 con 0x7fd9d405f080 2023-11-28T03:08:29.860 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:08:29.860 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:08:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-11-28T03:08:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 msgr2=0x7fd9d4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 0x7fd9d4061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd9e400d390 tx=0x7fd9e4007510).stop 2023-11-28T03:08:29.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 msgr2=0x7fd9e80c2990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 rx=0x7fd9e0009eb0 tx=0x7fd9e0017000).stop 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 shutdown_connections 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd9d405f080 0x7fd9d4061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9e80c4760 0x7fd9e80c2990 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9e80aa1c0 0x7fd9e80b0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 --2- 172.21.15.73:0/2866084766 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9e8154610 0x7fd9e8168e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 >> 172.21.15.73:0/2866084766 conn(0x7fd9e80655a0 msgr2=0x7fd9e80c9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 shutdown_connections 2023-11-28T03:08:29.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:08:29.861+0000 7fd9eee43700 1 -- 172.21.15.73:0/2866084766 wait complete. 2023-11-28T03:08:29.874 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:08:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-11-28T03:08:52.987 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:08:52.989 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:09:00.528 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:00.527+0000 7f6967161700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:00.529 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:00.527+0000 7f6967161700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:00.529 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:00.527+0000 7f6967161700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:00.529 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:00.527+0000 7f6967161700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:00.529 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:00.527+0000 7f6967161700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:23.060 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:09:23.063 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:09:29.875 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /bad-path 2023-11-28T03:09:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.002+0000 7f16d1d75700 1 Processor -- start 2023-11-28T03:09:30.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.002+0000 7f16d1d75700 1 -- start start 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 0x7f16cc154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16cc0a7ff0 0x7f16cc0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0c6390 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16cc0c6630 con 0x7f16cc154530 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16d1d75700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0aaba0 con 0x7f16cc0aa220 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16cc0a7ff0 0x7f16cc0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f16cc0a7ff0 0x7f16cc0c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53572/0 (socket says 172.21.15.73:53572) 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 -- 172.21.15.73:0/1257049486 learned_addr learned my addr 172.21.15.73:0/1257049486 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16cbfff700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 -- 172.21.15.73:0/1257049486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 msgr2=0x7f16cc0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16cb7fe700 1 --2- 172.21.15.73:0/1257049486 >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 0x7f16cc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 -- 172.21.15.73:0/1257049486 >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 msgr2=0x7f16cc154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 --2- 172.21.15.73:0/1257049486 >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 0x7f16cc154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 -- 172.21.15.73:0/1257049486 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16cc0a8470 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16cbfff700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16cb7fe700 1 --2- 172.21.15.73:0/1257049486 >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 0x7f16cc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16caffd700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc0c8580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f16cc0c9080 tx=0x7f16c00090f0).ready entity=mon.0 client_cookie=f3b7d4ae04fc506d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16ca7fc700 1 -- 172.21.15.73:0/1257049486 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c000b600 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.003+0000 7f16ca7fc700 1 -- 172.21.15.73:0/1257049486 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c000b760 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16ca7fc700 1 -- 172.21.15.73:0/1257049486 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c000bae0 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 -- 172.21.15.73:0/1257049486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc0a7ff0 msgr2=0x7f16cc0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc0c8580 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 rx=0x7f16cc0c9080 tx=0x7f16c00090f0).stop 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 -- 172.21.15.73:0/1257049486 shutdown_connections 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1257049486 >> v2:172.21.15.150:3300/0 conn(0x7f16cc154530 0x7f16cc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1257049486 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc0c8580 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 -- 172.21.15.73:0/1257049486 >> 172.21.15.73:0/1257049486 conn(0x7f16cc094930 msgr2=0x7f16cc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:30.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 -- 172.21.15.73:0/1257049486 shutdown_connections 2023-11-28T03:09:30.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.004+0000 7f16d1d75700 1 -- 172.21.15.73:0/1257049486 wait complete. 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.005+0000 7f16d1d75700 1 Processor -- start 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.005+0000 7f16d1d75700 1 -- start start 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc1510c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0ab730 con 0x7f16cc154530 2023-11-28T03:09:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0c5130 con 0x7f16cc0a7ff0 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16d1d75700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16cc0ac9b0 con 0x7f16cc0aa220 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53584/0 (socket says 172.21.15.73:53584) 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 -- 172.21.15.73:0/1181225695 learned_addr learned my addr 172.21.15.73:0/1181225695 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16caffd700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cb7fe700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc1510c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 -- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 msgr2=0x7f16cc0c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 -- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc0a7ff0 msgr2=0x7f16cc1510c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc1510c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.006+0000 7f16cbfff700 1 -- 172.21.15.73:0/1181225695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c0009370 con 0x7f16cc154530 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16cbfff700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f16c400c880 tx=0x7f16c4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4010580 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16c40106e0 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16cc0ab910 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16c4010a60 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.007+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16cc0abed0 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.008+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16cc0cc110 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.008+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f16c401b460 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.008+0000 7f16b37fe700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 0x7f16b80618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.009+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f16c407a860 con 0x7f16cc154530 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.009+0000 7f16cb7fe700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 0x7f16b80618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.009+0000 7f16cb7fe700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 0x7f16b80618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f16cc0caab0 tx=0x7f16bc0076c0).ready entity=mgr.7323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.013+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f16c404b580 con 0x7f16cc154530 2023-11-28T03:09:30.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.200+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f16cc06c860 con 0x7f16b805f3e0 2023-11-28T03:09:30.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.201+0000 7f16b37fe700 1 -- 172.21.15.73:0/1181225695 <== mgr.7323 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f16cc06c860 con 0x7f16b805f3e0 2023-11-28T03:09:30.203 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:30.203 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:08:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-11-28T03:09:30.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 msgr2=0x7f16b80618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 0x7f16b80618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f16cc0caab0 tx=0x7f16bc0076c0).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 msgr2=0x7f16cc0c4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 rx=0x7f16c400c880 tx=0x7f16c4008040).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 shutdown_connections 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f16b805f3e0 0x7f16b80618a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f16cc154530 0x7f16cc0c4c40 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f16cc0a7ff0 0x7f16cc1510c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 --2- 172.21.15.73:0/1181225695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f16cc0aa220 0x7f16cc0c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 >> 172.21.15.73:0/1181225695 conn(0x7f16cc094930 msgr2=0x7f16cc0a8650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 shutdown_connections 2023-11-28T03:09:30.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.203+0000 7f16d1d75700 1 -- 172.21.15.73:0/1181225695 wait complete. 2023-11-28T03:09:30.216 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:08:29", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-11-28T03:09:30.216 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /bad-path 2023-11-28T03:09:30.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 Processor -- start 2023-11-28T03:09:30.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 -- start start 2023-11-28T03:09:30.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40a41373a0 0x7f40a40c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f40a40c5f30 0x7f40a40c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a4066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a40c68d0 con 0x7f40a41373a0 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a4158e20 con 0x7f40a40c5f30 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.336+0000 7f40a9ee8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a41582a0 con 0x7f40a40669a0 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40a41373a0 0x7f40a40c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40a41373a0 0x7f40a40c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53586/0 (socket says 172.21.15.73:53586) 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 -- 172.21.15.73:0/3413260497 learned_addr learned my addr 172.21.15.73:0/3413260497 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a3fff700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a4066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a2ffd700 1 --2- 172.21.15.73:0/3413260497 >> v2:172.21.15.150:3300/0 conn(0x7f40a40c5f30 0x7f40a40c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 -- 172.21.15.73:0/3413260497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 msgr2=0x7f40a4066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a4066e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 -- 172.21.15.73:0/3413260497 >> v2:172.21.15.150:3300/0 conn(0x7f40a40c5f30 msgr2=0x7f40a40c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 --2- 172.21.15.73:0/3413260497 >> v2:172.21.15.150:3300/0 conn(0x7f40a40c5f30 0x7f40a40c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 -- 172.21.15.73:0/3413260497 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a4158580 con 0x7f40a41373a0 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a37fe700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40c72e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f40a4069110 tx=0x7f4094009840).ready entity=mon.0 client_cookie=b656efb2b1ccaaf8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a27fc700 1 -- 172.21.15.73:0/3413260497 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400e070 con 0x7f40a41373a0 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a27fc700 1 -- 172.21.15.73:0/3413260497 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409400ba50 con 0x7f40a41373a0 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.337+0000 7f40a27fc700 1 -- 172.21.15.73:0/3413260497 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409400bdd0 con 0x7f40a41373a0 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 -- 172.21.15.73:0/3413260497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 msgr2=0x7f40a40c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40c72e0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 rx=0x7f40a4069110 tx=0x7f4094009840).stop 2023-11-28T03:09:30.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 -- 172.21.15.73:0/3413260497 shutdown_connections 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40c72e0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/3413260497 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a4066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/3413260497 >> v2:172.21.15.150:3300/0 conn(0x7f40a40c5f30 0x7f40a40c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 -- 172.21.15.73:0/3413260497 >> 172.21.15.73:0/3413260497 conn(0x7f40a40628f0 msgr2=0x7f40a4062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 -- 172.21.15.73:0/3413260497 shutdown_connections 2023-11-28T03:09:30.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.338+0000 7f40a9ee8700 1 -- 172.21.15.73:0/3413260497 wait complete. 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.339+0000 7f40a9ee8700 1 Processor -- start 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.339+0000 7f40a9ee8700 1 -- start start 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a40aa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a40c5f30 0x7f40a40ac040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40acd70 con 0x7f40a41373a0 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40ad050 con 0x7f40a40c5f30 2023-11-28T03:09:30.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a40ad330 con 0x7f40a40669a0 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53602/0 (socket says 172.21.15.73:53602) 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 -- 172.21.15.73:0/2704982653 learned_addr learned my addr 172.21.15.73:0/2704982653 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a37fe700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a40aa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a2ffd700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a40c5f30 0x7f40a40ac040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 -- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 msgr2=0x7f40a40aa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a40aa2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 -- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a40c5f30 msgr2=0x7f40a40ac040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a40c5f30 0x7f40a40ac040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 -- 172.21.15.73:0/2704982653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4094009ae0 con 0x7f40a41373a0 2023-11-28T03:09:30.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a3fff700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f409c00c650 tx=0x7f409c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c010580 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f409c0106e0 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.340+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a40ad610 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.341+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409c010a60 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.341+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a40aca20 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a406c860 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f409c01b460 con 0x7f40a41373a0 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f408b7fe700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 0x7f40900618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f409c07a660 con 0x7f40a41373a0 2023-11-28T03:09:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f40a37fe700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 0x7f40900618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.342+0000 7f40a37fe700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 0x7f40900618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f40a4069110 tx=0x7f4094009840).ready entity=mgr.7323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.346+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f409c04b380 con 0x7f40a41373a0 2023-11-28T03:09:30.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.536+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f40a4158860 con 0x7f409005f3e0 2023-11-28T03:09:30.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.539+0000 7f408b7fe700 1 -- 172.21.15.73:0/2704982653 <== mgr.7323 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f40a4158860 con 0x7f409005f3e0 2023-11-28T03:09:30.540 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /bad-path 2023-11-28T03:09:30.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 msgr2=0x7f40900618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 0x7f40900618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f40a4069110 tx=0x7f4094009840).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 msgr2=0x7f40a40ac530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 rx=0x7f409c00c650 tx=0x7f409c008040).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 shutdown_connections 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f409005f3e0 0x7f40900618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40a41373a0 0x7f40a40ac530 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40a40669a0 0x7f40a40aa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 --2- 172.21.15.73:0/2704982653 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40a40c5f30 0x7f40a40ac040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 >> 172.21.15.73:0/2704982653 conn(0x7f40a40628f0 msgr2=0x7f40a4153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 shutdown_connections 2023-11-28T03:09:30.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.541+0000 7f40a9ee8700 1 -- 172.21.15.73:0/2704982653 wait complete. 2023-11-28T03:09:30.554 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-11-28T03:09:30.554 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:09:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.668+0000 7fdd8202c700 1 Processor -- start 2023-11-28T03:09:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.668+0000 7fdd8202c700 1 -- start start 2023-11-28T03:09:30.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd8202c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c1560e0 0x7fdd7c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd8202c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd7c0638b0 0x7fdd7c063d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd8202c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd7c062810 con 0x7fdd7c0650a0 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd8202c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd7c0670e0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd8202c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c062af0 con 0x7fdd7c1560e0 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd7c0638b0 0x7fdd7c063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdd7c0638b0 0x7fdd7c063d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53616/0 (socket says 172.21.15.73:53616) 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 -- 172.21.15.73:0/1750963972 learned_addr learned my addr 172.21.15.73:0/1750963972 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7affd700 1 --1- 172.21.15.73:0/1750963972 >> v1:172.21.15.73:6789/0 conn(0x7fdd7c0650a0 0x7fdd7c1564c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33402/0 (socket says 172.21.15.73:33402) 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7b7fe700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c1560e0 0x7fdd7c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 -- 172.21.15.73:0/1750963972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c1560e0 msgr2=0x7fdd7c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c1560e0 0x7fdd7c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 -- 172.21.15.73:0/1750963972 >> v1:172.21.15.73:6789/0 conn(0x7fdd7c0650a0 legacy=0x7fdd7c1564c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7bfff700 1 -- 172.21.15.73:0/1750963972 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd7c062dd0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.669+0000 7fdd7a7fc700 1 -- 172.21.15.73:0/1750963972 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 587812924 0 0) 0x7fdd7c062810 con 0x7fdd7c0650a0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd7bfff700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c063d40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fdd74009f20 tx=0x7fdd74009840).ready entity=mon.0 client_cookie=7b788e11e18e0e19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd7a7fc700 1 -- 172.21.15.73:0/1750963972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400e070 con 0x7fdd7c0638b0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd7a7fc700 1 -- 172.21.15.73:0/1750963972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd7400b9d0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd7a7fc700 1 -- 172.21.15.73:0/1750963972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd7400bd50 con 0x7fdd7c0638b0 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 -- 172.21.15.73:0/1750963972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 msgr2=0x7fdd7c063d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c063d40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 rx=0x7fdd74009f20 tx=0x7fdd74009840).stop 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 -- 172.21.15.73:0/1750963972 shutdown_connections 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c1560e0 0x7fdd7c156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 --2- 172.21.15.73:0/1750963972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c063d40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 -- 172.21.15.73:0/1750963972 >> 172.21.15.73:0/1750963972 conn(0x7fdd7c0c4200 msgr2=0x7fdd7c0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:30.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 -- 172.21.15.73:0/1750963972 shutdown_connections 2023-11-28T03:09:30.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.670+0000 7fdd8202c700 1 -- 172.21.15.73:0/1750963972 wait complete. 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.672+0000 7fdd8202c700 1 Processor -- start 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.672+0000 7fdd8202c700 1 -- start start 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c0650a0 0x7fdd7c168d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 0x7fdd7c1442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c151680 con 0x7fdd7c0638b0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1550e0 con 0x7fdd7c1560e0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd7c1547f0 con 0x7fdd7c0650a0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53624/0 (socket says 172.21.15.73:53624) 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 -- 172.21.15.73:0/2036113381 learned_addr learned my addr 172.21.15.73:0/2036113381 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7affd700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c0650a0 0x7fdd7c168d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7bfff700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 0x7fdd7c1442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 -- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c0650a0 msgr2=0x7fdd7c168d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c0650a0 0x7fdd7c168d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 -- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 msgr2=0x7fdd7c1442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 0x7fdd7c1442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:30.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 -- 172.21.15.73:0/2036113381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd74009ae0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7bfff700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 0x7fdd7c1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd7b7fe700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fdd6c00e8d0 tx=0x7fdd6c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c012600 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd6c012760 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd7c1513a0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd6c012ae0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.673+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd7c154d70 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.674+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd7c1553c0 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.675+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 43) v1 ==== 75679+0+0 (secure 0 0 0) 0x7fdd6c01b400 con 0x7fdd7c0638b0 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.675+0000 7fdd637fe700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 0x7fdd680618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:30.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.675+0000 7fdd7affd700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 0x7fdd680618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:30.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.675+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fdd6c07ab20 con 0x7fdd7c0638b0 2023-11-28T03:09:30.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.676+0000 7fdd7affd700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 0x7fdd680618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdd7c0c3c30 tx=0x7fdd70007d70).ready entity=mgr.7323 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:30.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.679+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdd6c04b950 con 0x7fdd7c0638b0 2023-11-28T03:09:30.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:30.886+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fdd7c06c860 con 0x7fdd7c0638b0 2023-11-28T03:09:31.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.321+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fdd6c0784b0 con 0x7fdd7c0638b0 2023-11-28T03:09:31.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.322+0000 7fdd637fe700 1 -- 172.21.15.73:0/2036113381 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v44) v1 ==== 93+0+0 (secure 0 0 0) 0x7fdd6c072810 con 0x7fdd7c0638b0 2023-11-28T03:09:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 msgr2=0x7fdd680618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 0x7fdd680618a0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fdd7c0c3c30 tx=0x7fdd70007d70).stop 2023-11-28T03:09:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 msgr2=0x7fdd7c150e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 rx=0x7fdd6c00e8d0 tx=0x7fdd6c006f40).stop 2023-11-28T03:09:31.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 shutdown_connections 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdd6805f3e0 0x7fdd680618a0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdd7c1560e0 0x7fdd7c1442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdd7c0638b0 0x7fdd7c150e00 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 --2- 172.21.15.73:0/2036113381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdd7c0650a0 0x7fdd7c168d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 >> 172.21.15.73:0/2036113381 conn(0x7fdd7c0c4200 msgr2=0x7fdd7c146570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 shutdown_connections 2023-11-28T03:09:31.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.324+0000 7fdd8202c700 1 -- 172.21.15.73:0/2036113381 wait complete. 2023-11-28T03:09:31.338 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:09:31.339 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:09:31.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:09:31.433 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:09:31.433 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:09:31.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:09:31.434 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:09:31.466 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:31.464+0000 7fadbf74e680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:09:31.468 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:31.468+0000 7f20bbdbe680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:09:31.476 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:09:31.476 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:31.477 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:09:31.531 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:31.530+0000 7fadbf74e680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:09:31.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:31.532 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:31.532+0000 7f20bbdbe680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:09:31.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.688+0000 7fd90d90e700 1 Processor -- start 2023-11-28T03:09:31.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.688+0000 7fd90d90e700 1 -- start start 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd90d90e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd90815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd90d90e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9080a9a90 con 0x7fd9080caab0 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd90d90e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd908154ce0 con 0x7fd9080cae90 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd90d90e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080aab80 con 0x7fd9080ca180 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd906ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd9080caab0 0x7fd9080aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33432/0 (socket says 172.21.15.73:33432) 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd906ffd700 1 -- 172.21.15.73:0/998637697 learned_addr learned my addr 172.21.15.73:0/998637697 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.689+0000 7fd9077fe700 1 --2- 172.21.15.73:0/998637697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd90815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1975922655 0 0) 0x7fd9080a9a90 con 0x7fd9080caab0 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8f00033f0 con 0x7fd9080caab0 2023-11-28T03:09:31.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477335958 0 0) 0x7fd908154ce0 con 0x7fd9080cae90 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9080a9a90 con 0x7fd9080cae90 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd9077fe700 1 -- 172.21.15.73:0/998637697 >> v1:172.21.15.150:6789/0 conn(0x7fd9080cae90 legacy=0x7fd9080c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3707970269 0 0) 0x7fd8f00033f0 con 0x7fd9080caab0 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd9077fe700 1 -- 172.21.15.73:0/998637697 >> v1:172.21.15.73:6789/0 conn(0x7fd9080caab0 legacy=0x7fd9080aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd9077fe700 1 -- 172.21.15.73:0/998637697 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9080a8e80 con 0x7fd9080ca180 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd9077fe700 1 --2- 172.21.15.73:0/998637697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd90815f3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd900009e90 tx=0x7fd900009780).ready entity=mon.2 client_cookie=ae54b3df3c89db78 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90000e070 con 0x7fd9080ca180 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90000b7a0 con 0x7fd9080ca180 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.690+0000 7fd905ffb700 1 -- 172.21.15.73:0/998637697 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8fc0015f0 con 0x7fd9080ca180 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 -- 172.21.15.73:0/998637697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 msgr2=0x7fd90815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 --2- 172.21.15.73:0/998637697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd90815f3d0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 rx=0x7fd900009e90 tx=0x7fd900009780).stop 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 -- 172.21.15.73:0/998637697 shutdown_connections 2023-11-28T03:09:31.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 --2- 172.21.15.73:0/998637697 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd90815f3d0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 -- 172.21.15.73:0/998637697 >> 172.21.15.73:0/998637697 conn(0x7fd908094920 msgr2=0x7fd90806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:31.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 -- 172.21.15.73:0/998637697 shutdown_connections 2023-11-28T03:09:31.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.691+0000 7fd90d90e700 1 -- 172.21.15.73:0/998637697 wait complete. 2023-11-28T03:09:31.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.692+0000 7fd90d90e700 1 Processor -- start 2023-11-28T03:09:31.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.692+0000 7fd90d90e700 1 -- start start 2023-11-28T03:09:31.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd9080afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:31.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9080caab0 0x7fd9080c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:31.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:31.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080b00c0 con 0x7fd9080cae90 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080b0ec0 con 0x7fd9080caab0 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9080c4ee0 con 0x7fd9080ca180 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53642/0 (socket says 172.21.15.73:53642) 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 -- 172.21.15.73:0/2056130137 learned_addr learned my addr 172.21.15.73:0/2056130137 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd906ffd700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd9080afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9067fc700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9080caab0 0x7fd9080c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 msgr2=0x7fd9080afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd9080afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9080caab0 msgr2=0x7fd9080c49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9080caab0 0x7fd9080c49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 -- 172.21.15.73:0/2056130137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd900009a20 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd9077fe700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fd908083c30 tx=0x7fd900009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90001d440 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd90001d5a0 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9080b02b0 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.693+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd90001d920 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.694+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9080b0870 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.694+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9080ca8f0 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.695+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fd90001dd10 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.695+0000 7fd8eeffd700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 0x7fd8f4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.695+0000 7fd906ffd700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 msgr2=0x7fd8f4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.695+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fd90007c990 con 0x7fd9080cae90 2023-11-28T03:09:31.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.695+0000 7fd906ffd700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 0x7fd8f4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.700+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd900051200 con 0x7fd9080cae90 2023-11-28T03:09:31.806 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:31.805+0000 7f20bbdbe680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:09:31.818 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:31.817+0000 7fadbf74e680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:09:31.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.896+0000 7fd906ffd700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 msgr2=0x7fd8f4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:31.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.896+0000 7fd906ffd700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 0x7fd8f4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:31.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.913+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd90806c860 con 0x7fd9080cae90 2023-11-28T03:09:31.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.913+0000 7fd8eeffd700 1 -- 172.21.15.73:0/2056130137 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fd900050df0 con 0x7fd9080cae90 2023-11-28T03:09:31.914 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:31.915 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 msgr2=0x7fd8f4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 0x7fd8f4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 msgr2=0x7fd908149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 rx=0x7fd908083c30 tx=0x7fd900009780).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 shutdown_connections 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd8f405f390 0x7fd8f4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9080cae90 0x7fd908149050 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9080ca180 0x7fd9080afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 --2- 172.21.15.73:0/2056130137 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9080caab0 0x7fd9080c49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 >> 172.21.15.73:0/2056130137 conn(0x7fd908094920 msgr2=0x7fd9080ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 shutdown_connections 2023-11-28T03:09:31.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:31.915+0000 7fd90d90e700 1 -- 172.21.15.73:0/2056130137 wait complete. 2023-11-28T03:09:31.918 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:09:31.929 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:32.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.043+0000 7f0e7d80f700 1 Processor -- start 2023-11-28T03:09:32.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.043+0000 7f0e7d80f700 1 -- start start 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e7d80f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 0x7f0e780c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e7d80f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 0x7f0e780c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e7d80f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780a9a90 con 0x7f0e780caab0 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e7d80f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e780c1490 con 0x7f0e780cae90 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e7d80f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78154ce0 con 0x7f0e780ca180 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e76ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e780caab0 0x7f0e780aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33448/0 (socket says 172.21.15.73:33448) 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e76ffd700 1 -- 172.21.15.73:0/3078285143 learned_addr learned my addr 172.21.15.73:0/3078285143 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:32.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e767fc700 1 --2- 172.21.15.73:0/3078285143 >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 0x7f0e780c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.044+0000 7f0e777fe700 1 --2- 172.21.15.73:0/3078285143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 0x7f0e780c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e75ffb700 1 -- 172.21.15.73:0/3078285143 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175708087 0 0) 0x7f0e780a9a90 con 0x7f0e780caab0 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e75ffb700 1 -- 172.21.15.73:0/3078285143 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e600033f0 con 0x7f0e780caab0 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e777fe700 1 -- 172.21.15.73:0/3078285143 >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 msgr2=0x7f0e780c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e777fe700 1 --2- 172.21.15.73:0/3078285143 >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 0x7f0e780c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e777fe700 1 -- 172.21.15.73:0/3078285143 >> v1:172.21.15.73:6789/0 conn(0x7f0e780caab0 legacy=0x7f0e780aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e777fe700 1 -- 172.21.15.73:0/3078285143 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780aab80 con 0x7f0e780ca180 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e767fc700 1 --2- 172.21.15.73:0/3078285143 >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 0x7f0e780c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e777fe700 1 --2- 172.21.15.73:0/3078285143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 0x7f0e780c0fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0e70009e60 tx=0x7f0e70009780).ready entity=mon.2 client_cookie=26121952c43133b2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e75ffb700 1 -- 172.21.15.73:0/3078285143 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000e070 con 0x7f0e780ca180 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e75ffb700 1 -- 172.21.15.73:0/3078285143 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e7000bab0 con 0x7f0e780ca180 2023-11-28T03:09:32.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.045+0000 7f0e75ffb700 1 -- 172.21.15.73:0/3078285143 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e7000be30 con 0x7f0e780ca180 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 -- 172.21.15.73:0/3078285143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 msgr2=0x7f0e780c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/3078285143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 0x7f0e780c0fa0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 rx=0x7f0e70009e60 tx=0x7f0e70009780).stop 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 -- 172.21.15.73:0/3078285143 shutdown_connections 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/3078285143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780ca180 0x7f0e780c0fa0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/3078285143 >> v2:172.21.15.150:3300/0 conn(0x7f0e780cae90 0x7f0e780c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 -- 172.21.15.73:0/3078285143 >> 172.21.15.73:0/3078285143 conn(0x7f0e78094920 msgr2=0x7f0e7806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 -- 172.21.15.73:0/3078285143 shutdown_connections 2023-11-28T03:09:32.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.046+0000 7f0e7d80f700 1 -- 172.21.15.73:0/3078285143 wait complete. 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.047+0000 7f0e7d80f700 1 Processor -- start 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.047+0000 7f0e7d80f700 1 -- start start 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 0x7f0e78154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e780cae90 0x7f0e780c4460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e781558c0 con 0x7f0e780ca180 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78155ba0 con 0x7f0e780cae90 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78155e80 con 0x7f0e780caab0 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53644/0 (socket says 172.21.15.73:53644) 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 -- 172.21.15.73:0/2137737673 learned_addr learned my addr 172.21.15.73:0/2137737673 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e767fc700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 0x7f0e78154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e777fe700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e780cae90 0x7f0e780c4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 -- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 msgr2=0x7f0e78154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 0x7f0e78154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 -- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e780cae90 msgr2=0x7f0e780c4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e780cae90 0x7f0e780c4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 -- 172.21.15.73:0/2137737673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e680092a0 con 0x7f0e780ca180 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e767fc700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 0x7f0e78154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e76ffd700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f0e6c002ee0 tx=0x7f0e6c007ae0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c00ce10 con 0x7f0e780ca180 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0072e0 con 0x7f0e780ca180 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c005670 con 0x7f0e780ca180 2023-11-28T03:09:32.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e70009a20 con 0x7f0e780ca180 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.048+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e78164520 con 0x7f0e780ca180 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e78164a50 con 0x7f0e780ca180 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f0e6c005a60 con 0x7f0e780ca180 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e5effd700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 0x7f0e64061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e767fc700 1 -- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 msgr2=0x7f0e64061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f0e6c079cd0 con 0x7f0e780ca180 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.050+0000 7f0e767fc700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 0x7f0e64061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:32.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.054+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e78164a50 con 0x7f0e780ca180 2023-11-28T03:09:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.224+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e78164a50 con 0x7f0e780ca180 2023-11-28T03:09:32.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.225+0000 7f0e5effd700 1 -- 172.21.15.73:0/2137737673 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17428 (secure 0 0 0) 0x7f0e78164a50 con 0x7f0e780ca180 2023-11-28T03:09:32.227 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:32.227 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":113,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:30.299893+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3656330984":"2023-11-29T03:08:28.497043+0000","172.21.15.73:0/2798049488":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:28.497043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.227+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 msgr2=0x7f0e64061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.227+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 0x7f0e64061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 msgr2=0x7f0e78152270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 rx=0x7f0e6c002ee0 tx=0x7f0e6c007ae0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 shutdown_connections 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e6405f360 0x7f0e64061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e780cae90 0x7f0e780c4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e780ca180 0x7f0e78152270 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 --2- 172.21.15.73:0/2137737673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e780caab0 0x7f0e78154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 >> 172.21.15.73:0/2137737673 conn(0x7f0e78094920 msgr2=0x7f0e780ad590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 shutdown_connections 2023-11-28T03:09:32.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.228+0000 7f0e7d80f700 1 -- 172.21.15.73:0/2137737673 wait complete. 2023-11-28T03:09:32.242 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '113', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 13, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:08:16.209998+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '109', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 14, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:08:16.638888+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '110', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:09:32.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 Processor -- start 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 -- start start 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc06c06a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c0c3470 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc06c151ea0 con 0x7fc06c069d10 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.364+0000 7fc072ea3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c152180 con 0x7fc06c151a20 2023-11-28T03:09:32.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc071440700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc06c0bd1f0 0x7fc06c0c38e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33454/0 (socket says 172.21.15.73:33454) 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc071440700 1 -- 172.21.15.73:0/1830081278 learned_addr learned my addr 172.21.15.73:0/1830081278 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06bfff700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc06c06a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 384391017 0 0) 0x7fc06c0c3470 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc05c0033f0 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2482382294 0 0) 0x7fc06c151ea0 con 0x7fc06c069d10 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc06c0c3470 con 0x7fc06c069d10 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 679923791 0 0) 0x7fc05c0033f0 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 msgr2=0x7fc06c06a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:32.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc06c06a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 >> v1:172.21.15.150:6789/0 conn(0x7fc06c069d10 legacy=0x7fc06c0be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c152460 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc0640017f0 con 0x7fc06c0bd1f0 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 >> v1:172.21.15.73:6789/0 conn(0x7fc06c0bd1f0 legacy=0x7fc06c0c38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc05c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc05c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc05c004b80 0x7fc05c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c152180 con 0x7fc05c004000 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00b9c0 con 0x7fc05c004b80 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc05c00bb80 con 0x7fc06c151a20 2023-11-28T03:09:32.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc06bfff700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc05c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.365+0000 7fc071440700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc05c004b80 0x7fc05c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc070c3f700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc05c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 -- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 msgr2=0x7fc05c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c151a20 0x7fc05c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 -- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc05c004b80 msgr2=0x7fc05c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:32.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc05c004b80 0x7fc05c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 -- 172.21.15.73:0/1830081278 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc05c0033f0 con 0x7fc05c004000 2023-11-28T03:09:32.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06bfff700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc05c004690 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc058006d80 tx=0x7fc058009060).ready entity=mon.0 client_cookie=c68f897958da0423 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc058009430 con 0x7fc05c004000 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc058009e70 con 0x7fc05c004000 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.366+0000 7fc06b7fe700 1 -- 172.21.15.73:0/1830081278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc05800e850 con 0x7fc05c004000 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 -- 172.21.15.73:0/1830081278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 msgr2=0x7fc05c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 --2- 172.21.15.73:0/1830081278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc05c004690 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 rx=0x7fc058006d80 tx=0x7fc058009060).stop 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc071440700 1 -- 172.21.15.73:0/1830081278 reap_dead start 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 -- 172.21.15.73:0/1830081278 shutdown_connections 2023-11-28T03:09:32.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 -- 172.21.15.73:0/1830081278 >> 172.21.15.73:0/1830081278 conn(0x7fc06c0642d0 msgr2=0x7fc06c0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 -- 172.21.15.73:0/1830081278 shutdown_connections 2023-11-28T03:09:32.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.367+0000 7fc072ea3700 1 -- 172.21.15.73:0/1830081278 wait complete. 2023-11-28T03:09:32.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.368+0000 7fc072ea3700 1 Processor -- start 2023-11-28T03:09:32.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.368+0000 7fc072ea3700 1 -- start start 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c0c3810 0x7fc06c1491b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c14a7b0 0x7fc06c14ac30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c14d2d0 con 0x7fc05c004000 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c14a320 con 0x7fc06c14a7b0 2023-11-28T03:09:32.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06c154950 con 0x7fc06c0c3810 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53668/0 (socket says 172.21.15.73:53668) 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 -- 172.21.15.73:0/2994944330 learned_addr learned my addr 172.21.15.73:0/2994944330 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc06bfff700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c0c3810 0x7fc06c1491b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc071440700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c14a7b0 0x7fc06c14ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c0c3810 msgr2=0x7fc06c1491b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c0c3810 0x7fc06c1491b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c14a7b0 msgr2=0x7fc06c14ac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c14a7b0 0x7fc06c14ac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 -- 172.21.15.73:0/2994944330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc06c1499f0 con 0x7fc05c004000 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc070c3f700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fc06000e5c0 tx=0x7fc060009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060001570 con 0x7fc05c004000 2023-11-28T03:09:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0600016d0 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc06c149cd0 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.369+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc060001a50 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.370+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc06c0c3c90 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.370+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc06c0c2360 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.371+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fc060014430 con 0x7fc05c004000 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.371+0000 7fc068ff9700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 0x7fc050061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.371+0000 7fc06bfff700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 msgr2=0x7fc050061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:32.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.371+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fc06007a410 con 0x7fc05c004000 2023-11-28T03:09:32.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.371+0000 7fc06bfff700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 0x7fc050061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:32.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.375+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc060049af0 con 0x7fc05c004000 2023-11-28T03:09:32.421 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:32.420+0000 7f20bbdbe680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:09:32.461 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:32.459+0000 7fadbf74e680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:09:32.493 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:32.493+0000 7f20bbdbe680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:09:32.533 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:32.532+0000 7fadbf74e680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:09:32.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.572+0000 7fc06bfff700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 msgr2=0x7fc050061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:32.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.572+0000 7fc06bfff700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 0x7fc050061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:32.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.579+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc06c14a600 con 0x7fc05c004000 2023-11-28T03:09:32.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.579+0000 7fc068ff9700 1 -- 172.21.15.73:0/2994944330 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v113) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc06004b240 con 0x7fc05c004000 2023-11-28T03:09:32.583 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:09:32.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 msgr2=0x7fc050061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:32.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 0x7fc050061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 msgr2=0x7fc06c154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 rx=0x7fc06000e5c0 tx=0x7fc060009040).stop 2023-11-28T03:09:32.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 shutdown_connections 2023-11-28T03:09:32.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc05005f390 0x7fc050061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc06c14a7b0 0x7fc06c14ac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc05c004000 0x7fc06c154390 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 --2- 172.21.15.73:0/2994944330 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc06c0c3810 0x7fc06c1491b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 >> 172.21.15.73:0/2994944330 conn(0x7fc06c0642d0 msgr2=0x7fc06c066040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 shutdown_connections 2023-11-28T03:09:32.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.581+0000 7fc072ea3700 1 -- 172.21.15.73:0/2994944330 wait complete. 2023-11-28T03:09:32.596 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:09:32.680 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:32.679+0000 7f20bbdbe680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:09:32.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.715+0000 7f060301e700 1 Processor -- start 2023-11-28T03:09:32.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.715+0000 7f060301e700 1 -- start start 2023-11-28T03:09:32.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f060301e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 0x7f05fc0a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f060301e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0c7e30 con 0x7f05fc062700 2023-11-28T03:09:32.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f060301e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05fc0bcf40 con 0x7f05fc0c86f0 2023-11-28T03:09:32.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f060301e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc0c6b20 con 0x7f05fc1547f0 2023-11-28T03:09:32.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f06015bb700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f05fc062700 0x7f05fc0a4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:33464/0 (socket says 172.21.15.73:33464) 2023-11-28T03:09:32.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f06015bb700 1 -- 172.21.15.73:0/831997414 learned_addr learned my addr 172.21.15.73:0/831997414 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:32.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.716+0000 7f0600dba700 1 --2- 172.21.15.73:0/831997414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 0x7f05fc0a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25725190 0 0) 0x7f05fc0c7e30 con 0x7f05fc062700 2023-11-28T03:09:32.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05ec0033f0 con 0x7f05fc062700 2023-11-28T03:09:32.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2822997234 0 0) 0x7f05fc0bcf40 con 0x7f05fc0c86f0 2023-11-28T03:09:32.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05fc0c7e30 con 0x7f05fc0c86f0 2023-11-28T03:09:32.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f0600dba700 1 -- 172.21.15.73:0/831997414 >> v1:172.21.15.150:6789/0 conn(0x7f05fc0c86f0 legacy=0x7f05fc0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f0600dba700 1 -- 172.21.15.73:0/831997414 >> v1:172.21.15.73:6789/0 conn(0x7f05fc062700 legacy=0x7f05fc0a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f0600dba700 1 -- 172.21.15.73:0/831997414 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05fc0a77d0 con 0x7f05fc1547f0 2023-11-28T03:09:32.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f0600dba700 1 --2- 172.21.15.73:0/831997414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 0x7f05fc0a72e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f05f0009790 tx=0x7f05f0009060).ready entity=mon.2 client_cookie=cebd0e9b0122ee5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f00090d0 con 0x7f05fc1547f0 2023-11-28T03:09:32.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000f510 con 0x7f05fc1547f0 2023-11-28T03:09:32.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.717+0000 7f05fb7fe700 1 -- 172.21.15.73:0/831997414 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000f890 con 0x7f05fc1547f0 2023-11-28T03:09:32.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 -- 172.21.15.73:0/831997414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 msgr2=0x7f05fc0a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 --2- 172.21.15.73:0/831997414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 0x7f05fc0a72e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 rx=0x7f05f0009790 tx=0x7f05f0009060).stop 2023-11-28T03:09:32.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 -- 172.21.15.73:0/831997414 shutdown_connections 2023-11-28T03:09:32.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 --2- 172.21.15.73:0/831997414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc1547f0 0x7f05fc0a72e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 -- 172.21.15.73:0/831997414 >> 172.21.15.73:0/831997414 conn(0x7f05fc0642d0 msgr2=0x7f05fc0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 -- 172.21.15.73:0/831997414 shutdown_connections 2023-11-28T03:09:32.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.718+0000 7f060301e700 1 -- 172.21.15.73:0/831997414 wait complete. 2023-11-28T03:09:32.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.719+0000 7f060301e700 1 Processor -- start 2023-11-28T03:09:32.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.719+0000 7f060301e700 1 -- start start 2023-11-28T03:09:32.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc0c86f0 0x7f05fc0c3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc1547f0 0x7f05fc0c3540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc168dd0 con 0x7f05fc062700 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc167ee0 con 0x7f05fc1547f0 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f060301e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05fc1681c0 con 0x7f05fc0c86f0 2023-11-28T03:09:32.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53678/0 (socket says 172.21.15.73:53678) 2023-11-28T03:09:32.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 -- 172.21.15.73:0/532651676 learned_addr learned my addr 172.21.15.73:0/532651676 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:32.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f06015bb700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc1547f0 0x7f05fc0c3540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f05fbfff700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc0c86f0 0x7f05fc0c3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:32.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 -- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc0c86f0 msgr2=0x7f05fc0c3050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc0c86f0 0x7f05fc0c3050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 -- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc1547f0 msgr2=0x7f05fc0c3540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc1547f0 0x7f05fc0c3540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 -- 172.21.15.73:0/532651676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05f00092a0 con 0x7f05fc062700 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f0600dba700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f05fc0695b0 tx=0x7f05f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.720+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f000f890 con 0x7f05fc062700 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.721+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05f000fc50 con 0x7f05fc062700 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.721+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05fc1684a0 con 0x7f05fc062700 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.721+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05f001b410 con 0x7f05fc062700 2023-11-28T03:09:32.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.721+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05fc168a60 con 0x7f05fc062700 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.722+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05fc06c860 con 0x7f05fc062700 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.722+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f05f001b780 con 0x7f05fc062700 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.723+0000 7f05f8ff9700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 0x7f05e0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.723+0000 7f05fbfff700 1 -- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 msgr2=0x7f05e0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.723+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f05f007be30 con 0x7f05fc062700 2023-11-28T03:09:32.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.723+0000 7f05fbfff700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 0x7f05e0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:32.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.727+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f05f004ccc0 con 0x7f05fc062700 2023-11-28T03:09:32.735 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:32.731+0000 7fadbf74e680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:09:32.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.897+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05fc0ab970 con 0x7f05fc062700 2023-11-28T03:09:32.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.897+0000 7f05f8ff9700 1 -- 172.21.15.73:0/532651676 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v113) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05f0029020 con 0x7f05fc062700 2023-11-28T03:09:32.899 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 msgr2=0x7f05e0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 0x7f05e0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 msgr2=0x7f05fc152610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 rx=0x7f05fc0695b0 tx=0x7f05f0017000).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 shutdown_connections 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f05e005f390 0x7f05e0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05fc1547f0 0x7f05fc0c3540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05fc062700 0x7f05fc152610 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 --2- 172.21.15.73:0/532651676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05fc0c86f0 0x7f05fc0c3050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 >> 172.21.15.73:0/532651676 conn(0x7f05fc0642d0 msgr2=0x7f05fc062bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.899+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 shutdown_connections 2023-11-28T03:09:32.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:32.900+0000 7f060301e700 1 -- 172.21.15.73:0/532651676 wait complete. 2023-11-28T03:09:32.912 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:09:33.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.033+0000 7f85bd12a700 1 Processor -- start 2023-11-28T03:09:33.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.033+0000 7f85bd12a700 1 -- start start 2023-11-28T03:09:33.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bd12a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f85b80bcc10 0x7f85b80bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bd12a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bd12a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85b8151f90 con 0x7f85b8062c20 2023-11-28T03:09:33.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bd12a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85b8062790 con 0x7f85b80bcc10 2023-11-28T03:09:33.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bd12a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8065e90 con 0x7f85b80653d0 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85b7fff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f85b80bcc10 0x7f85b80bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bc929700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f85b8062c20 0x7f85b80630b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:47906/0 (socket says 172.21.15.73:47906) 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85bc929700 1 -- 172.21.15.73:0/798810614 learned_addr learned my addr 172.21.15.73:0/798810614 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85b7fff700 1 --2- 172.21.15.73:0/798810614 >> v2:172.21.15.150:3300/0 conn(0x7f85b80bcc10 0x7f85b80bcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:45730/0 (socket says 172.21.15.73:45730) 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.034+0000 7f85b77fe700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:33552/0 (socket says 172.21.15.73:33552) 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b6ffd700 1 -- 172.21.15.73:0/798810614 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4068687834 0 0) 0x7f85b8151f90 con 0x7f85b8062c20 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b6ffd700 1 -- 172.21.15.73:0/798810614 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f85a00033f0 con 0x7f85b8062c20 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b7fff700 1 -- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 msgr2=0x7f85b8065850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b7fff700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b7fff700 1 -- 172.21.15.73:0/798810614 >> v1:172.21.15.150:6789/0 conn(0x7f85b8062c20 legacy=0x7f85b80630b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b7fff700 1 -- 172.21.15.73:0/798810614 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85b8150e50 con 0x7f85b80bcc10 2023-11-28T03:09:33.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b77fe700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:33.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b7fff700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b80bcc10 0x7f85b80bcff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f85a8009760 tx=0x7f85a8009060).ready entity=mon.1 client_cookie=f7edd8afe0a68353 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b6ffd700 1 -- 172.21.15.73:0/798810614 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a80090d0 con 0x7f85b80bcc10 2023-11-28T03:09:33.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b6ffd700 1 -- 172.21.15.73:0/798810614 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85a800d4a0 con 0x7f85b80bcc10 2023-11-28T03:09:33.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.035+0000 7f85b6ffd700 1 -- 172.21.15.73:0/798810614 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85a800d820 con 0x7f85b80bcc10 2023-11-28T03:09:33.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 -- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b80bcc10 msgr2=0x7f85b80bcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b80bcc10 0x7f85b80bcff0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 rx=0x7f85a8009760 tx=0x7f85a8009060).stop 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 -- 172.21.15.73:0/798810614 shutdown_connections 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b80bcc10 0x7f85b80bcff0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 --2- 172.21.15.73:0/798810614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80653d0 0x7f85b8065850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 -- 172.21.15.73:0/798810614 >> 172.21.15.73:0/798810614 conn(0x7f85b80c4010 msgr2=0x7f85b80c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 -- 172.21.15.73:0/798810614 shutdown_connections 2023-11-28T03:09:33.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.036+0000 7f85bd12a700 1 -- 172.21.15.73:0/798810614 wait complete. 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.037+0000 7f85bd12a700 1 Processor -- start 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.037+0000 7f85bd12a700 1 -- start start 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b8062c20 0x7f85b8148dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80bcc10 0x7f85b8156210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b81492c0 con 0x7f85b80653d0 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85b8160600 con 0x7f85b8062c20 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bd12a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85b80abbe0 con 0x7f85b80bcc10 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53694/0 (socket says 172.21.15.73:53694) 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 -- 172.21.15.73:0/1210825211 learned_addr learned my addr 172.21.15.73:0/1210825211 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:33.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85bc929700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80bcc10 0x7f85b8156210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b7fff700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b8062c20 0x7f85b8148dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 -- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80bcc10 msgr2=0x7f85b8156210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80bcc10 0x7f85b8156210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 -- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b8062c20 msgr2=0x7f85b8148dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b8062c20 0x7f85b8148dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.038+0000 7f85b77fe700 1 -- 172.21.15.73:0/1210825211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85ac00b540 con 0x7f85b80653d0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f85b77fe700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f85ac010f70 tx=0x7f85ac00b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac00ee10 con 0x7f85b80653d0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85ac011450 con 0x7f85b80653d0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a80092a0 con 0x7f85b80653d0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85ac0115c0 con 0x7f85b80653d0 2023-11-28T03:09:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.039+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85b80ab2b0 con 0x7f85b80653d0 2023-11-28T03:09:33.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85b8066970 con 0x7f85b80653d0 2023-11-28T03:09:33.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f85ac017020 con 0x7f85b80653d0 2023-11-28T03:09:33.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f859ffff700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 0x7f85a4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f85b7fff700 1 -- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 msgr2=0x7f85a4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:33.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f85ac07d0b0 con 0x7f85b80653d0 2023-11-28T03:09:33.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.040+0000 7f85b7fff700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 0x7f85a4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:33.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.048+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f85ac04df40 con 0x7f85b80653d0 2023-11-28T03:09:33.101 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.100+0000 7f20bbdbe680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:09:33.176 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.175+0000 7fadbf74e680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:09:33.205 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.204+0000 7f20bbdbe680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:09:33.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.220+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85b806c7c0 con 0x7f85b80653d0 2023-11-28T03:09:33.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.220+0000 7f859ffff700 1 -- 172.21.15.73:0/1210825211 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v113) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85ac022350 con 0x7f85b80653d0 2023-11-28T03:09:33.221 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:09:33.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 msgr2=0x7f85a4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 0x7f85a4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 msgr2=0x7f85b80ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 rx=0x7f85ac010f70 tx=0x7f85ac00b300).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 shutdown_connections 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f85a405f390 0x7f85a4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85b80bcc10 0x7f85b8156210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85b8062c20 0x7f85b8148dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 --2- 172.21.15.73:0/1210825211 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85b80653d0 0x7f85b80ac070 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 >> 172.21.15.73:0/1210825211 conn(0x7f85b80c4010 msgr2=0x7f85b80c67c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 shutdown_connections 2023-11-28T03:09:33.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.222+0000 7f85bd12a700 1 -- 172.21.15.73:0/1210825211 wait complete. 2023-11-28T03:09:33.236 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:09:33.287 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.286+0000 7fadbf74e680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:09:33.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 Processor -- start 2023-11-28T03:09:33.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 -- start start 2023-11-28T03:09:33.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 0x7f7e080c6620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e08062910 con 0x7f7e08150d10 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e080c6c00 con 0x7f7e080bcfa0 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.355+0000 7f7e0ef54700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e080c2460 con 0x7f7e08150930 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e07fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7e08150d10 0x7f7e080a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35346/0 (socket says 172.21.15.73:35346) 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e07fff700 1 -- 172.21.15.73:0/1795699026 learned_addr learned my addr 172.21.15.73:0/1795699026 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e0ccf0700 1 --2- 172.21.15.73:0/1795699026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 0x7f7e080c6620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3737137493 0 0) 0x7f7e08062910 con 0x7f7e08150d10 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7df80033f0 con 0x7f7e08150d10 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817388703 0 0) 0x7f7e080c6c00 con 0x7f7e080bcfa0 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7e08062910 con 0x7f7e080bcfa0 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e0ccf0700 1 -- 172.21.15.73:0/1795699026 >> v1:172.21.15.150:6789/0 conn(0x7f7e080bcfa0 legacy=0x7f7e08062da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 288765804 0 0) 0x7f7df80033f0 con 0x7f7e08150d10 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e0ccf0700 1 -- 172.21.15.73:0/1795699026 >> v1:172.21.15.73:6789/0 conn(0x7f7e08150d10 legacy=0x7f7e080a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e0ccf0700 1 -- 172.21.15.73:0/1795699026 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e080c6ee0 con 0x7f7e08150930 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e0ccf0700 1 --2- 172.21.15.73:0/1795699026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 0x7f7e080c6620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7dfc00a900 tx=0x7f7dfc00a170).ready entity=mon.2 client_cookie=cb3b8bdd37cbe5d6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.356+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df40015f0 con 0x7f7e08150930 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7dfc005880 con 0x7f7e08150930 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e077fe700 1 -- 172.21.15.73:0/1795699026 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7dfc005c20 con 0x7f7e08150930 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1795699026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 msgr2=0x7f7e080c6620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1795699026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 0x7f7e080c6620 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 rx=0x7f7dfc00a900 tx=0x7f7dfc00a170).stop 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1795699026 shutdown_connections 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1795699026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150930 0x7f7e080c6620 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1795699026 >> 172.21.15.73:0/1795699026 conn(0x7f7e080c8230 msgr2=0x7f7e080c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1795699026 shutdown_connections 2023-11-28T03:09:33.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.357+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1795699026 wait complete. 2023-11-28T03:09:33.361 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.358+0000 7f20bbdbe680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:09:33.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.358+0000 7f7e0ef54700 1 Processor -- start 2023-11-28T03:09:33.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 -- start start 2023-11-28T03:09:33.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 0x7f7e0814b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150d10 0x7f7e0814d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e0814d760 con 0x7f7e08150930 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08163fe0 con 0x7f7e080bcfa0 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.359+0000 7f7e0ef54700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e08164e40 con 0x7f7e08150d10 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54956/0 (socket says 172.21.15.73:54956) 2023-11-28T03:09:33.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 -- 172.21.15.73:0/1132932568 learned_addr learned my addr 172.21.15.73:0/1132932568 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e0d4f1700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150d10 0x7f7e0814d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e0ccf0700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 0x7f7e0814b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 -- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150d10 msgr2=0x7f7e0814d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150d10 0x7f7e0814d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 -- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 msgr2=0x7f7e0814b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 0x7f7e0814b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 -- 172.21.15.73:0/1132932568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7dfc00a3f0 con 0x7f7e08150930 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e0ccf0700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 0x7f7e0814b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e07fff700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f7df400c810 tx=0x7f7df4002ac0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df40072e0 con 0x7f7e08150930 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7df4007460 con 0x7f7e08150930 2023-11-28T03:09:33.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e0814baa0 con 0x7f7e08150930 2023-11-28T03:09:33.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7df40058c0 con 0x7f7e08150930 2023-11-28T03:09:33.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.360+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e08164510 con 0x7f7e08150930 2023-11-28T03:09:33.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.361+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e0806c860 con 0x7f7e08150930 2023-11-28T03:09:33.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.362+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f7df4005c70 con 0x7f7e08150930 2023-11-28T03:09:33.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.363+0000 7f7e04ff9700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 0x7f7dec061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.363+0000 7f7e0ccf0700 1 -- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 msgr2=0x7f7dec061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:33.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.363+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f7df407a1c0 con 0x7f7e08150930 2023-11-28T03:09:33.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.363+0000 7f7e0ccf0700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 0x7f7dec061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:33.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.366+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7df404aff0 con 0x7f7e08150930 2023-11-28T03:09:33.435 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.434+0000 7f20bbdbe680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:09:33.447 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.446+0000 7fadbf74e680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:09:33.502 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.501+0000 7f20bbdbe680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:09:33.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.523+0000 7fadbf74e680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:09:33.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.540+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7e08066bb0 con 0x7f7e08150930 2023-11-28T03:09:33.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.540+0000 7f7e04ff9700 1 -- 172.21.15.73:0/1132932568 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v113) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7df4018070 con 0x7f7e08150930 2023-11-28T03:09:33.541 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:09:33.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 msgr2=0x7f7dec061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 0x7f7dec061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 msgr2=0x7f7e0814cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 rx=0x7f7df400c810 tx=0x7f7df4002ac0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 shutdown_connections 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7dec05f390 0x7f7dec061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7e08150d10 0x7f7e0814d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7e080bcfa0 0x7f7e0814b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 --2- 172.21.15.73:0/1132932568 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7e08150930 0x7f7e0814cd80 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 >> 172.21.15.73:0/1132932568 conn(0x7f7e080c8230 msgr2=0x7f7e080ac720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 shutdown_connections 2023-11-28T03:09:33.545 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.542+0000 7f7e0ef54700 1 -- 172.21.15.73:0/1132932568 wait complete. 2023-11-28T03:09:33.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:33.596 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.595+0000 7fadbf74e680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:09:33.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.672+0000 7f783e98c700 1 Processor -- start 2023-11-28T03:09:33.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f783e98c700 1 -- start start 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f783e98c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838150720 0x7f7838150b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f783e98c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7838062910 con 0x7f78381545c0 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f783e98c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7838154ac0 con 0x7f78380c2b10 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f783e98c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78380ab1d0 con 0x7f7838150720 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.673+0000 7f7837fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838150720 0x7f7838150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7838150720 0x7f7838150b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54960/0 (socket says 172.21.15.73:54960) 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 -- 172.21.15.73:0/3557201065 learned_addr learned my addr 172.21.15.73:0/3557201065 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:33.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f78377fe700 1 --1- 172.21.15.73:0/3557201065 >> v1:172.21.15.73:6789/0 conn(0x7f78381545c0 0x7f78380a9c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35352/0 (socket says 172.21.15.73:35352) 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 -- 172.21.15.73:0/3557201065 >> v1:172.21.15.150:6789/0 conn(0x7f78380c2b10 legacy=0x7f7838062da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 -- 172.21.15.73:0/3557201065 >> v1:172.21.15.73:6789/0 conn(0x7f78381545c0 legacy=0x7f78380a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 -- 172.21.15.73:0/3557201065 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7838154da0 con 0x7f7838150720 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7837fff700 1 --2- 172.21.15.73:0/3557201065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838150720 0x7f7838150b00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f782c006fd0 tx=0x7f782c0090f0).ready entity=mon.0 client_cookie=fe05182488d7c065 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7836ffd700 1 -- 172.21.15.73:0/3557201065 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00b600 con 0x7f7838150720 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7836ffd700 1 -- 172.21.15.73:0/3557201065 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f782c00b760 con 0x7f7838150720 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.674+0000 7f7836ffd700 1 -- 172.21.15.73:0/3557201065 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f782c00bae0 con 0x7f7838150720 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 -- 172.21.15.73:0/3557201065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838150720 msgr2=0x7f7838150b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 --2- 172.21.15.73:0/3557201065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838150720 0x7f7838150b00 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 rx=0x7f782c006fd0 tx=0x7f782c0090f0).stop 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 -- 172.21.15.73:0/3557201065 shutdown_connections 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 --2- 172.21.15.73:0/3557201065 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7838150720 0x7f7838150b00 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 -- 172.21.15.73:0/3557201065 >> 172.21.15.73:0/3557201065 conn(0x7f78380c7760 msgr2=0x7f78380c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 -- 172.21.15.73:0/3557201065 shutdown_connections 2023-11-28T03:09:33.678 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.675+0000 7f783e98c700 1 -- 172.21.15.73:0/3557201065 wait complete. 2023-11-28T03:09:33.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.676+0000 7f783e98c700 1 Processor -- start 2023-11-28T03:09:33.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.676+0000 7f783e98c700 1 -- start start 2023-11-28T03:09:33.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78380c2b10 0x7f7838145220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7838150720 0x7f7838148550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838146640 con 0x7f78381545c0 2023-11-28T03:09:33.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7838145a30 con 0x7f78380c2b10 2023-11-28T03:09:33.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783e98c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7838145d10 con 0x7f7838150720 2023-11-28T03:09:33.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54970/0 (socket says 172.21.15.73:54970) 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 -- 172.21.15.73:0/3996327208 learned_addr learned my addr 172.21.15.73:0/3996327208 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f7837fff700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78380c2b10 0x7f7838145220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f78377fe700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7838150720 0x7f7838148550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7838150720 msgr2=0x7f7838148550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7838150720 0x7f7838148550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78380c2b10 msgr2=0x7f7838145220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78380c2b10 0x7f7838145220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 -- 172.21.15.73:0/3996327208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f782c009370 con 0x7f78381545c0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f783cf29700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f783000c7c0 tx=0x7f78300015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7830010510 con 0x7f78381545c0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.677+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7830010670 con 0x7f78381545c0 2023-11-28T03:09:33.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.678+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7838145ff0 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.678+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78300109f0 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.678+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7838145710 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.678+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f783806c860 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.680+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f7830025430 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.680+0000 7f781ffff700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 0x7f7824061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.680+0000 7f7837fff700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 msgr2=0x7f7824061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.680+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f783007aee0 con 0x7f78381545c0 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.680+0000 7f7837fff700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 0x7f7824061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:33.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.683+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f783004bdd0 con 0x7f78381545c0 2023-11-28T03:09:33.763 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.762+0000 7f20bbdbe680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:09:33.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:33.867+0000 7fadbf74e680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:09:33.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.881+0000 7f7837fff700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 msgr2=0x7f7824061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:33.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.881+0000 7f7837fff700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 0x7f7824061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:33.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.897+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7838146780 con 0x7f78381545c0 2023-11-28T03:09:33.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.897+0000 7f781ffff700 1 -- 172.21.15.73:0/3996327208 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f783004f3f0 con 0x7f78381545c0 2023-11-28T03:09:33.899 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:33.899 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:09:33.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 msgr2=0x7f7824061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:33.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 0x7f7824061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 msgr2=0x7f783814d4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 rx=0x7f783000c7c0 tx=0x7f78300015e0).stop 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 shutdown_connections 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f782405f360 0x7f7824061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78381545c0 0x7f783814d4a0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78380c2b10 0x7f7838145220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 --2- 172.21.15.73:0/3996327208 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7838150720 0x7f7838148550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:33.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 >> 172.21.15.73:0/3996327208 conn(0x7f78380c7760 msgr2=0x7f78380c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:33.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 shutdown_connections 2023-11-28T03:09:33.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:33.899+0000 7f783e98c700 1 -- 172.21.15.73:0/3996327208 wait complete. 2023-11-28T03:09:33.903 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:09:33.913 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:33.945 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:33.945+0000 7f20bbdbe680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:09:34.059 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:34.058+0000 7fadbf74e680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:09:34.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.071+0000 7f5ac2e56700 1 Processor -- start 2023-11-28T03:09:34.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.071+0000 7f5ac2e56700 1 -- start start 2023-11-28T03:09:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2e56700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2e56700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 0x7f5abc0c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2e56700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5abc0c12c0 con 0x7f5abc0c0180 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2e56700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5abc0c06b0 con 0x7f5abc0c1750 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2e56700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0c0990 con 0x7f5abc157660 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac1e54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac2655700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 0x7f5abc0c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac1e54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:46884/0 (socket says 172.21.15.73:46884) 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac1e54700 1 -- 172.21.15.73:0/4170311244 learned_addr learned my addr 172.21.15.73:0/4170311244 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.072+0000 7f5ac1653700 1 --1- 172.21.15.73:0/4170311244 >> v1:172.21.15.150:6789/0 conn(0x7f5abc0c0180 0x7f5abc1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:57874/0 (socket says 172.21.15.73:57874) 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac1e54700 1 -- 172.21.15.73:0/4170311244 >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 msgr2=0x7f5abc0c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac1e54700 1 --2- 172.21.15.73:0/4170311244 >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 0x7f5abc0c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac1e54700 1 -- 172.21.15.73:0/4170311244 >> v1:172.21.15.150:6789/0 conn(0x7f5abc0c0180 legacy=0x7f5abc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac0e52700 1 -- 172.21.15.73:0/4170311244 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1870549707 0 0) 0x7f5abc0c12c0 con 0x7f5abc0c0180 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac1e54700 1 -- 172.21.15.73:0/4170311244 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5abc0c0c70 con 0x7f5abc157660 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac2655700 1 --2- 172.21.15.73:0/4170311244 >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 0x7f5abc0c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac1e54700 1 --2- 172.21.15.73:0/4170311244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5aac006fd0 tx=0x7f5aac0090f0).ready entity=mon.2 client_cookie=e2b96dee6ce09a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac0e52700 1 -- 172.21.15.73:0/4170311244 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00b580 con 0x7f5abc157660 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac0e52700 1 -- 172.21.15.73:0/4170311244 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00b6e0 con 0x7f5abc157660 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.073+0000 7f5ac0e52700 1 -- 172.21.15.73:0/4170311244 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00ba60 con 0x7f5abc157660 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 -- 172.21.15.73:0/4170311244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 msgr2=0x7f5abc157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/4170311244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 rx=0x7f5aac006fd0 tx=0x7f5aac0090f0).stop 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 -- 172.21.15.73:0/4170311244 shutdown_connections 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/4170311244 >> v2:172.21.15.150:3300/0 conn(0x7f5abc0c1750 0x7f5abc0c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/4170311244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc157660 0x7f5abc157a40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 -- 172.21.15.73:0/4170311244 >> 172.21.15.73:0/4170311244 conn(0x7f5abc0cb000 msgr2=0x7f5abc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 -- 172.21.15.73:0/4170311244 shutdown_connections 2023-11-28T03:09:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.074+0000 7f5ac2e56700 1 -- 172.21.15.73:0/4170311244 wait complete. 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.075+0000 7f5ac2e56700 1 Processor -- start 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.075+0000 7f5ac2e56700 1 -- start start 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc0c1750 0x7f5abc156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5abc157660 0x7f5abc146b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc0c1fd0 con 0x7f5abc0c0180 2023-11-28T03:09:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc154ff0 con 0x7f5abc157660 2023-11-28T03:09:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5abc154a30 con 0x7f5abc0c1750 2023-11-28T03:09:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54976/0 (socket says 172.21.15.73:54976) 2023-11-28T03:09:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 -- 172.21.15.73:0/185767376 learned_addr learned my addr 172.21.15.73:0/185767376 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1653700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc0c1750 0x7f5abc156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2655700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5abc157660 0x7f5abc146b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 -- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc0c1750 msgr2=0x7f5abc156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc0c1750 0x7f5abc156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 -- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5abc157660 msgr2=0x7f5abc146b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5abc157660 0x7f5abc146b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 -- 172.21.15.73:0/185767376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ab4009a20 con 0x7f5abc0c0180 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac1e54700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f5abc0c0ad0 tx=0x7f5aac017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac00b850 con 0x7f5abc0c0180 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aac009370 con 0x7f5abc0c0180 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5aac00b9b0 con 0x7f5abc0c0180 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.076+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5aac01c670 con 0x7f5abc0c0180 2023-11-28T03:09:34.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.077+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5abc0c1c60 con 0x7f5abc0c0180 2023-11-28T03:09:34.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.078+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5abc0c22b0 con 0x7f5abc0c0180 2023-11-28T03:09:34.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.078+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f5aac01ca20 con 0x7f5abc0c0180 2023-11-28T03:09:34.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.079+0000 7f5ab1ffb700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 0x7f5aa0061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.079+0000 7f5ac1653700 1 -- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 msgr2=0x7f5aa0061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:34.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.079+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f5aac07bf40 con 0x7f5abc0c0180 2023-11-28T03:09:34.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.079+0000 7f5ac1653700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 0x7f5aa0061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:34.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.083+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5aac04ce80 con 0x7f5abc0c0180 2023-11-28T03:09:34.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.253+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5abc06c7c0 con 0x7f5abc0c0180 2023-11-28T03:09:34.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.254+0000 7f5ab1ffb700 1 -- 172.21.15.73:0/185767376 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17428 (secure 0 0 0) 0x7f5aac0504a0 con 0x7f5abc0c0180 2023-11-28T03:09:34.256 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:34.256 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":113,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:30.299893+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3656330984":"2023-11-29T03:08:28.497043+0000","172.21.15.73:0/2798049488":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:28.497043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 msgr2=0x7f5aa0061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 0x7f5aa0061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 msgr2=0x7f5abc0c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 rx=0x7f5abc0c0ad0 tx=0x7f5aac017000).stop 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 shutdown_connections 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5aa005f360 0x7f5aa0061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5abc157660 0x7f5abc146b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5abc0c0180 0x7f5abc0c2460 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 --2- 172.21.15.73:0/185767376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5abc0c1750 0x7f5abc156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 >> 172.21.15.73:0/185767376 conn(0x7f5abc0cb000 msgr2=0x7f5abc0ac780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 shutdown_connections 2023-11-28T03:09:34.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.256+0000 7f5ac2e56700 1 -- 172.21.15.73:0/185767376 wait complete. 2023-11-28T03:09:34.271 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:09:34.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.387+0000 7f3fd5a44700 1 Processor -- start 2023-11-28T03:09:34.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.387+0000 7f3fd5a44700 1 -- start start 2023-11-28T03:09:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fd5a44700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fd5a44700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00a6720 con 0x7f3fd0062700 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fd5a44700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3fd00c2540 con 0x7f3fd0154230 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fd5a44700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00a4f70 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54980/0 (socket says 172.21.15.73:54980) 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 -- 172.21.15.73:0/2658500038 learned_addr learned my addr 172.21.15.73:0/2658500038 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fd5243700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3fd0062700 0x7f3fd0062b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35364/0 (socket says 172.21.15.73:35364) 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 -- 172.21.15.73:0/2658500038 >> v1:172.21.15.150:6789/0 conn(0x7f3fd0154230 legacy=0x7f3fd00a6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 -- 172.21.15.73:0/2658500038 >> v1:172.21.15.73:6789/0 conn(0x7f3fd0062700 legacy=0x7f3fd0062b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcffff700 1 -- 172.21.15.73:0/2658500038 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00a5250 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.388+0000 7f3fcf7fe700 1 -- 172.21.15.73:0/2658500038 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1536753520 0 0) 0x7f3fd00a6720 con 0x7f3fd0062700 2023-11-28T03:09:34.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fcffff700 1 --2- 172.21.15.73:0/2658500038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f3fbc006fd0 tx=0x7f3fbc0090f0).ready entity=mon.0 client_cookie=66e810c97fada88c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fcf7fe700 1 -- 172.21.15.73:0/2658500038 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00b600 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fcf7fe700 1 -- 172.21.15.73:0/2658500038 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fbc00b760 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fcf7fe700 1 -- 172.21.15.73:0/2658500038 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc00bae0 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 -- 172.21.15.73:0/2658500038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd00c2cb0 msgr2=0x7f3fd00a48e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/2658500038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 rx=0x7f3fbc006fd0 tx=0x7f3fbc0090f0).stop 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 -- 172.21.15.73:0/2658500038 shutdown_connections 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/2658500038 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd00c2cb0 0x7f3fd00a48e0 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 -- 172.21.15.73:0/2658500038 >> 172.21.15.73:0/2658500038 conn(0x7f3fd00c7ca0 msgr2=0x7f3fd00c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 -- 172.21.15.73:0/2658500038 shutdown_connections 2023-11-28T03:09:34.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.389+0000 7f3fd5a44700 1 -- 172.21.15.73:0/2658500038 wait complete. 2023-11-28T03:09:34.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.390+0000 7f3fd5a44700 1 Processor -- start 2023-11-28T03:09:34.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 -- start start 2023-11-28T03:09:34.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3fd00c2cb0 0x7f3fd014f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd0154230 0x7f3fd00ac350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0150bb0 con 0x7f3fd0062700 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd014ffa0 con 0x7f3fd0154230 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd5a44700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd0150280 con 0x7f3fd00c2cb0 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd4a42700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd4a42700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:54992/0 (socket says 172.21.15.73:54992) 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.391+0000 7f3fd4a42700 1 -- 172.21.15.73:0/408072504 learned_addr learned my addr 172.21.15.73:0/408072504 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd5243700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd0154230 0x7f3fd00ac350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fcffff700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3fd00c2cb0 0x7f3fd014f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3fd00c2cb0 msgr2=0x7f3fd014f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3fd00c2cb0 0x7f3fd014f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd0154230 msgr2=0x7f3fd00ac350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd0154230 0x7f3fd00ac350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 -- 172.21.15.73:0/408072504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fbc009370 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd4a42700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f3fc400c440 tx=0x7f3fc4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4007020 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fc4007180 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd0150560 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fc4007520 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.392+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00ad6f0 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.393+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd006c7c0 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.394+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3fc40068b0 con 0x7f3fd0062700 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.394+0000 7f3fccff9700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 0x7f3fb4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.394+0000 7f3fcffff700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 msgr2=0x7f3fb4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:34.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.394+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f3fc4079990 con 0x7f3fd0062700 2023-11-28T03:09:34.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.394+0000 7f3fcffff700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 0x7f3fb4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:34.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.401+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3fc4047af0 con 0x7f3fd0062700 2023-11-28T03:09:34.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.595+0000 7f3fcffff700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 msgr2=0x7f3fb4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:34.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.595+0000 7f3fcffff700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 0x7f3fb4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:34.600 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:34.599+0000 7f20bbdbe680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:09:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.613+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3fd00a5530 con 0x7f3fd0062700 2023-11-28T03:09:34.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.613+0000 7f3fccff9700 1 -- 172.21.15.73:0/408072504 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f3fc404a820 con 0x7f3fd0062700 2023-11-28T03:09:34.615 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:34.615 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:09:34.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 msgr2=0x7f3fb4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:34.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 0x7f3fb4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 msgr2=0x7f3fd014f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 rx=0x7f3fc400c440 tx=0x7f3fc4007e00).stop 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 shutdown_connections 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3fb405f390 0x7f3fb4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3fd0154230 0x7f3fd00ac350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3fd0062700 0x7f3fd014f430 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 --2- 172.21.15.73:0/408072504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3fd00c2cb0 0x7f3fd014f920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 >> 172.21.15.73:0/408072504 conn(0x7f3fd00c7ca0 msgr2=0x7f3fd0152900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 shutdown_connections 2023-11-28T03:09:34.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.615+0000 7f3fd5a44700 1 -- 172.21.15.73:0/408072504 wait complete. 2023-11-28T03:09:34.619 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:09:34.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:34.668 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:34.667+0000 7f20bbdbe680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:09:34.742 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:34.742+0000 7f20bbdbe680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:09:34.746 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:34.745+0000 7fadbf74e680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:09:34.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.751+0000 7ffa3aa7f700 1 Processor -- start 2023-11-28T03:09:34.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.751+0000 7ffa3aa7f700 1 -- start start 2023-11-28T03:09:34.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3aa7f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa34062700 0x7ffa34062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3aa7f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ffa340661b0 0x7ffa34066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3aa7f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa340630a0 con 0x7ffa341542d0 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3aa7f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa34065760 con 0x7ffa34062700 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3aa7f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa340c3250 con 0x7ffa340661b0 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa34062700 0x7ffa34062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ffa34062700 0x7ffa34062bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55008/0 (socket says 172.21.15.73:55008) 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 -- 172.21.15.73:0/2820764118 learned_addr learned my addr 172.21.15.73:0/2820764118 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa3901c700 1 --2- 172.21.15.73:0/2820764118 >> v2:172.21.15.150:3300/0 conn(0x7ffa340661b0 0x7ffa34066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 -- 172.21.15.73:0/2820764118 >> v2:172.21.15.150:3300/0 conn(0x7ffa340661b0 msgr2=0x7ffa34066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 --2- 172.21.15.73:0/2820764118 >> v2:172.21.15.150:3300/0 conn(0x7ffa340661b0 0x7ffa34066640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 -- 172.21.15.73:0/2820764118 >> v1:172.21.15.150:6789/0 conn(0x7ffa341542d0 legacy=0x7ffa340697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 -- 172.21.15.73:0/2820764118 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa3406a340 con 0x7ffa34062700 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa33fff700 1 --2- 172.21.15.73:0/2820764118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa34062700 0x7ffa34062bb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7ffa3405e680 tx=0x7ffa200090f0).ready entity=mon.0 client_cookie=14fe69194152c50f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa337fe700 1 -- 172.21.15.73:0/2820764118 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000b520 con 0x7ffa34062700 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.752+0000 7ffa337fe700 1 -- 172.21.15.73:0/2820764118 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2000b680 con 0x7ffa34062700 2023-11-28T03:09:34.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa337fe700 1 -- 172.21.15.73:0/2820764118 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2000ba00 con 0x7ffa34062700 2023-11-28T03:09:34.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/2820764118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa34062700 msgr2=0x7ffa34062bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/2820764118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa34062700 0x7ffa34062bb0 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 rx=0x7ffa3405e680 tx=0x7ffa200090f0).stop 2023-11-28T03:09:34.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/2820764118 shutdown_connections 2023-11-28T03:09:34.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/2820764118 >> v2:172.21.15.150:3300/0 conn(0x7ffa340661b0 0x7ffa34066640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/2820764118 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa34062700 0x7ffa34062bb0 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/2820764118 >> 172.21.15.73:0/2820764118 conn(0x7ffa340c7fd0 msgr2=0x7ffa340c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/2820764118 shutdown_connections 2023-11-28T03:09:34.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.753+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/2820764118 wait complete. 2023-11-28T03:09:34.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.754+0000 7ffa3aa7f700 1 Processor -- start 2023-11-28T03:09:34.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.754+0000 7ffa3aa7f700 1 -- start start 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa34062700 0x7ffa34151290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa340661b0 0x7ffa34164cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340aa050 con 0x7ffa341542d0 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa34151780 con 0x7ffa34062700 2023-11-28T03:09:34.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3aa7f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa340ad5b0 con 0x7ffa340661b0 2023-11-28T03:09:34.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3901c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3901c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55018/0 (socket says 172.21.15.73:55018) 2023-11-28T03:09:34.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3901c700 1 -- 172.21.15.73:0/4189231135 learned_addr learned my addr 172.21.15.73:0/4189231135 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:34.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa3881b700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa34062700 0x7ffa34151290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.755+0000 7ffa33fff700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa340661b0 0x7ffa34164cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:34.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa340661b0 msgr2=0x7ffa34164cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa340661b0 0x7ffa34164cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa34062700 msgr2=0x7ffa34151290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa34062700 0x7ffa34151290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 -- 172.21.15.73:0/4189231135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa20009370 con 0x7ffa341542d0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3901c700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7ffa2c009e90 tx=0x7ffa2c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00ca40 con 0x7ffa341542d0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa2c00f040 con 0x7ffa341542d0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa340aca30 con 0x7ffa341542d0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c00cdb0 con 0x7ffa341542d0 2023-11-28T03:09:34.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.756+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa340acff0 con 0x7ffa341542d0 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.757+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa3406c860 con 0x7ffa341542d0 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.757+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7ffa2c013610 con 0x7ffa341542d0 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.758+0000 7ffa30ff9700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 0x7ffa18061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.758+0000 7ffa3881b700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 msgr2=0x7ffa18061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.758+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7ffa2c07afb0 con 0x7ffa341542d0 2023-11-28T03:09:34.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.758+0000 7ffa3881b700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 0x7ffa18061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:34.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.762+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ffa2c0a9af0 con 0x7ffa341542d0 2023-11-28T03:09:34.822 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:34.819+0000 7fadbf74e680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:09:34.891 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:34.890+0000 7f20bbdbe680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:09:34.898 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:34.897+0000 7fadbf74e680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:09:34.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.959+0000 7ffa3881b700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 msgr2=0x7ffa18061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:34.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.959+0000 7ffa3881b700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 0x7ffa18061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:34.961 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:34.960+0000 7f20bbdbe680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:09:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.976+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffa3406a620 con 0x7ffa341542d0 2023-11-28T03:09:34.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.976+0000 7ffa30ff9700 1 -- 172.21.15.73:0/4189231135 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 40 v40) v1 ==== 94+0+4494 (secure 0 0 0) 0x7ffa2c04bd30 con 0x7ffa341542d0 2023-11-28T03:09:34.978 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:34.978 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":40,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37}],"filesystems":[{"mdsmap":{"epoch":40,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:08:17.668973+0000","modified":"2023-11-28T03:08:19.679287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7128},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7128":{"gid":7128,"name":"c","rank":0,"incarnation":38,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/2863774554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2863774554},{"type":"v1","addr":"172.21.15.73:6836","nonce":2863774554}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[14],"metadata_pool":13,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":6}]} 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 msgr2=0x7ffa18061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 0x7ffa18061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 msgr2=0x7ffa340aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 rx=0x7ffa2c009e90 tx=0x7ffa2c00b040).stop 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 shutdown_connections 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ffa1805f390 0x7ffa18061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ffa341542d0 0x7ffa340aaf60 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ffa34062700 0x7ffa34151290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 --2- 172.21.15.73:0/4189231135 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ffa340661b0 0x7ffa34164cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:34.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 >> 172.21.15.73:0/4189231135 conn(0x7ffa340c7fd0 msgr2=0x7ffa340be420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:34.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 shutdown_connections 2023-11-28T03:09:34.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:34.979+0000 7ffa3aa7f700 1 -- 172.21.15.73:0/4189231135 wait complete. 2023-11-28T03:09:34.981 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 40 2023-11-28T03:09:34.992 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:35.053 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.052+0000 7fadbf74e680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:09:35.075 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:35.074+0000 7f20bbdbe680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:09:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.107+0000 7f54dcfb0700 1 Processor -- start 2023-11-28T03:09:35.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.107+0000 7f54dcfb0700 1 -- start start 2023-11-28T03:09:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54dcfb0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54d8066e00 0x7f54d80671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54dcfb0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d80bd590 con 0x7f54d8155e40 2023-11-28T03:09:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54dcfb0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54d80a6090 con 0x7f54d8067980 2023-11-28T03:09:35.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54dcfb0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54d8156370 con 0x7f54d8066e00 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54d5d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f54d8155e40 0x7f54d80a5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35370/0 (socket says 172.21.15.73:35370) 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.108+0000 7f54d5d9b700 1 -- 172.21.15.73:0/428724995 learned_addr learned my addr 172.21.15.73:0/428724995 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d659c700 1 --2- 172.21.15.73:0/428724995 >> v2:172.21.15.150:3300/0 conn(0x7f54d8066e00 0x7f54d80671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3990953505 0 0) 0x7f54d80bd590 con 0x7f54d8155e40 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54c00033f0 con 0x7f54d8155e40 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685236491 0 0) 0x7f54d80a6090 con 0x7f54d8067980 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54d80bd590 con 0x7f54d8067980 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2372127593 0 0) 0x7f54c00033f0 con 0x7f54d8155e40 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d659c700 1 -- 172.21.15.73:0/428724995 >> v1:172.21.15.150:6789/0 conn(0x7f54d8067980 legacy=0x7f54d80bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d659c700 1 -- 172.21.15.73:0/428724995 >> v1:172.21.15.73:6789/0 conn(0x7f54d8155e40 legacy=0x7f54d80a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d659c700 1 -- 172.21.15.73:0/428724995 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54d8156650 con 0x7f54d8066e00 2023-11-28T03:09:35.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d659c700 1 --2- 172.21.15.73:0/428724995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80671e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f54c8009760 tx=0x7f54c8009060).ready entity=mon.1 client_cookie=8329df355754f667 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc001470 con 0x7f54d8066e00 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54c8011400 con 0x7f54d8066e00 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.109+0000 7f54d559a700 1 -- 172.21.15.73:0/428724995 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54c8011780 con 0x7f54d8066e00 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 -- 172.21.15.73:0/428724995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 msgr2=0x7f54d80671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/428724995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80671e0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 rx=0x7f54c8009760 tx=0x7f54c8009060).stop 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 -- 172.21.15.73:0/428724995 shutdown_connections 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/428724995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80671e0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 -- 172.21.15.73:0/428724995 >> 172.21.15.73:0/428724995 conn(0x7f54d80c7fd0 msgr2=0x7f54d80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 -- 172.21.15.73:0/428724995 shutdown_connections 2023-11-28T03:09:35.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.110+0000 7f54dcfb0700 1 -- 172.21.15.73:0/428724995 wait complete. 2023-11-28T03:09:35.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.111+0000 7f54dcfb0700 1 Processor -- start 2023-11-28T03:09:35.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.111+0000 7f54dcfb0700 1 -- start start 2023-11-28T03:09:35.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80c0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54d8155e40 0x7f54d80c0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80c0b70 con 0x7f54d8067980 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54d81511b0 con 0x7f54d8066e00 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54dcfb0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54d80c14a0 con 0x7f54d8155e40 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55024/0 (socket says 172.21.15.73:55024) 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 -- 172.21.15.73:0/3154284909 learned_addr learned my addr 172.21.15.73:0/3154284909 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d659c700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80c0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d6d9d700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54d8155e40 0x7f54d80c0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 -- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54d8155e40 msgr2=0x7f54d80c0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54d8155e40 0x7f54d80c0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 -- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 msgr2=0x7f54d80c0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80c0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 -- 172.21.15.73:0/3154284909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54c80092a0 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.112+0000 7f54d5d9b700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f54cc00c800 tx=0x7f54cc001ab0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.113+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc007170 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.113+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54cc0072d0 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.113+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54cc0048c0 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.113+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54d80c22c0 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.113+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54d80c1990 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f54cc004c30 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54d806c860 con 0x7f54d8067980 2023-11-28T03:09:35.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54c67fc700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 0x7f54b8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54d659c700 1 -- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 msgr2=0x7f54b8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:35.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7f54cc079fc0 con 0x7f54d8067980 2023-11-28T03:09:35.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.114+0000 7f54d659c700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 0x7f54b8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:35.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.118+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f54cc04ae50 con 0x7f54d8067980 2023-11-28T03:09:35.136 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.135+0000 7fadbf74e680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:09:35.184 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:35.183+0000 7f20bbdbe680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:09:35.257 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.256+0000 7fadbf74e680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:09:35.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.301+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f54d8156930 con 0x7f54d8067980 2023-11-28T03:09:35.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.302+0000 7f54c67fc700 1 -- 172.21.15.73:0/3154284909 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v113) v1 ==== 74+0+17428 (secure 0 0 0) 0x7f54cc019030 con 0x7f54d8067980 2023-11-28T03:09:35.303 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:35.304 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":113,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:30.299893+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"113","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":13,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:08:16.209998+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"109","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":14,"pool_name":"cephfs_data","create_time":"2023-11-28T03:08:16.638888+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"110","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.27","mappings":[{"from":1,"to":3}]},{"pgid":"14.17","mappings":[{"from":1,"to":2}]},{"pgid":"14.2a","mappings":[{"from":1,"to":7}]},{"pgid":"14.36","mappings":[{"from":1,"to":5}]},{"pgid":"14.38","mappings":[{"from":1,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3656330984":"2023-11-29T03:08:28.497043+0000","172.21.15.73:0/2798049488":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6833/90017":"2023-11-29T03:08:28.497043+0000","172.21.15.73:6832/90017":"2023-11-29T03:08:28.497043+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:35.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 msgr2=0x7f54b8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 0x7f54b8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 msgr2=0x7f54d8150cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 rx=0x7f54cc00c800 tx=0x7f54cc001ab0).stop 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 shutdown_connections 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54b805f390 0x7f54b8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:35.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54d8155e40 0x7f54d80c0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54d8066e00 0x7f54d80c0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 --2- 172.21.15.73:0/3154284909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54d8067980 0x7f54d8150cc0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 >> 172.21.15.73:0/3154284909 conn(0x7f54d80c7fd0 msgr2=0x7f54d80adb50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:35.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 shutdown_connections 2023-11-28T03:09:35.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.304+0000 7f54dcfb0700 1 -- 172.21.15.73:0/3154284909 wait complete. 2023-11-28T03:09:35.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:09:35.372 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.371+0000 7fadbf74e680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:09:35.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.437+0000 7fa3e2744700 1 Processor -- start 2023-11-28T03:09:35.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.437+0000 7fa3e2744700 1 -- start start 2023-11-28T03:09:35.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3e2744700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa3dc155ae0 0x7fa3dc0bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3e2744700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc155650 con 0x7fa3dc0c6b30 2023-11-28T03:09:35.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3e2744700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3dc155f60 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3e2744700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3dc14bff0 con 0x7fa3dc155ae0 2023-11-28T03:09:35.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3dbfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa3dc0c6b30 0x7fa3dc1524b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35384/0 (socket says 172.21.15.73:35384) 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3dbfff700 1 -- 172.21.15.73:0/1069840934 learned_addr learned my addr 172.21.15.73:0/1069840934 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3db7fe700 1 --2- 172.21.15.73:0/1069840934 >> v2:172.21.15.150:3300/0 conn(0x7fa3dc155ae0 0x7fa3dc0bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.438+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2824280737 0 0) 0x7fa3dc155650 con 0x7fa3dc0c6b30 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3c40033f0 con 0x7fa3dc0c6b30 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3db7fe700 1 -- 172.21.15.73:0/1069840934 >> v1:172.21.15.150:6789/0 conn(0x7fa3dc0bdc30 legacy=0x7fa3dc0c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3db7fe700 1 -- 172.21.15.73:0/1069840934 >> v1:172.21.15.73:6789/0 conn(0x7fa3dc0c6b30 legacy=0x7fa3dc1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2706332289 0 0) 0x7fa3c40033f0 con 0x7fa3dc0c6b30 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3db7fe700 1 -- 172.21.15.73:0/1069840934 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3dc0be0a0 con 0x7fa3dc155ae0 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3db7fe700 1 --2- 172.21.15.73:0/1069840934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc0bd740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa3cc006fd0 tx=0x7fa3cc006f40).ready entity=mon.1 client_cookie=8438450803995f7b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00015f0 con 0x7fa3dc155ae0 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3cc005880 con 0x7fa3dc155ae0 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.439+0000 7fa3daffd700 1 -- 172.21.15.73:0/1069840934 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3cc005c00 con 0x7fa3dc155ae0 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 -- 172.21.15.73:0/1069840934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 msgr2=0x7fa3dc0bd740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 --2- 172.21.15.73:0/1069840934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc0bd740 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 rx=0x7fa3cc006fd0 tx=0x7fa3cc006f40).stop 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 -- 172.21.15.73:0/1069840934 shutdown_connections 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 --2- 172.21.15.73:0/1069840934 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc0bd740 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 -- 172.21.15.73:0/1069840934 >> 172.21.15.73:0/1069840934 conn(0x7fa3dc0c8230 msgr2=0x7fa3dc0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 -- 172.21.15.73:0/1069840934 shutdown_connections 2023-11-28T03:09:35.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.440+0000 7fa3e2744700 1 -- 172.21.15.73:0/1069840934 wait complete. 2023-11-28T03:09:35.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.441+0000 7fa3e2744700 1 Processor -- start 2023-11-28T03:09:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.441+0000 7fa3e2744700 1 -- start start 2023-11-28T03:09:35.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3dc0c6b30 0x7fa3dc0a9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc1542a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0bfd90 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0a9c00 con 0x7fa3dc155ae0 2023-11-28T03:09:35.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e2744700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3dc0a9ee0 con 0x7fa3dc0c6b30 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55040/0 (socket says 172.21.15.73:55040) 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 -- 172.21.15.73:0/116030195 learned_addr learned my addr 172.21.15.73:0/116030195 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3db7fe700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3dc0c6b30 0x7fa3dc0a9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3e0ce1700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc1542a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:35.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3dc0c6b30 msgr2=0x7fa3dc0a9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3dc0c6b30 0x7fa3dc0a9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 msgr2=0x7fa3dc1542a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc1542a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 -- 172.21.15.73:0/116030195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3cc009250 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.442+0000 7fa3dbfff700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fa3d00010e0 tx=0x7fa3d0001440).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:35.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.443+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d0007320 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.443+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d00074c0 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.443+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00058c0 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.443+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3dc0aa1c0 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.443+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3dc149f30 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.444+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3dc06c860 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.444+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 44) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa3d0005c30 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.445+0000 7fa3c3fff700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 0x7fa3c8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:35.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.445+0000 7fa3db7fe700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 msgr2=0x7fa3c8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:35.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.445+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6438+0+0 (secure 0 0 0) 0x7fa3d007a630 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.445+0000 7fa3db7fe700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 0x7fa3c8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:35.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.449+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa3d004b4c0 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.477 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:35.476+0000 7f20bbdbe680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:09:35.550 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:35.549+0000 7f20bbdbe680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:09:35.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.646+0000 7fa3db7fe700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 msgr2=0x7fa3c8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:35.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.646+0000 7fa3db7fe700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 0x7fa3c8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:09:35.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.662+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa3dc063b60 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.675 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.674+0000 7fadbf74e680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:09:35.746 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.745+0000 7fadbf74e680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:09:35.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.764+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 45) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fa3d0077fc0 con 0x7fa3dc0bdc30 2023-11-28T03:09:35.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.764+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 msgr2=0x7fa3c8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:35.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:35.764+0000 7fa3c3fff700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 0x7fa3c8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T03:09:35.771 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:09:35.775 INFO:tasks.ceph.mds.c.smithi073.stderr: -5296> 2023-11-28T03:08:23.599+0000 7f5fb6828700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -10> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -9> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -8> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -7> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -6> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -5> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -4> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:09:35.782 INFO:tasks.ceph.mds.c.smithi073.stderr: -3> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T03:09:35.783 INFO:tasks.ceph.mds.c.smithi073.stderr: -1> 2023-11-28T03:09:35.770+0000 7f5fb3021700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:09:35.801 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.819+0000 7fad3cee6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.819+0000 7fad3cee6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.819+0000 7fad3cee6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.819+0000 7fad3cee6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.819+0000 7fad3cee6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.820+0000 7fad396df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.820+0000 7fad396df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.820+0000 7fad396df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.822 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.820+0000 7fad396df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:35.822 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:35.820+0000 7fad396df700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:09:36.323 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:09:36.322+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:09:36.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.330+0000 7fa3c3fff700 1 -- 172.21.15.73:0/116030195 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v41) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa3d006f0a0 con 0x7fa3dc0bdc30 2023-11-28T03:09:36.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 msgr2=0x7fa3dc149a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 rx=0x7fa3d00010e0 tx=0x7fa3d0001440).stop 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 shutdown_connections 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa3c805f390 0x7fa3c8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa3dc155ae0 0x7fa3dc1542a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa3dc0bdc30 0x7fa3dc149a40 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 --2- 172.21.15.73:0/116030195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa3dc0c6b30 0x7fa3dc0a9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 >> 172.21.15.73:0/116030195 conn(0x7fa3dc0c8230 msgr2=0x7fa3dc14a800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 shutdown_connections 2023-11-28T03:09:36.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.333+0000 7fa3e2744700 1 -- 172.21.15.73:0/116030195 wait complete. 2023-11-28T03:09:36.336 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:09:36.348 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:09:36.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.462+0000 7f519bfdc700 1 Processor -- start 2023-11-28T03:09:36.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.462+0000 7f519bfdc700 1 -- start start 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519bfdc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5194158420 0x7f5194064350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519bfdc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 0x7f5194098c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519bfdc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51940c12d0 con 0x7f5194158800 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519bfdc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51940c61c0 con 0x7f5194158420 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519bfdc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5194099130 con 0x7f51940987b0 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5194158420 0x7f5194064350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5194158420 0x7f5194064350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55114/0 (socket says 172.21.15.73:55114) 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 -- 172.21.15.73:0/1234788606 learned_addr learned my addr 172.21.15.73:0/1234788606 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 -- 172.21.15.73:0/1234788606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 msgr2=0x7f5194098c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519a579700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 0x7f5194098c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5198d76700 1 -- 172.21.15.73:0/1234788606 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 350694199 0 0) 0x7f51940c12d0 con 0x7f5194158800 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 0x7f5194098c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 -- 172.21.15.73:0/1234788606 >> v1:172.21.15.73:6789/0 conn(0x7f5194158800 legacy=0x7f51940c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 -- 172.21.15.73:0/1234788606 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51940c5e90 con 0x7f5194158420 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f519a579700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 0x7f5194098c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:36.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5199d78700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5194158420 0x7f5194064350 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f518c009f20 tx=0x7f518c009840).ready entity=mon.0 client_cookie=f0c4918aaf4fbaa3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.463+0000 7f5198d76700 1 -- 172.21.15.73:0/1234788606 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00e070 con 0x7f5194158420 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f5198d76700 1 -- 172.21.15.73:0/1234788606 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c00ba80 con 0x7f5194158420 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f5198d76700 1 -- 172.21.15.73:0/1234788606 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c00be00 con 0x7f5194158420 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 -- 172.21.15.73:0/1234788606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5194158420 msgr2=0x7f5194064350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5194158420 0x7f5194064350 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 rx=0x7f518c009f20 tx=0x7f518c009840).stop 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 -- 172.21.15.73:0/1234788606 shutdown_connections 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51940987b0 0x7f5194098c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 --2- 172.21.15.73:0/1234788606 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5194158420 0x7f5194064350 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 -- 172.21.15.73:0/1234788606 >> 172.21.15.73:0/1234788606 conn(0x7f51940cb290 msgr2=0x7f51940cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 -- 172.21.15.73:0/1234788606 shutdown_connections 2023-11-28T03:09:36.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.464+0000 7f519bfdc700 1 -- 172.21.15.73:0/1234788606 wait complete. 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.465+0000 7f519bfdc700 1 Processor -- start 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.465+0000 7f519bfdc700 1 -- start start 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5194158420 0x7f519414f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5194158800 0x7f51940a9040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940ca400 con 0x7f51940987b0 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51940c9f70 con 0x7f5194158420 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519bfdc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51940c9360 con 0x7f5194158800 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55130/0 (socket says 172.21.15.73:55130) 2023-11-28T03:09:36.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 -- 172.21.15.73:0/4130325561 learned_addr learned my addr 172.21.15.73:0/4130325561 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f519a579700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5194158800 0x7f51940a9040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199577700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5194158420 0x7f519414f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 -- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5194158800 msgr2=0x7f51940a9040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5194158800 0x7f51940a9040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 -- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5194158420 msgr2=0x7f519414f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5194158420 0x7f519414f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 -- 172.21.15.73:0/4130325561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f518c009ae0 con 0x7f51940987b0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.466+0000 7f5199d78700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f5194062e00 tx=0x7f51940a7fc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.467+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c0092e0 con 0x7f51940987b0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.467+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c00bbf0 con 0x7f51940987b0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.467+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c01e680 con 0x7f51940987b0 2023-11-28T03:09:36.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.467+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51940c9640 con 0x7f51940987b0 2023-11-28T03:09:36.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.467+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51940c9c00 con 0x7f51940987b0 2023-11-28T03:09:36.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.468+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51940ca250 con 0x7f51940987b0 2023-11-28T03:09:36.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.468+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 45) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f518c003040 con 0x7f51940987b0 2023-11-28T03:09:36.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.468+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f518c07b530 con 0x7f51940987b0 2023-11-28T03:09:36.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.472+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f518c04c870 con 0x7f51940987b0 2023-11-28T03:09:36.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.677+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f51940c9d90 con 0x7f51940987b0 2023-11-28T03:09:36.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.766+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f518c0792c0 con 0x7f51940987b0 2023-11-28T03:09:36.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.767+0000 7f5189ffb700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 0x7f5178067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:36.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.767+0000 7f5199577700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 0x7f5178067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:36.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:36.768+0000 7f5199577700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 0x7f5178067d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f51940aa0f0 tx=0x7f5184009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:37.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.333+0000 7f5189ffb700 1 -- 172.21.15.73:0/4130325561 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v42) v1 ==== 104+0+0 (secure 0 0 0) 0x7f518c055970 con 0x7f51940987b0 2023-11-28T03:09:37.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 msgr2=0x7f5178067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 0x7f5178067d70 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 rx=0x7f51940aa0f0 tx=0x7f5184009040).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 msgr2=0x7f51940caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 rx=0x7f5194062e00 tx=0x7f51940a7fc0).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 shutdown_connections 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51780302c0 0x7f5178067d70 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5194158800 0x7f51940a9040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51940987b0 0x7f51940caaf0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 --2- 172.21.15.73:0/4130325561 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5194158420 0x7f519414f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 >> 172.21.15.73:0/4130325561 conn(0x7f51940cb290 msgr2=0x7f51940af920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.335+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 shutdown_connections 2023-11-28T03:09:37.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.336+0000 7f519bfdc700 1 -- 172.21.15.73:0/4130325561 wait complete. 2023-11-28T03:09:37.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:09:37.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.464+0000 7f18d7f47700 1 Processor -- start 2023-11-28T03:09:37.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.464+0000 7f18d7f47700 1 -- start start 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d7f47700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18d0154910 0x7f18d00a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d7f47700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d00aa220 0x7f18d00aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d7f47700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18d00c6630 con 0x7f18d0154530 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d7f47700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18d00c6390 con 0x7f18d0154910 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d7f47700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d00a84e0 con 0x7f18d00aa220 2023-11-28T03:09:37.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18d0154910 0x7f18d00a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f18d0154910 0x7f18d00a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55134/0 (socket says 172.21.15.73:55134) 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 -- 172.21.15.73:0/2575117364 learned_addr learned my addr 172.21.15.73:0/2575117364 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d64e4700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d00aa220 0x7f18d00aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 -- 172.21.15.73:0/2575117364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d00aa220 msgr2=0x7f18d00aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d00aa220 0x7f18d00aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 -- 172.21.15.73:0/2575117364 >> v1:172.21.15.150:6789/0 conn(0x7f18d0154530 legacy=0x7f18d00c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 -- 172.21.15.73:0/2575117364 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18d00aaba0 con 0x7f18d0154910 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.465+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d0154910 0x7f18d00a7ff0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f18c0006fd0 tx=0x7f18c00090f0).ready entity=mon.0 client_cookie=d87791bc42270485 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d4ce1700 1 -- 172.21.15.73:0/2575117364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000b580 con 0x7f18d0154910 2023-11-28T03:09:37.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d4ce1700 1 -- 172.21.15.73:0/2575117364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18c000b6e0 con 0x7f18d0154910 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d4ce1700 1 -- 172.21.15.73:0/2575117364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18c000ba60 con 0x7f18d0154910 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 -- 172.21.15.73:0/2575117364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d0154910 msgr2=0x7f18d00a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d0154910 0x7f18d00a7ff0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 rx=0x7f18c0006fd0 tx=0x7f18c00090f0).stop 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 -- 172.21.15.73:0/2575117364 shutdown_connections 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d00aa220 0x7f18d00aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2575117364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d0154910 0x7f18d00a7ff0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 -- 172.21.15.73:0/2575117364 >> 172.21.15.73:0/2575117364 conn(0x7f18d0094930 msgr2=0x7f18d006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 -- 172.21.15.73:0/2575117364 shutdown_connections 2023-11-28T03:09:37.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.466+0000 7f18d7f47700 1 -- 172.21.15.73:0/2575117364 wait complete. 2023-11-28T03:09:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.467+0000 7f18d7f47700 1 Processor -- start 2023-11-28T03:09:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.467+0000 7f18d7f47700 1 -- start start 2023-11-28T03:09:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 0x7f18d0150e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d0154910 0x7f18d0144fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d01652c0 con 0x7f18d00aa220 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18d0145490 con 0x7f18d0154530 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d7f47700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18d01646b0 con 0x7f18d0154910 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55140/0 (socket says 172.21.15.73:55140) 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 -- 172.21.15.73:0/2536532153 learned_addr learned my addr 172.21.15.73:0/2536532153 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d64e4700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d0154910 0x7f18d0144fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 0x7f18d0150e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 -- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d0154910 msgr2=0x7f18d0144fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d0154910 0x7f18d0144fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 -- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 msgr2=0x7f18d0150e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.468+0000 7f18d5ce3700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 0x7f18d0150e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:37.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18d5ce3700 1 -- 172.21.15.73:0/2536532153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18c0009370 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 0x7f18d0150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18d5ce3700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f18cc00c4f0 tx=0x7f18cc007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007020 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc007180 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc007520 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18d0164990 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.469+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18d0164f50 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.470+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f18cc006900 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.470+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18d01655a0 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.470+0000 7f18c5ffb700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 0x7f18b40614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.470+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(115..115 src has 1..115) v4 ==== 6782+0+0 (secure 0 0 0) 0x7f18cc079c60 con 0x7f18d00aa220 2023-11-28T03:09:37.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.470+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 0x7f18b40614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:37.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.471+0000 7f18d54e2700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 0x7f18b40614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f18c0009130 tx=0x7f18c0017000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:37.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.475+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f18cc04ae70 con 0x7f18d00aa220 2023-11-28T03:09:37.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:37.638+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f18d006c860 con 0x7f18d00aa220 2023-11-28T03:09:38.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.335+0000 7f18c5ffb700 1 -- 172.21.15.73:0/2536532153 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v116) v1 ==== 181+0+0 (secure 0 0 0) 0x7f18cc079020 con 0x7f18d00aa220 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 msgr2=0x7f18b40614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 0x7f18b40614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f18c0009130 tx=0x7f18c0017000).stop 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 msgr2=0x7f18d00aad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 rx=0x7f18cc00c4f0 tx=0x7f18cc007df0).stop 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 shutdown_connections 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f18b405f030 0x7f18b40614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f18d0154910 0x7f18d0144fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f18d00aa220 0x7f18d00aad60 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 --2- 172.21.15.73:0/2536532153 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f18d0154530 0x7f18d0150e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 >> 172.21.15.73:0/2536532153 conn(0x7f18d0094930 msgr2=0x7f18d0153240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 shutdown_connections 2023-11-28T03:09:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.339+0000 7f18d7f47700 1 -- 172.21.15.73:0/2536532153 wait complete. 2023-11-28T03:09:38.341 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:09:38.354 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:09:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.470+0000 7f9dbcdc3700 1 Processor -- start 2023-11-28T03:09:38.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.470+0000 7f9dbcdc3700 1 -- start start 2023-11-28T03:09:38.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9dbcdc3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db8154610 0x7f9db81582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9dbcdc3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9db80aa1c0 0x7f9db80aa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9dbcdc3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9db8159210 con 0x7f9db80ab3b0 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9dbcdc3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9db80c0c70 con 0x7f9db80aa1c0 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9dbcdc3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80c2110 con 0x7f9db8154610 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9db80aa1c0 0x7f9db80aa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9db80aa1c0 0x7f9db80aa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55156/0 (socket says 172.21.15.73:55156) 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 -- 172.21.15.73:0/609684352 learned_addr learned my addr 172.21.15.73:0/609684352 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db659c700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db8154610 0x7f9db81582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 -- 172.21.15.73:0/609684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db8154610 msgr2=0x7f9db81582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:38.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db8154610 0x7f9db81582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 -- 172.21.15.73:0/609684352 >> v1:172.21.15.150:6789/0 conn(0x7f9db80ab3b0 legacy=0x7f9db80c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 -- 172.21.15.73:0/609684352 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db80aaae0 con 0x7f9db80aa1c0 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.471+0000 7f9db6d9d700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db80aa1c0 0x7f9db80aa5f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9db0009f20 tx=0x7f9db0009840).ready entity=mon.0 client_cookie=ec3b0ea60194f26f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9db5d9b700 1 -- 172.21.15.73:0/609684352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000e070 con 0x7f9db80aa1c0 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9db5d9b700 1 -- 172.21.15.73:0/609684352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9db000ba40 con 0x7f9db80aa1c0 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9db5d9b700 1 -- 172.21.15.73:0/609684352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9db000bdc0 con 0x7f9db80aa1c0 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/609684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db80aa1c0 msgr2=0x7f9db80aa5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db80aa1c0 0x7f9db80aa5f0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 rx=0x7f9db0009f20 tx=0x7f9db0009840).stop 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/609684352 shutdown_connections 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db80aa1c0 0x7f9db80aa5f0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/609684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db8154610 0x7f9db81582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/609684352 >> 172.21.15.73:0/609684352 conn(0x7f9db80655a0 msgr2=0x7f9db8069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/609684352 shutdown_connections 2023-11-28T03:09:38.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.472+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/609684352 wait complete. 2023-11-28T03:09:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.473+0000 7f9dbcdc3700 1 Processor -- start 2023-11-28T03:09:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 -- start start 2023-11-28T03:09:38.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db80aa1c0 0x7f9db80b0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 0x7f9db80c9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80af020 con 0x7f9db8154610 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80af300 con 0x7f9db80aeb40 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dbcdc3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9db80af5e0 con 0x7f9db80aa1c0 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dadd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dadd9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55160/0 (socket says 172.21.15.73:55160) 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9dadd9b700 1 -- 172.21.15.73:0/971099502 learned_addr learned my addr 172.21.15.73:0/971099502 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9db659c700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db80aa1c0 0x7f9db80b0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.474+0000 7f9db6d9d700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 0x7f9db80c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 -- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db80aa1c0 msgr2=0x7f9db80b0e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db80aa1c0 0x7f9db80b0e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 -- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 msgr2=0x7f9db80c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 0x7f9db80c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 -- 172.21.15.73:0/971099502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9db0009ae0 con 0x7f9db8154610 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9db6d9d700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 0x7f9db80c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dadd9b700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f9da0009460 tx=0x7f9da0006ed0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0009e60 con 0x7f9db8154610 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da0013460 con 0x7f9db8154610 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9db80af8c0 con 0x7f9db8154610 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da0013800 con 0x7f9db8154610 2023-11-28T03:09:38.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.475+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9db80ae340 con 0x7f9db8154610 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.476+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9db806c860 con 0x7f9db8154610 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.476+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 46) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f9da0013bb0 con 0x7f9db8154610 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.477+0000 7f9daeffd700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 0x7f9d980614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.477+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(116..116 src has 1..116) v4 ==== 6315+0+0 (secure 0 0 0) 0x7f9da00797b0 con 0x7f9db8154610 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.477+0000 7f9db659c700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 0x7f9d980614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.477+0000 7f9db659c700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 0x7f9d980614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9da800ab30 tx=0x7f9da8008040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:38.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.481+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9da004ab90 con 0x7f9db8154610 2023-11-28T03:09:38.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.645+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9db80c0540 con 0x7f9db8154610 2023-11-28T03:09:38.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:38.775+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9da0077690 con 0x7f9db8154610 2023-11-28T03:09:39.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.348+0000 7f9daeffd700 1 -- 172.21.15.73:0/971099502 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v117) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9da001d3f0 con 0x7f9db8154610 2023-11-28T03:09:39.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 msgr2=0x7f9d980614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 0x7f9d980614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9da800ab30 tx=0x7f9da8008040).stop 2023-11-28T03:09:39.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 msgr2=0x7f9db81451c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 rx=0x7f9da0009460 tx=0x7f9da0006ed0).stop 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 shutdown_connections 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9d9805f030 0x7f9d980614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9db80aeb40 0x7f9db80c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9db80aa1c0 0x7f9db80b0e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 --2- 172.21.15.73:0/971099502 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9db8154610 0x7f9db81451c0 secure :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 rx=0x7f9da0009460 tx=0x7f9da0006ed0).stop 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.351+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 >> 172.21.15.73:0/971099502 conn(0x7f9db80655a0 msgr2=0x7f9db814dcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.352+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 shutdown_connections 2023-11-28T03:09:39.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.352+0000 7f9dbcdc3700 1 -- 172.21.15.73:0/971099502 wait complete. 2023-11-28T03:09:39.354 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:09:39.367 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:09:39.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.483+0000 7f19217d0700 1 Processor -- start 2023-11-28T03:09:39.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.483+0000 7f19217d0700 1 -- start start 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f19217d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f191c14e530 0x7f191c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f19217d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 0x7f191c14e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f19217d0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f191c0c7bc0 con 0x7f191c158570 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f19217d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c155920 con 0x7f191c14e530 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f19217d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f191c154e00 con 0x7f191c154950 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f191c14e530 0x7f191c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f1920fcf700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f191c158570 0x7f191c0aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35398/0 (socket says 172.21.15.73:35398) 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f1920fcf700 1 -- 172.21.15.73:0/1589600581 learned_addr learned my addr 172.21.15.73:0/1589600581 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191b7fe700 1 --2- 172.21.15.73:0/1589600581 >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 0x7f191c14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 -- 172.21.15.73:0/1589600581 >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 msgr2=0x7f191c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 --2- 172.21.15.73:0/1589600581 >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 0x7f191c14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 -- 172.21.15.73:0/1589600581 >> v1:172.21.15.73:6789/0 conn(0x7f191c158570 legacy=0x7f191c0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 -- 172.21.15.73:0/1589600581 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f191c155080 con 0x7f191c14e530 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191b7fe700 1 --2- 172.21.15.73:0/1589600581 >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 0x7f191c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191affd700 1 -- 172.21.15.73:0/1589600581 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3873882726 0 0) 0x7f191c0c7bc0 con 0x7f191c158570 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.484+0000 7f191bfff700 1 --2- 172.21.15.73:0/1589600581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c66f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f191c157f20 tx=0x7f190c009060).ready entity=mon.0 client_cookie=e7ff36d2a8d7508 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f191affd700 1 -- 172.21.15.73:0/1589600581 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00b580 con 0x7f191c14e530 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f191affd700 1 -- 172.21.15.73:0/1589600581 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c00b6e0 con 0x7f191c14e530 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f191affd700 1 -- 172.21.15.73:0/1589600581 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00ba60 con 0x7f191c14e530 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 -- 172.21.15.73:0/1589600581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 msgr2=0x7f191c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 --2- 172.21.15.73:0/1589600581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c66f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 rx=0x7f191c157f20 tx=0x7f190c009060).stop 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 -- 172.21.15.73:0/1589600581 shutdown_connections 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 --2- 172.21.15.73:0/1589600581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c66f0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 --2- 172.21.15.73:0/1589600581 >> v2:172.21.15.150:3300/0 conn(0x7f191c154950 0x7f191c14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 -- 172.21.15.73:0/1589600581 >> 172.21.15.73:0/1589600581 conn(0x7f191c064990 msgr2=0x7f191c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 -- 172.21.15.73:0/1589600581 shutdown_connections 2023-11-28T03:09:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.485+0000 7f19217d0700 1 -- 172.21.15.73:0/1589600581 wait complete. 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.486+0000 7f19217d0700 1 Processor -- start 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 -- start start 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f191c154950 0x7f191c0aff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f191c158570 0x7f191c0b0410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c148600 con 0x7f191c14e530 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0b0900 con 0x7f191c154950 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f19217d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f191c0af350 con 0x7f191c158570 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f191bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f191bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55176/0 (socket says 172.21.15.73:55176) 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.487+0000 7f191bfff700 1 -- 172.21.15.73:0/3413301794 learned_addr learned my addr 172.21.15.73:0/3413301794 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 -- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f191c158570 msgr2=0x7f191c0b0410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f191c158570 0x7f191c0b0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 -- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f191c154950 msgr2=0x7f191c0aff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f191c154950 0x7f191c0aff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 -- 172.21.15.73:0/3413301794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c0092a0 con 0x7f191c14e530 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f191bfff700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f191c064480 tx=0x7f190c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c00b850 con 0x7f191c14e530 2023-11-28T03:09:39.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c00b9b0 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f191c0c3690 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c01b690 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.488+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f191c0c2d60 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.489+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f191c0642c0 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.489+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f190c01ba40 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.489+0000 7f1903fff700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 0x7f1908061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.489+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f190c07b9f0 con 0x7f191c14e530 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.490+0000 7f191b7fe700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 0x7f1908061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.490+0000 7f191b7fe700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 0x7f1908061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1910006fd0 tx=0x7f1910008040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.494+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f190c04ce10 con 0x7f191c14e530 2023-11-28T03:09:39.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.658+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f191c062ee0 con 0x7f191c14e530 2023-11-28T03:09:39.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.662+0000 7f1903fff700 1 -- 172.21.15.73:0/3413301794 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v20) v1 ==== 120+0+0 (secure 0 0 0) 0x7f190c050430 con 0x7f191c14e530 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 msgr2=0x7f1908061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 0x7f1908061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f1910006fd0 tx=0x7f1910008040).stop 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 msgr2=0x7f191c0c5090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 rx=0x7f191c064480 tx=0x7f190c017000).stop 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 shutdown_connections 2023-11-28T03:09:39.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f190805f150 0x7f1908061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f191c158570 0x7f191c0b0410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f191c14e530 0x7f191c0c5090 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 --2- 172.21.15.73:0/3413301794 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f191c154950 0x7f191c0aff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 >> 172.21.15.73:0/3413301794 conn(0x7f191c064990 msgr2=0x7f191c156e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 shutdown_connections 2023-11-28T03:09:39.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.664+0000 7f19217d0700 1 -- 172.21.15.73:0/3413301794 wait complete. 2023-11-28T03:09:39.677 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:09:39.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.792+0000 7f7eb94ca700 1 Processor -- start 2023-11-28T03:09:39.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.792+0000 7f7eb94ca700 1 -- start start 2023-11-28T03:09:39.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb94ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4098d10 0x7f7eb40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb94ca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb4158320 0x7f7eb41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb94ca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb4158cc0 con 0x7f7eb40c6190 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb94ca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb40c72d0 con 0x7f7eb4158320 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb94ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4064570 con 0x7f7eb4098d10 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb4158320 0x7f7eb41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7eb4158320 0x7f7eb41587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55186/0 (socket says 172.21.15.73:55186) 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb37fe700 1 -- 172.21.15.73:0/2296022153 learned_addr learned my addr 172.21.15.73:0/2296022153 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb2ffd700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4098d10 0x7f7eb40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb37fe700 1 -- 172.21.15.73:0/2296022153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4098d10 msgr2=0x7f7eb40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.793+0000 7f7eb37fe700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4098d10 0x7f7eb40c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb37fe700 1 -- 172.21.15.73:0/2296022153 >> v1:172.21.15.150:6789/0 conn(0x7f7eb40c6190 legacy=0x7f7eb4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb37fe700 1 -- 172.21.15.73:0/2296022153 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb4154bc0 con 0x7f7eb4158320 2023-11-28T03:09:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb37fe700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4158320 0x7f7eb41587d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7eac009f20 tx=0x7f7eac009840).ready entity=mon.0 client_cookie=c32e241507d085c4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb27fc700 1 -- 172.21.15.73:0/2296022153 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00e070 con 0x7f7eb4158320 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb27fc700 1 -- 172.21.15.73:0/2296022153 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac00ba40 con 0x7f7eb4158320 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb27fc700 1 -- 172.21.15.73:0/2296022153 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac00bdc0 con 0x7f7eb4158320 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 -- 172.21.15.73:0/2296022153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4158320 msgr2=0x7f7eb41587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4158320 0x7f7eb41587d0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 rx=0x7f7eac009f20 tx=0x7f7eac009840).stop 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 -- 172.21.15.73:0/2296022153 shutdown_connections 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4158320 0x7f7eb41587d0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/2296022153 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4098d10 0x7f7eb40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 -- 172.21.15.73:0/2296022153 >> 172.21.15.73:0/2296022153 conn(0x7f7eb40cb310 msgr2=0x7f7eb40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 -- 172.21.15.73:0/2296022153 shutdown_connections 2023-11-28T03:09:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.794+0000 7f7eb94ca700 1 -- 172.21.15.73:0/2296022153 wait complete. 2023-11-28T03:09:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 Processor -- start 2023-11-28T03:09:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 -- start start 2023-11-28T03:09:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb40c6190 0x7f7eb4156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4158320 0x7f7eb4152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4155240 con 0x7f7eb4098d10 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb40b02b0 con 0x7f7eb40c6190 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb94ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb4168d10 con 0x7f7eb4158320 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55190/0 (socket says 172.21.15.73:55190) 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.796+0000 7f7eb2ffd700 1 -- 172.21.15.73:0/752519172 learned_addr learned my addr 172.21.15.73:0/752519172 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb37fe700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4158320 0x7f7eb4152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eaa7fc700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb40c6190 0x7f7eb4156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 -- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4158320 msgr2=0x7f7eb4152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4158320 0x7f7eb4152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 -- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb40c6190 msgr2=0x7f7eb4156ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb40c6190 0x7f7eb4156ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 -- 172.21.15.73:0/752519172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac009ae0 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb2ffd700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7ea400c850 tx=0x7f7ea4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4010580 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea40106e0 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb40afc60 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea4010a60 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.797+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb40af910 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.798+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb4046de0 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.798+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7ea401b460 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.799+0000 7f7eab7fe700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.799+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f7ea4079eb0 con 0x7f7eb4098d10 2023-11-28T03:09:39.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.799+0000 7f7eaa7fc700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:39.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.799+0000 7f7eaa7fc700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7eb40c5330 tx=0x7f7e9c007d20).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:39.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.803+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7ea404b270 con 0x7f7eb4098d10 2023-11-28T03:09:39.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.967+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7eb40cb150 con 0x7f7eb4098d10 2023-11-28T03:09:39.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.970+0000 7f7eab7fe700 1 -- 172.21.15.73:0/752519172 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7ea404e890 con 0x7f7eb4098d10 2023-11-28T03:09:39.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 msgr2=0x7f7e94061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7eb40c5330 tx=0x7f7e9c007d20).stop 2023-11-28T03:09:39.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 msgr2=0x7f7eb4154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:39.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 rx=0x7f7ea400c850 tx=0x7f7ea4008040).stop 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 shutdown_connections 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7e9405f150 0x7f7e94061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7eb4158320 0x7f7eb4152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7eb4098d10 0x7f7eb4154d50 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 --2- 172.21.15.73:0/752519172 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7eb40c6190 0x7f7eb4156ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:39.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.972+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 >> 172.21.15.73:0/752519172 conn(0x7f7eb40cb310 msgr2=0x7f7eb41536d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:39.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.973+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 shutdown_connections 2023-11-28T03:09:39.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:39.973+0000 7f7eb94ca700 1 -- 172.21.15.73:0/752519172 wait complete. 2023-11-28T03:09:39.986 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-11-28T03:09:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.103+0000 7fab73de6700 1 Processor -- start 2023-11-28T03:09:40.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.103+0000 7fab73de6700 1 -- start start 2023-11-28T03:09:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab73de6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fab6c152450 0x7fab6c0acad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab73de6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fab6c0acfc0 0x7fab6c0c3220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab73de6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab6c0ad450 con 0x7fab6c152070 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab73de6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c06a3f0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab73de6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab6c069070 con 0x7fab6c152450 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fab6c0acfc0 0x7fab6c0c3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab71b82700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fab6c152070 0x7fab6c062850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35406/0 (socket says 172.21.15.73:35406) 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fab6c0acfc0 0x7fab6c0c3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55206/0 (socket says 172.21.15.73:55206) 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab71b82700 1 -- 172.21.15.73:0/44998169 learned_addr learned my addr 172.21.15.73:0/44998169 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 -- 172.21.15.73:0/44998169 >> v2:172.21.15.150:3300/0 conn(0x7fab6c152450 msgr2=0x7fab6c0acad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 --2- 172.21.15.73:0/44998169 >> v2:172.21.15.150:3300/0 conn(0x7fab6c152450 0x7fab6c0acad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 -- 172.21.15.73:0/44998169 >> v1:172.21.15.73:6789/0 conn(0x7fab6c152070 legacy=0x7fab6c062850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab72383700 1 -- 172.21.15.73:0/44998169 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab6c069350 con 0x7fab6c0acfc0 2023-11-28T03:09:40.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.104+0000 7fab70b80700 1 -- 172.21.15.73:0/44998169 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438102901 0 0) 0x7fab6c0ad450 con 0x7fab6c152070 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab72383700 1 --2- 172.21.15.73:0/44998169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c0c3220 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fab6c068a80 tx=0x7fab64009060).ready entity=mon.0 client_cookie=534792a7b2bd76f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab70b80700 1 -- 172.21.15.73:0/44998169 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab640090d0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab70b80700 1 -- 172.21.15.73:0/44998169 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab6400b580 con 0x7fab6c0acfc0 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab70b80700 1 -- 172.21.15.73:0/44998169 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab6400b900 con 0x7fab6c0acfc0 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 -- 172.21.15.73:0/44998169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 msgr2=0x7fab6c0c3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 --2- 172.21.15.73:0/44998169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c0c3220 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 rx=0x7fab6c068a80 tx=0x7fab64009060).stop 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 -- 172.21.15.73:0/44998169 shutdown_connections 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 --2- 172.21.15.73:0/44998169 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c0c3220 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 --2- 172.21.15.73:0/44998169 >> v2:172.21.15.150:3300/0 conn(0x7fab6c152450 0x7fab6c0acad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 -- 172.21.15.73:0/44998169 >> 172.21.15.73:0/44998169 conn(0x7fab6c0c7760 msgr2=0x7fab6c0c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 -- 172.21.15.73:0/44998169 shutdown_connections 2023-11-28T03:09:40.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.105+0000 7fab73de6700 1 -- 172.21.15.73:0/44998169 wait complete. 2023-11-28T03:09:40.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.106+0000 7fab73de6700 1 Processor -- start 2023-11-28T03:09:40.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.106+0000 7fab73de6700 1 -- start start 2023-11-28T03:09:40.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.111 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6c152070 0x7fab6c13d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab6c152450 0x7fab6c154910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c166810 con 0x7fab6c0acfc0 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c165e90 con 0x7fab6c152070 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab73de6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab6c154510 con 0x7fab6c152450 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab71b82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab71b82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55220/0 (socket says 172.21.15.73:55220) 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab71b82700 1 -- 172.21.15.73:0/3021417251 learned_addr learned my addr 172.21.15.73:0/3021417251 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab71b82700 1 -- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab6c152450 msgr2=0x7fab6c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.107+0000 7fab71b82700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab6c152450 0x7fab6c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab71b82700 1 -- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6c152070 msgr2=0x7fab6c13d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab71b82700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6c152070 0x7fab6c13d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab71b82700 1 -- 172.21.15.73:0/3021417251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab640092a0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab71b82700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fab6800c960 tx=0x7fab68001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68005d30 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab68005e90 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab6c14a7f0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab68007320 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.108+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab6c165840 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.109+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab6c06c860 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.109+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fab68007750 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.110+0000 7fab61ffb700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 0x7fab500615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.110+0000 7fab71381700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 0x7fab500615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.110+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7fab6807a2c0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.110+0000 7fab71381700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 0x7fab500615e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fab6c155340 tx=0x7fab5c006c30).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:40.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.114+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fab6804b6e0 con 0x7fab6c0acfc0 2023-11-28T03:09:40.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:40.341+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7fab6c06b820 con 0x7fab6c0acfc0 2023-11-28T03:09:41.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.344+0000 7fab61ffb700 1 -- 172.21.15.73:0/3021417251 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v624) v1 ==== 183+0+0 (secure 0 0 0) 0x7fab6804ed00 con 0x7fab6c0acfc0 2023-11-28T03:09:41.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 msgr2=0x7fab500615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 0x7fab500615e0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7fab6c155340 tx=0x7fab5c006c30).stop 2023-11-28T03:09:41.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 msgr2=0x7fab6c166320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 rx=0x7fab6800c960 tx=0x7fab68001630).stop 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 shutdown_connections 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fab5005f120 0x7fab500615e0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fab6c152450 0x7fab6c154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fab6c0acfc0 0x7fab6c166320 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 --2- 172.21.15.73:0/3021417251 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fab6c152070 0x7fab6c13d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 >> 172.21.15.73:0/3021417251 conn(0x7fab6c0c7760 msgr2=0x7fab6c0c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 shutdown_connections 2023-11-28T03:09:41.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.347+0000 7fab73de6700 1 -- 172.21.15.73:0/3021417251 wait complete. 2023-11-28T03:09:41.361 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-11-28T03:09:41.362 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-11-28T03:09:41.362 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:09:41.362 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-11-28T03:09:41.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 Processor -- start 2023-11-28T03:09:41.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 -- start start 2023-11-28T03:09:41.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d2c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2d2c095e80 0x7f2d2c096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d2c096800 con 0x7f2d2c1544d0 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d2c067650 con 0x7f2d2c095e80 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.478+0000 7f2d324c8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0683d0 con 0x7f2d2c064e20 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2d2c1544d0 0x7f2d2c066f40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35418/0 (socket says 172.21.15.73:35418) 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2bfff700 1 -- 172.21.15.73:0/672756078 learned_addr learned my addr 172.21.15.73:0/672756078 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d30a65700 1 --2- 172.21.15.73:0/672756078 >> v2:172.21.15.150:3300/0 conn(0x7f2d2c095e80 0x7f2d2c096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d2c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3710565744 0 0) 0x7f2d2c096800 con 0x7f2d2c1544d0 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d140033f0 con 0x7f2d2c1544d0 2023-11-28T03:09:41.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2771705295 0 0) 0x7f2d140033f0 con 0x7f2d2c1544d0 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 msgr2=0x7f2d2c155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d2c155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 >> v2:172.21.15.150:3300/0 conn(0x7f2d2c095e80 msgr2=0x7f2d2c096310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 --2- 172.21.15.73:0/672756078 >> v2:172.21.15.150:3300/0 conn(0x7f2d2c095e80 0x7f2d2c096310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c067850 con 0x7f2d2c1544d0 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2d1c0015f0 con 0x7f2d2c1544d0 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 >> v1:172.21.15.73:6789/0 conn(0x7f2d2c1544d0 legacy=0x7f2d2c066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d140038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d14003e10 0x7f2d14004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d14004b80 0x7f2d14005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c067650 con 0x7f2d14003e10 2023-11-28T03:09:41.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0683d0 con 0x7f2d14004b80 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d1400b8e0 con 0x7f2d2c064e20 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d14003e10 0x7f2d14004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2bfff700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d140038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d30a65700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d14004b80 0x7f2d14005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 msgr2=0x7f2d140038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d2c064e20 0x7f2d140038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d14004b80 msgr2=0x7f2d14005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d14004b80 0x7f2d14005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.479+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/672756078 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d140033f0 con 0x7f2d14003e10 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d14003e10 0x7f2d14004690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f2d20009ab0 tx=0x7f2d20009060).ready entity=mon.0 client_cookie=a49f0d8cf9eb449 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20009430 con 0x7f2d14003e10 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d2000e640 con 0x7f2d14003e10 2023-11-28T03:09:41.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d2affd700 1 -- 172.21.15.73:0/672756078 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000e9c0 con 0x7f2d14003e10 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 -- 172.21.15.73:0/672756078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d14003e10 msgr2=0x7f2d14004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 --2- 172.21.15.73:0/672756078 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d14003e10 0x7f2d14004690 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 rx=0x7f2d20009ab0 tx=0x7f2d20009060).stop 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d30a65700 1 -- 172.21.15.73:0/672756078 reap_dead start 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 -- 172.21.15.73:0/672756078 shutdown_connections 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 -- 172.21.15.73:0/672756078 >> 172.21.15.73:0/672756078 conn(0x7f2d2c0631d0 msgr2=0x7f2d2c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 -- 172.21.15.73:0/672756078 shutdown_connections 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.480+0000 7f2d324c8700 1 -- 172.21.15.73:0/672756078 wait complete. 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.481+0000 7f2d324c8700 1 Processor -- start 2023-11-28T03:09:41.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 -- start start 2023-11-28T03:09:41.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d14003e10 0x7f2d2c153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c063e10 0x7f2d2c064290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c064780 con 0x7f2d2c0aa470 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0ac160 con 0x7f2d2c063e10 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d324c8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d2c0aac20 con 0x7f2d14003e10 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.482+0000 7f2d2b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55240/0 (socket says 172.21.15.73:55240) 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/687735047 learned_addr learned my addr 172.21.15.73:0/687735047 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2bfff700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d14003e10 0x7f2d2c153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d30a65700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c063e10 0x7f2d2c064290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d14003e10 msgr2=0x7f2d2c153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d14003e10 0x7f2d2c153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c063e10 msgr2=0x7f2d2c064290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c063e10 0x7f2d2c064290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 -- 172.21.15.73:0/687735047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d2c0ac400 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d2b7fe700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f2d2000e4b0 tx=0x7f2d2001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d20020070 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d20009430 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d2c0ac6e0 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d2000e7b0 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.483+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d2c0aa8f0 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.484+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d2c06c860 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.484+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2d200123f0 con 0x7f2d2c0aa470 2023-11-28T03:09:41.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.485+0000 7f2d13fff700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 0x7f2d18061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:41.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.485+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f2d2007b0e0 con 0x7f2d2c0aa470 2023-11-28T03:09:41.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.485+0000 7f2d2bfff700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 0x7f2d18061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:41.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.485+0000 7f2d2bfff700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 0x7f2d18061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2d1c001860 tx=0x7f2d1c009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:41.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.489+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2d2004ff80 con 0x7f2d2c0aa470 2023-11-28T03:09:41.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:41.718+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f2d2c067b30 con 0x7f2d2c0aa470 2023-11-28T03:09:42.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.349+0000 7f2d13fff700 1 -- 172.21.15.73:0/687735047 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v625) v1 ==== 151+0+0 (secure 0 0 0) 0x7f2d20029070 con 0x7f2d2c0aa470 2023-11-28T03:09:42.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 msgr2=0x7f2d18061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 0x7f2d18061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2d1c001860 tx=0x7f2d1c009040).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 msgr2=0x7f2d2c14ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 rx=0x7f2d2000e4b0 tx=0x7f2d2001a000).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 shutdown_connections 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2d1805f150 0x7f2d18061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d2c063e10 0x7f2d2c064290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d14003e10 0x7f2d2c153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 --2- 172.21.15.73:0/687735047 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d2c0aa470 0x7f2d2c14ebf0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 >> 172.21.15.73:0/687735047 conn(0x7f2d2c0631d0 msgr2=0x7f2d2c0a7e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 shutdown_connections 2023-11-28T03:09:42.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.351+0000 7f2d324c8700 1 -- 172.21.15.73:0/687735047 wait complete. 2023-11-28T03:09:42.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:09:42.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.481+0000 7fd9ab6a2700 1 Processor -- start 2023-11-28T03:09:42.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.481+0000 7fd9ab6a2700 1 -- start start 2023-11-28T03:09:42.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9ab6a2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9a4098d10 0x7fd9a40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9ab6a2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd9a4158320 0x7fd9a41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9ab6a2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9a4158cc0 con 0x7fd9a40c6190 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9ab6a2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a40c72d0 con 0x7fd9a4098d10 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9ab6a2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9a4064570 con 0x7fd9a4158320 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a943e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9a4098d10 0x7fd9a40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a943e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd9a4098d10 0x7fd9a40c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55256/0 (socket says 172.21.15.73:55256) 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a943e700 1 -- 172.21.15.73:0/3650640665 learned_addr learned my addr 172.21.15.73:0/3650640665 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a9c3f700 1 --2- 172.21.15.73:0/3650640665 >> v2:172.21.15.150:3300/0 conn(0x7fd9a4158320 0x7fd9a41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a943e700 1 -- 172.21.15.73:0/3650640665 >> v2:172.21.15.150:3300/0 conn(0x7fd9a4158320 msgr2=0x7fd9a41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.482+0000 7fd9a943e700 1 --2- 172.21.15.73:0/3650640665 >> v2:172.21.15.150:3300/0 conn(0x7fd9a4158320 0x7fd9a41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9a943e700 1 -- 172.21.15.73:0/3650640665 >> v1:172.21.15.150:6789/0 conn(0x7fd9a40c6190 legacy=0x7fd9a4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9a943e700 1 -- 172.21.15.73:0/3650640665 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a4154bc0 con 0x7fd9a4098d10 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9a943e700 1 --2- 172.21.15.73:0/3650640665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a4098d10 0x7fd9a40c7760 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fd9a40657a0 tx=0x7fd9a00090f0).ready entity=mon.0 client_cookie=4ecee5de2770639b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd99bfff700 1 -- 172.21.15.73:0/3650640665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000b600 con 0x7fd9a4098d10 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd99bfff700 1 -- 172.21.15.73:0/3650640665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9a000b760 con 0x7fd9a4098d10 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd99bfff700 1 -- 172.21.15.73:0/3650640665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9a000bae0 con 0x7fd9a4098d10 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3650640665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a4098d10 msgr2=0x7fd9a40c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3650640665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a4098d10 0x7fd9a40c7760 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 rx=0x7fd9a40657a0 tx=0x7fd9a00090f0).stop 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3650640665 shutdown_connections 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3650640665 >> v2:172.21.15.150:3300/0 conn(0x7fd9a4158320 0x7fd9a41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3650640665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a4098d10 0x7fd9a40c7760 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3650640665 >> 172.21.15.73:0/3650640665 conn(0x7fd9a40cb310 msgr2=0x7fd9a40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3650640665 shutdown_connections 2023-11-28T03:09:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.483+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3650640665 wait complete. 2023-11-28T03:09:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.484+0000 7fd9ab6a2700 1 Processor -- start 2023-11-28T03:09:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.484+0000 7fd9ab6a2700 1 -- start start 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a4098d10 0x7fd9a4155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a4158320 0x7fd9a40a9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a4156280 con 0x7fd9a40c6190 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40a9fc0 con 0x7fd9a4158320 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9ab6a2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a40c38e0 con 0x7fd9a4098d10 2023-11-28T03:09:42.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9a8c3d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9a8c3d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55260/0 (socket says 172.21.15.73:55260) 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.485+0000 7fd9a8c3d700 1 -- 172.21.15.73:0/3353372024 learned_addr learned my addr 172.21.15.73:0/3353372024 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a9c3f700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a4158320 0x7fd9a40a9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a943e700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a4098d10 0x7fd9a4155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 -- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a4098d10 msgr2=0x7fd9a4155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a4098d10 0x7fd9a4155090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 -- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a4158320 msgr2=0x7fd9a40a9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a4158320 0x7fd9a40a9ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 -- 172.21.15.73:0/3353372024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0009370 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9a8c3d700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fd99400c4c0 tx=0x7fd994007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994007020 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd994007180 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd994007520 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9a41559e0 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.486+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9a4155fa0 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.487+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9a40c36b0 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.487+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd9940068b0 con 0x7fd9a40c6190 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.488+0000 7fd9997fa700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 0x7fd988061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.488+0000 7fd9a943e700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 0x7fd988061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.488+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7fd9940792e0 con 0x7fd9a40c6190 2023-11-28T03:09:42.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.488+0000 7fd9a943e700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 0x7fd988061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd9a0009130 tx=0x7fd9a0017000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.492+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd994047af0 con 0x7fd9a40c6190 2023-11-28T03:09:42.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.655+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fd9a4158cc0 con 0x7fd9a40c6190 2023-11-28T03:09:42.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.659+0000 7fd9997fa700 1 -- 172.21.15.73:0/3353372024 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fd9a4158cc0 con 0x7fd9a40c6190 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 msgr2=0x7fd988061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 0x7fd988061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fd9a0009130 tx=0x7fd9a0017000).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 msgr2=0x7fd9a40a95e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 rx=0x7fd99400c4c0 tx=0x7fd994007dd0).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 shutdown_connections 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd98805f150 0x7fd988061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd9a4158320 0x7fd9a40a9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd9a4098d10 0x7fd9a4155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 --2- 172.21.15.73:0/3353372024 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd9a40c6190 0x7fd9a40a95e0 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 >> 172.21.15.73:0/3353372024 conn(0x7fd9a40cb310 msgr2=0x7fd9a414c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:42.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 shutdown_connections 2023-11-28T03:09:42.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.661+0000 7fd9ab6a2700 1 -- 172.21.15.73:0/3353372024 wait complete. 2023-11-28T03:09:42.674 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:09:42.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:09:42.709 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:09:42.754 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:09:42.755 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:09:42.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:42.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.866+0000 7f0d19c72700 1 Processor -- start 2023-11-28T03:09:42.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.866+0000 7f0d19c72700 1 -- start start 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d19c72700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d140caab0 0x7f0d140aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d19c72700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d140ca180 0x7f0d1415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d19c72700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140a9a90 con 0x7f0d140cae90 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d19c72700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d14154ce0 con 0x7f0d140ca180 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d19c72700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d140a8e80 con 0x7f0d140caab0 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d13fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d140ca180 0x7f0d1415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d12ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0d140cae90 0x7f0d140c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35430/0 (socket says 172.21.15.73:35430) 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d13fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d140ca180 0x7f0d1415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55272/0 (socket says 172.21.15.73:55272) 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d12ffd700 1 -- 172.21.15.73:0/1710749085 learned_addr learned my addr 172.21.15.73:0/1710749085 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.867+0000 7f0d137fe700 1 --2- 172.21.15.73:0/1710749085 >> v2:172.21.15.150:3300/0 conn(0x7f0d140caab0 0x7f0d140aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d13fff700 1 -- 172.21.15.73:0/1710749085 >> v2:172.21.15.150:3300/0 conn(0x7f0d140caab0 msgr2=0x7f0d140aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d13fff700 1 --2- 172.21.15.73:0/1710749085 >> v2:172.21.15.150:3300/0 conn(0x7f0d140caab0 0x7f0d140aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d13fff700 1 -- 172.21.15.73:0/1710749085 >> v1:172.21.15.73:6789/0 conn(0x7f0d140cae90 legacy=0x7f0d140c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d13fff700 1 -- 172.21.15.73:0/1710749085 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140a9160 con 0x7f0d140ca180 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d127fc700 1 -- 172.21.15.73:0/1710749085 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3840118433 0 0) 0x7f0d140a9a90 con 0x7f0d140cae90 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d13fff700 1 --2- 172.21.15.73:0/1710749085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140ca180 0x7f0d1415e3c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f0d140848c0 tx=0x7f0d0c009060).ready entity=mon.0 client_cookie=7f8b0ec206c196cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d127fc700 1 -- 172.21.15.73:0/1710749085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00b580 con 0x7f0d140ca180 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d127fc700 1 -- 172.21.15.73:0/1710749085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00b6e0 con 0x7f0d140ca180 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d127fc700 1 -- 172.21.15.73:0/1710749085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00ba60 con 0x7f0d140ca180 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 -- 172.21.15.73:0/1710749085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140ca180 msgr2=0x7f0d1415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 --2- 172.21.15.73:0/1710749085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140ca180 0x7f0d1415e3c0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 rx=0x7f0d140848c0 tx=0x7f0d0c009060).stop 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 -- 172.21.15.73:0/1710749085 shutdown_connections 2023-11-28T03:09:42.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 --2- 172.21.15.73:0/1710749085 >> v2:172.21.15.150:3300/0 conn(0x7f0d140caab0 0x7f0d140aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 --2- 172.21.15.73:0/1710749085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140ca180 0x7f0d1415e3c0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 -- 172.21.15.73:0/1710749085 >> 172.21.15.73:0/1710749085 conn(0x7f0d14094920 msgr2=0x7f0d1406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:42.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 -- 172.21.15.73:0/1710749085 shutdown_connections 2023-11-28T03:09:42.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.868+0000 7f0d19c72700 1 -- 172.21.15.73:0/1710749085 wait complete. 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 Processor -- start 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 -- start start 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140ca180 0x7f0d140aa440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140caab0 0x7f0d140c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14156a40 con 0x7f0d140cae90 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140c7030 con 0x7f0d140caab0 2023-11-28T03:09:42.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.870+0000 7f0d19c72700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140aaa20 con 0x7f0d140ca180 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55274/0 (socket says 172.21.15.73:55274) 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 -- 172.21.15.73:0/2742435477 learned_addr learned my addr 172.21.15.73:0/2742435477 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d137fe700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140ca180 0x7f0d140aa440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d12ffd700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140caab0 0x7f0d140c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 -- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140ca180 msgr2=0x7f0d140aa440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140ca180 0x7f0d140aa440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 -- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140caab0 msgr2=0x7f0d140c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140caab0 0x7f0d140c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 -- 172.21.15.73:0/2742435477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c0092a0 con 0x7f0d140cae90 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d13fff700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f0d14083c30 tx=0x7f0d0c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c00b850 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c00b9b0 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d14156c90 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c01b660 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.871+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14157250 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.872+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d1406c860 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.872+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0d0c01bc60 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.873+0000 7f0cfb7fe700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.873+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(117..117 src has 1..117) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f0d0c07bdf0 con 0x7f0d140cae90 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.873+0000 7f0d137fe700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:42.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.873+0000 7f0d137fe700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0d04009fd0 tx=0x7f0d040098a0).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:42.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:42.877+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d0c04d250 con 0x7f0d140cae90 2023-11-28T03:09:43.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.080+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0d140a9440 con 0x7f0d140cae90 2023-11-28T03:09:43.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.081+0000 7f0cfb7fe700 1 -- 172.21.15.73:0/2742435477 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 42 v42) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f0d0c029020 con 0x7f0d140cae90 2023-11-28T03:09:43.082 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:43.082 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":42,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5963,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/498898579","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":498898579},{"type":"v1","addr":"172.21.15.73:6837","nonce":498898579}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5969,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/2596390498","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2596390498},{"type":"v1","addr":"172.21.15.150:6833","nonce":2596390498}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":5972,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3311386652","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":3311386652},{"type":"v1","addr":"172.21.15.150:6835","nonce":3311386652}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":37},{"gid":7497,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/1923819893","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":1923819893},{"type":"v1","addr":"172.21.15.73:6836","nonce":1923819893}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[]} 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 msgr2=0x7f0d00061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f0d04009fd0 tx=0x7f0d040098a0).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 msgr2=0x7f0d140c7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 rx=0x7f0d14083c30 tx=0x7f0d0c017000).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 shutdown_connections 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d0005f150 0x7f0d00061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d140cae90 0x7f0d140c7260 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d140ca180 0x7f0d140aa440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 --2- 172.21.15.73:0/2742435477 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d140caab0 0x7f0d140c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.082+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 >> 172.21.15.73:0/2742435477 conn(0x7f0d14094920 msgr2=0x7f0d140637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:43.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.083+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 shutdown_connections 2023-11-28T03:09:43.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.083+0000 7f0d19c72700 1 -- 172.21.15.73:0/2742435477 wait complete. 2023-11-28T03:09:43.085 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 42 2023-11-28T03:09:43.096 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:09:43.096 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:09:43.096 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:09:43.096 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:09:43.096 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:09:43.097 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:09:43.097 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:09:43.097 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:09:43.097 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:09:43.097 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:09:43.097 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:09:43.097 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:09:43.098 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:09:43.098 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:09:43.098 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:09:43.098 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:09:43.105 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:09:43.105 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:09:43.106 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:09:43.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:09:43.108 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:09:43.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:09:43.109 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:09:43.109 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:09:43.147 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:09:43.148 DEBUG:teuthology.parallel:result is None 2023-11-28T03:09:43.149 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:09:43.149 DEBUG:teuthology.parallel:result is None 2023-11-28T03:09:43.152 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:09:43.152 DEBUG:teuthology.parallel:result is None 2023-11-28T03:09:43.153 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:09:43.153 DEBUG:teuthology.parallel:result is None 2023-11-28T03:09:43.153 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:09:43.220 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:09:43.220 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:09:43.237 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:09:43.237 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:09:43.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.275+0000 7f00ae2ca700 1 Processor -- start 2023-11-28T03:09:43.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.275+0000 7f00ae2ca700 1 -- start start 2023-11-28T03:09:43.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00ae2ca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f00a80c51a0 0x7f00a8153390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00ae2ca700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a8155f20 con 0x7f00a80c5ac0 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00ae2ca700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00a8156200 con 0x7f00a80c5ea0 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00ae2ca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00a8153990 con 0x7f00a80c51a0 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f00a80c51a0 0x7f00a8153390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f00a80c5ac0 0x7f00a8156690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35446/0 (socket says 172.21.15.73:35446) 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a7fff700 1 -- 172.21.15.73:0/807030834 learned_addr learned my addr 172.21.15.73:0/807030834 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a77fe700 1 -- 172.21.15.73:0/807030834 >> v1:172.21.15.150:6789/0 conn(0x7f00a80c5ea0 legacy=0x7f00a8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:43.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a77fe700 1 -- 172.21.15.73:0/807030834 >> v1:172.21.15.73:6789/0 conn(0x7f00a80c5ac0 legacy=0x7f00a8156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a6ffd700 1 -- 172.21.15.73:0/807030834 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405206645 0 0) 0x7f00a8155f20 con 0x7f00a80c5ac0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.276+0000 7f00a77fe700 1 -- 172.21.15.73:0/807030834 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00a80bff70 con 0x7f00a80c51a0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00a77fe700 1 --2- 172.21.15.73:0/807030834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c51a0 0x7f00a8153390 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f009c009760 tx=0x7f009c009060).ready entity=mon.0 client_cookie=31ed8582f9fd64a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00a6ffd700 1 -- 172.21.15.73:0/807030834 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0090d0 con 0x7f00a80c51a0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00a6ffd700 1 -- 172.21.15.73:0/807030834 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c00b580 con 0x7f00a80c51a0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00a6ffd700 1 -- 172.21.15.73:0/807030834 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c00b900 con 0x7f00a80c51a0 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 -- 172.21.15.73:0/807030834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c51a0 msgr2=0x7f00a8153390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/807030834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c51a0 0x7f00a8153390 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 rx=0x7f009c009760 tx=0x7f009c009060).stop 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 -- 172.21.15.73:0/807030834 shutdown_connections 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/807030834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c51a0 0x7f00a8153390 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 -- 172.21.15.73:0/807030834 >> 172.21.15.73:0/807030834 conn(0x7f00a8062700 msgr2=0x7f00a8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:43.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 -- 172.21.15.73:0/807030834 shutdown_connections 2023-11-28T03:09:43.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.277+0000 7f00ae2ca700 1 -- 172.21.15.73:0/807030834 wait complete. 2023-11-28T03:09:43.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.278+0000 7f00ae2ca700 1 Processor -- start 2023-11-28T03:09:43.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 -- start start 2023-11-28T03:09:43.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00a80c51a0 0x7f00a80a5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:43.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:43.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00a80c5ea0 0x7f00a80ae010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80c1b50 con 0x7f00a80c5ac0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80ae500 con 0x7f00a80c5ea0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00ae2ca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00a80c0bf0 con 0x7f00a80c51a0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00a77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.279+0000 7f00a77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:55308/0 (socket says 172.21.15.73:55308) 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 -- 172.21.15.73:0/3014748560 learned_addr learned my addr 172.21.15.73:0/3014748560 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a7fff700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00a80c51a0 0x7f00a80a5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00ac867700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00a80c5ea0 0x7f00a80ae010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 -- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00a80c51a0 msgr2=0x7f00a80a5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00a80c51a0 0x7f00a80a5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 -- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00a80c5ea0 msgr2=0x7f00a80ae010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:43.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00a80c5ea0 0x7f00a80ae010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 -- 172.21.15.73:0/3014748560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0092a0 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00a77fe700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f009c009eb0 tx=0x7f009c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c006950 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c011b70 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c01b410 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00a80c0ed0 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.280+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00a80c1490 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.281+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00a80c1c90 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.281+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f009c01b7c0 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.282+0000 7f008ffff700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 0x7f00940615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.282+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f009c07baf0 con 0x7f00a80c5ac0 2023-11-28T03:09:43.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.282+0000 7f00a7fff700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 0x7f00940615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:43.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.282+0000 7f00a7fff700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 0x7f00940615e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f00980011e0 tx=0x7f0098007bb0).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:43.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.286+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f009c04cf10 con 0x7f00a80c5ac0 2023-11-28T03:09:43.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:43.465+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f00a806c860 con 0x7f00a80c5ac0 2023-11-28T03:09:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.356+0000 7f008ffff700 1 -- 172.21.15.73:0/3014748560 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v122) v1 ==== 97+0+0 (secure 0 0 0) 0x7f009c050530 con 0x7f00a80c5ac0 2023-11-28T03:09:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 msgr2=0x7f00940615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 0x7f00940615e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f00980011e0 tx=0x7f0098007bb0).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 msgr2=0x7f00a80adb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 rx=0x7f009c009eb0 tx=0x7f009c017000).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 shutdown_connections 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f009405f120 0x7f00940615e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00a80c5ea0 0x7f00a80ae010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00a80c51a0 0x7f00a80a5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 --2- 172.21.15.73:0/3014748560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00a80c5ac0 0x7f00a80adb20 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 >> 172.21.15.73:0/3014748560 conn(0x7f00a8062700 msgr2=0x7f00a80c39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 shutdown_connections 2023-11-28T03:09:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.358+0000 7f00ae2ca700 1 -- 172.21.15.73:0/3014748560 wait complete. 2023-11-28T03:09:44.361 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:09:44.372 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:09:44.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.487+0000 7f712566b700 1 Processor -- start 2023-11-28T03:09:44.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.487+0000 7f712566b700 1 -- start start 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f712566b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7120154120 0x7f71200be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f712566b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7120154500 0x7f71200bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f712566b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71200bfc00 con 0x7f71200c3fd0 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f712566b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200beff0 con 0x7f7120154500 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f712566b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71200bf2d0 con 0x7f7120154120 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f711f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7120154500 0x7f71200bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f711f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7120154500 0x7f71200bea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53294/0 (socket says 172.21.15.73:53294) 2023-11-28T03:09:44.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.488+0000 7f711f7fe700 1 -- 172.21.15.73:0/3567119565 learned_addr learned my addr 172.21.15.73:0/3567119565 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711f7fe700 1 -- 172.21.15.73:0/3567119565 >> v2:172.21.15.150:3300/0 conn(0x7f7120154120 msgr2=0x7f71200be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711f7fe700 1 --2- 172.21.15.73:0/3567119565 >> v2:172.21.15.150:3300/0 conn(0x7f7120154120 0x7f71200be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711f7fe700 1 -- 172.21.15.73:0/3567119565 >> v1:172.21.15.150:6789/0 conn(0x7f71200c3fd0 legacy=0x7f71200c43f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711f7fe700 1 -- 172.21.15.73:0/3567119565 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71200bf5b0 con 0x7f7120154500 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711f7fe700 1 --2- 172.21.15.73:0/3567119565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120154500 0x7f71200bea00 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f712006a550 tx=0x7f71140090f0).ready entity=mon.0 client_cookie=41fb753c3faee19f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711effd700 1 -- 172.21.15.73:0/3567119565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400b5a0 con 0x7f7120154500 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711effd700 1 -- 172.21.15.73:0/3567119565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711400b700 con 0x7f7120154500 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f711effd700 1 -- 172.21.15.73:0/3567119565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711400ba80 con 0x7f7120154500 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f712566b700 1 -- 172.21.15.73:0/3567119565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120154500 msgr2=0x7f71200bea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.489+0000 7f712566b700 1 --2- 172.21.15.73:0/3567119565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120154500 0x7f71200bea00 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 rx=0x7f712006a550 tx=0x7f71140090f0).stop 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 -- 172.21.15.73:0/3567119565 shutdown_connections 2023-11-28T03:09:44.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 --2- 172.21.15.73:0/3567119565 >> v2:172.21.15.150:3300/0 conn(0x7f7120154120 0x7f71200be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 --2- 172.21.15.73:0/3567119565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7120154500 0x7f71200bea00 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 -- 172.21.15.73:0/3567119565 >> 172.21.15.73:0/3567119565 conn(0x7f7120083000 msgr2=0x7f7120092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 -- 172.21.15.73:0/3567119565 shutdown_connections 2023-11-28T03:09:44.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.490+0000 7f712566b700 1 -- 172.21.15.73:0/3567119565 wait complete. 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.491+0000 7f712566b700 1 Processor -- start 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.491+0000 7f712566b700 1 -- start start 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 0x7f71201555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 0x7f7120155ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120155fc0 con 0x7f71200c3fd0 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7120157a30 con 0x7f7120154500 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7120156e20 con 0x7f7120154120 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53298/0 (socket says 172.21.15.73:53298) 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 -- 172.21.15.73:0/2475550237 learned_addr learned my addr 172.21.15.73:0/2475550237 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f7124e6a700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 0x7f7120155ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711f7fe700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 0x7f71201555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 -- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 msgr2=0x7f71201555e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 0x7f71201555e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 -- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 msgr2=0x7f7120155ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 0x7f7120155ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 -- 172.21.15.73:0/2475550237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7114009370 con 0x7f71200c3fd0 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f7124e6a700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 0x7f7120155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711f7fe700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 0x7f71201555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f711ffff700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f711000b5b0 tx=0x7f71100090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000d4a0 con 0x7f71200c3fd0 2023-11-28T03:09:44.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f711000d600 con 0x7f71200c3fd0 2023-11-28T03:09:44.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7120157100 con 0x7f71200c3fd0 2023-11-28T03:09:44.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f711000d9c0 con 0x7f71200c3fd0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.492+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71201576c0 con 0x7f71200c3fd0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.493+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f712006c7c0 con 0x7f71200c3fd0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.493+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7110012410 con 0x7f71200c3fd0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.494+0000 7f7107fff700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 0x7f710c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.494+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7110079d80 con 0x7f71200c3fd0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.494+0000 7f711f7fe700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 0x7f710c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.494+0000 7f711f7fe700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 0x7f710c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7114009130 tx=0x7f7114015000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.498+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f711004b340 con 0x7f71200c3fd0 2023-11-28T03:09:44.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.726+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f71200bf890 con 0x7f71200c3fd0 2023-11-28T03:09:44.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.727+0000 7f7107fff700 1 -- 172.21.15.73:0/2475550237 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f711004e960 con 0x7f71200c3fd0 2023-11-28T03:09:44.728 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:44.728 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.730 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:09:44.731 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.734 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:09:44.735 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:09:44.738 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:09:44.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:09:44.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.741 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:09:44.742 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:09:44.743 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:09:44.744 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:09:44.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.744 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:44.744 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:09:44.744 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:09:44.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 msgr2=0x7f710c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 0x7f710c061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f7114009130 tx=0x7f7114015000).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 msgr2=0x7f7120152e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 rx=0x7f711000b5b0 tx=0x7f71100090f0).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 shutdown_connections 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f710c05f150 0x7f710c061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7120154500 0x7f7120155ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71200c3fd0 0x7f7120152e10 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 --2- 172.21.15.73:0/2475550237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7120154120 0x7f71201555e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.728+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 >> 172.21.15.73:0/2475550237 conn(0x7f7120083000 msgr2=0x7f7120063810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.729+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 shutdown_connections 2023-11-28T03:09:44.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.729+0000 7f712566b700 1 -- 172.21.15.73:0/2475550237 wait complete. 2023-11-28T03:09:44.747 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:44.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.858+0000 7f40dd3f5700 1 Processor -- start 2023-11-28T03:09:44.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.858+0000 7f40dd3f5700 1 -- start start 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40dd3f5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d80697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40dd3f5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40d8062e10 0x7f40d80661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40dd3f5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d8065760 con 0x7f40d8069ce0 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40dd3f5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d806a160 con 0x7f40d8062e10 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40dd3f5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80c3250 con 0x7f40d81542d0 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40d77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40d8062e10 0x7f40d80661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40d77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f40d8062e10 0x7f40d80661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53304/0 (socket says 172.21.15.73:53304) 2023-11-28T03:09:44.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.859+0000 7f40d77fe700 1 -- 172.21.15.73:0/2731916598 learned_addr learned my addr 172.21.15.73:0/2731916598 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d6ffd700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d80697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d77fe700 1 -- 172.21.15.73:0/2731916598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 msgr2=0x7f40d80697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d77fe700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d80697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d77fe700 1 -- 172.21.15.73:0/2731916598 >> v1:172.21.15.150:6789/0 conn(0x7f40d8069ce0 legacy=0x7f40d8062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d77fe700 1 -- 172.21.15.73:0/2731916598 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d806a340 con 0x7f40d8062e10 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d77fe700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d80661b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f40d0009f20 tx=0x7f40d0009840).ready entity=mon.0 client_cookie=28a942825df8fd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d67fc700 1 -- 172.21.15.73:0/2731916598 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000e070 con 0x7f40d8062e10 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d67fc700 1 -- 172.21.15.73:0/2731916598 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d000ba80 con 0x7f40d8062e10 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40d67fc700 1 -- 172.21.15.73:0/2731916598 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d000be00 con 0x7f40d8062e10 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 -- 172.21.15.73:0/2731916598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 msgr2=0x7f40d80661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d80661b0 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 rx=0x7f40d0009f20 tx=0x7f40d0009840).stop 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 -- 172.21.15.73:0/2731916598 shutdown_connections 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d80661b0 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/2731916598 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d80697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 -- 172.21.15.73:0/2731916598 >> 172.21.15.73:0/2731916598 conn(0x7f40d80c7fd0 msgr2=0x7f40d80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.860+0000 7f40dd3f5700 1 -- 172.21.15.73:0/2731916598 shutdown_connections 2023-11-28T03:09:44.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.861+0000 7f40dd3f5700 1 -- 172.21.15.73:0/2731916598 wait complete. 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 Processor -- start 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 -- start start 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 0x7f40d8167940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d8167e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d8168320 con 0x7f40d8062e10 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80a8620 con 0x7f40d8069ce0 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.862+0000 7f40dd3f5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d80a88e0 con 0x7f40d81542d0 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53312/0 (socket says 172.21.15.73:53312) 2023-11-28T03:09:44.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 -- 172.21.15.73:0/3632204858 learned_addr learned my addr 172.21.15.73:0/3632204858 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d77fe700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d8167e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40ce7fc700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 0x7f40d8167940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 -- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 msgr2=0x7f40d8167e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d8167e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 -- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 msgr2=0x7f40d8167940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 0x7f40d8167940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 -- 172.21.15.73:0/3632204858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d0009ae0 con 0x7f40d8062e10 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40ce7fc700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 0x7f40d8167940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40d6ffd700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f40c800d860 tx=0x7f40c8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011590 con 0x7f40d8062e10 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c80116f0 con 0x7f40d8062e10 2023-11-28T03:09:44.868 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d80a8bc0 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c8011a70 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.863+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d814dfa0 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.864+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40d806a620 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.864+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f40c8012420 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.865+0000 7f40cf7fe700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 0x7f40b8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.865+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f40c8079880 con 0x7f40d8062e10 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.865+0000 7f40ce7fc700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 0x7f40b8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.865+0000 7f40ce7fc700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 0x7f40b8061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f40d814e320 tx=0x7f40c0009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:44.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:44.869+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f40c804ae40 con 0x7f40d8062e10 2023-11-28T03:09:45.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.032+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f40d805f660 con 0x7f40d8062e10 2023-11-28T03:09:45.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.033+0000 7f40cf7fe700 1 -- 172.21.15.73:0/3632204858 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f40c8011e60 con 0x7f40d8062e10 2023-11-28T03:09:45.034 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:45.035 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":122,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:44.350810+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":14,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":112,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":108,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:45.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 msgr2=0x7f40b8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 0x7f40b8061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f40d814e320 tx=0x7f40c0009040).stop 2023-11-28T03:09:45.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 msgr2=0x7f40d814dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 rx=0x7f40c800d860 tx=0x7f40c8008040).stop 2023-11-28T03:09:45.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 shutdown_connections 2023-11-28T03:09:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f40b805f150 0x7f40b8061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f40d81542d0 0x7f40d8167e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40d8062e10 0x7f40d814dab0 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 --2- 172.21.15.73:0/3632204858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40d8069ce0 0x7f40d8167940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.035+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 >> 172.21.15.73:0/3632204858 conn(0x7f40d80c7fd0 msgr2=0x7f40d80be2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.036+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 shutdown_connections 2023-11-28T03:09:45.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.036+0000 7f40dd3f5700 1 -- 172.21.15.73:0/3632204858 wait complete. 2023-11-28T03:09:45.049 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:09:45.049 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:09:45.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.164+0000 7f2633782700 1 Processor -- start 2023-11-28T03:09:45.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.164+0000 7f2633782700 1 -- start start 2023-11-28T03:09:45.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f2633782700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f262c158a30 0x7f262c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f2633782700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c0c72d0 con 0x7f262c098d10 2023-11-28T03:09:45.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f2633782700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f262c064570 con 0x7f262c0c6190 2023-11-28T03:09:45.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f2633782700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f262c154bc0 con 0x7f262c158a30 2023-11-28T03:09:45.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f2631d1f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f262c158a30 0x7f262c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f263151e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f262c098d10 0x7f262c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56416/0 (socket says 172.21.15.73:56416) 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.165+0000 7f263151e700 1 -- 172.21.15.73:0/2183026273 learned_addr learned my addr 172.21.15.73:0/2183026273 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2631d1f700 1 -- 172.21.15.73:0/2183026273 >> v1:172.21.15.150:6789/0 conn(0x7f262c0c6190 legacy=0x7f262c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2631d1f700 1 -- 172.21.15.73:0/2183026273 >> v1:172.21.15.73:6789/0 conn(0x7f262c098d10 legacy=0x7f262c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2631d1f700 1 -- 172.21.15.73:0/2183026273 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f262c0c66c0 con 0x7f262c158a30 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2623fff700 1 -- 172.21.15.73:0/2183026273 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2991652525 0 0) 0x7f262c0c72d0 con 0x7f262c098d10 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2631d1f700 1 --2- 172.21.15.73:0/2183026273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c158a30 0x7f262c0c7760 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2624009760 tx=0x7f2624009060).ready entity=mon.0 client_cookie=44c62cf44a4b27ac server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2623fff700 1 -- 172.21.15.73:0/2183026273 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400b520 con 0x7f262c158a30 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2623fff700 1 -- 172.21.15.73:0/2183026273 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f262400b680 con 0x7f262c158a30 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2623fff700 1 -- 172.21.15.73:0/2183026273 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f262400ba00 con 0x7f262c158a30 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2633782700 1 -- 172.21.15.73:0/2183026273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c158a30 msgr2=0x7f262c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2633782700 1 --2- 172.21.15.73:0/2183026273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c158a30 0x7f262c0c7760 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 rx=0x7f2624009760 tx=0x7f2624009060).stop 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2633782700 1 -- 172.21.15.73:0/2183026273 shutdown_connections 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2633782700 1 --2- 172.21.15.73:0/2183026273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c158a30 0x7f262c0c7760 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.166+0000 7f2633782700 1 -- 172.21.15.73:0/2183026273 >> 172.21.15.73:0/2183026273 conn(0x7f262c0cb310 msgr2=0x7f262c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.167+0000 7f2633782700 1 -- 172.21.15.73:0/2183026273 shutdown_connections 2023-11-28T03:09:45.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.167+0000 7f2633782700 1 -- 172.21.15.73:0/2183026273 wait complete. 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.168+0000 7f2633782700 1 Processor -- start 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.168+0000 7f2633782700 1 -- start start 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.168+0000 7f2633782700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f262c0c6190 0x7f262c144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262c158a30 0x7f262c0c45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0abd60 con 0x7f262c098d10 2023-11-28T03:09:45.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0af1b0 con 0x7f262c158a30 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f262c0ae2c0 con 0x7f262c0c6190 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53324/0 (socket says 172.21.15.73:53324) 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 -- 172.21.15.73:0/2289415636 learned_addr learned my addr 172.21.15.73:0/2289415636 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2631d1f700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262c158a30 0x7f262c0c45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2630d1d700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f262c0c6190 0x7f262c144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 -- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f262c0c6190 msgr2=0x7f262c144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f262c0c6190 0x7f262c144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 -- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262c158a30 msgr2=0x7f262c0c45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262c158a30 0x7f262c0c45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.175 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 -- 172.21.15.73:0/2289415636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26240092a0 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f263151e700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f262800c960 tx=0x7f2628001630).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628005d30 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2628005e90 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f262c0ae5a0 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628007320 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.169+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f262c0aeb60 con 0x7f262c098d10 2023-11-28T03:09:45.176 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.170+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f262c06c860 con 0x7f262c098d10 2023-11-28T03:09:45.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.170+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2628007710 con 0x7f262c098d10 2023-11-28T03:09:45.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.171+0000 7f26217fa700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 0x7f26100615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.171+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f262807a030 con 0x7f262c098d10 2023-11-28T03:09:45.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.171+0000 7f2630d1d700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 0x7f26100615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.171+0000 7f2630d1d700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 0x7f26100615e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f262c0ae010 tx=0x7f261c009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.178 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.175+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f262804b650 con 0x7f262c098d10 2023-11-28T03:09:45.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.338+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f262c0c4ed0 con 0x7f262c098d10 2023-11-28T03:09:45.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.339+0000 7f26217fa700 1 -- 172.21.15.73:0/2289415636 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v122) v1 ==== 110+0+10 (secure 0 0 0) 0x7f262804ec70 con 0x7f262c098d10 2023-11-28T03:09:45.340 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:09:45.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 msgr2=0x7f26100615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 0x7f26100615e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f262c0ae010 tx=0x7f261c009040).stop 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 msgr2=0x7f262c0ab870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 rx=0x7f262800c960 tx=0x7f2628001630).stop 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 shutdown_connections 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f261005f120 0x7f26100615e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f262c158a30 0x7f262c0c45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f262c098d10 0x7f262c0ab870 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 --2- 172.21.15.73:0/2289415636 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f262c0c6190 0x7f262c144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.340+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 >> 172.21.15.73:0/2289415636 conn(0x7f262c0cb310 msgr2=0x7f262c153d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.341+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 shutdown_connections 2023-11-28T03:09:45.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.341+0000 7f2633782700 1 -- 172.21.15.73:0/2289415636 wait complete. 2023-11-28T03:09:45.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:09:45.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.468+0000 7f0b5b005700 1 Processor -- start 2023-11-28T03:09:45.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.468+0000 7f0b5b005700 1 -- start start 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b5b005700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b54098d10 0x7f0b540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b5b005700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b540c72d0 con 0x7f0b54158320 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b5b005700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b54064570 con 0x7f0b540c6190 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b5b005700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b54154bc0 con 0x7f0b54098d10 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b58da1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b54098d10 0x7f0b540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b595a2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0b54158320 0x7f0b540a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56432/0 (socket says 172.21.15.73:56432) 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b595a2700 1 -- 172.21.15.73:0/189716867 learned_addr learned my addr 172.21.15.73:0/189716867 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b58da1700 1 -- 172.21.15.73:0/189716867 >> v1:172.21.15.150:6789/0 conn(0x7f0b540c6190 legacy=0x7f0b54063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b58da1700 1 -- 172.21.15.73:0/189716867 >> v1:172.21.15.73:6789/0 conn(0x7f0b54158320 legacy=0x7f0b540a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b4b7fe700 1 -- 172.21.15.73:0/189716867 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2141822744 0 0) 0x7f0b540c72d0 con 0x7f0b54158320 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b58da1700 1 -- 172.21.15.73:0/189716867 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b540c66c0 con 0x7f0b54098d10 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b58da1700 1 --2- 172.21.15.73:0/189716867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b54098d10 0x7f0b540c7760 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f0b50009e60 tx=0x7f0b50009780).ready entity=mon.0 client_cookie=6b29265b88879f1d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b4b7fe700 1 -- 172.21.15.73:0/189716867 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b5000e070 con 0x7f0b54098d10 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b4b7fe700 1 -- 172.21.15.73:0/189716867 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b5000bbf0 con 0x7f0b54098d10 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.469+0000 7f0b4b7fe700 1 -- 172.21.15.73:0/189716867 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b50012410 con 0x7f0b54098d10 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 -- 172.21.15.73:0/189716867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b54098d10 msgr2=0x7f0b540c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 --2- 172.21.15.73:0/189716867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b54098d10 0x7f0b540c7760 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 rx=0x7f0b50009e60 tx=0x7f0b50009780).stop 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 -- 172.21.15.73:0/189716867 shutdown_connections 2023-11-28T03:09:45.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 --2- 172.21.15.73:0/189716867 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b54098d10 0x7f0b540c7760 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 -- 172.21.15.73:0/189716867 >> 172.21.15.73:0/189716867 conn(0x7f0b540cb310 msgr2=0x7f0b540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 -- 172.21.15.73:0/189716867 shutdown_connections 2023-11-28T03:09:45.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.470+0000 7f0b5b005700 1 -- 172.21.15.73:0/189716867 wait complete. 2023-11-28T03:09:45.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.471+0000 7f0b5b005700 1 Processor -- start 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.471+0000 7f0b5b005700 1 -- start start 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b54098d10 0x7f0b54155fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b54158320 0x7f0b5414e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b540c29c0 con 0x7f0b540c6190 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b541571b0 con 0x7f0b54098d10 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b5b005700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b54157b30 con 0x7f0b54158320 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53328/0 (socket says 172.21.15.73:53328) 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 -- 172.21.15.73:0/1705981538 learned_addr learned my addr 172.21.15.73:0/1705981538 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 -- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b54158320 msgr2=0x7f0b5414e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b54158320 0x7f0b5414e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 -- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b54098d10 msgr2=0x7f0b54155fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b54098d10 0x7f0b54155fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.472+0000 7f0b4bfff700 1 -- 172.21.15.73:0/1705981538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b50009a20 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b4bfff700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f0b4400c4c0 tx=0x7f0b44007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400f470 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b4400f5d0 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b5414ed50 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b4400f990 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.473+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b54156880 con 0x7f0b540c6190 2023-11-28T03:09:45.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.474+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b5406c860 con 0x7f0b540c6190 2023-11-28T03:09:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.474+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0b44012400 con 0x7f0b540c6190 2023-11-28T03:09:45.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.474+0000 7f0b48ff9700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 0x7f0b40061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.474+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0b44079350 con 0x7f0b540c6190 2023-11-28T03:09:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.475+0000 7f0b58da1700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 0x7f0b40061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.475+0000 7f0b58da1700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 0x7f0b40061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0b50009e60 tx=0x7f0b50009780).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.478+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0b44048af0 con 0x7f0b540c6190 2023-11-28T03:09:45.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.642+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b54157340 con 0x7f0b540c6190 2023-11-28T03:09:45.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.642+0000 7f0b48ff9700 1 -- 172.21.15.73:0/1705981538 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v122) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b44017070 con 0x7f0b540c6190 2023-11-28T03:09:45.644 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:09:45.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 msgr2=0x7f0b40061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 0x7f0b40061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f0b50009e60 tx=0x7f0b50009780).stop 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 msgr2=0x7f0b54157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 rx=0x7f0b4400c4c0 tx=0x7f0b44007db0).stop 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 shutdown_connections 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0b4005f150 0x7f0b40061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b54158320 0x7f0b5414e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:45.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b54098d10 0x7f0b54155fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:45.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 --2- 172.21.15.73:0/1705981538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b540c6190 0x7f0b54157640 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 >> 172.21.15.73:0/1705981538 conn(0x7f0b540cb310 msgr2=0x7f0b540ca280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 shutdown_connections 2023-11-28T03:09:45.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.644+0000 7f0b5b005700 1 -- 172.21.15.73:0/1705981538 wait complete. 2023-11-28T03:09:45.657 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:09:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.769+0000 7fe71b365700 1 Processor -- start 2023-11-28T03:09:45.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.769+0000 7fe71b365700 1 -- start start 2023-11-28T03:09:45.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe71b365700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 0x7fe7141548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe71b365700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe7140c6e30 0x7fe7140c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe71b365700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7140c69a0 con 0x7fe71406a610 2023-11-28T03:09:45.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe71b365700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7140c13a0 con 0x7fe7140c6e30 2023-11-28T03:09:45.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe71b365700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7140c1680 con 0x7fe7141544e0 2023-11-28T03:09:45.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe718900700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe71406a610 0x7fe7140c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56442/0 (socket says 172.21.15.73:56442) 2023-11-28T03:09:45.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe718900700 1 -- 172.21.15.73:0/3664940848 learned_addr learned my addr 172.21.15.73:0/3664940848 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe719902700 1 --2- 172.21.15.73:0/3664940848 >> v2:172.21.15.150:3300/0 conn(0x7fe7140c6e30 0x7fe7140c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.770+0000 7fe719101700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 0x7fe7141548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3877834449 0 0) 0x7fe7140c69a0 con 0x7fe71406a610 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7000033f0 con 0x7fe71406a610 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719902700 1 -- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 msgr2=0x7fe7141548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:45.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719902700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 0x7fe7141548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719902700 1 -- 172.21.15.73:0/3664940848 >> v1:172.21.15.73:6789/0 conn(0x7fe71406a610 legacy=0x7fe7140c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1651754652 0 0) 0x7fe7000033f0 con 0x7fe71406a610 2023-11-28T03:09:45.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719902700 1 -- 172.21.15.73:0/3664940848 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7140c6630 con 0x7fe7140c6e30 2023-11-28T03:09:45.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719101700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 0x7fe7141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:45.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe719902700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe7140c72e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe70c009760 tx=0x7fe70c009060).ready entity=mon.1 client_cookie=69d9c7d48a3735bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe704001470 con 0x7fe7140c6e30 2023-11-28T03:09:45.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c011400 con 0x7fe7140c6e30 2023-11-28T03:09:45.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.771+0000 7fe70bfff700 1 -- 172.21.15.73:0/3664940848 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c011780 con 0x7fe7140c6e30 2023-11-28T03:09:45.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 -- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 msgr2=0x7fe7140c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe7140c72e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 rx=0x7fe70c009760 tx=0x7fe70c009060).stop 2023-11-28T03:09:45.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 -- 172.21.15.73:0/3664940848 shutdown_connections 2023-11-28T03:09:45.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe7140c72e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 --2- 172.21.15.73:0/3664940848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe7141544e0 0x7fe7141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 -- 172.21.15.73:0/3664940848 >> 172.21.15.73:0/3664940848 conn(0x7fe714085020 msgr2=0x7fe714094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 -- 172.21.15.73:0/3664940848 shutdown_connections 2023-11-28T03:09:45.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.772+0000 7fe71b365700 1 -- 172.21.15.73:0/3664940848 wait complete. 2023-11-28T03:09:45.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.773+0000 7fe71b365700 1 Processor -- start 2023-11-28T03:09:45.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.773+0000 7fe71b365700 1 -- start start 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe71406a610 0x7fe7141508f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe714096430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714151260 con 0x7fe7141544e0 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe714151540 con 0x7fe7140c6e30 2023-11-28T03:09:45.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe71b365700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe714151820 con 0x7fe71406a610 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53330/0 (socket says 172.21.15.73:53330) 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 -- 172.21.15.73:0/1600189633 learned_addr learned my addr 172.21.15.73:0/1600189633 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719101700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe71406a610 0x7fe7141508f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe718900700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe714096430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 -- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe71406a610 msgr2=0x7fe7141508f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe71406a610 0x7fe7141508f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 -- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 msgr2=0x7fe714096430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe714096430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 -- 172.21.15.73:0/1600189633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe710009a20 con 0x7fe7141544e0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.774+0000 7fe719902700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fe70c009d50 tx=0x7fe70c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.775+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c010bb0 con 0x7fe7141544e0 2023-11-28T03:09:45.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.775+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe70c010d10 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.775+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe70c0092a0 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.775+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe70c01d640 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.775+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe714151de0 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.776+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe71406c860 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.776+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe70c01d9f0 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.776+0000 7fe7097fa700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 0x7fe6f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.776+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fe70c07ad40 con 0x7fe7141544e0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.776+0000 7fe719101700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 0x7fe6f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.777+0000 7fe719101700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 0x7fe6f8061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe7100099f0 tx=0x7fe71000b040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:45.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.780+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe70c04c410 con 0x7fe7141544e0 2023-11-28T03:09:45.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.946+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe7140990e0 con 0x7fe7141544e0 2023-11-28T03:09:45.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.947+0000 7fe7097fa700 1 -- 172.21.15.73:0/1600189633 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v122) v1 ==== 84+0+143 (secure 0 0 0) 0x7fe70c022030 con 0x7fe7141544e0 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:09:45.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:09:45.950 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:45.950 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:09:45.950 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:09:45.950 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:45.950 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:09:45.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 msgr2=0x7fe6f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 0x7fe6f8061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fe7100099f0 tx=0x7fe71000b040).stop 2023-11-28T03:09:45.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 msgr2=0x7fe71414fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 rx=0x7fe70c009d50 tx=0x7fe70c016000).stop 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 shutdown_connections 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe6f805f150 0x7fe6f8061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe7141544e0 0x7fe71414fcc0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe71406a610 0x7fe7141508f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 --2- 172.21.15.73:0/1600189633 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe7140c6e30 0x7fe714096430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:45.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 >> 172.21.15.73:0/1600189633 conn(0x7fe714085020 msgr2=0x7fe7140c97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:45.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 shutdown_connections 2023-11-28T03:09:45.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:45.949+0000 7fe71b365700 1 -- 172.21.15.73:0/1600189633 wait complete. 2023-11-28T03:09:45.962 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:09:45.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:09:46.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.078+0000 7f193d900700 1 Processor -- start 2023-11-28T03:09:46.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.078+0000 7f193d900700 1 -- start start 2023-11-28T03:09:46.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f193d900700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f193d900700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f193d900700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19380c6630 con 0x7f1938154530 2023-11-28T03:09:46.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f193d900700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19380c6390 con 0x7f19380aa220 2023-11-28T03:09:46.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f193d900700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380a84e0 con 0x7f1938154910 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f19377fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f1936ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1938154530 0x7f19380c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56456/0 (socket says 172.21.15.73:56456) 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f1936ffd700 1 -- 172.21.15.73:0/987422478 learned_addr learned my addr 172.21.15.73:0/987422478 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.079+0000 7f192e7fc700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19377fe700 1 -- 172.21.15.73:0/987422478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 msgr2=0x7f19380a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19377fe700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19377fe700 1 -- 172.21.15.73:0/987422478 >> v1:172.21.15.73:6789/0 conn(0x7f1938154530 legacy=0x7f19380c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19377fe700 1 -- 172.21.15.73:0/987422478 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19380aaba0 con 0x7f19380aa220 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19367fc700 1 -- 172.21.15.73:0/987422478 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1511143001 0 0) 0x7f19380c6630 con 0x7f1938154530 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f192e7fc700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19377fe700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19380aa220 0x7f19380aa6b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f1930009e60 tx=0x7f1930009780).ready entity=mon.0 client_cookie=62f3b70d51203e80 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19367fc700 1 -- 172.21.15.73:0/987422478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193000e070 con 0x7f19380aa220 2023-11-28T03:09:46.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19367fc700 1 -- 172.21.15.73:0/987422478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f193000bc30 con 0x7f19380aa220 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f19367fc700 1 -- 172.21.15.73:0/987422478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1930012410 con 0x7f19380aa220 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 -- 172.21.15.73:0/987422478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19380aa220 msgr2=0x7f19380aa6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19380aa220 0x7f19380aa6b0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 rx=0x7f1930009e60 tx=0x7f1930009780).stop 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 -- 172.21.15.73:0/987422478 shutdown_connections 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f19380aa220 0x7f19380aa6b0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 --2- 172.21.15.73:0/987422478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f19380a7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 -- 172.21.15.73:0/987422478 >> 172.21.15.73:0/987422478 conn(0x7f1938094930 msgr2=0x7f193806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:46.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 -- 172.21.15.73:0/987422478 shutdown_connections 2023-11-28T03:09:46.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.080+0000 7f193d900700 1 -- 172.21.15.73:0/987422478 wait complete. 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 Processor -- start 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 -- start start 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 0x7f1938152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f1938148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19381523b0 con 0x7f1938154530 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19380c4260 con 0x7f19380aa220 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f193d900700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19380c4470 con 0x7f1938154910 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.082+0000 7f192e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53350/0 (socket says 172.21.15.73:53350) 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 -- 172.21.15.73:0/735665310 learned_addr learned my addr 172.21.15.73:0/735665310 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f1936ffd700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 0x7f1938152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f19377fe700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f1938148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 -- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 msgr2=0x7f1938148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f1938148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 -- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 msgr2=0x7f1938152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 0x7f1938152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 -- 172.21.15.73:0/735665310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1930009a20 con 0x7f1938154530 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f19377fe700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f1938148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f1936ffd700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 0x7f1938152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192e7fc700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f192000c880 tx=0x7f1920008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920010580 con 0x7f1938154530 2023-11-28T03:09:46.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19200106e0 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19380c4750 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1920010a80 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.083+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19380c4d10 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.084+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19380c5080 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.084+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f192001b460 con 0x7f1938154530 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.085+0000 7f192f7fe700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 0x7f1918061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.085+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1920079ec0 con 0x7f1938154530 2023-11-28T03:09:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.085+0000 7f1936ffd700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 0x7f1918061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.085+0000 7f1936ffd700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 0x7f1918061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1928001360 tx=0x7f1928009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.089+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f192004b480 con 0x7f1938154530 2023-11-28T03:09:46.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.252+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f193806c860 con 0x7f1938154530 2023-11-28T03:09:46.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.361+0000 7f192f7fe700 1 -- 172.21.15.73:0/735665310 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v123) v1 ==== 152+0+0 (secure 0 0 0) 0x7f192004eaa0 con 0x7f1938154530 2023-11-28T03:09:46.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 msgr2=0x7f1918061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 0x7f1918061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f1928001360 tx=0x7f1928009040).stop 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 msgr2=0x7f19380c3d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 rx=0x7f192000c880 tx=0x7f1920008040).stop 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 shutdown_connections 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f191805f150 0x7f1918061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1938154910 0x7f1938148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f19380aa220 0x7f1938152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 --2- 172.21.15.73:0/735665310 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1938154530 0x7f19380c3d70 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 >> 172.21.15.73:0/735665310 conn(0x7f1938094930 msgr2=0x7f19380c0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:46.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 shutdown_connections 2023-11-28T03:09:46.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.364+0000 7f193d900700 1 -- 172.21.15.73:0/735665310 wait complete. 2023-11-28T03:09:46.367 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:09:46.382 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:09:46.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.500+0000 7f9ba1e4e700 1 Processor -- start 2023-11-28T03:09:46.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.500+0000 7f9ba1e4e700 1 -- start start 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9ba1e4e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 0x7f9b9c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9ba1e4e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c0c6190 0x7f9b9c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9ba1e4e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b9c0c72d0 con 0x7f9b9c064350 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9ba1e4e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c154bc0 con 0x7f9b9c0c6190 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9ba1e4e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b9c0c66c0 con 0x7f9b9c098d10 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9b9affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c0c6190 0x7f9b9c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9b9affd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9b9c0c6190 0x7f9b9c063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53354/0 (socket says 172.21.15.73:53354) 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9b9bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9b9c064350 0x7f9b9c158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56460/0 (socket says 172.21.15.73:56460) 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.501+0000 7f9b9affd700 1 -- 172.21.15.73:0/4123551130 learned_addr learned my addr 172.21.15.73:0/4123551130 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/4123551130 >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 0x7f9b9c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9affd700 1 -- 172.21.15.73:0/4123551130 >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 msgr2=0x7f9b9c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9affd700 1 --2- 172.21.15.73:0/4123551130 >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 0x7f9b9c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9affd700 1 -- 172.21.15.73:0/4123551130 >> v1:172.21.15.73:6789/0 conn(0x7f9b9c064350 legacy=0x7f9b9c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9affd700 1 -- 172.21.15.73:0/4123551130 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b9c0c69a0 con 0x7f9b9c0c6190 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/4123551130 >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 0x7f9b9c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9affd700 1 --2- 172.21.15.73:0/4123551130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0c6190 0x7f9b9c063e60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9b9c083f80 tx=0x7f9b900090f0).ready entity=mon.0 client_cookie=a11edad95ceb002f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9a7fc700 1 -- 172.21.15.73:0/4123551130 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000b600 con 0x7f9b9c0c6190 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9a7fc700 1 -- 172.21.15.73:0/4123551130 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9000b760 con 0x7f9b9c0c6190 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9b9a7fc700 1 -- 172.21.15.73:0/4123551130 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000bae0 con 0x7f9b9c0c6190 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/4123551130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0c6190 msgr2=0x7f9b9c063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/4123551130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0c6190 0x7f9b9c063e60 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 rx=0x7f9b9c083f80 tx=0x7f9b900090f0).stop 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/4123551130 shutdown_connections 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/4123551130 >> v2:172.21.15.150:3300/0 conn(0x7f9b9c098d10 0x7f9b9c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/4123551130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c0c6190 0x7f9b9c063e60 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/4123551130 >> 172.21.15.73:0/4123551130 conn(0x7f9b9c0cb310 msgr2=0x7f9b9c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.502+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/4123551130 shutdown_connections 2023-11-28T03:09:46.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.503+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/4123551130 wait complete. 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.504+0000 7f9ba1e4e700 1 Processor -- start 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.504+0000 7f9ba1e4e700 1 -- start start 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 0x7f9b9c149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b9c0c6190 0x7f9b9c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1562e0 con 0x7f9b9c098d10 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1556d0 con 0x7f9b9c064350 2023-11-28T03:09:46.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b9c1559b0 con 0x7f9b9c0c6190 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53366/0 (socket says 172.21.15.73:53366) 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 -- 172.21.15.73:0/319807681 learned_addr learned my addr 172.21.15.73:0/319807681 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 0x7f9b9c149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9bfff700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b9c0c6190 0x7f9b9c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 -- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b9c0c6190 msgr2=0x7f9b9c0aec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b9c0c6190 0x7f9b9c0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 -- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 msgr2=0x7f9b9c149070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 0x7f9b9c149070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 -- 172.21.15.73:0/319807681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b90009370 con 0x7f9b9c098d10 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 0x7f9b9c149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:46.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b9affd700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9b90011000 tx=0x7f9b90017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9000bae0 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b90000c90 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9001b640 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9b9c155c90 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.505+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b9c149560 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.506+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b9c156130 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.507+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9b9001ba30 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.507+0000 7f9b837fe700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 0x7f9b88061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.507+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f9b9007ba20 con 0x7f9b9c098d10 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.507+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 0x7f9b88061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:46.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.507+0000 7f9b9b7fe700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 0x7f9b88061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b8c00d640 tx=0x7f9b8c007670).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:46.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.511+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9b9004ced0 con 0x7f9b9c098d10 2023-11-28T03:09:46.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:46.675+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9b9c06c860 con 0x7f9b9c098d10 2023-11-28T03:09:47.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.370+0000 7f9b837fe700 1 -- 172.21.15.73:0/319807681 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v124) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9b900504f0 con 0x7f9b9c098d10 2023-11-28T03:09:47.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.384+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 msgr2=0x7f9b88061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.384+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 0x7f9b88061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9b8c00d640 tx=0x7f9b8c007670).stop 2023-11-28T03:09:47.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.384+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 msgr2=0x7f9b9c154d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.384+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 rx=0x7f9b90011000 tx=0x7f9b90017000).stop 2023-11-28T03:09:47.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 shutdown_connections 2023-11-28T03:09:47.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9b8805f150 0x7f9b88061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9b9c0c6190 0x7f9b9c0aec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9b9c064350 0x7f9b9c149070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 --2- 172.21.15.73:0/319807681 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9b9c098d10 0x7f9b9c154d60 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.387+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 >> 172.21.15.73:0/319807681 conn(0x7f9b9c0cb310 msgr2=0x7f9b9c14c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:47.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.389+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 shutdown_connections 2023-11-28T03:09:47.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.389+0000 7f9ba1e4e700 1 -- 172.21.15.73:0/319807681 wait complete. 2023-11-28T03:09:47.395 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:09:47.407 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:09:47.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.523+0000 7f4127fff700 1 Processor -- start 2023-11-28T03:09:47.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.523+0000 7f4127fff700 1 -- start start 2023-11-28T03:09:47.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4127fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4127fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4128158570 0x7f4128158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4127fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41280c7bc0 con 0x7f4128154950 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4127fff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4128155920 con 0x7f4128158570 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4127fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4128154dd0 con 0x7f412814e530 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f41277fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4128158570 0x7f4128158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f411ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4128154950 0x7f412814e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56462/0 (socket says 172.21.15.73:56462) 2023-11-28T03:09:47.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f411ffff700 1 -- 172.21.15.73:0/52598692 learned_addr learned my addr 172.21.15.73:0/52598692 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f4126ffd700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.524+0000 7f41277fe700 1 -- 172.21.15.73:0/52598692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 msgr2=0x7f41280c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41277fe700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41277fe700 1 -- 172.21.15.73:0/52598692 >> v1:172.21.15.73:6789/0 conn(0x7f4128154950 legacy=0x7f412814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41267fc700 1 -- 172.21.15.73:0/52598692 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1246459764 0 0) 0x7f41280c7bc0 con 0x7f4128154950 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41277fe700 1 -- 172.21.15.73:0/52598692 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4128155080 con 0x7f4128158570 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41277fe700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f4128158a20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f4120009e60 tx=0x7f4120009780).ready entity=mon.0 client_cookie=a57741725af8f7de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41267fc700 1 -- 172.21.15.73:0/52598692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000e070 con 0x7f4128158570 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41267fc700 1 -- 172.21.15.73:0/52598692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412000bb60 con 0x7f4128158570 2023-11-28T03:09:47.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f41267fc700 1 -- 172.21.15.73:0/52598692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4120012410 con 0x7f4128158570 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 -- 172.21.15.73:0/52598692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 msgr2=0x7f4128158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f4128158a20 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 rx=0x7f4120009e60 tx=0x7f4120009780).stop 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 -- 172.21.15.73:0/52598692 shutdown_connections 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 --2- 172.21.15.73:0/52598692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f4128158a20 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 -- 172.21.15.73:0/52598692 >> 172.21.15.73:0/52598692 conn(0x7f4128064990 msgr2=0x7f4128064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 -- 172.21.15.73:0/52598692 shutdown_connections 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.525+0000 7f4127fff700 1 -- 172.21.15.73:0/52598692 wait complete. 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 Processor -- start 2023-11-28T03:09:47.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 -- start start 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128154950 0x7f412814d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41281666f0 con 0x7f4128158570 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41280c55a0 con 0x7f4128154950 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f4127fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4128165dc0 con 0x7f412814e530 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f41277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f41277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53386/0 (socket says 172.21.15.73:53386) 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f41277fe700 1 -- 172.21.15.73:0/79935380 learned_addr learned my addr 172.21.15.73:0/79935380 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.527+0000 7f411ffff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128154950 0x7f412814d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f4126ffd700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 -- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 msgr2=0x7f41280c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 -- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128154950 msgr2=0x7f412814d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128154950 0x7f412814d8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 -- 172.21.15.73:0/79935380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4120009a20 con 0x7f4128158570 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f41277fe700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f4128064480 tx=0x7f4120009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412000b6f0 con 0x7f4128158570 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412001d630 con 0x7f4128158570 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412001d9d0 con 0x7f4128158570 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41281660a0 con 0x7f4128158570 2023-11-28T03:09:47.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.528+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41280c38f0 con 0x7f4128158570 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.529+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41281669d0 con 0x7f4128158570 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.529+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4120003040 con 0x7f4128158570 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.530+0000 7f411effd700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 0x7f4108061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.530+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f412007be80 con 0x7f4128158570 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.530+0000 7f4126ffd700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 0x7f4108061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.530+0000 7f4126ffd700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 0x7f4108061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f41280c6850 tx=0x7f4118009060).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.534+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f412004d250 con 0x7f4128158570 2023-11-28T03:09:47.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.740+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4128062ee0 con 0x7f4128158570 2023-11-28T03:09:47.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.762+0000 7f411effd700 1 -- 172.21.15.73:0/79935380 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 15 and data pool 16 v44) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4120050870 con 0x7f4128158570 2023-11-28T03:09:47.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 msgr2=0x7f4108061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 0x7f4108061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f41280c6850 tx=0x7f4118009060).stop 2023-11-28T03:09:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 msgr2=0x7f41280ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 rx=0x7f4128064480 tx=0x7f4120009780).stop 2023-11-28T03:09:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 shutdown_connections 2023-11-28T03:09:47.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f410805f150 0x7f4108061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4128158570 0x7f41280ae030 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f412814e530 0x7f41280c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 --2- 172.21.15.73:0/79935380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4128154950 0x7f412814d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.766+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 >> 172.21.15.73:0/79935380 conn(0x7f4128064990 msgr2=0x7f4128153d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.767+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 shutdown_connections 2023-11-28T03:09:47.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.767+0000 7f4127fff700 1 -- 172.21.15.73:0/79935380 wait complete. 2023-11-28T03:09:47.769 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 15 and data pool 16 2023-11-28T03:09:47.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:47.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.894+0000 7f44f68d7700 1 Processor -- start 2023-11-28T03:09:47.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.894+0000 7f44f68d7700 1 -- start start 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44f68d7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f44f0063e50 0x7f44f0069070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44f68d7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01514a0 0x7f44f0069560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44f68d7700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44f00a9ea0 con 0x7f44f01510c0 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44f68d7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44f0069a50 con 0x7f44f0063e50 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44f68d7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f00ac4d0 con 0x7f44f01514a0 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44ef7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f44f0063e50 0x7f44f0069070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44effff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f44f01510c0 0x7f44f00a9790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56466/0 (socket says 172.21.15.73:56466) 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.895+0000 7f44effff700 1 -- 172.21.15.73:0/1395287272 learned_addr learned my addr 172.21.15.73:0/1395287272 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44ef7fe700 1 -- 172.21.15.73:0/1395287272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01514a0 msgr2=0x7f44f0069560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44ef7fe700 1 --2- 172.21.15.73:0/1395287272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01514a0 0x7f44f0069560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:47.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44ef7fe700 1 -- 172.21.15.73:0/1395287272 >> v1:172.21.15.73:6789/0 conn(0x7f44f01510c0 legacy=0x7f44f00a9790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44ef7fe700 1 -- 172.21.15.73:0/1395287272 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44f00ad4a0 con 0x7f44f0063e50 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44eeffd700 1 -- 172.21.15.73:0/1395287272 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3320530998 0 0) 0x7f44f00a9ea0 con 0x7f44f01510c0 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44ef7fe700 1 --2- 172.21.15.73:0/1395287272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f0063e50 0x7f44f0069070 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f44e4009760 tx=0x7f44e4009060).ready entity=mon.0 client_cookie=ba69224a2ef2d570 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44eeffd700 1 -- 172.21.15.73:0/1395287272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400b580 con 0x7f44f0063e50 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44eeffd700 1 -- 172.21.15.73:0/1395287272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e400b6e0 con 0x7f44f0063e50 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44eeffd700 1 -- 172.21.15.73:0/1395287272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e400ba60 con 0x7f44f0063e50 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44f68d7700 1 -- 172.21.15.73:0/1395287272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f0063e50 msgr2=0x7f44f0069070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.896+0000 7f44f68d7700 1 --2- 172.21.15.73:0/1395287272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f0063e50 0x7f44f0069070 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 rx=0x7f44e4009760 tx=0x7f44e4009060).stop 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 -- 172.21.15.73:0/1395287272 shutdown_connections 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 --2- 172.21.15.73:0/1395287272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01514a0 0x7f44f0069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 --2- 172.21.15.73:0/1395287272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f0063e50 0x7f44f0069070 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 -- 172.21.15.73:0/1395287272 >> 172.21.15.73:0/1395287272 conn(0x7f44f00c7760 msgr2=0x7f44f00c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 -- 172.21.15.73:0/1395287272 shutdown_connections 2023-11-28T03:09:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.897+0000 7f44f68d7700 1 -- 172.21.15.73:0/1395287272 wait complete. 2023-11-28T03:09:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.898+0000 7f44f68d7700 1 Processor -- start 2023-11-28T03:09:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.898+0000 7f44f68d7700 1 -- start start 2023-11-28T03:09:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0063e50 0x7f44f013f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01510c0 0x7f44f0145a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0148590 con 0x7f44f01514a0 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44f0149110 con 0x7f44f0063e50 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44f01495a0 con 0x7f44f01510c0 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53398/0 (socket says 172.21.15.73:53398) 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 -- 172.21.15.73:0/4206026352 learned_addr learned my addr 172.21.15.73:0/4206026352 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44effff700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0063e50 0x7f44f013f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44ef7fe700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01510c0 0x7f44f0145a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 -- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01510c0 msgr2=0x7f44f0145a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01510c0 0x7f44f0145a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 -- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0063e50 msgr2=0x7f44f013f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0063e50 0x7f44f013f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 -- 172.21.15.73:0/4206026352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44e40092a0 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f4e74700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f44e800b840 tx=0x7f44e80092d0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800d5a0 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44e800d720 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44f01487e0 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44e800dae0 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.899+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44f0148da0 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.900+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44f006c860 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.901+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f44e8019460 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.901+0000 7f44d7fff700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 0x7f44dc0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.901+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f44e807a230 con 0x7f44f01514a0 2023-11-28T03:09:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.901+0000 7f44effff700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 0x7f44dc0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:47.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.901+0000 7f44effff700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 0x7f44dc0615e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f44e00011e0 tx=0x7f44e0009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:47.905+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44e804b4a0 con 0x7f44f01514a0 2023-11-28T03:09:48.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.068+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44f00aaa40 con 0x7f44f01514a0 2023-11-28T03:09:48.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.069+0000 7f44d7fff700 1 -- 172.21.15.73:0/4206026352 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f44e804eac0 con 0x7f44f01514a0 2023-11-28T03:09:48.071 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:48.071 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":125,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:47.742082+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:48.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 msgr2=0x7f44dc0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 0x7f44dc0615e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f44e00011e0 tx=0x7f44e0009040).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 msgr2=0x7f44f0144340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 rx=0x7f44e800b840 tx=0x7f44e80092d0).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 shutdown_connections 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44dc05f120 0x7f44dc0615e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44f01514a0 0x7f44f0144340 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44f0063e50 0x7f44f013f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 --2- 172.21.15.73:0/4206026352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44f01510c0 0x7f44f0145a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 >> 172.21.15.73:0/4206026352 conn(0x7f44f00c7760 msgr2=0x7f44f0064ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 shutdown_connections 2023-11-28T03:09:48.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.071+0000 7f44f68d7700 1 -- 172.21.15.73:0/4206026352 wait complete. 2023-11-28T03:09:48.085 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:48.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.198+0000 7f3140f4b700 1 Processor -- start 2023-11-28T03:09:48.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.198+0000 7f3140f4b700 1 -- start start 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f3140f4b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f313c157660 0x7f313c157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f3140f4b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 0x7f313c0c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f3140f4b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f313c0c12c0 con 0x7f313c0c0180 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f3140f4b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313c0c06b0 con 0x7f313c157660 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f3140f4b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f313c0c0990 con 0x7f313c0c1750 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f313c157660 0x7f313c157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f313c0c0180 0x7f313c1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56478/0 (socket says 172.21.15.73:56478) 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f313c157660 0x7f313c157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53400/0 (socket says 172.21.15.73:53400) 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313affd700 1 -- 172.21.15.73:0/3942551768 learned_addr learned my addr 172.21.15.73:0/3942551768 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:48.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313bfff700 1 --2- 172.21.15.73:0/3942551768 >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 0x7f313c0c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 -- 172.21.15.73:0/3942551768 >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 msgr2=0x7f313c0c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 --2- 172.21.15.73:0/3942551768 >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 0x7f313c0c1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 -- 172.21.15.73:0/3942551768 >> v1:172.21.15.73:6789/0 conn(0x7f313c0c0180 legacy=0x7f313c1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313a7fc700 1 -- 172.21.15.73:0/3942551768 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3055006101 0 0) 0x7f313c0c12c0 con 0x7f313c0c0180 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 -- 172.21.15.73:0/3942551768 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f313c0c0c70 con 0x7f313c157660 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313bfff700 1 --2- 172.21.15.73:0/3942551768 >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 0x7f313c0c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.199+0000 7f313b7fe700 1 --2- 172.21.15.73:0/3942551768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c157660 0x7f313c157a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f313c084650 tx=0x7f312c009780).ready entity=mon.0 client_cookie=f0c758199bb84cfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f313a7fc700 1 -- 172.21.15.73:0/3942551768 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c00e070 con 0x7f313c157660 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f313a7fc700 1 -- 172.21.15.73:0/3942551768 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f312c00bc30 con 0x7f313c157660 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f313a7fc700 1 -- 172.21.15.73:0/3942551768 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f312c012410 con 0x7f313c157660 2023-11-28T03:09:48.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 -- 172.21.15.73:0/3942551768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c157660 msgr2=0x7f313c157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 --2- 172.21.15.73:0/3942551768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c157660 0x7f313c157a40 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 rx=0x7f313c084650 tx=0x7f312c009780).stop 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 -- 172.21.15.73:0/3942551768 shutdown_connections 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 --2- 172.21.15.73:0/3942551768 >> v2:172.21.15.150:3300/0 conn(0x7f313c0c1750 0x7f313c0c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 --2- 172.21.15.73:0/3942551768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c157660 0x7f313c157a40 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 -- 172.21.15.73:0/3942551768 >> 172.21.15.73:0/3942551768 conn(0x7f313c0cb000 msgr2=0x7f313c0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 -- 172.21.15.73:0/3942551768 shutdown_connections 2023-11-28T03:09:48.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.200+0000 7f3140f4b700 1 -- 172.21.15.73:0/3942551768 wait complete. 2023-11-28T03:09:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.201+0000 7f3140f4b700 1 Processor -- start 2023-11-28T03:09:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 -- start start 2023-11-28T03:09:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f313c0c0180 0x7f313c150190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 0x7f313c1564f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c0c46a0 con 0x7f313c0c1750 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1569e0 con 0x7f313c157660 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f3140f4b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f313c1539c0 con 0x7f313c0c0180 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f313affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f313affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53402/0 (socket says 172.21.15.73:53402) 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.202+0000 7f313affd700 1 -- 172.21.15.73:0/1901371971 learned_addr learned my addr 172.21.15.73:0/1901371971 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313b7fe700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f313c0c0180 0x7f313c150190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313bfff700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 0x7f313c1564f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 -- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f313c0c0180 msgr2=0x7f313c150190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f313c0c0180 0x7f313c150190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 -- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 msgr2=0x7f313c1564f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 0x7f313c1564f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 -- 172.21.15.73:0/1901371971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f312c009a20 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313bfff700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 0x7f313c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f313affd700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f313000c510 tx=0x7f3130007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300072e0 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3130007440 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f313c150680 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31300057a0 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.203+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f313c0b0730 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.204+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f313c153b50 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.204+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3130005bd0 con 0x7f313c0c1750 2023-11-28T03:09:48.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.205+0000 7f31237fe700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 0x7f3128061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.205+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3130079bf0 con 0x7f313c0c1750 2023-11-28T03:09:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.205+0000 7f313b7fe700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 0x7f3128061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.205+0000 7f313b7fe700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 0x7f3128061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f312c0097c0 tx=0x7f313c0c1230).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.209+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f313004aec0 con 0x7f313c0c1750 2023-11-28T03:09:48.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.373+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f313c06c7c0 con 0x7f313c0c1750 2023-11-28T03:09:48.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.374+0000 7f31237fe700 1 -- 172.21.15.73:0/1901371971 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v125) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f3130018030 con 0x7f313c0c1750 2023-11-28T03:09:48.375 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:48.376 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":125,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:47.742082+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:48.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 msgr2=0x7f3128061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 0x7f3128061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f312c0097c0 tx=0x7f313c0c1230).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 msgr2=0x7f313c156000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 rx=0x7f313000c510 tx=0x7f3130007c70).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 shutdown_connections 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f312805f150 0x7f3128061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f313c157660 0x7f313c1564f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f313c0c0180 0x7f313c150190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 --2- 172.21.15.73:0/1901371971 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f313c0c1750 0x7f313c156000 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 >> 172.21.15.73:0/1901371971 conn(0x7f313c0cb000 msgr2=0x7f313c0639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 shutdown_connections 2023-11-28T03:09:48.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.376+0000 7f3140f4b700 1 -- 172.21.15.73:0/1901371971 wait complete. 2023-11-28T03:09:48.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:09:48.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.504+0000 7f895a8b1700 1 Processor -- start 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.504+0000 7f895a8b1700 1 -- start start 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f895a8b1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f89540c6190 0x7f89540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f895a8b1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89540c72d0 con 0x7f8954158320 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f895a8b1700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8954064570 con 0x7f8954098d10 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f895a8b1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8954154bc0 con 0x7f89540c6190 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f8958e4e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8954158320 0x7f89540a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56490/0 (socket says 172.21.15.73:56490) 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f8958e4e700 1 -- 172.21.15.73:0/1868868676 learned_addr learned my addr 172.21.15.73:0/1868868676 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:48.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f89537fe700 1 --2- 172.21.15.73:0/1868868676 >> v2:172.21.15.150:3300/0 conn(0x7f89540c6190 0x7f89540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2042493618 0 0) 0x7f89540c72d0 con 0x7f8954158320 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.505+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f893c0033f0 con 0x7f8954158320 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 301878318 0 0) 0x7f893c0033f0 con 0x7f8954158320 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 >> v2:172.21.15.150:3300/0 conn(0x7f89540c6190 msgr2=0x7f89540c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 --2- 172.21.15.73:0/1868868676 >> v2:172.21.15.150:3300/0 conn(0x7f89540c6190 0x7f89540c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 >> v1:172.21.15.150:6789/0 conn(0x7f8954098d10 legacy=0x7f8954063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540c66c0 con 0x7f8954158320 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f894c0017f0 con 0x7f8954158320 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 >> v1:172.21.15.73:6789/0 conn(0x7f8954158320 legacy=0x7f89540a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f893c003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c004690 0x7f893c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c005000 0x7f893c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8954154bc0 con 0x7f893c0038e0 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00be00 con 0x7f893c004690 2023-11-28T03:09:48.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f893c00bfc0 con 0x7f893c005000 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f893c003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f89537fe700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c004690 0x7f893c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8958e4e700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c005000 0x7f893c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 -- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c005000 msgr2=0x7f893c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f893c005000 0x7f893c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 -- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c004690 msgr2=0x7f893c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f893c004690 0x7f893c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 -- 172.21.15.73:0/1868868676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f893c0033f0 con 0x7f893c0038e0 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8958e4e700 1 -- 172.21.15.73:0/1868868676 reap_dead start 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8953fff700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f893c003cc0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f8944001820 tx=0x7f8944007b90).ready entity=mon.0 client_cookie=fa743393dbff77d2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944007ea0 con 0x7f893c0038e0 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944012400 con 0x7f893c0038e0 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.506+0000 7f8952ffd700 1 -- 172.21.15.73:0/1868868676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440127a0 con 0x7f893c0038e0 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 -- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 msgr2=0x7f893c003cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f893c003cc0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 rx=0x7f8944001820 tx=0x7f8944007b90).stop 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 -- 172.21.15.73:0/1868868676 shutdown_connections 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1868868676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f893c003cc0 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 -- 172.21.15.73:0/1868868676 >> 172.21.15.73:0/1868868676 conn(0x7f89540cb310 msgr2=0x7f89540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 -- 172.21.15.73:0/1868868676 shutdown_connections 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.507+0000 7f895a8b1700 1 -- 172.21.15.73:0/1868868676 wait complete. 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.508+0000 7f895a8b1700 1 Processor -- start 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.508+0000 7f895a8b1700 1 -- start start 2023-11-28T03:09:48.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8954157a10 0x7f89540aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89540c0270 0x7f89540c06d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89540c0bc0 con 0x7f893c0038e0 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89541558c0 con 0x7f89540c0270 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f895a8b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89540ab6c0 con 0x7f8954157a10 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53434/0 (socket says 172.21.15.73:53434) 2023-11-28T03:09:48.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 -- 172.21.15.73:0/1626776032 learned_addr learned my addr 172.21.15.73:0/1626776032 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:48.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 -- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8954157a10 msgr2=0x7f89540aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8954157a10 0x7f89540aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:48.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 -- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89540c0270 msgr2=0x7f89540c06d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89540c0270 0x7f89540c06d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:48.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 -- 172.21.15.73:0/1626776032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89540ab9a0 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f8953fff700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f893c00bdd0 tx=0x7f8944016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.509+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8944007ea0 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.510+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944007240 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.510+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440075c0 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.510+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89540c0bc0 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.510+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8954147dd0 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.511+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f895406c860 con 0x7f893c0038e0 2023-11-28T03:09:48.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.511+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f89440068b0 con 0x7f893c0038e0 2023-11-28T03:09:48.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.511+0000 7f893bfff700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 0x7f8940061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.511+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(125..125 src has 1..125) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f894407ab90 con 0x7f893c0038e0 2023-11-28T03:09:48.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.511+0000 7f89537fe700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 0x7f8940061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.512+0000 7f89537fe700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 0x7f8940061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f89540b0ef0 tx=0x7f8948009060).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.515+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f894404be60 con 0x7f893c0038e0 2023-11-28T03:09:48.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.717+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f89540c69a0 con 0x7f893c0038e0 2023-11-28T03:09:48.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.749+0000 7f893bfff700 1 -- 172.21.15.73:0/1626776032 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8944023020 con 0x7f893c0038e0 2023-11-28T03:09:48.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 msgr2=0x7f8940061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 0x7f8940061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f89540b0ef0 tx=0x7f8948009060).stop 2023-11-28T03:09:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 msgr2=0x7f89541478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 rx=0x7f893c00bdd0 tx=0x7f8944016000).stop 2023-11-28T03:09:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 shutdown_connections 2023-11-28T03:09:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f894005f150 0x7f8940061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f89540c0270 0x7f89540c06d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f893c0038e0 0x7f89541478e0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 --2- 172.21.15.73:0/1626776032 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8954157a10 0x7f89540aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.751+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 >> 172.21.15.73:0/1626776032 conn(0x7f89540cb310 msgr2=0x7f89540ca990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.752+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 shutdown_connections 2023-11-28T03:09:48.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.752+0000 7f895a8b1700 1 -- 172.21.15.73:0/1626776032 wait complete. 2023-11-28T03:09:48.764 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:09:48.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:09:48.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.881+0000 7f8f68373700 1 Processor -- start 2023-11-28T03:09:48.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.881+0000 7f8f68373700 1 -- start start 2023-11-28T03:09:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.882+0000 7f8f68373700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f600a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.882+0000 7f8f68373700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f60067770 con 0x7f8f6014e600 2023-11-28T03:09:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.882+0000 7f8f68373700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f600be8d0 con 0x7f8f60158160 2023-11-28T03:09:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.882+0000 7f8f68373700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60159180 con 0x7f8f600a8280 2023-11-28T03:09:48.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f66910700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f6014e600 0x7f8f6014eab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56496/0 (socket says 172.21.15.73:56496) 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f66910700 1 -- 172.21.15.73:0/3650394727 learned_addr learned my addr 172.21.15.73:0/3650394727 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6610f700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f600a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3700863679 0 0) 0x7f8f60067770 con 0x7f8f6014e600 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f4c0033f0 con 0x7f8f6014e600 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1884186595 0 0) 0x7f8f600be8d0 con 0x7f8f60158160 2023-11-28T03:09:48.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f60067770 con 0x7f8f60158160 2023-11-28T03:09:48.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4001395741 0 0) 0x7f8f4c0033f0 con 0x7f8f6014e600 2023-11-28T03:09:48.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 msgr2=0x7f8f600a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f600a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 >> v1:172.21.15.150:6789/0 conn(0x7f8f60158160 legacy=0x7f8f60155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f60158600 con 0x7f8f6014e600 2023-11-28T03:09:48.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8f580017f0 con 0x7f8f6014e600 2023-11-28T03:09:48.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 >> v1:172.21.15.73:6789/0 conn(0x7f8f6014e600 legacy=0x7f8f6014eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f4c003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f4c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f4c004b80 0x7f8f4c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60159180 con 0x7f8f4c003fb0 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00b990 con 0x7f8f4c004b80 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f4c00bb50 con 0x7f8f600a8280 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f4c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f6610f700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f4c003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.883+0000 7f8f66910700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f4c004b80 0x7f8f4c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 -- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 msgr2=0x7f8f4c003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f600a8280 0x7f8f4c003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 -- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f4c004b80 msgr2=0x7f8f4c005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:48.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f4c004b80 0x7f8f4c005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 -- 172.21.15.73:0/3650394727 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f4c0033f0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f4c004690 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f8f50001860 tx=0x7f8f50007bb0).ready entity=mon.0 client_cookie=e35247b3507d0e0d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50007020 con 0x7f8f4c003fb0 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f50007180 con 0x7f8f4c003fb0 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f6510d700 1 -- 172.21.15.73:0/3650394727 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f50007500 con 0x7f8f4c003fb0 2023-11-28T03:09:48.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f68373700 1 -- 172.21.15.73:0/3650394727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 msgr2=0x7f8f4c004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f68373700 1 --2- 172.21.15.73:0/3650394727 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f4c004690 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 rx=0x7f8f50001860 tx=0x7f8f50007bb0).stop 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f66910700 1 -- 172.21.15.73:0/3650394727 reap_dead start 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f68373700 1 -- 172.21.15.73:0/3650394727 shutdown_connections 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.884+0000 7f8f68373700 1 -- 172.21.15.73:0/3650394727 >> 172.21.15.73:0/3650394727 conn(0x7f8f60062700 msgr2=0x7f8f60062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.885+0000 7f8f68373700 1 -- 172.21.15.73:0/3650394727 shutdown_connections 2023-11-28T03:09:48.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.885+0000 7f8f68373700 1 -- 172.21.15.73:0/3650394727 wait complete. 2023-11-28T03:09:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 Processor -- start 2023-11-28T03:09:48.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 -- start start 2023-11-28T03:09:48.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f6014dce0 0x7f8f60157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f600647e0 0x7f8f60064c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c42c0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f60065150 con 0x7f8f600647e0 2023-11-28T03:09:48.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f68373700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f600c2010 con 0x7f8f6014dce0 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.886+0000 7f8f6610f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53442/0 (socket says 172.21.15.73:53442) 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 -- 172.21.15.73:0/3407502353 learned_addr learned my addr 172.21.15.73:0/3407502353 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f6014dce0 0x7f8f60157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f66910700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f600647e0 0x7f8f60064c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 -- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f6014dce0 msgr2=0x7f8f60157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f6014dce0 0x7f8f60157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 -- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f600647e0 msgr2=0x7f8f60064c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f600647e0 0x7f8f60064c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 -- 172.21.15.73:0/3407502353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f600c22f0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f6610f700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f8f5c0119f0 tx=0x7f8f5c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00f940 con 0x7f8f4c003fb0 2023-11-28T03:09:48.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5c00faa0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f600c25d0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5c00fe20 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.887+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f601578f0 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.888+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f6006c860 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.888+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8f5c014650 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.889+0000 7f8f567fc700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 0x7f8f44061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.889+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8f5c07ad90 con 0x7f8f4c003fb0 2023-11-28T03:09:48.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.889+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 0x7f8f44061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:48.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.889+0000 7f8f6590e700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 0x7f8f44061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8f600655d0 tx=0x7f8f50016000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:48.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:48.893+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8f5c04c060 con 0x7f8f4c003fb0 2023-11-28T03:09:49.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.098+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f8f600c9f60 con 0x7f8f4c003fb0 2023-11-28T03:09:49.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.751+0000 7f8f567fc700 1 -- 172.21.15.73:0/3407502353 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8f5c04f680 con 0x7f8f4c003fb0 2023-11-28T03:09:49.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 msgr2=0x7f8f44061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:49.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 0x7f8f44061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f8f600655d0 tx=0x7f8f50016000).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 msgr2=0x7f8f600c1a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 rx=0x7f8f5c0119f0 tx=0x7f8f5c009060).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 shutdown_connections 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8f4405f150 0x7f8f44061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f600647e0 0x7f8f60064c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f4c003fb0 0x7f8f600c1a60 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 --2- 172.21.15.73:0/3407502353 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f6014dce0 0x7f8f60157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 >> 172.21.15.73:0/3407502353 conn(0x7f8f60062700 msgr2=0x7f8f60155360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 shutdown_connections 2023-11-28T03:09:49.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.753+0000 7f8f68373700 1 -- 172.21.15.73:0/3407502353 wait complete. 2023-11-28T03:09:49.767 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:09:49.767 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:49.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.881+0000 7fbd1f2a1700 1 Processor -- start 2023-11-28T03:09:49.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.881+0000 7fbd1f2a1700 1 -- start start 2023-11-28T03:09:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd180aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 0x7fbd180c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd180aa960 0x7fbd180c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180c1490 con 0x7fbd180aa960 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd180a9a90 con 0x7fbd180cae90 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1f2a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd18154ce0 con 0x7fbd180caab0 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd180aa960 0x7fbd180c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbd180aa960 0x7fbd180c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53448/0 (socket says 172.21.15.73:53448) 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 -- 172.21.15.73:0/1410119464 learned_addr learned my addr 172.21.15.73:0/1410119464 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d03d700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd180aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1c83c700 1 --2- 172.21.15.73:0/1410119464 >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 0x7fbd180c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 -- 172.21.15.73:0/1410119464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 msgr2=0x7fbd180aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd180aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 -- 172.21.15.73:0/1410119464 >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 msgr2=0x7fbd180c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/1410119464 >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 0x7fbd180c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 -- 172.21.15.73:0/1410119464 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd180a8e80 con 0x7fbd180aa960 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d03d700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd180aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1c83c700 1 --2- 172.21.15.73:0/1410119464 >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 0x7fbd180c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aa960 0x7fbd180c0fa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fbd180c9070 tx=0x7fbd100090f0).ready entity=mon.0 client_cookie=857c732dc4ebe81e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd0ffff700 1 -- 172.21.15.73:0/1410119464 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000b600 con 0x7fbd180aa960 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd0ffff700 1 -- 172.21.15.73:0/1410119464 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1000b760 con 0x7fbd180aa960 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.882+0000 7fbd0ffff700 1 -- 172.21.15.73:0/1410119464 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000bae0 con 0x7fbd180aa960 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/1410119464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aa960 msgr2=0x7fbd180c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aa960 0x7fbd180c0fa0 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 rx=0x7fbd180c9070 tx=0x7fbd100090f0).stop 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/1410119464 shutdown_connections 2023-11-28T03:09:49.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aa960 0x7fbd180c0fa0 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/1410119464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd180aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/1410119464 >> v2:172.21.15.150:3300/0 conn(0x7fbd180cae90 0x7fbd180c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/1410119464 >> 172.21.15.73:0/1410119464 conn(0x7fbd18094920 msgr2=0x7fbd1806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/1410119464 shutdown_connections 2023-11-28T03:09:49.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.883+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/1410119464 wait complete. 2023-11-28T03:09:49.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.884+0000 7fbd1f2a1700 1 Processor -- start 2023-11-28T03:09:49.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.884+0000 7fbd1f2a1700 1 -- start start 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd181541a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd180cae90 0x7fbd181592d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd18159ab0 con 0x7fbd180aafa0 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd18159d90 con 0x7fbd180cae90 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1f2a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd1815a070 con 0x7fbd180caab0 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53452/0 (socket says 172.21.15.73:53452) 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 -- 172.21.15.73:0/3045125190 learned_addr learned my addr 172.21.15.73:0/3045125190 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d03d700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd181541a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1c83c700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd180cae90 0x7fbd181592d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 -- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 msgr2=0x7fbd181541a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd181541a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 -- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd180cae90 msgr2=0x7fbd181592d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd180cae90 0x7fbd181592d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.885+0000 7fbd1d83e700 1 -- 172.21.15.73:0/3045125190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd10009370 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd1d83e700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fbd18083c30 tx=0x7fbd10017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd10006d90 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd1000bea0 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd1000bae0 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd1815a350 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.886+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd181597c0 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.887+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd1815a870 con 0x7fbd180aafa0 2023-11-28T03:09:49.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.887+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbd1001b8f0 con 0x7fbd180aafa0 2023-11-28T03:09:49.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.887+0000 7fbd0d7fa700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 0x7fbcfc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:49.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.888+0000 7fbd1d03d700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 0x7fbcfc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:49.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.888+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbd1007bfd0 con 0x7fbd180aafa0 2023-11-28T03:09:49.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.888+0000 7fbd1d03d700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 0x7fbcfc061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbd140077a0 tx=0x7fbd14009a20).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:49.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:49.892+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbd1815a870 con 0x7fbd180aafa0 2023-11-28T03:09:50.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.093+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd1815a870 con 0x7fbd180aafa0 2023-11-28T03:09:50.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.094+0000 7fbd0d7fa700 1 -- 172.21.15.73:0/3045125190 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7fbd1815a870 con 0x7fbd180aafa0 2023-11-28T03:09:50.095 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:50.095 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:09:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 msgr2=0x7fbcfc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 0x7fbcfc061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7fbd140077a0 tx=0x7fbd14009a20).stop 2023-11-28T03:09:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 msgr2=0x7fbd18167b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 rx=0x7fbd18083c30 tx=0x7fbd10017000).stop 2023-11-28T03:09:50.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 shutdown_connections 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbcfc05f150 0x7fbcfc061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbd180aafa0 0x7fbd18167b90 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbd180caab0 0x7fbd181541a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 --2- 172.21.15.73:0/3045125190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbd180cae90 0x7fbd181592d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 >> 172.21.15.73:0/3045125190 conn(0x7fbd18094920 msgr2=0x7fbd180a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 shutdown_connections 2023-11-28T03:09:50.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.096+0000 7fbd1f2a1700 1 -- 172.21.15.73:0/3045125190 wait complete. 2023-11-28T03:09:50.099 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:09:50.109 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:09:50.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.220+0000 7fd2bffff700 1 Processor -- start 2023-11-28T03:09:50.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.220+0000 7fd2bffff700 1 -- start start 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2bffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2c0157660 0x7fd2c0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2bffff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c00c12c0 con 0x7fd2c00c0180 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2bffff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2c00c06b0 con 0x7fd2c00c1750 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2bffff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2c00c0990 con 0x7fd2c0157660 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2c0157660 0x7fd2c0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd2c0157660 0x7fd2c0157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53462/0 (socket says 172.21.15.73:53462) 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2b67fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd2c00c0180 0x7fd2c01589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56498/0 (socket says 172.21.15.73:56498) 2023-11-28T03:09:50.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 -- 172.21.15.73:0/2128552710 learned_addr learned my addr 172.21.15.73:0/2128552710 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 -- 172.21.15.73:0/2128552710 >> v1:172.21.15.150:6789/0 conn(0x7fd2c00c1750 legacy=0x7fd2c00648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 -- 172.21.15.73:0/2128552710 >> v1:172.21.15.73:6789/0 conn(0x7fd2c00c0180 legacy=0x7fd2c01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2be7fc700 1 -- 172.21.15.73:0/2128552710 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2589480374 0 0) 0x7fd2c00c12c0 con 0x7fd2c00c0180 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.221+0000 7fd2beffd700 1 -- 172.21.15.73:0/2128552710 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2c00c0c70 con 0x7fd2c0157660 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2beffd700 1 --2- 172.21.15.73:0/2128552710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c0157660 0x7fd2c0157a40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fd2b0006fd0 tx=0x7fd2b00090f0).ready entity=mon.0 client_cookie=af5bf4b1bece50df server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2be7fc700 1 -- 172.21.15.73:0/2128552710 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000b600 con 0x7fd2c0157660 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2be7fc700 1 -- 172.21.15.73:0/2128552710 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000b760 con 0x7fd2c0157660 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2be7fc700 1 -- 172.21.15.73:0/2128552710 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000bae0 con 0x7fd2c0157660 2023-11-28T03:09:50.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 -- 172.21.15.73:0/2128552710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c0157660 msgr2=0x7fd2c0157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 --2- 172.21.15.73:0/2128552710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c0157660 0x7fd2c0157a40 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 rx=0x7fd2b0006fd0 tx=0x7fd2b00090f0).stop 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 -- 172.21.15.73:0/2128552710 shutdown_connections 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 --2- 172.21.15.73:0/2128552710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c0157660 0x7fd2c0157a40 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 -- 172.21.15.73:0/2128552710 >> 172.21.15.73:0/2128552710 conn(0x7fd2c00cb000 msgr2=0x7fd2c00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 -- 172.21.15.73:0/2128552710 shutdown_connections 2023-11-28T03:09:50.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.222+0000 7fd2bffff700 1 -- 172.21.15.73:0/2128552710 wait complete. 2023-11-28T03:09:50.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.223+0000 7fd2bffff700 1 Processor -- start 2023-11-28T03:09:50.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 -- start start 2023-11-28T03:09:50.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 0x7fd2c00aa1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2c0157660 0x7fd2c0164c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0156b30 con 0x7fd2c00c0180 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0064140 con 0x7fd2c00c1750 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2bffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2c0154840 con 0x7fd2c0157660 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2beffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2beffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53478/0 (socket says 172.21.15.73:53478) 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.224+0000 7fd2beffd700 1 -- 172.21.15.73:0/4207830732 learned_addr learned my addr 172.21.15.73:0/4207830732 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2b67fc700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 0x7fd2c00aa1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2bf7fe700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2c0157660 0x7fd2c0164c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 -- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2c0157660 msgr2=0x7fd2c0164c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2c0157660 0x7fd2c0164c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 -- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 msgr2=0x7fd2c00aa1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 0x7fd2c00aa1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 -- 172.21.15.73:0/4207830732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2b0009370 con 0x7fd2c00c0180 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2b67fc700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 0x7fd2c00aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2beffd700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fd2c0084570 tx=0x7fd2b0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b000b8d0 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2b000ba30 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2c0156fb0 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2b001b660 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.225+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2c0156750 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.226+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2c0156c70 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.226+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd2b001b9d0 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.227+0000 7fd2b77fe700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 0x7fd2a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.227+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd2b007be20 con 0x7fd2c00c0180 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.227+0000 7fd2b67fc700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 0x7fd2a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.227+0000 7fd2b67fc700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 0x7fd2a0061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd2c00c69a0 tx=0x7fd2a8007b10).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.231+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd2b004d0f0 con 0x7fd2c00c0180 2023-11-28T03:09:50.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.395+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2c01562a0 con 0x7fd2c00c0180 2023-11-28T03:09:50.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.396+0000 7fd2b77fe700 1 -- 172.21.15.73:0/4207830732 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fd2b001f070 con 0x7fd2c00c0180 2023-11-28T03:09:50.397 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:50.398 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":126,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:09:48.744434+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:09:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 msgr2=0x7fd2a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 0x7fd2a0061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fd2c00c69a0 tx=0x7fd2a8007b10).stop 2023-11-28T03:09:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 msgr2=0x7fd2c0154350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 rx=0x7fd2c0084570 tx=0x7fd2b0017000).stop 2023-11-28T03:09:50.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 shutdown_connections 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd2a005f150 0x7fd2a0061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd2c0157660 0x7fd2c0164c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2c00c0180 0x7fd2c0154350 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 --2- 172.21.15.73:0/4207830732 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd2c00c1750 0x7fd2c00aa1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 >> 172.21.15.73:0/4207830732 conn(0x7fd2c00cb000 msgr2=0x7fd2c01510b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 shutdown_connections 2023-11-28T03:09:50.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.398+0000 7fd2bffff700 1 -- 172.21.15.73:0/4207830732 wait complete. 2023-11-28T03:09:50.411 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:09:50.411 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:09:50.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.524+0000 7f0b181d0700 1 Processor -- start 2023-11-28T03:09:50.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.524+0000 7f0b181d0700 1 -- start start 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b181d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0b101542e0 0x7f0b101546c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b181d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b10158a80 0x7f0b100ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b181d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b100ca200 con 0x7f0b100cab80 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b181d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b100cafd0 con 0x7f0b10158a80 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b181d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b100c95f0 con 0x7f0b101542e0 2023-11-28T03:09:50.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b10158a80 0x7f0b100ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b10158a80 0x7f0b100ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53482/0 (socket says 172.21.15.73:53482) 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 -- 172.21.15.73:0/3747008778 learned_addr learned my addr 172.21.15.73:0/3747008778 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b15f6c700 1 --2- 172.21.15.73:0/3747008778 >> v2:172.21.15.150:3300/0 conn(0x7f0b101542e0 0x7f0b101546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 -- 172.21.15.73:0/3747008778 >> v2:172.21.15.150:3300/0 conn(0x7f0b101542e0 msgr2=0x7f0b101546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3747008778 >> v2:172.21.15.150:3300/0 conn(0x7f0b101542e0 0x7f0b101546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 -- 172.21.15.73:0/3747008778 >> v1:172.21.15.150:6789/0 conn(0x7f0b100cab80 legacy=0x7f0b100ab0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 -- 172.21.15.73:0/3747008778 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b100c98d0 con 0x7f0b10158a80 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.525+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3747008778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b10158a80 0x7f0b100ca690 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f0b100cc210 tx=0x7f0b000090f0).ready entity=mon.0 client_cookie=7477b2dc6b2248f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b14f6a700 1 -- 172.21.15.73:0/3747008778 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000b520 con 0x7f0b10158a80 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b14f6a700 1 -- 172.21.15.73:0/3747008778 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0000b680 con 0x7f0b10158a80 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b14f6a700 1 -- 172.21.15.73:0/3747008778 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000ba00 con 0x7f0b10158a80 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 -- 172.21.15.73:0/3747008778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b10158a80 msgr2=0x7f0b100ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3747008778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b10158a80 0x7f0b100ca690 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 rx=0x7f0b100cc210 tx=0x7f0b000090f0).stop 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 -- 172.21.15.73:0/3747008778 shutdown_connections 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3747008778 >> v2:172.21.15.150:3300/0 conn(0x7f0b101542e0 0x7f0b101546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3747008778 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b10158a80 0x7f0b100ca690 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 -- 172.21.15.73:0/3747008778 >> 172.21.15.73:0/3747008778 conn(0x7f0b100649c0 msgr2=0x7f0b10064dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 -- 172.21.15.73:0/3747008778 shutdown_connections 2023-11-28T03:09:50.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.526+0000 7f0b181d0700 1 -- 172.21.15.73:0/3747008778 wait complete. 2023-11-28T03:09:50.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.527+0000 7f0b181d0700 1 Processor -- start 2023-11-28T03:09:50.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.527+0000 7f0b181d0700 1 -- start start 2023-11-28T03:09:50.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 0x7f0b100b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 0x7f0b100a9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b100b11b0 con 0x7f0b100cab80 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10168ca0 con 0x7f0b101542e0 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b181d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b10168390 con 0x7f0b10158a80 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53488/0 (socket says 172.21.15.73:53488) 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 -- 172.21.15.73:0/3032793468 learned_addr learned my addr 172.21.15.73:0/3032793468 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 0x7f0b100b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b1676d700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 0x7f0b100a9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 -- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 msgr2=0x7f0b100a9700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 0x7f0b100a9700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 -- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 msgr2=0x7f0b100b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 0x7f0b100b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b15f6c700 1 -- 172.21.15.73:0/3032793468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b00009370 con 0x7f0b100cab80 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 0x7f0b100b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.528+0000 7f0b1676d700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 0x7f0b100a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b15f6c700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f0b0c00e980 tx=0x7f0b0c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00c850 con 0x7f0b100cab80 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b0c00c9b0 con 0x7f0b100cab80 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b0c00cd30 con 0x7f0b100cab80 2023-11-28T03:09:50.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b100b09a0 con 0x7f0b100cab80 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.529+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b10168930 con 0x7f0b100cab80 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.530+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b10152610 con 0x7f0b100cab80 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.530+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0b0c012600 con 0x7f0b100cab80 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.531+0000 7f0b05ffb700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 0x7f0af4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.531+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0b0c07a7a0 con 0x7f0b100cab80 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.531+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 0x7f0af4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.537 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.531+0000 7f0b1576b700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 0x7f0af4061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0b100af9c0 tx=0x7f0b00017000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.535+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0b0c04ba70 con 0x7f0b100cab80 2023-11-28T03:09:50.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.694+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0b100c9bb0 con 0x7f0af405f150 2023-11-28T03:09:50.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.704+0000 7f0b05ffb700 1 -- 172.21.15.73:0/3032793468 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353236 (secure 0 0 0) 0x7f0b100c9bb0 con 0x7f0af405f150 2023-11-28T03:09:50.709 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:50.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 msgr2=0x7f0af4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 0x7f0af4061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f0b100af9c0 tx=0x7f0b00017000).stop 2023-11-28T03:09:50.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 msgr2=0x7f0b100b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 rx=0x7f0b0c00e980 tx=0x7f0b0c009060).stop 2023-11-28T03:09:50.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 shutdown_connections 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0af405f150 0x7f0af4061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b10158a80 0x7f0b100a9700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b100cab80 0x7f0b100b0330 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 --2- 172.21.15.73:0/3032793468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b101542e0 0x7f0b100b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 >> 172.21.15.73:0/3032793468 conn(0x7f0b100649c0 msgr2=0x7f0b10151f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 shutdown_connections 2023-11-28T03:09:50.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.707+0000 7f0b181d0700 1 -- 172.21.15.73:0/3032793468 wait complete. 2023-11-28T03:09:50.715 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:09:50.720 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:09:49.773048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2421348,"kb_used_data":11812,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752553372,"statfs":{"total":773094113280,"available":770614652928,"internally_reserved":0,"allocated":12095488,"data_stored":6017119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.560879"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369372+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365278+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368673+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368605+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364443+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368661+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368544+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364792+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367095+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365829+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369140+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366373+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368253+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.376360+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370856+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369474+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364740+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369025+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374317+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365064+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373899+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364709+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364722+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368751+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366569+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364670+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370707+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364484+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364340+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373871+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368283+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368192+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368705+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374385+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365285+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370208+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368677+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367186+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364493+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369798+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366689+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364307+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369982+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366518+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368210+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368382+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374352+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364750+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369024+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.363898+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365339+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369869+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365009+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374623+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364460+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374654+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366070+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456983+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:09:44.456983+0000","last_peered":"2023-11-28T03:09:44.456983+0000","last_clean":"2023-11-28T03:09:44.456983+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:09:44.456983+0000","last_undegraded":"2023-11-28T03:09:44.456983+0000","last_fullsized":"2023-11-28T03:09:44.456983+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366105+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367145+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456816+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:09:44.456816+0000","last_peered":"2023-11-28T03:09:44.456816+0000","last_clean":"2023-11-28T03:09:44.456816+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:09:44.456816+0000","last_undegraded":"2023-11-28T03:09:44.456816+0000","last_fullsized":"2023-11-28T03:09:44.456816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373990+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457268+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:09:44.457268+0000","last_peered":"2023-11-28T03:09:44.457268+0000","last_clean":"2023-11-28T03:09:44.457268+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:09:44.457268+0000","last_undegraded":"2023-11-28T03:09:44.457268+0000","last_fullsized":"2023-11-28T03:09:44.457268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369550+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457272+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:09:44.457272+0000","last_peered":"2023-11-28T03:09:44.457272+0000","last_clean":"2023-11-28T03:09:44.457272+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:09:44.457272+0000","last_undegraded":"2023-11-28T03:09:44.457272+0000","last_fullsized":"2023-11-28T03:09:44.457272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374287+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457279+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:09:44.457279+0000","last_peered":"2023-11-28T03:09:44.457279+0000","last_clean":"2023-11-28T03:09:44.457279+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:09:44.457279+0000","last_undegraded":"2023-11-28T03:09:44.457279+0000","last_fullsized":"2023-11-28T03:09:44.457279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374286+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457012+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:09:44.457012+0000","last_peered":"2023-11-28T03:09:44.457012+0000","last_clean":"2023-11-28T03:09:44.457012+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:09:44.457012+0000","last_undegraded":"2023-11-28T03:09:44.457012+0000","last_fullsized":"2023-11-28T03:09:44.457012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374262+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.357942+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:09:44.357942+0000","last_peered":"2023-11-28T03:09:44.357942+0000","last_clean":"2023-11-28T03:09:44.357942+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:09:44.357942+0000","last_undegraded":"2023-11-28T03:09:44.357942+0000","last_fullsized":"2023-11-28T03:09:44.357942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.34499999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46600000000000003,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.628},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.71399999999999997,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":1.506,"15min":1.506},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.53300000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58699999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.60399999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.64500000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.54700000000000004,"5min":0.67800000000000005,"15min":0.70399999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.371},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.67100000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.307,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.64900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.628,"15min":0.63400000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.65200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.75600000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.97799999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.91900000000000004,"15min":0.99299999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.629,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.94199999999999995,"15min":1.038},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78700000000000003,"5min":0.89400000000000002,"15min":0.94099999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":1.024},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.88300000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.217,"15min":0.184},"max":{"1min":0.46100000000000002,"5min":0.56599999999999995,"15min":0.56999999999999995},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47499999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.377,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.41699999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.61499999999999999,"5min":0.621,"15min":0.65400000000000003},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":1.0489999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":1.679,"15min":1.679},"last":0.498},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.97499999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.97399999999999998,"15min":1.046},"last":0.69299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34799999999999998,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.48499999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.41699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.53800000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34599999999999997,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.5,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.58499999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.28499999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.41799999999999998},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.58199999999999996,"15min":0.67400000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88600000000000001,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.98599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":1.004},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.875,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.65500000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.63},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575052,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52300000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35699999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.379}]},{"osd":1,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60099999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.42099999999999999},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.68200000000000005},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82599999999999996,"5min":1.581,"15min":1.581},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.94199999999999995},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.97899999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.93600000000000005},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":0.90900000000000003,"15min":0.996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.63}]},{"osd":7,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":0.93500000000000005,"15min":0.97199999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52100000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.505}]},{"osd":1,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53600000000000003,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.84599999999999997,"15min":0.878},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.496,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68400000000000005,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.77900000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70099999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47599999999999998,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78200000000000003,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:09:50.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:09:50.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.872+0000 7f4043150700 1 Processor -- start 2023-11-28T03:09:50.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.872+0000 7f4043150700 1 -- start start 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4043150700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f403c0a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4043150700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0bbed0 con 0x7f403c152230 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4043150700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0bc1b0 con 0x7f403c0686e0 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4043150700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a4290 con 0x7f403c156a80 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4040eec700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f403c152230 0x7f403c066250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56514/0 (socket says 172.21.15.73:56514) 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4040eec700 1 -- 172.21.15.73:0/185527721 learned_addr learned my addr 172.21.15.73:0/185527721 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4066001752 0 0) 0x7f403c0bbed0 con 0x7f403c152230 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40240033f0 con 0x7f403c152230 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403bfff700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f403c0a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1138132322 0 0) 0x7f40240033f0 con 0x7f403c152230 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 msgr2=0x7f403c0a3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f403c0a3da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 >> v1:172.21.15.150:6789/0 conn(0x7f403c0686e0 legacy=0x7f403c0bcc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0bc490 con 0x7f403c152230 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f402c0017f0 con 0x7f403c152230 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 >> v1:172.21.15.73:6789/0 conn(0x7f403c152230 legacy=0x7f403c066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f4024003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f4024004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a4290 con 0x7f4024003910 2023-11-28T03:09:50.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f402400b910 con 0x7f4024004b80 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402400bad0 con 0x7f403c156a80 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f4040eec700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f4024003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.873+0000 7f403bfff700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f4024004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f40416ed700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 -- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 msgr2=0x7f4024004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c156a80 0x7f4024004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 -- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 msgr2=0x7f4024005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 -- 172.21.15.73:0/185527721 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240033f0 con 0x7f4024003910 2023-11-28T03:09:50.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f40416ed700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4040eec700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f4024003d30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f402c00c480 tx=0x7f402c00c790).ready entity=mon.0 client_cookie=926e3ed8be0b631 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c002030 con 0x7f4024003910 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c00ce20 con 0x7f4024003910 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f403b7fe700 1 -- 172.21.15.73:0/185527721 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c005d70 con 0x7f4024003910 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4043150700 1 -- 172.21.15.73:0/185527721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 msgr2=0x7f4024003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4043150700 1 --2- 172.21.15.73:0/185527721 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f4024003d30 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 rx=0x7f402c00c480 tx=0x7f402c00c790).stop 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f40416ed700 1 -- 172.21.15.73:0/185527721 reap_dead start 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4043150700 1 -- 172.21.15.73:0/185527721 shutdown_connections 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.874+0000 7f4043150700 1 -- 172.21.15.73:0/185527721 >> 172.21.15.73:0/185527721 conn(0x7f403c0631d0 msgr2=0x7f403c0c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.875+0000 7f4043150700 1 -- 172.21.15.73:0/185527721 shutdown_connections 2023-11-28T03:09:50.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.875+0000 7f4043150700 1 -- 172.21.15.73:0/185527721 wait complete. 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.876+0000 7f4043150700 1 Processor -- start 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.876+0000 7f4043150700 1 -- start start 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.876+0000 7f4043150700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.876+0000 7f4043150700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 0x7f403c0a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 0x7f403c0bff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0ac9d0 con 0x7f4024003910 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a7320 con 0x7f403c0a69b0 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0c0840 con 0x7f403c0bfb10 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53500/0 (socket says 172.21.15.73:53500) 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 -- 172.21.15.73:0/1865483194 learned_addr learned my addr 172.21.15.73:0/1865483194 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:50.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f403bfff700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 0x7f403c0a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f40416ed700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 0x7f403c0bff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 -- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 msgr2=0x7f403c0bff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 0x7f403c0bff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 -- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 msgr2=0x7f403c0a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 0x7f403c0a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 -- 172.21.15.73:0/1865483194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0c14d0 con 0x7f4024003910 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f40416ed700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 0x7f403c0bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f403bfff700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 0x7f403c0a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4040eec700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f402c00c7d0 tx=0x7f402c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c004a30 con 0x7f4024003910 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c004b90 con 0x7f4024003910 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c0c17b0 con 0x7f4024003910 2023-11-28T03:09:50.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c020450 con 0x7f4024003910 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.877+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0c1d70 con 0x7f4024003910 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.878+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c06c860 con 0x7f4024003910 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.878+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f402c013450 con 0x7f4024003910 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.879+0000 7f4038ff9700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 0x7f4028061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.879+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f402c07b4c0 con 0x7f4024003910 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.879+0000 7f403bfff700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 0x7f4028061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:50.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.879+0000 7f403bfff700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 0x7f4028061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4030006c40 tx=0x7f4030008040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:50.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:50.883+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f402c04c7e0 con 0x7f4024003910 2023-11-28T03:09:51.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.042+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f403c0bc770 con 0x7f402805f150 2023-11-28T03:09:51.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.051+0000 7f4038ff9700 1 -- 172.21.15.73:0/1865483194 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353236 (secure 0 0 0) 0x7f403c0bc770 con 0x7f402805f150 2023-11-28T03:09:51.057 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.054+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 msgr2=0x7f4028061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.054+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 0x7f4028061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f4030006c40 tx=0x7f4030008040).stop 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.054+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 msgr2=0x7f403c0ac4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.054+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 rx=0x7f402c00c7d0 tx=0x7f402c015000).stop 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 shutdown_connections 2023-11-28T03:09:51.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f150 0x7f4028061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0bfb10 0x7f403c0bff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4024003910 0x7f403c0ac4e0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 --2- 172.21.15.73:0/1865483194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f403c0a69b0 0x7f403c0a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 >> 172.21.15.73:0/1865483194 conn(0x7f403c0631d0 msgr2=0x7f403c0a4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 shutdown_connections 2023-11-28T03:09:51.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.055+0000 7f4043150700 1 -- 172.21.15.73:0/1865483194 wait complete. 2023-11-28T03:09:51.062 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:09:51.067 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:09:49.773048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2421348,"kb_used_data":11812,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752553372,"statfs":{"total":773094113280,"available":770614652928,"internally_reserved":0,"allocated":12095488,"data_stored":6017119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.560879"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369372+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365278+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368673+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368605+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364443+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368661+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368544+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364792+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367095+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365829+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369140+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366373+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368253+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.376360+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370856+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369474+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364740+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369025+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374317+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365064+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373899+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364709+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364722+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368751+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366569+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364670+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370707+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364484+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364340+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373871+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368283+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368192+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368705+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374385+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365285+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370208+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368677+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367186+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364493+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369798+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366689+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364307+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369982+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366518+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368210+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368382+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374352+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364750+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369024+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.363898+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365339+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369869+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365009+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374623+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364460+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374654+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366070+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456983+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:09:44.456983+0000","last_peered":"2023-11-28T03:09:44.456983+0000","last_clean":"2023-11-28T03:09:44.456983+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:09:44.456983+0000","last_undegraded":"2023-11-28T03:09:44.456983+0000","last_fullsized":"2023-11-28T03:09:44.456983+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366105+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367145+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456816+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:09:44.456816+0000","last_peered":"2023-11-28T03:09:44.456816+0000","last_clean":"2023-11-28T03:09:44.456816+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:09:44.456816+0000","last_undegraded":"2023-11-28T03:09:44.456816+0000","last_fullsized":"2023-11-28T03:09:44.456816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373990+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457268+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:09:44.457268+0000","last_peered":"2023-11-28T03:09:44.457268+0000","last_clean":"2023-11-28T03:09:44.457268+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:09:44.457268+0000","last_undegraded":"2023-11-28T03:09:44.457268+0000","last_fullsized":"2023-11-28T03:09:44.457268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369550+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457272+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:09:44.457272+0000","last_peered":"2023-11-28T03:09:44.457272+0000","last_clean":"2023-11-28T03:09:44.457272+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:09:44.457272+0000","last_undegraded":"2023-11-28T03:09:44.457272+0000","last_fullsized":"2023-11-28T03:09:44.457272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374287+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457279+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:09:44.457279+0000","last_peered":"2023-11-28T03:09:44.457279+0000","last_clean":"2023-11-28T03:09:44.457279+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:09:44.457279+0000","last_undegraded":"2023-11-28T03:09:44.457279+0000","last_fullsized":"2023-11-28T03:09:44.457279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374286+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457012+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:09:44.457012+0000","last_peered":"2023-11-28T03:09:44.457012+0000","last_clean":"2023-11-28T03:09:44.457012+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:09:44.457012+0000","last_undegraded":"2023-11-28T03:09:44.457012+0000","last_fullsized":"2023-11-28T03:09:44.457012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374262+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.357942+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:09:44.357942+0000","last_peered":"2023-11-28T03:09:44.357942+0000","last_clean":"2023-11-28T03:09:44.357942+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:09:44.357942+0000","last_undegraded":"2023-11-28T03:09:44.357942+0000","last_fullsized":"2023-11-28T03:09:44.357942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.34499999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46600000000000003,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.628},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.71399999999999997,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":1.506,"15min":1.506},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.53300000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58699999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.60399999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.64500000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.54700000000000004,"5min":0.67800000000000005,"15min":0.70399999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.371},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.67100000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.307,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.64900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.628,"15min":0.63400000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.65200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.75600000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.97799999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.91900000000000004,"15min":0.99299999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.629,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.94199999999999995,"15min":1.038},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78700000000000003,"5min":0.89400000000000002,"15min":0.94099999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":1.024},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.88300000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.217,"15min":0.184},"max":{"1min":0.46100000000000002,"5min":0.56599999999999995,"15min":0.56999999999999995},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47499999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.377,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.41699999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.61499999999999999,"5min":0.621,"15min":0.65400000000000003},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":1.0489999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":1.679,"15min":1.679},"last":0.498},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.97499999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.97399999999999998,"15min":1.046},"last":0.69299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34799999999999998,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.48499999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.41699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.53800000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34599999999999997,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.5,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.58499999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.28499999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.41799999999999998},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.58199999999999996,"15min":0.67400000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88600000000000001,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.98599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":1.004},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.875,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.65500000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.63},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575052,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52300000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35699999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.379}]},{"osd":1,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60099999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.42099999999999999},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.68200000000000005},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82599999999999996,"5min":1.581,"15min":1.581},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.94199999999999995},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.97899999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.93600000000000005},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":0.90900000000000003,"15min":0.996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.63}]},{"osd":7,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":0.93500000000000005,"15min":0.97199999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52100000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.505}]},{"osd":1,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53600000000000003,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.84599999999999997,"15min":0.878},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.496,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68400000000000005,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.77900000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70099999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47599999999999998,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78200000000000003,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:09:51.073 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:09:51.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 Processor -- start 2023-11-28T03:09:51.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 -- start start 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f47981548c0 0x7f479806a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980c69a0 con 0x7f47981544e0 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47980c13a0 con 0x7f47980c6e30 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.221+0000 7f479d4fc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47980c1680 con 0x7f47981548c0 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f4796ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f47981544e0 0x7f47980c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56524/0 (socket says 172.21.15.73:56524) 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f4796ffd700 1 -- 172.21.15.73:0/4075617210 learned_addr learned my addr 172.21.15.73:0/4075617210 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f478e7fc700 1 --2- 172.21.15.73:0/4075617210 >> v2:172.21.15.150:3300/0 conn(0x7f47981548c0 0x7f479806a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 428365311 0 0) 0x7f47980c69a0 con 0x7f47981544e0 2023-11-28T03:09:51.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f47840033f0 con 0x7f47981544e0 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f478e7fc700 1 -- 172.21.15.73:0/4075617210 >> v1:172.21.15.150:6789/0 conn(0x7f47980c6e30 legacy=0x7f47980a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1987192992 0 0) 0x7f47980c13a0 con 0x7f47980c6e30 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f478e7fc700 1 -- 172.21.15.73:0/4075617210 >> v1:172.21.15.73:6789/0 conn(0x7f47981544e0 legacy=0x7f47980c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f478e7fc700 1 -- 172.21.15.73:0/4075617210 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980c6630 con 0x7f47981548c0 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f478e7fc700 1 --2- 172.21.15.73:0/4075617210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981548c0 0x7f479806a610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f4780009760 tx=0x7f4780009060).ready entity=mon.1 client_cookie=bb5c047dbee893cc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000f510 con 0x7f47981548c0 2023-11-28T03:09:51.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f478000f670 con 0x7f47981548c0 2023-11-28T03:09:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.222+0000 7f47967fc700 1 -- 172.21.15.73:0/4075617210 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f478000f9f0 con 0x7f47981548c0 2023-11-28T03:09:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 -- 172.21.15.73:0/4075617210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981548c0 msgr2=0x7f479806a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 --2- 172.21.15.73:0/4075617210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981548c0 0x7f479806a610 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 rx=0x7f4780009760 tx=0x7f4780009060).stop 2023-11-28T03:09:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 -- 172.21.15.73:0/4075617210 shutdown_connections 2023-11-28T03:09:51.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 --2- 172.21.15.73:0/4075617210 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981548c0 0x7f479806a610 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 -- 172.21.15.73:0/4075617210 >> 172.21.15.73:0/4075617210 conn(0x7f4798085020 msgr2=0x7f4798094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 -- 172.21.15.73:0/4075617210 shutdown_connections 2023-11-28T03:09:51.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.223+0000 7f479d4fc700 1 -- 172.21.15.73:0/4075617210 wait complete. 2023-11-28T03:09:51.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.224+0000 7f479d4fc700 1 Processor -- start 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.224+0000 7f479d4fc700 1 -- start start 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 0x7f47980acfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981544e0 0x7f47981658e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47980ad4a0 con 0x7f47981548c0 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4798097eb0 con 0x7f47981544e0 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f479d4fc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47980950a0 con 0x7f47980c6e30 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53506/0 (socket says 172.21.15.73:53506) 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 -- 172.21.15.73:0/34332342 learned_addr learned my addr 172.21.15.73:0/34332342 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f478e7fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981544e0 0x7f47981658e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f4796ffd700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 0x7f47980acfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 -- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 msgr2=0x7f47980acfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 0x7f47980acfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 -- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981544e0 msgr2=0x7f47981658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981544e0 0x7f47981658e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f47977fe700 1 -- 172.21.15.73:0/34332342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800092a0 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.225+0000 7f4796ffd700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 0x7f47980acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f47977fe700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f479000c810 tx=0x7f47900014a0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790010510 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4790010690 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4798095380 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4790010a50 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.226+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4798095940 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.227+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4798095cb0 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.227+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f479001b400 con 0x7f47981548c0 2023-11-28T03:09:51.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.227+0000 7f478f7fe700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 0x7f4778061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.227+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4790079fe0 con 0x7f47981548c0 2023-11-28T03:09:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.227+0000 7f4796ffd700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 0x7f4778061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.228+0000 7f4796ffd700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 0x7f4778061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4788000c00 tx=0x7f4788009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.231+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f479004b250 con 0x7f47981548c0 2023-11-28T03:09:51.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.393+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f47980c6250 con 0x7f477805f150 2023-11-28T03:09:51.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.401+0000 7f478f7fe700 1 -- 172.21.15.73:0/34332342 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353236 (secure 0 0 0) 0x7f47980c6250 con 0x7f477805f150 2023-11-28T03:09:51.407 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:51.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 msgr2=0x7f4778061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 0x7f4778061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4788000c00 tx=0x7f4788009040).stop 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 msgr2=0x7f47980973b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 rx=0x7f479000c810 tx=0x7f47900014a0).stop 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 shutdown_connections 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f477805f150 0x7f4778061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f47981548c0 0x7f47980973b0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f47980c6e30 0x7f47980acfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 --2- 172.21.15.73:0/34332342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f47981544e0 0x7f47981658e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 >> 172.21.15.73:0/34332342 conn(0x7f4798085020 msgr2=0x7f47980cba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 shutdown_connections 2023-11-28T03:09:51.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.405+0000 7f479d4fc700 1 -- 172.21.15.73:0/34332342 wait complete. 2023-11-28T03:09:51.412 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:09:51.418 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:09:49.773048+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2421348,"kb_used_data":11812,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752553372,"statfs":{"total":773094113280,"available":770614652928,"internally_reserved":0,"allocated":12095488,"data_stored":6017119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.560879"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369372+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365278+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368673+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368605+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364443+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368661+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368544+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364792+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367095+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365829+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369140+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366373+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368253+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.376360+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370856+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369474+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364740+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369025+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374317+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365064+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373899+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364709+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364722+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368751+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366569+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364670+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370707+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364484+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364340+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373871+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368283+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368192+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368705+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374385+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365285+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.370208+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368677+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367186+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364493+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369798+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366689+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364307+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369982+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366518+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368210+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.368382+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374352+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364750+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369024+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.363898+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365339+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369869+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365009+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374623+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364460+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374654+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366070+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456983+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:09:44.456983+0000","last_peered":"2023-11-28T03:09:44.456983+0000","last_clean":"2023-11-28T03:09:44.456983+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:09:44.456983+0000","last_undegraded":"2023-11-28T03:09:44.456983+0000","last_fullsized":"2023-11-28T03:09:44.456983+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366105+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367145+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":101,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.456816+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:09:44.456816+0000","last_peered":"2023-11-28T03:09:44.456816+0000","last_clean":"2023-11-28T03:09:44.456816+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:09:44.456816+0000","last_undegraded":"2023-11-28T03:09:44.456816+0000","last_fullsized":"2023-11-28T03:09:44.456816+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.373990+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":96,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457268+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:09:44.457268+0000","last_peered":"2023-11-28T03:09:44.457268+0000","last_clean":"2023-11-28T03:09:44.457268+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:09:44.457268+0000","last_undegraded":"2023-11-28T03:09:44.457268+0000","last_fullsized":"2023-11-28T03:09:44.457268+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.369550+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457272+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:09:44.457272+0000","last_peered":"2023-11-28T03:09:44.457272+0000","last_clean":"2023-11-28T03:09:44.457272+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:09:44.457272+0000","last_undegraded":"2023-11-28T03:09:44.457272+0000","last_fullsized":"2023-11-28T03:09:44.457272+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374287+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457279+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:09:44.457279+0000","last_peered":"2023-11-28T03:09:44.457279+0000","last_clean":"2023-11-28T03:09:44.457279+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:09:44.457279+0000","last_undegraded":"2023-11-28T03:09:44.457279+0000","last_fullsized":"2023-11-28T03:09:44.457279+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374286+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457012+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:09:44.457012+0000","last_peered":"2023-11-28T03:09:44.457012+0000","last_clean":"2023-11-28T03:09:44.457012+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:09:44.457012+0000","last_undegraded":"2023-11-28T03:09:44.457012+0000","last_fullsized":"2023-11-28T03:09:44.457012+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.374262+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":109,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.357942+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:09:44.357942+0000","last_peered":"2023-11-28T03:09:44.357942+0000","last_clean":"2023-11-28T03:09:44.357942+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:09:44.357942+0000","last_undegraded":"2023-11-28T03:09:44.357942+0000","last_fullsized":"2023-11-28T03:09:44.357942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575052,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.59799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.34499999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.48699999999999999}]},{"osd":3,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.504},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53400000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46600000000000003,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.628},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.60999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.57399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575052,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.71399999999999997,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.5},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.51300000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":1.506,"15min":1.506},"last":0.54400000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47399999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.53300000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58699999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63300000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.60399999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.64700000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.64500000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.69399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575051,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.54700000000000004,"5min":0.67800000000000005,"15min":0.70399999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.371},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.67100000000000004},"last":0.38300000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.307,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.64900000000000002},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.628,"15min":0.63400000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.65200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.75600000000000001},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.97799999999999998},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.91900000000000004,"15min":0.99299999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.629,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.94199999999999995,"15min":1.038},"last":0.69499999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78700000000000003,"5min":0.89400000000000002,"15min":0.94099999999999995},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.67600000000000005}]},{"osd":7,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":1.024},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.88300000000000001},"last":0.63700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575052,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.217,"15min":0.184},"max":{"1min":0.46100000000000002,"5min":0.56599999999999995,"15min":0.56999999999999995},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47499999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.34300000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.441}]},{"osd":3,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.377,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.41699999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.61499999999999999,"5min":0.621,"15min":0.65400000000000003},"last":0.41199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":1.0489999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":1.679,"15min":1.679},"last":0.498},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.51900000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.97499999999999998},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.97399999999999998,"15min":1.046},"last":0.69299999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302676,"kb_used_data":1484,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069164,"statfs":{"total":96636764160,"available":96326823936,"internally_reserved":0,"allocated":1519616,"data_stored":756028,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34799999999999998,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.48499999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.41699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.53800000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.39800000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34599999999999997,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.5,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.58499999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.28499999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.41799999999999998},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.58199999999999996,"15min":0.67400000000000004},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88600000000000001,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.59699999999999998}]},{"osd":5,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.98599999999999999},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":1.004},"last":0.60399999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.875,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.65500000000000003}]},{"osd":7,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.63},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.621}]}]},{"osd":3,"up_from":13,"seq":55834575052,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52300000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.35699999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.379}]},{"osd":1,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60099999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.39800000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.42099999999999999},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.39200000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.68200000000000005},"last":0.42699999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82599999999999996,"5min":1.581,"15min":1.581},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.94199999999999995},"last":0.59299999999999997}]},{"osd":5,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.97899999999999998},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.93600000000000005},"last":0.60999999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":0.90900000000000003,"15min":0.996},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.63}]},{"osd":7,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":0.93500000000000005,"15min":0.97199999999999998},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.57999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52100000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.505}]},{"osd":1,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53600000000000003,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.84599999999999997,"15min":0.878},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.496,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68400000000000005,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.77900000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70099999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47599999999999998,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78200000000000003,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:09:51.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.20 is not active+clean 2023-11-28T03:09:51.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3f is not active+clean 2023-11-28T03:09:51.427 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369372+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.427 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.21 is not active+clean 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3e is not active+clean 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365278+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.22 is not active+clean 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3d is not active+clean 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368673+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.23 is not active+clean 2023-11-28T03:09:51.428 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3c is not active+clean 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368605+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.24 is not active+clean 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3b is not active+clean 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364443+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.25 is not active+clean 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3a is not active+clean 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368661+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.429 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.26 is not active+clean 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.39 is not active+clean 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368544+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.27 is not active+clean 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.38 is not active+clean 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364792+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.28 is not active+clean 2023-11-28T03:09:51.430 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.37 is not active+clean 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.367095+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.29 is not active+clean 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.36 is not active+clean 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365829+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2a is not active+clean 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.35 is not active+clean 2023-11-28T03:09:51.431 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369140+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2b is not active+clean 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.34 is not active+clean 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366373+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2c is not active+clean 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.33 is not active+clean 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368253+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2d is not active+clean 2023-11-28T03:09:51.432 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.32 is not active+clean 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.376360+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2e is not active+clean 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.31 is not active+clean 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.370856+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2f is not active+clean 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.30 is not active+clean 2023-11-28T03:09:51.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369474+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.30 is not active+clean 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2f is not active+clean 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364740+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.31 is not active+clean 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2e is not active+clean 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369025+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.32 is not active+clean 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2d is not active+clean 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374317+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.33 is not active+clean 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2c is not active+clean 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365064+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.34 is not active+clean 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2b is not active+clean 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.373899+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.35 is not active+clean 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2a is not active+clean 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364709+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.36 is not active+clean 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.29 is not active+clean 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364722+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.37 is not active+clean 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.28 is not active+clean 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368751+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.38 is not active+clean 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.27 is not active+clean 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366569+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.39 is not active+clean 2023-11-28T03:09:51.437 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.26 is not active+clean 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364670+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3a is not active+clean 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.25 is not active+clean 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.370707+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3b is not active+clean 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.24 is not active+clean 2023-11-28T03:09:51.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364484+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3c is not active+clean 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.23 is not active+clean 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364340+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3d is not active+clean 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.22 is not active+clean 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.373871+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3e is not active+clean 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.21 is not active+clean 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368283+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3f is not active+clean 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.20 is not active+clean 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368192+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.0 is not active+clean 2023-11-28T03:09:51.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1f is not active+clean 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368705+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1 is not active+clean 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1e is not active+clean 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374385+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.2 is not active+clean 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1d is not active+clean 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365285+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.3 is not active+clean 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1c is not active+clean 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.370208+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.4 is not active+clean 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1b is not active+clean 2023-11-28T03:09:51.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368677+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 6], 'acting': [7, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.5 is not active+clean 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1a is not active+clean 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.367186+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4], 'acting': [5, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.6 is not active+clean 2023-11-28T03:09:51.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.19 is not active+clean 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364493+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.7 is not active+clean 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.18 is not active+clean 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369798+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.8 is not active+clean 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.17 is not active+clean 2023-11-28T03:09:51.444 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366689+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.9 is not active+clean 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.16 is not active+clean 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364307+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.a is not active+clean 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.15 is not active+clean 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369982+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.b is not active+clean 2023-11-28T03:09:51.445 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.14 is not active+clean 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366518+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.c is not active+clean 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.13 is not active+clean 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368210+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.d is not active+clean 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.446 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.12 is not active+clean 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.368382+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.e is not active+clean 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.11 is not active+clean 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374352+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.f is not active+clean 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.10 is not active+clean 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364750+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.447 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.18 is not active+clean 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.7 is not active+clean 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369024+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.19 is not active+clean 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.6 is not active+clean 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.363898+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1a is not active+clean 2023-11-28T03:09:51.448 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.5 is not active+clean 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365339+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1b is not active+clean 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.4 is not active+clean 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369869+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1c is not active+clean 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.3 is not active+clean 2023-11-28T03:09:51.449 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.365009+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1d is not active+clean 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.2 is not active+clean 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374623+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1e is not active+clean 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.1 is not active+clean 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.364460+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.450 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.1f is not active+clean 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.0 is not active+clean 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374654+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.16 is not active+clean 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.9 is not active+clean 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366070+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.10 is not active+clean 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.451 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.f is not active+clean 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.366105+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.13 is not active+clean 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.c is not active+clean 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.367145+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.11 is not active+clean 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.e is not active+clean 2023-11-28T03:09:51.452 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.373990+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.12 is not active+clean 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.d is not active+clean 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.369550+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.17 is not active+clean 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.8 is not active+clean 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374287+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.14 is not active+clean 2023-11-28T03:09:51.453 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.b is not active+clean 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374286+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 16.15 is not active+clean 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '16.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T03:09:47.361629+0000', 'last_change': '2023-11-28T03:09:47.361629+0000', 'last_active': '2023-11-28T03:09:47.361629+0000', 'last_peered': '2023-11-28T03:09:47.361629+0000', 'last_clean': '2023-11-28T03:09:47.361629+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:47.361629+0000', 'last_undegraded': '2023-11-28T03:09:47.361629+0000', 'last_fullsized': '2023-11-28T03:09:47.361629+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:47.361629+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:PG 15.a is not active+clean 2023-11-28T03:09:51.454 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '15.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T03:09:46.356969+0000', 'last_change': '2023-11-28T03:09:46.374262+0000', 'last_active': '2023-11-28T03:09:46.356969+0000', 'last_peered': '2023-11-28T03:09:46.356969+0000', 'last_clean': '2023-11-28T03:09:46.356969+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:09:46.356969+0000', 'last_undegraded': '2023-11-28T03:09:46.356969+0000', 'last_fullsized': '2023-11-28T03:09:46.356969+0000', 'mapping_epoch': 123, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 123, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'last_clean_scrub_stamp': '2023-11-28T03:09:46.356969+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:09:51.455 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T03:09:51.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.570+0000 7f142137e700 1 Processor -- start 2023-11-28T03:09:51.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.570+0000 7f142137e700 1 -- start start 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f142137e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f141c0c3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f142137e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0bfc00 con 0x7f141c154120 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f142137e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f141c0beff0 con 0x7f141c0c7db0 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f142137e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0bf2d0 con 0x7f141c154500 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f141bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f141c154120 0x7f141c0be510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:56534/0 (socket says 172.21.15.73:56534) 2023-11-28T03:09:51.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f141bfff700 1 -- 172.21.15.73:0/3640769996 learned_addr learned my addr 172.21.15.73:0/3640769996 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3331147081 0 0) 0x7f141c0bfc00 con 0x7f141c154120 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14040033f0 con 0x7f141c154120 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.571+0000 7f141b7fe700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f141c0c3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 729575903 0 0) 0x7f14040033f0 con 0x7f141c154120 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 msgr2=0x7f141c0c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f141c0c3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 >> v1:172.21.15.150:6789/0 conn(0x7f141c0c7db0 legacy=0x7f141c15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c0bf5b0 con 0x7f141c154120 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f140c0015f0 con 0x7f141c154120 2023-11-28T03:09:51.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 >> v1:172.21.15.73:6789/0 conn(0x7f141c154120 legacy=0x7f141c0be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404003910 0x7f1404003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1404004690 0x7f1404004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f1404005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0bf2d0 con 0x7f1404004690 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f140400b9c0 con 0x7f1404003910 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f140400bb80 con 0x7f141c154500 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1404004690 0x7f1404004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141bfff700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404003910 0x7f1404003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f1420b7d700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f1404005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 -- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 msgr2=0x7f1404005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:51.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f1404005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 -- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404003910 msgr2=0x7f1404003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1404003910 0x7f1404003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 -- 172.21.15.73:0/3640769996 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14040033f0 con 0x7f1404004690 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f1420b7d700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c154500 0x7f1404005090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141b7fe700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1404004690 0x7f1404004b10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f141000ba00 tx=0x7f1410008040).ready entity=mon.0 client_cookie=4f2ec024cfd7d184 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000b340 con 0x7f1404004690 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1410013480 con 0x7f1404004690 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.572+0000 7f141affd700 1 -- 172.21.15.73:0/3640769996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1410013800 con 0x7f1404004690 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 -- 172.21.15.73:0/3640769996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1404004690 msgr2=0x7f1404004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 --2- 172.21.15.73:0/3640769996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1404004690 0x7f1404004b10 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 rx=0x7f141000ba00 tx=0x7f1410008040).stop 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f1420b7d700 1 -- 172.21.15.73:0/3640769996 reap_dead start 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 -- 172.21.15.73:0/3640769996 shutdown_connections 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 -- 172.21.15.73:0/3640769996 >> 172.21.15.73:0/3640769996 conn(0x7f141c083000 msgr2=0x7f141c092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 -- 172.21.15.73:0/3640769996 shutdown_connections 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.573+0000 7f142137e700 1 -- 172.21.15.73:0/3640769996 wait complete. 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.574+0000 7f142137e700 1 Processor -- start 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.574+0000 7f142137e700 1 -- start start 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 0x7f141c14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 0x7f141c168100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c1685f0 con 0x7f141c0ab510 2023-11-28T03:09:51.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f141c0a9bc0 con 0x7f141c167c80 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f142137e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f141c150390 con 0x7f141c14f000 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53516/0 (socket says 172.21.15.73:53516) 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 -- 172.21.15.73:0/1141409045 learned_addr learned my addr 172.21.15.73:0/1141409045 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141bfff700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 0x7f141c14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f1420b7d700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 0x7f141c168100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 -- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 msgr2=0x7f141c14f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 0x7f141c14f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 -- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 msgr2=0x7f141c168100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 0x7f141c168100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141b7fe700 1 -- 172.21.15.73:0/1141409045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f141c150670 con 0x7f141c0ab510 2023-11-28T03:09:51.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f141bfff700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 0x7f141c14f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.575+0000 7f1420b7d700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 0x7f141c168100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f141b7fe700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f1410008010 tx=0x7f1410008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141000dbf0 con 0x7f141c0ab510 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f141000dd50 con 0x7f141c0ab510 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f141001d660 con 0x7f141c0ab510 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f141c0a9e60 con 0x7f141c0ab510 2023-11-28T03:09:51.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.576+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f141c150950 con 0x7f141c0ab510 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.577+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f141c06c7c0 con 0x7f141c0ab510 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.577+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f141001a030 con 0x7f141c0ab510 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.577+0000 7f1403fff700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 0x7f1408061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.577+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f141007b870 con 0x7f141c0ab510 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.577+0000 7f141bfff700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 0x7f1408061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.578+0000 7f141bfff700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 0x7f1408061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f140c001720 tx=0x7f140c009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.581+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f141004cae0 con 0x7f141c0ab510 2023-11-28T03:09:51.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.809+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f141c1685f0 con 0x7f141c0ab510 2023-11-28T03:09:51.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.810+0000 7f1403fff700 1 -- 172.21.15.73:0/1141409045 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1110 (secure 0 0 0) 0x7f141002a470 con 0x7f141c0ab510 2023-11-28T03:09:51.812 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:51.812 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1045,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":126,"num_osds":8,"num_up_osds":8,"osd_up_since":1701139955,"num_in_osds":8,"osd_in_since":1701139948,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"unknown","count":64},{"state_name":"active+clean","count":9}],"num_pgs":137,"num_pools":4,"num_objects":0,"data_bytes":0,"bytes_used":2479460352,"bytes_avail":770614652928,"bytes_total":773094113280,"unknown_pgs_ratio":0.46715328097343445,"inactive_pgs_ratio":0.46715328097343445},"fsmap":{"epoch":46,"id":7,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":7,"rank":0,"name":"c","status":"up:active","gid":7596}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-28T03:09:30.526016+0000","services":{}},"progress_events":{}} 2023-11-28T03:09:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 msgr2=0x7f1408061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 0x7f1408061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f140c001720 tx=0x7f140c009040).stop 2023-11-28T03:09:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 msgr2=0x7f141c0aa380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 rx=0x7f1410008010 tx=0x7f1410008040).stop 2023-11-28T03:09:51.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 shutdown_connections 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f140805f150 0x7f1408061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f141c167c80 0x7f141c168100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f141c14f000 0x7f141c14f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 --2- 172.21.15.73:0/1141409045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f141c0ab510 0x7f141c0aa380 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.811+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 >> 172.21.15.73:0/1141409045 conn(0x7f141c083000 msgr2=0x7f141c067a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.812+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 shutdown_connections 2023-11-28T03:09:51.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.812+0000 7f142137e700 1 -- 172.21.15.73:0/1141409045 wait complete. 2023-11-28T03:09:51.825 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T03:09:51.825 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:09:51.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 Processor -- start 2023-11-28T03:09:51.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 -- start start 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f62d80cae90 0x7f62d80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62d80aa960 0x7f62d80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80c1490 con 0x7f62d80aa960 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62d80a9a90 con 0x7f62d80cae90 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.936+0000 7f62de4e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8154ce0 con 0x7f62d80caab0 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62d80aa960 0x7f62d80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f62d80aa960 0x7f62d80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53532/0 (socket says 172.21.15.73:53532) 2023-11-28T03:09:51.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 -- 172.21.15.73:0/4039864695 learned_addr learned my addr 172.21.15.73:0/4039864695 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62d7fff700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62d77fe700 1 --2- 172.21.15.73:0/4039864695 >> v2:172.21.15.150:3300/0 conn(0x7f62d80cae90 0x7f62d80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 -- 172.21.15.73:0/4039864695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 msgr2=0x7f62d80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 -- 172.21.15.73:0/4039864695 >> v2:172.21.15.150:3300/0 conn(0x7f62d80cae90 msgr2=0x7f62d80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 --2- 172.21.15.73:0/4039864695 >> v2:172.21.15.150:3300/0 conn(0x7f62d80cae90 0x7f62d80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 -- 172.21.15.73:0/4039864695 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d80a8e80 con 0x7f62d80aa960 2023-11-28T03:09:51.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62dca83700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d80aa960 0x7f62d80c0fa0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f62d80c9070 tx=0x7f62d0009840).ready entity=mon.0 client_cookie=3878408d5bcbcea0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62d6ffd700 1 -- 172.21.15.73:0/4039864695 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000e070 con 0x7f62d80aa960 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62d6ffd700 1 -- 172.21.15.73:0/4039864695 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d000b970 con 0x7f62d80aa960 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.937+0000 7f62d6ffd700 1 -- 172.21.15.73:0/4039864695 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bcf0 con 0x7f62d80aa960 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 -- 172.21.15.73:0/4039864695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d80aa960 msgr2=0x7f62d80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d80aa960 0x7f62d80c0fa0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 rx=0x7f62d80c9070 tx=0x7f62d0009840).stop 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 -- 172.21.15.73:0/4039864695 shutdown_connections 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d80aa960 0x7f62d80c0fa0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 --2- 172.21.15.73:0/4039864695 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 --2- 172.21.15.73:0/4039864695 >> v2:172.21.15.150:3300/0 conn(0x7f62d80cae90 0x7f62d80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 -- 172.21.15.73:0/4039864695 >> 172.21.15.73:0/4039864695 conn(0x7f62d8094920 msgr2=0x7f62d806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:51.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 -- 172.21.15.73:0/4039864695 shutdown_connections 2023-11-28T03:09:51.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.938+0000 7f62de4e6700 1 -- 172.21.15.73:0/4039864695 wait complete. 2023-11-28T03:09:51.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.939+0000 7f62de4e6700 1 Processor -- start 2023-11-28T03:09:51.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.939+0000 7f62de4e6700 1 -- start start 2023-11-28T03:09:51.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80af4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d80cae90 0x7f62d8153c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8155170 con 0x7f62d8154150 2023-11-28T03:09:51.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8155430 con 0x7f62d80cae90 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62de4e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62d8155710 con 0x7f62d80caab0 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53540/0 (socket says 172.21.15.73:53540) 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 -- 172.21.15.73:0/334437958 learned_addr learned my addr 172.21.15.73:0/334437958 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62d7fff700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80af4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62d77fe700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d80cae90 0x7f62d8153c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 -- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 msgr2=0x7f62d80af4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80af4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 -- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d80cae90 msgr2=0x7f62d8153c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:51.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d80cae90 0x7f62d8153c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 -- 172.21.15.73:0/334437958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d0009ae0 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.940+0000 7f62dca83700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f62d000b610 tx=0x7f62d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.941+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d000bee0 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.941+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0013e60 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.941+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d001e6c0 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.941+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62d81559f0 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.941+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62d80af9a0 con 0x7f62d8154150 2023-11-28T03:09:51.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.942+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62d806c860 con 0x7f62d8154150 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.942+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f62d0003040 con 0x7f62d8154150 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.942+0000 7f62bffff700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 0x7f62c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.942+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f62d007b5a0 con 0x7f62d8154150 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.942+0000 7f62d7fff700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 0x7f62c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.943+0000 7f62d7fff700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 0x7f62c4061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f62cc006fd0 tx=0x7f62cc006f40).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:51.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:51.946+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f62d004c920 con 0x7f62d8154150 2023-11-28T03:09:52.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.104+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f62d80a9160 con 0x7f62c405f150 2023-11-28T03:09:52.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.114+0000 7f62bffff700 1 -- 172.21.15.73:0/334437958 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360243 (secure 0 0 0) 0x7f62d80a9160 con 0x7f62c405f150 2023-11-28T03:09:52.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 msgr2=0x7f62c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 0x7f62c4061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f62cc006fd0 tx=0x7f62cc006f40).stop 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 msgr2=0x7f62d81646e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 rx=0x7f62d000b610 tx=0x7f62d0009840).stop 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 shutdown_connections 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f62c405f150 0x7f62c4061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f62d8154150 0x7f62d81646e0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:52.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f62d80caab0 0x7f62d80af4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:52.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 --2- 172.21.15.73:0/334437958 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f62d80cae90 0x7f62d8153c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:52.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 >> 172.21.15.73:0/334437958 conn(0x7f62d8094920 msgr2=0x7f62d80c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:52.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 shutdown_connections 2023-11-28T03:09:52.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:52.117+0000 7f62de4e6700 1 -- 172.21.15.73:0/334437958 wait complete. 2023-11-28T03:09:52.123 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:09:52.130 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":22,"stamp":"2023-11-28T03:09:51.773353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1414,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":246,"acting":246,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":217,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2421652,"kb_used_data":12116,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752553068,"statfs":{"total":773094113280,"available":770614341632,"internally_reserved":0,"allocated":12406784,"data_stored":6162298,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5589,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-56,"num_write_kb":-24,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.560879"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772877+0000","last_change":"2023-11-28T03:09:47.767383+0000","last_active":"2023-11-28T03:09:47.772877+0000","last_peered":"2023-11-28T03:09:47.772877+0000","last_clean":"2023-11-28T03:09:47.772877+0000","last_became_active":"2023-11-28T03:09:47.767079+0000","last_became_peered":"2023-11-28T03:09:47.767079+0000","last_unstale":"2023-11-28T03:09:47.772877+0000","last_undegraded":"2023-11-28T03:09:47.772877+0000","last_fullsized":"2023-11-28T03:09:47.772877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3f","version":"125'3","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768089+0000","last_change":"2023-11-28T03:09:47.394206+0000","last_active":"2023-11-28T03:09:47.768089+0000","last_peered":"2023-11-28T03:09:47.768089+0000","last_clean":"2023-11-28T03:09:47.768089+0000","last_became_active":"2023-11-28T03:09:47.393934+0000","last_became_peered":"2023-11-28T03:09:47.393934+0000","last_unstale":"2023-11-28T03:09:47.768089+0000","last_undegraded":"2023-11-28T03:09:47.768089+0000","last_fullsized":"2023-11-28T03:09:47.768089+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768202+0000","last_change":"2023-11-28T03:09:47.763844+0000","last_active":"2023-11-28T03:09:47.768202+0000","last_peered":"2023-11-28T03:09:47.768202+0000","last_clean":"2023-11-28T03:09:47.768202+0000","last_became_active":"2023-11-28T03:09:47.763586+0000","last_became_peered":"2023-11-28T03:09:47.763586+0000","last_unstale":"2023-11-28T03:09:47.768202+0000","last_undegraded":"2023-11-28T03:09:47.768202+0000","last_fullsized":"2023-11-28T03:09:47.768202+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750907+0000","last_change":"2023-11-28T03:09:47.392119+0000","last_active":"2023-11-28T03:09:47.750907+0000","last_peered":"2023-11-28T03:09:47.750907+0000","last_clean":"2023-11-28T03:09:47.750907+0000","last_became_active":"2023-11-28T03:09:47.391850+0000","last_became_peered":"2023-11-28T03:09:47.391850+0000","last_unstale":"2023-11-28T03:09:47.750907+0000","last_undegraded":"2023-11-28T03:09:47.750907+0000","last_fullsized":"2023-11-28T03:09:47.750907+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748005+0000","last_change":"2023-11-28T03:09:47.385738+0000","last_active":"2023-11-28T03:09:47.748005+0000","last_peered":"2023-11-28T03:09:47.748005+0000","last_clean":"2023-11-28T03:09:47.748005+0000","last_became_active":"2023-11-28T03:09:47.385479+0000","last_became_peered":"2023-11-28T03:09:47.385479+0000","last_unstale":"2023-11-28T03:09:47.748005+0000","last_undegraded":"2023-11-28T03:09:47.748005+0000","last_fullsized":"2023-11-28T03:09:47.748005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767699+0000","last_change":"2023-11-28T03:09:47.766548+0000","last_active":"2023-11-28T03:09:47.767699+0000","last_peered":"2023-11-28T03:09:47.767699+0000","last_clean":"2023-11-28T03:09:47.767699+0000","last_became_active":"2023-11-28T03:09:47.766247+0000","last_became_peered":"2023-11-28T03:09:47.766247+0000","last_unstale":"2023-11-28T03:09:47.767699+0000","last_undegraded":"2023-11-28T03:09:47.767699+0000","last_fullsized":"2023-11-28T03:09:47.767699+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3c","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.761978+0000","last_change":"2023-11-28T03:09:47.390788+0000","last_active":"2023-11-28T03:09:47.761978+0000","last_peered":"2023-11-28T03:09:47.761978+0000","last_clean":"2023-11-28T03:09:47.761978+0000","last_became_active":"2023-11-28T03:09:47.390517+0000","last_became_peered":"2023-11-28T03:09:47.390517+0000","last_unstale":"2023-11-28T03:09:47.761978+0000","last_undegraded":"2023-11-28T03:09:47.761978+0000","last_fullsized":"2023-11-28T03:09:47.761978+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772433+0000","last_change":"2023-11-28T03:09:47.766560+0000","last_active":"2023-11-28T03:09:47.772433+0000","last_peered":"2023-11-28T03:09:47.772433+0000","last_clean":"2023-11-28T03:09:47.772433+0000","last_became_active":"2023-11-28T03:09:47.766303+0000","last_became_peered":"2023-11-28T03:09:47.766303+0000","last_unstale":"2023-11-28T03:09:47.772433+0000","last_undegraded":"2023-11-28T03:09:47.772433+0000","last_fullsized":"2023-11-28T03:09:47.772433+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764007+0000","last_change":"2023-11-28T03:09:47.383307+0000","last_active":"2023-11-28T03:09:47.764007+0000","last_peered":"2023-11-28T03:09:47.764007+0000","last_clean":"2023-11-28T03:09:47.764007+0000","last_became_active":"2023-11-28T03:09:47.382990+0000","last_became_peered":"2023-11-28T03:09:47.382990+0000","last_unstale":"2023-11-28T03:09:47.764007+0000","last_undegraded":"2023-11-28T03:09:47.764007+0000","last_fullsized":"2023-11-28T03:09:47.764007+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772251+0000","last_change":"2023-11-28T03:09:47.385556+0000","last_active":"2023-11-28T03:09:47.772251+0000","last_peered":"2023-11-28T03:09:47.772251+0000","last_clean":"2023-11-28T03:09:47.772251+0000","last_became_active":"2023-11-28T03:09:47.385208+0000","last_became_peered":"2023-11-28T03:09:47.385208+0000","last_unstale":"2023-11-28T03:09:47.772251+0000","last_undegraded":"2023-11-28T03:09:47.772251+0000","last_fullsized":"2023-11-28T03:09:47.772251+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752152+0000","last_change":"2023-11-28T03:09:47.385411+0000","last_active":"2023-11-28T03:09:47.752152+0000","last_peered":"2023-11-28T03:09:47.752152+0000","last_clean":"2023-11-28T03:09:47.752152+0000","last_became_active":"2023-11-28T03:09:47.385116+0000","last_became_peered":"2023-11-28T03:09:47.385116+0000","last_unstale":"2023-11-28T03:09:47.752152+0000","last_undegraded":"2023-11-28T03:09:47.752152+0000","last_fullsized":"2023-11-28T03:09:47.752152+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.749361+0000","last_change":"2023-11-28T03:09:47.389535+0000","last_active":"2023-11-28T03:09:47.749361+0000","last_peered":"2023-11-28T03:09:47.749361+0000","last_clean":"2023-11-28T03:09:47.749361+0000","last_became_active":"2023-11-28T03:09:47.389266+0000","last_became_peered":"2023-11-28T03:09:47.389266+0000","last_unstale":"2023-11-28T03:09:47.749361+0000","last_undegraded":"2023-11-28T03:09:47.749361+0000","last_fullsized":"2023-11-28T03:09:47.749361+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770613+0000","last_change":"2023-11-28T03:09:47.755826+0000","last_active":"2023-11-28T03:09:47.770613+0000","last_peered":"2023-11-28T03:09:47.770613+0000","last_clean":"2023-11-28T03:09:47.770613+0000","last_became_active":"2023-11-28T03:09:47.755588+0000","last_became_peered":"2023-11-28T03:09:47.755588+0000","last_unstale":"2023-11-28T03:09:47.770613+0000","last_undegraded":"2023-11-28T03:09:47.770613+0000","last_fullsized":"2023-11-28T03:09:47.770613+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367095+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772936+0000","last_change":"2023-11-28T03:09:47.766559+0000","last_active":"2023-11-28T03:09:47.772936+0000","last_peered":"2023-11-28T03:09:47.772936+0000","last_clean":"2023-11-28T03:09:47.772936+0000","last_became_active":"2023-11-28T03:09:47.766260+0000","last_became_peered":"2023-11-28T03:09:47.766260+0000","last_unstale":"2023-11-28T03:09:47.772936+0000","last_undegraded":"2023-11-28T03:09:47.772936+0000","last_fullsized":"2023-11-28T03:09:47.772936+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755692+0000","last_change":"2023-11-28T03:09:47.387044+0000","last_active":"2023-11-28T03:09:47.755692+0000","last_peered":"2023-11-28T03:09:47.755692+0000","last_clean":"2023-11-28T03:09:47.755692+0000","last_became_active":"2023-11-28T03:09:47.386766+0000","last_became_peered":"2023-11-28T03:09:47.386766+0000","last_unstale":"2023-11-28T03:09:47.755692+0000","last_undegraded":"2023-11-28T03:09:47.755692+0000","last_fullsized":"2023-11-28T03:09:47.755692+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774677+0000","last_change":"2023-11-28T03:09:47.768197+0000","last_active":"2023-11-28T03:09:47.774677+0000","last_peered":"2023-11-28T03:09:47.774677+0000","last_clean":"2023-11-28T03:09:47.774677+0000","last_became_active":"2023-11-28T03:09:47.767959+0000","last_became_peered":"2023-11-28T03:09:47.767959+0000","last_unstale":"2023-11-28T03:09:47.774677+0000","last_undegraded":"2023-11-28T03:09:47.774677+0000","last_fullsized":"2023-11-28T03:09:47.774677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752716+0000","last_change":"2023-11-28T03:09:47.386088+0000","last_active":"2023-11-28T03:09:47.752716+0000","last_peered":"2023-11-28T03:09:47.752716+0000","last_clean":"2023-11-28T03:09:47.752716+0000","last_became_active":"2023-11-28T03:09:47.385812+0000","last_became_peered":"2023-11-28T03:09:47.385812+0000","last_unstale":"2023-11-28T03:09:47.752716+0000","last_undegraded":"2023-11-28T03:09:47.752716+0000","last_fullsized":"2023-11-28T03:09:47.752716+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775413+0000","last_change":"2023-11-28T03:09:47.770395+0000","last_active":"2023-11-28T03:09:47.775413+0000","last_peered":"2023-11-28T03:09:47.775413+0000","last_clean":"2023-11-28T03:09:47.775413+0000","last_became_active":"2023-11-28T03:09:47.770125+0000","last_became_peered":"2023-11-28T03:09:47.770125+0000","last_unstale":"2023-11-28T03:09:47.775413+0000","last_undegraded":"2023-11-28T03:09:47.775413+0000","last_fullsized":"2023-11-28T03:09:47.775413+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366373+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.33","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.760910+0000","last_change":"2023-11-28T03:09:47.383115+0000","last_active":"2023-11-28T03:09:47.760910+0000","last_peered":"2023-11-28T03:09:47.760910+0000","last_clean":"2023-11-28T03:09:47.760910+0000","last_became_active":"2023-11-28T03:09:47.382846+0000","last_became_peered":"2023-11-28T03:09:47.382846+0000","last_unstale":"2023-11-28T03:09:47.760910+0000","last_undegraded":"2023-11-28T03:09:47.760910+0000","last_fullsized":"2023-11-28T03:09:47.760910+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768007+0000","last_change":"2023-11-28T03:09:47.763500+0000","last_active":"2023-11-28T03:09:47.768007+0000","last_peered":"2023-11-28T03:09:47.768007+0000","last_clean":"2023-11-28T03:09:47.768007+0000","last_became_active":"2023-11-28T03:09:47.763151+0000","last_became_peered":"2023-11-28T03:09:47.763151+0000","last_unstale":"2023-11-28T03:09:47.768007+0000","last_undegraded":"2023-11-28T03:09:47.768007+0000","last_fullsized":"2023-11-28T03:09:47.768007+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755467+0000","last_change":"2023-11-28T03:09:47.392189+0000","last_active":"2023-11-28T03:09:47.755467+0000","last_peered":"2023-11-28T03:09:47.755467+0000","last_clean":"2023-11-28T03:09:47.755467+0000","last_became_active":"2023-11-28T03:09:47.389101+0000","last_became_peered":"2023-11-28T03:09:47.389101+0000","last_unstale":"2023-11-28T03:09:47.755467+0000","last_undegraded":"2023-11-28T03:09:47.755467+0000","last_fullsized":"2023-11-28T03:09:47.755467+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768702+0000","last_change":"2023-11-28T03:09:47.765962+0000","last_active":"2023-11-28T03:09:47.768702+0000","last_peered":"2023-11-28T03:09:47.768702+0000","last_clean":"2023-11-28T03:09:47.768702+0000","last_became_active":"2023-11-28T03:09:47.765570+0000","last_became_peered":"2023-11-28T03:09:47.765570+0000","last_unstale":"2023-11-28T03:09:47.768702+0000","last_undegraded":"2023-11-28T03:09:47.768702+0000","last_fullsized":"2023-11-28T03:09:47.768702+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752707+0000","last_change":"2023-11-28T03:09:47.384811+0000","last_active":"2023-11-28T03:09:47.752707+0000","last_peered":"2023-11-28T03:09:47.752707+0000","last_clean":"2023-11-28T03:09:47.752707+0000","last_became_active":"2023-11-28T03:09:47.384543+0000","last_became_peered":"2023-11-28T03:09:47.384543+0000","last_unstale":"2023-11-28T03:09:47.752707+0000","last_undegraded":"2023-11-28T03:09:47.752707+0000","last_fullsized":"2023-11-28T03:09:47.752707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774397+0000","last_change":"2023-11-28T03:09:47.767269+0000","last_active":"2023-11-28T03:09:47.774397+0000","last_peered":"2023-11-28T03:09:47.774397+0000","last_clean":"2023-11-28T03:09:47.774397+0000","last_became_active":"2023-11-28T03:09:47.766860+0000","last_became_peered":"2023-11-28T03:09:47.766860+0000","last_unstale":"2023-11-28T03:09:47.774397+0000","last_undegraded":"2023-11-28T03:09:47.774397+0000","last_fullsized":"2023-11-28T03:09:47.774397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"125'4","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772536+0000","last_change":"2023-11-28T03:09:47.395446+0000","last_active":"2023-11-28T03:09:47.772536+0000","last_peered":"2023-11-28T03:09:47.772536+0000","last_clean":"2023-11-28T03:09:47.772536+0000","last_became_active":"2023-11-28T03:09:47.395212+0000","last_became_peered":"2023-11-28T03:09:47.395212+0000","last_unstale":"2023-11-28T03:09:47.772536+0000","last_undegraded":"2023-11-28T03:09:47.772536+0000","last_fullsized":"2023-11-28T03:09:47.772536+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772596+0000","last_change":"2023-11-28T03:09:47.761229+0000","last_active":"2023-11-28T03:09:47.772596+0000","last_peered":"2023-11-28T03:09:47.772596+0000","last_clean":"2023-11-28T03:09:47.772596+0000","last_became_active":"2023-11-28T03:09:47.760944+0000","last_became_peered":"2023-11-28T03:09:47.760944+0000","last_unstale":"2023-11-28T03:09:47.772596+0000","last_undegraded":"2023-11-28T03:09:47.772596+0000","last_fullsized":"2023-11-28T03:09:47.772596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750997+0000","last_change":"2023-11-28T03:09:47.391999+0000","last_active":"2023-11-28T03:09:47.750997+0000","last_peered":"2023-11-28T03:09:47.750997+0000","last_clean":"2023-11-28T03:09:47.750997+0000","last_became_active":"2023-11-28T03:09:47.389903+0000","last_became_peered":"2023-11-28T03:09:47.389903+0000","last_unstale":"2023-11-28T03:09:47.750997+0000","last_undegraded":"2023-11-28T03:09:47.750997+0000","last_fullsized":"2023-11-28T03:09:47.750997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775095+0000","last_change":"2023-11-28T03:09:47.766531+0000","last_active":"2023-11-28T03:09:47.775095+0000","last_peered":"2023-11-28T03:09:47.775095+0000","last_clean":"2023-11-28T03:09:47.775095+0000","last_became_active":"2023-11-28T03:09:47.766212+0000","last_became_peered":"2023-11-28T03:09:47.766212+0000","last_unstale":"2023-11-28T03:09:47.775095+0000","last_undegraded":"2023-11-28T03:09:47.775095+0000","last_fullsized":"2023-11-28T03:09:47.775095+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753010+0000","last_change":"2023-11-28T03:09:47.393114+0000","last_active":"2023-11-28T03:09:47.753010+0000","last_peered":"2023-11-28T03:09:47.753010+0000","last_clean":"2023-11-28T03:09:47.753010+0000","last_became_active":"2023-11-28T03:09:47.392832+0000","last_became_peered":"2023-11-28T03:09:47.392832+0000","last_unstale":"2023-11-28T03:09:47.753010+0000","last_undegraded":"2023-11-28T03:09:47.753010+0000","last_fullsized":"2023-11-28T03:09:47.753010+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772568+0000","last_change":"2023-11-28T03:09:47.762910+0000","last_active":"2023-11-28T03:09:47.772568+0000","last_peered":"2023-11-28T03:09:47.772568+0000","last_clean":"2023-11-28T03:09:47.772568+0000","last_became_active":"2023-11-28T03:09:47.762627+0000","last_became_peered":"2023-11-28T03:09:47.762627+0000","last_unstale":"2023-11-28T03:09:47.772568+0000","last_undegraded":"2023-11-28T03:09:47.772568+0000","last_fullsized":"2023-11-28T03:09:47.772568+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2d","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774624+0000","last_change":"2023-11-28T03:09:47.387477+0000","last_active":"2023-11-28T03:09:47.774624+0000","last_peered":"2023-11-28T03:09:47.774624+0000","last_clean":"2023-11-28T03:09:47.774624+0000","last_became_active":"2023-11-28T03:09:47.387191+0000","last_became_peered":"2023-11-28T03:09:47.387191+0000","last_unstale":"2023-11-28T03:09:47.774624+0000","last_undegraded":"2023-11-28T03:09:47.774624+0000","last_fullsized":"2023-11-28T03:09:47.774624+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775556+0000","last_change":"2023-11-28T03:09:47.770678+0000","last_active":"2023-11-28T03:09:47.775556+0000","last_peered":"2023-11-28T03:09:47.775556+0000","last_clean":"2023-11-28T03:09:47.775556+0000","last_became_active":"2023-11-28T03:09:47.770403+0000","last_became_peered":"2023-11-28T03:09:47.770403+0000","last_unstale":"2023-11-28T03:09:47.775556+0000","last_undegraded":"2023-11-28T03:09:47.775556+0000","last_fullsized":"2023-11-28T03:09:47.775556+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365064+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770142+0000","last_change":"2023-11-28T03:09:47.385173+0000","last_active":"2023-11-28T03:09:47.770142+0000","last_peered":"2023-11-28T03:09:47.770142+0000","last_clean":"2023-11-28T03:09:47.770142+0000","last_became_active":"2023-11-28T03:09:47.384367+0000","last_became_peered":"2023-11-28T03:09:47.384367+0000","last_unstale":"2023-11-28T03:09:47.770142+0000","last_undegraded":"2023-11-28T03:09:47.770142+0000","last_fullsized":"2023-11-28T03:09:47.770142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748627+0000","last_change":"2023-11-28T03:09:47.393622+0000","last_active":"2023-11-28T03:09:47.748627+0000","last_peered":"2023-11-28T03:09:47.748627+0000","last_clean":"2023-11-28T03:09:47.748627+0000","last_became_active":"2023-11-28T03:09:47.390278+0000","last_became_peered":"2023-11-28T03:09:47.390278+0000","last_unstale":"2023-11-28T03:09:47.748627+0000","last_undegraded":"2023-11-28T03:09:47.748627+0000","last_fullsized":"2023-11-28T03:09:47.748627+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773662+0000","last_change":"2023-11-28T03:09:47.765563+0000","last_active":"2023-11-28T03:09:47.773662+0000","last_peered":"2023-11-28T03:09:47.773662+0000","last_clean":"2023-11-28T03:09:47.773662+0000","last_became_active":"2023-11-28T03:09:47.765387+0000","last_became_peered":"2023-11-28T03:09:47.765387+0000","last_unstale":"2023-11-28T03:09:47.773662+0000","last_undegraded":"2023-11-28T03:09:47.773662+0000","last_fullsized":"2023-11-28T03:09:47.773662+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748538+0000","last_change":"2023-11-28T03:09:47.388908+0000","last_active":"2023-11-28T03:09:47.748538+0000","last_peered":"2023-11-28T03:09:47.748538+0000","last_clean":"2023-11-28T03:09:47.748538+0000","last_became_active":"2023-11-28T03:09:47.388654+0000","last_became_peered":"2023-11-28T03:09:47.388654+0000","last_unstale":"2023-11-28T03:09:47.748538+0000","last_undegraded":"2023-11-28T03:09:47.748538+0000","last_fullsized":"2023-11-28T03:09:47.748538+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772302+0000","last_change":"2023-11-28T03:09:47.767149+0000","last_active":"2023-11-28T03:09:47.772302+0000","last_peered":"2023-11-28T03:09:47.772302+0000","last_clean":"2023-11-28T03:09:47.772302+0000","last_became_active":"2023-11-28T03:09:47.766835+0000","last_became_peered":"2023-11-28T03:09:47.766835+0000","last_unstale":"2023-11-28T03:09:47.772302+0000","last_undegraded":"2023-11-28T03:09:47.772302+0000","last_fullsized":"2023-11-28T03:09:47.772302+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.751817+0000","last_change":"2023-11-28T03:09:47.393032+0000","last_active":"2023-11-28T03:09:47.751817+0000","last_peered":"2023-11-28T03:09:47.751817+0000","last_clean":"2023-11-28T03:09:47.751817+0000","last_became_active":"2023-11-28T03:09:47.392759+0000","last_became_peered":"2023-11-28T03:09:47.392759+0000","last_unstale":"2023-11-28T03:09:47.751817+0000","last_undegraded":"2023-11-28T03:09:47.751817+0000","last_fullsized":"2023-11-28T03:09:47.751817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776213+0000","last_change":"2023-11-28T03:09:47.774084+0000","last_active":"2023-11-28T03:09:47.776213+0000","last_peered":"2023-11-28T03:09:47.776213+0000","last_clean":"2023-11-28T03:09:47.776213+0000","last_became_active":"2023-11-28T03:09:47.773856+0000","last_became_peered":"2023-11-28T03:09:47.773856+0000","last_unstale":"2023-11-28T03:09:47.776213+0000","last_undegraded":"2023-11-28T03:09:47.776213+0000","last_fullsized":"2023-11-28T03:09:47.776213+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755914+0000","last_change":"2023-11-28T03:09:47.387880+0000","last_active":"2023-11-28T03:09:47.755914+0000","last_peered":"2023-11-28T03:09:47.755914+0000","last_clean":"2023-11-28T03:09:47.755914+0000","last_became_active":"2023-11-28T03:09:47.387590+0000","last_became_peered":"2023-11-28T03:09:47.387590+0000","last_unstale":"2023-11-28T03:09:47.755914+0000","last_undegraded":"2023-11-28T03:09:47.755914+0000","last_fullsized":"2023-11-28T03:09:47.755914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768226+0000","last_change":"2023-11-28T03:09:47.765943+0000","last_active":"2023-11-28T03:09:47.768226+0000","last_peered":"2023-11-28T03:09:47.768226+0000","last_clean":"2023-11-28T03:09:47.768226+0000","last_became_active":"2023-11-28T03:09:47.765585+0000","last_became_peered":"2023-11-28T03:09:47.765585+0000","last_unstale":"2023-11-28T03:09:47.768226+0000","last_undegraded":"2023-11-28T03:09:47.768226+0000","last_fullsized":"2023-11-28T03:09:47.768226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750729+0000","last_change":"2023-11-28T03:09:47.393496+0000","last_active":"2023-11-28T03:09:47.750729+0000","last_peered":"2023-11-28T03:09:47.750729+0000","last_clean":"2023-11-28T03:09:47.750729+0000","last_became_active":"2023-11-28T03:09:47.393219+0000","last_became_peered":"2023-11-28T03:09:47.393219+0000","last_unstale":"2023-11-28T03:09:47.750729+0000","last_undegraded":"2023-11-28T03:09:47.750729+0000","last_fullsized":"2023-11-28T03:09:47.750729+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775173+0000","last_change":"2023-11-28T03:09:47.768144+0000","last_active":"2023-11-28T03:09:47.775173+0000","last_peered":"2023-11-28T03:09:47.775173+0000","last_clean":"2023-11-28T03:09:47.775173+0000","last_became_active":"2023-11-28T03:09:47.767603+0000","last_became_peered":"2023-11-28T03:09:47.767603+0000","last_unstale":"2023-11-28T03:09:47.775173+0000","last_undegraded":"2023-11-28T03:09:47.775173+0000","last_fullsized":"2023-11-28T03:09:47.775173+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752344+0000","last_change":"2023-11-28T03:09:47.389257+0000","last_active":"2023-11-28T03:09:47.752344+0000","last_peered":"2023-11-28T03:09:47.752344+0000","last_clean":"2023-11-28T03:09:47.752344+0000","last_became_active":"2023-11-28T03:09:47.388904+0000","last_became_peered":"2023-11-28T03:09:47.388904+0000","last_unstale":"2023-11-28T03:09:47.752344+0000","last_undegraded":"2023-11-28T03:09:47.752344+0000","last_fullsized":"2023-11-28T03:09:47.752344+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773460+0000","last_change":"2023-11-28T03:09:47.765311+0000","last_active":"2023-11-28T03:09:47.773460+0000","last_peered":"2023-11-28T03:09:47.773460+0000","last_clean":"2023-11-28T03:09:47.773460+0000","last_became_active":"2023-11-28T03:09:47.765037+0000","last_became_peered":"2023-11-28T03:09:47.765037+0000","last_unstale":"2023-11-28T03:09:47.773460+0000","last_undegraded":"2023-11-28T03:09:47.773460+0000","last_fullsized":"2023-11-28T03:09:47.773460+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.754226+0000","last_change":"2023-11-28T03:09:47.385356+0000","last_active":"2023-11-28T03:09:47.754226+0000","last_peered":"2023-11-28T03:09:47.754226+0000","last_clean":"2023-11-28T03:09:47.754226+0000","last_became_active":"2023-11-28T03:09:47.385087+0000","last_became_peered":"2023-11-28T03:09:47.385087+0000","last_unstale":"2023-11-28T03:09:47.754226+0000","last_undegraded":"2023-11-28T03:09:47.754226+0000","last_fullsized":"2023-11-28T03:09:47.754226+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768762+0000","last_change":"2023-11-28T03:09:47.762089+0000","last_active":"2023-11-28T03:09:47.768762+0000","last_peered":"2023-11-28T03:09:47.768762+0000","last_clean":"2023-11-28T03:09:47.768762+0000","last_became_active":"2023-11-28T03:09:47.761503+0000","last_became_peered":"2023-11-28T03:09:47.761503+0000","last_unstale":"2023-11-28T03:09:47.768762+0000","last_undegraded":"2023-11-28T03:09:47.768762+0000","last_fullsized":"2023-11-28T03:09:47.768762+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748921+0000","last_change":"2023-11-28T03:09:47.393246+0000","last_active":"2023-11-28T03:09:47.748921+0000","last_peered":"2023-11-28T03:09:47.748921+0000","last_clean":"2023-11-28T03:09:47.748921+0000","last_became_active":"2023-11-28T03:09:47.392973+0000","last_became_peered":"2023-11-28T03:09:47.392973+0000","last_unstale":"2023-11-28T03:09:47.748921+0000","last_undegraded":"2023-11-28T03:09:47.748921+0000","last_fullsized":"2023-11-28T03:09:47.748921+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775496+0000","last_change":"2023-11-28T03:09:47.768410+0000","last_active":"2023-11-28T03:09:47.775496+0000","last_peered":"2023-11-28T03:09:47.775496+0000","last_clean":"2023-11-28T03:09:47.775496+0000","last_became_active":"2023-11-28T03:09:47.767775+0000","last_became_peered":"2023-11-28T03:09:47.767775+0000","last_unstale":"2023-11-28T03:09:47.775496+0000","last_undegraded":"2023-11-28T03:09:47.775496+0000","last_fullsized":"2023-11-28T03:09:47.775496+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755760+0000","last_change":"2023-11-28T03:09:47.385915+0000","last_active":"2023-11-28T03:09:47.755760+0000","last_peered":"2023-11-28T03:09:47.755760+0000","last_clean":"2023-11-28T03:09:47.755760+0000","last_became_active":"2023-11-28T03:09:47.385628+0000","last_became_peered":"2023-11-28T03:09:47.385628+0000","last_unstale":"2023-11-28T03:09:47.755760+0000","last_undegraded":"2023-11-28T03:09:47.755760+0000","last_fullsized":"2023-11-28T03:09:47.755760+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775660+0000","last_change":"2023-11-28T03:09:47.773362+0000","last_active":"2023-11-28T03:09:47.775660+0000","last_peered":"2023-11-28T03:09:47.775660+0000","last_clean":"2023-11-28T03:09:47.775660+0000","last_became_active":"2023-11-28T03:09:47.773071+0000","last_became_peered":"2023-11-28T03:09:47.773071+0000","last_unstale":"2023-11-28T03:09:47.775660+0000","last_undegraded":"2023-11-28T03:09:47.775660+0000","last_fullsized":"2023-11-28T03:09:47.775660+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752888+0000","last_change":"2023-11-28T03:09:47.386808+0000","last_active":"2023-11-28T03:09:47.752888+0000","last_peered":"2023-11-28T03:09:47.752888+0000","last_clean":"2023-11-28T03:09:47.752888+0000","last_became_active":"2023-11-28T03:09:47.386529+0000","last_became_peered":"2023-11-28T03:09:47.386529+0000","last_unstale":"2023-11-28T03:09:47.752888+0000","last_undegraded":"2023-11-28T03:09:47.752888+0000","last_fullsized":"2023-11-28T03:09:47.752888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.765393+0000","last_change":"2023-11-28T03:09:47.760194+0000","last_active":"2023-11-28T03:09:47.765393+0000","last_peered":"2023-11-28T03:09:47.765393+0000","last_clean":"2023-11-28T03:09:47.765393+0000","last_became_active":"2023-11-28T03:09:47.759941+0000","last_became_peered":"2023-11-28T03:09:47.759941+0000","last_unstale":"2023-11-28T03:09:47.765393+0000","last_undegraded":"2023-11-28T03:09:47.765393+0000","last_fullsized":"2023-11-28T03:09:47.765393+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747508+0000","last_change":"2023-11-28T03:09:47.392973+0000","last_active":"2023-11-28T03:09:47.747508+0000","last_peered":"2023-11-28T03:09:47.747508+0000","last_clean":"2023-11-28T03:09:47.747508+0000","last_became_active":"2023-11-28T03:09:47.392709+0000","last_became_peered":"2023-11-28T03:09:47.392709+0000","last_unstale":"2023-11-28T03:09:47.747508+0000","last_undegraded":"2023-11-28T03:09:47.747508+0000","last_fullsized":"2023-11-28T03:09:47.747508+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766656+0000","last_change":"2023-11-28T03:09:47.761218+0000","last_active":"2023-11-28T03:09:47.766656+0000","last_peered":"2023-11-28T03:09:47.766656+0000","last_clean":"2023-11-28T03:09:47.766656+0000","last_became_active":"2023-11-28T03:09:47.760887+0000","last_became_peered":"2023-11-28T03:09:47.760887+0000","last_unstale":"2023-11-28T03:09:47.766656+0000","last_undegraded":"2023-11-28T03:09:47.766656+0000","last_fullsized":"2023-11-28T03:09:47.766656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767480+0000","last_change":"2023-11-28T03:09:47.388373+0000","last_active":"2023-11-28T03:09:47.767480+0000","last_peered":"2023-11-28T03:09:47.767480+0000","last_clean":"2023-11-28T03:09:47.767480+0000","last_became_active":"2023-11-28T03:09:47.388085+0000","last_became_peered":"2023-11-28T03:09:47.388085+0000","last_unstale":"2023-11-28T03:09:47.767480+0000","last_undegraded":"2023-11-28T03:09:47.767480+0000","last_fullsized":"2023-11-28T03:09:47.767480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775454+0000","last_change":"2023-11-28T03:09:47.771133+0000","last_active":"2023-11-28T03:09:47.775454+0000","last_peered":"2023-11-28T03:09:47.775454+0000","last_clean":"2023-11-28T03:09:47.775454+0000","last_became_active":"2023-11-28T03:09:47.770868+0000","last_became_peered":"2023-11-28T03:09:47.770868+0000","last_unstale":"2023-11-28T03:09:47.775454+0000","last_undegraded":"2023-11-28T03:09:47.775454+0000","last_fullsized":"2023-11-28T03:09:47.775454+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767540+0000","last_change":"2023-11-28T03:09:47.386456+0000","last_active":"2023-11-28T03:09:47.767540+0000","last_peered":"2023-11-28T03:09:47.767540+0000","last_clean":"2023-11-28T03:09:47.767540+0000","last_became_active":"2023-11-28T03:09:47.385555+0000","last_became_peered":"2023-11-28T03:09:47.385555+0000","last_unstale":"2023-11-28T03:09:47.767540+0000","last_undegraded":"2023-11-28T03:09:47.767540+0000","last_fullsized":"2023-11-28T03:09:47.767540+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365285+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747764+0000","last_change":"2023-11-28T03:09:47.389262+0000","last_active":"2023-11-28T03:09:47.747764+0000","last_peered":"2023-11-28T03:09:47.747764+0000","last_clean":"2023-11-28T03:09:47.747764+0000","last_became_active":"2023-11-28T03:09:47.388988+0000","last_became_peered":"2023-11-28T03:09:47.388988+0000","last_unstale":"2023-11-28T03:09:47.747764+0000","last_undegraded":"2023-11-28T03:09:47.747764+0000","last_fullsized":"2023-11-28T03:09:47.747764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.763347+0000","last_change":"2023-11-28T03:09:47.760154+0000","last_active":"2023-11-28T03:09:47.763347+0000","last_peered":"2023-11-28T03:09:47.763347+0000","last_clean":"2023-11-28T03:09:47.763347+0000","last_became_active":"2023-11-28T03:09:47.759874+0000","last_became_peered":"2023-11-28T03:09:47.759874+0000","last_unstale":"2023-11-28T03:09:47.763347+0000","last_undegraded":"2023-11-28T03:09:47.763347+0000","last_fullsized":"2023-11-28T03:09:47.763347+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752780+0000","last_change":"2023-11-28T03:09:47.381553+0000","last_active":"2023-11-28T03:09:47.752780+0000","last_peered":"2023-11-28T03:09:47.752780+0000","last_clean":"2023-11-28T03:09:47.752780+0000","last_became_active":"2023-11-28T03:09:47.381281+0000","last_became_peered":"2023-11-28T03:09:47.381281+0000","last_unstale":"2023-11-28T03:09:47.752780+0000","last_undegraded":"2023-11-28T03:09:47.752780+0000","last_fullsized":"2023-11-28T03:09:47.752780+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767987+0000","last_change":"2023-11-28T03:09:47.763223+0000","last_active":"2023-11-28T03:09:47.767987+0000","last_peered":"2023-11-28T03:09:47.767987+0000","last_clean":"2023-11-28T03:09:47.767987+0000","last_became_active":"2023-11-28T03:09:47.762958+0000","last_became_peered":"2023-11-28T03:09:47.762958+0000","last_unstale":"2023-11-28T03:09:47.767987+0000","last_undegraded":"2023-11-28T03:09:47.767987+0000","last_fullsized":"2023-11-28T03:09:47.767987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367186+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775459+0000","last_change":"2023-11-28T03:09:47.768612+0000","last_active":"2023-11-28T03:09:47.775459+0000","last_peered":"2023-11-28T03:09:47.775459+0000","last_clean":"2023-11-28T03:09:47.775459+0000","last_became_active":"2023-11-28T03:09:47.767705+0000","last_became_peered":"2023-11-28T03:09:47.767705+0000","last_unstale":"2023-11-28T03:09:47.775459+0000","last_undegraded":"2023-11-28T03:09:47.775459+0000","last_fullsized":"2023-11-28T03:09:47.775459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364493+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.762485+0000","last_change":"2023-11-28T03:09:47.760482+0000","last_active":"2023-11-28T03:09:47.762485+0000","last_peered":"2023-11-28T03:09:47.762485+0000","last_clean":"2023-11-28T03:09:47.762485+0000","last_became_active":"2023-11-28T03:09:47.759849+0000","last_became_peered":"2023-11-28T03:09:47.759849+0000","last_unstale":"2023-11-28T03:09:47.762485+0000","last_undegraded":"2023-11-28T03:09:47.762485+0000","last_fullsized":"2023-11-28T03:09:47.762485+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.754087+0000","last_change":"2023-11-28T03:09:47.387939+0000","last_active":"2023-11-28T03:09:47.754087+0000","last_peered":"2023-11-28T03:09:47.754087+0000","last_clean":"2023-11-28T03:09:47.754087+0000","last_became_active":"2023-11-28T03:09:47.387663+0000","last_became_peered":"2023-11-28T03:09:47.387663+0000","last_unstale":"2023-11-28T03:09:47.754087+0000","last_undegraded":"2023-11-28T03:09:47.754087+0000","last_fullsized":"2023-11-28T03:09:47.754087+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766637+0000","last_change":"2023-11-28T03:09:47.761146+0000","last_active":"2023-11-28T03:09:47.766637+0000","last_peered":"2023-11-28T03:09:47.766637+0000","last_clean":"2023-11-28T03:09:47.766637+0000","last_became_active":"2023-11-28T03:09:47.760689+0000","last_became_peered":"2023-11-28T03:09:47.760689+0000","last_unstale":"2023-11-28T03:09:47.766637+0000","last_undegraded":"2023-11-28T03:09:47.766637+0000","last_fullsized":"2023-11-28T03:09:47.766637+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.366689+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773247+0000","last_change":"2023-11-28T03:09:47.764684+0000","last_active":"2023-11-28T03:09:47.773247+0000","last_peered":"2023-11-28T03:09:47.773247+0000","last_clean":"2023-11-28T03:09:47.773247+0000","last_became_active":"2023-11-28T03:09:47.764420+0000","last_became_peered":"2023-11-28T03:09:47.764420+0000","last_unstale":"2023-11-28T03:09:47.773247+0000","last_undegraded":"2023-11-28T03:09:47.773247+0000","last_fullsized":"2023-11-28T03:09:47.773247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750830+0000","last_change":"2023-11-28T03:09:47.387921+0000","last_active":"2023-11-28T03:09:47.750830+0000","last_peered":"2023-11-28T03:09:47.750830+0000","last_clean":"2023-11-28T03:09:47.750830+0000","last_became_active":"2023-11-28T03:09:47.387346+0000","last_became_peered":"2023-11-28T03:09:47.387346+0000","last_unstale":"2023-11-28T03:09:47.750830+0000","last_undegraded":"2023-11-28T03:09:47.750830+0000","last_fullsized":"2023-11-28T03:09:47.750830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776192+0000","last_change":"2023-11-28T03:09:47.773540+0000","last_active":"2023-11-28T03:09:47.776192+0000","last_peered":"2023-11-28T03:09:47.776192+0000","last_clean":"2023-11-28T03:09:47.776192+0000","last_became_active":"2023-11-28T03:09:47.773261+0000","last_became_peered":"2023-11-28T03:09:47.773261+0000","last_unstale":"2023-11-28T03:09:47.776192+0000","last_undegraded":"2023-11-28T03:09:47.776192+0000","last_fullsized":"2023-11-28T03:09:47.776192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"125'3","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772664+0000","last_change":"2023-11-28T03:09:47.389637+0000","last_active":"2023-11-28T03:09:47.772664+0000","last_peered":"2023-11-28T03:09:47.772664+0000","last_clean":"2023-11-28T03:09:47.772664+0000","last_became_active":"2023-11-28T03:09:47.389333+0000","last_became_peered":"2023-11-28T03:09:47.389333+0000","last_unstale":"2023-11-28T03:09:47.772664+0000","last_undegraded":"2023-11-28T03:09:47.772664+0000","last_fullsized":"2023-11-28T03:09:47.772664+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773504+0000","last_change":"2023-11-28T03:09:47.766624+0000","last_active":"2023-11-28T03:09:47.773504+0000","last_peered":"2023-11-28T03:09:47.773504+0000","last_clean":"2023-11-28T03:09:47.773504+0000","last_became_active":"2023-11-28T03:09:47.765923+0000","last_became_peered":"2023-11-28T03:09:47.765923+0000","last_unstale":"2023-11-28T03:09:47.773504+0000","last_undegraded":"2023-11-28T03:09:47.773504+0000","last_fullsized":"2023-11-28T03:09:47.773504+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767181+0000","last_change":"2023-11-28T03:09:47.388066+0000","last_active":"2023-11-28T03:09:47.767181+0000","last_peered":"2023-11-28T03:09:47.767181+0000","last_clean":"2023-11-28T03:09:47.767181+0000","last_became_active":"2023-11-28T03:09:47.387767+0000","last_became_peered":"2023-11-28T03:09:47.387767+0000","last_unstale":"2023-11-28T03:09:47.767181+0000","last_undegraded":"2023-11-28T03:09:47.767181+0000","last_fullsized":"2023-11-28T03:09:47.767181+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.13","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770865+0000","last_change":"2023-11-28T03:09:47.395717+0000","last_active":"2023-11-28T03:09:47.770865+0000","last_peered":"2023-11-28T03:09:47.770865+0000","last_clean":"2023-11-28T03:09:47.770865+0000","last_became_active":"2023-11-28T03:09:47.395484+0000","last_became_peered":"2023-11-28T03:09:47.395484+0000","last_unstale":"2023-11-28T03:09:47.770865+0000","last_undegraded":"2023-11-28T03:09:47.770865+0000","last_fullsized":"2023-11-28T03:09:47.770865+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767178+0000","last_change":"2023-11-28T03:09:47.762679+0000","last_active":"2023-11-28T03:09:47.767178+0000","last_peered":"2023-11-28T03:09:47.767178+0000","last_clean":"2023-11-28T03:09:47.767178+0000","last_became_active":"2023-11-28T03:09:47.762384+0000","last_became_peered":"2023-11-28T03:09:47.762384+0000","last_unstale":"2023-11-28T03:09:47.767178+0000","last_undegraded":"2023-11-28T03:09:47.767178+0000","last_fullsized":"2023-11-28T03:09:47.767178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752947+0000","last_change":"2023-11-28T03:09:47.381351+0000","last_active":"2023-11-28T03:09:47.752947+0000","last_peered":"2023-11-28T03:09:47.752947+0000","last_clean":"2023-11-28T03:09:47.752947+0000","last_became_active":"2023-11-28T03:09:47.381096+0000","last_became_peered":"2023-11-28T03:09:47.381096+0000","last_unstale":"2023-11-28T03:09:47.752947+0000","last_undegraded":"2023-11-28T03:09:47.752947+0000","last_fullsized":"2023-11-28T03:09:47.752947+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767695+0000","last_change":"2023-11-28T03:09:47.766180+0000","last_active":"2023-11-28T03:09:47.767695+0000","last_peered":"2023-11-28T03:09:47.767695+0000","last_clean":"2023-11-28T03:09:47.767695+0000","last_became_active":"2023-11-28T03:09:47.765901+0000","last_became_peered":"2023-11-28T03:09:47.765901+0000","last_unstale":"2023-11-28T03:09:47.767695+0000","last_undegraded":"2023-11-28T03:09:47.767695+0000","last_fullsized":"2023-11-28T03:09:47.767695+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755399+0000","last_change":"2023-11-28T03:09:47.389094+0000","last_active":"2023-11-28T03:09:47.755399+0000","last_peered":"2023-11-28T03:09:47.755399+0000","last_clean":"2023-11-28T03:09:47.755399+0000","last_became_active":"2023-11-28T03:09:47.388840+0000","last_became_peered":"2023-11-28T03:09:47.388840+0000","last_unstale":"2023-11-28T03:09:47.755399+0000","last_undegraded":"2023-11-28T03:09:47.755399+0000","last_fullsized":"2023-11-28T03:09:47.755399+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773011+0000","last_change":"2023-11-28T03:09:47.767119+0000","last_active":"2023-11-28T03:09:47.773011+0000","last_peered":"2023-11-28T03:09:47.773011+0000","last_clean":"2023-11-28T03:09:47.773011+0000","last_became_active":"2023-11-28T03:09:47.766500+0000","last_became_peered":"2023-11-28T03:09:47.766500+0000","last_unstale":"2023-11-28T03:09:47.773011+0000","last_undegraded":"2023-11-28T03:09:47.773011+0000","last_fullsized":"2023-11-28T03:09:47.773011+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.364750+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766608+0000","last_change":"2023-11-28T03:09:47.761013+0000","last_active":"2023-11-28T03:09:47.766608+0000","last_peered":"2023-11-28T03:09:47.766608+0000","last_clean":"2023-11-28T03:09:47.766608+0000","last_became_active":"2023-11-28T03:09:47.760574+0000","last_became_peered":"2023-11-28T03:09:47.760574+0000","last_unstale":"2023-11-28T03:09:47.766608+0000","last_undegraded":"2023-11-28T03:09:47.766608+0000","last_fullsized":"2023-11-28T03:09:47.766608+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768209+0000","last_change":"2023-11-28T03:09:47.394060+0000","last_active":"2023-11-28T03:09:47.768209+0000","last_peered":"2023-11-28T03:09:47.768209+0000","last_clean":"2023-11-28T03:09:47.768209+0000","last_became_active":"2023-11-28T03:09:47.393707+0000","last_became_peered":"2023-11-28T03:09:47.393707+0000","last_unstale":"2023-11-28T03:09:47.768209+0000","last_undegraded":"2023-11-28T03:09:47.768209+0000","last_fullsized":"2023-11-28T03:09:47.768209+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750926+0000","last_change":"2023-11-28T03:09:47.388105+0000","last_active":"2023-11-28T03:09:47.750926+0000","last_peered":"2023-11-28T03:09:47.750926+0000","last_clean":"2023-11-28T03:09:47.750926+0000","last_became_active":"2023-11-28T03:09:47.387671+0000","last_became_peered":"2023-11-28T03:09:47.387671+0000","last_unstale":"2023-11-28T03:09:47.750926+0000","last_undegraded":"2023-11-28T03:09:47.750926+0000","last_fullsized":"2023-11-28T03:09:47.750926+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770329+0000","last_change":"2023-11-28T03:09:47.765129+0000","last_active":"2023-11-28T03:09:47.770329+0000","last_peered":"2023-11-28T03:09:47.770329+0000","last_clean":"2023-11-28T03:09:47.770329+0000","last_became_active":"2023-11-28T03:09:47.764865+0000","last_became_peered":"2023-11-28T03:09:47.764865+0000","last_unstale":"2023-11-28T03:09:47.770329+0000","last_undegraded":"2023-11-28T03:09:47.770329+0000","last_fullsized":"2023-11-28T03:09:47.770329+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753891+0000","last_change":"2023-11-28T03:09:47.383234+0000","last_active":"2023-11-28T03:09:47.753891+0000","last_peered":"2023-11-28T03:09:47.753891+0000","last_clean":"2023-11-28T03:09:47.753891+0000","last_became_active":"2023-11-28T03:09:47.382933+0000","last_became_peered":"2023-11-28T03:09:47.382933+0000","last_unstale":"2023-11-28T03:09:47.753891+0000","last_undegraded":"2023-11-28T03:09:47.753891+0000","last_fullsized":"2023-11-28T03:09:47.753891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747737+0000","last_change":"2023-11-28T03:09:47.391565+0000","last_active":"2023-11-28T03:09:47.747737+0000","last_peered":"2023-11-28T03:09:47.747737+0000","last_clean":"2023-11-28T03:09:47.747737+0000","last_became_active":"2023-11-28T03:09:47.391157+0000","last_became_peered":"2023-11-28T03:09:47.391157+0000","last_unstale":"2023-11-28T03:09:47.747737+0000","last_undegraded":"2023-11-28T03:09:47.747737+0000","last_fullsized":"2023-11-28T03:09:47.747737+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776195+0000","last_change":"2023-11-28T03:09:47.773506+0000","last_active":"2023-11-28T03:09:47.776195+0000","last_peered":"2023-11-28T03:09:47.776195+0000","last_clean":"2023-11-28T03:09:47.776195+0000","last_became_active":"2023-11-28T03:09:47.773240+0000","last_became_peered":"2023-11-28T03:09:47.773240+0000","last_unstale":"2023-11-28T03:09:47.776195+0000","last_undegraded":"2023-11-28T03:09:47.776195+0000","last_fullsized":"2023-11-28T03:09:47.776195+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.365009+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776171+0000","last_change":"2023-11-28T03:09:47.773468+0000","last_active":"2023-11-28T03:09:47.776171+0000","last_peered":"2023-11-28T03:09:47.776171+0000","last_clean":"2023-11-28T03:09:47.776171+0000","last_became_active":"2023-11-28T03:09:47.773190+0000","last_became_peered":"2023-11-28T03:09:47.773190+0000","last_unstale":"2023-11-28T03:09:47.776171+0000","last_undegraded":"2023-11-28T03:09:47.776171+0000","last_fullsized":"2023-11-28T03:09:47.776171+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755888+0000","last_change":"2023-11-28T03:09:47.389380+0000","last_active":"2023-11-28T03:09:47.755888+0000","last_peered":"2023-11-28T03:09:47.755888+0000","last_clean":"2023-11-28T03:09:47.755888+0000","last_became_active":"2023-11-28T03:09:47.388732+0000","last_became_peered":"2023-11-28T03:09:47.388732+0000","last_unstale":"2023-11-28T03:09:47.755888+0000","last_undegraded":"2023-11-28T03:09:47.755888+0000","last_fullsized":"2023-11-28T03:09:47.755888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772694+0000","last_change":"2023-11-28T03:09:47.763808+0000","last_active":"2023-11-28T03:09:47.772694+0000","last_peered":"2023-11-28T03:09:47.772694+0000","last_clean":"2023-11-28T03:09:47.772694+0000","last_became_active":"2023-11-28T03:09:47.763538+0000","last_became_peered":"2023-11-28T03:09:47.763538+0000","last_unstale":"2023-11-28T03:09:47.772694+0000","last_undegraded":"2023-11-28T03:09:47.772694+0000","last_fullsized":"2023-11-28T03:09:47.772694+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755809+0000","last_change":"2023-11-28T03:09:47.384792+0000","last_active":"2023-11-28T03:09:47.755809+0000","last_peered":"2023-11-28T03:09:47.755809+0000","last_clean":"2023-11-28T03:09:47.755809+0000","last_became_active":"2023-11-28T03:09:47.384525+0000","last_became_peered":"2023-11-28T03:09:47.384525+0000","last_unstale":"2023-11-28T03:09:47.755809+0000","last_undegraded":"2023-11-28T03:09:47.755809+0000","last_fullsized":"2023-11-28T03:09:47.755809+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755834+0000","last_change":"2023-11-28T03:09:47.391584+0000","last_active":"2023-11-28T03:09:47.755834+0000","last_peered":"2023-11-28T03:09:47.755834+0000","last_clean":"2023-11-28T03:09:47.755834+0000","last_became_active":"2023-11-28T03:09:47.391149+0000","last_became_peered":"2023-11-28T03:09:47.391149+0000","last_unstale":"2023-11-28T03:09:47.755834+0000","last_undegraded":"2023-11-28T03:09:47.755834+0000","last_fullsized":"2023-11-28T03:09:47.755834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772139+0000","last_change":"2023-11-28T03:09:47.764924+0000","last_active":"2023-11-28T03:09:47.772139+0000","last_peered":"2023-11-28T03:09:47.772139+0000","last_clean":"2023-11-28T03:09:47.772139+0000","last_became_active":"2023-11-28T03:09:47.764678+0000","last_became_peered":"2023-11-28T03:09:47.764678+0000","last_unstale":"2023-11-28T03:09:47.772139+0000","last_undegraded":"2023-11-28T03:09:47.772139+0000","last_fullsized":"2023-11-28T03:09:47.772139+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755688+0000","last_change":"2023-11-28T03:09:47.387312+0000","last_active":"2023-11-28T03:09:47.755688+0000","last_peered":"2023-11-28T03:09:47.755688+0000","last_clean":"2023-11-28T03:09:47.755688+0000","last_became_active":"2023-11-28T03:09:47.386963+0000","last_became_peered":"2023-11-28T03:09:47.386963+0000","last_unstale":"2023-11-28T03:09:47.755688+0000","last_undegraded":"2023-11-28T03:09:47.755688+0000","last_fullsized":"2023-11-28T03:09:47.755688+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747844+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:09:47.747844+0000","last_peered":"2023-11-28T03:09:47.747844+0000","last_clean":"2023-11-28T03:09:47.747844+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:09:47.747844+0000","last_undegraded":"2023-11-28T03:09:47.747844+0000","last_fullsized":"2023-11-28T03:09:47.747844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764001+0000","last_change":"2023-11-28T03:09:47.762200+0000","last_active":"2023-11-28T03:09:47.764001+0000","last_peered":"2023-11-28T03:09:47.764001+0000","last_clean":"2023-11-28T03:09:47.764001+0000","last_became_active":"2023-11-28T03:09:47.761919+0000","last_became_peered":"2023-11-28T03:09:47.761919+0000","last_unstale":"2023-11-28T03:09:47.764001+0000","last_undegraded":"2023-11-28T03:09:47.764001+0000","last_fullsized":"2023-11-28T03:09:47.764001+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766654+0000","last_change":"2023-11-28T03:09:47.383551+0000","last_active":"2023-11-28T03:09:47.766654+0000","last_peered":"2023-11-28T03:09:47.766654+0000","last_clean":"2023-11-28T03:09:47.766654+0000","last_became_active":"2023-11-28T03:09:47.382466+0000","last_became_peered":"2023-11-28T03:09:47.382466+0000","last_unstale":"2023-11-28T03:09:47.766654+0000","last_undegraded":"2023-11-28T03:09:47.766654+0000","last_fullsized":"2023-11-28T03:09:47.766654+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773479+0000","last_change":"2023-11-28T03:09:47.765348+0000","last_active":"2023-11-28T03:09:47.773479+0000","last_peered":"2023-11-28T03:09:47.773479+0000","last_clean":"2023-11-28T03:09:47.773479+0000","last_became_active":"2023-11-28T03:09:47.765087+0000","last_became_peered":"2023-11-28T03:09:47.765087+0000","last_unstale":"2023-11-28T03:09:47.773479+0000","last_undegraded":"2023-11-28T03:09:47.773479+0000","last_fullsized":"2023-11-28T03:09:47.773479+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T03:09:46.356969+0000","last_change":"2023-11-28T03:09:46.367145+0000","last_active":"2023-11-28T03:09:46.356969+0000","last_peered":"2023-11-28T03:09:46.356969+0000","last_clean":"2023-11-28T03:09:46.356969+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:46.356969+0000","last_undegraded":"2023-11-28T03:09:46.356969+0000","last_fullsized":"2023-11-28T03:09:46.356969+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":104,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755522+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:09:47.755522+0000","last_peered":"2023-11-28T03:09:47.755522+0000","last_clean":"2023-11-28T03:09:47.755522+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:09:47.755522+0000","last_undegraded":"2023-11-28T03:09:47.755522+0000","last_fullsized":"2023-11-28T03:09:47.755522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T03:09:47.361629+0000","last_change":"2023-11-28T03:09:47.361629+0000","last_active":"2023-11-28T03:09:47.361629+0000","last_peered":"2023-11-28T03:09:47.361629+0000","last_clean":"2023-11-28T03:09:47.361629+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:09:47.361629+0000","last_undegraded":"2023-11-28T03:09:47.361629+0000","last_fullsized":"2023-11-28T03:09:47.361629+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.e","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.761975+0000","last_change":"2023-11-28T03:09:47.386633+0000","last_active":"2023-11-28T03:09:47.761975+0000","last_peered":"2023-11-28T03:09:47.761975+0000","last_clean":"2023-11-28T03:09:47.761975+0000","last_became_active":"2023-11-28T03:09:47.386182+0000","last_became_peered":"2023-11-28T03:09:47.386182+0000","last_unstale":"2023-11-28T03:09:47.761975+0000","last_undegraded":"2023-11-28T03:09:47.761975+0000","last_fullsized":"2023-11-28T03:09:47.761975+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":94,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-11-28T03:09:44.457086+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:09:44.457086+0000","last_peered":"2023-11-28T03:09:44.457086+0000","last_clean":"2023-11-28T03:09:44.457086+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:09:44.457086+0000","last_undegraded":"2023-11-28T03:09:44.457086+0000","last_fullsized":"2023-11-28T03:09:44.457086+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752931+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:09:47.752931+0000","last_peered":"2023-11-28T03:09:47.752931+0000","last_clean":"2023-11-28T03:09:47.752931+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:09:47.752931+0000","last_undegraded":"2023-11-28T03:09:47.752931+0000","last_fullsized":"2023-11-28T03:09:47.752931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764113+0000","last_change":"2023-11-28T03:09:47.758283+0000","last_active":"2023-11-28T03:09:47.764113+0000","last_peered":"2023-11-28T03:09:47.764113+0000","last_clean":"2023-11-28T03:09:47.764113+0000","last_became_active":"2023-11-28T03:09:47.757999+0000","last_became_peered":"2023-11-28T03:09:47.757999+0000","last_unstale":"2023-11-28T03:09:47.764113+0000","last_undegraded":"2023-11-28T03:09:47.764113+0000","last_fullsized":"2023-11-28T03:09:47.764113+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752605+0000","last_change":"2023-11-28T03:09:47.385187+0000","last_active":"2023-11-28T03:09:47.752605+0000","last_peered":"2023-11-28T03:09:47.752605+0000","last_clean":"2023-11-28T03:09:47.752605+0000","last_became_active":"2023-11-28T03:09:47.384894+0000","last_became_peered":"2023-11-28T03:09:47.384894+0000","last_unstale":"2023-11-28T03:09:47.752605+0000","last_undegraded":"2023-11-28T03:09:47.752605+0000","last_fullsized":"2023-11-28T03:09:47.752605+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753007+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:09:47.753007+0000","last_peered":"2023-11-28T03:09:47.753007+0000","last_clean":"2023-11-28T03:09:47.753007+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:09:47.753007+0000","last_undegraded":"2023-11-28T03:09:47.753007+0000","last_fullsized":"2023-11-28T03:09:47.753007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772799+0000","last_change":"2023-11-28T03:09:47.769354+0000","last_active":"2023-11-28T03:09:47.772799+0000","last_peered":"2023-11-28T03:09:47.772799+0000","last_clean":"2023-11-28T03:09:47.772799+0000","last_became_active":"2023-11-28T03:09:47.769091+0000","last_became_peered":"2023-11-28T03:09:47.769091+0000","last_unstale":"2023-11-28T03:09:47.772799+0000","last_undegraded":"2023-11-28T03:09:47.772799+0000","last_fullsized":"2023-11-28T03:09:47.772799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755619+0000","last_change":"2023-11-28T03:09:47.385514+0000","last_active":"2023-11-28T03:09:47.755619+0000","last_peered":"2023-11-28T03:09:47.755619+0000","last_clean":"2023-11-28T03:09:47.755619+0000","last_became_active":"2023-11-28T03:09:47.384387+0000","last_became_peered":"2023-11-28T03:09:47.384387+0000","last_unstale":"2023-11-28T03:09:47.755619+0000","last_undegraded":"2023-11-28T03:09:47.755619+0000","last_fullsized":"2023-11-28T03:09:47.755619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752806+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:09:47.752806+0000","last_peered":"2023-11-28T03:09:47.752806+0000","last_clean":"2023-11-28T03:09:47.752806+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:09:47.752806+0000","last_undegraded":"2023-11-28T03:09:47.752806+0000","last_fullsized":"2023-11-28T03:09:47.752806+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766944+0000","last_change":"2023-11-28T03:09:47.762725+0000","last_active":"2023-11-28T03:09:47.766944+0000","last_peered":"2023-11-28T03:09:47.766944+0000","last_clean":"2023-11-28T03:09:47.766944+0000","last_became_active":"2023-11-28T03:09:47.762436+0000","last_became_peered":"2023-11-28T03:09:47.762436+0000","last_unstale":"2023-11-28T03:09:47.766944+0000","last_undegraded":"2023-11-28T03:09:47.766944+0000","last_fullsized":"2023-11-28T03:09:47.766944+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766696+0000","last_change":"2023-11-28T03:09:47.388594+0000","last_active":"2023-11-28T03:09:47.766696+0000","last_peered":"2023-11-28T03:09:47.766696+0000","last_clean":"2023-11-28T03:09:47.766696+0000","last_became_active":"2023-11-28T03:09:47.387910+0000","last_became_peered":"2023-11-28T03:09:47.387910+0000","last_unstale":"2023-11-28T03:09:47.766696+0000","last_undegraded":"2023-11-28T03:09:47.766696+0000","last_fullsized":"2023-11-28T03:09:47.766696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748067+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:09:47.748067+0000","last_peered":"2023-11-28T03:09:47.748067+0000","last_clean":"2023-11-28T03:09:47.748067+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:09:47.748067+0000","last_undegraded":"2023-11-28T03:09:47.748067+0000","last_fullsized":"2023-11-28T03:09:47.748067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775515+0000","last_change":"2023-11-28T03:09:47.771650+0000","last_active":"2023-11-28T03:09:47.775515+0000","last_peered":"2023-11-28T03:09:47.775515+0000","last_clean":"2023-11-28T03:09:47.775515+0000","last_became_active":"2023-11-28T03:09:47.768197+0000","last_became_peered":"2023-11-28T03:09:47.768197+0000","last_unstale":"2023-11-28T03:09:47.775515+0000","last_undegraded":"2023-11-28T03:09:47.775515+0000","last_fullsized":"2023-11-28T03:09:47.775515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755657+0000","last_change":"2023-11-28T03:09:47.384548+0000","last_active":"2023-11-28T03:09:47.755657+0000","last_peered":"2023-11-28T03:09:47.755657+0000","last_clean":"2023-11-28T03:09:47.755657+0000","last_became_active":"2023-11-28T03:09:47.383855+0000","last_became_peered":"2023-11-28T03:09:47.383855+0000","last_unstale":"2023-11-28T03:09:47.755657+0000","last_undegraded":"2023-11-28T03:09:47.755657+0000","last_fullsized":"2023-11-28T03:09:47.755657+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.756127+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:09:47.756127+0000","last_peered":"2023-11-28T03:09:47.756127+0000","last_clean":"2023-11-28T03:09:47.756127+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:09:47.756127+0000","last_undegraded":"2023-11-28T03:09:47.756127+0000","last_fullsized":"2023-11-28T03:09:47.756127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":1414,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":31,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28,"ondisk_log_size":28,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1560576,"data_stored":776680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.34499999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.56799999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.5},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46600000000000003,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.72199999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302724,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069116,"statfs":{"total":96636764160,"available":96326774784,"internally_reserved":0,"allocated":1568768,"data_stored":777256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.71399999999999997,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.379},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":1.506,"15min":1.506},"last":0.438}]},{"osd":2,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.53300000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58699999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.60399999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.64500000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1572864,"data_stored":778140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.54700000000000004,"5min":0.67800000000000005,"15min":0.70399999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.371},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.67100000000000004},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.307,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.64900000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.628,"15min":0.63400000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.65200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"last":0.375},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.75600000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.97799999999999998},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.91900000000000004,"15min":0.99299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.629,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.94199999999999995,"15min":1.038},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78700000000000003,"5min":0.89400000000000002,"15min":0.94099999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":1.024},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.88300000000000001},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575053,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302724,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069116,"statfs":{"total":96636764160,"available":96326774784,"internally_reserved":0,"allocated":1568768,"data_stored":777324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.217,"15min":0.184},"max":{"1min":0.46100000000000002,"5min":0.56599999999999995,"15min":0.56999999999999995},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47499999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.246},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.377,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.311},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.41699999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.61499999999999999,"5min":0.621,"15min":0.65400000000000003},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":1.0489999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":1.679,"15min":1.679},"last":0.436},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42199999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.97499999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.97399999999999998,"15min":1.046},"last":0.38400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575054,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302720,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069120,"statfs":{"total":96636764160,"available":96326778880,"internally_reserved":0,"allocated":1564672,"data_stored":776694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34799999999999998,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.48499999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.41699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.53800000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33000000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34599999999999997,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.5,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.58499999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.371}]},{"osd":3,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.41799999999999998},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.58199999999999996,"15min":0.67400000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88600000000000001,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.98599999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":1.004},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.875,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.72399999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.63},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.60099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575053,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302712,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069128,"statfs":{"total":96636764160,"available":96326787072,"internally_reserved":0,"allocated":1556480,"data_stored":776590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52300000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.35699999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60099999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.42099999999999999},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.68200000000000005},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82599999999999996,"5min":1.581,"15min":1.581},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.94199999999999995},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.97899999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.93600000000000005},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":0.90900000000000003,"15min":0.996},"last":0.625},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":0.93500000000000005,"15min":0.97199999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.60299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575051,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52100000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.438},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.505}]},{"osd":1,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53600000000000003,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.39600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.372},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.495}]},{"osd":3,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.84599999999999997,"15min":0.878},"last":0.45800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.496,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.52600000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.53900000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575053,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302664,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069176,"statfs":{"total":96636764160,"available":96326836224,"internally_reserved":0,"allocated":1507328,"data_stored":749807,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68400000000000005,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.77900000000000003},"last":0.55700000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.53500000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.64200000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70099999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47599999999999998,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.69899999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78200000000000003,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.71999999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.73499999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:09:53.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:09:53.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:09:53.665 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:09:53.664+0000 7f6f6373a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:09:55.136 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:09:55.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.261+0000 7f9c3b451700 1 Processor -- start 2023-11-28T03:09:55.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 -- start start 2023-11-28T03:09:55.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c341544d0 0x7f9c34155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c340673c0 0x7f9c3415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c340683d0 con 0x7f9c34066f40 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c34067850 con 0x7f9c341544d0 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.262+0000 7f9c3b451700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c34067b30 con 0x7f9c340673c0 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9c341544d0 0x7f9c34155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c399ee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c340673c0 0x7f9c3415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c389ec700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9c34066f40 0x7f9c34095e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39274/0 (socket says 172.21.15.73:39274) 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c389ec700 1 -- 172.21.15.73:0/2728130829 learned_addr learned my addr 172.21.15.73:0/2728130829 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 -- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c340673c0 msgr2=0x7f9c3415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 --2- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c340673c0 0x7f9c3415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 -- 172.21.15.73:0/2728130829 >> v1:172.21.15.150:6789/0 conn(0x7f9c34066f40 legacy=0x7f9c34095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 -- 172.21.15.73:0/2728130829 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c34067e10 con 0x7f9c341544d0 2023-11-28T03:09:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c2bfff700 1 -- 172.21.15.73:0/2728130829 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1724171987 0 0) 0x7f9c340683d0 con 0x7f9c34066f40 2023-11-28T03:09:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c391ed700 1 --2- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c341544d0 0x7f9c34155750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f9c30009eb0 tx=0x7f9c30009780).ready entity=mon.1 client_cookie=e1a934e0d0fbe400 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c2bfff700 1 -- 172.21.15.73:0/2728130829 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c3000e070 con 0x7f9c341544d0 2023-11-28T03:09:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c2bfff700 1 -- 172.21.15.73:0/2728130829 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c3000bc30 con 0x7f9c341544d0 2023-11-28T03:09:55.266 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.263+0000 7f9c2bfff700 1 -- 172.21.15.73:0/2728130829 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c30012410 con 0x7f9c341544d0 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 -- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c341544d0 msgr2=0x7f9c34155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 --2- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c341544d0 0x7f9c34155750 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 rx=0x7f9c30009eb0 tx=0x7f9c30009780).stop 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 -- 172.21.15.73:0/2728130829 shutdown_connections 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 --2- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c340673c0 0x7f9c3415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 --2- 172.21.15.73:0/2728130829 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c341544d0 0x7f9c34155750 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 -- 172.21.15.73:0/2728130829 >> 172.21.15.73:0/2728130829 conn(0x7f9c340631d0 msgr2=0x7f9c340c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 -- 172.21.15.73:0/2728130829 shutdown_connections 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.264+0000 7f9c3b451700 1 -- 172.21.15.73:0/2728130829 wait complete. 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.265+0000 7f9c3b451700 1 Processor -- start 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.265+0000 7f9c3b451700 1 -- start start 2023-11-28T03:09:55.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34066f40 0x7f9c3414e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c341544d0 0x7f9c340ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340a96b0 con 0x7f9c340a6e30 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340a7760 con 0x7f9c34066f40 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c3b451700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c340a8aa0 con 0x7f9c341544d0 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c399ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c399ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44962/0 (socket says 172.21.15.73:44962) 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c399ee700 1 -- 172.21.15.73:0/1653416676 learned_addr learned my addr 172.21.15.73:0/1653416676 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c391ed700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34066f40 0x7f9c3414e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.266+0000 7f9c389ec700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c341544d0 0x7f9c340ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 -- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c341544d0 msgr2=0x7f9c340ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c341544d0 0x7f9c340ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 -- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34066f40 msgr2=0x7f9c3414e560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34066f40 0x7f9c3414e560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 -- 172.21.15.73:0/1653416676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c30009a20 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c399ee700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f9c2c00e8d0 tx=0x7f9c2c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c0094d0 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c2c012600 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c2c012a20 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c340a8d80 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.267+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c340a9340 con 0x7f9c340a6e30 2023-11-28T03:09:55.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.268+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c3406c860 con 0x7f9c340a6e30 2023-11-28T03:09:55.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.268+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9c2c01b400 con 0x7f9c340a6e30 2023-11-28T03:09:55.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.269+0000 7f9c297fa700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 0x7f9c18061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.269+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f9c2c07a550 con 0x7f9c340a6e30 2023-11-28T03:09:55.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.269+0000 7f9c391ed700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 0x7f9c18061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.269+0000 7f9c391ed700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 0x7f9c18061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9c340a85e0 tx=0x7f9c30009780).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.273+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9c2c04b820 con 0x7f9c340a6e30 2023-11-28T03:09:55.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.435+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9c340a97f0 con 0x7f9c1805f150 2023-11-28T03:09:55.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.444+0000 7f9c297fa700 1 -- 172.21.15.73:0/1653416676 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362073 (secure 0 0 0) 0x7f9c340a97f0 con 0x7f9c1805f150 2023-11-28T03:09:55.450 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:55.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 msgr2=0x7f9c18061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 0x7f9c18061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f9c340a85e0 tx=0x7f9c30009780).stop 2023-11-28T03:09:55.452 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 msgr2=0x7f9c340a7270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 rx=0x7f9c2c00e8d0 tx=0x7f9c2c009060).stop 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 shutdown_connections 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9c1805f150 0x7f9c18061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9c340a6e30 0x7f9c340a7270 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9c34066f40 0x7f9c3414e560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 --2- 172.21.15.73:0/1653416676 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9c341544d0 0x7f9c340ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 >> 172.21.15.73:0/1653416676 conn(0x7f9c340631d0 msgr2=0x7f9c340bf670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 shutdown_connections 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.448+0000 7f9c3b451700 1 -- 172.21.15.73:0/1653416676 wait complete. 2023-11-28T03:09:55.453 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:09:55.461 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T03:09:53.773852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2421760,"kb_used_data":12224,"kb_used_omap":0,"kb_used_meta":2409472,"kb_avail":752552960,"statfs":{"total":773094113280,"available":770614231040,"internally_reserved":0,"allocated":12517376,"data_stored":6217332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2467299328},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-4697,"num_objects":-18,"num_object_clones":0,"num_object_copies":-36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-18,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-43,"num_write_kb":-23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.561378"},"pg_stats":[{"pgid":"16.20","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772877+0000","last_change":"2023-11-28T03:09:47.767383+0000","last_active":"2023-11-28T03:09:47.772877+0000","last_peered":"2023-11-28T03:09:47.772877+0000","last_clean":"2023-11-28T03:09:47.772877+0000","last_became_active":"2023-11-28T03:09:47.767079+0000","last_became_peered":"2023-11-28T03:09:47.767079+0000","last_unstale":"2023-11-28T03:09:47.772877+0000","last_undegraded":"2023-11-28T03:09:47.772877+0000","last_fullsized":"2023-11-28T03:09:47.772877+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3f","version":"125'3","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768089+0000","last_change":"2023-11-28T03:09:47.394206+0000","last_active":"2023-11-28T03:09:47.768089+0000","last_peered":"2023-11-28T03:09:47.768089+0000","last_clean":"2023-11-28T03:09:47.768089+0000","last_became_active":"2023-11-28T03:09:47.393934+0000","last_became_peered":"2023-11-28T03:09:47.393934+0000","last_unstale":"2023-11-28T03:09:47.768089+0000","last_undegraded":"2023-11-28T03:09:47.768089+0000","last_fullsized":"2023-11-28T03:09:47.768089+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.21","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768202+0000","last_change":"2023-11-28T03:09:47.763844+0000","last_active":"2023-11-28T03:09:47.768202+0000","last_peered":"2023-11-28T03:09:47.768202+0000","last_clean":"2023-11-28T03:09:47.768202+0000","last_became_active":"2023-11-28T03:09:47.763586+0000","last_became_peered":"2023-11-28T03:09:47.763586+0000","last_unstale":"2023-11-28T03:09:47.768202+0000","last_undegraded":"2023-11-28T03:09:47.768202+0000","last_fullsized":"2023-11-28T03:09:47.768202+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750907+0000","last_change":"2023-11-28T03:09:47.392119+0000","last_active":"2023-11-28T03:09:47.750907+0000","last_peered":"2023-11-28T03:09:47.750907+0000","last_clean":"2023-11-28T03:09:47.750907+0000","last_became_active":"2023-11-28T03:09:47.391850+0000","last_became_peered":"2023-11-28T03:09:47.391850+0000","last_unstale":"2023-11-28T03:09:47.750907+0000","last_undegraded":"2023-11-28T03:09:47.750907+0000","last_fullsized":"2023-11-28T03:09:47.750907+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.22","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770230+0000","last_change":"2023-11-28T03:09:47.765843+0000","last_active":"2023-11-28T03:09:47.770230+0000","last_peered":"2023-11-28T03:09:47.770230+0000","last_clean":"2023-11-28T03:09:47.770230+0000","last_became_active":"2023-11-28T03:09:47.765425+0000","last_became_peered":"2023-11-28T03:09:47.765425+0000","last_unstale":"2023-11-28T03:09:47.770230+0000","last_undegraded":"2023-11-28T03:09:47.770230+0000","last_fullsized":"2023-11-28T03:09:47.770230+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748005+0000","last_change":"2023-11-28T03:09:47.385738+0000","last_active":"2023-11-28T03:09:47.748005+0000","last_peered":"2023-11-28T03:09:47.748005+0000","last_clean":"2023-11-28T03:09:47.748005+0000","last_became_active":"2023-11-28T03:09:47.385479+0000","last_became_peered":"2023-11-28T03:09:47.385479+0000","last_unstale":"2023-11-28T03:09:47.748005+0000","last_undegraded":"2023-11-28T03:09:47.748005+0000","last_fullsized":"2023-11-28T03:09:47.748005+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.23","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767699+0000","last_change":"2023-11-28T03:09:47.766548+0000","last_active":"2023-11-28T03:09:47.767699+0000","last_peered":"2023-11-28T03:09:47.767699+0000","last_clean":"2023-11-28T03:09:47.767699+0000","last_became_active":"2023-11-28T03:09:47.766247+0000","last_became_peered":"2023-11-28T03:09:47.766247+0000","last_unstale":"2023-11-28T03:09:47.767699+0000","last_undegraded":"2023-11-28T03:09:47.767699+0000","last_fullsized":"2023-11-28T03:09:47.767699+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3c","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.761978+0000","last_change":"2023-11-28T03:09:47.390788+0000","last_active":"2023-11-28T03:09:47.761978+0000","last_peered":"2023-11-28T03:09:47.761978+0000","last_clean":"2023-11-28T03:09:47.761978+0000","last_became_active":"2023-11-28T03:09:47.390517+0000","last_became_peered":"2023-11-28T03:09:47.390517+0000","last_unstale":"2023-11-28T03:09:47.761978+0000","last_undegraded":"2023-11-28T03:09:47.761978+0000","last_fullsized":"2023-11-28T03:09:47.761978+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.24","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772433+0000","last_change":"2023-11-28T03:09:47.766560+0000","last_active":"2023-11-28T03:09:47.772433+0000","last_peered":"2023-11-28T03:09:47.772433+0000","last_clean":"2023-11-28T03:09:47.772433+0000","last_became_active":"2023-11-28T03:09:47.766303+0000","last_became_peered":"2023-11-28T03:09:47.766303+0000","last_unstale":"2023-11-28T03:09:47.772433+0000","last_undegraded":"2023-11-28T03:09:47.772433+0000","last_fullsized":"2023-11-28T03:09:47.772433+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764007+0000","last_change":"2023-11-28T03:09:47.383307+0000","last_active":"2023-11-28T03:09:47.764007+0000","last_peered":"2023-11-28T03:09:47.764007+0000","last_clean":"2023-11-28T03:09:47.764007+0000","last_became_active":"2023-11-28T03:09:47.382990+0000","last_became_peered":"2023-11-28T03:09:47.382990+0000","last_unstale":"2023-11-28T03:09:47.764007+0000","last_undegraded":"2023-11-28T03:09:47.764007+0000","last_fullsized":"2023-11-28T03:09:47.764007+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.25","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775676+0000","last_change":"2023-11-28T03:09:47.773275+0000","last_active":"2023-11-28T03:09:47.775676+0000","last_peered":"2023-11-28T03:09:47.775676+0000","last_clean":"2023-11-28T03:09:47.775676+0000","last_became_active":"2023-11-28T03:09:47.773013+0000","last_became_peered":"2023-11-28T03:09:47.773013+0000","last_unstale":"2023-11-28T03:09:47.775676+0000","last_undegraded":"2023-11-28T03:09:47.775676+0000","last_fullsized":"2023-11-28T03:09:47.775676+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.3a","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772251+0000","last_change":"2023-11-28T03:09:47.385556+0000","last_active":"2023-11-28T03:09:47.772251+0000","last_peered":"2023-11-28T03:09:47.772251+0000","last_clean":"2023-11-28T03:09:47.772251+0000","last_became_active":"2023-11-28T03:09:47.385208+0000","last_became_peered":"2023-11-28T03:09:47.385208+0000","last_unstale":"2023-11-28T03:09:47.772251+0000","last_undegraded":"2023-11-28T03:09:47.772251+0000","last_fullsized":"2023-11-28T03:09:47.772251+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.26","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770118+0000","last_change":"2023-11-28T03:09:47.765880+0000","last_active":"2023-11-28T03:09:47.770118+0000","last_peered":"2023-11-28T03:09:47.770118+0000","last_clean":"2023-11-28T03:09:47.770118+0000","last_became_active":"2023-11-28T03:09:47.765532+0000","last_became_peered":"2023-11-28T03:09:47.765532+0000","last_unstale":"2023-11-28T03:09:47.770118+0000","last_undegraded":"2023-11-28T03:09:47.770118+0000","last_fullsized":"2023-11-28T03:09:47.770118+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752152+0000","last_change":"2023-11-28T03:09:47.385411+0000","last_active":"2023-11-28T03:09:47.752152+0000","last_peered":"2023-11-28T03:09:47.752152+0000","last_clean":"2023-11-28T03:09:47.752152+0000","last_became_active":"2023-11-28T03:09:47.385116+0000","last_became_peered":"2023-11-28T03:09:47.385116+0000","last_unstale":"2023-11-28T03:09:47.752152+0000","last_undegraded":"2023-11-28T03:09:47.752152+0000","last_fullsized":"2023-11-28T03:09:47.752152+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.27","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770180+0000","last_change":"2023-11-28T03:09:47.765998+0000","last_active":"2023-11-28T03:09:47.770180+0000","last_peered":"2023-11-28T03:09:47.770180+0000","last_clean":"2023-11-28T03:09:47.770180+0000","last_became_active":"2023-11-28T03:09:47.765689+0000","last_became_peered":"2023-11-28T03:09:47.765689+0000","last_unstale":"2023-11-28T03:09:47.770180+0000","last_undegraded":"2023-11-28T03:09:47.770180+0000","last_fullsized":"2023-11-28T03:09:47.770180+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.749361+0000","last_change":"2023-11-28T03:09:47.389535+0000","last_active":"2023-11-28T03:09:47.749361+0000","last_peered":"2023-11-28T03:09:47.749361+0000","last_clean":"2023-11-28T03:09:47.749361+0000","last_became_active":"2023-11-28T03:09:47.389266+0000","last_became_peered":"2023-11-28T03:09:47.389266+0000","last_unstale":"2023-11-28T03:09:47.749361+0000","last_undegraded":"2023-11-28T03:09:47.749361+0000","last_fullsized":"2023-11-28T03:09:47.749361+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.28","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770613+0000","last_change":"2023-11-28T03:09:47.755826+0000","last_active":"2023-11-28T03:09:47.770613+0000","last_peered":"2023-11-28T03:09:47.770613+0000","last_clean":"2023-11-28T03:09:47.770613+0000","last_became_active":"2023-11-28T03:09:47.755588+0000","last_became_peered":"2023-11-28T03:09:47.755588+0000","last_unstale":"2023-11-28T03:09:47.770613+0000","last_undegraded":"2023-11-28T03:09:47.770613+0000","last_fullsized":"2023-11-28T03:09:47.770613+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748676+0000","last_change":"2023-11-28T03:09:47.383050+0000","last_active":"2023-11-28T03:09:47.748676+0000","last_peered":"2023-11-28T03:09:47.748676+0000","last_clean":"2023-11-28T03:09:47.748676+0000","last_became_active":"2023-11-28T03:09:47.382767+0000","last_became_peered":"2023-11-28T03:09:47.382767+0000","last_unstale":"2023-11-28T03:09:47.748676+0000","last_undegraded":"2023-11-28T03:09:47.748676+0000","last_fullsized":"2023-11-28T03:09:47.748676+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.29","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772936+0000","last_change":"2023-11-28T03:09:47.766559+0000","last_active":"2023-11-28T03:09:47.772936+0000","last_peered":"2023-11-28T03:09:47.772936+0000","last_clean":"2023-11-28T03:09:47.772936+0000","last_became_active":"2023-11-28T03:09:47.766260+0000","last_became_peered":"2023-11-28T03:09:47.766260+0000","last_unstale":"2023-11-28T03:09:47.772936+0000","last_undegraded":"2023-11-28T03:09:47.772936+0000","last_fullsized":"2023-11-28T03:09:47.772936+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755692+0000","last_change":"2023-11-28T03:09:47.387044+0000","last_active":"2023-11-28T03:09:47.755692+0000","last_peered":"2023-11-28T03:09:47.755692+0000","last_clean":"2023-11-28T03:09:47.755692+0000","last_became_active":"2023-11-28T03:09:47.386766+0000","last_became_peered":"2023-11-28T03:09:47.386766+0000","last_unstale":"2023-11-28T03:09:47.755692+0000","last_undegraded":"2023-11-28T03:09:47.755692+0000","last_fullsized":"2023-11-28T03:09:47.755692+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.2a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774677+0000","last_change":"2023-11-28T03:09:47.768197+0000","last_active":"2023-11-28T03:09:47.774677+0000","last_peered":"2023-11-28T03:09:47.774677+0000","last_clean":"2023-11-28T03:09:47.774677+0000","last_became_active":"2023-11-28T03:09:47.767959+0000","last_became_peered":"2023-11-28T03:09:47.767959+0000","last_unstale":"2023-11-28T03:09:47.774677+0000","last_undegraded":"2023-11-28T03:09:47.774677+0000","last_fullsized":"2023-11-28T03:09:47.774677+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752716+0000","last_change":"2023-11-28T03:09:47.386088+0000","last_active":"2023-11-28T03:09:47.752716+0000","last_peered":"2023-11-28T03:09:47.752716+0000","last_clean":"2023-11-28T03:09:47.752716+0000","last_became_active":"2023-11-28T03:09:47.385812+0000","last_became_peered":"2023-11-28T03:09:47.385812+0000","last_unstale":"2023-11-28T03:09:47.752716+0000","last_undegraded":"2023-11-28T03:09:47.752716+0000","last_fullsized":"2023-11-28T03:09:47.752716+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.2b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775413+0000","last_change":"2023-11-28T03:09:47.770395+0000","last_active":"2023-11-28T03:09:47.775413+0000","last_peered":"2023-11-28T03:09:47.775413+0000","last_clean":"2023-11-28T03:09:47.775413+0000","last_became_active":"2023-11-28T03:09:47.770125+0000","last_became_peered":"2023-11-28T03:09:47.770125+0000","last_unstale":"2023-11-28T03:09:47.775413+0000","last_undegraded":"2023-11-28T03:09:47.775413+0000","last_fullsized":"2023-11-28T03:09:47.775413+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.34","version":"125'4","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776088+0000","last_change":"2023-11-28T03:09:47.386192+0000","last_active":"2023-11-28T03:09:47.776088+0000","last_peered":"2023-11-28T03:09:47.776088+0000","last_clean":"2023-11-28T03:09:47.776088+0000","last_became_active":"2023-11-28T03:09:47.385904+0000","last_became_peered":"2023-11-28T03:09:47.385904+0000","last_unstale":"2023-11-28T03:09:47.776088+0000","last_undegraded":"2023-11-28T03:09:47.776088+0000","last_fullsized":"2023-11-28T03:09:47.776088+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.769628+0000","last_change":"2023-11-28T03:09:47.761792+0000","last_active":"2023-11-28T03:09:47.769628+0000","last_peered":"2023-11-28T03:09:47.769628+0000","last_clean":"2023-11-28T03:09:47.769628+0000","last_became_active":"2023-11-28T03:09:47.761504+0000","last_became_peered":"2023-11-28T03:09:47.761504+0000","last_unstale":"2023-11-28T03:09:47.769628+0000","last_undegraded":"2023-11-28T03:09:47.769628+0000","last_fullsized":"2023-11-28T03:09:47.769628+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.760910+0000","last_change":"2023-11-28T03:09:47.383115+0000","last_active":"2023-11-28T03:09:47.760910+0000","last_peered":"2023-11-28T03:09:47.760910+0000","last_clean":"2023-11-28T03:09:47.760910+0000","last_became_active":"2023-11-28T03:09:47.382846+0000","last_became_peered":"2023-11-28T03:09:47.382846+0000","last_unstale":"2023-11-28T03:09:47.760910+0000","last_undegraded":"2023-11-28T03:09:47.760910+0000","last_fullsized":"2023-11-28T03:09:47.760910+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768007+0000","last_change":"2023-11-28T03:09:47.763500+0000","last_active":"2023-11-28T03:09:47.768007+0000","last_peered":"2023-11-28T03:09:47.768007+0000","last_clean":"2023-11-28T03:09:47.768007+0000","last_became_active":"2023-11-28T03:09:47.763151+0000","last_became_peered":"2023-11-28T03:09:47.763151+0000","last_unstale":"2023-11-28T03:09:47.768007+0000","last_undegraded":"2023-11-28T03:09:47.768007+0000","last_fullsized":"2023-11-28T03:09:47.768007+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755467+0000","last_change":"2023-11-28T03:09:47.392189+0000","last_active":"2023-11-28T03:09:47.755467+0000","last_peered":"2023-11-28T03:09:47.755467+0000","last_clean":"2023-11-28T03:09:47.755467+0000","last_became_active":"2023-11-28T03:09:47.389101+0000","last_became_peered":"2023-11-28T03:09:47.389101+0000","last_unstale":"2023-11-28T03:09:47.755467+0000","last_undegraded":"2023-11-28T03:09:47.755467+0000","last_fullsized":"2023-11-28T03:09:47.755467+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768702+0000","last_change":"2023-11-28T03:09:47.765962+0000","last_active":"2023-11-28T03:09:47.768702+0000","last_peered":"2023-11-28T03:09:47.768702+0000","last_clean":"2023-11-28T03:09:47.768702+0000","last_became_active":"2023-11-28T03:09:47.765570+0000","last_became_peered":"2023-11-28T03:09:47.765570+0000","last_unstale":"2023-11-28T03:09:47.768702+0000","last_undegraded":"2023-11-28T03:09:47.768702+0000","last_fullsized":"2023-11-28T03:09:47.768702+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752707+0000","last_change":"2023-11-28T03:09:47.384811+0000","last_active":"2023-11-28T03:09:47.752707+0000","last_peered":"2023-11-28T03:09:47.752707+0000","last_clean":"2023-11-28T03:09:47.752707+0000","last_became_active":"2023-11-28T03:09:47.384543+0000","last_became_peered":"2023-11-28T03:09:47.384543+0000","last_unstale":"2023-11-28T03:09:47.752707+0000","last_undegraded":"2023-11-28T03:09:47.752707+0000","last_fullsized":"2023-11-28T03:09:47.752707+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.2f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774397+0000","last_change":"2023-11-28T03:09:47.767269+0000","last_active":"2023-11-28T03:09:47.774397+0000","last_peered":"2023-11-28T03:09:47.774397+0000","last_clean":"2023-11-28T03:09:47.774397+0000","last_became_active":"2023-11-28T03:09:47.766860+0000","last_became_peered":"2023-11-28T03:09:47.766860+0000","last_unstale":"2023-11-28T03:09:47.774397+0000","last_undegraded":"2023-11-28T03:09:47.774397+0000","last_fullsized":"2023-11-28T03:09:47.774397+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.30","version":"125'4","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772536+0000","last_change":"2023-11-28T03:09:47.395446+0000","last_active":"2023-11-28T03:09:47.772536+0000","last_peered":"2023-11-28T03:09:47.772536+0000","last_clean":"2023-11-28T03:09:47.772536+0000","last_became_active":"2023-11-28T03:09:47.395212+0000","last_became_peered":"2023-11-28T03:09:47.395212+0000","last_unstale":"2023-11-28T03:09:47.772536+0000","last_undegraded":"2023-11-28T03:09:47.772536+0000","last_fullsized":"2023-11-28T03:09:47.772536+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.30","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772596+0000","last_change":"2023-11-28T03:09:47.761229+0000","last_active":"2023-11-28T03:09:47.772596+0000","last_peered":"2023-11-28T03:09:47.772596+0000","last_clean":"2023-11-28T03:09:47.772596+0000","last_became_active":"2023-11-28T03:09:47.760944+0000","last_became_peered":"2023-11-28T03:09:47.760944+0000","last_unstale":"2023-11-28T03:09:47.772596+0000","last_undegraded":"2023-11-28T03:09:47.772596+0000","last_fullsized":"2023-11-28T03:09:47.772596+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750997+0000","last_change":"2023-11-28T03:09:47.391999+0000","last_active":"2023-11-28T03:09:47.750997+0000","last_peered":"2023-11-28T03:09:47.750997+0000","last_clean":"2023-11-28T03:09:47.750997+0000","last_became_active":"2023-11-28T03:09:47.389903+0000","last_became_peered":"2023-11-28T03:09:47.389903+0000","last_unstale":"2023-11-28T03:09:47.750997+0000","last_undegraded":"2023-11-28T03:09:47.750997+0000","last_fullsized":"2023-11-28T03:09:47.750997+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.31","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775095+0000","last_change":"2023-11-28T03:09:47.766531+0000","last_active":"2023-11-28T03:09:47.775095+0000","last_peered":"2023-11-28T03:09:47.775095+0000","last_clean":"2023-11-28T03:09:47.775095+0000","last_became_active":"2023-11-28T03:09:47.766212+0000","last_became_peered":"2023-11-28T03:09:47.766212+0000","last_unstale":"2023-11-28T03:09:47.775095+0000","last_undegraded":"2023-11-28T03:09:47.775095+0000","last_fullsized":"2023-11-28T03:09:47.775095+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753010+0000","last_change":"2023-11-28T03:09:47.393114+0000","last_active":"2023-11-28T03:09:47.753010+0000","last_peered":"2023-11-28T03:09:47.753010+0000","last_clean":"2023-11-28T03:09:47.753010+0000","last_became_active":"2023-11-28T03:09:47.392832+0000","last_became_peered":"2023-11-28T03:09:47.392832+0000","last_unstale":"2023-11-28T03:09:47.753010+0000","last_undegraded":"2023-11-28T03:09:47.753010+0000","last_fullsized":"2023-11-28T03:09:47.753010+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.32","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772568+0000","last_change":"2023-11-28T03:09:47.762910+0000","last_active":"2023-11-28T03:09:47.772568+0000","last_peered":"2023-11-28T03:09:47.772568+0000","last_clean":"2023-11-28T03:09:47.772568+0000","last_became_active":"2023-11-28T03:09:47.762627+0000","last_became_peered":"2023-11-28T03:09:47.762627+0000","last_unstale":"2023-11-28T03:09:47.772568+0000","last_undegraded":"2023-11-28T03:09:47.772568+0000","last_fullsized":"2023-11-28T03:09:47.772568+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2d","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.774624+0000","last_change":"2023-11-28T03:09:47.387477+0000","last_active":"2023-11-28T03:09:47.774624+0000","last_peered":"2023-11-28T03:09:47.774624+0000","last_clean":"2023-11-28T03:09:47.774624+0000","last_became_active":"2023-11-28T03:09:47.387191+0000","last_became_peered":"2023-11-28T03:09:47.387191+0000","last_unstale":"2023-11-28T03:09:47.774624+0000","last_undegraded":"2023-11-28T03:09:47.774624+0000","last_fullsized":"2023-11-28T03:09:47.774624+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.33","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775556+0000","last_change":"2023-11-28T03:09:47.770678+0000","last_active":"2023-11-28T03:09:47.775556+0000","last_peered":"2023-11-28T03:09:47.775556+0000","last_clean":"2023-11-28T03:09:47.775556+0000","last_became_active":"2023-11-28T03:09:47.770403+0000","last_became_peered":"2023-11-28T03:09:47.770403+0000","last_unstale":"2023-11-28T03:09:47.775556+0000","last_undegraded":"2023-11-28T03:09:47.775556+0000","last_fullsized":"2023-11-28T03:09:47.775556+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748783+0000","last_change":"2023-11-28T03:09:47.384305+0000","last_active":"2023-11-28T03:09:47.748783+0000","last_peered":"2023-11-28T03:09:47.748783+0000","last_clean":"2023-11-28T03:09:47.748783+0000","last_became_active":"2023-11-28T03:09:47.383972+0000","last_became_peered":"2023-11-28T03:09:47.383972+0000","last_unstale":"2023-11-28T03:09:47.748783+0000","last_undegraded":"2023-11-28T03:09:47.748783+0000","last_fullsized":"2023-11-28T03:09:47.748783+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.34","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.769643+0000","last_change":"2023-11-28T03:09:47.763138+0000","last_active":"2023-11-28T03:09:47.769643+0000","last_peered":"2023-11-28T03:09:47.769643+0000","last_clean":"2023-11-28T03:09:47.769643+0000","last_became_active":"2023-11-28T03:09:47.762654+0000","last_became_peered":"2023-11-28T03:09:47.762654+0000","last_unstale":"2023-11-28T03:09:47.769643+0000","last_undegraded":"2023-11-28T03:09:47.769643+0000","last_fullsized":"2023-11-28T03:09:47.769643+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770142+0000","last_change":"2023-11-28T03:09:47.385173+0000","last_active":"2023-11-28T03:09:47.770142+0000","last_peered":"2023-11-28T03:09:47.770142+0000","last_clean":"2023-11-28T03:09:47.770142+0000","last_became_active":"2023-11-28T03:09:47.384367+0000","last_became_peered":"2023-11-28T03:09:47.384367+0000","last_unstale":"2023-11-28T03:09:47.770142+0000","last_undegraded":"2023-11-28T03:09:47.770142+0000","last_fullsized":"2023-11-28T03:09:47.770142+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.35","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773342+0000","last_change":"2023-11-28T03:09:47.769537+0000","last_active":"2023-11-28T03:09:47.773342+0000","last_peered":"2023-11-28T03:09:47.773342+0000","last_clean":"2023-11-28T03:09:47.773342+0000","last_became_active":"2023-11-28T03:09:47.769254+0000","last_became_peered":"2023-11-28T03:09:47.769254+0000","last_unstale":"2023-11-28T03:09:47.773342+0000","last_undegraded":"2023-11-28T03:09:47.773342+0000","last_fullsized":"2023-11-28T03:09:47.773342+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748627+0000","last_change":"2023-11-28T03:09:47.393622+0000","last_active":"2023-11-28T03:09:47.748627+0000","last_peered":"2023-11-28T03:09:47.748627+0000","last_clean":"2023-11-28T03:09:47.748627+0000","last_became_active":"2023-11-28T03:09:47.390278+0000","last_became_peered":"2023-11-28T03:09:47.390278+0000","last_unstale":"2023-11-28T03:09:47.748627+0000","last_undegraded":"2023-11-28T03:09:47.748627+0000","last_fullsized":"2023-11-28T03:09:47.748627+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.36","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773662+0000","last_change":"2023-11-28T03:09:47.765563+0000","last_active":"2023-11-28T03:09:47.773662+0000","last_peered":"2023-11-28T03:09:47.773662+0000","last_clean":"2023-11-28T03:09:47.773662+0000","last_became_active":"2023-11-28T03:09:47.765387+0000","last_became_peered":"2023-11-28T03:09:47.765387+0000","last_unstale":"2023-11-28T03:09:47.773662+0000","last_undegraded":"2023-11-28T03:09:47.773662+0000","last_fullsized":"2023-11-28T03:09:47.773662+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748538+0000","last_change":"2023-11-28T03:09:47.388908+0000","last_active":"2023-11-28T03:09:47.748538+0000","last_peered":"2023-11-28T03:09:47.748538+0000","last_clean":"2023-11-28T03:09:47.748538+0000","last_became_active":"2023-11-28T03:09:47.388654+0000","last_became_peered":"2023-11-28T03:09:47.388654+0000","last_unstale":"2023-11-28T03:09:47.748538+0000","last_undegraded":"2023-11-28T03:09:47.748538+0000","last_fullsized":"2023-11-28T03:09:47.748538+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.37","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772302+0000","last_change":"2023-11-28T03:09:47.767149+0000","last_active":"2023-11-28T03:09:47.772302+0000","last_peered":"2023-11-28T03:09:47.772302+0000","last_clean":"2023-11-28T03:09:47.772302+0000","last_became_active":"2023-11-28T03:09:47.766835+0000","last_became_peered":"2023-11-28T03:09:47.766835+0000","last_unstale":"2023-11-28T03:09:47.772302+0000","last_undegraded":"2023-11-28T03:09:47.772302+0000","last_fullsized":"2023-11-28T03:09:47.772302+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.751817+0000","last_change":"2023-11-28T03:09:47.393032+0000","last_active":"2023-11-28T03:09:47.751817+0000","last_peered":"2023-11-28T03:09:47.751817+0000","last_clean":"2023-11-28T03:09:47.751817+0000","last_became_active":"2023-11-28T03:09:47.392759+0000","last_became_peered":"2023-11-28T03:09:47.392759+0000","last_unstale":"2023-11-28T03:09:47.751817+0000","last_undegraded":"2023-11-28T03:09:47.751817+0000","last_fullsized":"2023-11-28T03:09:47.751817+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.38","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776213+0000","last_change":"2023-11-28T03:09:47.774084+0000","last_active":"2023-11-28T03:09:47.776213+0000","last_peered":"2023-11-28T03:09:47.776213+0000","last_clean":"2023-11-28T03:09:47.776213+0000","last_became_active":"2023-11-28T03:09:47.773856+0000","last_became_peered":"2023-11-28T03:09:47.773856+0000","last_unstale":"2023-11-28T03:09:47.776213+0000","last_undegraded":"2023-11-28T03:09:47.776213+0000","last_fullsized":"2023-11-28T03:09:47.776213+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755914+0000","last_change":"2023-11-28T03:09:47.387880+0000","last_active":"2023-11-28T03:09:47.755914+0000","last_peered":"2023-11-28T03:09:47.755914+0000","last_clean":"2023-11-28T03:09:47.755914+0000","last_became_active":"2023-11-28T03:09:47.387590+0000","last_became_peered":"2023-11-28T03:09:47.387590+0000","last_unstale":"2023-11-28T03:09:47.755914+0000","last_undegraded":"2023-11-28T03:09:47.755914+0000","last_fullsized":"2023-11-28T03:09:47.755914+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.39","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768226+0000","last_change":"2023-11-28T03:09:47.765943+0000","last_active":"2023-11-28T03:09:47.768226+0000","last_peered":"2023-11-28T03:09:47.768226+0000","last_clean":"2023-11-28T03:09:47.768226+0000","last_became_active":"2023-11-28T03:09:47.765585+0000","last_became_peered":"2023-11-28T03:09:47.765585+0000","last_unstale":"2023-11-28T03:09:47.768226+0000","last_undegraded":"2023-11-28T03:09:47.768226+0000","last_fullsized":"2023-11-28T03:09:47.768226+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750729+0000","last_change":"2023-11-28T03:09:47.393496+0000","last_active":"2023-11-28T03:09:47.750729+0000","last_peered":"2023-11-28T03:09:47.750729+0000","last_clean":"2023-11-28T03:09:47.750729+0000","last_became_active":"2023-11-28T03:09:47.393219+0000","last_became_peered":"2023-11-28T03:09:47.393219+0000","last_unstale":"2023-11-28T03:09:47.750729+0000","last_undegraded":"2023-11-28T03:09:47.750729+0000","last_fullsized":"2023-11-28T03:09:47.750729+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775173+0000","last_change":"2023-11-28T03:09:47.768144+0000","last_active":"2023-11-28T03:09:47.775173+0000","last_peered":"2023-11-28T03:09:47.775173+0000","last_clean":"2023-11-28T03:09:47.775173+0000","last_became_active":"2023-11-28T03:09:47.767603+0000","last_became_peered":"2023-11-28T03:09:47.767603+0000","last_unstale":"2023-11-28T03:09:47.775173+0000","last_undegraded":"2023-11-28T03:09:47.775173+0000","last_fullsized":"2023-11-28T03:09:47.775173+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752344+0000","last_change":"2023-11-28T03:09:47.389257+0000","last_active":"2023-11-28T03:09:47.752344+0000","last_peered":"2023-11-28T03:09:47.752344+0000","last_clean":"2023-11-28T03:09:47.752344+0000","last_became_active":"2023-11-28T03:09:47.388904+0000","last_became_peered":"2023-11-28T03:09:47.388904+0000","last_unstale":"2023-11-28T03:09:47.752344+0000","last_undegraded":"2023-11-28T03:09:47.752344+0000","last_fullsized":"2023-11-28T03:09:47.752344+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.3b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773460+0000","last_change":"2023-11-28T03:09:47.765311+0000","last_active":"2023-11-28T03:09:47.773460+0000","last_peered":"2023-11-28T03:09:47.773460+0000","last_clean":"2023-11-28T03:09:47.773460+0000","last_became_active":"2023-11-28T03:09:47.765037+0000","last_became_peered":"2023-11-28T03:09:47.765037+0000","last_unstale":"2023-11-28T03:09:47.773460+0000","last_undegraded":"2023-11-28T03:09:47.773460+0000","last_fullsized":"2023-11-28T03:09:47.773460+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.754226+0000","last_change":"2023-11-28T03:09:47.385356+0000","last_active":"2023-11-28T03:09:47.754226+0000","last_peered":"2023-11-28T03:09:47.754226+0000","last_clean":"2023-11-28T03:09:47.754226+0000","last_became_active":"2023-11-28T03:09:47.385087+0000","last_became_peered":"2023-11-28T03:09:47.385087+0000","last_unstale":"2023-11-28T03:09:47.754226+0000","last_undegraded":"2023-11-28T03:09:47.754226+0000","last_fullsized":"2023-11-28T03:09:47.754226+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.3c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768762+0000","last_change":"2023-11-28T03:09:47.762089+0000","last_active":"2023-11-28T03:09:47.768762+0000","last_peered":"2023-11-28T03:09:47.768762+0000","last_clean":"2023-11-28T03:09:47.768762+0000","last_became_active":"2023-11-28T03:09:47.761503+0000","last_became_peered":"2023-11-28T03:09:47.761503+0000","last_unstale":"2023-11-28T03:09:47.768762+0000","last_undegraded":"2023-11-28T03:09:47.768762+0000","last_fullsized":"2023-11-28T03:09:47.768762+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748921+0000","last_change":"2023-11-28T03:09:47.393246+0000","last_active":"2023-11-28T03:09:47.748921+0000","last_peered":"2023-11-28T03:09:47.748921+0000","last_clean":"2023-11-28T03:09:47.748921+0000","last_became_active":"2023-11-28T03:09:47.392973+0000","last_became_peered":"2023-11-28T03:09:47.392973+0000","last_unstale":"2023-11-28T03:09:47.748921+0000","last_undegraded":"2023-11-28T03:09:47.748921+0000","last_fullsized":"2023-11-28T03:09:47.748921+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.3d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775496+0000","last_change":"2023-11-28T03:09:47.768410+0000","last_active":"2023-11-28T03:09:47.775496+0000","last_peered":"2023-11-28T03:09:47.775496+0000","last_clean":"2023-11-28T03:09:47.775496+0000","last_became_active":"2023-11-28T03:09:47.767775+0000","last_became_peered":"2023-11-28T03:09:47.767775+0000","last_unstale":"2023-11-28T03:09:47.775496+0000","last_undegraded":"2023-11-28T03:09:47.775496+0000","last_fullsized":"2023-11-28T03:09:47.775496+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755760+0000","last_change":"2023-11-28T03:09:47.385915+0000","last_active":"2023-11-28T03:09:47.755760+0000","last_peered":"2023-11-28T03:09:47.755760+0000","last_clean":"2023-11-28T03:09:47.755760+0000","last_became_active":"2023-11-28T03:09:47.385628+0000","last_became_peered":"2023-11-28T03:09:47.385628+0000","last_unstale":"2023-11-28T03:09:47.755760+0000","last_undegraded":"2023-11-28T03:09:47.755760+0000","last_fullsized":"2023-11-28T03:09:47.755760+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.3e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775660+0000","last_change":"2023-11-28T03:09:47.773362+0000","last_active":"2023-11-28T03:09:47.775660+0000","last_peered":"2023-11-28T03:09:47.775660+0000","last_clean":"2023-11-28T03:09:47.775660+0000","last_became_active":"2023-11-28T03:09:47.773071+0000","last_became_peered":"2023-11-28T03:09:47.773071+0000","last_unstale":"2023-11-28T03:09:47.775660+0000","last_undegraded":"2023-11-28T03:09:47.775660+0000","last_fullsized":"2023-11-28T03:09:47.775660+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752888+0000","last_change":"2023-11-28T03:09:47.386808+0000","last_active":"2023-11-28T03:09:47.752888+0000","last_peered":"2023-11-28T03:09:47.752888+0000","last_clean":"2023-11-28T03:09:47.752888+0000","last_became_active":"2023-11-28T03:09:47.386529+0000","last_became_peered":"2023-11-28T03:09:47.386529+0000","last_unstale":"2023-11-28T03:09:47.752888+0000","last_undegraded":"2023-11-28T03:09:47.752888+0000","last_fullsized":"2023-11-28T03:09:47.752888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.3f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.765393+0000","last_change":"2023-11-28T03:09:47.760194+0000","last_active":"2023-11-28T03:09:47.765393+0000","last_peered":"2023-11-28T03:09:47.765393+0000","last_clean":"2023-11-28T03:09:47.765393+0000","last_became_active":"2023-11-28T03:09:47.759941+0000","last_became_peered":"2023-11-28T03:09:47.759941+0000","last_unstale":"2023-11-28T03:09:47.765393+0000","last_undegraded":"2023-11-28T03:09:47.765393+0000","last_fullsized":"2023-11-28T03:09:47.765393+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747508+0000","last_change":"2023-11-28T03:09:47.392973+0000","last_active":"2023-11-28T03:09:47.747508+0000","last_peered":"2023-11-28T03:09:47.747508+0000","last_clean":"2023-11-28T03:09:47.747508+0000","last_became_active":"2023-11-28T03:09:47.392709+0000","last_became_peered":"2023-11-28T03:09:47.392709+0000","last_unstale":"2023-11-28T03:09:47.747508+0000","last_undegraded":"2023-11-28T03:09:47.747508+0000","last_fullsized":"2023-11-28T03:09:47.747508+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766656+0000","last_change":"2023-11-28T03:09:47.761218+0000","last_active":"2023-11-28T03:09:47.766656+0000","last_peered":"2023-11-28T03:09:47.766656+0000","last_clean":"2023-11-28T03:09:47.766656+0000","last_became_active":"2023-11-28T03:09:47.760887+0000","last_became_peered":"2023-11-28T03:09:47.760887+0000","last_unstale":"2023-11-28T03:09:47.766656+0000","last_undegraded":"2023-11-28T03:09:47.766656+0000","last_fullsized":"2023-11-28T03:09:47.766656+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767480+0000","last_change":"2023-11-28T03:09:47.388373+0000","last_active":"2023-11-28T03:09:47.767480+0000","last_peered":"2023-11-28T03:09:47.767480+0000","last_clean":"2023-11-28T03:09:47.767480+0000","last_became_active":"2023-11-28T03:09:47.388085+0000","last_became_peered":"2023-11-28T03:09:47.388085+0000","last_unstale":"2023-11-28T03:09:47.767480+0000","last_undegraded":"2023-11-28T03:09:47.767480+0000","last_fullsized":"2023-11-28T03:09:47.767480+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775454+0000","last_change":"2023-11-28T03:09:47.771133+0000","last_active":"2023-11-28T03:09:47.775454+0000","last_peered":"2023-11-28T03:09:47.775454+0000","last_clean":"2023-11-28T03:09:47.775454+0000","last_became_active":"2023-11-28T03:09:47.770868+0000","last_became_peered":"2023-11-28T03:09:47.770868+0000","last_unstale":"2023-11-28T03:09:47.775454+0000","last_undegraded":"2023-11-28T03:09:47.775454+0000","last_fullsized":"2023-11-28T03:09:47.775454+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.1e","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767540+0000","last_change":"2023-11-28T03:09:47.386456+0000","last_active":"2023-11-28T03:09:47.767540+0000","last_peered":"2023-11-28T03:09:47.767540+0000","last_clean":"2023-11-28T03:09:47.767540+0000","last_became_active":"2023-11-28T03:09:47.385555+0000","last_became_peered":"2023-11-28T03:09:47.385555+0000","last_unstale":"2023-11-28T03:09:47.767540+0000","last_undegraded":"2023-11-28T03:09:47.767540+0000","last_fullsized":"2023-11-28T03:09:47.767540+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770194+0000","last_change":"2023-11-28T03:09:47.765979+0000","last_active":"2023-11-28T03:09:47.770194+0000","last_peered":"2023-11-28T03:09:47.770194+0000","last_clean":"2023-11-28T03:09:47.770194+0000","last_became_active":"2023-11-28T03:09:47.764903+0000","last_became_peered":"2023-11-28T03:09:47.764903+0000","last_unstale":"2023-11-28T03:09:47.770194+0000","last_undegraded":"2023-11-28T03:09:47.770194+0000","last_fullsized":"2023-11-28T03:09:47.770194+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.1d","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767357+0000","last_change":"2023-11-28T03:09:47.385551+0000","last_active":"2023-11-28T03:09:47.767357+0000","last_peered":"2023-11-28T03:09:47.767357+0000","last_clean":"2023-11-28T03:09:47.767357+0000","last_became_active":"2023-11-28T03:09:47.385227+0000","last_became_peered":"2023-11-28T03:09:47.385227+0000","last_unstale":"2023-11-28T03:09:47.767357+0000","last_undegraded":"2023-11-28T03:09:47.767357+0000","last_fullsized":"2023-11-28T03:09:47.767357+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.769734+0000","last_change":"2023-11-28T03:09:47.765411+0000","last_active":"2023-11-28T03:09:47.769734+0000","last_peered":"2023-11-28T03:09:47.769734+0000","last_clean":"2023-11-28T03:09:47.769734+0000","last_became_active":"2023-11-28T03:09:47.765119+0000","last_became_peered":"2023-11-28T03:09:47.765119+0000","last_unstale":"2023-11-28T03:09:47.769734+0000","last_undegraded":"2023-11-28T03:09:47.769734+0000","last_fullsized":"2023-11-28T03:09:47.769734+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747764+0000","last_change":"2023-11-28T03:09:47.389262+0000","last_active":"2023-11-28T03:09:47.747764+0000","last_peered":"2023-11-28T03:09:47.747764+0000","last_clean":"2023-11-28T03:09:47.747764+0000","last_became_active":"2023-11-28T03:09:47.388988+0000","last_became_peered":"2023-11-28T03:09:47.388988+0000","last_unstale":"2023-11-28T03:09:47.747764+0000","last_undegraded":"2023-11-28T03:09:47.747764+0000","last_fullsized":"2023-11-28T03:09:47.747764+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.763347+0000","last_change":"2023-11-28T03:09:47.760154+0000","last_active":"2023-11-28T03:09:47.763347+0000","last_peered":"2023-11-28T03:09:47.763347+0000","last_clean":"2023-11-28T03:09:47.763347+0000","last_became_active":"2023-11-28T03:09:47.759874+0000","last_became_peered":"2023-11-28T03:09:47.759874+0000","last_unstale":"2023-11-28T03:09:47.763347+0000","last_undegraded":"2023-11-28T03:09:47.763347+0000","last_fullsized":"2023-11-28T03:09:47.763347+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752780+0000","last_change":"2023-11-28T03:09:47.381553+0000","last_active":"2023-11-28T03:09:47.752780+0000","last_peered":"2023-11-28T03:09:47.752780+0000","last_clean":"2023-11-28T03:09:47.752780+0000","last_became_active":"2023-11-28T03:09:47.381281+0000","last_became_peered":"2023-11-28T03:09:47.381281+0000","last_unstale":"2023-11-28T03:09:47.752780+0000","last_undegraded":"2023-11-28T03:09:47.752780+0000","last_fullsized":"2023-11-28T03:09:47.752780+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767987+0000","last_change":"2023-11-28T03:09:47.763223+0000","last_active":"2023-11-28T03:09:47.767987+0000","last_peered":"2023-11-28T03:09:47.767987+0000","last_clean":"2023-11-28T03:09:47.767987+0000","last_became_active":"2023-11-28T03:09:47.762958+0000","last_became_peered":"2023-11-28T03:09:47.762958+0000","last_unstale":"2023-11-28T03:09:47.767987+0000","last_undegraded":"2023-11-28T03:09:47.767987+0000","last_fullsized":"2023-11-28T03:09:47.767987+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.757048+0000","last_change":"2023-11-28T03:09:47.380579+0000","last_active":"2023-11-28T03:09:47.757048+0000","last_peered":"2023-11-28T03:09:47.757048+0000","last_clean":"2023-11-28T03:09:47.757048+0000","last_became_active":"2023-11-28T03:09:47.379835+0000","last_became_peered":"2023-11-28T03:09:47.379835+0000","last_unstale":"2023-11-28T03:09:47.757048+0000","last_undegraded":"2023-11-28T03:09:47.757048+0000","last_fullsized":"2023-11-28T03:09:47.757048+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775459+0000","last_change":"2023-11-28T03:09:47.768612+0000","last_active":"2023-11-28T03:09:47.775459+0000","last_peered":"2023-11-28T03:09:47.775459+0000","last_clean":"2023-11-28T03:09:47.775459+0000","last_became_active":"2023-11-28T03:09:47.767705+0000","last_became_peered":"2023-11-28T03:09:47.767705+0000","last_unstale":"2023-11-28T03:09:47.775459+0000","last_undegraded":"2023-11-28T03:09:47.775459+0000","last_fullsized":"2023-11-28T03:09:47.775459+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748269+0000","last_change":"2023-11-28T03:09:47.383880+0000","last_active":"2023-11-28T03:09:47.748269+0000","last_peered":"2023-11-28T03:09:47.748269+0000","last_clean":"2023-11-28T03:09:47.748269+0000","last_became_active":"2023-11-28T03:09:47.383611+0000","last_became_peered":"2023-11-28T03:09:47.383611+0000","last_unstale":"2023-11-28T03:09:47.748269+0000","last_undegraded":"2023-11-28T03:09:47.748269+0000","last_fullsized":"2023-11-28T03:09:47.748269+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.7","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.762485+0000","last_change":"2023-11-28T03:09:47.760482+0000","last_active":"2023-11-28T03:09:47.762485+0000","last_peered":"2023-11-28T03:09:47.762485+0000","last_clean":"2023-11-28T03:09:47.762485+0000","last_became_active":"2023-11-28T03:09:47.759849+0000","last_became_peered":"2023-11-28T03:09:47.759849+0000","last_unstale":"2023-11-28T03:09:47.762485+0000","last_undegraded":"2023-11-28T03:09:47.762485+0000","last_fullsized":"2023-11-28T03:09:47.762485+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.754087+0000","last_change":"2023-11-28T03:09:47.387939+0000","last_active":"2023-11-28T03:09:47.754087+0000","last_peered":"2023-11-28T03:09:47.754087+0000","last_clean":"2023-11-28T03:09:47.754087+0000","last_became_active":"2023-11-28T03:09:47.387663+0000","last_became_peered":"2023-11-28T03:09:47.387663+0000","last_unstale":"2023-11-28T03:09:47.754087+0000","last_undegraded":"2023-11-28T03:09:47.754087+0000","last_fullsized":"2023-11-28T03:09:47.754087+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766637+0000","last_change":"2023-11-28T03:09:47.761146+0000","last_active":"2023-11-28T03:09:47.766637+0000","last_peered":"2023-11-28T03:09:47.766637+0000","last_clean":"2023-11-28T03:09:47.766637+0000","last_became_active":"2023-11-28T03:09:47.760689+0000","last_became_peered":"2023-11-28T03:09:47.760689+0000","last_unstale":"2023-11-28T03:09:47.766637+0000","last_undegraded":"2023-11-28T03:09:47.766637+0000","last_fullsized":"2023-11-28T03:09:47.766637+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748716+0000","last_change":"2023-11-28T03:09:47.385497+0000","last_active":"2023-11-28T03:09:47.748716+0000","last_peered":"2023-11-28T03:09:47.748716+0000","last_clean":"2023-11-28T03:09:47.748716+0000","last_became_active":"2023-11-28T03:09:47.385147+0000","last_became_peered":"2023-11-28T03:09:47.385147+0000","last_unstale":"2023-11-28T03:09:47.748716+0000","last_undegraded":"2023-11-28T03:09:47.748716+0000","last_fullsized":"2023-11-28T03:09:47.748716+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773247+0000","last_change":"2023-11-28T03:09:47.764684+0000","last_active":"2023-11-28T03:09:47.773247+0000","last_peered":"2023-11-28T03:09:47.773247+0000","last_clean":"2023-11-28T03:09:47.773247+0000","last_became_active":"2023-11-28T03:09:47.764420+0000","last_became_peered":"2023-11-28T03:09:47.764420+0000","last_unstale":"2023-11-28T03:09:47.773247+0000","last_undegraded":"2023-11-28T03:09:47.773247+0000","last_fullsized":"2023-11-28T03:09:47.773247+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750830+0000","last_change":"2023-11-28T03:09:47.387921+0000","last_active":"2023-11-28T03:09:47.750830+0000","last_peered":"2023-11-28T03:09:47.750830+0000","last_clean":"2023-11-28T03:09:47.750830+0000","last_became_active":"2023-11-28T03:09:47.387346+0000","last_became_peered":"2023-11-28T03:09:47.387346+0000","last_unstale":"2023-11-28T03:09:47.750830+0000","last_undegraded":"2023-11-28T03:09:47.750830+0000","last_fullsized":"2023-11-28T03:09:47.750830+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776192+0000","last_change":"2023-11-28T03:09:47.773540+0000","last_active":"2023-11-28T03:09:47.776192+0000","last_peered":"2023-11-28T03:09:47.776192+0000","last_clean":"2023-11-28T03:09:47.776192+0000","last_became_active":"2023-11-28T03:09:47.773261+0000","last_became_peered":"2023-11-28T03:09:47.773261+0000","last_unstale":"2023-11-28T03:09:47.776192+0000","last_undegraded":"2023-11-28T03:09:47.776192+0000","last_fullsized":"2023-11-28T03:09:47.776192+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.15","version":"125'3","reported_seq":13,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772664+0000","last_change":"2023-11-28T03:09:47.389637+0000","last_active":"2023-11-28T03:09:47.772664+0000","last_peered":"2023-11-28T03:09:47.772664+0000","last_clean":"2023-11-28T03:09:47.772664+0000","last_became_active":"2023-11-28T03:09:47.389333+0000","last_became_peered":"2023-11-28T03:09:47.389333+0000","last_unstale":"2023-11-28T03:09:47.772664+0000","last_undegraded":"2023-11-28T03:09:47.772664+0000","last_fullsized":"2023-11-28T03:09:47.772664+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773504+0000","last_change":"2023-11-28T03:09:47.766624+0000","last_active":"2023-11-28T03:09:47.773504+0000","last_peered":"2023-11-28T03:09:47.773504+0000","last_clean":"2023-11-28T03:09:47.773504+0000","last_became_active":"2023-11-28T03:09:47.765923+0000","last_became_peered":"2023-11-28T03:09:47.765923+0000","last_unstale":"2023-11-28T03:09:47.773504+0000","last_undegraded":"2023-11-28T03:09:47.773504+0000","last_fullsized":"2023-11-28T03:09:47.773504+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.14","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767181+0000","last_change":"2023-11-28T03:09:47.388066+0000","last_active":"2023-11-28T03:09:47.767181+0000","last_peered":"2023-11-28T03:09:47.767181+0000","last_clean":"2023-11-28T03:09:47.767181+0000","last_became_active":"2023-11-28T03:09:47.387767+0000","last_became_peered":"2023-11-28T03:09:47.387767+0000","last_unstale":"2023-11-28T03:09:47.767181+0000","last_undegraded":"2023-11-28T03:09:47.767181+0000","last_fullsized":"2023-11-28T03:09:47.767181+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770091+0000","last_change":"2023-11-28T03:09:47.764418+0000","last_active":"2023-11-28T03:09:47.770091+0000","last_peered":"2023-11-28T03:09:47.770091+0000","last_clean":"2023-11-28T03:09:47.770091+0000","last_became_active":"2023-11-28T03:09:47.763263+0000","last_became_peered":"2023-11-28T03:09:47.763263+0000","last_unstale":"2023-11-28T03:09:47.770091+0000","last_undegraded":"2023-11-28T03:09:47.770091+0000","last_fullsized":"2023-11-28T03:09:47.770091+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.13","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770865+0000","last_change":"2023-11-28T03:09:47.395717+0000","last_active":"2023-11-28T03:09:47.770865+0000","last_peered":"2023-11-28T03:09:47.770865+0000","last_clean":"2023-11-28T03:09:47.770865+0000","last_became_active":"2023-11-28T03:09:47.395484+0000","last_became_peered":"2023-11-28T03:09:47.395484+0000","last_unstale":"2023-11-28T03:09:47.770865+0000","last_undegraded":"2023-11-28T03:09:47.770865+0000","last_fullsized":"2023-11-28T03:09:47.770865+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767178+0000","last_change":"2023-11-28T03:09:47.762679+0000","last_active":"2023-11-28T03:09:47.767178+0000","last_peered":"2023-11-28T03:09:47.767178+0000","last_clean":"2023-11-28T03:09:47.767178+0000","last_became_active":"2023-11-28T03:09:47.762384+0000","last_became_peered":"2023-11-28T03:09:47.762384+0000","last_unstale":"2023-11-28T03:09:47.767178+0000","last_undegraded":"2023-11-28T03:09:47.767178+0000","last_fullsized":"2023-11-28T03:09:47.767178+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752947+0000","last_change":"2023-11-28T03:09:47.381351+0000","last_active":"2023-11-28T03:09:47.752947+0000","last_peered":"2023-11-28T03:09:47.752947+0000","last_clean":"2023-11-28T03:09:47.752947+0000","last_became_active":"2023-11-28T03:09:47.381096+0000","last_became_peered":"2023-11-28T03:09:47.381096+0000","last_unstale":"2023-11-28T03:09:47.752947+0000","last_undegraded":"2023-11-28T03:09:47.752947+0000","last_fullsized":"2023-11-28T03:09:47.752947+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.767695+0000","last_change":"2023-11-28T03:09:47.766180+0000","last_active":"2023-11-28T03:09:47.767695+0000","last_peered":"2023-11-28T03:09:47.767695+0000","last_clean":"2023-11-28T03:09:47.767695+0000","last_became_active":"2023-11-28T03:09:47.765901+0000","last_became_peered":"2023-11-28T03:09:47.765901+0000","last_unstale":"2023-11-28T03:09:47.767695+0000","last_undegraded":"2023-11-28T03:09:47.767695+0000","last_fullsized":"2023-11-28T03:09:47.767695+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755399+0000","last_change":"2023-11-28T03:09:47.389094+0000","last_active":"2023-11-28T03:09:47.755399+0000","last_peered":"2023-11-28T03:09:47.755399+0000","last_clean":"2023-11-28T03:09:47.755399+0000","last_became_active":"2023-11-28T03:09:47.388840+0000","last_became_peered":"2023-11-28T03:09:47.388840+0000","last_unstale":"2023-11-28T03:09:47.755399+0000","last_undegraded":"2023-11-28T03:09:47.755399+0000","last_fullsized":"2023-11-28T03:09:47.755399+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773011+0000","last_change":"2023-11-28T03:09:47.767119+0000","last_active":"2023-11-28T03:09:47.773011+0000","last_peered":"2023-11-28T03:09:47.773011+0000","last_clean":"2023-11-28T03:09:47.773011+0000","last_became_active":"2023-11-28T03:09:47.766500+0000","last_became_peered":"2023-11-28T03:09:47.766500+0000","last_unstale":"2023-11-28T03:09:47.773011+0000","last_undegraded":"2023-11-28T03:09:47.773011+0000","last_fullsized":"2023-11-28T03:09:47.773011+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.10","version":"125'4","reported_seq":14,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766554+0000","last_change":"2023-11-28T03:09:47.379880+0000","last_active":"2023-11-28T03:09:47.766554+0000","last_peered":"2023-11-28T03:09:47.766554+0000","last_clean":"2023-11-28T03:09:47.766554+0000","last_became_active":"2023-11-28T03:09:47.379590+0000","last_became_peered":"2023-11-28T03:09:47.379590+0000","last_unstale":"2023-11-28T03:09:47.766554+0000","last_undegraded":"2023-11-28T03:09:47.766554+0000","last_fullsized":"2023-11-28T03:09:47.766554+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766608+0000","last_change":"2023-11-28T03:09:47.761013+0000","last_active":"2023-11-28T03:09:47.766608+0000","last_peered":"2023-11-28T03:09:47.766608+0000","last_clean":"2023-11-28T03:09:47.766608+0000","last_became_active":"2023-11-28T03:09:47.760574+0000","last_became_peered":"2023-11-28T03:09:47.760574+0000","last_unstale":"2023-11-28T03:09:47.766608+0000","last_undegraded":"2023-11-28T03:09:47.766608+0000","last_fullsized":"2023-11-28T03:09:47.766608+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.7","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.768209+0000","last_change":"2023-11-28T03:09:47.394060+0000","last_active":"2023-11-28T03:09:47.768209+0000","last_peered":"2023-11-28T03:09:47.768209+0000","last_clean":"2023-11-28T03:09:47.768209+0000","last_became_active":"2023-11-28T03:09:47.393707+0000","last_became_peered":"2023-11-28T03:09:47.393707+0000","last_unstale":"2023-11-28T03:09:47.768209+0000","last_undegraded":"2023-11-28T03:09:47.768209+0000","last_fullsized":"2023-11-28T03:09:47.768209+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.765586+0000","last_change":"2023-11-28T03:09:47.759207+0000","last_active":"2023-11-28T03:09:47.765586+0000","last_peered":"2023-11-28T03:09:47.765586+0000","last_clean":"2023-11-28T03:09:47.765586+0000","last_became_active":"2023-11-28T03:09:47.758881+0000","last_became_peered":"2023-11-28T03:09:47.758881+0000","last_unstale":"2023-11-28T03:09:47.765586+0000","last_undegraded":"2023-11-28T03:09:47.765586+0000","last_fullsized":"2023-11-28T03:09:47.765586+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.750926+0000","last_change":"2023-11-28T03:09:47.388105+0000","last_active":"2023-11-28T03:09:47.750926+0000","last_peered":"2023-11-28T03:09:47.750926+0000","last_clean":"2023-11-28T03:09:47.750926+0000","last_became_active":"2023-11-28T03:09:47.387671+0000","last_became_peered":"2023-11-28T03:09:47.387671+0000","last_unstale":"2023-11-28T03:09:47.750926+0000","last_undegraded":"2023-11-28T03:09:47.750926+0000","last_fullsized":"2023-11-28T03:09:47.750926+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1a","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.770329+0000","last_change":"2023-11-28T03:09:47.765129+0000","last_active":"2023-11-28T03:09:47.770329+0000","last_peered":"2023-11-28T03:09:47.770329+0000","last_clean":"2023-11-28T03:09:47.770329+0000","last_became_active":"2023-11-28T03:09:47.764865+0000","last_became_peered":"2023-11-28T03:09:47.764865+0000","last_unstale":"2023-11-28T03:09:47.770329+0000","last_undegraded":"2023-11-28T03:09:47.770329+0000","last_fullsized":"2023-11-28T03:09:47.770329+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753891+0000","last_change":"2023-11-28T03:09:47.383234+0000","last_active":"2023-11-28T03:09:47.753891+0000","last_peered":"2023-11-28T03:09:47.753891+0000","last_clean":"2023-11-28T03:09:47.753891+0000","last_became_active":"2023-11-28T03:09:47.382933+0000","last_became_peered":"2023-11-28T03:09:47.382933+0000","last_unstale":"2023-11-28T03:09:47.753891+0000","last_undegraded":"2023-11-28T03:09:47.753891+0000","last_fullsized":"2023-11-28T03:09:47.753891+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1b","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775982+0000","last_change":"2023-11-28T03:09:47.775110+0000","last_active":"2023-11-28T03:09:47.775982+0000","last_peered":"2023-11-28T03:09:47.775982+0000","last_clean":"2023-11-28T03:09:47.775982+0000","last_became_active":"2023-11-28T03:09:47.774768+0000","last_became_peered":"2023-11-28T03:09:47.774768+0000","last_unstale":"2023-11-28T03:09:47.775982+0000","last_undegraded":"2023-11-28T03:09:47.775982+0000","last_fullsized":"2023-11-28T03:09:47.775982+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747737+0000","last_change":"2023-11-28T03:09:47.391565+0000","last_active":"2023-11-28T03:09:47.747737+0000","last_peered":"2023-11-28T03:09:47.747737+0000","last_clean":"2023-11-28T03:09:47.747737+0000","last_became_active":"2023-11-28T03:09:47.391157+0000","last_became_peered":"2023-11-28T03:09:47.391157+0000","last_unstale":"2023-11-28T03:09:47.747737+0000","last_undegraded":"2023-11-28T03:09:47.747737+0000","last_fullsized":"2023-11-28T03:09:47.747737+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776195+0000","last_change":"2023-11-28T03:09:47.773506+0000","last_active":"2023-11-28T03:09:47.776195+0000","last_peered":"2023-11-28T03:09:47.776195+0000","last_clean":"2023-11-28T03:09:47.776195+0000","last_became_active":"2023-11-28T03:09:47.773240+0000","last_became_peered":"2023-11-28T03:09:47.773240+0000","last_unstale":"2023-11-28T03:09:47.776195+0000","last_undegraded":"2023-11-28T03:09:47.776195+0000","last_fullsized":"2023-11-28T03:09:47.776195+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748903+0000","last_change":"2023-11-28T03:09:47.381999+0000","last_active":"2023-11-28T03:09:47.748903+0000","last_peered":"2023-11-28T03:09:47.748903+0000","last_clean":"2023-11-28T03:09:47.748903+0000","last_became_active":"2023-11-28T03:09:47.381649+0000","last_became_peered":"2023-11-28T03:09:47.381649+0000","last_unstale":"2023-11-28T03:09:47.748903+0000","last_undegraded":"2023-11-28T03:09:47.748903+0000","last_fullsized":"2023-11-28T03:09:47.748903+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1d","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.776171+0000","last_change":"2023-11-28T03:09:47.773468+0000","last_active":"2023-11-28T03:09:47.776171+0000","last_peered":"2023-11-28T03:09:47.776171+0000","last_clean":"2023-11-28T03:09:47.776171+0000","last_became_active":"2023-11-28T03:09:47.773190+0000","last_became_peered":"2023-11-28T03:09:47.773190+0000","last_unstale":"2023-11-28T03:09:47.776171+0000","last_undegraded":"2023-11-28T03:09:47.776171+0000","last_fullsized":"2023-11-28T03:09:47.776171+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755888+0000","last_change":"2023-11-28T03:09:47.389380+0000","last_active":"2023-11-28T03:09:47.755888+0000","last_peered":"2023-11-28T03:09:47.755888+0000","last_clean":"2023-11-28T03:09:47.755888+0000","last_became_active":"2023-11-28T03:09:47.388732+0000","last_became_peered":"2023-11-28T03:09:47.388732+0000","last_unstale":"2023-11-28T03:09:47.755888+0000","last_undegraded":"2023-11-28T03:09:47.755888+0000","last_fullsized":"2023-11-28T03:09:47.755888+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.1e","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772694+0000","last_change":"2023-11-28T03:09:47.763808+0000","last_active":"2023-11-28T03:09:47.772694+0000","last_peered":"2023-11-28T03:09:47.772694+0000","last_clean":"2023-11-28T03:09:47.772694+0000","last_became_active":"2023-11-28T03:09:47.763538+0000","last_became_peered":"2023-11-28T03:09:47.763538+0000","last_unstale":"2023-11-28T03:09:47.772694+0000","last_undegraded":"2023-11-28T03:09:47.772694+0000","last_fullsized":"2023-11-28T03:09:47.772694+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755809+0000","last_change":"2023-11-28T03:09:47.384792+0000","last_active":"2023-11-28T03:09:47.755809+0000","last_peered":"2023-11-28T03:09:47.755809+0000","last_clean":"2023-11-28T03:09:47.755809+0000","last_became_active":"2023-11-28T03:09:47.384525+0000","last_became_peered":"2023-11-28T03:09:47.384525+0000","last_unstale":"2023-11-28T03:09:47.755809+0000","last_undegraded":"2023-11-28T03:09:47.755809+0000","last_fullsized":"2023-11-28T03:09:47.755809+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.769571+0000","last_change":"2023-11-28T03:09:47.766442+0000","last_active":"2023-11-28T03:09:47.769571+0000","last_peered":"2023-11-28T03:09:47.769571+0000","last_clean":"2023-11-28T03:09:47.769571+0000","last_became_active":"2023-11-28T03:09:47.766117+0000","last_became_peered":"2023-11-28T03:09:47.766117+0000","last_unstale":"2023-11-28T03:09:47.769571+0000","last_undegraded":"2023-11-28T03:09:47.769571+0000","last_fullsized":"2023-11-28T03:09:47.769571+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755834+0000","last_change":"2023-11-28T03:09:47.391584+0000","last_active":"2023-11-28T03:09:47.755834+0000","last_peered":"2023-11-28T03:09:47.755834+0000","last_clean":"2023-11-28T03:09:47.755834+0000","last_became_active":"2023-11-28T03:09:47.391149+0000","last_became_peered":"2023-11-28T03:09:47.391149+0000","last_unstale":"2023-11-28T03:09:47.755834+0000","last_undegraded":"2023-11-28T03:09:47.755834+0000","last_fullsized":"2023-11-28T03:09:47.755834+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772139+0000","last_change":"2023-11-28T03:09:47.764924+0000","last_active":"2023-11-28T03:09:47.772139+0000","last_peered":"2023-11-28T03:09:47.772139+0000","last_clean":"2023-11-28T03:09:47.772139+0000","last_became_active":"2023-11-28T03:09:47.764678+0000","last_became_peered":"2023-11-28T03:09:47.764678+0000","last_unstale":"2023-11-28T03:09:47.772139+0000","last_undegraded":"2023-11-28T03:09:47.772139+0000","last_fullsized":"2023-11-28T03:09:47.772139+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755688+0000","last_change":"2023-11-28T03:09:47.387312+0000","last_active":"2023-11-28T03:09:47.755688+0000","last_peered":"2023-11-28T03:09:47.755688+0000","last_clean":"2023-11-28T03:09:47.755688+0000","last_became_active":"2023-11-28T03:09:47.386963+0000","last_became_peered":"2023-11-28T03:09:47.386963+0000","last_unstale":"2023-11-28T03:09:47.755688+0000","last_undegraded":"2023-11-28T03:09:47.755688+0000","last_fullsized":"2023-11-28T03:09:47.755688+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.747844+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:09:47.747844+0000","last_peered":"2023-11-28T03:09:47.747844+0000","last_clean":"2023-11-28T03:09:47.747844+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:09:47.747844+0000","last_undegraded":"2023-11-28T03:09:47.747844+0000","last_fullsized":"2023-11-28T03:09:47.747844+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.10","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764001+0000","last_change":"2023-11-28T03:09:47.762200+0000","last_active":"2023-11-28T03:09:47.764001+0000","last_peered":"2023-11-28T03:09:47.764001+0000","last_clean":"2023-11-28T03:09:47.764001+0000","last_became_active":"2023-11-28T03:09:47.761919+0000","last_became_peered":"2023-11-28T03:09:47.761919+0000","last_unstale":"2023-11-28T03:09:47.764001+0000","last_undegraded":"2023-11-28T03:09:47.764001+0000","last_fullsized":"2023-11-28T03:09:47.764001+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.f","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766654+0000","last_change":"2023-11-28T03:09:47.383551+0000","last_active":"2023-11-28T03:09:47.766654+0000","last_peered":"2023-11-28T03:09:47.766654+0000","last_clean":"2023-11-28T03:09:47.766654+0000","last_became_active":"2023-11-28T03:09:47.382466+0000","last_became_peered":"2023-11-28T03:09:47.382466+0000","last_unstale":"2023-11-28T03:09:47.766654+0000","last_undegraded":"2023-11-28T03:09:47.766654+0000","last_fullsized":"2023-11-28T03:09:47.766654+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.757098+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:09:47.757098+0000","last_peered":"2023-11-28T03:09:47.757098+0000","last_clean":"2023-11-28T03:09:47.757098+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:09:47.757098+0000","last_undegraded":"2023-11-28T03:09:47.757098+0000","last_fullsized":"2023-11-28T03:09:47.757098+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.13","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.773479+0000","last_change":"2023-11-28T03:09:47.765348+0000","last_active":"2023-11-28T03:09:47.773479+0000","last_peered":"2023-11-28T03:09:47.773479+0000","last_clean":"2023-11-28T03:09:47.773479+0000","last_became_active":"2023-11-28T03:09:47.765087+0000","last_became_peered":"2023-11-28T03:09:47.765087+0000","last_unstale":"2023-11-28T03:09:47.773479+0000","last_undegraded":"2023-11-28T03:09:47.773479+0000","last_fullsized":"2023-11-28T03:09:47.773479+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.754604+0000","last_change":"2023-11-28T03:09:47.384853+0000","last_active":"2023-11-28T03:09:47.754604+0000","last_peered":"2023-11-28T03:09:47.754604+0000","last_clean":"2023-11-28T03:09:47.754604+0000","last_became_active":"2023-11-28T03:09:47.384557+0000","last_became_peered":"2023-11-28T03:09:47.384557+0000","last_unstale":"2023-11-28T03:09:47.754604+0000","last_undegraded":"2023-11-28T03:09:47.754604+0000","last_fullsized":"2023-11-28T03:09:47.754604+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":104,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755522+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:09:47.755522+0000","last_peered":"2023-11-28T03:09:47.755522+0000","last_clean":"2023-11-28T03:09:47.755522+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:09:47.755522+0000","last_undegraded":"2023-11-28T03:09:47.755522+0000","last_fullsized":"2023-11-28T03:09:47.755522+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764696+0000","last_change":"2023-11-28T03:09:47.761186+0000","last_active":"2023-11-28T03:09:47.764696+0000","last_peered":"2023-11-28T03:09:47.764696+0000","last_clean":"2023-11-28T03:09:47.764696+0000","last_became_active":"2023-11-28T03:09:47.760876+0000","last_became_peered":"2023-11-28T03:09:47.760876+0000","last_unstale":"2023-11-28T03:09:47.764696+0000","last_undegraded":"2023-11-28T03:09:47.764696+0000","last_fullsized":"2023-11-28T03:09:47.764696+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.e","version":"125'2","reported_seq":12,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.761975+0000","last_change":"2023-11-28T03:09:47.386633+0000","last_active":"2023-11-28T03:09:47.761975+0000","last_peered":"2023-11-28T03:09:47.761975+0000","last_clean":"2023-11-28T03:09:47.761975+0000","last_became_active":"2023-11-28T03:09:47.386182+0000","last_became_peered":"2023-11-28T03:09:47.386182+0000","last_unstale":"2023-11-28T03:09:47.761975+0000","last_undegraded":"2023-11-28T03:09:47.761975+0000","last_fullsized":"2023-11-28T03:09:47.761975+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.758184+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:09:47.758184+0000","last_peered":"2023-11-28T03:09:47.758184+0000","last_clean":"2023-11-28T03:09:47.758184+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:09:47.758184+0000","last_undegraded":"2023-11-28T03:09:47.758184+0000","last_fullsized":"2023-11-28T03:09:47.758184+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":99,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752931+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:09:47.752931+0000","last_peered":"2023-11-28T03:09:47.752931+0000","last_clean":"2023-11-28T03:09:47.752931+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:09:47.752931+0000","last_undegraded":"2023-11-28T03:09:47.752931+0000","last_fullsized":"2023-11-28T03:09:47.752931+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.764113+0000","last_change":"2023-11-28T03:09:47.758283+0000","last_active":"2023-11-28T03:09:47.764113+0000","last_peered":"2023-11-28T03:09:47.764113+0000","last_clean":"2023-11-28T03:09:47.764113+0000","last_became_active":"2023-11-28T03:09:47.757999+0000","last_became_peered":"2023-11-28T03:09:47.757999+0000","last_unstale":"2023-11-28T03:09:47.764113+0000","last_undegraded":"2023-11-28T03:09:47.764113+0000","last_fullsized":"2023-11-28T03:09:47.764113+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752605+0000","last_change":"2023-11-28T03:09:47.385187+0000","last_active":"2023-11-28T03:09:47.752605+0000","last_peered":"2023-11-28T03:09:47.752605+0000","last_clean":"2023-11-28T03:09:47.752605+0000","last_became_active":"2023-11-28T03:09:47.384894+0000","last_became_peered":"2023-11-28T03:09:47.384894+0000","last_unstale":"2023-11-28T03:09:47.752605+0000","last_undegraded":"2023-11-28T03:09:47.752605+0000","last_fullsized":"2023-11-28T03:09:47.752605+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.753007+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:09:47.753007+0000","last_peered":"2023-11-28T03:09:47.753007+0000","last_clean":"2023-11-28T03:09:47.753007+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:09:47.753007+0000","last_undegraded":"2023-11-28T03:09:47.753007+0000","last_fullsized":"2023-11-28T03:09:47.753007+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.772799+0000","last_change":"2023-11-28T03:09:47.769354+0000","last_active":"2023-11-28T03:09:47.772799+0000","last_peered":"2023-11-28T03:09:47.772799+0000","last_clean":"2023-11-28T03:09:47.772799+0000","last_became_active":"2023-11-28T03:09:47.769091+0000","last_became_peered":"2023-11-28T03:09:47.769091+0000","last_unstale":"2023-11-28T03:09:47.772799+0000","last_undegraded":"2023-11-28T03:09:47.772799+0000","last_fullsized":"2023-11-28T03:09:47.772799+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755619+0000","last_change":"2023-11-28T03:09:47.385514+0000","last_active":"2023-11-28T03:09:47.755619+0000","last_peered":"2023-11-28T03:09:47.755619+0000","last_clean":"2023-11-28T03:09:47.755619+0000","last_became_active":"2023-11-28T03:09:47.384387+0000","last_became_peered":"2023-11-28T03:09:47.384387+0000","last_unstale":"2023-11-28T03:09:47.755619+0000","last_undegraded":"2023-11-28T03:09:47.755619+0000","last_fullsized":"2023-11-28T03:09:47.755619+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.752806+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:09:47.752806+0000","last_peered":"2023-11-28T03:09:47.752806+0000","last_clean":"2023-11-28T03:09:47.752806+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:09:47.752806+0000","last_undegraded":"2023-11-28T03:09:47.752806+0000","last_fullsized":"2023-11-28T03:09:47.752806+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.14","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766944+0000","last_change":"2023-11-28T03:09:47.762725+0000","last_active":"2023-11-28T03:09:47.766944+0000","last_peered":"2023-11-28T03:09:47.766944+0000","last_clean":"2023-11-28T03:09:47.766944+0000","last_became_active":"2023-11-28T03:09:47.762436+0000","last_became_peered":"2023-11-28T03:09:47.762436+0000","last_unstale":"2023-11-28T03:09:47.766944+0000","last_undegraded":"2023-11-28T03:09:47.766944+0000","last_fullsized":"2023-11-28T03:09:47.766944+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"125'1","reported_seq":11,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.766696+0000","last_change":"2023-11-28T03:09:47.388594+0000","last_active":"2023-11-28T03:09:47.766696+0000","last_peered":"2023-11-28T03:09:47.766696+0000","last_clean":"2023-11-28T03:09:47.766696+0000","last_became_active":"2023-11-28T03:09:47.387910+0000","last_became_peered":"2023-11-28T03:09:47.387910+0000","last_unstale":"2023-11-28T03:09:47.766696+0000","last_undegraded":"2023-11-28T03:09:47.766696+0000","last_fullsized":"2023-11-28T03:09:47.766696+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":97,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.748067+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:09:47.748067+0000","last_peered":"2023-11-28T03:09:47.748067+0000","last_clean":"2023-11-28T03:09:47.748067+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:09:47.748067+0000","last_undegraded":"2023-11-28T03:09:47.748067+0000","last_fullsized":"2023-11-28T03:09:47.748067+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.15","version":"0'0","reported_seq":9,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.775515+0000","last_change":"2023-11-28T03:09:47.771650+0000","last_active":"2023-11-28T03:09:47.775515+0000","last_peered":"2023-11-28T03:09:47.775515+0000","last_clean":"2023-11-28T03:09:47.775515+0000","last_became_active":"2023-11-28T03:09:47.768197+0000","last_became_peered":"2023-11-28T03:09:47.768197+0000","last_unstale":"2023-11-28T03:09:47.775515+0000","last_undegraded":"2023-11-28T03:09:47.775515+0000","last_fullsized":"2023-11-28T03:09:47.775515+0000","mapping_epoch":124,"log_start":"0'0","ondisk_log_start":"0'0","created":124,"last_epoch_clean":125,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:47.361629+0000","last_clean_scrub_stamp":"2023-11-28T03:09:47.361629+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":10,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.755657+0000","last_change":"2023-11-28T03:09:47.384548+0000","last_active":"2023-11-28T03:09:47.755657+0000","last_peered":"2023-11-28T03:09:47.755657+0000","last_clean":"2023-11-28T03:09:47.755657+0000","last_became_active":"2023-11-28T03:09:47.383855+0000","last_became_peered":"2023-11-28T03:09:47.383855+0000","last_unstale":"2023-11-28T03:09:47.755657+0000","last_undegraded":"2023-11-28T03:09:47.755657+0000","last_fullsized":"2023-11-28T03:09:47.755657+0000","mapping_epoch":123,"log_start":"0'0","ondisk_log_start":"0'0","created":123,"last_epoch_clean":124,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:09:46.356969+0000","last_clean_scrub_stamp":"2023-11-28T03:09:46.356969+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":112,"reported_epoch":125,"state":"active+clean","last_fresh":"2023-11-28T03:09:47.756127+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:09:47.756127+0000","last_peered":"2023-11-28T03:09:47.756127+0000","last_clean":"2023-11-28T03:09:47.756127+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:09:47.756127+0000","last_undegraded":"2023-11-28T03:09:47.756127+0000","last_fullsized":"2023-11-28T03:09:47.756127+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":16,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575053,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1560576,"data_stored":776680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.57699999999999996,"15min":0.59599999999999997},"min":{"1min":0.40300000000000002,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":1.46,"5min":1.46,"15min":1.46},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.58899999999999997,"15min":0.60499999999999998},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.59399999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.54600000000000004,"15min":0.54900000000000004},"min":{"1min":0.35699999999999998,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":1.5249999999999999,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.54000000000000004,"15min":0.55700000000000005},"min":{"1min":0.41199999999999998,"5min":0.318,"15min":0.318},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.54500000000000004}]},{"osd":2,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53400000000000003,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.378,"15min":0.34499999999999997},"max":{"1min":1.5009999999999999,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.54800000000000004,"15min":0.56200000000000006},"min":{"1min":0.40500000000000003,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.70999999999999996,"5min":0.80100000000000005,"15min":0.81899999999999995},"last":0.56799999999999995}]},{"osd":3,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55500000000000005,"15min":0.56599999999999995},"min":{"1min":0.39100000000000001,"5min":0.373,"15min":0.373},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.5},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57599999999999996},"min":{"1min":0.37,"5min":0.36599999999999999,"15min":0.34499999999999997},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.70199999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46600000000000003,"15min":0.46100000000000002},"min":{"1min":0.307,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70299999999999996,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.49099999999999999,"15min":0.48299999999999998},"min":{"1min":0.33800000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.78000000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.50900000000000001,"15min":0.495},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.78400000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.53400000000000003,"15min":0.53300000000000003},"min":{"1min":0.35399999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.69499999999999995,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.72199999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:09:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.749,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.36099999999999999,"15min":0.28999999999999998},"max":{"1min":0.746,"5min":0.78000000000000003,"15min":0.84499999999999997},"last":0.68100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575053,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302724,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069116,"statfs":{"total":96636764160,"available":96326774784,"internally_reserved":0,"allocated":1568768,"data_stored":777256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.53700000000000003,"15min":0.54700000000000004},"min":{"1min":0.42899999999999999,"5min":0.35399999999999998,"15min":0.34100000000000003},"max":{"1min":0.71399999999999997,"5min":1.4810000000000001,"15min":1.4810000000000001},"last":0.379},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55200000000000005,"15min":0.56699999999999995},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76800000000000002,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.47599999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56799999999999995,"15min":0.56599999999999995},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.29999999999999999},"max":{"1min":0.77800000000000002,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.499},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55600000000000005,"15min":0.55800000000000005},"min":{"1min":0.41299999999999998,"5min":0.375,"15min":0.35699999999999998},"max":{"1min":0.70299999999999996,"5min":1.506,"15min":1.506},"last":0.438}]},{"osd":2,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.55900000000000005,"15min":0.57099999999999995},"min":{"1min":0.40699999999999997,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.57899999999999996,"15min":0.57999999999999996},"min":{"1min":0.44800000000000001,"5min":0.34899999999999998,"15min":0.34100000000000003},"max":{"1min":0.68700000000000006,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.57299999999999995,"15min":0.58499999999999996},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72799999999999998,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.42599999999999999,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.79900000000000004,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.58199999999999996}]},{"osd":4,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.47099999999999997,"15min":0.45300000000000001},"min":{"1min":0.312,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.53300000000000003,"5min":1.7310000000000001,"15min":1.7310000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.496,"15min":0.48899999999999999},"min":{"1min":0.35899999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.58699999999999997,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.61899999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.53000000000000003,"15min":0.51300000000000001},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.67800000000000005,"5min":0.95099999999999996,"15min":0.95099999999999996},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.56299999999999994,"15min":0.54900000000000004},"min":{"1min":0.377,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.60399999999999998,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.51600000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:09:07 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.55400000000000005,"15min":0.53900000000000003},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.317},"max":{"1min":0.66100000000000003,"5min":1.3799999999999999,"15min":1.3799999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.56999999999999995,"15min":0.55500000000000005},"min":{"1min":0.38700000000000001,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.64500000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.66300000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575052,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302728,"kb_used_data":1536,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069112,"statfs":{"total":96636764160,"available":96326770688,"internally_reserved":0,"allocated":1572864,"data_stored":778140,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.33400000000000002,"15min":0.34499999999999997},"min":{"1min":0.253,"5min":0.189,"15min":0.189},"max":{"1min":0.54700000000000004,"5min":0.67800000000000005,"15min":0.70399999999999996},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.376,"5min":0.34999999999999998,"15min":0.371},"min":{"1min":0.23200000000000001,"5min":0.20899999999999999,"15min":0.20899999999999999},"max":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.67100000000000004},"last":0.30199999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.36899999999999999,"15min":0.38400000000000001},"min":{"1min":0.307,"5min":0.21199999999999999,"15min":0.21199999999999999},"max":{"1min":0.53800000000000003,"5min":0.57499999999999996,"15min":0.64900000000000002},"last":0.317},{"interface":"front","average":{"1min":0.41199999999999998,"5min":0.38700000000000001,"15min":0.40300000000000002},"min":{"1min":0.27800000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.61499999999999999,"5min":0.628,"15min":0.63400000000000001},"last":0.33800000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.42299999999999999,"15min":0.45400000000000001},"min":{"1min":0.32900000000000001,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.65200000000000002,"5min":0.69599999999999995,"15min":0.70799999999999996},"last":0.375},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39800000000000002,"15min":0.42399999999999999},"min":{"1min":0.29099999999999998,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.54300000000000004,"5min":0.68799999999999994,"15min":0.75600000000000001},"last":0.40500000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.57999999999999996,"15min":0.58999999999999997},"min":{"1min":0.44,"5min":0.313,"15min":0.313},"max":{"1min":0.89400000000000002,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.57699999999999996,"15min":0.59499999999999997},"min":{"1min":0.42499999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.97799999999999998},"last":0.63100000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.59899999999999998,"15min":0.62},"min":{"1min":0.47799999999999998,"5min":0.373,"15min":0.36199999999999999},"max":{"1min":0.81200000000000006,"5min":0.91900000000000004,"15min":0.99299999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.629,"15min":0.64700000000000002},"min":{"1min":0.48899999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.94199999999999995,"15min":1.038},"last":0.68700000000000006}]},{"osd":6,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.57499999999999996,"15min":0.59399999999999997},"min":{"1min":0.42999999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.78700000000000003,"5min":0.89400000000000002,"15min":0.94099999999999995},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.61399999999999999,"15min":0.621},"min":{"1min":0.46200000000000002,"5min":0.41199999999999998,"15min":0.40600000000000003},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:09:11 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.44600000000000001,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.84199999999999997,"5min":0.85199999999999998,"15min":1.024},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.59599999999999997,"15min":0.61699999999999999},"min":{"1min":0.437,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85099999999999998,"5min":0.87,"15min":0.88300000000000001},"last":0.61799999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575053,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302724,"kb_used_data":1532,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069116,"statfs":{"total":96636764160,"available":96326774784,"internally_reserved":0,"allocated":1568768,"data_stored":777324,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.33400000000000002,"15min":0.34399999999999997},"min":{"1min":0.221,"5min":0.217,"15min":0.184},"max":{"1min":0.46100000000000002,"5min":0.56599999999999995,"15min":0.56999999999999995},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.34200000000000003,"5min":0.34799999999999998,"15min":0.36499999999999999},"min":{"1min":0.27300000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.47499999999999998,"5min":0.70299999999999996,"15min":0.70299999999999996},"last":0.29799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.34000000000000002,"15min":0.36599999999999999},"min":{"1min":0.224,"5min":0.224,"15min":0.20100000000000001},"max":{"1min":0.503,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.246},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.41199999999999998,"15min":0.439},"min":{"1min":0.31900000000000001,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.56399999999999995,"5min":0.64200000000000002,"15min":0.73299999999999998},"last":0.33100000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.377,"15min":0.39100000000000001},"min":{"1min":0.28999999999999998,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.311},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.39300000000000002,"15min":0.41699999999999998},"min":{"1min":0.30599999999999999,"5min":0.217,"15min":0.217},"max":{"1min":0.61499999999999999,"5min":0.621,"15min":0.65400000000000003},"last":0.35199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.60999999999999999},"min":{"1min":0.33600000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.89600000000000002,"5min":0.94099999999999995,"15min":1.0489999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.60299999999999998,"15min":0.63300000000000001},"min":{"1min":0.41099999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.85899999999999999,"5min":1.2869999999999999,"15min":1.2869999999999999},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.59599999999999997,"15min":0.63},"min":{"1min":0.41099999999999998,"5min":0.374,"15min":0.374},"max":{"1min":0.83399999999999996,"5min":1.304,"15min":1.304},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59199999999999997,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.80900000000000005,"5min":0.90800000000000003,"15min":0.97299999999999998},"last":0.45300000000000001}]},{"osd":6,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.57499999999999996,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.79400000000000004,"5min":1.679,"15min":1.679},"last":0.436},{"interface":"front","average":{"1min":0.624,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.40400000000000003,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.86499999999999999,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.42199999999999999}]},{"osd":7,"last update":"Tue Nov 28 03:09:05 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58699999999999997,"15min":0.62},"min":{"1min":0.41199999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.84799999999999998,"5min":0.89700000000000002,"15min":0.97499999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.626,"15min":0.66700000000000004},"min":{"1min":0.44800000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.97399999999999998,"15min":1.046},"last":0.38400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575054,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302720,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069120,"statfs":{"total":96636764160,"available":96326778880,"internally_reserved":0,"allocated":1564672,"data_stored":776694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.34799999999999998,"15min":0.36599999999999999},"min":{"1min":0.23599999999999999,"5min":0.21099999999999999,"15min":0.20499999999999999},"max":{"1min":0.48499999999999999,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.38400000000000001,"15min":0.41699999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.53800000000000003,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.33000000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.34599999999999997,"15min":0.36699999999999999},"min":{"1min":0.24299999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.5,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.24299999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.36799999999999999,"15min":0.39400000000000002},"min":{"1min":0.245,"5min":0.218,"15min":0.218},"max":{"1min":0.58499999999999996,"5min":0.70399999999999996,"15min":0.73099999999999998},"last":0.371}]},{"osd":3,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.38800000000000001,"15min":0.41799999999999998},"min":{"1min":0.28199999999999997,"5min":0.19800000000000001,"15min":0.19800000000000001},"max":{"1min":0.55400000000000005,"5min":0.58199999999999996,"15min":0.67400000000000004},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.379,"15min":0.40000000000000002},"min":{"1min":0.248,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.53300000000000003,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.56999999999999995,"15min":0.58899999999999997},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.88600000000000001,"5min":0.91900000000000004,"15min":0.94599999999999995},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59099999999999997,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.61699999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.87,"15min":0.98599999999999999},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.622,"5min":0.623,"15min":0.63900000000000001},"min":{"1min":0.433,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.85799999999999998,"5min":0.94699999999999995,"15min":1.004},"last":0.67600000000000005}]},{"osd":6,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.60499999999999998,"15min":0.63200000000000001},"min":{"1min":0.41699999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.875,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.629,"15min":0.65500000000000003},"min":{"1min":0.44700000000000001,"5min":0.40699999999999997,"15min":0.40100000000000002},"max":{"1min":0.89200000000000002,"5min":0.90200000000000002,"15min":0.93999999999999995},"last":0.72399999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:08:57 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60199999999999998,"15min":0.63300000000000001},"min":{"1min":0.39700000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59999999999999998,"15min":0.63},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82599999999999996,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.60099999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575053,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302712,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069128,"statfs":{"total":96636764160,"available":96326787072,"internally_reserved":0,"allocated":1556480,"data_stored":776590,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.34499999999999997,"15min":0.36199999999999999},"min":{"1min":0.24299999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.52300000000000002,"5min":0.59899999999999998,"15min":0.59899999999999998},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.37,"5min":0.35699999999999998,"15min":0.377},"min":{"1min":0.27700000000000002,"5min":0.22500000000000001,"15min":0.218},"max":{"1min":0.56999999999999995,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.34499999999999997}]},{"osd":1,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.36099999999999999,"15min":0.38},"min":{"1min":0.245,"5min":0.23599999999999999,"15min":0.23499999999999999},"max":{"1min":0.50800000000000001,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.41799999999999998},"min":{"1min":0.30499999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.60099999999999998,"5min":0.68899999999999995,"15min":0.68899999999999995},"last":0.35799999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.39700000000000002,"15min":0.42099999999999999},"min":{"1min":0.25800000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.55300000000000005,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.44},"min":{"1min":0.309,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.68200000000000005},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.57599999999999996,"15min":0.60099999999999998},"min":{"1min":0.42599999999999999,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.82599999999999996,"5min":1.581,"15min":1.581},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.57999999999999996,"15min":0.61099999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.877,"5min":0.877,"15min":0.94199999999999995},"last":0.66500000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.57099999999999995,"15min":0.60099999999999998},"min":{"1min":0.40799999999999997,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.97899999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.621,"5min":0.59999999999999998,"15min":0.629},"min":{"1min":0.437,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81399999999999995,"5min":0.88300000000000001,"15min":0.93600000000000005},"last":0.71199999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.59299999999999997,"15min":0.63},"min":{"1min":0.42999999999999999,"5min":0.38300000000000001,"15min":0.35699999999999998},"max":{"1min":0.86899999999999999,"5min":0.90900000000000003,"15min":0.996},"last":0.625},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.59499999999999997,"15min":0.63400000000000001},"min":{"1min":0.42399999999999999,"5min":0.37,"15min":0.37},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.63800000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:09:00 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61399999999999999,"15min":0.64700000000000002},"min":{"1min":0.46400000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.78400000000000003,"5min":0.93500000000000005,"15min":0.97199999999999998},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.60599999999999998,"15min":0.64600000000000002},"min":{"1min":0.47899999999999998,"5min":0.376,"15min":0.376},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.60299999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575052,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302720,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069120,"statfs":{"total":96636764160,"available":96326778880,"internally_reserved":0,"allocated":1564672,"data_stored":777504,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.52100000000000002,"15min":0.54600000000000004},"min":{"1min":0.374,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.67700000000000005,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.378,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.752,"5min":0.78900000000000003,"15min":0.83199999999999996},"last":0.42999999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53500000000000003,"15min":0.55300000000000005},"min":{"1min":0.377,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.68700000000000006,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.53600000000000003,"15min":0.55900000000000005},"min":{"1min":0.34599999999999997,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.66900000000000004,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.51000000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.53000000000000003,"15min":0.55800000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.71299999999999997,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.56100000000000005},"min":{"1min":0.373,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.67700000000000005,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.497}]},{"osd":3,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.56899999999999995,"15min":0.59699999999999998},"min":{"1min":0.39000000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.70099999999999996,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.59599999999999997,"15min":0.625},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67600000000000005,"5min":0.84599999999999997,"15min":0.878},"last":0.56200000000000006}]},{"osd":5,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.437,"5min":0.47899999999999998,"15min":0.46899999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.28699999999999998},"max":{"1min":0.63900000000000001,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.441},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.496,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.72499999999999998,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.46100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.496,"15min":0.498},"min":{"1min":0.32600000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.63400000000000001,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.52300000000000002,"15min":0.52100000000000002},"min":{"1min":0.38200000000000001,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.71499999999999997,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.57499999999999996}]},{"osd":7,"last update":"Tue Nov 28 03:09:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.53800000000000003,"15min":0.54000000000000004},"min":{"1min":0.35899999999999999,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.81499999999999995,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.56100000000000005,"15min":0.56799999999999995},"min":{"1min":0.371,"5min":0.35399999999999998,"15min":0.32900000000000001},"max":{"1min":0.70499999999999996,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.59499999999999997}]}]},{"osd":5,"up_from":13,"seq":55834575054,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302716,"kb_used_data":1524,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94069124,"statfs":{"total":96636764160,"available":96326782976,"internally_reserved":0,"allocated":1560576,"data_stored":777144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52900000000000003,"15min":0.52800000000000002},"min":{"1min":0.34799999999999998,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.68400000000000005,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.52200000000000002,"15min":0.54400000000000004},"min":{"1min":0.33100000000000002,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.751,"5min":0.751,"15min":0.77900000000000003},"last":0.39000000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.54000000000000004,"15min":0.55900000000000005},"min":{"1min":0.34499999999999997,"5min":0.27000000000000002,"15min":0.25900000000000001},"max":{"1min":0.74099999999999999,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55300000000000005,"15min":0.57799999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.31900000000000001},"max":{"1min":0.72299999999999998,"5min":0.82499999999999996,"15min":0.82699999999999996},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.56299999999999994,"15min":0.58499999999999996},"min":{"1min":0.36899999999999999,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.746,"5min":0.79900000000000004,"15min":0.80500000000000005},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58299999999999996,"15min":0.59799999999999998},"min":{"1min":0.40000000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.77900000000000003,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.52400000000000002,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34599999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.57499999999999996,"15min":0.57899999999999996},"min":{"1min":0.35899999999999999,"5min":0.35299999999999998,"15min":0.33100000000000002},"max":{"1min":0.70099999999999996,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.47599999999999998,"15min":0.45300000000000001},"min":{"1min":0.307,"5min":0.28999999999999998,"15min":0.28699999999999998},"max":{"1min":0.79900000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.52500000000000002,"15min":0.53400000000000003},"min":{"1min":0.34999999999999998,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.71999999999999997,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.59399999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.48699999999999999,"15min":0.49299999999999999},"min":{"1min":0.28899999999999998,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.78200000000000003,"5min":0.79600000000000004,"15min":0.82899999999999996},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55400000000000005,"15min":0.55200000000000005},"min":{"1min":0.42699999999999999,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.80000000000000004,"5min":0.94399999999999995,"15min":0.94399999999999995},"last":0.52100000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:09:08 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.495,"15min":0.502},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.68000000000000005,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.53400000000000003,"15min":0.53600000000000003},"min":{"1min":0.32200000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.81799999999999995,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1550,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:09:55.467 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:09:55.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:09:55.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.620+0000 7f05a11d0700 1 Processor -- start 2023-11-28T03:09:55.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.620+0000 7f05a11d0700 1 -- start start 2023-11-28T03:09:55.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f05a11d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f059c0c6190 0x7f059c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f05a11d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f059c158320 0x7f059c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f05a11d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f059c158cc0 con 0x7f059c098d10 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f05a11d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c0c72d0 con 0x7f059c0c6190 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f05a11d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f059c064570 con 0x7f059c158320 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f059c0c6190 0x7f059c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f059c0c6190 0x7f059c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44968/0 (socket says 172.21.15.73:44968) 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 -- 172.21.15.73:0/1621773806 learned_addr learned my addr 172.21.15.73:0/1621773806 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059b59e700 1 --2- 172.21.15.73:0/1621773806 >> v2:172.21.15.150:3300/0 conn(0x7f059c158320 0x7f059c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 -- 172.21.15.73:0/1621773806 >> v2:172.21.15.150:3300/0 conn(0x7f059c158320 msgr2=0x7f059c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 --2- 172.21.15.73:0/1621773806 >> v2:172.21.15.150:3300/0 conn(0x7f059c158320 0x7f059c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 -- 172.21.15.73:0/1621773806 >> v1:172.21.15.150:6789/0 conn(0x7f059c098d10 legacy=0x7f059c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 -- 172.21.15.73:0/1621773806 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059c154bc0 con 0x7f059c0c6190 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f059a59c700 1 --2- 172.21.15.73:0/1621773806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c0c6190 0x7f059c0c7760 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f059c0657a0 tx=0x7f05900090f0).ready entity=mon.0 client_cookie=6962c2c380380b5b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f0599d9b700 1 -- 172.21.15.73:0/1621773806 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000b520 con 0x7f059c0c6190 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f0599d9b700 1 -- 172.21.15.73:0/1621773806 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059000b680 con 0x7f059c0c6190 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.621+0000 7f0599d9b700 1 -- 172.21.15.73:0/1621773806 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059000ba00 con 0x7f059c0c6190 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 -- 172.21.15.73:0/1621773806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c0c6190 msgr2=0x7f059c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 --2- 172.21.15.73:0/1621773806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c0c6190 0x7f059c0c7760 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 rx=0x7f059c0657a0 tx=0x7f05900090f0).stop 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 -- 172.21.15.73:0/1621773806 shutdown_connections 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 --2- 172.21.15.73:0/1621773806 >> v2:172.21.15.150:3300/0 conn(0x7f059c158320 0x7f059c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 --2- 172.21.15.73:0/1621773806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c0c6190 0x7f059c0c7760 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 -- 172.21.15.73:0/1621773806 >> 172.21.15.73:0/1621773806 conn(0x7f059c0cb310 msgr2=0x7f059c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 -- 172.21.15.73:0/1621773806 shutdown_connections 2023-11-28T03:09:55.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.622+0000 7f05a11d0700 1 -- 172.21.15.73:0/1621773806 wait complete. 2023-11-28T03:09:55.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.623+0000 7f05a11d0700 1 Processor -- start 2023-11-28T03:09:55.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.623+0000 7f05a11d0700 1 -- start start 2023-11-28T03:09:55.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f059c098d10 0x7f059c1578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f059c0c6190 0x7f059c151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c154d50 con 0x7f059c158320 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059c151910 con 0x7f059c0c6190 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f05a11d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059c168500 con 0x7f059c098d10 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44978/0 (socket says 172.21.15.73:44978) 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 -- 172.21.15.73:0/3718878088 learned_addr learned my addr 172.21.15.73:0/3718878088 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059a59c700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f059c0c6190 0x7f059c151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059ad9d700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f059c098d10 0x7f059c1578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 -- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f059c098d10 msgr2=0x7f059c1578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f059c098d10 0x7f059c1578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 -- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f059c0c6190 msgr2=0x7f059c151420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f059c0c6190 0x7f059c151420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.624+0000 7f059b59e700 1 -- 172.21.15.73:0/3718878088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590009370 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f059b59e700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f0594009df0 tx=0x7f059400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400caa0 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059400f040 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059c1687e0 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059400ce10 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.625+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059c168da0 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.626+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059c154ec0 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.626+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0594013610 con 0x7f059c158320 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.626+0000 7f0582ffd700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 0x7f0588061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:55.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.626+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f059407a8f0 con 0x7f059c158320 2023-11-28T03:09:55.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.626+0000 7f059ad9d700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 0x7f0588061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:55.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.627+0000 7f059ad9d700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 0x7f0588061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f058c003890 tx=0x7f058c000bc0).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:55.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.631+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f059404bbc0 con 0x7f059c158320 2023-11-28T03:09:55.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.866+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f059c06c860 con 0x7f059c158320 2023-11-28T03:09:55.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.870+0000 7f0582ffd700 1 -- 172.21.15.73:0/3718878088 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7f059404f1e0 con 0x7f059c158320 2023-11-28T03:09:55.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 msgr2=0x7f0588061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 0x7f0588061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f058c003890 tx=0x7f058c000bc0).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 msgr2=0x7f059c0c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 rx=0x7f0594009df0 tx=0x7f059400b040).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 shutdown_connections 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f058805f150 0x7f0588061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f059c158320 0x7f059c0c3ba0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f059c098d10 0x7f059c1578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 --2- 172.21.15.73:0/3718878088 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f059c0c6190 0x7f059c151420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 >> 172.21.15.73:0/3718878088 conn(0x7f059c0cb310 msgr2=0x7f059c0a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:55.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 shutdown_connections 2023-11-28T03:09:55.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.872+0000 7f05a11d0700 1 -- 172.21.15.73:0/3718878088 wait complete. 2023-11-28T03:09:55.875 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:09:55.885 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:09:55.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.997+0000 7f78ab215700 1 Processor -- start 2023-11-28T03:09:56.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.997+0000 7f78ab215700 1 -- start start 2023-11-28T03:09:56.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78ab215700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78a40c0180 0x7f78a4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78ab215700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40c12c0 con 0x7f78a4157660 2023-11-28T03:09:56.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78ab215700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78a40c06b0 con 0x7f78a40c1750 2023-11-28T03:09:56.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78ab215700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78a40c0990 con 0x7f78a40c0180 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9a12700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f78a40c0180 0x7f78a4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78aa213700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f78a4157660 0x7f78a41589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45668/0 (socket says 172.21.15.73:45668) 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78aa213700 1 -- 172.21.15.73:0/741038164 learned_addr learned my addr 172.21.15.73:0/741038164 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9a12700 1 -- 172.21.15.73:0/741038164 >> v1:172.21.15.150:6789/0 conn(0x7f78a40c1750 legacy=0x7f78a40648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9a12700 1 -- 172.21.15.73:0/741038164 >> v1:172.21.15.73:6789/0 conn(0x7f78a4157660 legacy=0x7f78a41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9a12700 1 -- 172.21.15.73:0/741038164 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78a40c0c70 con 0x7f78a40c0180 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9211700 1 -- 172.21.15.73:0/741038164 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531349490 0 0) 0x7f78a40c12c0 con 0x7f78a4157660 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.998+0000 7f78a9a12700 1 --2- 172.21.15.73:0/741038164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a40c0180 0x7f78a4157a40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f7894009760 tx=0x7f7894009060).ready entity=mon.0 client_cookie=10a9e25bbff2a0f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78a9211700 1 -- 172.21.15.73:0/741038164 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400b580 con 0x7f78a40c0180 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78a9211700 1 -- 172.21.15.73:0/741038164 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f789400b6e0 con 0x7f78a40c0180 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78a9211700 1 -- 172.21.15.73:0/741038164 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f789400ba60 con 0x7f78a40c0180 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 -- 172.21.15.73:0/741038164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a40c0180 msgr2=0x7f78a4157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 --2- 172.21.15.73:0/741038164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a40c0180 0x7f78a4157a40 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 rx=0x7f7894009760 tx=0x7f7894009060).stop 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 -- 172.21.15.73:0/741038164 shutdown_connections 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 --2- 172.21.15.73:0/741038164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a40c0180 0x7f78a4157a40 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 -- 172.21.15.73:0/741038164 >> 172.21.15.73:0/741038164 conn(0x7f78a40cb000 msgr2=0x7f78a40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 -- 172.21.15.73:0/741038164 shutdown_connections 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:55.999+0000 7f78ab215700 1 -- 172.21.15.73:0/741038164 wait complete. 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.000+0000 7f78ab215700 1 Processor -- start 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.000+0000 7f78ab215700 1 -- start start 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 0x7f78a40a9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 0x7f78a4153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40a9b00 con 0x7f78a4157660 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40b01d0 con 0x7f78a40c0180 2023-11-28T03:09:56.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78ab215700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40afba0 con 0x7f78a40c1750 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44986/0 (socket says 172.21.15.73:44986) 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 -- 172.21.15.73:0/2694367347 learned_addr learned my addr 172.21.15.73:0/2694367347 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78a9a12700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 0x7f78a4153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aa213700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 0x7f78a40a9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 -- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 msgr2=0x7f78a4153ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 0x7f78a4153ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 -- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 msgr2=0x7f78a40a9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 0x7f78a40a9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aaa14700 1 -- 172.21.15.73:0/2694367347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940092a0 con 0x7f78a4157660 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78a9a12700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 0x7f78a4153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.001+0000 7f78aa213700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 0x7f78a40a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:56.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f78aaa14700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f78a000c440 tx=0x7f78a0007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a000f470 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78a000f5d0 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a40aeca0 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a000f950 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.002+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a4153fa0 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.003+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a4154220 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.003+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f78a000fd40 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.003+0000 7f789a7fc700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 0x7f7888061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.003+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f78a0079670 con 0x7f78a4157660 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.003+0000 7f78aa213700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 0x7f7888061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.004+0000 7f78aa213700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 0x7f7888061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f789c007fd0 tx=0x7f789c009040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.007+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f78a0048af0 con 0x7f78a4157660 2023-11-28T03:09:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.212+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f78a406c7c0 con 0x7f78a4157660 2023-11-28T03:09:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.213+0000 7f789a7fc700 1 -- 172.21.15.73:0/2694367347 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f78a004a9f0 con 0x7f78a4157660 2023-11-28T03:09:56.214 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:09:56.214 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 msgr2=0x7f7888061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 0x7f7888061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f789c007fd0 tx=0x7f789c009040).stop 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 msgr2=0x7f78a40b0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 rx=0x7f78a000c440 tx=0x7f78a0007db0).stop 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 shutdown_connections 2023-11-28T03:09:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f788805f150 0x7f7888061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f78a4157660 0x7f78a40b0660 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f78a40c0180 0x7f78a40a9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 --2- 172.21.15.73:0/2694367347 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f78a40c1750 0x7f78a4153ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 >> 172.21.15.73:0/2694367347 conn(0x7f78a40cb000 msgr2=0x7f78a4152c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 shutdown_connections 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.215+0000 7f78ab215700 1 -- 172.21.15.73:0/2694367347 wait complete. 2023-11-28T03:09:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:09:56.630 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:09:47.742412+0000', 'modified': '2023-11-28T03:09:49.748867+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7596}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7596': {'gid': 7596, 'name': 'c', 'rank': 0, 'incarnation': 44, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/966840583', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 966840583}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 966840583}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [16], 'metadata_pool': 15, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:09:56.631 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:09:56.631 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-11-28T03:09:56.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.745+0000 7f2ee6d72700 1 Processor -- start 2023-11-28T03:09:56.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.745+0000 7f2ee6d72700 1 -- start start 2023-11-28T03:09:56.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee6d72700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ee014e530 0x7f2ee00c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee6d72700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 0x7f2ee014e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee6d72700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ee00c7bc0 con 0x7f2ee0158570 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee6d72700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee0155920 con 0x7f2ee014e530 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee6d72700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee0154e00 con 0x7f2ee0154950 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ee014e530 0x7f2ee00c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ee014e530 0x7f2ee00c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:44996/0 (socket says 172.21.15.73:44996) 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1345262694 learned_addr learned my addr 172.21.15.73:0/1345262694 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1345262694 >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 0x7f2ee014e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1345262694 >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 msgr2=0x7f2ee014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1345262694 >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 0x7f2ee014e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1345262694 >> v1:172.21.15.150:6789/0 conn(0x7f2ee0158570 legacy=0x7f2ee00aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1345262694 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee0155080 con 0x7f2ee014e530 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1345262694 >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 0x7f2ee014e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.746+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1345262694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee014e530 0x7f2ee00c66f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f2ee0157f20 tx=0x7f2ed80090f0).ready entity=mon.0 client_cookie=433f6e86e5e611f4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee4d6e700 1 -- 172.21.15.73:0/1345262694 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800b600 con 0x7f2ee014e530 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee4d6e700 1 -- 172.21.15.73:0/1345262694 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed800b760 con 0x7f2ee014e530 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee4d6e700 1 -- 172.21.15.73:0/1345262694 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800bae0 con 0x7f2ee014e530 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 -- 172.21.15.73:0/1345262694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee014e530 msgr2=0x7f2ee00c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/1345262694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee014e530 0x7f2ee00c66f0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 rx=0x7f2ee0157f20 tx=0x7f2ed80090f0).stop 2023-11-28T03:09:56.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 -- 172.21.15.73:0/1345262694 shutdown_connections 2023-11-28T03:09:56.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/1345262694 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee014e530 0x7f2ee00c66f0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/1345262694 >> v2:172.21.15.150:3300/0 conn(0x7f2ee0154950 0x7f2ee014e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 -- 172.21.15.73:0/1345262694 >> 172.21.15.73:0/1345262694 conn(0x7f2ee0064990 msgr2=0x7f2ee0064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 -- 172.21.15.73:0/1345262694 shutdown_connections 2023-11-28T03:09:56.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.747+0000 7f2ee6d72700 1 -- 172.21.15.73:0/1345262694 wait complete. 2023-11-28T03:09:56.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.748+0000 7f2ee6d72700 1 Processor -- start 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.748+0000 7f2ee6d72700 1 -- start start 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ee014e530 0x7f2ee01684e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0158570 0x7f2ee00c05a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee0165dc0 con 0x7f2ee0154950 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee01689d0 con 0x7f2ee0158570 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6d72700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee014ec80 con 0x7f2ee014e530 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45010/0 (socket says 172.21.15.73:45010) 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 -- 172.21.15.73:0/3824185570 learned_addr learned my addr 172.21.15.73:0/3824185570 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee6571700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0158570 0x7f2ee00c05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ee014e530 0x7f2ee01684e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 -- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ee014e530 msgr2=0x7f2ee01684e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ee014e530 0x7f2ee01684e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 -- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0158570 msgr2=0x7f2ee00c05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0158570 0x7f2ee00c05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.749+0000 7f2ee556f700 1 -- 172.21.15.73:0/3824185570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed8009370 con 0x7f2ee0154950 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f2ed400d1d0 tx=0x7f2ed4009a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2eddffb700 1 -- 172.21.15.73:0/3824185570 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed400de70 con 0x7f2ee0154950 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2eddffb700 1 -- 172.21.15.73:0/3824185570 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed400f040 con 0x7f2ee0154950 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee014ef60 con 0x7f2ee0154950 2023-11-28T03:09:56.755 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2eddffb700 1 -- 172.21.15.73:0/3824185570 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4013600 con 0x7f2ee0154950 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.750+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee0165a50 con 0x7f2ee0154950 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.751+0000 7f2eddffb700 1 -- 172.21.15.73:0/3824185570 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2ed40139f0 con 0x7f2ee0154950 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.751+0000 7f2eddffb700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 0x7f2ecc061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.751+0000 7f2eddffb700 1 -- 172.21.15.73:0/3824185570 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2ed407a940 con 0x7f2ee0154950 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.752+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 0x7f2ecc061660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.752+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 0x7f2ecc061660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2ed8009130 tx=0x7f2ed8017000).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.754+0000 7f2ef996cb80 1 Processor -- start 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.754+0000 7f2ef996cb80 1 -- start start 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.754+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.754+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e49725d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.754+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e496eb060 con 0x557e4974dcf0 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e496621f0 con 0x557e4976e4e0 2023-11-28T03:09:56.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e49677440 con 0x557e494e1e40 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45012/0 (socket says 172.21.15.73:45012) 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 -- 172.21.15.73:0/277085005 learned_addr learned my addr 172.21.15.73:0/277085005 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee556f700 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e49725d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 -- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 msgr2=0x557e49725880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 -- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 msgr2=0x557e49725d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e49725d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 -- 172.21.15.73:0/277085005 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e49670620 con 0x557e4974dcf0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ee6571700 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f2ed00034c0 tx=0x7f2ed0009040).ready entity=mon.0 client_cookie=cd32b35ec27497c9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ec6ffd700 1 -- 172.21.15.73:0/277085005 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0010450 con 0x557e4974dcf0 2023-11-28T03:09:56.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ec6ffd700 1 -- 172.21.15.73:0/277085005 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed00105b0 con 0x557e4974dcf0 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.755+0000 7f2ec6ffd700 1 -- 172.21.15.73:0/277085005 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed0010930 con 0x557e4974dcf0 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- 172.21.15.73:0/277085005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 msgr2=0x557e497207e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 rx=0x7f2ed00034c0 tx=0x7f2ed0009040).stop 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- 172.21.15.73:0/277085005 shutdown_connections 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e497207e0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/277085005 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e49725d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- 172.21.15.73:0/277085005 >> 172.21.15.73:0/277085005 conn(0x557e4958f4a0 msgr2=0x557e4976e970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- 172.21.15.73:0/277085005 shutdown_connections 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- 172.21.15.73:0/277085005 wait complete. 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 -- start start 2023-11-28T03:09:56.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.756+0000 7f2ef996cb80 1 Processor -- start 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e494e1e40 0x557e497472d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4976dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e49677580 con 0x557e4974dcf0 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e49677800 con 0x557e494e1e40 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e49642e10 con 0x557e4976e4e0 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ef996cb80 10 client.0 ll_register_callbacks cb 0x557e496e5640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.757+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45026/0 (socket says 172.21.15.73:45026) 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 -- 172.21.15.73:0/3555305652 learned_addr learned my addr 172.21.15.73:0/3555305652 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee6571700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4976dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e494e1e40 0x557e497472d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 -- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 msgr2=0x557e4976dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4976dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 -- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e494e1e40 msgr2=0x557e497472d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e494e1e40 0x557e497472d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 -- 172.21.15.73:0/3555305652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ed000f040 con 0x557e4974dcf0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ee556f700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f2ed4078f70 tx=0x7f2ed401e040).ready entity=mon.0 client_cookie=e6e5daab5d5a7568 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4013ca0 con 0x557e4974dcf0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2ed4013e00 con 0x557e4974dcf0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e49719720 con 0x557e4974dcf0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed4011500 con 0x557e4974dcf0 2023-11-28T03:09:56.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 10 client.7782 fetch_fsmap learned FSMap version 46 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x557e4966b9c0 con 0x557e4974dcf0 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e49719720 con 0x557e4974dcf0 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2ed404ee00 con 0x557e4974dcf0 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 10 client.7782 fetch_fsmap finished waiting for FSMap version 46 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 10 client.7782 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 20 client.7782 populate_metadata read hostname 'smithi073' 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4974b870 0x557e49720af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 4 client.7782 mds_command: new command op to 7596 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.758+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 --> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557e494ddca0 con 0x557e4974b870 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4974b870 0x557e49720af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ebffff700 10 client.7782 ms_handle_connect on v2:172.21.15.73:6835/966840583 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4974b870 0x557e49720af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=219b11bb10e676d3 server_cookie=6cbc204032e3ca47 in_seq=0 out_seq=0 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ebffff700 1 -- 172.21.15.73:0/3555305652 <== mds.0 v2:172.21.15.73:6835/966840583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x557e494ddca0 con 0x557e4974b870 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ebffff700 10 client.7782 handle_command_reply: tid=0 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ef996cb80 1 client.7782 shutdown 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 msgr2=0x557e4976d730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.759+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 rx=0x7f2ed4078f70 tx=0x7f2ed401e040).stop 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 shutdown_connections 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4974b870 0x557e49720af0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4976dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ebffff700 0 client.7782 ms_handle_reset on v2:172.21.15.73:6835/966840583 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e494e1e40 0x557e497472d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/3555305652 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e4976d730 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 >> 172.21.15.73:0/3555305652 conn(0x557e4958f4a0 msgr2=0x557e49725880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 shutdown_connections 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 1 -- 172.21.15.73:0/3555305652 wait complete. 2023-11-28T03:09:56.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.760+0000 7f2ef996cb80 20 client.7782 trim_cache size 0 max 16384 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.777+0000 7f2ef996cb80 1 Processor -- start 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.777+0000 7f2ef996cb80 1 -- start start 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e497501e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e49768340 con 0x557e4974dcf0 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4976bcc0 con 0x557e4976e4e0 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e4971cf10 con 0x557e494e1e40 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45038/0 (socket says 172.21.15.73:45038) 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 -- 172.21.15.73:0/1379845055 learned_addr learned my addr 172.21.15.73:0/1379845055 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee6571700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e497501e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 -- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 msgr2=0x557e49725880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 -- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 msgr2=0x557e497501e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e497501e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 -- 172.21.15.73:0/1379845055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557e4966b710 con 0x557e4974dcf0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f2ed404dc60 tx=0x7f2ed4057000).ready entity=mon.0 client_cookie=7ecc2907204103 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ec77fe700 1 -- 172.21.15.73:0/1379845055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed4011500 con 0x557e4974dcf0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ec77fe700 1 -- 172.21.15.73:0/1379845055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed404f8b0 con 0x557e4974dcf0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.778+0000 7f2ec77fe700 1 -- 172.21.15.73:0/1379845055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed404ee00 con 0x557e4974dcf0 2023-11-28T03:09:56.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1379845055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 msgr2=0x557e49725d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 rx=0x7f2ed404dc60 tx=0x7f2ed4057000).stop 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1379845055 shutdown_connections 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4976e4e0 0x557e497501e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e494e1e40 0x557e49725880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1379845055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e4974dcf0 0x557e49725d70 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1379845055 >> 172.21.15.73:0/1379845055 conn(0x557e4958f4a0 msgr2=0x557e49721c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1379845055 shutdown_connections 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.779+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1379845055 wait complete. 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.780+0000 7f2ef996cb80 1 -- start start 2023-11-28T03:09:56.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.780+0000 7f2ef996cb80 1 Processor -- start 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4974dcf0 0x557e4971f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4971fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e4974c670 con 0x557e494e1e40 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x557e497201d0 con 0x557e4974dcf0 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x557e497510d0 con 0x557e4976e4e0 2023-11-28T03:09:56.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 10 client.0 ll_register_callbacks cb 0x557e496e5640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45042/0 (socket says 172.21.15.73:45042) 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1543222966 learned_addr learned my addr 172.21.15.73:0/1543222966 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4974dcf0 0x557e4971f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee6571700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4971fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 msgr2=0x557e4971fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4971fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4974dcf0 msgr2=0x557e4971f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4974dcf0 0x557e4971f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 -- 172.21.15.73:0/1543222966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x557e4966b9c0 con 0x557e494e1e40 2023-11-28T03:09:56.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ee5d70700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f2ee0064480 tx=0x7f2ed802f280).ready entity=mon.0 client_cookie=5003ab7401cd4434 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ebffff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed800bb10 con 0x557e494e1e40 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2ed8028420 con 0x557e494e1e40 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557e49514ca0 con 0x557e494e1e40 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2ed80287f0 con 0x557e494e1e40 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 10 client.7788 fetch_fsmap learned FSMap version 46 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.781+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x557e494e2f80 con 0x557e494e1e40 2023-11-28T03:09:56.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557e49514ca0 con 0x557e494e1e40 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 46) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f2ed8028950 con 0x557e494e1e40 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 10 client.7788 fetch_fsmap finished waiting for FSMap version 46 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 10 client.7788 resolve_mds: resolved name 'c' to daemon mds.c 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 20 client.7788 populate_metadata read hostname 'smithi073' 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4978a2d0 0x557e4978a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 4 client.7788 mds_command: new command op to 7596 tid=0[{"prefix": "status"}] 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 --> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557e49543540 con 0x557e4978a2d0 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4978a2d0 0x557e4978a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ebffff700 10 client.7788 ms_handle_connect on v2:172.21.15.73:6835/966840583 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ee556f700 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4978a2d0 0x557e4978a6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=b6e092be4804af58 server_cookie=e2d6a655692d4e62 in_seq=0 out_seq=0 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ebffff700 1 -- 172.21.15.73:0/1543222966 <== mds.0 v2:172.21.15.73:6835/966840583 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x557e49543540 con 0x557e4978a2d0 2023-11-28T03:09:56.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ebffff700 10 client.7788 handle_command_reply: tid=0 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 1 client.7788 shutdown 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 msgr2=0x557e497cfd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.782+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 rx=0x7f2ee0064480 tx=0x7f2ed802f280).stop 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 shutdown_connections 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:6835/966840583,v1:172.21.15.73:6837/966840583] conn(0x557e4978a2d0 0x557e4978a6b0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x557e4976e4e0 0x557e4971fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x557e494e1e40 0x557e497cfd70 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 --2- 172.21.15.73:0/1543222966 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x557e4974dcf0 0x557e4971f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ebffff700 0 client.7788 ms_handle_reset on v2:172.21.15.73:6835/966840583 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 >> 172.21.15.73:0/1543222966 conn(0x557e4958f4a0 msgr2=0x557e49721ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 shutdown_connections 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 1 -- 172.21.15.73:0/1543222966 wait complete. 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ef996cb80 20 client.7788 trim_cache size 0 max 16384 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:09:56.793 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 7596, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 9.0395471779999994, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 46, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 126, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 126, 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 13.561857420000001 2023-11-28T03:09:56.794 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:09:56.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 msgr2=0x7f2ecc061660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 0x7f2ecc061660 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f2ed8009130 tx=0x7f2ed8017000).stop 2023-11-28T03:09:56.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 msgr2=0x7f2ee0153c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 rx=0x7f2ed400d1d0 tx=0x7f2ed4009a90).stop 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 shutdown_connections 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ecc05f1a0 0x7f2ecc061660 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ee0158570 0x7f2ee00c05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ee014e530 0x7f2ee01684e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 --2- 172.21.15.73:0/3824185570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ee0154950 0x7f2ee0153c30 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 >> 172.21.15.73:0/3824185570 conn(0x7f2ee0064990 msgr2=0x7f2ee00ac580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 shutdown_connections 2023-11-28T03:09:56.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.783+0000 7f2ee6d72700 1 -- 172.21.15.73:0/3824185570 wait complete. 2023-11-28T03:09:56.797 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:09:56.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.896+0000 7f60acb4e700 1 Processor -- start 2023-11-28T03:09:56.898 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.896+0000 7f60acb4e700 1 -- start start 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60acb4e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60a8154700 0x7f60a80c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60acb4e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a80c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60acb4e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60a80c6d80 con 0x7f60a80c7700 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60acb4e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60a80c85d0 con 0x7f60a8154700 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60acb4e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8154ae0 con 0x7f60a80c7c50 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60a659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60a8154700 0x7f60a80c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f609dd9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f60a80c7700 0x7f60a80aa1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45670/0 (socket says 172.21.15.73:45670) 2023-11-28T03:09:56.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60a659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f60a8154700 0x7f60a80c7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45050/0 (socket says 172.21.15.73:45050) 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f609dd9b700 1 -- 172.21.15.73:0/1443061468 learned_addr learned my addr 172.21.15.73:0/1443061468 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.897+0000 7f60a6d9d700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a80c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a659c700 1 -- 172.21.15.73:0/1443061468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 msgr2=0x7f60a80c80e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a659c700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a80c80e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a659c700 1 -- 172.21.15.73:0/1443061468 >> v1:172.21.15.73:6789/0 conn(0x7f60a80c7700 legacy=0x7f60a80aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a5d9b700 1 -- 172.21.15.73:0/1443061468 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3069182463 0 0) 0x7f60a80c6d80 con 0x7f60a80c7700 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a659c700 1 -- 172.21.15.73:0/1443061468 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a8064c30 con 0x7f60a8154700 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a6d9d700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a80c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a659c700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a8154700 0x7f60a80c7210 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f6098009760 tx=0x7f6098009060).ready entity=mon.0 client_cookie=2589ed53ee514016 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a5d9b700 1 -- 172.21.15.73:0/1443061468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800b580 con 0x7f60a8154700 2023-11-28T03:09:56.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a5d9b700 1 -- 172.21.15.73:0/1443061468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f609800b6e0 con 0x7f60a8154700 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60a5d9b700 1 -- 172.21.15.73:0/1443061468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800ba60 con 0x7f60a8154700 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 -- 172.21.15.73:0/1443061468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a8154700 msgr2=0x7f60a80c7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a8154700 0x7f60a80c7210 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 rx=0x7f6098009760 tx=0x7f6098009060).stop 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 -- 172.21.15.73:0/1443061468 shutdown_connections 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a80c80e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 --2- 172.21.15.73:0/1443061468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a8154700 0x7f60a80c7210 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 -- 172.21.15.73:0/1443061468 >> 172.21.15.73:0/1443061468 conn(0x7f60a80948b0 msgr2=0x7f60a806a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 -- 172.21.15.73:0/1443061468 shutdown_connections 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.898+0000 7f60acb4e700 1 -- 172.21.15.73:0/1443061468 wait complete. 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 Processor -- start 2023-11-28T03:09:56.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 -- start start 2023-11-28T03:09:56.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a8141e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a8154700 0x7f60a8150970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a8151fd0 con 0x7f60a80c7700 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a81513c0 con 0x7f60a8154700 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60acb4e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a81516a0 con 0x7f60a80c7c50 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.900+0000 7f60a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45052/0 (socket says 172.21.15.73:45052) 2023-11-28T03:09:56.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 -- 172.21.15.73:0/2595366413 learned_addr learned my addr 172.21.15.73:0/2595366413 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 -- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 msgr2=0x7f60a8141e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a8141e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 -- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a8154700 msgr2=0x7f60a8150970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a8154700 0x7f60a8150970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 -- 172.21.15.73:0/2595366413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60980092a0 con 0x7f60a80c7700 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60a659c700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f60a8062d60 tx=0x7f6098017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609800ba60 con 0x7f60a80c7700 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6098000c90 con 0x7f60a80c7700 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60a8151980 con 0x7f60a80c7700 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f609801b640 con 0x7f60a80c7700 2023-11-28T03:09:56.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.901+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60a8142310 con 0x7f60a80c7700 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.902+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60a80475e0 con 0x7f60a80c7700 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.902+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f609801ba30 con 0x7f60a80c7700 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.903+0000 7f609effd700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 0x7f6088061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.903+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f609807b200 con 0x7f60a80c7700 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.903+0000 7f609dd9b700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 0x7f6088061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.903+0000 7f609dd9b700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 0x7f6088061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f60900011e0 tx=0x7f6090007850).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:56.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:56.907+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f609804c4d0 con 0x7f60a80c7700 2023-11-28T03:09:57.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.111+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f60a80c6450 con 0x7f60a80c7700 2023-11-28T03:09:57.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.112+0000 7f609effd700 1 -- 172.21.15.73:0/2595366413 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6098021020 con 0x7f60a80c7700 2023-11-28T03:09:57.113 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 msgr2=0x7f6088061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 0x7f6088061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f60900011e0 tx=0x7f6090007850).stop 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 msgr2=0x7f60a8152460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 rx=0x7f60a8062d60 tx=0x7f6098017000).stop 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 shutdown_connections 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f608805f150 0x7f6088061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a8154700 0x7f60a8150970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:57.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a80c7700 0x7f60a8152460 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 --2- 172.21.15.73:0/2595366413 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a80c7c50 0x7f60a8141e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:57.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 >> 172.21.15.73:0/2595366413 conn(0x7f60a80948b0 msgr2=0x7f60a80640c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:57.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 shutdown_connections 2023-11-28T03:09:57.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.114+0000 7f60acb4e700 1 -- 172.21.15.73:0/2595366413 wait complete. 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:09:57.126 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:09:57.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.127 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:09:57.142 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41483sec preferred_lft 41483sec 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:09:57.143 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:09:57.144 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:09:57.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.145 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:09:57.204 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:09:57.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.206 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:09:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:09:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:09:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:09:57.261 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:09:57.262 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:09:57.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:09:57.262 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:09:57.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.361 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:09:57.375 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:09:57.376 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:09:57.376 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:09:57.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.376 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:09:57.440 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.440 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:09:57.496 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:09:57.496 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:09:57.574 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:09:57.575 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:09:57.600 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:09:57.601 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:09:57.603 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:09:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:09:57.603 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:09:57.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:09:57.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.822+0000 7f3a367ed700 1 Processor -- start 2023-11-28T03:09:57.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.822+0000 7f3a367ed700 1 -- start start 2023-11-28T03:09:57.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a367ed700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3a3014e7e0 0x7f3a300bccd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a367ed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a367ed700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a30063060 con 0x7f3a30062bf0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a367ed700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a3006a3f0 con 0x7f3a3014e7e0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a367ed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a30069560 con 0x7f3a3014ebc0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a34fea700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a34fea700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:59984/0 (socket says 172.21.15.73:59984) 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a34fea700 1 -- 172.21.15.73:0/2806946436 learned_addr learned my addr 172.21.15.73:0/2806946436 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.823+0000 7f3a357eb700 1 --2- 172.21.15.73:0/2806946436 >> v2:172.21.15.150:3300/0 conn(0x7f3a3014e7e0 0x7f3a300bccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a27fff700 1 -- 172.21.15.73:0/2806946436 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 707647584 0 0) 0x7f3a30063060 con 0x7f3a30062bf0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a34fea700 1 -- 172.21.15.73:0/2806946436 >> v2:172.21.15.150:3300/0 conn(0x7f3a3014e7e0 msgr2=0x7f3a300bccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a34fea700 1 --2- 172.21.15.73:0/2806946436 >> v2:172.21.15.150:3300/0 conn(0x7f3a3014e7e0 0x7f3a300bccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a34fea700 1 -- 172.21.15.73:0/2806946436 >> v1:172.21.15.150:6789/0 conn(0x7f3a30062bf0 legacy=0x7f3a300c59f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a34fea700 1 -- 172.21.15.73:0/2806946436 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a30069770 con 0x7f3a3014ebc0 2023-11-28T03:09:57.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a34fea700 1 --2- 172.21.15.73:0/2806946436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3a20006fd0 tx=0x7f3a200090f0).ready entity=mon.2 client_cookie=cff05a3419a614d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a27fff700 1 -- 172.21.15.73:0/2806946436 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000b580 con 0x7f3a3014ebc0 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a27fff700 1 -- 172.21.15.73:0/2806946436 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b6e0 con 0x7f3a3014ebc0 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a27fff700 1 -- 172.21.15.73:0/2806946436 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000ba60 con 0x7f3a3014ebc0 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 -- 172.21.15.73:0/2806946436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 msgr2=0x7f3a30062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2806946436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 rx=0x7f3a20006fd0 tx=0x7f3a200090f0).stop 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 -- 172.21.15.73:0/2806946436 shutdown_connections 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2806946436 >> v2:172.21.15.150:3300/0 conn(0x7f3a3014e7e0 0x7f3a300bccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2806946436 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a3014ebc0 0x7f3a30062700 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.824+0000 7f3a367ed700 1 -- 172.21.15.73:0/2806946436 >> 172.21.15.73:0/2806946436 conn(0x7f3a30064320 msgr2=0x7f3a300c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.825+0000 7f3a367ed700 1 -- 172.21.15.73:0/2806946436 shutdown_connections 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.825+0000 7f3a367ed700 1 -- 172.21.15.73:0/2806946436 wait complete. 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.826+0000 7f3a367ed700 1 Processor -- start 2023-11-28T03:09:57.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.826+0000 7f3a367ed700 1 -- start start 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a30062bf0 0x7f3a30167380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3014ebc0 0x7f3a300a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300a6290 con 0x7f3a3014e7e0 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300aa690 con 0x7f3a3014ebc0 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a300a9dd0 con 0x7f3a30062bf0 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45058/0 (socket says 172.21.15.73:45058) 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 -- 172.21.15.73:0/2438594232 learned_addr learned my addr 172.21.15.73:0/2438594232 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a357eb700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a30062bf0 0x7f3a30167380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a35fec700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3014ebc0 0x7f3a300a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 -- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a30062bf0 msgr2=0x7f3a30167380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a30062bf0 0x7f3a30167380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 -- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3014ebc0 msgr2=0x7f3a300a5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3014ebc0 0x7f3a300a5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 -- 172.21.15.73:0/2438594232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a20009370 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a34fea700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f3a20009ea0 tx=0x7f3a20017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2000b850 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a2000b9b0 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a300aa040 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a2001c670 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.827+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a30167870 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.828+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a3006c7c0 con 0x7f3a3014e7e0 2023-11-28T03:09:57.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.829+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3a2001c9e0 con 0x7f3a3014e7e0 2023-11-28T03:09:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.829+0000 7f3a257fa700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 0x7f3a14061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.829+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3a2007b8e0 con 0x7f3a3014e7e0 2023-11-28T03:09:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.829+0000 7f3a357eb700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 0x7f3a14061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.829+0000 7f3a357eb700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 0x7f3a14061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a28009fd0 tx=0x7f3a2800b040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:57.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:57.833+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3a2004cbb0 con 0x7f3a3014e7e0 2023-11-28T03:09:58.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.022+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3a30069a50 con 0x7f3a1405f150 2023-11-28T03:09:58.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.023+0000 7f3a257fa700 1 -- 172.21.15.73:0/2438594232 <== mgr.7416 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f3a30069a50 con 0x7f3a1405f150 2023-11-28T03:09:58.024 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:09:58.025 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:09:58.025 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:09:58.025 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:09:58.025 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:09:58.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 msgr2=0x7f3a14061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:58.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 0x7f3a14061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f3a28009fd0 tx=0x7f3a2800b040).stop 2023-11-28T03:09:58.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 msgr2=0x7f3a300a58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 rx=0x7f3a20009ea0 tx=0x7f3a20017000).stop 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 shutdown_connections 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3a1405f150 0x7f3a14061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3a3014ebc0 0x7f3a300a5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3a30062bf0 0x7f3a30167380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 --2- 172.21.15.73:0/2438594232 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3a3014e7e0 0x7f3a300a58b0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 >> 172.21.15.73:0/2438594232 conn(0x7f3a30064320 msgr2=0x7f3a300c3a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 shutdown_connections 2023-11-28T03:09:58.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.025+0000 7f3a367ed700 1 -- 172.21.15.73:0/2438594232 wait complete. 2023-11-28T03:09:58.039 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:09:58.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.152+0000 7f5bd7f93700 1 Processor -- start 2023-11-28T03:09:58.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.152+0000 7f5bd7f93700 1 -- start start 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd7f93700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5bd0098d10 0x7f5bd00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd7f93700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5bd0158320 0x7f5bd01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd7f93700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bd0158cc0 con 0x7f5bd00c6190 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd7f93700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd00c72d0 con 0x7f5bd0098d10 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd7f93700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bd0064570 con 0x7f5bd0158320 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd5d2f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5bd0098d10 0x7f5bd00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd5d2f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5bd0098d10 0x7f5bd00c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45070/0 (socket says 172.21.15.73:45070) 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.153+0000 7f5bd5d2f700 1 -- 172.21.15.73:0/1491720377 learned_addr learned my addr 172.21.15.73:0/1491720377 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:58.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd6530700 1 --2- 172.21.15.73:0/1491720377 >> v2:172.21.15.150:3300/0 conn(0x7f5bd0158320 0x7f5bd01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd5d2f700 1 -- 172.21.15.73:0/1491720377 >> v2:172.21.15.150:3300/0 conn(0x7f5bd0158320 msgr2=0x7f5bd01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd5d2f700 1 --2- 172.21.15.73:0/1491720377 >> v2:172.21.15.150:3300/0 conn(0x7f5bd0158320 0x7f5bd01587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd5d2f700 1 -- 172.21.15.73:0/1491720377 >> v1:172.21.15.150:6789/0 conn(0x7f5bd00c6190 legacy=0x7f5bd0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd5d2f700 1 -- 172.21.15.73:0/1491720377 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd0154bc0 con 0x7f5bd0098d10 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd5d2f700 1 --2- 172.21.15.73:0/1491720377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0098d10 0x7f5bd00c7760 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f5bd00657a0 tx=0x7f5bc8009840).ready entity=mon.0 client_cookie=490ec57fad59fc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd4d2d700 1 -- 172.21.15.73:0/1491720377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800e070 con 0x7f5bd0098d10 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd4d2d700 1 -- 172.21.15.73:0/1491720377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc800ba50 con 0x7f5bd0098d10 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd4d2d700 1 -- 172.21.15.73:0/1491720377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc800bdd0 con 0x7f5bd0098d10 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 -- 172.21.15.73:0/1491720377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0098d10 msgr2=0x7f5bd00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:58.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/1491720377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0098d10 0x7f5bd00c7760 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 rx=0x7f5bd00657a0 tx=0x7f5bc8009840).stop 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 -- 172.21.15.73:0/1491720377 shutdown_connections 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/1491720377 >> v2:172.21.15.150:3300/0 conn(0x7f5bd0158320 0x7f5bd01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/1491720377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0098d10 0x7f5bd00c7760 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.154+0000 7f5bd7f93700 1 -- 172.21.15.73:0/1491720377 >> 172.21.15.73:0/1491720377 conn(0x7f5bd00cb310 msgr2=0x7f5bd00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.155+0000 7f5bd7f93700 1 -- 172.21.15.73:0/1491720377 shutdown_connections 2023-11-28T03:09:58.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.155+0000 7f5bd7f93700 1 -- 172.21.15.73:0/1491720377 wait complete. 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 Processor -- start 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 -- start start 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd0098d10 0x7f5bd0146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bd00c6190 0x7f5bd00c4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00c54a0 con 0x7f5bd0158320 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd00ab680 con 0x7f5bd0098d10 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd7f93700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bd0146530 con 0x7f5bd00c6190 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.156+0000 7f5bd6530700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45076/0 (socket says 172.21.15.73:45076) 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 -- 172.21.15.73:0/4118996474 learned_addr learned my addr 172.21.15.73:0/4118996474 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd5d2f700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd0098d10 0x7f5bd0146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd552e700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bd00c6190 0x7f5bd00c4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 -- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bd00c6190 msgr2=0x7f5bd00c4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bd00c6190 0x7f5bd00c4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 -- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd0098d10 msgr2=0x7f5bd0146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd0098d10 0x7f5bd0146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 -- 172.21.15.73:0/4118996474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc8009ae0 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd6530700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f5bc000bfd0 tx=0x7f5bc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0011590 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc00116f0 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc0011a70 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bd00c51c0 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.157+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd00ab310 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.158+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 47) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5bc0012420 con 0x7f5bd0158320 2023-11-28T03:09:58.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.158+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bd00aae90 con 0x7f5bd0158320 2023-11-28T03:09:58.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.159+0000 7f5bcdffb700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 0x7f5bbc0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:58.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.159+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5bc0079a80 con 0x7f5bd0158320 2023-11-28T03:09:58.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.159+0000 7f5bd5d2f700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 0x7f5bbc0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:58.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.159+0000 7f5bd5d2f700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 0x7f5bbc0615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f5bc8009880 tx=0x7f5bd00a8040).ready entity=mgr.7416 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:58.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.163+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5bd00aae90 con 0x7f5bd0158320 2023-11-28T03:09:58.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:58.373+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f5bd00aae90 con 0x7f5bd0158320 2023-11-28T03:09:59.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.302+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 48) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f5bc0018d20 con 0x7f5bd0158320 2023-11-28T03:09:59.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.303+0000 7f5bcdffb700 1 -- 172.21.15.73:0/4118996474 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5bc00735b0 con 0x7f5bd0158320 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 msgr2=0x7f5bbc0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 0x7f5bbc0615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f5bc8009880 tx=0x7f5bd00a8040).stop 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 msgr2=0x7f5bd0153b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 rx=0x7f5bc000bfd0 tx=0x7f5bc0008040).stop 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 shutdown_connections 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5bbc05f120 0x7f5bbc0615e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5bd0158320 0x7f5bd0153b50 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5bd0098d10 0x7f5bd0146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 --2- 172.21.15.73:0/4118996474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5bd00c6190 0x7f5bd00c4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 >> 172.21.15.73:0/4118996474 conn(0x7f5bd00cb310 msgr2=0x7f5bd00ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:59.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 shutdown_connections 2023-11-28T03:09:59.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.306+0000 7f5bd7f93700 1 -- 172.21.15.73:0/4118996474 wait complete. 2023-11-28T03:09:59.326 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:09:59.445 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:59.444+0000 7f291d2d9680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:09:59.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.450+0000 7fc2bc154700 1 Processor -- start 2023-11-28T03:09:59.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.450+0000 7fc2bc154700 1 -- start start 2023-11-28T03:09:59.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2bc154700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2b4098d10 0x7fc2b40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2bc154700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 0x7fc2b4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2bc154700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2b40c72d0 con 0x7fc2b4064350 2023-11-28T03:09:59.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2bc154700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2b4154bc0 con 0x7fc2b4098d10 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2bc154700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40c66c0 con 0x7fc2b40c6190 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2b4098d10 0x7fc2b40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2b4098d10 0x7fc2b40c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45094/0 (socket says 172.21.15.73:45094) 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 -- 172.21.15.73:0/4254252901 learned_addr learned my addr 172.21.15.73:0/4254252901 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2ba6f1700 1 --1- 172.21.15.73:0/4254252901 >> v1:172.21.15.73:6789/0 conn(0x7fc2b4064350 0x7fc2b4158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45686/0 (socket says 172.21.15.73:45686) 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b96ef700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 0x7fc2b4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 -- 172.21.15.73:0/4254252901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 msgr2=0x7fc2b4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 0x7fc2b4063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 -- 172.21.15.73:0/4254252901 >> v1:172.21.15.73:6789/0 conn(0x7fc2b4064350 legacy=0x7fc2b4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 -- 172.21.15.73:0/4254252901 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b40c69a0 con 0x7fc2b4098d10 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b96ef700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 0x7fc2b4063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b9ef0700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b4098d10 0x7fc2b40c7760 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fc2b0006fd0 tx=0x7fc2b00090f0).ready entity=mon.0 client_cookie=6a4f50a6d13438eb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:59.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.451+0000 7fc2b8eee700 1 -- 172.21.15.73:0/4254252901 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000b600 con 0x7fc2b4098d10 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2b8eee700 1 -- 172.21.15.73:0/4254252901 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b000b760 con 0x7fc2b4098d10 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2b8eee700 1 -- 172.21.15.73:0/4254252901 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b000bae0 con 0x7fc2b4098d10 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 -- 172.21.15.73:0/4254252901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b4098d10 msgr2=0x7fc2b40c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b4098d10 0x7fc2b40c7760 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 rx=0x7fc2b0006fd0 tx=0x7fc2b00090f0).stop 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 -- 172.21.15.73:0/4254252901 shutdown_connections 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b4098d10 0x7fc2b40c7760 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 --2- 172.21.15.73:0/4254252901 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b40c6190 0x7fc2b4063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 -- 172.21.15.73:0/4254252901 >> 172.21.15.73:0/4254252901 conn(0x7fc2b40cb310 msgr2=0x7fc2b40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 -- 172.21.15.73:0/4254252901 shutdown_connections 2023-11-28T03:09:59.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.452+0000 7fc2bc154700 1 -- 172.21.15.73:0/4254252901 wait complete. 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.453+0000 7fc2bc154700 1 Processor -- start 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.453+0000 7fc2bc154700 1 -- start start 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b4064350 0x7fc2b414b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2b4098d10 0x7fc2b40a9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40aa130 con 0x7fc2b40c6190 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40a9e50 con 0x7fc2b4098d10 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2bc154700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2b40af470 con 0x7fc2b4064350 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45104/0 (socket says 172.21.15.73:45104) 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 -- 172.21.15.73:0/3586346108 learned_addr learned my addr 172.21.15.73:0/3586346108 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2b9ef0700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b4064350 0x7fc2b414b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2b96ef700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2b4098d10 0x7fc2b40a9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 -- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b4064350 msgr2=0x7fc2b414b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b4064350 0x7fc2b414b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.454+0000 7fc2ba6f1700 1 -- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2b4098d10 msgr2=0x7fc2b40a9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2ba6f1700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2b4098d10 0x7fc2b40a9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2ba6f1700 1 -- 172.21.15.73:0/3586346108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b0009370 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2ba6f1700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc2ac00c790 tx=0x7fc2ac0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac005c70 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ac005dd0 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2b40af750 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ac007320 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.455+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2b414b690 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.456+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2b406c860 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.457+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 48) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fc2ac007690 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.457+0000 7fc2a9ffb700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 0x7fc298061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.457+0000 7fc2b9ef0700 1 -- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 msgr2=0x7fc298061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.457+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc2ac07a0b0 con 0x7fc2b40c6190 2023-11-28T03:09:59.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.457+0000 7fc2b9ef0700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 0x7fc298061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:59.461 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:59.458+0000 7fc5ddd0a680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:09:59.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.461+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc2ac04b380 con 0x7fc2b40c6190 2023-11-28T03:09:59.510 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:59.509+0000 7f291d2d9680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:09:59.524 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:59.523+0000 7fc5ddd0a680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:09:59.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.635+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc2b40c6c80 con 0x7fc2b40c6190 2023-11-28T03:09:59.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.639+0000 7fc2a9ffb700 1 -- 172.21.15.73:0/3586346108 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc2ac04e9a0 con 0x7fc2b40c6190 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.640+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 msgr2=0x7fc298061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.640+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 0x7fc298061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 msgr2=0x7fc2b4154dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 rx=0x7fc2ac00c790 tx=0x7fc2ac0015e0).stop 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 shutdown_connections 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc29805f1a0 0x7fc298061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2b40c6190 0x7fc2b4154dc0 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2b4064350 0x7fc2b414b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 --2- 172.21.15.73:0/3586346108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2b4098d10 0x7fc2b40a9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 >> 172.21.15.73:0/3586346108 conn(0x7fc2b40cb310 msgr2=0x7fc2b40c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:59.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 shutdown_connections 2023-11-28T03:09:59.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.641+0000 7fc2bc154700 1 -- 172.21.15.73:0/3586346108 wait complete. 2023-11-28T03:09:59.654 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:09:59.654 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T03:09:59.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-28T03:09:59.787 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:09:59.786+0000 7f291d2d9680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:09:59.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:09:59.808+0000 7fc5ddd0a680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:09:59.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.832+0000 7fb4758f4700 1 Processor -- start 2023-11-28T03:09:59.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.832+0000 7fb4758f4700 1 -- start start 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb4758f4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb470154610 0x7fb4701582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb4758f4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 0x7fb4700aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb4758f4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb470159210 con 0x7fb4700ab3b0 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb4758f4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4700c0c70 con 0x7fb470154610 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb4758f4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4700c2110 con 0x7fb4700aa1c0 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb46effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb470154610 0x7fb4701582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb46e7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb4700ab3b0 0x7fb4700c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45690/0 (socket says 172.21.15.73:45690) 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb46e7fc700 1 -- 172.21.15.73:0/1574139388 learned_addr learned my addr 172.21.15.73:0/1574139388 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:09:59.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.833+0000 7fb46f7fe700 1 --2- 172.21.15.73:0/1574139388 >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 0x7fb4700aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46effd700 1 -- 172.21.15.73:0/1574139388 >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 msgr2=0x7fb4700aa600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46effd700 1 --2- 172.21.15.73:0/1574139388 >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 0x7fb4700aa600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46effd700 1 -- 172.21.15.73:0/1574139388 >> v1:172.21.15.73:6789/0 conn(0x7fb4700ab3b0 legacy=0x7fb4700c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46dffb700 1 -- 172.21.15.73:0/1574139388 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1401565243 0 0) 0x7fb470159210 con 0x7fb4700ab3b0 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46effd700 1 -- 172.21.15.73:0/1574139388 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4700aaaf0 con 0x7fb470154610 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46f7fe700 1 --2- 172.21.15.73:0/1574139388 >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 0x7fb4700aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46effd700 1 --2- 172.21.15.73:0/1574139388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470154610 0x7fb4701582e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fb4700cc270 tx=0x7fb458009060).ready entity=mon.0 client_cookie=3c7788762562d714 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46dffb700 1 -- 172.21.15.73:0/1574139388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800b580 con 0x7fb470154610 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46dffb700 1 -- 172.21.15.73:0/1574139388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b6e0 con 0x7fb470154610 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb46dffb700 1 -- 172.21.15.73:0/1574139388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb45800ba60 con 0x7fb470154610 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb4758f4700 1 -- 172.21.15.73:0/1574139388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470154610 msgr2=0x7fb4701582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.834+0000 7fb4758f4700 1 --2- 172.21.15.73:0/1574139388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470154610 0x7fb4701582e0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 rx=0x7fb4700cc270 tx=0x7fb458009060).stop 2023-11-28T03:09:59.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 -- 172.21.15.73:0/1574139388 shutdown_connections 2023-11-28T03:09:59.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 --2- 172.21.15.73:0/1574139388 >> v2:172.21.15.150:3300/0 conn(0x7fb4700aa1c0 0x7fb4700aa600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 --2- 172.21.15.73:0/1574139388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470154610 0x7fb4701582e0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 -- 172.21.15.73:0/1574139388 >> 172.21.15.73:0/1574139388 conn(0x7fb4700655a0 msgr2=0x7fb470069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:09:59.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 -- 172.21.15.73:0/1574139388 shutdown_connections 2023-11-28T03:09:59.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.835+0000 7fb4758f4700 1 -- 172.21.15.73:0/1574139388 wait complete. 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.836+0000 7fb4758f4700 1 Processor -- start 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.836+0000 7fb4758f4700 1 -- start start 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 0x7fb47013fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb470154610 0x7fb470156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4700af4a0 con 0x7fb470157330 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb470154f00 con 0x7fb470154610 2023-11-28T03:09:59.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb4758f4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4701577b0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:45110/0 (socket says 172.21.15.73:45110) 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46f7fe700 1 -- 172.21.15.73:0/3691031314 learned_addr learned my addr 172.21.15.73:0/3691031314 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 0x7fb47013fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb470154610 0x7fb470156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb470154610 msgr2=0x7fb470156e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb470154610 0x7fb470156e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 msgr2=0x7fb4700ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4580092a0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46f7fe700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:09:59.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb46effd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 0x7fb47013fc40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb458014000 tx=0x7fb458017000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.837+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458000bc0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.838+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb45800b850 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.838+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4700aeee0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.838+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb458000d20 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.838+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb470154b90 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.838+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4701549f0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.839+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 48) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fb45801b8e0 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.839+0000 7fb466ffd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.839+0000 7fb46e7fc700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.839+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb45807b730 con 0x7fb4700aa1c0 2023-11-28T03:09:59.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.839+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:09:59.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:09:59.844+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb4701549f0 con 0x7fb4700aa1c0 2023-11-28T03:10:00.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:00.040+0000 7fb46e7fc700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:10:00.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:00.040+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:10:00.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:00.043+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4701549f0 con 0x7fb45005f1a0 2023-11-28T03:10:00.405 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:00.402+0000 7f291d2d9680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:10:00.431 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:00.430+0000 7fc5ddd0a680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:10:00.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:00.441+0000 7fb46e7fc700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:10:00.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:00.441+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:10:00.475 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:00.474+0000 7f291d2d9680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:10:00.504 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:00.503+0000 7fc5ddd0a680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:10:00.661 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:00.660+0000 7f291d2d9680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:10:00.689 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:00.688+0000 7fc5ddd0a680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:10:01.082 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.081+0000 7f291d2d9680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:10:01.108 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.107+0000 7fc5ddd0a680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:10:01.185 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.184+0000 7f291d2d9680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:10:01.211 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.210+0000 7fc5ddd0a680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:10:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:01.242+0000 7fb46e7fc700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:10:01.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:01.242+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:10:01.340 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.338+0000 7f291d2d9680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:10:01.365 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.364+0000 7fc5ddd0a680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:10:01.414 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.413+0000 7f291d2d9680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:10:01.439 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.438+0000 7fc5ddd0a680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:10:01.481 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.480+0000 7f291d2d9680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:10:01.507 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.506+0000 7fc5ddd0a680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:10:01.740 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.739+0000 7f291d2d9680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:10:01.765 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.765+0000 7fc5ddd0a680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:10:01.923 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:01.922+0000 7f291d2d9680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:10:01.949 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:01.948+0000 7fc5ddd0a680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:10:02.576 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:02.575+0000 7f291d2d9680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:10:02.596 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:02.596+0000 7fc5ddd0a680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:10:02.644 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:02.643+0000 7f291d2d9680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:10:02.665 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:02.665+0000 7fc5ddd0a680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:10:02.719 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:02.718+0000 7f291d2d9680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:10:02.742 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:02.742+0000 7fc5ddd0a680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:10:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:02.844+0000 7fb46e7fc700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:10:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:02.844+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:10:02.864 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:02.864+0000 7f291d2d9680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:10:02.891 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:02.891+0000 7fc5ddd0a680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:10:02.934 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:02.933+0000 7f291d2d9680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:10:02.962 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:02.962+0000 7fc5ddd0a680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:10:03.050 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:03.049+0000 7f291d2d9680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:10:03.078 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.078+0000 7fc5ddd0a680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:10:03.159 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:03.158+0000 7f291d2d9680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:10:03.186 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.186+0000 7fc5ddd0a680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:10:03.451 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:03.450+0000 7f291d2d9680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:10:03.479 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.479+0000 7fc5ddd0a680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:10:03.522 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:10:03.521+0000 7f291d2d9680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:10:03.550 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.550+0000 7fc5ddd0a680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:10:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:03.570+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mgrmap(e 49) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fb458079500 con 0x7fb4700aa1c0 2023-11-28T03:10:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:03.571+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb450061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:10:03.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:03.571+0000 7fb466ffd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb450061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:10:03.617 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc555496700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.617 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc555496700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.617 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc555496700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.617 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc555496700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.617 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc555496700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc552490700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc552490700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc552490700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc552490700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:03.618 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:10:03.617+0000 7fc552490700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:10:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.574+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mon.2 v2:172.21.15.150:3301/0 8 ==== mgrmap(e 50) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fb458006a40 con 0x7fb4700aa1c0 2023-11-28T03:10:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.574+0000 7fb466ffd700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb4500615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.575+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb4701549f0 con 0x7fb45005f1a0 2023-11-28T03:10:04.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.586+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb4500615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.587+0000 7fb46e7fc700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb4500615e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb47015a490 tx=0x7fb460007bb0).ready entity=mgr.7818 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:10:04.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.593+0000 7fb466ffd700 1 -- 172.21.15.73:0/3691031314 <== mgr.7818 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb4701549f0 con 0x7fb45005f1a0 2023-11-28T03:10:04.593 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 msgr2=0x7fb4500615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb4500615e0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 rx=0x7fb47015a490 tx=0x7fb460007bb0).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 msgr2=0x7fb47013fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 0x7fb47013fc40 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 rx=0x7fb458014000 tx=0x7fb458017000).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 shutdown_connections 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb45005f1a0 0x7fb4500615e0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb470157330 0x7fb4700ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb4700aa1c0 0x7fb47013fc40 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 --2- 172.21.15.73:0/3691031314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb470154610 0x7fb470156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 >> 172.21.15.73:0/3691031314 conn(0x7fb4700655a0 msgr2=0x7fb47014d910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 shutdown_connections 2023-11-28T03:10:04.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.595+0000 7fb4758f4700 1 -- 172.21.15.73:0/3691031314 wait complete. 2023-11-28T03:10:04.610 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T03:10:04.610 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-28-03_11_00 in ~56s... 2023-11-28T03:10:04.610 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-11-28T03:10:04.610 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-28T03:10:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.733+0000 7f5514b1a700 1 Processor -- start 2023-11-28T03:10:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.733+0000 7f5514b1a700 1 -- start start 2023-11-28T03:10:04.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f5514b1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5510137550 0x7f55100c2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f5514b1a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55100c62f0 0x7f55100bbb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f5514b1a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f551006a440 con 0x7f5510137930 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f5514b1a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55100c3090 con 0x7f55100c62f0 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f5514b1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5510069830 con 0x7f5510137550 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55100c62f0 0x7f55100bbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550f7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5510137550 0x7f55100c2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55100c62f0 0x7f55100bbb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:50804/0 (socket says 172.21.15.73:50804) 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 -- 172.21.15.73:0/2987950558 learned_addr learned my addr 172.21.15.73:0/2987950558 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550effd700 1 --1- 172.21.15.73:0/2987950558 >> v1:172.21.15.150:6789/0 conn(0x7f5510137930 0x7f55100c5be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:48770/0 (socket says 172.21.15.73:48770) 2023-11-28T03:10:04.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 -- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5510137550 msgr2=0x7f55100c2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 --2- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5510137550 0x7f55100c2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 -- 172.21.15.73:0/2987950558 >> v1:172.21.15.150:6789/0 conn(0x7f5510137930 legacy=0x7f55100c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 -- 172.21.15.73:0/2987950558 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5510069b10 con 0x7f55100c62f0 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.734+0000 7f550ffff700 1 --2- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55100c62f0 0x7f55100bbb30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f5508009f20 tx=0x7f5508009840).ready entity=mon.1 client_cookie=49f350d5e2b43250 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f550e7fc700 1 -- 172.21.15.73:0/2987950558 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800e070 con 0x7f55100c62f0 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f550e7fc700 1 -- 172.21.15.73:0/2987950558 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550800b9d0 con 0x7f55100c62f0 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f550e7fc700 1 -- 172.21.15.73:0/2987950558 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550800bd50 con 0x7f55100c62f0 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 -- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55100c62f0 msgr2=0x7f55100bbb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 --2- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55100c62f0 0x7f55100bbb30 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 rx=0x7f5508009f20 tx=0x7f5508009840).stop 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 -- 172.21.15.73:0/2987950558 shutdown_connections 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 --2- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55100c62f0 0x7f55100bbb30 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 --2- 172.21.15.73:0/2987950558 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5510137550 0x7f55100c2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 -- 172.21.15.73:0/2987950558 >> 172.21.15.73:0/2987950558 conn(0x7f55100631d0 msgr2=0x7f55100c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:10:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 -- 172.21.15.73:0/2987950558 shutdown_connections 2023-11-28T03:10:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.735+0000 7f5514b1a700 1 -- 172.21.15.73:0/2987950558 wait complete. 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.736+0000 7f5514b1a700 1 Processor -- start 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.736+0000 7f5514b1a700 1 -- start start 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 0x7f55100ab4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5510137550 0x7f55100aada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100acb60 con 0x7f5510137930 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55100ac6d0 con 0x7f5510137550 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f5514b1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55100ab290 con 0x7f55100c62f0 2023-11-28T03:10:04.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f550ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f550ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36856/0 (socket says 172.21.15.73:36856) 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f550ffff700 1 -- 172.21.15.73:0/3369849964 learned_addr learned my addr 172.21.15.73:0/3369849964 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f550effd700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5510137550 0x7f55100aada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.737+0000 7f550f7fe700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 0x7f55100ab4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 -- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 msgr2=0x7f55100ab4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 0x7f55100ab4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 -- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5510137550 msgr2=0x7f55100aada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5510137550 0x7f55100aada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 -- 172.21.15.73:0/3369849964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5508009ae0 con 0x7f5510137930 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550f7fe700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 0x7f55100ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:10:04.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f550ffff700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f550800b610 tx=0x7f5508009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5508013e50 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f550801d460 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55100abac0 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f550801d800 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.738+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55100ac080 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.739+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55100ac860 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.739+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 50) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5508003040 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.739+0000 7f54f77fe700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 0x7f54fc061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.739+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f550807b480 con 0x7f5510137930 2023-11-28T03:10:04.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.740+0000 7f550f7fe700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 0x7f54fc061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:10:04.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.740+0000 7f550f7fe700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 0x7f54fc061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5500006d80 tx=0x7f5500008040).ready entity=mgr.7818 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:10:04.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.744+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f550804c870 con 0x7f5510137930 2023-11-28T03:10:04.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.929+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f551006c7c0 con 0x7f54fc05f050 2023-11-28T03:10:04.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.932+0000 7f54f77fe700 1 -- 172.21.15.73:0/3369849964 <== mgr.7818 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f551006c7c0 con 0x7f54fc05f050 2023-11-28T03:10:04.932 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:10:04.932 INFO:teuthology.orchestra.run.smithi073.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.933+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 msgr2=0x7f54fc061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.933+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 0x7f54fc061510 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5500006d80 tx=0x7f5500008040).stop 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.933+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 msgr2=0x7f551014d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 rx=0x7f550800b610 tx=0x7f5508009840).stop 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 shutdown_connections 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54fc05f050 0x7f54fc061510 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5510137930 0x7f551014d9d0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55100c62f0 0x7f55100ab4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 --2- 172.21.15.73:0/3369849964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5510137550 0x7f55100aada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:10:04.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 >> 172.21.15.73:0/3369849964 conn(0x7f55100631d0 msgr2=0x7f55100bbb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:10:04.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 shutdown_connections 2023-11-28T03:10:04.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:10:04.934+0000 7f5514b1a700 1 -- 172.21.15.73:0/3369849964 wait complete. 2023-11-28T03:10:04.946 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-11-28T03:10:04.949 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-11-28T03:10:04.949 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:04.950 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:05.965 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:05.966 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:06.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:06.985 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:08.004 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:08.005 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:09.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:09.024 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:10.045 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:10.045 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:11.068 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:11.068 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:12.088 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:12.088 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:13.108 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:13.108 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:14.129 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:14.129 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:15.149 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:15.149 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:16.168 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:16.169 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:17.188 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:17.188 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:18.208 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:18.208 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:19.228 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:19.228 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:20.247 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:20.248 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:21.268 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:21.268 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:22.287 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:22.287 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:23.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:10:23.270 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:10:23.306 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:23.306 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:24.329 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:24.330 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:25.349 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:25.349 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:26.367 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:26.368 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:27.388 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:27.389 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:28.408 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:28.408 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:29.428 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:29.428 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:30.448 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:30.448 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:31.468 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:31.468 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:32.488 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:32.488 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:33.508 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:33.508 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:34.527 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:34.527 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:35.547 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:35.548 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:36.567 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:36.567 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:37.587 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:37.587 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:38.607 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:38.608 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:39.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:39.628 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:40.648 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:40.648 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:41.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:41.668 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:42.688 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:42.688 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:43.707 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:43.707 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:44.729 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:44.729 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:45.750 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:45.751 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:46.771 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:46.771 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:47.791 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:47.791 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:48.812 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:48.812 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:49.834 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:49.835 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:50.854 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:50.855 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:51.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:51.874 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:52.894 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:52.894 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:53.337 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:10:53.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:10:53.914 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:53.914 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:54.934 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:54.934 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:55.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:55.954 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:56.974 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:56.974 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:57.994 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:57.994 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:10:59.014 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:10:59.014 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:11:00.035 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:00.036 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:11:00.597 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:00.596+0000 7fc544c75700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:00.598 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:00.596+0000 7fc544c75700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:00.598 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:00.596+0000 7fc544c75700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:00.598 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:00.596+0000 7fc544c75700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:00.598 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:00.596+0000 7fc544c75700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:01.056 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:01.056 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:11:01.075 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_11_00_UTC 2023-11-28T03:11:01.076 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-28-03_11_00_UTC'] 2023-11-28T03:11:01.076 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701141004.6106043, now = 1701141061.0762248, timo = 56 2023-11-28T03:11:02.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-28T03:11:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 Processor -- start 2023-11-28T03:11:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 -- start start 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21e4154610 0x7f21e41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f21e40ab3b0 0x7f21e40c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21e4159210 con 0x7f21e40aa1c0 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40c0c70 con 0x7f21e4154610 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.205+0000 7f21e9bb1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21e40c2110 con 0x7f21e40ab3b0 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f21e4154610 0x7f21e41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f21e40aa1c0 0x7f21e4063550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35424/0 (socket says 172.21.15.73:35424) 2023-11-28T03:11:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e3fff700 1 -- 172.21.15.73:0/3425032012 learned_addr learned my addr 172.21.15.73:0/3425032012 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e2ffd700 1 --2- 172.21.15.73:0/3425032012 >> v2:172.21.15.150:3300/0 conn(0x7f21e40ab3b0 0x7f21e40c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 -- 172.21.15.73:0/3425032012 >> v2:172.21.15.150:3300/0 conn(0x7f21e40ab3b0 msgr2=0x7f21e40c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 --2- 172.21.15.73:0/3425032012 >> v2:172.21.15.150:3300/0 conn(0x7f21e40ab3b0 0x7f21e40c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 -- 172.21.15.73:0/3425032012 >> v1:172.21.15.73:6789/0 conn(0x7f21e40aa1c0 legacy=0x7f21e4063550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 -- 172.21.15.73:0/3425032012 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21e4158970 con 0x7f21e4154610 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e27fc700 1 -- 172.21.15.73:0/3425032012 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 565211613 0 0) 0x7f21e4159210 con 0x7f21e40aa1c0 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e37fe700 1 --2- 172.21.15.73:0/3425032012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4154610 0x7f21e41582e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f21e40c5770 tx=0x7f21d4009780).ready entity=mon.0 client_cookie=bcd5ea978ce466d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e27fc700 1 -- 172.21.15.73:0/3425032012 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d400e070 con 0x7f21e4154610 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e27fc700 1 -- 172.21.15.73:0/3425032012 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21d400bc30 con 0x7f21e4154610 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.206+0000 7f21e27fc700 1 -- 172.21.15.73:0/3425032012 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21d4012410 con 0x7f21e4154610 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 -- 172.21.15.73:0/3425032012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4154610 msgr2=0x7f21e41582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/3425032012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4154610 0x7f21e41582e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 rx=0x7f21e40c5770 tx=0x7f21d4009780).stop 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 -- 172.21.15.73:0/3425032012 shutdown_connections 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/3425032012 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4154610 0x7f21e41582e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/3425032012 >> v2:172.21.15.150:3300/0 conn(0x7f21e40ab3b0 0x7f21e40c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 -- 172.21.15.73:0/3425032012 >> 172.21.15.73:0/3425032012 conn(0x7f21e40655a0 msgr2=0x7f21e4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 -- 172.21.15.73:0/3425032012 shutdown_connections 2023-11-28T03:11:02.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.207+0000 7f21e9bb1700 1 -- 172.21.15.73:0/3425032012 wait complete. 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.208+0000 7f21e9bb1700 1 Processor -- start 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.208+0000 7f21e9bb1700 1 -- start start 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 0x7f21e4157450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 0x7f21e4168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e40ac170 con 0x7f21e4168810 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21e41504d0 con 0x7f21e40aa1c0 2023-11-28T03:11:02.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e9bb1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21e414ff50 con 0x7f21e4154610 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35290/0 (socket says 172.21.15.73:35290) 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 -- 172.21.15.73:0/906089879 learned_addr learned my addr 172.21.15.73:0/906089879 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e2ffd700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 0x7f21e4168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e37fe700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 0x7f21e4157450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 -- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 msgr2=0x7f21e4168320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 0x7f21e4168320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 -- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 msgr2=0x7f21e4157450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 0x7f21e4157450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 -- 172.21.15.73:0/906089879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009a20 con 0x7f21e4168810 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e2ffd700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 0x7f21e4168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:02.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e37fe700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 0x7f21e4157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.209+0000 7f21e3fff700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f21dc00c490 tx=0x7f21dc007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.210+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0072e0 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.210+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21dc007440 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.210+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21e41501f0 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.210+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21dc0057a0 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.210+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21e40ac400 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.211+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21e40ac590 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.211+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 51) v1 ==== 75622+0+0 (secure 0 0 0) 0x7f21dc005b90 con 0x7f21e4168810 2023-11-28T03:11:02.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.211+0000 7f21cb7fe700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 0x7f21d0061840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.211+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f21dc079e00 con 0x7f21e4168810 2023-11-28T03:11:02.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.212+0000 7f21e37fe700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 0x7f21d0061840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.212+0000 7f21e37fe700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 0x7f21d0061840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f21d4009fd0 tx=0x7f21d4009780).ready entity=mgr.7818 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.216+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21e40ac590 con 0x7f21e4168810 2023-11-28T03:11:02.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.316+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 52) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f21dc077920 con 0x7f21e4168810 2023-11-28T03:11:02.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.405+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f21e40ac590 con 0x7f21d005f380 2023-11-28T03:11:02.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.409+0000 7f21cb7fe700 1 -- 172.21.15.73:0/906089879 <== mgr.7818 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f21e40ac590 con 0x7f21d005f380 2023-11-28T03:11:02.411 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path snap_test_dir1 2023-11-28T03:11:02.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 msgr2=0x7f21d0061840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 0x7f21d0061840 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f21d4009fd0 tx=0x7f21d4009780).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 msgr2=0x7f21e4154aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 rx=0x7f21dc00c490 tx=0x7f21dc007c10).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 shutdown_connections 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f21d005f380 0x7f21d0061840 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f21e4168810 0x7f21e4154aa0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f21e40aa1c0 0x7f21e4157450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 --2- 172.21.15.73:0/906089879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21e4154610 0x7f21e4168320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 >> 172.21.15.73:0/906089879 conn(0x7f21e40655a0 msgr2=0x7f21e40c9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:02.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 shutdown_connections 2023-11-28T03:11:02.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.411+0000 7f21e9bb1700 1 -- 172.21.15.73:0/906089879 wait complete. 2023-11-28T03:11:02.424 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-28T03:11:02.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:02.424 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T03:11:02.440 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_11_00_UTC 2023-11-28T03:11:02.440 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_11_00_UTC 2023-11-28T03:11:02.440 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:02.440 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_11_00_UTC) 2023-11-28T03:11:02.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:02.499 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T03:11:02.556 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T03:11:02.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.710+0000 7f3e37fff700 1 Processor -- start 2023-11-28T03:11:02.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.710+0000 7f3e37fff700 1 -- start start 2023-11-28T03:11:02.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e37fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e3814e530 0x7f3e380c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e37fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38154950 0x7f3e3814e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e37fff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e380c7bc0 con 0x7f3e38158570 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e37fff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e38155920 con 0x7f3e3814e530 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e37fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e38154dd0 con 0x7f3e38154950 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3e3814e530 0x7f3e380c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e377fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3e38158570 0x7f3e380aa030 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35432/0 (socket says 172.21.15.73:35432) 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e377fe700 1 -- 172.21.15.73:0/170637418 learned_addr learned my addr 172.21.15.73:0/170637418 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 -- 172.21.15.73:0/170637418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38154950 msgr2=0x7f3e3814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 --2- 172.21.15.73:0/170637418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38154950 0x7f3e3814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 -- 172.21.15.73:0/170637418 >> v1:172.21.15.73:6789/0 conn(0x7f3e38158570 legacy=0x7f3e380aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e35ffb700 1 -- 172.21.15.73:0/170637418 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1557050929 0 0) 0x7f3e380c7bc0 con 0x7f3e38158570 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 -- 172.21.15.73:0/170637418 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e38155080 con 0x7f3e3814e530 2023-11-28T03:11:02.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e36ffd700 1 --2- 172.21.15.73:0/170637418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e3814e530 0x7f3e380c66f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f3e30009760 tx=0x7f3e30009060).ready entity=mon.0 client_cookie=111c47ea6afa1c7d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e35ffb700 1 -- 172.21.15.73:0/170637418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e300090d0 con 0x7f3e3814e530 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e35ffb700 1 -- 172.21.15.73:0/170637418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e3000b580 con 0x7f3e3814e530 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.711+0000 7f3e35ffb700 1 -- 172.21.15.73:0/170637418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e3000b900 con 0x7f3e3814e530 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 -- 172.21.15.73:0/170637418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e3814e530 msgr2=0x7f3e380c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 --2- 172.21.15.73:0/170637418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e3814e530 0x7f3e380c66f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 rx=0x7f3e30009760 tx=0x7f3e30009060).stop 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 -- 172.21.15.73:0/170637418 shutdown_connections 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 --2- 172.21.15.73:0/170637418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e3814e530 0x7f3e380c66f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 --2- 172.21.15.73:0/170637418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38154950 0x7f3e3814e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 -- 172.21.15.73:0/170637418 >> 172.21.15.73:0/170637418 conn(0x7f3e38064990 msgr2=0x7f3e38064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 -- 172.21.15.73:0/170637418 shutdown_connections 2023-11-28T03:11:02.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.712+0000 7f3e37fff700 1 -- 172.21.15.73:0/170637418 wait complete. 2023-11-28T03:11:02.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.713+0000 7f3e37fff700 1 Processor -- start 2023-11-28T03:11:02.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.713+0000 7f3e37fff700 1 -- start start 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e3814e530 0x7f3e3814dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38158570 0x7f3e380c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c5d80 con 0x7f3e38154950 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380b0920 con 0x7f3e3814e530 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e380c33c0 con 0x7f3e38158570 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35316/0 (socket says 172.21.15.73:35316) 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 -- 172.21.15.73:0/1348499597 learned_addr learned my addr 172.21.15.73:0/1348499597 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 -- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38158570 msgr2=0x7f3e380c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38158570 0x7f3e380c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 -- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e3814e530 msgr2=0x7f3e3814dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:02.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e36ffd700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e3814e530 0x7f3e3814dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e3814e530 0x7f3e3814dac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 -- 172.21.15.73:0/1348499597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e300092a0 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e367fc700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f3e2c00b530 tx=0x7f3e2c0090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00d4a0 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e2c00d620 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e380c2a90 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.714+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e2c00d9e0 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.715+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e380c3050 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.715+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e380c3570 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.716+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 52) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f3e2c012410 con 0x7f3e38154950 2023-11-28T03:11:02.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.716+0000 7f3e1effd700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 0x7f3e240618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:02.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.716+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f3e2c07a5f0 con 0x7f3e38154950 2023-11-28T03:11:02.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.716+0000 7f3e36ffd700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 0x7f3e240618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:02.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.716+0000 7f3e36ffd700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 0x7f3e240618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3e3000b340 tx=0x7f3e30017000).ready entity=mgr.7818 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:02.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.720+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3e380c3570 con 0x7f3e38154950 2023-11-28T03:11:02.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:02.928+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f3e380aa190 con 0x7f3e38154950 2023-11-28T03:11:03.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.321+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3e2c019db0 con 0x7f3e38154950 2023-11-28T03:11:03.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.322+0000 7f3e1effd700 1 -- 172.21.15.73:0/1348499597 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v53) v1 ==== 93+0+0 (secure 0 0 0) 0x7f3e2c0736b0 con 0x7f3e38154950 2023-11-28T03:11:03.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 msgr2=0x7f3e240618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 0x7f3e240618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f3e3000b340 tx=0x7f3e30017000).stop 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 msgr2=0x7f3e380c53a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 rx=0x7f3e2c00b530 tx=0x7f3e2c0090f0).stop 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 shutdown_connections 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3e2405f3e0 0x7f3e240618a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3e38158570 0x7f3e380c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:03.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3e3814e530 0x7f3e3814dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 --2- 172.21.15.73:0/1348499597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3e38154950 0x7f3e380c53a0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 >> 172.21.15.73:0/1348499597 conn(0x7f3e38064990 msgr2=0x7f3e38156ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:03.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 shutdown_connections 2023-11-28T03:11:03.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.324+0000 7f3e37fff700 1 -- 172.21.15.73:0/1348499597 wait complete. 2023-11-28T03:11:03.337 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:11:03.338 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:11:03.374 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:11:03.420 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:11:03.421 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T03:11:03.421 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:03.421 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T03:11:03.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T03:11:03.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:03.460 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T03:11:03.465 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:03.465+0000 7faae096b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:11:03.469 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:03.467+0000 7ff428cdd680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:11:03.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:03.531 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:03.530+0000 7ff428cdd680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:11:03.532 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:03.531+0000 7faae096b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:11:03.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.637+0000 7f2dd177c700 1 Processor -- start 2023-11-28T03:11:03.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.637+0000 7f2dd177c700 1 -- start start 2023-11-28T03:11:03.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd177c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2dcc0a8380 0x7f2dcc0bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd177c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd177c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dcc0bf0e0 con 0x7f2dcc152190 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd177c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc0be510 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd177c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dcc152570 con 0x7f2dcc0a8380 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33598/0 (socket says 172.21.15.73:33598) 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 -- 172.21.15.73:0/450788726 learned_addr learned my addr 172.21.15.73:0/450788726 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dcb7fe700 1 --2- 172.21.15.73:0/450788726 >> v2:172.21.15.150:3300/0 conn(0x7f2dcc0a8380 0x7f2dcc0bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 -- 172.21.15.73:0/450788726 >> v2:172.21.15.150:3300/0 conn(0x7f2dcc0a8380 msgr2=0x7f2dcc0bf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 --2- 172.21.15.73:0/450788726 >> v2:172.21.15.150:3300/0 conn(0x7f2dcc0a8380 0x7f2dcc0bf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 -- 172.21.15.73:0/450788726 >> v1:172.21.15.73:6789/0 conn(0x7f2dcc152190 legacy=0x7f2dcc0c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.638+0000 7f2dd0f7b700 1 -- 172.21.15.73:0/450788726 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dcc0be7b0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd0f7b700 1 --2- 172.21.15.73:0/450788726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f2dcc068a40 tx=0x7f2dc4009840).ready entity=mon.0 client_cookie=16148ef367184530 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dcaffd700 1 -- 172.21.15.73:0/450788726 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400e070 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dcaffd700 1 -- 172.21.15.73:0/450788726 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dc400ba20 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dcaffd700 1 -- 172.21.15.73:0/450788726 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dc400bda0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 -- 172.21.15.73:0/450788726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 msgr2=0x7f2dcc0a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 --2- 172.21.15.73:0/450788726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 rx=0x7f2dcc068a40 tx=0x7f2dc4009840).stop 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 -- 172.21.15.73:0/450788726 shutdown_connections 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 --2- 172.21.15.73:0/450788726 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc0a5c00 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 --2- 172.21.15.73:0/450788726 >> v2:172.21.15.150:3300/0 conn(0x7f2dcc0a8380 0x7f2dcc0bf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 -- 172.21.15.73:0/450788726 >> 172.21.15.73:0/450788726 conn(0x7f2dcc083000 msgr2=0x7f2dcc092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:03.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 -- 172.21.15.73:0/450788726 shutdown_connections 2023-11-28T03:11:03.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.639+0000 7f2dd177c700 1 -- 172.21.15.73:0/450788726 wait complete. 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.640+0000 7f2dd177c700 1 Processor -- start 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 -- start start 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc0a8380 0x7f2dcc14a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 0x7f2dcc154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc1551f0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc154670 con 0x7f2dcc0a8380 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.641+0000 7f2dd177c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dcc154950 con 0x7f2dcc152190 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33610/0 (socket says 172.21.15.73:33610) 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 -- 172.21.15.73:0/2136886535 learned_addr learned my addr 172.21.15.73:0/2136886535 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcbfff700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc0a8380 0x7f2dcc14a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dd0f7b700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 0x7f2dcc154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 msgr2=0x7f2dcc154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 0x7f2dcc154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc0a8380 msgr2=0x7f2dcc14a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc0a8380 0x7f2dcc14a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 -- 172.21.15.73:0/2136886535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc4009ae0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dd0f7b700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 0x7f2dcc154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dcb7fe700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f2dbc00e900 tx=0x7f2dbc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012600 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dbc012760 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dbc012ae0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dcc154c30 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.642+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dcc14ece0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.643+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dcc06c7c0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.644+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f2dbc01b400 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.644+0000 7f2db3fff700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 0x7f2db8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.644+0000 7f2dcbfff700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 msgr2=0x7f2db8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:03.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.644+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2dbc07aad0 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.644+0000 7f2dcbfff700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 0x7f2db8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:03.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.652+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2dbc04b900 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.806 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:03.805+0000 7ff428cdd680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:11:03.821 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:03.820+0000 7faae096b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:11:03.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.845+0000 7f2dcbfff700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 msgr2=0x7f2db8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:03.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.845+0000 7f2dcbfff700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 0x7f2db8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:03.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.864+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2dcc0aee00 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.865+0000 7f2db3fff700 1 -- 172.21.15.73:0/2136886535 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f2dbc016070 con 0x7f2dcc0bfa60 2023-11-28T03:11:03.866 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:03.866 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:11:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 msgr2=0x7f2db8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 0x7f2db8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 msgr2=0x7f2dcc153b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:03.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 rx=0x7f2dbc00e900 tx=0x7f2dbc009060).stop 2023-11-28T03:11:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 shutdown_connections 2023-11-28T03:11:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2db805f390 0x7f2db8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2dcc152190 0x7f2dcc154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2dcc0a8380 0x7f2dcc14a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 --2- 172.21.15.73:0/2136886535 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2dcc0bfa60 0x7f2dcc153b20 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:03.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 >> 172.21.15.73:0/2136886535 conn(0x7f2dcc083000 msgr2=0x7f2dcc0a74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:03.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 shutdown_connections 2023-11-28T03:11:03.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:03.867+0000 7f2dd177c700 1 -- 172.21.15.73:0/2136886535 wait complete. 2023-11-28T03:11:03.873 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:11:03.881 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.036+0000 7fdfe57ab700 1 Processor -- start 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.036+0000 7fdfe57ab700 1 -- start start 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfe57ab700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdfe014ada0 0x7fdfe0064f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfe57ab700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfe0151880 0x7fdfe00c5900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfe57ab700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe0066240 con 0x7fdfe00c5df0 2023-11-28T03:11:04.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfe57ab700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe0065440 con 0x7fdfe0151880 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfe57ab700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe00666d0 con 0x7fdfe014ada0 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfe0151880 0x7fdfe00c5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfe0151880 0x7fdfe00c5900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33622/0 (socket says 172.21.15.73:33622) 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 -- 172.21.15.73:0/1273493453 learned_addr learned my addr 172.21.15.73:0/1273493453 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfdeffd700 1 --2- 172.21.15.73:0/1273493453 >> v2:172.21.15.150:3300/0 conn(0x7fdfe014ada0 0x7fdfe0064f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 -- 172.21.15.73:0/1273493453 >> v2:172.21.15.150:3300/0 conn(0x7fdfe014ada0 msgr2=0x7fdfe0064f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 --2- 172.21.15.73:0/1273493453 >> v2:172.21.15.150:3300/0 conn(0x7fdfe014ada0 0x7fdfe0064f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.037+0000 7fdfde7fc700 1 -- 172.21.15.73:0/1273493453 >> v1:172.21.15.73:6789/0 conn(0x7fdfe00c5df0 legacy=0x7fdfe00c2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfde7fc700 1 -- 172.21.15.73:0/1273493453 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe00c6220 con 0x7fdfe0151880 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfde7fc700 1 --2- 172.21.15.73:0/1273493453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe00c5900 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fdfe0064460 tx=0x7fdfd000a2b0).ready entity=mon.0 client_cookie=75d0ff0f00f0286e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfddffb700 1 -- 172.21.15.73:0/1273493453 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000ae90 con 0x7fdfe0151880 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfddffb700 1 -- 172.21.15.73:0/1273493453 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd000c940 con 0x7fdfe0151880 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfddffb700 1 -- 172.21.15.73:0/1273493453 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd000ccc0 con 0x7fdfe0151880 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1273493453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 msgr2=0x7fdfe00c5900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1273493453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe00c5900 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 rx=0x7fdfe0064460 tx=0x7fdfd000a2b0).stop 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1273493453 shutdown_connections 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1273493453 >> v2:172.21.15.150:3300/0 conn(0x7fdfe014ada0 0x7fdfe0064f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1273493453 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe00c5900 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1273493453 >> 172.21.15.73:0/1273493453 conn(0x7fdfe00c3f20 msgr2=0x7fdfe00c4330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1273493453 shutdown_connections 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.038+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1273493453 wait complete. 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 Processor -- start 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 -- start start 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfe00c5df0 0x7fdfe00beae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfe014ada0 0x7fdfe0167bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0164a40 con 0x7fdfe0151880 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe0164710 con 0x7fdfe00c5df0 2023-11-28T03:11:04.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfe57ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe01684d0 con 0x7fdfe014ada0 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfdf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfdf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33628/0 (socket says 172.21.15.73:33628) 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.040+0000 7fdfdf7fe700 1 -- 172.21.15.73:0/1241445597 learned_addr learned my addr 172.21.15.73:0/1241445597 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdeffd700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfe00c5df0 0x7fdfe00beae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfde7fc700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfe014ada0 0x7fdfe0167bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 -- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfe014ada0 msgr2=0x7fdfe0167bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfe014ada0 0x7fdfe0167bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 -- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfe00c5df0 msgr2=0x7fdfe00beae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfe00c5df0 0x7fdfe00beae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 -- 172.21.15.73:0/1241445597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfd000a510 con 0x7fdfe0151880 2023-11-28T03:11:04.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfdf7fe700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fdfd8002230 tx=0x7fdfd80015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800e4a0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfd800e600 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe01687b0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfd800e980 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.041+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe01680c0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.042+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfe0164bb0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.042+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fdfd801b3b0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.043+0000 7fdfc6ffd700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 0x7fdfcc061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.043+0000 7fdfdeffd700 1 -- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 msgr2=0x7fdfcc061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.043+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fdfd8079cf0 con 0x7fdfe0151880 2023-11-28T03:11:04.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.043+0000 7fdfdeffd700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 0x7fdfcc061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:04.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.047+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdfd8048af0 con 0x7fdfe0151880 2023-11-28T03:11:04.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.217+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdfe006c7c0 con 0x7fdfe0151880 2023-11-28T03:11:04.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.218+0000 7fdfc6ffd700 1 -- 172.21.15.73:0/1241445597 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17186 (secure 0 0 0) 0x7fdfd804ac60 con 0x7fdfe0151880 2023-11-28T03:11:04.219 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:04.219 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":127,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:10:03.562867+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:6832/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/3114748317":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/1437075559":"2023-11-29T03:10:03.562840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:04.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.219+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 msgr2=0x7fdfcc061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.219+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 0x7fdfcc061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 msgr2=0x7fdfe0164ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 rx=0x7fdfd8002230 tx=0x7fdfd80015e0).stop 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 shutdown_connections 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdfcc05f390 0x7fdfcc061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfe0151880 0x7fdfe0164ed0 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfe00c5df0 0x7fdfe00beae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 --2- 172.21.15.73:0/1241445597 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfe014ada0 0x7fdfe0167bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 >> 172.21.15.73:0/1241445597 conn(0x7fdfe00c3f20 msgr2=0x7fdfe00c2710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 shutdown_connections 2023-11-28T03:11:04.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.220+0000 7fdfe57ab700 1 -- 172.21.15.73:0/1241445597 wait complete. 2023-11-28T03:11:04.233 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '117', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 15, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:09:46.277212+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '125', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 16, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:09:46.698961+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '126', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T03:11:04.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:11:04.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 Processor -- start 2023-11-28T03:11:04.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 -- start start 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65640c6190 0x7f65640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6564158320 0x7f65641587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6564158cc0 con 0x7f6564098d10 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65640c72d0 con 0x7f65640c6190 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.354+0000 7f656a7a7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6564064570 con 0x7f6564158320 2023-11-28T03:11:04.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65640c6190 0x7f65640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f65640c6190 0x7f65640c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33632/0 (socket says 172.21.15.73:33632) 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 -- 172.21.15.73:0/1249499278 learned_addr learned my addr 172.21.15.73:0/1249499278 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f6568d44700 1 --2- 172.21.15.73:0/1249499278 >> v2:172.21.15.150:3300/0 conn(0x7f6564158320 0x7f65641587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 -- 172.21.15.73:0/1249499278 >> v2:172.21.15.150:3300/0 conn(0x7f6564158320 msgr2=0x7f65641587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 --2- 172.21.15.73:0/1249499278 >> v2:172.21.15.150:3300/0 conn(0x7f6564158320 0x7f65641587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 -- 172.21.15.73:0/1249499278 >> v1:172.21.15.150:6789/0 conn(0x7f6564098d10 legacy=0x7f6564063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 -- 172.21.15.73:0/1249499278 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6564154bc0 con 0x7f65640c6190 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f65637fe700 1 --2- 172.21.15.73:0/1249499278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f65640c7760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f65640657a0 tx=0x7f65540090f0).ready entity=mon.0 client_cookie=4f08042272703123 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f6562ffd700 1 -- 172.21.15.73:0/1249499278 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400b520 con 0x7f65640c6190 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f6562ffd700 1 -- 172.21.15.73:0/1249499278 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655400b680 con 0x7f65640c6190 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.355+0000 7f6562ffd700 1 -- 172.21.15.73:0/1249499278 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400ba00 con 0x7f65640c6190 2023-11-28T03:11:04.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 -- 172.21.15.73:0/1249499278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 msgr2=0x7f65640c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 --2- 172.21.15.73:0/1249499278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f65640c7760 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 rx=0x7f65640657a0 tx=0x7f65540090f0).stop 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 -- 172.21.15.73:0/1249499278 shutdown_connections 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 --2- 172.21.15.73:0/1249499278 >> v2:172.21.15.150:3300/0 conn(0x7f6564158320 0x7f65641587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 --2- 172.21.15.73:0/1249499278 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f65640c7760 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 -- 172.21.15.73:0/1249499278 >> 172.21.15.73:0/1249499278 conn(0x7f65640cb310 msgr2=0x7f65640cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 -- 172.21.15.73:0/1249499278 shutdown_connections 2023-11-28T03:11:04.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.356+0000 7f656a7a7700 1 -- 172.21.15.73:0/1249499278 wait complete. 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.357+0000 7f656a7a7700 1 Processor -- start 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.357+0000 7f656a7a7700 1 -- start start 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 0x7f6564154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6564158320 0x7f65640c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564164af0 con 0x7f65640c6190 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6564156520 con 0x7f6564158320 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f656a7a7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6564155560 con 0x7f6564098d10 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33634/0 (socket says 172.21.15.73:33634) 2023-11-28T03:11:04.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 -- 172.21.15.73:0/2199969768 learned_addr learned my addr 172.21.15.73:0/2199969768 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f6568d44700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6564158320 0x7f65640c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f6563fff700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 0x7f6564154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 msgr2=0x7f6564154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 0x7f6564154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6564158320 msgr2=0x7f65640c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6564158320 0x7f65640c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 -- 172.21.15.73:0/2199969768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6554009370 con 0x7f65640c6190 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f6563fff700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 0x7f6564154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.358+0000 7f65637fe700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f6554009130 tx=0x7f6554017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.359+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655400be90 con 0x7f65640c6190 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.359+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f655400b7f0 con 0x7f65640c6190 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.359+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f655401c670 con 0x7f65640c6190 2023-11-28T03:11:04.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.359+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6564155840 con 0x7f65640c6190 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.359+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6564155240 con 0x7f65640c6190 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.360+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6564155d60 con 0x7f65640c6190 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.360+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f655401ca20 con 0x7f65640c6190 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.360+0000 7f655bfff700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 0x7f6544061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.361+0000 7f6563fff700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 msgr2=0x7f6544061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.361+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f655407c3a0 con 0x7f65640c6190 2023-11-28T03:11:04.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.361+0000 7f6563fff700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 0x7f6544061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:04.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.365+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f655404d2c0 con 0x7f65640c6190 2023-11-28T03:11:04.420 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:04.419+0000 7ff428cdd680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:11:04.463 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:04.462+0000 7faae096b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:11:04.492 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:04.491+0000 7ff428cdd680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:11:04.536 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:04.535+0000 7faae096b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:11:04.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.562+0000 7f6563fff700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 msgr2=0x7f6544061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:04.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.562+0000 7f6563fff700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 0x7f6544061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:04.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.569+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f656406c860 con 0x7f65640c6190 2023-11-28T03:11:04.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.570+0000 7f655bfff700 1 -- 172.21.15.73:0/2199969768 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v127) v1 ==== 110+0+10 (secure 0 0 0) 0x7f655402a0b0 con 0x7f65640c6190 2023-11-28T03:11:04.571 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:11:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 msgr2=0x7f6544061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 0x7f6544061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 msgr2=0x7f6564147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 rx=0x7f6554009130 tx=0x7f6554017000).stop 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 shutdown_connections 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f654405f360 0x7f6544061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6564158320 0x7f65640c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6564098d10 0x7f6564154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 --2- 172.21.15.73:0/2199969768 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f65640c6190 0x7f6564147030 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 >> 172.21.15.73:0/2199969768 conn(0x7f65640cb310 msgr2=0x7f6564153000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 shutdown_connections 2023-11-28T03:11:04.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.572+0000 7f656a7a7700 1 -- 172.21.15.73:0/2199969768 wait complete. 2023-11-28T03:11:04.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:11:04.679 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:04.678+0000 7ff428cdd680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:11:04.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.706+0000 7f2c89aff700 1 Processor -- start 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.706+0000 7f2c89aff700 1 -- start start 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c89aff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c840c6b30 0x7f2c84155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c89aff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c840c2ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c89aff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c84155fd0 con 0x7f2c841524b0 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c89aff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c8414bff0 con 0x7f2c840c6b30 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c89aff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c84155650 con 0x7f2c840bde50 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c837fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c840c6b30 0x7f2c84155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c837fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c840c6b30 0x7f2c84155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33646/0 (socket says 172.21.15.73:33646) 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c837fe700 1 -- 172.21.15.73:0/3106668998 learned_addr learned my addr 172.21.15.73:0/3106668998 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.707+0000 7f2c83fff700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c840c2ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c837fe700 1 -- 172.21.15.73:0/3106668998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 msgr2=0x7f2c840c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c837fe700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c840c2ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c837fe700 1 -- 172.21.15.73:0/3106668998 >> v1:172.21.15.73:6789/0 conn(0x7f2c841524b0 legacy=0x7f2c840bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c837fe700 1 -- 172.21.15.73:0/3106668998 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c84063b60 con 0x7f2c840c6b30 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c837fe700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c840c6b30 0x7f2c84155ae0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f2c78009810 tx=0x7f2c78009060).ready entity=mon.0 client_cookie=8bcf3faff781d3d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c827fc700 1 -- 172.21.15.73:0/3106668998 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78005880 con 0x7f2c840c6b30 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c827fc700 1 -- 172.21.15.73:0/3106668998 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c780059e0 con 0x7f2c840c6b30 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c827fc700 1 -- 172.21.15.73:0/3106668998 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78005d60 con 0x7f2c840c6b30 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c89aff700 1 -- 172.21.15.73:0/3106668998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c840c6b30 msgr2=0x7f2c84155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.708+0000 7f2c89aff700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c840c6b30 0x7f2c84155ae0 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 rx=0x7f2c78009810 tx=0x7f2c78009060).stop 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 -- 172.21.15.73:0/3106668998 shutdown_connections 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c840c2ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 --2- 172.21.15.73:0/3106668998 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c840c6b30 0x7f2c84155ae0 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 -- 172.21.15.73:0/3106668998 >> 172.21.15.73:0/3106668998 conn(0x7f2c840c8230 msgr2=0x7f2c840c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 -- 172.21.15.73:0/3106668998 shutdown_connections 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.709+0000 7f2c89aff700 1 -- 172.21.15.73:0/3106668998 wait complete. 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.710+0000 7f2c89aff700 1 Processor -- start 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.710+0000 7f2c89aff700 1 -- start start 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c84165940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c840c6b30 0x7f2c84062700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840630e0 con 0x7f2c841524b0 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840bf570 con 0x7f2c840c6b30 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c840be9a0 con 0x7f2c840bde50 2023-11-28T03:11:04.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c83fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c83fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33648/0 (socket says 172.21.15.73:33648) 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c84165940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c83fff700 1 -- 172.21.15.73:0/2490545326 learned_addr learned my addr 172.21.15.73:0/2490545326 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c82ffd700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c840c6b30 0x7f2c84062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 -- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c840c6b30 msgr2=0x7f2c84062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c840c6b30 0x7f2c84062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 -- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 msgr2=0x7f2c84062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 -- 172.21.15.73:0/2490545326 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c78009300 con 0x7f2c840bde50 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c83fff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c837fe700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c84165940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f2c78013000 tx=0x7f2c78016000).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:04.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c7801f070 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c7800b580 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c78005880 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.711+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c840bec40 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.712+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c840bf200 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.712+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c8406c860 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.713+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f2c78018070 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.713+0000 7f2c6b7fe700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 0x7f2c70061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.713+0000 7f2c82ffd700 1 -- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 msgr2=0x7f2c70061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.713+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2c7807a800 con 0x7f2c840bde50 2023-11-28T03:11:04.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.713+0000 7f2c82ffd700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 0x7f2c70061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:04.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.717+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c7804b720 con 0x7f2c840bde50 2023-11-28T03:11:04.733 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:04.732+0000 7faae096b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:11:04.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.888+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f2c84063e40 con 0x7f2c840bde50 2023-11-28T03:11:04.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.889+0000 7f2c6b7fe700 1 -- 172.21.15.73:0/2490545326 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v127) v1 ==== 92+0+10 (secure 0 0 0) 0x7f2c7804ed40 con 0x7f2c840bde50 2023-11-28T03:11:04.890 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:11:04.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 msgr2=0x7f2c70061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:04.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 0x7f2c70061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 msgr2=0x7f2c84165940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c84165940 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 rx=0x7f2c78013000 tx=0x7f2c78016000).stop 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 shutdown_connections 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c7005f390 0x7f2c70061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c841524b0 0x7f2c84062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c840bde50 0x7f2c84165940 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 --2- 172.21.15.73:0/2490545326 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c840c6b30 0x7f2c84062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 >> 172.21.15.73:0/2490545326 conn(0x7f2c840c8230 msgr2=0x7f2c8414b350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 shutdown_connections 2023-11-28T03:11:04.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:04.891+0000 7f2c89aff700 1 -- 172.21.15.73:0/2490545326 wait complete. 2023-11-28T03:11:04.906 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.028+0000 7fa168bd5700 1 Processor -- start 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.028+0000 7fa168bd5700 1 -- start start 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa164098d10 0x7fa1640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1640c6190 0x7fa164063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1640c72d0 con 0x7fa1640c6190 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa164154bc0 con 0x7fa164098d10 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa168bd5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c66c0 con 0x7fa164064350 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa161d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1640c6190 0x7fa164063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa161d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa1640c6190 0x7fa164063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33650/0 (socket says 172.21.15.73:33650) 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.029+0000 7fa161d9b700 1 -- 172.21.15.73:0/3447714255 learned_addr learned my addr 172.21.15.73:0/3447714255 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa16259c700 1 --2- 172.21.15.73:0/3447714255 >> v2:172.21.15.150:3300/0 conn(0x7fa164098d10 0x7fa1640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa162d9d700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 -- 172.21.15.73:0/3447714255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 msgr2=0x7fa164158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 -- 172.21.15.73:0/3447714255 >> v2:172.21.15.150:3300/0 conn(0x7fa164098d10 msgr2=0x7fa1640c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 --2- 172.21.15.73:0/3447714255 >> v2:172.21.15.150:3300/0 conn(0x7fa164098d10 0x7fa1640c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 -- 172.21.15.73:0/3447714255 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1640c69a0 con 0x7fa1640c6190 2023-11-28T03:11:05.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa161d9b700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164063e60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fa164083f80 tx=0x7fa1540090f0).ready entity=mon.0 client_cookie=963d12f2ae5c38e5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa16159a700 1 -- 172.21.15.73:0/3447714255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400f510 con 0x7fa1640c6190 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa16159a700 1 -- 172.21.15.73:0/3447714255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa15400f670 con 0x7fa1640c6190 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.030+0000 7fa16159a700 1 -- 172.21.15.73:0/3447714255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15400f9f0 con 0x7fa1640c6190 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 -- 172.21.15.73:0/3447714255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 msgr2=0x7fa164063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164063e60 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 rx=0x7fa164083f80 tx=0x7fa1540090f0).stop 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 -- 172.21.15.73:0/3447714255 shutdown_connections 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 --2- 172.21.15.73:0/3447714255 >> v2:172.21.15.150:3300/0 conn(0x7fa164098d10 0x7fa1640c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 --2- 172.21.15.73:0/3447714255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164063e60 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 -- 172.21.15.73:0/3447714255 >> 172.21.15.73:0/3447714255 conn(0x7fa1640cb310 msgr2=0x7fa1640cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 -- 172.21.15.73:0/3447714255 shutdown_connections 2023-11-28T03:11:05.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.031+0000 7fa168bd5700 1 -- 172.21.15.73:0/3447714255 wait complete. 2023-11-28T03:11:05.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.032+0000 7fa168bd5700 1 Processor -- start 2023-11-28T03:11:05.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.032+0000 7fa168bd5700 1 -- start start 2023-11-28T03:11:05.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 0x7fa164140f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa164154dc0 con 0x7fa1640c6190 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c2660 con 0x7fa164098d10 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1640c4b20 con 0x7fa164064350 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa162d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa162d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33658/0 (socket says 172.21.15.73:33658) 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa162d9d700 1 -- 172.21.15.73:0/470015421 learned_addr learned my addr 172.21.15.73:0/470015421 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 0x7fa164140f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa16259c700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 -- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 msgr2=0x7fa164157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 -- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 msgr2=0x7fa164153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 -- 172.21.15.73:0/470015421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa154009370 con 0x7fa164098d10 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa162d9d700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa161d9b700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 0x7fa164140f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fa154014000 tx=0x7fa154017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa154011b40 con 0x7fa164098d10 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa154011cc0 con 0x7fa164098d10 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1640c4e00 con 0x7fa164098d10 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.033+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa15401c670 con 0x7fa164098d10 2023-11-28T03:11:05.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.034+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa164155010 con 0x7fa164098d10 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa164155380 con 0x7fa164098d10 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa15401ca60 con 0x7fa164098d10 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa1527fc700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 0x7fa144061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa16259c700 1 -- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 msgr2=0x7fa144061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa15407b590 con 0x7fa164098d10 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.035+0000 7fa16259c700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 0x7fa144061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:05.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.040+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa15404c560 con 0x7fa164098d10 2023-11-28T03:11:05.101 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.100+0000 7ff428cdd680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:11:05.173 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.172+0000 7faae096b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:11:05.205 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.204+0000 7ff428cdd680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:05.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.210+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa16406c860 con 0x7fa164098d10 2023-11-28T03:11:05.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.211+0000 7fa1527fc700 1 -- 172.21.15.73:0/470015421 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v127) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa15404fb80 con 0x7fa164098d10 2023-11-28T03:11:05.212 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.212+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 msgr2=0x7fa144061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.212+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 0x7fa144061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.212+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 msgr2=0x7fa164140f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.212+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 0x7fa164140f00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 rx=0x7fa154014000 tx=0x7fa154017000).stop 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 shutdown_connections 2023-11-28T03:11:05.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa14405f390 0x7fa144061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa1640c6190 0x7fa164153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa164064350 0x7fa164157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 --2- 172.21.15.73:0/470015421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa164098d10 0x7fa164140f00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 >> 172.21.15.73:0/470015421 conn(0x7fa1640cb310 msgr2=0x7fa164158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 shutdown_connections 2023-11-28T03:11:05.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.213+0000 7fa168bd5700 1 -- 172.21.15.73:0/470015421 wait complete. 2023-11-28T03:11:05.226 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T03:11:05.283 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.282+0000 7faae096b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:05.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.344+0000 7fe8c1d48700 1 Processor -- start 2023-11-28T03:11:05.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.344+0000 7fe8c1d48700 1 -- start start 2023-11-28T03:11:05.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8c1d48700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe8bc0a5ed0 0x7fe8bc0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8c1d48700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc067680 con 0x7fe8bc1559e0 2023-11-28T03:11:05.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8c1d48700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8bc155dc0 con 0x7fe8bc0be7d0 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8c1d48700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8bc065f60 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bb7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe8bc1559e0 0x7fe8bc0be0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51194/0 (socket says 172.21.15.73:51194) 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bb7fe700 1 -- 172.21.15.73:0/1339528938 learned_addr learned my addr 172.21.15.73:0/1339528938 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bbfff700 1 --2- 172.21.15.73:0/1339528938 >> v2:172.21.15.150:3300/0 conn(0x7fe8bc0a5ed0 0x7fe8bc0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1113494684 0 0) 0x7fe8bc067680 con 0x7fe8bc1559e0 2023-11-28T03:11:05.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8a80033f0 con 0x7fe8bc1559e0 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2355454694 0 0) 0x7fe8bc155dc0 con 0x7fe8bc0be7d0 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe8bc067680 con 0x7fe8bc0be7d0 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bbfff700 1 -- 172.21.15.73:0/1339528938 >> v1:172.21.15.150:6789/0 conn(0x7fe8bc0be7d0 legacy=0x7fe8bc0c49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bbfff700 1 -- 172.21.15.73:0/1339528938 >> v1:172.21.15.73:6789/0 conn(0x7fe8bc1559e0 legacy=0x7fe8bc0be0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bbfff700 1 -- 172.21.15.73:0/1339528938 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8bc066a70 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.346+0000 7fe8bbfff700 1 --2- 172.21.15.73:0/1339528938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0c50d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe8b4009e60 tx=0x7fe8b4009780).ready entity=mon.1 client_cookie=f6264ec641662781 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400e070 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8b400bab0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8baffd700 1 -- 172.21.15.73:0/1339528938 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b400be30 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 -- 172.21.15.73:0/1339528938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc0a5ed0 msgr2=0x7fe8bc0c50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/1339528938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0c50d0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 rx=0x7fe8b4009e60 tx=0x7fe8b4009780).stop 2023-11-28T03:11:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 -- 172.21.15.73:0/1339528938 shutdown_connections 2023-11-28T03:11:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/1339528938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0c50d0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 -- 172.21.15.73:0/1339528938 >> 172.21.15.73:0/1339528938 conn(0x7fe8bc0631d0 msgr2=0x7fe8bc0c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 -- 172.21.15.73:0/1339528938 shutdown_connections 2023-11-28T03:11:05.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.347+0000 7fe8c1d48700 1 -- 172.21.15.73:0/1339528938 wait complete. 2023-11-28T03:11:05.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.349+0000 7fe8c1d48700 1 Processor -- start 2023-11-28T03:11:05.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.349+0000 7fe8c1d48700 1 -- start start 2023-11-28T03:11:05.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8bc0be7d0 0x7fe8bc0c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 0x7fe8bc0c2f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0ac6e0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0c3470 con 0x7fe8bc1559e0 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.350+0000 7fe8c1d48700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8bc0aada0 con 0x7fe8bc0be7d0 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33668/0 (socket says 172.21.15.73:33668) 2023-11-28T03:11:05.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 -- 172.21.15.73:0/2575154303 learned_addr learned my addr 172.21.15.73:0/2575154303 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8b2ffd700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8bc0be7d0 0x7fe8bc0c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bbfff700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 0x7fe8bc0c2f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 -- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8bc0be7d0 msgr2=0x7fe8bc0c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8bc0be7d0 0x7fe8bc0c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 -- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 msgr2=0x7fe8bc0c2f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 0x7fe8bc0c2f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 -- 172.21.15.73:0/2575154303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b4009a20 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bbfff700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 0x7fe8bc0c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8bb7fe700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fe8ac00c560 tx=0x7fe8ac007d00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac00ce70 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8bc0aa500 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8ac0072e0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8ac007660 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.351+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8bc0aaac0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.352+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8bc06c860 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.353+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fe8ac0059b0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.353+0000 7fe8b3fff700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 0x7fe89c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.353+0000 7fe8b2ffd700 1 -- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 msgr2=0x7fe89c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:05.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.353+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe8ac079cd0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.353+0000 7fe8b2ffd700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 0x7fe89c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:05.363 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.359+0000 7ff428cdd680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:05.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.360+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe8ac04abf0 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.435 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.434+0000 7ff428cdd680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:11:05.446 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.445+0000 7faae096b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:05.502 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.501+0000 7ff428cdd680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:11:05.523 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.522+0000 7faae096b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:11:05.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.531+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe8bc066d50 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.531+0000 7fe8b3fff700 1 -- 172.21.15.73:0/2575154303 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v127) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe8ac018030 con 0x7fe8bc0a5ed0 2023-11-28T03:11:05.532 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T03:11:05.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 msgr2=0x7fe89c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:05.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 0x7fe89c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 msgr2=0x7fe8bc0ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 rx=0x7fe8ac00c560 tx=0x7fe8ac007d00).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 shutdown_connections 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe89c05f390 0x7fe89c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe8bc1559e0 0x7fe8bc0c2f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe8bc0a5ed0 0x7fe8bc0ae620 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 --2- 172.21.15.73:0/2575154303 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe8bc0be7d0 0x7fe8bc0c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 >> 172.21.15.73:0/2575154303 conn(0x7fe8bc0631d0 msgr2=0x7fe8bc0c7480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 shutdown_connections 2023-11-28T03:11:05.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.533+0000 7fe8c1d48700 1 -- 172.21.15.73:0/2575154303 wait complete. 2023-11-28T03:11:05.547 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:05.596 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.594+0000 7faae096b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:11:05.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.661+0000 7f512f59e700 1 Processor -- start 2023-11-28T03:11:05.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.661+0000 7f512f59e700 1 -- start start 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512f59e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5130157660 0x7f5130157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f51300c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512f59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51300c12c0 con 0x7f51300c0180 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512f59e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51300c06b0 con 0x7f5130157660 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51300c0990 con 0x7f51300c1750 2023-11-28T03:11:05.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512e59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5130157660 0x7f5130157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512e59c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5130157660 0x7f5130157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33670/0 (socket says 172.21.15.73:33670) 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.662+0000 7f512e59c700 1 -- 172.21.15.73:0/1709203802 learned_addr learned my addr 172.21.15.73:0/1709203802 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512ed9d700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f51300c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512e59c700 1 -- 172.21.15.73:0/1709203802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 msgr2=0x7f51300c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512e59c700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f51300c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512e59c700 1 -- 172.21.15.73:0/1709203802 >> v1:172.21.15.150:6789/0 conn(0x7f51300c0180 legacy=0x7f51301589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512e59c700 1 -- 172.21.15.73:0/1709203802 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51300c0c70 con 0x7f5130157660 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512e59c700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f5130157a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f5124006fd0 tx=0x7f51240090f0).ready entity=mon.0 client_cookie=6ddb94911e80ddd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512d59a700 1 -- 172.21.15.73:0/1709203802 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400b580 con 0x7f5130157660 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512d59a700 1 -- 172.21.15.73:0/1709203802 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512400b6e0 con 0x7f5130157660 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512d59a700 1 -- 172.21.15.73:0/1709203802 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512400ba60 con 0x7f5130157660 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.663+0000 7f512f59e700 1 -- 172.21.15.73:0/1709203802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 msgr2=0x7f5130157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f5130157a40 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 rx=0x7f5124006fd0 tx=0x7f51240090f0).stop 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 -- 172.21.15.73:0/1709203802 shutdown_connections 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f51300c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 --2- 172.21.15.73:0/1709203802 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f5130157a40 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 -- 172.21.15.73:0/1709203802 >> 172.21.15.73:0/1709203802 conn(0x7f51300cb000 msgr2=0x7f51300cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 -- 172.21.15.73:0/1709203802 shutdown_connections 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.664+0000 7f512f59e700 1 -- 172.21.15.73:0/1709203802 wait complete. 2023-11-28T03:11:05.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.665+0000 7f512f59e700 1 Processor -- start 2023-11-28T03:11:05.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.665+0000 7f512f59e700 1 -- start start 2023-11-28T03:11:05.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 0x7f51301514c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f5130167ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5130153f50 con 0x7f5130157660 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51301519b0 con 0x7f51300c0180 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5130168e00 con 0x7f51300c1750 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33684/0 (socket says 172.21.15.73:33684) 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512e59c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 0x7f51301514c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 -- 172.21.15.73:0/78481108 learned_addr learned my addr 172.21.15.73:0/78481108 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512dd9b700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f5130167ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 msgr2=0x7f5130167ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f5130167ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 msgr2=0x7f51301514c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 0x7f51301514c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 -- 172.21.15.73:0/78481108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5124009370 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512e59c700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 0x7f51301514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512ed9d700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f5128009e90 tx=0x7f512800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800caa0 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f512800f040 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5130168840 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512800ce10 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.666+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5130153be0 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f513006c7c0 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f5128013610 con 0x7f5130157660 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f511e7fc700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 0x7f5110061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:05.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f512e59c700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 msgr2=0x7f5110061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:05.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f512807af00 con 0x7f5130157660 2023-11-28T03:11:05.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.668+0000 7f512e59c700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 0x7f5110061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:05.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.672+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f512804bd10 con 0x7f5130157660 2023-11-28T03:11:05.762 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.761+0000 7ff428cdd680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:11:05.864 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:05.863+0000 7faae096b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:11:05.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.869+0000 7f512e59c700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 msgr2=0x7f5110061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:05.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.869+0000 7f512e59c700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 0x7f5110061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:05.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.885+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f51300c0f50 con 0x7f5130157660 2023-11-28T03:11:05.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.886+0000 7f511e7fc700 1 -- 172.21.15.73:0/78481108 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f512804f330 con 0x7f5130157660 2023-11-28T03:11:05.887 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:05.887 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:11:05.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 msgr2=0x7f5110061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:05.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 0x7f5110061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 msgr2=0x7f51300aa070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 rx=0x7f5128009e90 tx=0x7f512800b040).stop 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 shutdown_connections 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f511005f390 0x7f5110061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5130157660 0x7f51300aa070 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51300c0180 0x7f51301514c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 --2- 172.21.15.73:0/78481108 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51300c1750 0x7f5130167ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 >> 172.21.15.73:0/78481108 conn(0x7f51300cb000 msgr2=0x7f51300aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 shutdown_connections 2023-11-28T03:11:05.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:05.888+0000 7f512f59e700 1 -- 172.21.15.73:0/78481108 wait complete. 2023-11-28T03:11:05.891 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:11:05.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:05.945 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:05.944+0000 7ff428cdd680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:06.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.020+0000 7f11f84ff700 1 Processor -- start 2023-11-28T03:11:06.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.020+0000 7f11f84ff700 1 -- start start 2023-11-28T03:11:06.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.021+0000 7f11f84ff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f11f006b510 0x7f11f01546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.021+0000 7f11f84ff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11f00c10f0 0x7f11f0158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.021+0000 7f11f84ff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f0063380 con 0x7f11f00c0d10 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.021+0000 7f11f84ff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f00c19e0 con 0x7f11f00c10f0 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.021+0000 7f11f84ff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f0159060 con 0x7f11f006b510 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11f00c10f0 0x7f11f0158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11f00c10f0 0x7f11f0158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33698/0 (socket says 172.21.15.73:33698) 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f629b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f11f00c0d10 0x7f11f0153fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51206/0 (socket says 172.21.15.73:51206) 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 -- 172.21.15.73:0/2746087183 learned_addr learned my addr 172.21.15.73:0/2746087183 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f5a9a700 1 --2- 172.21.15.73:0/2746087183 >> v2:172.21.15.150:3300/0 conn(0x7f11f006b510 0x7f11f01546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 -- 172.21.15.73:0/2746087183 >> v2:172.21.15.150:3300/0 conn(0x7f11f006b510 msgr2=0x7f11f01546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:06.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 --2- 172.21.15.73:0/2746087183 >> v2:172.21.15.150:3300/0 conn(0x7f11f006b510 0x7f11f01546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 -- 172.21.15.73:0/2746087183 >> v1:172.21.15.73:6789/0 conn(0x7f11f00c0d10 legacy=0x7f11f0153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f5299700 1 -- 172.21.15.73:0/2746087183 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2661800000 0 0) 0x7f11f0063380 con 0x7f11f00c0d10 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 -- 172.21.15.73:0/2746087183 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f0062770 con 0x7f11f00c10f0 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f6a9c700 1 --2- 172.21.15.73:0/2746087183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f0158b70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f11e8009f20 tx=0x7f11e8009840).ready entity=mon.0 client_cookie=2a27635257365a3d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f5299700 1 -- 172.21.15.73:0/2746087183 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800e070 con 0x7f11f00c10f0 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f5299700 1 -- 172.21.15.73:0/2746087183 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800b990 con 0x7f11f00c10f0 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.022+0000 7f11f5299700 1 -- 172.21.15.73:0/2746087183 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800bd10 con 0x7f11f00c10f0 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 -- 172.21.15.73:0/2746087183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 msgr2=0x7f11f0158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 --2- 172.21.15.73:0/2746087183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f0158b70 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 rx=0x7f11e8009f20 tx=0x7f11e8009840).stop 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 -- 172.21.15.73:0/2746087183 shutdown_connections 2023-11-28T03:11:06.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 --2- 172.21.15.73:0/2746087183 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f0158b70 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 --2- 172.21.15.73:0/2746087183 >> v2:172.21.15.150:3300/0 conn(0x7f11f006b510 0x7f11f01546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 -- 172.21.15.73:0/2746087183 >> 172.21.15.73:0/2746087183 conn(0x7f11f00c7760 msgr2=0x7f11f00c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 -- 172.21.15.73:0/2746087183 shutdown_connections 2023-11-28T03:11:06.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.023+0000 7f11f84ff700 1 -- 172.21.15.73:0/2746087183 wait complete. 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.024+0000 7f11f84ff700 1 Processor -- start 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.024+0000 7f11f84ff700 1 -- start start 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f006b510 0x7f11f01368a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f00c0d10 0x7f11f015a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01592c0 con 0x7f11f00c10f0 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01595a0 con 0x7f11f00c0d10 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f84ff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0159880 con 0x7f11f006b510 2023-11-28T03:11:06.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33712/0 (socket says 172.21.15.73:33712) 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 -- 172.21.15.73:0/900977217 learned_addr learned my addr 172.21.15.73:0/900977217 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f629b700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f006b510 0x7f11f01368a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f5a9a700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f00c0d10 0x7f11f015a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 -- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f006b510 msgr2=0x7f11f01368a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f006b510 0x7f11f01368a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 -- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f00c0d10 msgr2=0x7f11f015a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f00c0d10 0x7f11f015a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 -- 172.21.15.73:0/900977217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e8009ae0 con 0x7f11f00c10f0 2023-11-28T03:11:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11f6a9c700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f11e800b610 tx=0x7f11e8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.025+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e800bd10 con 0x7f11f00c10f0 2023-11-28T03:11:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.026+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e800bec0 con 0x7f11f00c10f0 2023-11-28T03:11:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.026+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f0159b60 con 0x7f11f00c10f0 2023-11-28T03:11:06.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.026+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e801c610 con 0x7f11f00c10f0 2023-11-28T03:11:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.026+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f015a120 con 0x7f11f00c10f0 2023-11-28T03:11:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.027+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f006c860 con 0x7f11f00c10f0 2023-11-28T03:11:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.027+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f11e801ca00 con 0x7f11f00c10f0 2023-11-28T03:11:06.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.028+0000 7f11e67fc700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 0x7f11d4061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.028+0000 7f11f629b700 1 -- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 msgr2=0x7f11d4061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.028+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f11e807c000 con 0x7f11f00c10f0 2023-11-28T03:11:06.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.028+0000 7f11f629b700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 0x7f11d4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:06.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.032+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11e804ce10 con 0x7f11f00c10f0 2023-11-28T03:11:06.057 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:06.056+0000 7faae096b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:06.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.203+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f11f0062a50 con 0x7f11f00c10f0 2023-11-28T03:11:06.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.204+0000 7f11e67fc700 1 -- 172.21.15.73:0/900977217 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17186 (secure 0 0 0) 0x7f11e8023310 con 0x7f11f00c10f0 2023-11-28T03:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:06.205 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":127,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:10:03.562867+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:6832/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/3114748317":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/1437075559":"2023-11-29T03:10:03.562840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 msgr2=0x7f11d4061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 0x7f11d4061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 msgr2=0x7f11f014ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 rx=0x7f11e800b610 tx=0x7f11e8009840).stop 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 shutdown_connections 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f11d405f390 0x7f11d4061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11f00c10f0 0x7f11f014ac10 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11f006b510 0x7f11f01368a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 --2- 172.21.15.73:0/900977217 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11f00c0d10 0x7f11f015a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 >> 172.21.15.73:0/900977217 conn(0x7f11f00c7760 msgr2=0x7f11f00c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 shutdown_connections 2023-11-28T03:11:06.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.206+0000 7f11f84ff700 1 -- 172.21.15.73:0/900977217 wait complete. 2023-11-28T03:11:06.219 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T03:11:06.220 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:06.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.335+0000 7f3ed97d6700 1 Processor -- start 2023-11-28T03:11:06.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.335+0000 7f3ed97d6700 1 -- start start 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3ed4067cb0 0x7f3ed40c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4068090 0x7f3ed40c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3ed40c2640 0x7f3ed40c2ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed4151670 con 0x7f3ed40c2640 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ed40c48a0 con 0x7f3ed4067cb0 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed97d6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed4154d20 con 0x7f3ed4068090 2023-11-28T03:11:06.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3ed40c2640 0x7f3ed40c2ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3ed40c2640 0x7f3ed40c2ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33720/0 (socket says 172.21.15.73:33720) 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed37fe700 1 -- 172.21.15.73:0/2501662240 learned_addr learned my addr 172.21.15.73:0/2501662240 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.336+0000 7f3ed27fc700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4068090 0x7f3ed40c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed2ffd700 1 --2- 172.21.15.73:0/2501662240 >> v2:172.21.15.150:3300/0 conn(0x7f3ed4067cb0 0x7f3ed40c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 -- 172.21.15.73:0/2501662240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4068090 msgr2=0x7f3ed40c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4068090 0x7f3ed40c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 -- 172.21.15.73:0/2501662240 >> v2:172.21.15.150:3300/0 conn(0x7f3ed4067cb0 msgr2=0x7f3ed40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 --2- 172.21.15.73:0/2501662240 >> v2:172.21.15.150:3300/0 conn(0x7f3ed4067cb0 0x7f3ed40c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 -- 172.21.15.73:0/2501662240 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed41555c0 con 0x7f3ed40c2640 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed37fe700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40c2640 0x7f3ed40c2ad0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f3ed40c5b30 tx=0x7f3ecc009840).ready entity=mon.0 client_cookie=eabdfad392487e8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed1ffb700 1 -- 172.21.15.73:0/2501662240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00e070 con 0x7f3ed40c2640 2023-11-28T03:11:06.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed1ffb700 1 -- 172.21.15.73:0/2501662240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ecc00b970 con 0x7f3ed40c2640 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed1ffb700 1 -- 172.21.15.73:0/2501662240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ecc00bcf0 con 0x7f3ed40c2640 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed97d6700 1 -- 172.21.15.73:0/2501662240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40c2640 msgr2=0x7f3ed40c2ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.337+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40c2640 0x7f3ed40c2ad0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 rx=0x7f3ed40c5b30 tx=0x7f3ecc009840).stop 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 -- 172.21.15.73:0/2501662240 shutdown_connections 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed40c2640 0x7f3ed40c2ad0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/2501662240 >> v2:172.21.15.150:3300/0 conn(0x7f3ed4067cb0 0x7f3ed40c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/2501662240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4068090 0x7f3ed40c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 -- 172.21.15.73:0/2501662240 >> 172.21.15.73:0/2501662240 conn(0x7f3ed40c7fd0 msgr2=0x7f3ed40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 -- 172.21.15.73:0/2501662240 shutdown_connections 2023-11-28T03:11:06.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.338+0000 7f3ed97d6700 1 -- 172.21.15.73:0/2501662240 wait complete. 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.339+0000 7f3ed97d6700 1 Processor -- start 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.339+0000 7f3ed97d6700 1 -- start start 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4067cb0 0x7f3ed40c1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ed40c2640 0x7f3ed40ab900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed41635c0 con 0x7f3ed4068090 2023-11-28T03:11:06.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40ab470 con 0x7f3ed40c2640 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed40c22c0 con 0x7f3ed4067cb0 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33722/0 (socket says 172.21.15.73:33722) 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 -- 172.21.15.73:0/1747554328 learned_addr learned my addr 172.21.15.73:0/1747554328 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed2ffd700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4067cb0 0x7f3ed40c1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed37fe700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ed40c2640 0x7f3ed40ab900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4067cb0 msgr2=0x7f3ed40c1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4067cb0 0x7f3ed40c1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ed40c2640 msgr2=0x7f3ed40ab900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ed40c2640 0x7f3ed40ab900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 -- 172.21.15.73:0/1747554328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ecc009ae0 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed27fc700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f3ec400e950 tx=0x7f3ec4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4012600 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ec4012760 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ed4163000 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ec4012ae0 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.340+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ed40ab100 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ed40aaf60 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3ec401b400 con 0x7f3ed4068090 2023-11-28T03:11:06.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ebaffd700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 0x7f3ec0061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ed2ffd700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 msgr2=0x7f3ec0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f3ec407ab40 con 0x7f3ed4068090 2023-11-28T03:11:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.342+0000 7f3ed2ffd700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 0x7f3ec0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:06.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.346+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3ed40aaf60 con 0x7f3ed4068090 2023-11-28T03:11:06.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.543+0000 7f3ed2ffd700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 msgr2=0x7f3ec0061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:06.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.543+0000 7f3ed2ffd700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 0x7f3ec0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:06.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.562+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3ed40aaf60 con 0x7f3ed4068090 2023-11-28T03:11:06.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.562+0000 7f3ebaffd700 1 -- 172.21.15.73:0/1747554328 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f3ed40aaf60 con 0x7f3ed4068090 2023-11-28T03:11:06.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:06.564 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.564+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 msgr2=0x7f3ec0061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.564+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 0x7f3ec0061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.564+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 msgr2=0x7f3ed41625e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.564+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 rx=0x7f3ec400e950 tx=0x7f3ec4009060).stop 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 shutdown_connections 2023-11-28T03:11:06.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3ec005f390 0x7f3ec0061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3ed40c2640 0x7f3ed40ab900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3ed4067cb0 0x7f3ed40c1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 --2- 172.21.15.73:0/1747554328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3ed4068090 0x7f3ed41625e0 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 >> 172.21.15.73:0/1747554328 conn(0x7f3ed40c7fd0 msgr2=0x7f3ed414f750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 shutdown_connections 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.565+0000 7f3ed97d6700 1 -- 172.21.15.73:0/1747554328 wait complete. 2023-11-28T03:11:06.567 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:11:06.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:06.601 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:06.600+0000 7ff428cdd680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:11:06.669 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:06.668+0000 7ff428cdd680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:11:06.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.698+0000 7f2f0ae1c700 1 Processor -- start 2023-11-28T03:11:06.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.698+0000 7f2f0ae1c700 1 -- start start 2023-11-28T03:11:06.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f0ae1c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f0ae1c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f040630a0 con 0x7f2f040661b0 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f0ae1c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f04065760 con 0x7f2f041542d0 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f0ae1c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f040c3250 con 0x7f2f04062700 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f03fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f03fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33726/0 (socket says 172.21.15.73:33726) 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.699+0000 7f2f03fff700 1 -- 172.21.15.73:0/3187736236 learned_addr learned my addr 172.21.15.73:0/3187736236 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f03fff700 1 -- 172.21.15.73:0/3187736236 >> v1:172.21.15.150:6789/0 conn(0x7f2f041542d0 legacy=0x7f2f040697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f03fff700 1 -- 172.21.15.73:0/3187736236 >> v1:172.21.15.73:6789/0 conn(0x7f2f040661b0 legacy=0x7f2f04066640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f03fff700 1 -- 172.21.15.73:0/3187736236 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f0406a340 con 0x7f2f04062700 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f03fff700 1 --2- 172.21.15.73:0/3187736236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f04062bb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f2ef0006fd0 tx=0x7f2ef00090f0).ready entity=mon.0 client_cookie=ba62c4366afa39d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f037fe700 1 -- 172.21.15.73:0/3187736236 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000b600 con 0x7f2f04062700 2023-11-28T03:11:06.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f037fe700 1 -- 172.21.15.73:0/3187736236 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ef000b760 con 0x7f2f04062700 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f037fe700 1 -- 172.21.15.73:0/3187736236 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000bae0 con 0x7f2f04062700 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/3187736236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 msgr2=0x7f2f04062bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/3187736236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f04062bb0 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 rx=0x7f2ef0006fd0 tx=0x7f2ef00090f0).stop 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/3187736236 shutdown_connections 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/3187736236 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f04062700 0x7f2f04062bb0 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.700+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/3187736236 >> 172.21.15.73:0/3187736236 conn(0x7f2f040c7fd0 msgr2=0x7f2f040c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.701+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/3187736236 shutdown_connections 2023-11-28T03:11:06.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.701+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/3187736236 wait complete. 2023-11-28T03:11:06.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 Processor -- start 2023-11-28T03:11:06.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 -- start start 2023-11-28T03:11:06.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f04062700 0x7f2f040bfa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f040ac110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c0990 con 0x7f2f041542d0 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040a7df0 con 0x7f2f04062700 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.702+0000 7f2f0ae1c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f040c03d0 con 0x7f2f040661b0 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33734/0 (socket says 172.21.15.73:33734) 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 -- 172.21.15.73:0/208494827 learned_addr learned my addr 172.21.15.73:0/208494827 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f08bb8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f04062700 0x7f2f040bfa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f03fff700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f040ac110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 msgr2=0x7f2f040ac110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f040ac110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f04062700 msgr2=0x7f2f040bfa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f04062700 0x7f2f040bfa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 -- 172.21.15.73:0/208494827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef0009370 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f093b9700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f2efc002230 tx=0x7f2efc0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e4a0 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2efc00e600 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2efc00e980 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f040c06b0 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.703+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f0414d1d0 con 0x7f2f041542d0 2023-11-28T03:11:06.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f040a7f60 con 0x7f2f041542d0 2023-11-28T03:11:06.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f2efc01b3b0 con 0x7f2f041542d0 2023-11-28T03:11:06.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f00ff9700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 0x7f2ee8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:06.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f08bb8700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 msgr2=0x7f2ee8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:06.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2efc079af0 con 0x7f2f041542d0 2023-11-28T03:11:06.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.705+0000 7f2f08bb8700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 0x7f2ee8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:06.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.709+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2efc048af0 con 0x7f2f041542d0 2023-11-28T03:11:06.740 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:06.739+0000 7faae096b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:11:06.744 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:06.743+0000 7ff428cdd680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:11:06.812 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:06.811+0000 7faae096b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:11:06.890 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:06.889+0000 7faae096b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:11:06.892 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:06.891+0000 7ff428cdd680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:11:06.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.906+0000 7f2f08bb8700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 msgr2=0x7f2ee8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:06.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.906+0000 7f2f08bb8700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 0x7f2ee8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:06.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.923+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2f040630a0 con 0x7f2f041542d0 2023-11-28T03:11:06.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.924+0000 7f2f00ff9700 1 -- 172.21.15.73:0/208494827 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4494 (secure 0 0 0) 0x7f2f040630a0 con 0x7f2f041542d0 2023-11-28T03:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:06.925 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":46,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:09:47.742412+0000","modified":"2023-11-28T03:09:49.748867+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7596},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7596":{"gid":7596,"name":"c","rank":0,"incarnation":44,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/966840583","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":966840583},{"type":"v1","addr":"172.21.15.73:6837","nonce":966840583}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[16],"metadata_pool":15,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":7}]} 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 msgr2=0x7f2ee8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 0x7f2ee8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 msgr2=0x7f2f040ac600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 rx=0x7f2efc002230 tx=0x7f2efc0015e0).stop 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 shutdown_connections 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ee805f390 0x7f2ee8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2f041542d0 0x7f2f040ac600 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2f04062700 0x7f2f040bfa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 --2- 172.21.15.73:0/208494827 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2f040661b0 0x7f2f040ac110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.925+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 >> 172.21.15.73:0/208494827 conn(0x7f2f040c7fd0 msgr2=0x7f2f040bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.926+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 shutdown_connections 2023-11-28T03:11:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:06.926+0000 7f2f0ae1c700 1 -- 172.21.15.73:0/208494827 wait complete. 2023-11-28T03:11:06.928 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 46 2023-11-28T03:11:06.939 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:06.962 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:06.961+0000 7ff428cdd680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:11:07.046 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.045+0000 7faae096b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:11:07.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.054+0000 7f442ea08700 1 Processor -- start 2023-11-28T03:11:07.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.054+0000 7f442ea08700 1 -- start start 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ea08700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4428065f50 0x7f4428155c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ea08700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 0x7f4428156110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ea08700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44280c2a90 con 0x7f442813b790 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ea08700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4428156bf0 con 0x7f4428065f50 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ea08700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4428156910 con 0x7f44280a7890 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4428065f50 0x7f4428155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4428065f50 0x7f4428155c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33750/0 (socket says 172.21.15.73:33750) 2023-11-28T03:11:07.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 -- 172.21.15.73:0/963912513 learned_addr learned my addr 172.21.15.73:0/963912513 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442e207700 1 --2- 172.21.15.73:0/963912513 >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 0x7f4428156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ca04700 1 -- 172.21.15.73:0/963912513 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3638189090 0 0) 0x7f44280c2a90 con 0x7f442813b790 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442ca04700 1 -- 172.21.15.73:0/963912513 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44140033f0 con 0x7f442813b790 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 -- 172.21.15.73:0/963912513 >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 msgr2=0x7f4428156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 --2- 172.21.15.73:0/963912513 >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 0x7f4428156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 -- 172.21.15.73:0/963912513 >> v1:172.21.15.150:6789/0 conn(0x7f442813b790 legacy=0x7f4428065840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442d205700 1 -- 172.21.15.73:0/963912513 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44280c2440 con 0x7f4428065f50 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.055+0000 7f442e207700 1 --2- 172.21.15.73:0/963912513 >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 0x7f4428156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442d205700 1 --2- 172.21.15.73:0/963912513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4428065f50 0x7f4428155c20 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f44280c2a00 tx=0x7f44240090f0).ready entity=mon.0 client_cookie=40f84de1d7a230ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ca04700 1 -- 172.21.15.73:0/963912513 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400f510 con 0x7f4428065f50 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ca04700 1 -- 172.21.15.73:0/963912513 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400f670 con 0x7f4428065f50 2023-11-28T03:11:07.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ca04700 1 -- 172.21.15.73:0/963912513 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400f9f0 con 0x7f4428065f50 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 -- 172.21.15.73:0/963912513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4428065f50 msgr2=0x7f4428155c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 --2- 172.21.15.73:0/963912513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4428065f50 0x7f4428155c20 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 rx=0x7f44280c2a00 tx=0x7f44240090f0).stop 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 -- 172.21.15.73:0/963912513 shutdown_connections 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 --2- 172.21.15.73:0/963912513 >> v2:172.21.15.150:3300/0 conn(0x7f44280a7890 0x7f4428156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 --2- 172.21.15.73:0/963912513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4428065f50 0x7f4428155c20 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 -- 172.21.15.73:0/963912513 >> 172.21.15.73:0/963912513 conn(0x7f44280c7ca0 msgr2=0x7f44280c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 -- 172.21.15.73:0/963912513 shutdown_connections 2023-11-28T03:11:07.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.056+0000 7f442ea08700 1 -- 172.21.15.73:0/963912513 wait complete. 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 Processor -- start 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 -- start start 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4428065f50 0x7f44281639a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442813b790 0x7f4428167ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428168090 con 0x7f44280a7890 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4428152740 con 0x7f442813b790 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.058+0000 7f442ea08700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4428152fe0 con 0x7f4428065f50 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33758/0 (socket says 172.21.15.73:33758) 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 -- 172.21.15.73:0/424308061 learned_addr learned my addr 172.21.15.73:0/424308061 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:07.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442e207700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442813b790 0x7f4428167ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442da06700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4428065f50 0x7f44281639a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 -- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4428065f50 msgr2=0x7f44281639a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4428065f50 0x7f44281639a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 -- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442813b790 msgr2=0x7f4428167ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442813b790 0x7f4428167ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 -- 172.21.15.73:0/424308061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4420009a20 con 0x7f44280a7890 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442d205700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f4428068d10 tx=0x7f4424017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442400f7e0 con 0x7f44280a7890 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f442400f940 con 0x7f44280a7890 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4424009370 con 0x7f44280a7890 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f442401c670 con 0x7f44280a7890 2023-11-28T03:11:07.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.059+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4428152990 con 0x7f44280a7890 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.060+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f442806c7c0 con 0x7f44280a7890 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.061+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f442401ca60 con 0x7f44280a7890 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.061+0000 7f441dffb700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 0x7f440c061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.061+0000 7f442da06700 1 -- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 msgr2=0x7f440c061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.061+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f442407c320 con 0x7f44280a7890 2023-11-28T03:11:07.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.061+0000 7f442da06700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 0x7f440c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:07.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.065+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f442404d2f0 con 0x7f44280a7890 2023-11-28T03:11:07.078 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:07.077+0000 7ff428cdd680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:11:07.122 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.120+0000 7faae096b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:11:07.186 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:07.185+0000 7ff428cdd680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:07.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.238+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44280c2720 con 0x7f44280a7890 2023-11-28T03:11:07.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.239+0000 7f441dffb700 1 -- 172.21.15.73:0/424308061 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v127) v1 ==== 74+0+17186 (secure 0 0 0) 0x7f4424050910 con 0x7f44280a7890 2023-11-28T03:11:07.241 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:07.241 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":127,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:10:03.562867+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"117","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":15,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:09:46.277212+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":16,"pool_name":"cephfs_data","create_time":"2023-11-28T03:09:46.698961+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"126","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:6832/90017":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/3114748317":"2023-11-29T03:10:03.562840+0000","172.21.15.73:0/1437075559":"2023-11-29T03:10:03.562840+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:07.241 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.241+0000 7faae096b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:11:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 msgr2=0x7f440c061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 0x7f440c061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 msgr2=0x7f44281676b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 rx=0x7f4428068d10 tx=0x7f4424017000).stop 2023-11-28T03:11:07.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 shutdown_connections 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f440c05f390 0x7f440c061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f442813b790 0x7f4428167ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4428065f50 0x7f44281639a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 --2- 172.21.15.73:0/424308061 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44280a7890 0x7f44281676b0 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 >> 172.21.15.73:0/424308061 conn(0x7f44280c7ca0 msgr2=0x7f44281365f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 shutdown_connections 2023-11-28T03:11:07.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.242+0000 7f442ea08700 1 -- 172.21.15.73:0/424308061 wait complete. 2023-11-28T03:11:07.255 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T03:11:07.358 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.357+0000 7faae096b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:07.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.372+0000 7fcbac58d700 1 Processor -- start 2023-11-28T03:11:07.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.372+0000 7fcbac58d700 1 -- start start 2023-11-28T03:11:07.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcbac58d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcbac58d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba40c72d0 con 0x7fcba40c6190 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcbac58d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcba4064570 con 0x7fcba4098d10 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcbac58d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba4154bc0 con 0x7fcba4158a30 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcba9b28700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcba40c6190 0x7fcba4158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51222/0 (socket says 172.21.15.73:51222) 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcba9b28700 1 -- 172.21.15.73:0/999904613 learned_addr learned my addr 172.21.15.73:0/999904613 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.373+0000 7fcbaab2a700 1 --2- 172.21.15.73:0/999904613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25726982 0 0) 0x7fcba40c72d0 con 0x7fcba40c6190 2023-11-28T03:11:07.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb900033f0 con 0x7fcba40c6190 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1255347144 0 0) 0x7fcba4064570 con 0x7fcba4098d10 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcba40c72d0 con 0x7fcba4098d10 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3600999136 0 0) 0x7fcb900033f0 con 0x7fcba40c6190 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcbaab2a700 1 -- 172.21.15.73:0/999904613 >> v1:172.21.15.150:6789/0 conn(0x7fcba4098d10 legacy=0x7fcba4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcbaab2a700 1 -- 172.21.15.73:0/999904613 >> v1:172.21.15.73:6789/0 conn(0x7fcba40c6190 legacy=0x7fcba4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcbaab2a700 1 -- 172.21.15.73:0/999904613 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcba40c66c0 con 0x7fcba4158a30 2023-11-28T03:11:07.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcbaab2a700 1 --2- 172.21.15.73:0/999904613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c7760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcb9c009e90 tx=0x7fcb9c009780).ready entity=mon.2 client_cookie=3cf75a29b2a44ffa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb9c00e070 con 0x7fcba4158a30 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb9c00b7a0 con 0x7fcba4158a30 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.374+0000 7fcba9327700 1 -- 172.21.15.73:0/999904613 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb94001470 con 0x7fcba4158a30 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 -- 172.21.15.73:0/999904613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 msgr2=0x7fcba40c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 --2- 172.21.15.73:0/999904613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c7760 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 rx=0x7fcb9c009e90 tx=0x7fcb9c009780).stop 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 -- 172.21.15.73:0/999904613 shutdown_connections 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 --2- 172.21.15.73:0/999904613 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c7760 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 -- 172.21.15.73:0/999904613 >> 172.21.15.73:0/999904613 conn(0x7fcba40cb310 msgr2=0x7fcba40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:07.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 -- 172.21.15.73:0/999904613 shutdown_connections 2023-11-28T03:11:07.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.375+0000 7fcbac58d700 1 -- 172.21.15.73:0/999904613 wait complete. 2023-11-28T03:11:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.376+0000 7fcbac58d700 1 Processor -- start 2023-11-28T03:11:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.376+0000 7fcbac58d700 1 -- start start 2023-11-28T03:11:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4098d10 0x7fcba40afc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c9310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b0dc0 con 0x7fcba40c6190 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40ae560 con 0x7fcba4098d10 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbac58d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcba40b01b0 con 0x7fcba4158a30 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33762/0 (socket says 172.21.15.73:33762) 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 -- 172.21.15.73:0/1178515511 learned_addr learned my addr 172.21.15.73:0/1178515511 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbaa329700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4098d10 0x7fcba40afc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcbaab2a700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 msgr2=0x7fcba40c9310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c9310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4098d10 msgr2=0x7fcba40afc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:07.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4098d10 0x7fcba40afc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.377+0000 7fcba9b28700 1 -- 172.21.15.73:0/1178515511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb9c009a20 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcba9b28700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fcb9400c790 tx=0x7fcb94001a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940072e0 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb94007460 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb940059f0 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcba40b0490 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.378+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcba40b0a50 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.379+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcba40b0f30 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.379+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 53) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fcb94013450 con 0x7fcba40c6190 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.380+0000 7fcb9a7fc700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 0x7fcb88061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.380+0000 7fcbaa329700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 msgr2=0x7fcb88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:07.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.380+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fcb94079ee0 con 0x7fcba40c6190 2023-11-28T03:11:07.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.380+0000 7fcbaa329700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 0x7fcb88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:07.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.384+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcb9404ada0 con 0x7fcba40c6190 2023-11-28T03:11:07.482 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:07.481+0000 7ff428cdd680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:11:07.552 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:07.551+0000 7ff428cdd680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:11:07.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.581+0000 7fcbaa329700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 msgr2=0x7fcb88061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:07.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.581+0000 7fcbaa329700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 0x7fcb88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:07.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.598+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fcba406c860 con 0x7fcba40c6190 2023-11-28T03:11:07.659 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.658+0000 7faae096b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:11:07.730 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.729+0000 7faae096b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:11:07.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.749+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 54) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fcb94077900 con 0x7fcba40c6190 2023-11-28T03:11:07.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.749+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 msgr2=0x7fcb88061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:07.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:07.749+0000 7fcb9a7fc700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 0x7fcb88061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:07.755 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:11:07.755 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:11:07.756 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:11:07.762 INFO:tasks.ceph.mds.c.smithi073.stderr: -8763> 2023-11-28T03:09:53.664+0000 7f6f6373a700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:11:07.772 INFO:tasks.ceph.mds.c.smithi073.stderr: -6> 2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T03:11:07.772 INFO:tasks.ceph.mds.c.smithi073.stderr: -4> 2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T03:11:07.773 INFO:tasks.ceph.mds.c.smithi073.stderr: -2> 2023-11-28T03:11:07.754+0000 7f6f5ff33700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T03:11:07.793 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:11:07.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5a0fb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5a0fb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.809 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5a0fb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5a0fb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5d101700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5a0fb700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5d101700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5d101700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5d101700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:07.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:07.808+0000 7faa5d101700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:08.323 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T03:11:08.322+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T03:11:08.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.330+0000 7fcb9a7fc700 1 -- 172.21.15.73:0/1178515511 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fcb94073440 con 0x7fcba40c6190 2023-11-28T03:11:08.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 msgr2=0x7fcba4168c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:08.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 rx=0x7fcb9400c790 tx=0x7fcb94001a50).stop 2023-11-28T03:11:08.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 shutdown_connections 2023-11-28T03:11:08.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcb8805f390 0x7fcb88061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:08.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcba4158a30 0x7fcba40c9310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcba4098d10 0x7fcba40afc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 --2- 172.21.15.73:0/1178515511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcba40c6190 0x7fcba4168c00 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 >> 172.21.15.73:0/1178515511 conn(0x7fcba40cb310 msgr2=0x7fcba40ca390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 shutdown_connections 2023-11-28T03:11:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.332+0000 7fcbac58d700 1 -- 172.21.15.73:0/1178515511 wait complete. 2023-11-28T03:11:08.335 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T03:11:08.346 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T03:11:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.461+0000 7f5468a32700 1 Processor -- start 2023-11-28T03:11:08.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.461+0000 7f5468a32700 1 -- start start 2023-11-28T03:11:08.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5468a32700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5464098d10 0x7f54640c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5468a32700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f54640c6190 0x7f5464063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5468a32700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54640c72d0 con 0x7f5464064350 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5468a32700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5464154bc0 con 0x7f5464098d10 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5468a32700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54640c66c0 con 0x7f54640c6190 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5464098d10 0x7f54640c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5464098d10 0x7f54640c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33870/0 (socket says 172.21.15.73:33870) 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 -- 172.21.15.73:0/269917906 learned_addr learned my addr 172.21.15.73:0/269917906 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f5461d9b700 1 --2- 172.21.15.73:0/269917906 >> v2:172.21.15.150:3300/0 conn(0x7f54640c6190 0x7f5464063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 -- 172.21.15.73:0/269917906 >> v2:172.21.15.150:3300/0 conn(0x7f54640c6190 msgr2=0x7f5464063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 --2- 172.21.15.73:0/269917906 >> v2:172.21.15.150:3300/0 conn(0x7f54640c6190 0x7f5464063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 -- 172.21.15.73:0/269917906 >> v1:172.21.15.150:6789/0 conn(0x7f5464064350 legacy=0x7f5464158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.462+0000 7f546259c700 1 -- 172.21.15.73:0/269917906 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54640c69a0 con 0x7f5464098d10 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f546259c700 1 --2- 172.21.15.73:0/269917906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5464098d10 0x7f54640c7760 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f54640c5b40 tx=0x7f5454009840).ready entity=mon.0 client_cookie=ea1c6fc0b96b916b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f546159a700 1 -- 172.21.15.73:0/269917906 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545400e070 con 0x7f5464098d10 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f546159a700 1 -- 172.21.15.73:0/269917906 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545400ba50 con 0x7f5464098d10 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f546159a700 1 -- 172.21.15.73:0/269917906 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545400bdd0 con 0x7f5464098d10 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 -- 172.21.15.73:0/269917906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5464098d10 msgr2=0x7f54640c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:08.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 --2- 172.21.15.73:0/269917906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5464098d10 0x7f54640c7760 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 rx=0x7f54640c5b40 tx=0x7f5454009840).stop 2023-11-28T03:11:08.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 -- 172.21.15.73:0/269917906 shutdown_connections 2023-11-28T03:11:08.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 --2- 172.21.15.73:0/269917906 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5464098d10 0x7f54640c7760 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 --2- 172.21.15.73:0/269917906 >> v2:172.21.15.150:3300/0 conn(0x7f54640c6190 0x7f5464063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 -- 172.21.15.73:0/269917906 >> 172.21.15.73:0/269917906 conn(0x7f54640cb310 msgr2=0x7f54640cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:08.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 -- 172.21.15.73:0/269917906 shutdown_connections 2023-11-28T03:11:08.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.463+0000 7f5468a32700 1 -- 172.21.15.73:0/269917906 wait complete. 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.464+0000 7f5468a32700 1 Processor -- start 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.464+0000 7f5468a32700 1 -- start start 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5464064350 0x7f5464150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5464098d10 0x7f5464155ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464155a10 con 0x7f54640c6190 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5464154e00 con 0x7f5464098d10 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5468a32700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54641550e0 con 0x7f5464064350 2023-11-28T03:11:08.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5462d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5461d9b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5464098d10 0x7f5464155ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5462d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33872/0 (socket says 172.21.15.73:33872) 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f5462d9d700 1 -- 172.21.15.73:0/2233600842 learned_addr learned my addr 172.21.15.73:0/2233600842 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.465+0000 7f546259c700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5464064350 0x7f5464150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 -- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5464064350 msgr2=0x7f5464150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5464064350 0x7f5464150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 -- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5464098d10 msgr2=0x7f5464155ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5464098d10 0x7f5464155ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 -- 172.21.15.73:0/2233600842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5454009ae0 con 0x7f54640c6190 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5462d9d700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f545c00c440 tx=0x7f545c007d70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c0123f0 con 0x7f54640c6190 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f545c001e90 con 0x7f54640c6190 2023-11-28T03:11:08.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54641553c0 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f545c012760 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.466+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54640aaf80 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.467+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5464155bc0 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.467+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 54) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f545c012b50 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.468+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f545c079570 con 0x7f54640c6190 2023-11-28T03:11:08.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.472+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f545c047af0 con 0x7f54640c6190 2023-11-28T03:11:08.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.676+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f546406c860 con 0x7f54640c6190 2023-11-28T03:11:08.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.755+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 55) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f545c0771c0 con 0x7f54640c6190 2023-11-28T03:11:08.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.755+0000 7f545a7fc700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 0x7f5444067c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:08.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.756+0000 7f546259c700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 0x7f5444067c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:08.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:08.757+0000 7f546259c700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 0x7f5444067c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f5454009880 tx=0x7f54640a8040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:09.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.332+0000 7f545a7fc700 1 -- 172.21.15.73:0/2233600842 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f545c071490 con 0x7f54640c6190 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 msgr2=0x7f5444067c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 0x7f5444067c70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 rx=0x7f5454009880 tx=0x7f54640a8040).stop 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 msgr2=0x7f5464156390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 rx=0x7f545c00c440 tx=0x7f545c007d70).stop 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 shutdown_connections 2023-11-28T03:11:09.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f54440302f0 0x7f5444067c70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54640c6190 0x7f5464156390 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.334+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5464064350 0x7f5464150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.335+0000 7f5468a32700 1 --2- 172.21.15.73:0/2233600842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5464098d10 0x7f5464155ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.335+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 >> 172.21.15.73:0/2233600842 conn(0x7f54640cb310 msgr2=0x7f54640aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.335+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 shutdown_connections 2023-11-28T03:11:09.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.335+0000 7f5468a32700 1 -- 172.21.15.73:0/2233600842 wait complete. 2023-11-28T03:11:09.349 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T03:11:09.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.466+0000 7f346b31b700 1 Processor -- start 2023-11-28T03:11:09.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.467+0000 7f346b31b700 1 -- start start 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f346b31b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3464067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f346b31b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34640c3510 con 0x7f34641556f0 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f346b31b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34640bdfe0 con 0x7f346413af40 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f346b31b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3464067c70 con 0x7f346413ab60 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f34698b8700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f34641556f0 0x7f34640c39b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:51242/0 (socket says 172.21.15.73:51242) 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f34698b8700 1 -- 172.21.15.73:0/4189627848 learned_addr learned my addr 172.21.15.73:0/4189627848 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:09.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f34690b7700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3464067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1051245626 0 0) 0x7f34640c3510 con 0x7f34641556f0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34500033f0 con 0x7f34641556f0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 690252023 0 0) 0x7f34500033f0 con 0x7f34641556f0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 msgr2=0x7f3464067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3464067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 >> v1:172.21.15.150:6789/0 conn(0x7f346413af40 legacy=0x7f3464154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640be2c0 con 0x7f34641556f0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f345c001470 con 0x7f34641556f0 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 >> v1:172.21.15.73:6789/0 conn(0x7f34641556f0 legacy=0x7f34640c39b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3450003910 0x7f3450003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3450004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3450004b80 0x7f3450005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3464067c70 con 0x7f3450003910 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345000b910 con 0x7f3450004b80 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345000bad0 con 0x7f346413ab60 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.468+0000 7f34690b7700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3450003910 0x7f3450003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34698b8700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3450004b80 0x7f3450005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34688b6700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3450004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 -- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 msgr2=0x7f3450004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f346413ab60 0x7f3450004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 -- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3450004b80 msgr2=0x7f3450005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3450004b80 0x7f3450005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 -- 172.21.15.73:0/4189627848 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34500033f0 con 0x7f3450003910 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34690b7700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3450003910 0x7f3450003d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f346000e210 tx=0x7f346000b040).ready entity=mon.0 client_cookie=bef2748cc3bca8d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3460009ac0 con 0x7f3450003910 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346000f040 con 0x7f3450003910 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f345bfff700 1 -- 172.21.15.73:0/4189627848 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346000ecd0 con 0x7f3450003910 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f346b31b700 1 -- 172.21.15.73:0/4189627848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3450003910 msgr2=0x7f3450003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f346b31b700 1 --2- 172.21.15.73:0/4189627848 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3450003910 0x7f3450003d30 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 rx=0x7f346000e210 tx=0x7f346000b040).stop 2023-11-28T03:11:09.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f34698b8700 1 -- 172.21.15.73:0/4189627848 reap_dead start 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f346b31b700 1 -- 172.21.15.73:0/4189627848 shutdown_connections 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.469+0000 7f346b31b700 1 -- 172.21.15.73:0/4189627848 >> 172.21.15.73:0/4189627848 conn(0x7f3464063810 msgr2=0x7f3464063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.470+0000 7f346b31b700 1 -- 172.21.15.73:0/4189627848 shutdown_connections 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.470+0000 7f346b31b700 1 -- 172.21.15.73:0/4189627848 wait complete. 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.471+0000 7f346b31b700 1 Processor -- start 2023-11-28T03:11:09.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.471+0000 7f346b31b700 1 -- start start 2023-11-28T03:11:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450003910 0x7f34640aaf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34640c88f0 0x7f34640acff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640ad4e0 con 0x7f3464151c60 2023-11-28T03:11:09.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34640c8d70 con 0x7f34640c88f0 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34641525d0 con 0x7f3450003910 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33890/0 (socket says 172.21.15.73:33890) 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 -- 172.21.15.73:0/3647517580 learned_addr learned my addr 172.21.15.73:0/3647517580 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34690b7700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450003910 0x7f34640aaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34688b6700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34640c88f0 0x7f34640acff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 -- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450003910 msgr2=0x7f34640aaf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450003910 0x7f34640aaf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 -- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34640c88f0 msgr2=0x7f34640acff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34640c88f0 0x7f34640acff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 -- 172.21.15.73:0/3647517580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34640ab5b0 con 0x7f3464151c60 2023-11-28T03:11:09.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34698b8700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f345c0015a0 tx=0x7f345c000f80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c010520 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345c0106a0 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34640ab890 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345c010a60 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.472+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34640abe50 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.473+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34640c6b20 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.474+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 55) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f345c014430 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.474+0000 7f34597fa700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 0x7f3448061520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.474+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 6693+0+0 (secure 0 0 0) 0x7f345c079e70 con 0x7f3464151c60 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.474+0000 7f34690b7700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 0x7f3448061520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:09.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.475+0000 7f34690b7700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 0x7f3448061520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f346000afd0 tx=0x7f346001a000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:09.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.478+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f345c04b070 con 0x7f3464151c60 2023-11-28T03:11:09.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:09.640+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f346406c860 con 0x7f3464151c60 2023-11-28T03:11:10.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.334+0000 7f34597fa700 1 -- 172.21.15.73:0/3647517580 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v130) v1 ==== 181+0+0 (secure 0 0 0) 0x7f345c04e690 con 0x7f3464151c60 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 msgr2=0x7f3448061520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 0x7f3448061520 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f346000afd0 tx=0x7f346001a000).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 msgr2=0x7f34641520e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 rx=0x7f345c0015a0 tx=0x7f345c000f80).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 shutdown_connections 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f344805f060 0x7f3448061520 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3464151c60 0x7f34641520e0 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3450003910 0x7f34640aaf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 --2- 172.21.15.73:0/3647517580 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34640c88f0 0x7f34640acff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 >> 172.21.15.73:0/3647517580 conn(0x7f3464063810 msgr2=0x7f3464153750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:10.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 shutdown_connections 2023-11-28T03:11:10.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.337+0000 7f346b31b700 1 -- 172.21.15.73:0/3647517580 wait complete. 2023-11-28T03:11:10.343 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T03:11:10.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T03:11:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.470+0000 7f558fd2f700 1 Processor -- start 2023-11-28T03:11:10.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.470+0000 7f558fd2f700 1 -- start start 2023-11-28T03:11:10.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5588154530 0x7f5588154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f55880c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55880aa220 0x7f55880aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55880c6390 con 0x7f5588154530 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55880c6630 con 0x7f55880aa220 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558fd2f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55880aaba0 con 0x7f55880a7ff0 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5588154530 0x7f5588154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5588154530 0x7f5588154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33902/0 (socket says 172.21.15.73:33902) 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 -- 172.21.15.73:0/116660592 learned_addr learned my addr 172.21.15.73:0/116660592 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558e2cc700 1 --2- 172.21.15.73:0/116660592 >> v2:172.21.15.150:3300/0 conn(0x7f55880aa220 0x7f55880aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558d2ca700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f55880c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 -- 172.21.15.73:0/116660592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 msgr2=0x7f55880c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f55880c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 -- 172.21.15.73:0/116660592 >> v2:172.21.15.150:3300/0 conn(0x7f55880aa220 msgr2=0x7f55880aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 --2- 172.21.15.73:0/116660592 >> v2:172.21.15.150:3300/0 conn(0x7f55880aa220 0x7f55880aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 -- 172.21.15.73:0/116660592 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55880a8470 con 0x7f5588154530 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.471+0000 7f558dacb700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f5588154910 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f55880a9fc0 tx=0x7f5584009060).ready entity=mon.0 client_cookie=21aec6a92c2e8db3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558cac9700 1 -- 172.21.15.73:0/116660592 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584005880 con 0x7f5588154530 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558cac9700 1 -- 172.21.15.73:0/116660592 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55840059e0 con 0x7f5588154530 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558cac9700 1 -- 172.21.15.73:0/116660592 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5584005d60 con 0x7f5588154530 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 -- 172.21.15.73:0/116660592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 msgr2=0x7f5588154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f5588154910 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 rx=0x7f55880a9fc0 tx=0x7f5584009060).stop 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 -- 172.21.15.73:0/116660592 shutdown_connections 2023-11-28T03:11:10.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f5588154910 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 --2- 172.21.15.73:0/116660592 >> v2:172.21.15.150:3300/0 conn(0x7f55880aa220 0x7f55880aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 --2- 172.21.15.73:0/116660592 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f55880c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 -- 172.21.15.73:0/116660592 >> 172.21.15.73:0/116660592 conn(0x7f5588094930 msgr2=0x7f558806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:10.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 -- 172.21.15.73:0/116660592 shutdown_connections 2023-11-28T03:11:10.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.472+0000 7f558fd2f700 1 -- 172.21.15.73:0/116660592 wait complete. 2023-11-28T03:11:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.473+0000 7f558fd2f700 1 Processor -- start 2023-11-28T03:11:10.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.473+0000 7f558fd2f700 1 -- start start 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f5588144d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55880aa220 0x7f5588150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880c3640 con 0x7f5588154530 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55880c4460 con 0x7f55880aa220 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558fd2f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5588145240 con 0x7f55880a7ff0 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558e2cc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558e2cc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33914/0 (socket says 172.21.15.73:33914) 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.474+0000 7f558e2cc700 1 -- 172.21.15.73:0/3017591242 learned_addr learned my addr 172.21.15.73:0/3017591242 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558d2ca700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55880aa220 0x7f5588150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 -- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 msgr2=0x7f5588144d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:10.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f5588144d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 -- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55880aa220 msgr2=0x7f5588150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55880aa220 0x7f5588150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 -- 172.21.15.73:0/3017591242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5584009300 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558e2cc700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f5580009fa0 tx=0x7f5580009900).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558000ed20 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558000ee80 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55880c3850 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5580014610 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.475+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55880c3e10 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.476+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f558806c860 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.476+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 55) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f558001d3b0 con 0x7f5588154530 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.477+0000 7f557dffb700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 0x7f556c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:10.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.477+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(130..130 src has 1..130) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f558007a570 con 0x7f5588154530 2023-11-28T03:11:10.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.477+0000 7f558dacb700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 0x7f556c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:10.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.477+0000 7f558dacb700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 0x7f556c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f558400b3c0 tx=0x7f5584016000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:10.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.481+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f558004b940 con 0x7f5588154530 2023-11-28T03:11:10.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.646+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f55880c45d0 con 0x7f5588154530 2023-11-28T03:11:10.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:10.760+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f55800784a0 con 0x7f5588154530 2023-11-28T03:11:11.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.300+0000 7f557dffb700 1 -- 172.21.15.73:0/3017591242 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v131) v1 ==== 169+0+0 (secure 0 0 0) 0x7f55800736b0 con 0x7f5588154530 2023-11-28T03:11:11.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.302+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 msgr2=0x7f556c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.302+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 0x7f556c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f558400b3c0 tx=0x7f5584016000).stop 2023-11-28T03:11:11.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.302+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 msgr2=0x7f55880c48f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.302+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 rx=0x7f5580009fa0 tx=0x7f5580009900).stop 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 shutdown_connections 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f556c05f030 0x7f556c0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5588154530 0x7f55880c48f0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55880a7ff0 0x7f5588144d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:11.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 --2- 172.21.15.73:0/3017591242 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55880aa220 0x7f5588150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 >> 172.21.15.73:0/3017591242 conn(0x7f5588094930 msgr2=0x7f5588152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:11.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 shutdown_connections 2023-11-28T03:11:11.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.303+0000 7f558fd2f700 1 -- 172.21.15.73:0/3017591242 wait complete. 2023-11-28T03:11:11.311 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T03:11:11.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T03:11:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.431+0000 7f278d398700 1 Processor -- start 2023-11-28T03:11:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.431+0000 7f278d398700 1 -- start start 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f278d398700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2788098d10 0x7f27880c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f278d398700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2788158320 0x7f27881587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f278d398700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2788158cc0 con 0x7f27880c6190 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f278d398700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27880c72d0 con 0x7f2788098d10 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f278d398700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2788064570 con 0x7f2788158320 2023-11-28T03:11:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2788098d10 0x7f27880c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2788098d10 0x7f27880c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33920/0 (socket says 172.21.15.73:33920) 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 -- 172.21.15.73:0/1800325379 learned_addr learned my addr 172.21.15.73:0/1800325379 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f27877fe700 1 --2- 172.21.15.73:0/1800325379 >> v2:172.21.15.150:3300/0 conn(0x7f2788158320 0x7f27881587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 -- 172.21.15.73:0/1800325379 >> v2:172.21.15.150:3300/0 conn(0x7f2788158320 msgr2=0x7f27881587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 --2- 172.21.15.73:0/1800325379 >> v2:172.21.15.150:3300/0 conn(0x7f2788158320 0x7f27881587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 -- 172.21.15.73:0/1800325379 >> v1:172.21.15.150:6789/0 conn(0x7f27880c6190 legacy=0x7f2788063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 -- 172.21.15.73:0/1800325379 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2788154bc0 con 0x7f2788098d10 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2786ffd700 1 --2- 172.21.15.73:0/1800325379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2788098d10 0x7f27880c7760 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f27880657a0 tx=0x7f27780090f0).ready entity=mon.0 client_cookie=fc10f8c28c37446a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2785ffb700 1 -- 172.21.15.73:0/1800325379 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800b5a0 con 0x7f2788098d10 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2785ffb700 1 -- 172.21.15.73:0/1800325379 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277800b700 con 0x7f2788098d10 2023-11-28T03:11:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.432+0000 7f2785ffb700 1 -- 172.21.15.73:0/1800325379 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277800ba80 con 0x7f2788098d10 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 -- 172.21.15.73:0/1800325379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2788098d10 msgr2=0x7f27880c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 --2- 172.21.15.73:0/1800325379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2788098d10 0x7f27880c7760 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 rx=0x7f27880657a0 tx=0x7f27780090f0).stop 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 -- 172.21.15.73:0/1800325379 shutdown_connections 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 --2- 172.21.15.73:0/1800325379 >> v2:172.21.15.150:3300/0 conn(0x7f2788158320 0x7f27881587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 --2- 172.21.15.73:0/1800325379 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2788098d10 0x7f27880c7760 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 -- 172.21.15.73:0/1800325379 >> 172.21.15.73:0/1800325379 conn(0x7f27880cb310 msgr2=0x7f27880cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 -- 172.21.15.73:0/1800325379 shutdown_connections 2023-11-28T03:11:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.433+0000 7f278d398700 1 -- 172.21.15.73:0/1800325379 wait complete. 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.434+0000 7f278d398700 1 Processor -- start 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.434+0000 7f278d398700 1 -- start start 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2788098d10 0x7f278814c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2788158320 0x7f27880a8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880a99c0 con 0x7f27880c6190 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27880a8b30 con 0x7f2788158320 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f278d398700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27880a8db0 con 0x7f2788098d10 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33922/0 (socket says 172.21.15.73:33922) 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 -- 172.21.15.73:0/1071888733 learned_addr learned my addr 172.21.15.73:0/1071888733 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f2786ffd700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2788098d10 0x7f278814c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27877fe700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2788158320 0x7f27880a8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 -- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2788098d10 msgr2=0x7f278814c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2788098d10 0x7f278814c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 -- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2788158320 msgr2=0x7f27880a8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2788158320 0x7f27880a8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.435+0000 7f27867fc700 1 -- 172.21.15.73:0/1071888733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2778009370 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f27867fc700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f277c00c4c0 tx=0x7f277c007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c012400 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f277c012560 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27880a9090 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f277c012900 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.436+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27880a9650 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.437+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27880a9b70 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.437+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f277c012cf0 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.437+0000 7f276effd700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 0x7f27740615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.437+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f277c079410 con 0x7f27880c6190 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.437+0000 7f2786ffd700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 0x7f27740615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.438+0000 7f2786ffd700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 0x7f27740615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2778009130 tx=0x7f2778015000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.441+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f277c048af0 con 0x7f27880c6190 2023-11-28T03:11:11.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.619+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f278806c860 con 0x7f27880c6190 2023-11-28T03:11:11.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.622+0000 7f276effd700 1 -- 172.21.15.73:0/1071888733 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v24) v1 ==== 120+0+0 (secure 0 0 0) 0x7f277c017020 con 0x7f27880c6190 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.624+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 msgr2=0x7f27740615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.624+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 0x7f27740615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2778009130 tx=0x7f2778015000).stop 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.624+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 msgr2=0x7f278814e0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.624+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 rx=0x7f277c00c4c0 tx=0x7f277c007db0).stop 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 shutdown_connections 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f277405f120 0x7f27740615e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2788158320 0x7f27880a8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2788098d10 0x7f278814c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 --2- 172.21.15.73:0/1071888733 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f27880c6190 0x7f278814e0f0 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 >> 172.21.15.73:0/1071888733 conn(0x7f27880cb310 msgr2=0x7f278814f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:11.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 shutdown_connections 2023-11-28T03:11:11.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.625+0000 7f278d398700 1 -- 172.21.15.73:0/1071888733 wait complete. 2023-11-28T03:11:11.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T03:11:11.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.753+0000 7f7a3ccec700 1 Processor -- start 2023-11-28T03:11:11.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.753+0000 7f7a3ccec700 1 -- start start 2023-11-28T03:11:11.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a380c6190 0x7f7a38063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7a38064350 0x7f7a38158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a380c72d0 con 0x7f7a380c6190 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a38154bc0 con 0x7f7a38064350 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3ccec700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c66c0 con 0x7f7a38098d10 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a380c6190 0x7f7a38063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7a380c6190 0x7f7a38063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33938/0 (socket says 172.21.15.73:33938) 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 -- 172.21.15.73:0/277722536 learned_addr learned my addr 172.21.15.73:0/277722536 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a3659c700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a36d9d700 1 --2- 172.21.15.73:0/277722536 >> v2:172.21.15.150:3300/0 conn(0x7f7a38064350 0x7f7a38158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 -- 172.21.15.73:0/277722536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 msgr2=0x7f7a380c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:11.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 -- 172.21.15.73:0/277722536 >> v2:172.21.15.150:3300/0 conn(0x7f7a38064350 msgr2=0x7f7a38158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 --2- 172.21.15.73:0/277722536 >> v2:172.21.15.150:3300/0 conn(0x7f7a38064350 0x7f7a38158320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.754+0000 7f7a35d9b700 1 -- 172.21.15.73:0/277722536 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a380c69a0 con 0x7f7a380c6190 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a35d9b700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a38063e60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f7a38084570 tx=0x7f7a2c0090f0).ready entity=mon.0 client_cookie=b56324ee2a9451e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3559a700 1 -- 172.21.15.73:0/277722536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00b600 con 0x7f7a380c6190 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3559a700 1 -- 172.21.15.73:0/277722536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a2c00b760 con 0x7f7a380c6190 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3559a700 1 -- 172.21.15.73:0/277722536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c00bae0 con 0x7f7a380c6190 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 -- 172.21.15.73:0/277722536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 msgr2=0x7f7a38063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a38063e60 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 rx=0x7f7a38084570 tx=0x7f7a2c0090f0).stop 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 -- 172.21.15.73:0/277722536 shutdown_connections 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/277722536 >> v2:172.21.15.150:3300/0 conn(0x7f7a38064350 0x7f7a38158320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/277722536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a38063e60 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 -- 172.21.15.73:0/277722536 >> 172.21.15.73:0/277722536 conn(0x7f7a380cb310 msgr2=0x7f7a380cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:11.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 -- 172.21.15.73:0/277722536 shutdown_connections 2023-11-28T03:11:11.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.755+0000 7f7a3ccec700 1 -- 172.21.15.73:0/277722536 wait complete. 2023-11-28T03:11:11.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.756+0000 7f7a3ccec700 1 Processor -- start 2023-11-28T03:11:11.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.756+0000 7f7a3ccec700 1 -- start start 2023-11-28T03:11:11.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a38064350 0x7f7a381480e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380c9ea0 con 0x7f7a380c6190 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a380b0ad0 con 0x7f7a38064350 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3ccec700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a381485d0 con 0x7f7a38098d10 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a36d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a36d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33954/0 (socket says 172.21.15.73:33954) 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a36d9d700 1 -- 172.21.15.73:0/1062260981 learned_addr learned my addr 172.21.15.73:0/1062260981 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a35d9b700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.757+0000 7f7a3659c700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a38064350 0x7f7a381480e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 -- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 msgr2=0x7f7a380c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 -- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a38064350 msgr2=0x7f7a381480e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a38064350 0x7f7a381480e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 -- 172.21.15.73:0/1062260981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a2c009370 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a36d9d700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f7a2800d890 tx=0x7f7a28008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28011590 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a280116f0 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a380b0510 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a28011a70 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.758+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a380c24e0 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.759+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a3806c860 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.759+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7a28012420 con 0x7f7a380c6190 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.760+0000 7f7a267fc700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 0x7f7a180615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:11.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.760+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f7a28079a20 con 0x7f7a380c6190 2023-11-28T03:11:11.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.760+0000 7f7a3659c700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 0x7f7a180615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:11.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.760+0000 7f7a3659c700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 0x7f7a180615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7a30007640 tx=0x7f7a30009850).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:11.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.764+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7a2804adc0 con 0x7f7a380c6190 2023-11-28T03:11:11.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.928+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f7a380c6c80 con 0x7f7a380c6190 2023-11-28T03:11:11.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.931+0000 7f7a267fc700 1 -- 172.21.15.73:0/1062260981 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v25) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7a2804e3e0 con 0x7f7a380c6190 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 msgr2=0x7f7a180615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 0x7f7a180615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f7a30007640 tx=0x7f7a30009850).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 msgr2=0x7f7a380c99b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 rx=0x7f7a2800d890 tx=0x7f7a28008040).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 shutdown_connections 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7a1805f120 0x7f7a180615e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a380c6190 0x7f7a380c99b0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a38064350 0x7f7a381480e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 --2- 172.21.15.73:0/1062260981 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a38098d10 0x7f7a380c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 >> 172.21.15.73:0/1062260981 conn(0x7f7a380cb310 msgr2=0x7f7a380ca8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 shutdown_connections 2023-11-28T03:11:11.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:11.933+0000 7f7a3ccec700 1 -- 172.21.15.73:0/1062260981 wait complete. 2023-11-28T03:11:11.946 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-11-28T03:11:12.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.058+0000 7f58e4184700 1 Processor -- start 2023-11-28T03:11:12.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.058+0000 7f58e4184700 1 -- start start 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e4184700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58dc0caab0 0x7f58dc0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e4184700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e4184700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58dc0a9a90 con 0x7f58dc0cae90 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e4184700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58dc154ce0 con 0x7f58dc0caab0 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e4184700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0a8e80 con 0x7f58dc0ca180 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e1f20700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58dc0caab0 0x7f58dc0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e1f20700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58dc0caab0 0x7f58dc0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33970/0 (socket says 172.21.15.73:33970) 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.059+0000 7f58e1f20700 1 -- 172.21.15.73:0/2361572284 learned_addr learned my addr 172.21.15.73:0/2361572284 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e2721700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e1f20700 1 -- 172.21.15.73:0/2361572284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 msgr2=0x7f58dc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:12.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e1f20700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e1f20700 1 -- 172.21.15.73:0/2361572284 >> v1:172.21.15.150:6789/0 conn(0x7f58dc0cae90 legacy=0x7f58dc0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e1f20700 1 -- 172.21.15.73:0/2361572284 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58dc0a9160 con 0x7f58dc0caab0 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e1f20700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0caab0 0x7f58dc0aa470 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f58cc009f70 tx=0x7f58cc009840).ready entity=mon.0 client_cookie=3d463599a3ba0eab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e0f1e700 1 -- 172.21.15.73:0/2361572284 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00e070 con 0x7f58dc0caab0 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e0f1e700 1 -- 172.21.15.73:0/2361572284 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58cc00ba20 con 0x7f58dc0caab0 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e0f1e700 1 -- 172.21.15.73:0/2361572284 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58cc00bda0 con 0x7f58dc0caab0 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e4184700 1 -- 172.21.15.73:0/2361572284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0caab0 msgr2=0x7f58dc0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.060+0000 7f58e4184700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0caab0 0x7f58dc0aa470 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 rx=0x7f58cc009f70 tx=0x7f58cc009840).stop 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 -- 172.21.15.73:0/2361572284 shutdown_connections 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 --2- 172.21.15.73:0/2361572284 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0caab0 0x7f58dc0aa470 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 -- 172.21.15.73:0/2361572284 >> 172.21.15.73:0/2361572284 conn(0x7f58dc094920 msgr2=0x7f58dc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:12.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 -- 172.21.15.73:0/2361572284 shutdown_connections 2023-11-28T03:11:12.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.061+0000 7f58e4184700 1 -- 172.21.15.73:0/2361572284 wait complete. 2023-11-28T03:11:12.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 Processor -- start 2023-11-28T03:11:12.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 -- start start 2023-11-28T03:11:12.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc1482c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0caab0 0x7f58dc0ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0ad0e0 con 0x7f58dc0cae90 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0aa7d0 con 0x7f58dc0caab0 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.062+0000 7f58e4184700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58dc0c40b0 con 0x7f58dc0ca180 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33982/0 (socket says 172.21.15.73:33982) 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e1f20700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc1482c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 -- 172.21.15.73:0/439376625 learned_addr learned my addr 172.21.15.73:0/439376625 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e171f700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0caab0 0x7f58dc0ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 -- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 msgr2=0x7f58dc1482c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc1482c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 -- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0caab0 msgr2=0x7f58dc0ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0caab0 0x7f58dc0ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 -- 172.21.15.73:0/439376625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58cc009ae0 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e2721700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f58d400e930 tx=0x7f58d4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4012600 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d4012760 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58dc0c3900 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d4012ae0 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.063+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58dc0aa460 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.065+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58dc0aa960 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.065+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f58d401b400 con 0x7f58dc0cae90 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.065+0000 7f58d9ffb700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 0x7f58c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.065+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f58d407a470 con 0x7f58dc0cae90 2023-11-28T03:11:12.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.065+0000 7f58e1f20700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 0x7f58c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.066+0000 7f58e1f20700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 0x7f58c8061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f58cc00c010 tx=0x7f58cc009840).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.069+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58d404b810 con 0x7f58dc0cae90 2023-11-28T03:11:12.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.294+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f58dc06c860 con 0x7f58dc0cae90 2023-11-28T03:11:12.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.337+0000 7f58d9ffb700 1 -- 172.21.15.73:0/439376625 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v681) v1 ==== 148+0+0 (secure 0 0 0) 0x7f58d4016020 con 0x7f58dc0cae90 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 msgr2=0x7f58c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 0x7f58c8061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f58cc00c010 tx=0x7f58cc009840).stop 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 msgr2=0x7f58dc0acbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 rx=0x7f58d400e930 tx=0x7f58d4009060).stop 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 shutdown_connections 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f58c805f150 0x7f58c8061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58dc0cae90 0x7f58dc0acbf0 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58dc0ca180 0x7f58dc1482c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 --2- 172.21.15.73:0/439376625 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58dc0caab0 0x7f58dc0ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 >> 172.21.15.73:0/439376625 conn(0x7f58dc094920 msgr2=0x7f58dc0b0040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:12.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 shutdown_connections 2023-11-28T03:11:12.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.339+0000 7f58e4184700 1 -- 172.21.15.73:0/439376625 wait complete. 2023-11-28T03:11:12.353 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-11-28T03:11:12.353 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-11-28T03:11:12.353 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T03:11:12.353 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-11-28T03:11:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.465+0000 7f7fa925f700 1 Processor -- start 2023-11-28T03:11:12.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.465+0000 7f7fa925f700 1 -- start start 2023-11-28T03:11:12.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.465+0000 7f7fa925f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7fa40caab0 0x7f7fa40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa925f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fa40cae90 0x7f7fa40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa925f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fa40a9a90 con 0x7f7fa40aa960 2023-11-28T03:11:12.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa925f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa4154ce0 con 0x7f7fa40cae90 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa925f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fa40a8e80 con 0x7f7fa40caab0 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fa40cae90 0x7f7fa40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fa40cae90 0x7f7fa40c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:33984/0 (socket says 172.21.15.73:33984) 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 -- 172.21.15.73:0/3800999819 learned_addr learned my addr 172.21.15.73:0/3800999819 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa2d9d700 1 --2- 172.21.15.73:0/3800999819 >> v2:172.21.15.150:3300/0 conn(0x7f7fa40caab0 0x7f7fa40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 -- 172.21.15.73:0/3800999819 >> v2:172.21.15.150:3300/0 conn(0x7f7fa40caab0 msgr2=0x7f7fa40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 --2- 172.21.15.73:0/3800999819 >> v2:172.21.15.150:3300/0 conn(0x7f7fa40caab0 0x7f7fa40aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 -- 172.21.15.73:0/3800999819 >> v1:172.21.15.73:6789/0 conn(0x7f7fa40aa960 legacy=0x7f7fa40c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 -- 172.21.15.73:0/3800999819 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fa40a9160 con 0x7f7fa40cae90 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa259c700 1 --2- 172.21.15.73:0/3800999819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40cae90 0x7f7fa40c0ab0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f7fa40848c0 tx=0x7f7f980090f0).ready entity=mon.0 client_cookie=a2c06a0701e2ef28 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa1d9b700 1 -- 172.21.15.73:0/3800999819 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800b600 con 0x7f7fa40cae90 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa1d9b700 1 -- 172.21.15.73:0/3800999819 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9800b760 con 0x7f7fa40cae90 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.466+0000 7f7fa1d9b700 1 -- 172.21.15.73:0/3800999819 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9800bae0 con 0x7f7fa40cae90 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 -- 172.21.15.73:0/3800999819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40cae90 msgr2=0x7f7fa40c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 --2- 172.21.15.73:0/3800999819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40cae90 0x7f7fa40c0ab0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 rx=0x7f7fa40848c0 tx=0x7f7f980090f0).stop 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 -- 172.21.15.73:0/3800999819 shutdown_connections 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 --2- 172.21.15.73:0/3800999819 >> v2:172.21.15.150:3300/0 conn(0x7f7fa40caab0 0x7f7fa40aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 --2- 172.21.15.73:0/3800999819 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40cae90 0x7f7fa40c0ab0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 -- 172.21.15.73:0/3800999819 >> 172.21.15.73:0/3800999819 conn(0x7f7fa4094920 msgr2=0x7f7fa406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 -- 172.21.15.73:0/3800999819 shutdown_connections 2023-11-28T03:11:12.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.467+0000 7f7fa925f700 1 -- 172.21.15.73:0/3800999819 wait complete. 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.468+0000 7f7fa925f700 1 Processor -- start 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.468+0000 7f7fa925f700 1 -- start start 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 0x7f7fa4154eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 0x7f7fa4153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4155ef0 con 0x7f7fa40caab0 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4155930 con 0x7f7fa40cae90 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa925f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fa4155c10 con 0x7f7fa41553a0 2023-11-28T03:11:12.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34000/0 (socket says 172.21.15.73:34000) 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 -- 172.21.15.73:0/1212637099 learned_addr learned my addr 172.21.15.73:0/1212637099 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa259c700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 0x7f7fa4154eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa359e700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 0x7f7fa4153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 -- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 msgr2=0x7f7fa4153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 0x7f7fa4153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 -- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 msgr2=0x7f7fa4154eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 0x7f7fa4154eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 -- 172.21.15.73:0/1212637099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f98009370 con 0x7f7fa40caab0 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa359e700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 0x7f7fa4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa259c700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 0x7f7fa4154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:12.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7fa2d9d700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f7f9c00c880 tx=0x7f7f9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.469+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c010580 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.470+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f9c0106e0 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.470+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c010a60 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.470+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fa40c2fd0 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.470+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fa414c3e0 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.470+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fa40c2e30 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.471+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7f9c01b460 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.471+0000 7f7f8affd700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 0x7f7f90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.471+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f7f9c07a030 con 0x7f7fa40caab0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.471+0000 7f7fa259c700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 0x7f7f90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:12.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.471+0000 7f7fa259c700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 0x7f7f90061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7f98009130 tx=0x7f7f98017000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:12.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.475+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7fa40c2e30 con 0x7f7fa40caab0 2023-11-28T03:11:12.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:12.706+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f7fa40c2e30 con 0x7f7fa40caab0 2023-11-28T03:11:13.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.339+0000 7f7f8affd700 1 -- 172.21.15.73:0/1212637099 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v682) v1 ==== 181+0+0 (secure 0 0 0) 0x7f7fa40c2e30 con 0x7f7fa40caab0 2023-11-28T03:11:13.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 msgr2=0x7f7f90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 0x7f7f90061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f7f98009130 tx=0x7f7f98017000).stop 2023-11-28T03:11:13.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 msgr2=0x7f7fa414bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 rx=0x7f7f9c00c880 tx=0x7f7f9c008040).stop 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 shutdown_connections 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7f9005f150 0x7f7f90061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fa41553a0 0x7f7fa4153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fa40caab0 0x7f7fa414bef0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 --2- 172.21.15.73:0/1212637099 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fa40cae90 0x7f7fa4154eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 >> 172.21.15.73:0/1212637099 conn(0x7f7fa4094920 msgr2=0x7f7fa40ac010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 shutdown_connections 2023-11-28T03:11:13.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.341+0000 7f7fa925f700 1 -- 172.21.15.73:0/1212637099 wait complete. 2023-11-28T03:11:13.355 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T03:11:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.471+0000 7faf51ebc700 1 Processor -- start 2023-11-28T03:11:13.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.471+0000 7faf51ebc700 1 -- start start 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf51ebc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faf4c0c6190 0x7faf4c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf51ebc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c158320 0x7faf4c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf51ebc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf4c158cc0 con 0x7faf4c098d10 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf51ebc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf4c0c72d0 con 0x7faf4c0c6190 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf51ebc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c064570 con 0x7faf4c158320 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf4b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faf4c098d10 0x7faf4c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37200/0 (socket says 172.21.15.73:37200) 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.472+0000 7faf4b7fe700 1 -- 172.21.15.73:0/1830031414 learned_addr learned my addr 172.21.15.73:0/1830031414 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf43fff700 1 --2- 172.21.15.73:0/1830031414 >> v2:172.21.15.150:3300/0 conn(0x7faf4c0c6190 0x7faf4c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c158320 0x7faf4c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 347254455 0 0) 0x7faf3c001110 con 0x7faf4c098d10 2023-11-28T03:11:13.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf380033f0 con 0x7faf4c098d10 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3384600842 0 0) 0x7faf380033f0 con 0x7faf4c098d10 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c158320 msgr2=0x7faf4c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c158320 0x7faf4c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 >> v2:172.21.15.150:3300/0 conn(0x7faf4c0c6190 msgr2=0x7faf4c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 --2- 172.21.15.73:0/1830031414 >> v2:172.21.15.150:3300/0 conn(0x7faf4c0c6190 0x7faf4c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c154bc0 con 0x7faf4c098d10 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7faf3c001600 con 0x7faf4c098d10 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 >> v1:172.21.15.73:6789/0 conn(0x7faf4c098d10 legacy=0x7faf4c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380038e0 0x7faf38003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38004690 0x7faf38004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf38005000 0x7faf3800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0c72d0 con 0x7faf38005000 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c064570 con 0x7faf380038e0 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf3800bdd0 con 0x7faf38004690 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c158320 0x7faf4c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf43fff700 1 --2- 172.21.15.73:0/1830031414 >> v2:172.21.15.150:3300/0 conn(0x7faf4c0c6190 0x7faf4c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf38005000 0x7faf3800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4b7fe700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380038e0 0x7faf38003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf43fff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38004690 0x7faf38004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 -- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38004690 msgr2=0x7faf38004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf38004690 0x7faf38004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 -- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380038e0 msgr2=0x7faf38003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf380038e0 0x7faf38003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.473+0000 7faf4bfff700 1 -- 172.21.15.73:0/1830031414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf380033f0 con 0x7faf38005000 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf4bfff700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf38005000 0x7faf3800b8e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7faf4400e280 tx=0x7faf4400b040).ready entity=mon.0 client_cookie=fff7360d7e24f1a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ead0 con 0x7faf38005000 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4400f040 con 0x7faf38005000 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf4affd700 1 -- 172.21.15.73:0/1830031414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4400ee60 con 0x7faf38005000 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 -- 172.21.15.73:0/1830031414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf38005000 msgr2=0x7faf3800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 --2- 172.21.15.73:0/1830031414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf38005000 0x7faf3800b8e0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 rx=0x7faf4400e280 tx=0x7faf4400b040).stop 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf4bfff700 1 -- 172.21.15.73:0/1830031414 reap_dead start 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 -- 172.21.15.73:0/1830031414 shutdown_connections 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 -- 172.21.15.73:0/1830031414 >> 172.21.15.73:0/1830031414 conn(0x7faf4c0cb310 msgr2=0x7faf4c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 -- 172.21.15.73:0/1830031414 shutdown_connections 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.474+0000 7faf51ebc700 1 -- 172.21.15.73:0/1830031414 wait complete. 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.475+0000 7faf51ebc700 1 Processor -- start 2023-11-28T03:11:13.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.475+0000 7faf51ebc700 1 -- start start 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 0x7faf4c156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c149360 0x7faf4c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c156f30 con 0x7faf4c1682b0 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0b00c0 con 0x7faf4c148f80 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf51ebc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf4c0b0c40 con 0x7faf4c149360 2023-11-28T03:11:13.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41288/0 (socket says 172.21.15.73:41288) 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 -- 172.21.15.73:0/4060555925 learned_addr learned my addr 172.21.15.73:0/4060555925 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4b7fe700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 0x7faf4c156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf43fff700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c149360 0x7faf4c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 -- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c149360 msgr2=0x7faf4c156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c149360 0x7faf4c156a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 -- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 msgr2=0x7faf4c156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 0x7faf4c156550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4bfff700 1 -- 172.21.15.73:0/4060555925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf4c0ab570 con 0x7faf4c1682b0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.476+0000 7faf4b7fe700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 0x7faf4c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf4bfff700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7faf380044b0 tx=0x7faf44017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf44014e90 con 0x7faf4c1682b0 2023-11-28T03:11:13.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf4401f420 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4c0b0310 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf4401f7a0 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.477+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4c0b08d0 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.478+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf4c06c860 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.478+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faf4401fbd0 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.478+0000 7faf437fe700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 0x7faf2c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.478+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7faf4407b640 con 0x7faf4c1682b0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.479+0000 7faf4b7fe700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 0x7faf2c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.479+0000 7faf4b7fe700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 0x7faf2c0615e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faf3c001050 tx=0x7faf3c009040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.483+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faf4404ca90 con 0x7faf4c1682b0 2023-11-28T03:11:13.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.646+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7faf4c0b0fd0 con 0x7faf4c1682b0 2023-11-28T03:11:13.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.652+0000 7faf437fe700 1 -- 172.21.15.73:0/4060555925 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v26)=0 v26) v1 ==== 122+0+0 (secure 0 0 0) 0x7faf440190c0 con 0x7faf4c1682b0 2023-11-28T03:11:13.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 msgr2=0x7faf2c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 0x7faf2c0615e0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7faf3c001050 tx=0x7faf3c009040).stop 2023-11-28T03:11:13.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 msgr2=0x7faf4c168730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 rx=0x7faf380044b0 tx=0x7faf44017000).stop 2023-11-28T03:11:13.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 shutdown_connections 2023-11-28T03:11:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7faf2c05f120 0x7faf2c0615e0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf4c1682b0 0x7faf4c168730 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf4c148f80 0x7faf4c156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 --2- 172.21.15.73:0/4060555925 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf4c149360 0x7faf4c156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 >> 172.21.15.73:0/4060555925 conn(0x7faf4c0cb310 msgr2=0x7faf4c151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 shutdown_connections 2023-11-28T03:11:13.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.654+0000 7faf51ebc700 1 -- 172.21.15.73:0/4060555925 wait complete. 2023-11-28T03:11:13.667 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T03:11:13.667 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:11:13.707 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T03:11:13.750 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T03:11:13.750 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T03:11:13.751 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:13.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.863+0000 7ff5115ad700 1 Processor -- start 2023-11-28T03:11:13.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.863+0000 7ff5115ad700 1 -- start start 2023-11-28T03:11:13.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff5115ad700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff50c154c00 0x7ff50c069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff5115ad700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c06a270 con 0x7ff50c150890 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff5115ad700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff50c0c28c0 con 0x7ff50c069900 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff5115ad700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff50c1552e0 con 0x7ff50c154c00 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff50affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff50c150890 0x7ff50c0bbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37214/0 (socket says 172.21.15.73:37214) 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff50affd700 1 -- 172.21.15.73:0/1865014742 learned_addr learned my addr 172.21.15.73:0/1865014742 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.864+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/1865014742 >> v2:172.21.15.150:3300/0 conn(0x7ff50c154c00 0x7ff50c069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2151777 0 0) 0x7ff4fc001490 con 0x7ff50c150890 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4f40033f0 con 0x7ff50c150890 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199627674 0 0) 0x7ff50c0c28c0 con 0x7ff50c069900 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff4fc001490 con 0x7ff50c069900 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1962073193 0 0) 0x7ff4f40033f0 con 0x7ff50c150890 2023-11-28T03:11:13.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 >> v2:172.21.15.150:3300/0 conn(0x7ff50c154c00 msgr2=0x7ff50c069d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 --2- 172.21.15.73:0/1865014742 >> v2:172.21.15.150:3300/0 conn(0x7ff50c154c00 0x7ff50c069d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 >> v1:172.21.15.150:6789/0 conn(0x7ff50c069900 legacy=0x7ff50c1544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0bc540 con 0x7ff50c150890 2023-11-28T03:11:13.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff4fc001980 con 0x7ff50c150890 2023-11-28T03:11:13.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 >> v1:172.21.15.73:6789/0 conn(0x7ff50c150890 legacy=0x7ff50c0bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4003a70 0x7ff4f4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 0x7ff4f4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4f4005000 0x7ff4f400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c1552e0 con 0x7ff4f4004690 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f400be80 con 0x7ff4f4003a70 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff4f400c040 con 0x7ff4f4005000 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 0x7ff4f4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50affd700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4003a70 0x7ff4f4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4f4005000 0x7ff4f400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 -- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4f4005000 msgr2=0x7ff4f400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4f4005000 0x7ff4f400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 -- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4003a70 msgr2=0x7ff4f4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4003a70 0x7ff4f4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 -- 172.21.15.73:0/1865014742 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4f40033f0 con 0x7ff4f4004690 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff4f4005000 0x7ff4f400b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50b7fe700 1 -- 172.21.15.73:0/1865014742 reap_dead start 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff50a7fc700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 0x7ff4f4004b10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7ff500001860 tx=0x7ff500007bb0).ready entity=mon.0 client_cookie=427be3409d1dcf2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.865+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500007020 con 0x7ff4f4004690 2023-11-28T03:11:13.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff500007180 con 0x7ff4f4004690 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff509ffb700 1 -- 172.21.15.73:0/1865014742 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff500007520 con 0x7ff4f4004690 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 -- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 msgr2=0x7ff4f4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 0x7ff4f4004b10 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 rx=0x7ff500001860 tx=0x7ff500007bb0).stop 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 -- 172.21.15.73:0/1865014742 shutdown_connections 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 --2- 172.21.15.73:0/1865014742 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff4f4004690 0x7ff4f4004b10 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 -- 172.21.15.73:0/1865014742 >> 172.21.15.73:0/1865014742 conn(0x7ff50c0631d0 msgr2=0x7ff50c0c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 -- 172.21.15.73:0/1865014742 shutdown_connections 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.866+0000 7ff5115ad700 1 -- 172.21.15.73:0/1865014742 wait complete. 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.867+0000 7ff5115ad700 1 Processor -- start 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.867+0000 7ff5115ad700 1 -- start start 2023-11-28T03:11:13.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 0x7ff50c0ac9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff50c092c30 0x7ff50c0930b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c094300 con 0x7ff50c0aa350 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0aacc0 con 0x7ff4f4004690 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff5115ad700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff50c0acea0 con 0x7ff50c092c30 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41304/0 (socket says 172.21.15.73:41304) 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 -- 172.21.15.73:0/3780818487 learned_addr learned my addr 172.21.15.73:0/3780818487 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50a7fc700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff50c092c30 0x7ff50c0930b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50affd700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 0x7ff50c0ac9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 -- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff50c092c30 msgr2=0x7ff50c0930b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff50c092c30 0x7ff50c0930b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 -- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 msgr2=0x7ff50c0ac9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 0x7ff50c0ac9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 -- 172.21.15.73:0/3780818487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff50c0936f0 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50affd700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 0x7ff50c0ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.868+0000 7ff50b7fe700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7ff50c0c1330 tx=0x7ff504008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.869+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504013550 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.869+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5040136b0 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.869+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff50c0939d0 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.869+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff504013a50 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.869+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff50c093f90 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.870+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff50c06c860 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.870+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff504015440 con 0x7ff50c0aa350 2023-11-28T03:11:13.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.870+0000 7ff4f2ffd700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 0x7ff4f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:13.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.870+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(131..131 src has 1..131) v4 ==== 5813+0+0 (secure 0 0 0) 0x7ff504079d40 con 0x7ff50c0aa350 2023-11-28T03:11:13.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.870+0000 7ff50affd700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 0x7ff4f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:13.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.871+0000 7ff50affd700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 0x7ff4f8061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff4fc001460 tx=0x7ff4fc009040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:13.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:13.874+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff50404b0e0 con 0x7ff50c0aa350 2023-11-28T03:11:14.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.075+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff50c0c1cd0 con 0x7ff50c0aa350 2023-11-28T03:11:14.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.075+0000 7ff4f2ffd700 1 -- 172.21.15.73:0/3780818487 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3535 (secure 0 0 0) 0x7ff50404e700 con 0x7ff50c0aa350 2023-11-28T03:11:14.077 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:14.077 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6263,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1805785459","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1805785459},{"type":"v1","addr":"172.21.15.150:6834","nonce":1805785459}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":6266,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/3336098918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":3336098918},{"type":"v1","addr":"172.21.15.150:6835","nonce":3336098918}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":7593,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2230426074","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2230426074},{"type":"v1","addr":"172.21.15.73:6836","nonce":2230426074}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":43},{"gid":8004,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2595863191","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2595863191},{"type":"v1","addr":"172.21.15.73:6837","nonce":2595863191}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":47}],"filesystems":[]} 2023-11-28T03:11:14.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 msgr2=0x7ff4f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 0x7ff4f8061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff4fc001460 tx=0x7ff4fc009040).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 msgr2=0x7ff50c0aa7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 rx=0x7ff50c0c1330 tx=0x7ff504008040).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 shutdown_connections 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff4f805f150 0x7ff4f8061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff50c0aa350 0x7ff50c0aa7d0 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff4f4004690 0x7ff50c0ac9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 --2- 172.21.15.73:0/3780818487 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff50c092c30 0x7ff50c0930b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 >> 172.21.15.73:0/3780818487 conn(0x7ff50c0631d0 msgr2=0x7ff50c0bbe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:14.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 shutdown_connections 2023-11-28T03:11:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.077+0000 7ff5115ad700 1 -- 172.21.15.73:0/3780818487 wait complete. 2023-11-28T03:11:14.080 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 48 2023-11-28T03:11:14.091 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T03:11:14.091 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T03:11:14.091 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T03:11:14.091 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:11:14.091 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T03:11:14.092 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T03:11:14.092 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T03:11:14.092 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:11:14.092 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T03:11:14.092 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T03:11:14.092 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T03:11:14.092 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:11:14.093 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T03:11:14.093 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T03:11:14.093 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T03:11:14.093 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T03:11:14.101 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T03:11:14.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T03:11:14.102 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T03:11:14.103 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T03:11:14.104 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T03:11:14.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T03:11:14.105 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T03:11:14.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T03:11:14.144 INFO:tasks.ceph.mds.a:Started 2023-11-28T03:11:14.144 DEBUG:teuthology.parallel:result is None 2023-11-28T03:11:14.145 INFO:tasks.ceph.mds.c:Started 2023-11-28T03:11:14.145 DEBUG:teuthology.parallel:result is None 2023-11-28T03:11:14.146 INFO:tasks.ceph.mds.b:Started 2023-11-28T03:11:14.146 DEBUG:teuthology.parallel:result is None 2023-11-28T03:11:14.147 INFO:tasks.ceph.mds.d:Started 2023-11-28T03:11:14.147 DEBUG:teuthology.parallel:result is None 2023-11-28T03:11:14.147 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T03:11:14.216 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T03:11:14.217 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T03:11:14.231 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T03:11:14.232 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T03:11:14.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.268+0000 7fd7f72c9700 1 Processor -- start 2023-11-28T03:11:14.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.268+0000 7fd7f72c9700 1 -- start start 2023-11-28T03:11:14.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f72c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f0066a30 0x7fd7f0066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.271 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f72c9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7f00ac3d0 0x7fd7f013bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f72c9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7f00c75e0 con 0x7fd7f00abf50 2023-11-28T03:11:14.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f72c9700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7f00c6d20 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f72c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00c6f90 con 0x7fd7f0066a30 2023-11-28T03:11:14.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7f00ac3d0 0x7fd7f013bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7f00ac3d0 0x7fd7f013bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41326/0 (socket says 172.21.15.73:41326) 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/1177806918 learned_addr learned my addr 172.21.15.73:0/1177806918 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f62c7700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f0066a30 0x7fd7f0066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/1177806918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f0066a30 msgr2=0x7fd7f0066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f0066a30 0x7fd7f0066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/1177806918 >> v1:172.21.15.150:6789/0 conn(0x7fd7f00abf50 legacy=0x7fd7f013b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/1177806918 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00c7270 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.269+0000 7fd7f6ac8700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f013bb40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fd7e8009f20 tx=0x7fd7e8009840).ready entity=mon.0 client_cookie=23a9c4752e9b50b3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:14.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f52c5700 1 -- 172.21.15.73:0/1177806918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800e070 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f52c5700 1 -- 172.21.15.73:0/1177806918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e800ba80 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f52c5700 1 -- 172.21.15.73:0/1177806918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e800be00 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 -- 172.21.15.73:0/1177806918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 msgr2=0x7fd7f013bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f013bb40 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 rx=0x7fd7e8009f20 tx=0x7fd7e8009840).stop 2023-11-28T03:11:14.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 -- 172.21.15.73:0/1177806918 shutdown_connections 2023-11-28T03:11:14.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f013bb40 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/1177806918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f0066a30 0x7fd7f0066e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 -- 172.21.15.73:0/1177806918 >> 172.21.15.73:0/1177806918 conn(0x7fd7f00c8ed0 msgr2=0x7fd7f00c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:14.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 -- 172.21.15.73:0/1177806918 shutdown_connections 2023-11-28T03:11:14.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.270+0000 7fd7f72c9700 1 -- 172.21.15.73:0/1177806918 wait complete. 2023-11-28T03:11:14.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.271+0000 7fd7f72c9700 1 Processor -- start 2023-11-28T03:11:14.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.271+0000 7fd7f72c9700 1 -- start start 2023-11-28T03:11:14.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 0x7fd7f00ab1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 0x7fd7f0151970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f00aba80 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0063810 con 0x7fd7f0066a30 2023-11-28T03:11:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f72c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7f0063a20 con 0x7fd7f00abf50 2023-11-28T03:11:14.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41334/0 (socket says 172.21.15.73:41334) 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/3111832669 learned_addr learned my addr 172.21.15.73:0/3111832669 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f5ac6700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 0x7fd7f0151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f62c7700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 0x7fd7f00ab1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 msgr2=0x7fd7f0151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 0x7fd7f0151970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 msgr2=0x7fd7f00ab1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 0x7fd7f00ab1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f6ac8700 1 -- 172.21.15.73:0/3111832669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7e8009ae0 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f62c7700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 0x7fd7f00ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.272+0000 7fd7f5ac6700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 0x7fd7f0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7f6ac8700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fd7e800b6c0 tx=0x7fd7e8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80092e0 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7e8007610 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7f0063d00 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7e80079d0 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.273+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7f00647e0 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.274+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7f0064630 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.274+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd7e8003040 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.274+0000 7fd7e67fc700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 0x7fd7d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.274+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5785+0+0 (secure 0 0 0) 0x7fd7e807bcf0 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.274+0000 7fd7f62c7700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 0x7fd7d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.275+0000 7fd7f62c7700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 0x7fd7d4061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd7ec006fd0 tx=0x7fd7ec008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:14.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.278+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd7e8080070 con 0x7fd7f00ac3d0 2023-11-28T03:11:14.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b is failed for ~0s 2023-11-28T03:11:14.307 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-11-28T03:11:14.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:14.459+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fd7f0063ef0 con 0x7fd7f00ac3d0 2023-11-28T03:11:15.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.351+0000 7fd7e67fc700 1 -- 172.21.15.73:0/3111832669 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v136) v1 ==== 97+0+0 (secure 0 0 0) 0x7fd7e8024030 con 0x7fd7f00ac3d0 2023-11-28T03:11:15.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 msgr2=0x7fd7d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 0x7fd7d4061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7fd7ec006fd0 tx=0x7fd7ec008040).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 msgr2=0x7fd7f0137ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 rx=0x7fd7e800b6c0 tx=0x7fd7e8009840).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 shutdown_connections 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd7d405f150 0x7fd7d4061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7f00ac3d0 0x7fd7f0137ba0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd7f0066a30 0x7fd7f00ab1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 --2- 172.21.15.73:0/3111832669 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd7f00abf50 0x7fd7f0151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 >> 172.21.15.73:0/3111832669 conn(0x7fd7f00c8ed0 msgr2=0x7fd7f013a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 shutdown_connections 2023-11-28T03:11:15.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.353+0000 7fd7f72c9700 1 -- 172.21.15.73:0/3111832669 wait complete. 2023-11-28T03:11:15.356 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T03:11:15.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T03:11:15.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.482+0000 7f841b2e8700 1 Processor -- start 2023-11-28T03:11:15.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.482+0000 7f841b2e8700 1 -- start start 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f841b2e8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f841b2e8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f841b2e8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84140c69a0 con 0x7f841406a610 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f841b2e8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84140c13a0 con 0x7f84141544e0 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f841b2e8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84140c1680 con 0x7f84140c6e30 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8419885700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8419084700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8418883700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f841406a610 0x7f84140c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:50732/0 (socket says 172.21.15.73:50732) 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8419084700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:46990/0 (socket says 172.21.15.73:46990) 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8419885700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:40592/0 (socket says 172.21.15.73:40592) 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.483+0000 7f8418883700 1 -- 172.21.15.73:0/667479419 learned_addr learned my addr 172.21.15.73:0/667479419 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419885700 1 -- 172.21.15.73:0/667479419 >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 msgr2=0x7f84141548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419885700 1 --2- 172.21.15.73:0/667479419 >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419885700 1 -- 172.21.15.73:0/667479419 >> v1:172.21.15.150:6789/0 conn(0x7f841406a610 legacy=0x7f84140c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f840bfff700 1 -- 172.21.15.73:0/667479419 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1792692173 0 0) 0x7f84140c69a0 con 0x7f841406a610 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419885700 1 -- 172.21.15.73:0/667479419 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84140c6630 con 0x7f84140c6e30 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419084700 1 --2- 172.21.15.73:0/667479419 >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f8419885700 1 --2- 172.21.15.73:0/667479419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f840c009760 tx=0x7f840c009060).ready entity=mon.2 client_cookie=a2a0a4693693f55b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f840bfff700 1 -- 172.21.15.73:0/667479419 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00f510 con 0x7f84140c6e30 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f840bfff700 1 -- 172.21.15.73:0/667479419 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00f670 con 0x7f84140c6e30 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.484+0000 7f840bfff700 1 -- 172.21.15.73:0/667479419 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00f9f0 con 0x7f84140c6e30 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 -- 172.21.15.73:0/667479419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 msgr2=0x7f84140c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 --2- 172.21.15.73:0/667479419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 rx=0x7f840c009760 tx=0x7f840c009060).stop 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 -- 172.21.15.73:0/667479419 shutdown_connections 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 --2- 172.21.15.73:0/667479419 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84140c6e30 0x7f84140c72e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 --2- 172.21.15.73:0/667479419 >> v2:172.21.15.150:3300/0 conn(0x7f84141544e0 0x7f84141548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 -- 172.21.15.73:0/667479419 >> 172.21.15.73:0/667479419 conn(0x7f8414085020 msgr2=0x7f8414094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 -- 172.21.15.73:0/667479419 shutdown_connections 2023-11-28T03:11:15.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.485+0000 7f841b2e8700 1 -- 172.21.15.73:0/667479419 wait complete. 2023-11-28T03:11:15.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.486+0000 7f841b2e8700 1 Processor -- start 2023-11-28T03:11:15.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.486+0000 7f841b2e8700 1 -- start start 2023-11-28T03:11:15.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f841406a610 0x7f8414165510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84140c6e30 0x7f8414094940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414164d90 con 0x7f84141544e0 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8414095320 con 0x7f84140c6e30 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8414095550 con 0x7f841406a610 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41350/0 (socket says 172.21.15.73:41350) 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 -- 172.21.15.73:0/4119768388 learned_addr learned my addr 172.21.15.73:0/4119768388 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419084700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f841406a610 0x7f8414165510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8418883700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84140c6e30 0x7f8414094940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 -- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f841406a610 msgr2=0x7f8414165510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f841406a610 0x7f8414165510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 -- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84140c6e30 msgr2=0x7f8414094940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84140c6e30 0x7f8414094940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 -- 172.21.15.73:0/4119768388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8410009a20 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f8419885700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f840c009e00 tx=0x7f840c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c00f7e0 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840c00f960 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f840c0092a0 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840c01b690 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.487+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8414095b10 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.488+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f841406c860 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.489+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f840c01ba00 con 0x7f84141544e0 2023-11-28T03:11:15.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.489+0000 7f84097fa700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 0x7f83f8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.489+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f840c07b770 con 0x7f84141544e0 2023-11-28T03:11:15.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.489+0000 7f8419084700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 0x7f83f8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.489+0000 7f8419084700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 0x7f83f8061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8414167ea0 tx=0x7f841000b040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.493+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f840c04cd90 con 0x7f84141544e0 2023-11-28T03:11:15.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.719+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f84140c5890 con 0x7f84141544e0 2023-11-28T03:11:15.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.720+0000 7f84097fa700 1 -- 172.21.15.73:0/4119768388 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f840c01bcb0 con 0x7f84141544e0 2023-11-28T03:11:15.721 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:15.721 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:11:15.721 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T03:11:15.721 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.722 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T03:11:15.723 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T03:11:15.724 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.725 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.726 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.727 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.727 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.727 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.727 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T03:11:15.728 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.729 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.730 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.730 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.732 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.733 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T03:11:15.734 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.735 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:11:15.736 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.737 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.738 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T03:11:15.739 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T03:11:15.740 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:11:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 msgr2=0x7f83f8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 0x7f83f8061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f8414167ea0 tx=0x7f841000b040).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 msgr2=0x7f8414094e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 rx=0x7f840c009e00 tx=0x7f840c017000).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 shutdown_connections 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f83f805f150 0x7f83f8061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84141544e0 0x7f8414094e30 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f841406a610 0x7f8414165510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.721+0000 7f841b2e8700 1 --2- 172.21.15.73:0/4119768388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84140c6e30 0x7f8414094940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.722+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 >> 172.21.15.73:0/4119768388 conn(0x7f8414085020 msgr2=0x7f84140ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.722+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 shutdown_connections 2023-11-28T03:11:15.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.722+0000 7f841b2e8700 1 -- 172.21.15.73:0/4119768388 wait complete. 2023-11-28T03:11:15.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.857+0000 7fd703580700 1 Processor -- start 2023-11-28T03:11:15.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.857+0000 7fd703580700 1 -- start start 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd703580700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6fc0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd703580700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd6fc0cae90 0x7fd6fc0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd703580700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6fc0a9a90 con 0x7fd6fc0aa960 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd703580700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6fc154ce0 con 0x7fd6fc0cae90 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd703580700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0a8e80 con 0x7fd6fc0caab0 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd701b1d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd6fc0aa960 0x7fd6fc0c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37244/0 (socket says 172.21.15.73:37244) 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd701b1d700 1 -- 172.21.15.73:0/6118364 learned_addr learned my addr 172.21.15.73:0/6118364 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd70131c700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6fc0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3255141541 0 0) 0x7fd6fc0a9a90 con 0x7fd6fc0aa960 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e80033f0 con 0x7fd6fc0aa960 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd700b1b700 1 --2- 172.21.15.73:0/6118364 >> v2:172.21.15.150:3300/0 conn(0x7fd6fc0cae90 0x7fd6fc0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 965859027 0 0) 0x7fd6e80033f0 con 0x7fd6fc0aa960 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 msgr2=0x7fd6fc0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6fc0aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 >> v2:172.21.15.150:3300/0 conn(0x7fd6fc0cae90 msgr2=0x7fd6fc0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 --2- 172.21.15.73:0/6118364 >> v2:172.21.15.150:3300/0 conn(0x7fd6fc0cae90 0x7fd6fc0c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0a9160 con 0x7fd6fc0aa960 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fd6f40017f0 con 0x7fd6fc0aa960 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 >> v1:172.21.15.73:6789/0 conn(0x7fd6fc0aa960 legacy=0x7fd6fc0c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6e80038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6e8003e10 0x7fd6e8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6e8004b80 0x7fd6e8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc154ce0 con 0x7fd6e8004b80 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0a8e80 con 0x7fd6e8003e10 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.858+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6e800b8e0 con 0x7fd6fc0caab0 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6e8004b80 0x7fd6e8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd700b1b700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6e8003e10 0x7fd6e8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd70131c700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6e80038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 -- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 msgr2=0x7fd6e80038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc0caab0 0x7fd6e80038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 -- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6e8003e10 msgr2=0x7fd6e8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:15.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6e8003e10 0x7fd6e8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 -- 172.21.15.73:0/6118364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6e80033f0 con 0x7fd6e8004b80 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6e8004b80 0x7fd6e8005000 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fd6f400c560 tx=0x7fd6f4002210).ready entity=mon.0 client_cookie=eac4f21da4d6d7fa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4007de0 con 0x7fd6e8004b80 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f400ce50 con 0x7fd6e8004b80 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd6f3fff700 1 -- 172.21.15.73:0/6118364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4005d30 con 0x7fd6e8004b80 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd703580700 1 -- 172.21.15.73:0/6118364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6e8004b80 msgr2=0x7fd6e8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd703580700 1 --2- 172.21.15.73:0/6118364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6e8004b80 0x7fd6e8005000 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 rx=0x7fd6f400c560 tx=0x7fd6f4002210).stop 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd701b1d700 1 -- 172.21.15.73:0/6118364 reap_dead start 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd703580700 1 -- 172.21.15.73:0/6118364 shutdown_connections 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.859+0000 7fd703580700 1 -- 172.21.15.73:0/6118364 >> 172.21.15.73:0/6118364 conn(0x7fd6fc094920 msgr2=0x7fd6fc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.860+0000 7fd703580700 1 -- 172.21.15.73:0/6118364 shutdown_connections 2023-11-28T03:11:15.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.860+0000 7fd703580700 1 -- 172.21.15.73:0/6118364 wait complete. 2023-11-28T03:11:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.861+0000 7fd703580700 1 Processor -- start 2023-11-28T03:11:15.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.861+0000 7fd703580700 1 -- start start 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.861+0000 7fd703580700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6fc0b0150 0x7fd6fc0c4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc151a30 0x7fd6fc0b0530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0c2a00 con 0x7fd6fc0aec30 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0af480 con 0x7fd6fc0b0150 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6fc0af760 con 0x7fd6fc151a30 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41372/0 (socket says 172.21.15.73:41372) 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 -- 172.21.15.73:0/1387105157 learned_addr learned my addr 172.21.15.73:0/1387105157 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd70131c700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6fc0b0150 0x7fd6fc0c4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd700b1b700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc151a30 0x7fd6fc0b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 -- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc151a30 msgr2=0x7fd6fc0b0530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc151a30 0x7fd6fc0b0530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 -- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6fc0b0150 msgr2=0x7fd6fc0c4da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6fc0b0150 0x7fd6fc0c4da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 -- 172.21.15.73:0/1387105157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc0c23b0 con 0x7fd6fc0aec30 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd701b1d700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fd6f400c480 tx=0x7fd6f4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f4007de0 con 0x7fd6fc0aec30 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6f40017b0 con 0x7fd6fc0aec30 2023-11-28T03:11:15.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6fc0c2690 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6f40136a0 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.862+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6fc0af230 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.863+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6fc0c2b70 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.864+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd6f4013a50 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.864+0000 7fd6f17fa700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 0x7fd6e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.864+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd6f407b120 con 0x7fd6fc0aec30 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.864+0000 7fd70131c700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 0x7fd6e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.864+0000 7fd70131c700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 0x7fd6e0061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fd6f8006cf0 tx=0x7fd6f8006f40).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:15.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:15.868+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd6f404c740 con 0x7fd6fc0aec30 2023-11-28T03:11:16.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.031+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd6fc06c860 con 0x7fd6fc0aec30 2023-11-28T03:11:16.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.032+0000 7fd6f17fa700 1 -- 172.21.15.73:0/1387105157 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v136) v1 ==== 74+0+13723 (secure 0 0 0) 0x7fd6f4025070 con 0x7fd6fc0aec30 2023-11-28T03:11:16.033 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:16.033 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":136,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:11:15.341745+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":16,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":124,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:16.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 msgr2=0x7fd6e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 0x7fd6e0061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fd6f8006cf0 tx=0x7fd6f8006f40).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 msgr2=0x7fd6fc13fa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 rx=0x7fd6f400c480 tx=0x7fd6f4015000).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 shutdown_connections 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd6e005f150 0x7fd6e0061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd6fc0aec30 0x7fd6fc13fa60 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd6fc0b0150 0x7fd6fc0c4da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 --2- 172.21.15.73:0/1387105157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd6fc151a30 0x7fd6fc0b0530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 >> 172.21.15.73:0/1387105157 conn(0x7fd6fc094920 msgr2=0x7fd6fc0abe10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 shutdown_connections 2023-11-28T03:11:16.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.034+0000 7fd703580700 1 -- 172.21.15.73:0/1387105157 wait complete. 2023-11-28T03:11:16.047 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '131', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T03:11:16.048 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T03:11:16.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.162+0000 7fc810616700 1 Processor -- start 2023-11-28T03:11:16.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.162+0000 7fc810616700 1 -- start start 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc810616700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 0x7fc8080c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc810616700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc808158320 0x7fc8081587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc810616700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc808158cc0 con 0x7fc808098d10 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc810616700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8080c72d0 con 0x7fc808158320 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc810616700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc808064570 con 0x7fc8080c6190 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc80e3b2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc808098d10 0x7fc808063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37246/0 (socket says 172.21.15.73:37246) 2023-11-28T03:11:16.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc80e3b2700 1 -- 172.21.15.73:0/2359034455 learned_addr learned my addr 172.21.15.73:0/2359034455 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/2359034455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 0x7fc8080c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc80ebb3700 1 --2- 172.21.15.73:0/2359034455 >> v2:172.21.15.150:3300/0 conn(0x7fc808158320 0x7fc8081587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.163+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2543553749 0 0) 0x7fc804001110 con 0x7fc808098d10 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7f40033f0 con 0x7fc808098d10 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80dbb1700 1 -- 172.21.15.73:0/2359034455 >> v2:172.21.15.150:3300/0 conn(0x7fc808158320 msgr2=0x7fc8081587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/2359034455 >> v2:172.21.15.150:3300/0 conn(0x7fc808158320 0x7fc8081587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 214246986 0 0) 0x7fc7f40033f0 con 0x7fc808098d10 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80dbb1700 1 -- 172.21.15.73:0/2359034455 >> v1:172.21.15.73:6789/0 conn(0x7fc808098d10 legacy=0x7fc808063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80dbb1700 1 -- 172.21.15.73:0/2359034455 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc808154bc0 con 0x7fc8080c6190 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/2359034455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 0x7fc8080c7760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc7f8009760 tx=0x7fc7f8009060).ready entity=mon.2 client_cookie=1888027f3098522c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc804001600 con 0x7fc8080c6190 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f8011400 con 0x7fc8080c6190 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.164+0000 7fc80d3b0700 1 -- 172.21.15.73:0/2359034455 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8011780 con 0x7fc8080c6190 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 -- 172.21.15.73:0/2359034455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 msgr2=0x7fc8080c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 --2- 172.21.15.73:0/2359034455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 0x7fc8080c7760 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 rx=0x7fc7f8009760 tx=0x7fc7f8009060).stop 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 -- 172.21.15.73:0/2359034455 shutdown_connections 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 --2- 172.21.15.73:0/2359034455 >> v2:172.21.15.150:3300/0 conn(0x7fc808158320 0x7fc8081587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 --2- 172.21.15.73:0/2359034455 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8080c6190 0x7fc8080c7760 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 -- 172.21.15.73:0/2359034455 >> 172.21.15.73:0/2359034455 conn(0x7fc8080cb310 msgr2=0x7fc8080cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 -- 172.21.15.73:0/2359034455 shutdown_connections 2023-11-28T03:11:16.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.165+0000 7fc810616700 1 -- 172.21.15.73:0/2359034455 wait complete. 2023-11-28T03:11:16.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.166+0000 7fc810616700 1 Processor -- start 2023-11-28T03:11:16.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.166+0000 7fc810616700 1 -- start start 2023-11-28T03:11:16.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 0x7fc808165440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.171 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc808158320 0x7fc80814e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808155490 con 0x7fc8080c6190 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc808165930 con 0x7fc808098d10 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc808157280 con 0x7fc808158320 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41388/0 (socket says 172.21.15.73:41388) 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 -- 172.21.15.73:0/4140883134 learned_addr learned my addr 172.21.15.73:0/4140883134 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80e3b2700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 0x7fc808165440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80ebb3700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc808158320 0x7fc80814e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 -- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc808158320 msgr2=0x7fc80814e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc808158320 0x7fc80814e490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 -- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 msgr2=0x7fc808165440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 0x7fc808165440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 -- 172.21.15.73:0/4140883134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f80092a0 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80e3b2700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 0x7fc808165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc80dbb1700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fc8080646b0 tx=0x7fc7f8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8010870 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f80109d0 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc808155e10 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f8010d70 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.167+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc808156fa0 con 0x7fc8080c6190 2023-11-28T03:11:16.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.168+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc80806c860 con 0x7fc8080c6190 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.169+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc7f801d640 con 0x7fc8080c6190 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.169+0000 7fc7fe7fc700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 0x7fc7ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.169+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc7f807aae0 con 0x7fc8080c6190 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.169+0000 7fc80e3b2700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 0x7fc7ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.169+0000 7fc80e3b2700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 0x7fc7ec061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc804005b40 tx=0x7fc804001240).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.173+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc7f804c1b0 con 0x7fc8080c6190 2023-11-28T03:11:16.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.339+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fc8080c66c0 con 0x7fc8080c6190 2023-11-28T03:11:16.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.340+0000 7fc7fe7fc700 1 -- 172.21.15.73:0/4140883134 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v136) v1 ==== 110+0+10 (secure 0 0 0) 0x7fc7f8022030 con 0x7fc8080c6190 2023-11-28T03:11:16.341 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T03:11:16.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 msgr2=0x7fc7ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 0x7fc7ec061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fc804005b40 tx=0x7fc804001240).stop 2023-11-28T03:11:16.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 msgr2=0x7fc808155920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 rx=0x7fc8080646b0 tx=0x7fc7f8016000).stop 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 shutdown_connections 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7ec05f150 0x7fc7ec061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc808158320 0x7fc80814e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc808098d10 0x7fc808165440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 --2- 172.21.15.73:0/4140883134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8080c6190 0x7fc808155920 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 >> 172.21.15.73:0/4140883134 conn(0x7fc8080cb310 msgr2=0x7fc8080caa30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 shutdown_connections 2023-11-28T03:11:16.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.342+0000 7fc810616700 1 -- 172.21.15.73:0/4140883134 wait complete. 2023-11-28T03:11:16.356 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T03:11:16.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.472+0000 7f76f8038700 1 Processor -- start 2023-11-28T03:11:16.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.472+0000 7f76f8038700 1 -- start start 2023-11-28T03:11:16.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f8038700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76f00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f8038700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f76f00c6190 0x7f76f0063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f8038700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76f00c72d0 con 0x7f76f0064350 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f8038700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76f0154bc0 con 0x7f76f00c6190 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f8038700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c66c0 con 0x7f76f0098d10 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f65d5700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f76f0064350 0x7f76f0158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37248/0 (socket says 172.21.15.73:37248) 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f65d5700 1 -- 172.21.15.73:0/943354399 learned_addr learned my addr 172.21.15.73:0/943354399 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f55d3700 1 --2- 172.21.15.73:0/943354399 >> v2:172.21.15.150:3300/0 conn(0x7f76f00c6190 0x7f76f0063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76f00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330249896 0 0) 0x7f76f00c72d0 con 0x7f76f0064350 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.473+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f76d80033f0 con 0x7f76f0064350 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 54003615 0 0) 0x7f76d80033f0 con 0x7f76f0064350 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 msgr2=0x7f76f00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76f00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 >> v2:172.21.15.150:3300/0 conn(0x7f76f00c6190 msgr2=0x7f76f0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 --2- 172.21.15.73:0/943354399 >> v2:172.21.15.150:3300/0 conn(0x7f76f00c6190 0x7f76f0063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00c69a0 con 0x7f76f0064350 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f76e8001470 con 0x7f76f0064350 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 >> v1:172.21.15.73:6789/0 conn(0x7f76f0064350 legacy=0x7f76f0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d80038e0 0x7f76d8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d8004690 0x7f76d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76d8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f0154bc0 con 0x7f76d80038e0 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c66c0 con 0x7f76d8004690 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76d800b8e0 con 0x7f76f0098d10 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d80038e0 0x7f76d8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f65d5700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76d8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f55d3700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d8004690 0x7f76d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 -- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 msgr2=0x7f76d8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f0098d10 0x7f76d8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 -- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d8004690 msgr2=0x7f76d8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d8004690 0x7f76d8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 -- 172.21.15.73:0/943354399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76d80033f0 con 0x7f76d80038e0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d80038e0 0x7f76d8003d00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f76e000e210 tx=0x7f76e000b040).ready entity=mon.0 client_cookie=9c60d5e7c175f903 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e0009ac0 con 0x7f76d80038e0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e000f040 con 0x7f76d80038e0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.474+0000 7f76f4dd2700 1 -- 172.21.15.73:0/943354399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e000ecd0 con 0x7f76d80038e0 2023-11-28T03:11:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 -- 172.21.15.73:0/943354399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d80038e0 msgr2=0x7f76d8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 --2- 172.21.15.73:0/943354399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76d80038e0 0x7f76d8003d00 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 rx=0x7f76e000e210 tx=0x7f76e000b040).stop 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f65d5700 1 -- 172.21.15.73:0/943354399 reap_dead start 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 -- 172.21.15.73:0/943354399 shutdown_connections 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 -- 172.21.15.73:0/943354399 >> 172.21.15.73:0/943354399 conn(0x7f76f00cb310 msgr2=0x7f76f00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 -- 172.21.15.73:0/943354399 shutdown_connections 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.475+0000 7f76f8038700 1 -- 172.21.15.73:0/943354399 wait complete. 2023-11-28T03:11:16.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.476+0000 7f76f8038700 1 Processor -- start 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.476+0000 7f76f8038700 1 -- start start 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d80038e0 0x7f76f00ae330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f00c8320 0x7f76f00c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00ae820 con 0x7f76f00c75e0 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00aff50 con 0x7f76d80038e0 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f8038700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76f00c8c70 con 0x7f76f00c8320 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41412/0 (socket says 172.21.15.73:41412) 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 -- 172.21.15.73:0/95442143 learned_addr learned my addr 172.21.15.73:0/95442143 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 -- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f00c8320 msgr2=0x7f76f00c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f00c8320 0x7f76f00c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d80038e0 0x7f76f00ae330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 -- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d80038e0 msgr2=0x7f76f00ae330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d80038e0 0x7f76f00ae330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 -- 172.21.15.73:0/95442143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76f00c9bb0 con 0x7f76f00c75e0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76f55d3700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f76e4011a20 tx=0x7f76e4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.477+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400f940 con 0x7f76f00c75e0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.478+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76e400faa0 con 0x7f76f00c75e0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.478+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76f00afc20 con 0x7f76f00c75e0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.478+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76e400fe20 con 0x7f76f00c75e0 2023-11-28T03:11:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.478+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76f00ca120 con 0x7f76f00c75e0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.479+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76f00b00e0 con 0x7f76f00c75e0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.479+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f76e4014650 con 0x7f76f00c75e0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.479+0000 7f76edffb700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 0x7f76dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.479+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f76e407a6c0 con 0x7f76f00c75e0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.479+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 0x7f76dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.480+0000 7f76f5dd4700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 0x7f76dc061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f76f0154750 tx=0x7f76e001a000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.483+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76e404bce0 con 0x7f76f00c75e0 2023-11-28T03:11:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.645+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f76f006c860 con 0x7f76f00c75e0 2023-11-28T03:11:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.646+0000 7f76edffb700 1 -- 172.21.15.73:0/95442143 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v136) v1 ==== 92+0+10 (secure 0 0 0) 0x7f76e40220a0 con 0x7f76f00c75e0 2023-11-28T03:11:16.647 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T03:11:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 msgr2=0x7f76dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 0x7f76dc061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f76f0154750 tx=0x7f76e001a000).stop 2023-11-28T03:11:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 msgr2=0x7f76f00c7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 rx=0x7f76e4011a20 tx=0x7f76e4009060).stop 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 shutdown_connections 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f76dc05f150 0x7f76dc061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f76f00c8320 0x7f76f00c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f76d80038e0 0x7f76f00ae330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 --2- 172.21.15.73:0/95442143 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76f00c75e0 0x7f76f00c7e30 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 >> 172.21.15.73:0/95442143 conn(0x7f76f00cb310 msgr2=0x7f76f00c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 shutdown_connections 2023-11-28T03:11:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.648+0000 7f76f8038700 1 -- 172.21.15.73:0/95442143 wait complete. 2023-11-28T03:11:16.661 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T03:11:16.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.772+0000 7f95b6a4d700 1 Processor -- start 2023-11-28T03:11:16.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.773+0000 7f95b6a4d700 1 -- start start 2023-11-28T03:11:16.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.773+0000 7f95b6a4d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.773+0000 7f95b6a4d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00a9a90 con 0x7f95b00caab0 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.773+0000 7f95b6a4d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b0154ce0 con 0x7f95b00cae90 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.773+0000 7f95b6a4d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00aab80 con 0x7f95b00ca180 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95affff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f95b00caab0 0x7f95b00aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37262/0 (socket says 172.21.15.73:37262) 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95affff700 1 -- 172.21.15.73:0/3322934633 learned_addr learned my addr 172.21.15.73:0/3322934633 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95b4fea700 1 --2- 172.21.15.73:0/3322934633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3616206138 0 0) 0x7f95b00a9a90 con 0x7f95b00caab0 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f959c0033f0 con 0x7f95b00caab0 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4266129477 0 0) 0x7f95b0154ce0 con 0x7f95b00cae90 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f95b00a9a90 con 0x7f95b00cae90 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059052593 0 0) 0x7f959c0033f0 con 0x7f95b00caab0 2023-11-28T03:11:16.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95b4fea700 1 -- 172.21.15.73:0/3322934633 >> v1:172.21.15.150:6789/0 conn(0x7f95b00cae90 legacy=0x7f95b00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95b4fea700 1 -- 172.21.15.73:0/3322934633 >> v1:172.21.15.73:6789/0 conn(0x7f95b00caab0 legacy=0x7f95b00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95b4fea700 1 -- 172.21.15.73:0/3322934633 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00a8e80 con 0x7f95b00ca180 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95b4fea700 1 --2- 172.21.15.73:0/3322934633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b015f3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f95a8009e90 tx=0x7f95a8009780).ready entity=mon.2 client_cookie=f528a1f1a8a131ea server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a800e070 con 0x7f95b00ca180 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a800b7a0 con 0x7f95b00ca180 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.774+0000 7f95af7fe700 1 -- 172.21.15.73:0/3322934633 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a00015f0 con 0x7f95b00ca180 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 -- 172.21.15.73:0/3322934633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 msgr2=0x7f95b015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/3322934633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b015f3d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 rx=0x7f95a8009e90 tx=0x7f95a8009780).stop 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 -- 172.21.15.73:0/3322934633 shutdown_connections 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/3322934633 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b015f3d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 -- 172.21.15.73:0/3322934633 >> 172.21.15.73:0/3322934633 conn(0x7f95b0094920 msgr2=0x7f95b006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 -- 172.21.15.73:0/3322934633 shutdown_connections 2023-11-28T03:11:16.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.775+0000 7f95b6a4d700 1 -- 172.21.15.73:0/3322934633 wait complete. 2023-11-28T03:11:16.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.776+0000 7f95b6a4d700 1 Processor -- start 2023-11-28T03:11:16.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.776+0000 7f95b6a4d700 1 -- start start 2023-11-28T03:11:16.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b00afbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00cae90 0x7f95b0149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b00c0 con 0x7f95b00caab0 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00b0ec0 con 0x7f95b00cae90 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00c4ee0 con 0x7f95b00ca180 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41424/0 (socket says 172.21.15.73:41424) 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 -- 172.21.15.73:0/1003664190 learned_addr learned my addr 172.21.15.73:0/1003664190 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95affff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b00afbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b4fea700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00cae90 0x7f95b0149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 -- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 msgr2=0x7f95b00afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b00afbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 -- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00cae90 msgr2=0x7f95b0149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00cae90 0x7f95b0149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 -- 172.21.15.73:0/1003664190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a8009a20 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95a7fff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f959800c440 tx=0x7f9598007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800f470 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959800f5d0 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b00b02b0 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.777+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959800f990 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.778+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00b0870 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.778+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00ca8f0 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.779+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f959800fd40 con 0x7f95b00caab0 2023-11-28T03:11:16.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.779+0000 7f95acff9700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 0x7f9590061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.779+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f95980794a0 con 0x7f95b00caab0 2023-11-28T03:11:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.779+0000 7f95affff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 0x7f9590061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.780+0000 7f95affff700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 0x7f9590061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f95a0000c00 tx=0x7f95a0009040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:16.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.783+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f959804aac0 con 0x7f95b00caab0 2023-11-28T03:11:16.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.946+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f95b006c860 con 0x7f95b00caab0 2023-11-28T03:11:16.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.947+0000 7f95acff9700 1 -- 172.21.15.73:0/1003664190 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v136) v1 ==== 84+0+143 (secure 0 0 0) 0x7f959804e0e0 con 0x7f95b00caab0 2023-11-28T03:11:16.948 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:16.948 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:11:16.948 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:16.948 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:16.949 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:11:16.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 msgr2=0x7f9590061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 0x7f9590061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f95a0000c00 tx=0x7f95a0009040).stop 2023-11-28T03:11:16.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 msgr2=0x7f95b00c49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 rx=0x7f959800c440 tx=0x7f9598007c90).stop 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 shutdown_connections 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f959005f150 0x7f9590061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b00cae90 0x7f95b0149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00ca180 0x7f95b00afbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 --2- 172.21.15.73:0/1003664190 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b00caab0 0x7f95b00c49f0 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 >> 172.21.15.73:0/1003664190 conn(0x7f95b0094920 msgr2=0x7f95b00ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 shutdown_connections 2023-11-28T03:11:16.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:16.949+0000 7f95b6a4d700 1 -- 172.21.15.73:0/1003664190 wait complete. 2023-11-28T03:11:16.962 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T03:11:16.962 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T03:11:17.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.079+0000 7f6f7253e700 1 Processor -- start 2023-11-28T03:11:17.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.079+0000 7f6f7253e700 1 -- start start 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f7253e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f6c0aa930 0x7f6f6c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f7253e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c0c6630 con 0x7f6f6c154910 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f7253e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f6c064230 con 0x7f6f6c154530 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f7253e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f6c0c6390 con 0x7f6f6c0aa930 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f70adb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6f6c0aa930 0x7f6f6c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6f6c154910 0x7f6f6c0aa220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37264/0 (socket says 172.21.15.73:37264) 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6b7fe700 1 -- 172.21.15.73:0/2473718610 learned_addr learned my addr 172.21.15.73:0/2473718610 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f70adb700 1 -- 172.21.15.73:0/2473718610 >> v1:172.21.15.150:6789/0 conn(0x7f6f6c154530 legacy=0x7f6f6c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:17.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f70adb700 1 -- 172.21.15.73:0/2473718610 >> v1:172.21.15.73:6789/0 conn(0x7f6f6c154910 legacy=0x7f6f6c0aa220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6affd700 1 -- 172.21.15.73:0/2473718610 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3372703520 0 0) 0x7f6f6c0c6630 con 0x7f6f6c154910 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f70adb700 1 -- 172.21.15.73:0/2473718610 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6c064510 con 0x7f6f6c0aa930 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f70adb700 1 --2- 172.21.15.73:0/2473718610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c0a7ff0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f6f64009e60 tx=0x7f6f64009780).ready entity=mon.0 client_cookie=27b6aa7474078f36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6affd700 1 -- 172.21.15.73:0/2473718610 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f6400e070 con 0x7f6f6c0aa930 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6affd700 1 -- 172.21.15.73:0/2473718610 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f6400bbf0 con 0x7f6f6c0aa930 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.080+0000 7f6f6affd700 1 -- 172.21.15.73:0/2473718610 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f64012410 con 0x7f6f6c0aa930 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 -- 172.21.15.73:0/2473718610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 msgr2=0x7f6f6c0a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 --2- 172.21.15.73:0/2473718610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c0a7ff0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 rx=0x7f6f64009e60 tx=0x7f6f64009780).stop 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 -- 172.21.15.73:0/2473718610 shutdown_connections 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 --2- 172.21.15.73:0/2473718610 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c0a7ff0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 -- 172.21.15.73:0/2473718610 >> 172.21.15.73:0/2473718610 conn(0x7f6f6c094930 msgr2=0x7f6f6c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 -- 172.21.15.73:0/2473718610 shutdown_connections 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.081+0000 7f6f7253e700 1 -- 172.21.15.73:0/2473718610 wait complete. 2023-11-28T03:11:17.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.082+0000 7f6f7253e700 1 Processor -- start 2023-11-28T03:11:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.082+0000 7f6f7253e700 1 -- start start 2023-11-28T03:11:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 0x7f6f6c0ae230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f6c154910 0x7f6f6c14fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0af160 con 0x7f6f6c0aa930 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0ab120 con 0x7f6f6c154530 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f7253e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f6c0aeba0 con 0x7f6f6c154910 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41438/0 (socket says 172.21.15.73:41438) 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 -- 172.21.15.73:0/3903457631 learned_addr learned my addr 172.21.15.73:0/3903457631 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6b7fe700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 0x7f6f6c0ae230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f70adb700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f6c154910 0x7f6f6c14fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 -- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f6c154910 msgr2=0x7f6f6c14fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f6c154910 0x7f6f6c14fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 -- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 msgr2=0x7f6f6c0ae230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 0x7f6f6c0ae230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 -- 172.21.15.73:0/3903457631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f64009a20 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6b7fe700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 0x7f6f6c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.083+0000 7f6f6bfff700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f6f5c00c460 tx=0x7f6f5c007e30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.084+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00f470 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.084+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c00f5d0 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.084+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f6c0aee80 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.084+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c00f990 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.084+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f6c1432f0 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.085+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6c06c860 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.085+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6f5c012400 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.085+0000 7f6f53fff700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 0x7f6f58061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.085+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f6f5c079340 con 0x7f6f6c0aa930 2023-11-28T03:11:17.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.085+0000 7f6f6b7fe700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 0x7f6f58061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.086+0000 7f6f6b7fe700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 0x7f6f58061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6f60007fd0 tx=0x7f6f60009040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.089+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6f5c048af0 con 0x7f6f6c0aa930 2023-11-28T03:11:17.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.252+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f6f6c0aaf70 con 0x7f6f6c0aa930 2023-11-28T03:11:17.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.363+0000 7f6f53fff700 1 -- 172.21.15.73:0/3903457631 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v137) v1 ==== 152+0+0 (secure 0 0 0) 0x7f6f5c04a9b0 con 0x7f6f6c0aa930 2023-11-28T03:11:17.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 msgr2=0x7f6f58061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 0x7f6f58061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f6f60007fd0 tx=0x7f6f60009040).stop 2023-11-28T03:11:17.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 msgr2=0x7f6f6c142e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 rx=0x7f6f5c00c460 tx=0x7f6f5c007e30).stop 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 shutdown_connections 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6f5805f150 0x7f6f58061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6f6c154910 0x7f6f6c14fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6f6c0aa930 0x7f6f6c142e00 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 --2- 172.21.15.73:0/3903457631 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6f6c154530 0x7f6f6c0ae230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.365+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 >> 172.21.15.73:0/3903457631 conn(0x7f6f6c094930 msgr2=0x7f6f6c0c1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:17.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.366+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 shutdown_connections 2023-11-28T03:11:17.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.366+0000 7f6f7253e700 1 -- 172.21.15.73:0/3903457631 wait complete. 2023-11-28T03:11:17.368 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T03:11:17.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T03:11:17.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 Processor -- start 2023-11-28T03:11:17.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 -- start start 2023-11-28T03:11:17.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 0x7f9ff80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ff80c72d0 con 0x7f9ff8064350 2023-11-28T03:11:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ff8154bc0 con 0x7f9ff8098d10 2023-11-28T03:11:17.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.502+0000 7f9fff144700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80c66c0 con 0x7f9ff80c6190 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9ffcee0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 0x7f9ff80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:40714/0 (socket says 172.21.15.73:40714) 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 -- 172.21.15.73:0/1747403 learned_addr learned my addr 172.21.15.73:0/1747403 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9ffd6e1700 1 --1- 172.21.15.73:0/1747403 >> v1:172.21.15.150:6789/0 conn(0x7f9ff8064350 0x7f9ff8158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:50766/0 (socket says 172.21.15.73:50766) 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9fef7fe700 1 -- 172.21.15.73:0/1747403 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2946968733 0 0) 0x7f9ff80c72d0 con 0x7f9ff8064350 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9fef7fe700 1 -- 172.21.15.73:0/1747403 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9fe00033f0 con 0x7f9ff8064350 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 -- 172.21.15.73:0/1747403 >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 msgr2=0x7f9ff80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 --2- 172.21.15.73:0/1747403 >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 0x7f9ff80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 -- 172.21.15.73:0/1747403 >> v1:172.21.15.150:6789/0 conn(0x7f9ff8064350 legacy=0x7f9ff8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 -- 172.21.15.73:0/1747403 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff80c69a0 con 0x7f9ff80c6190 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9ffcee0700 1 --2- 172.21.15.73:0/1747403 >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 0x7f9ff80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.503+0000 7f9feffff700 1 --2- 172.21.15.73:0/1747403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9fe8006fd0 tx=0x7f9fe80090f0).ready entity=mon.2 client_cookie=2780a98ec1f59a82 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fef7fe700 1 -- 172.21.15.73:0/1747403 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800f510 con 0x7f9ff80c6190 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fef7fe700 1 -- 172.21.15.73:0/1747403 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fe800f670 con 0x7f9ff80c6190 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fef7fe700 1 -- 172.21.15.73:0/1747403 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe800f9f0 con 0x7f9ff80c6190 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 -- 172.21.15.73:0/1747403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 msgr2=0x7f9ff8063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 --2- 172.21.15.73:0/1747403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 rx=0x7f9fe8006fd0 tx=0x7f9fe80090f0).stop 2023-11-28T03:11:17.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 -- 172.21.15.73:0/1747403 shutdown_connections 2023-11-28T03:11:17.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 --2- 172.21.15.73:0/1747403 >> v2:172.21.15.150:3300/0 conn(0x7f9ff8098d10 0x7f9ff80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 --2- 172.21.15.73:0/1747403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff80c6190 0x7f9ff8063e60 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 -- 172.21.15.73:0/1747403 >> 172.21.15.73:0/1747403 conn(0x7f9ff80cb310 msgr2=0x7f9ff80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:17.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 -- 172.21.15.73:0/1747403 shutdown_connections 2023-11-28T03:11:17.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.504+0000 7f9fff144700 1 -- 172.21.15.73:0/1747403 wait complete. 2023-11-28T03:11:17.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.505+0000 7f9fff144700 1 Processor -- start 2023-11-28T03:11:17.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 -- start start 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff8098d10 0x7f9ff80ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff8148da0 0x7f9ff80afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80af720 con 0x7f9ff8064350 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80aeb10 con 0x7f9ff8148da0 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9fff144700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff80aedf0 con 0x7f9ff8098d10 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9ffcee0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9ffcee0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41452/0 (socket says 172.21.15.73:41452) 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.506+0000 7f9ffcee0700 1 -- 172.21.15.73:0/443179784 learned_addr learned my addr 172.21.15.73:0/443179784 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9feffff700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff8098d10 0x7f9ff80ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffd6e1700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff8148da0 0x7f9ff80afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 -- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff8098d10 msgr2=0x7f9ff80ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff8098d10 0x7f9ff80ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 -- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff8148da0 msgr2=0x7f9ff80afbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff8148da0 0x7f9ff80afbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 -- 172.21.15.73:0/443179784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff4009a20 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9ffcee0700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f9ff400c0c0 tx=0x7f9ff400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff400cd60 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ff400f040 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fe8009370 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff4013610 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.507+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff80af3b0 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.508+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ff806c860 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.508+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9ff40139c0 con 0x7f9ff8064350 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.509+0000 7f9fecff9700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 0x7f9fe4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:17.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.509+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f9ff407a9e0 con 0x7f9ff8064350 2023-11-28T03:11:17.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.509+0000 7f9feffff700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 0x7f9fe4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:17.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.509+0000 7f9feffff700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 0x7f9fe4061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fe8006fd0 tx=0x7f9fe8017000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:17.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.513+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9ff404be90 con 0x7f9ff8064350 2023-11-28T03:11:17.680 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:17.679+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f9ff80c6c80 con 0x7f9ff8064350 2023-11-28T03:11:18.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.369+0000 7f9fecff9700 1 -- 172.21.15.73:0/443179784 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v138) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9ff404f4b0 con 0x7f9ff8064350 2023-11-28T03:11:18.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.379+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 msgr2=0x7f9fe4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.379+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 0x7f9fe4061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f9fe8006fd0 tx=0x7f9fe8017000).stop 2023-11-28T03:11:18.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.379+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 msgr2=0x7f9ff80aa670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.379+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 rx=0x7f9ff400c0c0 tx=0x7f9ff400b040).stop 2023-11-28T03:11:18.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 shutdown_connections 2023-11-28T03:11:18.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9fe405f150 0x7f9fe4061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ff8148da0 0x7f9ff80afbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ff8064350 0x7f9ff80aa670 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 --2- 172.21.15.73:0/443179784 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ff8098d10 0x7f9ff80ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.380+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 >> 172.21.15.73:0/443179784 conn(0x7f9ff80cb310 msgr2=0x7f9ff814c360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:18.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.383+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 shutdown_connections 2023-11-28T03:11:18.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.383+0000 7f9fff144700 1 -- 172.21.15.73:0/443179784 wait complete. 2023-11-28T03:11:18.389 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T03:11:18.405 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T03:11:18.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.521+0000 7f210c938700 1 Processor -- start 2023-11-28T03:11:18.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.521+0000 7f210c938700 1 -- start start 2023-11-28T03:11:18.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f210c938700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f210814e530 0x7f21080c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f210c938700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f2108158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f210c938700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21080c7bc0 con 0x7f2108154950 2023-11-28T03:11:18.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f210c938700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2108155920 con 0x7f210814e530 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f210c938700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2108154e00 con 0x7f2108158570 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f210814e530 0x7f21080c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f210814e530 0x7f21080c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41460/0 (socket says 172.21.15.73:41460) 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 -- 172.21.15.73:0/3300115258 learned_addr learned my addr 172.21.15.73:0/3300115258 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 -- 172.21.15.73:0/3300115258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 msgr2=0x7f2108158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 --2- 172.21.15.73:0/3300115258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f2108158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:18.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 -- 172.21.15.73:0/3300115258 >> v1:172.21.15.150:6789/0 conn(0x7f2108154950 legacy=0x7f210814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:18.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 -- 172.21.15.73:0/3300115258 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2108155080 con 0x7f210814e530 2023-11-28T03:11:18.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21077fe700 1 --2- 172.21.15.73:0/3300115258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210814e530 0x7f21080c66f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f20f8009f20 tx=0x7f20f8009840).ready entity=mon.0 client_cookie=d156481d29db1699 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.522+0000 7f21067fc700 1 -- 172.21.15.73:0/3300115258 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800e070 con 0x7f210814e530 2023-11-28T03:11:18.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f21067fc700 1 -- 172.21.15.73:0/3300115258 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20f800ba40 con 0x7f210814e530 2023-11-28T03:11:18.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f21067fc700 1 -- 172.21.15.73:0/3300115258 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20f800bdc0 con 0x7f210814e530 2023-11-28T03:11:18.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 -- 172.21.15.73:0/3300115258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210814e530 msgr2=0x7f21080c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 --2- 172.21.15.73:0/3300115258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210814e530 0x7f21080c66f0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 rx=0x7f20f8009f20 tx=0x7f20f8009840).stop 2023-11-28T03:11:18.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 -- 172.21.15.73:0/3300115258 shutdown_connections 2023-11-28T03:11:18.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 --2- 172.21.15.73:0/3300115258 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f2108158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:18.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 --2- 172.21.15.73:0/3300115258 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210814e530 0x7f21080c66f0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 -- 172.21.15.73:0/3300115258 >> 172.21.15.73:0/3300115258 conn(0x7f2108064990 msgr2=0x7f2108064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:18.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 -- 172.21.15.73:0/3300115258 shutdown_connections 2023-11-28T03:11:18.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.523+0000 7f210c938700 1 -- 172.21.15.73:0/3300115258 wait complete. 2023-11-28T03:11:18.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.524+0000 7f210c938700 1 Processor -- start 2023-11-28T03:11:18.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.524+0000 7f210c938700 1 -- start start 2023-11-28T03:11:18.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210814e530 0x7f210814dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f21080af4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210814e130 con 0x7f2108154950 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21080c3510 con 0x7f210814e530 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f210c938700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2108169c70 con 0x7f2108158570 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41476/0 (socket says 172.21.15.73:41476) 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 -- 172.21.15.73:0/527812795 learned_addr learned my addr 172.21.15.73:0/527812795 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f21077fe700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210814e530 0x7f210814dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2107fff700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f21080af4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 -- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 msgr2=0x7f21080af4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f21080af4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 -- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210814e530 msgr2=0x7f210814dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210814e530 0x7f210814dc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.525+0000 7f2106ffd700 1 -- 172.21.15.73:0/527812795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20f8009ae0 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f2106ffd700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f20fc00c440 tx=0x7f20fc007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00cea0 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00f470 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00f810 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21080c2be0 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.526+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21080c31a0 con 0x7f2108154950 2023-11-28T03:11:18.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.527+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2108062ee0 con 0x7f2108154950 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.527+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f20fc00fc40 con 0x7f2108154950 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.527+0000 7f20ef7fe700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 0x7f20f4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.528+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f20fc079690 con 0x7f2108154950 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.528+0000 7f21077fe700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 0x7f20f4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.528+0000 7f21077fe700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 0x7f20f4061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f20f800c010 tx=0x7f20f8009840).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.532+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f20fc048af0 con 0x7f2108154950 2023-11-28T03:11:18.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.738+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f210806c7c0 con 0x7f2108154950 2023-11-28T03:11:18.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.744+0000 7f20ef7fe700 1 -- 172.21.15.73:0/527812795 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 17 and data pool 18 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7f20fc04a9b0 con 0x7f2108154950 2023-11-28T03:11:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.749+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 msgr2=0x7f20f4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.749+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 0x7f20f4061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f20f800c010 tx=0x7f20f8009840).stop 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.749+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 msgr2=0x7f21080af000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.749+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 rx=0x7f20fc00c440 tx=0x7f20fc007de0).stop 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 shutdown_connections 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20f405f150 0x7f20f4061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2108158570 0x7f21080af4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210814e530 0x7f210814dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 --2- 172.21.15.73:0/527812795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2108154950 0x7f21080af000 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.750+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 >> 172.21.15.73:0/527812795 conn(0x7f2108064990 msgr2=0x7f21080ad1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:18.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.753+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 shutdown_connections 2023-11-28T03:11:18.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.754+0000 7f210c938700 1 -- 172.21.15.73:0/527812795 wait complete. 2023-11-28T03:11:18.767 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 17 and data pool 18 2023-11-28T03:11:18.780 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.893+0000 7fb05564a700 1 Processor -- start 2023-11-28T03:11:18.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.893+0000 7fb05564a700 1 -- start start 2023-11-28T03:11:18.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.893+0000 7fb05564a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.893+0000 7fb05564a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb05564a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0500a9a90 con 0x7fb0500cae90 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb05564a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb050154ce0 con 0x7fb0500caab0 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb05564a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500a8e80 con 0x7fb0500ca180 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04effd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04effd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:47116/0 (socket says 172.21.15.73:47116) 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:40748/0 (socket says 172.21.15.73:40748) 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04effd700 1 -- 172.21.15.73:0/1425807235 learned_addr learned my addr 172.21.15.73:0/1425807235 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 -- 172.21.15.73:0/1425807235 >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 msgr2=0x7fb0500aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:18.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 --2- 172.21.15.73:0/1425807235 >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 -- 172.21.15.73:0/1425807235 >> v1:172.21.15.150:6789/0 conn(0x7fb0500cae90 legacy=0x7fb0500c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04f7fe700 1 -- 172.21.15.73:0/1425807235 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0500a9160 con 0x7fb0500ca180 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.894+0000 7fb04effd700 1 --2- 172.21.15.73:0/1425807235 >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb04f7fe700 1 --2- 172.21.15.73:0/1425807235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb048009760 tx=0x7fb048009060).ready entity=mon.2 client_cookie=d1e6b186b98acdd2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb04dffb700 1 -- 172.21.15.73:0/1425807235 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800f510 con 0x7fb0500ca180 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb04dffb700 1 -- 172.21.15.73:0/1425807235 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04800f670 con 0x7fb0500ca180 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb04dffb700 1 -- 172.21.15.73:0/1425807235 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04800f9f0 con 0x7fb0500ca180 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 -- 172.21.15.73:0/1425807235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 msgr2=0x7fb05015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 --2- 172.21.15.73:0/1425807235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 rx=0x7fb048009760 tx=0x7fb048009060).stop 2023-11-28T03:11:18.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 -- 172.21.15.73:0/1425807235 shutdown_connections 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 --2- 172.21.15.73:0/1425807235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500ca180 0x7fb05015e3c0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 --2- 172.21.15.73:0/1425807235 >> v2:172.21.15.150:3300/0 conn(0x7fb0500caab0 0x7fb0500aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 -- 172.21.15.73:0/1425807235 >> 172.21.15.73:0/1425807235 conn(0x7fb050094920 msgr2=0x7fb05006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 -- 172.21.15.73:0/1425807235 shutdown_connections 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.895+0000 7fb05564a700 1 -- 172.21.15.73:0/1425807235 wait complete. 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 Processor -- start 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 -- start start 2023-11-28T03:11:18.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500caab0 0x7fb0501683f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0500cae90 0x7fb0500ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0501688e0 con 0x7fb0500ca180 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500acd60 con 0x7fb0500cae90 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb05564a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0500ac4c0 con 0x7fb0500caab0 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb04effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb04effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41480/0 (socket says 172.21.15.73:41480) 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.897+0000 7fb04effd700 1 -- 172.21.15.73:0/3461120755 learned_addr learned my addr 172.21.15.73:0/3461120755 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04e7fc700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500caab0 0x7fb0501683f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04f7fe700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0500cae90 0x7fb0500ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 -- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500caab0 msgr2=0x7fb0501683f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500caab0 0x7fb0501683f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 -- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0500cae90 msgr2=0x7fb0500ab790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0500cae90 0x7fb0500ab790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 -- 172.21.15.73:0/3461120755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb040009ae0 con 0x7fb0500ca180 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb04effd700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fb04000c170 tx=0x7fb04000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000caa0 con 0x7fb0500ca180 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb04000f040 con 0x7fb0500ca180 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb04000ce10 con 0x7fb0500ca180 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0480092a0 con 0x7fb0500ca180 2023-11-28T03:11:18.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.898+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb050140d60 con 0x7fb0500ca180 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.899+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb05006c860 con 0x7fb0500ca180 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.899+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb040014670 con 0x7fb0500ca180 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.900+0000 7fb036ffd700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 0x7fb03c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.900+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb04007a190 con 0x7fb0500ca180 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.900+0000 7fb04e7fc700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 0x7fb03c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.900+0000 7fb04e7fc700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 0x7fb03c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb0500b0590 tx=0x7fb044007d40).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:18.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:18.904+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb04004aaf0 con 0x7fb0500ca180 2023-11-28T03:11:19.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.066+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb0500a9440 con 0x7fb0500ca180 2023-11-28T03:11:19.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.067+0000 7fb036ffd700 1 -- 172.21.15.73:0/3461120755 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16956 (secure 0 0 0) 0x7fb040054250 con 0x7fb0500ca180 2023-11-28T03:11:19.068 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:19.069 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":139,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:11:18.740195+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 msgr2=0x7fb03c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 0x7fb03c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb0500b0590 tx=0x7fb044007d40).stop 2023-11-28T03:11:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 msgr2=0x7fb050167f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 rx=0x7fb04000c170 tx=0x7fb04000b040).stop 2023-11-28T03:11:19.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 shutdown_connections 2023-11-28T03:11:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb03c05f150 0x7fb03c061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0500cae90 0x7fb0500ab790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0500ca180 0x7fb050167f00 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 --2- 172.21.15.73:0/3461120755 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0500caab0 0x7fb0501683f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 >> 172.21.15.73:0/3461120755 conn(0x7fb050094920 msgr2=0x7fb0500a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 shutdown_connections 2023-11-28T03:11:19.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.069+0000 7fb05564a700 1 -- 172.21.15.73:0/3461120755 wait complete. 2023-11-28T03:11:19.082 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.195+0000 7f2165fe0700 1 Processor -- start 2023-11-28T03:11:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.195+0000 7f2165fe0700 1 -- start start 2023-11-28T03:11:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f2165fe0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2160067cb0 0x7f21600c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f2165fe0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2160068090 0x7f21600c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f2165fe0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2160154d20 con 0x7f21600c2640 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f2165fe0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2160151670 con 0x7f2160068090 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f2165fe0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600c48a0 con 0x7f2160067cb0 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f215effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2160068090 0x7f21600c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f215effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2160068090 0x7f21600c43b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41492/0 (socket says 172.21.15.73:41492) 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.196+0000 7f215effd700 1 -- 172.21.15.73:0/3001935605 learned_addr learned my addr 172.21.15.73:0/3001935605 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215f7fe700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2160067cb0 0x7f21600c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215effd700 1 -- 172.21.15.73:0/3001935605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2160067cb0 msgr2=0x7f21600c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215effd700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2160067cb0 0x7f21600c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215effd700 1 -- 172.21.15.73:0/3001935605 >> v1:172.21.15.150:6789/0 conn(0x7f21600c2640 legacy=0x7f21600c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215effd700 1 -- 172.21.15.73:0/3001935605 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21600a7490 con 0x7f2160068090 2023-11-28T03:11:19.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215effd700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600c43b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f2154006fd0 tx=0x7f21540090f0).ready entity=mon.0 client_cookie=1916d345eeb63f9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215e7fc700 1 -- 172.21.15.73:0/3001935605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400b600 con 0x7f2160068090 2023-11-28T03:11:19.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215e7fc700 1 -- 172.21.15.73:0/3001935605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215400b760 con 0x7f2160068090 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f215e7fc700 1 -- 172.21.15.73:0/3001935605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215400bae0 con 0x7f2160068090 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f2165fe0700 1 -- 172.21.15.73:0/3001935605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 msgr2=0x7f21600c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.197+0000 7f2165fe0700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600c43b0 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 rx=0x7f2154006fd0 tx=0x7f21540090f0).stop 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 -- 172.21.15.73:0/3001935605 shutdown_connections 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2160067cb0 0x7f21600c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 --2- 172.21.15.73:0/3001935605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600c43b0 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 -- 172.21.15.73:0/3001935605 >> 172.21.15.73:0/3001935605 conn(0x7f21600c7fd0 msgr2=0x7f21600c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 -- 172.21.15.73:0/3001935605 shutdown_connections 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.198+0000 7f2165fe0700 1 -- 172.21.15.73:0/3001935605 wait complete. 2023-11-28T03:11:19.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.199+0000 7f2165fe0700 1 Processor -- start 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.199+0000 7f2165fe0700 1 -- start start 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 0x7f21600ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21600c2640 0x7f21600c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600a7e90 con 0x7f2160068090 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21600a8170 con 0x7f2160067cb0 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21600a8450 con 0x7f21600c2640 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41502/0 (socket says 172.21.15.73:41502) 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 -- 172.21.15.73:0/2784625663 learned_addr learned my addr 172.21.15.73:0/2784625663 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:19.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215ffff700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21600c2640 0x7f21600c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215f7fe700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 0x7f21600ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 -- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21600c2640 msgr2=0x7f21600c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21600c2640 0x7f21600c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 -- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 msgr2=0x7f21600ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 0x7f21600ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 -- 172.21.15.73:0/2784625663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2154009370 con 0x7f2160068090 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215f7fe700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 0x7f21600ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f215effd700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f2154011000 tx=0x7f2154017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2154006a40 con 0x7f2160068090 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f215400b8d0 con 0x7f2160068090 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f215401c670 con 0x7f2160068090 2023-11-28T03:11:19.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21600a8730 con 0x7f2160068090 2023-11-28T03:11:19.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.200+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21600a7ab0 con 0x7f2160068090 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.201+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21600a8ce0 con 0x7f2160068090 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.202+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f215401ca20 con 0x7f2160068090 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.202+0000 7f21477fe700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 0x7f214c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.202+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f215407b8d0 con 0x7f2160068090 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.202+0000 7f215f7fe700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 0x7f214c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.202+0000 7f215f7fe700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 0x7f214c0615e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2150007540 tx=0x7f215000b040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.206+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f21600a8ce0 con 0x7f2160068090 2023-11-28T03:11:19.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.368+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21600a7490 con 0x7f2160068090 2023-11-28T03:11:19.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.371+0000 7f21477fe700 1 -- 172.21.15.73:0/2784625663 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v139) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f21600a7490 con 0x7f2160068090 2023-11-28T03:11:19.374 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:19.374 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":139,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:11:18.740195+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 msgr2=0x7f214c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 0x7f214c0615e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f2150007540 tx=0x7f215000b040).stop 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 msgr2=0x7f21600bf0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 rx=0x7f2154011000 tx=0x7f2154017000).stop 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 shutdown_connections 2023-11-28T03:11:19.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f214c05f120 0x7f214c0615e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f21600c2640 0x7f21600c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2160067cb0 0x7f21600ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 --2- 172.21.15.73:0/2784625663 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2160068090 0x7f21600bf0e0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 >> 172.21.15.73:0/2784625663 conn(0x7f21600c7fd0 msgr2=0x7f21600bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 shutdown_connections 2023-11-28T03:11:19.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.373+0000 7f2165fe0700 1 -- 172.21.15.73:0/2784625663 wait complete. 2023-11-28T03:11:19.388 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T03:11:19.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.503+0000 7f46052f8700 1 Processor -- start 2023-11-28T03:11:19.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.503+0000 7f46052f8700 1 -- start start 2023-11-28T03:11:19.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f46052f8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4600153390 0x7f46000c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f46052f8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4600155f20 con 0x7f46000c5ac0 2023-11-28T03:11:19.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f46052f8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4600156200 con 0x7f46000c51a0 2023-11-28T03:11:19.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f46052f8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4600153990 con 0x7f4600153390 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f45feffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f46000c5ac0 0x7f4600156690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37272/0 (socket says 172.21.15.73:37272) 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f45feffd700 1 -- 172.21.15.73:0/2908810465 learned_addr learned my addr 172.21.15.73:0/2908810465 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.504+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1798121128 0 0) 0x7f4600155f20 con 0x7f46000c5ac0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f45ec0033f0 con 0x7f46000c5ac0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45ff7fe700 1 --2- 172.21.15.73:0/2908810465 >> v2:172.21.15.150:3300/0 conn(0x7f4600153390 0x7f46000c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2113046218 0 0) 0x7f45ec0033f0 con 0x7f46000c5ac0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 >> v2:172.21.15.150:3300/0 conn(0x7f4600153390 msgr2=0x7f46000c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 --2- 172.21.15.73:0/2908810465 >> v2:172.21.15.150:3300/0 conn(0x7f4600153390 0x7f46000c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 >> v1:172.21.15.150:6789/0 conn(0x7f46000c51a0 legacy=0x7f4600066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000bff70 con 0x7f46000c5ac0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f45f0001470 con 0x7f46000c5ac0 2023-11-28T03:11:19.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 >> v1:172.21.15.73:6789/0 conn(0x7f46000c5ac0 legacy=0x7f4600156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45ec0038e0 0x7f45ec003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f45ec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45ec005120 0x7f45ec00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4600153990 con 0x7f45ec004690 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec00c010 con 0x7f45ec0038e0 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45ec00c1d0 con 0x7f45ec005120 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f45ec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45feffd700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45ec0038e0 0x7f45ec003ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45ff7fe700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45ec005120 0x7f45ec00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 -- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45ec005120 msgr2=0x7f45ec00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45ec005120 0x7f45ec00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 -- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45ec0038e0 msgr2=0x7f45ec003ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f45ec0038e0 0x7f45ec003ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 -- 172.21.15.73:0/2908810465 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45ec0033f0 con 0x7f45ec004690 2023-11-28T03:11:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45ff7fe700 1 -- 172.21.15.73:0/2908810465 reap_dead start 2023-11-28T03:11:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f45ec004b10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f45e8002640 tx=0x7f45e8007bb0).ready entity=mon.0 client_cookie=a738282b92cc97ca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8002ea0 con 0x7f45ec004690 2023-11-28T03:11:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45e8013450 con 0x7f45ec004690 2023-11-28T03:11:19.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.505+0000 7f45fe7fc700 1 -- 172.21.15.73:0/2908810465 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45e8013810 con 0x7f45ec004690 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 -- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 msgr2=0x7f45ec004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f45ec004b10 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 rx=0x7f45e8002640 tx=0x7f45e8007bb0).stop 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 -- 172.21.15.73:0/2908810465 shutdown_connections 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 --2- 172.21.15.73:0/2908810465 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f45ec004b10 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 -- 172.21.15.73:0/2908810465 >> 172.21.15.73:0/2908810465 conn(0x7f4600062700 msgr2=0x7f4600062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 -- 172.21.15.73:0/2908810465 shutdown_connections 2023-11-28T03:11:19.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.506+0000 7f46052f8700 1 -- 172.21.15.73:0/2908810465 wait complete. 2023-11-28T03:11:19.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.507+0000 7f46052f8700 1 Processor -- start 2023-11-28T03:11:19.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.507+0000 7f46052f8700 1 -- start start 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46000aea70 0x7f46000c39f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 0x7f46000c3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000c43d0 con 0x7f45ec004690 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46000c1770 con 0x7f46000aeef0 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f46052f8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46000aab50 con 0x7f46000aea70 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41522/0 (socket says 172.21.15.73:41522) 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 -- 172.21.15.73:0/2741326425 learned_addr learned my addr 172.21.15.73:0/2741326425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46000aea70 0x7f46000c39f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45ff7fe700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 0x7f46000c3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 -- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46000aea70 msgr2=0x7f46000c39f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46000aea70 0x7f46000c39f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 -- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 msgr2=0x7f46000c3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 0x7f46000c3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45feffd700 1 -- 172.21.15.73:0/2741326425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46000c0b60 con 0x7f45ec004690 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.508+0000 7f45ff7fe700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 0x7f46000c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:19.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f45feffd700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f45f00049e0 tx=0x7f46000c0100).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f0004e50 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45f000f870 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45f000fbf0 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46000c0e40 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.509+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46000c1400 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.510+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460006c860 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.510+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f45f001c360 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.510+0000 7f45f77fe700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 0x7f45e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.510+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f45f00797c0 con 0x7f45ec004690 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.510+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 0x7f45e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.511+0000 7f45f67fc700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 0x7f45e0061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f46000add70 tx=0x7f45e8015000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.514+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f45f004aa30 con 0x7f45ec004690 2023-11-28T03:11:19.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.719+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f46000c4f80 con 0x7f45ec004690 2023-11-28T03:11:19.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.743+0000 7f45f77fe700 1 -- 172.21.15.73:0/2741326425 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v51) v1 ==== 118+0+0 (secure 0 0 0) 0x7f45f0019020 con 0x7f45ec004690 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 msgr2=0x7f45e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 0x7f45e0061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f46000add70 tx=0x7f45e8015000).stop 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 msgr2=0x7f46000aa660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 rx=0x7f45f00049e0 tx=0x7f46000c0100).stop 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 shutdown_connections 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45e005f150 0x7f45e0061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46000aeef0 0x7f46000c3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45ec004690 0x7f46000aa660 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 --2- 172.21.15.73:0/2741326425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46000aea70 0x7f46000c39f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.745+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 >> 172.21.15.73:0/2741326425 conn(0x7f4600062700 msgr2=0x7f4600155400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.746+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 shutdown_connections 2023-11-28T03:11:19.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.746+0000 7f46052f8700 1 -- 172.21.15.73:0/2741326425 wait complete. 2023-11-28T03:11:19.759 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T03:11:19.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T03:11:19.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.874+0000 7fba9a54c700 1 Processor -- start 2023-11-28T03:11:19.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.874+0000 7fba9a54c700 1 -- start start 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba9a54c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba940c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba9a54c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fba94158320 0x7fba941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba9a54c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba94158cc0 con 0x7fba94098d10 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba9a54c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba940c72d0 con 0x7fba94158320 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba9a54c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba94064570 con 0x7fba940c6190 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba98ae9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fba94158320 0x7fba941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba93fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fba94098d10 0x7fba94063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37282/0 (socket says 172.21.15.73:37282) 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.875+0000 7fba93fff700 1 -- 172.21.15.73:0/2390579096 learned_addr learned my addr 172.21.15.73:0/2390579096 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba98ae9700 1 -- 172.21.15.73:0/2390579096 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 msgr2=0x7fba940c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:19.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba98ae9700 1 --2- 172.21.15.73:0/2390579096 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba940c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba98ae9700 1 -- 172.21.15.73:0/2390579096 >> v1:172.21.15.73:6789/0 conn(0x7fba94098d10 legacy=0x7fba94063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba98ae9700 1 -- 172.21.15.73:0/2390579096 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94154bc0 con 0x7fba94158320 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba92ffd700 1 -- 172.21.15.73:0/2390579096 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1905213038 0 0) 0x7fba84001110 con 0x7fba94098d10 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba98ae9700 1 --2- 172.21.15.73:0/2390579096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94158320 0x7fba941587d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fba8c009e60 tx=0x7fba8c009780).ready entity=mon.0 client_cookie=e52982d4fbf68dc5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba92ffd700 1 -- 172.21.15.73:0/2390579096 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c00e070 con 0x7fba94158320 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba92ffd700 1 -- 172.21.15.73:0/2390579096 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba8c00bc30 con 0x7fba94158320 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba92ffd700 1 -- 172.21.15.73:0/2390579096 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba8c012410 con 0x7fba94158320 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 -- 172.21.15.73:0/2390579096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94158320 msgr2=0x7fba941587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 --2- 172.21.15.73:0/2390579096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94158320 0x7fba941587d0 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 rx=0x7fba8c009e60 tx=0x7fba8c009780).stop 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 -- 172.21.15.73:0/2390579096 shutdown_connections 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 --2- 172.21.15.73:0/2390579096 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94158320 0x7fba941587d0 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 --2- 172.21.15.73:0/2390579096 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:19.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 -- 172.21.15.73:0/2390579096 >> 172.21.15.73:0/2390579096 conn(0x7fba940cb310 msgr2=0x7fba940cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:19.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 -- 172.21.15.73:0/2390579096 shutdown_connections 2023-11-28T03:11:19.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.876+0000 7fba9a54c700 1 -- 172.21.15.73:0/2390579096 wait complete. 2023-11-28T03:11:19.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.877+0000 7fba9a54c700 1 Processor -- start 2023-11-28T03:11:19.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 -- start start 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba94168cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 0x7fba9414d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9414e380 con 0x7fba94098d10 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9414de30 con 0x7fba94158320 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba9a54c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba940c23b0 con 0x7fba940c6190 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba93fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba93fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41544/0 (socket says 172.21.15.73:41544) 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.878+0000 7fba93fff700 1 -- 172.21.15.73:0/3705840377 learned_addr learned my addr 172.21.15.73:0/3705840377 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:19.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba937fe700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba94168cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba98ae9700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 0x7fba9414d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 -- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 msgr2=0x7fba94168cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba94168cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 -- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 msgr2=0x7fba9414d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 0x7fba9414d940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 -- 172.21.15.73:0/3705840377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba8c009a20 con 0x7fba94098d10 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba937fe700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba94168cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba98ae9700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 0x7fba9414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba93fff700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fba8400c510 tx=0x7fba84007ad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840072e0 con 0x7fba94098d10 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba84007440 con 0x7fba94098d10 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba940c2f30 con 0x7fba94098d10 2023-11-28T03:11:19.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba840057a0 con 0x7fba94098d10 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.879+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba940c28e0 con 0x7fba94098d10 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.880+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9406c860 con 0x7fba94098d10 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.880+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fba84005bd0 con 0x7fba94098d10 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.881+0000 7fba7bfff700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 0x7fba800615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.881+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fba840799a0 con 0x7fba94098d10 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.881+0000 7fba937fe700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 0x7fba800615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.881+0000 7fba937fe700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 0x7fba800615e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fba94150bb0 tx=0x7fba88008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:19.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:19.885+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fba8404ac70 con 0x7fba94098d10 2023-11-28T03:11:20.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.091+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fba94158cc0 con 0x7fba94098d10 2023-11-28T03:11:20.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.753+0000 7fba7bfff700 1 -- 172.21.15.73:0/3705840377 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fba94158cc0 con 0x7fba94098d10 2023-11-28T03:11:20.756 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 msgr2=0x7fba800615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 0x7fba800615e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fba94150bb0 tx=0x7fba88008040).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 msgr2=0x7fba941506c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 rx=0x7fba8400c510 tx=0x7fba84007ad0).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 shutdown_connections 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba8005f120 0x7fba800615e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba94158320 0x7fba9414d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba94098d10 0x7fba941506c0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 --2- 172.21.15.73:0/3705840377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba940c6190 0x7fba94168cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 >> 172.21.15.73:0/3705840377 conn(0x7fba940cb310 msgr2=0x7fba940c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 shutdown_connections 2023-11-28T03:11:20.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.755+0000 7fba9a54c700 1 -- 172.21.15.73:0/3705840377 wait complete. 2023-11-28T03:11:20.769 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T03:11:20.769 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:20.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.882+0000 7f71f68a5700 1 Processor -- start 2023-11-28T03:11:20.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.882+0000 7f71f68a5700 1 -- start start 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71f68a5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f71f00ca180 0x7f71f015f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71f68a5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f00a9a90 con 0x7f71f00caab0 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71f68a5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f0154ce0 con 0x7f71f00cae90 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71f68a5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f00aab80 con 0x7f71f00ca180 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71effff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f71f00caab0 0x7f71f00aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37286/0 (socket says 172.21.15.73:37286) 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71effff700 1 -- 172.21.15.73:0/229110378 learned_addr learned my addr 172.21.15.73:0/229110378 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3129749965 0 0) 0x7f71f00a9a90 con 0x7f71f00caab0 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f71d80033f0 con 0x7f71f00caab0 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71f4e42700 1 --2- 172.21.15.73:0/229110378 >> v2:172.21.15.150:3300/0 conn(0x7f71f00ca180 0x7f71f015f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1242560336 0 0) 0x7f71d80033f0 con 0x7f71f00caab0 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 >> v2:172.21.15.150:3300/0 conn(0x7f71f00ca180 msgr2=0x7f71f015f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:20.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 --2- 172.21.15.73:0/229110378 >> v2:172.21.15.150:3300/0 conn(0x7f71f00ca180 0x7f71f015f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 >> v1:172.21.15.150:6789/0 conn(0x7f71f00cae90 legacy=0x7f71f00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00a8e80 con 0x7f71f00caab0 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f71e00015f0 con 0x7f71f00caab0 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.883+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 >> v1:172.21.15.73:6789/0 conn(0x7f71f00caab0 legacy=0x7f71f00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 0x7f71d8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d8004690 0x7f71d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d8005050 0x7f71d800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00aab80 con 0x7f71d80038e0 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d800bfe0 con 0x7f71d8005050 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d800c1a0 con 0x7f71d8004690 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 0x7f71d8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71f4e42700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d8005050 0x7f71d800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71ef7fe700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d8004690 0x7f71d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 -- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d8004690 msgr2=0x7f71d8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71d8004690 0x7f71d8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 -- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d8005050 msgr2=0x7f71d800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d8005050 0x7f71d800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 -- 172.21.15.73:0/229110378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d80033f0 con 0x7f71d80038e0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71f4e42700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d8005050 0x7f71d800b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71f4e42700 1 -- 172.21.15.73:0/229110378 reap_dead start 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71effff700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 0x7f71d8003d00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f71e000c160 tx=0x7f71e00077a0).ready entity=mon.0 client_cookie=fcaddb70818fd557 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00079d0 con 0x7f71d80038e0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e000ca90 con 0x7f71d80038e0 2023-11-28T03:11:20.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.884+0000 7f71eeffd700 1 -- 172.21.15.73:0/229110378 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e000ce10 con 0x7f71d80038e0 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 -- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 msgr2=0x7f71d8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 0x7f71d8003d00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 rx=0x7f71e000c160 tx=0x7f71e00077a0).stop 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 -- 172.21.15.73:0/229110378 shutdown_connections 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 --2- 172.21.15.73:0/229110378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71d80038e0 0x7f71d8003d00 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 -- 172.21.15.73:0/229110378 >> 172.21.15.73:0/229110378 conn(0x7f71f0094920 msgr2=0x7f71f006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 -- 172.21.15.73:0/229110378 shutdown_connections 2023-11-28T03:11:20.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.885+0000 7f71f68a5700 1 -- 172.21.15.73:0/229110378 wait complete. 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.886+0000 7f71f68a5700 1 Processor -- start 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.886+0000 7f71f68a5700 1 -- start start 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d80038e0 0x7f71f01577d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f01481d0 0x7f71f0164fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0164b10 con 0x7f71f0165490 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f0164800 con 0x7f71d80038e0 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f00c3240 con 0x7f71f01481d0 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.893 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41554/0 (socket says 172.21.15.73:41554) 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 -- 172.21.15.73:0/2644219235 learned_addr learned my addr 172.21.15.73:0/2644219235 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71ef7fe700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f01481d0 0x7f71f0164fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71effff700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d80038e0 0x7f71f01577d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 -- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f01481d0 msgr2=0x7f71f0164fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f01481d0 0x7f71f0164fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 -- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d80038e0 msgr2=0x7f71f01577d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d80038e0 0x7f71f01577d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 -- 172.21.15.73:0/2644219235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f00c3520 con 0x7f71f0165490 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f4e42700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f71f00c1c40 tx=0x7f71e80097f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8009860 con 0x7f71f0165490 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e800f030 con 0x7f71f0165490 2023-11-28T03:11:20.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f00c3800 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e8002e90 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.887+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f0153de0 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.888+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f00c39f0 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.889+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f71e8003040 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.889+0000 7f71d7fff700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 0x7f71dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.889+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f71e807a030 con 0x7f71f0165490 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.889+0000 7f71effff700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 0x7f71dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:20.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.889+0000 7f71effff700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 0x7f71dc061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f71e0000ef0 tx=0x7f71e0015000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:20.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:20.893+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71e804aaf0 con 0x7f71f0165490 2023-11-28T03:11:21.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.b has been restored 2023-11-28T03:11:21.024 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-11-28T03:11:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.096+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f71f006c860 con 0x7f71f0165490 2023-11-28T03:11:21.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.097+0000 7f71d7fff700 1 -- 172.21.15.73:0/2644219235 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7f71f00c39f0 con 0x7f71f0165490 2023-11-28T03:11:21.098 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:21.098 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 msgr2=0x7f71dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 0x7f71dc061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f71e0000ef0 tx=0x7f71e0015000).stop 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 msgr2=0x7f71f00b0230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 rx=0x7f71f00c1c40 tx=0x7f71e80097f0).stop 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 shutdown_connections 2023-11-28T03:11:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f71dc05f150 0x7f71dc061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f0165490 0x7f71f00b0230 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71d80038e0 0x7f71f01577d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 --2- 172.21.15.73:0/2644219235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f01481d0 0x7f71f0164fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 >> 172.21.15.73:0/2644219235 conn(0x7f71f0094920 msgr2=0x7f71f00abd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 shutdown_connections 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.099+0000 7f71f68a5700 1 -- 172.21.15.73:0/2644219235 wait complete. 2023-11-28T03:11:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T03:11:21.112 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T03:11:21.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.224+0000 7fc8589e7700 1 Processor -- start 2023-11-28T03:11:21.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.224+0000 7fc8589e7700 1 -- start start 2023-11-28T03:11:21.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc8589e7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8540caab0 0x7fc8540aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc8589e7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8540cae90 0x7fc8540c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc8589e7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8540a9a90 con 0x7fc8540aa960 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc8589e7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc854154ce0 con 0x7fc8540caab0 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc8589e7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8540a8e80 con 0x7fc8540cae90 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8540caab0 0x7fc8540aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc8540caab0 0x7fc8540aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41560/0 (socket says 172.21.15.73:41560) 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 -- 172.21.15.73:0/1219266736 learned_addr learned my addr 172.21.15.73:0/1219266736 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 -- 172.21.15.73:0/1219266736 >> v2:172.21.15.150:3300/0 conn(0x7fc8540cae90 msgr2=0x7fc8540c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc851d9b700 1 --2- 172.21.15.73:0/1219266736 >> v2:172.21.15.150:3300/0 conn(0x7fc8540cae90 0x7fc8540c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 --2- 172.21.15.73:0/1219266736 >> v2:172.21.15.150:3300/0 conn(0x7fc8540cae90 0x7fc8540c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.225+0000 7fc85259c700 1 -- 172.21.15.73:0/1219266736 >> v1:172.21.15.150:6789/0 conn(0x7fc8540aa960 legacy=0x7fc8540c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc85259c700 1 -- 172.21.15.73:0/1219266736 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8540a9160 con 0x7fc8540caab0 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc85259c700 1 --2- 172.21.15.73:0/1219266736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540caab0 0x7fc8540aa470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fc8540b1440 tx=0x7fc844009840).ready entity=mon.0 client_cookie=f314062282f138f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc85159a700 1 -- 172.21.15.73:0/1219266736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84400e070 con 0x7fc8540caab0 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc85159a700 1 -- 172.21.15.73:0/1219266736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc84400ba90 con 0x7fc8540caab0 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc85159a700 1 -- 172.21.15.73:0/1219266736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc84400be10 con 0x7fc8540caab0 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 -- 172.21.15.73:0/1219266736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540caab0 msgr2=0x7fc8540aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 --2- 172.21.15.73:0/1219266736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540caab0 0x7fc8540aa470 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 rx=0x7fc8540b1440 tx=0x7fc844009840).stop 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 -- 172.21.15.73:0/1219266736 shutdown_connections 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 --2- 172.21.15.73:0/1219266736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540caab0 0x7fc8540aa470 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 --2- 172.21.15.73:0/1219266736 >> v2:172.21.15.150:3300/0 conn(0x7fc8540cae90 0x7fc8540c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 -- 172.21.15.73:0/1219266736 >> 172.21.15.73:0/1219266736 conn(0x7fc854094920 msgr2=0x7fc85406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 -- 172.21.15.73:0/1219266736 shutdown_connections 2023-11-28T03:11:21.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.226+0000 7fc8589e7700 1 -- 172.21.15.73:0/1219266736 wait complete. 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.227+0000 7fc8589e7700 1 Processor -- start 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 -- start start 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540caab0 0x7fc8540c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c2310 0x7fc8540c2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c3970 con 0x7fc8540cae90 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c2d60 con 0x7fc8540c2310 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc8589e7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8540c3040 con 0x7fc8540caab0 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc851d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc851d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41568/0 (socket says 172.21.15.73:41568) 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.228+0000 7fc851d9b700 1 -- 172.21.15.73:0/732105738 learned_addr learned my addr 172.21.15.73:0/732105738 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc85259c700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540caab0 0x7fc8540c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 -- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540caab0 msgr2=0x7fc8540c3e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540caab0 0x7fc8540c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 -- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c2310 msgr2=0x7fc8540c2790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc852d9d700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c2310 0x7fc8540c2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c2310 0x7fc8540c2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 -- 172.21.15.73:0/732105738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844009ae0 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc851d9b700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fc84800c650 tx=0x7fc848008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848010580 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8480106e0 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8540c3320 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc848010a60 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.229+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc854150ef0 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.230+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8540c3ae0 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.230+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc84801b460 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.230+0000 7fc8427fc700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 0x7fc834061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.231+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc84807a0e0 con 0x7fc8540cae90 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.231+0000 7fc85259c700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 0x7fc834061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.231+0000 7fc85259c700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 0x7fc834061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc844009fd0 tx=0x7fc844009840).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.235+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc84804b350 con 0x7fc8540cae90 2023-11-28T03:11:21.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.399+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc85406c860 con 0x7fc8540cae90 2023-11-28T03:11:21.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.400+0000 7fc8427fc700 1 -- 172.21.15.73:0/732105738 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v140) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fc84804e970 con 0x7fc8540cae90 2023-11-28T03:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:21.401 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":140,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T03:11:19.742241+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"140","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 msgr2=0x7fc834061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 0x7fc834061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fc844009fd0 tx=0x7fc844009840).stop 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 msgr2=0x7fc854150a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 rx=0x7fc84800c650 tx=0x7fc848008040).stop 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 shutdown_connections 2023-11-28T03:11:21.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc83405f150 0x7fc834061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8540c2310 0x7fc8540c2790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8540caab0 0x7fc8540c3e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 --2- 172.21.15.73:0/732105738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8540cae90 0x7fc854150a00 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 >> 172.21.15.73:0/732105738 conn(0x7fc854094920 msgr2=0x7fc8540af6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 shutdown_connections 2023-11-28T03:11:21.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.402+0000 7fc8589e7700 1 -- 172.21.15.73:0/732105738 wait complete. 2023-11-28T03:11:21.415 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T03:11:21.415 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:11:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.527+0000 7f92ec90e700 1 Processor -- start 2023-11-28T03:11:21.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.527+0000 7f92ec90e700 1 -- start start 2023-11-28T03:11:21.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.528+0000 7f92ec90e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92e80671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.528+0000 7f92ec90e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e80bd590 con 0x7f92e8066e00 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.528+0000 7f92ec90e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e80a6090 con 0x7f92e8067980 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.528+0000 7f92ec90e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8156370 con 0x7f92e8155e40 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e659c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92e8066e00 0x7f92e80a5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37300/0 (socket says 172.21.15.73:37300) 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e659c700 1 -- 172.21.15.73:0/1987480362 learned_addr learned my addr 172.21.15.73:0/1987480362 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92e80671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2214601236 0 0) 0x7f92e80bd590 con 0x7f92e8066e00 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92d40033f0 con 0x7f92e8066e00 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3349586264 0 0) 0x7f92e80a6090 con 0x7f92e8067980 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e80bd590 con 0x7f92e8067980 2023-11-28T03:11:21.533 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1265621350 0 0) 0x7f92d40033f0 con 0x7f92e8066e00 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 msgr2=0x7f92e80671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92e80671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 >> v1:172.21.15.150:6789/0 conn(0x7f92e8067980 legacy=0x7f92e80bda20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8156650 con 0x7f92e8066e00 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f92d80017f0 con 0x7f92e8066e00 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92e80671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 >> v1:172.21.15.73:6789/0 conn(0x7f92e8066e00 legacy=0x7f92e80a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d4003aa0 0x7f92d4003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92d4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92d4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e8156370 con 0x7f92d4004b80 2023-11-28T03:11:21.534 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400b8e0 con 0x7f92d4003aa0 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d400baa0 con 0x7f92e8155e40 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92d4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e659c700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d4003aa0 0x7f92d4003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.529+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92d4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 -- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 msgr2=0x7f92d4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e8155e40 0x7f92d4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 -- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d4003aa0 msgr2=0x7f92d4003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92d4003aa0 0x7f92d4003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 -- 172.21.15.73:0/1987480362 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d00092a0 con 0x7f92d4004b80 2023-11-28T03:11:21.535 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92d4005000 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f92e0001890 tx=0x7f92e0007bb0).ready entity=mon.0 client_cookie=fc409bf2fa7e4220 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0007020 con 0x7f92d4004b80 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e0007180 con 0x7f92d4004b80 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e559a700 1 -- 172.21.15.73:0/1987480362 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e0007500 con 0x7f92d4004b80 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 -- 172.21.15.73:0/1987480362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 msgr2=0x7f92d4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1987480362 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92d4005000 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 rx=0x7f92e0001890 tx=0x7f92e0007bb0).stop 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92e6d9d700 1 -- 172.21.15.73:0/1987480362 reap_dead start 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 -- 172.21.15.73:0/1987480362 shutdown_connections 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 -- 172.21.15.73:0/1987480362 >> 172.21.15.73:0/1987480362 conn(0x7f92e80c7fd0 msgr2=0x7f92e80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 -- 172.21.15.73:0/1987480362 shutdown_connections 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.530+0000 7f92ec90e700 1 -- 172.21.15.73:0/1987480362 wait complete. 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 Processor -- start 2023-11-28T03:11:21.536 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 -- start start 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 0x7f92e8155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 0x7f92e80c6de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80ac240 con 0x7f92d4004b80 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80ac6d0 con 0x7f92e80c69a0 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.532+0000 7f92ec90e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e80c72d0 con 0x7f92e80ab170 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41590/0 (socket says 172.21.15.73:41590) 2023-11-28T03:11:21.540 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 -- 172.21.15.73:0/1454006637 learned_addr learned my addr 172.21.15.73:0/1454006637 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 0x7f92e80c6de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 0x7f92e8155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 -- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 msgr2=0x7f92e8155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 0x7f92e8155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 -- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 msgr2=0x7f92e80c6de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 0x7f92e80c6de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 -- 172.21.15.73:0/1454006637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e80ab630 con 0x7f92d4004b80 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e6d9d700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 0x7f92e80c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 0x7f92e8155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92e659c700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f92d8007710 tx=0x7f92d8007f20).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.541 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800f870 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d800f9d0 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e80ab910 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800fd50 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.533+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e80abed0 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.534+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e806c860 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.534+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f92d8012560 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.535+0000 7f92de7fc700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 0x7f92c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.535+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f92d8079e30 con 0x7f92d4004b80 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.535+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 0x7f92c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.535+0000 7f92e5d9b700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 0x7f92c8061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f92d0006d50 tx=0x7f92d0008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.539+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f92d804b0a0 con 0x7f92d4004b80 2023-11-28T03:11:21.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.698+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92e80ac380 con 0x7f92c805f150 2023-11-28T03:11:21.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.708+0000 7f92de7fc700 1 -- 172.21.15.73:0/1454006637 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360699 (secure 0 0 0) 0x7f92d0009b80 con 0x7f92c805f150 2023-11-28T03:11:21.714 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:21.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 msgr2=0x7f92c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 0x7f92c8061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f92d0006d50 tx=0x7f92d0008040).stop 2023-11-28T03:11:21.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 msgr2=0x7f92e80c13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 rx=0x7f92d8007710 tx=0x7f92d8007f20).stop 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 shutdown_connections 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f92c805f150 0x7f92c8061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92e80c69a0 0x7f92e80c6de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92d4004b80 0x7f92e80c13c0 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 --2- 172.21.15.73:0/1454006637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f92e80ab170 0x7f92e8155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.711+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 >> 172.21.15.73:0/1454006637 conn(0x7f92e80c7fd0 msgr2=0x7f92e80c5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.712+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 shutdown_connections 2023-11-28T03:11:21.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.712+0000 7f92ec90e700 1 -- 172.21.15.73:0/1454006637 wait complete. 2023-11-28T03:11:21.719 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:11:21.725 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T03:11:19.763166+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426948,"kb_used_data":13316,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752547772,"statfs":{"total":773094113280,"available":770608918528,"internally_reserved":0,"allocated":13635584,"data_stored":6798157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19375,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.586173"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757468+0000","last_change":"2023-11-28T03:11:18.757597+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757468+0000","last_undegraded":"2023-11-28T03:11:18.757468+0000","last_fullsized":"2023-11-28T03:11:18.757468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388461+0000","last_change":"2023-11-28T03:11:18.385568+0000","last_active":"2023-11-28T03:11:18.388461+0000","last_peered":"2023-11-28T03:11:18.388461+0000","last_clean":"2023-11-28T03:11:18.388461+0000","last_became_active":"2023-11-28T03:11:18.385211+0000","last_became_peered":"2023-11-28T03:11:18.385211+0000","last_unstale":"2023-11-28T03:11:18.388461+0000","last_undegraded":"2023-11-28T03:11:18.388461+0000","last_fullsized":"2023-11-28T03:11:18.388461+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759645+0000","last_change":"2023-11-28T03:11:18.759718+0000","last_active":"2023-11-28T03:11:18.759645+0000","last_peered":"2023-11-28T03:11:18.759645+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759448+0000","last_became_peered":"2023-11-28T03:11:18.759448+0000","last_unstale":"2023-11-28T03:11:18.759645+0000","last_undegraded":"2023-11-28T03:11:18.759645+0000","last_fullsized":"2023-11-28T03:11:18.759645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391029+0000","last_change":"2023-11-28T03:11:18.389741+0000","last_active":"2023-11-28T03:11:18.391029+0000","last_peered":"2023-11-28T03:11:18.391029+0000","last_clean":"2023-11-28T03:11:18.391029+0000","last_became_active":"2023-11-28T03:11:18.389440+0000","last_became_peered":"2023-11-28T03:11:18.389440+0000","last_unstale":"2023-11-28T03:11:18.391029+0000","last_undegraded":"2023-11-28T03:11:18.391029+0000","last_fullsized":"2023-11-28T03:11:18.391029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768741+0000","last_change":"2023-11-28T03:11:18.768823+0000","last_active":"2023-11-28T03:11:18.768741+0000","last_peered":"2023-11-28T03:11:18.768741+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766758+0000","last_became_peered":"2023-11-28T03:11:18.766758+0000","last_unstale":"2023-11-28T03:11:18.768741+0000","last_undegraded":"2023-11-28T03:11:18.768741+0000","last_fullsized":"2023-11-28T03:11:18.768741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397054+0000","last_change":"2023-11-28T03:11:18.394730+0000","last_active":"2023-11-28T03:11:18.397054+0000","last_peered":"2023-11-28T03:11:18.397054+0000","last_clean":"2023-11-28T03:11:18.397054+0000","last_became_active":"2023-11-28T03:11:18.394473+0000","last_became_peered":"2023-11-28T03:11:18.394473+0000","last_unstale":"2023-11-28T03:11:18.397054+0000","last_undegraded":"2023-11-28T03:11:18.397054+0000","last_fullsized":"2023-11-28T03:11:18.397054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759861+0000","last_change":"2023-11-28T03:11:18.759965+0000","last_active":"2023-11-28T03:11:18.759861+0000","last_peered":"2023-11-28T03:11:18.759861+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759675+0000","last_became_peered":"2023-11-28T03:11:18.759675+0000","last_unstale":"2023-11-28T03:11:18.759861+0000","last_undegraded":"2023-11-28T03:11:18.759861+0000","last_fullsized":"2023-11-28T03:11:18.759861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392731+0000","last_change":"2023-11-28T03:11:18.390367+0000","last_active":"2023-11-28T03:11:18.392731+0000","last_peered":"2023-11-28T03:11:18.392731+0000","last_clean":"2023-11-28T03:11:18.392731+0000","last_became_active":"2023-11-28T03:11:18.390086+0000","last_became_peered":"2023-11-28T03:11:18.390086+0000","last_unstale":"2023-11-28T03:11:18.392731+0000","last_undegraded":"2023-11-28T03:11:18.392731+0000","last_fullsized":"2023-11-28T03:11:18.392731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757310+0000","last_change":"2023-11-28T03:11:18.757382+0000","last_active":"2023-11-28T03:11:18.757310+0000","last_peered":"2023-11-28T03:11:18.757310+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.756962+0000","last_became_peered":"2023-11-28T03:11:18.756962+0000","last_unstale":"2023-11-28T03:11:18.757310+0000","last_undegraded":"2023-11-28T03:11:18.757310+0000","last_fullsized":"2023-11-28T03:11:18.757310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387008+0000","last_change":"2023-11-28T03:11:18.385649+0000","last_active":"2023-11-28T03:11:18.387008+0000","last_peered":"2023-11-28T03:11:18.387008+0000","last_clean":"2023-11-28T03:11:18.387008+0000","last_became_active":"2023-11-28T03:11:18.385386+0000","last_became_peered":"2023-11-28T03:11:18.385386+0000","last_unstale":"2023-11-28T03:11:18.387008+0000","last_undegraded":"2023-11-28T03:11:18.387008+0000","last_fullsized":"2023-11-28T03:11:18.387008+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748324+0000","last_change":"2023-11-28T03:11:18.748460+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748324+0000","last_undegraded":"2023-11-28T03:11:18.748324+0000","last_fullsized":"2023-11-28T03:11:18.748324+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394106+0000","last_change":"2023-11-28T03:11:18.392909+0000","last_active":"2023-11-28T03:11:18.394106+0000","last_peered":"2023-11-28T03:11:18.394106+0000","last_clean":"2023-11-28T03:11:18.394106+0000","last_became_active":"2023-11-28T03:11:18.392552+0000","last_became_peered":"2023-11-28T03:11:18.392552+0000","last_unstale":"2023-11-28T03:11:18.394106+0000","last_undegraded":"2023-11-28T03:11:18.394106+0000","last_fullsized":"2023-11-28T03:11:18.394106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748559+0000","last_change":"2023-11-28T03:11:18.748669+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748559+0000","last_undegraded":"2023-11-28T03:11:18.748559+0000","last_fullsized":"2023-11-28T03:11:18.748559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392365+0000","last_change":"2023-11-28T03:11:18.389689+0000","last_active":"2023-11-28T03:11:18.392365+0000","last_peered":"2023-11-28T03:11:18.392365+0000","last_clean":"2023-11-28T03:11:18.392365+0000","last_became_active":"2023-11-28T03:11:18.388755+0000","last_became_peered":"2023-11-28T03:11:18.388755+0000","last_unstale":"2023-11-28T03:11:18.392365+0000","last_undegraded":"2023-11-28T03:11:18.392365+0000","last_fullsized":"2023-11-28T03:11:18.392365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757626+0000","last_change":"2023-11-28T03:11:18.757695+0000","last_active":"2023-11-28T03:11:18.757626+0000","last_peered":"2023-11-28T03:11:18.757626+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757427+0000","last_became_peered":"2023-11-28T03:11:18.757427+0000","last_unstale":"2023-11-28T03:11:18.757626+0000","last_undegraded":"2023-11-28T03:11:18.757626+0000","last_fullsized":"2023-11-28T03:11:18.757626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395299+0000","last_change":"2023-11-28T03:11:18.391234+0000","last_active":"2023-11-28T03:11:18.395299+0000","last_peered":"2023-11-28T03:11:18.395299+0000","last_clean":"2023-11-28T03:11:18.395299+0000","last_became_active":"2023-11-28T03:11:18.390855+0000","last_became_peered":"2023-11-28T03:11:18.390855+0000","last_unstale":"2023-11-28T03:11:18.395299+0000","last_undegraded":"2023-11-28T03:11:18.395299+0000","last_fullsized":"2023-11-28T03:11:18.395299+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751698+0000","last_change":"2023-11-28T03:11:18.751811+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751698+0000","last_undegraded":"2023-11-28T03:11:18.751698+0000","last_fullsized":"2023-11-28T03:11:18.751698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389124+0000","last_change":"2023-11-28T03:11:18.385956+0000","last_active":"2023-11-28T03:11:18.389124+0000","last_peered":"2023-11-28T03:11:18.389124+0000","last_clean":"2023-11-28T03:11:18.389124+0000","last_became_active":"2023-11-28T03:11:18.385634+0000","last_became_peered":"2023-11-28T03:11:18.385634+0000","last_unstale":"2023-11-28T03:11:18.389124+0000","last_undegraded":"2023-11-28T03:11:18.389124+0000","last_fullsized":"2023-11-28T03:11:18.389124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761439+0000","last_change":"2023-11-28T03:11:18.761511+0000","last_active":"2023-11-28T03:11:18.761439+0000","last_peered":"2023-11-28T03:11:18.761439+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761234+0000","last_became_peered":"2023-11-28T03:11:18.761234+0000","last_unstale":"2023-11-28T03:11:18.761439+0000","last_undegraded":"2023-11-28T03:11:18.761439+0000","last_fullsized":"2023-11-28T03:11:18.761439+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386815+0000","last_change":"2023-11-28T03:11:18.384000+0000","last_active":"2023-11-28T03:11:18.386815+0000","last_peered":"2023-11-28T03:11:18.386815+0000","last_clean":"2023-11-28T03:11:18.386815+0000","last_became_active":"2023-11-28T03:11:18.383737+0000","last_became_peered":"2023-11-28T03:11:18.383737+0000","last_unstale":"2023-11-28T03:11:18.386815+0000","last_undegraded":"2023-11-28T03:11:18.386815+0000","last_fullsized":"2023-11-28T03:11:18.386815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749608+0000","last_change":"2023-11-28T03:11:18.749746+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749608+0000","last_undegraded":"2023-11-28T03:11:18.749608+0000","last_fullsized":"2023-11-28T03:11:18.749608+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390548+0000","last_change":"2023-11-28T03:11:18.386918+0000","last_active":"2023-11-28T03:11:18.390548+0000","last_peered":"2023-11-28T03:11:18.390548+0000","last_clean":"2023-11-28T03:11:18.390548+0000","last_became_active":"2023-11-28T03:11:18.386386+0000","last_became_peered":"2023-11-28T03:11:18.386386+0000","last_unstale":"2023-11-28T03:11:18.390548+0000","last_undegraded":"2023-11-28T03:11:18.390548+0000","last_fullsized":"2023-11-28T03:11:18.390548+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761388+0000","last_change":"2023-11-28T03:11:18.761551+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761388+0000","last_undegraded":"2023-11-28T03:11:18.761388+0000","last_fullsized":"2023-11-28T03:11:18.761388+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386802+0000","last_change":"2023-11-28T03:11:18.384007+0000","last_active":"2023-11-28T03:11:18.386802+0000","last_peered":"2023-11-28T03:11:18.386802+0000","last_clean":"2023-11-28T03:11:18.386802+0000","last_became_active":"2023-11-28T03:11:18.383671+0000","last_became_peered":"2023-11-28T03:11:18.383671+0000","last_unstale":"2023-11-28T03:11:18.386802+0000","last_undegraded":"2023-11-28T03:11:18.386802+0000","last_fullsized":"2023-11-28T03:11:18.386802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753548+0000","last_change":"2023-11-28T03:11:18.753676+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753548+0000","last_undegraded":"2023-11-28T03:11:18.753548+0000","last_fullsized":"2023-11-28T03:11:18.753548+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395509+0000","last_change":"2023-11-28T03:11:18.391315+0000","last_active":"2023-11-28T03:11:18.395509+0000","last_peered":"2023-11-28T03:11:18.395509+0000","last_clean":"2023-11-28T03:11:18.395509+0000","last_became_active":"2023-11-28T03:11:18.391053+0000","last_became_peered":"2023-11-28T03:11:18.391053+0000","last_unstale":"2023-11-28T03:11:18.395509+0000","last_undegraded":"2023-11-28T03:11:18.395509+0000","last_fullsized":"2023-11-28T03:11:18.395509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756085+0000","last_change":"2023-11-28T03:11:18.756179+0000","last_active":"2023-11-28T03:11:18.756085+0000","last_peered":"2023-11-28T03:11:18.756085+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755853+0000","last_became_peered":"2023-11-28T03:11:18.755853+0000","last_unstale":"2023-11-28T03:11:18.756085+0000","last_undegraded":"2023-11-28T03:11:18.756085+0000","last_fullsized":"2023-11-28T03:11:18.756085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395277+0000","last_change":"2023-11-28T03:11:18.391280+0000","last_active":"2023-11-28T03:11:18.395277+0000","last_peered":"2023-11-28T03:11:18.395277+0000","last_clean":"2023-11-28T03:11:18.395277+0000","last_became_active":"2023-11-28T03:11:18.387894+0000","last_became_peered":"2023-11-28T03:11:18.387894+0000","last_unstale":"2023-11-28T03:11:18.395277+0000","last_undegraded":"2023-11-28T03:11:18.395277+0000","last_fullsized":"2023-11-28T03:11:18.395277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759232+0000","last_change":"2023-11-28T03:11:18.764061+0000","last_active":"2023-11-28T03:11:18.759232+0000","last_peered":"2023-11-28T03:11:18.759232+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758596+0000","last_became_peered":"2023-11-28T03:11:18.758596+0000","last_unstale":"2023-11-28T03:11:18.759232+0000","last_undegraded":"2023-11-28T03:11:18.759232+0000","last_fullsized":"2023-11-28T03:11:18.759232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393196+0000","last_change":"2023-11-28T03:11:18.392016+0000","last_active":"2023-11-28T03:11:18.393196+0000","last_peered":"2023-11-28T03:11:18.393196+0000","last_clean":"2023-11-28T03:11:18.393196+0000","last_became_active":"2023-11-28T03:11:18.391714+0000","last_became_peered":"2023-11-28T03:11:18.391714+0000","last_unstale":"2023-11-28T03:11:18.393196+0000","last_undegraded":"2023-11-28T03:11:18.393196+0000","last_fullsized":"2023-11-28T03:11:18.393196+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755843+0000","last_change":"2023-11-28T03:11:18.755924+0000","last_active":"2023-11-28T03:11:18.755843+0000","last_peered":"2023-11-28T03:11:18.755843+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755604+0000","last_became_peered":"2023-11-28T03:11:18.755604+0000","last_unstale":"2023-11-28T03:11:18.755843+0000","last_undegraded":"2023-11-28T03:11:18.755843+0000","last_fullsized":"2023-11-28T03:11:18.755843+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400025+0000","last_change":"2023-11-28T03:11:18.396374+0000","last_active":"2023-11-28T03:11:18.400025+0000","last_peered":"2023-11-28T03:11:18.400025+0000","last_clean":"2023-11-28T03:11:18.400025+0000","last_became_active":"2023-11-28T03:11:18.396165+0000","last_became_peered":"2023-11-28T03:11:18.396165+0000","last_unstale":"2023-11-28T03:11:18.400025+0000","last_undegraded":"2023-11-28T03:11:18.400025+0000","last_fullsized":"2023-11-28T03:11:18.400025+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748189+0000","last_change":"2023-11-28T03:11:18.748320+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748189+0000","last_undegraded":"2023-11-28T03:11:18.748189+0000","last_fullsized":"2023-11-28T03:11:18.748189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.383966+0000","last_change":"2023-11-28T03:11:18.380395+0000","last_active":"2023-11-28T03:11:18.383966+0000","last_peered":"2023-11-28T03:11:18.383966+0000","last_clean":"2023-11-28T03:11:18.383966+0000","last_became_active":"2023-11-28T03:11:18.379748+0000","last_became_peered":"2023-11-28T03:11:18.379748+0000","last_unstale":"2023-11-28T03:11:18.383966+0000","last_undegraded":"2023-11-28T03:11:18.383966+0000","last_fullsized":"2023-11-28T03:11:18.383966+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746878+0000","last_change":"2023-11-28T03:11:18.747007+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746878+0000","last_undegraded":"2023-11-28T03:11:18.746878+0000","last_fullsized":"2023-11-28T03:11:18.746878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386571+0000","last_change":"2023-11-28T03:11:18.382167+0000","last_active":"2023-11-28T03:11:18.386571+0000","last_peered":"2023-11-28T03:11:18.386571+0000","last_clean":"2023-11-28T03:11:18.386571+0000","last_became_active":"2023-11-28T03:11:18.381842+0000","last_became_peered":"2023-11-28T03:11:18.381842+0000","last_unstale":"2023-11-28T03:11:18.386571+0000","last_undegraded":"2023-11-28T03:11:18.386571+0000","last_fullsized":"2023-11-28T03:11:18.386571+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766948+0000","last_change":"2023-11-28T03:11:18.767015+0000","last_active":"2023-11-28T03:11:18.766948+0000","last_peered":"2023-11-28T03:11:18.766948+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766723+0000","last_became_peered":"2023-11-28T03:11:18.766723+0000","last_unstale":"2023-11-28T03:11:18.766948+0000","last_undegraded":"2023-11-28T03:11:18.766948+0000","last_fullsized":"2023-11-28T03:11:18.766948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.401250+0000","last_change":"2023-11-28T03:11:18.400594+0000","last_active":"2023-11-28T03:11:18.401250+0000","last_peered":"2023-11-28T03:11:18.401250+0000","last_clean":"2023-11-28T03:11:18.401250+0000","last_became_active":"2023-11-28T03:11:18.400354+0000","last_became_peered":"2023-11-28T03:11:18.400354+0000","last_unstale":"2023-11-28T03:11:18.401250+0000","last_undegraded":"2023-11-28T03:11:18.401250+0000","last_fullsized":"2023-11-28T03:11:18.401250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755842+0000","last_change":"2023-11-28T03:11:18.755965+0000","last_active":"2023-11-28T03:11:18.755842+0000","last_peered":"2023-11-28T03:11:18.755842+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755630+0000","last_became_peered":"2023-11-28T03:11:18.755630+0000","last_unstale":"2023-11-28T03:11:18.755842+0000","last_undegraded":"2023-11-28T03:11:18.755842+0000","last_fullsized":"2023-11-28T03:11:18.755842+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390934+0000","last_change":"2023-11-28T03:11:18.387489+0000","last_active":"2023-11-28T03:11:18.390934+0000","last_peered":"2023-11-28T03:11:18.390934+0000","last_clean":"2023-11-28T03:11:18.390934+0000","last_became_active":"2023-11-28T03:11:18.386665+0000","last_became_peered":"2023-11-28T03:11:18.386665+0000","last_unstale":"2023-11-28T03:11:18.390934+0000","last_undegraded":"2023-11-28T03:11:18.390934+0000","last_fullsized":"2023-11-28T03:11:18.390934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755670+0000","last_change":"2023-11-28T03:11:18.755670+0000","last_active":"2023-11-28T03:11:18.755670+0000","last_peered":"2023-11-28T03:11:18.755670+0000","last_clean":"2023-11-28T03:11:18.755670+0000","last_became_active":"2023-11-28T03:11:18.755386+0000","last_became_peered":"2023-11-28T03:11:18.755386+0000","last_unstale":"2023-11-28T03:11:18.755670+0000","last_undegraded":"2023-11-28T03:11:18.755670+0000","last_fullsized":"2023-11-28T03:11:18.755670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393816+0000","last_change":"2023-11-28T03:11:18.391627+0000","last_active":"2023-11-28T03:11:18.393816+0000","last_peered":"2023-11-28T03:11:18.393816+0000","last_clean":"2023-11-28T03:11:18.393816+0000","last_became_active":"2023-11-28T03:11:18.391309+0000","last_became_peered":"2023-11-28T03:11:18.391309+0000","last_unstale":"2023-11-28T03:11:18.393816+0000","last_undegraded":"2023-11-28T03:11:18.393816+0000","last_fullsized":"2023-11-28T03:11:18.393816+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757097+0000","last_change":"2023-11-28T03:11:18.757255+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757097+0000","last_undegraded":"2023-11-28T03:11:18.757097+0000","last_fullsized":"2023-11-28T03:11:18.757097+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395478+0000","last_change":"2023-11-28T03:11:18.391689+0000","last_active":"2023-11-28T03:11:18.395478+0000","last_peered":"2023-11-28T03:11:18.395478+0000","last_clean":"2023-11-28T03:11:18.395478+0000","last_became_active":"2023-11-28T03:11:18.390946+0000","last_became_peered":"2023-11-28T03:11:18.390946+0000","last_unstale":"2023-11-28T03:11:18.395478+0000","last_undegraded":"2023-11-28T03:11:18.395478+0000","last_fullsized":"2023-11-28T03:11:18.395478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751684+0000","last_change":"2023-11-28T03:11:18.751840+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751684+0000","last_undegraded":"2023-11-28T03:11:18.751684+0000","last_fullsized":"2023-11-28T03:11:18.751684+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390577+0000","last_change":"2023-11-28T03:11:18.386639+0000","last_active":"2023-11-28T03:11:18.390577+0000","last_peered":"2023-11-28T03:11:18.390577+0000","last_clean":"2023-11-28T03:11:18.390577+0000","last_became_active":"2023-11-28T03:11:18.386342+0000","last_became_peered":"2023-11-28T03:11:18.386342+0000","last_unstale":"2023-11-28T03:11:18.390577+0000","last_undegraded":"2023-11-28T03:11:18.390577+0000","last_fullsized":"2023-11-28T03:11:18.390577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758112+0000","last_change":"2023-11-28T03:11:18.758266+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758112+0000","last_undegraded":"2023-11-28T03:11:18.758112+0000","last_fullsized":"2023-11-28T03:11:18.758112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399942+0000","last_change":"2023-11-28T03:11:18.395025+0000","last_active":"2023-11-28T03:11:18.399942+0000","last_peered":"2023-11-28T03:11:18.399942+0000","last_clean":"2023-11-28T03:11:18.399942+0000","last_became_active":"2023-11-28T03:11:18.394807+0000","last_became_peered":"2023-11-28T03:11:18.394807+0000","last_unstale":"2023-11-28T03:11:18.399942+0000","last_undegraded":"2023-11-28T03:11:18.399942+0000","last_fullsized":"2023-11-28T03:11:18.399942+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751381+0000","last_change":"2023-11-28T03:11:18.751504+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751381+0000","last_undegraded":"2023-11-28T03:11:18.751381+0000","last_fullsized":"2023-11-28T03:11:18.751381+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.384795+0000","last_change":"2023-11-28T03:11:18.380585+0000","last_active":"2023-11-28T03:11:18.384795+0000","last_peered":"2023-11-28T03:11:18.384795+0000","last_clean":"2023-11-28T03:11:18.384795+0000","last_became_active":"2023-11-28T03:11:18.380095+0000","last_became_peered":"2023-11-28T03:11:18.380095+0000","last_unstale":"2023-11-28T03:11:18.384795+0000","last_undegraded":"2023-11-28T03:11:18.384795+0000","last_fullsized":"2023-11-28T03:11:18.384795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746528+0000","last_change":"2023-11-28T03:11:18.746656+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746528+0000","last_undegraded":"2023-11-28T03:11:18.746528+0000","last_fullsized":"2023-11-28T03:11:18.746528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389225+0000","last_change":"2023-11-28T03:11:18.387312+0000","last_active":"2023-11-28T03:11:18.389225+0000","last_peered":"2023-11-28T03:11:18.389225+0000","last_clean":"2023-11-28T03:11:18.389225+0000","last_became_active":"2023-11-28T03:11:18.386958+0000","last_became_peered":"2023-11-28T03:11:18.386958+0000","last_unstale":"2023-11-28T03:11:18.389225+0000","last_undegraded":"2023-11-28T03:11:18.389225+0000","last_fullsized":"2023-11-28T03:11:18.389225+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749305+0000","last_change":"2023-11-28T03:11:18.749305+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749305+0000","last_undegraded":"2023-11-28T03:11:18.749305+0000","last_fullsized":"2023-11-28T03:11:18.749305+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389375+0000","last_change":"2023-11-28T03:11:18.386227+0000","last_active":"2023-11-28T03:11:18.389375+0000","last_peered":"2023-11-28T03:11:18.389375+0000","last_clean":"2023-11-28T03:11:18.389375+0000","last_became_active":"2023-11-28T03:11:18.385925+0000","last_became_peered":"2023-11-28T03:11:18.385925+0000","last_unstale":"2023-11-28T03:11:18.389375+0000","last_undegraded":"2023-11-28T03:11:18.389375+0000","last_fullsized":"2023-11-28T03:11:18.389375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757939+0000","last_change":"2023-11-28T03:11:18.757939+0000","last_active":"2023-11-28T03:11:18.757939+0000","last_peered":"2023-11-28T03:11:18.757939+0000","last_clean":"2023-11-28T03:11:18.757939+0000","last_became_active":"2023-11-28T03:11:18.756692+0000","last_became_peered":"2023-11-28T03:11:18.756692+0000","last_unstale":"2023-11-28T03:11:18.757939+0000","last_undegraded":"2023-11-28T03:11:18.757939+0000","last_fullsized":"2023-11-28T03:11:18.757939+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395297+0000","last_change":"2023-11-28T03:11:18.386046+0000","last_active":"2023-11-28T03:11:18.395297+0000","last_peered":"2023-11-28T03:11:18.395297+0000","last_clean":"2023-11-28T03:11:18.395297+0000","last_became_active":"2023-11-28T03:11:18.385393+0000","last_became_peered":"2023-11-28T03:11:18.385393+0000","last_unstale":"2023-11-28T03:11:18.395297+0000","last_undegraded":"2023-11-28T03:11:18.395297+0000","last_fullsized":"2023-11-28T03:11:18.395297+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755485+0000","last_change":"2023-11-28T03:11:18.755485+0000","last_active":"2023-11-28T03:11:18.755485+0000","last_peered":"2023-11-28T03:11:18.755485+0000","last_clean":"2023-11-28T03:11:18.755485+0000","last_became_active":"2023-11-28T03:11:18.755182+0000","last_became_peered":"2023-11-28T03:11:18.755182+0000","last_unstale":"2023-11-28T03:11:18.755485+0000","last_undegraded":"2023-11-28T03:11:18.755485+0000","last_fullsized":"2023-11-28T03:11:18.755485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397939+0000","last_change":"2023-11-28T03:11:18.396889+0000","last_active":"2023-11-28T03:11:18.397939+0000","last_peered":"2023-11-28T03:11:18.397939+0000","last_clean":"2023-11-28T03:11:18.397939+0000","last_became_active":"2023-11-28T03:11:18.396619+0000","last_became_peered":"2023-11-28T03:11:18.396619+0000","last_unstale":"2023-11-28T03:11:18.397939+0000","last_undegraded":"2023-11-28T03:11:18.397939+0000","last_fullsized":"2023-11-28T03:11:18.397939+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761321+0000","last_change":"2023-11-28T03:11:18.761438+0000","last_active":"2023-11-28T03:11:18.761321+0000","last_peered":"2023-11-28T03:11:18.761321+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760711+0000","last_became_peered":"2023-11-28T03:11:18.760711+0000","last_unstale":"2023-11-28T03:11:18.761321+0000","last_undegraded":"2023-11-28T03:11:18.761321+0000","last_fullsized":"2023-11-28T03:11:18.761321+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396985+0000","last_change":"2023-11-28T03:11:18.396051+0000","last_active":"2023-11-28T03:11:18.396985+0000","last_peered":"2023-11-28T03:11:18.396985+0000","last_clean":"2023-11-28T03:11:18.396985+0000","last_became_active":"2023-11-28T03:11:18.395777+0000","last_became_peered":"2023-11-28T03:11:18.395777+0000","last_unstale":"2023-11-28T03:11:18.396985+0000","last_undegraded":"2023-11-28T03:11:18.396985+0000","last_fullsized":"2023-11-28T03:11:18.396985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764649+0000","last_change":"2023-11-28T03:11:18.764724+0000","last_active":"2023-11-28T03:11:18.764649+0000","last_peered":"2023-11-28T03:11:18.764649+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.764373+0000","last_became_peered":"2023-11-28T03:11:18.764373+0000","last_unstale":"2023-11-28T03:11:18.764649+0000","last_undegraded":"2023-11-28T03:11:18.764649+0000","last_fullsized":"2023-11-28T03:11:18.764649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390603+0000","last_change":"2023-11-28T03:11:18.388277+0000","last_active":"2023-11-28T03:11:18.390603+0000","last_peered":"2023-11-28T03:11:18.390603+0000","last_clean":"2023-11-28T03:11:18.390603+0000","last_became_active":"2023-11-28T03:11:18.387116+0000","last_became_peered":"2023-11-28T03:11:18.387116+0000","last_unstale":"2023-11-28T03:11:18.390603+0000","last_undegraded":"2023-11-28T03:11:18.390603+0000","last_fullsized":"2023-11-28T03:11:18.390603+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749235+0000","last_change":"2023-11-28T03:11:18.749361+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749235+0000","last_undegraded":"2023-11-28T03:11:18.749235+0000","last_fullsized":"2023-11-28T03:11:18.749235+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390576+0000","last_change":"2023-11-28T03:11:18.388476+0000","last_active":"2023-11-28T03:11:18.390576+0000","last_peered":"2023-11-28T03:11:18.390576+0000","last_clean":"2023-11-28T03:11:18.390576+0000","last_became_active":"2023-11-28T03:11:18.387333+0000","last_became_peered":"2023-11-28T03:11:18.387333+0000","last_unstale":"2023-11-28T03:11:18.390576+0000","last_undegraded":"2023-11-28T03:11:18.390576+0000","last_fullsized":"2023-11-28T03:11:18.390576+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750948+0000","last_change":"2023-11-28T03:11:18.751069+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750948+0000","last_undegraded":"2023-11-28T03:11:18.750948+0000","last_fullsized":"2023-11-28T03:11:18.750948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396753+0000","last_change":"2023-11-28T03:11:18.395421+0000","last_active":"2023-11-28T03:11:18.396753+0000","last_peered":"2023-11-28T03:11:18.396753+0000","last_clean":"2023-11-28T03:11:18.396753+0000","last_became_active":"2023-11-28T03:11:18.394055+0000","last_became_peered":"2023-11-28T03:11:18.394055+0000","last_unstale":"2023-11-28T03:11:18.396753+0000","last_undegraded":"2023-11-28T03:11:18.396753+0000","last_fullsized":"2023-11-28T03:11:18.396753+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748206+0000","last_change":"2023-11-28T03:11:18.748338+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748206+0000","last_undegraded":"2023-11-28T03:11:18.748206+0000","last_fullsized":"2023-11-28T03:11:18.748206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388103+0000","last_change":"2023-11-28T03:11:18.378416+0000","last_active":"2023-11-28T03:11:18.388103+0000","last_peered":"2023-11-28T03:11:18.388103+0000","last_clean":"2023-11-28T03:11:18.388103+0000","last_became_active":"2023-11-28T03:11:18.378168+0000","last_became_peered":"2023-11-28T03:11:18.378168+0000","last_unstale":"2023-11-28T03:11:18.388103+0000","last_undegraded":"2023-11-28T03:11:18.388103+0000","last_fullsized":"2023-11-28T03:11:18.388103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760205+0000","last_change":"2023-11-28T03:11:18.760275+0000","last_active":"2023-11-28T03:11:18.760205+0000","last_peered":"2023-11-28T03:11:18.760205+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759999+0000","last_became_peered":"2023-11-28T03:11:18.759999+0000","last_unstale":"2023-11-28T03:11:18.760205+0000","last_undegraded":"2023-11-28T03:11:18.760205+0000","last_fullsized":"2023-11-28T03:11:18.760205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.402009+0000","last_change":"2023-11-28T03:11:18.400805+0000","last_active":"2023-11-28T03:11:18.402009+0000","last_peered":"2023-11-28T03:11:18.402009+0000","last_clean":"2023-11-28T03:11:18.402009+0000","last_became_active":"2023-11-28T03:11:18.400469+0000","last_became_peered":"2023-11-28T03:11:18.400469+0000","last_unstale":"2023-11-28T03:11:18.402009+0000","last_undegraded":"2023-11-28T03:11:18.402009+0000","last_fullsized":"2023-11-28T03:11:18.402009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761738+0000","last_change":"2023-11-28T03:11:18.761855+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761738+0000","last_undegraded":"2023-11-28T03:11:18.761738+0000","last_fullsized":"2023-11-28T03:11:18.761738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390532+0000","last_change":"2023-11-28T03:11:18.386676+0000","last_active":"2023-11-28T03:11:18.390532+0000","last_peered":"2023-11-28T03:11:18.390532+0000","last_clean":"2023-11-28T03:11:18.390532+0000","last_became_active":"2023-11-28T03:11:18.386406+0000","last_became_peered":"2023-11-28T03:11:18.386406+0000","last_unstale":"2023-11-28T03:11:18.390532+0000","last_undegraded":"2023-11-28T03:11:18.390532+0000","last_fullsized":"2023-11-28T03:11:18.390532+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756246+0000","last_change":"2023-11-28T03:11:18.756315+0000","last_active":"2023-11-28T03:11:18.756246+0000","last_peered":"2023-11-28T03:11:18.756246+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755546+0000","last_became_peered":"2023-11-28T03:11:18.755546+0000","last_unstale":"2023-11-28T03:11:18.756246+0000","last_undegraded":"2023-11-28T03:11:18.756246+0000","last_fullsized":"2023-11-28T03:11:18.756246+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393795+0000","last_change":"2023-11-28T03:11:18.390764+0000","last_active":"2023-11-28T03:11:18.393795+0000","last_peered":"2023-11-28T03:11:18.393795+0000","last_clean":"2023-11-28T03:11:18.393795+0000","last_became_active":"2023-11-28T03:11:18.390219+0000","last_became_peered":"2023-11-28T03:11:18.390219+0000","last_unstale":"2023-11-28T03:11:18.393795+0000","last_undegraded":"2023-11-28T03:11:18.393795+0000","last_fullsized":"2023-11-28T03:11:18.393795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763333+0000","last_change":"2023-11-28T03:11:18.763422+0000","last_active":"2023-11-28T03:11:18.763333+0000","last_peered":"2023-11-28T03:11:18.763333+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.763080+0000","last_became_peered":"2023-11-28T03:11:18.763080+0000","last_unstale":"2023-11-28T03:11:18.763333+0000","last_undegraded":"2023-11-28T03:11:18.763333+0000","last_fullsized":"2023-11-28T03:11:18.763333+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399936+0000","last_change":"2023-11-28T03:11:18.395919+0000","last_active":"2023-11-28T03:11:18.399936+0000","last_peered":"2023-11-28T03:11:18.399936+0000","last_clean":"2023-11-28T03:11:18.399936+0000","last_became_active":"2023-11-28T03:11:18.395621+0000","last_became_peered":"2023-11-28T03:11:18.395621+0000","last_unstale":"2023-11-28T03:11:18.399936+0000","last_undegraded":"2023-11-28T03:11:18.399936+0000","last_fullsized":"2023-11-28T03:11:18.399936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762081+0000","last_change":"2023-11-28T03:11:18.762163+0000","last_active":"2023-11-28T03:11:18.762081+0000","last_peered":"2023-11-28T03:11:18.762081+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759034+0000","last_became_peered":"2023-11-28T03:11:18.759034+0000","last_unstale":"2023-11-28T03:11:18.762081+0000","last_undegraded":"2023-11-28T03:11:18.762081+0000","last_fullsized":"2023-11-28T03:11:18.762081+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391545+0000","last_change":"2023-11-28T03:11:18.389034+0000","last_active":"2023-11-28T03:11:18.391545+0000","last_peered":"2023-11-28T03:11:18.391545+0000","last_clean":"2023-11-28T03:11:18.391545+0000","last_became_active":"2023-11-28T03:11:18.388710+0000","last_became_peered":"2023-11-28T03:11:18.388710+0000","last_unstale":"2023-11-28T03:11:18.391545+0000","last_undegraded":"2023-11-28T03:11:18.391545+0000","last_fullsized":"2023-11-28T03:11:18.391545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750502+0000","last_change":"2023-11-28T03:11:18.750640+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750502+0000","last_undegraded":"2023-11-28T03:11:18.750502+0000","last_fullsized":"2023-11-28T03:11:18.750502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394539+0000","last_change":"2023-11-28T03:11:18.391283+0000","last_active":"2023-11-28T03:11:18.394539+0000","last_peered":"2023-11-28T03:11:18.394539+0000","last_clean":"2023-11-28T03:11:18.394539+0000","last_became_active":"2023-11-28T03:11:18.390915+0000","last_became_peered":"2023-11-28T03:11:18.390915+0000","last_unstale":"2023-11-28T03:11:18.394539+0000","last_undegraded":"2023-11-28T03:11:18.394539+0000","last_fullsized":"2023-11-28T03:11:18.394539+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745744+0000","last_change":"2023-11-28T03:11:18.745874+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745744+0000","last_undegraded":"2023-11-28T03:11:18.745744+0000","last_fullsized":"2023-11-28T03:11:18.745744+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394088+0000","last_change":"2023-11-28T03:11:18.392679+0000","last_active":"2023-11-28T03:11:18.394088+0000","last_peered":"2023-11-28T03:11:18.394088+0000","last_clean":"2023-11-28T03:11:18.394088+0000","last_became_active":"2023-11-28T03:11:18.392366+0000","last_became_peered":"2023-11-28T03:11:18.392366+0000","last_unstale":"2023-11-28T03:11:18.394088+0000","last_undegraded":"2023-11-28T03:11:18.394088+0000","last_fullsized":"2023-11-28T03:11:18.394088+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751787+0000","last_change":"2023-11-28T03:11:18.751924+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751787+0000","last_undegraded":"2023-11-28T03:11:18.751787+0000","last_fullsized":"2023-11-28T03:11:18.751787+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387291+0000","last_change":"2023-11-28T03:11:18.385219+0000","last_active":"2023-11-28T03:11:18.387291+0000","last_peered":"2023-11-28T03:11:18.387291+0000","last_clean":"2023-11-28T03:11:18.387291+0000","last_became_active":"2023-11-28T03:11:18.384858+0000","last_became_peered":"2023-11-28T03:11:18.384858+0000","last_unstale":"2023-11-28T03:11:18.387291+0000","last_undegraded":"2023-11-28T03:11:18.387291+0000","last_fullsized":"2023-11-28T03:11:18.387291+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757565+0000","last_change":"2023-11-28T03:11:18.757640+0000","last_active":"2023-11-28T03:11:18.757565+0000","last_peered":"2023-11-28T03:11:18.757565+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757397+0000","last_became_peered":"2023-11-28T03:11:18.757397+0000","last_unstale":"2023-11-28T03:11:18.757565+0000","last_undegraded":"2023-11-28T03:11:18.757565+0000","last_fullsized":"2023-11-28T03:11:18.757565+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397201+0000","last_change":"2023-11-28T03:11:18.395814+0000","last_active":"2023-11-28T03:11:18.397201+0000","last_peered":"2023-11-28T03:11:18.397201+0000","last_clean":"2023-11-28T03:11:18.397201+0000","last_became_active":"2023-11-28T03:11:18.395516+0000","last_became_peered":"2023-11-28T03:11:18.395516+0000","last_unstale":"2023-11-28T03:11:18.397201+0000","last_undegraded":"2023-11-28T03:11:18.397201+0000","last_fullsized":"2023-11-28T03:11:18.397201+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745455+0000","last_change":"2023-11-28T03:11:18.745591+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745455+0000","last_undegraded":"2023-11-28T03:11:18.745455+0000","last_fullsized":"2023-11-28T03:11:18.745455+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397153+0000","last_change":"2023-11-28T03:11:18.395387+0000","last_active":"2023-11-28T03:11:18.397153+0000","last_peered":"2023-11-28T03:11:18.397153+0000","last_clean":"2023-11-28T03:11:18.397153+0000","last_became_active":"2023-11-28T03:11:18.394994+0000","last_became_peered":"2023-11-28T03:11:18.394994+0000","last_unstale":"2023-11-28T03:11:18.397153+0000","last_undegraded":"2023-11-28T03:11:18.397153+0000","last_fullsized":"2023-11-28T03:11:18.397153+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753511+0000","last_change":"2023-11-28T03:11:18.753635+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753511+0000","last_undegraded":"2023-11-28T03:11:18.753511+0000","last_fullsized":"2023-11-28T03:11:18.753511+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748519+0000","last_change":"2023-11-28T03:11:18.394301+0000","last_active":"2023-11-28T03:11:18.748519+0000","last_peered":"2023-11-28T03:11:18.748519+0000","last_clean":"2023-11-28T03:11:18.748519+0000","last_became_active":"2023-11-28T03:11:18.393966+0000","last_became_peered":"2023-11-28T03:11:18.393966+0000","last_unstale":"2023-11-28T03:11:18.748519+0000","last_undegraded":"2023-11-28T03:11:18.748519+0000","last_fullsized":"2023-11-28T03:11:18.748519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761932+0000","last_change":"2023-11-28T03:11:18.761997+0000","last_active":"2023-11-28T03:11:18.761932+0000","last_peered":"2023-11-28T03:11:18.761932+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761245+0000","last_became_peered":"2023-11-28T03:11:18.761245+0000","last_unstale":"2023-11-28T03:11:18.761932+0000","last_undegraded":"2023-11-28T03:11:18.761932+0000","last_fullsized":"2023-11-28T03:11:18.761932+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396237+0000","last_change":"2023-11-28T03:11:18.394666+0000","last_active":"2023-11-28T03:11:18.396237+0000","last_peered":"2023-11-28T03:11:18.396237+0000","last_clean":"2023-11-28T03:11:18.396237+0000","last_became_active":"2023-11-28T03:11:18.393583+0000","last_became_peered":"2023-11-28T03:11:18.393583+0000","last_unstale":"2023-11-28T03:11:18.396237+0000","last_undegraded":"2023-11-28T03:11:18.396237+0000","last_fullsized":"2023-11-28T03:11:18.396237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761889+0000","last_change":"2023-11-28T03:11:18.761969+0000","last_active":"2023-11-28T03:11:18.761889+0000","last_peered":"2023-11-28T03:11:18.761889+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758807+0000","last_became_peered":"2023-11-28T03:11:18.758807+0000","last_unstale":"2023-11-28T03:11:18.761889+0000","last_undegraded":"2023-11-28T03:11:18.761889+0000","last_fullsized":"2023-11-28T03:11:18.761889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395238+0000","last_change":"2023-11-28T03:11:18.384737+0000","last_active":"2023-11-28T03:11:18.395238+0000","last_peered":"2023-11-28T03:11:18.395238+0000","last_clean":"2023-11-28T03:11:18.395238+0000","last_became_active":"2023-11-28T03:11:18.384433+0000","last_became_peered":"2023-11-28T03:11:18.384433+0000","last_unstale":"2023-11-28T03:11:18.395238+0000","last_undegraded":"2023-11-28T03:11:18.395238+0000","last_fullsized":"2023-11-28T03:11:18.395238+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759794+0000","last_change":"2023-11-28T03:11:18.759866+0000","last_active":"2023-11-28T03:11:18.759794+0000","last_peered":"2023-11-28T03:11:18.759794+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759595+0000","last_became_peered":"2023-11-28T03:11:18.759595+0000","last_unstale":"2023-11-28T03:11:18.759794+0000","last_undegraded":"2023-11-28T03:11:18.759794+0000","last_fullsized":"2023-11-28T03:11:18.759794+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395339+0000","last_change":"2023-11-28T03:11:18.391939+0000","last_active":"2023-11-28T03:11:18.395339+0000","last_peered":"2023-11-28T03:11:18.395339+0000","last_clean":"2023-11-28T03:11:18.395339+0000","last_became_active":"2023-11-28T03:11:18.391661+0000","last_became_peered":"2023-11-28T03:11:18.391661+0000","last_unstale":"2023-11-28T03:11:18.395339+0000","last_undegraded":"2023-11-28T03:11:18.395339+0000","last_fullsized":"2023-11-28T03:11:18.395339+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.752002+0000","last_change":"2023-11-28T03:11:18.752124+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.752002+0000","last_undegraded":"2023-11-28T03:11:18.752002+0000","last_fullsized":"2023-11-28T03:11:18.752002+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392802+0000","last_change":"2023-11-28T03:11:18.390407+0000","last_active":"2023-11-28T03:11:18.392802+0000","last_peered":"2023-11-28T03:11:18.392802+0000","last_clean":"2023-11-28T03:11:18.392802+0000","last_became_active":"2023-11-28T03:11:18.390103+0000","last_became_peered":"2023-11-28T03:11:18.390103+0000","last_unstale":"2023-11-28T03:11:18.392802+0000","last_undegraded":"2023-11-28T03:11:18.392802+0000","last_fullsized":"2023-11-28T03:11:18.392802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756058+0000","last_change":"2023-11-28T03:11:18.756132+0000","last_active":"2023-11-28T03:11:18.756058+0000","last_peered":"2023-11-28T03:11:18.756058+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755858+0000","last_became_peered":"2023-11-28T03:11:18.755858+0000","last_unstale":"2023-11-28T03:11:18.756058+0000","last_undegraded":"2023-11-28T03:11:18.756058+0000","last_fullsized":"2023-11-28T03:11:18.756058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392637+0000","last_change":"2023-11-28T03:11:18.387271+0000","last_active":"2023-11-28T03:11:18.392637+0000","last_peered":"2023-11-28T03:11:18.392637+0000","last_clean":"2023-11-28T03:11:18.392637+0000","last_became_active":"2023-11-28T03:11:18.386973+0000","last_became_peered":"2023-11-28T03:11:18.386973+0000","last_unstale":"2023-11-28T03:11:18.392637+0000","last_undegraded":"2023-11-28T03:11:18.392637+0000","last_fullsized":"2023-11-28T03:11:18.392637+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.762229+0000","last_change":"2023-11-28T03:11:18.762360+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.762229+0000","last_undegraded":"2023-11-28T03:11:18.762229+0000","last_fullsized":"2023-11-28T03:11:18.762229+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392472+0000","last_change":"2023-11-28T03:11:18.387360+0000","last_active":"2023-11-28T03:11:18.392472+0000","last_peered":"2023-11-28T03:11:18.392472+0000","last_clean":"2023-11-28T03:11:18.392472+0000","last_became_active":"2023-11-28T03:11:18.386943+0000","last_became_peered":"2023-11-28T03:11:18.386943+0000","last_unstale":"2023-11-28T03:11:18.392472+0000","last_undegraded":"2023-11-28T03:11:18.392472+0000","last_fullsized":"2023-11-28T03:11:18.392472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.747957+0000","last_change":"2023-11-28T03:11:18.748091+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.747957+0000","last_undegraded":"2023-11-28T03:11:18.747957+0000","last_fullsized":"2023-11-28T03:11:18.747957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392710+0000","last_change":"2023-11-28T03:11:18.390261+0000","last_active":"2023-11-28T03:11:18.392710+0000","last_peered":"2023-11-28T03:11:18.392710+0000","last_clean":"2023-11-28T03:11:18.392710+0000","last_became_active":"2023-11-28T03:11:18.389960+0000","last_became_peered":"2023-11-28T03:11:18.389960+0000","last_unstale":"2023-11-28T03:11:18.392710+0000","last_undegraded":"2023-11-28T03:11:18.392710+0000","last_fullsized":"2023-11-28T03:11:18.392710+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753207+0000","last_change":"2023-11-28T03:11:18.753332+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753207+0000","last_undegraded":"2023-11-28T03:11:18.753207+0000","last_fullsized":"2023-11-28T03:11:18.753207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380619+0000","last_change":"2023-11-28T03:11:18.379037+0000","last_active":"2023-11-28T03:11:18.380619+0000","last_peered":"2023-11-28T03:11:18.380619+0000","last_clean":"2023-11-28T03:11:18.380619+0000","last_became_active":"2023-11-28T03:11:18.378709+0000","last_became_peered":"2023-11-28T03:11:18.378709+0000","last_unstale":"2023-11-28T03:11:18.380619+0000","last_undegraded":"2023-11-28T03:11:18.380619+0000","last_fullsized":"2023-11-28T03:11:18.380619+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751193+0000","last_change":"2023-11-28T03:11:18.751193+0000","last_active":"2023-11-28T03:11:18.751193+0000","last_peered":"2023-11-28T03:11:18.751193+0000","last_clean":"2023-11-28T03:11:18.751193+0000","last_became_active":"2023-11-28T03:11:18.750949+0000","last_became_peered":"2023-11-28T03:11:18.750949+0000","last_unstale":"2023-11-28T03:11:18.751193+0000","last_undegraded":"2023-11-28T03:11:18.751193+0000","last_fullsized":"2023-11-28T03:11:18.751193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390906+0000","last_change":"2023-11-28T03:11:18.388975+0000","last_active":"2023-11-28T03:11:18.390906+0000","last_peered":"2023-11-28T03:11:18.390906+0000","last_clean":"2023-11-28T03:11:18.390906+0000","last_became_active":"2023-11-28T03:11:18.388685+0000","last_became_peered":"2023-11-28T03:11:18.388685+0000","last_unstale":"2023-11-28T03:11:18.390906+0000","last_undegraded":"2023-11-28T03:11:18.390906+0000","last_fullsized":"2023-11-28T03:11:18.390906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757330+0000","last_change":"2023-11-28T03:11:18.757471+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757330+0000","last_undegraded":"2023-11-28T03:11:18.757330+0000","last_fullsized":"2023-11-28T03:11:18.757330+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392301+0000","last_change":"2023-11-28T03:11:18.390087+0000","last_active":"2023-11-28T03:11:18.392301+0000","last_peered":"2023-11-28T03:11:18.392301+0000","last_clean":"2023-11-28T03:11:18.392301+0000","last_became_active":"2023-11-28T03:11:18.389794+0000","last_became_peered":"2023-11-28T03:11:18.389794+0000","last_unstale":"2023-11-28T03:11:18.392301+0000","last_undegraded":"2023-11-28T03:11:18.392301+0000","last_fullsized":"2023-11-28T03:11:18.392301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750181+0000","last_change":"2023-11-28T03:11:18.750339+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750181+0000","last_undegraded":"2023-11-28T03:11:18.750181+0000","last_fullsized":"2023-11-28T03:11:18.750181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395320+0000","last_change":"2023-11-28T03:11:18.388300+0000","last_active":"2023-11-28T03:11:18.395320+0000","last_peered":"2023-11-28T03:11:18.395320+0000","last_clean":"2023-11-28T03:11:18.395320+0000","last_became_active":"2023-11-28T03:11:18.387929+0000","last_became_peered":"2023-11-28T03:11:18.387929+0000","last_unstale":"2023-11-28T03:11:18.395320+0000","last_undegraded":"2023-11-28T03:11:18.395320+0000","last_fullsized":"2023-11-28T03:11:18.395320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745039+0000","last_change":"2023-11-28T03:11:18.745192+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745039+0000","last_undegraded":"2023-11-28T03:11:18.745039+0000","last_fullsized":"2023-11-28T03:11:18.745039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388843+0000","last_change":"2023-11-28T03:11:18.384508+0000","last_active":"2023-11-28T03:11:18.388843+0000","last_peered":"2023-11-28T03:11:18.388843+0000","last_clean":"2023-11-28T03:11:18.388843+0000","last_became_active":"2023-11-28T03:11:18.384214+0000","last_became_peered":"2023-11-28T03:11:18.384214+0000","last_unstale":"2023-11-28T03:11:18.388843+0000","last_undegraded":"2023-11-28T03:11:18.388843+0000","last_fullsized":"2023-11-28T03:11:18.388843+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373322+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:11:18.373322+0000","last_peered":"2023-11-28T03:11:18.373322+0000","last_clean":"2023-11-28T03:11:18.373322+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:11:18.373322+0000","last_undegraded":"2023-11-28T03:11:18.373322+0000","last_fullsized":"2023-11-28T03:11:18.373322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750870+0000","last_change":"2023-11-28T03:11:18.750870+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750870+0000","last_undegraded":"2023-11-28T03:11:18.750870+0000","last_fullsized":"2023-11-28T03:11:18.750870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395340+0000","last_change":"2023-11-28T03:11:18.387081+0000","last_active":"2023-11-28T03:11:18.395340+0000","last_peered":"2023-11-28T03:11:18.395340+0000","last_clean":"2023-11-28T03:11:18.395340+0000","last_became_active":"2023-11-28T03:11:18.386827+0000","last_became_peered":"2023-11-28T03:11:18.386827+0000","last_unstale":"2023-11-28T03:11:18.395340+0000","last_undegraded":"2023-11-28T03:11:18.395340+0000","last_fullsized":"2023-11-28T03:11:18.395340+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.377275+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:11:18.377275+0000","last_peered":"2023-11-28T03:11:18.377275+0000","last_clean":"2023-11-28T03:11:18.377275+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:11:18.377275+0000","last_undegraded":"2023-11-28T03:11:18.377275+0000","last_fullsized":"2023-11-28T03:11:18.377275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749941+0000","last_change":"2023-11-28T03:11:18.750076+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749941+0000","last_undegraded":"2023-11-28T03:11:18.749941+0000","last_fullsized":"2023-11-28T03:11:18.749941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395335+0000","last_change":"2023-11-28T03:11:18.386867+0000","last_active":"2023-11-28T03:11:18.395335+0000","last_peered":"2023-11-28T03:11:18.395335+0000","last_clean":"2023-11-28T03:11:18.395335+0000","last_became_active":"2023-11-28T03:11:18.386623+0000","last_became_peered":"2023-11-28T03:11:18.386623+0000","last_unstale":"2023-11-28T03:11:18.395335+0000","last_undegraded":"2023-11-28T03:11:18.395335+0000","last_fullsized":"2023-11-28T03:11:18.395335+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.370453+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:11:18.370453+0000","last_peered":"2023-11-28T03:11:18.370453+0000","last_clean":"2023-11-28T03:11:18.370453+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:11:18.370453+0000","last_undegraded":"2023-11-28T03:11:18.370453+0000","last_fullsized":"2023-11-28T03:11:18.370453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758040+0000","last_change":"2023-11-28T03:11:18.758171+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758040+0000","last_undegraded":"2023-11-28T03:11:18.758040+0000","last_fullsized":"2023-11-28T03:11:18.758040+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396967+0000","last_change":"2023-11-28T03:11:18.394496+0000","last_active":"2023-11-28T03:11:18.396967+0000","last_peered":"2023-11-28T03:11:18.396967+0000","last_clean":"2023-11-28T03:11:18.396967+0000","last_became_active":"2023-11-28T03:11:18.394294+0000","last_became_peered":"2023-11-28T03:11:18.394294+0000","last_unstale":"2023-11-28T03:11:18.396967+0000","last_undegraded":"2023-11-28T03:11:18.396967+0000","last_fullsized":"2023-11-28T03:11:18.396967+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.382976+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:11:18.382976+0000","last_peered":"2023-11-28T03:11:18.382976+0000","last_clean":"2023-11-28T03:11:18.382976+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:11:18.382976+0000","last_undegraded":"2023-11-28T03:11:18.382976+0000","last_fullsized":"2023-11-28T03:11:18.382976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380178+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:11:18.380178+0000","last_peered":"2023-11-28T03:11:18.380178+0000","last_clean":"2023-11-28T03:11:18.380178+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:11:18.380178+0000","last_undegraded":"2023-11-28T03:11:18.380178+0000","last_fullsized":"2023-11-28T03:11:18.380178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760444+0000","last_change":"2023-11-28T03:11:18.760540+0000","last_active":"2023-11-28T03:11:18.760444+0000","last_peered":"2023-11-28T03:11:18.760444+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760079+0000","last_became_peered":"2023-11-28T03:11:18.760079+0000","last_unstale":"2023-11-28T03:11:18.760444+0000","last_undegraded":"2023-11-28T03:11:18.760444+0000","last_fullsized":"2023-11-28T03:11:18.760444+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400010+0000","last_change":"2023-11-28T03:11:18.396476+0000","last_active":"2023-11-28T03:11:18.400010+0000","last_peered":"2023-11-28T03:11:18.400010+0000","last_clean":"2023-11-28T03:11:18.400010+0000","last_became_active":"2023-11-28T03:11:18.396309+0000","last_became_peered":"2023-11-28T03:11:18.396309+0000","last_unstale":"2023-11-28T03:11:18.400010+0000","last_undegraded":"2023-11-28T03:11:18.400010+0000","last_fullsized":"2023-11-28T03:11:18.400010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.381718+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:11:18.381718+0000","last_peered":"2023-11-28T03:11:18.381718+0000","last_clean":"2023-11-28T03:11:18.381718+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:11:18.381718+0000","last_undegraded":"2023-11-28T03:11:18.381718+0000","last_fullsized":"2023-11-28T03:11:18.381718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.744775+0000","last_change":"2023-11-28T03:11:18.744919+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.744775+0000","last_undegraded":"2023-11-28T03:11:18.744775+0000","last_fullsized":"2023-11-28T03:11:18.744775+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389393+0000","last_change":"2023-11-28T03:11:18.387795+0000","last_active":"2023-11-28T03:11:18.389393+0000","last_peered":"2023-11-28T03:11:18.389393+0000","last_clean":"2023-11-28T03:11:18.389393+0000","last_became_active":"2023-11-28T03:11:18.387481+0000","last_became_peered":"2023-11-28T03:11:18.387481+0000","last_unstale":"2023-11-28T03:11:18.389393+0000","last_undegraded":"2023-11-28T03:11:18.389393+0000","last_fullsized":"2023-11-28T03:11:18.389393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392461+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:11:18.392461+0000","last_peered":"2023-11-28T03:11:18.392461+0000","last_clean":"2023-11-28T03:11:18.392461+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:11:18.392461+0000","last_undegraded":"2023-11-28T03:11:18.392461+0000","last_fullsized":"2023-11-28T03:11:18.392461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757177+0000","last_change":"2023-11-28T03:11:18.757177+0000","last_active":"2023-11-28T03:11:18.757177+0000","last_peered":"2023-11-28T03:11:18.757177+0000","last_clean":"2023-11-28T03:11:18.757177+0000","last_became_active":"2023-11-28T03:11:18.756875+0000","last_became_peered":"2023-11-28T03:11:18.756875+0000","last_unstale":"2023-11-28T03:11:18.757177+0000","last_undegraded":"2023-11-28T03:11:18.757177+0000","last_fullsized":"2023-11-28T03:11:18.757177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755179+0000","last_change":"2023-11-28T03:11:18.392352+0000","last_active":"2023-11-28T03:11:18.755179+0000","last_peered":"2023-11-28T03:11:18.755179+0000","last_clean":"2023-11-28T03:11:18.755179+0000","last_became_active":"2023-11-28T03:11:18.392050+0000","last_became_peered":"2023-11-28T03:11:18.392050+0000","last_unstale":"2023-11-28T03:11:18.755179+0000","last_undegraded":"2023-11-28T03:11:18.755179+0000","last_fullsized":"2023-11-28T03:11:18.755179+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373107+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:11:18.373107+0000","last_peered":"2023-11-28T03:11:18.373107+0000","last_clean":"2023-11-28T03:11:18.373107+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:11:18.373107+0000","last_undegraded":"2023-11-28T03:11:18.373107+0000","last_fullsized":"2023-11-28T03:11:18.373107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761405+0000","last_change":"2023-11-28T03:11:18.761490+0000","last_active":"2023-11-28T03:11:18.761405+0000","last_peered":"2023-11-28T03:11:18.761405+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761053+0000","last_became_peered":"2023-11-28T03:11:18.761053+0000","last_unstale":"2023-11-28T03:11:18.761405+0000","last_undegraded":"2023-11-28T03:11:18.761405+0000","last_fullsized":"2023-11-28T03:11:18.761405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395315+0000","last_change":"2023-11-28T03:11:18.386215+0000","last_active":"2023-11-28T03:11:18.395315+0000","last_peered":"2023-11-28T03:11:18.395315+0000","last_clean":"2023-11-28T03:11:18.395315+0000","last_became_active":"2023-11-28T03:11:18.385765+0000","last_became_peered":"2023-11-28T03:11:18.385765+0000","last_unstale":"2023-11-28T03:11:18.395315+0000","last_undegraded":"2023-11-28T03:11:18.395315+0000","last_fullsized":"2023-11-28T03:11:18.395315+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.378564+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:11:18.378564+0000","last_peered":"2023-11-28T03:11:18.378564+0000","last_clean":"2023-11-28T03:11:18.378564+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:11:18.378564+0000","last_undegraded":"2023-11-28T03:11:18.378564+0000","last_fullsized":"2023-11-28T03:11:18.378564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69899999999999995,"5min":1.46,"15min":1.46},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69799999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60999999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":0.64400000000000002,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.76900000000000002,"15min":0.84499999999999997},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.73699999999999999,"15min":1.4810000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.94399999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":1.125,"15min":1.506},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93000000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.90900000000000003},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.441,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.627,"5min":0.78100000000000003,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":0.59699999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.874,"15min":0.95099999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.4490000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.91100000000000003,"15min":1.3799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.90400000000000003,"15min":1.1040000000000001},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.67800000000000005},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.218,"15min":0.20899999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.67100000000000004},"last":0.22}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.21199999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.25800000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.45600000000000002},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41399999999999998,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97799999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.99299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.038},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.94099999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83199999999999996,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.024},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88300000000000001},"last":0.373}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.56599999999999995},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34399999999999997,"15min":0.36499999999999999},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53700000000000003,"5min":0.58999999999999997,"15min":0.70299999999999996},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33900000000000002,"15min":0.36299999999999999},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.439},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.73299999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.371,"15min":0.39300000000000002},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47999999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.41899999999999998},"min":{"1min":0.318,"5min":0.217,"15min":0.217},"max":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.65400000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79500000000000004,"5min":1.427,"15min":1.427},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.628},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.747,"5min":0.85899999999999999,"15min":1.2869999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.629},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":1.304},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":1.456,"15min":1.456},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.4399999999999999,"15min":1.679},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59099999999999997,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80700000000000005,"5min":1.341,"15min":1.341},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":1.474,"15min":1.474},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.315},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.39700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.19800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.67400000000000004},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.627,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.25900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.35799999999999998},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.98599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":1.004},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.91500000000000004,"15min":0.93300000000000005},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20799999999999999},"max":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.59899999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68899999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.74199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.34799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.878,"15min":1.581},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94199999999999995},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.35699999999999998},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.97899999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.93100000000000005,"15min":0.996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38,"15min":0.376},"max":{"1min":0.87,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60499999999999998,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.878},"last":0.621}]},{"osd":5,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80000000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.27900000000000003,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.28000000000000003},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.78300000000000003,"15min":0.83299999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.82499999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.747,"15min":0.80500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.81499999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.94399999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63700000000000001,"5min":0.77800000000000002,"15min":0.89900000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.91200000000000003},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:11:21.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:11:21.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.878+0000 7f41c359e700 1 Processor -- start 2023-11-28T03:11:21.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.878+0000 7f41c359e700 1 -- start start 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c359e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 0x7f41c40694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c359e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41c4069e60 con 0x7f41c40699d0 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c359e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41c4066e60 con 0x7f41c4155510 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c4067140 con 0x7f41c413a8d0 2023-11-28T03:11:21.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41b9d9b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 0x7f41c40694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c259c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c259c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:40888/0 (socket says 172.21.15.73:40888) 2023-11-28T03:11:21.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c259c700 1 -- 172.21.15.73:0/2083963967 learned_addr learned my addr 172.21.15.73:0/2083963967 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.879+0000 7f41c2d9d700 1 --1- 172.21.15.73:0/2083963967 >> v1:172.21.15.150:6789/0 conn(0x7f41c40699d0 0x7f41c40c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:50850/0 (socket says 172.21.15.73:50850) 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c259c700 1 -- 172.21.15.73:0/2083963967 >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 msgr2=0x7f41c40694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c259c700 1 --2- 172.21.15.73:0/2083963967 >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 0x7f41c40694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c259c700 1 -- 172.21.15.73:0/2083963967 >> v1:172.21.15.150:6789/0 conn(0x7f41c40699d0 legacy=0x7f41c40c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c259c700 1 -- 172.21.15.73:0/2083963967 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41c4067420 con 0x7f41c413a8d0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c1d9b700 1 -- 172.21.15.73:0/2083963967 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182210864 0 0) 0x7f41c4069e60 con 0x7f41c40699d0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41b9d9b700 1 --2- 172.21.15.73:0/2083963967 >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 0x7f41c40694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c259c700 1 --2- 172.21.15.73:0/2083963967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f41b4006fd0 tx=0x7f41b40090f0).ready entity=mon.2 client_cookie=65bf16d031eab8bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c1d9b700 1 -- 172.21.15.73:0/2083963967 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b400b580 con 0x7f41c413a8d0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c1d9b700 1 -- 172.21.15.73:0/2083963967 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b400b6e0 con 0x7f41c413a8d0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c1d9b700 1 -- 172.21.15.73:0/2083963967 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b400ba60 con 0x7f41c413a8d0 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c359e700 1 -- 172.21.15.73:0/2083963967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 msgr2=0x7f41c4066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.880+0000 7f41c359e700 1 --2- 172.21.15.73:0/2083963967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 rx=0x7f41b4006fd0 tx=0x7f41b40090f0).stop 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 -- 172.21.15.73:0/2083963967 shutdown_connections 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 --2- 172.21.15.73:0/2083963967 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c413a8d0 0x7f41c4066880 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 --2- 172.21.15.73:0/2083963967 >> v2:172.21.15.150:3300/0 conn(0x7f41c4155510 0x7f41c40694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 -- 172.21.15.73:0/2083963967 >> 172.21.15.73:0/2083963967 conn(0x7f41c40c7ca0 msgr2=0x7f41c40c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 -- 172.21.15.73:0/2083963967 shutdown_connections 2023-11-28T03:11:21.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.881+0000 7f41c359e700 1 -- 172.21.15.73:0/2083963967 wait complete. 2023-11-28T03:11:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.882+0000 7f41c359e700 1 Processor -- start 2023-11-28T03:11:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.882+0000 7f41c359e700 1 -- start start 2023-11-28T03:11:21.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.882+0000 7f41c359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c413a8d0 0x7f41c40c2b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c4155510 0x7f41c40a8510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40a8080 con 0x7f41c40699d0 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40c3000 con 0x7f41c413a8d0 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41c40c32a0 con 0x7f41c4155510 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41606/0 (socket says 172.21.15.73:41606) 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 -- 172.21.15.73:0/320834244 learned_addr learned my addr 172.21.15.73:0/320834244 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c2d9d700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c4155510 0x7f41c40a8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41b9d9b700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c413a8d0 0x7f41c40c2b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 -- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c4155510 msgr2=0x7f41c40a8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c4155510 0x7f41c40a8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 -- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c413a8d0 msgr2=0x7f41c40c2b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c413a8d0 0x7f41c40c2b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 -- 172.21.15.73:0/320834244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0092a0 con 0x7f41c40699d0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c259c700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f41b4009fd0 tx=0x7f41b4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b4006a40 con 0x7f41c40699d0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b4000bc0 con 0x7f41c40699d0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b4009370 con 0x7f41c40699d0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b401c420 con 0x7f41c40699d0 2023-11-28T03:11:21.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.883+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41c40a7a30 con 0x7f41c40699d0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.884+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41c40a8360 con 0x7f41c40699d0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.885+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f41b401ca20 con 0x7f41c40699d0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.885+0000 7f41baffd700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 0x7f41a4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.885+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f41b407bf90 con 0x7f41c40699d0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.885+0000 7f41b9d9b700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 0x7f41a4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.885+0000 7f41b9d9b700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 0x7f41a4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f41c40a7510 tx=0x7f41ac009060).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:21.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:21.889+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41b40aaaf0 con 0x7f41c40699d0 2023-11-28T03:11:22.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.047+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f41c40c6db0 con 0x7f41a405f150 2023-11-28T03:11:22.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.057+0000 7f41baffd700 1 -- 172.21.15.73:0/320834244 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360699 (secure 0 0 0) 0x7f41c40c6db0 con 0x7f41a405f150 2023-11-28T03:11:22.062 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 msgr2=0x7f41a4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 0x7f41a4061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f41c40a7510 tx=0x7f41ac009060).stop 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 msgr2=0x7f41c40a04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 rx=0x7f41b4009fd0 tx=0x7f41b4017000).stop 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 shutdown_connections 2023-11-28T03:11:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f41a405f150 0x7f41a4061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41c4155510 0x7f41c40a8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41c40699d0 0x7f41c40a04d0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 --2- 172.21.15.73:0/320834244 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41c413a8d0 0x7f41c40c2b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 >> 172.21.15.73:0/320834244 conn(0x7f41c40c7ca0 msgr2=0x7f41c40aac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 shutdown_connections 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.060+0000 7f41c359e700 1 -- 172.21.15.73:0/320834244 wait complete. 2023-11-28T03:11:22.066 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:11:22.073 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:11:21.763486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426948,"kb_used_data":13316,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752547772,"statfs":{"total":773094113280,"available":770608918528,"internally_reserved":0,"allocated":13635584,"data_stored":6798157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19375,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.586494"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757468+0000","last_change":"2023-11-28T03:11:18.757597+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757468+0000","last_undegraded":"2023-11-28T03:11:18.757468+0000","last_fullsized":"2023-11-28T03:11:18.757468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388461+0000","last_change":"2023-11-28T03:11:18.385568+0000","last_active":"2023-11-28T03:11:18.388461+0000","last_peered":"2023-11-28T03:11:18.388461+0000","last_clean":"2023-11-28T03:11:18.388461+0000","last_became_active":"2023-11-28T03:11:18.385211+0000","last_became_peered":"2023-11-28T03:11:18.385211+0000","last_unstale":"2023-11-28T03:11:18.388461+0000","last_undegraded":"2023-11-28T03:11:18.388461+0000","last_fullsized":"2023-11-28T03:11:18.388461+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759645+0000","last_change":"2023-11-28T03:11:18.759718+0000","last_active":"2023-11-28T03:11:18.759645+0000","last_peered":"2023-11-28T03:11:18.759645+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759448+0000","last_became_peered":"2023-11-28T03:11:18.759448+0000","last_unstale":"2023-11-28T03:11:18.759645+0000","last_undegraded":"2023-11-28T03:11:18.759645+0000","last_fullsized":"2023-11-28T03:11:18.759645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391029+0000","last_change":"2023-11-28T03:11:18.389741+0000","last_active":"2023-11-28T03:11:18.391029+0000","last_peered":"2023-11-28T03:11:18.391029+0000","last_clean":"2023-11-28T03:11:18.391029+0000","last_became_active":"2023-11-28T03:11:18.389440+0000","last_became_peered":"2023-11-28T03:11:18.389440+0000","last_unstale":"2023-11-28T03:11:18.391029+0000","last_undegraded":"2023-11-28T03:11:18.391029+0000","last_fullsized":"2023-11-28T03:11:18.391029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768741+0000","last_change":"2023-11-28T03:11:18.768823+0000","last_active":"2023-11-28T03:11:18.768741+0000","last_peered":"2023-11-28T03:11:18.768741+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766758+0000","last_became_peered":"2023-11-28T03:11:18.766758+0000","last_unstale":"2023-11-28T03:11:18.768741+0000","last_undegraded":"2023-11-28T03:11:18.768741+0000","last_fullsized":"2023-11-28T03:11:18.768741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397054+0000","last_change":"2023-11-28T03:11:18.394730+0000","last_active":"2023-11-28T03:11:18.397054+0000","last_peered":"2023-11-28T03:11:18.397054+0000","last_clean":"2023-11-28T03:11:18.397054+0000","last_became_active":"2023-11-28T03:11:18.394473+0000","last_became_peered":"2023-11-28T03:11:18.394473+0000","last_unstale":"2023-11-28T03:11:18.397054+0000","last_undegraded":"2023-11-28T03:11:18.397054+0000","last_fullsized":"2023-11-28T03:11:18.397054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759861+0000","last_change":"2023-11-28T03:11:18.759965+0000","last_active":"2023-11-28T03:11:18.759861+0000","last_peered":"2023-11-28T03:11:18.759861+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759675+0000","last_became_peered":"2023-11-28T03:11:18.759675+0000","last_unstale":"2023-11-28T03:11:18.759861+0000","last_undegraded":"2023-11-28T03:11:18.759861+0000","last_fullsized":"2023-11-28T03:11:18.759861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392731+0000","last_change":"2023-11-28T03:11:18.390367+0000","last_active":"2023-11-28T03:11:18.392731+0000","last_peered":"2023-11-28T03:11:18.392731+0000","last_clean":"2023-11-28T03:11:18.392731+0000","last_became_active":"2023-11-28T03:11:18.390086+0000","last_became_peered":"2023-11-28T03:11:18.390086+0000","last_unstale":"2023-11-28T03:11:18.392731+0000","last_undegraded":"2023-11-28T03:11:18.392731+0000","last_fullsized":"2023-11-28T03:11:18.392731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757310+0000","last_change":"2023-11-28T03:11:18.757382+0000","last_active":"2023-11-28T03:11:18.757310+0000","last_peered":"2023-11-28T03:11:18.757310+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.756962+0000","last_became_peered":"2023-11-28T03:11:18.756962+0000","last_unstale":"2023-11-28T03:11:18.757310+0000","last_undegraded":"2023-11-28T03:11:18.757310+0000","last_fullsized":"2023-11-28T03:11:18.757310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387008+0000","last_change":"2023-11-28T03:11:18.385649+0000","last_active":"2023-11-28T03:11:18.387008+0000","last_peered":"2023-11-28T03:11:18.387008+0000","last_clean":"2023-11-28T03:11:18.387008+0000","last_became_active":"2023-11-28T03:11:18.385386+0000","last_became_peered":"2023-11-28T03:11:18.385386+0000","last_unstale":"2023-11-28T03:11:18.387008+0000","last_undegraded":"2023-11-28T03:11:18.387008+0000","last_fullsized":"2023-11-28T03:11:18.387008+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748324+0000","last_change":"2023-11-28T03:11:18.748460+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748324+0000","last_undegraded":"2023-11-28T03:11:18.748324+0000","last_fullsized":"2023-11-28T03:11:18.748324+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394106+0000","last_change":"2023-11-28T03:11:18.392909+0000","last_active":"2023-11-28T03:11:18.394106+0000","last_peered":"2023-11-28T03:11:18.394106+0000","last_clean":"2023-11-28T03:11:18.394106+0000","last_became_active":"2023-11-28T03:11:18.392552+0000","last_became_peered":"2023-11-28T03:11:18.392552+0000","last_unstale":"2023-11-28T03:11:18.394106+0000","last_undegraded":"2023-11-28T03:11:18.394106+0000","last_fullsized":"2023-11-28T03:11:18.394106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748559+0000","last_change":"2023-11-28T03:11:18.748669+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748559+0000","last_undegraded":"2023-11-28T03:11:18.748559+0000","last_fullsized":"2023-11-28T03:11:18.748559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392365+0000","last_change":"2023-11-28T03:11:18.389689+0000","last_active":"2023-11-28T03:11:18.392365+0000","last_peered":"2023-11-28T03:11:18.392365+0000","last_clean":"2023-11-28T03:11:18.392365+0000","last_became_active":"2023-11-28T03:11:18.388755+0000","last_became_peered":"2023-11-28T03:11:18.388755+0000","last_unstale":"2023-11-28T03:11:18.392365+0000","last_undegraded":"2023-11-28T03:11:18.392365+0000","last_fullsized":"2023-11-28T03:11:18.392365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757626+0000","last_change":"2023-11-28T03:11:18.757695+0000","last_active":"2023-11-28T03:11:18.757626+0000","last_peered":"2023-11-28T03:11:18.757626+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757427+0000","last_became_peered":"2023-11-28T03:11:18.757427+0000","last_unstale":"2023-11-28T03:11:18.757626+0000","last_undegraded":"2023-11-28T03:11:18.757626+0000","last_fullsized":"2023-11-28T03:11:18.757626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395299+0000","last_change":"2023-11-28T03:11:18.391234+0000","last_active":"2023-11-28T03:11:18.395299+0000","last_peered":"2023-11-28T03:11:18.395299+0000","last_clean":"2023-11-28T03:11:18.395299+0000","last_became_active":"2023-11-28T03:11:18.390855+0000","last_became_peered":"2023-11-28T03:11:18.390855+0000","last_unstale":"2023-11-28T03:11:18.395299+0000","last_undegraded":"2023-11-28T03:11:18.395299+0000","last_fullsized":"2023-11-28T03:11:18.395299+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751698+0000","last_change":"2023-11-28T03:11:18.751811+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751698+0000","last_undegraded":"2023-11-28T03:11:18.751698+0000","last_fullsized":"2023-11-28T03:11:18.751698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389124+0000","last_change":"2023-11-28T03:11:18.385956+0000","last_active":"2023-11-28T03:11:18.389124+0000","last_peered":"2023-11-28T03:11:18.389124+0000","last_clean":"2023-11-28T03:11:18.389124+0000","last_became_active":"2023-11-28T03:11:18.385634+0000","last_became_peered":"2023-11-28T03:11:18.385634+0000","last_unstale":"2023-11-28T03:11:18.389124+0000","last_undegraded":"2023-11-28T03:11:18.389124+0000","last_fullsized":"2023-11-28T03:11:18.389124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761439+0000","last_change":"2023-11-28T03:11:18.761511+0000","last_active":"2023-11-28T03:11:18.761439+0000","last_peered":"2023-11-28T03:11:18.761439+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761234+0000","last_became_peered":"2023-11-28T03:11:18.761234+0000","last_unstale":"2023-11-28T03:11:18.761439+0000","last_undegraded":"2023-11-28T03:11:18.761439+0000","last_fullsized":"2023-11-28T03:11:18.761439+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386815+0000","last_change":"2023-11-28T03:11:18.384000+0000","last_active":"2023-11-28T03:11:18.386815+0000","last_peered":"2023-11-28T03:11:18.386815+0000","last_clean":"2023-11-28T03:11:18.386815+0000","last_became_active":"2023-11-28T03:11:18.383737+0000","last_became_peered":"2023-11-28T03:11:18.383737+0000","last_unstale":"2023-11-28T03:11:18.386815+0000","last_undegraded":"2023-11-28T03:11:18.386815+0000","last_fullsized":"2023-11-28T03:11:18.386815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749608+0000","last_change":"2023-11-28T03:11:18.749746+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749608+0000","last_undegraded":"2023-11-28T03:11:18.749608+0000","last_fullsized":"2023-11-28T03:11:18.749608+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390548+0000","last_change":"2023-11-28T03:11:18.386918+0000","last_active":"2023-11-28T03:11:18.390548+0000","last_peered":"2023-11-28T03:11:18.390548+0000","last_clean":"2023-11-28T03:11:18.390548+0000","last_became_active":"2023-11-28T03:11:18.386386+0000","last_became_peered":"2023-11-28T03:11:18.386386+0000","last_unstale":"2023-11-28T03:11:18.390548+0000","last_undegraded":"2023-11-28T03:11:18.390548+0000","last_fullsized":"2023-11-28T03:11:18.390548+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761388+0000","last_change":"2023-11-28T03:11:18.761551+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761388+0000","last_undegraded":"2023-11-28T03:11:18.761388+0000","last_fullsized":"2023-11-28T03:11:18.761388+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386802+0000","last_change":"2023-11-28T03:11:18.384007+0000","last_active":"2023-11-28T03:11:18.386802+0000","last_peered":"2023-11-28T03:11:18.386802+0000","last_clean":"2023-11-28T03:11:18.386802+0000","last_became_active":"2023-11-28T03:11:18.383671+0000","last_became_peered":"2023-11-28T03:11:18.383671+0000","last_unstale":"2023-11-28T03:11:18.386802+0000","last_undegraded":"2023-11-28T03:11:18.386802+0000","last_fullsized":"2023-11-28T03:11:18.386802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753548+0000","last_change":"2023-11-28T03:11:18.753676+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753548+0000","last_undegraded":"2023-11-28T03:11:18.753548+0000","last_fullsized":"2023-11-28T03:11:18.753548+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395509+0000","last_change":"2023-11-28T03:11:18.391315+0000","last_active":"2023-11-28T03:11:18.395509+0000","last_peered":"2023-11-28T03:11:18.395509+0000","last_clean":"2023-11-28T03:11:18.395509+0000","last_became_active":"2023-11-28T03:11:18.391053+0000","last_became_peered":"2023-11-28T03:11:18.391053+0000","last_unstale":"2023-11-28T03:11:18.395509+0000","last_undegraded":"2023-11-28T03:11:18.395509+0000","last_fullsized":"2023-11-28T03:11:18.395509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756085+0000","last_change":"2023-11-28T03:11:18.756179+0000","last_active":"2023-11-28T03:11:18.756085+0000","last_peered":"2023-11-28T03:11:18.756085+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755853+0000","last_became_peered":"2023-11-28T03:11:18.755853+0000","last_unstale":"2023-11-28T03:11:18.756085+0000","last_undegraded":"2023-11-28T03:11:18.756085+0000","last_fullsized":"2023-11-28T03:11:18.756085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395277+0000","last_change":"2023-11-28T03:11:18.391280+0000","last_active":"2023-11-28T03:11:18.395277+0000","last_peered":"2023-11-28T03:11:18.395277+0000","last_clean":"2023-11-28T03:11:18.395277+0000","last_became_active":"2023-11-28T03:11:18.387894+0000","last_became_peered":"2023-11-28T03:11:18.387894+0000","last_unstale":"2023-11-28T03:11:18.395277+0000","last_undegraded":"2023-11-28T03:11:18.395277+0000","last_fullsized":"2023-11-28T03:11:18.395277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759232+0000","last_change":"2023-11-28T03:11:18.764061+0000","last_active":"2023-11-28T03:11:18.759232+0000","last_peered":"2023-11-28T03:11:18.759232+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758596+0000","last_became_peered":"2023-11-28T03:11:18.758596+0000","last_unstale":"2023-11-28T03:11:18.759232+0000","last_undegraded":"2023-11-28T03:11:18.759232+0000","last_fullsized":"2023-11-28T03:11:18.759232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393196+0000","last_change":"2023-11-28T03:11:18.392016+0000","last_active":"2023-11-28T03:11:18.393196+0000","last_peered":"2023-11-28T03:11:18.393196+0000","last_clean":"2023-11-28T03:11:18.393196+0000","last_became_active":"2023-11-28T03:11:18.391714+0000","last_became_peered":"2023-11-28T03:11:18.391714+0000","last_unstale":"2023-11-28T03:11:18.393196+0000","last_undegraded":"2023-11-28T03:11:18.393196+0000","last_fullsized":"2023-11-28T03:11:18.393196+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755843+0000","last_change":"2023-11-28T03:11:18.755924+0000","last_active":"2023-11-28T03:11:18.755843+0000","last_peered":"2023-11-28T03:11:18.755843+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755604+0000","last_became_peered":"2023-11-28T03:11:18.755604+0000","last_unstale":"2023-11-28T03:11:18.755843+0000","last_undegraded":"2023-11-28T03:11:18.755843+0000","last_fullsized":"2023-11-28T03:11:18.755843+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400025+0000","last_change":"2023-11-28T03:11:18.396374+0000","last_active":"2023-11-28T03:11:18.400025+0000","last_peered":"2023-11-28T03:11:18.400025+0000","last_clean":"2023-11-28T03:11:18.400025+0000","last_became_active":"2023-11-28T03:11:18.396165+0000","last_became_peered":"2023-11-28T03:11:18.396165+0000","last_unstale":"2023-11-28T03:11:18.400025+0000","last_undegraded":"2023-11-28T03:11:18.400025+0000","last_fullsized":"2023-11-28T03:11:18.400025+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748189+0000","last_change":"2023-11-28T03:11:18.748320+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748189+0000","last_undegraded":"2023-11-28T03:11:18.748189+0000","last_fullsized":"2023-11-28T03:11:18.748189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.383966+0000","last_change":"2023-11-28T03:11:18.380395+0000","last_active":"2023-11-28T03:11:18.383966+0000","last_peered":"2023-11-28T03:11:18.383966+0000","last_clean":"2023-11-28T03:11:18.383966+0000","last_became_active":"2023-11-28T03:11:18.379748+0000","last_became_peered":"2023-11-28T03:11:18.379748+0000","last_unstale":"2023-11-28T03:11:18.383966+0000","last_undegraded":"2023-11-28T03:11:18.383966+0000","last_fullsized":"2023-11-28T03:11:18.383966+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746878+0000","last_change":"2023-11-28T03:11:18.747007+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746878+0000","last_undegraded":"2023-11-28T03:11:18.746878+0000","last_fullsized":"2023-11-28T03:11:18.746878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386571+0000","last_change":"2023-11-28T03:11:18.382167+0000","last_active":"2023-11-28T03:11:18.386571+0000","last_peered":"2023-11-28T03:11:18.386571+0000","last_clean":"2023-11-28T03:11:18.386571+0000","last_became_active":"2023-11-28T03:11:18.381842+0000","last_became_peered":"2023-11-28T03:11:18.381842+0000","last_unstale":"2023-11-28T03:11:18.386571+0000","last_undegraded":"2023-11-28T03:11:18.386571+0000","last_fullsized":"2023-11-28T03:11:18.386571+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766948+0000","last_change":"2023-11-28T03:11:18.767015+0000","last_active":"2023-11-28T03:11:18.766948+0000","last_peered":"2023-11-28T03:11:18.766948+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766723+0000","last_became_peered":"2023-11-28T03:11:18.766723+0000","last_unstale":"2023-11-28T03:11:18.766948+0000","last_undegraded":"2023-11-28T03:11:18.766948+0000","last_fullsized":"2023-11-28T03:11:18.766948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.401250+0000","last_change":"2023-11-28T03:11:18.400594+0000","last_active":"2023-11-28T03:11:18.401250+0000","last_peered":"2023-11-28T03:11:18.401250+0000","last_clean":"2023-11-28T03:11:18.401250+0000","last_became_active":"2023-11-28T03:11:18.400354+0000","last_became_peered":"2023-11-28T03:11:18.400354+0000","last_unstale":"2023-11-28T03:11:18.401250+0000","last_undegraded":"2023-11-28T03:11:18.401250+0000","last_fullsized":"2023-11-28T03:11:18.401250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755842+0000","last_change":"2023-11-28T03:11:18.755965+0000","last_active":"2023-11-28T03:11:18.755842+0000","last_peered":"2023-11-28T03:11:18.755842+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755630+0000","last_became_peered":"2023-11-28T03:11:18.755630+0000","last_unstale":"2023-11-28T03:11:18.755842+0000","last_undegraded":"2023-11-28T03:11:18.755842+0000","last_fullsized":"2023-11-28T03:11:18.755842+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390934+0000","last_change":"2023-11-28T03:11:18.387489+0000","last_active":"2023-11-28T03:11:18.390934+0000","last_peered":"2023-11-28T03:11:18.390934+0000","last_clean":"2023-11-28T03:11:18.390934+0000","last_became_active":"2023-11-28T03:11:18.386665+0000","last_became_peered":"2023-11-28T03:11:18.386665+0000","last_unstale":"2023-11-28T03:11:18.390934+0000","last_undegraded":"2023-11-28T03:11:18.390934+0000","last_fullsized":"2023-11-28T03:11:18.390934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755670+0000","last_change":"2023-11-28T03:11:18.755670+0000","last_active":"2023-11-28T03:11:18.755670+0000","last_peered":"2023-11-28T03:11:18.755670+0000","last_clean":"2023-11-28T03:11:18.755670+0000","last_became_active":"2023-11-28T03:11:18.755386+0000","last_became_peered":"2023-11-28T03:11:18.755386+0000","last_unstale":"2023-11-28T03:11:18.755670+0000","last_undegraded":"2023-11-28T03:11:18.755670+0000","last_fullsized":"2023-11-28T03:11:18.755670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393816+0000","last_change":"2023-11-28T03:11:18.391627+0000","last_active":"2023-11-28T03:11:18.393816+0000","last_peered":"2023-11-28T03:11:18.393816+0000","last_clean":"2023-11-28T03:11:18.393816+0000","last_became_active":"2023-11-28T03:11:18.391309+0000","last_became_peered":"2023-11-28T03:11:18.391309+0000","last_unstale":"2023-11-28T03:11:18.393816+0000","last_undegraded":"2023-11-28T03:11:18.393816+0000","last_fullsized":"2023-11-28T03:11:18.393816+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757097+0000","last_change":"2023-11-28T03:11:18.757255+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757097+0000","last_undegraded":"2023-11-28T03:11:18.757097+0000","last_fullsized":"2023-11-28T03:11:18.757097+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395478+0000","last_change":"2023-11-28T03:11:18.391689+0000","last_active":"2023-11-28T03:11:18.395478+0000","last_peered":"2023-11-28T03:11:18.395478+0000","last_clean":"2023-11-28T03:11:18.395478+0000","last_became_active":"2023-11-28T03:11:18.390946+0000","last_became_peered":"2023-11-28T03:11:18.390946+0000","last_unstale":"2023-11-28T03:11:18.395478+0000","last_undegraded":"2023-11-28T03:11:18.395478+0000","last_fullsized":"2023-11-28T03:11:18.395478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751684+0000","last_change":"2023-11-28T03:11:18.751840+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751684+0000","last_undegraded":"2023-11-28T03:11:18.751684+0000","last_fullsized":"2023-11-28T03:11:18.751684+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390577+0000","last_change":"2023-11-28T03:11:18.386639+0000","last_active":"2023-11-28T03:11:18.390577+0000","last_peered":"2023-11-28T03:11:18.390577+0000","last_clean":"2023-11-28T03:11:18.390577+0000","last_became_active":"2023-11-28T03:11:18.386342+0000","last_became_peered":"2023-11-28T03:11:18.386342+0000","last_unstale":"2023-11-28T03:11:18.390577+0000","last_undegraded":"2023-11-28T03:11:18.390577+0000","last_fullsized":"2023-11-28T03:11:18.390577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758112+0000","last_change":"2023-11-28T03:11:18.758266+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758112+0000","last_undegraded":"2023-11-28T03:11:18.758112+0000","last_fullsized":"2023-11-28T03:11:18.758112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399942+0000","last_change":"2023-11-28T03:11:18.395025+0000","last_active":"2023-11-28T03:11:18.399942+0000","last_peered":"2023-11-28T03:11:18.399942+0000","last_clean":"2023-11-28T03:11:18.399942+0000","last_became_active":"2023-11-28T03:11:18.394807+0000","last_became_peered":"2023-11-28T03:11:18.394807+0000","last_unstale":"2023-11-28T03:11:18.399942+0000","last_undegraded":"2023-11-28T03:11:18.399942+0000","last_fullsized":"2023-11-28T03:11:18.399942+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751381+0000","last_change":"2023-11-28T03:11:18.751504+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751381+0000","last_undegraded":"2023-11-28T03:11:18.751381+0000","last_fullsized":"2023-11-28T03:11:18.751381+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.384795+0000","last_change":"2023-11-28T03:11:18.380585+0000","last_active":"2023-11-28T03:11:18.384795+0000","last_peered":"2023-11-28T03:11:18.384795+0000","last_clean":"2023-11-28T03:11:18.384795+0000","last_became_active":"2023-11-28T03:11:18.380095+0000","last_became_peered":"2023-11-28T03:11:18.380095+0000","last_unstale":"2023-11-28T03:11:18.384795+0000","last_undegraded":"2023-11-28T03:11:18.384795+0000","last_fullsized":"2023-11-28T03:11:18.384795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746528+0000","last_change":"2023-11-28T03:11:18.746656+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746528+0000","last_undegraded":"2023-11-28T03:11:18.746528+0000","last_fullsized":"2023-11-28T03:11:18.746528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389225+0000","last_change":"2023-11-28T03:11:18.387312+0000","last_active":"2023-11-28T03:11:18.389225+0000","last_peered":"2023-11-28T03:11:18.389225+0000","last_clean":"2023-11-28T03:11:18.389225+0000","last_became_active":"2023-11-28T03:11:18.386958+0000","last_became_peered":"2023-11-28T03:11:18.386958+0000","last_unstale":"2023-11-28T03:11:18.389225+0000","last_undegraded":"2023-11-28T03:11:18.389225+0000","last_fullsized":"2023-11-28T03:11:18.389225+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749305+0000","last_change":"2023-11-28T03:11:18.749305+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749305+0000","last_undegraded":"2023-11-28T03:11:18.749305+0000","last_fullsized":"2023-11-28T03:11:18.749305+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389375+0000","last_change":"2023-11-28T03:11:18.386227+0000","last_active":"2023-11-28T03:11:18.389375+0000","last_peered":"2023-11-28T03:11:18.389375+0000","last_clean":"2023-11-28T03:11:18.389375+0000","last_became_active":"2023-11-28T03:11:18.385925+0000","last_became_peered":"2023-11-28T03:11:18.385925+0000","last_unstale":"2023-11-28T03:11:18.389375+0000","last_undegraded":"2023-11-28T03:11:18.389375+0000","last_fullsized":"2023-11-28T03:11:18.389375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757939+0000","last_change":"2023-11-28T03:11:18.757939+0000","last_active":"2023-11-28T03:11:18.757939+0000","last_peered":"2023-11-28T03:11:18.757939+0000","last_clean":"2023-11-28T03:11:18.757939+0000","last_became_active":"2023-11-28T03:11:18.756692+0000","last_became_peered":"2023-11-28T03:11:18.756692+0000","last_unstale":"2023-11-28T03:11:18.757939+0000","last_undegraded":"2023-11-28T03:11:18.757939+0000","last_fullsized":"2023-11-28T03:11:18.757939+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395297+0000","last_change":"2023-11-28T03:11:18.386046+0000","last_active":"2023-11-28T03:11:18.395297+0000","last_peered":"2023-11-28T03:11:18.395297+0000","last_clean":"2023-11-28T03:11:18.395297+0000","last_became_active":"2023-11-28T03:11:18.385393+0000","last_became_peered":"2023-11-28T03:11:18.385393+0000","last_unstale":"2023-11-28T03:11:18.395297+0000","last_undegraded":"2023-11-28T03:11:18.395297+0000","last_fullsized":"2023-11-28T03:11:18.395297+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755485+0000","last_change":"2023-11-28T03:11:18.755485+0000","last_active":"2023-11-28T03:11:18.755485+0000","last_peered":"2023-11-28T03:11:18.755485+0000","last_clean":"2023-11-28T03:11:18.755485+0000","last_became_active":"2023-11-28T03:11:18.755182+0000","last_became_peered":"2023-11-28T03:11:18.755182+0000","last_unstale":"2023-11-28T03:11:18.755485+0000","last_undegraded":"2023-11-28T03:11:18.755485+0000","last_fullsized":"2023-11-28T03:11:18.755485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397939+0000","last_change":"2023-11-28T03:11:18.396889+0000","last_active":"2023-11-28T03:11:18.397939+0000","last_peered":"2023-11-28T03:11:18.397939+0000","last_clean":"2023-11-28T03:11:18.397939+0000","last_became_active":"2023-11-28T03:11:18.396619+0000","last_became_peered":"2023-11-28T03:11:18.396619+0000","last_unstale":"2023-11-28T03:11:18.397939+0000","last_undegraded":"2023-11-28T03:11:18.397939+0000","last_fullsized":"2023-11-28T03:11:18.397939+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761321+0000","last_change":"2023-11-28T03:11:18.761438+0000","last_active":"2023-11-28T03:11:18.761321+0000","last_peered":"2023-11-28T03:11:18.761321+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760711+0000","last_became_peered":"2023-11-28T03:11:18.760711+0000","last_unstale":"2023-11-28T03:11:18.761321+0000","last_undegraded":"2023-11-28T03:11:18.761321+0000","last_fullsized":"2023-11-28T03:11:18.761321+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396985+0000","last_change":"2023-11-28T03:11:18.396051+0000","last_active":"2023-11-28T03:11:18.396985+0000","last_peered":"2023-11-28T03:11:18.396985+0000","last_clean":"2023-11-28T03:11:18.396985+0000","last_became_active":"2023-11-28T03:11:18.395777+0000","last_became_peered":"2023-11-28T03:11:18.395777+0000","last_unstale":"2023-11-28T03:11:18.396985+0000","last_undegraded":"2023-11-28T03:11:18.396985+0000","last_fullsized":"2023-11-28T03:11:18.396985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764649+0000","last_change":"2023-11-28T03:11:18.764724+0000","last_active":"2023-11-28T03:11:18.764649+0000","last_peered":"2023-11-28T03:11:18.764649+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.764373+0000","last_became_peered":"2023-11-28T03:11:18.764373+0000","last_unstale":"2023-11-28T03:11:18.764649+0000","last_undegraded":"2023-11-28T03:11:18.764649+0000","last_fullsized":"2023-11-28T03:11:18.764649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390603+0000","last_change":"2023-11-28T03:11:18.388277+0000","last_active":"2023-11-28T03:11:18.390603+0000","last_peered":"2023-11-28T03:11:18.390603+0000","last_clean":"2023-11-28T03:11:18.390603+0000","last_became_active":"2023-11-28T03:11:18.387116+0000","last_became_peered":"2023-11-28T03:11:18.387116+0000","last_unstale":"2023-11-28T03:11:18.390603+0000","last_undegraded":"2023-11-28T03:11:18.390603+0000","last_fullsized":"2023-11-28T03:11:18.390603+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749235+0000","last_change":"2023-11-28T03:11:18.749361+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749235+0000","last_undegraded":"2023-11-28T03:11:18.749235+0000","last_fullsized":"2023-11-28T03:11:18.749235+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390576+0000","last_change":"2023-11-28T03:11:18.388476+0000","last_active":"2023-11-28T03:11:18.390576+0000","last_peered":"2023-11-28T03:11:18.390576+0000","last_clean":"2023-11-28T03:11:18.390576+0000","last_became_active":"2023-11-28T03:11:18.387333+0000","last_became_peered":"2023-11-28T03:11:18.387333+0000","last_unstale":"2023-11-28T03:11:18.390576+0000","last_undegraded":"2023-11-28T03:11:18.390576+0000","last_fullsized":"2023-11-28T03:11:18.390576+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750948+0000","last_change":"2023-11-28T03:11:18.751069+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750948+0000","last_undegraded":"2023-11-28T03:11:18.750948+0000","last_fullsized":"2023-11-28T03:11:18.750948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396753+0000","last_change":"2023-11-28T03:11:18.395421+0000","last_active":"2023-11-28T03:11:18.396753+0000","last_peered":"2023-11-28T03:11:18.396753+0000","last_clean":"2023-11-28T03:11:18.396753+0000","last_became_active":"2023-11-28T03:11:18.394055+0000","last_became_peered":"2023-11-28T03:11:18.394055+0000","last_unstale":"2023-11-28T03:11:18.396753+0000","last_undegraded":"2023-11-28T03:11:18.396753+0000","last_fullsized":"2023-11-28T03:11:18.396753+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748206+0000","last_change":"2023-11-28T03:11:18.748338+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748206+0000","last_undegraded":"2023-11-28T03:11:18.748206+0000","last_fullsized":"2023-11-28T03:11:18.748206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388103+0000","last_change":"2023-11-28T03:11:18.378416+0000","last_active":"2023-11-28T03:11:18.388103+0000","last_peered":"2023-11-28T03:11:18.388103+0000","last_clean":"2023-11-28T03:11:18.388103+0000","last_became_active":"2023-11-28T03:11:18.378168+0000","last_became_peered":"2023-11-28T03:11:18.378168+0000","last_unstale":"2023-11-28T03:11:18.388103+0000","last_undegraded":"2023-11-28T03:11:18.388103+0000","last_fullsized":"2023-11-28T03:11:18.388103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760205+0000","last_change":"2023-11-28T03:11:18.760275+0000","last_active":"2023-11-28T03:11:18.760205+0000","last_peered":"2023-11-28T03:11:18.760205+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759999+0000","last_became_peered":"2023-11-28T03:11:18.759999+0000","last_unstale":"2023-11-28T03:11:18.760205+0000","last_undegraded":"2023-11-28T03:11:18.760205+0000","last_fullsized":"2023-11-28T03:11:18.760205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.402009+0000","last_change":"2023-11-28T03:11:18.400805+0000","last_active":"2023-11-28T03:11:18.402009+0000","last_peered":"2023-11-28T03:11:18.402009+0000","last_clean":"2023-11-28T03:11:18.402009+0000","last_became_active":"2023-11-28T03:11:18.400469+0000","last_became_peered":"2023-11-28T03:11:18.400469+0000","last_unstale":"2023-11-28T03:11:18.402009+0000","last_undegraded":"2023-11-28T03:11:18.402009+0000","last_fullsized":"2023-11-28T03:11:18.402009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761738+0000","last_change":"2023-11-28T03:11:18.761855+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761738+0000","last_undegraded":"2023-11-28T03:11:18.761738+0000","last_fullsized":"2023-11-28T03:11:18.761738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390532+0000","last_change":"2023-11-28T03:11:18.386676+0000","last_active":"2023-11-28T03:11:18.390532+0000","last_peered":"2023-11-28T03:11:18.390532+0000","last_clean":"2023-11-28T03:11:18.390532+0000","last_became_active":"2023-11-28T03:11:18.386406+0000","last_became_peered":"2023-11-28T03:11:18.386406+0000","last_unstale":"2023-11-28T03:11:18.390532+0000","last_undegraded":"2023-11-28T03:11:18.390532+0000","last_fullsized":"2023-11-28T03:11:18.390532+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756246+0000","last_change":"2023-11-28T03:11:18.756315+0000","last_active":"2023-11-28T03:11:18.756246+0000","last_peered":"2023-11-28T03:11:18.756246+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755546+0000","last_became_peered":"2023-11-28T03:11:18.755546+0000","last_unstale":"2023-11-28T03:11:18.756246+0000","last_undegraded":"2023-11-28T03:11:18.756246+0000","last_fullsized":"2023-11-28T03:11:18.756246+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393795+0000","last_change":"2023-11-28T03:11:18.390764+0000","last_active":"2023-11-28T03:11:18.393795+0000","last_peered":"2023-11-28T03:11:18.393795+0000","last_clean":"2023-11-28T03:11:18.393795+0000","last_became_active":"2023-11-28T03:11:18.390219+0000","last_became_peered":"2023-11-28T03:11:18.390219+0000","last_unstale":"2023-11-28T03:11:18.393795+0000","last_undegraded":"2023-11-28T03:11:18.393795+0000","last_fullsized":"2023-11-28T03:11:18.393795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763333+0000","last_change":"2023-11-28T03:11:18.763422+0000","last_active":"2023-11-28T03:11:18.763333+0000","last_peered":"2023-11-28T03:11:18.763333+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.763080+0000","last_became_peered":"2023-11-28T03:11:18.763080+0000","last_unstale":"2023-11-28T03:11:18.763333+0000","last_undegraded":"2023-11-28T03:11:18.763333+0000","last_fullsized":"2023-11-28T03:11:18.763333+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399936+0000","last_change":"2023-11-28T03:11:18.395919+0000","last_active":"2023-11-28T03:11:18.399936+0000","last_peered":"2023-11-28T03:11:18.399936+0000","last_clean":"2023-11-28T03:11:18.399936+0000","last_became_active":"2023-11-28T03:11:18.395621+0000","last_became_peered":"2023-11-28T03:11:18.395621+0000","last_unstale":"2023-11-28T03:11:18.399936+0000","last_undegraded":"2023-11-28T03:11:18.399936+0000","last_fullsized":"2023-11-28T03:11:18.399936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762081+0000","last_change":"2023-11-28T03:11:18.762163+0000","last_active":"2023-11-28T03:11:18.762081+0000","last_peered":"2023-11-28T03:11:18.762081+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759034+0000","last_became_peered":"2023-11-28T03:11:18.759034+0000","last_unstale":"2023-11-28T03:11:18.762081+0000","last_undegraded":"2023-11-28T03:11:18.762081+0000","last_fullsized":"2023-11-28T03:11:18.762081+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391545+0000","last_change":"2023-11-28T03:11:18.389034+0000","last_active":"2023-11-28T03:11:18.391545+0000","last_peered":"2023-11-28T03:11:18.391545+0000","last_clean":"2023-11-28T03:11:18.391545+0000","last_became_active":"2023-11-28T03:11:18.388710+0000","last_became_peered":"2023-11-28T03:11:18.388710+0000","last_unstale":"2023-11-28T03:11:18.391545+0000","last_undegraded":"2023-11-28T03:11:18.391545+0000","last_fullsized":"2023-11-28T03:11:18.391545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750502+0000","last_change":"2023-11-28T03:11:18.750640+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750502+0000","last_undegraded":"2023-11-28T03:11:18.750502+0000","last_fullsized":"2023-11-28T03:11:18.750502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394539+0000","last_change":"2023-11-28T03:11:18.391283+0000","last_active":"2023-11-28T03:11:18.394539+0000","last_peered":"2023-11-28T03:11:18.394539+0000","last_clean":"2023-11-28T03:11:18.394539+0000","last_became_active":"2023-11-28T03:11:18.390915+0000","last_became_peered":"2023-11-28T03:11:18.390915+0000","last_unstale":"2023-11-28T03:11:18.394539+0000","last_undegraded":"2023-11-28T03:11:18.394539+0000","last_fullsized":"2023-11-28T03:11:18.394539+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745744+0000","last_change":"2023-11-28T03:11:18.745874+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745744+0000","last_undegraded":"2023-11-28T03:11:18.745744+0000","last_fullsized":"2023-11-28T03:11:18.745744+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394088+0000","last_change":"2023-11-28T03:11:18.392679+0000","last_active":"2023-11-28T03:11:18.394088+0000","last_peered":"2023-11-28T03:11:18.394088+0000","last_clean":"2023-11-28T03:11:18.394088+0000","last_became_active":"2023-11-28T03:11:18.392366+0000","last_became_peered":"2023-11-28T03:11:18.392366+0000","last_unstale":"2023-11-28T03:11:18.394088+0000","last_undegraded":"2023-11-28T03:11:18.394088+0000","last_fullsized":"2023-11-28T03:11:18.394088+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751787+0000","last_change":"2023-11-28T03:11:18.751924+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751787+0000","last_undegraded":"2023-11-28T03:11:18.751787+0000","last_fullsized":"2023-11-28T03:11:18.751787+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387291+0000","last_change":"2023-11-28T03:11:18.385219+0000","last_active":"2023-11-28T03:11:18.387291+0000","last_peered":"2023-11-28T03:11:18.387291+0000","last_clean":"2023-11-28T03:11:18.387291+0000","last_became_active":"2023-11-28T03:11:18.384858+0000","last_became_peered":"2023-11-28T03:11:18.384858+0000","last_unstale":"2023-11-28T03:11:18.387291+0000","last_undegraded":"2023-11-28T03:11:18.387291+0000","last_fullsized":"2023-11-28T03:11:18.387291+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757565+0000","last_change":"2023-11-28T03:11:18.757640+0000","last_active":"2023-11-28T03:11:18.757565+0000","last_peered":"2023-11-28T03:11:18.757565+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757397+0000","last_became_peered":"2023-11-28T03:11:18.757397+0000","last_unstale":"2023-11-28T03:11:18.757565+0000","last_undegraded":"2023-11-28T03:11:18.757565+0000","last_fullsized":"2023-11-28T03:11:18.757565+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397201+0000","last_change":"2023-11-28T03:11:18.395814+0000","last_active":"2023-11-28T03:11:18.397201+0000","last_peered":"2023-11-28T03:11:18.397201+0000","last_clean":"2023-11-28T03:11:18.397201+0000","last_became_active":"2023-11-28T03:11:18.395516+0000","last_became_peered":"2023-11-28T03:11:18.395516+0000","last_unstale":"2023-11-28T03:11:18.397201+0000","last_undegraded":"2023-11-28T03:11:18.397201+0000","last_fullsized":"2023-11-28T03:11:18.397201+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745455+0000","last_change":"2023-11-28T03:11:18.745591+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745455+0000","last_undegraded":"2023-11-28T03:11:18.745455+0000","last_fullsized":"2023-11-28T03:11:18.745455+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397153+0000","last_change":"2023-11-28T03:11:18.395387+0000","last_active":"2023-11-28T03:11:18.397153+0000","last_peered":"2023-11-28T03:11:18.397153+0000","last_clean":"2023-11-28T03:11:18.397153+0000","last_became_active":"2023-11-28T03:11:18.394994+0000","last_became_peered":"2023-11-28T03:11:18.394994+0000","last_unstale":"2023-11-28T03:11:18.397153+0000","last_undegraded":"2023-11-28T03:11:18.397153+0000","last_fullsized":"2023-11-28T03:11:18.397153+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753511+0000","last_change":"2023-11-28T03:11:18.753635+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753511+0000","last_undegraded":"2023-11-28T03:11:18.753511+0000","last_fullsized":"2023-11-28T03:11:18.753511+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748519+0000","last_change":"2023-11-28T03:11:18.394301+0000","last_active":"2023-11-28T03:11:18.748519+0000","last_peered":"2023-11-28T03:11:18.748519+0000","last_clean":"2023-11-28T03:11:18.748519+0000","last_became_active":"2023-11-28T03:11:18.393966+0000","last_became_peered":"2023-11-28T03:11:18.393966+0000","last_unstale":"2023-11-28T03:11:18.748519+0000","last_undegraded":"2023-11-28T03:11:18.748519+0000","last_fullsized":"2023-11-28T03:11:18.748519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761932+0000","last_change":"2023-11-28T03:11:18.761997+0000","last_active":"2023-11-28T03:11:18.761932+0000","last_peered":"2023-11-28T03:11:18.761932+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761245+0000","last_became_peered":"2023-11-28T03:11:18.761245+0000","last_unstale":"2023-11-28T03:11:18.761932+0000","last_undegraded":"2023-11-28T03:11:18.761932+0000","last_fullsized":"2023-11-28T03:11:18.761932+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396237+0000","last_change":"2023-11-28T03:11:18.394666+0000","last_active":"2023-11-28T03:11:18.396237+0000","last_peered":"2023-11-28T03:11:18.396237+0000","last_clean":"2023-11-28T03:11:18.396237+0000","last_became_active":"2023-11-28T03:11:18.393583+0000","last_became_peered":"2023-11-28T03:11:18.393583+0000","last_unstale":"2023-11-28T03:11:18.396237+0000","last_undegraded":"2023-11-28T03:11:18.396237+0000","last_fullsized":"2023-11-28T03:11:18.396237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761889+0000","last_change":"2023-11-28T03:11:18.761969+0000","last_active":"2023-11-28T03:11:18.761889+0000","last_peered":"2023-11-28T03:11:18.761889+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758807+0000","last_became_peered":"2023-11-28T03:11:18.758807+0000","last_unstale":"2023-11-28T03:11:18.761889+0000","last_undegraded":"2023-11-28T03:11:18.761889+0000","last_fullsized":"2023-11-28T03:11:18.761889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395238+0000","last_change":"2023-11-28T03:11:18.384737+0000","last_active":"2023-11-28T03:11:18.395238+0000","last_peered":"2023-11-28T03:11:18.395238+0000","last_clean":"2023-11-28T03:11:18.395238+0000","last_became_active":"2023-11-28T03:11:18.384433+0000","last_became_peered":"2023-11-28T03:11:18.384433+0000","last_unstale":"2023-11-28T03:11:18.395238+0000","last_undegraded":"2023-11-28T03:11:18.395238+0000","last_fullsized":"2023-11-28T03:11:18.395238+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759794+0000","last_change":"2023-11-28T03:11:18.759866+0000","last_active":"2023-11-28T03:11:18.759794+0000","last_peered":"2023-11-28T03:11:18.759794+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759595+0000","last_became_peered":"2023-11-28T03:11:18.759595+0000","last_unstale":"2023-11-28T03:11:18.759794+0000","last_undegraded":"2023-11-28T03:11:18.759794+0000","last_fullsized":"2023-11-28T03:11:18.759794+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395339+0000","last_change":"2023-11-28T03:11:18.391939+0000","last_active":"2023-11-28T03:11:18.395339+0000","last_peered":"2023-11-28T03:11:18.395339+0000","last_clean":"2023-11-28T03:11:18.395339+0000","last_became_active":"2023-11-28T03:11:18.391661+0000","last_became_peered":"2023-11-28T03:11:18.391661+0000","last_unstale":"2023-11-28T03:11:18.395339+0000","last_undegraded":"2023-11-28T03:11:18.395339+0000","last_fullsized":"2023-11-28T03:11:18.395339+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.752002+0000","last_change":"2023-11-28T03:11:18.752124+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.752002+0000","last_undegraded":"2023-11-28T03:11:18.752002+0000","last_fullsized":"2023-11-28T03:11:18.752002+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392802+0000","last_change":"2023-11-28T03:11:18.390407+0000","last_active":"2023-11-28T03:11:18.392802+0000","last_peered":"2023-11-28T03:11:18.392802+0000","last_clean":"2023-11-28T03:11:18.392802+0000","last_became_active":"2023-11-28T03:11:18.390103+0000","last_became_peered":"2023-11-28T03:11:18.390103+0000","last_unstale":"2023-11-28T03:11:18.392802+0000","last_undegraded":"2023-11-28T03:11:18.392802+0000","last_fullsized":"2023-11-28T03:11:18.392802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756058+0000","last_change":"2023-11-28T03:11:18.756132+0000","last_active":"2023-11-28T03:11:18.756058+0000","last_peered":"2023-11-28T03:11:18.756058+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755858+0000","last_became_peered":"2023-11-28T03:11:18.755858+0000","last_unstale":"2023-11-28T03:11:18.756058+0000","last_undegraded":"2023-11-28T03:11:18.756058+0000","last_fullsized":"2023-11-28T03:11:18.756058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392637+0000","last_change":"2023-11-28T03:11:18.387271+0000","last_active":"2023-11-28T03:11:18.392637+0000","last_peered":"2023-11-28T03:11:18.392637+0000","last_clean":"2023-11-28T03:11:18.392637+0000","last_became_active":"2023-11-28T03:11:18.386973+0000","last_became_peered":"2023-11-28T03:11:18.386973+0000","last_unstale":"2023-11-28T03:11:18.392637+0000","last_undegraded":"2023-11-28T03:11:18.392637+0000","last_fullsized":"2023-11-28T03:11:18.392637+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.762229+0000","last_change":"2023-11-28T03:11:18.762360+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.762229+0000","last_undegraded":"2023-11-28T03:11:18.762229+0000","last_fullsized":"2023-11-28T03:11:18.762229+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392472+0000","last_change":"2023-11-28T03:11:18.387360+0000","last_active":"2023-11-28T03:11:18.392472+0000","last_peered":"2023-11-28T03:11:18.392472+0000","last_clean":"2023-11-28T03:11:18.392472+0000","last_became_active":"2023-11-28T03:11:18.386943+0000","last_became_peered":"2023-11-28T03:11:18.386943+0000","last_unstale":"2023-11-28T03:11:18.392472+0000","last_undegraded":"2023-11-28T03:11:18.392472+0000","last_fullsized":"2023-11-28T03:11:18.392472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.747957+0000","last_change":"2023-11-28T03:11:18.748091+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.747957+0000","last_undegraded":"2023-11-28T03:11:18.747957+0000","last_fullsized":"2023-11-28T03:11:18.747957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392710+0000","last_change":"2023-11-28T03:11:18.390261+0000","last_active":"2023-11-28T03:11:18.392710+0000","last_peered":"2023-11-28T03:11:18.392710+0000","last_clean":"2023-11-28T03:11:18.392710+0000","last_became_active":"2023-11-28T03:11:18.389960+0000","last_became_peered":"2023-11-28T03:11:18.389960+0000","last_unstale":"2023-11-28T03:11:18.392710+0000","last_undegraded":"2023-11-28T03:11:18.392710+0000","last_fullsized":"2023-11-28T03:11:18.392710+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753207+0000","last_change":"2023-11-28T03:11:18.753332+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753207+0000","last_undegraded":"2023-11-28T03:11:18.753207+0000","last_fullsized":"2023-11-28T03:11:18.753207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380619+0000","last_change":"2023-11-28T03:11:18.379037+0000","last_active":"2023-11-28T03:11:18.380619+0000","last_peered":"2023-11-28T03:11:18.380619+0000","last_clean":"2023-11-28T03:11:18.380619+0000","last_became_active":"2023-11-28T03:11:18.378709+0000","last_became_peered":"2023-11-28T03:11:18.378709+0000","last_unstale":"2023-11-28T03:11:18.380619+0000","last_undegraded":"2023-11-28T03:11:18.380619+0000","last_fullsized":"2023-11-28T03:11:18.380619+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751193+0000","last_change":"2023-11-28T03:11:18.751193+0000","last_active":"2023-11-28T03:11:18.751193+0000","last_peered":"2023-11-28T03:11:18.751193+0000","last_clean":"2023-11-28T03:11:18.751193+0000","last_became_active":"2023-11-28T03:11:18.750949+0000","last_became_peered":"2023-11-28T03:11:18.750949+0000","last_unstale":"2023-11-28T03:11:18.751193+0000","last_undegraded":"2023-11-28T03:11:18.751193+0000","last_fullsized":"2023-11-28T03:11:18.751193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390906+0000","last_change":"2023-11-28T03:11:18.388975+0000","last_active":"2023-11-28T03:11:18.390906+0000","last_peered":"2023-11-28T03:11:18.390906+0000","last_clean":"2023-11-28T03:11:18.390906+0000","last_became_active":"2023-11-28T03:11:18.388685+0000","last_became_peered":"2023-11-28T03:11:18.388685+0000","last_unstale":"2023-11-28T03:11:18.390906+0000","last_undegraded":"2023-11-28T03:11:18.390906+0000","last_fullsized":"2023-11-28T03:11:18.390906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757330+0000","last_change":"2023-11-28T03:11:18.757471+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757330+0000","last_undegraded":"2023-11-28T03:11:18.757330+0000","last_fullsized":"2023-11-28T03:11:18.757330+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392301+0000","last_change":"2023-11-28T03:11:18.390087+0000","last_active":"2023-11-28T03:11:18.392301+0000","last_peered":"2023-11-28T03:11:18.392301+0000","last_clean":"2023-11-28T03:11:18.392301+0000","last_became_active":"2023-11-28T03:11:18.389794+0000","last_became_peered":"2023-11-28T03:11:18.389794+0000","last_unstale":"2023-11-28T03:11:18.392301+0000","last_undegraded":"2023-11-28T03:11:18.392301+0000","last_fullsized":"2023-11-28T03:11:18.392301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750181+0000","last_change":"2023-11-28T03:11:18.750339+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750181+0000","last_undegraded":"2023-11-28T03:11:18.750181+0000","last_fullsized":"2023-11-28T03:11:18.750181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395320+0000","last_change":"2023-11-28T03:11:18.388300+0000","last_active":"2023-11-28T03:11:18.395320+0000","last_peered":"2023-11-28T03:11:18.395320+0000","last_clean":"2023-11-28T03:11:18.395320+0000","last_became_active":"2023-11-28T03:11:18.387929+0000","last_became_peered":"2023-11-28T03:11:18.387929+0000","last_unstale":"2023-11-28T03:11:18.395320+0000","last_undegraded":"2023-11-28T03:11:18.395320+0000","last_fullsized":"2023-11-28T03:11:18.395320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745039+0000","last_change":"2023-11-28T03:11:18.745192+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745039+0000","last_undegraded":"2023-11-28T03:11:18.745039+0000","last_fullsized":"2023-11-28T03:11:18.745039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388843+0000","last_change":"2023-11-28T03:11:18.384508+0000","last_active":"2023-11-28T03:11:18.388843+0000","last_peered":"2023-11-28T03:11:18.388843+0000","last_clean":"2023-11-28T03:11:18.388843+0000","last_became_active":"2023-11-28T03:11:18.384214+0000","last_became_peered":"2023-11-28T03:11:18.384214+0000","last_unstale":"2023-11-28T03:11:18.388843+0000","last_undegraded":"2023-11-28T03:11:18.388843+0000","last_fullsized":"2023-11-28T03:11:18.388843+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373322+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:11:18.373322+0000","last_peered":"2023-11-28T03:11:18.373322+0000","last_clean":"2023-11-28T03:11:18.373322+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:11:18.373322+0000","last_undegraded":"2023-11-28T03:11:18.373322+0000","last_fullsized":"2023-11-28T03:11:18.373322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750870+0000","last_change":"2023-11-28T03:11:18.750870+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750870+0000","last_undegraded":"2023-11-28T03:11:18.750870+0000","last_fullsized":"2023-11-28T03:11:18.750870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395340+0000","last_change":"2023-11-28T03:11:18.387081+0000","last_active":"2023-11-28T03:11:18.395340+0000","last_peered":"2023-11-28T03:11:18.395340+0000","last_clean":"2023-11-28T03:11:18.395340+0000","last_became_active":"2023-11-28T03:11:18.386827+0000","last_became_peered":"2023-11-28T03:11:18.386827+0000","last_unstale":"2023-11-28T03:11:18.395340+0000","last_undegraded":"2023-11-28T03:11:18.395340+0000","last_fullsized":"2023-11-28T03:11:18.395340+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.377275+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:11:18.377275+0000","last_peered":"2023-11-28T03:11:18.377275+0000","last_clean":"2023-11-28T03:11:18.377275+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:11:18.377275+0000","last_undegraded":"2023-11-28T03:11:18.377275+0000","last_fullsized":"2023-11-28T03:11:18.377275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749941+0000","last_change":"2023-11-28T03:11:18.750076+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749941+0000","last_undegraded":"2023-11-28T03:11:18.749941+0000","last_fullsized":"2023-11-28T03:11:18.749941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395335+0000","last_change":"2023-11-28T03:11:18.386867+0000","last_active":"2023-11-28T03:11:18.395335+0000","last_peered":"2023-11-28T03:11:18.395335+0000","last_clean":"2023-11-28T03:11:18.395335+0000","last_became_active":"2023-11-28T03:11:18.386623+0000","last_became_peered":"2023-11-28T03:11:18.386623+0000","last_unstale":"2023-11-28T03:11:18.395335+0000","last_undegraded":"2023-11-28T03:11:18.395335+0000","last_fullsized":"2023-11-28T03:11:18.395335+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.370453+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:11:18.370453+0000","last_peered":"2023-11-28T03:11:18.370453+0000","last_clean":"2023-11-28T03:11:18.370453+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:11:18.370453+0000","last_undegraded":"2023-11-28T03:11:18.370453+0000","last_fullsized":"2023-11-28T03:11:18.370453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758040+0000","last_change":"2023-11-28T03:11:18.758171+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758040+0000","last_undegraded":"2023-11-28T03:11:18.758040+0000","last_fullsized":"2023-11-28T03:11:18.758040+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396967+0000","last_change":"2023-11-28T03:11:18.394496+0000","last_active":"2023-11-28T03:11:18.396967+0000","last_peered":"2023-11-28T03:11:18.396967+0000","last_clean":"2023-11-28T03:11:18.396967+0000","last_became_active":"2023-11-28T03:11:18.394294+0000","last_became_peered":"2023-11-28T03:11:18.394294+0000","last_unstale":"2023-11-28T03:11:18.396967+0000","last_undegraded":"2023-11-28T03:11:18.396967+0000","last_fullsized":"2023-11-28T03:11:18.396967+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.382976+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:11:18.382976+0000","last_peered":"2023-11-28T03:11:18.382976+0000","last_clean":"2023-11-28T03:11:18.382976+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:11:18.382976+0000","last_undegraded":"2023-11-28T03:11:18.382976+0000","last_fullsized":"2023-11-28T03:11:18.382976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380178+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:11:18.380178+0000","last_peered":"2023-11-28T03:11:18.380178+0000","last_clean":"2023-11-28T03:11:18.380178+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:11:18.380178+0000","last_undegraded":"2023-11-28T03:11:18.380178+0000","last_fullsized":"2023-11-28T03:11:18.380178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760444+0000","last_change":"2023-11-28T03:11:18.760540+0000","last_active":"2023-11-28T03:11:18.760444+0000","last_peered":"2023-11-28T03:11:18.760444+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760079+0000","last_became_peered":"2023-11-28T03:11:18.760079+0000","last_unstale":"2023-11-28T03:11:18.760444+0000","last_undegraded":"2023-11-28T03:11:18.760444+0000","last_fullsized":"2023-11-28T03:11:18.760444+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400010+0000","last_change":"2023-11-28T03:11:18.396476+0000","last_active":"2023-11-28T03:11:18.400010+0000","last_peered":"2023-11-28T03:11:18.400010+0000","last_clean":"2023-11-28T03:11:18.400010+0000","last_became_active":"2023-11-28T03:11:18.396309+0000","last_became_peered":"2023-11-28T03:11:18.396309+0000","last_unstale":"2023-11-28T03:11:18.400010+0000","last_undegraded":"2023-11-28T03:11:18.400010+0000","last_fullsized":"2023-11-28T03:11:18.400010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.381718+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:11:18.381718+0000","last_peered":"2023-11-28T03:11:18.381718+0000","last_clean":"2023-11-28T03:11:18.381718+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:11:18.381718+0000","last_undegraded":"2023-11-28T03:11:18.381718+0000","last_fullsized":"2023-11-28T03:11:18.381718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.744775+0000","last_change":"2023-11-28T03:11:18.744919+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.744775+0000","last_undegraded":"2023-11-28T03:11:18.744775+0000","last_fullsized":"2023-11-28T03:11:18.744775+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389393+0000","last_change":"2023-11-28T03:11:18.387795+0000","last_active":"2023-11-28T03:11:18.389393+0000","last_peered":"2023-11-28T03:11:18.389393+0000","last_clean":"2023-11-28T03:11:18.389393+0000","last_became_active":"2023-11-28T03:11:18.387481+0000","last_became_peered":"2023-11-28T03:11:18.387481+0000","last_unstale":"2023-11-28T03:11:18.389393+0000","last_undegraded":"2023-11-28T03:11:18.389393+0000","last_fullsized":"2023-11-28T03:11:18.389393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392461+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:11:18.392461+0000","last_peered":"2023-11-28T03:11:18.392461+0000","last_clean":"2023-11-28T03:11:18.392461+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:11:18.392461+0000","last_undegraded":"2023-11-28T03:11:18.392461+0000","last_fullsized":"2023-11-28T03:11:18.392461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757177+0000","last_change":"2023-11-28T03:11:18.757177+0000","last_active":"2023-11-28T03:11:18.757177+0000","last_peered":"2023-11-28T03:11:18.757177+0000","last_clean":"2023-11-28T03:11:18.757177+0000","last_became_active":"2023-11-28T03:11:18.756875+0000","last_became_peered":"2023-11-28T03:11:18.756875+0000","last_unstale":"2023-11-28T03:11:18.757177+0000","last_undegraded":"2023-11-28T03:11:18.757177+0000","last_fullsized":"2023-11-28T03:11:18.757177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755179+0000","last_change":"2023-11-28T03:11:18.392352+0000","last_active":"2023-11-28T03:11:18.755179+0000","last_peered":"2023-11-28T03:11:18.755179+0000","last_clean":"2023-11-28T03:11:18.755179+0000","last_became_active":"2023-11-28T03:11:18.392050+0000","last_became_peered":"2023-11-28T03:11:18.392050+0000","last_unstale":"2023-11-28T03:11:18.755179+0000","last_undegraded":"2023-11-28T03:11:18.755179+0000","last_fullsized":"2023-11-28T03:11:18.755179+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373107+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:11:18.373107+0000","last_peered":"2023-11-28T03:11:18.373107+0000","last_clean":"2023-11-28T03:11:18.373107+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:11:18.373107+0000","last_undegraded":"2023-11-28T03:11:18.373107+0000","last_fullsized":"2023-11-28T03:11:18.373107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761405+0000","last_change":"2023-11-28T03:11:18.761490+0000","last_active":"2023-11-28T03:11:18.761405+0000","last_peered":"2023-11-28T03:11:18.761405+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761053+0000","last_became_peered":"2023-11-28T03:11:18.761053+0000","last_unstale":"2023-11-28T03:11:18.761405+0000","last_undegraded":"2023-11-28T03:11:18.761405+0000","last_fullsized":"2023-11-28T03:11:18.761405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395315+0000","last_change":"2023-11-28T03:11:18.386215+0000","last_active":"2023-11-28T03:11:18.395315+0000","last_peered":"2023-11-28T03:11:18.395315+0000","last_clean":"2023-11-28T03:11:18.395315+0000","last_became_active":"2023-11-28T03:11:18.385765+0000","last_became_peered":"2023-11-28T03:11:18.385765+0000","last_unstale":"2023-11-28T03:11:18.395315+0000","last_undegraded":"2023-11-28T03:11:18.395315+0000","last_fullsized":"2023-11-28T03:11:18.395315+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.378564+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:11:18.378564+0000","last_peered":"2023-11-28T03:11:18.378564+0000","last_clean":"2023-11-28T03:11:18.378564+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:11:18.378564+0000","last_undegraded":"2023-11-28T03:11:18.378564+0000","last_fullsized":"2023-11-28T03:11:18.378564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69899999999999995,"5min":1.46,"15min":1.46},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69799999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60999999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":0.64400000000000002,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.76900000000000002,"15min":0.84499999999999997},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.73699999999999999,"15min":1.4810000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.94399999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":1.125,"15min":1.506},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93000000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.90900000000000003},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.441,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.627,"5min":0.78100000000000003,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":0.59699999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.874,"15min":0.95099999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.4490000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.91100000000000003,"15min":1.3799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.90400000000000003,"15min":1.1040000000000001},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.67800000000000005},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.218,"15min":0.20899999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.67100000000000004},"last":0.22}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.21199999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.25800000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.45600000000000002},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41399999999999998,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97799999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.99299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.038},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.94099999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83199999999999996,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.024},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88300000000000001},"last":0.373}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.56599999999999995},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34399999999999997,"15min":0.36499999999999999},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53700000000000003,"5min":0.58999999999999997,"15min":0.70299999999999996},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33900000000000002,"15min":0.36299999999999999},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.439},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.73299999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.371,"15min":0.39300000000000002},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47999999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.41899999999999998},"min":{"1min":0.318,"5min":0.217,"15min":0.217},"max":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.65400000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79500000000000004,"5min":1.427,"15min":1.427},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.628},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.747,"5min":0.85899999999999999,"15min":1.2869999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.629},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":1.304},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":1.456,"15min":1.456},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.4399999999999999,"15min":1.679},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59099999999999997,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80700000000000005,"5min":1.341,"15min":1.341},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":1.474,"15min":1.474},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.315},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.39700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.19800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.67400000000000004},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.627,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.25900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.35799999999999998},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.98599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":1.004},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.91500000000000004,"15min":0.93300000000000005},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20799999999999999},"max":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.59899999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68899999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.74199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.34799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.878,"15min":1.581},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94199999999999995},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.35699999999999998},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.97899999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.93100000000000005,"15min":0.996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38,"15min":0.376},"max":{"1min":0.87,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60499999999999998,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.878},"last":0.621}]},{"osd":5,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80000000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.27900000000000003,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.28000000000000003},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.78300000000000003,"15min":0.83299999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.82499999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.747,"15min":0.80500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.81499999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.94399999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63700000000000001,"5min":0.77800000000000002,"15min":0.89900000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.91200000000000003},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:11:22.114 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:11:22.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.227+0000 7f7366ec3700 1 Processor -- start 2023-11-28T03:11:22.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.227+0000 7f7366ec3700 1 -- start start 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7360157660 0x7f7360157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 0x7f73601589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73600c1750 0x7f73600c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73600c12c0 con 0x7f73600c1750 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73600c06b0 con 0x7f73600c0180 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7366ec3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600c0990 con 0x7f7360157660 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73600c1750 0x7f73600c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73600c1750 0x7f73600c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41622/0 (socket says 172.21.15.73:41622) 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 -- 172.21.15.73:0/858502405 learned_addr learned my addr 172.21.15.73:0/858502405 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73656c0700 1 --2- 172.21.15.73:0/858502405 >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 0x7f73601589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f7365ec1700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7360157660 0x7f7360157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 -- 172.21.15.73:0/858502405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7360157660 msgr2=0x7f7360157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7360157660 0x7f7360157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 -- 172.21.15.73:0/858502405 >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 msgr2=0x7f73601589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 --2- 172.21.15.73:0/858502405 >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 0x7f73601589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73666c2700 1 -- 172.21.15.73:0/858502405 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73600c0c70 con 0x7f73600c1750 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.228+0000 7f73656c0700 1 --2- 172.21.15.73:0/858502405 >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 0x7f73601589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f73666c2700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c1750 0x7f73600c1bc0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f7360084330 tx=0x7f735c0090f0).ready entity=mon.0 client_cookie=5e7d4e2eef3c0e1b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7364ebf700 1 -- 172.21.15.73:0/858502405 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00b520 con 0x7f73600c1750 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7364ebf700 1 -- 172.21.15.73:0/858502405 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735c00b680 con 0x7f73600c1750 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7364ebf700 1 -- 172.21.15.73:0/858502405 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735c00ba00 con 0x7f73600c1750 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 -- 172.21.15.73:0/858502405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c1750 msgr2=0x7f73600c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c1750 0x7f73600c1bc0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 rx=0x7f7360084330 tx=0x7f735c0090f0).stop 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 -- 172.21.15.73:0/858502405 shutdown_connections 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c1750 0x7f73600c1bc0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 --2- 172.21.15.73:0/858502405 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7360157660 0x7f7360157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 --2- 172.21.15.73:0/858502405 >> v2:172.21.15.150:3300/0 conn(0x7f73600c0180 0x7f73601589c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 -- 172.21.15.73:0/858502405 >> 172.21.15.73:0/858502405 conn(0x7f73600cb000 msgr2=0x7f73600cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 -- 172.21.15.73:0/858502405 shutdown_connections 2023-11-28T03:11:22.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.229+0000 7f7366ec3700 1 -- 172.21.15.73:0/858502405 wait complete. 2023-11-28T03:11:22.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.230+0000 7f7366ec3700 1 Processor -- start 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 -- start start 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600c1750 0x7f73600adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 0x7f73600afb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600af180 con 0x7f73600c0180 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73600c3070 con 0x7f7360157660 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7366ec3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73600ae8e0 con 0x7f73600c1750 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7365ec1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7365ec1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41632/0 (socket says 172.21.15.73:41632) 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.231+0000 7f7365ec1700 1 -- 172.21.15.73:0/1721431433 learned_addr learned my addr 172.21.15.73:0/1721431433 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:22.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f73656c0700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600c1750 0x7f73600adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f73666c2700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 0x7f73600afb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 -- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600c1750 msgr2=0x7f73600adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600c1750 0x7f73600adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 -- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 msgr2=0x7f73600afb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 0x7f73600afb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 -- 172.21.15.73:0/1721431433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f735c009370 con 0x7f73600c0180 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f73666c2700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 0x7f73600afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7365ec1700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f7358009e70 tx=0x7f735800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f735800cbd0 con 0x7f73600c0180 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f735800f040 con 0x7f73600c0180 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73600aebc0 con 0x7f73600c0180 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7358013400 con 0x7f73600c0180 2023-11-28T03:11:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.232+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73600aa940 con 0x7f73600c0180 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.233+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f736006c7c0 con 0x7f73600c0180 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.233+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f73580137b0 con 0x7f73600c0180 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.234+0000 7f7355ffb700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 0x7f7344061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.234+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f735807ab10 con 0x7f73600c0180 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.234+0000 7f73656c0700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 0x7f7344061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.234+0000 7f73656c0700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 0x7f7344061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7350006d80 tx=0x7f7350008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.238+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f735804be80 con 0x7f73600c0180 2023-11-28T03:11:22.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.400+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f73600c0f50 con 0x7f734405f150 2023-11-28T03:11:22.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.409+0000 7f7355ffb700 1 -- 172.21.15.73:0/1721431433 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360699 (secure 0 0 0) 0x7f73600c0f50 con 0x7f734405f150 2023-11-28T03:11:22.414 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 msgr2=0x7f7344061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 0x7f7344061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f7350006d80 tx=0x7f7350008040).stop 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 msgr2=0x7f73600aa450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 rx=0x7f7358009e70 tx=0x7f735800b040).stop 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 shutdown_connections 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f734405f150 0x7f7344061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7360157660 0x7f73600afb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73600c0180 0x7f73600aa450 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 --2- 172.21.15.73:0/1721431433 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73600c1750 0x7f73600adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 >> 172.21.15.73:0/1721431433 conn(0x7f73600cb000 msgr2=0x7f73600ad090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 shutdown_connections 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.412+0000 7f7366ec3700 1 -- 172.21.15.73:0/1721431433 wait complete. 2023-11-28T03:11:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:11:22.425 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:11:21.763486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426948,"kb_used_data":13316,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752547772,"statfs":{"total":773094113280,"available":770608918528,"internally_reserved":0,"allocated":13635584,"data_stored":6798157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19375,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.586494"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757468+0000","last_change":"2023-11-28T03:11:18.757597+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757468+0000","last_undegraded":"2023-11-28T03:11:18.757468+0000","last_fullsized":"2023-11-28T03:11:18.757468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388461+0000","last_change":"2023-11-28T03:11:18.385568+0000","last_active":"2023-11-28T03:11:18.388461+0000","last_peered":"2023-11-28T03:11:18.388461+0000","last_clean":"2023-11-28T03:11:18.388461+0000","last_became_active":"2023-11-28T03:11:18.385211+0000","last_became_peered":"2023-11-28T03:11:18.385211+0000","last_unstale":"2023-11-28T03:11:18.388461+0000","last_undegraded":"2023-11-28T03:11:18.388461+0000","last_fullsized":"2023-11-28T03:11:18.388461+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759645+0000","last_change":"2023-11-28T03:11:18.759718+0000","last_active":"2023-11-28T03:11:18.759645+0000","last_peered":"2023-11-28T03:11:18.759645+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759448+0000","last_became_peered":"2023-11-28T03:11:18.759448+0000","last_unstale":"2023-11-28T03:11:18.759645+0000","last_undegraded":"2023-11-28T03:11:18.759645+0000","last_fullsized":"2023-11-28T03:11:18.759645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391029+0000","last_change":"2023-11-28T03:11:18.389741+0000","last_active":"2023-11-28T03:11:18.391029+0000","last_peered":"2023-11-28T03:11:18.391029+0000","last_clean":"2023-11-28T03:11:18.391029+0000","last_became_active":"2023-11-28T03:11:18.389440+0000","last_became_peered":"2023-11-28T03:11:18.389440+0000","last_unstale":"2023-11-28T03:11:18.391029+0000","last_undegraded":"2023-11-28T03:11:18.391029+0000","last_fullsized":"2023-11-28T03:11:18.391029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768741+0000","last_change":"2023-11-28T03:11:18.768823+0000","last_active":"2023-11-28T03:11:18.768741+0000","last_peered":"2023-11-28T03:11:18.768741+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766758+0000","last_became_peered":"2023-11-28T03:11:18.766758+0000","last_unstale":"2023-11-28T03:11:18.768741+0000","last_undegraded":"2023-11-28T03:11:18.768741+0000","last_fullsized":"2023-11-28T03:11:18.768741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397054+0000","last_change":"2023-11-28T03:11:18.394730+0000","last_active":"2023-11-28T03:11:18.397054+0000","last_peered":"2023-11-28T03:11:18.397054+0000","last_clean":"2023-11-28T03:11:18.397054+0000","last_became_active":"2023-11-28T03:11:18.394473+0000","last_became_peered":"2023-11-28T03:11:18.394473+0000","last_unstale":"2023-11-28T03:11:18.397054+0000","last_undegraded":"2023-11-28T03:11:18.397054+0000","last_fullsized":"2023-11-28T03:11:18.397054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759861+0000","last_change":"2023-11-28T03:11:18.759965+0000","last_active":"2023-11-28T03:11:18.759861+0000","last_peered":"2023-11-28T03:11:18.759861+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759675+0000","last_became_peered":"2023-11-28T03:11:18.759675+0000","last_unstale":"2023-11-28T03:11:18.759861+0000","last_undegraded":"2023-11-28T03:11:18.759861+0000","last_fullsized":"2023-11-28T03:11:18.759861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392731+0000","last_change":"2023-11-28T03:11:18.390367+0000","last_active":"2023-11-28T03:11:18.392731+0000","last_peered":"2023-11-28T03:11:18.392731+0000","last_clean":"2023-11-28T03:11:18.392731+0000","last_became_active":"2023-11-28T03:11:18.390086+0000","last_became_peered":"2023-11-28T03:11:18.390086+0000","last_unstale":"2023-11-28T03:11:18.392731+0000","last_undegraded":"2023-11-28T03:11:18.392731+0000","last_fullsized":"2023-11-28T03:11:18.392731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757310+0000","last_change":"2023-11-28T03:11:18.757382+0000","last_active":"2023-11-28T03:11:18.757310+0000","last_peered":"2023-11-28T03:11:18.757310+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.756962+0000","last_became_peered":"2023-11-28T03:11:18.756962+0000","last_unstale":"2023-11-28T03:11:18.757310+0000","last_undegraded":"2023-11-28T03:11:18.757310+0000","last_fullsized":"2023-11-28T03:11:18.757310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387008+0000","last_change":"2023-11-28T03:11:18.385649+0000","last_active":"2023-11-28T03:11:18.387008+0000","last_peered":"2023-11-28T03:11:18.387008+0000","last_clean":"2023-11-28T03:11:18.387008+0000","last_became_active":"2023-11-28T03:11:18.385386+0000","last_became_peered":"2023-11-28T03:11:18.385386+0000","last_unstale":"2023-11-28T03:11:18.387008+0000","last_undegraded":"2023-11-28T03:11:18.387008+0000","last_fullsized":"2023-11-28T03:11:18.387008+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748324+0000","last_change":"2023-11-28T03:11:18.748460+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748324+0000","last_undegraded":"2023-11-28T03:11:18.748324+0000","last_fullsized":"2023-11-28T03:11:18.748324+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394106+0000","last_change":"2023-11-28T03:11:18.392909+0000","last_active":"2023-11-28T03:11:18.394106+0000","last_peered":"2023-11-28T03:11:18.394106+0000","last_clean":"2023-11-28T03:11:18.394106+0000","last_became_active":"2023-11-28T03:11:18.392552+0000","last_became_peered":"2023-11-28T03:11:18.392552+0000","last_unstale":"2023-11-28T03:11:18.394106+0000","last_undegraded":"2023-11-28T03:11:18.394106+0000","last_fullsized":"2023-11-28T03:11:18.394106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748559+0000","last_change":"2023-11-28T03:11:18.748669+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748559+0000","last_undegraded":"2023-11-28T03:11:18.748559+0000","last_fullsized":"2023-11-28T03:11:18.748559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392365+0000","last_change":"2023-11-28T03:11:18.389689+0000","last_active":"2023-11-28T03:11:18.392365+0000","last_peered":"2023-11-28T03:11:18.392365+0000","last_clean":"2023-11-28T03:11:18.392365+0000","last_became_active":"2023-11-28T03:11:18.388755+0000","last_became_peered":"2023-11-28T03:11:18.388755+0000","last_unstale":"2023-11-28T03:11:18.392365+0000","last_undegraded":"2023-11-28T03:11:18.392365+0000","last_fullsized":"2023-11-28T03:11:18.392365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757626+0000","last_change":"2023-11-28T03:11:18.757695+0000","last_active":"2023-11-28T03:11:18.757626+0000","last_peered":"2023-11-28T03:11:18.757626+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757427+0000","last_became_peered":"2023-11-28T03:11:18.757427+0000","last_unstale":"2023-11-28T03:11:18.757626+0000","last_undegraded":"2023-11-28T03:11:18.757626+0000","last_fullsized":"2023-11-28T03:11:18.757626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395299+0000","last_change":"2023-11-28T03:11:18.391234+0000","last_active":"2023-11-28T03:11:18.395299+0000","last_peered":"2023-11-28T03:11:18.395299+0000","last_clean":"2023-11-28T03:11:18.395299+0000","last_became_active":"2023-11-28T03:11:18.390855+0000","last_became_peered":"2023-11-28T03:11:18.390855+0000","last_unstale":"2023-11-28T03:11:18.395299+0000","last_undegraded":"2023-11-28T03:11:18.395299+0000","last_fullsized":"2023-11-28T03:11:18.395299+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751698+0000","last_change":"2023-11-28T03:11:18.751811+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751698+0000","last_undegraded":"2023-11-28T03:11:18.751698+0000","last_fullsized":"2023-11-28T03:11:18.751698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389124+0000","last_change":"2023-11-28T03:11:18.385956+0000","last_active":"2023-11-28T03:11:18.389124+0000","last_peered":"2023-11-28T03:11:18.389124+0000","last_clean":"2023-11-28T03:11:18.389124+0000","last_became_active":"2023-11-28T03:11:18.385634+0000","last_became_peered":"2023-11-28T03:11:18.385634+0000","last_unstale":"2023-11-28T03:11:18.389124+0000","last_undegraded":"2023-11-28T03:11:18.389124+0000","last_fullsized":"2023-11-28T03:11:18.389124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761439+0000","last_change":"2023-11-28T03:11:18.761511+0000","last_active":"2023-11-28T03:11:18.761439+0000","last_peered":"2023-11-28T03:11:18.761439+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761234+0000","last_became_peered":"2023-11-28T03:11:18.761234+0000","last_unstale":"2023-11-28T03:11:18.761439+0000","last_undegraded":"2023-11-28T03:11:18.761439+0000","last_fullsized":"2023-11-28T03:11:18.761439+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386815+0000","last_change":"2023-11-28T03:11:18.384000+0000","last_active":"2023-11-28T03:11:18.386815+0000","last_peered":"2023-11-28T03:11:18.386815+0000","last_clean":"2023-11-28T03:11:18.386815+0000","last_became_active":"2023-11-28T03:11:18.383737+0000","last_became_peered":"2023-11-28T03:11:18.383737+0000","last_unstale":"2023-11-28T03:11:18.386815+0000","last_undegraded":"2023-11-28T03:11:18.386815+0000","last_fullsized":"2023-11-28T03:11:18.386815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749608+0000","last_change":"2023-11-28T03:11:18.749746+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749608+0000","last_undegraded":"2023-11-28T03:11:18.749608+0000","last_fullsized":"2023-11-28T03:11:18.749608+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390548+0000","last_change":"2023-11-28T03:11:18.386918+0000","last_active":"2023-11-28T03:11:18.390548+0000","last_peered":"2023-11-28T03:11:18.390548+0000","last_clean":"2023-11-28T03:11:18.390548+0000","last_became_active":"2023-11-28T03:11:18.386386+0000","last_became_peered":"2023-11-28T03:11:18.386386+0000","last_unstale":"2023-11-28T03:11:18.390548+0000","last_undegraded":"2023-11-28T03:11:18.390548+0000","last_fullsized":"2023-11-28T03:11:18.390548+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761388+0000","last_change":"2023-11-28T03:11:18.761551+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761388+0000","last_undegraded":"2023-11-28T03:11:18.761388+0000","last_fullsized":"2023-11-28T03:11:18.761388+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386802+0000","last_change":"2023-11-28T03:11:18.384007+0000","last_active":"2023-11-28T03:11:18.386802+0000","last_peered":"2023-11-28T03:11:18.386802+0000","last_clean":"2023-11-28T03:11:18.386802+0000","last_became_active":"2023-11-28T03:11:18.383671+0000","last_became_peered":"2023-11-28T03:11:18.383671+0000","last_unstale":"2023-11-28T03:11:18.386802+0000","last_undegraded":"2023-11-28T03:11:18.386802+0000","last_fullsized":"2023-11-28T03:11:18.386802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753548+0000","last_change":"2023-11-28T03:11:18.753676+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753548+0000","last_undegraded":"2023-11-28T03:11:18.753548+0000","last_fullsized":"2023-11-28T03:11:18.753548+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395509+0000","last_change":"2023-11-28T03:11:18.391315+0000","last_active":"2023-11-28T03:11:18.395509+0000","last_peered":"2023-11-28T03:11:18.395509+0000","last_clean":"2023-11-28T03:11:18.395509+0000","last_became_active":"2023-11-28T03:11:18.391053+0000","last_became_peered":"2023-11-28T03:11:18.391053+0000","last_unstale":"2023-11-28T03:11:18.395509+0000","last_undegraded":"2023-11-28T03:11:18.395509+0000","last_fullsized":"2023-11-28T03:11:18.395509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756085+0000","last_change":"2023-11-28T03:11:18.756179+0000","last_active":"2023-11-28T03:11:18.756085+0000","last_peered":"2023-11-28T03:11:18.756085+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755853+0000","last_became_peered":"2023-11-28T03:11:18.755853+0000","last_unstale":"2023-11-28T03:11:18.756085+0000","last_undegraded":"2023-11-28T03:11:18.756085+0000","last_fullsized":"2023-11-28T03:11:18.756085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395277+0000","last_change":"2023-11-28T03:11:18.391280+0000","last_active":"2023-11-28T03:11:18.395277+0000","last_peered":"2023-11-28T03:11:18.395277+0000","last_clean":"2023-11-28T03:11:18.395277+0000","last_became_active":"2023-11-28T03:11:18.387894+0000","last_became_peered":"2023-11-28T03:11:18.387894+0000","last_unstale":"2023-11-28T03:11:18.395277+0000","last_undegraded":"2023-11-28T03:11:18.395277+0000","last_fullsized":"2023-11-28T03:11:18.395277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759232+0000","last_change":"2023-11-28T03:11:18.764061+0000","last_active":"2023-11-28T03:11:18.759232+0000","last_peered":"2023-11-28T03:11:18.759232+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758596+0000","last_became_peered":"2023-11-28T03:11:18.758596+0000","last_unstale":"2023-11-28T03:11:18.759232+0000","last_undegraded":"2023-11-28T03:11:18.759232+0000","last_fullsized":"2023-11-28T03:11:18.759232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393196+0000","last_change":"2023-11-28T03:11:18.392016+0000","last_active":"2023-11-28T03:11:18.393196+0000","last_peered":"2023-11-28T03:11:18.393196+0000","last_clean":"2023-11-28T03:11:18.393196+0000","last_became_active":"2023-11-28T03:11:18.391714+0000","last_became_peered":"2023-11-28T03:11:18.391714+0000","last_unstale":"2023-11-28T03:11:18.393196+0000","last_undegraded":"2023-11-28T03:11:18.393196+0000","last_fullsized":"2023-11-28T03:11:18.393196+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755843+0000","last_change":"2023-11-28T03:11:18.755924+0000","last_active":"2023-11-28T03:11:18.755843+0000","last_peered":"2023-11-28T03:11:18.755843+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755604+0000","last_became_peered":"2023-11-28T03:11:18.755604+0000","last_unstale":"2023-11-28T03:11:18.755843+0000","last_undegraded":"2023-11-28T03:11:18.755843+0000","last_fullsized":"2023-11-28T03:11:18.755843+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400025+0000","last_change":"2023-11-28T03:11:18.396374+0000","last_active":"2023-11-28T03:11:18.400025+0000","last_peered":"2023-11-28T03:11:18.400025+0000","last_clean":"2023-11-28T03:11:18.400025+0000","last_became_active":"2023-11-28T03:11:18.396165+0000","last_became_peered":"2023-11-28T03:11:18.396165+0000","last_unstale":"2023-11-28T03:11:18.400025+0000","last_undegraded":"2023-11-28T03:11:18.400025+0000","last_fullsized":"2023-11-28T03:11:18.400025+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748189+0000","last_change":"2023-11-28T03:11:18.748320+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748189+0000","last_undegraded":"2023-11-28T03:11:18.748189+0000","last_fullsized":"2023-11-28T03:11:18.748189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.383966+0000","last_change":"2023-11-28T03:11:18.380395+0000","last_active":"2023-11-28T03:11:18.383966+0000","last_peered":"2023-11-28T03:11:18.383966+0000","last_clean":"2023-11-28T03:11:18.383966+0000","last_became_active":"2023-11-28T03:11:18.379748+0000","last_became_peered":"2023-11-28T03:11:18.379748+0000","last_unstale":"2023-11-28T03:11:18.383966+0000","last_undegraded":"2023-11-28T03:11:18.383966+0000","last_fullsized":"2023-11-28T03:11:18.383966+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746878+0000","last_change":"2023-11-28T03:11:18.747007+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746878+0000","last_undegraded":"2023-11-28T03:11:18.746878+0000","last_fullsized":"2023-11-28T03:11:18.746878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386571+0000","last_change":"2023-11-28T03:11:18.382167+0000","last_active":"2023-11-28T03:11:18.386571+0000","last_peered":"2023-11-28T03:11:18.386571+0000","last_clean":"2023-11-28T03:11:18.386571+0000","last_became_active":"2023-11-28T03:11:18.381842+0000","last_became_peered":"2023-11-28T03:11:18.381842+0000","last_unstale":"2023-11-28T03:11:18.386571+0000","last_undegraded":"2023-11-28T03:11:18.386571+0000","last_fullsized":"2023-11-28T03:11:18.386571+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766948+0000","last_change":"2023-11-28T03:11:18.767015+0000","last_active":"2023-11-28T03:11:18.766948+0000","last_peered":"2023-11-28T03:11:18.766948+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766723+0000","last_became_peered":"2023-11-28T03:11:18.766723+0000","last_unstale":"2023-11-28T03:11:18.766948+0000","last_undegraded":"2023-11-28T03:11:18.766948+0000","last_fullsized":"2023-11-28T03:11:18.766948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.401250+0000","last_change":"2023-11-28T03:11:18.400594+0000","last_active":"2023-11-28T03:11:18.401250+0000","last_peered":"2023-11-28T03:11:18.401250+0000","last_clean":"2023-11-28T03:11:18.401250+0000","last_became_active":"2023-11-28T03:11:18.400354+0000","last_became_peered":"2023-11-28T03:11:18.400354+0000","last_unstale":"2023-11-28T03:11:18.401250+0000","last_undegraded":"2023-11-28T03:11:18.401250+0000","last_fullsized":"2023-11-28T03:11:18.401250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755842+0000","last_change":"2023-11-28T03:11:18.755965+0000","last_active":"2023-11-28T03:11:18.755842+0000","last_peered":"2023-11-28T03:11:18.755842+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755630+0000","last_became_peered":"2023-11-28T03:11:18.755630+0000","last_unstale":"2023-11-28T03:11:18.755842+0000","last_undegraded":"2023-11-28T03:11:18.755842+0000","last_fullsized":"2023-11-28T03:11:18.755842+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390934+0000","last_change":"2023-11-28T03:11:18.387489+0000","last_active":"2023-11-28T03:11:18.390934+0000","last_peered":"2023-11-28T03:11:18.390934+0000","last_clean":"2023-11-28T03:11:18.390934+0000","last_became_active":"2023-11-28T03:11:18.386665+0000","last_became_peered":"2023-11-28T03:11:18.386665+0000","last_unstale":"2023-11-28T03:11:18.390934+0000","last_undegraded":"2023-11-28T03:11:18.390934+0000","last_fullsized":"2023-11-28T03:11:18.390934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755670+0000","last_change":"2023-11-28T03:11:18.755670+0000","last_active":"2023-11-28T03:11:18.755670+0000","last_peered":"2023-11-28T03:11:18.755670+0000","last_clean":"2023-11-28T03:11:18.755670+0000","last_became_active":"2023-11-28T03:11:18.755386+0000","last_became_peered":"2023-11-28T03:11:18.755386+0000","last_unstale":"2023-11-28T03:11:18.755670+0000","last_undegraded":"2023-11-28T03:11:18.755670+0000","last_fullsized":"2023-11-28T03:11:18.755670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393816+0000","last_change":"2023-11-28T03:11:18.391627+0000","last_active":"2023-11-28T03:11:18.393816+0000","last_peered":"2023-11-28T03:11:18.393816+0000","last_clean":"2023-11-28T03:11:18.393816+0000","last_became_active":"2023-11-28T03:11:18.391309+0000","last_became_peered":"2023-11-28T03:11:18.391309+0000","last_unstale":"2023-11-28T03:11:18.393816+0000","last_undegraded":"2023-11-28T03:11:18.393816+0000","last_fullsized":"2023-11-28T03:11:18.393816+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757097+0000","last_change":"2023-11-28T03:11:18.757255+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757097+0000","last_undegraded":"2023-11-28T03:11:18.757097+0000","last_fullsized":"2023-11-28T03:11:18.757097+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395478+0000","last_change":"2023-11-28T03:11:18.391689+0000","last_active":"2023-11-28T03:11:18.395478+0000","last_peered":"2023-11-28T03:11:18.395478+0000","last_clean":"2023-11-28T03:11:18.395478+0000","last_became_active":"2023-11-28T03:11:18.390946+0000","last_became_peered":"2023-11-28T03:11:18.390946+0000","last_unstale":"2023-11-28T03:11:18.395478+0000","last_undegraded":"2023-11-28T03:11:18.395478+0000","last_fullsized":"2023-11-28T03:11:18.395478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751684+0000","last_change":"2023-11-28T03:11:18.751840+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751684+0000","last_undegraded":"2023-11-28T03:11:18.751684+0000","last_fullsized":"2023-11-28T03:11:18.751684+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390577+0000","last_change":"2023-11-28T03:11:18.386639+0000","last_active":"2023-11-28T03:11:18.390577+0000","last_peered":"2023-11-28T03:11:18.390577+0000","last_clean":"2023-11-28T03:11:18.390577+0000","last_became_active":"2023-11-28T03:11:18.386342+0000","last_became_peered":"2023-11-28T03:11:18.386342+0000","last_unstale":"2023-11-28T03:11:18.390577+0000","last_undegraded":"2023-11-28T03:11:18.390577+0000","last_fullsized":"2023-11-28T03:11:18.390577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758112+0000","last_change":"2023-11-28T03:11:18.758266+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758112+0000","last_undegraded":"2023-11-28T03:11:18.758112+0000","last_fullsized":"2023-11-28T03:11:18.758112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399942+0000","last_change":"2023-11-28T03:11:18.395025+0000","last_active":"2023-11-28T03:11:18.399942+0000","last_peered":"2023-11-28T03:11:18.399942+0000","last_clean":"2023-11-28T03:11:18.399942+0000","last_became_active":"2023-11-28T03:11:18.394807+0000","last_became_peered":"2023-11-28T03:11:18.394807+0000","last_unstale":"2023-11-28T03:11:18.399942+0000","last_undegraded":"2023-11-28T03:11:18.399942+0000","last_fullsized":"2023-11-28T03:11:18.399942+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751381+0000","last_change":"2023-11-28T03:11:18.751504+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751381+0000","last_undegraded":"2023-11-28T03:11:18.751381+0000","last_fullsized":"2023-11-28T03:11:18.751381+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.384795+0000","last_change":"2023-11-28T03:11:18.380585+0000","last_active":"2023-11-28T03:11:18.384795+0000","last_peered":"2023-11-28T03:11:18.384795+0000","last_clean":"2023-11-28T03:11:18.384795+0000","last_became_active":"2023-11-28T03:11:18.380095+0000","last_became_peered":"2023-11-28T03:11:18.380095+0000","last_unstale":"2023-11-28T03:11:18.384795+0000","last_undegraded":"2023-11-28T03:11:18.384795+0000","last_fullsized":"2023-11-28T03:11:18.384795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746528+0000","last_change":"2023-11-28T03:11:18.746656+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746528+0000","last_undegraded":"2023-11-28T03:11:18.746528+0000","last_fullsized":"2023-11-28T03:11:18.746528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389225+0000","last_change":"2023-11-28T03:11:18.387312+0000","last_active":"2023-11-28T03:11:18.389225+0000","last_peered":"2023-11-28T03:11:18.389225+0000","last_clean":"2023-11-28T03:11:18.389225+0000","last_became_active":"2023-11-28T03:11:18.386958+0000","last_became_peered":"2023-11-28T03:11:18.386958+0000","last_unstale":"2023-11-28T03:11:18.389225+0000","last_undegraded":"2023-11-28T03:11:18.389225+0000","last_fullsized":"2023-11-28T03:11:18.389225+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749305+0000","last_change":"2023-11-28T03:11:18.749305+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749305+0000","last_undegraded":"2023-11-28T03:11:18.749305+0000","last_fullsized":"2023-11-28T03:11:18.749305+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389375+0000","last_change":"2023-11-28T03:11:18.386227+0000","last_active":"2023-11-28T03:11:18.389375+0000","last_peered":"2023-11-28T03:11:18.389375+0000","last_clean":"2023-11-28T03:11:18.389375+0000","last_became_active":"2023-11-28T03:11:18.385925+0000","last_became_peered":"2023-11-28T03:11:18.385925+0000","last_unstale":"2023-11-28T03:11:18.389375+0000","last_undegraded":"2023-11-28T03:11:18.389375+0000","last_fullsized":"2023-11-28T03:11:18.389375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757939+0000","last_change":"2023-11-28T03:11:18.757939+0000","last_active":"2023-11-28T03:11:18.757939+0000","last_peered":"2023-11-28T03:11:18.757939+0000","last_clean":"2023-11-28T03:11:18.757939+0000","last_became_active":"2023-11-28T03:11:18.756692+0000","last_became_peered":"2023-11-28T03:11:18.756692+0000","last_unstale":"2023-11-28T03:11:18.757939+0000","last_undegraded":"2023-11-28T03:11:18.757939+0000","last_fullsized":"2023-11-28T03:11:18.757939+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395297+0000","last_change":"2023-11-28T03:11:18.386046+0000","last_active":"2023-11-28T03:11:18.395297+0000","last_peered":"2023-11-28T03:11:18.395297+0000","last_clean":"2023-11-28T03:11:18.395297+0000","last_became_active":"2023-11-28T03:11:18.385393+0000","last_became_peered":"2023-11-28T03:11:18.385393+0000","last_unstale":"2023-11-28T03:11:18.395297+0000","last_undegraded":"2023-11-28T03:11:18.395297+0000","last_fullsized":"2023-11-28T03:11:18.395297+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755485+0000","last_change":"2023-11-28T03:11:18.755485+0000","last_active":"2023-11-28T03:11:18.755485+0000","last_peered":"2023-11-28T03:11:18.755485+0000","last_clean":"2023-11-28T03:11:18.755485+0000","last_became_active":"2023-11-28T03:11:18.755182+0000","last_became_peered":"2023-11-28T03:11:18.755182+0000","last_unstale":"2023-11-28T03:11:18.755485+0000","last_undegraded":"2023-11-28T03:11:18.755485+0000","last_fullsized":"2023-11-28T03:11:18.755485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397939+0000","last_change":"2023-11-28T03:11:18.396889+0000","last_active":"2023-11-28T03:11:18.397939+0000","last_peered":"2023-11-28T03:11:18.397939+0000","last_clean":"2023-11-28T03:11:18.397939+0000","last_became_active":"2023-11-28T03:11:18.396619+0000","last_became_peered":"2023-11-28T03:11:18.396619+0000","last_unstale":"2023-11-28T03:11:18.397939+0000","last_undegraded":"2023-11-28T03:11:18.397939+0000","last_fullsized":"2023-11-28T03:11:18.397939+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761321+0000","last_change":"2023-11-28T03:11:18.761438+0000","last_active":"2023-11-28T03:11:18.761321+0000","last_peered":"2023-11-28T03:11:18.761321+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760711+0000","last_became_peered":"2023-11-28T03:11:18.760711+0000","last_unstale":"2023-11-28T03:11:18.761321+0000","last_undegraded":"2023-11-28T03:11:18.761321+0000","last_fullsized":"2023-11-28T03:11:18.761321+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396985+0000","last_change":"2023-11-28T03:11:18.396051+0000","last_active":"2023-11-28T03:11:18.396985+0000","last_peered":"2023-11-28T03:11:18.396985+0000","last_clean":"2023-11-28T03:11:18.396985+0000","last_became_active":"2023-11-28T03:11:18.395777+0000","last_became_peered":"2023-11-28T03:11:18.395777+0000","last_unstale":"2023-11-28T03:11:18.396985+0000","last_undegraded":"2023-11-28T03:11:18.396985+0000","last_fullsized":"2023-11-28T03:11:18.396985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764649+0000","last_change":"2023-11-28T03:11:18.764724+0000","last_active":"2023-11-28T03:11:18.764649+0000","last_peered":"2023-11-28T03:11:18.764649+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.764373+0000","last_became_peered":"2023-11-28T03:11:18.764373+0000","last_unstale":"2023-11-28T03:11:18.764649+0000","last_undegraded":"2023-11-28T03:11:18.764649+0000","last_fullsized":"2023-11-28T03:11:18.764649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390603+0000","last_change":"2023-11-28T03:11:18.388277+0000","last_active":"2023-11-28T03:11:18.390603+0000","last_peered":"2023-11-28T03:11:18.390603+0000","last_clean":"2023-11-28T03:11:18.390603+0000","last_became_active":"2023-11-28T03:11:18.387116+0000","last_became_peered":"2023-11-28T03:11:18.387116+0000","last_unstale":"2023-11-28T03:11:18.390603+0000","last_undegraded":"2023-11-28T03:11:18.390603+0000","last_fullsized":"2023-11-28T03:11:18.390603+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749235+0000","last_change":"2023-11-28T03:11:18.749361+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749235+0000","last_undegraded":"2023-11-28T03:11:18.749235+0000","last_fullsized":"2023-11-28T03:11:18.749235+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390576+0000","last_change":"2023-11-28T03:11:18.388476+0000","last_active":"2023-11-28T03:11:18.390576+0000","last_peered":"2023-11-28T03:11:18.390576+0000","last_clean":"2023-11-28T03:11:18.390576+0000","last_became_active":"2023-11-28T03:11:18.387333+0000","last_became_peered":"2023-11-28T03:11:18.387333+0000","last_unstale":"2023-11-28T03:11:18.390576+0000","last_undegraded":"2023-11-28T03:11:18.390576+0000","last_fullsized":"2023-11-28T03:11:18.390576+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750948+0000","last_change":"2023-11-28T03:11:18.751069+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750948+0000","last_undegraded":"2023-11-28T03:11:18.750948+0000","last_fullsized":"2023-11-28T03:11:18.750948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396753+0000","last_change":"2023-11-28T03:11:18.395421+0000","last_active":"2023-11-28T03:11:18.396753+0000","last_peered":"2023-11-28T03:11:18.396753+0000","last_clean":"2023-11-28T03:11:18.396753+0000","last_became_active":"2023-11-28T03:11:18.394055+0000","last_became_peered":"2023-11-28T03:11:18.394055+0000","last_unstale":"2023-11-28T03:11:18.396753+0000","last_undegraded":"2023-11-28T03:11:18.396753+0000","last_fullsized":"2023-11-28T03:11:18.396753+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748206+0000","last_change":"2023-11-28T03:11:18.748338+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748206+0000","last_undegraded":"2023-11-28T03:11:18.748206+0000","last_fullsized":"2023-11-28T03:11:18.748206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388103+0000","last_change":"2023-11-28T03:11:18.378416+0000","last_active":"2023-11-28T03:11:18.388103+0000","last_peered":"2023-11-28T03:11:18.388103+0000","last_clean":"2023-11-28T03:11:18.388103+0000","last_became_active":"2023-11-28T03:11:18.378168+0000","last_became_peered":"2023-11-28T03:11:18.378168+0000","last_unstale":"2023-11-28T03:11:18.388103+0000","last_undegraded":"2023-11-28T03:11:18.388103+0000","last_fullsized":"2023-11-28T03:11:18.388103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760205+0000","last_change":"2023-11-28T03:11:18.760275+0000","last_active":"2023-11-28T03:11:18.760205+0000","last_peered":"2023-11-28T03:11:18.760205+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759999+0000","last_became_peered":"2023-11-28T03:11:18.759999+0000","last_unstale":"2023-11-28T03:11:18.760205+0000","last_undegraded":"2023-11-28T03:11:18.760205+0000","last_fullsized":"2023-11-28T03:11:18.760205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.402009+0000","last_change":"2023-11-28T03:11:18.400805+0000","last_active":"2023-11-28T03:11:18.402009+0000","last_peered":"2023-11-28T03:11:18.402009+0000","last_clean":"2023-11-28T03:11:18.402009+0000","last_became_active":"2023-11-28T03:11:18.400469+0000","last_became_peered":"2023-11-28T03:11:18.400469+0000","last_unstale":"2023-11-28T03:11:18.402009+0000","last_undegraded":"2023-11-28T03:11:18.402009+0000","last_fullsized":"2023-11-28T03:11:18.402009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761738+0000","last_change":"2023-11-28T03:11:18.761855+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761738+0000","last_undegraded":"2023-11-28T03:11:18.761738+0000","last_fullsized":"2023-11-28T03:11:18.761738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390532+0000","last_change":"2023-11-28T03:11:18.386676+0000","last_active":"2023-11-28T03:11:18.390532+0000","last_peered":"2023-11-28T03:11:18.390532+0000","last_clean":"2023-11-28T03:11:18.390532+0000","last_became_active":"2023-11-28T03:11:18.386406+0000","last_became_peered":"2023-11-28T03:11:18.386406+0000","last_unstale":"2023-11-28T03:11:18.390532+0000","last_undegraded":"2023-11-28T03:11:18.390532+0000","last_fullsized":"2023-11-28T03:11:18.390532+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756246+0000","last_change":"2023-11-28T03:11:18.756315+0000","last_active":"2023-11-28T03:11:18.756246+0000","last_peered":"2023-11-28T03:11:18.756246+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755546+0000","last_became_peered":"2023-11-28T03:11:18.755546+0000","last_unstale":"2023-11-28T03:11:18.756246+0000","last_undegraded":"2023-11-28T03:11:18.756246+0000","last_fullsized":"2023-11-28T03:11:18.756246+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393795+0000","last_change":"2023-11-28T03:11:18.390764+0000","last_active":"2023-11-28T03:11:18.393795+0000","last_peered":"2023-11-28T03:11:18.393795+0000","last_clean":"2023-11-28T03:11:18.393795+0000","last_became_active":"2023-11-28T03:11:18.390219+0000","last_became_peered":"2023-11-28T03:11:18.390219+0000","last_unstale":"2023-11-28T03:11:18.393795+0000","last_undegraded":"2023-11-28T03:11:18.393795+0000","last_fullsized":"2023-11-28T03:11:18.393795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763333+0000","last_change":"2023-11-28T03:11:18.763422+0000","last_active":"2023-11-28T03:11:18.763333+0000","last_peered":"2023-11-28T03:11:18.763333+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.763080+0000","last_became_peered":"2023-11-28T03:11:18.763080+0000","last_unstale":"2023-11-28T03:11:18.763333+0000","last_undegraded":"2023-11-28T03:11:18.763333+0000","last_fullsized":"2023-11-28T03:11:18.763333+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399936+0000","last_change":"2023-11-28T03:11:18.395919+0000","last_active":"2023-11-28T03:11:18.399936+0000","last_peered":"2023-11-28T03:11:18.399936+0000","last_clean":"2023-11-28T03:11:18.399936+0000","last_became_active":"2023-11-28T03:11:18.395621+0000","last_became_peered":"2023-11-28T03:11:18.395621+0000","last_unstale":"2023-11-28T03:11:18.399936+0000","last_undegraded":"2023-11-28T03:11:18.399936+0000","last_fullsized":"2023-11-28T03:11:18.399936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762081+0000","last_change":"2023-11-28T03:11:18.762163+0000","last_active":"2023-11-28T03:11:18.762081+0000","last_peered":"2023-11-28T03:11:18.762081+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759034+0000","last_became_peered":"2023-11-28T03:11:18.759034+0000","last_unstale":"2023-11-28T03:11:18.762081+0000","last_undegraded":"2023-11-28T03:11:18.762081+0000","last_fullsized":"2023-11-28T03:11:18.762081+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391545+0000","last_change":"2023-11-28T03:11:18.389034+0000","last_active":"2023-11-28T03:11:18.391545+0000","last_peered":"2023-11-28T03:11:18.391545+0000","last_clean":"2023-11-28T03:11:18.391545+0000","last_became_active":"2023-11-28T03:11:18.388710+0000","last_became_peered":"2023-11-28T03:11:18.388710+0000","last_unstale":"2023-11-28T03:11:18.391545+0000","last_undegraded":"2023-11-28T03:11:18.391545+0000","last_fullsized":"2023-11-28T03:11:18.391545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750502+0000","last_change":"2023-11-28T03:11:18.750640+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750502+0000","last_undegraded":"2023-11-28T03:11:18.750502+0000","last_fullsized":"2023-11-28T03:11:18.750502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394539+0000","last_change":"2023-11-28T03:11:18.391283+0000","last_active":"2023-11-28T03:11:18.394539+0000","last_peered":"2023-11-28T03:11:18.394539+0000","last_clean":"2023-11-28T03:11:18.394539+0000","last_became_active":"2023-11-28T03:11:18.390915+0000","last_became_peered":"2023-11-28T03:11:18.390915+0000","last_unstale":"2023-11-28T03:11:18.394539+0000","last_undegraded":"2023-11-28T03:11:18.394539+0000","last_fullsized":"2023-11-28T03:11:18.394539+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745744+0000","last_change":"2023-11-28T03:11:18.745874+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745744+0000","last_undegraded":"2023-11-28T03:11:18.745744+0000","last_fullsized":"2023-11-28T03:11:18.745744+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394088+0000","last_change":"2023-11-28T03:11:18.392679+0000","last_active":"2023-11-28T03:11:18.394088+0000","last_peered":"2023-11-28T03:11:18.394088+0000","last_clean":"2023-11-28T03:11:18.394088+0000","last_became_active":"2023-11-28T03:11:18.392366+0000","last_became_peered":"2023-11-28T03:11:18.392366+0000","last_unstale":"2023-11-28T03:11:18.394088+0000","last_undegraded":"2023-11-28T03:11:18.394088+0000","last_fullsized":"2023-11-28T03:11:18.394088+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751787+0000","last_change":"2023-11-28T03:11:18.751924+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751787+0000","last_undegraded":"2023-11-28T03:11:18.751787+0000","last_fullsized":"2023-11-28T03:11:18.751787+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387291+0000","last_change":"2023-11-28T03:11:18.385219+0000","last_active":"2023-11-28T03:11:18.387291+0000","last_peered":"2023-11-28T03:11:18.387291+0000","last_clean":"2023-11-28T03:11:18.387291+0000","last_became_active":"2023-11-28T03:11:18.384858+0000","last_became_peered":"2023-11-28T03:11:18.384858+0000","last_unstale":"2023-11-28T03:11:18.387291+0000","last_undegraded":"2023-11-28T03:11:18.387291+0000","last_fullsized":"2023-11-28T03:11:18.387291+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757565+0000","last_change":"2023-11-28T03:11:18.757640+0000","last_active":"2023-11-28T03:11:18.757565+0000","last_peered":"2023-11-28T03:11:18.757565+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757397+0000","last_became_peered":"2023-11-28T03:11:18.757397+0000","last_unstale":"2023-11-28T03:11:18.757565+0000","last_undegraded":"2023-11-28T03:11:18.757565+0000","last_fullsized":"2023-11-28T03:11:18.757565+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397201+0000","last_change":"2023-11-28T03:11:18.395814+0000","last_active":"2023-11-28T03:11:18.397201+0000","last_peered":"2023-11-28T03:11:18.397201+0000","last_clean":"2023-11-28T03:11:18.397201+0000","last_became_active":"2023-11-28T03:11:18.395516+0000","last_became_peered":"2023-11-28T03:11:18.395516+0000","last_unstale":"2023-11-28T03:11:18.397201+0000","last_undegraded":"2023-11-28T03:11:18.397201+0000","last_fullsized":"2023-11-28T03:11:18.397201+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745455+0000","last_change":"2023-11-28T03:11:18.745591+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745455+0000","last_undegraded":"2023-11-28T03:11:18.745455+0000","last_fullsized":"2023-11-28T03:11:18.745455+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397153+0000","last_change":"2023-11-28T03:11:18.395387+0000","last_active":"2023-11-28T03:11:18.397153+0000","last_peered":"2023-11-28T03:11:18.397153+0000","last_clean":"2023-11-28T03:11:18.397153+0000","last_became_active":"2023-11-28T03:11:18.394994+0000","last_became_peered":"2023-11-28T03:11:18.394994+0000","last_unstale":"2023-11-28T03:11:18.397153+0000","last_undegraded":"2023-11-28T03:11:18.397153+0000","last_fullsized":"2023-11-28T03:11:18.397153+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753511+0000","last_change":"2023-11-28T03:11:18.753635+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753511+0000","last_undegraded":"2023-11-28T03:11:18.753511+0000","last_fullsized":"2023-11-28T03:11:18.753511+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748519+0000","last_change":"2023-11-28T03:11:18.394301+0000","last_active":"2023-11-28T03:11:18.748519+0000","last_peered":"2023-11-28T03:11:18.748519+0000","last_clean":"2023-11-28T03:11:18.748519+0000","last_became_active":"2023-11-28T03:11:18.393966+0000","last_became_peered":"2023-11-28T03:11:18.393966+0000","last_unstale":"2023-11-28T03:11:18.748519+0000","last_undegraded":"2023-11-28T03:11:18.748519+0000","last_fullsized":"2023-11-28T03:11:18.748519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761932+0000","last_change":"2023-11-28T03:11:18.761997+0000","last_active":"2023-11-28T03:11:18.761932+0000","last_peered":"2023-11-28T03:11:18.761932+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761245+0000","last_became_peered":"2023-11-28T03:11:18.761245+0000","last_unstale":"2023-11-28T03:11:18.761932+0000","last_undegraded":"2023-11-28T03:11:18.761932+0000","last_fullsized":"2023-11-28T03:11:18.761932+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396237+0000","last_change":"2023-11-28T03:11:18.394666+0000","last_active":"2023-11-28T03:11:18.396237+0000","last_peered":"2023-11-28T03:11:18.396237+0000","last_clean":"2023-11-28T03:11:18.396237+0000","last_became_active":"2023-11-28T03:11:18.393583+0000","last_became_peered":"2023-11-28T03:11:18.393583+0000","last_unstale":"2023-11-28T03:11:18.396237+0000","last_undegraded":"2023-11-28T03:11:18.396237+0000","last_fullsized":"2023-11-28T03:11:18.396237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761889+0000","last_change":"2023-11-28T03:11:18.761969+0000","last_active":"2023-11-28T03:11:18.761889+0000","last_peered":"2023-11-28T03:11:18.761889+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758807+0000","last_became_peered":"2023-11-28T03:11:18.758807+0000","last_unstale":"2023-11-28T03:11:18.761889+0000","last_undegraded":"2023-11-28T03:11:18.761889+0000","last_fullsized":"2023-11-28T03:11:18.761889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395238+0000","last_change":"2023-11-28T03:11:18.384737+0000","last_active":"2023-11-28T03:11:18.395238+0000","last_peered":"2023-11-28T03:11:18.395238+0000","last_clean":"2023-11-28T03:11:18.395238+0000","last_became_active":"2023-11-28T03:11:18.384433+0000","last_became_peered":"2023-11-28T03:11:18.384433+0000","last_unstale":"2023-11-28T03:11:18.395238+0000","last_undegraded":"2023-11-28T03:11:18.395238+0000","last_fullsized":"2023-11-28T03:11:18.395238+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759794+0000","last_change":"2023-11-28T03:11:18.759866+0000","last_active":"2023-11-28T03:11:18.759794+0000","last_peered":"2023-11-28T03:11:18.759794+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759595+0000","last_became_peered":"2023-11-28T03:11:18.759595+0000","last_unstale":"2023-11-28T03:11:18.759794+0000","last_undegraded":"2023-11-28T03:11:18.759794+0000","last_fullsized":"2023-11-28T03:11:18.759794+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395339+0000","last_change":"2023-11-28T03:11:18.391939+0000","last_active":"2023-11-28T03:11:18.395339+0000","last_peered":"2023-11-28T03:11:18.395339+0000","last_clean":"2023-11-28T03:11:18.395339+0000","last_became_active":"2023-11-28T03:11:18.391661+0000","last_became_peered":"2023-11-28T03:11:18.391661+0000","last_unstale":"2023-11-28T03:11:18.395339+0000","last_undegraded":"2023-11-28T03:11:18.395339+0000","last_fullsized":"2023-11-28T03:11:18.395339+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.752002+0000","last_change":"2023-11-28T03:11:18.752124+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.752002+0000","last_undegraded":"2023-11-28T03:11:18.752002+0000","last_fullsized":"2023-11-28T03:11:18.752002+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392802+0000","last_change":"2023-11-28T03:11:18.390407+0000","last_active":"2023-11-28T03:11:18.392802+0000","last_peered":"2023-11-28T03:11:18.392802+0000","last_clean":"2023-11-28T03:11:18.392802+0000","last_became_active":"2023-11-28T03:11:18.390103+0000","last_became_peered":"2023-11-28T03:11:18.390103+0000","last_unstale":"2023-11-28T03:11:18.392802+0000","last_undegraded":"2023-11-28T03:11:18.392802+0000","last_fullsized":"2023-11-28T03:11:18.392802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756058+0000","last_change":"2023-11-28T03:11:18.756132+0000","last_active":"2023-11-28T03:11:18.756058+0000","last_peered":"2023-11-28T03:11:18.756058+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755858+0000","last_became_peered":"2023-11-28T03:11:18.755858+0000","last_unstale":"2023-11-28T03:11:18.756058+0000","last_undegraded":"2023-11-28T03:11:18.756058+0000","last_fullsized":"2023-11-28T03:11:18.756058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392637+0000","last_change":"2023-11-28T03:11:18.387271+0000","last_active":"2023-11-28T03:11:18.392637+0000","last_peered":"2023-11-28T03:11:18.392637+0000","last_clean":"2023-11-28T03:11:18.392637+0000","last_became_active":"2023-11-28T03:11:18.386973+0000","last_became_peered":"2023-11-28T03:11:18.386973+0000","last_unstale":"2023-11-28T03:11:18.392637+0000","last_undegraded":"2023-11-28T03:11:18.392637+0000","last_fullsized":"2023-11-28T03:11:18.392637+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.762229+0000","last_change":"2023-11-28T03:11:18.762360+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.762229+0000","last_undegraded":"2023-11-28T03:11:18.762229+0000","last_fullsized":"2023-11-28T03:11:18.762229+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392472+0000","last_change":"2023-11-28T03:11:18.387360+0000","last_active":"2023-11-28T03:11:18.392472+0000","last_peered":"2023-11-28T03:11:18.392472+0000","last_clean":"2023-11-28T03:11:18.392472+0000","last_became_active":"2023-11-28T03:11:18.386943+0000","last_became_peered":"2023-11-28T03:11:18.386943+0000","last_unstale":"2023-11-28T03:11:18.392472+0000","last_undegraded":"2023-11-28T03:11:18.392472+0000","last_fullsized":"2023-11-28T03:11:18.392472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.747957+0000","last_change":"2023-11-28T03:11:18.748091+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.747957+0000","last_undegraded":"2023-11-28T03:11:18.747957+0000","last_fullsized":"2023-11-28T03:11:18.747957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392710+0000","last_change":"2023-11-28T03:11:18.390261+0000","last_active":"2023-11-28T03:11:18.392710+0000","last_peered":"2023-11-28T03:11:18.392710+0000","last_clean":"2023-11-28T03:11:18.392710+0000","last_became_active":"2023-11-28T03:11:18.389960+0000","last_became_peered":"2023-11-28T03:11:18.389960+0000","last_unstale":"2023-11-28T03:11:18.392710+0000","last_undegraded":"2023-11-28T03:11:18.392710+0000","last_fullsized":"2023-11-28T03:11:18.392710+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753207+0000","last_change":"2023-11-28T03:11:18.753332+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753207+0000","last_undegraded":"2023-11-28T03:11:18.753207+0000","last_fullsized":"2023-11-28T03:11:18.753207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380619+0000","last_change":"2023-11-28T03:11:18.379037+0000","last_active":"2023-11-28T03:11:18.380619+0000","last_peered":"2023-11-28T03:11:18.380619+0000","last_clean":"2023-11-28T03:11:18.380619+0000","last_became_active":"2023-11-28T03:11:18.378709+0000","last_became_peered":"2023-11-28T03:11:18.378709+0000","last_unstale":"2023-11-28T03:11:18.380619+0000","last_undegraded":"2023-11-28T03:11:18.380619+0000","last_fullsized":"2023-11-28T03:11:18.380619+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751193+0000","last_change":"2023-11-28T03:11:18.751193+0000","last_active":"2023-11-28T03:11:18.751193+0000","last_peered":"2023-11-28T03:11:18.751193+0000","last_clean":"2023-11-28T03:11:18.751193+0000","last_became_active":"2023-11-28T03:11:18.750949+0000","last_became_peered":"2023-11-28T03:11:18.750949+0000","last_unstale":"2023-11-28T03:11:18.751193+0000","last_undegraded":"2023-11-28T03:11:18.751193+0000","last_fullsized":"2023-11-28T03:11:18.751193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390906+0000","last_change":"2023-11-28T03:11:18.388975+0000","last_active":"2023-11-28T03:11:18.390906+0000","last_peered":"2023-11-28T03:11:18.390906+0000","last_clean":"2023-11-28T03:11:18.390906+0000","last_became_active":"2023-11-28T03:11:18.388685+0000","last_became_peered":"2023-11-28T03:11:18.388685+0000","last_unstale":"2023-11-28T03:11:18.390906+0000","last_undegraded":"2023-11-28T03:11:18.390906+0000","last_fullsized":"2023-11-28T03:11:18.390906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757330+0000","last_change":"2023-11-28T03:11:18.757471+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757330+0000","last_undegraded":"2023-11-28T03:11:18.757330+0000","last_fullsized":"2023-11-28T03:11:18.757330+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392301+0000","last_change":"2023-11-28T03:11:18.390087+0000","last_active":"2023-11-28T03:11:18.392301+0000","last_peered":"2023-11-28T03:11:18.392301+0000","last_clean":"2023-11-28T03:11:18.392301+0000","last_became_active":"2023-11-28T03:11:18.389794+0000","last_became_peered":"2023-11-28T03:11:18.389794+0000","last_unstale":"2023-11-28T03:11:18.392301+0000","last_undegraded":"2023-11-28T03:11:18.392301+0000","last_fullsized":"2023-11-28T03:11:18.392301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750181+0000","last_change":"2023-11-28T03:11:18.750339+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750181+0000","last_undegraded":"2023-11-28T03:11:18.750181+0000","last_fullsized":"2023-11-28T03:11:18.750181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395320+0000","last_change":"2023-11-28T03:11:18.388300+0000","last_active":"2023-11-28T03:11:18.395320+0000","last_peered":"2023-11-28T03:11:18.395320+0000","last_clean":"2023-11-28T03:11:18.395320+0000","last_became_active":"2023-11-28T03:11:18.387929+0000","last_became_peered":"2023-11-28T03:11:18.387929+0000","last_unstale":"2023-11-28T03:11:18.395320+0000","last_undegraded":"2023-11-28T03:11:18.395320+0000","last_fullsized":"2023-11-28T03:11:18.395320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745039+0000","last_change":"2023-11-28T03:11:18.745192+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745039+0000","last_undegraded":"2023-11-28T03:11:18.745039+0000","last_fullsized":"2023-11-28T03:11:18.745039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388843+0000","last_change":"2023-11-28T03:11:18.384508+0000","last_active":"2023-11-28T03:11:18.388843+0000","last_peered":"2023-11-28T03:11:18.388843+0000","last_clean":"2023-11-28T03:11:18.388843+0000","last_became_active":"2023-11-28T03:11:18.384214+0000","last_became_peered":"2023-11-28T03:11:18.384214+0000","last_unstale":"2023-11-28T03:11:18.388843+0000","last_undegraded":"2023-11-28T03:11:18.388843+0000","last_fullsized":"2023-11-28T03:11:18.388843+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373322+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:11:18.373322+0000","last_peered":"2023-11-28T03:11:18.373322+0000","last_clean":"2023-11-28T03:11:18.373322+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:11:18.373322+0000","last_undegraded":"2023-11-28T03:11:18.373322+0000","last_fullsized":"2023-11-28T03:11:18.373322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750870+0000","last_change":"2023-11-28T03:11:18.750870+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750870+0000","last_undegraded":"2023-11-28T03:11:18.750870+0000","last_fullsized":"2023-11-28T03:11:18.750870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395340+0000","last_change":"2023-11-28T03:11:18.387081+0000","last_active":"2023-11-28T03:11:18.395340+0000","last_peered":"2023-11-28T03:11:18.395340+0000","last_clean":"2023-11-28T03:11:18.395340+0000","last_became_active":"2023-11-28T03:11:18.386827+0000","last_became_peered":"2023-11-28T03:11:18.386827+0000","last_unstale":"2023-11-28T03:11:18.395340+0000","last_undegraded":"2023-11-28T03:11:18.395340+0000","last_fullsized":"2023-11-28T03:11:18.395340+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.377275+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:11:18.377275+0000","last_peered":"2023-11-28T03:11:18.377275+0000","last_clean":"2023-11-28T03:11:18.377275+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:11:18.377275+0000","last_undegraded":"2023-11-28T03:11:18.377275+0000","last_fullsized":"2023-11-28T03:11:18.377275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749941+0000","last_change":"2023-11-28T03:11:18.750076+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749941+0000","last_undegraded":"2023-11-28T03:11:18.749941+0000","last_fullsized":"2023-11-28T03:11:18.749941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395335+0000","last_change":"2023-11-28T03:11:18.386867+0000","last_active":"2023-11-28T03:11:18.395335+0000","last_peered":"2023-11-28T03:11:18.395335+0000","last_clean":"2023-11-28T03:11:18.395335+0000","last_became_active":"2023-11-28T03:11:18.386623+0000","last_became_peered":"2023-11-28T03:11:18.386623+0000","last_unstale":"2023-11-28T03:11:18.395335+0000","last_undegraded":"2023-11-28T03:11:18.395335+0000","last_fullsized":"2023-11-28T03:11:18.395335+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.370453+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:11:18.370453+0000","last_peered":"2023-11-28T03:11:18.370453+0000","last_clean":"2023-11-28T03:11:18.370453+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:11:18.370453+0000","last_undegraded":"2023-11-28T03:11:18.370453+0000","last_fullsized":"2023-11-28T03:11:18.370453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758040+0000","last_change":"2023-11-28T03:11:18.758171+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758040+0000","last_undegraded":"2023-11-28T03:11:18.758040+0000","last_fullsized":"2023-11-28T03:11:18.758040+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396967+0000","last_change":"2023-11-28T03:11:18.394496+0000","last_active":"2023-11-28T03:11:18.396967+0000","last_peered":"2023-11-28T03:11:18.396967+0000","last_clean":"2023-11-28T03:11:18.396967+0000","last_became_active":"2023-11-28T03:11:18.394294+0000","last_became_peered":"2023-11-28T03:11:18.394294+0000","last_unstale":"2023-11-28T03:11:18.396967+0000","last_undegraded":"2023-11-28T03:11:18.396967+0000","last_fullsized":"2023-11-28T03:11:18.396967+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.382976+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:11:18.382976+0000","last_peered":"2023-11-28T03:11:18.382976+0000","last_clean":"2023-11-28T03:11:18.382976+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:11:18.382976+0000","last_undegraded":"2023-11-28T03:11:18.382976+0000","last_fullsized":"2023-11-28T03:11:18.382976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380178+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:11:18.380178+0000","last_peered":"2023-11-28T03:11:18.380178+0000","last_clean":"2023-11-28T03:11:18.380178+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:11:18.380178+0000","last_undegraded":"2023-11-28T03:11:18.380178+0000","last_fullsized":"2023-11-28T03:11:18.380178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760444+0000","last_change":"2023-11-28T03:11:18.760540+0000","last_active":"2023-11-28T03:11:18.760444+0000","last_peered":"2023-11-28T03:11:18.760444+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760079+0000","last_became_peered":"2023-11-28T03:11:18.760079+0000","last_unstale":"2023-11-28T03:11:18.760444+0000","last_undegraded":"2023-11-28T03:11:18.760444+0000","last_fullsized":"2023-11-28T03:11:18.760444+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400010+0000","last_change":"2023-11-28T03:11:18.396476+0000","last_active":"2023-11-28T03:11:18.400010+0000","last_peered":"2023-11-28T03:11:18.400010+0000","last_clean":"2023-11-28T03:11:18.400010+0000","last_became_active":"2023-11-28T03:11:18.396309+0000","last_became_peered":"2023-11-28T03:11:18.396309+0000","last_unstale":"2023-11-28T03:11:18.400010+0000","last_undegraded":"2023-11-28T03:11:18.400010+0000","last_fullsized":"2023-11-28T03:11:18.400010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.381718+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:11:18.381718+0000","last_peered":"2023-11-28T03:11:18.381718+0000","last_clean":"2023-11-28T03:11:18.381718+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:11:18.381718+0000","last_undegraded":"2023-11-28T03:11:18.381718+0000","last_fullsized":"2023-11-28T03:11:18.381718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.744775+0000","last_change":"2023-11-28T03:11:18.744919+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.744775+0000","last_undegraded":"2023-11-28T03:11:18.744775+0000","last_fullsized":"2023-11-28T03:11:18.744775+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389393+0000","last_change":"2023-11-28T03:11:18.387795+0000","last_active":"2023-11-28T03:11:18.389393+0000","last_peered":"2023-11-28T03:11:18.389393+0000","last_clean":"2023-11-28T03:11:18.389393+0000","last_became_active":"2023-11-28T03:11:18.387481+0000","last_became_peered":"2023-11-28T03:11:18.387481+0000","last_unstale":"2023-11-28T03:11:18.389393+0000","last_undegraded":"2023-11-28T03:11:18.389393+0000","last_fullsized":"2023-11-28T03:11:18.389393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392461+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:11:18.392461+0000","last_peered":"2023-11-28T03:11:18.392461+0000","last_clean":"2023-11-28T03:11:18.392461+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:11:18.392461+0000","last_undegraded":"2023-11-28T03:11:18.392461+0000","last_fullsized":"2023-11-28T03:11:18.392461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757177+0000","last_change":"2023-11-28T03:11:18.757177+0000","last_active":"2023-11-28T03:11:18.757177+0000","last_peered":"2023-11-28T03:11:18.757177+0000","last_clean":"2023-11-28T03:11:18.757177+0000","last_became_active":"2023-11-28T03:11:18.756875+0000","last_became_peered":"2023-11-28T03:11:18.756875+0000","last_unstale":"2023-11-28T03:11:18.757177+0000","last_undegraded":"2023-11-28T03:11:18.757177+0000","last_fullsized":"2023-11-28T03:11:18.757177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755179+0000","last_change":"2023-11-28T03:11:18.392352+0000","last_active":"2023-11-28T03:11:18.755179+0000","last_peered":"2023-11-28T03:11:18.755179+0000","last_clean":"2023-11-28T03:11:18.755179+0000","last_became_active":"2023-11-28T03:11:18.392050+0000","last_became_peered":"2023-11-28T03:11:18.392050+0000","last_unstale":"2023-11-28T03:11:18.755179+0000","last_undegraded":"2023-11-28T03:11:18.755179+0000","last_fullsized":"2023-11-28T03:11:18.755179+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373107+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:11:18.373107+0000","last_peered":"2023-11-28T03:11:18.373107+0000","last_clean":"2023-11-28T03:11:18.373107+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:11:18.373107+0000","last_undegraded":"2023-11-28T03:11:18.373107+0000","last_fullsized":"2023-11-28T03:11:18.373107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761405+0000","last_change":"2023-11-28T03:11:18.761490+0000","last_active":"2023-11-28T03:11:18.761405+0000","last_peered":"2023-11-28T03:11:18.761405+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761053+0000","last_became_peered":"2023-11-28T03:11:18.761053+0000","last_unstale":"2023-11-28T03:11:18.761405+0000","last_undegraded":"2023-11-28T03:11:18.761405+0000","last_fullsized":"2023-11-28T03:11:18.761405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395315+0000","last_change":"2023-11-28T03:11:18.386215+0000","last_active":"2023-11-28T03:11:18.395315+0000","last_peered":"2023-11-28T03:11:18.395315+0000","last_clean":"2023-11-28T03:11:18.395315+0000","last_became_active":"2023-11-28T03:11:18.385765+0000","last_became_peered":"2023-11-28T03:11:18.385765+0000","last_unstale":"2023-11-28T03:11:18.395315+0000","last_undegraded":"2023-11-28T03:11:18.395315+0000","last_fullsized":"2023-11-28T03:11:18.395315+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.378564+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:11:18.378564+0000","last_peered":"2023-11-28T03:11:18.378564+0000","last_clean":"2023-11-28T03:11:18.378564+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:11:18.378564+0000","last_undegraded":"2023-11-28T03:11:18.378564+0000","last_fullsized":"2023-11-28T03:11:18.378564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69899999999999995,"5min":1.46,"15min":1.46},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69799999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60999999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":0.64400000000000002,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.76900000000000002,"15min":0.84499999999999997},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.73699999999999999,"15min":1.4810000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.94399999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":1.125,"15min":1.506},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93000000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.90900000000000003},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.441,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.627,"5min":0.78100000000000003,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":0.59699999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.874,"15min":0.95099999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.4490000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.91100000000000003,"15min":1.3799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.90400000000000003,"15min":1.1040000000000001},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.67800000000000005},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.218,"15min":0.20899999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.67100000000000004},"last":0.22}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.21199999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.25800000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.45600000000000002},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41399999999999998,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97799999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.99299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.038},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.94099999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83199999999999996,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.024},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88300000000000001},"last":0.373}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.56599999999999995},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34399999999999997,"15min":0.36499999999999999},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53700000000000003,"5min":0.58999999999999997,"15min":0.70299999999999996},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33900000000000002,"15min":0.36299999999999999},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.439},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.73299999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.371,"15min":0.39300000000000002},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47999999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.41899999999999998},"min":{"1min":0.318,"5min":0.217,"15min":0.217},"max":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.65400000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79500000000000004,"5min":1.427,"15min":1.427},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.628},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.747,"5min":0.85899999999999999,"15min":1.2869999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.629},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":1.304},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":1.456,"15min":1.456},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.4399999999999999,"15min":1.679},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59099999999999997,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80700000000000005,"5min":1.341,"15min":1.341},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":1.474,"15min":1.474},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.315},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.39700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.19800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.67400000000000004},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.627,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.25900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.35799999999999998},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.98599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":1.004},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.91500000000000004,"15min":0.93300000000000005},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20799999999999999},"max":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.59899999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68899999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.74199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.34799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.878,"15min":1.581},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94199999999999995},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.35699999999999998},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.97899999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.93100000000000005,"15min":0.996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38,"15min":0.376},"max":{"1min":0.87,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60499999999999998,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.878},"last":0.621}]},{"osd":5,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80000000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.27900000000000003,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.28000000000000003},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.78300000000000003,"15min":0.83299999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.82499999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.747,"15min":0.80500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.81499999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.94399999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63700000000000001,"5min":0.77800000000000002,"15min":0.89900000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.91200000000000003},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:11:22.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3c is not active+clean 2023-11-28T03:11:22.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3c', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.757468+0000', 'last_change': '2023-11-28T03:11:18.757597+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.757468+0000', 'last_undegraded': '2023-11-28T03:11:18.757468+0000', 'last_fullsized': '2023-11-28T03:11:18.757468+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.433 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.39 is not active+clean 2023-11-28T03:11:22.433 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.39', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.748324+0000', 'last_change': '2023-11-28T03:11:18.748460+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.748324+0000', 'last_undegraded': '2023-11-28T03:11:18.748324+0000', 'last_fullsized': '2023-11-28T03:11:18.748324+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': ['0', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3a is not active+clean 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3a', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.748559+0000', 'last_change': '2023-11-28T03:11:18.748669+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.748559+0000', 'last_undegraded': '2023-11-28T03:11:18.748559+0000', 'last_fullsized': '2023-11-28T03:11:18.748559+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.34 is not active+clean 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.34', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.751698+0000', 'last_change': '2023-11-28T03:11:18.751811+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.751698+0000', 'last_undegraded': '2023-11-28T03:11:18.751698+0000', 'last_fullsized': '2023-11-28T03:11:18.751698+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.36 is not active+clean 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.36', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.749608+0000', 'last_change': '2023-11-28T03:11:18.749746+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.749608+0000', 'last_undegraded': '2023-11-28T03:11:18.749608+0000', 'last_fullsized': '2023-11-28T03:11:18.749608+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': ['1', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.37 is not active+clean 2023-11-28T03:11:22.434 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.37', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.761388+0000', 'last_change': '2023-11-28T03:11:18.761551+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.761388+0000', 'last_undegraded': '2023-11-28T03:11:18.761388+0000', 'last_fullsized': '2023-11-28T03:11:18.761388+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': ['7', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.30 is not active+clean 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.30', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.753548+0000', 'last_change': '2023-11-28T03:11:18.753676+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.753548+0000', 'last_undegraded': '2023-11-28T03:11:18.753548+0000', 'last_fullsized': '2023-11-28T03:11:18.753548+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': ['0', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2c is not active+clean 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2c', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.748189+0000', 'last_change': '2023-11-28T03:11:18.748320+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.748189+0000', 'last_undegraded': '2023-11-28T03:11:18.748189+0000', 'last_fullsized': '2023-11-28T03:11:18.748189+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': ['2', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2d is not active+clean 2023-11-28T03:11:22.435 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2d', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.746878+0000', 'last_change': '2023-11-28T03:11:18.747007+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.746878+0000', 'last_undegraded': '2023-11-28T03:11:18.746878+0000', 'last_fullsized': '2023-11-28T03:11:18.746878+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:11:22.436 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.29 is not active+clean 2023-11-28T03:11:22.436 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.29', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.757097+0000', 'last_change': '2023-11-28T03:11:18.757255+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.757097+0000', 'last_undegraded': '2023-11-28T03:11:18.757097+0000', 'last_fullsized': '2023-11-28T03:11:18.757097+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': ['7', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2a is not active+clean 2023-11-28T03:11:22.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2a', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.751684+0000', 'last_change': '2023-11-28T03:11:18.751840+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.751684+0000', 'last_undegraded': '2023-11-28T03:11:18.751684+0000', 'last_fullsized': '2023-11-28T03:11:18.751684+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:11:22.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2b is not active+clean 2023-11-28T03:11:22.438 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2b', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.758112+0000', 'last_change': '2023-11-28T03:11:18.758266+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.758112+0000', 'last_undegraded': '2023-11-28T03:11:18.758112+0000', 'last_fullsized': '2023-11-28T03:11:18.758112+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.438 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.24 is not active+clean 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.24', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.751381+0000', 'last_change': '2023-11-28T03:11:18.751504+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.751381+0000', 'last_undegraded': '2023-11-28T03:11:18.751381+0000', 'last_fullsized': '2023-11-28T03:11:18.751381+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': ['0', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.25 is not active+clean 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.25', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.746528+0000', 'last_change': '2023-11-28T03:11:18.746656+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.746528+0000', 'last_undegraded': '2023-11-28T03:11:18.746528+0000', 'last_fullsized': '2023-11-28T03:11:18.746528+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.26 is not active+clean 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.26', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.749305+0000', 'last_change': '2023-11-28T03:11:18.749305+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.749305+0000', 'last_undegraded': '2023-11-28T03:11:18.749305+0000', 'last_fullsized': '2023-11-28T03:11:18.749305+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': ['2', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.23 is not active+clean 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.23', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.749235+0000', 'last_change': '2023-11-28T03:11:18.749361+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.749235+0000', 'last_undegraded': '2023-11-28T03:11:18.749235+0000', 'last_fullsized': '2023-11-28T03:11:18.749235+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': ['2', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1c is not active+clean 2023-11-28T03:11:22.439 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1c', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.750948+0000', 'last_change': '2023-11-28T03:11:18.751069+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.750948+0000', 'last_undegraded': '2023-11-28T03:11:18.750948+0000', 'last_fullsized': '2023-11-28T03:11:18.750948+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': ['1', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1d is not active+clean 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1d', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.748206+0000', 'last_change': '2023-11-28T03:11:18.748338+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.748206+0000', 'last_undegraded': '2023-11-28T03:11:18.748206+0000', 'last_fullsized': '2023-11-28T03:11:18.748206+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': ['0', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1f is not active+clean 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1f', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.761738+0000', 'last_change': '2023-11-28T03:11:18.761855+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.761738+0000', 'last_undegraded': '2023-11-28T03:11:18.761738+0000', 'last_fullsized': '2023-11-28T03:11:18.761738+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.1b is not active+clean 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.1b', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.750502+0000', 'last_change': '2023-11-28T03:11:18.750640+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.750502+0000', 'last_undegraded': '2023-11-28T03:11:18.750502+0000', 'last_fullsized': '2023-11-28T03:11:18.750502+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': ['1', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.c is not active+clean 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.c', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.745744+0000', 'last_change': '2023-11-28T03:11:18.745874+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.745744+0000', 'last_undegraded': '2023-11-28T03:11:18.745744+0000', 'last_fullsized': '2023-11-28T03:11:18.745744+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': ['5', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T03:11:22.440 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.d is not active+clean 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.d', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.751787+0000', 'last_change': '2023-11-28T03:11:18.751924+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.751787+0000', 'last_undegraded': '2023-11-28T03:11:18.751787+0000', 'last_fullsized': '2023-11-28T03:11:18.751787+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': ['4', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.f is not active+clean 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.f', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.745455+0000', 'last_change': '2023-11-28T03:11:18.745591+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.745455+0000', 'last_undegraded': '2023-11-28T03:11:18.745455+0000', 'last_fullsized': '2023-11-28T03:11:18.745455+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': ['5', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.8 is not active+clean 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.8', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.753511+0000', 'last_change': '2023-11-28T03:11:18.753635+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.753511+0000', 'last_undegraded': '2023-11-28T03:11:18.753511+0000', 'last_fullsized': '2023-11-28T03:11:18.753511+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': ['0', '6'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.4 is not active+clean 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.4', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.752002+0000', 'last_change': '2023-11-28T03:11:18.752124+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.752002+0000', 'last_undegraded': '2023-11-28T03:11:18.752002+0000', 'last_fullsized': '2023-11-28T03:11:18.752002+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': ['0', '1'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.441 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.6 is not active+clean 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.6', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.762229+0000', 'last_change': '2023-11-28T03:11:18.762360+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.762229+0000', 'last_undegraded': '2023-11-28T03:11:18.762229+0000', 'last_fullsized': '2023-11-28T03:11:18.762229+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': ['7', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.7 is not active+clean 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.7', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.747957+0000', 'last_change': '2023-11-28T03:11:18.748091+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.747957+0000', 'last_undegraded': '2023-11-28T03:11:18.747957+0000', 'last_fullsized': '2023-11-28T03:11:18.747957+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': ['2', '5'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.0 is not active+clean 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.0', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.753207+0000', 'last_change': '2023-11-28T03:11:18.753332+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.753207+0000', 'last_undegraded': '2023-11-28T03:11:18.753207+0000', 'last_fullsized': '2023-11-28T03:11:18.753207+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': ['0', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.2 is not active+clean 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.2', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.757330+0000', 'last_change': '2023-11-28T03:11:18.757471+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.757330+0000', 'last_undegraded': '2023-11-28T03:11:18.757330+0000', 'last_fullsized': '2023-11-28T03:11:18.757330+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': ['7', '0'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.3 is not active+clean 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.3', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.750181+0000', 'last_change': '2023-11-28T03:11:18.750339+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.750181+0000', 'last_undegraded': '2023-11-28T03:11:18.750181+0000', 'last_fullsized': '2023-11-28T03:11:18.750181+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': ['1', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:11:22.442 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.14 is not active+clean 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.14', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.745039+0000', 'last_change': '2023-11-28T03:11:18.745192+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.745039+0000', 'last_undegraded': '2023-11-28T03:11:18.745039+0000', 'last_fullsized': '2023-11-28T03:11:18.745039+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': ['6', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.12 is not active+clean 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.12', 'version': "0'0", 'reported_seq': 3, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.750870+0000', 'last_change': '2023-11-28T03:11:18.750870+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.750870+0000', 'last_undegraded': '2023-11-28T03:11:18.750870+0000', 'last_fullsized': '2023-11-28T03:11:18.750870+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': ['2', '4'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.11 is not active+clean 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.11', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.749941+0000', 'last_change': '2023-11-28T03:11:18.750076+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.749941+0000', 'last_undegraded': '2023-11-28T03:11:18.749941+0000', 'last_fullsized': '2023-11-28T03:11:18.749941+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': ['1', '7'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.13 is not active+clean 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.13', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.758040+0000', 'last_change': '2023-11-28T03:11:18.758171+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.758040+0000', 'last_undegraded': '2023-11-28T03:11:18.758040+0000', 'last_fullsized': '2023-11-28T03:11:18.758040+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': ['7', '2'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:PG 18.15 is not active+clean 2023-11-28T03:11:22.443 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '18.15', 'version': "0'0", 'reported_seq': 2, 'reported_epoch': 139, 'state': 'creating+activating', 'last_fresh': '2023-11-28T03:11:18.744775+0000', 'last_change': '2023-11-28T03:11:18.744919+0000', 'last_active': '2023-11-28T03:11:18.363215+0000', 'last_peered': '2023-11-28T03:11:18.363215+0000', 'last_clean': '2023-11-28T03:11:18.363215+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T03:11:18.744775+0000', 'last_undegraded': '2023-11-28T03:11:18.744775+0000', 'last_fullsized': '2023-11-28T03:11:18.744775+0000', 'mapping_epoch': 138, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 138, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'last_clean_scrub_stamp': '2023-11-28T03:11:18.363215+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 3], 'acting': [5, 3], 'avail_no_missing': ['5', '3'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T03:11:22.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T03:11:22.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.576+0000 7f2c53365700 1 Processor -- start 2023-11-28T03:11:22.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 -- start start 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2c4c0caab0 0x7f2c4c0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0aa960 0x7f2c4c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0c1490 con 0x7f2c4c0cae90 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c4c0a9a90 con 0x7f2c4c0caab0 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.577+0000 7f2c53365700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c154ce0 con 0x7f2c4c0aa960 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41646/0 (socket says 172.21.15.73:41646) 2023-11-28T03:11:22.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 -- 172.21.15.73:0/1434460343 learned_addr learned my addr 172.21.15.73:0/1434460343 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c51902700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0aa960 0x7f2c4c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c51101700 1 --2- 172.21.15.73:0/1434460343 >> v2:172.21.15.150:3300/0 conn(0x7f2c4c0caab0 0x7f2c4c0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 -- 172.21.15.73:0/1434460343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0aa960 msgr2=0x7f2c4c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0aa960 0x7f2c4c0c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 -- 172.21.15.73:0/1434460343 >> v2:172.21.15.150:3300/0 conn(0x7f2c4c0caab0 msgr2=0x7f2c4c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 --2- 172.21.15.73:0/1434460343 >> v2:172.21.15.150:3300/0 conn(0x7f2c4c0caab0 0x7f2c4c0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 -- 172.21.15.73:0/1434460343 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c4c0a8e80 con 0x7f2c4c0cae90 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c50900700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f2c4c0848c0 tx=0x7f2c3c0090f0).ready entity=mon.0 client_cookie=eef649471ab44ac6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c43fff700 1 -- 172.21.15.73:0/1434460343 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00b600 con 0x7f2c4c0cae90 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c43fff700 1 -- 172.21.15.73:0/1434460343 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c3c00b760 con 0x7f2c4c0cae90 2023-11-28T03:11:22.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.578+0000 7f2c43fff700 1 -- 172.21.15.73:0/1434460343 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c3c00bae0 con 0x7f2c4c0cae90 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 -- 172.21.15.73:0/1434460343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0cae90 msgr2=0x7f2c4c0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 rx=0x7f2c4c0848c0 tx=0x7f2c3c0090f0).stop 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 -- 172.21.15.73:0/1434460343 shutdown_connections 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0aa960 0x7f2c4c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 --2- 172.21.15.73:0/1434460343 >> v2:172.21.15.150:3300/0 conn(0x7f2c4c0caab0 0x7f2c4c0aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 --2- 172.21.15.73:0/1434460343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0cae90 0x7f2c4c0c0ab0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 -- 172.21.15.73:0/1434460343 >> 172.21.15.73:0/1434460343 conn(0x7f2c4c094920 msgr2=0x7f2c4c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 -- 172.21.15.73:0/1434460343 shutdown_connections 2023-11-28T03:11:22.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.579+0000 7f2c53365700 1 -- 172.21.15.73:0/1434460343 wait complete. 2023-11-28T03:11:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.580+0000 7f2c53365700 1 Processor -- start 2023-11-28T03:11:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.580+0000 7f2c53365700 1 -- start start 2023-11-28T03:11:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 0x7f2c4c13f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 0x7f2c4c0ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c156b90 con 0x7f2c4c0acc20 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c1579d0 con 0x7f2c4c0caab0 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c4c156dc0 con 0x7f2c4c0cae90 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41650/0 (socket says 172.21.15.73:41650) 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 -- 172.21.15.73:0/270398543 learned_addr learned my addr 172.21.15.73:0/270398543 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51101700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 0x7f2c4c13f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c50900700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 0x7f2c4c0ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 -- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 msgr2=0x7f2c4c0ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 0x7f2c4c0ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 -- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 msgr2=0x7f2c4c13f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 0x7f2c4c13f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 -- 172.21.15.73:0/270398543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c3c009370 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c50900700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 0x7f2c4c0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51101700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 0x7f2c4c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c51902700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f2c4400ce80 tx=0x7f2c44007840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c4400ae40 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c44016400 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c4c1570a0 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.581+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c44016780 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.582+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c4c157660 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.582+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c4c157b40 con 0x7f2c4c0acc20 2023-11-28T03:11:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.583+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2c44009030 con 0x7f2c4c0acc20 2023-11-28T03:11:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.583+0000 7f2c417fa700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 0x7f2c30061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.583+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2c4407aa00 con 0x7f2c4c0acc20 2023-11-28T03:11:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.583+0000 7f2c51101700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 0x7f2c30061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.584+0000 7f2c51101700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 0x7f2c30061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2c48006fd0 tx=0x7f2c48008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.587+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c4404bcd0 con 0x7f2c4c0acc20 2023-11-28T03:11:22.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.815+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f2c4c06c860 con 0x7f2c4c0acc20 2023-11-28T03:11:22.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.816+0000 7f2c417fa700 1 -- 172.21.15.73:0/270398543 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1040 (secure 0 0 0) 0x7f2c4404f2f0 con 0x7f2c4c0acc20 2023-11-28T03:11:22.817 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:22.817 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":1136,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":140,"num_osds":8,"num_up_osds":8,"osd_up_since":1701139955,"num_in_osds":8,"osd_in_since":1701139948,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":102},{"state_name":"creating+activating","count":35}],"num_pgs":137,"num_pools":4,"num_objects":3,"data_bytes":90,"bytes_used":2485194752,"bytes_avail":770608918528,"bytes_total":773094113280,"inactive_pgs_ratio":0.25547444820404053},"fsmap":{"epoch":52,"id":8,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":8,"rank":0,"name":"a","status":"up:active","gid":8094}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":7,"modified":"2023-11-28T03:09:30.526016+0000","services":{}},"progress_events":{}} 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.817+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 msgr2=0x7f2c30061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.817+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 0x7f2c30061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f2c48006fd0 tx=0x7f2c48008040).stop 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 msgr2=0x7f2c4c0b0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 rx=0x7f2c4400ce80 tx=0x7f2c44007840).stop 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 shutdown_connections 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c3005f150 0x7f2c30061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2c4c0acc20 0x7f2c4c0b0630 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2c4c0caab0 0x7f2c4c13f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 --2- 172.21.15.73:0/270398543 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2c4c0cae90 0x7f2c4c0ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 >> 172.21.15.73:0/270398543 conn(0x7f2c4c094920 msgr2=0x7f2c4c0c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 shutdown_connections 2023-11-28T03:11:22.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.818+0000 7f2c53365700 1 -- 172.21.15.73:0/270398543 wait complete. 2023-11-28T03:11:22.831 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T03:11:22.831 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:11:22.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.943+0000 7f527f581700 1 Processor -- start 2023-11-28T03:11:22.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.943+0000 7f527f581700 1 -- start start 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527f581700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52780ca180 0x7f527815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527f581700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52780a9a90 con 0x7f52780caab0 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527f581700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5278154ce0 con 0x7f52780cae90 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527f581700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52780aab80 con 0x7f52780ca180 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527d31d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f52780caab0 0x7f52780aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37316/0 (socket says 172.21.15.73:37316) 2023-11-28T03:11:22.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527d31d700 1 -- 172.21.15.73:0/337365622 learned_addr learned my addr 172.21.15.73:0/337365622 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f527db1e700 1 --2- 172.21.15.73:0/337365622 >> v2:172.21.15.150:3300/0 conn(0x7f52780ca180 0x7f527815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3979772069 0 0) 0x7f52780a9a90 con 0x7f52780caab0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52640033f0 con 0x7f52780caab0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3951439749 0 0) 0x7f52640033f0 con 0x7f52780caab0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 >> v2:172.21.15.150:3300/0 conn(0x7f52780ca180 msgr2=0x7f527815f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 --2- 172.21.15.73:0/337365622 >> v2:172.21.15.150:3300/0 conn(0x7f52780ca180 0x7f527815f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 >> v1:172.21.15.150:6789/0 conn(0x7f52780cae90 legacy=0x7f52780c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.944+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780a8e80 con 0x7f52780caab0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f52700015f0 con 0x7f52780caab0 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 >> v1:172.21.15.73:6789/0 conn(0x7f52780caab0 legacy=0x7f52780aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52640038e0 0x7f5264003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264004690 0x7f5264004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 0x7f526400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780aab80 con 0x7f5264005000 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f526400bf60 con 0x7f5264004690 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f526400c0f0 con 0x7f52640038e0 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 0x7f526400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527d31d700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52640038e0 0x7f5264003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527cb1c700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264004690 0x7f5264004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 -- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52640038e0 msgr2=0x7f5264003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52640038e0 0x7f5264003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 -- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264004690 msgr2=0x7f5264004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264004690 0x7f5264004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 -- 172.21.15.73:0/337365622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52640033f0 con 0x7f5264005000 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527d31d700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52640038e0 0x7f5264003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:22.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527cb1c700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264004690 0x7f5264004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 -- 172.21.15.73:0/337365622 reap_dead start 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f527db1e700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 0x7f526400b8e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f527400e1c0 tx=0x7f527400b040).ready entity=mon.0 client_cookie=a5930b621b582e0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274017070 con 0x7f5264005000 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527400f040 con 0x7f5264005000 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.945+0000 7f526ffff700 1 -- 172.21.15.73:0/337365622 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527400eca0 con 0x7f5264005000 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 -- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 msgr2=0x7f526400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 0x7f526400b8e0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 rx=0x7f527400e1c0 tx=0x7f527400b040).stop 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 -- 172.21.15.73:0/337365622 shutdown_connections 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 --2- 172.21.15.73:0/337365622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5264005000 0x7f526400b8e0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 -- 172.21.15.73:0/337365622 >> 172.21.15.73:0/337365622 conn(0x7f5278094920 msgr2=0x7f527806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:22.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 -- 172.21.15.73:0/337365622 shutdown_connections 2023-11-28T03:11:22.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.946+0000 7f527f581700 1 -- 172.21.15.73:0/337365622 wait complete. 2023-11-28T03:11:22.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.947+0000 7f527f581700 1 Processor -- start 2023-11-28T03:11:22.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.947+0000 7f527f581700 1 -- start start 2023-11-28T03:11:22.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 0x7f52780b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52780c2310 0x7f5278157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780b0670 con 0x7f5278157530 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52780ae2c0 con 0x7f5264005000 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527f581700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52780c2a80 con 0x7f52780c2310 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41664/0 (socket says 172.21.15.73:41664) 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 -- 172.21.15.73:0/1473106355 learned_addr learned my addr 172.21.15.73:0/1473106355 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527d31d700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 0x7f52780b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527cb1c700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52780c2310 0x7f5278157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 -- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52780c2310 msgr2=0x7f5278157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52780c2310 0x7f5278157040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 -- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 msgr2=0x7f52780b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 0x7f52780b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 -- 172.21.15.73:0/1473106355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52780c2d60 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527d31d700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 0x7f52780b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.948+0000 7f527db1e700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f5274000f80 tx=0x7f5274019000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.949+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5274009ac0 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.949+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f527401e430 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.949+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f527401e7f0 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.949+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52780c3040 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.949+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52780c3320 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.950+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52780ae430 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.950+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5274003040 con 0x7f5278157530 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.950+0000 7f526d7fa700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 0x7f525c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:22.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.951+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f527407bf50 con 0x7f5278157530 2023-11-28T03:11:22.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.951+0000 7f527d31d700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 0x7f525c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:22.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.951+0000 7f527d31d700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 0x7f525c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f5270000e10 tx=0x7f5270009040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:22.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:22.955+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52780ae430 con 0x7f5278157530 2023-11-28T03:11:23.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.112+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f52780ae430 con 0x7f525c05f150 2023-11-28T03:11:23.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.122+0000 7f526d7fa700 1 -- 172.21.15.73:0/1473106355 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+360699 (secure 0 0 0) 0x7f52780ae430 con 0x7f525c05f150 2023-11-28T03:11:23.127 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 msgr2=0x7f525c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 0x7f525c061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f5270000e10 tx=0x7f5270009040).stop 2023-11-28T03:11:23.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 msgr2=0x7f52780ae7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 rx=0x7f5274000f80 tx=0x7f5274019000).stop 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 shutdown_connections 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f525c05f150 0x7f525c061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5278157530 0x7f52780ae7a0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5264005000 0x7f52780b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 --2- 172.21.15.73:0/1473106355 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52780c2310 0x7f5278157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.125+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 >> 172.21.15.73:0/1473106355 conn(0x7f5278094920 msgr2=0x7f52780ab9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.126+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 shutdown_connections 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:23.126+0000 7f527f581700 1 -- 172.21.15.73:0/1473106355 wait complete. 2023-11-28T03:11:23.130 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:11:23.139 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T03:11:21.763486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":70,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2426948,"kb_used_data":13316,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752547772,"statfs":{"total":773094113280,"available":770608918528,"internally_reserved":0,"allocated":13635584,"data_stored":6798157,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":25,"apply_latency_ms":25,"commit_latency_ns":25000000,"apply_latency_ns":25000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-19375,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-67,"num_write_kb":-43,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.586494"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757468+0000","last_change":"2023-11-28T03:11:18.757597+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757468+0000","last_undegraded":"2023-11-28T03:11:18.757468+0000","last_fullsized":"2023-11-28T03:11:18.757468+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388461+0000","last_change":"2023-11-28T03:11:18.385568+0000","last_active":"2023-11-28T03:11:18.388461+0000","last_peered":"2023-11-28T03:11:18.388461+0000","last_clean":"2023-11-28T03:11:18.388461+0000","last_became_active":"2023-11-28T03:11:18.385211+0000","last_became_peered":"2023-11-28T03:11:18.385211+0000","last_unstale":"2023-11-28T03:11:18.388461+0000","last_undegraded":"2023-11-28T03:11:18.388461+0000","last_fullsized":"2023-11-28T03:11:18.388461+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759645+0000","last_change":"2023-11-28T03:11:18.759718+0000","last_active":"2023-11-28T03:11:18.759645+0000","last_peered":"2023-11-28T03:11:18.759645+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759448+0000","last_became_peered":"2023-11-28T03:11:18.759448+0000","last_unstale":"2023-11-28T03:11:18.759645+0000","last_undegraded":"2023-11-28T03:11:18.759645+0000","last_fullsized":"2023-11-28T03:11:18.759645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391029+0000","last_change":"2023-11-28T03:11:18.389741+0000","last_active":"2023-11-28T03:11:18.391029+0000","last_peered":"2023-11-28T03:11:18.391029+0000","last_clean":"2023-11-28T03:11:18.391029+0000","last_became_active":"2023-11-28T03:11:18.389440+0000","last_became_peered":"2023-11-28T03:11:18.389440+0000","last_unstale":"2023-11-28T03:11:18.391029+0000","last_undegraded":"2023-11-28T03:11:18.391029+0000","last_fullsized":"2023-11-28T03:11:18.391029+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768741+0000","last_change":"2023-11-28T03:11:18.768823+0000","last_active":"2023-11-28T03:11:18.768741+0000","last_peered":"2023-11-28T03:11:18.768741+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766758+0000","last_became_peered":"2023-11-28T03:11:18.766758+0000","last_unstale":"2023-11-28T03:11:18.768741+0000","last_undegraded":"2023-11-28T03:11:18.768741+0000","last_fullsized":"2023-11-28T03:11:18.768741+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397054+0000","last_change":"2023-11-28T03:11:18.394730+0000","last_active":"2023-11-28T03:11:18.397054+0000","last_peered":"2023-11-28T03:11:18.397054+0000","last_clean":"2023-11-28T03:11:18.397054+0000","last_became_active":"2023-11-28T03:11:18.394473+0000","last_became_peered":"2023-11-28T03:11:18.394473+0000","last_unstale":"2023-11-28T03:11:18.397054+0000","last_undegraded":"2023-11-28T03:11:18.397054+0000","last_fullsized":"2023-11-28T03:11:18.397054+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759861+0000","last_change":"2023-11-28T03:11:18.759965+0000","last_active":"2023-11-28T03:11:18.759861+0000","last_peered":"2023-11-28T03:11:18.759861+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759675+0000","last_became_peered":"2023-11-28T03:11:18.759675+0000","last_unstale":"2023-11-28T03:11:18.759861+0000","last_undegraded":"2023-11-28T03:11:18.759861+0000","last_fullsized":"2023-11-28T03:11:18.759861+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392731+0000","last_change":"2023-11-28T03:11:18.390367+0000","last_active":"2023-11-28T03:11:18.392731+0000","last_peered":"2023-11-28T03:11:18.392731+0000","last_clean":"2023-11-28T03:11:18.392731+0000","last_became_active":"2023-11-28T03:11:18.390086+0000","last_became_peered":"2023-11-28T03:11:18.390086+0000","last_unstale":"2023-11-28T03:11:18.392731+0000","last_undegraded":"2023-11-28T03:11:18.392731+0000","last_fullsized":"2023-11-28T03:11:18.392731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757310+0000","last_change":"2023-11-28T03:11:18.757382+0000","last_active":"2023-11-28T03:11:18.757310+0000","last_peered":"2023-11-28T03:11:18.757310+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.756962+0000","last_became_peered":"2023-11-28T03:11:18.756962+0000","last_unstale":"2023-11-28T03:11:18.757310+0000","last_undegraded":"2023-11-28T03:11:18.757310+0000","last_fullsized":"2023-11-28T03:11:18.757310+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387008+0000","last_change":"2023-11-28T03:11:18.385649+0000","last_active":"2023-11-28T03:11:18.387008+0000","last_peered":"2023-11-28T03:11:18.387008+0000","last_clean":"2023-11-28T03:11:18.387008+0000","last_became_active":"2023-11-28T03:11:18.385386+0000","last_became_peered":"2023-11-28T03:11:18.385386+0000","last_unstale":"2023-11-28T03:11:18.387008+0000","last_undegraded":"2023-11-28T03:11:18.387008+0000","last_fullsized":"2023-11-28T03:11:18.387008+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748324+0000","last_change":"2023-11-28T03:11:18.748460+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748324+0000","last_undegraded":"2023-11-28T03:11:18.748324+0000","last_fullsized":"2023-11-28T03:11:18.748324+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":["0","7"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394106+0000","last_change":"2023-11-28T03:11:18.392909+0000","last_active":"2023-11-28T03:11:18.394106+0000","last_peered":"2023-11-28T03:11:18.394106+0000","last_clean":"2023-11-28T03:11:18.394106+0000","last_became_active":"2023-11-28T03:11:18.392552+0000","last_became_peered":"2023-11-28T03:11:18.392552+0000","last_unstale":"2023-11-28T03:11:18.394106+0000","last_undegraded":"2023-11-28T03:11:18.394106+0000","last_fullsized":"2023-11-28T03:11:18.394106+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748559+0000","last_change":"2023-11-28T03:11:18.748669+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748559+0000","last_undegraded":"2023-11-28T03:11:18.748559+0000","last_fullsized":"2023-11-28T03:11:18.748559+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392365+0000","last_change":"2023-11-28T03:11:18.389689+0000","last_active":"2023-11-28T03:11:18.392365+0000","last_peered":"2023-11-28T03:11:18.392365+0000","last_clean":"2023-11-28T03:11:18.392365+0000","last_became_active":"2023-11-28T03:11:18.388755+0000","last_became_peered":"2023-11-28T03:11:18.388755+0000","last_unstale":"2023-11-28T03:11:18.392365+0000","last_undegraded":"2023-11-28T03:11:18.392365+0000","last_fullsized":"2023-11-28T03:11:18.392365+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757626+0000","last_change":"2023-11-28T03:11:18.757695+0000","last_active":"2023-11-28T03:11:18.757626+0000","last_peered":"2023-11-28T03:11:18.757626+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757427+0000","last_became_peered":"2023-11-28T03:11:18.757427+0000","last_unstale":"2023-11-28T03:11:18.757626+0000","last_undegraded":"2023-11-28T03:11:18.757626+0000","last_fullsized":"2023-11-28T03:11:18.757626+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395299+0000","last_change":"2023-11-28T03:11:18.391234+0000","last_active":"2023-11-28T03:11:18.395299+0000","last_peered":"2023-11-28T03:11:18.395299+0000","last_clean":"2023-11-28T03:11:18.395299+0000","last_became_active":"2023-11-28T03:11:18.390855+0000","last_became_peered":"2023-11-28T03:11:18.390855+0000","last_unstale":"2023-11-28T03:11:18.395299+0000","last_undegraded":"2023-11-28T03:11:18.395299+0000","last_fullsized":"2023-11-28T03:11:18.395299+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751698+0000","last_change":"2023-11-28T03:11:18.751811+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751698+0000","last_undegraded":"2023-11-28T03:11:18.751698+0000","last_fullsized":"2023-11-28T03:11:18.751698+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389124+0000","last_change":"2023-11-28T03:11:18.385956+0000","last_active":"2023-11-28T03:11:18.389124+0000","last_peered":"2023-11-28T03:11:18.389124+0000","last_clean":"2023-11-28T03:11:18.389124+0000","last_became_active":"2023-11-28T03:11:18.385634+0000","last_became_peered":"2023-11-28T03:11:18.385634+0000","last_unstale":"2023-11-28T03:11:18.389124+0000","last_undegraded":"2023-11-28T03:11:18.389124+0000","last_fullsized":"2023-11-28T03:11:18.389124+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761439+0000","last_change":"2023-11-28T03:11:18.761511+0000","last_active":"2023-11-28T03:11:18.761439+0000","last_peered":"2023-11-28T03:11:18.761439+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761234+0000","last_became_peered":"2023-11-28T03:11:18.761234+0000","last_unstale":"2023-11-28T03:11:18.761439+0000","last_undegraded":"2023-11-28T03:11:18.761439+0000","last_fullsized":"2023-11-28T03:11:18.761439+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386815+0000","last_change":"2023-11-28T03:11:18.384000+0000","last_active":"2023-11-28T03:11:18.386815+0000","last_peered":"2023-11-28T03:11:18.386815+0000","last_clean":"2023-11-28T03:11:18.386815+0000","last_became_active":"2023-11-28T03:11:18.383737+0000","last_became_peered":"2023-11-28T03:11:18.383737+0000","last_unstale":"2023-11-28T03:11:18.386815+0000","last_undegraded":"2023-11-28T03:11:18.386815+0000","last_fullsized":"2023-11-28T03:11:18.386815+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749608+0000","last_change":"2023-11-28T03:11:18.749746+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749608+0000","last_undegraded":"2023-11-28T03:11:18.749608+0000","last_fullsized":"2023-11-28T03:11:18.749608+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390548+0000","last_change":"2023-11-28T03:11:18.386918+0000","last_active":"2023-11-28T03:11:18.390548+0000","last_peered":"2023-11-28T03:11:18.390548+0000","last_clean":"2023-11-28T03:11:18.390548+0000","last_became_active":"2023-11-28T03:11:18.386386+0000","last_became_peered":"2023-11-28T03:11:18.386386+0000","last_unstale":"2023-11-28T03:11:18.390548+0000","last_undegraded":"2023-11-28T03:11:18.390548+0000","last_fullsized":"2023-11-28T03:11:18.390548+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761388+0000","last_change":"2023-11-28T03:11:18.761551+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761388+0000","last_undegraded":"2023-11-28T03:11:18.761388+0000","last_fullsized":"2023-11-28T03:11:18.761388+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386802+0000","last_change":"2023-11-28T03:11:18.384007+0000","last_active":"2023-11-28T03:11:18.386802+0000","last_peered":"2023-11-28T03:11:18.386802+0000","last_clean":"2023-11-28T03:11:18.386802+0000","last_became_active":"2023-11-28T03:11:18.383671+0000","last_became_peered":"2023-11-28T03:11:18.383671+0000","last_unstale":"2023-11-28T03:11:18.386802+0000","last_undegraded":"2023-11-28T03:11:18.386802+0000","last_fullsized":"2023-11-28T03:11:18.386802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753548+0000","last_change":"2023-11-28T03:11:18.753676+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753548+0000","last_undegraded":"2023-11-28T03:11:18.753548+0000","last_fullsized":"2023-11-28T03:11:18.753548+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":["0","2"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395509+0000","last_change":"2023-11-28T03:11:18.391315+0000","last_active":"2023-11-28T03:11:18.395509+0000","last_peered":"2023-11-28T03:11:18.395509+0000","last_clean":"2023-11-28T03:11:18.395509+0000","last_became_active":"2023-11-28T03:11:18.391053+0000","last_became_peered":"2023-11-28T03:11:18.391053+0000","last_unstale":"2023-11-28T03:11:18.395509+0000","last_undegraded":"2023-11-28T03:11:18.395509+0000","last_fullsized":"2023-11-28T03:11:18.395509+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756085+0000","last_change":"2023-11-28T03:11:18.756179+0000","last_active":"2023-11-28T03:11:18.756085+0000","last_peered":"2023-11-28T03:11:18.756085+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755853+0000","last_became_peered":"2023-11-28T03:11:18.755853+0000","last_unstale":"2023-11-28T03:11:18.756085+0000","last_undegraded":"2023-11-28T03:11:18.756085+0000","last_fullsized":"2023-11-28T03:11:18.756085+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395277+0000","last_change":"2023-11-28T03:11:18.391280+0000","last_active":"2023-11-28T03:11:18.395277+0000","last_peered":"2023-11-28T03:11:18.395277+0000","last_clean":"2023-11-28T03:11:18.395277+0000","last_became_active":"2023-11-28T03:11:18.387894+0000","last_became_peered":"2023-11-28T03:11:18.387894+0000","last_unstale":"2023-11-28T03:11:18.395277+0000","last_undegraded":"2023-11-28T03:11:18.395277+0000","last_fullsized":"2023-11-28T03:11:18.395277+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759232+0000","last_change":"2023-11-28T03:11:18.764061+0000","last_active":"2023-11-28T03:11:18.759232+0000","last_peered":"2023-11-28T03:11:18.759232+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758596+0000","last_became_peered":"2023-11-28T03:11:18.758596+0000","last_unstale":"2023-11-28T03:11:18.759232+0000","last_undegraded":"2023-11-28T03:11:18.759232+0000","last_fullsized":"2023-11-28T03:11:18.759232+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393196+0000","last_change":"2023-11-28T03:11:18.392016+0000","last_active":"2023-11-28T03:11:18.393196+0000","last_peered":"2023-11-28T03:11:18.393196+0000","last_clean":"2023-11-28T03:11:18.393196+0000","last_became_active":"2023-11-28T03:11:18.391714+0000","last_became_peered":"2023-11-28T03:11:18.391714+0000","last_unstale":"2023-11-28T03:11:18.393196+0000","last_undegraded":"2023-11-28T03:11:18.393196+0000","last_fullsized":"2023-11-28T03:11:18.393196+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755843+0000","last_change":"2023-11-28T03:11:18.755924+0000","last_active":"2023-11-28T03:11:18.755843+0000","last_peered":"2023-11-28T03:11:18.755843+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755604+0000","last_became_peered":"2023-11-28T03:11:18.755604+0000","last_unstale":"2023-11-28T03:11:18.755843+0000","last_undegraded":"2023-11-28T03:11:18.755843+0000","last_fullsized":"2023-11-28T03:11:18.755843+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400025+0000","last_change":"2023-11-28T03:11:18.396374+0000","last_active":"2023-11-28T03:11:18.400025+0000","last_peered":"2023-11-28T03:11:18.400025+0000","last_clean":"2023-11-28T03:11:18.400025+0000","last_became_active":"2023-11-28T03:11:18.396165+0000","last_became_peered":"2023-11-28T03:11:18.396165+0000","last_unstale":"2023-11-28T03:11:18.400025+0000","last_undegraded":"2023-11-28T03:11:18.400025+0000","last_fullsized":"2023-11-28T03:11:18.400025+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748189+0000","last_change":"2023-11-28T03:11:18.748320+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748189+0000","last_undegraded":"2023-11-28T03:11:18.748189+0000","last_fullsized":"2023-11-28T03:11:18.748189+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":["2","0"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.383966+0000","last_change":"2023-11-28T03:11:18.380395+0000","last_active":"2023-11-28T03:11:18.383966+0000","last_peered":"2023-11-28T03:11:18.383966+0000","last_clean":"2023-11-28T03:11:18.383966+0000","last_became_active":"2023-11-28T03:11:18.379748+0000","last_became_peered":"2023-11-28T03:11:18.379748+0000","last_unstale":"2023-11-28T03:11:18.383966+0000","last_undegraded":"2023-11-28T03:11:18.383966+0000","last_fullsized":"2023-11-28T03:11:18.383966+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746878+0000","last_change":"2023-11-28T03:11:18.747007+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746878+0000","last_undegraded":"2023-11-28T03:11:18.746878+0000","last_fullsized":"2023-11-28T03:11:18.746878+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.386571+0000","last_change":"2023-11-28T03:11:18.382167+0000","last_active":"2023-11-28T03:11:18.386571+0000","last_peered":"2023-11-28T03:11:18.386571+0000","last_clean":"2023-11-28T03:11:18.386571+0000","last_became_active":"2023-11-28T03:11:18.381842+0000","last_became_peered":"2023-11-28T03:11:18.381842+0000","last_unstale":"2023-11-28T03:11:18.386571+0000","last_undegraded":"2023-11-28T03:11:18.386571+0000","last_fullsized":"2023-11-28T03:11:18.386571+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766948+0000","last_change":"2023-11-28T03:11:18.767015+0000","last_active":"2023-11-28T03:11:18.766948+0000","last_peered":"2023-11-28T03:11:18.766948+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.766723+0000","last_became_peered":"2023-11-28T03:11:18.766723+0000","last_unstale":"2023-11-28T03:11:18.766948+0000","last_undegraded":"2023-11-28T03:11:18.766948+0000","last_fullsized":"2023-11-28T03:11:18.766948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.401250+0000","last_change":"2023-11-28T03:11:18.400594+0000","last_active":"2023-11-28T03:11:18.401250+0000","last_peered":"2023-11-28T03:11:18.401250+0000","last_clean":"2023-11-28T03:11:18.401250+0000","last_became_active":"2023-11-28T03:11:18.400354+0000","last_became_peered":"2023-11-28T03:11:18.400354+0000","last_unstale":"2023-11-28T03:11:18.401250+0000","last_undegraded":"2023-11-28T03:11:18.401250+0000","last_fullsized":"2023-11-28T03:11:18.401250+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755842+0000","last_change":"2023-11-28T03:11:18.755965+0000","last_active":"2023-11-28T03:11:18.755842+0000","last_peered":"2023-11-28T03:11:18.755842+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755630+0000","last_became_peered":"2023-11-28T03:11:18.755630+0000","last_unstale":"2023-11-28T03:11:18.755842+0000","last_undegraded":"2023-11-28T03:11:18.755842+0000","last_fullsized":"2023-11-28T03:11:18.755842+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390934+0000","last_change":"2023-11-28T03:11:18.387489+0000","last_active":"2023-11-28T03:11:18.390934+0000","last_peered":"2023-11-28T03:11:18.390934+0000","last_clean":"2023-11-28T03:11:18.390934+0000","last_became_active":"2023-11-28T03:11:18.386665+0000","last_became_peered":"2023-11-28T03:11:18.386665+0000","last_unstale":"2023-11-28T03:11:18.390934+0000","last_undegraded":"2023-11-28T03:11:18.390934+0000","last_fullsized":"2023-11-28T03:11:18.390934+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755670+0000","last_change":"2023-11-28T03:11:18.755670+0000","last_active":"2023-11-28T03:11:18.755670+0000","last_peered":"2023-11-28T03:11:18.755670+0000","last_clean":"2023-11-28T03:11:18.755670+0000","last_became_active":"2023-11-28T03:11:18.755386+0000","last_became_peered":"2023-11-28T03:11:18.755386+0000","last_unstale":"2023-11-28T03:11:18.755670+0000","last_undegraded":"2023-11-28T03:11:18.755670+0000","last_fullsized":"2023-11-28T03:11:18.755670+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393816+0000","last_change":"2023-11-28T03:11:18.391627+0000","last_active":"2023-11-28T03:11:18.393816+0000","last_peered":"2023-11-28T03:11:18.393816+0000","last_clean":"2023-11-28T03:11:18.393816+0000","last_became_active":"2023-11-28T03:11:18.391309+0000","last_became_peered":"2023-11-28T03:11:18.391309+0000","last_unstale":"2023-11-28T03:11:18.393816+0000","last_undegraded":"2023-11-28T03:11:18.393816+0000","last_fullsized":"2023-11-28T03:11:18.393816+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757097+0000","last_change":"2023-11-28T03:11:18.757255+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757097+0000","last_undegraded":"2023-11-28T03:11:18.757097+0000","last_fullsized":"2023-11-28T03:11:18.757097+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":["7","3"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395478+0000","last_change":"2023-11-28T03:11:18.391689+0000","last_active":"2023-11-28T03:11:18.395478+0000","last_peered":"2023-11-28T03:11:18.395478+0000","last_clean":"2023-11-28T03:11:18.395478+0000","last_became_active":"2023-11-28T03:11:18.390946+0000","last_became_peered":"2023-11-28T03:11:18.390946+0000","last_unstale":"2023-11-28T03:11:18.395478+0000","last_undegraded":"2023-11-28T03:11:18.395478+0000","last_fullsized":"2023-11-28T03:11:18.395478+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751684+0000","last_change":"2023-11-28T03:11:18.751840+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751684+0000","last_undegraded":"2023-11-28T03:11:18.751684+0000","last_fullsized":"2023-11-28T03:11:18.751684+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390577+0000","last_change":"2023-11-28T03:11:18.386639+0000","last_active":"2023-11-28T03:11:18.390577+0000","last_peered":"2023-11-28T03:11:18.390577+0000","last_clean":"2023-11-28T03:11:18.390577+0000","last_became_active":"2023-11-28T03:11:18.386342+0000","last_became_peered":"2023-11-28T03:11:18.386342+0000","last_unstale":"2023-11-28T03:11:18.390577+0000","last_undegraded":"2023-11-28T03:11:18.390577+0000","last_fullsized":"2023-11-28T03:11:18.390577+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758112+0000","last_change":"2023-11-28T03:11:18.758266+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758112+0000","last_undegraded":"2023-11-28T03:11:18.758112+0000","last_fullsized":"2023-11-28T03:11:18.758112+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399942+0000","last_change":"2023-11-28T03:11:18.395025+0000","last_active":"2023-11-28T03:11:18.399942+0000","last_peered":"2023-11-28T03:11:18.399942+0000","last_clean":"2023-11-28T03:11:18.399942+0000","last_became_active":"2023-11-28T03:11:18.394807+0000","last_became_peered":"2023-11-28T03:11:18.394807+0000","last_unstale":"2023-11-28T03:11:18.399942+0000","last_undegraded":"2023-11-28T03:11:18.399942+0000","last_fullsized":"2023-11-28T03:11:18.399942+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751381+0000","last_change":"2023-11-28T03:11:18.751504+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751381+0000","last_undegraded":"2023-11-28T03:11:18.751381+0000","last_fullsized":"2023-11-28T03:11:18.751381+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":["0","5"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.384795+0000","last_change":"2023-11-28T03:11:18.380585+0000","last_active":"2023-11-28T03:11:18.384795+0000","last_peered":"2023-11-28T03:11:18.384795+0000","last_clean":"2023-11-28T03:11:18.384795+0000","last_became_active":"2023-11-28T03:11:18.380095+0000","last_became_peered":"2023-11-28T03:11:18.380095+0000","last_unstale":"2023-11-28T03:11:18.384795+0000","last_undegraded":"2023-11-28T03:11:18.384795+0000","last_fullsized":"2023-11-28T03:11:18.384795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.746528+0000","last_change":"2023-11-28T03:11:18.746656+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.746528+0000","last_undegraded":"2023-11-28T03:11:18.746528+0000","last_fullsized":"2023-11-28T03:11:18.746528+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389225+0000","last_change":"2023-11-28T03:11:18.387312+0000","last_active":"2023-11-28T03:11:18.389225+0000","last_peered":"2023-11-28T03:11:18.389225+0000","last_clean":"2023-11-28T03:11:18.389225+0000","last_became_active":"2023-11-28T03:11:18.386958+0000","last_became_peered":"2023-11-28T03:11:18.386958+0000","last_unstale":"2023-11-28T03:11:18.389225+0000","last_undegraded":"2023-11-28T03:11:18.389225+0000","last_fullsized":"2023-11-28T03:11:18.389225+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749305+0000","last_change":"2023-11-28T03:11:18.749305+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749305+0000","last_undegraded":"2023-11-28T03:11:18.749305+0000","last_fullsized":"2023-11-28T03:11:18.749305+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":["2","1"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389375+0000","last_change":"2023-11-28T03:11:18.386227+0000","last_active":"2023-11-28T03:11:18.389375+0000","last_peered":"2023-11-28T03:11:18.389375+0000","last_clean":"2023-11-28T03:11:18.389375+0000","last_became_active":"2023-11-28T03:11:18.385925+0000","last_became_peered":"2023-11-28T03:11:18.385925+0000","last_unstale":"2023-11-28T03:11:18.389375+0000","last_undegraded":"2023-11-28T03:11:18.389375+0000","last_fullsized":"2023-11-28T03:11:18.389375+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757939+0000","last_change":"2023-11-28T03:11:18.757939+0000","last_active":"2023-11-28T03:11:18.757939+0000","last_peered":"2023-11-28T03:11:18.757939+0000","last_clean":"2023-11-28T03:11:18.757939+0000","last_became_active":"2023-11-28T03:11:18.756692+0000","last_became_peered":"2023-11-28T03:11:18.756692+0000","last_unstale":"2023-11-28T03:11:18.757939+0000","last_undegraded":"2023-11-28T03:11:18.757939+0000","last_fullsized":"2023-11-28T03:11:18.757939+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395297+0000","last_change":"2023-11-28T03:11:18.386046+0000","last_active":"2023-11-28T03:11:18.395297+0000","last_peered":"2023-11-28T03:11:18.395297+0000","last_clean":"2023-11-28T03:11:18.395297+0000","last_became_active":"2023-11-28T03:11:18.385393+0000","last_became_peered":"2023-11-28T03:11:18.385393+0000","last_unstale":"2023-11-28T03:11:18.395297+0000","last_undegraded":"2023-11-28T03:11:18.395297+0000","last_fullsized":"2023-11-28T03:11:18.395297+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755485+0000","last_change":"2023-11-28T03:11:18.755485+0000","last_active":"2023-11-28T03:11:18.755485+0000","last_peered":"2023-11-28T03:11:18.755485+0000","last_clean":"2023-11-28T03:11:18.755485+0000","last_became_active":"2023-11-28T03:11:18.755182+0000","last_became_peered":"2023-11-28T03:11:18.755182+0000","last_unstale":"2023-11-28T03:11:18.755485+0000","last_undegraded":"2023-11-28T03:11:18.755485+0000","last_fullsized":"2023-11-28T03:11:18.755485+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397939+0000","last_change":"2023-11-28T03:11:18.396889+0000","last_active":"2023-11-28T03:11:18.397939+0000","last_peered":"2023-11-28T03:11:18.397939+0000","last_clean":"2023-11-28T03:11:18.397939+0000","last_became_active":"2023-11-28T03:11:18.396619+0000","last_became_peered":"2023-11-28T03:11:18.396619+0000","last_unstale":"2023-11-28T03:11:18.397939+0000","last_undegraded":"2023-11-28T03:11:18.397939+0000","last_fullsized":"2023-11-28T03:11:18.397939+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761321+0000","last_change":"2023-11-28T03:11:18.761438+0000","last_active":"2023-11-28T03:11:18.761321+0000","last_peered":"2023-11-28T03:11:18.761321+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760711+0000","last_became_peered":"2023-11-28T03:11:18.760711+0000","last_unstale":"2023-11-28T03:11:18.761321+0000","last_undegraded":"2023-11-28T03:11:18.761321+0000","last_fullsized":"2023-11-28T03:11:18.761321+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396985+0000","last_change":"2023-11-28T03:11:18.396051+0000","last_active":"2023-11-28T03:11:18.396985+0000","last_peered":"2023-11-28T03:11:18.396985+0000","last_clean":"2023-11-28T03:11:18.396985+0000","last_became_active":"2023-11-28T03:11:18.395777+0000","last_became_peered":"2023-11-28T03:11:18.395777+0000","last_unstale":"2023-11-28T03:11:18.396985+0000","last_undegraded":"2023-11-28T03:11:18.396985+0000","last_fullsized":"2023-11-28T03:11:18.396985+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764649+0000","last_change":"2023-11-28T03:11:18.764724+0000","last_active":"2023-11-28T03:11:18.764649+0000","last_peered":"2023-11-28T03:11:18.764649+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.764373+0000","last_became_peered":"2023-11-28T03:11:18.764373+0000","last_unstale":"2023-11-28T03:11:18.764649+0000","last_undegraded":"2023-11-28T03:11:18.764649+0000","last_fullsized":"2023-11-28T03:11:18.764649+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390603+0000","last_change":"2023-11-28T03:11:18.388277+0000","last_active":"2023-11-28T03:11:18.390603+0000","last_peered":"2023-11-28T03:11:18.390603+0000","last_clean":"2023-11-28T03:11:18.390603+0000","last_became_active":"2023-11-28T03:11:18.387116+0000","last_became_peered":"2023-11-28T03:11:18.387116+0000","last_unstale":"2023-11-28T03:11:18.390603+0000","last_undegraded":"2023-11-28T03:11:18.390603+0000","last_fullsized":"2023-11-28T03:11:18.390603+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749235+0000","last_change":"2023-11-28T03:11:18.749361+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749235+0000","last_undegraded":"2023-11-28T03:11:18.749235+0000","last_fullsized":"2023-11-28T03:11:18.749235+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":["2","3"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390576+0000","last_change":"2023-11-28T03:11:18.388476+0000","last_active":"2023-11-28T03:11:18.390576+0000","last_peered":"2023-11-28T03:11:18.390576+0000","last_clean":"2023-11-28T03:11:18.390576+0000","last_became_active":"2023-11-28T03:11:18.387333+0000","last_became_peered":"2023-11-28T03:11:18.387333+0000","last_unstale":"2023-11-28T03:11:18.390576+0000","last_undegraded":"2023-11-28T03:11:18.390576+0000","last_fullsized":"2023-11-28T03:11:18.390576+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750948+0000","last_change":"2023-11-28T03:11:18.751069+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750948+0000","last_undegraded":"2023-11-28T03:11:18.750948+0000","last_fullsized":"2023-11-28T03:11:18.750948+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":["1","0"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396753+0000","last_change":"2023-11-28T03:11:18.395421+0000","last_active":"2023-11-28T03:11:18.396753+0000","last_peered":"2023-11-28T03:11:18.396753+0000","last_clean":"2023-11-28T03:11:18.396753+0000","last_became_active":"2023-11-28T03:11:18.394055+0000","last_became_peered":"2023-11-28T03:11:18.394055+0000","last_unstale":"2023-11-28T03:11:18.396753+0000","last_undegraded":"2023-11-28T03:11:18.396753+0000","last_fullsized":"2023-11-28T03:11:18.396753+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.748206+0000","last_change":"2023-11-28T03:11:18.748338+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.748206+0000","last_undegraded":"2023-11-28T03:11:18.748206+0000","last_fullsized":"2023-11-28T03:11:18.748206+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":["0","3"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388103+0000","last_change":"2023-11-28T03:11:18.378416+0000","last_active":"2023-11-28T03:11:18.388103+0000","last_peered":"2023-11-28T03:11:18.388103+0000","last_clean":"2023-11-28T03:11:18.388103+0000","last_became_active":"2023-11-28T03:11:18.378168+0000","last_became_peered":"2023-11-28T03:11:18.378168+0000","last_unstale":"2023-11-28T03:11:18.388103+0000","last_undegraded":"2023-11-28T03:11:18.388103+0000","last_fullsized":"2023-11-28T03:11:18.388103+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760205+0000","last_change":"2023-11-28T03:11:18.760275+0000","last_active":"2023-11-28T03:11:18.760205+0000","last_peered":"2023-11-28T03:11:18.760205+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759999+0000","last_became_peered":"2023-11-28T03:11:18.759999+0000","last_unstale":"2023-11-28T03:11:18.760205+0000","last_undegraded":"2023-11-28T03:11:18.760205+0000","last_fullsized":"2023-11-28T03:11:18.760205+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.402009+0000","last_change":"2023-11-28T03:11:18.400805+0000","last_active":"2023-11-28T03:11:18.402009+0000","last_peered":"2023-11-28T03:11:18.402009+0000","last_clean":"2023-11-28T03:11:18.402009+0000","last_became_active":"2023-11-28T03:11:18.400469+0000","last_became_peered":"2023-11-28T03:11:18.400469+0000","last_unstale":"2023-11-28T03:11:18.402009+0000","last_undegraded":"2023-11-28T03:11:18.402009+0000","last_fullsized":"2023-11-28T03:11:18.402009+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.761738+0000","last_change":"2023-11-28T03:11:18.761855+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.761738+0000","last_undegraded":"2023-11-28T03:11:18.761738+0000","last_fullsized":"2023-11-28T03:11:18.761738+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390532+0000","last_change":"2023-11-28T03:11:18.386676+0000","last_active":"2023-11-28T03:11:18.390532+0000","last_peered":"2023-11-28T03:11:18.390532+0000","last_clean":"2023-11-28T03:11:18.390532+0000","last_became_active":"2023-11-28T03:11:18.386406+0000","last_became_peered":"2023-11-28T03:11:18.386406+0000","last_unstale":"2023-11-28T03:11:18.390532+0000","last_undegraded":"2023-11-28T03:11:18.390532+0000","last_fullsized":"2023-11-28T03:11:18.390532+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756246+0000","last_change":"2023-11-28T03:11:18.756315+0000","last_active":"2023-11-28T03:11:18.756246+0000","last_peered":"2023-11-28T03:11:18.756246+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755546+0000","last_became_peered":"2023-11-28T03:11:18.755546+0000","last_unstale":"2023-11-28T03:11:18.756246+0000","last_undegraded":"2023-11-28T03:11:18.756246+0000","last_fullsized":"2023-11-28T03:11:18.756246+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.393795+0000","last_change":"2023-11-28T03:11:18.390764+0000","last_active":"2023-11-28T03:11:18.393795+0000","last_peered":"2023-11-28T03:11:18.393795+0000","last_clean":"2023-11-28T03:11:18.393795+0000","last_became_active":"2023-11-28T03:11:18.390219+0000","last_became_peered":"2023-11-28T03:11:18.390219+0000","last_unstale":"2023-11-28T03:11:18.393795+0000","last_undegraded":"2023-11-28T03:11:18.393795+0000","last_fullsized":"2023-11-28T03:11:18.393795+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763333+0000","last_change":"2023-11-28T03:11:18.763422+0000","last_active":"2023-11-28T03:11:18.763333+0000","last_peered":"2023-11-28T03:11:18.763333+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.763080+0000","last_became_peered":"2023-11-28T03:11:18.763080+0000","last_unstale":"2023-11-28T03:11:18.763333+0000","last_undegraded":"2023-11-28T03:11:18.763333+0000","last_fullsized":"2023-11-28T03:11:18.763333+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.399936+0000","last_change":"2023-11-28T03:11:18.395919+0000","last_active":"2023-11-28T03:11:18.399936+0000","last_peered":"2023-11-28T03:11:18.399936+0000","last_clean":"2023-11-28T03:11:18.399936+0000","last_became_active":"2023-11-28T03:11:18.395621+0000","last_became_peered":"2023-11-28T03:11:18.395621+0000","last_unstale":"2023-11-28T03:11:18.399936+0000","last_undegraded":"2023-11-28T03:11:18.399936+0000","last_fullsized":"2023-11-28T03:11:18.399936+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762081+0000","last_change":"2023-11-28T03:11:18.762163+0000","last_active":"2023-11-28T03:11:18.762081+0000","last_peered":"2023-11-28T03:11:18.762081+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759034+0000","last_became_peered":"2023-11-28T03:11:18.759034+0000","last_unstale":"2023-11-28T03:11:18.762081+0000","last_undegraded":"2023-11-28T03:11:18.762081+0000","last_fullsized":"2023-11-28T03:11:18.762081+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.391545+0000","last_change":"2023-11-28T03:11:18.389034+0000","last_active":"2023-11-28T03:11:18.391545+0000","last_peered":"2023-11-28T03:11:18.391545+0000","last_clean":"2023-11-28T03:11:18.391545+0000","last_became_active":"2023-11-28T03:11:18.388710+0000","last_became_peered":"2023-11-28T03:11:18.388710+0000","last_unstale":"2023-11-28T03:11:18.391545+0000","last_undegraded":"2023-11-28T03:11:18.391545+0000","last_fullsized":"2023-11-28T03:11:18.391545+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750502+0000","last_change":"2023-11-28T03:11:18.750640+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750502+0000","last_undegraded":"2023-11-28T03:11:18.750502+0000","last_fullsized":"2023-11-28T03:11:18.750502+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":["1","3"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394539+0000","last_change":"2023-11-28T03:11:18.391283+0000","last_active":"2023-11-28T03:11:18.394539+0000","last_peered":"2023-11-28T03:11:18.394539+0000","last_clean":"2023-11-28T03:11:18.394539+0000","last_became_active":"2023-11-28T03:11:18.390915+0000","last_became_peered":"2023-11-28T03:11:18.390915+0000","last_unstale":"2023-11-28T03:11:18.394539+0000","last_undegraded":"2023-11-28T03:11:18.394539+0000","last_fullsized":"2023-11-28T03:11:18.394539+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745744+0000","last_change":"2023-11-28T03:11:18.745874+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745744+0000","last_undegraded":"2023-11-28T03:11:18.745744+0000","last_fullsized":"2023-11-28T03:11:18.745744+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":["5","0"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.394088+0000","last_change":"2023-11-28T03:11:18.392679+0000","last_active":"2023-11-28T03:11:18.394088+0000","last_peered":"2023-11-28T03:11:18.394088+0000","last_clean":"2023-11-28T03:11:18.394088+0000","last_became_active":"2023-11-28T03:11:18.392366+0000","last_became_peered":"2023-11-28T03:11:18.392366+0000","last_unstale":"2023-11-28T03:11:18.394088+0000","last_undegraded":"2023-11-28T03:11:18.394088+0000","last_fullsized":"2023-11-28T03:11:18.394088+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.751787+0000","last_change":"2023-11-28T03:11:18.751924+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.751787+0000","last_undegraded":"2023-11-28T03:11:18.751787+0000","last_fullsized":"2023-11-28T03:11:18.751787+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":["4","3"],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.387291+0000","last_change":"2023-11-28T03:11:18.385219+0000","last_active":"2023-11-28T03:11:18.387291+0000","last_peered":"2023-11-28T03:11:18.387291+0000","last_clean":"2023-11-28T03:11:18.387291+0000","last_became_active":"2023-11-28T03:11:18.384858+0000","last_became_peered":"2023-11-28T03:11:18.384858+0000","last_unstale":"2023-11-28T03:11:18.387291+0000","last_undegraded":"2023-11-28T03:11:18.387291+0000","last_fullsized":"2023-11-28T03:11:18.387291+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757565+0000","last_change":"2023-11-28T03:11:18.757640+0000","last_active":"2023-11-28T03:11:18.757565+0000","last_peered":"2023-11-28T03:11:18.757565+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.757397+0000","last_became_peered":"2023-11-28T03:11:18.757397+0000","last_unstale":"2023-11-28T03:11:18.757565+0000","last_undegraded":"2023-11-28T03:11:18.757565+0000","last_fullsized":"2023-11-28T03:11:18.757565+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397201+0000","last_change":"2023-11-28T03:11:18.395814+0000","last_active":"2023-11-28T03:11:18.397201+0000","last_peered":"2023-11-28T03:11:18.397201+0000","last_clean":"2023-11-28T03:11:18.397201+0000","last_became_active":"2023-11-28T03:11:18.395516+0000","last_became_peered":"2023-11-28T03:11:18.395516+0000","last_unstale":"2023-11-28T03:11:18.397201+0000","last_undegraded":"2023-11-28T03:11:18.397201+0000","last_fullsized":"2023-11-28T03:11:18.397201+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745455+0000","last_change":"2023-11-28T03:11:18.745591+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745455+0000","last_undegraded":"2023-11-28T03:11:18.745455+0000","last_fullsized":"2023-11-28T03:11:18.745455+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":["5","7"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.397153+0000","last_change":"2023-11-28T03:11:18.395387+0000","last_active":"2023-11-28T03:11:18.397153+0000","last_peered":"2023-11-28T03:11:18.397153+0000","last_clean":"2023-11-28T03:11:18.397153+0000","last_became_active":"2023-11-28T03:11:18.394994+0000","last_became_peered":"2023-11-28T03:11:18.394994+0000","last_unstale":"2023-11-28T03:11:18.397153+0000","last_undegraded":"2023-11-28T03:11:18.397153+0000","last_fullsized":"2023-11-28T03:11:18.397153+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753511+0000","last_change":"2023-11-28T03:11:18.753635+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753511+0000","last_undegraded":"2023-11-28T03:11:18.753511+0000","last_fullsized":"2023-11-28T03:11:18.753511+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":["0","6"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"139'1","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748519+0000","last_change":"2023-11-28T03:11:18.394301+0000","last_active":"2023-11-28T03:11:18.748519+0000","last_peered":"2023-11-28T03:11:18.748519+0000","last_clean":"2023-11-28T03:11:18.748519+0000","last_became_active":"2023-11-28T03:11:18.393966+0000","last_became_peered":"2023-11-28T03:11:18.393966+0000","last_unstale":"2023-11-28T03:11:18.748519+0000","last_undegraded":"2023-11-28T03:11:18.748519+0000","last_fullsized":"2023-11-28T03:11:18.748519+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761932+0000","last_change":"2023-11-28T03:11:18.761997+0000","last_active":"2023-11-28T03:11:18.761932+0000","last_peered":"2023-11-28T03:11:18.761932+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761245+0000","last_became_peered":"2023-11-28T03:11:18.761245+0000","last_unstale":"2023-11-28T03:11:18.761932+0000","last_undegraded":"2023-11-28T03:11:18.761932+0000","last_fullsized":"2023-11-28T03:11:18.761932+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396237+0000","last_change":"2023-11-28T03:11:18.394666+0000","last_active":"2023-11-28T03:11:18.396237+0000","last_peered":"2023-11-28T03:11:18.396237+0000","last_clean":"2023-11-28T03:11:18.396237+0000","last_became_active":"2023-11-28T03:11:18.393583+0000","last_became_peered":"2023-11-28T03:11:18.393583+0000","last_unstale":"2023-11-28T03:11:18.396237+0000","last_undegraded":"2023-11-28T03:11:18.396237+0000","last_fullsized":"2023-11-28T03:11:18.396237+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761889+0000","last_change":"2023-11-28T03:11:18.761969+0000","last_active":"2023-11-28T03:11:18.761889+0000","last_peered":"2023-11-28T03:11:18.761889+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.758807+0000","last_became_peered":"2023-11-28T03:11:18.758807+0000","last_unstale":"2023-11-28T03:11:18.761889+0000","last_undegraded":"2023-11-28T03:11:18.761889+0000","last_fullsized":"2023-11-28T03:11:18.761889+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395238+0000","last_change":"2023-11-28T03:11:18.384737+0000","last_active":"2023-11-28T03:11:18.395238+0000","last_peered":"2023-11-28T03:11:18.395238+0000","last_clean":"2023-11-28T03:11:18.395238+0000","last_became_active":"2023-11-28T03:11:18.384433+0000","last_became_peered":"2023-11-28T03:11:18.384433+0000","last_unstale":"2023-11-28T03:11:18.395238+0000","last_undegraded":"2023-11-28T03:11:18.395238+0000","last_fullsized":"2023-11-28T03:11:18.395238+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759794+0000","last_change":"2023-11-28T03:11:18.759866+0000","last_active":"2023-11-28T03:11:18.759794+0000","last_peered":"2023-11-28T03:11:18.759794+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.759595+0000","last_became_peered":"2023-11-28T03:11:18.759595+0000","last_unstale":"2023-11-28T03:11:18.759794+0000","last_undegraded":"2023-11-28T03:11:18.759794+0000","last_fullsized":"2023-11-28T03:11:18.759794+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395339+0000","last_change":"2023-11-28T03:11:18.391939+0000","last_active":"2023-11-28T03:11:18.395339+0000","last_peered":"2023-11-28T03:11:18.395339+0000","last_clean":"2023-11-28T03:11:18.395339+0000","last_became_active":"2023-11-28T03:11:18.391661+0000","last_became_peered":"2023-11-28T03:11:18.391661+0000","last_unstale":"2023-11-28T03:11:18.395339+0000","last_undegraded":"2023-11-28T03:11:18.395339+0000","last_fullsized":"2023-11-28T03:11:18.395339+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.752002+0000","last_change":"2023-11-28T03:11:18.752124+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.752002+0000","last_undegraded":"2023-11-28T03:11:18.752002+0000","last_fullsized":"2023-11-28T03:11:18.752002+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":["0","1"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392802+0000","last_change":"2023-11-28T03:11:18.390407+0000","last_active":"2023-11-28T03:11:18.392802+0000","last_peered":"2023-11-28T03:11:18.392802+0000","last_clean":"2023-11-28T03:11:18.392802+0000","last_became_active":"2023-11-28T03:11:18.390103+0000","last_became_peered":"2023-11-28T03:11:18.390103+0000","last_unstale":"2023-11-28T03:11:18.392802+0000","last_undegraded":"2023-11-28T03:11:18.392802+0000","last_fullsized":"2023-11-28T03:11:18.392802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.756058+0000","last_change":"2023-11-28T03:11:18.756132+0000","last_active":"2023-11-28T03:11:18.756058+0000","last_peered":"2023-11-28T03:11:18.756058+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.755858+0000","last_became_peered":"2023-11-28T03:11:18.755858+0000","last_unstale":"2023-11-28T03:11:18.756058+0000","last_undegraded":"2023-11-28T03:11:18.756058+0000","last_fullsized":"2023-11-28T03:11:18.756058+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392637+0000","last_change":"2023-11-28T03:11:18.387271+0000","last_active":"2023-11-28T03:11:18.392637+0000","last_peered":"2023-11-28T03:11:18.392637+0000","last_clean":"2023-11-28T03:11:18.392637+0000","last_became_active":"2023-11-28T03:11:18.386973+0000","last_became_peered":"2023-11-28T03:11:18.386973+0000","last_unstale":"2023-11-28T03:11:18.392637+0000","last_undegraded":"2023-11-28T03:11:18.392637+0000","last_fullsized":"2023-11-28T03:11:18.392637+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.762229+0000","last_change":"2023-11-28T03:11:18.762360+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.762229+0000","last_undegraded":"2023-11-28T03:11:18.762229+0000","last_fullsized":"2023-11-28T03:11:18.762229+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":["7","4"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392472+0000","last_change":"2023-11-28T03:11:18.387360+0000","last_active":"2023-11-28T03:11:18.392472+0000","last_peered":"2023-11-28T03:11:18.392472+0000","last_clean":"2023-11-28T03:11:18.392472+0000","last_became_active":"2023-11-28T03:11:18.386943+0000","last_became_peered":"2023-11-28T03:11:18.386943+0000","last_unstale":"2023-11-28T03:11:18.392472+0000","last_undegraded":"2023-11-28T03:11:18.392472+0000","last_fullsized":"2023-11-28T03:11:18.392472+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.747957+0000","last_change":"2023-11-28T03:11:18.748091+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.747957+0000","last_undegraded":"2023-11-28T03:11:18.747957+0000","last_fullsized":"2023-11-28T03:11:18.747957+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":["2","5"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392710+0000","last_change":"2023-11-28T03:11:18.390261+0000","last_active":"2023-11-28T03:11:18.392710+0000","last_peered":"2023-11-28T03:11:18.392710+0000","last_clean":"2023-11-28T03:11:18.392710+0000","last_became_active":"2023-11-28T03:11:18.389960+0000","last_became_peered":"2023-11-28T03:11:18.389960+0000","last_unstale":"2023-11-28T03:11:18.392710+0000","last_undegraded":"2023-11-28T03:11:18.392710+0000","last_fullsized":"2023-11-28T03:11:18.392710+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.753207+0000","last_change":"2023-11-28T03:11:18.753332+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.753207+0000","last_undegraded":"2023-11-28T03:11:18.753207+0000","last_fullsized":"2023-11-28T03:11:18.753207+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":["0","4"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380619+0000","last_change":"2023-11-28T03:11:18.379037+0000","last_active":"2023-11-28T03:11:18.380619+0000","last_peered":"2023-11-28T03:11:18.380619+0000","last_clean":"2023-11-28T03:11:18.380619+0000","last_became_active":"2023-11-28T03:11:18.378709+0000","last_became_peered":"2023-11-28T03:11:18.378709+0000","last_unstale":"2023-11-28T03:11:18.380619+0000","last_undegraded":"2023-11-28T03:11:18.380619+0000","last_fullsized":"2023-11-28T03:11:18.380619+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751193+0000","last_change":"2023-11-28T03:11:18.751193+0000","last_active":"2023-11-28T03:11:18.751193+0000","last_peered":"2023-11-28T03:11:18.751193+0000","last_clean":"2023-11-28T03:11:18.751193+0000","last_became_active":"2023-11-28T03:11:18.750949+0000","last_became_peered":"2023-11-28T03:11:18.750949+0000","last_unstale":"2023-11-28T03:11:18.751193+0000","last_undegraded":"2023-11-28T03:11:18.751193+0000","last_fullsized":"2023-11-28T03:11:18.751193+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.390906+0000","last_change":"2023-11-28T03:11:18.388975+0000","last_active":"2023-11-28T03:11:18.390906+0000","last_peered":"2023-11-28T03:11:18.390906+0000","last_clean":"2023-11-28T03:11:18.390906+0000","last_became_active":"2023-11-28T03:11:18.388685+0000","last_became_peered":"2023-11-28T03:11:18.388685+0000","last_unstale":"2023-11-28T03:11:18.390906+0000","last_undegraded":"2023-11-28T03:11:18.390906+0000","last_fullsized":"2023-11-28T03:11:18.390906+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.757330+0000","last_change":"2023-11-28T03:11:18.757471+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.757330+0000","last_undegraded":"2023-11-28T03:11:18.757330+0000","last_fullsized":"2023-11-28T03:11:18.757330+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":["7","0"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392301+0000","last_change":"2023-11-28T03:11:18.390087+0000","last_active":"2023-11-28T03:11:18.392301+0000","last_peered":"2023-11-28T03:11:18.392301+0000","last_clean":"2023-11-28T03:11:18.392301+0000","last_became_active":"2023-11-28T03:11:18.389794+0000","last_became_peered":"2023-11-28T03:11:18.389794+0000","last_unstale":"2023-11-28T03:11:18.392301+0000","last_undegraded":"2023-11-28T03:11:18.392301+0000","last_fullsized":"2023-11-28T03:11:18.392301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750181+0000","last_change":"2023-11-28T03:11:18.750339+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750181+0000","last_undegraded":"2023-11-28T03:11:18.750181+0000","last_fullsized":"2023-11-28T03:11:18.750181+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395320+0000","last_change":"2023-11-28T03:11:18.388300+0000","last_active":"2023-11-28T03:11:18.395320+0000","last_peered":"2023-11-28T03:11:18.395320+0000","last_clean":"2023-11-28T03:11:18.395320+0000","last_became_active":"2023-11-28T03:11:18.387929+0000","last_became_peered":"2023-11-28T03:11:18.387929+0000","last_unstale":"2023-11-28T03:11:18.395320+0000","last_undegraded":"2023-11-28T03:11:18.395320+0000","last_fullsized":"2023-11-28T03:11:18.395320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.745039+0000","last_change":"2023-11-28T03:11:18.745192+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.745039+0000","last_undegraded":"2023-11-28T03:11:18.745039+0000","last_fullsized":"2023-11-28T03:11:18.745039+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":["6","7"],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.388843+0000","last_change":"2023-11-28T03:11:18.384508+0000","last_active":"2023-11-28T03:11:18.388843+0000","last_peered":"2023-11-28T03:11:18.388843+0000","last_clean":"2023-11-28T03:11:18.388843+0000","last_became_active":"2023-11-28T03:11:18.384214+0000","last_became_peered":"2023-11-28T03:11:18.384214+0000","last_unstale":"2023-11-28T03:11:18.388843+0000","last_undegraded":"2023-11-28T03:11:18.388843+0000","last_fullsized":"2023-11-28T03:11:18.388843+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373322+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:11:18.373322+0000","last_peered":"2023-11-28T03:11:18.373322+0000","last_clean":"2023-11-28T03:11:18.373322+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:11:18.373322+0000","last_undegraded":"2023-11-28T03:11:18.373322+0000","last_fullsized":"2023-11-28T03:11:18.373322+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":3,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.750870+0000","last_change":"2023-11-28T03:11:18.750870+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.750870+0000","last_undegraded":"2023-11-28T03:11:18.750870+0000","last_fullsized":"2023-11-28T03:11:18.750870+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":["2","4"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395340+0000","last_change":"2023-11-28T03:11:18.387081+0000","last_active":"2023-11-28T03:11:18.395340+0000","last_peered":"2023-11-28T03:11:18.395340+0000","last_clean":"2023-11-28T03:11:18.395340+0000","last_became_active":"2023-11-28T03:11:18.386827+0000","last_became_peered":"2023-11-28T03:11:18.386827+0000","last_unstale":"2023-11-28T03:11:18.395340+0000","last_undegraded":"2023-11-28T03:11:18.395340+0000","last_fullsized":"2023-11-28T03:11:18.395340+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.377275+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:11:18.377275+0000","last_peered":"2023-11-28T03:11:18.377275+0000","last_clean":"2023-11-28T03:11:18.377275+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:11:18.377275+0000","last_undegraded":"2023-11-28T03:11:18.377275+0000","last_fullsized":"2023-11-28T03:11:18.377275+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.749941+0000","last_change":"2023-11-28T03:11:18.750076+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.749941+0000","last_undegraded":"2023-11-28T03:11:18.749941+0000","last_fullsized":"2023-11-28T03:11:18.749941+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":["1","7"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395335+0000","last_change":"2023-11-28T03:11:18.386867+0000","last_active":"2023-11-28T03:11:18.395335+0000","last_peered":"2023-11-28T03:11:18.395335+0000","last_clean":"2023-11-28T03:11:18.395335+0000","last_became_active":"2023-11-28T03:11:18.386623+0000","last_became_peered":"2023-11-28T03:11:18.386623+0000","last_unstale":"2023-11-28T03:11:18.395335+0000","last_undegraded":"2023-11-28T03:11:18.395335+0000","last_fullsized":"2023-11-28T03:11:18.395335+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":115,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.370453+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:11:18.370453+0000","last_peered":"2023-11-28T03:11:18.370453+0000","last_clean":"2023-11-28T03:11:18.370453+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:11:18.370453+0000","last_undegraded":"2023-11-28T03:11:18.370453+0000","last_fullsized":"2023-11-28T03:11:18.370453+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.758040+0000","last_change":"2023-11-28T03:11:18.758171+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.758040+0000","last_undegraded":"2023-11-28T03:11:18.758040+0000","last_fullsized":"2023-11-28T03:11:18.758040+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":["7","2"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.396967+0000","last_change":"2023-11-28T03:11:18.394496+0000","last_active":"2023-11-28T03:11:18.396967+0000","last_peered":"2023-11-28T03:11:18.396967+0000","last_clean":"2023-11-28T03:11:18.396967+0000","last_became_active":"2023-11-28T03:11:18.394294+0000","last_became_peered":"2023-11-28T03:11:18.394294+0000","last_unstale":"2023-11-28T03:11:18.396967+0000","last_undegraded":"2023-11-28T03:11:18.396967+0000","last_fullsized":"2023-11-28T03:11:18.396967+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":107,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.382976+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:11:18.382976+0000","last_peered":"2023-11-28T03:11:18.382976+0000","last_clean":"2023-11-28T03:11:18.382976+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:11:18.382976+0000","last_undegraded":"2023-11-28T03:11:18.382976+0000","last_fullsized":"2023-11-28T03:11:18.382976+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":110,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.380178+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:11:18.380178+0000","last_peered":"2023-11-28T03:11:18.380178+0000","last_clean":"2023-11-28T03:11:18.380178+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:11:18.380178+0000","last_undegraded":"2023-11-28T03:11:18.380178+0000","last_fullsized":"2023-11-28T03:11:18.380178+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760444+0000","last_change":"2023-11-28T03:11:18.760540+0000","last_active":"2023-11-28T03:11:18.760444+0000","last_peered":"2023-11-28T03:11:18.760444+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.760079+0000","last_became_peered":"2023-11-28T03:11:18.760079+0000","last_unstale":"2023-11-28T03:11:18.760444+0000","last_undegraded":"2023-11-28T03:11:18.760444+0000","last_fullsized":"2023-11-28T03:11:18.760444+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.400010+0000","last_change":"2023-11-28T03:11:18.396476+0000","last_active":"2023-11-28T03:11:18.400010+0000","last_peered":"2023-11-28T03:11:18.400010+0000","last_clean":"2023-11-28T03:11:18.400010+0000","last_became_active":"2023-11-28T03:11:18.396309+0000","last_became_peered":"2023-11-28T03:11:18.396309+0000","last_unstale":"2023-11-28T03:11:18.400010+0000","last_undegraded":"2023-11-28T03:11:18.400010+0000","last_fullsized":"2023-11-28T03:11:18.400010+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.381718+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:11:18.381718+0000","last_peered":"2023-11-28T03:11:18.381718+0000","last_clean":"2023-11-28T03:11:18.381718+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:11:18.381718+0000","last_undegraded":"2023-11-28T03:11:18.381718+0000","last_fullsized":"2023-11-28T03:11:18.381718+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":2,"reported_epoch":139,"state":"creating+activating","last_fresh":"2023-11-28T03:11:18.744775+0000","last_change":"2023-11-28T03:11:18.744919+0000","last_active":"2023-11-28T03:11:18.363215+0000","last_peered":"2023-11-28T03:11:18.363215+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T03:11:18.744775+0000","last_undegraded":"2023-11-28T03:11:18.744775+0000","last_fullsized":"2023-11-28T03:11:18.744775+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":["5","3"],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.389393+0000","last_change":"2023-11-28T03:11:18.387795+0000","last_active":"2023-11-28T03:11:18.389393+0000","last_peered":"2023-11-28T03:11:18.389393+0000","last_clean":"2023-11-28T03:11:18.389393+0000","last_became_active":"2023-11-28T03:11:18.387481+0000","last_became_peered":"2023-11-28T03:11:18.387481+0000","last_unstale":"2023-11-28T03:11:18.389393+0000","last_undegraded":"2023-11-28T03:11:18.389393+0000","last_fullsized":"2023-11-28T03:11:18.389393+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":108,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.392461+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:11:18.392461+0000","last_peered":"2023-11-28T03:11:18.392461+0000","last_clean":"2023-11-28T03:11:18.392461+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:11:18.392461+0000","last_undegraded":"2023-11-28T03:11:18.392461+0000","last_fullsized":"2023-11-28T03:11:18.392461+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":8,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757177+0000","last_change":"2023-11-28T03:11:18.757177+0000","last_active":"2023-11-28T03:11:18.757177+0000","last_peered":"2023-11-28T03:11:18.757177+0000","last_clean":"2023-11-28T03:11:18.757177+0000","last_became_active":"2023-11-28T03:11:18.756875+0000","last_became_peered":"2023-11-28T03:11:18.756875+0000","last_unstale":"2023-11-28T03:11:18.757177+0000","last_undegraded":"2023-11-28T03:11:18.757177+0000","last_fullsized":"2023-11-28T03:11:18.757177+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"139'3","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755179+0000","last_change":"2023-11-28T03:11:18.392352+0000","last_active":"2023-11-28T03:11:18.755179+0000","last_peered":"2023-11-28T03:11:18.755179+0000","last_clean":"2023-11-28T03:11:18.755179+0000","last_became_active":"2023-11-28T03:11:18.392050+0000","last_became_peered":"2023-11-28T03:11:18.392050+0000","last_unstale":"2023-11-28T03:11:18.755179+0000","last_undegraded":"2023-11-28T03:11:18.755179+0000","last_fullsized":"2023-11-28T03:11:18.755179+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":109,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.373107+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:11:18.373107+0000","last_peered":"2023-11-28T03:11:18.373107+0000","last_clean":"2023-11-28T03:11:18.373107+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:11:18.373107+0000","last_undegraded":"2023-11-28T03:11:18.373107+0000","last_fullsized":"2023-11-28T03:11:18.373107+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":7,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761405+0000","last_change":"2023-11-28T03:11:18.761490+0000","last_active":"2023-11-28T03:11:18.761405+0000","last_peered":"2023-11-28T03:11:18.761405+0000","last_clean":"2023-11-28T03:11:18.363215+0000","last_became_active":"2023-11-28T03:11:18.761053+0000","last_became_peered":"2023-11-28T03:11:18.761053+0000","last_unstale":"2023-11-28T03:11:18.761405+0000","last_undegraded":"2023-11-28T03:11:18.761405+0000","last_fullsized":"2023-11-28T03:11:18.761405+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":9,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.395315+0000","last_change":"2023-11-28T03:11:18.386215+0000","last_active":"2023-11-28T03:11:18.395315+0000","last_peered":"2023-11-28T03:11:18.395315+0000","last_clean":"2023-11-28T03:11:18.395315+0000","last_became_active":"2023-11-28T03:11:18.385765+0000","last_became_peered":"2023-11-28T03:11:18.385765+0000","last_unstale":"2023-11-28T03:11:18.395315+0000","last_undegraded":"2023-11-28T03:11:18.395315+0000","last_fullsized":"2023-11-28T03:11:18.395315+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":121,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.378564+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:11:18.378564+0000","last_peered":"2023-11-28T03:11:18.378564+0000","last_clean":"2023-11-28T03:11:18.378564+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:11:18.378564+0000","last_undegraded":"2023-11-28T03:11:18.378564+0000","last_fullsized":"2023-11-28T03:11:18.378564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":2204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575069,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69899999999999995,"5min":1.46,"15min":1.46},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.61199999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.52700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69799999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.433},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.45500000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.48999999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60999999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":0.64400000000000002,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.57699999999999996}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.76900000000000002,"15min":0.84499999999999997},"last":0.56899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575069,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.73699999999999999,"15min":1.4810000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48799999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.94399999999999995},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":1.125,"15min":1.506},"last":0.63100000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93000000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.90900000000000003},"last":0.67600000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.64300000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.441,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.627,"5min":0.78100000000000003,"15min":1.7310000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":0.59699999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.78700000000000003}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.874,"15min":0.95099999999999996},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.4490000000000001},"last":0.69899999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.91100000000000003,"15min":1.3799999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.90400000000000003,"15min":1.1040000000000001},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.67800000000000005},"last":0.20399999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.218,"15min":0.20899999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.67100000000000004},"last":0.22}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.21199999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.25800000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.45600000000000002},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.32500000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41399999999999998,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"last":0.308}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.91100000000000003},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97799999999999998},"last":0.52800000000000002}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.99299999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.038},"last":0.45900000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.94099999999999995},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83199999999999996,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.47999999999999998}]},{"osd":7,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.024},"last":0.504},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88300000000000001},"last":0.373}]}]},{"osd":0,"up_from":13,"seq":55834575069,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.56599999999999995},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.34399999999999997,"15min":0.36499999999999999},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53700000000000003,"5min":0.58999999999999997,"15min":0.70299999999999996},"last":0.28000000000000003}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33900000000000002,"15min":0.36299999999999999},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.439},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.73299999999999998},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.371,"15min":0.39300000000000002},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47999999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.41899999999999998},"min":{"1min":0.318,"5min":0.217,"15min":0.217},"max":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.65400000000000003},"last":0.34999999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79500000000000004,"5min":1.427,"15min":1.427},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.628},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.747,"5min":0.85899999999999999,"15min":1.2869999999999999},"last":0.68700000000000006}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.629},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":1.304},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":1.456,"15min":1.456},"last":0.71899999999999997}]},{"osd":6,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.4399999999999999,"15min":1.679},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59099999999999997,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80700000000000005,"5min":1.341,"15min":1.341},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":1.474,"15min":1.474},"last":0.628}]}]},{"osd":2,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302852,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068988,"statfs":{"total":96636764160,"available":96326643712,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.315},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.35799999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.39700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.28799999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.19800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.67400000000000004},"last":0.371},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.627,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.25900000000000001}]},{"osd":4,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.35799999999999998},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.70299999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.98599999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":1.004},"last":0.66200000000000003}]},{"osd":6,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.91500000000000004,"15min":0.93300000000000005},"last":0.72099999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.61899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575069,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306948,"kb_used_data":1660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064892,"statfs":{"total":96636764160,"available":96322449408,"internally_reserved":0,"allocated":1699840,"data_stored":847265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20799999999999999},"max":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.59899999999999998},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68899999999999995},"last":0.39700000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.74199999999999999},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.34799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.35999999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.878,"15min":1.581},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94199999999999995},"last":0.61499999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.35699999999999998},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.97899999999999998},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.76800000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.93100000000000005,"15min":0.996},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.67100000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.97199999999999998},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38,"15min":0.376},"max":{"1min":0.87,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.66200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575068,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.52600000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.5},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.60699999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60499999999999998,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.878},"last":0.621}]},{"osd":5,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80000000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.42099999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.45500000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.64400000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575070,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302864,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068976,"statfs":{"total":96636764160,"available":96326631424,"internally_reserved":0,"allocated":1712128,"data_stored":853944,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.27900000000000003,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.28000000000000003},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.77900000000000003},"last":0.50700000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.78300000000000003,"15min":0.83299999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.82499999999999996},"last":0.47999999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.747,"15min":0.80500000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"last":0.53100000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.438},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.54700000000000004}]},{"osd":4,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.85699999999999998},"last":0.62},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.81499999999999995},"last":0.499}]},{"osd":6,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.94399999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63700000000000001,"5min":0.77800000000000002,"15min":0.89900000000000002},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.91200000000000003},"last":0.60599999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:11:23.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:11:23.480 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:11:24.660 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T03:11:24.660+0000 7fb353d75700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T03:11:26.146 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T03:11:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.272+0000 7f7dab348700 1 Processor -- start 2023-11-28T03:11:26.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.272+0000 7f7dab348700 1 -- start start 2023-11-28T03:11:26.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 0x7f7da40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7da40cae90 0x7f7da40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40aa960 0x7f7da40c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40c1490 con 0x7f7da40cae90 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7da40a9a90 con 0x7f7da40caab0 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7dab348700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da4154ce0 con 0x7f7da40aa960 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7da40cae90 0x7f7da40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7da40cae90 0x7f7da40c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35484/0 (socket says 172.21.15.73:35484) 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 -- 172.21.15.73:0/2089725563 learned_addr learned my addr 172.21.15.73:0/2089725563 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da98e5700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40aa960 0x7f7da40c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da90e4700 1 --2- 172.21.15.73:0/2089725563 >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 0x7f7da40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 -- 172.21.15.73:0/2089725563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40aa960 msgr2=0x7f7da40c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40aa960 0x7f7da40c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 -- 172.21.15.73:0/2089725563 >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 msgr2=0x7f7da40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:26.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 --2- 172.21.15.73:0/2089725563 >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 0x7f7da40aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 -- 172.21.15.73:0/2089725563 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da40a8e80 con 0x7f7da40cae90 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da90e4700 1 --2- 172.21.15.73:0/2089725563 >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 0x7f7da40aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.273+0000 7f7da88e3700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c0ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f7da40848c0 tx=0x7f7d940090f0).ready entity=mon.0 client_cookie=4324f95054454c98 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7d9bfff700 1 -- 172.21.15.73:0/2089725563 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400b520 con 0x7f7da40cae90 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7d9bfff700 1 -- 172.21.15.73:0/2089725563 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400b680 con 0x7f7da40cae90 2023-11-28T03:11:26.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7d9bfff700 1 -- 172.21.15.73:0/2089725563 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400ba00 con 0x7f7da40cae90 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 -- 172.21.15.73:0/2089725563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 msgr2=0x7f7da40c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c0ab0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 rx=0x7f7da40848c0 tx=0x7f7d940090f0).stop 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 -- 172.21.15.73:0/2089725563 shutdown_connections 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40aa960 0x7f7da40c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 --2- 172.21.15.73:0/2089725563 >> v2:172.21.15.150:3300/0 conn(0x7f7da40caab0 0x7f7da40aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 --2- 172.21.15.73:0/2089725563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c0ab0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 -- 172.21.15.73:0/2089725563 >> 172.21.15.73:0/2089725563 conn(0x7f7da4094920 msgr2=0x7f7da406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 -- 172.21.15.73:0/2089725563 shutdown_connections 2023-11-28T03:11:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.274+0000 7f7dab348700 1 -- 172.21.15.73:0/2089725563 wait complete. 2023-11-28T03:11:26.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.275+0000 7f7dab348700 1 Processor -- start 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.275+0000 7f7dab348700 1 -- start start 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40caab0 0x7f7da40ab7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7da40acec0 0x7f7da414c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c39b0 con 0x7f7da40cae90 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40c2950 con 0x7f7da40acec0 2023-11-28T03:11:26.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7dab348700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7da40aca30 con 0x7f7da40caab0 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7da88e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7da88e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35486/0 (socket says 172.21.15.73:35486) 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7da88e3700 1 -- 172.21.15.73:0/4285802640 learned_addr learned my addr 172.21.15.73:0/4285802640 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7da90e4700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40caab0 0x7f7da40ab7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.276+0000 7f7da98e5700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7da40acec0 0x7f7da414c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 -- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40caab0 msgr2=0x7f7da40ab7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40caab0 0x7f7da40ab7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 -- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7da40acec0 msgr2=0x7f7da414c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7da40acec0 0x7f7da414c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 -- 172.21.15.73:0/4285802640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d94009370 con 0x7f7da40cae90 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7da88e3700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f7da4083c30 tx=0x7f7d94017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9400be90 con 0x7f7da40cae90 2023-11-28T03:11:26.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d9400b7f0 con 0x7f7da40cae90 2023-11-28T03:11:26.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7da40ad320 con 0x7f7da40cae90 2023-11-28T03:11:26.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7da40ac100 con 0x7f7da40cae90 2023-11-28T03:11:26.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.277+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d9401b620 con 0x7f7da40cae90 2023-11-28T03:11:26.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.278+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7da406c860 con 0x7f7da40cae90 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.278+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7d9401ba10 con 0x7f7da40cae90 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.279+0000 7f7d997fa700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 0x7f7d88061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.279+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7d9407af40 con 0x7f7da40cae90 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.279+0000 7f7da90e4700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 0x7f7d88061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.279+0000 7f7da90e4700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 0x7f7d88061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7d9c007780 tx=0x7f7d9c009a20).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.283+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d9404c2c0 con 0x7f7da40cae90 2023-11-28T03:11:26.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.448+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7da40acb70 con 0x7f7d8805f150 2023-11-28T03:11:26.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.457+0000 7f7d997fa700 1 -- 172.21.15.73:0/4285802640 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362014 (secure 0 0 0) 0x7f7d9c00f040 con 0x7f7d8805f150 2023-11-28T03:11:26.462 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:26.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 msgr2=0x7f7d88061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 0x7f7d88061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f7d9c007780 tx=0x7f7d9c009a20).stop 2023-11-28T03:11:26.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 msgr2=0x7f7da40c34c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 rx=0x7f7da4083c30 tx=0x7f7d94017000).stop 2023-11-28T03:11:26.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 shutdown_connections 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7d8805f150 0x7f7d88061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7da40acec0 0x7f7da414c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7da40caab0 0x7f7da40ab7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 --2- 172.21.15.73:0/4285802640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7da40cae90 0x7f7da40c34c0 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 >> 172.21.15.73:0/4285802640 conn(0x7f7da4094920 msgr2=0x7f7da40c0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 shutdown_connections 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.461+0000 7f7dab348700 1 -- 172.21.15.73:0/4285802640 wait complete. 2023-11-28T03:11:26.466 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T03:11:26.473 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T03:11:25.764264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2427264,"kb_used_data":13632,"kb_used_omap":0,"kb_used_meta":2413568,"kb_avail":752547456,"statfs":{"total":773094113280,"available":770608594944,"internally_reserved":0,"allocated":13959168,"data_stored":6947228,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2471493632},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-17159,"num_objects":-5,"num_object_clones":0,"num_object_copies":-10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-28,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.587271"},"pg_stats":[{"pgid":"18.3c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772258+0000","last_change":"2023-11-28T03:11:18.767758+0000","last_active":"2023-11-28T03:11:18.772258+0000","last_peered":"2023-11-28T03:11:18.772258+0000","last_clean":"2023-11-28T03:11:18.772258+0000","last_became_active":"2023-11-28T03:11:18.767501+0000","last_became_peered":"2023-11-28T03:11:18.767501+0000","last_unstale":"2023-11-28T03:11:18.772258+0000","last_undegraded":"2023-11-28T03:11:18.772258+0000","last_fullsized":"2023-11-28T03:11:18.772258+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3f","version":"139'3","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766514+0000","last_change":"2023-11-28T03:11:18.385568+0000","last_active":"2023-11-28T03:11:18.766514+0000","last_peered":"2023-11-28T03:11:18.766514+0000","last_clean":"2023-11-28T03:11:18.766514+0000","last_became_active":"2023-11-28T03:11:18.385211+0000","last_became_peered":"2023-11-28T03:11:18.385211+0000","last_unstale":"2023-11-28T03:11:18.766514+0000","last_undegraded":"2023-11-28T03:11:18.766514+0000","last_fullsized":"2023-11-28T03:11:18.766514+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769080+0000","last_change":"2023-11-28T03:11:18.759718+0000","last_active":"2023-11-28T03:11:18.769080+0000","last_peered":"2023-11-28T03:11:18.769080+0000","last_clean":"2023-11-28T03:11:18.769080+0000","last_became_active":"2023-11-28T03:11:18.759448+0000","last_became_peered":"2023-11-28T03:11:18.759448+0000","last_unstale":"2023-11-28T03:11:18.769080+0000","last_undegraded":"2023-11-28T03:11:18.769080+0000","last_fullsized":"2023-11-28T03:11:18.769080+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752440+0000","last_change":"2023-11-28T03:11:18.389741+0000","last_active":"2023-11-28T03:11:18.752440+0000","last_peered":"2023-11-28T03:11:18.752440+0000","last_clean":"2023-11-28T03:11:18.752440+0000","last_became_active":"2023-11-28T03:11:18.389440+0000","last_became_peered":"2023-11-28T03:11:18.389440+0000","last_unstale":"2023-11-28T03:11:18.752440+0000","last_undegraded":"2023-11-28T03:11:18.752440+0000","last_fullsized":"2023-11-28T03:11:18.752440+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.3e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771789+0000","last_change":"2023-11-28T03:11:18.768823+0000","last_active":"2023-11-28T03:11:18.771789+0000","last_peered":"2023-11-28T03:11:18.771789+0000","last_clean":"2023-11-28T03:11:18.771789+0000","last_became_active":"2023-11-28T03:11:18.766758+0000","last_became_peered":"2023-11-28T03:11:18.766758+0000","last_unstale":"2023-11-28T03:11:18.771789+0000","last_undegraded":"2023-11-28T03:11:18.771789+0000","last_fullsized":"2023-11-28T03:11:18.771789+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759729+0000","last_change":"2023-11-28T03:11:18.394730+0000","last_active":"2023-11-28T03:11:18.759729+0000","last_peered":"2023-11-28T03:11:18.759729+0000","last_clean":"2023-11-28T03:11:18.759729+0000","last_became_active":"2023-11-28T03:11:18.394473+0000","last_became_peered":"2023-11-28T03:11:18.394473+0000","last_unstale":"2023-11-28T03:11:18.759729+0000","last_undegraded":"2023-11-28T03:11:18.759729+0000","last_fullsized":"2023-11-28T03:11:18.759729+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.767016+0000","last_change":"2023-11-28T03:11:18.759965+0000","last_active":"2023-11-28T03:11:18.767016+0000","last_peered":"2023-11-28T03:11:18.767016+0000","last_clean":"2023-11-28T03:11:18.767016+0000","last_became_active":"2023-11-28T03:11:18.759675+0000","last_became_peered":"2023-11-28T03:11:18.759675+0000","last_unstale":"2023-11-28T03:11:18.767016+0000","last_undegraded":"2023-11-28T03:11:18.767016+0000","last_fullsized":"2023-11-28T03:11:18.767016+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3c","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.767711+0000","last_change":"2023-11-28T03:11:18.390367+0000","last_active":"2023-11-28T03:11:18.767711+0000","last_peered":"2023-11-28T03:11:18.767711+0000","last_clean":"2023-11-28T03:11:18.767711+0000","last_became_active":"2023-11-28T03:11:18.390086+0000","last_became_peered":"2023-11-28T03:11:18.390086+0000","last_unstale":"2023-11-28T03:11:18.767711+0000","last_undegraded":"2023-11-28T03:11:18.767711+0000","last_fullsized":"2023-11-28T03:11:18.767711+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.38","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769326+0000","last_change":"2023-11-28T03:11:18.757382+0000","last_active":"2023-11-28T03:11:18.769326+0000","last_peered":"2023-11-28T03:11:18.769326+0000","last_clean":"2023-11-28T03:11:18.769326+0000","last_became_active":"2023-11-28T03:11:18.756962+0000","last_became_peered":"2023-11-28T03:11:18.756962+0000","last_unstale":"2023-11-28T03:11:18.769326+0000","last_undegraded":"2023-11-28T03:11:18.769326+0000","last_fullsized":"2023-11-28T03:11:18.769326+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3b","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770839+0000","last_change":"2023-11-28T03:11:18.385649+0000","last_active":"2023-11-28T03:11:18.770839+0000","last_peered":"2023-11-28T03:11:18.770839+0000","last_clean":"2023-11-28T03:11:18.770839+0000","last_became_active":"2023-11-28T03:11:18.385386+0000","last_became_peered":"2023-11-28T03:11:18.385386+0000","last_unstale":"2023-11-28T03:11:18.770839+0000","last_undegraded":"2023-11-28T03:11:18.770839+0000","last_fullsized":"2023-11-28T03:11:18.770839+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.39","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772057+0000","last_change":"2023-11-28T03:11:18.767971+0000","last_active":"2023-11-28T03:11:18.772057+0000","last_peered":"2023-11-28T03:11:18.772057+0000","last_clean":"2023-11-28T03:11:18.772057+0000","last_became_active":"2023-11-28T03:11:18.767753+0000","last_became_peered":"2023-11-28T03:11:18.767753+0000","last_unstale":"2023-11-28T03:11:18.772057+0000","last_undegraded":"2023-11-28T03:11:18.772057+0000","last_fullsized":"2023-11-28T03:11:18.772057+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3a","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769975+0000","last_change":"2023-11-28T03:11:18.392909+0000","last_active":"2023-11-28T03:11:18.769975+0000","last_peered":"2023-11-28T03:11:18.769975+0000","last_clean":"2023-11-28T03:11:18.769975+0000","last_became_active":"2023-11-28T03:11:18.392552+0000","last_became_peered":"2023-11-28T03:11:18.392552+0000","last_unstale":"2023-11-28T03:11:18.769975+0000","last_undegraded":"2023-11-28T03:11:18.769975+0000","last_fullsized":"2023-11-28T03:11:18.769975+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.3a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772137+0000","last_change":"2023-11-28T03:11:18.770341+0000","last_active":"2023-11-28T03:11:18.772137+0000","last_peered":"2023-11-28T03:11:18.772137+0000","last_clean":"2023-11-28T03:11:18.772137+0000","last_became_active":"2023-11-28T03:11:18.770073+0000","last_became_peered":"2023-11-28T03:11:18.770073+0000","last_unstale":"2023-11-28T03:11:18.772137+0000","last_undegraded":"2023-11-28T03:11:18.772137+0000","last_fullsized":"2023-11-28T03:11:18.772137+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.747144+0000","last_change":"2023-11-28T03:11:18.389689+0000","last_active":"2023-11-28T03:11:18.747144+0000","last_peered":"2023-11-28T03:11:18.747144+0000","last_clean":"2023-11-28T03:11:18.747144+0000","last_became_active":"2023-11-28T03:11:18.388755+0000","last_became_peered":"2023-11-28T03:11:18.388755+0000","last_unstale":"2023-11-28T03:11:18.747144+0000","last_undegraded":"2023-11-28T03:11:18.747144+0000","last_fullsized":"2023-11-28T03:11:18.747144+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.3b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766915+0000","last_change":"2023-11-28T03:11:18.757695+0000","last_active":"2023-11-28T03:11:18.766915+0000","last_peered":"2023-11-28T03:11:18.766915+0000","last_clean":"2023-11-28T03:11:18.766915+0000","last_became_active":"2023-11-28T03:11:18.757427+0000","last_became_peered":"2023-11-28T03:11:18.757427+0000","last_unstale":"2023-11-28T03:11:18.766915+0000","last_undegraded":"2023-11-28T03:11:18.766915+0000","last_fullsized":"2023-11-28T03:11:18.766915+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.758623+0000","last_change":"2023-11-28T03:11:18.391234+0000","last_active":"2023-11-28T03:11:18.758623+0000","last_peered":"2023-11-28T03:11:18.758623+0000","last_clean":"2023-11-28T03:11:18.758623+0000","last_became_active":"2023-11-28T03:11:18.390855+0000","last_became_peered":"2023-11-28T03:11:18.390855+0000","last_unstale":"2023-11-28T03:11:18.758623+0000","last_undegraded":"2023-11-28T03:11:18.758623+0000","last_fullsized":"2023-11-28T03:11:18.758623+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.34","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771882+0000","last_change":"2023-11-28T03:11:18.765622+0000","last_active":"2023-11-28T03:11:18.771882+0000","last_peered":"2023-11-28T03:11:18.771882+0000","last_clean":"2023-11-28T03:11:18.771882+0000","last_became_active":"2023-11-28T03:11:18.765104+0000","last_became_peered":"2023-11-28T03:11:18.765104+0000","last_unstale":"2023-11-28T03:11:18.771882+0000","last_undegraded":"2023-11-28T03:11:18.771882+0000","last_fullsized":"2023-11-28T03:11:18.771882+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753315+0000","last_change":"2023-11-28T03:11:18.385956+0000","last_active":"2023-11-28T03:11:18.753315+0000","last_peered":"2023-11-28T03:11:18.753315+0000","last_clean":"2023-11-28T03:11:18.753315+0000","last_became_active":"2023-11-28T03:11:18.385634+0000","last_became_peered":"2023-11-28T03:11:18.385634+0000","last_unstale":"2023-11-28T03:11:18.753315+0000","last_undegraded":"2023-11-28T03:11:18.753315+0000","last_fullsized":"2023-11-28T03:11:18.753315+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.35","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.767077+0000","last_change":"2023-11-28T03:11:18.761511+0000","last_active":"2023-11-28T03:11:18.767077+0000","last_peered":"2023-11-28T03:11:18.767077+0000","last_clean":"2023-11-28T03:11:18.767077+0000","last_became_active":"2023-11-28T03:11:18.761234+0000","last_became_peered":"2023-11-28T03:11:18.761234+0000","last_unstale":"2023-11-28T03:11:18.767077+0000","last_undegraded":"2023-11-28T03:11:18.767077+0000","last_fullsized":"2023-11-28T03:11:18.767077+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749457+0000","last_change":"2023-11-28T03:11:18.384000+0000","last_active":"2023-11-28T03:11:18.749457+0000","last_peered":"2023-11-28T03:11:18.749457+0000","last_clean":"2023-11-28T03:11:18.749457+0000","last_became_active":"2023-11-28T03:11:18.383737+0000","last_became_peered":"2023-11-28T03:11:18.383737+0000","last_unstale":"2023-11-28T03:11:18.749457+0000","last_undegraded":"2023-11-28T03:11:18.749457+0000","last_fullsized":"2023-11-28T03:11:18.749457+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.36","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768433+0000","last_change":"2023-11-28T03:11:18.761053+0000","last_active":"2023-11-28T03:11:18.768433+0000","last_peered":"2023-11-28T03:11:18.768433+0000","last_clean":"2023-11-28T03:11:18.768433+0000","last_became_active":"2023-11-28T03:11:18.760757+0000","last_became_peered":"2023-11-28T03:11:18.760757+0000","last_unstale":"2023-11-28T03:11:18.768433+0000","last_undegraded":"2023-11-28T03:11:18.768433+0000","last_fullsized":"2023-11-28T03:11:18.768433+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752764+0000","last_change":"2023-11-28T03:11:18.386918+0000","last_active":"2023-11-28T03:11:18.752764+0000","last_peered":"2023-11-28T03:11:18.752764+0000","last_clean":"2023-11-28T03:11:18.752764+0000","last_became_active":"2023-11-28T03:11:18.386386+0000","last_became_peered":"2023-11-28T03:11:18.386386+0000","last_unstale":"2023-11-28T03:11:18.752764+0000","last_undegraded":"2023-11-28T03:11:18.752764+0000","last_fullsized":"2023-11-28T03:11:18.752764+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.37","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773822+0000","last_change":"2023-11-28T03:11:18.772062+0000","last_active":"2023-11-28T03:11:18.773822+0000","last_peered":"2023-11-28T03:11:18.773822+0000","last_clean":"2023-11-28T03:11:18.773822+0000","last_became_active":"2023-11-28T03:11:18.771857+0000","last_became_peered":"2023-11-28T03:11:18.771857+0000","last_unstale":"2023-11-28T03:11:18.773822+0000","last_undegraded":"2023-11-28T03:11:18.773822+0000","last_fullsized":"2023-11-28T03:11:18.773822+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.34","version":"139'4","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.777698+0000","last_change":"2023-11-28T03:11:18.384007+0000","last_active":"2023-11-28T03:11:18.777698+0000","last_peered":"2023-11-28T03:11:18.777698+0000","last_clean":"2023-11-28T03:11:18.777698+0000","last_became_active":"2023-11-28T03:11:18.383671+0000","last_became_peered":"2023-11-28T03:11:18.383671+0000","last_unstale":"2023-11-28T03:11:18.777698+0000","last_undegraded":"2023-11-28T03:11:18.777698+0000","last_fullsized":"2023-11-28T03:11:18.777698+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.30","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772120+0000","last_change":"2023-11-28T03:11:18.768400+0000","last_active":"2023-11-28T03:11:18.772120+0000","last_peered":"2023-11-28T03:11:18.772120+0000","last_clean":"2023-11-28T03:11:18.772120+0000","last_became_active":"2023-11-28T03:11:18.767686+0000","last_became_peered":"2023-11-28T03:11:18.767686+0000","last_unstale":"2023-11-28T03:11:18.772120+0000","last_undegraded":"2023-11-28T03:11:18.772120+0000","last_fullsized":"2023-11-28T03:11:18.772120+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768055+0000","last_change":"2023-11-28T03:11:18.391315+0000","last_active":"2023-11-28T03:11:18.768055+0000","last_peered":"2023-11-28T03:11:18.768055+0000","last_clean":"2023-11-28T03:11:18.768055+0000","last_became_active":"2023-11-28T03:11:18.391053+0000","last_became_peered":"2023-11-28T03:11:18.391053+0000","last_unstale":"2023-11-28T03:11:18.768055+0000","last_undegraded":"2023-11-28T03:11:18.768055+0000","last_fullsized":"2023-11-28T03:11:18.768055+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.31","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.766495+0000","last_change":"2023-11-28T03:11:18.756179+0000","last_active":"2023-11-28T03:11:18.766495+0000","last_peered":"2023-11-28T03:11:18.766495+0000","last_clean":"2023-11-28T03:11:18.766495+0000","last_became_active":"2023-11-28T03:11:18.755853+0000","last_became_peered":"2023-11-28T03:11:18.755853+0000","last_unstale":"2023-11-28T03:11:18.766495+0000","last_undegraded":"2023-11-28T03:11:18.766495+0000","last_fullsized":"2023-11-28T03:11:18.766495+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757592+0000","last_change":"2023-11-28T03:11:18.391280+0000","last_active":"2023-11-28T03:11:18.757592+0000","last_peered":"2023-11-28T03:11:18.757592+0000","last_clean":"2023-11-28T03:11:18.757592+0000","last_became_active":"2023-11-28T03:11:18.387894+0000","last_became_peered":"2023-11-28T03:11:18.387894+0000","last_unstale":"2023-11-28T03:11:18.757592+0000","last_undegraded":"2023-11-28T03:11:18.757592+0000","last_fullsized":"2023-11-28T03:11:18.757592+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.32","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769664+0000","last_change":"2023-11-28T03:11:18.764061+0000","last_active":"2023-11-28T03:11:18.769664+0000","last_peered":"2023-11-28T03:11:18.769664+0000","last_clean":"2023-11-28T03:11:18.769664+0000","last_became_active":"2023-11-28T03:11:18.758596+0000","last_became_peered":"2023-11-28T03:11:18.758596+0000","last_unstale":"2023-11-28T03:11:18.769664+0000","last_undegraded":"2023-11-28T03:11:18.769664+0000","last_fullsized":"2023-11-28T03:11:18.769664+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.750431+0000","last_change":"2023-11-28T03:11:18.392016+0000","last_active":"2023-11-28T03:11:18.750431+0000","last_peered":"2023-11-28T03:11:18.750431+0000","last_clean":"2023-11-28T03:11:18.750431+0000","last_became_active":"2023-11-28T03:11:18.391714+0000","last_became_peered":"2023-11-28T03:11:18.391714+0000","last_unstale":"2023-11-28T03:11:18.750431+0000","last_undegraded":"2023-11-28T03:11:18.750431+0000","last_fullsized":"2023-11-28T03:11:18.750431+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.33","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762898+0000","last_change":"2023-11-28T03:11:18.755924+0000","last_active":"2023-11-28T03:11:18.762898+0000","last_peered":"2023-11-28T03:11:18.762898+0000","last_clean":"2023-11-28T03:11:18.762898+0000","last_became_active":"2023-11-28T03:11:18.755604+0000","last_became_peered":"2023-11-28T03:11:18.755604+0000","last_unstale":"2023-11-28T03:11:18.762898+0000","last_undegraded":"2023-11-28T03:11:18.762898+0000","last_fullsized":"2023-11-28T03:11:18.762898+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"139'4","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769227+0000","last_change":"2023-11-28T03:11:18.396374+0000","last_active":"2023-11-28T03:11:18.769227+0000","last_peered":"2023-11-28T03:11:18.769227+0000","last_clean":"2023-11-28T03:11:18.769227+0000","last_became_active":"2023-11-28T03:11:18.396165+0000","last_became_peered":"2023-11-28T03:11:18.396165+0000","last_unstale":"2023-11-28T03:11:18.769227+0000","last_undegraded":"2023-11-28T03:11:18.769227+0000","last_fullsized":"2023-11-28T03:11:18.769227+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.765993+0000","last_change":"2023-11-28T03:11:18.764664+0000","last_active":"2023-11-28T03:11:18.765993+0000","last_peered":"2023-11-28T03:11:18.765993+0000","last_clean":"2023-11-28T03:11:18.765993+0000","last_became_active":"2023-11-28T03:11:18.764391+0000","last_became_peered":"2023-11-28T03:11:18.764391+0000","last_unstale":"2023-11-28T03:11:18.765993+0000","last_undegraded":"2023-11-28T03:11:18.765993+0000","last_fullsized":"2023-11-28T03:11:18.765993+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753573+0000","last_change":"2023-11-28T03:11:18.380395+0000","last_active":"2023-11-28T03:11:18.753573+0000","last_peered":"2023-11-28T03:11:18.753573+0000","last_clean":"2023-11-28T03:11:18.753573+0000","last_became_active":"2023-11-28T03:11:18.379748+0000","last_became_peered":"2023-11-28T03:11:18.379748+0000","last_unstale":"2023-11-28T03:11:18.753573+0000","last_undegraded":"2023-11-28T03:11:18.753573+0000","last_fullsized":"2023-11-28T03:11:18.753573+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770964+0000","last_change":"2023-11-28T03:11:18.764145+0000","last_active":"2023-11-28T03:11:18.770964+0000","last_peered":"2023-11-28T03:11:18.770964+0000","last_clean":"2023-11-28T03:11:18.770964+0000","last_became_active":"2023-11-28T03:11:18.763898+0000","last_became_peered":"2023-11-28T03:11:18.763898+0000","last_unstale":"2023-11-28T03:11:18.770964+0000","last_undegraded":"2023-11-28T03:11:18.770964+0000","last_fullsized":"2023-11-28T03:11:18.770964+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749296+0000","last_change":"2023-11-28T03:11:18.382167+0000","last_active":"2023-11-28T03:11:18.749296+0000","last_peered":"2023-11-28T03:11:18.749296+0000","last_clean":"2023-11-28T03:11:18.749296+0000","last_became_active":"2023-11-28T03:11:18.381842+0000","last_became_peered":"2023-11-28T03:11:18.381842+0000","last_unstale":"2023-11-28T03:11:18.749296+0000","last_undegraded":"2023-11-28T03:11:18.749296+0000","last_fullsized":"2023-11-28T03:11:18.749296+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.2e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770082+0000","last_change":"2023-11-28T03:11:18.767015+0000","last_active":"2023-11-28T03:11:18.770082+0000","last_peered":"2023-11-28T03:11:18.770082+0000","last_clean":"2023-11-28T03:11:18.770082+0000","last_became_active":"2023-11-28T03:11:18.766723+0000","last_became_peered":"2023-11-28T03:11:18.766723+0000","last_unstale":"2023-11-28T03:11:18.770082+0000","last_undegraded":"2023-11-28T03:11:18.770082+0000","last_fullsized":"2023-11-28T03:11:18.770082+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768229+0000","last_change":"2023-11-28T03:11:18.400594+0000","last_active":"2023-11-28T03:11:18.768229+0000","last_peered":"2023-11-28T03:11:18.768229+0000","last_clean":"2023-11-28T03:11:18.768229+0000","last_became_active":"2023-11-28T03:11:18.400354+0000","last_became_peered":"2023-11-28T03:11:18.400354+0000","last_unstale":"2023-11-28T03:11:18.768229+0000","last_undegraded":"2023-11-28T03:11:18.768229+0000","last_fullsized":"2023-11-28T03:11:18.768229+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.2f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763673+0000","last_change":"2023-11-28T03:11:18.755965+0000","last_active":"2023-11-28T03:11:18.763673+0000","last_peered":"2023-11-28T03:11:18.763673+0000","last_clean":"2023-11-28T03:11:18.763673+0000","last_became_active":"2023-11-28T03:11:18.755630+0000","last_became_peered":"2023-11-28T03:11:18.755630+0000","last_unstale":"2023-11-28T03:11:18.763673+0000","last_undegraded":"2023-11-28T03:11:18.763673+0000","last_fullsized":"2023-11-28T03:11:18.763673+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752407+0000","last_change":"2023-11-28T03:11:18.387489+0000","last_active":"2023-11-28T03:11:18.752407+0000","last_peered":"2023-11-28T03:11:18.752407+0000","last_clean":"2023-11-28T03:11:18.752407+0000","last_became_active":"2023-11-28T03:11:18.386665+0000","last_became_peered":"2023-11-28T03:11:18.386665+0000","last_unstale":"2023-11-28T03:11:18.752407+0000","last_undegraded":"2023-11-28T03:11:18.752407+0000","last_fullsized":"2023-11-28T03:11:18.752407+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.28","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760828+0000","last_change":"2023-11-28T03:11:18.755670+0000","last_active":"2023-11-28T03:11:18.760828+0000","last_peered":"2023-11-28T03:11:18.760828+0000","last_clean":"2023-11-28T03:11:18.760828+0000","last_became_active":"2023-11-28T03:11:18.755386+0000","last_became_peered":"2023-11-28T03:11:18.755386+0000","last_unstale":"2023-11-28T03:11:18.760828+0000","last_undegraded":"2023-11-28T03:11:18.760828+0000","last_fullsized":"2023-11-28T03:11:18.760828+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.2b","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770950+0000","last_change":"2023-11-28T03:11:18.391627+0000","last_active":"2023-11-28T03:11:18.770950+0000","last_peered":"2023-11-28T03:11:18.770950+0000","last_clean":"2023-11-28T03:11:18.770950+0000","last_became_active":"2023-11-28T03:11:18.391309+0000","last_became_peered":"2023-11-28T03:11:18.391309+0000","last_unstale":"2023-11-28T03:11:18.770950+0000","last_undegraded":"2023-11-28T03:11:18.770950+0000","last_fullsized":"2023-11-28T03:11:18.770950+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.29","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773811+0000","last_change":"2023-11-28T03:11:18.771621+0000","last_active":"2023-11-28T03:11:18.773811+0000","last_peered":"2023-11-28T03:11:18.773811+0000","last_clean":"2023-11-28T03:11:18.773811+0000","last_became_active":"2023-11-28T03:11:18.771317+0000","last_became_peered":"2023-11-28T03:11:18.771317+0000","last_unstale":"2023-11-28T03:11:18.773811+0000","last_undegraded":"2023-11-28T03:11:18.773811+0000","last_fullsized":"2023-11-28T03:11:18.773811+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.745132+0000","last_change":"2023-11-28T03:11:18.391689+0000","last_active":"2023-11-28T03:11:18.745132+0000","last_peered":"2023-11-28T03:11:18.745132+0000","last_clean":"2023-11-28T03:11:18.745132+0000","last_became_active":"2023-11-28T03:11:18.390946+0000","last_became_peered":"2023-11-28T03:11:18.390946+0000","last_unstale":"2023-11-28T03:11:18.745132+0000","last_undegraded":"2023-11-28T03:11:18.745132+0000","last_fullsized":"2023-11-28T03:11:18.745132+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.2a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772627+0000","last_change":"2023-11-28T03:11:18.771494+0000","last_active":"2023-11-28T03:11:18.772627+0000","last_peered":"2023-11-28T03:11:18.772627+0000","last_clean":"2023-11-28T03:11:18.772627+0000","last_became_active":"2023-11-28T03:11:18.771108+0000","last_became_peered":"2023-11-28T03:11:18.771108+0000","last_unstale":"2023-11-28T03:11:18.772627+0000","last_undegraded":"2023-11-28T03:11:18.772627+0000","last_fullsized":"2023-11-28T03:11:18.772627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752523+0000","last_change":"2023-11-28T03:11:18.386639+0000","last_active":"2023-11-28T03:11:18.752523+0000","last_peered":"2023-11-28T03:11:18.752523+0000","last_clean":"2023-11-28T03:11:18.752523+0000","last_became_active":"2023-11-28T03:11:18.386342+0000","last_became_peered":"2023-11-28T03:11:18.386342+0000","last_unstale":"2023-11-28T03:11:18.752523+0000","last_undegraded":"2023-11-28T03:11:18.752523+0000","last_fullsized":"2023-11-28T03:11:18.752523+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771510+0000","last_change":"2023-11-28T03:11:18.767028+0000","last_active":"2023-11-28T03:11:18.771510+0000","last_peered":"2023-11-28T03:11:18.771510+0000","last_clean":"2023-11-28T03:11:18.771510+0000","last_became_active":"2023-11-28T03:11:18.766789+0000","last_became_peered":"2023-11-28T03:11:18.766789+0000","last_unstale":"2023-11-28T03:11:18.771510+0000","last_undegraded":"2023-11-28T03:11:18.771510+0000","last_fullsized":"2023-11-28T03:11:18.771510+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.743864+0000","last_change":"2023-11-28T03:11:18.395025+0000","last_active":"2023-11-28T03:11:18.743864+0000","last_peered":"2023-11-28T03:11:18.743864+0000","last_clean":"2023-11-28T03:11:18.743864+0000","last_became_active":"2023-11-28T03:11:18.394807+0000","last_became_peered":"2023-11-28T03:11:18.394807+0000","last_unstale":"2023-11-28T03:11:18.743864+0000","last_undegraded":"2023-11-28T03:11:18.743864+0000","last_fullsized":"2023-11-28T03:11:18.743864+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.24","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769421+0000","last_change":"2023-11-28T03:11:18.763610+0000","last_active":"2023-11-28T03:11:18.769421+0000","last_peered":"2023-11-28T03:11:18.769421+0000","last_clean":"2023-11-28T03:11:18.769421+0000","last_became_active":"2023-11-28T03:11:18.762947+0000","last_became_peered":"2023-11-28T03:11:18.762947+0000","last_unstale":"2023-11-28T03:11:18.769421+0000","last_undegraded":"2023-11-28T03:11:18.769421+0000","last_fullsized":"2023-11-28T03:11:18.769421+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753477+0000","last_change":"2023-11-28T03:11:18.380585+0000","last_active":"2023-11-28T03:11:18.753477+0000","last_peered":"2023-11-28T03:11:18.753477+0000","last_clean":"2023-11-28T03:11:18.753477+0000","last_became_active":"2023-11-28T03:11:18.380095+0000","last_became_peered":"2023-11-28T03:11:18.380095+0000","last_unstale":"2023-11-28T03:11:18.753477+0000","last_undegraded":"2023-11-28T03:11:18.753477+0000","last_fullsized":"2023-11-28T03:11:18.753477+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.25","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770983+0000","last_change":"2023-11-28T03:11:18.766659+0000","last_active":"2023-11-28T03:11:18.770983+0000","last_peered":"2023-11-28T03:11:18.770983+0000","last_clean":"2023-11-28T03:11:18.770983+0000","last_became_active":"2023-11-28T03:11:18.766277+0000","last_became_peered":"2023-11-28T03:11:18.766277+0000","last_unstale":"2023-11-28T03:11:18.770983+0000","last_undegraded":"2023-11-28T03:11:18.770983+0000","last_fullsized":"2023-11-28T03:11:18.770983+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749546+0000","last_change":"2023-11-28T03:11:18.387312+0000","last_active":"2023-11-28T03:11:18.749546+0000","last_peered":"2023-11-28T03:11:18.749546+0000","last_clean":"2023-11-28T03:11:18.749546+0000","last_became_active":"2023-11-28T03:11:18.386958+0000","last_became_peered":"2023-11-28T03:11:18.386958+0000","last_unstale":"2023-11-28T03:11:18.749546+0000","last_undegraded":"2023-11-28T03:11:18.749546+0000","last_fullsized":"2023-11-28T03:11:18.749546+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.26","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.765121+0000","last_change":"2023-11-28T03:11:18.759723+0000","last_active":"2023-11-28T03:11:18.765121+0000","last_peered":"2023-11-28T03:11:18.765121+0000","last_clean":"2023-11-28T03:11:18.765121+0000","last_became_active":"2023-11-28T03:11:18.759456+0000","last_became_peered":"2023-11-28T03:11:18.759456+0000","last_unstale":"2023-11-28T03:11:18.765121+0000","last_undegraded":"2023-11-28T03:11:18.765121+0000","last_fullsized":"2023-11-28T03:11:18.765121+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.750038+0000","last_change":"2023-11-28T03:11:18.386227+0000","last_active":"2023-11-28T03:11:18.750038+0000","last_peered":"2023-11-28T03:11:18.750038+0000","last_clean":"2023-11-28T03:11:18.750038+0000","last_became_active":"2023-11-28T03:11:18.385925+0000","last_became_peered":"2023-11-28T03:11:18.385925+0000","last_unstale":"2023-11-28T03:11:18.750038+0000","last_undegraded":"2023-11-28T03:11:18.750038+0000","last_fullsized":"2023-11-28T03:11:18.750038+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.27","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.761587+0000","last_change":"2023-11-28T03:11:18.757939+0000","last_active":"2023-11-28T03:11:18.761587+0000","last_peered":"2023-11-28T03:11:18.761587+0000","last_clean":"2023-11-28T03:11:18.761587+0000","last_became_active":"2023-11-28T03:11:18.756692+0000","last_became_peered":"2023-11-28T03:11:18.756692+0000","last_unstale":"2023-11-28T03:11:18.761587+0000","last_undegraded":"2023-11-28T03:11:18.761587+0000","last_fullsized":"2023-11-28T03:11:18.761587+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.745448+0000","last_change":"2023-11-28T03:11:18.386046+0000","last_active":"2023-11-28T03:11:18.745448+0000","last_peered":"2023-11-28T03:11:18.745448+0000","last_clean":"2023-11-28T03:11:18.745448+0000","last_became_active":"2023-11-28T03:11:18.385393+0000","last_became_peered":"2023-11-28T03:11:18.385393+0000","last_unstale":"2023-11-28T03:11:18.745448+0000","last_undegraded":"2023-11-28T03:11:18.745448+0000","last_fullsized":"2023-11-28T03:11:18.745448+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.20","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760806+0000","last_change":"2023-11-28T03:11:18.755485+0000","last_active":"2023-11-28T03:11:18.760806+0000","last_peered":"2023-11-28T03:11:18.760806+0000","last_clean":"2023-11-28T03:11:18.760806+0000","last_became_active":"2023-11-28T03:11:18.755182+0000","last_became_peered":"2023-11-28T03:11:18.755182+0000","last_unstale":"2023-11-28T03:11:18.760806+0000","last_undegraded":"2023-11-28T03:11:18.760806+0000","last_fullsized":"2023-11-28T03:11:18.760806+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752201+0000","last_change":"2023-11-28T03:11:18.396889+0000","last_active":"2023-11-28T03:11:18.752201+0000","last_peered":"2023-11-28T03:11:18.752201+0000","last_clean":"2023-11-28T03:11:18.752201+0000","last_became_active":"2023-11-28T03:11:18.396619+0000","last_became_peered":"2023-11-28T03:11:18.396619+0000","last_unstale":"2023-11-28T03:11:18.752201+0000","last_undegraded":"2023-11-28T03:11:18.752201+0000","last_fullsized":"2023-11-28T03:11:18.752201+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.21","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764479+0000","last_change":"2023-11-28T03:11:18.761438+0000","last_active":"2023-11-28T03:11:18.764479+0000","last_peered":"2023-11-28T03:11:18.764479+0000","last_clean":"2023-11-28T03:11:18.764479+0000","last_became_active":"2023-11-28T03:11:18.760711+0000","last_became_peered":"2023-11-28T03:11:18.760711+0000","last_unstale":"2023-11-28T03:11:18.764479+0000","last_undegraded":"2023-11-28T03:11:18.764479+0000","last_fullsized":"2023-11-28T03:11:18.764479+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.746654+0000","last_change":"2023-11-28T03:11:18.396051+0000","last_active":"2023-11-28T03:11:18.746654+0000","last_peered":"2023-11-28T03:11:18.746654+0000","last_clean":"2023-11-28T03:11:18.746654+0000","last_became_active":"2023-11-28T03:11:18.395777+0000","last_became_peered":"2023-11-28T03:11:18.395777+0000","last_unstale":"2023-11-28T03:11:18.746654+0000","last_undegraded":"2023-11-28T03:11:18.746654+0000","last_fullsized":"2023-11-28T03:11:18.746654+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.22","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769680+0000","last_change":"2023-11-28T03:11:18.764724+0000","last_active":"2023-11-28T03:11:18.769680+0000","last_peered":"2023-11-28T03:11:18.769680+0000","last_clean":"2023-11-28T03:11:18.769680+0000","last_became_active":"2023-11-28T03:11:18.764373+0000","last_became_peered":"2023-11-28T03:11:18.764373+0000","last_unstale":"2023-11-28T03:11:18.769680+0000","last_undegraded":"2023-11-28T03:11:18.769680+0000","last_fullsized":"2023-11-28T03:11:18.769680+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753272+0000","last_change":"2023-11-28T03:11:18.388277+0000","last_active":"2023-11-28T03:11:18.753272+0000","last_peered":"2023-11-28T03:11:18.753272+0000","last_clean":"2023-11-28T03:11:18.753272+0000","last_became_active":"2023-11-28T03:11:18.387116+0000","last_became_peered":"2023-11-28T03:11:18.387116+0000","last_unstale":"2023-11-28T03:11:18.753272+0000","last_undegraded":"2023-11-28T03:11:18.753272+0000","last_fullsized":"2023-11-28T03:11:18.753272+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.23","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.765772+0000","last_change":"2023-11-28T03:11:18.762845+0000","last_active":"2023-11-28T03:11:18.765772+0000","last_peered":"2023-11-28T03:11:18.765772+0000","last_clean":"2023-11-28T03:11:18.765772+0000","last_became_active":"2023-11-28T03:11:18.761917+0000","last_became_peered":"2023-11-28T03:11:18.761917+0000","last_unstale":"2023-11-28T03:11:18.765772+0000","last_undegraded":"2023-11-28T03:11:18.765772+0000","last_fullsized":"2023-11-28T03:11:18.765772+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749981+0000","last_change":"2023-11-28T03:11:18.388476+0000","last_active":"2023-11-28T03:11:18.749981+0000","last_peered":"2023-11-28T03:11:18.749981+0000","last_clean":"2023-11-28T03:11:18.749981+0000","last_became_active":"2023-11-28T03:11:18.387333+0000","last_became_peered":"2023-11-28T03:11:18.387333+0000","last_unstale":"2023-11-28T03:11:18.749981+0000","last_undegraded":"2023-11-28T03:11:18.749981+0000","last_fullsized":"2023-11-28T03:11:18.749981+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768963+0000","last_change":"2023-11-28T03:11:18.765317+0000","last_active":"2023-11-28T03:11:18.768963+0000","last_peered":"2023-11-28T03:11:18.768963+0000","last_clean":"2023-11-28T03:11:18.768963+0000","last_became_active":"2023-11-28T03:11:18.764904+0000","last_became_peered":"2023-11-28T03:11:18.764904+0000","last_unstale":"2023-11-28T03:11:18.768963+0000","last_undegraded":"2023-11-28T03:11:18.768963+0000","last_fullsized":"2023-11-28T03:11:18.768963+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1f","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763213+0000","last_change":"2023-11-28T03:11:18.395421+0000","last_active":"2023-11-28T03:11:18.763213+0000","last_peered":"2023-11-28T03:11:18.763213+0000","last_clean":"2023-11-28T03:11:18.763213+0000","last_became_active":"2023-11-28T03:11:18.394055+0000","last_became_peered":"2023-11-28T03:11:18.394055+0000","last_unstale":"2023-11-28T03:11:18.763213+0000","last_undegraded":"2023-11-28T03:11:18.763213+0000","last_fullsized":"2023-11-28T03:11:18.763213+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769313+0000","last_change":"2023-11-28T03:11:18.762565+0000","last_active":"2023-11-28T03:11:18.769313+0000","last_peered":"2023-11-28T03:11:18.769313+0000","last_clean":"2023-11-28T03:11:18.769313+0000","last_became_active":"2023-11-28T03:11:18.762273+0000","last_became_peered":"2023-11-28T03:11:18.762273+0000","last_unstale":"2023-11-28T03:11:18.769313+0000","last_undegraded":"2023-11-28T03:11:18.769313+0000","last_fullsized":"2023-11-28T03:11:18.769313+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1e","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762741+0000","last_change":"2023-11-28T03:11:18.378416+0000","last_active":"2023-11-28T03:11:18.762741+0000","last_peered":"2023-11-28T03:11:18.762741+0000","last_clean":"2023-11-28T03:11:18.762741+0000","last_became_active":"2023-11-28T03:11:18.378168+0000","last_became_peered":"2023-11-28T03:11:18.378168+0000","last_unstale":"2023-11-28T03:11:18.762741+0000","last_undegraded":"2023-11-28T03:11:18.762741+0000","last_fullsized":"2023-11-28T03:11:18.762741+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764109+0000","last_change":"2023-11-28T03:11:18.760275+0000","last_active":"2023-11-28T03:11:18.764109+0000","last_peered":"2023-11-28T03:11:18.764109+0000","last_clean":"2023-11-28T03:11:18.764109+0000","last_became_active":"2023-11-28T03:11:18.759999+0000","last_became_peered":"2023-11-28T03:11:18.759999+0000","last_unstale":"2023-11-28T03:11:18.764109+0000","last_undegraded":"2023-11-28T03:11:18.764109+0000","last_fullsized":"2023-11-28T03:11:18.764109+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1d","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.765919+0000","last_change":"2023-11-28T03:11:18.400805+0000","last_active":"2023-11-28T03:11:18.765919+0000","last_peered":"2023-11-28T03:11:18.765919+0000","last_clean":"2023-11-28T03:11:18.765919+0000","last_became_active":"2023-11-28T03:11:18.400469+0000","last_became_peered":"2023-11-28T03:11:18.400469+0000","last_unstale":"2023-11-28T03:11:18.765919+0000","last_undegraded":"2023-11-28T03:11:18.765919+0000","last_fullsized":"2023-11-28T03:11:18.765919+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773499+0000","last_change":"2023-11-28T03:11:18.768593+0000","last_active":"2023-11-28T03:11:18.773499+0000","last_peered":"2023-11-28T03:11:18.773499+0000","last_clean":"2023-11-28T03:11:18.773499+0000","last_became_active":"2023-11-28T03:11:18.768254+0000","last_became_peered":"2023-11-28T03:11:18.768254+0000","last_unstale":"2023-11-28T03:11:18.773499+0000","last_undegraded":"2023-11-28T03:11:18.773499+0000","last_fullsized":"2023-11-28T03:11:18.773499+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751301+0000","last_change":"2023-11-28T03:11:18.386676+0000","last_active":"2023-11-28T03:11:18.751301+0000","last_peered":"2023-11-28T03:11:18.751301+0000","last_clean":"2023-11-28T03:11:18.751301+0000","last_became_active":"2023-11-28T03:11:18.386406+0000","last_became_peered":"2023-11-28T03:11:18.386406+0000","last_unstale":"2023-11-28T03:11:18.751301+0000","last_undegraded":"2023-11-28T03:11:18.751301+0000","last_fullsized":"2023-11-28T03:11:18.751301+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763042+0000","last_change":"2023-11-28T03:11:18.756315+0000","last_active":"2023-11-28T03:11:18.763042+0000","last_peered":"2023-11-28T03:11:18.763042+0000","last_clean":"2023-11-28T03:11:18.763042+0000","last_became_active":"2023-11-28T03:11:18.755546+0000","last_became_peered":"2023-11-28T03:11:18.755546+0000","last_unstale":"2023-11-28T03:11:18.763042+0000","last_undegraded":"2023-11-28T03:11:18.763042+0000","last_fullsized":"2023-11-28T03:11:18.763042+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752320+0000","last_change":"2023-11-28T03:11:18.390764+0000","last_active":"2023-11-28T03:11:18.752320+0000","last_peered":"2023-11-28T03:11:18.752320+0000","last_clean":"2023-11-28T03:11:18.752320+0000","last_became_active":"2023-11-28T03:11:18.390219+0000","last_became_peered":"2023-11-28T03:11:18.390219+0000","last_unstale":"2023-11-28T03:11:18.752320+0000","last_undegraded":"2023-11-28T03:11:18.752320+0000","last_fullsized":"2023-11-28T03:11:18.752320+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.767379+0000","last_change":"2023-11-28T03:11:18.763422+0000","last_active":"2023-11-28T03:11:18.767379+0000","last_peered":"2023-11-28T03:11:18.767379+0000","last_clean":"2023-11-28T03:11:18.767379+0000","last_became_active":"2023-11-28T03:11:18.763080+0000","last_became_peered":"2023-11-28T03:11:18.763080+0000","last_unstale":"2023-11-28T03:11:18.767379+0000","last_undegraded":"2023-11-28T03:11:18.767379+0000","last_fullsized":"2023-11-28T03:11:18.767379+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748874+0000","last_change":"2023-11-28T03:11:18.395919+0000","last_active":"2023-11-28T03:11:18.748874+0000","last_peered":"2023-11-28T03:11:18.748874+0000","last_clean":"2023-11-28T03:11:18.748874+0000","last_became_active":"2023-11-28T03:11:18.395621+0000","last_became_peered":"2023-11-28T03:11:18.395621+0000","last_unstale":"2023-11-28T03:11:18.748874+0000","last_undegraded":"2023-11-28T03:11:18.748874+0000","last_fullsized":"2023-11-28T03:11:18.748874+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769646+0000","last_change":"2023-11-28T03:11:18.762163+0000","last_active":"2023-11-28T03:11:18.769646+0000","last_peered":"2023-11-28T03:11:18.769646+0000","last_clean":"2023-11-28T03:11:18.769646+0000","last_became_active":"2023-11-28T03:11:18.759034+0000","last_became_peered":"2023-11-28T03:11:18.759034+0000","last_unstale":"2023-11-28T03:11:18.769646+0000","last_undegraded":"2023-11-28T03:11:18.769646+0000","last_fullsized":"2023-11-28T03:11:18.769646+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.746677+0000","last_change":"2023-11-28T03:11:18.389034+0000","last_active":"2023-11-28T03:11:18.746677+0000","last_peered":"2023-11-28T03:11:18.746677+0000","last_clean":"2023-11-28T03:11:18.746677+0000","last_became_active":"2023-11-28T03:11:18.388710+0000","last_became_peered":"2023-11-28T03:11:18.388710+0000","last_unstale":"2023-11-28T03:11:18.746677+0000","last_undegraded":"2023-11-28T03:11:18.746677+0000","last_fullsized":"2023-11-28T03:11:18.746677+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771916+0000","last_change":"2023-11-28T03:11:18.770420+0000","last_active":"2023-11-28T03:11:18.771916+0000","last_peered":"2023-11-28T03:11:18.771916+0000","last_clean":"2023-11-28T03:11:18.771916+0000","last_became_active":"2023-11-28T03:11:18.769842+0000","last_became_peered":"2023-11-28T03:11:18.769842+0000","last_unstale":"2023-11-28T03:11:18.771916+0000","last_undegraded":"2023-11-28T03:11:18.771916+0000","last_fullsized":"2023-11-28T03:11:18.771916+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753991+0000","last_change":"2023-11-28T03:11:18.391283+0000","last_active":"2023-11-28T03:11:18.753991+0000","last_peered":"2023-11-28T03:11:18.753991+0000","last_clean":"2023-11-28T03:11:18.753991+0000","last_became_active":"2023-11-28T03:11:18.390915+0000","last_became_peered":"2023-11-28T03:11:18.390915+0000","last_unstale":"2023-11-28T03:11:18.753991+0000","last_undegraded":"2023-11-28T03:11:18.753991+0000","last_fullsized":"2023-11-28T03:11:18.753991+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773304+0000","last_change":"2023-11-28T03:11:18.765486+0000","last_active":"2023-11-28T03:11:18.773304+0000","last_peered":"2023-11-28T03:11:18.773304+0000","last_clean":"2023-11-28T03:11:18.773304+0000","last_became_active":"2023-11-28T03:11:18.765002+0000","last_became_peered":"2023-11-28T03:11:18.765002+0000","last_unstale":"2023-11-28T03:11:18.773304+0000","last_undegraded":"2023-11-28T03:11:18.773304+0000","last_fullsized":"2023-11-28T03:11:18.773304+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760972+0000","last_change":"2023-11-28T03:11:18.392679+0000","last_active":"2023-11-28T03:11:18.760972+0000","last_peered":"2023-11-28T03:11:18.760972+0000","last_clean":"2023-11-28T03:11:18.760972+0000","last_became_active":"2023-11-28T03:11:18.392366+0000","last_became_peered":"2023-11-28T03:11:18.392366+0000","last_unstale":"2023-11-28T03:11:18.760972+0000","last_undegraded":"2023-11-28T03:11:18.760972+0000","last_fullsized":"2023-11-28T03:11:18.760972+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772612+0000","last_change":"2023-11-28T03:11:18.771452+0000","last_active":"2023-11-28T03:11:18.772612+0000","last_peered":"2023-11-28T03:11:18.772612+0000","last_clean":"2023-11-28T03:11:18.772612+0000","last_became_active":"2023-11-28T03:11:18.771105+0000","last_became_peered":"2023-11-28T03:11:18.771105+0000","last_unstale":"2023-11-28T03:11:18.772612+0000","last_undegraded":"2023-11-28T03:11:18.772612+0000","last_fullsized":"2023-11-28T03:11:18.772612+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.e","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762726+0000","last_change":"2023-11-28T03:11:18.385219+0000","last_active":"2023-11-28T03:11:18.762726+0000","last_peered":"2023-11-28T03:11:18.762726+0000","last_clean":"2023-11-28T03:11:18.762726+0000","last_became_active":"2023-11-28T03:11:18.384858+0000","last_became_peered":"2023-11-28T03:11:18.384858+0000","last_unstale":"2023-11-28T03:11:18.762726+0000","last_undegraded":"2023-11-28T03:11:18.762726+0000","last_fullsized":"2023-11-28T03:11:18.762726+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762241+0000","last_change":"2023-11-28T03:11:18.757640+0000","last_active":"2023-11-28T03:11:18.762241+0000","last_peered":"2023-11-28T03:11:18.762241+0000","last_clean":"2023-11-28T03:11:18.762241+0000","last_became_active":"2023-11-28T03:11:18.757397+0000","last_became_peered":"2023-11-28T03:11:18.757397+0000","last_unstale":"2023-11-28T03:11:18.762241+0000","last_undegraded":"2023-11-28T03:11:18.762241+0000","last_fullsized":"2023-11-28T03:11:18.762241+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760062+0000","last_change":"2023-11-28T03:11:18.395814+0000","last_active":"2023-11-28T03:11:18.760062+0000","last_peered":"2023-11-28T03:11:18.760062+0000","last_clean":"2023-11-28T03:11:18.760062+0000","last_became_active":"2023-11-28T03:11:18.395516+0000","last_became_peered":"2023-11-28T03:11:18.395516+0000","last_unstale":"2023-11-28T03:11:18.760062+0000","last_undegraded":"2023-11-28T03:11:18.760062+0000","last_fullsized":"2023-11-28T03:11:18.760062+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768500+0000","last_change":"2023-11-28T03:11:18.764443+0000","last_active":"2023-11-28T03:11:18.768500+0000","last_peered":"2023-11-28T03:11:18.768500+0000","last_clean":"2023-11-28T03:11:18.768500+0000","last_became_active":"2023-11-28T03:11:18.764171+0000","last_became_peered":"2023-11-28T03:11:18.764171+0000","last_unstale":"2023-11-28T03:11:18.768500+0000","last_undegraded":"2023-11-28T03:11:18.768500+0000","last_fullsized":"2023-11-28T03:11:18.768500+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757802+0000","last_change":"2023-11-28T03:11:18.395387+0000","last_active":"2023-11-28T03:11:18.757802+0000","last_peered":"2023-11-28T03:11:18.757802+0000","last_clean":"2023-11-28T03:11:18.757802+0000","last_became_active":"2023-11-28T03:11:18.394994+0000","last_became_peered":"2023-11-28T03:11:18.394994+0000","last_unstale":"2023-11-28T03:11:18.757802+0000","last_undegraded":"2023-11-28T03:11:18.757802+0000","last_fullsized":"2023-11-28T03:11:18.757802+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772088+0000","last_change":"2023-11-28T03:11:18.767939+0000","last_active":"2023-11-28T03:11:18.772088+0000","last_peered":"2023-11-28T03:11:18.772088+0000","last_clean":"2023-11-28T03:11:18.772088+0000","last_became_active":"2023-11-28T03:11:18.767368+0000","last_became_peered":"2023-11-28T03:11:18.767368+0000","last_unstale":"2023-11-28T03:11:18.772088+0000","last_undegraded":"2023-11-28T03:11:18.772088+0000","last_fullsized":"2023-11-28T03:11:18.772088+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.b","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757356+0000","last_change":"2023-11-28T03:11:18.394301+0000","last_active":"2023-11-28T03:11:18.757356+0000","last_peered":"2023-11-28T03:11:18.757356+0000","last_clean":"2023-11-28T03:11:18.757356+0000","last_became_active":"2023-11-28T03:11:18.393966+0000","last_became_peered":"2023-11-28T03:11:18.393966+0000","last_unstale":"2023-11-28T03:11:18.757356+0000","last_undegraded":"2023-11-28T03:11:18.757356+0000","last_fullsized":"2023-11-28T03:11:18.757356+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769255+0000","last_change":"2023-11-28T03:11:18.761997+0000","last_active":"2023-11-28T03:11:18.769255+0000","last_peered":"2023-11-28T03:11:18.769255+0000","last_clean":"2023-11-28T03:11:18.769255+0000","last_became_active":"2023-11-28T03:11:18.761245+0000","last_became_peered":"2023-11-28T03:11:18.761245+0000","last_unstale":"2023-11-28T03:11:18.769255+0000","last_undegraded":"2023-11-28T03:11:18.769255+0000","last_fullsized":"2023-11-28T03:11:18.769255+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757696+0000","last_change":"2023-11-28T03:11:18.394666+0000","last_active":"2023-11-28T03:11:18.757696+0000","last_peered":"2023-11-28T03:11:18.757696+0000","last_clean":"2023-11-28T03:11:18.757696+0000","last_became_active":"2023-11-28T03:11:18.393583+0000","last_became_peered":"2023-11-28T03:11:18.393583+0000","last_unstale":"2023-11-28T03:11:18.757696+0000","last_undegraded":"2023-11-28T03:11:18.757696+0000","last_fullsized":"2023-11-28T03:11:18.757696+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769627+0000","last_change":"2023-11-28T03:11:18.761969+0000","last_active":"2023-11-28T03:11:18.769627+0000","last_peered":"2023-11-28T03:11:18.769627+0000","last_clean":"2023-11-28T03:11:18.769627+0000","last_became_active":"2023-11-28T03:11:18.758807+0000","last_became_peered":"2023-11-28T03:11:18.758807+0000","last_unstale":"2023-11-28T03:11:18.769627+0000","last_undegraded":"2023-11-28T03:11:18.769627+0000","last_fullsized":"2023-11-28T03:11:18.769627+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.744278+0000","last_change":"2023-11-28T03:11:18.384737+0000","last_active":"2023-11-28T03:11:18.744278+0000","last_peered":"2023-11-28T03:11:18.744278+0000","last_clean":"2023-11-28T03:11:18.744278+0000","last_became_active":"2023-11-28T03:11:18.384433+0000","last_became_peered":"2023-11-28T03:11:18.384433+0000","last_unstale":"2023-11-28T03:11:18.744278+0000","last_undegraded":"2023-11-28T03:11:18.744278+0000","last_fullsized":"2023-11-28T03:11:18.744278+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.763906+0000","last_change":"2023-11-28T03:11:18.759866+0000","last_active":"2023-11-28T03:11:18.763906+0000","last_peered":"2023-11-28T03:11:18.763906+0000","last_clean":"2023-11-28T03:11:18.763906+0000","last_became_active":"2023-11-28T03:11:18.759595+0000","last_became_peered":"2023-11-28T03:11:18.759595+0000","last_unstale":"2023-11-28T03:11:18.763906+0000","last_undegraded":"2023-11-28T03:11:18.763906+0000","last_fullsized":"2023-11-28T03:11:18.763906+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.758719+0000","last_change":"2023-11-28T03:11:18.391939+0000","last_active":"2023-11-28T03:11:18.758719+0000","last_peered":"2023-11-28T03:11:18.758719+0000","last_clean":"2023-11-28T03:11:18.758719+0000","last_became_active":"2023-11-28T03:11:18.391661+0000","last_became_peered":"2023-11-28T03:11:18.391661+0000","last_unstale":"2023-11-28T03:11:18.758719+0000","last_undegraded":"2023-11-28T03:11:18.758719+0000","last_fullsized":"2023-11-28T03:11:18.758719+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769443+0000","last_change":"2023-11-28T03:11:18.763802+0000","last_active":"2023-11-28T03:11:18.769443+0000","last_peered":"2023-11-28T03:11:18.769443+0000","last_clean":"2023-11-28T03:11:18.769443+0000","last_became_active":"2023-11-28T03:11:18.763260+0000","last_became_peered":"2023-11-28T03:11:18.763260+0000","last_unstale":"2023-11-28T03:11:18.769443+0000","last_undegraded":"2023-11-28T03:11:18.769443+0000","last_fullsized":"2023-11-28T03:11:18.769443+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.7","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.767617+0000","last_change":"2023-11-28T03:11:18.390407+0000","last_active":"2023-11-28T03:11:18.767617+0000","last_peered":"2023-11-28T03:11:18.767617+0000","last_clean":"2023-11-28T03:11:18.767617+0000","last_became_active":"2023-11-28T03:11:18.390103+0000","last_became_peered":"2023-11-28T03:11:18.390103+0000","last_unstale":"2023-11-28T03:11:18.767617+0000","last_undegraded":"2023-11-28T03:11:18.767617+0000","last_fullsized":"2023-11-28T03:11:18.767617+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.762952+0000","last_change":"2023-11-28T03:11:18.756132+0000","last_active":"2023-11-28T03:11:18.762952+0000","last_peered":"2023-11-28T03:11:18.762952+0000","last_clean":"2023-11-28T03:11:18.762952+0000","last_became_active":"2023-11-28T03:11:18.755858+0000","last_became_peered":"2023-11-28T03:11:18.755858+0000","last_unstale":"2023-11-28T03:11:18.762952+0000","last_undegraded":"2023-11-28T03:11:18.762952+0000","last_fullsized":"2023-11-28T03:11:18.762952+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749193+0000","last_change":"2023-11-28T03:11:18.387271+0000","last_active":"2023-11-28T03:11:18.749193+0000","last_peered":"2023-11-28T03:11:18.749193+0000","last_clean":"2023-11-28T03:11:18.749193+0000","last_became_active":"2023-11-28T03:11:18.386973+0000","last_became_peered":"2023-11-28T03:11:18.386973+0000","last_unstale":"2023-11-28T03:11:18.749193+0000","last_undegraded":"2023-11-28T03:11:18.749193+0000","last_fullsized":"2023-11-28T03:11:18.749193+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773511+0000","last_change":"2023-11-28T03:11:18.768781+0000","last_active":"2023-11-28T03:11:18.773511+0000","last_peered":"2023-11-28T03:11:18.773511+0000","last_clean":"2023-11-28T03:11:18.773511+0000","last_became_active":"2023-11-28T03:11:18.768547+0000","last_became_peered":"2023-11-28T03:11:18.768547+0000","last_unstale":"2023-11-28T03:11:18.773511+0000","last_undegraded":"2023-11-28T03:11:18.773511+0000","last_fullsized":"2023-11-28T03:11:18.773511+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.750133+0000","last_change":"2023-11-28T03:11:18.387360+0000","last_active":"2023-11-28T03:11:18.750133+0000","last_peered":"2023-11-28T03:11:18.750133+0000","last_clean":"2023-11-28T03:11:18.750133+0000","last_became_active":"2023-11-28T03:11:18.386943+0000","last_became_peered":"2023-11-28T03:11:18.386943+0000","last_unstale":"2023-11-28T03:11:18.750133+0000","last_undegraded":"2023-11-28T03:11:18.750133+0000","last_fullsized":"2023-11-28T03:11:18.750133+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.765063+0000","last_change":"2023-11-28T03:11:18.759218+0000","last_active":"2023-11-28T03:11:18.765063+0000","last_peered":"2023-11-28T03:11:18.765063+0000","last_clean":"2023-11-28T03:11:18.765063+0000","last_became_active":"2023-11-28T03:11:18.758988+0000","last_became_peered":"2023-11-28T03:11:18.758988+0000","last_unstale":"2023-11-28T03:11:18.765063+0000","last_undegraded":"2023-11-28T03:11:18.765063+0000","last_fullsized":"2023-11-28T03:11:18.765063+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752856+0000","last_change":"2023-11-28T03:11:18.390261+0000","last_active":"2023-11-28T03:11:18.752856+0000","last_peered":"2023-11-28T03:11:18.752856+0000","last_clean":"2023-11-28T03:11:18.752856+0000","last_became_active":"2023-11-28T03:11:18.389960+0000","last_became_peered":"2023-11-28T03:11:18.389960+0000","last_unstale":"2023-11-28T03:11:18.752856+0000","last_undegraded":"2023-11-28T03:11:18.752856+0000","last_fullsized":"2023-11-28T03:11:18.752856+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.772140+0000","last_change":"2023-11-28T03:11:18.768583+0000","last_active":"2023-11-28T03:11:18.772140+0000","last_peered":"2023-11-28T03:11:18.772140+0000","last_clean":"2023-11-28T03:11:18.772140+0000","last_became_active":"2023-11-28T03:11:18.768203+0000","last_became_peered":"2023-11-28T03:11:18.768203+0000","last_unstale":"2023-11-28T03:11:18.772140+0000","last_undegraded":"2023-11-28T03:11:18.772140+0000","last_fullsized":"2023-11-28T03:11:18.772140+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.752454+0000","last_change":"2023-11-28T03:11:18.379037+0000","last_active":"2023-11-28T03:11:18.752454+0000","last_peered":"2023-11-28T03:11:18.752454+0000","last_clean":"2023-11-28T03:11:18.752454+0000","last_became_active":"2023-11-28T03:11:18.378709+0000","last_became_peered":"2023-11-28T03:11:18.378709+0000","last_unstale":"2023-11-28T03:11:18.752454+0000","last_undegraded":"2023-11-28T03:11:18.752454+0000","last_fullsized":"2023-11-28T03:11:18.752454+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.757667+0000","last_change":"2023-11-28T03:11:18.751193+0000","last_active":"2023-11-28T03:11:18.757667+0000","last_peered":"2023-11-28T03:11:18.757667+0000","last_clean":"2023-11-28T03:11:18.757667+0000","last_became_active":"2023-11-28T03:11:18.750949+0000","last_became_peered":"2023-11-28T03:11:18.750949+0000","last_unstale":"2023-11-28T03:11:18.757667+0000","last_undegraded":"2023-11-28T03:11:18.757667+0000","last_fullsized":"2023-11-28T03:11:18.757667+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753542+0000","last_change":"2023-11-28T03:11:18.388975+0000","last_active":"2023-11-28T03:11:18.753542+0000","last_peered":"2023-11-28T03:11:18.753542+0000","last_clean":"2023-11-28T03:11:18.753542+0000","last_became_active":"2023-11-28T03:11:18.388685+0000","last_became_peered":"2023-11-28T03:11:18.388685+0000","last_unstale":"2023-11-28T03:11:18.753542+0000","last_undegraded":"2023-11-28T03:11:18.753542+0000","last_fullsized":"2023-11-28T03:11:18.753542+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770910+0000","last_change":"2023-11-28T03:11:18.767145+0000","last_active":"2023-11-28T03:11:18.770910+0000","last_peered":"2023-11-28T03:11:18.770910+0000","last_clean":"2023-11-28T03:11:18.770910+0000","last_became_active":"2023-11-28T03:11:18.766817+0000","last_became_peered":"2023-11-28T03:11:18.766817+0000","last_unstale":"2023-11-28T03:11:18.770910+0000","last_undegraded":"2023-11-28T03:11:18.770910+0000","last_fullsized":"2023-11-28T03:11:18.770910+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.753369+0000","last_change":"2023-11-28T03:11:18.390087+0000","last_active":"2023-11-28T03:11:18.753369+0000","last_peered":"2023-11-28T03:11:18.753369+0000","last_clean":"2023-11-28T03:11:18.753369+0000","last_became_active":"2023-11-28T03:11:18.389794+0000","last_became_peered":"2023-11-28T03:11:18.389794+0000","last_unstale":"2023-11-28T03:11:18.753369+0000","last_undegraded":"2023-11-28T03:11:18.753369+0000","last_fullsized":"2023-11-28T03:11:18.753369+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768645+0000","last_change":"2023-11-28T03:11:18.763569+0000","last_active":"2023-11-28T03:11:18.768645+0000","last_peered":"2023-11-28T03:11:18.768645+0000","last_clean":"2023-11-28T03:11:18.768645+0000","last_became_active":"2023-11-28T03:11:18.763311+0000","last_became_peered":"2023-11-28T03:11:18.763311+0000","last_unstale":"2023-11-28T03:11:18.768645+0000","last_undegraded":"2023-11-28T03:11:18.768645+0000","last_fullsized":"2023-11-28T03:11:18.768645+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.758819+0000","last_change":"2023-11-28T03:11:18.388300+0000","last_active":"2023-11-28T03:11:18.758819+0000","last_peered":"2023-11-28T03:11:18.758819+0000","last_clean":"2023-11-28T03:11:18.758819+0000","last_became_active":"2023-11-28T03:11:18.387929+0000","last_became_peered":"2023-11-28T03:11:18.387929+0000","last_unstale":"2023-11-28T03:11:18.758819+0000","last_undegraded":"2023-11-28T03:11:18.758819+0000","last_fullsized":"2023-11-28T03:11:18.758819+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771079+0000","last_change":"2023-11-28T03:11:18.763454+0000","last_active":"2023-11-28T03:11:18.771079+0000","last_peered":"2023-11-28T03:11:18.771079+0000","last_clean":"2023-11-28T03:11:18.771079+0000","last_became_active":"2023-11-28T03:11:18.763179+0000","last_became_peered":"2023-11-28T03:11:18.763179+0000","last_unstale":"2023-11-28T03:11:18.771079+0000","last_undegraded":"2023-11-28T03:11:18.771079+0000","last_fullsized":"2023-11-28T03:11:18.771079+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749332+0000","last_change":"2023-11-28T03:11:18.384508+0000","last_active":"2023-11-28T03:11:18.749332+0000","last_peered":"2023-11-28T03:11:18.749332+0000","last_clean":"2023-11-28T03:11:18.749332+0000","last_became_active":"2023-11-28T03:11:18.384214+0000","last_became_peered":"2023-11-28T03:11:18.384214+0000","last_unstale":"2023-11-28T03:11:18.749332+0000","last_undegraded":"2023-11-28T03:11:18.749332+0000","last_fullsized":"2023-11-28T03:11:18.749332+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":110,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751331+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T03:11:18.751331+0000","last_peered":"2023-11-28T03:11:18.751331+0000","last_clean":"2023-11-28T03:11:18.751331+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T03:11:18.751331+0000","last_undegraded":"2023-11-28T03:11:18.751331+0000","last_fullsized":"2023-11-28T03:11:18.751331+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764320+0000","last_change":"2023-11-28T03:11:18.758697+0000","last_active":"2023-11-28T03:11:18.764320+0000","last_peered":"2023-11-28T03:11:18.764320+0000","last_clean":"2023-11-28T03:11:18.764320+0000","last_became_active":"2023-11-28T03:11:18.758435+0000","last_became_peered":"2023-11-28T03:11:18.758435+0000","last_unstale":"2023-11-28T03:11:18.764320+0000","last_undegraded":"2023-11-28T03:11:18.764320+0000","last_fullsized":"2023-11-28T03:11:18.764320+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.744222+0000","last_change":"2023-11-28T03:11:18.387081+0000","last_active":"2023-11-28T03:11:18.744222+0000","last_peered":"2023-11-28T03:11:18.744222+0000","last_clean":"2023-11-28T03:11:18.744222+0000","last_became_active":"2023-11-28T03:11:18.386827+0000","last_became_peered":"2023-11-28T03:11:18.386827+0000","last_unstale":"2023-11-28T03:11:18.744222+0000","last_undegraded":"2023-11-28T03:11:18.744222+0000","last_fullsized":"2023-11-28T03:11:18.744222+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":108,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.746719+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T03:11:18.746719+0000","last_peered":"2023-11-28T03:11:18.746719+0000","last_clean":"2023-11-28T03:11:18.746719+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T03:11:18.746719+0000","last_undegraded":"2023-11-28T03:11:18.746719+0000","last_fullsized":"2023-11-28T03:11:18.746719+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768326+0000","last_change":"2023-11-28T03:11:18.763515+0000","last_active":"2023-11-28T03:11:18.768326+0000","last_peered":"2023-11-28T03:11:18.768326+0000","last_clean":"2023-11-28T03:11:18.768326+0000","last_became_active":"2023-11-28T03:11:18.763207+0000","last_became_peered":"2023-11-28T03:11:18.763207+0000","last_unstale":"2023-11-28T03:11:18.768326+0000","last_undegraded":"2023-11-28T03:11:18.768326+0000","last_fullsized":"2023-11-28T03:11:18.768326+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.745221+0000","last_change":"2023-11-28T03:11:18.386867+0000","last_active":"2023-11-28T03:11:18.745221+0000","last_peered":"2023-11-28T03:11:18.745221+0000","last_clean":"2023-11-28T03:11:18.745221+0000","last_became_active":"2023-11-28T03:11:18.386623+0000","last_became_peered":"2023-11-28T03:11:18.386623+0000","last_unstale":"2023-11-28T03:11:18.745221+0000","last_undegraded":"2023-11-28T03:11:18.745221+0000","last_fullsized":"2023-11-28T03:11:18.745221+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":116,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749327+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T03:11:18.749327+0000","last_peered":"2023-11-28T03:11:18.749327+0000","last_clean":"2023-11-28T03:11:18.749327+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T03:11:18.749327+0000","last_undegraded":"2023-11-28T03:11:18.749327+0000","last_fullsized":"2023-11-28T03:11:18.749327+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771489+0000","last_change":"2023-11-28T03:11:18.766525+0000","last_active":"2023-11-28T03:11:18.771489+0000","last_peered":"2023-11-28T03:11:18.771489+0000","last_clean":"2023-11-28T03:11:18.771489+0000","last_became_active":"2023-11-28T03:11:18.766223+0000","last_became_peered":"2023-11-28T03:11:18.766223+0000","last_unstale":"2023-11-28T03:11:18.771489+0000","last_undegraded":"2023-11-28T03:11:18.771489+0000","last_fullsized":"2023-11-28T03:11:18.771489+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.10","version":"139'4","reported_seq":14,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.771063+0000","last_change":"2023-11-28T03:11:18.394496+0000","last_active":"2023-11-28T03:11:18.771063+0000","last_peered":"2023-11-28T03:11:18.771063+0000","last_clean":"2023-11-28T03:11:18.771063+0000","last_became_active":"2023-11-28T03:11:18.394294+0000","last_became_peered":"2023-11-28T03:11:18.394294+0000","last_unstale":"2023-11-28T03:11:18.771063+0000","last_undegraded":"2023-11-28T03:11:18.771063+0000","last_fullsized":"2023-11-28T03:11:18.771063+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":108,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.746912+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T03:11:18.746912+0000","last_peered":"2023-11-28T03:11:18.746912+0000","last_clean":"2023-11-28T03:11:18.746912+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T03:11:18.746912+0000","last_undegraded":"2023-11-28T03:11:18.746912+0000","last_fullsized":"2023-11-28T03:11:18.746912+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":111,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759074+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T03:11:18.759074+0000","last_peered":"2023-11-28T03:11:18.759074+0000","last_clean":"2023-11-28T03:11:18.759074+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T03:11:18.759074+0000","last_undegraded":"2023-11-28T03:11:18.759074+0000","last_fullsized":"2023-11-28T03:11:18.759074+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770796+0000","last_change":"2023-11-28T03:11:18.760540+0000","last_active":"2023-11-28T03:11:18.770796+0000","last_peered":"2023-11-28T03:11:18.770796+0000","last_clean":"2023-11-28T03:11:18.770796+0000","last_became_active":"2023-11-28T03:11:18.760079+0000","last_became_peered":"2023-11-28T03:11:18.760079+0000","last_unstale":"2023-11-28T03:11:18.770796+0000","last_undegraded":"2023-11-28T03:11:18.770796+0000","last_fullsized":"2023-11-28T03:11:18.770796+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.13","version":"139'1","reported_seq":11,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.769615+0000","last_change":"2023-11-28T03:11:18.396476+0000","last_active":"2023-11-28T03:11:18.769615+0000","last_peered":"2023-11-28T03:11:18.769615+0000","last_clean":"2023-11-28T03:11:18.769615+0000","last_became_active":"2023-11-28T03:11:18.396309+0000","last_became_peered":"2023-11-28T03:11:18.396309+0000","last_unstale":"2023-11-28T03:11:18.769615+0000","last_undegraded":"2023-11-28T03:11:18.769615+0000","last_fullsized":"2023-11-28T03:11:18.769615+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":109,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.759156+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T03:11:18.759156+0000","last_peered":"2023-11-28T03:11:18.759156+0000","last_clean":"2023-11-28T03:11:18.759156+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T03:11:18.759156+0000","last_undegraded":"2023-11-28T03:11:18.759156+0000","last_fullsized":"2023-11-28T03:11:18.759156+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.773629+0000","last_change":"2023-11-28T03:11:18.770142+0000","last_active":"2023-11-28T03:11:18.773629+0000","last_peered":"2023-11-28T03:11:18.773629+0000","last_clean":"2023-11-28T03:11:18.773629+0000","last_became_active":"2023-11-28T03:11:18.769833+0000","last_became_peered":"2023-11-28T03:11:18.769833+0000","last_unstale":"2023-11-28T03:11:18.773629+0000","last_undegraded":"2023-11-28T03:11:18.773629+0000","last_fullsized":"2023-11-28T03:11:18.773629+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.748731+0000","last_change":"2023-11-28T03:11:18.387795+0000","last_active":"2023-11-28T03:11:18.748731+0000","last_peered":"2023-11-28T03:11:18.748731+0000","last_clean":"2023-11-28T03:11:18.748731+0000","last_became_active":"2023-11-28T03:11:18.387481+0000","last_became_peered":"2023-11-28T03:11:18.387481+0000","last_unstale":"2023-11-28T03:11:18.748731+0000","last_undegraded":"2023-11-28T03:11:18.748731+0000","last_fullsized":"2023-11-28T03:11:18.748731+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":109,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.760146+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T03:11:18.760146+0000","last_peered":"2023-11-28T03:11:18.760146+0000","last_clean":"2023-11-28T03:11:18.760146+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T03:11:18.760146+0000","last_undegraded":"2023-11-28T03:11:18.760146+0000","last_fullsized":"2023-11-28T03:11:18.760146+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.768914+0000","last_change":"2023-11-28T03:11:18.757177+0000","last_active":"2023-11-28T03:11:18.768914+0000","last_peered":"2023-11-28T03:11:18.768914+0000","last_clean":"2023-11-28T03:11:18.768914+0000","last_became_active":"2023-11-28T03:11:18.756875+0000","last_became_peered":"2023-11-28T03:11:18.756875+0000","last_unstale":"2023-11-28T03:11:18.768914+0000","last_undegraded":"2023-11-28T03:11:18.768914+0000","last_fullsized":"2023-11-28T03:11:18.768914+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.15","version":"139'3","reported_seq":13,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.755275+0000","last_change":"2023-11-28T03:11:18.392352+0000","last_active":"2023-11-28T03:11:18.755275+0000","last_peered":"2023-11-28T03:11:18.755275+0000","last_clean":"2023-11-28T03:11:18.755275+0000","last_became_active":"2023-11-28T03:11:18.392050+0000","last_became_peered":"2023-11-28T03:11:18.392050+0000","last_unstale":"2023-11-28T03:11:18.755275+0000","last_undegraded":"2023-11-28T03:11:18.755275+0000","last_fullsized":"2023-11-28T03:11:18.755275+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":110,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.751587+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T03:11:18.751587+0000","last_peered":"2023-11-28T03:11:18.751587+0000","last_clean":"2023-11-28T03:11:18.751587+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T03:11:18.751587+0000","last_undegraded":"2023-11-28T03:11:18.751587+0000","last_fullsized":"2023-11-28T03:11:18.751587+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":9,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.764544+0000","last_change":"2023-11-28T03:11:18.761490+0000","last_active":"2023-11-28T03:11:18.764544+0000","last_peered":"2023-11-28T03:11:18.764544+0000","last_clean":"2023-11-28T03:11:18.764544+0000","last_became_active":"2023-11-28T03:11:18.761053+0000","last_became_peered":"2023-11-28T03:11:18.761053+0000","last_unstale":"2023-11-28T03:11:18.764544+0000","last_undegraded":"2023-11-28T03:11:18.764544+0000","last_fullsized":"2023-11-28T03:11:18.764544+0000","mapping_epoch":138,"log_start":"0'0","ondisk_log_start":"0'0","created":138,"last_epoch_clean":139,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:18.363215+0000","last_clean_scrub_stamp":"2023-11-28T03:11:18.363215+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.14","version":"139'2","reported_seq":12,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.770942+0000","last_change":"2023-11-28T03:11:18.386215+0000","last_active":"2023-11-28T03:11:18.770942+0000","last_peered":"2023-11-28T03:11:18.770942+0000","last_clean":"2023-11-28T03:11:18.770942+0000","last_became_active":"2023-11-28T03:11:18.385765+0000","last_became_peered":"2023-11-28T03:11:18.385765+0000","last_unstale":"2023-11-28T03:11:18.770942+0000","last_undegraded":"2023-11-28T03:11:18.770942+0000","last_fullsized":"2023-11-28T03:11:18.770942+0000","mapping_epoch":137,"log_start":"0'0","ondisk_log_start":"0'0","created":137,"last_epoch_clean":138,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T03:11:17.358460+0000","last_clean_scrub_stamp":"2023-11-28T03:11:17.358460+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":139,"state":"active+clean","last_fresh":"2023-11-28T03:11:18.749415+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T03:11:18.749415+0000","last_peered":"2023-11-28T03:11:18.749415+0000","last_clean":"2023-11-28T03:11:18.749415+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T03:11:18.749415+0000","last_undegraded":"2023-11-28T03:11:18.749415+0000","last_fullsized":"2023-11-28T03:11:18.749415+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":18,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575070,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302900,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068940,"statfs":{"total":96636764160,"available":96326594560,"internally_reserved":0,"allocated":1748992,"data_stored":868021,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58099999999999996,"15min":0.59399999999999997},"min":{"1min":0.436,"5min":0.32700000000000001,"15min":0.30199999999999999},"max":{"1min":0.69899999999999995,"5min":1.46,"15min":1.46},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58599999999999997,"15min":0.60299999999999998},"min":{"1min":0.47199999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.70399999999999996,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.54800000000000004},"min":{"1min":0.38800000000000001,"5min":0.32800000000000001,"15min":0.32000000000000001},"max":{"1min":0.65200000000000002,"5min":1.5249999999999999,"15min":1.5249999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55300000000000005,"15min":0.55900000000000005},"min":{"1min":0.42699999999999999,"5min":0.318,"15min":0.318},"max":{"1min":0.66000000000000003,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.47599999999999998}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54000000000000004,"15min":0.54900000000000004},"min":{"1min":0.40500000000000003,"5min":0.38900000000000001,"15min":0.34499999999999997},"max":{"1min":0.69799999999999995,"5min":1.5009999999999999,"15min":1.5009999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.54200000000000004,"15min":0.56200000000000006},"min":{"1min":0.39400000000000002,"5min":0.35999999999999999,"15min":0.34899999999999998},"max":{"1min":0.72399999999999998,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.52900000000000003}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.35899999999999999,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.623,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.57099999999999995,"15min":0.57899999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.75,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.49099999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46500000000000002},"min":{"1min":0.33100000000000002,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66500000000000004,"5min":0.79100000000000004,"15min":0.83099999999999996},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.498,"15min":0.48999999999999999},"min":{"1min":0.34699999999999998,"5min":0.32500000000000001,"15min":0.31},"max":{"1min":0.70699999999999996,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.49099999999999999,"15min":0.495},"min":{"1min":0.316,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60999999999999999,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.623},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.52800000000000002,"15min":0.53600000000000003},"min":{"1min":0.38300000000000001,"5min":0.34100000000000003,"15min":0.34000000000000002},"max":{"1min":0.64400000000000002,"5min":0.81499999999999995,"15min":0.83099999999999996},"last":0.65100000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56100000000000005,"15min":0.56000000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.76300000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.66800000000000004,"5min":0.76900000000000002,"15min":0.84499999999999997},"last":0.64200000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575070,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302892,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068948,"statfs":{"total":96636764160,"available":96326602752,"internally_reserved":0,"allocated":1740800,"data_stored":868381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.38800000000000001,"5min":0.35099999999999998,"15min":0.34100000000000003},"max":{"1min":0.70199999999999996,"5min":0.73699999999999999,"15min":1.4810000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.55700000000000005,"15min":0.56699999999999995},"min":{"1min":0.35499999999999998,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.52000000000000002}]},{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55100000000000005,"15min":0.56599999999999995},"min":{"1min":0.374,"5min":0.32000000000000001,"15min":0.29999999999999999},"max":{"1min":0.755,"5min":0.79000000000000004,"15min":0.94399999999999995},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.54500000000000004,"15min":0.55800000000000005},"min":{"1min":0.36199999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.77600000000000002,"5min":1.125,"15min":1.506},"last":0.49199999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.56499999999999995,"15min":0.57299999999999995},"min":{"1min":0.443,"5min":0.35399999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.93000000000000005},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.56399999999999995,"15min":0.58099999999999996},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.76100000000000001,"5min":0.77600000000000002,"15min":0.90900000000000003},"last":0.46300000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.435,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.71699999999999997,"5min":0.77900000000000003,"15min":0.98599999999999999},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.58099999999999996,"15min":0.58199999999999996},"min":{"1min":0.38800000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.76400000000000001,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.48199999999999998}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.441,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.309,"15min":0.27900000000000003},"max":{"1min":0.627,"5min":0.78100000000000003,"15min":1.7310000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.33600000000000002,"15min":0.30499999999999999},"max":{"1min":0.59699999999999998,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.55500000000000005}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.52200000000000002,"15min":0.51600000000000001},"min":{"1min":0.33000000000000002,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.747,"5min":0.874,"15min":0.95099999999999996},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.54300000000000004,"15min":0.54900000000000004},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.311},"max":{"1min":0.73599999999999999,"5min":0.82099999999999995,"15min":1.4490000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.54300000000000004},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":0.71399999999999997,"5min":0.91100000000000003,"15min":1.3799999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55200000000000005,"15min":0.55900000000000005},"min":{"1min":0.40699999999999997,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.71999999999999997,"5min":0.90400000000000003,"15min":1.1040000000000001},"last":0.60899999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575069,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302892,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068948,"statfs":{"total":96636764160,"available":96326602752,"internally_reserved":0,"allocated":1740800,"data_stored":867849,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.34599999999999997,"15min":0.34799999999999998},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.189},"max":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.67800000000000005},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.36199999999999999,"15min":0.374},"min":{"1min":0.22,"5min":0.218,"15min":0.20899999999999999},"max":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.67100000000000004},"last":0.45600000000000002}]},{"osd":2,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.38400000000000001,"15min":0.38700000000000001},"min":{"1min":0.23999999999999999,"5min":0.22700000000000001,"15min":0.21199999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.39200000000000002,"15min":0.40799999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.34899999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.432,"15min":0.45600000000000002},"min":{"1min":0.309,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.41399999999999998,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.68799999999999994},"last":0.377}]},{"osd":4,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.40899999999999997,"5min":0.33300000000000002,"15min":0.313},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.91100000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.57499999999999996,"15min":0.60099999999999998},"min":{"1min":0.443,"5min":0.40899999999999997,"15min":0.36699999999999999},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97799999999999998},"last":0.70899999999999996}]},{"osd":5,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.622},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36199999999999999},"max":{"1min":0.81799999999999995,"5min":0.84699999999999998,"15min":0.99299999999999999},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.627,"15min":0.64800000000000002},"min":{"1min":0.45800000000000002,"5min":0.39400000000000002,"15min":0.38},"max":{"1min":0.84899999999999998,"5min":0.94199999999999995,"15min":1.038},"last":0.74199999999999999}]},{"osd":6,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.57399999999999995,"15min":0.59599999999999997},"min":{"1min":0.41099999999999998,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.875,"5min":0.875,"15min":0.94099999999999995},"last":0.63},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.60399999999999998,"15min":0.622},"min":{"1min":0.38700000000000001,"5min":0.378,"15min":0.378},"max":{"1min":0.83199999999999996,"5min":0.92000000000000004,"15min":0.94599999999999995},"last":0.69999999999999996}]},{"osd":7,"last update":"Tue Nov 28 03:11:17 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.59799999999999998,"15min":0.61799999999999999},"min":{"1min":0.49399999999999999,"5min":0.40799999999999997,"15min":0.40000000000000002},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":1.024},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.59499999999999997,"15min":0.61899999999999999},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.88300000000000001},"last":0.72799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834575070,"num_pgs":42,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302900,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068940,"statfs":{"total":96636764160,"available":96326594560,"internally_reserved":0,"allocated":1748992,"data_stored":869295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.36199999999999999,"5min":0.32700000000000001,"15min":0.34399999999999997},"min":{"1min":0.25700000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.56599999999999995},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.378,"5min":0.34399999999999997,"15min":0.36499999999999999},"min":{"1min":0.27700000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.53700000000000003,"5min":0.58999999999999997,"15min":0.70299999999999996},"last":0.32700000000000001}]},{"osd":2,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.374,"5min":0.33900000000000002,"15min":0.36299999999999999},"min":{"1min":0.27200000000000002,"5min":0.224,"15min":0.224},"max":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.60699999999999998},"last":0.25900000000000001},{"interface":"front","average":{"1min":0.46200000000000002,"5min":0.41399999999999998,"15min":0.439},"min":{"1min":0.34499999999999997,"5min":0.27800000000000002,"15min":0.26300000000000001},"max":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.73299999999999998},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.371,"15min":0.39300000000000002},"min":{"1min":0.311,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.47999999999999998,"5min":0.60699999999999998,"15min":0.61099999999999999},"last":0.314},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.40000000000000002,"15min":0.41899999999999998},"min":{"1min":0.318,"5min":0.217,"15min":0.217},"max":{"1min":0.59999999999999998,"5min":0.61499999999999999,"15min":0.65400000000000003},"last":0.29899999999999999}]},{"osd":4,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.58399999999999996,"15min":0.61099999999999999},"min":{"1min":0.45400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.79500000000000004,"5min":1.427,"15min":1.427},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.58899999999999997,"15min":0.628},"min":{"1min":0.433,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.747,"5min":0.85899999999999999,"15min":1.2869999999999999},"last":0.42799999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.59299999999999997,"15min":0.629},"min":{"1min":0.434,"5min":0.39700000000000002,"15min":0.374},"max":{"1min":0.82999999999999996,"5min":0.83399999999999996,"15min":1.304},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.59799999999999998,"15min":0.63100000000000001},"min":{"1min":0.46000000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.82299999999999995,"5min":1.456,"15min":1.456},"last":0.51700000000000002}]},{"osd":6,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.57799999999999996,"15min":0.59799999999999998},"min":{"1min":0.42799999999999999,"5min":0.307,"15min":0.307},"max":{"1min":0.77900000000000003,"5min":1.4399999999999999,"15min":1.679},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.58599999999999997,"15min":0.61599999999999999},"min":{"1min":0.42699999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.78300000000000003,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.54500000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:09 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.59099999999999997,"15min":0.622},"min":{"1min":0.45100000000000001,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.80700000000000005,"5min":1.341,"15min":1.341},"last":0.441},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63,"15min":0.66600000000000004},"min":{"1min":0.49299999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.76400000000000001,"5min":1.474,"15min":1.474},"last":0.56000000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575071,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302896,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068944,"statfs":{"total":96636764160,"available":96326598656,"internally_reserved":0,"allocated":1744896,"data_stored":868789,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.36699999999999999,"15min":0.36899999999999999},"min":{"1min":0.25700000000000001,"5min":0.22,"15min":0.21099999999999999},"max":{"1min":0.54800000000000004,"5min":0.59499999999999997,"15min":0.59499999999999997},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.41399999999999998,"15min":0.41999999999999998},"min":{"1min":0.312,"5min":0.25900000000000001,"15min":0.23599999999999999},"max":{"1min":0.59799999999999998,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.32600000000000001}]},{"osd":1,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.36599999999999999,"15min":0.37},"min":{"1min":0.28000000000000003,"5min":0.24199999999999999,"15min":0.22700000000000001},"max":{"1min":0.52600000000000002,"5min":0.56100000000000005,"15min":0.60599999999999998},"last":0.30599999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.38800000000000001,"15min":0.39700000000000002},"min":{"1min":0.245,"5min":0.245,"15min":0.218},"max":{"1min":0.61299999999999999,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.41199999999999998,"15min":0.41999999999999998},"min":{"1min":0.33300000000000002,"5min":0.27400000000000002,"15min":0.19800000000000001},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.67400000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.42099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.27100000000000002,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.627,"5min":0.64200000000000002,"15min":0.65500000000000003},"last":0.318}]},{"osd":4,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58799999999999997,"15min":0.59599999999999997},"min":{"1min":0.42899999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.80100000000000005,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.60799999999999998,"15min":0.61299999999999999},"min":{"1min":0.39200000000000002,"5min":0.379,"15min":0.35799999999999998},"max":{"1min":0.754,"5min":1.087,"15min":1.087},"last":0.44800000000000001}]},{"osd":5,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.625,"15min":0.63500000000000001},"min":{"1min":0.44,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.86399999999999999,"5min":0.89600000000000002,"15min":0.98599999999999999},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63900000000000001,"15min":0.64600000000000002},"min":{"1min":0.40100000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.79900000000000004,"5min":0.94699999999999995,"15min":1.004},"last":0.505}]},{"osd":6,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.61899999999999999,"15min":0.63300000000000001},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.38500000000000001},"max":{"1min":0.878,"5min":0.91000000000000003,"15min":0.95199999999999996},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.64500000000000002,"15min":0.65700000000000003},"min":{"1min":0.47099999999999997,"5min":0.44700000000000001,"15min":0.40699999999999997},"max":{"1min":0.84999999999999998,"5min":0.91500000000000004,"15min":0.93300000000000005},"last":0.439}]},{"osd":7,"last update":"Tue Nov 28 03:11:00 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61599999999999999,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.39700000000000002,"15min":0.33000000000000002},"max":{"1min":0.83999999999999997,"5min":0.86799999999999999,"15min":0.99299999999999999},"last":0.432},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.623,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.82099999999999995,"5min":0.93200000000000005,"15min":0.94499999999999995},"last":0.46700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575070,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306996,"kb_used_data":1708,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064844,"statfs":{"total":96636764160,"available":96322400256,"internally_reserved":0,"allocated":1748992,"data_stored":868505,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35699999999999998,"15min":0.36199999999999999},"min":{"1min":0.28999999999999998,"5min":0.22,"15min":0.20799999999999999},"max":{"1min":0.51400000000000001,"5min":0.53600000000000003,"15min":0.59899999999999998},"last":0.314},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.36599999999999999,"15min":0.378},"min":{"1min":0.27300000000000002,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55400000000000005,"5min":0.59799999999999998,"15min":0.68899999999999995},"last":0.24099999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.374,"15min":0.38200000000000001},"min":{"1min":0.247,"5min":0.245,"15min":0.23599999999999999},"max":{"1min":0.60099999999999998,"5min":0.68600000000000005,"15min":0.68600000000000005},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61599999999999999,"5min":0.622,"15min":0.68899999999999995},"last":0.28299999999999997}]},{"osd":2,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.31900000000000001,"5min":0.25800000000000001,"15min":0.223},"max":{"1min":0.54300000000000004,"5min":0.60699999999999998,"15min":0.74199999999999999},"last":0.307},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.42799999999999999,"15min":0.44},"min":{"1min":0.34799999999999998,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"last":0.33300000000000002}]},{"osd":4,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.36399999999999999,"15min":0.34599999999999997},"max":{"1min":0.751,"5min":0.878,"15min":1.581},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.59999999999999998,"15min":0.61299999999999999},"min":{"1min":0.45000000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.94199999999999995},"last":0.42299999999999999}]},{"osd":5,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.59399999999999997,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.373,"15min":0.35699999999999998},"max":{"1min":0.85099999999999998,"5min":0.93999999999999995,"15min":0.97899999999999998},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.623,"5min":0.61499999999999999,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.40000000000000002,"15min":0.34999999999999998},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"last":0.41499999999999998}]},{"osd":6,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.47999999999999998,"5min":0.40500000000000003,"15min":0.38300000000000001},"max":{"1min":0.876,"5min":0.93100000000000005,"15min":0.996},"last":0.439},{"interface":"front","average":{"1min":0.626,"5min":0.61199999999999999,"15min":0.63300000000000001},"min":{"1min":0.40799999999999997,"5min":0.38400000000000001,"15min":0.37},"max":{"1min":0.92700000000000005,"5min":0.95199999999999996,"15min":1.0409999999999999},"last":0.51200000000000001}]},{"osd":7,"last update":"Tue Nov 28 03:11:01 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.62,"15min":0.64300000000000002},"min":{"1min":0.41499999999999998,"5min":0.39500000000000002,"15min":0.37},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.97199999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63100000000000001,"15min":0.64500000000000002},"min":{"1min":0.48699999999999999,"5min":0.38,"15min":0.376},"max":{"1min":0.87,"5min":0.90000000000000002,"15min":0.98899999999999999},"last":0.501}]}]},{"osd":4,"up_from":13,"seq":55834575069,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302896,"kb_used_data":1704,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068944,"statfs":{"total":96636764160,"available":96326598656,"internally_reserved":0,"allocated":1744896,"data_stored":868471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.54000000000000004,"15min":0.55100000000000005},"min":{"1min":0.438,"5min":0.33300000000000002,"15min":0.29499999999999998},"max":{"1min":0.71499999999999997,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.54400000000000004,"15min":0.56200000000000006},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.34799999999999998},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.83199999999999996},"last":0.46800000000000003}]},{"osd":1,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54200000000000004,"15min":0.55300000000000005},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.32800000000000001},"max":{"1min":0.748,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.54400000000000004,"15min":0.56000000000000005},"min":{"1min":0.39600000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.749,"5min":0.80500000000000005,"15min":0.81200000000000006},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.372,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.76600000000000001,"5min":0.81899999999999995,"15min":0.82199999999999995},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54600000000000004,"15min":0.56299999999999994},"min":{"1min":0.41899999999999998,"5min":0.373,"15min":0.33500000000000002},"max":{"1min":0.78600000000000003,"5min":0.80900000000000005,"15min":0.80900000000000005},"last":0.55500000000000005}]},{"osd":3,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.57799999999999996,"15min":0.59999999999999998},"min":{"1min":0.40200000000000002,"5min":0.376,"15min":0.376},"max":{"1min":0.79300000000000004,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.60499999999999998,"15min":0.628},"min":{"1min":0.45800000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.878},"last":0.56499999999999995}]},{"osd":5,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.28699999999999998},"max":{"1min":0.64200000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.48899999999999999,"15min":0.499},"min":{"1min":0.32300000000000001,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.80000000000000004,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.48899999999999999,"15min":0.498},"min":{"1min":0.30599999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.71499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.51700000000000002,"15min":0.52100000000000002},"min":{"1min":0.375,"5min":0.35999999999999999,"15min":0.33600000000000002},"max":{"1min":0.70099999999999996,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Nov 28 03:10:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53300000000000003,"15min":0.54000000000000004},"min":{"1min":0.39600000000000002,"5min":0.34300000000000003,"15min":0.33400000000000002},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55400000000000005,"15min":0.56699999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32900000000000001},"max":{"1min":0.88400000000000001,"5min":0.90600000000000003,"15min":0.91300000000000003},"last":0.63700000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575071,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302892,"kb_used_data":1700,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068948,"statfs":{"total":96636764160,"available":96326602752,"internally_reserved":0,"allocated":1740800,"data_stored":867917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.53900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34799999999999998,"5min":0.27900000000000003,"15min":0.23499999999999999},"max":{"1min":0.753,"5min":2.8959999999999999,"15min":2.8959999999999999},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.51800000000000002,"15min":0.54200000000000004},"min":{"1min":0.38400000000000001,"5min":0.33100000000000002,"15min":0.28000000000000003},"max":{"1min":0.71499999999999997,"5min":0.751,"15min":0.77900000000000003},"last":0.61599999999999999}]},{"osd":1,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.54000000000000004,"15min":0.56100000000000005},"min":{"1min":0.46500000000000002,"5min":0.34399999999999997,"15min":0.27000000000000002},"max":{"1min":0.76100000000000001,"5min":0.78300000000000003,"15min":0.83299999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.55600000000000005,"15min":0.57899999999999996},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.82499999999999996},"last":0.48699999999999999}]},{"osd":2,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.56599999999999995,"15min":0.58499999999999996},"min":{"1min":0.44800000000000001,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.73299999999999998,"5min":0.747,"15min":0.80500000000000005},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.59699999999999998},"min":{"1min":0.40200000000000002,"5min":0.40000000000000002,"15min":0.38100000000000001},"max":{"1min":0.75700000000000001,"5min":0.82299999999999995,"15min":0.83399999999999996},"last":0.64400000000000002}]},{"osd":3,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.52700000000000002,"15min":0.54300000000000004},"min":{"1min":0.41399999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.67900000000000005,"5min":0.77900000000000003,"15min":0.79800000000000004},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57499999999999996,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.35899999999999999,"15min":0.35299999999999998},"max":{"1min":0.74199999999999999,"5min":2.9079999999999999,"15min":2.9079999999999999},"last":0.46700000000000003}]},{"osd":4,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.46400000000000002,"15min":0.44900000000000001},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.60199999999999998,"5min":0.80600000000000005,"15min":0.85699999999999998},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.52400000000000002,"15min":0.53200000000000003},"min":{"1min":0.38300000000000001,"5min":0.32900000000000001,"15min":0.32400000000000001},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.81499999999999995},"last":0.69599999999999995}]},{"osd":6,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48599999999999999,"15min":0.48899999999999999},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.27200000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.374,"5min":0.36199999999999999,"15min":0.33700000000000002},"max":{"1min":0.75800000000000001,"5min":0.80000000000000004,"15min":0.94399999999999995},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Nov 28 03:11:12 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.48499999999999999,"15min":0.499},"min":{"1min":0.33900000000000002,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.63700000000000001,"5min":0.77800000000000002,"15min":0.89900000000000002},"last":0.628},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.52900000000000003,"15min":0.53100000000000003},"min":{"1min":0.35099999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.91200000000000003},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":962,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T03:11:26.480 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T03:11:26.480 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T03:11:26.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.634+0000 7f0e31f1a700 1 Processor -- start 2023-11-28T03:11:26.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.634+0000 7f0e31f1a700 1 -- start start 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e31f1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e2c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e31f1a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c0c72d0 con 0x7f0e2c098d10 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e31f1a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e2c064570 con 0x7f0e2c158320 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e31f1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c154bc0 con 0x7f0e2c0c6190 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0e2c098d10 0x7f0e2c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44270/0 (socket says 172.21.15.73:44270) 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/2223483009 learned_addr learned my addr 172.21.15.73:0/2223483009 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3401146379 0 0) 0x7f0e2c0c72d0 con 0x7f0e2c098d10 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e2c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0e180033f0 con 0x7f0e2c098d10 2023-11-28T03:11:26.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1951641885 0 0) 0x7f0e180033f0 con 0x7f0e2c098d10 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 msgr2=0x7f0e2c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e2c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 >> v1:172.21.15.150:6789/0 conn(0x7f0e2c158320 legacy=0x7f0e2c0a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0c66c0 con 0x7f0e2c098d10 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f0e1c001470 con 0x7f0e2c098d10 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 >> v1:172.21.15.73:6789/0 conn(0x7f0e2c098d10 legacy=0x7f0e2c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e18003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e18004690 0x7f0e18004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e18005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c154bc0 con 0x7f0e18003910 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1800b910 con 0x7f0e18004690 2023-11-28T03:11:26.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1800bad0 con 0x7f0e2c0c6190 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.635+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e18003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e18004690 0x7f0e18004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2bfff700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e18005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 msgr2=0x7f0e18005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c0c6190 0x7f0e18005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e18004690 msgr2=0x7f0e18004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e18004690 0x7f0e18004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/2223483009 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e180033f0 con 0x7f0e18003910 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e18004690 0x7f0e18004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e18003d30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f0e1c00c190 tx=0x7f0e1c001e50).ready entity=mon.0 client_cookie=d0c762f4e1496077 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c007a20 con 0x7f0e18003910 2023-11-28T03:11:26.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c00cac0 con 0x7f0e18003910 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2affd700 1 -- 172.21.15.73:0/2223483009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c00ce40 con 0x7f0e18003910 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e31f1a700 1 -- 172.21.15.73:0/2223483009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 msgr2=0x7f0e18003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/2223483009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e18003d30 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 rx=0x7f0e1c00c190 tx=0x7f0e1c001e50).stop 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e2bfff700 1 -- 172.21.15.73:0/2223483009 reap_dead start 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e31f1a700 1 -- 172.21.15.73:0/2223483009 shutdown_connections 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.636+0000 7f0e31f1a700 1 -- 172.21.15.73:0/2223483009 >> 172.21.15.73:0/2223483009 conn(0x7f0e2c0cb310 msgr2=0x7f0e2c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.637+0000 7f0e31f1a700 1 -- 172.21.15.73:0/2223483009 shutdown_connections 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.637+0000 7f0e31f1a700 1 -- 172.21.15.73:0/2223483009 wait complete. 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.638+0000 7f0e31f1a700 1 Processor -- start 2023-11-28T03:11:26.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.638+0000 7f0e31f1a700 1 -- start start 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 0x7f0e2c0cab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c155c50 0x7f0e2c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c0c8ea0 con 0x7f0e18003910 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c168030 con 0x7f0e2c0c86c0 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2c1688d0 con 0x7f0e2c155c50 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35504/0 (socket says 172.21.15.73:35504) 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/1835183149 learned_addr learned my addr 172.21.15.73:0/1835183149 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 0x7f0e2c0cab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2bfff700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c155c50 0x7f0e2c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c155c50 msgr2=0x7f0e2c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c155c50 0x7f0e2c1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 msgr2=0x7f0e2c0cab70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 0x7f0e2c0cab70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 -- 172.21.15.73:0/1835183149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2c0cb060 con 0x7f0e18003910 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 0x7f0e2c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e2b7fe700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f0e1c00c780 tx=0x7f0e1c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0076c0 con 0x7f0e18003910 2023-11-28T03:11:26.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0046b0 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e2c1565a0 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c004a70 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.639+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e2c168280 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.640+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2c06c860 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.641+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0e1c020480 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.641+0000 7f0e23fff700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 0x7f0e0c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.641+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0e1c07afa0 con 0x7f0e18003910 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.641+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 0x7f0e0c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:26.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.641+0000 7f0e22ffd700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 0x7f0e0c0615e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0e14006c20 tx=0x7f0e14008040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:26.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.645+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e1c0a9af0 con 0x7f0e18003910 2023-11-28T03:11:26.874 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.874+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f0e2c0c69a0 con 0x7f0e18003910 2023-11-28T03:11:26.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.878+0000 7f0e23fff700 1 -- 172.21.15.73:0/1835183149 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f0e1c04c210 con 0x7f0e18003910 2023-11-28T03:11:26.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 msgr2=0x7f0e0c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 0x7f0e0c0615e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f0e14006c20 tx=0x7f0e14008040).stop 2023-11-28T03:11:26.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 msgr2=0x7f0e2c1535f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:26.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 rx=0x7f0e1c00c780 tx=0x7f0e1c015000).stop 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 shutdown_connections 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0e0c05f120 0x7f0e0c0615e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e2c155c50 0x7f0e2c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e18003910 0x7f0e2c1535f0 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 --2- 172.21.15.73:0/1835183149 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e2c0c86c0 0x7f0e2c0cab70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 >> 172.21.15.73:0/1835183149 conn(0x7f0e2c0cb310 msgr2=0x7f0e2c1550f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 shutdown_connections 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:26.880+0000 7f0e31f1a700 1 -- 172.21.15.73:0/1835183149 wait complete. 2023-11-28T03:11:26.881 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T03:11:26.892 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T03:11:27.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.005+0000 7f34be25b700 1 Processor -- start 2023-11-28T03:11:27.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.005+0000 7f34be25b700 1 -- start start 2023-11-28T03:11:27.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34be25b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f34b8064e20 0x7f34b8155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34be25b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b80683d0 con 0x7f34b8095e80 2023-11-28T03:11:27.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34be25b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b8067850 con 0x7f34b81544d0 2023-11-28T03:11:27.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34be25b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b8067650 con 0x7f34b8064e20 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34b7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f34b8095e80 0x7f34b815e0a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44282/0 (socket says 172.21.15.73:44282) 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.006+0000 7f34b7fff700 1 -- 172.21.15.73:0/848902622 learned_addr learned my addr 172.21.15.73:0/848902622 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b6ffd700 1 --2- 172.21.15.73:0/848902622 >> v2:172.21.15.150:3300/0 conn(0x7f34b8064e20 0x7f34b8155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4228840111 0 0) 0x7f34b80683d0 con 0x7f34b8095e80 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34a00033f0 con 0x7f34b8095e80 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1369086136 0 0) 0x7f34a00033f0 con 0x7f34b8095e80 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 >> v2:172.21.15.150:3300/0 conn(0x7f34b8064e20 msgr2=0x7f34b8155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 --2- 172.21.15.73:0/848902622 >> v2:172.21.15.150:3300/0 conn(0x7f34b8064e20 0x7f34b8155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 >> v1:172.21.15.150:6789/0 conn(0x7f34b81544d0 legacy=0x7f34b8066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b8067b30 con 0x7f34b8095e80 2023-11-28T03:11:27.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f34b00017f0 con 0x7f34b8095e80 2023-11-28T03:11:27.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 >> v1:172.21.15.73:6789/0 conn(0x7f34b8095e80 legacy=0x7f34b815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a00038e0 0x7f34a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0004690 0x7f34a0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 0x7f34a000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8067650 con 0x7f34a0005000 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34a000beb0 con 0x7f34a00038e0 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34a000c070 con 0x7f34a0004690 2023-11-28T03:11:27.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 0x7f34a000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b77fe700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a00038e0 0x7f34a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b6ffd700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0004690 0x7f34a0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 -- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0004690 msgr2=0x7f34a0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0004690 0x7f34a0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 -- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a00038e0 msgr2=0x7f34a0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a00038e0 0x7f34a0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 -- 172.21.15.73:0/848902622 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a00033f0 con 0x7f34a0005000 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b77fe700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34a00038e0 0x7f34a0003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 -- 172.21.15.73:0/848902622 reap_dead start 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.007+0000 7f34b7fff700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 0x7f34a000b8e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f34b000c480 tx=0x7f34b000c790).ready entity=mon.0 client_cookie=bbeaa8d7663de2bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0002030 con 0x7f34a0005000 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b000ce20 con 0x7f34a0005000 2023-11-28T03:11:27.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34b67fc700 1 -- 172.21.15.73:0/848902622 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b0005d30 con 0x7f34a0005000 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 -- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 msgr2=0x7f34a000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 0x7f34a000b8e0 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 rx=0x7f34b000c480 tx=0x7f34b000c790).stop 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 -- 172.21.15.73:0/848902622 shutdown_connections 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 --2- 172.21.15.73:0/848902622 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34a0005000 0x7f34a000b8e0 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 -- 172.21.15.73:0/848902622 >> 172.21.15.73:0/848902622 conn(0x7f34b80631d0 msgr2=0x7f34b80c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 -- 172.21.15.73:0/848902622 shutdown_connections 2023-11-28T03:11:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.008+0000 7f34be25b700 1 -- 172.21.15.73:0/848902622 wait complete. 2023-11-28T03:11:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.009+0000 7f34be25b700 1 Processor -- start 2023-11-28T03:11:27.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.009+0000 7f34be25b700 1 -- start start 2023-11-28T03:11:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 0x7f34b80ae0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 0x7f34b814dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b81555d0 con 0x7f34b814fa10 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b80ae5a0 con 0x7f34b80ac6f0 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34be25b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b8150340 con 0x7f34a0005000 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35526/0 (socket says 172.21.15.73:35526) 2023-11-28T03:11:27.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 -- 172.21.15.73:0/1480373160 learned_addr learned my addr 172.21.15.73:0/1480373160 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b77fe700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 0x7f34b80ae0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 -- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 msgr2=0x7f34b80ae0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b6ffd700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 0x7f34b814dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 0x7f34b80ae0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 -- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 msgr2=0x7f34b814dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 0x7f34b814dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b7fff700 1 -- 172.21.15.73:0/1480373160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b80af0c0 con 0x7f34b814fa10 2023-11-28T03:11:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b77fe700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 0x7f34b80ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.010+0000 7f34b6ffd700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 0x7f34b814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:27.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f34b7fff700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f34b000c7d0 tx=0x7f34b0015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b00074f0 con 0x7f34b814fa10 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34b0007650 con 0x7f34b814fa10 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b80ae820 con 0x7f34b814fa10 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34b00079d0 con 0x7f34b814fa10 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.011+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b80aede0 con 0x7f34b814fa10 2023-11-28T03:11:27.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.012+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b806c860 con 0x7f34b814fa10 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.012+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f34b0013660 con 0x7f34b814fa10 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.012+0000 7f349f7fe700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 0x7f34a4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.012+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f34b007b2f0 con 0x7f34b814fa10 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.012+0000 7f34b77fe700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 0x7f34a4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.013+0000 7f34b77fe700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 0x7f34a4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f34a8007950 tx=0x7f34a800c040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.016+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f34b004c5c0 con 0x7f34b814fa10 2023-11-28T03:11:27.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.221+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34b8067e10 con 0x7f34b814fa10 2023-11-28T03:11:27.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.221+0000 7f349f7fe700 1 -- 172.21.15.73:0/1480373160 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7f34b001a020 con 0x7f34b814fa10 2023-11-28T03:11:27.222 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:27.222 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T03:11:27.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 msgr2=0x7f34a4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 0x7f34a4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f34a8007950 tx=0x7f34a800c040).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 msgr2=0x7f34b814fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 rx=0x7f34b000c7d0 tx=0x7f34b0015000).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 shutdown_connections 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f34a405f150 0x7f34a4061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f34b814fa10 0x7f34b814fe50 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f34a0005000 0x7f34b80ae0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 --2- 172.21.15.73:0/1480373160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f34b80ac6f0 0x7f34b814dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 >> 172.21.15.73:0/1480373160 conn(0x7f34b80631d0 msgr2=0x7f34b815a140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 shutdown_connections 2023-11-28T03:11:27.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.223+0000 7f34be25b700 1 -- 172.21.15.73:0/1480373160 wait complete. 2023-11-28T03:11:27.225 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T03:11:27.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 52, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T03:11:18.740523+0000', 'modified': '2023-11-28T03:11:20.749057+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8094}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8094': {'gid': 8094, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/3840072732', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 3840072732}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3840072732}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [18], 'metadata_pool': 17, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T03:11:27.638 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T03:11:27.638 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T03:11:27.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.754+0000 7f68ba159700 1 Processor -- start 2023-11-28T03:11:27.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.754+0000 7f68ba159700 1 -- start start 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68ba159700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f68b40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68ba159700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b40c72d0 con 0x7f68b4098d10 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68ba159700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68b4064570 con 0x7f68b40c6190 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68ba159700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4154bc0 con 0x7f68b4158a30 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b37fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f68b4098d10 0x7f68b4063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44296/0 (socket says 172.21.15.73:44296) 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b37fe700 1 -- 172.21.15.73:0/363345930 learned_addr learned my addr 172.21.15.73:0/363345930 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b3fff700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f68b40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 409264095 0 0) 0x7f68b40c72d0 con 0x7f68b4098d10 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f689c0033f0 con 0x7f68b4098d10 2023-11-28T03:11:27.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2312144773 0 0) 0x7f689c0033f0 con 0x7f68b4098d10 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 msgr2=0x7f68b40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f68b40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 >> v1:172.21.15.150:6789/0 conn(0x7f68b40c6190 legacy=0x7f68b4158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b40c66c0 con 0x7f68b4098d10 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f68ac0017f0 con 0x7f68b4098d10 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 >> v1:172.21.15.73:6789/0 conn(0x7f68b4098d10 legacy=0x7f68b4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f689c003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f689c003e40 0x7f689c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f689c004b80 0x7f689c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4154bc0 con 0x7f689c004b80 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f689c00b990 con 0x7f689c003e40 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f689c00bb50 con 0x7f68b4158a30 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.755+0000 7f68b3fff700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f689c004b80 0x7f689c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f689c003e40 0x7f689c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b37fe700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f689c003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 -- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 msgr2=0x7f689c003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b4158a30 0x7f689c003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 -- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f689c003e40 msgr2=0x7f689c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f689c003e40 0x7f689c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 -- 172.21.15.73:0/363345930 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f689c0033f0 con 0x7f689c004b80 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f689c003e40 0x7f689c004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f689c004b80 0x7f689c005000 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f68a4009a60 tx=0x7f68a4009060).ready entity=mon.0 client_cookie=f818a6cfd48078bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.760 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4009ea0 con 0x7f689c004b80 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a4009430 con 0x7f689c004b80 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b27fc700 1 -- 172.21.15.73:0/363345930 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a400e870 con 0x7f689c004b80 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68ba159700 1 -- 172.21.15.73:0/363345930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f689c004b80 msgr2=0x7f689c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68ba159700 1 --2- 172.21.15.73:0/363345930 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f689c004b80 0x7f689c005000 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 rx=0x7f68a4009a60 tx=0x7f68a4009060).stop 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.756+0000 7f68b3fff700 1 -- 172.21.15.73:0/363345930 reap_dead start 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.757+0000 7f68ba159700 1 -- 172.21.15.73:0/363345930 shutdown_connections 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.757+0000 7f68ba159700 1 -- 172.21.15.73:0/363345930 >> 172.21.15.73:0/363345930 conn(0x7f68b40cb310 msgr2=0x7f68b40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.757+0000 7f68ba159700 1 -- 172.21.15.73:0/363345930 shutdown_connections 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.757+0000 7f68ba159700 1 -- 172.21.15.73:0/363345930 wait complete. 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.758+0000 7f68ba159700 1 Processor -- start 2023-11-28T03:11:27.761 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.758+0000 7f68ba159700 1 -- start start 2023-11-28T03:11:27.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b40c8d80 0x7f68b4148d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b40c9160 0x7f68b4155300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40c76e0 con 0x7f68b41649d0 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68b40af780 con 0x7f68b40c8d80 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68b4164fe0 con 0x7f68b40c9160 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35540/0 (socket says 172.21.15.73:35540) 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 -- 172.21.15.73:0/3710588418 learned_addr learned my addr 172.21.15.73:0/3710588418 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b37fe700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b40c8d80 0x7f68b4148d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b3fff700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b40c9160 0x7f68b4155300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 -- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b40c9160 msgr2=0x7f68b4155300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b40c9160 0x7f68b4155300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 -- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b40c8d80 msgr2=0x7f68b4148d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b40c8d80 0x7f68b4148d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 -- 172.21.15.73:0/3710588418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b41652c0 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f68a800e4b0 tx=0x7f68a8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f689b7fe700 1 -- 172.21.15.73:0/3710588418 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8015440 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f689b7fe700 1 -- 172.21.15.73:0/3710588418 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a8010040 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b41655a0 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f689b7fe700 1 -- 172.21.15.73:0/3710588418 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80157b0 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.759+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40c95e0 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.761+0000 7f689b7fe700 1 -- 172.21.15.73:0/3710588418 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f68a8015be0 con 0x7f68b41649d0 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.761+0000 7f689b7fe700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 0x7f68a0061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.761+0000 7f689b7fe700 1 -- 172.21.15.73:0/3710588418 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68a8079770 con 0x7f68b41649d0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.761+0000 7f68b37fe700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 0x7f68a0061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.761+0000 7f68b37fe700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 0x7f68a0061640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68ac007fd0 tx=0x7f68ac00c040).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.763+0000 7f68cbaf1b80 1 Processor -- start 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.763+0000 7f68cbaf1b80 1 -- start start 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9728d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db96fb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db96c6060 con 0x558db9749ea0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db963d1f0 con 0x558db94bce40 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db9652440 con 0x558db9700cb0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35554/0 (socket says 172.21.15.73:35554) 2023-11-28T03:11:27.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 -- 172.21.15.73:0/2368487629 learned_addr learned my addr 172.21.15.73:0/2368487629 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9728d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 -- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 msgr2=0x558db96fb600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b3fff700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db96fb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db96fb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 -- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 msgr2=0x558db9728d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9728d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b2ffd700 1 -- 172.21.15.73:0/2368487629 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db964b620 con 0x558db9749ea0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.764+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9728d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.765+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f68a800f010 tx=0x7f68a8011000).ready entity=mon.0 client_cookie=f3088bdba00d8b60 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.765+0000 7f6898ff9700 1 -- 172.21.15.73:0/2368487629 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a804e060 con 0x558db9749ea0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.765+0000 7f6898ff9700 1 -- 172.21.15.73:0/2368487629 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a804e6a0 con 0x558db9749ea0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.765+0000 7f6898ff9700 1 -- 172.21.15.73:0/2368487629 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a804ae10 con 0x558db9749ea0 2023-11-28T03:11:27.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2368487629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 msgr2=0x558db97007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 rx=0x7f68a800f010 tx=0x7f68a8011000).stop 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2368487629 shutdown_connections 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db96fb600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9728d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2368487629 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97007c0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2368487629 >> 172.21.15.73:0/2368487629 conn(0x558db956a4a0 msgr2=0x558db9749790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2368487629 shutdown_connections 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2368487629 wait complete. 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 -- start start 2023-11-28T03:11:27.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.766+0000 7f68cbaf1b80 1 Processor -- start 2023-11-28T03:11:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db97283b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db972f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db9652580 con 0x558db9749ea0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db9652800 con 0x558db94bce40 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db961de10 con 0x558db9700cb0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68cbaf1b80 10 client.0 ll_register_callbacks cb 0x558db96c0460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35566/0 (socket says 172.21.15.73:35566) 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68b3fff700 1 -- 172.21.15.73:0/1629251516 learned_addr learned my addr 172.21.15.73:0/1629251516 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68b37fe700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db97283b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.767+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db972f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 -- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 msgr2=0x558db972f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db972f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 -- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 msgr2=0x558db97283b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db97283b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 -- 172.21.15.73:0/1629251516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68b40c95e0 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b3fff700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f689c005650 tx=0x7f68a401a000).ready entity=mon.0 client_cookie=b34546db454b9307 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4021070 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68a400edf0 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558db96f4720 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a4009430 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 10 client.8253 fetch_fsmap learned FSMap version 52 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x558db96469c0 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558db96f4720 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 52) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f68a4011070 con 0x558db9749ea0 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 10 client.8253 fetch_fsmap finished waiting for FSMap version 52 2023-11-28T03:11:27.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 10 client.8253 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 20 client.8253 populate_metadata read hostname 'smithi073' 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9758390 0x558db9758770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 4 client.8253 mds_command: new command op to 8094 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 --> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558db94b8ca0 con 0x558db9758390 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.768+0000 7f68b37fe700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9758390 0x558db9758770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f6891ffb700 10 client.8253 ms_handle_connect on v2:172.21.15.73:6835/3840072732 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68b37fe700 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9758390 0x558db9758770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=cb8f432ea6cd983b server_cookie=5d1df55abf37e36b in_seq=0 out_seq=0 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f6891ffb700 1 -- 172.21.15.73:0/1629251516 <== mds.0 v2:172.21.15.73:6835/3840072732 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x558db94b8ca0 con 0x558db9758390 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f6891ffb700 10 client.8253 handle_command_reply: tid=0 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 client.8253 shutdown 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 msgr2=0x558db972c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 rx=0x7f689c005650 tx=0x7f68a401a000).stop 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 shutdown_connections 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9758390 0x558db9758770 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db972c170 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db97283b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f6891ffb700 0 client.8253 ms_handle_reset on v2:172.21.15.73:6835/3840072732 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1629251516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db972f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.769+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 >> 172.21.15.73:0/1629251516 conn(0x558db956a4a0 msgr2=0x558db9701110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.770+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 shutdown_connections 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.770+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1629251516 wait complete. 2023-11-28T03:11:27.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.770+0000 7f68cbaf1b80 20 client.8253 trim_cache size 0 max 16384 2023-11-28T03:11:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.786+0000 7f68cbaf1b80 1 Processor -- start 2023-11-28T03:11:27.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.786+0000 7f68cbaf1b80 1 -- start start 2023-11-28T03:11:27.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db972dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9749ea0 0x558db9757540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db96fcf90 con 0x558db9700cb0 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db9727da0 con 0x558db94bce40 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db97415c0 con 0x558db9749ea0 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35580/0 (socket says 172.21.15.73:35580) 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 -- 172.21.15.73:0/1354218031 learned_addr learned my addr 172.21.15.73:0/1354218031 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 -- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9749ea0 msgr2=0x558db9757540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9749ea0 0x558db9757540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b37fe700 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db972dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 -- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 msgr2=0x558db972dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db972dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 -- 172.21.15.73:0/1354218031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558db96f4340 con 0x558db9700cb0 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f68a800e480 tx=0x7f68a8059000).ready entity=mon.0 client_cookie=2e9564d3e9c8545d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68997fa700 1 -- 172.21.15.73:0/1354218031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8060070 con 0x558db9700cb0 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68997fa700 1 -- 172.21.15.73:0/1354218031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a804e060 con 0x558db9700cb0 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.787+0000 7f68997fa700 1 -- 172.21.15.73:0/1354218031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8063070 con 0x558db9700cb0 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1354218031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 msgr2=0x558db97a1710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 rx=0x7f68a800e480 tx=0x7f68a8059000).stop 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1354218031 shutdown_connections 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9749ea0 0x558db9757540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db972dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/1354218031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9700cb0 0x558db97a1710 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1354218031 >> 172.21.15.73:0/1354218031 conn(0x558db956a4a0 msgr2=0x558db972e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1354218031 shutdown_connections 2023-11-28T03:11:27.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/1354218031 wait complete. 2023-11-28T03:11:27.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 -- start start 2023-11-28T03:11:27.791 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.788+0000 7f68cbaf1b80 1 Processor -- start 2023-11-28T03:11:27.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9757920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db97a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db9761b30 con 0x558db9749ea0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x558db9756420 con 0x558db94bce40 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x558db9756670 con 0x558db9700cb0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68cbaf1b80 10 client.0 ll_register_callbacks cb 0x558db96c0460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68b3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35590/0 (socket says 172.21.15.73:35590) 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68b3fff700 1 -- 172.21.15.73:0/2319336614 learned_addr learned my addr 172.21.15.73:0/2319336614 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9757920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.789+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db97a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 -- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 msgr2=0x558db97a2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db97a2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 -- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 msgr2=0x558db9757920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9757920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 -- 172.21.15.73:0/2319336614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558db964b620 con 0x558db9749ea0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b2ffd700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db97a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9757920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68b3fff700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f68a4001eb0 tx=0x7f68a4021000).ready entity=mon.0 client_cookie=8ef772c1f342d045 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f6891ffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a4009430 con 0x558db9749ea0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68a400edf0 con 0x558db9749ea0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68a804e6a0 con 0x558db9749ea0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=52) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68a4026620 con 0x558db9749ea0 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68cbaf1b80 10 client.8259 fetch_fsmap learned FSMap version 52 2023-11-28T03:11:27.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=52,monmap=2+}) v3 -- 0x558db9646710 con 0x558db9749ea0 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.790+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a804e6a0 con 0x558db9749ea0 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 52) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f68a4026780 con 0x558db9749ea0 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 10 client.8259 fetch_fsmap finished waiting for FSMap version 52 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 10 client.8259 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 20 client.8259 populate_metadata read hostname 'smithi073' 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9756f80 0x558db9741cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 4 client.8259 mds_command: new command op to 8094 tid=0[{"prefix": "status"}] 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 --> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558db94efca0 con 0x558db9756f80 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9756f80 0x558db9741cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f6891ffb700 10 client.8259 ms_handle_connect on v2:172.21.15.73:6835/3840072732 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68b37fe700 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9756f80 0x558db9741cd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=41369cc0cc011aa2 server_cookie=db168e675f6791a3 in_seq=0 out_seq=0 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f6891ffb700 1 -- 172.21.15.73:0/2319336614 <== mds.0 v2:172.21.15.73:6835/3840072732 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x558db94efca0 con 0x558db9756f80 2023-11-28T03:11:27.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f6891ffb700 10 client.8259 handle_command_reply: tid=0 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 client.8259 shutdown 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 msgr2=0x558db97a2b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 rx=0x7f68a4001eb0 tx=0x7f68a4021000).stop 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 shutdown_connections 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:6835/3840072732,v1:172.21.15.73:6837/3840072732] conn(0x558db9756f80 0x558db9741cd0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x558db9749ea0 0x558db97a2b60 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x558db94bce40 0x558db9757920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f6891ffb700 0 client.8259 ms_handle_reset on v2:172.21.15.73:6835/3840072732 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 --2- 172.21.15.73:0/2319336614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x558db9700cb0 0x558db97a2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 >> 172.21.15.73:0/2319336614 conn(0x558db956a4a0 msgr2=0x558db975e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 shutdown_connections 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.791+0000 7f68cbaf1b80 1 -- 172.21.15.73:0/2319336614 wait complete. 2023-11-28T03:11:27.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68cbaf1b80 20 client.8259 trim_cache size 0 max 16384 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 8094, 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 9.0494288600000008, 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 52, 2023-11-28T03:11:27.800 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 140, 2023-11-28T03:11:27.801 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 140, 2023-11-28T03:11:27.801 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 13.573072466999999 2023-11-28T03:11:27.801 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 msgr2=0x7f68a0061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 0x7f68a0061640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68ac007fd0 tx=0x7f68ac00c040).stop 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 msgr2=0x7f68b40af290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 rx=0x7f68a800e4b0 tx=0x7f68a8009040).stop 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 shutdown_connections 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f68a005f180 0x7f68a0061640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68b40c9160 0x7f68b4155300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68b40c8d80 0x7f68b4148d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 --2- 172.21.15.73:0/3710588418 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68b41649d0 0x7f68b40af290 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 >> 172.21.15.73:0/3710588418 conn(0x7f68b40cb310 msgr2=0x7f68b40cabf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 shutdown_connections 2023-11-28T03:11:27.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.792+0000 7f68ba159700 1 -- 172.21.15.73:0/3710588418 wait complete. 2023-11-28T03:11:27.804 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T03:11:27.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.904+0000 7f57090cd700 1 Processor -- start 2023-11-28T03:11:27.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.904+0000 7f57090cd700 1 -- start start 2023-11-28T03:11:27.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f57090cd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57040c8360 0x7f57040aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f57090cd700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57040c6d80 con 0x7f57040aa1a0 2023-11-28T03:11:27.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f57090cd700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5704154ae0 con 0x7f5704154700 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f57090cd700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5704064c30 con 0x7f57040c8360 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f570359e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57040c8360 0x7f57040aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f56fa59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f57040aa1a0 0x7f57040c7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44300/0 (socket says 172.21.15.73:44300) 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f56fa59c700 1 -- 172.21.15.73:0/728634538 learned_addr learned my addr 172.21.15.73:0/728634538 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f570359e700 1 -- 172.21.15.73:0/728634538 >> v1:172.21.15.150:6789/0 conn(0x7f5704154700 legacy=0x7f57040c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f570359e700 1 -- 172.21.15.73:0/728634538 >> v1:172.21.15.73:6789/0 conn(0x7f57040aa1a0 legacy=0x7f57040c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f570259c700 1 -- 172.21.15.73:0/728634538 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 472744080 0 0) 0x7f57040c6d80 con 0x7f57040aa1a0 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.905+0000 7f570359e700 1 -- 172.21.15.73:0/728634538 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57040c6450 con 0x7f57040c8360 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f570359e700 1 --2- 172.21.15.73:0/728634538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040c8360 0x7f57040aa620 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f56fc009e60 tx=0x7f56fc009780).ready entity=mon.0 client_cookie=925b17d76b8a8c76 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f570259c700 1 -- 172.21.15.73:0/728634538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc00e070 con 0x7f57040c8360 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f570259c700 1 -- 172.21.15.73:0/728634538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56fc00bc30 con 0x7f57040c8360 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f570259c700 1 -- 172.21.15.73:0/728634538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56fc012410 con 0x7f57040c8360 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 -- 172.21.15.73:0/728634538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040c8360 msgr2=0x7f57040aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 --2- 172.21.15.73:0/728634538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040c8360 0x7f57040aa620 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 rx=0x7f56fc009e60 tx=0x7f56fc009780).stop 2023-11-28T03:11:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 -- 172.21.15.73:0/728634538 shutdown_connections 2023-11-28T03:11:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 --2- 172.21.15.73:0/728634538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040c8360 0x7f57040aa620 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 -- 172.21.15.73:0/728634538 >> 172.21.15.73:0/728634538 conn(0x7f57040948b0 msgr2=0x7f570406a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 -- 172.21.15.73:0/728634538 shutdown_connections 2023-11-28T03:11:27.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.906+0000 7f57090cd700 1 -- 172.21.15.73:0/728634538 wait complete. 2023-11-28T03:11:27.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 Processor -- start 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 -- start start 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57040c8360 0x7f5704141e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5704154700 0x7f5704167d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5704157770 con 0x7f57040aa1a0 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57040ab0d0 con 0x7f57040c8360 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f57090cd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57041571b0 con 0x7f5704154700 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f5702d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f5702d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35602/0 (socket says 172.21.15.73:35602) 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f5702d9d700 1 -- 172.21.15.73:0/70679331 learned_addr learned my addr 172.21.15.73:0/70679331 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f570359e700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5704154700 0x7f5704167d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.908+0000 7f56fa59c700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57040c8360 0x7f5704141e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 -- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5704154700 msgr2=0x7f5704167d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5704154700 0x7f5704167d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 -- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57040c8360 msgr2=0x7f5704141e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57040c8360 0x7f5704141e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 -- 172.21.15.73:0/70679331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56fc009a20 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f5702d9d700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f56f400c4c0 tx=0x7f56f4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f400ce70 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f4001e90 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4007250 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5704157490 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.909+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5704142330 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.910+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57040475e0 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.910+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f56f40066a0 con 0x7f57040aa1a0 2023-11-28T03:11:27.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.911+0000 7f56fb7fe700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 0x7f56e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:27.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.911+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f56f4079a40 con 0x7f57040aa1a0 2023-11-28T03:11:27.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.911+0000 7f56fa59c700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 0x7f56e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:27.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.911+0000 7f56fa59c700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 0x7f56e4061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f57040856d0 tx=0x7f56ec007db0).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:27.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:27.915+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f56f407be60 con 0x7f57040aa1a0 2023-11-28T03:11:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.118+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f57040630f0 con 0x7f57040aa1a0 2023-11-28T03:11:28.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.118+0000 7f56fb7fe700 1 -- 172.21.15.73:0/70679331 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v52) v1 ==== 53+0+73 (secure 0 0 0) 0x7f56f404ad10 con 0x7f57040aa1a0 2023-11-28T03:11:28.118 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 msgr2=0x7f56e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 0x7f56e4061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f57040856d0 tx=0x7f56ec007db0).stop 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 msgr2=0x7f5704156bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 rx=0x7f56f400c4c0 tx=0x7f56f4007e00).stop 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 shutdown_connections 2023-11-28T03:11:28.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f56e405f150 0x7f56e4061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5704154700 0x7f5704167d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57040aa1a0 0x7f5704156bd0 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 --2- 172.21.15.73:0/70679331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57040c8360 0x7f5704141e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 >> 172.21.15.73:0/70679331 conn(0x7f57040948b0 msgr2=0x7f57040c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 shutdown_connections 2023-11-28T03:11:28.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.120+0000 7f57090cd700 1 -- 172.21.15.73:0/70679331 wait complete. 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T03:11:28.131 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:11:28.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.132 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T03:11:28.147 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T03:11:28.147 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T03:11:28.148 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 41392sec preferred_lft 41392sec 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T03:11:28.149 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T03:11:28.150 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T03:11:28.151 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T03:11:28.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.151 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T03:11:28.211 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T03:11:28.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.213 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T03:11:28.269 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T03:11:28.270 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T03:11:28.369 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.369 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T03:11:28.383 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T03:11:28.384 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T03:11:28.384 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T03:11:28.384 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.384 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T03:11:28.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.448 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T03:11:28.503 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T03:11:28.503 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:11:28.581 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:11:28.582 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:11:28.606 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T03:11:28.607 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T03:11:28.609 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T03:11:28.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T03:11:28.609 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T03:11:28.674 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T03:11:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.829+0000 7ff1eedb0700 1 Processor -- start 2023-11-28T03:11:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.829+0000 7ff1eedb0700 1 -- start start 2023-11-28T03:11:28.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1eedb0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1e80c6190 0x7ff1e80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:28.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1eedb0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80c72d0 con 0x7ff1e8158320 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1eedb0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e8064570 con 0x7ff1e8098d10 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1eedb0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e8154bc0 con 0x7ff1e80c6190 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1dffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1e80c6190 0x7ff1e80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1ed34d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff1e8158320 0x7ff1e80a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44316/0 (socket says 172.21.15.73:44316) 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1ed34d700 1 -- 172.21.15.73:0/318712699 learned_addr learned my addr 172.21.15.73:0/318712699 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:28.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1dffff700 1 -- 172.21.15.73:0/318712699 >> v1:172.21.15.150:6789/0 conn(0x7ff1e8098d10 legacy=0x7ff1e8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1dffff700 1 -- 172.21.15.73:0/318712699 >> v1:172.21.15.73:6789/0 conn(0x7ff1e8158320 legacy=0x7ff1e80a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.830+0000 7ff1dffff700 1 -- 172.21.15.73:0/318712699 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80c66c0 con 0x7ff1e80c6190 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1dffff700 1 --2- 172.21.15.73:0/318712699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e80c6190 0x7ff1e80c7760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7ff1d8009760 tx=0x7ff1d8009060).ready entity=mon.0 client_cookie=a700cdec2f9e02b6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1df7fe700 1 -- 172.21.15.73:0/318712699 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800b580 con 0x7ff1e80c6190 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1df7fe700 1 -- 172.21.15.73:0/318712699 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1d800b6e0 con 0x7ff1e80c6190 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1df7fe700 1 -- 172.21.15.73:0/318712699 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1d800ba60 con 0x7ff1e80c6190 2023-11-28T03:11:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 -- 172.21.15.73:0/318712699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e80c6190 msgr2=0x7ff1e80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/318712699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e80c6190 0x7ff1e80c7760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 rx=0x7ff1d8009760 tx=0x7ff1d8009060).stop 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 -- 172.21.15.73:0/318712699 shutdown_connections 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/318712699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e80c6190 0x7ff1e80c7760 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 -- 172.21.15.73:0/318712699 >> 172.21.15.73:0/318712699 conn(0x7ff1e80cb310 msgr2=0x7ff1e80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 -- 172.21.15.73:0/318712699 shutdown_connections 2023-11-28T03:11:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.831+0000 7ff1eedb0700 1 -- 172.21.15.73:0/318712699 wait complete. 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.832+0000 7ff1eedb0700 1 Processor -- start 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 -- start start 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80c6190 0x7ff1e8156030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8158320 0x7ff1e80a9680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e814b860 con 0x7ff1e8098d10 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80ac030 con 0x7ff1e80c6190 2023-11-28T03:11:28.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1eedb0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8156520 con 0x7ff1e8158320 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1ecb4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1ecb4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35614/0 (socket says 172.21.15.73:35614) 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.833+0000 7ff1ecb4c700 1 -- 172.21.15.73:0/652588289 learned_addr learned my addr 172.21.15.73:0/652588289 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ed34d700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8158320 0x7ff1e80a9680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1dffff700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80c6190 0x7ff1e8156030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 -- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8158320 msgr2=0x7ff1e80a9680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8158320 0x7ff1e80a9680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 -- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80c6190 msgr2=0x7ff1e8156030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80c6190 0x7ff1e8156030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 -- 172.21.15.73:0/652588289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1d80092a0 con 0x7ff1e8098d10 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1ecb4c700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7ff1e400c4f0 tx=0x7ff1e4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e4007020 con 0x7ff1e8098d10 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e4001e90 con 0x7ff1e8098d10 2023-11-28T03:11:28.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e80ab140 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e40073b0 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.834+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e80ab700 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.835+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e806c860 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.835+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff1e40068b0 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.835+0000 7ff1dcff9700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 0x7ff1d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.836+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff1e4079b60 con 0x7ff1e8098d10 2023-11-28T03:11:28.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.836+0000 7ff1dffff700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 0x7ff1d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:28.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.836+0000 7ff1dffff700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 0x7ff1d4061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff1d8009760 tx=0x7ff1d8017000).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:28.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:28.839+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1e404ae30 con 0x7ff1e8098d10 2023-11-28T03:11:29.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.029+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff1e80c69a0 con 0x7ff1d405f150 2023-11-28T03:11:29.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.031+0000 7ff1dcff9700 1 -- 172.21.15.73:0/652588289 <== mgr.7908 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff1e80c69a0 con 0x7ff1d405f150 2023-11-28T03:11:29.031 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T03:11:29.031 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T03:11:29.031 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T03:11:29.031 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T03:11:29.031 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 msgr2=0x7ff1d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 0x7ff1d4061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7ff1d8009760 tx=0x7ff1d8017000).stop 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 msgr2=0x7ff1e814b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 rx=0x7ff1e400c4f0 tx=0x7ff1e4007e00).stop 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 shutdown_connections 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1d405f150 0x7ff1d4061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e8158320 0x7ff1e80a9680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8098d10 0x7ff1e814b370 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 --2- 172.21.15.73:0/652588289 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80c6190 0x7ff1e8156030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 >> 172.21.15.73:0/652588289 conn(0x7ff1e80cb310 msgr2=0x7ff1e8151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 shutdown_connections 2023-11-28T03:11:29.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.033+0000 7ff1eedb0700 1 -- 172.21.15.73:0/652588289 wait complete. 2023-11-28T03:11:29.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T03:11:29.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.160+0000 7f55e3c48700 1 Processor -- start 2023-11-28T03:11:29.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.160+0000 7f55e3c48700 1 -- start start 2023-11-28T03:11:29.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e3c48700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c6190 0x7f55dc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e3c48700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f55dc158320 0x7f55dc1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e3c48700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc158cc0 con 0x7f55dc098d10 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e3c48700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0c72d0 con 0x7f55dc158320 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e3c48700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc064570 con 0x7f55dc0c6190 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e21e5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f55dc158320 0x7f55dc1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e21e5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f55dc158320 0x7f55dc1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35616/0 (socket says 172.21.15.73:35616) 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e21e5700 1 -- 172.21.15.73:0/4114653590 learned_addr learned my addr 172.21.15.73:0/4114653590 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.161+0000 7f55e11e3700 1 --2- 172.21.15.73:0/4114653590 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c6190 0x7f55dc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e21e5700 1 -- 172.21.15.73:0/4114653590 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c6190 msgr2=0x7f55dc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e21e5700 1 --2- 172.21.15.73:0/4114653590 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c6190 0x7f55dc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e21e5700 1 -- 172.21.15.73:0/4114653590 >> v1:172.21.15.150:6789/0 conn(0x7f55dc098d10 legacy=0x7f55dc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e21e5700 1 -- 172.21.15.73:0/4114653590 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc154bc0 con 0x7f55dc158320 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e21e5700 1 --2- 172.21.15.73:0/4114653590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc158320 0x7f55dc1587d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f55dc0c5b40 tx=0x7f55d4009840).ready entity=mon.0 client_cookie=af63bfe9be0cb717 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e09e2700 1 -- 172.21.15.73:0/4114653590 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d400e070 con 0x7f55dc158320 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e09e2700 1 -- 172.21.15.73:0/4114653590 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d400ba20 con 0x7f55dc158320 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e09e2700 1 -- 172.21.15.73:0/4114653590 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d400bda0 con 0x7f55dc158320 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 -- 172.21.15.73:0/4114653590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc158320 msgr2=0x7f55dc1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 --2- 172.21.15.73:0/4114653590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc158320 0x7f55dc1587d0 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 rx=0x7f55dc0c5b40 tx=0x7f55d4009840).stop 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 -- 172.21.15.73:0/4114653590 shutdown_connections 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 --2- 172.21.15.73:0/4114653590 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc158320 0x7f55dc1587d0 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 --2- 172.21.15.73:0/4114653590 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c6190 0x7f55dc0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.162+0000 7f55e3c48700 1 -- 172.21.15.73:0/4114653590 >> 172.21.15.73:0/4114653590 conn(0x7f55dc0cb310 msgr2=0x7f55dc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.163+0000 7f55e3c48700 1 -- 172.21.15.73:0/4114653590 shutdown_connections 2023-11-28T03:11:29.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.163+0000 7f55e3c48700 1 -- 172.21.15.73:0/4114653590 wait complete. 2023-11-28T03:11:29.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 Processor -- start 2023-11-28T03:11:29.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 -- start start 2023-11-28T03:11:29.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc098d10 0x7f55dc157460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc158320 0x7f55dc148c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc156450 con 0x7f55dc0c6190 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1566a0 con 0x7f55dc158320 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e3c48700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc156980 con 0x7f55dc098d10 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e11e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e11e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35626/0 (socket says 172.21.15.73:35626) 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.164+0000 7f55e11e3700 1 -- 172.21.15.73:0/2751445399 learned_addr learned my addr 172.21.15.73:0/2751445399 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e21e5700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc158320 0x7f55dc148c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e19e4700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc098d10 0x7f55dc157460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 -- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc098d10 msgr2=0x7f55dc157460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc098d10 0x7f55dc157460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 -- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc158320 msgr2=0x7f55dc148c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc158320 0x7f55dc148c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 -- 172.21.15.73:0/2751445399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55d4009ae0 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e11e3700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f55cc00e950 tx=0x7f55cc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc012600 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc012760 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc012b00 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc156c60 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.165+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc157950 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.166+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc157180 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.166+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 56) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f55cc01a3b0 con 0x7f55dc0c6190 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.167+0000 7f55d1ffb700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 0x7f55c00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:29.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.167+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f55cc07a2d0 con 0x7f55dc0c6190 2023-11-28T03:11:29.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.167+0000 7f55e19e4700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 0x7f55c00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:29.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.167+0000 7f55e19e4700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 0x7f55c00615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f55d8003900 tx=0x7f55d8007d40).ready entity=mgr.7908 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:29.170 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.171+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55dc157180 con 0x7f55dc0c6190 2023-11-28T03:11:29.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:29.379+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f55dc157180 con 0x7f55dc0c6190 2023-11-28T03:11:30.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.315+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 57) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f55cc0780a0 con 0x7f55dc0c6190 2023-11-28T03:11:30.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.317+0000 7f55d1ffb700 1 -- 172.21.15.73:0/2751445399 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v57) v1 ==== 92+0+0 (secure 0 0 0) 0x7f55cc073230 con 0x7f55dc0c6190 2023-11-28T03:11:30.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 msgr2=0x7f55c00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 0x7f55c00615e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f55d8003900 tx=0x7f55d8007d40).stop 2023-11-28T03:11:30.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 msgr2=0x7f55dc148730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 rx=0x7f55cc00e950 tx=0x7f55cc009060).stop 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 shutdown_connections 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f55c005f120 0x7f55c00615e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc158320 0x7f55dc148c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc098d10 0x7f55dc157460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 --2- 172.21.15.73:0/2751445399 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55dc0c6190 0x7f55dc148730 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 >> 172.21.15.73:0/2751445399 conn(0x7f55dc0cb310 msgr2=0x7f55dc0ca320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:30.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 shutdown_connections 2023-11-28T03:11:30.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.319+0000 7f55e3c48700 1 -- 172.21.15.73:0/2751445399 wait complete. 2023-11-28T03:11:30.334 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T03:11:30.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.452+0000 7fd266837700 1 Processor -- start 2023-11-28T03:11:30.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.452+0000 7fd266837700 1 -- start start 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd266837700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd260155830 0x7fd260155c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd266837700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd2600c2980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd266837700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2601529c0 con 0x7fd2600c2e70 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd266837700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2601554b0 con 0x7fd260155830 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd266837700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2600c3300 con 0x7fd260155030 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd25ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd260155830 0x7fd260155c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd25ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd260155830 0x7fd260155c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35644/0 (socket says 172.21.15.73:35644) 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd25ffff700 1 -- 172.21.15.73:0/548725945 learned_addr learned my addr 172.21.15.73:0/548725945 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.453+0000 7fd25f7fe700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd2600c2980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25ffff700 1 -- 172.21.15.73:0/548725945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 msgr2=0x7fd2600c2980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25ffff700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd2600c2980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25ffff700 1 -- 172.21.15.73:0/548725945 >> v1:172.21.15.150:6789/0 conn(0x7fd2600c2e70 legacy=0x7fd2600c5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25ffff700 1 -- 172.21.15.73:0/548725945 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd260151db0 con 0x7fd260155830 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25ffff700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd260155830 0x7fd260155c10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7fd254006fd0 tx=0x7fd2540090f0).ready entity=mon.0 client_cookie=d6cbb5ead4fd43bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25effd700 1 -- 172.21.15.73:0/548725945 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400b580 con 0x7fd260155830 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25effd700 1 -- 172.21.15.73:0/548725945 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd25400b6e0 con 0x7fd260155830 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd25effd700 1 -- 172.21.15.73:0/548725945 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400ba80 con 0x7fd260155830 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 -- 172.21.15.73:0/548725945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd260155830 msgr2=0x7fd260155c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd260155830 0x7fd260155c10 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 rx=0x7fd254006fd0 tx=0x7fd2540090f0).stop 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 -- 172.21.15.73:0/548725945 shutdown_connections 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd260155830 0x7fd260155c10 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 --2- 172.21.15.73:0/548725945 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd2600c2980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.454+0000 7fd266837700 1 -- 172.21.15.73:0/548725945 >> 172.21.15.73:0/548725945 conn(0x7fd260062700 msgr2=0x7fd260062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.455+0000 7fd266837700 1 -- 172.21.15.73:0/548725945 shutdown_connections 2023-11-28T03:11:30.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.455+0000 7fd266837700 1 -- 172.21.15.73:0/548725945 wait complete. 2023-11-28T03:11:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.456+0000 7fd266837700 1 Processor -- start 2023-11-28T03:11:30.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.456+0000 7fd266837700 1 -- start start 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd260168b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd260155830 0x7fd260164b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd260168100 con 0x7fd2600c2e70 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2601686c0 con 0x7fd260155830 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd26014d230 con 0x7fd260155030 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35652/0 (socket says 172.21.15.73:35652) 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 -- 172.21.15.73:0/1911313214 learned_addr learned my addr 172.21.15.73:0/1911313214 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:30.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25f7fe700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd260168b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd264dd4700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd260155830 0x7fd260164b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 msgr2=0x7fd260168b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd260168b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd260155830 msgr2=0x7fd260164b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd260155830 0x7fd260164b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 -- 172.21.15.73:0/1911313214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd254009370 con 0x7fd2600c2e70 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd25ffff700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fd254009ea0 tx=0x7fd254017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25400be40 con 0x7fd2600c2e70 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd254006990 con 0x7fd2600c2e70 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd260169040 con 0x7fd2600c2e70 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.457+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd25401c420 con 0x7fd2600c2e70 2023-11-28T03:11:30.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.458+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd260165940 con 0x7fd2600c2e70 2023-11-28T03:11:30.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd26006c860 con 0x7fd2600c2e70 2023-11-28T03:11:30.463 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:30.459+0000 7f889d04f680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:11:30.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 57) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fd25401c850 con 0x7fd2600c2e70 2023-11-28T03:11:30.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd247fff700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 0x7fd24c061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd25f7fe700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 msgr2=0x7fd24c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:30.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd25407b970 con 0x7fd2600c2e70 2023-11-28T03:11:30.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.459+0000 7fd25f7fe700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 0x7fd24c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:30.464 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:30.459+0000 7f0d42761680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T03:11:30.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.467+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd25404cc40 con 0x7fd2600c2e70 2023-11-28T03:11:30.524 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:30.523+0000 7f0d42761680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:11:30.544 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:30.543+0000 7f889d04f680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T03:11:30.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.660+0000 7fd25f7fe700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 msgr2=0x7fd24c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:30.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.660+0000 7fd25f7fe700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 0x7fd24c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:30.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.661+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fd260165410 con 0x7fd2600c2e70 2023-11-28T03:11:30.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.665+0000 7fd247fff700 1 -- 172.21.15.73:0/1911313214 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v27)=0 v27) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd254020070 con 0x7fd2600c2e70 2023-11-28T03:11:30.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 msgr2=0x7fd24c061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:30.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 0x7fd24c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 msgr2=0x7fd26014cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 rx=0x7fd254009ea0 tx=0x7fd254017000).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 shutdown_connections 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd24c05f1a0 0x7fd24c061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd260155830 0x7fd260164b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd2600c2e70 0x7fd26014cd40 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 --2- 172.21.15.73:0/1911313214 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd260155030 0x7fd260168b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 >> 172.21.15.73:0/1911313214 conn(0x7fd260062700 msgr2=0x7fd26014ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:30.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 shutdown_connections 2023-11-28T03:11:30.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.667+0000 7fd266837700 1 -- 172.21.15.73:0/1911313214 wait complete. 2023-11-28T03:11:30.680 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T03:11:30.681 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T03:11:30.699 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1 1M 2023-11-28T03:11:30.797 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:30.796+0000 7f0d42761680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:11:30.828 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:30.827+0000 7f889d04f680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T03:11:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.858+0000 7f5873d6e700 1 Processor -- start 2023-11-28T03:11:30.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.858+0000 7f5873d6e700 1 -- start start 2023-11-28T03:11:30.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5873d6e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f586c154610 0x7f586c1582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5873d6e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c159210 con 0x7f586c0aa1c0 2023-11-28T03:11:30.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5873d6e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f586c0c0c70 con 0x7f586c0ab3b0 2023-11-28T03:11:30.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5873d6e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f586c158970 con 0x7f586c154610 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f587230b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f586c0aa1c0 0x7f586c0c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44320/0 (socket says 172.21.15.73:44320) 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f587230b700 1 -- 172.21.15.73:0/2297129026 learned_addr learned my addr 172.21.15.73:0/2297129026 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5871b0a700 1 --2- 172.21.15.73:0/2297129026 >> v2:172.21.15.150:3300/0 conn(0x7f586c154610 0x7f586c1582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390990623 0 0) 0x7f586c159210 con 0x7f586c0aa1c0 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f58580033f0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2445810606 0 0) 0x7f586c0c0c70 con 0x7f586c0ab3b0 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f586c159210 con 0x7f586c0ab3b0 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5871b0a700 1 -- 172.21.15.73:0/2297129026 >> v1:172.21.15.150:6789/0 conn(0x7f586c0ab3b0 legacy=0x7f586c0c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5871b0a700 1 -- 172.21.15.73:0/2297129026 >> v1:172.21.15.73:6789/0 conn(0x7f586c0aa1c0 legacy=0x7f586c0c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.859+0000 7f5871b0a700 1 -- 172.21.15.73:0/2297129026 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f586c158c50 con 0x7f586c154610 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5871b0a700 1 --2- 172.21.15.73:0/2297129026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c154610 0x7f586c1582e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f585c009760 tx=0x7f585c009060).ready entity=mon.1 client_cookie=d5d866a30d167ff5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00d4a0 con 0x7f586c154610 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00d600 con 0x7f586c154610 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5870b08700 1 -- 172.21.15.73:0/2297129026 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00d980 con 0x7f586c154610 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 -- 172.21.15.73:0/2297129026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c154610 msgr2=0x7f586c1582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 --2- 172.21.15.73:0/2297129026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c154610 0x7f586c1582e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 rx=0x7f585c009760 tx=0x7f585c009060).stop 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 -- 172.21.15.73:0/2297129026 shutdown_connections 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 --2- 172.21.15.73:0/2297129026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c154610 0x7f586c1582e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 -- 172.21.15.73:0/2297129026 >> 172.21.15.73:0/2297129026 conn(0x7f586c0655a0 msgr2=0x7f586c069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:30.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 -- 172.21.15.73:0/2297129026 shutdown_connections 2023-11-28T03:11:30.867 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.860+0000 7f5873d6e700 1 -- 172.21.15.73:0/2297129026 wait complete. 2023-11-28T03:11:30.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 Processor -- start 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 -- start start 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 0x7f586c13b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586c154610 0x7f586c0ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1575a0 con 0x7f586c0ae6e0 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586c156500 con 0x7f586c0aa1c0 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f5873d6e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f586c1567e0 con 0x7f586c154610 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f587230b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.862+0000 7f587230b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35658/0 (socket says 172.21.15.73:35658) 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f587230b700 1 -- 172.21.15.73:0/1031249630 learned_addr learned my addr 172.21.15.73:0/1031249630 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:30.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 0x7f586c13b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586c154610 0x7f586c0ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586c154610 msgr2=0x7f586c0ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586c154610 0x7f586c0ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 msgr2=0x7f586c168040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c0092a0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f587230b700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5871b0a700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 0x7f586c13b5e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f585c013000 tx=0x7f585c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00dcf0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c00de50 con 0x7f586c0aa1c0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f586c156ac0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00d980 con 0x7f586c0aa1c0 2023-11-28T03:11:30.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.863+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f586c157080 con 0x7f586c0aa1c0 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.864+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f586c06c860 con 0x7f586c0aa1c0 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.865+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 57) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f585c0263b0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.865+0000 7f5861ffb700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.865+0000 7f5871309700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.865+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(140..140 src has 1..140) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f585c07aff0 con 0x7f586c0aa1c0 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.865+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T03:11:30.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:30.869+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f585c04c260 con 0x7f586c0aa1c0 2023-11-28T03:11:31.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:31.061+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f586c158f30 con 0x7f585005f1a0 2023-11-28T03:11:31.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:31.066+0000 7f5871309700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:31.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:31.066+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T03:11:31.410 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:31.409+0000 7f0d42761680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:11:31.445 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:31.444+0000 7f889d04f680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T03:11:31.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:31.467+0000 7f5871309700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:31.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:31.467+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T03:11:31.481 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:31.480+0000 7f0d42761680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:11:31.517 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:31.516+0000 7f889d04f680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T03:11:31.670 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:31.669+0000 7f0d42761680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:11:31.703 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:31.702+0000 7f889d04f680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T03:11:32.090 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.089+0000 7f0d42761680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:11:32.120 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.119+0000 7f889d04f680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T03:11:32.194 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.193+0000 7f0d42761680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:32.224 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.222+0000 7f889d04f680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:32.268+0000 7f5871309700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:32.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:32.268+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T03:11:32.348 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.347+0000 7f0d42761680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:32.376 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.375+0000 7f889d04f680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:32.423 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.421+0000 7f0d42761680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:11:32.451 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.450+0000 7f889d04f680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T03:11:32.489 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.488+0000 7f0d42761680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:11:32.518 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.517+0000 7f889d04f680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T03:11:32.747 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.746+0000 7f0d42761680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:11:32.777 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.776+0000 7f889d04f680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T03:11:32.930 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:32.928+0000 7f0d42761680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:32.962 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:32.961+0000 7f889d04f680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T03:11:33.578 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:33.577+0000 7f0d42761680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:11:33.611 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:33.610+0000 7f889d04f680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T03:11:33.647 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:33.646+0000 7f0d42761680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:11:33.678 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:33.677+0000 7f889d04f680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T03:11:33.723 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:33.722+0000 7f0d42761680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:11:33.753 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:33.752+0000 7f889d04f680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T03:11:33.870 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:33.869+0000 7f0d42761680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:11:33.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:33.869+0000 7f5871309700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T03:11:33.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:33.869+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T03:11:33.899 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:33.898+0000 7f889d04f680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T03:11:33.941 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:33.940+0000 7f0d42761680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:11:33.968 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:33.967+0000 7f889d04f680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T03:11:34.055 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:34.054+0000 7f0d42761680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:11:34.084 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.083+0000 7f889d04f680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T03:11:34.164 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:34.163+0000 7f0d42761680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:34.191 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.190+0000 7f889d04f680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T03:11:34.458 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:34.457+0000 7f0d42761680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:11:34.483 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.482+0000 7f889d04f680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T03:11:34.530 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T03:11:34.528+0000 7f0d42761680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:11:34.555 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.554+0000 7f889d04f680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T03:11:34.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:34.576+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mgrmap(e 58) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f585c078dc0 con 0x7f586c0aa1c0 2023-11-28T03:11:34.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:34.576+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f5850061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T03:11:34.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:34.576+0000 7f5861ffb700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f5850061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T03:11:34.636 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f8810fd4700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.637 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f8810fd4700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.637 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f8810fd4700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.637 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f8810fd4700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.638 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f8810fd4700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.638 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f88147db700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.638 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f88147db700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.638 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f88147db700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.639 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f88147db700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:34.639 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:11:34.635+0000 7f88147db700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:11:35.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.576+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mon.1 v2:172.21.15.150:3300/0 8 ==== mgrmap(e 59) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f585c03d820 con 0x7f586c0aa1c0 2023-11-28T03:11:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.576+0000 7f5861ffb700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f58500615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.576+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f586c158f30 con 0x7f585005f1a0 2023-11-28T03:11:35.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.578+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f58500615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.597+0000 7f5871309700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f58500615e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f586c14f630 tx=0x7f5868007bd0).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:35.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.602+0000 7f5861ffb700 1 -- 172.21.15.73:0/1031249630 <== mgr.8289 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f586c158f30 con 0x7f585005f1a0 2023-11-28T03:11:35.604 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1 2023-11-28T03:11:35.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 msgr2=0x7f58500615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f58500615e0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 rx=0x7f586c14f630 tx=0x7f5868007bd0).stop 2023-11-28T03:11:35.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 msgr2=0x7f586c13b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 0x7f586c13b5e0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 rx=0x7f585c013000 tx=0x7f585c016000).stop 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 shutdown_connections 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f585005f1a0 0x7f58500615e0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f586c0ae6e0 0x7f586c168040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f586c0aa1c0 0x7f586c13b5e0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 --2- 172.21.15.73:0/1031249630 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f586c154610 0x7f586c0ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 >> 172.21.15.73:0/1031249630 conn(0x7f586c0655a0 msgr2=0x7f586c14dd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 shutdown_connections 2023-11-28T03:11:35.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.605+0000 7f5873d6e700 1 -- 172.21.15.73:0/1031249630 wait complete. 2023-11-28T03:11:35.619 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-11-28T03:11:35.620 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs /snap_test_dir1 55n 2023-11-28T03:11:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.740+0000 7f8c63391700 1 Processor -- start 2023-11-28T03:11:35.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.740+0000 7f8c63391700 1 -- start start 2023-11-28T03:11:35.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c63391700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 0x7f8c5c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c63391700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c0c72d0 con 0x7f8c5c098d10 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c63391700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c5c064570 con 0x7f8c5c0c6190 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c63391700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c154bc0 con 0x7f8c5c158a30 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6112d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8c5c098d10 0x7f8c5c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47470/0 (socket says 172.21.15.73:47470) 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6112d700 1 -- 172.21.15.73:0/1706757531 learned_addr learned my addr 172.21.15.73:0/1706757531 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6192e700 1 --2- 172.21.15.73:0/1706757531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 0x7f8c5c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080392230 0 0) 0x7f8c5c0c72d0 con 0x7f8c5c098d10 2023-11-28T03:11:35.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c480033f0 con 0x7f8c5c098d10 2023-11-28T03:11:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1239359519 0 0) 0x7f8c5c064570 con 0x7f8c5c0c6190 2023-11-28T03:11:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c5c0c72d0 con 0x7f8c5c0c6190 2023-11-28T03:11:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2733863462 0 0) 0x7f8c480033f0 con 0x7f8c5c098d10 2023-11-28T03:11:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6192e700 1 -- 172.21.15.73:0/1706757531 >> v1:172.21.15.150:6789/0 conn(0x7f8c5c0c6190 legacy=0x7f8c5c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6192e700 1 -- 172.21.15.73:0/1706757531 >> v1:172.21.15.73:6789/0 conn(0x7f8c5c098d10 legacy=0x7f8c5c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.741+0000 7f8c6192e700 1 -- 172.21.15.73:0/1706757531 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c5c0c66c0 con 0x7f8c5c158a30 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c6192e700 1 --2- 172.21.15.73:0/1706757531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 0x7f8c5c0c7760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8c58009790 tx=0x7f8c58009060).ready entity=mon.2 client_cookie=a89b5634ba0682d0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c540017f0 con 0x7f8c5c158a30 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58011400 con 0x7f8c5c158a30 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c53fff700 1 -- 172.21.15.73:0/1706757531 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58011780 con 0x7f8c5c158a30 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 -- 172.21.15.73:0/1706757531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 msgr2=0x7f8c5c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 --2- 172.21.15.73:0/1706757531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 0x7f8c5c0c7760 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 rx=0x7f8c58009790 tx=0x7f8c58009060).stop 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 -- 172.21.15.73:0/1706757531 shutdown_connections 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 --2- 172.21.15.73:0/1706757531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c158a30 0x7f8c5c0c7760 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 -- 172.21.15.73:0/1706757531 >> 172.21.15.73:0/1706757531 conn(0x7f8c5c0cb310 msgr2=0x7f8c5c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 -- 172.21.15.73:0/1706757531 shutdown_connections 2023-11-28T03:11:35.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.742+0000 7f8c63391700 1 -- 172.21.15.73:0/1706757531 wait complete. 2023-11-28T03:11:35.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.743+0000 7f8c63391700 1 Processor -- start 2023-11-28T03:11:35.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.743+0000 7f8c63391700 1 -- start start 2023-11-28T03:11:35.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c098d10 0x7f8c5c14b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c5c0c6190 0x7f8c5c0c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0afd20 con 0x7f8c5c158a30 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0c51c0 con 0x7f8c5c0c6190 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c63391700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c5c0c4920 con 0x7f8c5c098d10 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c6192e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c6192e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46150/0 (socket says 172.21.15.73:46150) 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c6192e700 1 -- 172.21.15.73:0/782648468 learned_addr learned my addr 172.21.15.73:0/782648468 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.744+0000 7f8c6112d700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c098d10 0x7f8c5c14b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6092c700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c5c0c6190 0x7f8c5c0c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 -- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c098d10 msgr2=0x7f8c5c14b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c098d10 0x7f8c5c14b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 -- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c5c0c6190 msgr2=0x7f8c5c0c3e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c5c0c6190 0x7f8c5c0c3e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 -- 172.21.15.73:0/782648468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c580092a0 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c6192e700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f8c5c0c2360 tx=0x7f8c58016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58021070 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c58010a20 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c5c0c4c00 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c58010dc0 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.745+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c5c0c23b0 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.746+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c5c06c860 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.746+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 59) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f8c5801d640 con 0x7f8c5c158a30 2023-11-28T03:11:35.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.747+0000 7f8c517fa700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 0x7f8c40061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:35.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.747+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f8c5807b360 con 0x7f8c5c158a30 2023-11-28T03:11:35.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.747+0000 7f8c6112d700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 0x7f8c40061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:35.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.747+0000 7f8c6112d700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 0x7f8c40061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8c54001920 tx=0x7f8c54000f40).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:35.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.751+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8c5804c750 con 0x7f8c5c158a30 2023-11-28T03:11:35.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.937+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f8c5c0afe60 con 0x7f8c4005f080 2023-11-28T03:11:35.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.942+0000 7f8c517fa700 1 -- 172.21.15.73:0/782648468 <== mgr.8289 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f8c540020d0 con 0x7f8c4005f080 2023-11-28T03:11:35.943 INFO:teuthology.orchestra.run.smithi073.stdout:Retention added to path /snap_test_dir1 2023-11-28T03:11:35.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 msgr2=0x7f8c40061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 0x7f8c40061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8c54001920 tx=0x7f8c54000f40).stop 2023-11-28T03:11:35.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 msgr2=0x7f8c5c0c4350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 rx=0x7f8c5c0c2360 tx=0x7f8c58016000).stop 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 shutdown_connections 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8c4005f080 0x7f8c40061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8c5c158a30 0x7f8c5c0c4350 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8c5c098d10 0x7f8c5c14b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 --2- 172.21.15.73:0/782648468 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8c5c0c6190 0x7f8c5c0c3e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 >> 172.21.15.73:0/782648468 conn(0x7f8c5c0cb310 msgr2=0x7f8c5c14bf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 shutdown_connections 2023-11-28T03:11:35.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:35.944+0000 7f8c63391700 1 -- 172.21.15.73:0/782648468 wait complete. 2023-11-28T03:11:35.957 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-11-28T03:11:35.957 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-11-28T03:11:35.957 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json /snap_test_dir1 2023-11-28T03:11:36.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.072+0000 7f1871230700 1 Processor -- start 2023-11-28T03:11:36.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.072+0000 7f1871230700 1 -- start start 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f186c1373a0 0x7f186c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 0x7f186c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 0x7f186c066e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c0c68d0 con 0x7f186c1373a0 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f186c158e20 con 0x7f186c0c5f30 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f1871230700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1582a0 con 0x7f186c0669a0 2023-11-28T03:11:36.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f186ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f186c1373a0 0x7f186c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f186ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f186c1373a0 0x7f186c0c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46160/0 (socket says 172.21.15.73:46160) 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.073+0000 7f186ad9d700 1 -- 172.21.15.73:0/2046466165 learned_addr learned my addr 172.21.15.73:0/2046466165 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186259c700 1 --2- 172.21.15.73:0/2046466165 >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 0x7f186c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 -- 172.21.15.73:0/2046466165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 msgr2=0x7f186c066e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186b59e700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 0x7f186c066e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 0x7f186c066e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 -- 172.21.15.73:0/2046466165 >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 msgr2=0x7f186c0c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 --2- 172.21.15.73:0/2046466165 >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 0x7f186c0c63e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 -- 172.21.15.73:0/2046466165 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f186c158580 con 0x7f186c1373a0 2023-11-28T03:11:36.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186259c700 1 --2- 172.21.15.73:0/2046466165 >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 0x7f186c0c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186b59e700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 0x7f186c066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186ad9d700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c1373a0 0x7f186c0c72e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f186c069110 tx=0x7f185c0090f0).ready entity=mon.0 client_cookie=cba52e3a646aadca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186a59c700 1 -- 172.21.15.73:0/2046466165 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00b600 con 0x7f186c1373a0 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186a59c700 1 -- 172.21.15.73:0/2046466165 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c00b760 con 0x7f186c1373a0 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f186a59c700 1 -- 172.21.15.73:0/2046466165 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00bae0 con 0x7f186c1373a0 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 -- 172.21.15.73:0/2046466165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c1373a0 msgr2=0x7f186c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c1373a0 0x7f186c0c72e0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 rx=0x7f186c069110 tx=0x7f185c0090f0).stop 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 -- 172.21.15.73:0/2046466165 shutdown_connections 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c1373a0 0x7f186c0c72e0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 --2- 172.21.15.73:0/2046466165 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c0669a0 0x7f186c066e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 --2- 172.21.15.73:0/2046466165 >> v2:172.21.15.150:3300/0 conn(0x7f186c0c5f30 0x7f186c0c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 -- 172.21.15.73:0/2046466165 >> 172.21.15.73:0/2046466165 conn(0x7f186c0628f0 msgr2=0x7f186c062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:36.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 -- 172.21.15.73:0/2046466165 shutdown_connections 2023-11-28T03:11:36.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.074+0000 7f1871230700 1 -- 172.21.15.73:0/2046466165 wait complete. 2023-11-28T03:11:36.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 Processor -- start 2023-11-28T03:11:36.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 -- start start 2023-11-28T03:11:36.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f186c0c5f30 0x7f186c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c1373a0 0x7f186c167f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1574b0 con 0x7f186c0669a0 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f186c1568a0 con 0x7f186c0c5f30 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.076+0000 7f1871230700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f186c156b80 con 0x7f186c1373a0 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46170/0 (socket says 172.21.15.73:46170) 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 -- 172.21.15.73:0/3406391736 learned_addr learned my addr 172.21.15.73:0/3406391736 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186b59e700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c1373a0 0x7f186c167f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186259c700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f186c0c5f30 0x7f186c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 -- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c1373a0 msgr2=0x7f186c167f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c1373a0 0x7f186c167f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 -- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f186c0c5f30 msgr2=0x7f186c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f186c0c5f30 0x7f186c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 -- 172.21.15.73:0/3406391736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c009370 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f186ad9d700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f185c009130 tx=0x7f185c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c00bae0 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f185c000c90 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f186c156e60 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f185c01c650 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.077+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f186c14e380 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.078+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f186c06c860 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.078+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 59) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f185c01ca00 con 0x7f186c0669a0 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.079+0000 7f18637fe700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 0x7f184c061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T03:11:36.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.079+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(141..141 src has 1..141) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f185c07ba10 con 0x7f186c0669a0 2023-11-28T03:11:36.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.079+0000 7f186259c700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 0x7f184c061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T03:11:36.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.079+0000 7f186259c700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 0x7f184c061510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f186c0adb60 tx=0x7f1854008040).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T03:11:36.084 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.083+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f185c04ce00 con 0x7f186c0669a0 2023-11-28T03:11:36.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.272+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f186c14e5a0 con 0x7f184c05f050 2023-11-28T03:11:36.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.275+0000 7f18637fe700 1 -- 172.21.15.73:0/3406391736 <== mgr.8289 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7f186c14e5a0 con 0x7f184c05f050 2023-11-28T03:11:36.276 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T03:11:36.276 INFO:teuthology.orchestra.run.smithi073.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-11-28T03:11:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 msgr2=0x7f184c061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:36.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 0x7f184c061510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f186c0adb60 tx=0x7f1854008040).stop 2023-11-28T03:11:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 msgr2=0x7f186c14de90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T03:11:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 rx=0x7f185c009130 tx=0x7f185c017000).stop 2023-11-28T03:11:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 shutdown_connections 2023-11-28T03:11:36.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f184c05f050 0x7f184c061510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f186c1373a0 0x7f186c167f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f186c0669a0 0x7f186c14de90 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 --2- 172.21.15.73:0/3406391736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f186c0c5f30 0x7f186c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T03:11:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 >> 172.21.15.73:0/3406391736 conn(0x7f186c0628f0 msgr2=0x7f186c066e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T03:11:36.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 shutdown_connections 2023-11-28T03:11:36.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T03:11:36.277+0000 7f1871230700 1 -- 172.21.15.73:0/3406391736 wait complete. 2023-11-28T03:11:36.290 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-11-28T03:11:36.290 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-11-28T03:11:53.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:11:53.578 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:12:00.947 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:12:00.946+0000 7f88037b9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:12:00.948 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:12:00.946+0000 7f88037b9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:12:00.948 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:12:00.946+0000 7f88037b9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:12:00.948 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:12:00.946+0000 7f88037b9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:12:00.948 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T03:12:00.946+0000 7f88037b9700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T03:12:23.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:12:23.672 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:12:53.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:12:53.755 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:13:23.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:13:24.001 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:13:54.056 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:13:54.059 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:14:24.204 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:14:24.208 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:14:54.336 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:14:54.340 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:15:24.474 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:15:24.478 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:15:54.614 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:15:54.617 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:16:24.757 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:16:24.760 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:16:54.880 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:16:54.883 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:17:24.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:17:24.997 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:17:55.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:17:55.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:18:25.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:18:25.318 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:18:55.463 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:18:55.466 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:19:25.652 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:19:25.655 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:19:55.726 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:19:55.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:20:25.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:20:25.875 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:20:56.045 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:20:56.048 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:21:26.174 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:21:26.177 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:21:56.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:21:56.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:22:26.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:22:26.448 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:22:56.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:22:56.596 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:23:26.723 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:23:26.726 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:23:56.879 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:23:56.882 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:24:27.028 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:24:27.031 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:24:57.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:24:57.145 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:25:27.287 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:25:27.290 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:25:57.428 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:25:57.434 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:26:27.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:26:27.722 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:26:57.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:26:57.880 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:27:28.030 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:27:28.033 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:27:58.203 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:27:58.206 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:28:28.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:28:28.354 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:28:58.467 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:28:58.470 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:29:28.606 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:29:28.609 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:29:58.742 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:29:58.745 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:30:28.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:30:28.852 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:30:58.988 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:30:58.991 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:31:29.127 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:31:29.130 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:31:59.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:31:59.294 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:32:29.523 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:32:29.528 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:32:59.659 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:32:59.663 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:33:29.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:33:29.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:33:59.894 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:33:59.898 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:34:30.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:34:32.194 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:35:02.331 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:35:02.334 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:35:32.458 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:35:32.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:36:02.621 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:36:02.625 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:36:32.770 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:36:32.773 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:37:02.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:37:02.929 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:37:32.998 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:37:33.003 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:38:03.078 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:38:03.081 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:38:33.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:38:33.228 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:39:03.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:39:03.511 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:39:33.643 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:39:33.647 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:40:03.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:40:03.777 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:40:33.888 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:40:33.891 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:41:03.993 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:41:04.000 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:41:34.136 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:41:34.140 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:42:04.284 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:42:04.288 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:42:34.375 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:42:34.378 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:43:04.530 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:43:04.533 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:43:34.684 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:43:34.687 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:44:04.808 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:44:04.811 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:44:34.930 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:44:34.933 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:45:05.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:45:05.089 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:45:35.242 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:45:35.245 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:46:05.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:46:05.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:46:35.462 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:46:35.465 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:47:05.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:47:05.593 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:47:35.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:47:35.731 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:48:05.860 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:48:05.863 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:48:35.979 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:48:35.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:49:06.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:49:06.113 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:49:36.258 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:49:36.261 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:50:06.429 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:50:06.433 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:50:36.519 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:50:36.522 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:51:06.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:51:06.586 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:51:36.789 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:51:36.793 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:52:06.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:52:06.938 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:52:37.057 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:52:37.061 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:53:07.198 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:53:07.202 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:53:37.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:53:37.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:54:07.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:54:07.496 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:54:37.607 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:54:37.610 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:55:07.950 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:55:07.953 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:55:38.096 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:55:38.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:56:08.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:56:08.161 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:56:38.306 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:56:38.309 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:57:08.436 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:57:08.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:57:38.600 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:57:38.603 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:58:08.740 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:58:08.743 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:58:38.878 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:58:38.881 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:59:09.025 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:59:09.028 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:59:39.088 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T03:59:39.092 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:00:09.239 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:00:09.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:00:39.386 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:00:39.389 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:01:09.526 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:01:09.529 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:01:39.682 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:01:39.685 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:02:09.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:02:09.818 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:02:39.936 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:02:39.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:03:10.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:03:10.079 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:03:40.141 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:03:40.144 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:04:10.321 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:04:10.324 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:04:40.444 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:04:40.447 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:05:10.592 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:05:11.714 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:05:42.011 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:05:42.015 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:06:12.095 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:06:12.099 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:06:42.260 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:06:42.263 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:07:12.403 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:07:12.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:07:16.290 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:16.291 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_13_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_14_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_15_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_16_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_17_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_18_00_UTC 2023-11-28T04:07:16.313 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_19_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_20_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_21_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_22_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_23_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_24_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_25_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_26_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_27_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_28_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_29_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_30_00_UTC 2023-11-28T04:07:16.314 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_31_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_32_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_33_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_34_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_35_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_36_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_37_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_38_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_39_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_40_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_41_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_42_00_UTC 2023-11-28T04:07:16.315 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_43_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_44_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_45_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_46_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_47_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_48_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_49_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_50_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_51_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_52_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_53_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_54_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_55_00_UTC 2023-11-28T04:07:16.316 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_56_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_57_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_58_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_59_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_00_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_01_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_02_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_03_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_04_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_05_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_06_00_UTC 2023-11-28T04:07:16.317 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_07_00_UTC 2023-11-28T04:07:16.318 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-03_13_00_UTC', 'scheduled-2023-11-28-03_14_00_UTC', 'scheduled-2023-11-28-03_15_00_UTC', 'scheduled-2023-11-28-03_16_00_UTC', 'scheduled-2023-11-28-03_17_00_UTC', 'scheduled-2023-11-28-03_18_00_UTC', 'scheduled-2023-11-28-03_19_00_UTC', 'scheduled-2023-11-28-03_20_00_UTC', 'scheduled-2023-11-28-03_21_00_UTC', 'scheduled-2023-11-28-03_22_00_UTC', 'scheduled-2023-11-28-03_23_00_UTC', 'scheduled-2023-11-28-03_24_00_UTC', 'scheduled-2023-11-28-03_25_00_UTC', 'scheduled-2023-11-28-03_26_00_UTC', 'scheduled-2023-11-28-03_27_00_UTC', 'scheduled-2023-11-28-03_28_00_UTC', 'scheduled-2023-11-28-03_29_00_UTC', 'scheduled-2023-11-28-03_30_00_UTC', 'scheduled-2023-11-28-03_31_00_UTC', 'scheduled-2023-11-28-03_32_00_UTC', 'scheduled-2023-11-28-03_33_00_UTC', 'scheduled-2023-11-28-03_34_00_UTC', 'scheduled-2023-11-28-03_35_00_UTC', 'scheduled-2023-11-28-03_36_00_UTC', 'scheduled-2023-11-28-03_37_00_UTC', 'scheduled-2023-11-28-03_38_00_UTC', 'scheduled-2023-11-28-03_39_00_UTC', 'scheduled-2023-11-28-03_40_00_UTC', 'scheduled-2023-11-28-03_41_00_UTC', 'scheduled-2023-11-28-03_42_00_UTC', 'scheduled-2023-11-28-03_43_00_UTC', 'scheduled-2023-11-28-03_44_00_UTC', 'scheduled-2023-11-28-03_45_00_UTC', 'scheduled-2023-11-28-03_46_00_UTC', 'scheduled-2023-11-28-03_47_00_UTC', 'scheduled-2023-11-28-03_48_00_UTC', 'scheduled-2023-11-28-03_49_00_UTC', 'scheduled-2023-11-28-03_50_00_UTC', 'scheduled-2023-11-28-03_51_00_UTC', 'scheduled-2023-11-28-03_52_00_UTC', 'scheduled-2023-11-28-03_53_00_UTC', 'scheduled-2023-11-28-03_54_00_UTC', 'scheduled-2023-11-28-03_55_00_UTC', 'scheduled-2023-11-28-03_56_00_UTC', 'scheduled-2023-11-28-03_57_00_UTC', 'scheduled-2023-11-28-03_58_00_UTC', 'scheduled-2023-11-28-03_59_00_UTC', 'scheduled-2023-11-28-04_00_00_UTC', 'scheduled-2023-11-28-04_01_00_UTC', 'scheduled-2023-11-28-04_02_00_UTC', 'scheduled-2023-11-28-04_03_00_UTC', 'scheduled-2023-11-28-04_04_00_UTC', 'scheduled-2023-11-28-04_05_00_UTC', 'scheduled-2023-11-28-04_06_00_UTC', 'scheduled-2023-11-28-04_07_00_UTC'] 2023-11-28T04:07:16.318 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1 2023-11-28T04:07:16.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.479+0000 7f5418a19700 1 Processor -- start 2023-11-28T04:07:16.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.479+0000 7f5418a19700 1 -- start start 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5418a19700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 0x7f54140a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5418a19700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5418a19700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5414158d50 con 0x7f54140c5fc0 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5418a19700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54140c6b60 con 0x7f54141542d0 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5418a19700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54140aa950 con 0x7f54141546b0 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:36538/0 (socket says 172.21.15.73:36538) 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 -- 172.21.15.73:0/3638541529 learned_addr learned my addr 172.21.15.73:0/3638541529 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:16.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5412d9d700 1 --2- 172.21.15.73:0/3638541529 >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 0x7f54140a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f541259c700 1 -- 172.21.15.73:0/3638541529 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 261004698 0 0) 0x7f5414158d50 con 0x7f54140c5fc0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f541259c700 1 -- 172.21.15.73:0/3638541529 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f54000033f0 con 0x7f54140c5fc0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 -- 172.21.15.73:0/3638541529 >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 msgr2=0x7f54140a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 --2- 172.21.15.73:0/3638541529 >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 0x7f54140a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 -- 172.21.15.73:0/3638541529 >> v1:172.21.15.73:6789/0 conn(0x7f54140c5fc0 legacy=0x7f54140c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f540a59c700 1 -- 172.21.15.73:0/3638541529 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5414158140 con 0x7f54141546b0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.480+0000 7f5412d9d700 1 --2- 172.21.15.73:0/3638541529 >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 0x7f54140a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f540a59c700 1 --2- 172.21.15.73:0/3638541529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f53fc006fd0 tx=0x7f53fc0090f0).ready entity=mon.2 client_cookie=4e765e46c076aaf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f541259c700 1 -- 172.21.15.73:0/3638541529 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00f510 con 0x7f54141546b0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f541259c700 1 -- 172.21.15.73:0/3638541529 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53fc00f670 con 0x7f54141546b0 2023-11-28T04:07:16.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f541259c700 1 -- 172.21.15.73:0/3638541529 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53fc00f9f0 con 0x7f54141546b0 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 -- 172.21.15.73:0/3638541529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 msgr2=0x7f54140aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 --2- 172.21.15.73:0/3638541529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 rx=0x7f53fc006fd0 tx=0x7f53fc0090f0).stop 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 -- 172.21.15.73:0/3638541529 shutdown_connections 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 --2- 172.21.15.73:0/3638541529 >> v2:172.21.15.73:3300/0 conn(0x7f54141542d0 0x7f54140a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 --2- 172.21.15.73:0/3638541529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140aa460 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 -- 172.21.15.73:0/3638541529 >> 172.21.15.73:0/3638541529 conn(0x7f54140cb010 msgr2=0x7f54140cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 -- 172.21.15.73:0/3638541529 shutdown_connections 2023-11-28T04:07:16.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.481+0000 7f5418a19700 1 -- 172.21.15.73:0/3638541529 wait complete. 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.483+0000 7f5418a19700 1 Processor -- start 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.483+0000 7f5418a19700 1 -- start start 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.483+0000 7f5418a19700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.483+0000 7f5418a19700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54141542d0 0x7f5414147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140ac910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414152580 con 0x7f54140c5fc0 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5414151970 con 0x7f54141542d0 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5414151c50 con 0x7f54141546b0 2023-11-28T04:07:16.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47834/0 (socket says 172.21.15.73:47834) 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 -- 172.21.15.73:0/12829073 learned_addr learned my addr 172.21.15.73:0/12829073 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f540a59c700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54141542d0 0x7f5414147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f541359e700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140ac910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 -- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 msgr2=0x7f54140ac910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140ac910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 -- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54141542d0 msgr2=0x7f5414147d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54141542d0 0x7f5414147d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 -- 172.21.15.73:0/12829073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54040092a0 con 0x7f54140c5fc0 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5412d9d700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f540400ea20 tx=0x7f5404009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400c850 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f540400c9b0 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53fc009370 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f540400cd30 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.484+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5414152210 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.485+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54140c23f0 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.485+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 62) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f5404012600 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.486+0000 7f540b7fe700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 0x7f53f4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.486+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6395+0+0 (secure 0 0 0) 0x7f540407ab90 con 0x7f54140c5fc0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.486+0000 7f540a59c700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 0x7f53f4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.486+0000 7f540a59c700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 0x7f53f4061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f53fc009130 tx=0x7f53fc017000).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.490+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f540404bbd0 con 0x7f54140c5fc0 2023-11-28T04:07:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.676+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f54140c59c0 con 0x7f53f405f2c0 2023-11-28T04:07:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.678+0000 7f540b7fe700 1 -- 172.21.15.73:0/12829073 <== mgr.8289 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7f54140c59c0 con 0x7f53f405f2c0 2023-11-28T04:07:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:16.679 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:11:35", "first": "2023-11-28T03:12:00", "last": "2023-11-28T04:07:00", "last_pruned": "2023-11-28T04:07:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 msgr2=0x7f53f4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 0x7f53f4061780 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 rx=0x7f53fc009130 tx=0x7f53fc017000).stop 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 msgr2=0x7f5414151170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 rx=0x7f540400ea20 tx=0x7f5404009060).stop 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 shutdown_connections 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f53f405f2c0 0x7f53f4061780 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f54141546b0 0x7f54140ac910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f54140c5fc0 0x7f5414151170 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 --2- 172.21.15.73:0/12829073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f54141542d0 0x7f5414147d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 >> 172.21.15.73:0/12829073 conn(0x7f54140cb010 msgr2=0x7f5414152ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 shutdown_connections 2023-11-28T04:07:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.680+0000 7f5418a19700 1 -- 172.21.15.73:0/12829073 wait complete. 2023-11-28T04:07:16.692 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T03:11:35", "first": "2023-11-28T03:12:00", "last": "2023-11-28T04:07:00", "last_pruned": "2023-11-28T04:07:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-11-28T04:07:16.693 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T03:11:35', 'first': '2023-11-28T03:12:00', 'last': '2023-11-28T04:07:00', 'last_pruned': '2023-11-28T04:07:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-11-28T04:07:16.693 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-11-28T04:07:16.693 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-11-28T04:07:16.693 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1 2023-11-28T04:07:16.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.809+0000 7fa6cb8b1700 1 Processor -- start 2023-11-28T04:07:16.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.809+0000 7fa6cb8b1700 1 -- start start 2023-11-28T04:07:16.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6cb8b1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6c4154520 0x7fa6c406a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.812 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6cb8b1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c40bd600 0x7fa6c406a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6cb8b1700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6c4067840 con 0x7fa6c4154900 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6cb8b1700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c40c50e0 con 0x7fa6c40bd600 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6cb8b1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6c4066c30 con 0x7fa6c4154520 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c40bd600 0x7fa6c406a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa6c40bd600 0x7fa6c406a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47846/0 (socket says 172.21.15.73:47846) 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/2240335538 learned_addr learned my addr 172.21.15.73:0/2240335538 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c964d700 1 --2- 172.21.15.73:0/2240335538 >> v2:172.21.15.150:3300/0 conn(0x7fa6c4154520 0x7fa6c406a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/2240335538 >> v2:172.21.15.150:3300/0 conn(0x7fa6c4154520 msgr2=0x7fa6c406a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 --2- 172.21.15.73:0/2240335538 >> v2:172.21.15.150:3300/0 conn(0x7fa6c4154520 0x7fa6c406a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/2240335538 >> v1:172.21.15.150:6789/0 conn(0x7fa6c4154900 legacy=0x7fa6c40bcef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.810+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/2240335538 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6c4066f10 con 0x7fa6c40bd600 2023-11-28T04:07:16.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6c9e4e700 1 --2- 172.21.15.73:0/2240335538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c40bd600 0x7fa6c406a8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fa6c4069660 tx=0x7fa6bc009840).ready entity=mon.0 client_cookie=ef1206c354afc660 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6bbfff700 1 -- 172.21.15.73:0/2240335538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00e070 con 0x7fa6c40bd600 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6bbfff700 1 -- 172.21.15.73:0/2240335538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc00ba20 con 0x7fa6c40bd600 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6bbfff700 1 -- 172.21.15.73:0/2240335538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc00bda0 con 0x7fa6c40bd600 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/2240335538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c40bd600 msgr2=0x7fa6c406a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/2240335538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c40bd600 0x7fa6c406a8a0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 rx=0x7fa6c4069660 tx=0x7fa6bc009840).stop 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/2240335538 shutdown_connections 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/2240335538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c40bd600 0x7fa6c406a8a0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/2240335538 >> v2:172.21.15.150:3300/0 conn(0x7fa6c4154520 0x7fa6c406a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/2240335538 >> 172.21.15.73:0/2240335538 conn(0x7fa6c4063810 msgr2=0x7fa6c4063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/2240335538 shutdown_connections 2023-11-28T04:07:16.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.811+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/2240335538 wait complete. 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.812+0000 7fa6cb8b1700 1 Processor -- start 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.812+0000 7fa6cb8b1700 1 -- start start 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 0x7fa6c414d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4154520 0x7fa6c40a6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c414d890 con 0x7fa6c4154900 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40a67c0 con 0x7fa6c40bd600 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6cb8b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6c40a5bb0 con 0x7fa6c4154520 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.813+0000 7fa6c9e4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47854/0 (socket says 172.21.15.73:47854) 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/3610764569 learned_addr learned my addr 172.21.15.73:0/3610764569 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c8e4c700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4154520 0x7fa6c40a6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c964d700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 0x7fa6c414d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4154520 msgr2=0x7fa6c40a6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4154520 0x7fa6c40a6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 msgr2=0x7fa6c414d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 0x7fa6c414d3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 -- 172.21.15.73:0/3610764569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6bc009ae0 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c964d700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 0x7fa6c414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6c9e4e700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fa6bc010000 tx=0x7fa6bc009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc007080 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6bc013e50 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6c40a5e90 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6bc01d660 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.814+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6c40a6450 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.815+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6c40a6950 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.815+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 62) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fa6bc01da50 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.816+0000 7fa6b97fa700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 0x7fa6a8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.816+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6395+0+0 (secure 0 0 0) 0x7fa6bc07c1c0 con 0x7fa6c4154900 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.816+0000 7fa6c964d700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 0x7fa6a8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.816+0000 7fa6c964d700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 0x7fa6a8061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa6c0009c00 tx=0x7fa6c0008040).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:16.820+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6bc050cf0 con 0x7fa6c4154900 2023-11-28T04:07:17.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.007+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa6c40671f0 con 0x7fa6a805f2c0 2023-11-28T04:07:17.012 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.011+0000 7fa6b97fa700 1 -- 172.21.15.73:0/3610764569 <== mgr.8289 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7fa6c40671f0 con 0x7fa6a805f2c0 2023-11-28T04:07:17.012 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1 2023-11-28T04:07:17.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 msgr2=0x7fa6a8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 0x7fa6a8061780 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 rx=0x7fa6c0009c00 tx=0x7fa6c0008040).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 msgr2=0x7fa6c4153110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 rx=0x7fa6bc010000 tx=0x7fa6bc009840).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 shutdown_connections 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6a805f2c0 0x7fa6a8061780 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6c4154900 0x7fa6c4153110 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6c40bd600 0x7fa6c414d3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 --2- 172.21.15.73:0/3610764569 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6c4154520 0x7fa6c40a6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 >> 172.21.15.73:0/3610764569 conn(0x7fa6c4063810 msgr2=0x7fa6c40c46c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 shutdown_connections 2023-11-28T04:07:17.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:17.013+0000 7fa6cb8b1700 1 -- 172.21.15.73:0/3610764569 wait complete. 2023-11-28T04:07:17.027 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-11-28T04:07:17.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.028 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:07:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_13_00_UTC 2023-11-28T04:07:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_14_00_UTC 2023-11-28T04:07:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_15_00_UTC 2023-11-28T04:07:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_16_00_UTC 2023-11-28T04:07:17.045 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_17_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_18_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_19_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_20_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_21_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_22_00_UTC 2023-11-28T04:07:17.046 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_23_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_24_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_25_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_26_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_27_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_28_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_29_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_30_00_UTC 2023-11-28T04:07:17.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_31_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_32_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_33_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_34_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_35_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_36_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_37_00_UTC 2023-11-28T04:07:17.048 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_38_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_39_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_40_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_41_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_42_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_43_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_44_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_45_00_UTC 2023-11-28T04:07:17.049 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_46_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_47_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_48_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_49_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_50_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_51_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_52_00_UTC 2023-11-28T04:07:17.050 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_53_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_54_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_55_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_56_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_57_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_58_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-03_59_00_UTC 2023-11-28T04:07:17.051 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_00_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_01_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_02_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_03_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_04_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_05_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_06_00_UTC 2023-11-28T04:07:17.052 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_07_00_UTC 2023-11-28T04:07:17.053 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_13_00_UTC 2023-11-28T04:07:17.053 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.053 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_13_00_UTC) 2023-11-28T04:07:17.105 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_14_00_UTC 2023-11-28T04:07:17.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.106 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_14_00_UTC) 2023-11-28T04:07:17.166 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_15_00_UTC 2023-11-28T04:07:17.166 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.166 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_15_00_UTC) 2023-11-28T04:07:17.226 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_16_00_UTC 2023-11-28T04:07:17.226 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.227 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_16_00_UTC) 2023-11-28T04:07:17.286 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_17_00_UTC 2023-11-28T04:07:17.286 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.286 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_17_00_UTC) 2023-11-28T04:07:17.347 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_18_00_UTC 2023-11-28T04:07:17.347 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.347 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_18_00_UTC) 2023-11-28T04:07:17.409 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_19_00_UTC 2023-11-28T04:07:17.410 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.410 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_19_00_UTC) 2023-11-28T04:07:17.469 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_20_00_UTC 2023-11-28T04:07:17.469 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.470 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_20_00_UTC) 2023-11-28T04:07:17.530 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_21_00_UTC 2023-11-28T04:07:17.530 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.530 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_21_00_UTC) 2023-11-28T04:07:17.589 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_22_00_UTC 2023-11-28T04:07:17.590 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.590 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_22_00_UTC) 2023-11-28T04:07:17.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_23_00_UTC 2023-11-28T04:07:17.650 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.651 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_23_00_UTC) 2023-11-28T04:07:17.710 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_24_00_UTC 2023-11-28T04:07:17.710 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.710 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_24_00_UTC) 2023-11-28T04:07:17.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_25_00_UTC 2023-11-28T04:07:17.770 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.770 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_25_00_UTC) 2023-11-28T04:07:17.829 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_26_00_UTC 2023-11-28T04:07:17.830 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.830 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_26_00_UTC) 2023-11-28T04:07:17.889 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_27_00_UTC 2023-11-28T04:07:17.889 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.890 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_27_00_UTC) 2023-11-28T04:07:17.949 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_28_00_UTC 2023-11-28T04:07:17.950 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:17.950 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_28_00_UTC) 2023-11-28T04:07:18.010 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_29_00_UTC 2023-11-28T04:07:18.010 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.010 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_29_00_UTC) 2023-11-28T04:07:18.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_30_00_UTC 2023-11-28T04:07:18.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.073 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_30_00_UTC) 2023-11-28T04:07:18.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_31_00_UTC 2023-11-28T04:07:18.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.133 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_31_00_UTC) 2023-11-28T04:07:18.192 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_32_00_UTC 2023-11-28T04:07:18.193 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.193 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_32_00_UTC) 2023-11-28T04:07:18.252 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_33_00_UTC 2023-11-28T04:07:18.253 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.253 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_33_00_UTC) 2023-11-28T04:07:18.312 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_34_00_UTC 2023-11-28T04:07:18.313 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.313 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_34_00_UTC) 2023-11-28T04:07:18.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_35_00_UTC 2023-11-28T04:07:18.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.373 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_35_00_UTC) 2023-11-28T04:07:18.433 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_36_00_UTC 2023-11-28T04:07:18.433 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.433 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_36_00_UTC) 2023-11-28T04:07:18.492 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_37_00_UTC 2023-11-28T04:07:18.493 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.493 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_37_00_UTC) 2023-11-28T04:07:18.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_38_00_UTC 2023-11-28T04:07:18.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.554 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_38_00_UTC) 2023-11-28T04:07:18.614 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_39_00_UTC 2023-11-28T04:07:18.614 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.614 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_39_00_UTC) 2023-11-28T04:07:18.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_40_00_UTC 2023-11-28T04:07:18.673 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.674 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_40_00_UTC) 2023-11-28T04:07:18.734 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_41_00_UTC 2023-11-28T04:07:18.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.735 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_41_00_UTC) 2023-11-28T04:07:18.796 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_42_00_UTC 2023-11-28T04:07:18.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.796 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_42_00_UTC) 2023-11-28T04:07:18.857 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_43_00_UTC 2023-11-28T04:07:18.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.857 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_43_00_UTC) 2023-11-28T04:07:18.920 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_44_00_UTC 2023-11-28T04:07:18.920 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.920 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_44_00_UTC) 2023-11-28T04:07:18.979 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_45_00_UTC 2023-11-28T04:07:18.979 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:18.980 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_45_00_UTC) 2023-11-28T04:07:19.040 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_46_00_UTC 2023-11-28T04:07:19.040 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.040 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_46_00_UTC) 2023-11-28T04:07:19.102 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_47_00_UTC 2023-11-28T04:07:19.102 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.102 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_47_00_UTC) 2023-11-28T04:07:19.162 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_48_00_UTC 2023-11-28T04:07:19.162 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.163 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_48_00_UTC) 2023-11-28T04:07:19.223 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_49_00_UTC 2023-11-28T04:07:19.223 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.224 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_49_00_UTC) 2023-11-28T04:07:19.284 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_50_00_UTC 2023-11-28T04:07:19.284 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.284 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_50_00_UTC) 2023-11-28T04:07:19.343 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_51_00_UTC 2023-11-28T04:07:19.343 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.344 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_51_00_UTC) 2023-11-28T04:07:19.403 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_52_00_UTC 2023-11-28T04:07:19.403 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.403 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_52_00_UTC) 2023-11-28T04:07:19.464 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_53_00_UTC 2023-11-28T04:07:19.464 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.464 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_53_00_UTC) 2023-11-28T04:07:19.523 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_54_00_UTC 2023-11-28T04:07:19.523 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.524 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_54_00_UTC) 2023-11-28T04:07:19.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_55_00_UTC 2023-11-28T04:07:19.598 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.598 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_55_00_UTC) 2023-11-28T04:07:19.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_56_00_UTC 2023-11-28T04:07:19.665 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.665 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_56_00_UTC) 2023-11-28T04:07:19.733 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_57_00_UTC 2023-11-28T04:07:19.733 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.733 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_57_00_UTC) 2023-11-28T04:07:19.811 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_58_00_UTC 2023-11-28T04:07:19.811 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.812 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_58_00_UTC) 2023-11-28T04:07:19.891 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-03_59_00_UTC 2023-11-28T04:07:19.891 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.892 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-03_59_00_UTC) 2023-11-28T04:07:19.952 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_00_00_UTC 2023-11-28T04:07:19.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:19.953 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_00_00_UTC) 2023-11-28T04:07:20.013 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_01_00_UTC 2023-11-28T04:07:20.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.013 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_01_00_UTC) 2023-11-28T04:07:20.072 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_02_00_UTC 2023-11-28T04:07:20.073 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.073 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_02_00_UTC) 2023-11-28T04:07:20.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_03_00_UTC 2023-11-28T04:07:20.132 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.133 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_03_00_UTC) 2023-11-28T04:07:20.192 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_04_00_UTC 2023-11-28T04:07:20.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.192 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_04_00_UTC) 2023-11-28T04:07:20.252 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_05_00_UTC 2023-11-28T04:07:20.252 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.253 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_05_00_UTC) 2023-11-28T04:07:20.312 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_06_00_UTC 2023-11-28T04:07:20.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.312 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_06_00_UTC) 2023-11-28T04:07:20.372 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_07_00_UTC 2023-11-28T04:07:20.372 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.372 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_07_00_UTC) 2023-11-28T04:07:20.430 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:20.431 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T04:07:20.488 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T04:07:20.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.660+0000 7fb0f15ee700 1 Processor -- start 2023-11-28T04:07:20.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.660+0000 7fb0f15ee700 1 -- start start 2023-11-28T04:07:20.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 0x7fb0ec0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec154950 0x7fb0ec14e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0ec158570 0x7fb0ec158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c7bc0 con 0x7fb0ec158570 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0ec155920 con 0x7fb0ec14e530 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f15ee700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec154dd0 con 0x7fb0ec154950 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0ec158570 0x7fb0ec158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb0ec158570 0x7fb0ec158a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47858/0 (socket says 172.21.15.73:47858) 2023-11-28T04:07:20.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 -- 172.21.15.73:0/2877087395 learned_addr learned my addr 172.21.15.73:0/2877087395 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:20.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0eb7fe700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec154950 0x7fb0ec14e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0ebfff700 1 --2- 172.21.15.73:0/2877087395 >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 0x7fb0ec0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 -- 172.21.15.73:0/2877087395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec154950 msgr2=0x7fb0ec14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:20.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec154950 0x7fb0ec14e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 -- 172.21.15.73:0/2877087395 >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 msgr2=0x7fb0ec0c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 --2- 172.21.15.73:0/2877087395 >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 0x7fb0ec0c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 -- 172.21.15.73:0/2877087395 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0ec155080 con 0x7fb0ec158570 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0ebfff700 1 --2- 172.21.15.73:0/2877087395 >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 0x7fb0ec0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0f0ded700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec158570 0x7fb0ec158a00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fb0ec064450 tx=0x7fb0e4009840).ready entity=mon.0 client_cookie=681ca8f942c1f603 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0eaffd700 1 -- 172.21.15.73:0/2877087395 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400e070 con 0x7fb0ec158570 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0eaffd700 1 -- 172.21.15.73:0/2877087395 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e400ba50 con 0x7fb0ec158570 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.661+0000 7fb0eaffd700 1 -- 172.21.15.73:0/2877087395 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e400bdd0 con 0x7fb0ec158570 2023-11-28T04:07:20.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 -- 172.21.15.73:0/2877087395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec158570 msgr2=0x7fb0ec158a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec158570 0x7fb0ec158a00 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 rx=0x7fb0ec064450 tx=0x7fb0e4009840).stop 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 -- 172.21.15.73:0/2877087395 shutdown_connections 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec158570 0x7fb0ec158a00 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/2877087395 >> v2:172.21.15.150:3300/0 conn(0x7fb0ec14e530 0x7fb0ec0c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/2877087395 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec154950 0x7fb0ec14e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 -- 172.21.15.73:0/2877087395 >> 172.21.15.73:0/2877087395 conn(0x7fb0ec064990 msgr2=0x7fb0ec064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 -- 172.21.15.73:0/2877087395 shutdown_connections 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.662+0000 7fb0f15ee700 1 -- 172.21.15.73:0/2877087395 wait complete. 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.663+0000 7fb0f15ee700 1 Processor -- start 2023-11-28T04:07:20.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.663+0000 7fb0f15ee700 1 -- start start 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec14e530 0x7fb0ec0c3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ec158570 0x7fb0ec0c1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c2dd0 con 0x7fb0ec154950 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c21c0 con 0x7fb0ec158570 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f15ee700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0ec0c24a0 con 0x7fb0ec14e530 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47864/0 (socket says 172.21.15.73:47864) 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 -- 172.21.15.73:0/3754153603 learned_addr learned my addr 172.21.15.73:0/3754153603 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0f0ded700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ec158570 0x7fb0ec0c1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0ebfff700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec14e530 0x7fb0ec0c3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 -- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec14e530 msgr2=0x7fb0ec0c3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec14e530 0x7fb0ec0c3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 -- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ec158570 msgr2=0x7fb0ec0c1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ec158570 0x7fb0ec0c1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 -- 172.21.15.73:0/3754153603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e4009ae0 con 0x7fb0ec154950 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.664+0000 7fb0eb7fe700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7fb0e000d890 tx=0x7fb0e0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.665+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0011590 con 0x7fb0ec154950 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.665+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0e00116f0 con 0x7fb0ec154950 2023-11-28T04:07:20.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.665+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0ec0c2780 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.665+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0011a70 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.665+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0ec14dfa0 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.666+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0ec062ee0 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.666+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 62) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fb0e001c460 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.666+0000 7fb0d3fff700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 0x7fb0d8061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.666+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fb0e007a240 con 0x7fb0ec154950 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.666+0000 7fb0ebfff700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 0x7fb0d8061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.667+0000 7fb0ebfff700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 0x7fb0d8061780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc008040).ready entity=mgr.8289 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:20.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.670+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0e004b1f0 con 0x7fb0ec154950 2023-11-28T04:07:20.879 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:20.878+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb0ec06c7c0 con 0x7fb0ec154950 2023-11-28T04:07:21.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.555+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fb0e0077d50 con 0x7fb0ec154950 2023-11-28T04:07:21.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.556+0000 7fb0d3fff700 1 -- 172.21.15.73:0/3754153603 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v63) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb0e0072590 con 0x7fb0ec154950 2023-11-28T04:07:21.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.558+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 msgr2=0x7fb0d8061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.558+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 0x7fb0d8061780 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 rx=0x7fb0dc006fd0 tx=0x7fb0dc008040).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.558+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 msgr2=0x7fb0ec14dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.558+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 rx=0x7fb0e000d890 tx=0x7fb0e0008040).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 shutdown_connections 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb0d805f2c0 0x7fb0d8061780 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0ec158570 0x7fb0ec0c1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0ec14e530 0x7fb0ec0c3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 --2- 172.21.15.73:0/3754153603 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0ec154950 0x7fb0ec14dab0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 >> 172.21.15.73:0/3754153603 conn(0x7fb0ec064990 msgr2=0x7fb0ec153e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 shutdown_connections 2023-11-28T04:07:21.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.559+0000 7fb0f15ee700 1 -- 172.21.15.73:0/3754153603 wait complete. 2023-11-28T04:07:21.573 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:07:21.573 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:07:21.621 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:07:21.657 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:07:21.658 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T04:07:21.658 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:21.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T04:07:21.710 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:07:21.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:21.710 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T04:07:21.715 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:21.715+0000 7fcd5764b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:07:21.718 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:21.712+0000 7f4ce2c9b680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:07:21.738 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:21.806 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:21.805+0000 7f4ce2c9b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:07:21.807 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:21.807+0000 7fcd5764b680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:07:21.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.896+0000 7fe3fb934700 1 Processor -- start 2023-11-28T04:07:21.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.896+0000 7fe3fb934700 1 -- start start 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe3f41542d0 0x7fe3f40697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3f4069ce0 0x7fe3f4062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f4157fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f4065760 con 0x7fe3f4069ce0 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3f4063080 con 0x7fe3f41542d0 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3fb934700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40c3250 con 0x7fe3f4062bf0 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3f8ecf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3f4069ce0 0x7fe3f4062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3f8ecf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe3f4069ce0 0x7fe3f4062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47904/0 (socket says 172.21.15.73:47904) 2023-11-28T04:07:21.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/2753499478 learned_addr learned my addr 172.21.15.73:0/2753499478 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3f9ed1700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f4157fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.897+0000 7fe3f96d0700 1 --2- 172.21.15.73:0/2753499478 >> v2:172.21.15.150:3300/0 conn(0x7fe3f41542d0 0x7fe3f40697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/2753499478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 msgr2=0x7fe3f4157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f4157fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/2753499478 >> v2:172.21.15.150:3300/0 conn(0x7fe3f41542d0 msgr2=0x7fe3f40697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/2753499478 >> v2:172.21.15.150:3300/0 conn(0x7fe3f41542d0 0x7fe3f40697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/2753499478 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f406a340 con 0x7fe3f4069ce0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4062700 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fe3f40961f0 tx=0x7fe3f00090f0).ready entity=mon.0 client_cookie=779c86642c06097d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3ebfff700 1 -- 172.21.15.73:0/2753499478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000f510 con 0x7fe3f4069ce0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3ebfff700 1 -- 172.21.15.73:0/2753499478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000f670 con 0x7fe3f4069ce0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3ebfff700 1 -- 172.21.15.73:0/2753499478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f000fa10 con 0x7fe3f4069ce0 2023-11-28T04:07:21.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3fb934700 1 -- 172.21.15.73:0/2753499478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 msgr2=0x7fe3f4062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.898+0000 7fe3fb934700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4062700 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 rx=0x7fe3f40961f0 tx=0x7fe3f00090f0).stop 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 -- 172.21.15.73:0/2753499478 shutdown_connections 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f4157fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 --2- 172.21.15.73:0/2753499478 >> v2:172.21.15.150:3300/0 conn(0x7fe3f41542d0 0x7fe3f40697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 --2- 172.21.15.73:0/2753499478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4062700 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 -- 172.21.15.73:0/2753499478 >> 172.21.15.73:0/2753499478 conn(0x7fe3f40c7fd0 msgr2=0x7fe3f40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 -- 172.21.15.73:0/2753499478 shutdown_connections 2023-11-28T04:07:21.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.899+0000 7fe3fb934700 1 -- 172.21.15.73:0/2753499478 wait complete. 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.900+0000 7fe3fb934700 1 Processor -- start 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.900+0000 7fe3fb934700 1 -- start start 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f40a94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 0x7fe3f4167d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40aaf80 con 0x7fe3f4069ce0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40a99b0 con 0x7fe3f41542d0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f40aa930 con 0x7fe3f4062bf0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47916/0 (socket says 172.21.15.73:47916) 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/1416418583 learned_addr learned my addr 172.21.15.73:0/1416418583 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f96d0700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f40a94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f9ed1700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 0x7fe3f4167d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 msgr2=0x7fe3f40a94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f40a94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 msgr2=0x7fe3f4167d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 0x7fe3f4167d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:21.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 -- 172.21.15.73:0/1416418583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3f0009370 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f9ed1700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 0x7fe3f4167d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3f8ecf700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fe3f0006fd0 tx=0x7fe3f0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0000bc0 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3f000f7e0 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f40ab220 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.901+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3f0024470 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.902+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f40a8fa0 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.903+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f40a9310 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.903+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fe3f0021070 con 0x7fe3f4069ce0 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.903+0000 7fe3e97fa700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 0x7fe3d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:21.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.903+0000 7fe3f96d0700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 msgr2=0x7fe3d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:21.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.903+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fe3f007bdc0 con 0x7fe3f4069ce0 2023-11-28T04:07:21.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.904+0000 7fe3f96d0700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 0x7fe3d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:21.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:21.908+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe3f004cdd0 con 0x7fe3f4069ce0 2023-11-28T04:07:22.093 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:22.093+0000 7fcd5764b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:07:22.105 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:22.104+0000 7f4ce2c9b680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:07:22.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.105+0000 7fe3f96d0700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 msgr2=0x7fe3d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:22.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.105+0000 7fe3f96d0700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 0x7fe3d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.134+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe3f406c860 con 0x7fe3f4069ce0 2023-11-28T04:07:22.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.134+0000 7fe3e97fa700 1 -- 172.21.15.73:0/1416418583 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7fe3f007d070 con 0x7fe3f4069ce0 2023-11-28T04:07:22.136 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:22.136 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.136+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 msgr2=0x7fe3d8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.136+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 0x7fe3d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.136+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 msgr2=0x7fe3f4163e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.136+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 rx=0x7fe3f0006fd0 tx=0x7fe3f0017000).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 shutdown_connections 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe3d805f270 0x7fe3d8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe3f41542d0 0x7fe3f4167d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe3f4062bf0 0x7fe3f40a94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 --2- 172.21.15.73:0/1416418583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe3f4069ce0 0x7fe3f4163e70 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 >> 172.21.15.73:0/1416418583 conn(0x7fe3f40c7fd0 msgr2=0x7fe3f40a6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 shutdown_connections 2023-11-28T04:07:22.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.137+0000 7fe3fb934700 1 -- 172.21.15.73:0/1416418583 wait complete. 2023-11-28T04:07:22.139 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T04:07:22.150 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:22.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.308+0000 7fb2bf2ab700 1 Processor -- start 2023-11-28T04:07:22.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.308+0000 7fb2bf2ab700 1 -- start start 2023-11-28T04:07:22.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bf2ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80c2fe0 0x7fb2b8155280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bf2ab700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb2b80683d0 0x7fb2b8068860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bf2ab700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b813a5b0 con 0x7fb2b813a1d0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bf2ab700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b80c3460 con 0x7fb2b80683d0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bf2ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8155a90 con 0x7fb2b80c2fe0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2be2a9700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb2b813a1d0 0x7fb2b8154b70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45824/0 (socket says 172.21.15.73:45824) 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2be2a9700 1 -- 172.21.15.73:0/3514412999 learned_addr learned my addr 172.21.15.73:0/3514412999 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> v2:172.21.15.150:3300/0 conn(0x7fb2b80683d0 0x7fb2b8068860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80c2fe0 0x7fb2b8155280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 601661586 0 0) 0x7fb2b813a5b0 con 0x7fb2b813a1d0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.310+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a40033f0 con 0x7fb2b813a1d0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2788694150 0 0) 0x7fb2a40033f0 con 0x7fb2b813a1d0 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80c2fe0 msgr2=0x7fb2b8155280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80c2fe0 0x7fb2b8155280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 >> v2:172.21.15.150:3300/0 conn(0x7fb2b80683d0 msgr2=0x7fb2b8068860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 --2- 172.21.15.73:0/3514412999 >> v2:172.21.15.150:3300/0 conn(0x7fb2b80683d0 0x7fb2b8068860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80c1c50 con 0x7fb2b813a1d0 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb2b4001470 con 0x7fb2b813a1d0 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 >> v1:172.21.15.73:6789/0 conn(0x7fb2b813a1d0 legacy=0x7fb2b8154b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a40038e0 0x7fb2a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4004690 0x7fb2a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2a4005000 0x7fb2a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c3460 con 0x7fb2a4005000 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8155a90 con 0x7fb2a40038e0 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a400bdd0 con 0x7fb2a4004690 2023-11-28T04:07:22.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> v2:172.21.15.150:3300/0 conn(0x7fb2b80683d0 0x7fb2b8068860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80c2fe0 0x7fb2b8155280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2a4005000 0x7fb2a400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2be2a9700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a40038e0 0x7fb2a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4004690 0x7fb2a4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 -- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4004690 msgr2=0x7fb2a4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4004690 0x7fb2a4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 -- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a40038e0 msgr2=0x7fb2a4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2a40038e0 0x7fb2a4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 -- 172.21.15.73:0/3514412999 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a40033f0 con 0x7fb2a4005000 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2a4005000 0x7fb2a400b8e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fb2b000e3c0 tx=0x7fb2b000b040).ready entity=mon.0 client_cookie=29d13c74286b83b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0017070 con 0x7fb2a4005000 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b000f040 con 0x7fb2a4005000 2023-11-28T04:07:22.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.311+0000 7fb2bd2a7700 1 -- 172.21.15.73:0/3514412999 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b000ee80 con 0x7fb2a4005000 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.312+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/3514412999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2a4005000 msgr2=0x7fb2a400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.312+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/3514412999 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2a4005000 0x7fb2a400b8e0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 rx=0x7fb2b000e3c0 tx=0x7fb2b000b040).stop 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.312+0000 7fb2beaaa700 1 -- 172.21.15.73:0/3514412999 reap_dead start 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.312+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/3514412999 shutdown_connections 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.312+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/3514412999 >> 172.21.15.73:0/3514412999 conn(0x7fb2b80c6ba0 msgr2=0x7fb2b80c6fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.313+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/3514412999 shutdown_connections 2023-11-28T04:07:22.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.313+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/3514412999 wait complete. 2023-11-28T04:07:22.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.314+0000 7fb2bf2ab700 1 Processor -- start 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.314+0000 7fb2bf2ab700 1 -- start start 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4005000 0x7fb2b8150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80bfeb0 0x7fb2b806a050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b806a540 con 0x7fb2b80bfa30 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8136cd0 con 0x7fb2b80bfeb0 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8151130 con 0x7fb2a4005000 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47936/0 (socket says 172.21.15.73:47936) 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 -- 172.21.15.73:0/2414092552 learned_addr learned my addr 172.21.15.73:0/2414092552 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2be2a9700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4005000 0x7fb2b8150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2beaaa700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80bfeb0 0x7fb2b806a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 -- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4005000 msgr2=0x7fb2b8150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4005000 0x7fb2b8150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 -- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80bfeb0 msgr2=0x7fb2b806a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80bfeb0 0x7fb2b806a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 -- 172.21.15.73:0/2414092552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80c0410 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bdaa8700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fb2a80119f0 tx=0x7fb2a8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800f760 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a800f8c0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80c06f0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800fc60 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.315+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b80c0cb0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.316+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b8136aa0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.317+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fb2a8014650 con 0x7fb2b80bfa30 2023-11-28T04:07:22.321 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.317+0000 7fb2ae7fc700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 0x7fb29c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.317+0000 7fb2be2a9700 1 -- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 msgr2=0x7fb29c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.317+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fb2a807ad20 con 0x7fb2b80bfa30 2023-11-28T04:07:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.317+0000 7fb2be2a9700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 0x7fb29c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:22.322 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.321+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2b8136aa0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.490+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2b8136aa0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.491+0000 7fb2ae7fc700 1 -- 172.21.15.73:0/2414092552 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v149) v1 ==== 74+0+17448 (secure 0 0 0) 0x7fb2b8136aa0 con 0x7fb2b80bfa30 2023-11-28T04:07:22.492 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:22.492 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":149,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:22.020677+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"148","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":112,"snap_epoch":148,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":5}]},{"pgid":"17.17","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/603729121":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6833/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6832/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:0/1414794090":"2023-11-29T03:11:34.567794+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":41,"length":16},{"begin":95,"length":18}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":18,"snaps":[{"begin":3,"length":38},{"begin":59,"length":36}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:22.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 msgr2=0x7fb29c061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:22.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 0x7fb29c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 msgr2=0x7fb2b8069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 rx=0x7fb2a80119f0 tx=0x7fb2a8009060).stop 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 shutdown_connections 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb29c05f240 0x7fb29c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b80bfeb0 0x7fb2b806a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2a4005000 0x7fb2b8150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 --2- 172.21.15.73:0/2414092552 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80bfa30 0x7fb2b8069b60 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 >> 172.21.15.73:0/2414092552 conn(0x7fb2b80c6ba0 msgr2=0x7fb2b80bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 shutdown_connections 2023-11-28T04:07:22.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.493+0000 7fb2bf2ab700 1 -- 172.21.15.73:0/2414092552 wait complete. 2023-11-28T04:07:22.506 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '143', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 17, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T03:11:17.276027+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '139', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 18, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T03:11:17.703654+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '148', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 112, 'snap_epoch': 148, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T04:07:22.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:07:22.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.626+0000 7fd822aed700 1 Processor -- start 2023-11-28T04:07:22.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.626+0000 7fd822aed700 1 -- start start 2023-11-28T04:07:22.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd822aed700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd81c0a8280 0x7fd81c0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd822aed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd822aed700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd81c067770 con 0x7fd81c14e600 2023-11-28T04:07:22.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd822aed700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd81c159180 con 0x7fd81c0a8280 2023-11-28T04:07:22.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd822aed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0be8d0 con 0x7fd81c158160 2023-11-28T04:07:22.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd820889700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd81c0a8280 0x7fd81c0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd82108a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd81c14e600 0x7fd81c14ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:45834/0 (socket says 172.21.15.73:45834) 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd82108a700 1 -- 172.21.15.73:0/2216989996 learned_addr learned my addr 172.21.15.73:0/2216989996 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.627+0000 7fd81bfff700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd820889700 1 -- 172.21.15.73:0/2216989996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 msgr2=0x7fd81c155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd820889700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd820889700 1 -- 172.21.15.73:0/2216989996 >> v1:172.21.15.73:6789/0 conn(0x7fd81c14e600 legacy=0x7fd81c14ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd820889700 1 -- 172.21.15.73:0/2216989996 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd81c158600 con 0x7fd81c0a8280 2023-11-28T04:07:22.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd820889700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c0a8660 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fd810009760 tx=0x7fd810009060).ready entity=mon.0 client_cookie=772824b67c65c2b9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd81b7fe700 1 -- 172.21.15.73:0/2216989996 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000b580 con 0x7fd81c0a8280 2023-11-28T04:07:22.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd81b7fe700 1 -- 172.21.15.73:0/2216989996 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b6e0 con 0x7fd81c0a8280 2023-11-28T04:07:22.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd81b7fe700 1 -- 172.21.15.73:0/2216989996 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000ba60 con 0x7fd81c0a8280 2023-11-28T04:07:22.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 -- 172.21.15.73:0/2216989996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 msgr2=0x7fd81c0a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c0a8660 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 rx=0x7fd810009760 tx=0x7fd810009060).stop 2023-11-28T04:07:22.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 -- 172.21.15.73:0/2216989996 shutdown_connections 2023-11-28T04:07:22.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c0a8660 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 --2- 172.21.15.73:0/2216989996 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 -- 172.21.15.73:0/2216989996 >> 172.21.15.73:0/2216989996 conn(0x7fd81c062700 msgr2=0x7fd81c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 -- 172.21.15.73:0/2216989996 shutdown_connections 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.628+0000 7fd822aed700 1 -- 172.21.15.73:0/2216989996 wait complete. 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 Processor -- start 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 -- start start 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 0x7fd81c064aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c14b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c2300 con 0x7fd81c0a8280 2023-11-28T04:07:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c1a40 con 0x7fd81c14e600 2023-11-28T04:07:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.630+0000 7fd822aed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd81c0c1cb0 con 0x7fd81c158160 2023-11-28T04:07:22.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47960/0 (socket says 172.21.15.73:47960) 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 -- 172.21.15.73:0/1545189185 learned_addr learned my addr 172.21.15.73:0/1545189185 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd82108a700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c14b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd81bfff700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 0x7fd81c064aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 msgr2=0x7fd81c14b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c14b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 msgr2=0x7fd81c064aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 0x7fd81c064aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 -- 172.21.15.73:0/1545189185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8100092a0 con 0x7fd81c0a8280 2023-11-28T04:07:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd81bfff700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 0x7fd81c064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:22.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd820889700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fd810009eb0 tx=0x7fd810017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81000be60 con 0x7fd81c0a8280 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd81000b850 con 0x7fd81c0a8280 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd81c0c1f90 con 0x7fd81c0a8280 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd81001b410 con 0x7fd81c0a8280 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.631+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd81c065460 con 0x7fd81c0a8280 2023-11-28T04:07:22.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.632+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd81c0a7990 con 0x7fd81c0a8280 2023-11-28T04:07:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.633+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd81001b840 con 0x7fd81c0a8280 2023-11-28T04:07:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.633+0000 7fd818ff9700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 0x7fd800061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.633+0000 7fd81bfff700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 msgr2=0x7fd800061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.633+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fd81007c0c0 con 0x7fd81c0a8280 2023-11-28T04:07:22.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.633+0000 7fd81bfff700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 0x7fd800061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:22.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.637+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd81004d0d0 con 0x7fd81c0a8280 2023-11-28T04:07:22.712 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:22.712+0000 7fcd5764b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:07:22.757 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:22.748+0000 7f4ce2c9b680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:07:22.784 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:22.784+0000 7fcd5764b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:07:22.822 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:22.821+0000 7f4ce2c9b680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:07:22.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.834+0000 7fd81bfff700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 msgr2=0x7fd800061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:22.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.834+0000 7fd81bfff700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 0x7fd800061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:22.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.838+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd81c06c860 con 0x7fd81c0a8280 2023-11-28T04:07:22.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.838+0000 7fd818ff9700 1 -- 172.21.15.73:0/1545189185 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v149) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd8100290b0 con 0x7fd81c0a8280 2023-11-28T04:07:22.839 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:07:22.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 msgr2=0x7fd800061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:22.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 0x7fd800061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 msgr2=0x7fd81c14ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 rx=0x7fd810009eb0 tx=0x7fd810017000).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 shutdown_connections 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd80005f270 0x7fd800061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd81c158160 0x7fd81c14b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd81c0a8280 0x7fd81c14ad50 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 --2- 172.21.15.73:0/1545189185 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd81c14e600 0x7fd81c064aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 >> 172.21.15.73:0/1545189185 conn(0x7fd81c062700 msgr2=0x7fd81c0becb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 shutdown_connections 2023-11-28T04:07:22.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.840+0000 7fd822aed700 1 -- 172.21.15.73:0/1545189185 wait complete. 2023-11-28T04:07:22.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:07:22.969 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:22.969+0000 7fcd5764b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:07:22.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.974+0000 7f101544b700 1 Processor -- start 2023-11-28T04:07:22.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.974+0000 7f101544b700 1 -- start start 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f101544b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f10100a8280 0x7f10100a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f101544b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f101014e600 0x7f101014ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f101544b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1010067770 con 0x7f1010158160 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f101544b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1010159180 con 0x7f101014e600 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f101544b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10100be8d0 con 0x7f10100a8280 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f100f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f101014e600 0x7f101014ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f100f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f101014e600 0x7f101014ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47976/0 (socket says 172.21.15.73:47976) 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f100f7fe700 1 -- 172.21.15.73:0/38311625 learned_addr learned my addr 172.21.15.73:0/38311625 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:22.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.975+0000 7f100effd700 1 --2- 172.21.15.73:0/38311625 >> v2:172.21.15.150:3300/0 conn(0x7f10100a8280 0x7f10100a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100f7fe700 1 -- 172.21.15.73:0/38311625 >> v2:172.21.15.150:3300/0 conn(0x7f10100a8280 msgr2=0x7f10100a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100f7fe700 1 --2- 172.21.15.73:0/38311625 >> v2:172.21.15.150:3300/0 conn(0x7f10100a8280 0x7f10100a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100f7fe700 1 -- 172.21.15.73:0/38311625 >> v1:172.21.15.150:6789/0 conn(0x7f1010158160 legacy=0x7f1010155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100dffb700 1 -- 172.21.15.73:0/38311625 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1967811301 0 0) 0x7f1010067770 con 0x7f1010158160 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100f7fe700 1 -- 172.21.15.73:0/38311625 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1010158600 con 0x7f101014e600 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100f7fe700 1 --2- 172.21.15.73:0/38311625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f101014e600 0x7f101014ea90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f10100c9050 tx=0x7f1008009840).ready entity=mon.0 client_cookie=95ca6abb612331e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100dffb700 1 -- 172.21.15.73:0/38311625 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800e070 con 0x7f101014e600 2023-11-28T04:07:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100dffb700 1 -- 172.21.15.73:0/38311625 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f100800ba20 con 0x7f101014e600 2023-11-28T04:07:22.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f100dffb700 1 -- 172.21.15.73:0/38311625 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f100800bda0 con 0x7f101014e600 2023-11-28T04:07:22.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f101544b700 1 -- 172.21.15.73:0/38311625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f101014e600 msgr2=0x7f101014ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.976+0000 7f101544b700 1 --2- 172.21.15.73:0/38311625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f101014e600 0x7f101014ea90 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 rx=0x7f10100c9050 tx=0x7f1008009840).stop 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 -- 172.21.15.73:0/38311625 shutdown_connections 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 --2- 172.21.15.73:0/38311625 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f101014e600 0x7f101014ea90 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 --2- 172.21.15.73:0/38311625 >> v2:172.21.15.150:3300/0 conn(0x7f10100a8280 0x7f10100a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 -- 172.21.15.73:0/38311625 >> 172.21.15.73:0/38311625 conn(0x7f1010062700 msgr2=0x7f1010062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 -- 172.21.15.73:0/38311625 shutdown_connections 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.977+0000 7f101544b700 1 -- 172.21.15.73:0/38311625 wait complete. 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.978+0000 7f101544b700 1 Processor -- start 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.978+0000 7f101544b700 1 -- start start 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 0x7f101014bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1010158160 0x7f10100641d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10100a90f0 con 0x7f10100a8280 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1010157410 con 0x7f1010158160 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f101015b220 con 0x7f101014e600 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47986/0 (socket says 172.21.15.73:47986) 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 -- 172.21.15.73:0/481534884 learned_addr learned my addr 172.21.15.73:0/481534884 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100f7fe700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1010158160 0x7f10100641d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100e7fc700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 0x7f101014bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 -- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 msgr2=0x7f101014bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 0x7f101014bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 -- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1010158160 msgr2=0x7f10100641d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1010158160 0x7f10100641d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 -- 172.21.15.73:0/481534884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1008009ae0 con 0x7f10100a8280 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100e7fc700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 0x7f101014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:22.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f100effd700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f100400e930 tx=0x7f1004009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004012600 con 0x7f10100a8280 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1004012760 con 0x7f10100a8280 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1010159c90 con 0x7f10100a8280 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.979+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1004012ae0 con 0x7f10100a8280 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.980+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f101015a250 con 0x7f10100a8280 2023-11-28T04:07:22.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.981+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff0005120 con 0x7f10100a8280 2023-11-28T04:07:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.981+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f100401b400 con 0x7f10100a8280 2023-11-28T04:07:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.982+0000 7f0ff6ffd700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 0x7f0ffc061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.982+0000 7f100e7fc700 1 -- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 msgr2=0x7f0ffc061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.982+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6439+0+0 (secure 0 0 0) 0x7f100407a860 con 0x7f10100a8280 2023-11-28T04:07:22.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.982+0000 7f100e7fc700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 0x7f0ffc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:22.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:22.986+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f100404b810 con 0x7f10100a8280 2023-11-28T04:07:23.020 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.019+0000 7f4ce2c9b680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:07:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.159+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0ff0005d70 con 0x7f10100a8280 2023-11-28T04:07:23.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.159+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/481534884 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v150) v1 ==== 92+0+10 (secure 0 0 0) 0x7f1004016020 con 0x7f10100a8280 2023-11-28T04:07:23.160 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 msgr2=0x7f0ffc061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 0x7f0ffc061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 msgr2=0x7f10100a8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 rx=0x7f100400e930 tx=0x7f1004009060).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 shutdown_connections 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.161+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0ffc05f270 0x7f0ffc061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1010158160 0x7f10100641d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f10100a8280 0x7f10100a8c00 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 --2- 172.21.15.73:0/481534884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f101014e600 0x7f101014bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 >> 172.21.15.73:0/481534884 conn(0x7f1010062700 msgr2=0x7f10100bf7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 shutdown_connections 2023-11-28T04:07:23.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.162+0000 7f101544b700 1 -- 172.21.15.73:0/481534884 wait complete. 2023-11-28T04:07:23.175 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T04:07:23.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.298+0000 7f1e45308700 1 Processor -- start 2023-11-28T04:07:23.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.298+0000 7f1e45308700 1 -- start start 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e45308700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e40158160 0x7f1e400a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e45308700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e4014e600 0x7f1e4014ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e45308700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e40067770 con 0x7f1e400a8280 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e45308700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e40159180 con 0x7f1e40158160 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e45308700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e400be8d0 con 0x7f1e4014e600 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e3e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e40158160 0x7f1e400a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e3e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e40158160 0x7f1e400a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48000/0 (socket says 172.21.15.73:48000) 2023-11-28T04:07:23.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.299+0000 7f1e3e7fc700 1 -- 172.21.15.73:0/3750810394 learned_addr learned my addr 172.21.15.73:0/3750810394 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3f7fe700 1 --2- 172.21.15.73:0/3750810394 >> v2:172.21.15.150:3300/0 conn(0x7f1e4014e600 0x7f1e4014ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3e7fc700 1 -- 172.21.15.73:0/3750810394 >> v2:172.21.15.150:3300/0 conn(0x7f1e4014e600 msgr2=0x7f1e4014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3e7fc700 1 --2- 172.21.15.73:0/3750810394 >> v2:172.21.15.150:3300/0 conn(0x7f1e4014e600 0x7f1e4014ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3e7fc700 1 -- 172.21.15.73:0/3750810394 >> v1:172.21.15.150:6789/0 conn(0x7f1e400a8280 legacy=0x7f1e40155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3e7fc700 1 -- 172.21.15.73:0/3750810394 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40158600 con 0x7f1e40158160 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3e7fc700 1 --2- 172.21.15.73:0/3750810394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e400a8660 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f1e40067fe0 tx=0x7f1e300090f0).ready entity=mon.0 client_cookie=51e9249a7b20fec7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3dffb700 1 -- 172.21.15.73:0/3750810394 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000b520 con 0x7f1e40158160 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3dffb700 1 -- 172.21.15.73:0/3750810394 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000b680 con 0x7f1e40158160 2023-11-28T04:07:23.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.300+0000 7f1e3dffb700 1 -- 172.21.15.73:0/3750810394 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000ba00 con 0x7f1e40158160 2023-11-28T04:07:23.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 -- 172.21.15.73:0/3750810394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 msgr2=0x7f1e400a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 --2- 172.21.15.73:0/3750810394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e400a8660 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 rx=0x7f1e40067fe0 tx=0x7f1e300090f0).stop 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 -- 172.21.15.73:0/3750810394 shutdown_connections 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 --2- 172.21.15.73:0/3750810394 >> v2:172.21.15.150:3300/0 conn(0x7f1e4014e600 0x7f1e4014ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 --2- 172.21.15.73:0/3750810394 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e400a8660 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 -- 172.21.15.73:0/3750810394 >> 172.21.15.73:0/3750810394 conn(0x7f1e40062700 msgr2=0x7f1e40062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 -- 172.21.15.73:0/3750810394 shutdown_connections 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.301+0000 7f1e45308700 1 -- 172.21.15.73:0/3750810394 wait complete. 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.302+0000 7f1e45308700 1 Processor -- start 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.302+0000 7f1e45308700 1 -- start start 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e400a8280 0x7f1e4014afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4014e600 0x7f1e400ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40156ba0 con 0x7f1e40158160 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4014b4a0 con 0x7f1e4014e600 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40155f90 con 0x7f1e400a8280 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:48014/0 (socket says 172.21.15.73:48014) 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 -- 172.21.15.73:0/442396554 learned_addr learned my addr 172.21.15.73:0/442396554 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3effd700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e400a8280 0x7f1e4014afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3e7fc700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4014e600 0x7f1e400ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 -- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e400a8280 msgr2=0x7f1e4014afb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e400a8280 0x7f1e4014afb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 -- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4014e600 msgr2=0x7f1e400ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4014e600 0x7f1e400ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 -- 172.21.15.73:0/442396554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e30009370 con 0x7f1e40158160 2023-11-28T04:07:23.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e3f7fe700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f1e38009e90 tx=0x7f1e3800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800caa0 con 0x7f1e40158160 2023-11-28T04:07:23.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800f040 con 0x7f1e40158160 2023-11-28T04:07:23.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e40156270 con 0x7f1e40158160 2023-11-28T04:07:23.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.303+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800ce10 con 0x7f1e40158160 2023-11-28T04:07:23.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.304+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e40156830 con 0x7f1e40158160 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e40156d10 con 0x7f1e40158160 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f1e38013610 con 0x7f1e40158160 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e26ffd700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 0x7f1e2c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e3effd700 1 -- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 msgr2=0x7f1e2c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7f1e3807ad10 con 0x7f1e40158160 2023-11-28T04:07:23.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.305+0000 7f1e3effd700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 0x7f1e2c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:23.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.309+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e3804bd20 con 0x7f1e40158160 2023-11-28T04:07:23.396 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:23.395+0000 7fcd5764b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:07:23.463 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.462+0000 7f4ce2c9b680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:07:23.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.479+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1e4006c860 con 0x7f1e40158160 2023-11-28T04:07:23.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.480+0000 7f1e26ffd700 1 -- 172.21.15.73:0/442396554 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v150) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1e3804f340 con 0x7f1e40158160 2023-11-28T04:07:23.481 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:07:23.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 msgr2=0x7f1e2c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:23.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 0x7f1e2c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 msgr2=0x7f1e40064990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 rx=0x7f1e38009e90 tx=0x7f1e3800b040).stop 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 shutdown_connections 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1e2c05f270 0x7f1e2c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e40158160 0x7f1e40064990 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e400a8280 0x7f1e4014afb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 --2- 172.21.15.73:0/442396554 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e4014e600 0x7f1e400ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 >> 172.21.15.73:0/442396554 conn(0x7f1e40062700 msgr2=0x7f1e4014bd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 shutdown_connections 2023-11-28T04:07:23.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.482+0000 7f1e45308700 1 -- 172.21.15.73:0/442396554 wait complete. 2023-11-28T04:07:23.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T04:07:23.500 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:23.500+0000 7fcd5764b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:23.579 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.575+0000 7f4ce2c9b680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:23.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.616+0000 7f336febb700 1 Processor -- start 2023-11-28T04:07:23.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.616+0000 7f336febb700 1 -- start start 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336febb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f33680c86f0 0x7f33680a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336febb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680c7e30 con 0x7f3368062700 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336febb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33680bcf40 con 0x7f33681547f0 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336febb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33680c6b20 con 0x7f33680c86f0 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336e458700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3368062700 0x7f33680a4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54576/0 (socket says 172.21.15.73:54576) 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336e458700 1 -- 172.21.15.73:0/1511051603 learned_addr learned my addr 172.21.15.73:0/1511051603 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336d456700 1 --2- 172.21.15.73:0/1511051603 >> v2:172.21.15.150:3300/0 conn(0x7f33680c86f0 0x7f33680a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460781754 0 0) 0x7f33680c7e30 con 0x7f3368062700 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.617+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33540033f0 con 0x7f3368062700 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847418483 0 0) 0x7f33680bcf40 con 0x7f33681547f0 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33680c7e30 con 0x7f33681547f0 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051157524 0 0) 0x7f33540033f0 con 0x7f3368062700 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336d456700 1 -- 172.21.15.73:0/1511051603 >> v1:172.21.15.150:6789/0 conn(0x7f33681547f0 legacy=0x7f33680c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336d456700 1 -- 172.21.15.73:0/1511051603 >> v1:172.21.15.73:6789/0 conn(0x7f3368062700 legacy=0x7f33680a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336d456700 1 -- 172.21.15.73:0/1511051603 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33680a77d0 con 0x7f33680c86f0 2023-11-28T04:07:23.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336d456700 1 --2- 172.21.15.73:0/1511051603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f33680a72e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f3364009760 tx=0x7f3364009060).ready entity=mon.1 client_cookie=1caad773b5a4ce3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33600017f0 con 0x7f33680c86f0 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33640090d0 con 0x7f33680c86f0 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.618+0000 7f336cc55700 1 -- 172.21.15.73:0/1511051603 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3364011610 con 0x7f33680c86f0 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 -- 172.21.15.73:0/1511051603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 msgr2=0x7f33680a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 --2- 172.21.15.73:0/1511051603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f33680a72e0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 rx=0x7f3364009760 tx=0x7f3364009060).stop 2023-11-28T04:07:23.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 -- 172.21.15.73:0/1511051603 shutdown_connections 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 --2- 172.21.15.73:0/1511051603 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f33680a72e0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 -- 172.21.15.73:0/1511051603 >> 172.21.15.73:0/1511051603 conn(0x7f33680642d0 msgr2=0x7f33680c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 -- 172.21.15.73:0/1511051603 shutdown_connections 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.619+0000 7f336febb700 1 -- 172.21.15.73:0/1511051603 wait complete. 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.620+0000 7f336febb700 1 Processor -- start 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.620+0000 7f336febb700 1 -- start start 2023-11-28T04:07:23.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3368062700 0x7f336813a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f3368150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3368151880 con 0x7f33681547f0 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33681528b0 con 0x7f33680c86f0 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33681581f0 con 0x7f3368062700 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46590/0 (socket says 172.21.15.73:46590) 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 -- 172.21.15.73:0/3923293274 learned_addr learned my addr 172.21.15.73:0/3923293274 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336d456700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f3368150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336dc57700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3368062700 0x7f336813a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 -- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3368062700 msgr2=0x7f336813a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3368062700 0x7f336813a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 -- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 msgr2=0x7f3368150ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f3368150ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 -- 172.21.15.73:0/3923293274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33640092a0 con 0x7f33681547f0 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336d456700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f3368150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336e458700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f336000cb90 tx=0x7f3360007bd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360004c40 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3360004da0 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f336814ef40 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.621+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3360007380 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.622+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f336814f500 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.623+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f336806c860 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.623+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f33600077b0 con 0x7f33681547f0 2023-11-28T04:07:23.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.623+0000 7f335dffb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 0x7f334c061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.623+0000 7f336dc57700 1 -- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 msgr2=0x7f334c061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.623+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7f336007a370 con 0x7f33681547f0 2023-11-28T04:07:23.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.624+0000 7f336dc57700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 0x7f334c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:23.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.627+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f336004b380 con 0x7f33681547f0 2023-11-28T04:07:23.659 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:23.658+0000 7fcd5764b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:23.734 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:23.733+0000 7fcd5764b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:07:23.736 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.735+0000 7f4ce2c9b680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:23.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.799+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3368157f40 con 0x7f33681547f0 2023-11-28T04:07:23.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.800+0000 7f335dffb700 1 -- 172.21.15.73:0/3923293274 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v150) v1 ==== 100+0+11 (secure 0 0 0) 0x7f336004e9a0 con 0x7f33681547f0 2023-11-28T04:07:23.801 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:07:23.802 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:23.800+0000 7fcd5764b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:07:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 msgr2=0x7f334c061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 0x7f334c061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 msgr2=0x7f3368151390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 rx=0x7f336000cb90 tx=0x7f3360007bd0).stop 2023-11-28T04:07:23.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 shutdown_connections 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f334c05f240 0x7f334c061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f33681547f0 0x7f3368151390 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3368062700 0x7f336813a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 --2- 172.21.15.73:0/3923293274 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f33680c86f0 0x7f3368150ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 >> 172.21.15.73:0/3923293274 conn(0x7f33680642d0 msgr2=0x7f33680a6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 shutdown_connections 2023-11-28T04:07:23.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.802+0000 7f336febb700 1 -- 172.21.15.73:0/3923293274 wait complete. 2023-11-28T04:07:23.813 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.812+0000 7f4ce2c9b680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:07:23.815 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:23.884 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:23.883+0000 7f4ce2c9b680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:07:23.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.930+0000 7fce3dcdb700 1 Processor -- start 2023-11-28T04:07:23.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.930+0000 7fce3dcdb700 1 -- start start 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 0x7fce38066310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce38152450 0x7fce380bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce380bd430 0x7fce380c14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce380c2470 con 0x7fce38152450 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce38151740 con 0x7fce38065f30 2023-11-28T04:07:23.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce3dcdb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce380bd8a0 con 0x7fce380bd430 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce38152450 0x7fce380bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fce38152450 0x7fce380bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46592/0 (socket says 172.21.15.73:46592) 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 -- 172.21.15.73:0/1954872049 learned_addr learned my addr 172.21.15.73:0/1954872049 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce37fff700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce380bd430 0x7fce380c14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce377fe700 1 --2- 172.21.15.73:0/1954872049 >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 0x7fce38066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 -- 172.21.15.73:0/1954872049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce380bd430 msgr2=0x7fce380c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce380bd430 0x7fce380c14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 -- 172.21.15.73:0/1954872049 >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 msgr2=0x7fce38066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 --2- 172.21.15.73:0/1954872049 >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 0x7fce38066310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 -- 172.21.15.73:0/1954872049 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce38066800 con 0x7fce38152450 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce377fe700 1 --2- 172.21.15.73:0/1954872049 >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 0x7fce38066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:23.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce36ffd700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce38152450 0x7fce380bcf40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fce380c6e30 tx=0x7fce2c0090f0).ready entity=mon.0 client_cookie=f5de656a60329e9f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce367fc700 1 -- 172.21.15.73:0/1954872049 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00b520 con 0x7fce38152450 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce367fc700 1 -- 172.21.15.73:0/1954872049 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c00b680 con 0x7fce38152450 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.931+0000 7fce367fc700 1 -- 172.21.15.73:0/1954872049 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c00ba00 con 0x7fce38152450 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 -- 172.21.15.73:0/1954872049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce38152450 msgr2=0x7fce380bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce38152450 0x7fce380bcf40 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 rx=0x7fce380c6e30 tx=0x7fce2c0090f0).stop 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 -- 172.21.15.73:0/1954872049 shutdown_connections 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce380bd430 0x7fce380c14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/1954872049 >> v2:172.21.15.150:3300/0 conn(0x7fce38065f30 0x7fce38066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/1954872049 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce38152450 0x7fce380bcf40 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 -- 172.21.15.73:0/1954872049 >> 172.21.15.73:0/1954872049 conn(0x7fce380631d0 msgr2=0x7fce380c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 -- 172.21.15.73:0/1954872049 shutdown_connections 2023-11-28T04:07:23.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.932+0000 7fce3dcdb700 1 -- 172.21.15.73:0/1954872049 wait complete. 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.933+0000 7fce3dcdb700 1 Processor -- start 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.933+0000 7fce3dcdb700 1 -- start start 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce38065f30 0x7fce381409c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38152450 0x7fce380ab2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce380ab870 con 0x7fce380bd430 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce38165320 con 0x7fce38152450 2023-11-28T04:07:23.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce3dcdb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce38140eb0 con 0x7fce38065f30 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce36ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce36ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46596/0 (socket says 172.21.15.73:46596) 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce36ffd700 1 -- 172.21.15.73:0/3958656922 learned_addr learned my addr 172.21.15.73:0/3958656922 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce377fe700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce38065f30 0x7fce381409c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.934+0000 7fce37fff700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38152450 0x7fce380ab2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce38065f30 msgr2=0x7fce381409c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce38065f30 0x7fce381409c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38152450 msgr2=0x7fce380ab2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38152450 0x7fce380ab2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 -- 172.21.15.73:0/3958656922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c009370 con 0x7fce380bd430 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce36ffd700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fce2c006fd0 tx=0x7fce2c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:23.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c000bc0 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce2c00b7f0 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce2c000d20 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce381649f0 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.935+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce38164fb0 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.936+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce3806c860 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.936+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fce2c01c8c0 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.937+0000 7fce1f7fe700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 0x7fce24061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.937+0000 7fce377fe700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 msgr2=0x7fce24061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.937+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fce2c07be50 con 0x7fce380bd430 2023-11-28T04:07:23.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.937+0000 7fce377fe700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 0x7fce24061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:23.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:23.941+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fce2c050890 con 0x7fce380bd430 2023-11-28T04:07:24.064 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:24.064+0000 7fcd5764b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:07:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.138+0000 7fce377fe700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 msgr2=0x7fce24061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:24.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.138+0000 7fce377fe700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 0x7fce24061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.150+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fce38152960 con 0x7fce380bd430 2023-11-28T04:07:24.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.151+0000 7fce1f7fe700 1 -- 172.21.15.73:0/3958656922 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7fce2c050480 con 0x7fce380bd430 2023-11-28T04:07:24.152 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:24.152 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 msgr2=0x7fce24061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 0x7fce24061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 msgr2=0x7fce380aadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 rx=0x7fce2c006fd0 tx=0x7fce2c017000).stop 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 shutdown_connections 2023-11-28T04:07:24.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fce2405f240 0x7fce24061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fce38152450 0x7fce380ab2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fce38065f30 0x7fce381409c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 --2- 172.21.15.73:0/3958656922 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fce380bd430 0x7fce380aadb0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 >> 172.21.15.73:0/3958656922 conn(0x7fce380631d0 msgr2=0x7fce3814fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 shutdown_connections 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.153+0000 7fce3dcdb700 1 -- 172.21.15.73:0/3958656922 wait complete. 2023-11-28T04:07:24.155 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:24.153+0000 7f4ce2c9b680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:07:24.155 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T04:07:24.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:24.247 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:24.246+0000 7fcd5764b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:24.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.284+0000 7fc81f2f5700 1 Processor -- start 2023-11-28T04:07:24.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.284+0000 7fc81f2f5700 1 -- start start 2023-11-28T04:07:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81f2f5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 0x7fc818155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81f2f5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8180673c0 0x7fc81815e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81f2f5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8180683d0 con 0x7fc818066f40 2023-11-28T04:07:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81f2f5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc818067850 con 0x7fc8180673c0 2023-11-28T04:07:24.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81f2f5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc818067b30 con 0x7fc8181544d0 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81d091700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 0x7fc818155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81c890700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc818066f40 0x7fc818095e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:59528/0 (socket says 172.21.15.73:59528) 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81d892700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8180673c0 0x7fc81815e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc81c890700 1 -- 172.21.15.73:0/2572903472 learned_addr learned my addr 172.21.15.73:0/2572903472 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.285+0000 7fc80ffff700 1 -- 172.21.15.73:0/2572903472 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 376299453 0 0) 0x7fc8180683d0 con 0x7fc818066f40 2023-11-28T04:07:24.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc80ffff700 1 -- 172.21.15.73:0/2572903472 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8040033f0 con 0x7fc818066f40 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d892700 1 -- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 msgr2=0x7fc818155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d892700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 0x7fc818155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d892700 1 -- 172.21.15.73:0/2572903472 >> v1:172.21.15.150:6789/0 conn(0x7fc818066f40 legacy=0x7fc818095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d892700 1 -- 172.21.15.73:0/2572903472 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc818067e10 con 0x7fc8180673c0 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d091700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 0x7fc818155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc81d892700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8180673c0 0x7fc81815e0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc810009e60 tx=0x7fc810009780).ready entity=mon.1 client_cookie=dda951efb7e2f33c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc80ffff700 1 -- 172.21.15.73:0/2572903472 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000e070 con 0x7fc8180673c0 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc80ffff700 1 -- 172.21.15.73:0/2572903472 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc81000bab0 con 0x7fc8180673c0 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.286+0000 7fc80ffff700 1 -- 172.21.15.73:0/2572903472 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc81000be30 con 0x7fc8180673c0 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 -- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8180673c0 msgr2=0x7fc81815e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8180673c0 0x7fc81815e0a0 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 rx=0x7fc810009e60 tx=0x7fc810009780).stop 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 -- 172.21.15.73:0/2572903472 shutdown_connections 2023-11-28T04:07:24.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8180673c0 0x7fc81815e0a0 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/2572903472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8181544d0 0x7fc818155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 -- 172.21.15.73:0/2572903472 >> 172.21.15.73:0/2572903472 conn(0x7fc8180631d0 msgr2=0x7fc8180c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 -- 172.21.15.73:0/2572903472 shutdown_connections 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.287+0000 7fc81f2f5700 1 -- 172.21.15.73:0/2572903472 wait complete. 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.288+0000 7fc81f2f5700 1 Processor -- start 2023-11-28T04:07:24.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.288+0000 7fc81f2f5700 1 -- start start 2023-11-28T04:07:24.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc818066f40 0x7fc8180a9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc818145b60 0x7fc81814e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc818157f50 con 0x7fc8181544d0 2023-11-28T04:07:24.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180ab730 con 0x7fc818066f40 2023-11-28T04:07:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81f2f5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8180aba10 con 0x7fc818145b60 2023-11-28T04:07:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46606/0 (socket says 172.21.15.73:46606) 2023-11-28T04:07:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81d091700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc818066f40 0x7fc8180a9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 -- 172.21.15.73:0/3442845397 learned_addr learned my addr 172.21.15.73:0/3442845397 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81d892700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc818145b60 0x7fc81814e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 -- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc818145b60 msgr2=0x7fc81814e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc818145b60 0x7fc81814e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 -- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc818066f40 msgr2=0x7fc8180a9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc818066f40 0x7fc8180a9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 -- 172.21.15.73:0/3442845397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc814009250 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.289+0000 7fc81c890700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fc80800c4c0 tx=0x7fc808007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.290+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc80800ce70 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.290+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8080072e0 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.290+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc810009a20 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.290+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8080057a0 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.290+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8180ac440 con 0x7fc8181544d0 2023-11-28T04:07:24.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.291+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc81806c860 con 0x7fc8181544d0 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.292+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc808005bd0 con 0x7fc8181544d0 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.292+0000 7fc80d7fa700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 0x7fc7fc061760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.292+0000 7fc81d091700 1 -- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 msgr2=0x7fc7fc061760 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.292+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fc80807a2b0 con 0x7fc8181544d0 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.292+0000 7fc81d091700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 0x7fc7fc061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:24.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.296+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc80804b380 con 0x7fc8181544d0 2023-11-28T04:07:24.349 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:24.347+0000 7f4ce2c9b680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:24.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.468+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8180680f0 con 0x7fc8181544d0 2023-11-28T04:07:24.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.469+0000 7fc80d7fa700 1 -- 172.21.15.73:0/3442845397 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17448 (secure 0 0 0) 0x7fc808018070 con 0x7fc8181544d0 2023-11-28T04:07:24.470 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:24.470 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":150,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:23.021862+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":5}]},{"pgid":"17.17","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/603729121":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6833/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6832/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:0/1414794090":"2023-11-29T03:11:34.567794+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":41,"length":17},{"begin":95,"length":19}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:24.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 msgr2=0x7fc7fc061760 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:24.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 0x7fc7fc061760 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 msgr2=0x7fc8180aaf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 rx=0x7fc80800c4c0 tx=0x7fc808007c50).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 shutdown_connections 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7fc05f2a0 0x7fc7fc061760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc818145b60 0x7fc81814e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc818066f40 0x7fc8180a9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 --2- 172.21.15.73:0/3442845397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8181544d0 0x7fc8180aaf50 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 >> 172.21.15.73:0/3442845397 conn(0x7fc8180631d0 msgr2=0x7fc818095e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 shutdown_connections 2023-11-28T04:07:24.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.471+0000 7fc81f2f5700 1 -- 172.21.15.73:0/3442845397 wait complete. 2023-11-28T04:07:24.484 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:07:24.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:24.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.601+0000 7f7699ba2700 1 Processor -- start 2023-11-28T04:07:24.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.601+0000 7f7699ba2700 1 -- start start 2023-11-28T04:07:24.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.602+0000 7f7699ba2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76940c35c0 0x7f76940c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.602+0000 7f7699ba2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7694063e30 0x7f7694062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.602+0000 7f7699ba2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f76940c59b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7699ba2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76940639a0 con 0x7f76940c35c0 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7699ba2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7694150d00 con 0x7f7694063e30 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7699ba2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7694062d90 con 0x7f7694069ce0 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76940c35c0 0x7f76940c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f76940c35c0 0x7f76940c39a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46622/0 (socket says 172.21.15.73:46622) 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 -- 172.21.15.73:0/239491237 learned_addr learned my addr 172.21.15.73:0/239491237 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7693fff700 1 --2- 172.21.15.73:0/239491237 >> v2:172.21.15.150:3300/0 conn(0x7f7694063e30 0x7f7694062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f76993a1700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f76940c59b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 -- 172.21.15.73:0/239491237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 msgr2=0x7f76940c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f76940c59b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 -- 172.21.15.73:0/239491237 >> v2:172.21.15.150:3300/0 conn(0x7f7694063e30 msgr2=0x7f7694062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 --2- 172.21.15.73:0/239491237 >> v2:172.21.15.150:3300/0 conn(0x7f7694063e30 0x7f7694062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 -- 172.21.15.73:0/239491237 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7694063070 con 0x7f76940c35c0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f76993a1700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f76940c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f7698ba0700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c39a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f7694154360 tx=0x7f76880090f0).ready entity=mon.0 client_cookie=c149ed6933deb21b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f76937fe700 1 -- 172.21.15.73:0/239491237 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800b520 con 0x7f76940c35c0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f76937fe700 1 -- 172.21.15.73:0/239491237 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768800b680 con 0x7f76940c35c0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.603+0000 7f76937fe700 1 -- 172.21.15.73:0/239491237 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768800ba00 con 0x7f76940c35c0 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 -- 172.21.15.73:0/239491237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 msgr2=0x7f76940c39a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c39a0 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 rx=0x7f7694154360 tx=0x7f76880090f0).stop 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 -- 172.21.15.73:0/239491237 shutdown_connections 2023-11-28T04:07:24.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c39a0 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 --2- 172.21.15.73:0/239491237 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f76940c59b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 --2- 172.21.15.73:0/239491237 >> v2:172.21.15.150:3300/0 conn(0x7f7694063e30 0x7f7694062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 -- 172.21.15.73:0/239491237 >> 172.21.15.73:0/239491237 conn(0x7f76940c7eb0 msgr2=0x7f76940c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 -- 172.21.15.73:0/239491237 shutdown_connections 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.604+0000 7f7699ba2700 1 -- 172.21.15.73:0/239491237 wait complete. 2023-11-28T04:07:24.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.605+0000 7f7699ba2700 1 Processor -- start 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.605+0000 7f7699ba2700 1 -- start start 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 0x7f769414c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f7694152fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76940bf290 con 0x7f76940c35c0 2023-11-28T04:07:24.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76940be680 con 0x7f7694063e30 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7699ba2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76940be960 con 0x7f7694069ce0 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46636/0 (socket says 172.21.15.73:46636) 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 -- 172.21.15.73:0/3308556637 learned_addr learned my addr 172.21.15.73:0/3308556637 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7698ba0700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 0x7f769414c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7693fff700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f7694152fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 msgr2=0x7f7694152fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f7694152fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 msgr2=0x7f769414c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 0x7f769414c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 -- 172.21.15.73:0/3308556637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7688009370 con 0x7f76940c35c0 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7693fff700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f7694152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:24.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f7698ba0700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 0x7f769414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.606+0000 7f76993a1700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f768c00d180 tx=0x7f768c009a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.607+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c00de20 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.607+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f768c00f040 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.607+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f768c013600 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.607+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76940bec40 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.607+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f769414c6f0 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.608+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76940bf440 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.609+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f768c0139f0 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.609+0000 7f7690ff9700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 0x7f7678061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.609+0000 7f7698ba0700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 msgr2=0x7f7678061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.609+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7f768c07aad0 con 0x7f76940c35c0 2023-11-28T04:07:24.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.609+0000 7f7698ba0700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 0x7f7678061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:24.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.613+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f76940bf440 con 0x7f76940c35c0 2023-11-28T04:07:24.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.810+0000 7f7698ba0700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 msgr2=0x7f7678061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:24.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.810+0000 7f7698ba0700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 0x7f7678061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:24.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.826+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f769406c7c0 con 0x7f76940c35c0 2023-11-28T04:07:24.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.826+0000 7f7690ff9700 1 -- 172.21.15.73:0/3308556637 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7f768c04ba80 con 0x7f76940c35c0 2023-11-28T04:07:24.827 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:24.828 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 msgr2=0x7f7678061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 0x7f7678061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 msgr2=0x7f76940c1f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 rx=0x7f768c00d180 tx=0x7f768c009a50).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 shutdown_connections 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f767805f270 0x7f7678061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f76940c35c0 0x7f76940c1f90 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7694063e30 0x7f769414c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 --2- 172.21.15.73:0/3308556637 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7694069ce0 0x7f7694152fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.828+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 >> 172.21.15.73:0/3308556637 conn(0x7f76940c7eb0 msgr2=0x7f76940bcf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.829+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 shutdown_connections 2023-11-28T04:07:24.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.829+0000 7f7699ba2700 1 -- 172.21.15.73:0/3308556637 wait complete. 2023-11-28T04:07:24.831 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T04:07:24.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:24.899 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:24.898+0000 7fcd5764b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:07:24.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.963+0000 7fba7ead4700 1 Processor -- start 2023-11-28T04:07:24.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.963+0000 7fba7ead4700 1 -- start start 2023-11-28T04:07:24.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7ead4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fba781542d0 0x7fba780697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7ead4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba78065760 con 0x7fba78069ce0 2023-11-28T04:07:24.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7ead4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba780c3250 con 0x7fba78062e10 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7ead4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba7806a340 con 0x7fba781542d0 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7c870700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fba781542d0 0x7fba780697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7c870700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fba781542d0 0x7fba780697f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46652/0 (socket says 172.21.15.73:46652) 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba77fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fba78069ce0 0x7fba78062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54586/0 (socket says 172.21.15.73:54586) 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.964+0000 7fba7c870700 1 -- 172.21.15.73:0/3286238085 learned_addr learned my addr 172.21.15.73:0/3286238085 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba7c870700 1 -- 172.21.15.73:0/3286238085 >> v1:172.21.15.150:6789/0 conn(0x7fba78062e10 legacy=0x7fba780661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba7c870700 1 -- 172.21.15.73:0/3286238085 >> v1:172.21.15.73:6789/0 conn(0x7fba78069ce0 legacy=0x7fba78062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba777fe700 1 -- 172.21.15.73:0/3286238085 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120434992 0 0) 0x7fba78065760 con 0x7fba78069ce0 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba7c870700 1 -- 172.21.15.73:0/3286238085 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba7806a620 con 0x7fba781542d0 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba7c870700 1 --2- 172.21.15.73:0/3286238085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba781542d0 0x7fba780697f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fba6c006fd0 tx=0x7fba6c0090f0).ready entity=mon.0 client_cookie=f2a06355b5354239 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.968 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba777fe700 1 -- 172.21.15.73:0/3286238085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00b600 con 0x7fba781542d0 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba777fe700 1 -- 172.21.15.73:0/3286238085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba6c00b760 con 0x7fba781542d0 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.965+0000 7fba777fe700 1 -- 172.21.15.73:0/3286238085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6c00bae0 con 0x7fba781542d0 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 -- 172.21.15.73:0/3286238085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba781542d0 msgr2=0x7fba780697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 --2- 172.21.15.73:0/3286238085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba781542d0 0x7fba780697f0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 rx=0x7fba6c006fd0 tx=0x7fba6c0090f0).stop 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 -- 172.21.15.73:0/3286238085 shutdown_connections 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 --2- 172.21.15.73:0/3286238085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba781542d0 0x7fba780697f0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 -- 172.21.15.73:0/3286238085 >> 172.21.15.73:0/3286238085 conn(0x7fba780c7fd0 msgr2=0x7fba780c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 -- 172.21.15.73:0/3286238085 shutdown_connections 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.966+0000 7fba7ead4700 1 -- 172.21.15.73:0/3286238085 wait complete. 2023-11-28T04:07:24.969 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.967+0000 7fba7ead4700 1 Processor -- start 2023-11-28T04:07:24.970 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:24.966+0000 7fcd5764b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:07:24.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.967+0000 7fba7ead4700 1 -- start start 2023-11-28T04:07:24.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba78062e10 0x7fba780abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba781542d0 0x7fba780ad590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780ab020 con 0x7fba78069ce0 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba780ab2c0 con 0x7fba781542d0 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba780ab5a0 con 0x7fba78062e10 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46658/0 (socket says 172.21.15.73:46658) 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 -- 172.21.15.73:0/2155198919 learned_addr learned my addr 172.21.15.73:0/2155198919 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7c870700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba78062e10 0x7fba780abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7d071700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba781542d0 0x7fba780ad590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba78062e10 msgr2=0x7fba780abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba78062e10 0x7fba780abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba781542d0 msgr2=0x7fba780ad590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba781542d0 0x7fba780ad590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 -- 172.21.15.73:0/2155198919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba6c009370 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba77fff700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fba6400c4c0 tx=0x7fba64007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba6400ce30 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba640072e0 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba780ab880 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.968+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba640056e0 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.969+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba780aa640 con 0x7fba78069ce0 2023-11-28T04:07:24.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.969+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba780aba70 con 0x7fba78069ce0 2023-11-28T04:07:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.970+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fba64005a90 con 0x7fba78069ce0 2023-11-28T04:07:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.970+0000 7fba74ff9700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 0x7fba5c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.970+0000 7fba7c870700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 msgr2=0x7fba5c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.970+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fba64079c10 con 0x7fba78069ce0 2023-11-28T04:07:24.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.970+0000 7fba7c870700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 0x7fba5c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:24.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:24.980+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fba6404ac20 con 0x7fba78069ce0 2023-11-28T04:07:25.033 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.032+0000 7f4ce2c9b680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:07:25.045 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.044+0000 7fcd5764b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:07:25.104 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.103+0000 7f4ce2c9b680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:07:25.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.171+0000 7fba7c870700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 msgr2=0x7fba5c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:25.172 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.171+0000 7fba7c870700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 0x7fba5c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:25.181 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.180+0000 7f4ce2c9b680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:07:25.190 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.189+0000 7fcd5764b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:07:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.193+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fba7806c860 con 0x7fba78069ce0 2023-11-28T04:07:25.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.193+0000 7fba74ff9700 1 -- 172.21.15.73:0/2155198919 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 52 v52) v1 ==== 94+0+4488 (secure 0 0 0) 0x7fba6404e240 con 0x7fba78069ce0 2023-11-28T04:07:25.195 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:25.195 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":52,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":52,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T03:11:18.740523+0000","modified":"2023-11-28T03:11:20.749057+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8094},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8094":{"gid":8094,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3840072732","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3840072732},{"type":"v1","addr":"172.21.15.73:6837","nonce":3840072732}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[18],"metadata_pool":17,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":8}]} 2023-11-28T04:07:25.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 msgr2=0x7fba5c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:25.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 0x7fba5c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:25.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 msgr2=0x7fba780aa150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 rx=0x7fba6400c4c0 tx=0x7fba64007c10).stop 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 shutdown_connections 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fba5c05f270 0x7fba5c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba781542d0 0x7fba780ad590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba78062e10 0x7fba780abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 --2- 172.21.15.73:0/2155198919 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba78069ce0 0x7fba780aa150 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.195+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 >> 172.21.15.73:0/2155198919 conn(0x7fba780c7fd0 msgr2=0x7fba780bd440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.196+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 shutdown_connections 2023-11-28T04:07:25.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.196+0000 7fba7ead4700 1 -- 172.21.15.73:0/2155198919 wait complete. 2023-11-28T04:07:25.199 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 52 2023-11-28T04:07:25.209 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:25.260 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.259+0000 7fcd5764b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:07:25.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.325+0000 7f08f12e5700 1 Processor -- start 2023-11-28T04:07:25.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.325+0000 7f08f12e5700 1 -- start start 2023-11-28T04:07:25.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08f12e5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 0x7f08ec065a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08f12e5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec0698f0 con 0x7f08ec0c1c60 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08f12e5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ec065f30 con 0x7f08ec1508e0 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08f12e5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec069d80 con 0x7f08ec0c2040 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08ebfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f08ec0c1c60 0x7f08ec1501d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54588/0 (socket says 172.21.15.73:54588) 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.326+0000 7f08ebfff700 1 -- 172.21.15.73:0/1661906268 learned_addr learned my addr 172.21.15.73:0/1661906268 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08f0ae4700 1 --2- 172.21.15.73:0/1661906268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 0x7f08ec065a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960154792 0 0) 0x7f08ec0698f0 con 0x7f08ec0c1c60 2023-11-28T04:07:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08d4004500 con 0x7f08ec0c1c60 2023-11-28T04:07:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2094102950 0 0) 0x7f08ec065f30 con 0x7f08ec1508e0 2023-11-28T04:07:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f08ec0698f0 con 0x7f08ec1508e0 2023-11-28T04:07:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1661906268 >> v1:172.21.15.150:6789/0 conn(0x7f08ec1508e0 legacy=0x7f08ec0c4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 766638364 0 0) 0x7f08d4004500 con 0x7f08ec0c1c60 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1661906268 >> v1:172.21.15.73:6789/0 conn(0x7f08ec0c1c60 legacy=0x7f08ec1501d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1661906268 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ec068ce0 con 0x7f08ec0c2040 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08f0ae4700 1 --2- 172.21.15.73:0/1661906268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 0x7f08ec065a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f08e4009e90 tx=0x7f08e4009780).ready entity=mon.2 client_cookie=f6a230024976f758 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e400e070 con 0x7f08ec0c2040 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e400b7a0 con 0x7f08ec0c2040 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.327+0000 7f08eaffd700 1 -- 172.21.15.73:0/1661906268 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08dc001470 con 0x7f08ec0c2040 2023-11-28T04:07:25.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 -- 172.21.15.73:0/1661906268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 msgr2=0x7f08ec065a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1661906268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 0x7f08ec065a40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 rx=0x7f08e4009e90 tx=0x7f08e4009780).stop 2023-11-28T04:07:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 -- 172.21.15.73:0/1661906268 shutdown_connections 2023-11-28T04:07:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1661906268 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c2040 0x7f08ec065a40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 -- 172.21.15.73:0/1661906268 >> 172.21.15.73:0/1661906268 conn(0x7f08ec0631d0 msgr2=0x7f08ec0c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:25.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 -- 172.21.15.73:0/1661906268 shutdown_connections 2023-11-28T04:07:25.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.328+0000 7f08f12e5700 1 -- 172.21.15.73:0/1661906268 wait complete. 2023-11-28T04:07:25.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.330+0000 7f08f12e5700 1 Processor -- start 2023-11-28T04:07:25.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.330+0000 7f08f12e5700 1 -- start start 2023-11-28T04:07:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 0x7f08ec14eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08ec0c2040 0x7f08ec147df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0bf120 con 0x7f08ec1508e0 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec0bf400 con 0x7f08ec0c2040 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.331+0000 7f08f12e5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ec14f290 con 0x7f08ec0c1c60 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46668/0 (socket says 172.21.15.73:46668) 2023-11-28T04:07:25.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1719807959 learned_addr learned my addr 172.21.15.73:0/1719807959 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08ebfff700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 0x7f08ec14eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08eb7fe700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08ec0c2040 0x7f08ec147df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 msgr2=0x7f08ec14eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 0x7f08ec14eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08ec0c2040 msgr2=0x7f08ec147df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08ec0c2040 0x7f08ec147df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.342 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 -- 172.21.15.73:0/1719807959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08e4009a20 con 0x7f08ec1508e0 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08ebfff700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 0x7f08ec14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f0ae4700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f08ec0636e0 tx=0x7f08ec069860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e401d440 con 0x7f08ec1508e0 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08e401d5a0 con 0x7f08ec1508e0 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ec1482e0 con 0x7f08ec1508e0 2023-11-28T04:07:25.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08e401d940 con 0x7f08ec1508e0 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.332+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ec14f7d0 con 0x7f08ec1508e0 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.333+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ec06c7c0 con 0x7f08ec1508e0 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.333+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f08e401dcf0 con 0x7f08ec1508e0 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.334+0000 7f08d3fff700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 0x7f08d8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.334+0000 7f08ebfff700 1 -- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 msgr2=0x7f08d8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.334+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7f08e407c530 con 0x7f08ec1508e0 2023-11-28T04:07:25.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.334+0000 7f08ebfff700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 0x7f08d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:25.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.338+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f08e404d500 con 0x7f08ec1508e0 2023-11-28T04:07:25.345 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.339+0000 7f4ce2c9b680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:07:25.376 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.375+0000 7fcd5764b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:07:25.415 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.414+0000 7f4ce2c9b680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:07:25.486 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.485+0000 7fcd5764b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:25.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.507+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08ec0655f0 con 0x7f08ec1508e0 2023-11-28T04:07:25.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.508+0000 7f08d3fff700 1 -- 172.21.15.73:0/1719807959 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v150) v1 ==== 74+0+17448 (secure 0 0 0) 0x7f08e4050b20 con 0x7f08ec1508e0 2023-11-28T04:07:25.509 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:25.509 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":150,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:23.021862+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"143","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":17,"pool_name":"cephfs_metadata","create_time":"2023-11-28T03:11:17.276027+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"139","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":18,"pool_name":"cephfs_data","create_time":"2023-11-28T03:11:17.703654+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"150","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":150,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.1","mappings":[{"from":0,"to":5}]},{"pgid":"17.17","mappings":[{"from":0,"to":2}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/603729121":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6833/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:6832/90017":"2023-11-29T03:11:34.567794+0000","172.21.15.73:0/1414794090":"2023-11-29T03:11:34.567794+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":18,"snaps":[{"begin":41,"length":17},{"begin":95,"length":19}]}],"new_removed_snaps":[{"pool":18,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 msgr2=0x7f08d8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 0x7f08d8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 msgr2=0x7f08ec14c580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 rx=0x7f08ec0636e0 tx=0x7f08ec069860).stop 2023-11-28T04:07:25.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 shutdown_connections 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f08d805f270 0x7f08d8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08ec1508e0 0x7f08ec14c580 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08ec0c1c60 0x7f08ec14eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 --2- 172.21.15.73:0/1719807959 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08ec0c2040 0x7f08ec147df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 >> 172.21.15.73:0/1719807959 conn(0x7f08ec0631d0 msgr2=0x7f08ec0bbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 shutdown_connections 2023-11-28T04:07:25.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.510+0000 7f08f12e5700 1 -- 172.21.15.73:0/1719807959 wait complete. 2023-11-28T04:07:25.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T04:07:25.533 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.532+0000 7f4ce2c9b680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:07:25.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 Processor -- start 2023-11-28T04:07:25.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 -- start start 2023-11-28T04:07:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe558098d10 0x7fe5580c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5580c72d0 con 0x7fe558158320 2023-11-28T04:07:25.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe558064570 con 0x7fe5580c6190 2023-11-28T04:07:25.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.641+0000 7fe55d2df700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe558154bc0 con 0x7fe558098d10 2023-11-28T04:07:25.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe5577fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe558158320 0x7fe5580a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:54604/0 (socket says 172.21.15.73:54604) 2023-11-28T04:07:25.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe5577fe700 1 -- 172.21.15.73:0/3967806830 learned_addr learned my addr 172.21.15.73:0/3967806830 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:25.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe556ffd700 1 --2- 172.21.15.73:0/3967806830 >> v2:172.21.15.150:3300/0 conn(0x7fe558098d10 0x7fe5580c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1715533175 0 0) 0x7fe5580c72d0 con 0x7fe558158320 2023-11-28T04:07:25.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5400033f0 con 0x7fe558158320 2023-11-28T04:07:25.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2194152677 0 0) 0x7fe558064570 con 0x7fe5580c6190 2023-11-28T04:07:25.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5580c72d0 con 0x7fe5580c6190 2023-11-28T04:07:25.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe556ffd700 1 -- 172.21.15.73:0/3967806830 >> v1:172.21.15.150:6789/0 conn(0x7fe5580c6190 legacy=0x7fe558063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe556ffd700 1 -- 172.21.15.73:0/3967806830 >> v1:172.21.15.73:6789/0 conn(0x7fe558158320 legacy=0x7fe5580a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe556ffd700 1 -- 172.21.15.73:0/3967806830 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5580c66c0 con 0x7fe558098d10 2023-11-28T04:07:25.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe556ffd700 1 --2- 172.21.15.73:0/3967806830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe5580c7760 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe548009760 tx=0x7fe548009060).ready entity=mon.1 client_cookie=f70681d53c5597bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800d4a0 con 0x7fe558098d10 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.642+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe54800d600 con 0x7fe558098d10 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe555ffb700 1 -- 172.21.15.73:0/3967806830 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe54800d980 con 0x7fe558098d10 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 -- 172.21.15.73:0/3967806830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 msgr2=0x7fe5580c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 --2- 172.21.15.73:0/3967806830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe5580c7760 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 rx=0x7fe548009760 tx=0x7fe548009060).stop 2023-11-28T04:07:25.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 -- 172.21.15.73:0/3967806830 shutdown_connections 2023-11-28T04:07:25.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 --2- 172.21.15.73:0/3967806830 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe5580c7760 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 -- 172.21.15.73:0/3967806830 >> 172.21.15.73:0/3967806830 conn(0x7fe5580cb310 msgr2=0x7fe5580cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:25.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 -- 172.21.15.73:0/3967806830 shutdown_connections 2023-11-28T04:07:25.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.643+0000 7fe55d2df700 1 -- 172.21.15.73:0/3967806830 wait complete. 2023-11-28T04:07:25.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.644+0000 7fe55d2df700 1 Processor -- start 2023-11-28T04:07:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.644+0000 7fe55d2df700 1 -- start start 2023-11-28T04:07:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe558148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5580c6190 0x7fe5580ad7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580b1370 con 0x7fe558158320 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe55814a650 con 0x7fe558098d10 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe55d2df700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5580ae6d0 con 0x7fe5580c6190 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe5577fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe5577fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46676/0 (socket says 172.21.15.73:46676) 2023-11-28T04:07:25.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe5577fe700 1 -- 172.21.15.73:0/2362486022 learned_addr learned my addr 172.21.15.73:0/2362486022 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe556ffd700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe558148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.645+0000 7fe5567fc700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5580c6190 0x7fe5580ad7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5580c6190 msgr2=0x7fe5580ad7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5580c6190 0x7fe5580ad7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 msgr2=0x7fe558148b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe558148b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:25.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 -- 172.21.15.73:0/2362486022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5480092a0 con 0x7fe558158320 2023-11-28T04:07:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe5577fe700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7fe550001560 tx=0x7fe5500015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe550005d30 con 0x7fe558158320 2023-11-28T04:07:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe550005e90 con 0x7fe558158320 2023-11-28T04:07:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5500072e0 con 0x7fe558158320 2023-11-28T04:07:25.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe558149010 con 0x7fe558158320 2023-11-28T04:07:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.646+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe55814a370 con 0x7fe558158320 2023-11-28T04:07:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.647+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe55806c860 con 0x7fe558158320 2023-11-28T04:07:25.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.647+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 63) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fe550007690 con 0x7fe558158320 2023-11-28T04:07:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.648+0000 7fe53effd700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 0x7fe544061700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.648+0000 7fe556ffd700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 msgr2=0x7fe544061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.648+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 6439+0+0 (secure 0 0 0) 0x7fe550079e50 con 0x7fe558158320 2023-11-28T04:07:25.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.648+0000 7fe556ffd700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 0x7fe544061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:25.660 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.650+0000 7f4ce2c9b680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:25.660 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.655+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe55004ae60 con 0x7fe558158320 2023-11-28T04:07:25.785 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.784+0000 7fcd5764b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:07:25.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.849+0000 7fe556ffd700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 msgr2=0x7fe544061700 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:25.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.849+0000 7fe556ffd700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 0x7fe544061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:25.856 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:25.856+0000 7fcd5764b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:07:25.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:25.868+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe5580c69a0 con 0x7fe558158320 2023-11-28T04:07:25.955 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:25.954+0000 7f4ce2c9b680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:07:26.027 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.025+0000 7f4ce2c9b680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:07:26.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.047+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 64) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fe55004ed50 con 0x7fe558158320 2023-11-28T04:07:26.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.047+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 msgr2=0x7fe544061700 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:26.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.047+0000 7fe53effd700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 0x7fe544061700 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:26.049 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:07:26.049+0000 7fb35056e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:07:26.050 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:07:26.049+0000 7fb35056e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:07:26.050 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:07:26.049+0000 7fb35056e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:07:26.065 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T04:07:26.049+0000 7fb35056e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:07:26.065 INFO:tasks.ceph.mds.a.smithi073.stderr: -3> 2023-11-28T04:07:26.049+0000 7fb35056e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:07:26.065 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T04:07:26.049+0000 7fb35056e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:07:26.086 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5e42f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5e42f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5e42f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5e42f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5e42f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5bc2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5bc2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5bc2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5bc2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.103 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:26.101+0000 7f4c5bc2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:07:26.556 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T04:07:26.556+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T04:07:26.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.563+0000 7fe53effd700 1 -- 172.21.15.73:0/2362486022 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v53) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe550073400 con 0x7fe558158320 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.565+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 msgr2=0x7fe5580aeb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.565+0000 7fe55d2df700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 rx=0x7fe550001560 tx=0x7fe5500015b0).stop 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 shutdown_connections 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe54405f240 0x7fe544061700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe558158320 0x7fe5580aeb60 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe558098d10 0x7fe558148b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 --2- 172.21.15.73:0/2362486022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe5580c6190 0x7fe5580ad7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 >> 172.21.15.73:0/2362486022 conn(0x7fe5580cb310 msgr2=0x7fe5580c0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:26.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 shutdown_connections 2023-11-28T04:07:26.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.566+0000 7fe55d2df700 1 -- 172.21.15.73:0/2362486022 wait complete. 2023-11-28T04:07:26.568 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T04:07:26.580 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T04:07:26.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 Processor -- start 2023-11-28T04:07:26.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 -- start start 2023-11-28T04:07:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbbdc1547f0 0x7fbbdc0a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc0c86f0 0x7fbbdc0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbdc062700 0x7fbbdc062b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0c7e30 con 0x7fbbdc062700 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbdc0bcf40 con 0x7fbbdc1547f0 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.735+0000 7fbbe16f0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc0c6b20 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbdc062700 0x7fbbdc062b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbbdc062700 0x7fbbdc062b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46802/0 (socket says 172.21.15.73:46802) 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 -- 172.21.15.73:0/2130665583 learned_addr learned my addr 172.21.15.73:0/2130665583 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbd27fc700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc0c86f0 0x7fbbdc0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdaffd700 1 --2- 172.21.15.73:0/2130665583 >> v2:172.21.15.150:3300/0 conn(0x7fbbdc1547f0 0x7fbbdc0a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 -- 172.21.15.73:0/2130665583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc0c86f0 msgr2=0x7fbbdc0c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc0c86f0 0x7fbbdc0c6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 -- 172.21.15.73:0/2130665583 >> v2:172.21.15.150:3300/0 conn(0x7fbbdc1547f0 msgr2=0x7fbbdc0a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 --2- 172.21.15.73:0/2130665583 >> v2:172.21.15.150:3300/0 conn(0x7fbbdc1547f0 0x7fbbdc0a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 -- 172.21.15.73:0/2130665583 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbdc0a77d0 con 0x7fbbdc062700 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbdb7fe700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc062700 0x7fbbdc062b90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fbbdc05f7f0 tx=0x7fbbd4009840).ready entity=mon.0 client_cookie=2977df8554085947 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbda7fc700 1 -- 172.21.15.73:0/2130665583 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400e070 con 0x7fbbdc062700 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbda7fc700 1 -- 172.21.15.73:0/2130665583 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd400b970 con 0x7fbbdc062700 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.736+0000 7fbbda7fc700 1 -- 172.21.15.73:0/2130665583 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400bcf0 con 0x7fbbdc062700 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 -- 172.21.15.73:0/2130665583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc062700 msgr2=0x7fbbdc062b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc062700 0x7fbbdc062b90 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 rx=0x7fbbdc05f7f0 tx=0x7fbbd4009840).stop 2023-11-28T04:07:26.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 -- 172.21.15.73:0/2130665583 shutdown_connections 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc062700 0x7fbbdc062b90 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/2130665583 >> v2:172.21.15.150:3300/0 conn(0x7fbbdc1547f0 0x7fbbdc0a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/2130665583 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc0c86f0 0x7fbbdc0c6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 -- 172.21.15.73:0/2130665583 >> 172.21.15.73:0/2130665583 conn(0x7fbbdc0642d0 msgr2=0x7fbbdc0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 -- 172.21.15.73:0/2130665583 shutdown_connections 2023-11-28T04:07:26.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.737+0000 7fbbe16f0700 1 -- 172.21.15.73:0/2130665583 wait complete. 2023-11-28T04:07:26.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.738+0000 7fbbe16f0700 1 Processor -- start 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.738+0000 7fbbe16f0700 1 -- start start 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc062700 0x7fbbdc140e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbdc1547f0 0x7fbbdc0a5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc164b70 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc164e50 con 0x7fbbdc1547f0 2023-11-28T04:07:26.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbdc165130 con 0x7fbbdc062700 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46810/0 (socket says 172.21.15.73:46810) 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 -- 172.21.15.73:0/1057668962 learned_addr learned my addr 172.21.15.73:0/1057668962 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbdaffd700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc062700 0x7fbbdc140e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbdb7fe700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbdc1547f0 0x7fbbdc0a5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:26.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 -- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc062700 msgr2=0x7fbbdc140e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc062700 0x7fbbdc140e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 -- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbdc1547f0 msgr2=0x7fbbdc0a5910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbdc1547f0 0x7fbbdc0a5910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 -- 172.21.15.73:0/1057668962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd4009ae0 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd27fc700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fbbc400e950 tx=0x7fbbc4009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400c850 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbc400c9b0 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbdc165410 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbc400cd30 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.739+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbdc164900 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.741+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbdc15b150 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.741+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 64) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fbbc4012600 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.741+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(152..152 src has 1..152) v4 ==== 6653+0+0 (secure 0 0 0) 0x7fbbc407a6e0 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.745+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbbc404b970 con 0x7fbbdc0c86f0 2023-11-28T04:07:26.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:26.950+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fbbdc06c860 con 0x7fbbdc0c86f0 2023-11-28T04:07:27.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.048+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 65) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fbbc4078470 con 0x7fbbdc0c86f0 2023-11-28T04:07:27.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.048+0000 7fbbd37fe700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 0x7fbbbc067bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:27.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.049+0000 7fbbdaffd700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 0x7fbbbc067bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:27.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.050+0000 7fbbdaffd700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 0x7fbbbc067bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fbbcc009bc0 tx=0x7fbbcc008040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:27.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.564+0000 7fbbd37fe700 1 -- 172.21.15.73:0/1057668962 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v54) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbbc406e7c0 con 0x7fbbdc0c86f0 2023-11-28T04:07:33.899 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 msgr2=0x7fbbbc067bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:33.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 0x7fbbbc067bb0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 rx=0x7fbbcc009bc0 tx=0x7fbbcc008040).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 msgr2=0x7fbbdc14e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 rx=0x7fbbc400e950 tx=0x7fbbc4009060).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 shutdown_connections 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbbbc0302c0 0x7fbbbc067bb0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbbdc1547f0 0x7fbbdc0a5910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbbdc062700 0x7fbbdc140e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 --2- 172.21.15.73:0/1057668962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbbdc0c86f0 0x7fbbdc14e0d0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 >> 172.21.15.73:0/1057668962 conn(0x7fbbdc0642d0 msgr2=0x7fbbdc065520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 shutdown_connections 2023-11-28T04:07:33.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:27.566+0000 7fbbe16f0700 1 -- 172.21.15.73:0/1057668962 wait complete. 2023-11-28T04:07:33.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T04:07:34.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.068+0000 7f01f5767700 1 Processor -- start 2023-11-28T04:07:34.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.068+0000 7f01f5767700 1 -- start start 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f5767700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f01f00aa680 0x7f01f00c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f5767700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f0158d50 con 0x7f01f01546b0 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f5767700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01f0158140 con 0x7f01f01542d0 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f5767700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01f0158420 con 0x7f01f00aa680 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f4f66700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f01f01546b0 0x7f01f00c64b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36672/0 (socket says 172.21.15.73:36672) 2023-11-28T04:07:34.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.069+0000 7f01f4f66700 1 -- 172.21.15.73:0/2820308463 learned_addr learned my addr 172.21.15.73:0/2820308463 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2920185796 0 0) 0x7f01f0158d50 con 0x7f01f01546b0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01d80033f0 con 0x7f01f01546b0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01ef7fe700 1 --2- 172.21.15.73:0/2820308463 >> v2:172.21.15.150:3300/0 conn(0x7f01f00aa680 0x7f01f00c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 76210618 0 0) 0x7f01d80033f0 con 0x7f01f01546b0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 >> v2:172.21.15.150:3300/0 conn(0x7f01f00aa680 msgr2=0x7f01f00c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 --2- 172.21.15.73:0/2820308463 >> v2:172.21.15.150:3300/0 conn(0x7f01f00aa680 0x7f01f00c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 >> v1:172.21.15.150:6789/0 conn(0x7f01f01542d0 legacy=0x7f01f00a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f0158700 con 0x7f01f01546b0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f01e80017f0 con 0x7f01f01546b0 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 >> v1:172.21.15.73:6789/0 conn(0x7f01f01546b0 legacy=0x7f01f00c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 0x7f01d8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d8004690 0x7f01d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01d8005000 0x7f01d800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f0158420 con 0x7f01d80038e0 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d800bf60 con 0x7f01d8004690 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d800c0f0 con 0x7f01d8005000 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 0x7f01d8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01ef7fe700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d8004690 0x7f01d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01f4f66700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01d8005000 0x7f01d800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 -- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01d8005000 msgr2=0x7f01d800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01d8005000 0x7f01d800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 -- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d8004690 msgr2=0x7f01d8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d8004690 0x7f01d8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01effff700 1 -- 172.21.15.73:0/2820308463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80033f0 con 0x7f01d80038e0 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.070+0000 7f01f4f66700 1 -- 172.21.15.73:0/2820308463 reap_dead start 2023-11-28T04:07:34.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01effff700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 0x7f01d8003d00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f01e0001730 tx=0x7f01e0007bb0).ready entity=mon.0 client_cookie=418f0f972859a7a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0007020 con 0x7f01d80038e0 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e0007180 con 0x7f01d80038e0 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01eeffd700 1 -- 172.21.15.73:0/2820308463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e0007520 con 0x7f01d80038e0 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 -- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 msgr2=0x7f01d8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 0x7f01d8003d00 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 rx=0x7f01e0001730 tx=0x7f01e0007bb0).stop 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 -- 172.21.15.73:0/2820308463 shutdown_connections 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 --2- 172.21.15.73:0/2820308463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01d80038e0 0x7f01d8003d00 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 -- 172.21.15.73:0/2820308463 >> 172.21.15.73:0/2820308463 conn(0x7f01f00cb010 msgr2=0x7f01f00cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:34.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 -- 172.21.15.73:0/2820308463 shutdown_connections 2023-11-28T04:07:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.071+0000 7f01f5767700 1 -- 172.21.15.73:0/2820308463 wait complete. 2023-11-28T04:07:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.072+0000 7f01f5767700 1 Processor -- start 2023-11-28T04:07:34.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 -- start start 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d80038e0 0x7f01f00b0120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01f00ae4b0 0x7f01f00ad000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00ae910 con 0x7f01f01532a0 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00af3e0 con 0x7f01d80038e0 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f5767700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f00af6b0 con 0x7f01f00ae4b0 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01ef7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52314/0 (socket says 172.21.15.73:52314) 2023-11-28T04:07:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01ef7fe700 1 -- 172.21.15.73:0/780144955 learned_addr learned my addr 172.21.15.73:0/780144955 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01f4f66700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01f00ae4b0 0x7f01f00ad000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.073+0000 7f01effff700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d80038e0 0x7f01f00b0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 -- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01f00ae4b0 msgr2=0x7f01f00ad000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:34.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01f00ae4b0 0x7f01f00ad000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 -- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d80038e0 msgr2=0x7f01f00b0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:34.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d80038e0 0x7f01f00b0120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 -- 172.21.15.73:0/780144955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f00aeb30 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01ef7fe700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f01e40116c0 tx=0x7f01e4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e40145f0 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01e4014750 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01f00af920 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01e4014ad0 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.074+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01f00aee10 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.075+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f01e4015480 con 0x7f01f01532a0 2023-11-28T04:07:34.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.075+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01f00c6c70 con 0x7f01f01532a0 2023-11-28T04:07:34.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.075+0000 7f01d7fff700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 0x7f01dc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:34.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.076+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 6609+0+0 (secure 0 0 0) 0x7f01e407aad0 con 0x7f01f01532a0 2023-11-28T04:07:34.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.076+0000 7f01effff700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 0x7f01dc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:34.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.076+0000 7f01effff700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 0x7f01dc061610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f01e0001010 tx=0x7f01e0015000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:34.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.080+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f01e404bbb0 con 0x7f01f01532a0 2023-11-28T04:07:34.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:34.242+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f01f006c7c0 con 0x7f01f01532a0 2023-11-28T04:07:35.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.049+0000 7f01d7fff700 1 -- 172.21.15.73:0/780144955 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v155) v1 ==== 181+0+0 (secure 0 0 0) 0x7f01e404f1d0 con 0x7f01f01532a0 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 msgr2=0x7f01dc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 0x7f01dc061610 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f01e0001010 tx=0x7f01e0015000).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 msgr2=0x7f01f00adfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 rx=0x7f01e40116c0 tx=0x7f01e4008040).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 shutdown_connections 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f01dc05f150 0x7f01dc061610 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f01f00ae4b0 0x7f01f00ad000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f01d80038e0 0x7f01f00b0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 --2- 172.21.15.73:0/780144955 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f01f01532a0 0x7f01f00adfc0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 >> 172.21.15.73:0/780144955 conn(0x7f01f00cb010 msgr2=0x7f01f00c1550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:35.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.052+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 shutdown_connections 2023-11-28T04:07:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.053+0000 7f01f5767700 1 -- 172.21.15.73:0/780144955 wait complete. 2023-11-28T04:07:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T04:07:35.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T04:07:35.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.189+0000 7f5045e71700 1 Processor -- start 2023-11-28T04:07:35.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.189+0000 7f5045e71700 1 -- start start 2023-11-28T04:07:35.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.189+0000 7f5045e71700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50400ab3b0 0x7f50401582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f5045e71700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5040159210 con 0x7f5040154610 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f5045e71700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50400c0c70 con 0x7f50400aa1c0 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f5045e71700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5040158970 con 0x7f50400ab3b0 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5040154610 0x7f50400c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36674/0 (socket says 172.21.15.73:36674) 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503f7fe700 1 -- 172.21.15.73:0/1522759692 learned_addr learned my addr 172.21.15.73:0/1522759692 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2433718815 0 0) 0x7f5040159210 con 0x7f5040154610 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f502c0033f0 con 0x7f5040154610 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50400ab3b0 0x7f50401582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4135927632 0 0) 0x7f502c0033f0 con 0x7f5040154610 2023-11-28T04:07:35.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50400ab3b0 msgr2=0x7f50401582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50400ab3b0 0x7f50401582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 >> v1:172.21.15.150:6789/0 conn(0x7f50400aa1c0 legacy=0x7f50400c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5040158c50 con 0x7f5040154610 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f50380017f0 con 0x7f5040154610 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50400ab3b0 0x7f50401582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 >> v1:172.21.15.73:6789/0 conn(0x7f5040154610 legacy=0x7f50400c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c003910 0x7f502c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502c004690 0x7f502c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c005050 0x7f502c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5040158970 con 0x7f502c004690 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f502c00bdd0 con 0x7f502c003910 2023-11-28T04:07:35.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f502c00bf40 con 0x7f502c005050 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502c004690 0x7f502c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503f7fe700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c003910 0x7f502c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.190+0000 7f503ffff700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c005050 0x7f502c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 -- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c005050 msgr2=0x7f502c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c005050 0x7f502c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 -- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c003910 msgr2=0x7f502c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f502c003910 0x7f502c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 -- 172.21.15.73:0/1522759692 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f502c0033f0 con 0x7f502c004690 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503ffff700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c005050 0x7f502c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503effd700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502c004690 0x7f502c004b10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f502800ba90 tx=0x7f5028008040).ready entity=mon.0 client_cookie=10bed774ff6b79a6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800b400 con 0x7f502c004690 2023-11-28T04:07:35.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f502800d5d0 con 0x7f502c004690 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503e7fc700 1 -- 172.21.15.73:0/1522759692 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f502800d950 con 0x7f502c004690 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 -- 172.21.15.73:0/1522759692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502c004690 msgr2=0x7f502c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 --2- 172.21.15.73:0/1522759692 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f502c004690 0x7f502c004b10 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 rx=0x7f502800ba90 tx=0x7f5028008040).stop 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f503ffff700 1 -- 172.21.15.73:0/1522759692 reap_dead start 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 -- 172.21.15.73:0/1522759692 shutdown_connections 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 -- 172.21.15.73:0/1522759692 >> 172.21.15.73:0/1522759692 conn(0x7f50400655a0 msgr2=0x7f5040069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 -- 172.21.15.73:0/1522759692 shutdown_connections 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.191+0000 7f5045e71700 1 -- 172.21.15.73:0/1522759692 wait complete. 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 Processor -- start 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 -- start start 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c004690 0x7f50400c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50400c97b0 0x7f5040153070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ae5b0 con 0x7f5040153560 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50400ae890 con 0x7f50400c97b0 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f5045e71700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50400aeb70 con 0x7f502c004690 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f503ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f503ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52342/0 (socket says 172.21.15.73:52342) 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.193+0000 7f503ffff700 1 -- 172.21.15.73:0/1930204822 learned_addr learned my addr 172.21.15.73:0/1930204822 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503effd700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50400c97b0 0x7f5040153070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503f7fe700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c004690 0x7f50400c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 -- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c004690 msgr2=0x7f50400c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c004690 0x7f50400c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 -- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50400c97b0 msgr2=0x7f5040153070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50400c97b0 0x7f5040153070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 -- 172.21.15.73:0/1930204822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50400aee50 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f503ffff700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f5030000c00 tx=0x7f5030009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030014430 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5030014590 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50400af130 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5030014910 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.194+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5040141dc0 con 0x7f5040153560 2023-11-28T04:07:35.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.195+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50400af320 con 0x7f5040153560 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.195+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5030014d40 con 0x7f5040153560 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.196+0000 7f50377fe700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 0x7f5020061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.196+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 6113+0+0 (secure 0 0 0) 0x7f50300795a0 con 0x7f5040153560 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.196+0000 7f503f7fe700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 0x7f5020061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.196+0000 7f503f7fe700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 0x7f5020061640 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5038004a30 tx=0x7f5038009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:35.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.200+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f503004a870 con 0x7f5040153560 2023-11-28T04:07:35.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:35.369+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f504006c860 con 0x7f5040153560 2023-11-28T04:07:36.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.052+0000 7f50377fe700 1 -- 172.21.15.73:0/1930204822 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v156) v1 ==== 169+0+0 (secure 0 0 0) 0x7f503004de90 con 0x7f5040153560 2023-11-28T04:07:36.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 msgr2=0x7f5020061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 0x7f5020061640 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f5038004a30 tx=0x7f5038009040).stop 2023-11-28T04:07:36.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 msgr2=0x7f50401418d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 rx=0x7f5030000c00 tx=0x7f5030009040).stop 2023-11-28T04:07:36.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 shutdown_connections 2023-11-28T04:07:36.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f502005f180 0x7f5020061640 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5040153560 0x7f50401418d0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f502c004690 0x7f50400c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 --2- 172.21.15.73:0/1930204822 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50400c97b0 0x7f5040153070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 >> 172.21.15.73:0/1930204822 conn(0x7f50400655a0 msgr2=0x7f50400ca210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:36.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 shutdown_connections 2023-11-28T04:07:36.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.055+0000 7f5045e71700 1 -- 172.21.15.73:0/1930204822 wait complete. 2023-11-28T04:07:36.060 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T04:07:36.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T04:07:36.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 Processor -- start 2023-11-28T04:07:36.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 -- start start 2023-11-28T04:07:36.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 0x7f73e80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e8158570 0x7f73e8158a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73e80c7bc0 con 0x7f73e8154950 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73e8155920 con 0x7f73e8158570 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.188+0000 7f73edd95700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8154dd0 con 0x7f73e814e530 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e8158570 0x7f73e8158a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f73e8158570 0x7f73e8158a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52346/0 (socket says 172.21.15.73:52346) 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 -- 172.21.15.73:0/3151112469 learned_addr learned my addr 172.21.15.73:0/3151112469 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ecd93700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 0x7f73e80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 -- 172.21.15.73:0/3151112469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 msgr2=0x7f73e80c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 0x7f73e80c66f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 -- 172.21.15.73:0/3151112469 >> v1:172.21.15.150:6789/0 conn(0x7f73e8154950 legacy=0x7f73e814e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 -- 172.21.15.73:0/3151112469 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e8155080 con 0x7f73e8158570 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ecd93700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 0x7f73e80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73ed594700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e8158a20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f73e0006fd0 tx=0x7f73e00090f0).ready entity=mon.0 client_cookie=dcae767811115b30 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73df7fe700 1 -- 172.21.15.73:0/3151112469 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000b600 con 0x7f73e8158570 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73df7fe700 1 -- 172.21.15.73:0/3151112469 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e000b760 con 0x7f73e8158570 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.189+0000 7f73df7fe700 1 -- 172.21.15.73:0/3151112469 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000bb00 con 0x7f73e8158570 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 -- 172.21.15.73:0/3151112469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 msgr2=0x7f73e8158a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e8158a20 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 rx=0x7f73e0006fd0 tx=0x7f73e00090f0).stop 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 -- 172.21.15.73:0/3151112469 shutdown_connections 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e8158a20 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 --2- 172.21.15.73:0/3151112469 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e814e530 0x7f73e80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 -- 172.21.15.73:0/3151112469 >> 172.21.15.73:0/3151112469 conn(0x7f73e8064990 msgr2=0x7f73e8064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:36.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 -- 172.21.15.73:0/3151112469 shutdown_connections 2023-11-28T04:07:36.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.190+0000 7f73edd95700 1 -- 172.21.15.73:0/3151112469 wait complete. 2023-11-28T04:07:36.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.191+0000 7f73edd95700 1 Processor -- start 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.191+0000 7f73edd95700 1 -- start start 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e814e530 0x7f73e813c1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 0x7f73e81672b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e8166e20 con 0x7f73e8158570 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e813c6d0 con 0x7f73e814e530 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73edd95700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73e80af450 con 0x7f73e8154950 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52362/0 (socket says 172.21.15.73:52362) 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 -- 172.21.15.73:0/2458258221 learned_addr learned my addr 172.21.15.73:0/2458258221 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:36.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ecd93700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e814e530 0x7f73e813c1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73dffff700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 0x7f73e81672b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 -- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 msgr2=0x7f73e81672b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 0x7f73e81672b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 -- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e814e530 msgr2=0x7f73e813c1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e814e530 0x7f73e813c1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 -- 172.21.15.73:0/2458258221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73e0009370 con 0x7f73e8158570 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73dffff700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 0x7f73e81672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.192+0000 7f73ed594700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f73e0009f50 tx=0x7f73e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.193+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e000bec0 con 0x7f73e8158570 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.193+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73e000bae0 con 0x7f73e8158570 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.193+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e8166210 con 0x7f73e8158570 2023-11-28T04:07:36.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.193+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73e0025420 con 0x7f73e8158570 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.193+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e81667d0 con 0x7f73e8158570 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.194+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e80ad6f0 con 0x7f73e8158570 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.194+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f73e0022020 con 0x7f73e8158570 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.194+0000 7f73dcff9700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 0x7f73d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.194+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7f73e007bb40 con 0x7f73e8158570 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.194+0000 7f73ecd93700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 0x7f73d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.195+0000 7f73ecd93700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 0x7f73d4061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f73e8157a80 tx=0x7f73e4007710).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.198+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f73e004cfb0 con 0x7f73e8158570 2023-11-28T04:07:36.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.365+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f73e8062ee0 con 0x7f73e8158570 2023-11-28T04:07:36.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.368+0000 7f73dcff9700 1 -- 172.21.15.73:0/2458258221 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v28) v1 ==== 120+0+0 (secure 0 0 0) 0x7f73e002a070 con 0x7f73e8158570 2023-11-28T04:07:36.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 msgr2=0x7f73d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 0x7f73d4061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f73e8157a80 tx=0x7f73e4007710).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 msgr2=0x7f73e81656a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 rx=0x7f73e0009f50 tx=0x7f73e0017000).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 shutdown_connections 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f73d405f150 0x7f73d4061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f73e8158570 0x7f73e81656a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f73e814e530 0x7f73e813c1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 --2- 172.21.15.73:0/2458258221 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f73e8154950 0x7f73e81672b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 >> 172.21.15.73:0/2458258221 conn(0x7f73e8064990 msgr2=0x7f73e8156400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 shutdown_connections 2023-11-28T04:07:36.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.370+0000 7f73edd95700 1 -- 172.21.15.73:0/2458258221 wait complete. 2023-11-28T04:07:36.383 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T04:07:36.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.497+0000 7f85aa13e700 1 Processor -- start 2023-11-28T04:07:36.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.497+0000 7f85aa13e700 1 -- start start 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85aa13e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f85a40c24a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85aa13e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a41554d0 con 0x7f85a40bd680 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85aa13e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85a41548c0 con 0x7f85a40bdb00 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85aa13e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4154ba0 con 0x7f85a40c20c0 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85a2ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f85a40bd680 0x7f85a4069f10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36684/0 (socket says 172.21.15.73:36684) 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.498+0000 7f85a2ffd700 1 -- 172.21.15.73:0/1957866505 learned_addr learned my addr 172.21.15.73:0/1957866505 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a37fe700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f85a40c24a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1589374795 0 0) 0x7f85a41554d0 con 0x7f85a40bd680 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f858c0033f0 con 0x7f85a40bd680 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2347301192 0 0) 0x7f858c0033f0 con 0x7f85a40bd680 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 msgr2=0x7f85a40c24a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f85a40c24a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 >> v1:172.21.15.150:6789/0 conn(0x7f85a40bdb00 legacy=0x7f85a4150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a4154e80 con 0x7f85a40bd680 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8598001470 con 0x7f85a40bd680 2023-11-28T04:07:36.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 >> v1:172.21.15.73:6789/0 conn(0x7f85a40bd680 legacy=0x7f85a4069f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f858c003910 0x7f858c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f858c004690 0x7f858c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f858c0050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4154ba0 con 0x7f858c004690 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f858c00cad0 con 0x7f858c003910 2023-11-28T04:07:36.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f858c00cc90 con 0x7f85a40c20c0 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f858c004690 0x7f858c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a37fe700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f858c003910 0x7f858c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a3fff700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f858c0050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 -- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 msgr2=0x7f858c0050b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f85a40c20c0 0x7f858c0050b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 -- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f858c003910 msgr2=0x7f858c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:36.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f858c003910 0x7f858c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.499+0000 7f85a2ffd700 1 -- 172.21.15.73:0/1957866505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f858c0033f0 con 0x7f858c004690 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85a2ffd700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f858c004690 0x7f858c004b10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f859800c160 tx=0x7f8598001e50).ready entity=mon.0 client_cookie=8966cb75d1d648f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8598007a20 con 0x7f858c004690 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859800ca90 con 0x7f858c004690 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85a27fc700 1 -- 172.21.15.73:0/1957866505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859800ce10 con 0x7f858c004690 2023-11-28T04:07:36.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 -- 172.21.15.73:0/1957866505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f858c004690 msgr2=0x7f858c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1957866505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f858c004690 0x7f858c004b10 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 rx=0x7f859800c160 tx=0x7f8598001e50).stop 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85a3fff700 1 -- 172.21.15.73:0/1957866505 reap_dead start 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 -- 172.21.15.73:0/1957866505 shutdown_connections 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 -- 172.21.15.73:0/1957866505 >> 172.21.15.73:0/1957866505 conn(0x7f85a4062700 msgr2=0x7f85a4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 -- 172.21.15.73:0/1957866505 shutdown_connections 2023-11-28T04:07:36.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.500+0000 7f85aa13e700 1 -- 172.21.15.73:0/1957866505 wait complete. 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 Processor -- start 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 -- start start 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f858c004690 0x7f85a4150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85a4150b40 0x7f85a40a5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4157300 con 0x7f85a40a5d10 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85a41527b0 con 0x7f85a4150b40 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85aa13e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85a4156a60 con 0x7f858c004690 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.502+0000 7f85a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52376/0 (socket says 172.21.15.73:52376) 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 -- 172.21.15.73:0/1652135224 learned_addr learned my addr 172.21.15.73:0/1652135224 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a37fe700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f858c004690 0x7f85a4150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a2ffd700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85a4150b40 0x7f85a40a5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 -- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f858c004690 msgr2=0x7f85a4150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f858c004690 0x7f85a4150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 -- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85a4150b40 msgr2=0x7f85a40a5820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85a4150b40 0x7f85a40a5820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 -- 172.21.15.73:0/1652135224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85a4156d40 con 0x7f85a40a5d10 2023-11-28T04:07:36.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85a3fff700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f858c004610 tx=0x7f859c009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c00ee40 con 0x7f85a40a5d10 2023-11-28T04:07:36.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f859c014430 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85a4157020 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f859c0147f0 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.503+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85a4156680 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.504+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85a406c860 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.504+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f859c014c20 con 0x7f85a40a5d10 2023-11-28T04:07:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.505+0000 7f858b7fe700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 0x7f8590061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.505+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7f859c079720 con 0x7f85a40a5d10 2023-11-28T04:07:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.505+0000 7f85a37fe700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 0x7f8590061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.505+0000 7f85a37fe700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 0x7f8590061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f85940094d0 tx=0x7f859400b040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:36.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.509+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f859c049af0 con 0x7f85a40a5d10 2023-11-28T04:07:36.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.673+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f85a4155160 con 0x7f85a40a5d10 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.677+0000 7f858b7fe700 1 -- 172.21.15.73:0/1652135224 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v29) v1 ==== 104+0+0 (secure 0 0 0) 0x7f859c04ab30 con 0x7f85a40a5d10 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 msgr2=0x7f8590061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 0x7f8590061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f85940094d0 tx=0x7f859400b040).stop 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 msgr2=0x7f85a4148660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 rx=0x7f858c004610 tx=0x7f859c009040).stop 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 shutdown_connections 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f859005f150 0x7f8590061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f85a40a5d10 0x7f85a4148660 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f858c004690 0x7f85a4150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 --2- 172.21.15.73:0/1652135224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f85a4150b40 0x7f85a40a5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 >> 172.21.15.73:0/1652135224 conn(0x7f85a4062700 msgr2=0x7f85a406a4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:37.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 shutdown_connections 2023-11-28T04:07:37.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:36.679+0000 7f85aa13e700 1 -- 172.21.15.73:0/1652135224 wait complete. 2023-11-28T04:07:37.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-11-28T04:07:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.238+0000 7f381053f700 1 Processor -- start 2023-11-28T04:07:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.238+0000 7f381053f700 1 -- start start 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f381053f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f38080aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f381053f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38080a9a90 con 0x7f38080cae90 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f381053f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3808154ce0 con 0x7f38080ca180 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f381053f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38080a8e80 con 0x7f38080caab0 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f380dada700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38080cae90 0x7f38080c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36694/0 (socket says 172.21.15.73:36694) 2023-11-28T04:07:37.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f380dada700 1 -- 172.21.15.73:0/3419707392 learned_addr learned my addr 172.21.15.73:0/3419707392 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 373476077 0 0) 0x7f38080a9a90 con 0x7f38080cae90 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37f40033f0 con 0x7f38080cae90 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.239+0000 7f380e2db700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f38080aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3118724647 0 0) 0x7f37f40033f0 con 0x7f38080cae90 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 msgr2=0x7f38080aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f38080aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 >> v1:172.21.15.150:6789/0 conn(0x7f38080ca180 legacy=0x7f380815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38080a9160 con 0x7f38080cae90 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f37f8001470 con 0x7f38080cae90 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 >> v1:172.21.15.73:6789/0 conn(0x7f38080cae90 legacy=0x7f38080c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f37f4003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37f4003ef0 0x7f37f4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f4004b80 0x7f37f4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080a8e80 con 0x7f37f4004b80 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400ba90 con 0x7f37f4003ef0 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37f400bc50 con 0x7f38080caab0 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f4004b80 0x7f37f4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380e2db700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f37f4003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380dada700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37f4003ef0 0x7f37f4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 -- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 msgr2=0x7f37f4003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38080caab0 0x7f37f4003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 -- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37f4003ef0 msgr2=0x7f37f4004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37f4003ef0 0x7f37f4004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 -- 172.21.15.73:0/3419707392 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37f40033f0 con 0x7f37f4004b80 2023-11-28T04:07:37.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380dada700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f37f4003ef0 0x7f37f4004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380eadc700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f4004b80 0x7f37f4005000 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f3800001730 tx=0x7f3800007bb0).ready entity=mon.0 client_cookie=5d64729965af1344 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800007020 con 0x7f37f4004b80 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3800007180 con 0x7f37f4004b80 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.240+0000 7f380d2d9700 1 -- 172.21.15.73:0/3419707392 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800007500 con 0x7f37f4004b80 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 -- 172.21.15.73:0/3419707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f4004b80 msgr2=0x7f37f4005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 --2- 172.21.15.73:0/3419707392 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f37f4004b80 0x7f37f4005000 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 rx=0x7f3800001730 tx=0x7f3800007bb0).stop 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f380eadc700 1 -- 172.21.15.73:0/3419707392 reap_dead start 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 -- 172.21.15.73:0/3419707392 shutdown_connections 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 -- 172.21.15.73:0/3419707392 >> 172.21.15.73:0/3419707392 conn(0x7f3808094920 msgr2=0x7f380806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 -- 172.21.15.73:0/3419707392 shutdown_connections 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.241+0000 7f381053f700 1 -- 172.21.15.73:0/3419707392 wait complete. 2023-11-28T04:07:37.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.242+0000 7f381053f700 1 Processor -- start 2023-11-28T04:07:37.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.242+0000 7f381053f700 1 -- start start 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37f4004b80 0x7f3808156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3808164b10 0x7f3808164f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38080ad0f0 con 0x7f38080ac780 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3808157190 con 0x7f3808164b10 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3808157470 con 0x7f37f4004b80 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52390/0 (socket says 172.21.15.73:52390) 2023-11-28T04:07:37.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 -- 172.21.15.73:0/2601171481 learned_addr learned my addr 172.21.15.73:0/2601171481 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380e2db700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37f4004b80 0x7f3808156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380dada700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3808164b10 0x7f3808164f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 -- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37f4004b80 msgr2=0x7f3808156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37f4004b80 0x7f3808156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 -- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3808164b10 msgr2=0x7f3808164f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3808164b10 0x7f3808164f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 -- 172.21.15.73:0/2601171481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3808157750 con 0x7f38080ac780 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f380eadc700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f3800010000 tx=0x7f3800015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800013670 con 0x7f38080ac780 2023-11-28T04:07:37.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38000137d0 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3808157a30 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.243+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3800013b90 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.244+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3808156d30 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.244+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3808156f50 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.245+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f380001f400 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.245+0000 7f37fe7fc700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 0x7f37ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.245+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7f380007ab70 con 0x7f38080ac780 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.245+0000 7f380e2db700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 0x7f37ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.245+0000 7f380e2db700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 0x7f37ec061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f38080abf70 tx=0x7f3804009760).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.249+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3808156f50 con 0x7f38080ac780 2023-11-28T04:07:37.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:37.477+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f3808156f50 con 0x7f38080ac780 2023-11-28T04:07:38.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.065+0000 7f37fe7fc700 1 -- 172.21.15.73:0/2601171481 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2393) v1 ==== 178+0+0 (secure 0 0 0) 0x7f3808156f50 con 0x7f38080ac780 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 msgr2=0x7f37ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 0x7f37ec061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f38080abf70 tx=0x7f3804009760).stop 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 msgr2=0x7f38080acc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 rx=0x7f3800010000 tx=0x7f3800015000).stop 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 shutdown_connections 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f37ec05f150 0x7f37ec061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38080ac780 0x7f38080acc00 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f37f4004b80 0x7f3808156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 --2- 172.21.15.73:0/2601171481 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3808164b10 0x7f3808164f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 >> 172.21.15.73:0/2601171481 conn(0x7f3808094920 msgr2=0x7f3808064130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 shutdown_connections 2023-11-28T04:07:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.068+0000 7f381053f700 1 -- 172.21.15.73:0/2601171481 wait complete. 2023-11-28T04:07:38.081 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-11-28T04:07:38.081 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-11-28T04:07:38.081 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T04:07:38.081 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-11-28T04:07:38.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.194+0000 7fde533c9700 1 Processor -- start 2023-11-28T04:07:38.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.194+0000 7fde533c9700 1 -- start start 2023-11-28T04:07:38.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde533c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde4c155c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde533c9700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c099020 con 0x7fde4c158050 2023-11-28T04:07:38.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde533c9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde4c063460 con 0x7fde4c0c0df0 2023-11-28T04:07:38.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde533c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c063d00 con 0x7fde4c0c6380 2023-11-28T04:07:38.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde523c7700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fde4c158050 0x7fde4c158430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36704/0 (socket says 172.21.15.73:36704) 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde523c7700 1 -- 172.21.15.73:0/1304214587 learned_addr learned my addr 172.21.15.73:0/1304214587 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde51bc6700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde4c155c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2382739214 0 0) 0x7fde4c099020 con 0x7fde4c158050 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.195+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde380033f0 con 0x7fde4c158050 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1045905464 0 0) 0x7fde380033f0 con 0x7fde4c158050 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 msgr2=0x7fde4c155c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde4c155c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 >> v1:172.21.15.150:6789/0 conn(0x7fde4c0c0df0 legacy=0x7fde4c1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0c1780 con 0x7fde4c158050 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fde480015f0 con 0x7fde4c158050 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 >> v1:172.21.15.73:6789/0 conn(0x7fde4c158050 legacy=0x7fde4c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde38003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde38003e00 0x7fde38004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38004b80 0x7fde38005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c063d00 con 0x7fde38003e00 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3800b990 con 0x7fde38004b80 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3800bb50 con 0x7fde4c0c6380 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde38003e00 0x7fde38004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde523c7700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde38003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde52bc8700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38004b80 0x7fde38005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 -- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 msgr2=0x7fde38003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:38.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c0c6380 0x7fde38003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 -- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38004b80 msgr2=0x7fde38005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38004b80 0x7fde38005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 -- 172.21.15.73:0/1304214587 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde380033f0 con 0x7fde38003e00 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde51bc6700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde38003e00 0x7fde38004690 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fde3c009a80 tx=0x7fde3c009060).ready entity=mon.0 client_cookie=88ffabe6eee13dd8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c009430 con 0x7fde38003e00 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde3c00e640 con 0x7fde38003e00 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.196+0000 7fde513c5700 1 -- 172.21.15.73:0/1304214587 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3c00e9e0 con 0x7fde38003e00 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 -- 172.21.15.73:0/1304214587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde38003e00 msgr2=0x7fde38004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 --2- 172.21.15.73:0/1304214587 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde38003e00 0x7fde38004690 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 rx=0x7fde3c009a80 tx=0x7fde3c009060).stop 2023-11-28T04:07:38.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde52bc8700 1 -- 172.21.15.73:0/1304214587 reap_dead start 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 -- 172.21.15.73:0/1304214587 shutdown_connections 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 -- 172.21.15.73:0/1304214587 >> 172.21.15.73:0/1304214587 conn(0x7fde4c0cb080 msgr2=0x7fde4c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 -- 172.21.15.73:0/1304214587 shutdown_connections 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.197+0000 7fde533c9700 1 -- 172.21.15.73:0/1304214587 wait complete. 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.198+0000 7fde533c9700 1 Processor -- start 2023-11-28T04:07:38.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.198+0000 7fde533c9700 1 -- start start 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 0x7fde4c150750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c150c40 0x7fde4c153970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0c2e60 con 0x7fde4c153e60 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c1542a0 con 0x7fde38003e00 2023-11-28T04:07:38.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde533c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde4c0c2250 con 0x7fde4c150c40 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52406/0 (socket says 172.21.15.73:52406) 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 -- 172.21.15.73:0/3539490331 learned_addr learned my addr 172.21.15.73:0/3539490331 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde523c7700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 0x7fde4c150750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 -- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c150c40 msgr2=0x7fde4c153970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c150c40 0x7fde4c153970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 -- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 msgr2=0x7fde4c150750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 0x7fde4c150750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 -- 172.21.15.73:0/3539490331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde4c0c2530 con 0x7fde4c153e60 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde523c7700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 0x7fde4c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.199+0000 7fde52bc8700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fde4c158b40 tx=0x7fde44009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:38.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.200+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440030d0 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.200+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde44003230 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.200+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde4c0c2810 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.200+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde440035b0 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.200+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde4c0c32f0 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.201+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde4c0c3010 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.201+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fde44001a60 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.201+0000 7fde427fc700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 0x7fde30061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.201+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7fde44079b60 con 0x7fde4c153e60 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.201+0000 7fde523c7700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 0x7fde30061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.202+0000 7fde523c7700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 0x7fde30061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fde48000ef0 tx=0x7fde48009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:38.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.205+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fde4c0c3010 con 0x7fde4c153e60 2023-11-28T04:07:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:38.433+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fde4c0c3010 con 0x7fde4c153e60 2023-11-28T04:07:39.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.068+0000 7fde427fc700 1 -- 172.21.15.73:0/3539490331 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2394) v1 ==== 177+0+0 (secure 0 0 0) 0x7fde4c0c3010 con 0x7fde4c153e60 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.069+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 msgr2=0x7fde30061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.069+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 0x7fde30061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fde48000ef0 tx=0x7fde48009040).stop 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.069+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 msgr2=0x7fde4c14eb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.069+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 rx=0x7fde4c158b40 tx=0x7fde44009040).stop 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 shutdown_connections 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fde3005f150 0x7fde30061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde4c153e60 0x7fde4c14eb70 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde38003e00 0x7fde4c150750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 --2- 172.21.15.73:0/3539490331 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde4c150c40 0x7fde4c153970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 >> 172.21.15.73:0/3539490331 conn(0x7fde4c0cb080 msgr2=0x7fde4c158430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 shutdown_connections 2023-11-28T04:07:39.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.070+0000 7fde533c9700 1 -- 172.21.15.73:0/3539490331 wait complete. 2023-11-28T04:07:39.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T04:07:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.199+0000 7fcc11381700 1 Processor -- start 2023-11-28T04:07:39.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.199+0000 7fcc11381700 1 -- start start 2023-11-28T04:07:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc11381700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcc0c154650 0x7fcc0c0c6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc11381700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0c9730 con 0x7fcc0c0aa770 2023-11-28T04:07:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc11381700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc0c0c99f0 con 0x7fcc0c154270 2023-11-28T04:07:39.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc11381700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c6b40 con 0x7fcc0c154650 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fcc0c0aa770 0x7fcc0c0c9e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36706/0 (socket says 172.21.15.73:36706) 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0b7fe700 1 -- 172.21.15.73:0/21170278 learned_addr learned my addr 172.21.15.73:0/21170278 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0a7fc700 1 --2- 172.21.15.73:0/21170278 >> v2:172.21.15.150:3300/0 conn(0x7fcc0c154650 0x7fcc0c0c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2048460061 0 0) 0x7fcc0c0c9730 con 0x7fcc0c0aa770 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcbf40033f0 con 0x7fcc0c0aa770 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/21170278 >> v1:172.21.15.150:6789/0 conn(0x7fcc0c154270 legacy=0x7fcc0c0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2289603296 0 0) 0x7fcbf40033f0 con 0x7fcc0c0aa770 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/21170278 >> v1:172.21.15.73:6789/0 conn(0x7fcc0c0aa770 legacy=0x7fcc0c0c9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/21170278 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc0c1586d0 con 0x7fcc0c154650 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.200+0000 7fcc0a7fc700 1 --2- 172.21.15.73:0/21170278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c6650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fcc00009e60 tx=0x7fcc00009780).ready entity=mon.1 client_cookie=800ab73bfedeb894 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000e070 con 0x7fcc0c154650 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000b850 con 0x7fcc0c154650 2023-11-28T04:07:39.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc09ffb700 1 -- 172.21.15.73:0/21170278 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc001470 con 0x7fcc0c154650 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 -- 172.21.15.73:0/21170278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 msgr2=0x7fcc0c0c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 --2- 172.21.15.73:0/21170278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c6650 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 rx=0x7fcc00009e60 tx=0x7fcc00009780).stop 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 -- 172.21.15.73:0/21170278 shutdown_connections 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 --2- 172.21.15.73:0/21170278 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c6650 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 -- 172.21.15.73:0/21170278 >> 172.21.15.73:0/21170278 conn(0x7fcc0c0649b0 msgr2=0x7fcc0c064dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 -- 172.21.15.73:0/21170278 shutdown_connections 2023-11-28T04:07:39.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.201+0000 7fcc11381700 1 -- 172.21.15.73:0/21170278 wait complete. 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.202+0000 7fcc11381700 1 Processor -- start 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.202+0000 7fcc11381700 1 -- start start 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 0x7fcc0c0b0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c3960 con 0x7fcc0c154270 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c5230 con 0x7fcc0c154650 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc11381700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0c0c2ce0 con 0x7fcc0c0aa770 2023-11-28T04:07:39.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0a7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52422/0 (socket says 172.21.15.73:52422) 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/2615501474 learned_addr learned my addr 172.21.15.73:0/2615501474 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 msgr2=0x7fcc0c0b0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0affd700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 0x7fcc0c0b0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.203+0000 7fcc0b7fe700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0a7fc700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 0x7fcc0c0b0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 msgr2=0x7fcc0c0c4d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0a7fc700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c4d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0a7fc700 1 -- 172.21.15.73:0/2615501474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc00009a20 con 0x7fcc0c154270 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0affd700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 0x7fcc0c0b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0b7fe700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc0a7fc700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fcc00010000 tx=0x7fcc00009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000ba90 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc00000f40 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc0c0c2fc0 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0001e680 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.204+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc0c0c3580 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.205+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0c0c3aa0 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.205+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcc00003040 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.205+0000 7fcbf2ffd700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 0x7fcbf8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.206+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7fcc0007b310 con 0x7fcc0c154270 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.206+0000 7fcc0affd700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 0x7fcbf8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.206+0000 7fcc0affd700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 0x7fcbf8061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fcc04007fd0 tx=0x7fcc04009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.210+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcc0c0c3aa0 con 0x7fcc0c154270 2023-11-28T04:07:39.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.374+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fcc0c0c3aa0 con 0x7fcc0c154270 2023-11-28T04:07:39.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.378+0000 7fcbf2ffd700 1 -- 172.21.15.73:0/2615501474 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v30)=0 v30) v1 ==== 122+0+0 (secure 0 0 0) 0x7fcc0c0c3aa0 con 0x7fcc0c154270 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 msgr2=0x7fcbf8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 0x7fcbf8061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7fcc04007fd0 tx=0x7fcc04009040).stop 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 msgr2=0x7fcc0c164180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 rx=0x7fcc00010000 tx=0x7fcc00009780).stop 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 shutdown_connections 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcbf805f150 0x7fcbf8061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcc0c154650 0x7fcc0c0c4d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcc0c0aa770 0x7fcc0c0b0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 --2- 172.21.15.73:0/2615501474 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcc0c154270 0x7fcc0c164180 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 >> 172.21.15.73:0/2615501474 conn(0x7fcc0c0649b0 msgr2=0x7fcc0c0cb510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:39.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 shutdown_connections 2023-11-28T04:07:39.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.380+0000 7fcc11381700 1 -- 172.21.15.73:0/2615501474 wait complete. 2023-11-28T04:07:39.392 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:07:39.393 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:07:39.429 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:07:39.473 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:07:39.473 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T04:07:39.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:39.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.584+0000 7feb9ea17700 1 Processor -- start 2023-11-28T04:07:39.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 -- start start 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feb98064e20 0x7feb98155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb98095e80 0x7feb980962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb980967e0 con 0x7feb981544d0 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb98067650 con 0x7feb98095e80 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9ea17700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb980683d0 con 0x7feb98064e20 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.585+0000 7feb9cfb4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb98095e80 0x7feb980962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7feb98095e80 0x7feb980962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52430/0 (socket says 172.21.15.73:52430) 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 -- 172.21.15.73:0/881017200 learned_addr learned my addr 172.21.15.73:0/881017200 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb977fe700 1 --2- 172.21.15.73:0/881017200 >> v2:172.21.15.150:3300/0 conn(0x7feb98064e20 0x7feb98155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 -- 172.21.15.73:0/881017200 >> v2:172.21.15.150:3300/0 conn(0x7feb98064e20 msgr2=0x7feb98155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 --2- 172.21.15.73:0/881017200 >> v2:172.21.15.150:3300/0 conn(0x7feb98064e20 0x7feb98155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 -- 172.21.15.73:0/881017200 >> v1:172.21.15.150:6789/0 conn(0x7feb981544d0 legacy=0x7feb98066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 -- 172.21.15.73:0/881017200 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb98067850 con 0x7feb98095e80 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb9cfb4700 1 --2- 172.21.15.73:0/881017200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb980962f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7feb9806a7d0 tx=0x7feb90009840).ready entity=mon.0 client_cookie=a761610c36d1ea59 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb96ffd700 1 -- 172.21.15.73:0/881017200 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000e070 con 0x7feb98095e80 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb96ffd700 1 -- 172.21.15.73:0/881017200 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb9000ba50 con 0x7feb98095e80 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.586+0000 7feb96ffd700 1 -- 172.21.15.73:0/881017200 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb9000bdd0 con 0x7feb98095e80 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 -- 172.21.15.73:0/881017200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 msgr2=0x7feb980962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 --2- 172.21.15.73:0/881017200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb980962f0 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 rx=0x7feb9806a7d0 tx=0x7feb90009840).stop 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 -- 172.21.15.73:0/881017200 shutdown_connections 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 --2- 172.21.15.73:0/881017200 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb980962f0 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 --2- 172.21.15.73:0/881017200 >> v2:172.21.15.150:3300/0 conn(0x7feb98064e20 0x7feb98155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 -- 172.21.15.73:0/881017200 >> 172.21.15.73:0/881017200 conn(0x7feb980631d0 msgr2=0x7feb980c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:39.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 -- 172.21.15.73:0/881017200 shutdown_connections 2023-11-28T04:07:39.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.587+0000 7feb9ea17700 1 -- 172.21.15.73:0/881017200 wait complete. 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.588+0000 7feb9ea17700 1 Processor -- start 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.588+0000 7feb9ea17700 1 -- start start 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb98064e20 0x7feb980c1830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981544d0 0x7feb980a6e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb981554a0 con 0x7feb98095e80 2023-11-28T04:07:39.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb98155a60 con 0x7feb981544d0 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb9814d330 con 0x7feb98064e20 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52440/0 (socket says 172.21.15.73:52440) 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 -- 172.21.15.73:0/4291853581 learned_addr learned my addr 172.21.15.73:0/4291853581 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9cfb4700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981544d0 0x7feb980a6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb97fff700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb98064e20 0x7feb980c1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 -- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb98064e20 msgr2=0x7feb980c1830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb98064e20 0x7feb980c1830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 -- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981544d0 msgr2=0x7feb980a6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981544d0 0x7feb980a6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 -- 172.21.15.73:0/4291853581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009ae0 con 0x7feb98095e80 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb977fe700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7feb8c00d8f0 tx=0x7feb8c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c0095b0 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb8c009710 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb8c009a90 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb9814d8e0 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.589+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb981556f0 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.590+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb9806c860 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.591+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feb8c013480 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.591+0000 7feb7ffff700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 0x7feb84061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.591+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5699+0+0 (secure 0 0 0) 0x7feb8c079e50 con 0x7feb98095e80 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.591+0000 7feb97fff700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 0x7feb84061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:39.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.592+0000 7feb97fff700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 0x7feb84061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7feb88007fd0 tx=0x7feb88009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:39.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.595+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feb8c04b260 con 0x7feb98095e80 2023-11-28T04:07:39.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.796+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feb98067b30 con 0x7feb98095e80 2023-11-28T04:07:39.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.796+0000 7feb7ffff700 1 -- 172.21.15.73:0/4291853581 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+3526 (secure 0 0 0) 0x7feb8c04e880 con 0x7feb98095e80 2023-11-28T04:07:39.797 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:39.798 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":54,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6545,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/90514872","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":90514872},{"type":"v1","addr":"172.21.15.150:6834","nonce":90514872}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7372,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/799233036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":799233036},{"type":"v1","addr":"172.21.15.150:6835","nonce":799233036}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8091,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/4197870683","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":4197870683},{"type":"v1","addr":"172.21.15.73:6836","nonce":4197870683}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8475,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3354099781","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3354099781},{"type":"v1","addr":"172.21.15.73:6837","nonce":3354099781}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":53}],"filesystems":[]} 2023-11-28T04:07:39.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 msgr2=0x7feb84061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 0x7feb84061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7feb88007fd0 tx=0x7feb88009040).stop 2023-11-28T04:07:39.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 msgr2=0x7feb9814ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:39.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 rx=0x7feb8c00d8f0 tx=0x7feb8c008040).stop 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 shutdown_connections 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feb8405f150 0x7feb84061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feb981544d0 0x7feb980a6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feb98064e20 0x7feb980c1830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 --2- 172.21.15.73:0/4291853581 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feb98095e80 0x7feb9814ce40 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 >> 172.21.15.73:0/4291853581 conn(0x7feb980631d0 msgr2=0x7feb980c86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 shutdown_connections 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.798+0000 7feb9ea17700 1 -- 172.21.15.73:0/4291853581 wait complete. 2023-11-28T04:07:39.800 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 54 2023-11-28T04:07:39.811 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T04:07:39.812 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T04:07:39.812 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T04:07:39.812 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:07:39.812 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T04:07:39.812 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T04:07:39.813 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T04:07:39.813 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:07:39.813 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T04:07:39.813 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T04:07:39.813 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T04:07:39.814 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:07:39.814 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T04:07:39.814 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T04:07:39.814 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T04:07:39.814 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:07:39.820 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T04:07:39.821 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T04:07:39.822 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T04:07:39.822 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T04:07:39.825 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T04:07:39.825 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T04:07:39.826 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T04:07:39.826 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T04:07:39.863 INFO:tasks.ceph.mds.a:Started 2023-11-28T04:07:39.864 DEBUG:teuthology.parallel:result is None 2023-11-28T04:07:39.865 INFO:tasks.ceph.mds.c:Started 2023-11-28T04:07:39.866 DEBUG:teuthology.parallel:result is None 2023-11-28T04:07:39.870 INFO:tasks.ceph.mds.d:Started 2023-11-28T04:07:39.870 DEBUG:teuthology.parallel:result is None 2023-11-28T04:07:39.870 INFO:tasks.ceph.mds.b:Started 2023-11-28T04:07:39.871 DEBUG:teuthology.parallel:result is None 2023-11-28T04:07:39.871 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T04:07:39.935 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:07:39.936 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T04:07:39.952 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T04:07:39.952 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T04:07:39.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.992+0000 7f1c1e095700 1 Processor -- start 2023-11-28T04:07:39.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.992+0000 7f1c1e095700 1 -- start start 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1e095700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c181510d0 0x7f1c180a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1e095700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c180c37c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1e095700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c18063b40 con 0x7f1c180bcea0 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1e095700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c180c2ce0 con 0x7f1c181510d0 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1e095700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c2fc0 con 0x7f1c181514b0 2023-11-28T04:07:40.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1d894700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1c180bcea0 0x7f1c180bd3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36724/0 (socket says 172.21.15.73:36724) 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.993+0000 7f1c1d894700 1 -- 172.21.15.73:0/1441903187 learned_addr learned my addr 172.21.15.73:0/1441903187 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1441903187 >> v2:172.21.15.150:3300/0 conn(0x7f1c181510d0 0x7f1c180a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1c892700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c180c37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2663779651 0 0) 0x7f1c18063b40 con 0x7f1c180bcea0 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c040033f0 con 0x7f1c180bcea0 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1104269402 0 0) 0x7f1c040033f0 con 0x7f1c180bcea0 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 msgr2=0x7f1c180c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c180c37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 >> v2:172.21.15.150:3300/0 conn(0x7f1c181510d0 msgr2=0x7f1c180a6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 --2- 172.21.15.73:0/1441903187 >> v2:172.21.15.150:3300/0 conn(0x7f1c181510d0 0x7f1c180a6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180c32a0 con 0x7f1c180bcea0 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f1c100017f0 con 0x7f1c180bcea0 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 >> v1:172.21.15.73:6789/0 conn(0x7f1c180bcea0 legacy=0x7f1c180bd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c040038e0 0x7f1c04003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c04004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c04004b80 0x7f1c04005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c2ce0 con 0x7f1c040038e0 2023-11-28T04:07:40.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180c2fc0 con 0x7f1c04004b80 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c0400b8e0 con 0x7f1c181514b0 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c040038e0 0x7f1c04003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1c892700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c04004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d894700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c04004b80 0x7f1c04005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 -- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 msgr2=0x7f1c04004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c181514b0 0x7f1c04004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 -- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c04004b80 msgr2=0x7f1c04005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c04004b80 0x7f1c04005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.994+0000 7f1c1d093700 1 -- 172.21.15.73:0/1441903187 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c040033f0 con 0x7f1c040038e0 2023-11-28T04:07:40.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c040038e0 0x7f1c04003d00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f1c14009a80 tx=0x7f1c14009060).ready entity=mon.0 client_cookie=c48dbb6c57e3cb52 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c14009430 con 0x7f1c040038e0 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1400e640 con 0x7f1c040038e0 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c0ffff700 1 -- 172.21.15.73:0/1441903187 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1400e9c0 con 0x7f1c040038e0 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 -- 172.21.15.73:0/1441903187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c040038e0 msgr2=0x7f1c04003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1441903187 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c040038e0 0x7f1c04003d00 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 rx=0x7f1c14009a80 tx=0x7f1c14009060).stop 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1d894700 1 -- 172.21.15.73:0/1441903187 reap_dead start 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 -- 172.21.15.73:0/1441903187 shutdown_connections 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 -- 172.21.15.73:0/1441903187 >> 172.21.15.73:0/1441903187 conn(0x7f1c180c4010 msgr2=0x7f1c180c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:40.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 -- 172.21.15.73:0/1441903187 shutdown_connections 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.995+0000 7f1c1e095700 1 -- 172.21.15.73:0/1441903187 wait complete. 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.996+0000 7f1c1e095700 1 Processor -- start 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.996+0000 7f1c1e095700 1 -- start start 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c040038e0 0x7f1c181522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c1814a120 0x7f1c180ac7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180a6760 con 0x7f1c180accc0 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c180ad120 con 0x7f1c040038e0 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1e095700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c181527d0 con 0x7f1c1814a120 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1d894700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.006 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1d894700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52484/0 (socket says 172.21.15.73:52484) 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1d894700 1 -- 172.21.15.73:0/1403484323 learned_addr learned my addr 172.21.15.73:0/1403484323 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c040038e0 0x7f1c181522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.997+0000 7f1c1c892700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c1814a120 0x7f1c180ac7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 -- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c1814a120 msgr2=0x7f1c180ac7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c1814a120 0x7f1c180ac7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 -- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c040038e0 msgr2=0x7f1c181522e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c040038e0 0x7f1c181522e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 -- 172.21.15.73:0/1403484323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c180a5b50 con 0x7f1c180accc0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1d894700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f1c10001350 tx=0x7f1c10001d90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000f8a0 con 0x7f1c180accc0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1000fa20 con 0x7f1c180accc0 2023-11-28T04:07:40.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1000fde0 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c180a5e30 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.998+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c180a63f0 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.999+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c180a6910 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.999+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1c10012560 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:39.999+0000 7f1c0d7fa700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 0x7f1bfc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:40.000+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 5699+0+0 (secure 0 0 0) 0x7f1c100798a0 con 0x7f1c180accc0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:40.000+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 0x7f1bfc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:40.000+0000 7f1c1d093700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 0x7f1bfc061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1c14006c40 tx=0x7f1c1401a000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:40.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:40.005+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c180a6910 con 0x7f1c180accc0 2023-11-28T04:07:40.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:40.184+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f1c180a6910 con 0x7f1c180accc0 2023-11-28T04:07:41.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.078+0000 7f1c0d7fa700 1 -- 172.21.15.73:0/1403484323 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v161) v1 ==== 97+0+0 (secure 0 0 0) 0x7f1c180a6910 con 0x7f1c180accc0 2023-11-28T04:07:41.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 msgr2=0x7f1bfc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 0x7f1bfc061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f1c14006c40 tx=0x7f1c1401a000).stop 2023-11-28T04:07:41.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 msgr2=0x7f1c18062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.081 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 rx=0x7f1c10001350 tx=0x7f1c10001d90).stop 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 shutdown_connections 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1bfc05f150 0x7f1bfc061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c180accc0 0x7f1c18062700 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c040038e0 0x7f1c181522e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 --2- 172.21.15.73:0/1403484323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c1814a120 0x7f1c180ac7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 >> 172.21.15.73:0/1403484323 conn(0x7f1c180c4010 msgr2=0x7f1c180c53a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 shutdown_connections 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.081+0000 7f1c1e095700 1 -- 172.21.15.73:0/1403484323 wait complete. 2023-11-28T04:07:41.082 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T04:07:41.094 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T04:07:41.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.221+0000 7f52abfff700 1 Processor -- start 2023-11-28T04:07:41.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.221+0000 7f52abfff700 1 -- start start 2023-11-28T04:07:41.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52abfff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52abfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52ac0c1750 0x7f52ac0c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52abfff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52ac0c12c0 con 0x7f52ac0c0180 2023-11-28T04:07:41.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52abfff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52ac0c06b0 con 0x7f52ac0c1750 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52abfff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0c0990 con 0x7f52ac157660 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52aaffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52ab7fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52ac0c1750 0x7f52ac0c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52aaffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:38520/0 (socket says 172.21.15.73:38520) 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52aa7fc700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f52ac0c0180 0x7f52ac1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:56988/0 (socket says 172.21.15.73:56988) 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52aaffd700 1 -- 172.21.15.73:0/335522445 learned_addr learned my addr 172.21.15.73:0/335522445 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52ab7fe700 1 -- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 msgr2=0x7f52ac157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52ab7fe700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52ab7fe700 1 -- 172.21.15.73:0/335522445 >> v1:172.21.15.150:6789/0 conn(0x7f52ac0c0180 legacy=0x7f52ac1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52a9ffb700 1 -- 172.21.15.73:0/335522445 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969274753 0 0) 0x7f52ac0c12c0 con 0x7f52ac0c0180 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52ab7fe700 1 -- 172.21.15.73:0/335522445 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52ac0c0c70 con 0x7f52ac0c1750 2023-11-28T04:07:41.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.222+0000 7f52aaffd700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52ab7fe700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c1750 0x7f52ac0c1bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f52a4009760 tx=0x7f52a4009060).ready entity=mon.1 client_cookie=cc75973546613688 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52a9ffb700 1 -- 172.21.15.73:0/335522445 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400f510 con 0x7f52ac0c1750 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52a9ffb700 1 -- 172.21.15.73:0/335522445 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a400f670 con 0x7f52ac0c1750 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52a9ffb700 1 -- 172.21.15.73:0/335522445 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400f9f0 con 0x7f52ac0c1750 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 -- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c1750 msgr2=0x7f52ac0c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c1750 0x7f52ac0c1bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 rx=0x7f52a4009760 tx=0x7f52a4009060).stop 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 -- 172.21.15.73:0/335522445 shutdown_connections 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c1750 0x7f52ac0c1bc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 --2- 172.21.15.73:0/335522445 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac157660 0x7f52ac157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 -- 172.21.15.73:0/335522445 >> 172.21.15.73:0/335522445 conn(0x7f52ac0cb000 msgr2=0x7f52ac0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 -- 172.21.15.73:0/335522445 shutdown_connections 2023-11-28T04:07:41.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.223+0000 7f52abfff700 1 -- 172.21.15.73:0/335522445 wait complete. 2023-11-28T04:07:41.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.224+0000 7f52abfff700 1 Processor -- start 2023-11-28T04:07:41.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.224+0000 7f52abfff700 1 -- start start 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c0180 0x7f52ac0ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac0c1750 0x7f52ac0c1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac0af330 con 0x7f52ac157660 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac164730 con 0x7f52ac0c0180 2023-11-28T04:07:41.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52abfff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52ac164a10 con 0x7f52ac0c1750 2023-11-28T04:07:41.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52ab7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52ab7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52500/0 (socket says 172.21.15.73:52500) 2023-11-28T04:07:41.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.225+0000 7f52ab7fe700 1 -- 172.21.15.73:0/1292435325 learned_addr learned my addr 172.21.15.73:0/1292435325 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:41.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52aa7fc700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac0c1750 0x7f52ac0c1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 -- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac0c1750 msgr2=0x7f52ac0c1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52aaffd700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c0180 0x7f52ac0ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac0c1750 0x7f52ac0c1b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 -- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c0180 msgr2=0x7f52ac0ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c0180 0x7f52ac0ad5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 -- 172.21.15.73:0/1292435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c009ae0 con 0x7f52ac157660 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52ab7fe700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f52ac084570 tx=0x7f52a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a400f9f0 con 0x7f52ac157660 2023-11-28T04:07:41.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52a401b410 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52a40092a0 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52a401b7d0 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.226+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52ac0aeec0 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.227+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52ac0af0e0 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.227+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f52a401bbc0 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.227+0000 7f5292ffd700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 0x7f5298061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.227+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f52a407ba30 con 0x7f52ac157660 2023-11-28T04:07:41.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.228+0000 7f52aaffd700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 0x7f5298061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.228+0000 7f52aaffd700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 0x7f5298061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f529c009ab0 tx=0x7f529c00b040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.232+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52a404d050 con 0x7f52ac157660 2023-11-28T04:07:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.458+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f52ac06c7c0 con 0x7f52ac157660 2023-11-28T04:07:41.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.458+0000 7f5292ffd700 1 -- 172.21.15.73:0/1292435325 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v37) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f52a4029020 con 0x7f52ac157660 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:07:41.459 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:07:41.460 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.461 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.462 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T04:07:41.463 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.464 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.465 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.466 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T04:07:41.467 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:07:41.468 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.469 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.470 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.471 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:07:41.472 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.473 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:41.474 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T04:07:41.475 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:07:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 msgr2=0x7f5298061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 0x7f5298061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f529c009ab0 tx=0x7f529c00b040).stop 2023-11-28T04:07:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 msgr2=0x7f52ac0c4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 rx=0x7f52ac084570 tx=0x7f52a4017000).stop 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 shutdown_connections 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f529805f150 0x7f5298061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52ac157660 0x7f52ac0c4220 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52ac0c0180 0x7f52ac0ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 --2- 172.21.15.73:0/1292435325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52ac0c1750 0x7f52ac0c1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 >> 172.21.15.73:0/1292435325 conn(0x7f52ac0cb000 msgr2=0x7f52ac0ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 shutdown_connections 2023-11-28T04:07:41.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.460+0000 7f52abfff700 1 -- 172.21.15.73:0/1292435325 wait complete. 2023-11-28T04:07:41.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:41.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.591+0000 7f44db59e700 1 Processor -- start 2023-11-28T04:07:41.591 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.591+0000 7f44db59e700 1 -- start start 2023-11-28T04:07:41.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44db59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44db59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44db59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44dc0c12c0 con 0x7f44dc157660 2023-11-28T04:07:41.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44db59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44dc0c06b0 con 0x7f44dc0c1750 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44db59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c0990 con 0x7f44dc0c0180 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44dad9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44dad9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:55484/0 (socket says 172.21.15.73:55484) 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:38540/0 (socket says 172.21.15.73:38540) 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44dad9d700 1 -- 172.21.15.73:0/1185568349 learned_addr learned my addr 172.21.15.73:0/1185568349 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44da59c700 1 --1- 172.21.15.73:0/1185568349 >> v1:172.21.15.150:6789/0 conn(0x7f44dc157660 0x7f44dc1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:57004/0 (socket says 172.21.15.73:57004) 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 -- 172.21.15.73:0/1185568349 >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 msgr2=0x7f44dc0c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:41.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 --2- 172.21.15.73:0/1185568349 >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 -- 172.21.15.73:0/1185568349 >> v1:172.21.15.150:6789/0 conn(0x7f44dc157660 legacy=0x7f44dc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d959a700 1 -- 172.21.15.73:0/1185568349 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 504373964 0 0) 0x7f44dc0c12c0 con 0x7f44dc157660 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44d9d9b700 1 -- 172.21.15.73:0/1185568349 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44dc0c0c70 con 0x7f44dc0c0180 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.592+0000 7f44dad9d700 1 --2- 172.21.15.73:0/1185568349 >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44d9d9b700 1 --2- 172.21.15.73:0/1185568349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f44d0009760 tx=0x7f44d0009060).ready entity=mon.2 client_cookie=948aa1b7e0d64b19 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44d959a700 1 -- 172.21.15.73:0/1185568349 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d00090d0 con 0x7f44dc0c0180 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44d959a700 1 -- 172.21.15.73:0/1185568349 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d000f510 con 0x7f44dc0c0180 2023-11-28T04:07:41.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44d959a700 1 -- 172.21.15.73:0/1185568349 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d000f890 con 0x7f44dc0c0180 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 -- 172.21.15.73:0/1185568349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 msgr2=0x7f44dc157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 --2- 172.21.15.73:0/1185568349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 rx=0x7f44d0009760 tx=0x7f44d0009060).stop 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 -- 172.21.15.73:0/1185568349 shutdown_connections 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 --2- 172.21.15.73:0/1185568349 >> v2:172.21.15.150:3300/0 conn(0x7f44dc0c1750 0x7f44dc0c1ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 --2- 172.21.15.73:0/1185568349 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c0180 0x7f44dc157a40 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 -- 172.21.15.73:0/1185568349 >> 172.21.15.73:0/1185568349 conn(0x7f44dc0cb000 msgr2=0x7f44dc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 -- 172.21.15.73:0/1185568349 shutdown_connections 2023-11-28T04:07:41.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.593+0000 7f44db59e700 1 -- 172.21.15.73:0/1185568349 wait complete. 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.594+0000 7f44db59e700 1 Processor -- start 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 -- start start 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44dc0c0180 0x7f44dc150d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 0x7f44dc0c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0c4d90 con 0x7f44dc157660 2023-11-28T04:07:41.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc1648a0 con 0x7f44dc0c0180 2023-11-28T04:07:41.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44db59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44dc0aec70 con 0x7f44dc0c1750 2023-11-28T04:07:41.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44dad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44dad9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52514/0 (socket says 172.21.15.73:52514) 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.595+0000 7f44dad9d700 1 -- 172.21.15.73:0/88350377 learned_addr learned my addr 172.21.15.73:0/88350377 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44da59c700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44dc0c0180 0x7f44dc150d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44d9d9b700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 0x7f44dc0c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 -- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 msgr2=0x7f44dc0c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 0x7f44dc0c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 -- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44dc0c0180 msgr2=0x7f44dc150d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44dc0c0180 0x7f44dc150d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 -- 172.21.15.73:0/88350377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d4009ae0 con 0x7f44dc157660 2023-11-28T04:07:41.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44d9d9b700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 0x7f44dc0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44dad9d700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f44d4009fd0 tx=0x7f44d400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d40092e0 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44d400f040 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d00092a0 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44d400cc80 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.596+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44dc0af230 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.597+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44dc164a30 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.597+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f44d4007400 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.598+0000 7f44ca7fc700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.598+0000 7f44da59c700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.598+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f44d4079e90 con 0x7f44dc157660 2023-11-28T04:07:41.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.598+0000 7f44da59c700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 0x7f44bc061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f44cc007fd0 tx=0x7f44cc007d70).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.602+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44d404aaf0 con 0x7f44dc157660 2023-11-28T04:07:41.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.763+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f44dc06c7c0 con 0x7f44dc157660 2023-11-28T04:07:41.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.764+0000 7f44ca7fc700 1 -- 172.21.15.73:0/88350377 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f44d404b500 con 0x7f44dc157660 2023-11-28T04:07:41.764 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:41.764 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":161,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:41.068828+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":18,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":142,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":138,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 msgr2=0x7f44bc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 0x7f44bc061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f44cc007fd0 tx=0x7f44cc007d70).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 msgr2=0x7f44dc0c48a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 rx=0x7f44d4009fd0 tx=0x7f44d400b040).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 shutdown_connections 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f44bc05f150 0x7f44bc061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44dc157660 0x7f44dc0c48a0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44dc0c0180 0x7f44dc150d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 --2- 172.21.15.73:0/88350377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44dc0c1750 0x7f44dc0c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 >> 172.21.15.73:0/88350377 conn(0x7f44dc0cb000 msgr2=0x7f44dc0c7990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 shutdown_connections 2023-11-28T04:07:41.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.765+0000 7f44db59e700 1 -- 172.21.15.73:0/88350377 wait complete. 2023-11-28T04:07:41.777 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T04:07:41.778 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:07:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.894+0000 7fb5ed4c9700 1 Processor -- start 2023-11-28T04:07:41.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.894+0000 7fb5ed4c9700 1 -- start start 2023-11-28T04:07:41.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5ed4c9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb5e80aa080 0x7fb5e80aa490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5ed4c9700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e80c0b20 con 0x7fb5e81551c0 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5ed4c9700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5e81548c0 con 0x7fb5e81555a0 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5ed4c9700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5e8155a00 con 0x7fb5e80aa080 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e7fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb5e81551c0 0x7fb5e80a7e50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36734/0 (socket says 172.21.15.73:36734) 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e7fff700 1 -- 172.21.15.73:0/1964889134 learned_addr learned my addr 172.21.15.73:0/1964889134 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2559901034 0 0) 0x7fb5e80c0b20 con 0x7fb5e81551c0 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5d00033f0 con 0x7fb5e81551c0 2023-11-28T04:07:41.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2777158567 0 0) 0x7fb5d00033f0 con 0x7fb5e81551c0 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 >> v2:172.21.15.150:3300/0 conn(0x7fb5e80aa080 msgr2=0x7fb5e80aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 --2- 172.21.15.73:0/1964889134 >> v2:172.21.15.150:3300/0 conn(0x7fb5e80aa080 0x7fb5e80aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 >> v1:172.21.15.150:6789/0 conn(0x7fb5e81555a0 legacy=0x7fb5e80c0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80c5aa0 con 0x7fb5e81551c0 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb5d8001470 con 0x7fb5e81551c0 2023-11-28T04:07:41.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 >> v1:172.21.15.73:6789/0 conn(0x7fb5e81551c0 legacy=0x7fb5e80a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5d00038e0 0x7fb5d0003ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5d0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5d00051b0 0x7fb5d000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e8155a00 con 0x7fb5d0004690 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d000c130 con 0x7fb5d00038e0 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.895+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5d000c2f0 con 0x7fb5d00051b0 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5d0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5eccc8700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5d00051b0 0x7fb5d000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 -- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5d00051b0 msgr2=0x7fb5d000b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5d00051b0 0x7fb5d000b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 -- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5d00038e0 msgr2=0x7fb5d0003ce0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5d00038e0 0x7fb5d0003ce0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 -- 172.21.15.73:0/1964889134 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5d00033f0 con 0x7fb5d0004690 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5eccc8700 1 -- 172.21.15.73:0/1964889134 reap_dead start 2023-11-28T04:07:41.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5d0004b10 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fb5dc001620 tx=0x7fb5dc007b90).ready entity=mon.0 client_cookie=6db6aa6c3005050 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc007ea0 con 0x7fb5d0004690 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5dc014410 con 0x7fb5d0004690 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5e6ffd700 1 -- 172.21.15.73:0/1964889134 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc0147b0 con 0x7fb5d0004690 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 msgr2=0x7fb5d0004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5d0004b10 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 rx=0x7fb5dc001620 tx=0x7fb5dc007b90).stop 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1964889134 shutdown_connections 2023-11-28T04:07:41.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.896+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1964889134 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5d0004b10 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.897+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1964889134 >> 172.21.15.73:0/1964889134 conn(0x7fb5e806a900 msgr2=0x7fb5e80cb100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.897+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1964889134 shutdown_connections 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.897+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1964889134 wait complete. 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.898+0000 7fb5ed4c9700 1 Processor -- start 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.898+0000 7fb5ed4c9700 1 -- start start 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5e80aa490 0x7fb5e8153b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5e8154000 0x7fb5e80c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80aac00 con 0x7fb5d0004690 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80accd0 con 0x7fb5e80aa490 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5e80ac090 con 0x7fb5e8154000 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52526/0 (socket says 172.21.15.73:52526) 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 -- 172.21.15.73:0/1374299103 learned_addr learned my addr 172.21.15.73:0/1374299103 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:41.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5eccc8700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5e8154000 0x7fb5e80c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 -- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5e8154000 msgr2=0x7fb5e80c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5e8154000 0x7fb5e80c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 -- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5e80aa490 msgr2=0x7fb5e8153b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5e80aa490 0x7fb5e8153b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 -- 172.21.15.73:0/1374299103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5e80ac330 con 0x7fb5d0004690 2023-11-28T04:07:41.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5e7fff700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fb5d8001620 tx=0x7fb5d8001aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d8007170 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5d80072f0 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5d80048c0 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5e80ac610 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.899+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5e80aa910 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.900+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5e806c7c0 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.900+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb5d8004c30 con 0x7fb5d0004690 2023-11-28T04:07:41.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.901+0000 7fb5cffff700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 0x7fb5d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:41.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.901+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb5d8079850 con 0x7fb5d0004690 2023-11-28T04:07:41.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.901+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 0x7fb5d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:41.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.901+0000 7fb5e77fe700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 0x7fb5d4061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb5e8168420 tx=0x7fb5dc015000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:41.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:41.905+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb5d804ae70 con 0x7fb5d0004690 2023-11-28T04:07:42.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.069+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb5e806c100 con 0x7fb5d0004690 2023-11-28T04:07:42.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.069+0000 7fb5cffff700 1 -- 172.21.15.73:0/1374299103 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v161) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb5d804e490 con 0x7fb5d0004690 2023-11-28T04:07:42.070 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 msgr2=0x7fb5d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 0x7fb5d4061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fb5e8168420 tx=0x7fb5dc015000).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 msgr2=0x7fb5e80c37e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 rx=0x7fb5d8001620 tx=0x7fb5d8001aa0).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 shutdown_connections 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5d405f150 0x7fb5d4061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5e8154000 0x7fb5e80c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5d0004690 0x7fb5e80c37e0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 --2- 172.21.15.73:0/1374299103 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5e80aa490 0x7fb5e8153b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 >> 172.21.15.73:0/1374299103 conn(0x7fb5e806a900 msgr2=0x7fb5e80ae5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 shutdown_connections 2023-11-28T04:07:42.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.071+0000 7fb5ed4c9700 1 -- 172.21.15.73:0/1374299103 wait complete. 2023-11-28T04:07:42.086 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:07:42.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.201+0000 7fb604a3a700 1 Processor -- start 2023-11-28T04:07:42.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.201+0000 7fb604a3a700 1 -- start start 2023-11-28T04:07:42.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb604a3a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb600158a30 0x7fb6000c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb604a3a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6000c72d0 con 0x7fb6000c6190 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb604a3a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb600064570 con 0x7fb600098d10 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb604a3a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb600154bc0 con 0x7fb600158a30 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb5f5d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb6000c6190 0x7fb600158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36744/0 (socket says 172.21.15.73:36744) 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/4072960444 learned_addr learned my addr 172.21.15.73:0/4072960444 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb5fed9d700 1 --2- 172.21.15.73:0/4072960444 >> v2:172.21.15.150:3300/0 conn(0x7fb600158a30 0x7fb6000c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598555444 0 0) 0x7fb6000c72d0 con 0x7fb6000c6190 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.202+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb5ec0033f0 con 0x7fb6000c6190 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1373080358 0 0) 0x7fb5ec0033f0 con 0x7fb6000c6190 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 >> v2:172.21.15.150:3300/0 conn(0x7fb600158a30 msgr2=0x7fb6000c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 --2- 172.21.15.73:0/4072960444 >> v2:172.21.15.150:3300/0 conn(0x7fb600158a30 0x7fb6000c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 >> v1:172.21.15.150:6789/0 conn(0x7fb600098d10 legacy=0x7fb600063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000c66c0 con 0x7fb6000c6190 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb5e8001470 con 0x7fb6000c6190 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 >> v1:172.21.15.73:6789/0 conn(0x7fb6000c6190 legacy=0x7fb600158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5ec0038e0 0x7fb5ec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 0x7fb5ec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec005000 0x7fb5ec00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb600154bc0 con 0x7fb5ec004690 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec00beb0 con 0x7fb5ec005000 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5ec00c070 con 0x7fb5ec0038e0 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 0x7fb5ec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fed9d700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec005000 0x7fb5ec00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fe59c700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5ec0038e0 0x7fb5ec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5ec0038e0 msgr2=0x7fb5ec003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb5ec0038e0 0x7fb5ec003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec005000 msgr2=0x7fb5ec00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec005000 0x7fb5ec00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/4072960444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5ec0033f0 con 0x7fb5ec004690 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fed9d700 1 -- 172.21.15.73:0/4072960444 reap_dead start 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 0x7fb5ec004b10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fb6000c65f0 tx=0x7fb6000a8040).ready entity=mon.0 client_cookie=c2cad0031715022a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800ca20 con 0x7fb5ec004690 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e800cb80 con 0x7fb5ec004690 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.203+0000 7fb5fdd9b700 1 -- 172.21.15.73:0/4072960444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e80070d0 con 0x7fb5ec004690 2023-11-28T04:07:42.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 -- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 msgr2=0x7fb5ec004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 0x7fb5ec004b10 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 rx=0x7fb6000c65f0 tx=0x7fb6000a8040).stop 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 -- 172.21.15.73:0/4072960444 shutdown_connections 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 --2- 172.21.15.73:0/4072960444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb5ec004690 0x7fb5ec004b10 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 -- 172.21.15.73:0/4072960444 >> 172.21.15.73:0/4072960444 conn(0x7fb6000cb310 msgr2=0x7fb6000cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 -- 172.21.15.73:0/4072960444 shutdown_connections 2023-11-28T04:07:42.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.204+0000 7fb604a3a700 1 -- 172.21.15.73:0/4072960444 wait complete. 2023-11-28T04:07:42.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.205+0000 7fb604a3a700 1 Processor -- start 2023-11-28T04:07:42.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.205+0000 7fb604a3a700 1 -- start start 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec004690 0x7fb600156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb600153980 0x7fb600153e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000c3940 con 0x7fb60014ac80 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6001542f0 con 0x7fb5ec004690 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb604a3a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6000c2d80 con 0x7fb600153980 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52552/0 (socket says 172.21.15.73:52552) 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/1044880575 learned_addr learned my addr 172.21.15.73:0/1044880575 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb600153980 msgr2=0x7fb600153e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb600153980 0x7fb600153e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec004690 msgr2=0x7fb600156fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec004690 0x7fb600156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 -- 172.21.15.73:0/1044880575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000c3010 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.206+0000 7fb5f5d9b700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fb5e800c720 tx=0x7fb5e8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.207+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e800cdc0 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.207+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5e80046b0 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.207+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6000c32f0 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.207+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5e8004a70 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.207+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60014b100 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.208+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6000c3790 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.208+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb5e801c020 con 0x7fb60014ac80 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.208+0000 7fb5f6ffd700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 0x7fb5e00615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.208+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fb5e807a390 con 0x7fb60014ac80 2023-11-28T04:07:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.208+0000 7fb5fe59c700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 0x7fb5e00615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.209+0000 7fb5fe59c700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 0x7fb5e00615e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb5f0007fd0 tx=0x7fb5f0009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.212+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb5e804b9b0 con 0x7fb60014ac80 2023-11-28T04:07:42.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.372+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fb60006c860 con 0x7fb60014ac80 2023-11-28T04:07:42.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.373+0000 7fb5f6ffd700 1 -- 172.21.15.73:0/1044880575 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v161) v1 ==== 92+0+10 (secure 0 0 0) 0x7fb5e8024070 con 0x7fb60014ac80 2023-11-28T04:07:42.373 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:07:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 msgr2=0x7fb5e00615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 0x7fb5e00615e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7fb5f0007fd0 tx=0x7fb5f0009040).stop 2023-11-28T04:07:42.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 msgr2=0x7fb6000caa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 rx=0x7fb5e800c720 tx=0x7fb5e8015000).stop 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 shutdown_connections 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb5e005f120 0x7fb5e00615e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb600153980 0x7fb600153e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb5ec004690 0x7fb600156fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 --2- 172.21.15.73:0/1044880575 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb60014ac80 0x7fb6000caa90 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 >> 172.21.15.73:0/1044880575 conn(0x7fb6000cb310 msgr2=0x7fb600152320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 shutdown_connections 2023-11-28T04:07:42.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.375+0000 7fb604a3a700 1 -- 172.21.15.73:0/1044880575 wait complete. 2023-11-28T04:07:42.387 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T04:07:42.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 Processor -- start 2023-11-28T04:07:42.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 -- start start 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 0x7fa79c0697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa79c065760 con 0x7fa79c069ce0 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.501+0000 7fa7a2b2e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa79c06a160 con 0x7fa79c062e10 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a2b2e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0c3250 con 0x7fa79c1542d0 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a10cb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 0x7fa79c0697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a10cb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:55556/0 (socket says 172.21.15.73:55556) 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a10cb700 1 -- 172.21.15.73:0/3373905280 learned_addr learned my addr 172.21.15.73:0/3373905280 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa79bfff700 1 --1- 172.21.15.73:0/3373905280 >> v1:172.21.15.150:6789/0 conn(0x7fa79c069ce0 0x7fa79c062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:57046/0 (socket says 172.21.15.73:57046) 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3373905280 >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 msgr2=0x7fa79c0661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 --2- 172.21.15.73:0/3373905280 >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3373905280 >> v1:172.21.15.150:6789/0 conn(0x7fa79c069ce0 legacy=0x7fa79c062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3373905280 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa79c06a340 con 0x7fa79c1542d0 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a10cb700 1 --2- 172.21.15.73:0/3373905280 >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa7a08ca700 1 --2- 172.21.15.73:0/3373905280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 0x7fa79c0697f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa78c009e60 tx=0x7fa78c009780).ready entity=mon.2 client_cookie=66865c25e3ff84bf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.502+0000 7fa79b7fe700 1 -- 172.21.15.73:0/3373905280 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c00e070 con 0x7fa79c1542d0 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa79b7fe700 1 -- 172.21.15.73:0/3373905280 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c00bb60 con 0x7fa79c1542d0 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa79b7fe700 1 -- 172.21.15.73:0/3373905280 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c012410 con 0x7fa79c1542d0 2023-11-28T04:07:42.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3373905280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 msgr2=0x7fa79c0697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3373905280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 0x7fa79c0697f0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 rx=0x7fa78c009e60 tx=0x7fa78c009780).stop 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3373905280 shutdown_connections 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3373905280 >> v2:172.21.15.150:3300/0 conn(0x7fa79c062e10 0x7fa79c0661b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3373905280 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c1542d0 0x7fa79c0697f0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3373905280 >> 172.21.15.73:0/3373905280 conn(0x7fa79c0c7fd0 msgr2=0x7fa79c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3373905280 shutdown_connections 2023-11-28T04:07:42.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.503+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3373905280 wait complete. 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.504+0000 7fa7a2b2e700 1 Processor -- start 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.504+0000 7fa7a2b2e700 1 -- start start 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 0x7fa79c0abd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79c1542d0 0x7fa79c0ac1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0ac6e0 con 0x7fa79c062e10 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0a9ea0 con 0x7fa79c1542d0 2023-11-28T04:07:42.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a2b2e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa79c0aa180 con 0x7fa79c069ce0 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a08ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a08ca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52562/0 (socket says 172.21.15.73:52562) 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.505+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3995793931 learned_addr learned my addr 172.21.15.73:0/3995793931 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa79bfff700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 0x7fa79c0abd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 msgr2=0x7fa79c0abd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 0x7fa79c0abd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79c1542d0 msgr2=0x7fa79c0ac1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79c1542d0 0x7fa79c0ac1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 -- 172.21.15.73:0/3995793931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa794009370 con 0x7fa79c062e10 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa79bfff700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 0x7fa79c0abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a08ca700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fa78c00b6c0 tx=0x7fa78c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c01d630 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78c01d790 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78c009a20 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78c01db50 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.506+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa79c0aa740 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.507+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa79c1691a0 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.507+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa78c003040 con 0x7fa79c062e10 2023-11-28T04:07:42.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.508+0000 7fa798ff9700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 0x7fa788061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.508+0000 7fa79bfff700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 0x7fa788061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.508+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fa78c07bb30 con 0x7fa79c062e10 2023-11-28T04:07:42.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.508+0000 7fa79bfff700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 0x7fa788061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa79c153d80 tx=0x7fa790009040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.512+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa78c04d150 con 0x7fa79c062e10 2023-11-28T04:07:42.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:07:42.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:07:42.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.683+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fa79c06a620 con 0x7fa79c062e10 2023-11-28T04:07:42.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.683+0000 7fa798ff9700 1 -- 172.21.15.73:0/3995793931 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v161) v1 ==== 84+0+143 (secure 0 0 0) 0x7fa78c050770 con 0x7fa79c062e10 2023-11-28T04:07:42.683 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:42.684 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:07:42.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 msgr2=0x7fa788061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 0x7fa788061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fa79c153d80 tx=0x7fa790009040).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 msgr2=0x7fa79c14ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 rx=0x7fa78c00b6c0 tx=0x7fa78c009780).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 shutdown_connections 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa78805f150 0x7fa788061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa79c1542d0 0x7fa79c0ac1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa79c062e10 0x7fa79c14ebc0 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 --2- 172.21.15.73:0/3995793931 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa79c069ce0 0x7fa79c0abd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 >> 172.21.15.73:0/3995793931 conn(0x7fa79c0c7fd0 msgr2=0x7fa79c0be7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 shutdown_connections 2023-11-28T04:07:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.685+0000 7fa7a2b2e700 1 -- 172.21.15.73:0/3995793931 wait complete. 2023-11-28T04:07:42.697 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T04:07:42.697 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 Processor -- start 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 -- start start 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f41400be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41400647c0 con 0x7f41400bed70 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41401559b0 con 0x7f41400c6530 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.815+0000 7f4144f5a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4140065680 con 0x7f41400c57e0 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f41400bed70 0x7f41400c35d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36746/0 (socket says 172.21.15.73:36746) 2023-11-28T04:07:42.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413ffff700 1 -- 172.21.15.73:0/4210302383 learned_addr learned my addr 172.21.15.73:0/4210302383 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f4136ffd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f41400be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3741829884 0 0) 0x7f4138001490 con 0x7f41400bed70 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f412c0033f0 con 0x7f41400bed70 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1780201899 0 0) 0x7f412c0033f0 con 0x7f41400bed70 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 msgr2=0x7f41400be880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f41400be880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 >> v1:172.21.15.150:6789/0 conn(0x7f41400c6530 legacy=0x7f41400640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400bf1f0 con 0x7f41400bed70 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f4138001980 con 0x7f41400bed70 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 >> v1:172.21.15.73:6789/0 conn(0x7f41400bed70 legacy=0x7f41400c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f412c003910 0x7f412c003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f412c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c004b80 0x7f412c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4140065680 con 0x7f412c004b80 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f412c00b8e0 con 0x7f412c003910 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f412c00baa0 con 0x7f41400c57e0 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413ffff700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c004b80 0x7f412c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f4136ffd700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f412c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.816+0000 7f413f7fe700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f412c003910 0x7f412c003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 -- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 msgr2=0x7f412c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41400c57e0 0x7f412c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 -- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f412c003910 msgr2=0x7f412c003cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f412c003910 0x7f412c003cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 -- 172.21.15.73:0/4210302383 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f412c0033f0 con 0x7f412c004b80 2023-11-28T04:07:42.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c004b80 0x7f412c005000 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f413800c5f0 tx=0x7f4138007bd0).ready entity=mon.0 client_cookie=60e7eff6b2d82c66 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4138007e00 con 0x7f412c004b80 2023-11-28T04:07:42.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4138005b20 con 0x7f412c004b80 2023-11-28T04:07:42.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413effd700 1 -- 172.21.15.73:0/4210302383 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41380070d0 con 0x7f412c004b80 2023-11-28T04:07:42.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f4144f5a700 1 -- 172.21.15.73:0/4210302383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c004b80 msgr2=0x7f412c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f4144f5a700 1 --2- 172.21.15.73:0/4210302383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f412c004b80 0x7f412c005000 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 rx=0x7f413800c5f0 tx=0x7f4138007bd0).stop 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f413ffff700 1 -- 172.21.15.73:0/4210302383 reap_dead start 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f4144f5a700 1 -- 172.21.15.73:0/4210302383 shutdown_connections 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.817+0000 7f4144f5a700 1 -- 172.21.15.73:0/4210302383 >> 172.21.15.73:0/4210302383 conn(0x7f4140092520 msgr2=0x7f4140062ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.818+0000 7f4144f5a700 1 -- 172.21.15.73:0/4210302383 shutdown_connections 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.818+0000 7f4144f5a700 1 -- 172.21.15.73:0/4210302383 wait complete. 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.819+0000 7f4144f5a700 1 Processor -- start 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.819+0000 7f4144f5a700 1 -- start start 2023-11-28T04:07:42.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f414014bf10 0x7f414014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41400bfdd0 0x7f41400c0250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400c2d80 con 0x7f4140168aa0 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41400adab0 con 0x7f41400bfdd0 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41400ad1c0 con 0x7f414014bf10 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52584/0 (socket says 172.21.15.73:52584) 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 -- 172.21.15.73:0/2960124156 learned_addr learned my addr 172.21.15.73:0/2960124156 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f413f7fe700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f414014bf10 0x7f414014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f413ffff700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41400bfdd0 0x7f41400c0250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 -- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f414014bf10 msgr2=0x7f414014c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f414014bf10 0x7f414014c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 -- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41400bfdd0 msgr2=0x7f41400c0250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41400bfdd0 0x7f41400c0250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 -- 172.21.15.73:0/2960124156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41400c2aa0 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4136ffd700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f4128006940 tx=0x7f4128009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800f940 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f412800faa0 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f412800fe20 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41400ad460 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.820+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41400c2740 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.821+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41400adc60 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.821+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4128014650 con 0x7f4140168aa0 2023-11-28T04:07:42.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.822+0000 7f4137fff700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 0x7f4120061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:42.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.822+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f412807aa70 con 0x7f4140168aa0 2023-11-28T04:07:42.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.822+0000 7f413f7fe700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 0x7f4120061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:42.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.822+0000 7f413f7fe700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 0x7f4120061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4130007cf0 tx=0x7f4130007be0).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:42.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.826+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41400adc60 con 0x7f4140168aa0 2023-11-28T04:07:42.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:42.988+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f41400adc20 con 0x7f4140168aa0 2023-11-28T04:07:43.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.086+0000 7f4137fff700 1 -- 172.21.15.73:0/2960124156 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v162) v1 ==== 152+0+0 (secure 0 0 0) 0x7f4128018070 con 0x7f4140168aa0 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 msgr2=0x7f4120061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 0x7f4120061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f4130007cf0 tx=0x7f4130007be0).stop 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 msgr2=0x7f41400c2250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 rx=0x7f4128006940 tx=0x7f4128009060).stop 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 shutdown_connections 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f412005f150 0x7f4120061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41400bfdd0 0x7f41400c0250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f414014bf10 0x7f414014c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 --2- 172.21.15.73:0/2960124156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4140168aa0 0x7f41400c2250 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 >> 172.21.15.73:0/2960124156 conn(0x7f4140092520 msgr2=0x7f41400a5ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:43.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.089+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 shutdown_connections 2023-11-28T04:07:43.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.091+0000 7f4144f5a700 1 -- 172.21.15.73:0/2960124156 wait complete. 2023-11-28T04:07:43.099 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T04:07:43.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T04:07:43.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.226+0000 7f522da0b700 1 Processor -- start 2023-11-28T04:07:43.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.226+0000 7f522da0b700 1 -- start start 2023-11-28T04:07:43.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f522da0b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5228098d10 0x7f52280c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f522da0b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5228158320 0x7f52281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f522da0b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5228158cc0 con 0x7f52280c6190 2023-11-28T04:07:43.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f522da0b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52280c72d0 con 0x7f5228098d10 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f522da0b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5228064570 con 0x7f5228158320 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5228098d10 0x7f52280c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f52267fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f52280c6190 0x7f5228063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36760/0 (socket says 172.21.15.73:36760) 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f52267fc700 1 -- 172.21.15.73:0/2090625676 learned_addr learned my addr 172.21.15.73:0/2090625676 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f52277fe700 1 --2- 172.21.15.73:0/2090625676 >> v2:172.21.15.150:3300/0 conn(0x7f5228158320 0x7f52281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 -- 172.21.15.73:0/2090625676 >> v2:172.21.15.150:3300/0 conn(0x7f5228158320 msgr2=0x7f52281587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 --2- 172.21.15.73:0/2090625676 >> v2:172.21.15.150:3300/0 conn(0x7f5228158320 0x7f52281587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 -- 172.21.15.73:0/2090625676 >> v1:172.21.15.73:6789/0 conn(0x7f52280c6190 legacy=0x7f5228063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5225ffb700 1 -- 172.21.15.73:0/2090625676 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4208705805 0 0) 0x7f521c001110 con 0x7f52280c6190 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 -- 172.21.15.73:0/2090625676 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5228154bc0 con 0x7f5228098d10 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5226ffd700 1 --2- 172.21.15.73:0/2090625676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228098d10 0x7f52280c7760 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f52280657a0 tx=0x7f5218009060).ready entity=mon.0 client_cookie=3f82bee9437d7071 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5225ffb700 1 -- 172.21.15.73:0/2090625676 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800b580 con 0x7f5228098d10 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5225ffb700 1 -- 172.21.15.73:0/2090625676 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f521800b6e0 con 0x7f5228098d10 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.227+0000 7f5225ffb700 1 -- 172.21.15.73:0/2090625676 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f521800ba60 con 0x7f5228098d10 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 -- 172.21.15.73:0/2090625676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228098d10 msgr2=0x7f52280c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 --2- 172.21.15.73:0/2090625676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228098d10 0x7f52280c7760 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 rx=0x7f52280657a0 tx=0x7f5218009060).stop 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 -- 172.21.15.73:0/2090625676 shutdown_connections 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 --2- 172.21.15.73:0/2090625676 >> v2:172.21.15.150:3300/0 conn(0x7f5228158320 0x7f52281587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 --2- 172.21.15.73:0/2090625676 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228098d10 0x7f52280c7760 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 -- 172.21.15.73:0/2090625676 >> 172.21.15.73:0/2090625676 conn(0x7f52280cb310 msgr2=0x7f52280cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 -- 172.21.15.73:0/2090625676 shutdown_connections 2023-11-28T04:07:43.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.228+0000 7f522da0b700 1 -- 172.21.15.73:0/2090625676 wait complete. 2023-11-28T04:07:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.229+0000 7f522da0b700 1 Processor -- start 2023-11-28T04:07:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.229+0000 7f522da0b700 1 -- start start 2023-11-28T04:07:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5228098d10 0x7f52280ab1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 0x7f52280ad5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52280aa530 con 0x7f5228158320 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f522814b7d0 con 0x7f5228098d10 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f522da0b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52280a9640 con 0x7f52280c6190 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:52600/0 (socket says 172.21.15.73:52600) 2023-11-28T04:07:43.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 -- 172.21.15.73:0/2130257614 learned_addr learned my addr 172.21.15.73:0/2130257614 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f5226ffd700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5228098d10 0x7f52280ab1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52267fc700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 0x7f52280ad5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 -- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 msgr2=0x7f52280ad5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 0x7f52280ad5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 -- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5228098d10 msgr2=0x7f52280ab1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5228098d10 0x7f52280ab1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 -- 172.21.15.73:0/2130257614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52180092a0 con 0x7f5228158320 2023-11-28T04:07:43.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52267fc700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 0x7f52280ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.230+0000 7f52277fe700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f5220007f20 tx=0x7f5220009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.231+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f522000ebf0 con 0x7f5228158320 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.231+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f522000ed50 con 0x7f5228158320 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.231+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52280a9920 con 0x7f5228158320 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.231+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5220014610 con 0x7f5228158320 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.231+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52280a9ee0 con 0x7f5228158320 2023-11-28T04:07:43.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.232+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52280aa6a0 con 0x7f5228158320 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.232+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5220014980 con 0x7f5228158320 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.233+0000 7f520effd700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 0x7f52140615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.233+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f522007a420 con 0x7f5228158320 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.233+0000 7f5226ffd700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 0x7f52140615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.233+0000 7f5226ffd700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 0x7f52140615e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f521800b340 tx=0x7f5218017000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:43.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.237+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f522004b8d0 con 0x7f5228158320 2023-11-28T04:07:43.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:43.399+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f522806c860 con 0x7f5228158320 2023-11-28T04:07:44.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.092+0000 7f520effd700 1 -- 172.21.15.73:0/2130257614 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v163) v1 ==== 170+0+0 (secure 0 0 0) 0x7f522004eef0 con 0x7f5228158320 2023-11-28T04:07:44.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.096+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 msgr2=0x7f52140615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.096+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 0x7f52140615e0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f521800b340 tx=0x7f5218017000).stop 2023-11-28T04:07:44.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.096+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 msgr2=0x7f5228153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.096+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 rx=0x7f5220007f20 tx=0x7f5220009860).stop 2023-11-28T04:07:44.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 shutdown_connections 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f521405f120 0x7f52140615e0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5228158320 0x7f5228153a60 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5228098d10 0x7f52280ab1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 --2- 172.21.15.73:0/2130257614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52280c6190 0x7f52280ad5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 >> 172.21.15.73:0/2130257614 conn(0x7f52280cb310 msgr2=0x7f522814c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 shutdown_connections 2023-11-28T04:07:44.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.099+0000 7f522da0b700 1 -- 172.21.15.73:0/2130257614 wait complete. 2023-11-28T04:07:44.108 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T04:07:44.124 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T04:07:44.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.239+0000 7f9cd8226700 1 Processor -- start 2023-11-28T04:07:44.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.239+0000 7f9cd8226700 1 -- start start 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd8226700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9cd00c6190 0x7f9cd00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd8226700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd00c72d0 con 0x7f9cd0098d10 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd8226700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd0064570 con 0x7f9cd0158320 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd8226700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cd0154bc0 con 0x7f9cd00c6190 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd57c1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9cd00c6190 0x7f9cd00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd5fc2700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9cd0098d10 0x7f9cd0063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55872/0 (socket says 172.21.15.73:55872) 2023-11-28T04:07:44.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.240+0000 7f9cd5fc2700 1 -- 172.21.15.73:0/1857074967 learned_addr learned my addr 172.21.15.73:0/1857074967 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd57c1700 1 -- 172.21.15.73:0/1857074967 >> v1:172.21.15.150:6789/0 conn(0x7f9cd0158320 legacy=0x7f9cd00a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd57c1700 1 -- 172.21.15.73:0/1857074967 >> v1:172.21.15.73:6789/0 conn(0x7f9cd0098d10 legacy=0x7f9cd0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd57c1700 1 -- 172.21.15.73:0/1857074967 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd00c66c0 con 0x7f9cd00c6190 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd4fc0700 1 -- 172.21.15.73:0/1857074967 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 761690916 0 0) 0x7f9cd00c72d0 con 0x7f9cd0098d10 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd57c1700 1 --2- 172.21.15.73:0/1857074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00c7760 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f9cc0009760 tx=0x7f9cc0009060).ready entity=mon.0 client_cookie=60c7684f04063f5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd4fc0700 1 -- 172.21.15.73:0/1857074967 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000b520 con 0x7f9cd00c6190 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd4fc0700 1 -- 172.21.15.73:0/1857074967 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000b680 con 0x7f9cd00c6190 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd4fc0700 1 -- 172.21.15.73:0/1857074967 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000ba00 con 0x7f9cd00c6190 2023-11-28T04:07:44.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 -- 172.21.15.73:0/1857074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 msgr2=0x7f9cd00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 --2- 172.21.15.73:0/1857074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00c7760 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 rx=0x7f9cc0009760 tx=0x7f9cc0009060).stop 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 -- 172.21.15.73:0/1857074967 shutdown_connections 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 --2- 172.21.15.73:0/1857074967 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00c7760 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 -- 172.21.15.73:0/1857074967 >> 172.21.15.73:0/1857074967 conn(0x7f9cd00cb310 msgr2=0x7f9cd00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.241+0000 7f9cd8226700 1 -- 172.21.15.73:0/1857074967 shutdown_connections 2023-11-28T04:07:44.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.242+0000 7f9cd8226700 1 -- 172.21.15.73:0/1857074967 wait complete. 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 Processor -- start 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 -- start start 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cd0098d10 0x7f9cd01468d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd0158320 0x7f9cd0152290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd01559a0 con 0x7f9cd00c6190 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0155bf0 con 0x7f9cd0158320 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd8226700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd0155ed0 con 0x7f9cd0098d10 2023-11-28T04:07:44.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd57c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd57c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58494/0 (socket says 172.21.15.73:58494) 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.243+0000 7f9cd57c1700 1 -- 172.21.15.73:0/529856738 learned_addr learned my addr 172.21.15.73:0/529856738 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd5fc2700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cd0098d10 0x7f9cd01468d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd67c3700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd0158320 0x7f9cd0152290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 -- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cd0098d10 msgr2=0x7f9cd01468d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cd0098d10 0x7f9cd01468d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 -- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd0158320 msgr2=0x7f9cd0152290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd0158320 0x7f9cd0152290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 -- 172.21.15.73:0/529856738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc00092a0 con 0x7f9cd00c6190 2023-11-28T04:07:44.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd57c1700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f9cd00c2360 tx=0x7f9cc0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc000b7f0 con 0x7f9cd00c6190 2023-11-28T04:07:44.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000b950 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd01561b0 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc001b620 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.244+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd0156770 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.245+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd0156990 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.246+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9cc001b9d0 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.246+0000 7f9cc5ffb700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 0x7f9cb40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.246+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f9cc007beb0 con 0x7f9cd00c6190 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.246+0000 7f9cd5fc2700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 0x7f9cb40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.246+0000 7f9cd5fc2700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 0x7f9cb40615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9ccc001560 tx=0x7f9ccc007f30).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.250+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9cc004d1e0 con 0x7f9cd00c6190 2023-11-28T04:07:44.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.457+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9cd006c860 con 0x7f9cd00c6190 2023-11-28T04:07:44.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.464+0000 7f9cc5ffb700 1 -- 172.21.15.73:0/529856738 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 19 and data pool 20 v56) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9cc0050800 con 0x7f9cd00c6190 2023-11-28T04:07:44.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.473+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 msgr2=0x7f9cb40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.473+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 0x7f9cb40615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f9ccc001560 tx=0x7f9ccc007f30).stop 2023-11-28T04:07:44.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.473+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 msgr2=0x7f9cd00aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.473+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 rx=0x7f9cd00c2360 tx=0x7f9cc0017000).stop 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 shutdown_connections 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9cb405f120 0x7f9cb40615e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9cd0158320 0x7f9cd0152290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9cd0098d10 0x7f9cd01468d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 --2- 172.21.15.73:0/529856738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9cd00c6190 0x7f9cd00aec90 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.474+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 >> 172.21.15.73:0/529856738 conn(0x7f9cd00cb310 msgr2=0x7f9cd00ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.475+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 shutdown_connections 2023-11-28T04:07:44.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.476+0000 7f9cd8226700 1 -- 172.21.15.73:0/529856738 wait complete. 2023-11-28T04:07:44.477 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 19 and data pool 20 2023-11-28T04:07:44.495 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.607+0000 7f0ffe45d700 1 Processor -- start 2023-11-28T04:07:44.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.607+0000 7f0ffe45d700 1 -- start start 2023-11-28T04:07:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffe45d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80ad6a0 0x7f0ff8155550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffe45d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff80ada80 0x7f0ff80c8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffe45d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ff8158ee0 con 0x7f0ff80c6710 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffe45d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ff8155a40 con 0x7f0ff80ada80 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffe45d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8158080 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffc9fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff80ada80 0x7f0ff80c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffc9fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0ff80ada80 0x7f0ff80c8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58498/0 (socket says 172.21.15.73:58498) 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.608+0000 7f0ffc9fa700 1 -- 172.21.15.73:0/1857256844 learned_addr learned my addr 172.21.15.73:0/1857256844 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffc9fa700 1 -- 172.21.15.73:0/1857256844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80ad6a0 msgr2=0x7f0ff8155550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffc9fa700 1 --2- 172.21.15.73:0/1857256844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80ad6a0 0x7f0ff8155550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffc9fa700 1 -- 172.21.15.73:0/1857256844 >> v1:172.21.15.150:6789/0 conn(0x7f0ff80c6710 legacy=0x7f0ff80c7980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffc9fa700 1 -- 172.21.15.73:0/1857256844 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff81582d0 con 0x7f0ff80ada80 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffc9fa700 1 --2- 172.21.15.73:0/1857256844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ada80 0x7f0ff80c8090 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f0ff0009f20 tx=0x7f0ff0009840).ready entity=mon.0 client_cookie=950ce259ad8fe073 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/1857256844 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000e070 con 0x7f0ff80ada80 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/1857256844 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ff000ba40 con 0x7f0ff80ada80 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ff6ffd700 1 -- 172.21.15.73:0/1857256844 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ff000bdc0 con 0x7f0ff80ada80 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1857256844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ada80 msgr2=0x7f0ff80c8090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1857256844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ada80 0x7f0ff80c8090 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 rx=0x7f0ff0009f20 tx=0x7f0ff0009840).stop 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1857256844 shutdown_connections 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1857256844 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ada80 0x7f0ff80c8090 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1857256844 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80ad6a0 0x7f0ff8155550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1857256844 >> 172.21.15.73:0/1857256844 conn(0x7f0ff80649c0 msgr2=0x7f0ff8064dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1857256844 shutdown_connections 2023-11-28T04:07:44.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.609+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1857256844 wait complete. 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 Processor -- start 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 -- start start 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 0x7f0ff81521e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80c6710 0x7f0ff81526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff814c540 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff8152bc0 con 0x7f0ff80ada80 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.611+0000 7f0ffe45d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff815ac50 con 0x7f0ff80c6710 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58502/0 (socket says 172.21.15.73:58502) 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 -- 172.21.15.73:0/1450562364 learned_addr learned my addr 172.21.15.73:0/1450562364 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff77fe700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 0x7f0ff81521e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ffc9fa700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80c6710 0x7f0ff81526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 -- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80c6710 msgr2=0x7f0ff81526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80c6710 0x7f0ff81526d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 -- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 msgr2=0x7f0ff81521e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 0x7f0ff81521e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 -- 172.21.15.73:0/1450562364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff0009ae0 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff77fe700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 0x7f0ff81521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ff7fff700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f0fe800b530 tx=0x7f0fe80090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800d4a0 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fe800d620 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ff815a040 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fe800d9e0 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.612+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ff815a600 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.613+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ff806c830 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.613+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0fe8012410 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.614+0000 7f0fdffff700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 0x7f0fe4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.614+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0fe807a110 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.614+0000 7f0ff77fe700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 0x7f0fe4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.614+0000 7f0ff77fe700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 0x7f0fe4061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0ff80aacb0 tx=0x7f0fec008040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.618+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0fe804b380 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.781+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0ff81585b0 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.782+0000 7f0fdffff700 1 -- 172.21.15.73:0/1450562364 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f0fe804e9a0 con 0x7f0ff80ad6a0 2023-11-28T04:07:44.782 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:44.782 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":164,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:44.458339+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:44.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 msgr2=0x7f0fe4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 0x7f0fe4061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f0ff80aacb0 tx=0x7f0fec008040).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 msgr2=0x7f0ff814c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 rx=0x7f0fe800b530 tx=0x7f0fe80090f0).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 shutdown_connections 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fe405f150 0x7f0fe4061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0ff80c6710 0x7f0ff81526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0ff80ad6a0 0x7f0ff814c050 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 --2- 172.21.15.73:0/1450562364 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0ff80ada80 0x7f0ff81521e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 >> 172.21.15.73:0/1450562364 conn(0x7f0ff80649c0 msgr2=0x7f0ff81571b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 shutdown_connections 2023-11-28T04:07:44.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.784+0000 7f0ffe45d700 1 -- 172.21.15.73:0/1450562364 wait complete. 2023-11-28T04:07:44.797 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:44.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.909+0000 7f70ad5d0700 1 Processor -- start 2023-11-28T04:07:44.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.909+0000 7f70ad5d0700 1 -- start start 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70a80caab0 0x7f70a80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80cae90 0x7f70a80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70a80aa960 0x7f70a80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a80c1490 con 0x7f70a80caab0 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70a80a9a90 con 0x7f70a80aa960 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70ad5d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8154ce0 con 0x7f70a80cae90 2023-11-28T04:07:44.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70a6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70a80caab0 0x7f70a80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70a6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f70a80caab0 0x7f70a80aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58506/0 (socket says 172.21.15.73:58506) 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.910+0000 7f70a6ffd700 1 -- 172.21.15.73:0/4000421257 learned_addr learned my addr 172.21.15.73:0/4000421257 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a77fe700 1 --2- 172.21.15.73:0/4000421257 >> v2:172.21.15.150:3300/0 conn(0x7f70a80aa960 0x7f70a80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 -- 172.21.15.73:0/4000421257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80cae90 msgr2=0x7f70a80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/4000421257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80cae90 0x7f70a80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 -- 172.21.15.73:0/4000421257 >> v2:172.21.15.150:3300/0 conn(0x7f70a80aa960 msgr2=0x7f70a80c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/4000421257 >> v2:172.21.15.150:3300/0 conn(0x7f70a80aa960 0x7f70a80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 -- 172.21.15.73:0/4000421257 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70a80a8e80 con 0x7f70a80caab0 2023-11-28T04:07:44.914 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/4000421257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80caab0 0x7f70a80aa470 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f70a805e5a0 tx=0x7f7098009060).ready entity=mon.0 client_cookie=9c6aa464c8f5a02f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a67fc700 1 -- 172.21.15.73:0/4000421257 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098005880 con 0x7f70a80caab0 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a67fc700 1 -- 172.21.15.73:0/4000421257 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70980059e0 con 0x7f70a80caab0 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70a67fc700 1 -- 172.21.15.73:0/4000421257 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7098005d60 con 0x7f70a80caab0 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 -- 172.21.15.73:0/4000421257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80caab0 msgr2=0x7f70a80aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/4000421257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80caab0 0x7f70a80aa470 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 rx=0x7f70a805e5a0 tx=0x7f7098009060).stop 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 -- 172.21.15.73:0/4000421257 shutdown_connections 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/4000421257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80caab0 0x7f70a80aa470 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/4000421257 >> v2:172.21.15.150:3300/0 conn(0x7f70a80aa960 0x7f70a80c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/4000421257 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80cae90 0x7f70a80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:44.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 -- 172.21.15.73:0/4000421257 >> 172.21.15.73:0/4000421257 conn(0x7f70a8094920 msgr2=0x7f70a806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:44.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 -- 172.21.15.73:0/4000421257 shutdown_connections 2023-11-28T04:07:44.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.911+0000 7f70ad5d0700 1 -- 172.21.15.73:0/4000421257 wait complete. 2023-11-28T04:07:44.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 Processor -- start 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 -- start start 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80caab0 0x7f70a8153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70a80b0650 0x7f70a80b0ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80b0fc0 con 0x7f70a80cae90 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a80c3ac0 con 0x7f70a80b0650 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f70ad5d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70a8155b60 con 0x7f70a80caab0 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.913+0000 7f709e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58512/0 (socket says 172.21.15.73:58512) 2023-11-28T04:07:44.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 -- 172.21.15.73:0/1121217475 learned_addr learned my addr 172.21.15.73:0/1121217475 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80caab0 0x7f70a8153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f70a77fe700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70a80b0650 0x7f70a80b0ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 -- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80caab0 msgr2=0x7f70a8153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80caab0 0x7f70a8153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 -- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70a80b0650 msgr2=0x7f70a80b0ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70a80b0650 0x7f70a80b0ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 -- 172.21.15.73:0/1121217475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7098009300 con 0x7f70a80cae90 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709e7fc700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f709000b490 tx=0x7f7090009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709000be40 con 0x7f70a80cae90 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7090002320 con 0x7f70a80cae90 2023-11-28T04:07:44.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70900026e0 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a8155e40 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.914+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a8153750 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.915+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70a80c3fa0 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.915+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7090002a90 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.916+0000 7f709f7fe700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 0x7f7088061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.916+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7090079ef0 con 0x7f70a80cae90 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.916+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 0x7f7088061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:44.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.916+0000 7f70a6ffd700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 0x7f7088061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f709800b3c0 tx=0x7f7098016000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:44.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:44.920+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f70a80c3fa0 con 0x7f70a80cae90 2023-11-28T04:07:45.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.085+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f70a80c3fa0 con 0x7f70a80cae90 2023-11-28T04:07:45.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.086+0000 7f709f7fe700 1 -- 172.21.15.73:0/1121217475 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v164) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f70a80c3fa0 con 0x7f70a80cae90 2023-11-28T04:07:45.086 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:45.086 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":164,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:44.458339+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.087+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 msgr2=0x7f7088061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.087+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 0x7f7088061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f709800b3c0 tx=0x7f7098016000).stop 2023-11-28T04:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.087+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 msgr2=0x7f70a8146e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.087+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 rx=0x7f709000b490 tx=0x7f7090009060).stop 2023-11-28T04:07:45.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.087+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 shutdown_connections 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f708805f150 0x7f7088061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70a80b0650 0x7f70a80b0ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70a80caab0 0x7f70a8153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 --2- 172.21.15.73:0/1121217475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70a80cae90 0x7f70a8146e50 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 >> 172.21.15.73:0/1121217475 conn(0x7f70a8094920 msgr2=0x7f70a80c0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 shutdown_connections 2023-11-28T04:07:45.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.088+0000 7f70ad5d0700 1 -- 172.21.15.73:0/1121217475 wait complete. 2023-11-28T04:07:45.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T04:07:45.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.218+0000 7efea233e700 1 Processor -- start 2023-11-28T04:07:45.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.218+0000 7efea233e700 1 -- start start 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efea233e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 0x7efe9c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efea233e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7efe9c158320 0x7efe9c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efea233e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe9c158cc0 con 0x7efe9c098d10 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efea233e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe9c0c72d0 con 0x7efe9c158320 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efea233e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c064570 con 0x7efe9c0c6190 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efe9bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efe9c098d10 0x7efe9c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55878/0 (socket says 172.21.15.73:55878) 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efe9bfff700 1 -- 172.21.15.73:0/2812024357 learned_addr learned my addr 172.21.15.73:0/2812024357 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:45.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3532187293 0 0) 0x7efe8c001110 con 0x7efe9c098d10 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.219+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe840033f0 con 0x7efe9c098d10 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4083198221 0 0) 0x7efe840033f0 con 0x7efe9c098d10 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 msgr2=0x7efe9c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 0x7efe9c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 >> v2:172.21.15.150:3300/0 conn(0x7efe9c158320 msgr2=0x7efe9c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 --2- 172.21.15.73:0/2812024357 >> v2:172.21.15.150:3300/0 conn(0x7efe9c158320 0x7efe9c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c154bc0 con 0x7efe9c098d10 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7efe8c001600 con 0x7efe9c098d10 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 >> v1:172.21.15.73:6789/0 conn(0x7efe9c098d10 legacy=0x7efe9c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 0x7efe840038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe84003fa0 0x7efe84004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe84004ca0 0x7efe84005120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0c72d0 con 0x7efe84004ca0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c064570 con 0x7efe84003fa0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8400bb70 con 0x7efe9c0c6190 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe84004ca0 0x7efe84005120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9bfff700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 0x7efe840038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9b7fe700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe84003fa0 0x7efe84004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 -- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 msgr2=0x7efe840038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c0c6190 0x7efe840038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 -- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe84003fa0 msgr2=0x7efe84004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe84003fa0 0x7efe84004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 -- 172.21.15.73:0/2812024357 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe840033f0 con 0x7efe84004ca0 2023-11-28T04:07:45.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efea08db700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe84004ca0 0x7efe84005120 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7efe94009f70 tx=0x7efe940095f0).ready entity=mon.0 client_cookie=9ca08d7aeed142c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe940098e0 con 0x7efe84004ca0 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.220+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9400bc60 con 0x7efe84004ca0 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efe9affd700 1 -- 172.21.15.73:0/2812024357 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe94015470 con 0x7efe84004ca0 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 -- 172.21.15.73:0/2812024357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe84004ca0 msgr2=0x7efe84005120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 --2- 172.21.15.73:0/2812024357 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe84004ca0 0x7efe84005120 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 rx=0x7efe94009f70 tx=0x7efe940095f0).stop 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea08db700 1 -- 172.21.15.73:0/2812024357 reap_dead start 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 -- 172.21.15.73:0/2812024357 shutdown_connections 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 -- 172.21.15.73:0/2812024357 >> 172.21.15.73:0/2812024357 conn(0x7efe9c0cb310 msgr2=0x7efe9c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 -- 172.21.15.73:0/2812024357 shutdown_connections 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.221+0000 7efea233e700 1 -- 172.21.15.73:0/2812024357 wait complete. 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.222+0000 7efea233e700 1 Processor -- start 2023-11-28T04:07:45.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.222+0000 7efea233e700 1 -- start start 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c14c480 0x7efe9c1512d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 0x7efe9c1517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c151cb0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0b0920 con 0x7efe9c0ae2c0 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea233e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe9c0afd10 con 0x7efe9c14c480 2023-11-28T04:07:45.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58540/0 (socket says 172.21.15.73:58540) 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 -- 172.21.15.73:0/2055541942 learned_addr learned my addr 172.21.15.73:0/2055541942 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea08db700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 0x7efe9c1517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9b7fe700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c14c480 0x7efe9c1512d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 -- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c14c480 msgr2=0x7efe9c1512d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c14c480 0x7efe9c1512d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 -- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 msgr2=0x7efe9c1517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 0x7efe9c1517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 -- 172.21.15.73:0/2055541942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe9c0afff0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efea08db700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 0x7efe9c1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.223+0000 7efe9bfff700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7efe8c001270 tx=0x7efe9c0a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.224+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00f8d0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.224+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8c00fa30 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.224+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe9c0b02d0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.224+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8c00fdb0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.224+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe9c0af9b0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.225+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe9c06c860 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.225+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efe8c012560 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.225+0000 7efe83fff700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 0x7efe880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.225+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efe8c079aa0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.225+0000 7efe9b7fe700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 0x7efe880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.226+0000 7efe9b7fe700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 0x7efe880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7efe90006f40 tx=0x7efe90008040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.229+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efe8c04ae20 con 0x7efe9c14c0a0 2023-11-28T04:07:45.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.432+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7efe9c158cc0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.463+0000 7efe83fff700 1 -- 172.21.15.73:0/2055541942 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v57) v1 ==== 118+0+0 (secure 0 0 0) 0x7efe9c158cc0 con 0x7efe9c14c0a0 2023-11-28T04:07:45.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 msgr2=0x7efe880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 0x7efe880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7efe90006f40 tx=0x7efe90008040).stop 2023-11-28T04:07:45.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 msgr2=0x7efe9c0af4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 rx=0x7efe8c001270 tx=0x7efe9c0a8040).stop 2023-11-28T04:07:45.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 shutdown_connections 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efe8805f120 0x7efe880615e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efe9c0ae2c0 0x7efe9c1517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efe9c14c0a0 0x7efe9c0af4c0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 --2- 172.21.15.73:0/2055541942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efe9c14c480 0x7efe9c1512d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 >> 172.21.15.73:0/2055541942 conn(0x7efe9c0cb310 msgr2=0x7efe9c155890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 shutdown_connections 2023-11-28T04:07:45.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.465+0000 7efea233e700 1 -- 172.21.15.73:0/2055541942 wait complete. 2023-11-28T04:07:45.478 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T04:07:45.478 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T04:07:45.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.592+0000 7fdbb3fe0700 1 Processor -- start 2023-11-28T04:07:45.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 -- start start 2023-11-28T04:07:45.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdbac158160 0x7fdbac0a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbac14e600 0x7fdbac14ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbac067770 con 0x7fdbac0a8280 2023-11-28T04:07:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac159180 con 0x7fdbac14e600 2023-11-28T04:07:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.593+0000 7fdbb3fe0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbac0be8d0 con 0x7fdbac158160 2023-11-28T04:07:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbac14e600 0x7fdbac14ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbac14e600 0x7fdbac14ea90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58544/0 (socket says 172.21.15.73:58544) 2023-11-28T04:07:45.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 -- 172.21.15.73:0/102497267 learned_addr learned my addr 172.21.15.73:0/102497267 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:45.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb157b700 1 --2- 172.21.15.73:0/102497267 >> v2:172.21.15.150:3300/0 conn(0x7fdbac158160 0x7fdbac0a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 -- 172.21.15.73:0/102497267 >> v2:172.21.15.150:3300/0 conn(0x7fdbac158160 msgr2=0x7fdbac0a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 --2- 172.21.15.73:0/102497267 >> v2:172.21.15.150:3300/0 conn(0x7fdbac158160 0x7fdbac0a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 -- 172.21.15.73:0/102497267 >> v1:172.21.15.150:6789/0 conn(0x7fdbac0a8280 legacy=0x7fdbac155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:45.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 -- 172.21.15.73:0/102497267 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbac158600 con 0x7fdbac14e600 2023-11-28T04:07:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb257d700 1 --2- 172.21.15.73:0/102497267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac14e600 0x7fdbac14ea90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fdbac0be860 tx=0x7fdba40090f0).ready entity=mon.0 client_cookie=b6a382f5f74a85a4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb0d7a700 1 -- 172.21.15.73:0/102497267 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400b600 con 0x7fdbac14e600 2023-11-28T04:07:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb0d7a700 1 -- 172.21.15.73:0/102497267 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400b760 con 0x7fdbac14e600 2023-11-28T04:07:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.594+0000 7fdbb0d7a700 1 -- 172.21.15.73:0/102497267 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400bae0 con 0x7fdbac14e600 2023-11-28T04:07:45.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/102497267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac14e600 msgr2=0x7fdbac14ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/102497267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac14e600 0x7fdbac14ea90 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 rx=0x7fdbac0be860 tx=0x7fdba40090f0).stop 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/102497267 shutdown_connections 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/102497267 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac14e600 0x7fdbac14ea90 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/102497267 >> v2:172.21.15.150:3300/0 conn(0x7fdbac158160 0x7fdbac0a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/102497267 >> 172.21.15.73:0/102497267 conn(0x7fdbac062700 msgr2=0x7fdbac062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:45.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/102497267 shutdown_connections 2023-11-28T04:07:45.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.595+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/102497267 wait complete. 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.596+0000 7fdbb3fe0700 1 Processor -- start 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.596+0000 7fdbb3fe0700 1 -- start start 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbac0a8280 0x7fdbac0a8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbac14e600 0x7fdbac0c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0ac0d0 con 0x7fdbac158160 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0ac930 con 0x7fdbac14e600 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbac0a90f0 con 0x7fdbac0a8280 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58560/0 (socket says 172.21.15.73:58560) 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 -- 172.21.15.73:0/1358876454 learned_addr learned my addr 172.21.15.73:0/1358876454 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb157b700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbac14e600 0x7fdbac0c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb1d7c700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbac0a8280 0x7fdbac0a8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 -- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbac0a8280 msgr2=0x7fdbac0a8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbac0a8280 0x7fdbac0a8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 -- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbac14e600 msgr2=0x7fdbac0c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbac14e600 0x7fdbac0c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 -- 172.21.15.73:0/1358876454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba4009370 con 0x7fdbac158160 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb257d700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fdba4009130 tx=0x7fdba4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400bae0 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400bea0 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbac0aefd0 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.597+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba401c670 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.598+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbac0ac5c0 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.598+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbac0c48d0 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.599+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdba401ca20 con 0x7fdbac158160 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.599+0000 7fdba1ffb700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 0x7fdb90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:45.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.599+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fdba407bff0 con 0x7fdbac158160 2023-11-28T04:07:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.599+0000 7fdbb1d7c700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 0x7fdb90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.600+0000 7fdbb1d7c700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 0x7fdb90061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdba8002c70 tx=0x7fdba80014c0).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:45.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.603+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdba404d2b0 con 0x7fdbac158160 2023-11-28T04:07:45.809 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:45.808+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fdbac06c860 con 0x7fdbac158160 2023-11-28T04:07:46.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.470+0000 7fdba1ffb700 1 -- 172.21.15.73:0/1358876454 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7fdba40508d0 con 0x7fdbac158160 2023-11-28T04:07:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 msgr2=0x7fdb90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 0x7fdb90061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fdba8002c70 tx=0x7fdba80014c0).stop 2023-11-28T04:07:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 msgr2=0x7fdbac14cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 rx=0x7fdba4009130 tx=0x7fdba4017000).stop 2023-11-28T04:07:46.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 shutdown_connections 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fdb9005f150 0x7fdb90061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbac158160 0x7fdbac14cd70 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbac0a8280 0x7fdbac0a8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 --2- 172.21.15.73:0/1358876454 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbac14e600 0x7fdbac0c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 >> 172.21.15.73:0/1358876454 conn(0x7fdbac062700 msgr2=0x7fdbac0becb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 shutdown_connections 2023-11-28T04:07:46.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.473+0000 7fdbb3fe0700 1 -- 172.21.15.73:0/1358876454 wait complete. 2023-11-28T04:07:46.487 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T04:07:46.487 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:46.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.601+0000 7f460ee97700 1 Processor -- start 2023-11-28T04:07:46.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.601+0000 7f460ee97700 1 -- start start 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460ee97700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4608155750 0x7f46080acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460ee97700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608155b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460ee97700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4608063ff0 con 0x7f460806b870 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460ee97700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46080696d0 con 0x7f4608155750 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460ee97700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080699b0 con 0x7f46080c2150 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4608155750 0x7f46080acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4608155750 0x7f46080acf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58576/0 (socket says 172.21.15.73:58576) 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 -- 172.21.15.73:0/1785264897 learned_addr learned my addr 172.21.15.73:0/1785264897 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 -- 172.21.15.73:0/1785264897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 msgr2=0x7f4608155b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 --2- 172.21.15.73:0/1785264897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608155b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.602+0000 7f460de95700 1 -- 172.21.15.73:0/1785264897 >> v1:172.21.15.150:6789/0 conn(0x7f460806b870 legacy=0x7f46080c1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:46.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460de95700 1 -- 172.21.15.73:0/1785264897 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4608063c80 con 0x7f4608155750 2023-11-28T04:07:46.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460de95700 1 --2- 172.21.15.73:0/1785264897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4608155750 0x7f46080acf40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f4604006fd0 tx=0x7f46040090f0).ready entity=mon.0 client_cookie=85b36a0ff5252d50 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ce93700 1 -- 172.21.15.73:0/1785264897 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400b600 con 0x7f4608155750 2023-11-28T04:07:46.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ce93700 1 -- 172.21.15.73:0/1785264897 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400b760 con 0x7f4608155750 2023-11-28T04:07:46.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ce93700 1 -- 172.21.15.73:0/1785264897 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400bae0 con 0x7f4608155750 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 -- 172.21.15.73:0/1785264897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4608155750 msgr2=0x7f46080acf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 --2- 172.21.15.73:0/1785264897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4608155750 0x7f46080acf40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 rx=0x7f4604006fd0 tx=0x7f46040090f0).stop 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 -- 172.21.15.73:0/1785264897 shutdown_connections 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 --2- 172.21.15.73:0/1785264897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608155b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 --2- 172.21.15.73:0/1785264897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4608155750 0x7f46080acf40 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 -- 172.21.15.73:0/1785264897 >> 172.21.15.73:0/1785264897 conn(0x7f46080c7eb0 msgr2=0x7f46080c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:46.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 -- 172.21.15.73:0/1785264897 shutdown_connections 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.603+0000 7f460ee97700 1 -- 172.21.15.73:0/1785264897 wait complete. 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.604+0000 7f460ee97700 1 Processor -- start 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 -- start start 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608166810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4608155750 0x7f46080bf4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4608166d00 con 0x7f460806b870 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46080be4e0 con 0x7f4608155750 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460ee97700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46080bf020 con 0x7f46080c2150 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460de95700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460de95700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58588/0 (socket says 172.21.15.73:58588) 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460de95700 1 -- 172.21.15.73:0/3032097780 learned_addr learned my addr 172.21.15.73:0/3032097780 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.605+0000 7f460d694700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608166810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 -- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 msgr2=0x7f4608166810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608166810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 -- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4608155750 msgr2=0x7f46080bf4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4608155750 0x7f46080bf4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 -- 172.21.15.73:0/3032097780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4604009370 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460de95700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f4608068730 tx=0x7f4604017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f460400b8d0 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f460400ba30 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46080bf9a0 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46040246c0 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.606+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46080be9d0 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.607+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f460806c7c0 con 0x7f460806b870 2023-11-28T04:07:46.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.607+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4604021070 con 0x7f460806b870 2023-11-28T04:07:46.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.608+0000 7f45fdffb700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 0x7f45ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.608+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f460407be30 con 0x7f460806b870 2023-11-28T04:07:46.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.608+0000 7f460d694700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 0x7f45ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.608+0000 7f460d694700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 0x7f45ec061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4608151550 tx=0x7f45f8008040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.612+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f460404d100 con 0x7f460806b870 2023-11-28T04:07:46.813 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.813+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f46080c46f0 con 0x7f460806b870 2023-11-28T04:07:46.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.813+0000 7f45fdffb700 1 -- 172.21.15.73:0/3032097780 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f4604050720 con 0x7f460806b870 2023-11-28T04:07:46.814 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:46.815 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:07:46.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 msgr2=0x7f45ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 0x7f45ec061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f4608151550 tx=0x7f45f8008040).stop 2023-11-28T04:07:46.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 msgr2=0x7f460814e230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 rx=0x7f4608068730 tx=0x7f4604017000).stop 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 shutdown_connections 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f45ec05f150 0x7f45ec061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4608155750 0x7f46080bf4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f460806b870 0x7f460814e230 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 --2- 172.21.15.73:0/3032097780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46080c2150 0x7f4608166810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 >> 172.21.15.73:0/3032097780 conn(0x7f46080c7eb0 msgr2=0x7f46080aa480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 shutdown_connections 2023-11-28T04:07:46.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.815+0000 7f460ee97700 1 -- 172.21.15.73:0/3032097780 wait complete. 2023-11-28T04:07:46.818 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:07:46.829 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:07:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.940+0000 7ff3dcfd2700 1 Processor -- start 2023-11-28T04:07:46.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.940+0000 7ff3dcfd2700 1 -- start start 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3dcfd2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3dcfd2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3dcfd2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d80c12c0 con 0x7ff3d80c0180 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3dcfd2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80c06b0 con 0x7ff3d8157660 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3dcfd2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80c0990 con 0x7ff3d80c1750 2023-11-28T04:07:46.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d77fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d7fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:34122/0 (socket says 172.21.15.73:34122) 2023-11-28T04:07:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d6ffd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7ff3d80c0180 0x7ff3d81589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:40716/0 (socket says 172.21.15.73:40716) 2023-11-28T04:07:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d77fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:57052/0 (socket says 172.21.15.73:57052) 2023-11-28T04:07:46.943 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.941+0000 7ff3d7fff700 1 -- 172.21.15.73:0/445812521 learned_addr learned my addr 172.21.15.73:0/445812521 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d7fff700 1 -- 172.21.15.73:0/445812521 >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 msgr2=0x7ff3d8157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d7fff700 1 --2- 172.21.15.73:0/445812521 >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d7fff700 1 -- 172.21.15.73:0/445812521 >> v1:172.21.15.150:6789/0 conn(0x7ff3d80c0180 legacy=0x7ff3d81589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d7fff700 1 -- 172.21.15.73:0/445812521 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80c0c70 con 0x7ff3d80c1750 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d77fe700 1 --2- 172.21.15.73:0/445812521 >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d7fff700 1 --2- 172.21.15.73:0/445812521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff3d0009f20 tx=0x7ff3d0009840).ready entity=mon.2 client_cookie=4d6f060fb2330501 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d67fc700 1 -- 172.21.15.73:0/445812521 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000e070 con 0x7ff3d80c1750 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d67fc700 1 -- 172.21.15.73:0/445812521 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d000b9d0 con 0x7ff3d80c1750 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3d67fc700 1 -- 172.21.15.73:0/445812521 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d000bd50 con 0x7ff3d80c1750 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/445812521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 msgr2=0x7ff3d80c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.945 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/445812521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 rx=0x7ff3d0009f20 tx=0x7ff3d0009840).stop 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/445812521 shutdown_connections 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/445812521 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d80c1bc0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/445812521 >> v2:172.21.15.150:3300/0 conn(0x7ff3d8157660 0x7ff3d8157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/445812521 >> 172.21.15.73:0/445812521 conn(0x7ff3d80cb000 msgr2=0x7ff3d80cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.942+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/445812521 shutdown_connections 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.943+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/445812521 wait complete. 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 Processor -- start 2023-11-28T04:07:46.946 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 -- start start 2023-11-28T04:07:46.948 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d8149900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3d8157660 0x7ff3d8149df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80af030 con 0x7ff3d80c0180 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d814a2e0 con 0x7ff3d8157660 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.944+0000 7ff3dcfd2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80a27b0 con 0x7ff3d80c1750 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58604/0 (socket says 172.21.15.73:58604) 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 -- 172.21.15.73:0/2126675619 learned_addr learned my addr 172.21.15.73:0/2126675619 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d6ffd700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d8149900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d7fff700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3d8157660 0x7ff3d8149df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 -- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 msgr2=0x7ff3d8149900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d8149900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 -- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3d8157660 msgr2=0x7ff3d8149df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3d8157660 0x7ff3d8149df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 -- 172.21.15.73:0/2126675619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3c80092a0 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3d77fe700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7ff3c800ea20 tx=0x7ff3c8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800ee60 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3c800c850 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d0009ae0 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3c800cbd0 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.945+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d80a2440 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.946+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d806c7c0 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.947+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff3c801b400 con 0x7ff3d80c0180 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.947+0000 7ff3bf7fe700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 0x7ff3c4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:46.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.947+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff3c807a710 con 0x7ff3d80c0180 2023-11-28T04:07:46.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.947+0000 7ff3d6ffd700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 0x7ff3c4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:46.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.947+0000 7ff3d6ffd700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 0x7ff3c4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff3d8155fe0 tx=0x7ff3cc000bc0).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:46.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:46.951+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3c804b980 con 0x7ff3d80c0180 2023-11-28T04:07:47.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.116+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff3d80c0f50 con 0x7ff3d80c0180 2023-11-28T04:07:47.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.117+0000 7ff3bf7fe700 1 -- 172.21.15.73:0/2126675619 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v165) v1 ==== 74+0+16943 (secure 0 0 0) 0x7ff3c804efa0 con 0x7ff3d80c0180 2023-11-28T04:07:47.119 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:47.119 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":165,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:07:45.460490+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:07:47.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 msgr2=0x7ff3c4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 0x7ff3c4061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7ff3d8155fe0 tx=0x7ff3cc000bc0).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 msgr2=0x7ff3d8159320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 rx=0x7ff3c800ea20 tx=0x7ff3c8009060).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 shutdown_connections 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff3c405f150 0x7ff3c4061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3d8157660 0x7ff3d8149df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3d80c0180 0x7ff3d8159320 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 --2- 172.21.15.73:0/2126675619 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3d80c1750 0x7ff3d8149900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 >> 172.21.15.73:0/2126675619 conn(0x7ff3d80cb000 msgr2=0x7ff3d81566d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 shutdown_connections 2023-11-28T04:07:47.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.119+0000 7ff3dcfd2700 1 -- 172.21.15.73:0/2126675619 wait complete. 2023-11-28T04:07:47.132 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T04:07:47.133 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:07:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.244+0000 7f96b4a45700 1 Processor -- start 2023-11-28T04:07:47.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.244+0000 7f96b4a45700 1 -- start start 2023-11-28T04:07:47.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.245+0000 7f96b4a45700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f96b00caab0 0x7f96b00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.245+0000 7f96b4a45700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f96b00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.249 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.245+0000 7f96b4a45700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96b00a9a90 con 0x7f96b00aa960 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.245+0000 7f96b4a45700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96b0154ce0 con 0x7f96b00caab0 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.245+0000 7f96b4a45700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a8e80 con 0x7f96b00cae90 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96aed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f96b00aa960 0x7f96b00c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55886/0 (socket says 172.21.15.73:55886) 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96aed9d700 1 -- 172.21.15.73:0/1930942716 learned_addr learned my addr 172.21.15.73:0/1930942716 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1930942716 >> v2:172.21.15.150:3300/0 conn(0x7f96b00caab0 0x7f96b00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1559184432 0 0) 0x7f96b00a9a90 con 0x7f96b00aa960 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96add9b700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f96b00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f969c0033f0 con 0x7f96b00aa960 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 652696223 0 0) 0x7f969c0033f0 con 0x7f96b00aa960 2023-11-28T04:07:47.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 msgr2=0x7f96b00c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f96b00c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 >> v2:172.21.15.150:3300/0 conn(0x7f96b00caab0 msgr2=0x7f96b00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 --2- 172.21.15.73:0/1930942716 >> v2:172.21.15.150:3300/0 conn(0x7f96b00caab0 0x7f96b00aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00a9160 con 0x7f96b00aa960 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f96a8001470 con 0x7f96b00aa960 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 >> v1:172.21.15.73:6789/0 conn(0x7f96b00aa960 legacy=0x7f96b00c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f969c0038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003e10 0x7f969c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0154ce0 con 0x7f969c004b80 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00a8e80 con 0x7f969c003e10 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f969c00b8e0 con 0x7f96b00cae90 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f969c0038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.246+0000 7f96add9b700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003e10 0x7f969c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 -- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 msgr2=0x7f969c0038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f96b00cae90 0x7f969c0038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 -- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003e10 msgr2=0x7f969c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f969c003e10 0x7f969c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 -- 172.21.15.73:0/1930942716 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f969c0033f0 con 0x7f969c004b80 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f96a800c1e0 tx=0x7f96a8001e90).ready entity=mon.0 client_cookie=73484fb953b4b69e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a8007a60 con 0x7f969c004b80 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96a800ca70 con 0x7f969c004b80 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96ad59a700 1 -- 172.21.15.73:0/1930942716 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96a800cdf0 con 0x7f969c004b80 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 -- 172.21.15.73:0/1930942716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 msgr2=0x7f969c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1930942716 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f969c004b80 0x7f969c005000 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 rx=0x7f96a800c1e0 tx=0x7f96a8001e90).stop 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96aed9d700 1 -- 172.21.15.73:0/1930942716 reap_dead start 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 -- 172.21.15.73:0/1930942716 shutdown_connections 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 -- 172.21.15.73:0/1930942716 >> 172.21.15.73:0/1930942716 conn(0x7f96b0094920 msgr2=0x7f96b006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 -- 172.21.15.73:0/1930942716 shutdown_connections 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.247+0000 7f96b4a45700 1 -- 172.21.15.73:0/1930942716 wait complete. 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.248+0000 7f96b4a45700 1 Processor -- start 2023-11-28T04:07:47.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 -- start start 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f969c004b80 0x7f96b0144f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 0x7f96b0064da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0159aa0 con 0x7f96b01587b0 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96b00c35c0 con 0x7f96b0064920 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96b4a45700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96b0065290 con 0x7f969c004b80 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58620/0 (socket says 172.21.15.73:58620) 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96add9b700 1 -- 172.21.15.73:0/1185672751 learned_addr learned my addr 172.21.15.73:0/1185672751 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.249+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f969c004b80 0x7f96b0144f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 0x7f96b0064da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 -- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f969c004b80 msgr2=0x7f96b0144f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f969c004b80 0x7f96b0144f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 -- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 msgr2=0x7f96b0064da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 0x7f96b0064da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 -- 172.21.15.73:0/1185672751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96b00c3290 con 0x7f96b01587b0 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96aed9d700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 0x7f96b0064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96add9b700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f9698011a20 tx=0x7f9698009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800f8e0 con 0x7f96b01587b0 2023-11-28T04:07:47.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f969800fa40 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96b0159200 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f969800fe00 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.250+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96b01597c0 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.251+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96b006c860 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.251+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9698014650 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.252+0000 7f96a67fc700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 0x7f9690061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.252+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 0x7f9690061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.252+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f969807aea0 con 0x7f96b01587b0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.252+0000 7f96ae59c700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 0x7f9690061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f96a00094a0 tx=0x7f96a000b040).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.256+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f969804c110 con 0x7f96b01587b0 2023-11-28T04:07:47.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.415+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f96b00c3420 con 0x7f969005f150 2023-11-28T04:07:47.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.425+0000 7f96a67fc700 1 -- 172.21.15.73:0/1185672751 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350228 (secure 0 0 0) 0x7f96a000f040 con 0x7f969005f150 2023-11-28T04:07:47.430 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 msgr2=0x7f9690061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 0x7f9690061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f96a00094a0 tx=0x7f96a000b040).stop 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 msgr2=0x7f96b0158c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 rx=0x7f9698011a20 tx=0x7f9698009060).stop 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 shutdown_connections 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f969005f150 0x7f9690061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f96b0064920 0x7f96b0064da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f969c004b80 0x7f96b0144f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 --2- 172.21.15.73:0/1185672751 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f96b01587b0 0x7f96b0158c30 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.428+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 >> 172.21.15.73:0/1185672751 conn(0x7f96b0094920 msgr2=0x7f96b00ac090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.429+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 shutdown_connections 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.429+0000 7f96b4a45700 1 -- 172.21.15.73:0/1185672751 wait complete. 2023-11-28T04:07:47.434 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:07:47.441 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-28T04:07:46.056908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2437448,"kb_used_data":15624,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752537272,"statfs":{"total":773094113280,"available":770598166528,"internally_reserved":0,"allocated":15998976,"data_stored":8044946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5035976,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-521,"num_write_kb":-5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-960,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995164"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.572057+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:07:40.572057+0000","last_peered":"2023-11-28T04:07:40.572057+0000","last_clean":"2023-11-28T04:07:40.572057+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:07:40.572057+0000","last_undegraded":"2023-11-28T04:07:40.572057+0000","last_fullsized":"2023-11-28T04:07:40.572057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:41.068514+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:07:41.068514+0000","last_peered":"2023-11-28T04:07:41.068514+0000","last_clean":"2023-11-28T04:07:41.068514+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:07:41.068514+0000","last_undegraded":"2023-11-28T04:07:41.068514+0000","last_fullsized":"2023-11-28T04:07:41.068514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.820489+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:07:40.820489+0000","last_peered":"2023-11-28T04:07:40.820489+0000","last_clean":"2023-11-28T04:07:40.820489+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:07:40.820489+0000","last_undegraded":"2023-11-28T04:07:40.820489+0000","last_fullsized":"2023-11-28T04:07:40.820489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:41.068514+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:07:41.068514+0000","last_peered":"2023-11-28T04:07:41.068514+0000","last_clean":"2023-11-28T04:07:41.068514+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:07:41.068514+0000","last_undegraded":"2023-11-28T04:07:41.068514+0000","last_fullsized":"2023-11-28T04:07:41.068514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":130,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577192+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:07:40.577192+0000","last_peered":"2023-11-28T04:07:40.577192+0000","last_clean":"2023-11-28T04:07:40.577192+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:07:40.577192+0000","last_undegraded":"2023-11-28T04:07:40.577192+0000","last_fullsized":"2023-11-28T04:07:40.577192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577239+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:07:40.577239+0000","last_peered":"2023-11-28T04:07:40.577239+0000","last_clean":"2023-11-28T04:07:40.577239+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:07:40.577239+0000","last_undegraded":"2023-11-28T04:07:40.577239+0000","last_fullsized":"2023-11-28T04:07:40.577239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577180+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:07:40.577180+0000","last_peered":"2023-11-28T04:07:40.577180+0000","last_clean":"2023-11-28T04:07:40.577180+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:07:40.577180+0000","last_undegraded":"2023-11-28T04:07:40.577180+0000","last_fullsized":"2023-11-28T04:07:40.577180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.572089+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:07:40.572089+0000","last_peered":"2023-11-28T04:07:40.572089+0000","last_clean":"2023-11-28T04:07:40.572089+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:07:40.572089+0000","last_undegraded":"2023-11-28T04:07:40.572089+0000","last_fullsized":"2023-11-28T04:07:40.572089+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:39.988385+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:07:39.988385+0000","last_peered":"2023-11-28T04:07:39.988385+0000","last_clean":"2023-11-28T04:07:39.988385+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:07:39.988385+0000","last_undegraded":"2023-11-28T04:07:39.988385+0000","last_fullsized":"2023-11-28T04:07:39.988385+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575745,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1990656,"data_stored":1001260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.379},"max":{"1min":0.72699999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.372}]},{"osd":3,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.875,"15min":0.875},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575745,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.81899999999999995},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.752,"15min":0.80300000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.33800000000000002},"max":{"1min":0.68600000000000005,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67900000000000005,"5min":0.79500000000000004,"15min":0.79600000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.78000000000000003,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.38600000000000001},"max":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79800000000000004},"last":0.68799999999999994}]},{"osd":4,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.35199999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.79200000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575744,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.61199999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.309,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.54600000000000004,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56299999999999994,"5min":0.621,"15min":0.67300000000000004},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.77500000000000002},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78500000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.45800000000000002,"15min":0.42499999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70599999999999996},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.42599999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834575746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307232,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064608,"statfs":{"total":96636764160,"available":96322158592,"internally_reserved":0,"allocated":1990656,"data_stored":1001260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.625,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.249,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.626},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.37},{"interface":"front","average":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41599999999999998},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.39900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":0.93400000000000005},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40200000000000002},"max":{"1min":0.85899999999999999,"5min":1.048,"15min":1.048},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.496}]},{"osd":6,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307244,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064596,"statfs":{"total":96636764160,"available":96322146304,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.621,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.39800000000000002},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.25700000000000001},"max":{"1min":0.59999999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.67600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.503,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.94799999999999995,"15min":1.085},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":1.1200000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.65900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.502}]},{"osd":6,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.145},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.434},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.47099999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.622}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.52200000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.505},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575746,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307244,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064596,"statfs":{"total":96636764160,"available":96322146304,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77800000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.89400000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.749}]},{"osd":3,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.378},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77000000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.69499999999999995,"15min":0.83799999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.84499999999999997},"last":0.83499999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.79000000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.90700000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.78500000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.81999999999999995},"last":0.871}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:07:47.448 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:07:47.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.593+0000 7fcfeb6d4700 1 Processor -- start 2023-11-28T04:07:47.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.593+0000 7fcfeb6d4700 1 -- start start 2023-11-28T04:07:47.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.594+0000 7fcfeb6d4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcfe40caab0 0x7fcfe40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.594+0000 7fcfeb6d4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40ca180 0x7fcfe415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.594+0000 7fcfeb6d4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfe40a9a90 con 0x7fcfe40cae90 2023-11-28T04:07:47.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.594+0000 7fcfeb6d4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfe4154ce0 con 0x7fcfe40caab0 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.594+0000 7fcfeb6d4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe40a8e80 con 0x7fcfe40ca180 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcfe40caab0 0x7fcfe40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fcfe40caab0 0x7fcfe40aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:57096/0 (socket says 172.21.15.73:57096) 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 -- 172.21.15.73:0/1286289141 learned_addr learned my addr 172.21.15.73:0/1286289141 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 -- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40ca180 msgr2=0x7fcfe415e3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 --2- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40ca180 0x7fcfe415e3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 -- 172.21.15.73:0/1286289141 >> v1:172.21.15.150:6789/0 conn(0x7fcfe40cae90 legacy=0x7fcfe40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfdbfff700 1 -- 172.21.15.73:0/1286289141 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3207692364 0 0) 0x7fcfe40a9a90 con 0x7fcfe40cae90 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 -- 172.21.15.73:0/1286289141 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe40a9160 con 0x7fcfe40caab0 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfe9470700 1 --2- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40caab0 0x7fcfe40aa470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fcfe0009810 tx=0x7fcfe0009060).ready entity=mon.1 client_cookie=5fbb08a6e17f5e11 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfdbfff700 1 -- 172.21.15.73:0/1286289141 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0005880 con 0x7fcfe40caab0 2023-11-28T04:07:47.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfdbfff700 1 -- 172.21.15.73:0/1286289141 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe00059e0 con 0x7fcfe40caab0 2023-11-28T04:07:47.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.595+0000 7fcfdbfff700 1 -- 172.21.15.73:0/1286289141 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe0005d60 con 0x7fcfe40caab0 2023-11-28T04:07:47.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40caab0 msgr2=0x7fcfe40aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40caab0 0x7fcfe40aa470 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 rx=0x7fcfe0009810 tx=0x7fcfe0009060).stop 2023-11-28T04:07:47.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/1286289141 shutdown_connections 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40ca180 0x7fcfe415e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/1286289141 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40caab0 0x7fcfe40aa470 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/1286289141 >> 172.21.15.73:0/1286289141 conn(0x7fcfe4094920 msgr2=0x7fcfe406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/1286289141 shutdown_connections 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.596+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/1286289141 wait complete. 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.597+0000 7fcfeb6d4700 1 Processor -- start 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.597+0000 7fcfeb6d4700 1 -- start start 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40ca180 0x7fcfe40ad060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40cae90 0x7fcfe41565c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4157760 con 0x7fcfe40caab0 2023-11-28T04:07:47.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4156b50 con 0x7fcfe40ca180 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfeb6d4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfe4156e30 con 0x7fcfe40cae90 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfe8c6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfe8c6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58622/0 (socket says 172.21.15.73:58622) 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.598+0000 7fcfe8c6f700 1 -- 172.21.15.73:0/2717048507 learned_addr learned my addr 172.21.15.73:0/2717048507 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 -- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40cae90 msgr2=0x7fcfe41565c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40cae90 0x7fcfe41565c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 -- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40ca180 msgr2=0x7fcfe40ad060 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40ca180 0x7fcfe40ad060 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 -- 172.21.15.73:0/2717048507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe0009300 con 0x7fcfe40caab0 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfe8c6f700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7fcfd400c410 tx=0x7fcfd4007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4007020 con 0x7fcfe40caab0 2023-11-28T04:07:47.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd40071a0 con 0x7fcfe40caab0 2023-11-28T04:07:47.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfe4157110 con 0x7fcfe40caab0 2023-11-28T04:07:47.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd4007560 con 0x7fcfe40caab0 2023-11-28T04:07:47.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.599+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfe4157d80 con 0x7fcfe40caab0 2023-11-28T04:07:47.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.600+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfe406c860 con 0x7fcfe40caab0 2023-11-28T04:07:47.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.600+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcfd40068b0 con 0x7fcfe40caab0 2023-11-28T04:07:47.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.601+0000 7fcfd97fa700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 0x7fcfc8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.608 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.601+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fcfd40795e0 con 0x7fcfe40caab0 2023-11-28T04:07:47.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.601+0000 7fcfe9470700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 0x7fcfc8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.601+0000 7fcfe9470700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 0x7fcfc8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcfe000b400 tx=0x7fcfe0016000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.605+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcfd4047af0 con 0x7fcfe40caab0 2023-11-28T04:07:47.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.762+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fcfe41578a0 con 0x7fcfc805f150 2023-11-28T04:07:47.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.771+0000 7fcfd97fa700 1 -- 172.21.15.73:0/2717048507 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+350228 (secure 0 0 0) 0x7fcfe0005b50 con 0x7fcfc805f150 2023-11-28T04:07:47.776 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 msgr2=0x7fcfc8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 0x7fcfc8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7fcfe000b400 tx=0x7fcfe0016000).stop 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 msgr2=0x7fcfe41560d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 rx=0x7fcfd400c410 tx=0x7fcfd4007e00).stop 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 shutdown_connections 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fcfc805f150 0x7fcfc8061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcfe40cae90 0x7fcfe41565c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcfe40ca180 0x7fcfe40ad060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:47.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 --2- 172.21.15.73:0/2717048507 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcfe40caab0 0x7fcfe41560d0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 >> 172.21.15.73:0/2717048507 conn(0x7fcfe4094920 msgr2=0x7fcfe40b0230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 shutdown_connections 2023-11-28T04:07:47.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.775+0000 7fcfeb6d4700 1 -- 172.21.15.73:0/2717048507 wait complete. 2023-11-28T04:07:47.782 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:07:47.788 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-11-28T04:07:46.056908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":35,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2437448,"kb_used_data":15624,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752537272,"statfs":{"total":773094113280,"available":770598166528,"internally_reserved":0,"allocated":15998976,"data_stored":8044946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5035976,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-521,"num_write_kb":-5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-960,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995164"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":129,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.572057+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:07:40.572057+0000","last_peered":"2023-11-28T04:07:40.572057+0000","last_clean":"2023-11-28T04:07:40.572057+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:07:40.572057+0000","last_undegraded":"2023-11-28T04:07:40.572057+0000","last_fullsized":"2023-11-28T04:07:40.572057+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:41.068514+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:07:41.068514+0000","last_peered":"2023-11-28T04:07:41.068514+0000","last_clean":"2023-11-28T04:07:41.068514+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:07:41.068514+0000","last_undegraded":"2023-11-28T04:07:41.068514+0000","last_fullsized":"2023-11-28T04:07:41.068514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":136,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.820489+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:07:40.820489+0000","last_peered":"2023-11-28T04:07:40.820489+0000","last_clean":"2023-11-28T04:07:40.820489+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:07:40.820489+0000","last_undegraded":"2023-11-28T04:07:40.820489+0000","last_fullsized":"2023-11-28T04:07:40.820489+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:41.068514+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:07:41.068514+0000","last_peered":"2023-11-28T04:07:41.068514+0000","last_clean":"2023-11-28T04:07:41.068514+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:07:41.068514+0000","last_undegraded":"2023-11-28T04:07:41.068514+0000","last_fullsized":"2023-11-28T04:07:41.068514+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":130,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577192+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:07:40.577192+0000","last_peered":"2023-11-28T04:07:40.577192+0000","last_clean":"2023-11-28T04:07:40.577192+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:07:40.577192+0000","last_undegraded":"2023-11-28T04:07:40.577192+0000","last_fullsized":"2023-11-28T04:07:40.577192+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577239+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:07:40.577239+0000","last_peered":"2023-11-28T04:07:40.577239+0000","last_clean":"2023-11-28T04:07:40.577239+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:07:40.577239+0000","last_undegraded":"2023-11-28T04:07:40.577239+0000","last_fullsized":"2023-11-28T04:07:40.577239+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":128,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.577180+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:07:40.577180+0000","last_peered":"2023-11-28T04:07:40.577180+0000","last_clean":"2023-11-28T04:07:40.577180+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:07:40.577180+0000","last_undegraded":"2023-11-28T04:07:40.577180+0000","last_fullsized":"2023-11-28T04:07:40.577180+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":129,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:40.572089+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:07:40.572089+0000","last_peered":"2023-11-28T04:07:40.572089+0000","last_clean":"2023-11-28T04:07:40.572089+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:07:40.572089+0000","last_undegraded":"2023-11-28T04:07:40.572089+0000","last_fullsized":"2023-11-28T04:07:40.572089+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:44.085647+0000","last_change":"2023-11-28T04:07:44.085647+0000","last_active":"2023-11-28T04:07:44.085647+0000","last_peered":"2023-11-28T04:07:44.085647+0000","last_clean":"2023-11-28T04:07:44.085647+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:44.085647+0000","last_undegraded":"2023-11-28T04:07:44.085647+0000","last_fullsized":"2023-11-28T04:07:44.085647+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:07:43.078371+0000","last_change":"2023-11-28T04:07:43.078371+0000","last_active":"2023-11-28T04:07:43.078371+0000","last_peered":"2023-11-28T04:07:43.078371+0000","last_clean":"2023-11-28T04:07:43.078371+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:07:43.078371+0000","last_undegraded":"2023-11-28T04:07:43.078371+0000","last_fullsized":"2023-11-28T04:07:43.078371+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":142,"reported_epoch":160,"state":"active+clean","last_fresh":"2023-11-28T04:07:39.988385+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:07:39.988385+0000","last_peered":"2023-11-28T04:07:39.988385+0000","last_clean":"2023-11-28T04:07:39.988385+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:07:39.988385+0000","last_undegraded":"2023-11-28T04:07:39.988385+0000","last_fullsized":"2023-11-28T04:07:39.988385+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575745,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303136,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068704,"statfs":{"total":96636764160,"available":96326352896,"internally_reserved":0,"allocated":1990656,"data_stored":1001260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.379},"max":{"1min":0.72699999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45400000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.44},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.372}]},{"osd":3,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.41599999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48199999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.318},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.38300000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.875,"15min":0.875},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.53600000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575745,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.81899999999999995},"last":0.49099999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.752,"15min":0.80300000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.33800000000000002},"max":{"1min":0.68600000000000005,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.46999999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67900000000000005,"5min":0.79500000000000004,"15min":0.79600000000000004},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.78000000000000003,"15min":0.83099999999999996},"last":0.50900000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.38600000000000001},"max":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79800000000000004},"last":0.68799999999999994}]},{"osd":4,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.35199999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.79200000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.41799999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.45400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575744,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.41999999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34200000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36099999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.70099999999999996}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.71599999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.82099999999999995}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.66300000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575745,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.61199999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.309,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.54600000000000004,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56299999999999994,"5min":0.621,"15min":0.67300000000000004},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.77500000000000002},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78500000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.45800000000000002,"15min":0.42499999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70599999999999996},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.42599999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834575746,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307232,"kb_used_data":1944,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064608,"statfs":{"total":96636764160,"available":96322158592,"internally_reserved":0,"allocated":1990656,"data_stored":1001260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.625,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.48199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.249,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.626},"last":0.38500000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.37},{"interface":"front","average":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41599999999999998},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.39900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":0.93400000000000005},"last":0.52200000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40200000000000002},"max":{"1min":0.85899999999999999,"5min":1.048,"15min":1.048},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.496}]},{"osd":6,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.997},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.63500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575745,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307244,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064596,"statfs":{"total":96636764160,"available":96322146304,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.621,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.45000000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.39800000000000002},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.25700000000000001},"max":{"1min":0.59999999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.36599999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.67600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.33100000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.503,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.94799999999999995,"15min":1.085},"last":0.46999999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":1.1200000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.65900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.502}]},{"osd":6,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.145},"last":0.53300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.434},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.47099999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.52800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834575744,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":303148,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068692,"statfs":{"total":96636764160,"available":96326340608,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.54200000000000004}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.69399999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.622}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.68100000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.55800000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.52200000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.505},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.63600000000000001}]}]},{"osd":5,"up_from":13,"seq":55834575746,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307244,"kb_used_data":1956,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064596,"statfs":{"total":96636764160,"available":96322146304,"internally_reserved":0,"allocated":2002944,"data_stored":1007071,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77800000000000002},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"last":0.57999999999999996}]},{"osd":1,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59699999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.89400000000000002},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.749}]},{"osd":3,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.378},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.77000000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.69499999999999995,"15min":0.83799999999999997},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.84499999999999997},"last":0.83499999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.79000000000000004},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.90700000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.78500000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.81999999999999995},"last":0.871}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:07:47.794 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:07:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.941+0000 7feec49eb700 1 Processor -- start 2023-11-28T04:07:47.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.941+0000 7feec49eb700 1 -- start start 2023-11-28T04:07:47.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feec49eb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feec0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feec49eb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7feec00c1750 0x7feec00c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feec49eb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feec00c12c0 con 0x7feec0157660 2023-11-28T04:07:47.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feec49eb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feec00c06b0 con 0x7feec00c1750 2023-11-28T04:07:47.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feec49eb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c0990 con 0x7feec00c0180 2023-11-28T04:07:47.950 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebf7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7feec0157660 0x7feec01589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55894/0 (socket says 172.21.15.73:55894) 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebf7fe700 1 -- 172.21.15.73:0/1716525509 learned_addr learned my addr 172.21.15.73:0/1716525509 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebeffd700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feec0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebffff700 1 --2- 172.21.15.73:0/1716525509 >> v2:172.21.15.150:3300/0 conn(0x7feec00c1750 0x7feec00c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1296910794 0 0) 0x7feec00c12c0 con 0x7feec0157660 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feea80033f0 con 0x7feec0157660 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 110812665 0 0) 0x7feea80033f0 con 0x7feec0157660 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 msgr2=0x7feec0157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feec0157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 >> v2:172.21.15.150:3300/0 conn(0x7feec00c1750 msgr2=0x7feec00c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 --2- 172.21.15.73:0/1716525509 >> v2:172.21.15.150:3300/0 conn(0x7feec00c1750 0x7feec00c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00c0c70 con 0x7feec0157660 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7feeb0001470 con 0x7feec0157660 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 >> v1:172.21.15.73:6789/0 conn(0x7feec0157660 legacy=0x7feec01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.942+0000 7feebe7fc700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80038e0 0x7feea8003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feea8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea8004b80 0x7feea8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c06b0 con 0x7feea80038e0 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c0990 con 0x7feea8004b80 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feea800b8e0 con 0x7feec00c0180 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80038e0 0x7feea8003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebffff700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea8004b80 0x7feea8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebeffd700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feea8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 -- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 msgr2=0x7feea8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feec00c0180 0x7feea8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 -- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea8004b80 msgr2=0x7feea8005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:47.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feea8004b80 0x7feea8005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 -- 172.21.15.73:0/1716525509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feea80033f0 con 0x7feea80038e0 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebf7fe700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80038e0 0x7feea8003d00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7feeb000c210 tx=0x7feeb0001e90).ready entity=mon.0 client_cookie=60b4b4c4ecbe79bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb0007a60 con 0x7feea80038e0 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb000cae0 con 0x7feea80038e0 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.943+0000 7feebe7fc700 1 -- 172.21.15.73:0/1716525509 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb000ce60 con 0x7feea80038e0 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 -- 172.21.15.73:0/1716525509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80038e0 msgr2=0x7feea8003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 --2- 172.21.15.73:0/1716525509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feea80038e0 0x7feea8003d00 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 rx=0x7feeb000c210 tx=0x7feeb0001e90).stop 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feebffff700 1 -- 172.21.15.73:0/1716525509 reap_dead start 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 -- 172.21.15.73:0/1716525509 shutdown_connections 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 -- 172.21.15.73:0/1716525509 >> 172.21.15.73:0/1716525509 conn(0x7feec00cb000 msgr2=0x7feec00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 -- 172.21.15.73:0/1716525509 shutdown_connections 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.944+0000 7feec49eb700 1 -- 172.21.15.73:0/1716525509 wait complete. 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.945+0000 7feec49eb700 1 Processor -- start 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.945+0000 7feec49eb700 1 -- start start 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea80038e0 0x7feec0063ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 0x7feec00ac730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c44b0 con 0x7feec00acc20 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feec00c4020 con 0x7feec014fd40 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feec00ad080 con 0x7feea80038e0 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58646/0 (socket says 172.21.15.73:58646) 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 -- 172.21.15.73:0/3806903898 learned_addr learned my addr 172.21.15.73:0/3806903898 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:47.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebf7fe700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea80038e0 0x7feec0063ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 -- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea80038e0 msgr2=0x7feec0063ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea80038e0 0x7feec0063ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebeffd700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 0x7feec00ac730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 -- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 msgr2=0x7feec00ac730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 0x7feec00ac730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 -- 172.21.15.73:0/3806903898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feec00c3410 con 0x7feec00acc20 2023-11-28T04:07:47.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebeffd700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 0x7feec00ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feebffff700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7feec0084680 tx=0x7feeb800b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb800e070 con 0x7feec00acc20 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeb8011bd0 con 0x7feec00acc20 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feec00c36f0 con 0x7feec00acc20 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeb8016450 con 0x7feec00acc20 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.946+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feec00c3cb0 con 0x7feec00acc20 2023-11-28T04:07:47.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.947+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feec006c7c0 con 0x7feec00acc20 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.948+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7feeb8016840 con 0x7feec00acc20 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.948+0000 7feea77fe700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 0x7feeac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.948+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7feeb807a640 con 0x7feec00acc20 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.948+0000 7feebf7fe700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 0x7feeac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.948+0000 7feebf7fe700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 0x7feeac061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feeb0007eb0 tx=0x7feeb0015000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:47.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:47.952+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7feeb804b9c0 con 0x7feec00acc20 2023-11-28T04:07:48.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.111+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feec00c0f50 con 0x7feeac05f150 2023-11-28T04:07:48.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.120+0000 7feea77fe700 1 -- 172.21.15.73:0/3806903898 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362037 (secure 0 0 0) 0x7feec00c0f50 con 0x7feeac05f150 2023-11-28T04:07:48.126 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 msgr2=0x7feeac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 0x7feeac061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7feeb0007eb0 tx=0x7feeb0015000).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 msgr2=0x7feec00c2120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 rx=0x7feec0084680 tx=0x7feeb800b040).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 shutdown_connections 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7feeac05f150 0x7feeac061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7feec00acc20 0x7feec00c2120 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7feea80038e0 0x7feec0063ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 --2- 172.21.15.73:0/3806903898 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7feec014fd40 0x7feec00ac730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.124+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 >> 172.21.15.73:0/3806903898 conn(0x7feec00cb000 msgr2=0x7feec00648d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:48.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.125+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 shutdown_connections 2023-11-28T04:07:48.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.125+0000 7feec49eb700 1 -- 172.21.15.73:0/3806903898 wait complete. 2023-11-28T04:07:48.130 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:07:48.138 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-28T04:07:48.057344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437872,"kb_used_data":16048,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752536848,"statfs":{"total":773094113280,"available":770597732352,"internally_reserved":0,"allocated":16433152,"data_stored":8248628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5035976,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-521,"num_write_kb":-5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-960,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995164"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480557+0000","last_change":"2023-11-28T04:07:44.477800+0000","last_active":"2023-11-28T04:07:44.480557+0000","last_peered":"2023-11-28T04:07:44.480557+0000","last_clean":"2023-11-28T04:07:44.480557+0000","last_became_active":"2023-11-28T04:07:44.477512+0000","last_became_peered":"2023-11-28T04:07:44.477512+0000","last_unstale":"2023-11-28T04:07:44.480557+0000","last_undegraded":"2023-11-28T04:07:44.480557+0000","last_fullsized":"2023-11-28T04:07:44.480557+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"164'3","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483535+0000","last_change":"2023-11-28T04:07:44.113370+0000","last_active":"2023-11-28T04:07:44.483535+0000","last_peered":"2023-11-28T04:07:44.483535+0000","last_clean":"2023-11-28T04:07:44.483535+0000","last_became_active":"2023-11-28T04:07:44.113093+0000","last_became_peered":"2023-11-28T04:07:44.113093+0000","last_unstale":"2023-11-28T04:07:44.483535+0000","last_undegraded":"2023-11-28T04:07:44.483535+0000","last_fullsized":"2023-11-28T04:07:44.483535+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491520+0000","last_change":"2023-11-28T04:07:44.491520+0000","last_active":"2023-11-28T04:07:44.491520+0000","last_peered":"2023-11-28T04:07:44.491520+0000","last_clean":"2023-11-28T04:07:44.491520+0000","last_became_active":"2023-11-28T04:07:44.491268+0000","last_became_peered":"2023-11-28T04:07:44.491268+0000","last_unstale":"2023-11-28T04:07:44.491520+0000","last_undegraded":"2023-11-28T04:07:44.491520+0000","last_fullsized":"2023-11-28T04:07:44.491520+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.115033+0000","last_change":"2023-11-28T04:07:44.111634+0000","last_active":"2023-11-28T04:07:44.115033+0000","last_peered":"2023-11-28T04:07:44.115033+0000","last_clean":"2023-11-28T04:07:44.115033+0000","last_became_active":"2023-11-28T04:07:44.111156+0000","last_became_peered":"2023-11-28T04:07:44.111156+0000","last_unstale":"2023-11-28T04:07:44.115033+0000","last_undegraded":"2023-11-28T04:07:44.115033+0000","last_fullsized":"2023-11-28T04:07:44.115033+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489125+0000","last_change":"2023-11-28T04:07:44.489125+0000","last_active":"2023-11-28T04:07:44.489125+0000","last_peered":"2023-11-28T04:07:44.489125+0000","last_clean":"2023-11-28T04:07:44.489125+0000","last_became_active":"2023-11-28T04:07:44.488874+0000","last_became_peered":"2023-11-28T04:07:44.488874+0000","last_unstale":"2023-11-28T04:07:44.489125+0000","last_undegraded":"2023-11-28T04:07:44.489125+0000","last_fullsized":"2023-11-28T04:07:44.489125+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467036+0000","last_change":"2023-11-28T04:07:44.106241+0000","last_active":"2023-11-28T04:07:44.467036+0000","last_peered":"2023-11-28T04:07:44.467036+0000","last_clean":"2023-11-28T04:07:44.467036+0000","last_became_active":"2023-11-28T04:07:44.105742+0000","last_became_peered":"2023-11-28T04:07:44.105742+0000","last_unstale":"2023-11-28T04:07:44.467036+0000","last_undegraded":"2023-11-28T04:07:44.467036+0000","last_fullsized":"2023-11-28T04:07:44.467036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479609+0000","last_change":"2023-11-28T04:07:44.477574+0000","last_active":"2023-11-28T04:07:44.479609+0000","last_peered":"2023-11-28T04:07:44.479609+0000","last_clean":"2023-11-28T04:07:44.479609+0000","last_became_active":"2023-11-28T04:07:44.477313+0000","last_became_peered":"2023-11-28T04:07:44.477313+0000","last_unstale":"2023-11-28T04:07:44.479609+0000","last_undegraded":"2023-11-28T04:07:44.479609+0000","last_fullsized":"2023-11-28T04:07:44.479609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482545+0000","last_change":"2023-11-28T04:07:44.108761+0000","last_active":"2023-11-28T04:07:44.482545+0000","last_peered":"2023-11-28T04:07:44.482545+0000","last_clean":"2023-11-28T04:07:44.482545+0000","last_became_active":"2023-11-28T04:07:44.108511+0000","last_became_peered":"2023-11-28T04:07:44.108511+0000","last_unstale":"2023-11-28T04:07:44.482545+0000","last_undegraded":"2023-11-28T04:07:44.482545+0000","last_fullsized":"2023-11-28T04:07:44.482545+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494783+0000","last_change":"2023-11-28T04:07:44.489084+0000","last_active":"2023-11-28T04:07:44.494783+0000","last_peered":"2023-11-28T04:07:44.494783+0000","last_clean":"2023-11-28T04:07:44.494783+0000","last_became_active":"2023-11-28T04:07:44.488814+0000","last_became_peered":"2023-11-28T04:07:44.488814+0000","last_unstale":"2023-11-28T04:07:44.494783+0000","last_undegraded":"2023-11-28T04:07:44.494783+0000","last_fullsized":"2023-11-28T04:07:44.494783+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470962+0000","last_change":"2023-11-28T04:07:44.112289+0000","last_active":"2023-11-28T04:07:44.470962+0000","last_peered":"2023-11-28T04:07:44.470962+0000","last_clean":"2023-11-28T04:07:44.470962+0000","last_became_active":"2023-11-28T04:07:44.111696+0000","last_became_peered":"2023-11-28T04:07:44.111696+0000","last_unstale":"2023-11-28T04:07:44.470962+0000","last_undegraded":"2023-11-28T04:07:44.470962+0000","last_fullsized":"2023-11-28T04:07:44.470962+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489146+0000","last_change":"2023-11-28T04:07:44.476209+0000","last_active":"2023-11-28T04:07:44.489146+0000","last_peered":"2023-11-28T04:07:44.489146+0000","last_clean":"2023-11-28T04:07:44.489146+0000","last_became_active":"2023-11-28T04:07:44.475591+0000","last_became_peered":"2023-11-28T04:07:44.475591+0000","last_unstale":"2023-11-28T04:07:44.489146+0000","last_undegraded":"2023-11-28T04:07:44.489146+0000","last_fullsized":"2023-11-28T04:07:44.489146+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3a","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478528+0000","last_change":"2023-11-28T04:07:44.109581+0000","last_active":"2023-11-28T04:07:44.478528+0000","last_peered":"2023-11-28T04:07:44.478528+0000","last_clean":"2023-11-28T04:07:44.478528+0000","last_became_active":"2023-11-28T04:07:44.109244+0000","last_became_peered":"2023-11-28T04:07:44.109244+0000","last_unstale":"2023-11-28T04:07:44.478528+0000","last_undegraded":"2023-11-28T04:07:44.478528+0000","last_fullsized":"2023-11-28T04:07:44.478528+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484728+0000","last_change":"2023-11-28T04:07:44.480799+0000","last_active":"2023-11-28T04:07:44.484728+0000","last_peered":"2023-11-28T04:07:44.484728+0000","last_clean":"2023-11-28T04:07:44.484728+0000","last_became_active":"2023-11-28T04:07:44.480511+0000","last_became_peered":"2023-11-28T04:07:44.480511+0000","last_unstale":"2023-11-28T04:07:44.484728+0000","last_undegraded":"2023-11-28T04:07:44.484728+0000","last_fullsized":"2023-11-28T04:07:44.484728+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470212+0000","last_change":"2023-11-28T04:07:44.106504+0000","last_active":"2023-11-28T04:07:44.470212+0000","last_peered":"2023-11-28T04:07:44.470212+0000","last_clean":"2023-11-28T04:07:44.470212+0000","last_became_active":"2023-11-28T04:07:44.106202+0000","last_became_peered":"2023-11-28T04:07:44.106202+0000","last_unstale":"2023-11-28T04:07:44.470212+0000","last_undegraded":"2023-11-28T04:07:44.470212+0000","last_fullsized":"2023-11-28T04:07:44.470212+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482773+0000","last_change":"2023-11-28T04:07:44.482773+0000","last_active":"2023-11-28T04:07:44.482773+0000","last_peered":"2023-11-28T04:07:44.482773+0000","last_clean":"2023-11-28T04:07:44.482773+0000","last_became_active":"2023-11-28T04:07:44.482483+0000","last_became_peered":"2023-11-28T04:07:44.482483+0000","last_unstale":"2023-11-28T04:07:44.482773+0000","last_undegraded":"2023-11-28T04:07:44.482773+0000","last_fullsized":"2023-11-28T04:07:44.482773+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466893+0000","last_change":"2023-11-28T04:07:44.110221+0000","last_active":"2023-11-28T04:07:44.466893+0000","last_peered":"2023-11-28T04:07:44.466893+0000","last_clean":"2023-11-28T04:07:44.466893+0000","last_became_active":"2023-11-28T04:07:44.109739+0000","last_became_peered":"2023-11-28T04:07:44.109739+0000","last_unstale":"2023-11-28T04:07:44.466893+0000","last_undegraded":"2023-11-28T04:07:44.466893+0000","last_fullsized":"2023-11-28T04:07:44.466893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481534+0000","last_change":"2023-11-28T04:07:44.481534+0000","last_active":"2023-11-28T04:07:44.481534+0000","last_peered":"2023-11-28T04:07:44.481534+0000","last_clean":"2023-11-28T04:07:44.481534+0000","last_became_active":"2023-11-28T04:07:44.481272+0000","last_became_peered":"2023-11-28T04:07:44.481272+0000","last_unstale":"2023-11-28T04:07:44.481534+0000","last_undegraded":"2023-11-28T04:07:44.481534+0000","last_fullsized":"2023-11-28T04:07:44.481534+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467360+0000","last_change":"2023-11-28T04:07:44.115088+0000","last_active":"2023-11-28T04:07:44.467360+0000","last_peered":"2023-11-28T04:07:44.467360+0000","last_clean":"2023-11-28T04:07:44.467360+0000","last_became_active":"2023-11-28T04:07:44.114010+0000","last_became_peered":"2023-11-28T04:07:44.114010+0000","last_unstale":"2023-11-28T04:07:44.467360+0000","last_undegraded":"2023-11-28T04:07:44.467360+0000","last_fullsized":"2023-11-28T04:07:44.467360+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487403+0000","last_change":"2023-11-28T04:07:44.487403+0000","last_active":"2023-11-28T04:07:44.487403+0000","last_peered":"2023-11-28T04:07:44.487403+0000","last_clean":"2023-11-28T04:07:44.487403+0000","last_became_active":"2023-11-28T04:07:44.487107+0000","last_became_peered":"2023-11-28T04:07:44.487107+0000","last_unstale":"2023-11-28T04:07:44.487403+0000","last_undegraded":"2023-11-28T04:07:44.487403+0000","last_fullsized":"2023-11-28T04:07:44.487403+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.115013+0000","last_change":"2023-11-28T04:07:44.111442+0000","last_active":"2023-11-28T04:07:44.115013+0000","last_peered":"2023-11-28T04:07:44.115013+0000","last_clean":"2023-11-28T04:07:44.115013+0000","last_became_active":"2023-11-28T04:07:44.110824+0000","last_became_peered":"2023-11-28T04:07:44.110824+0000","last_unstale":"2023-11-28T04:07:44.115013+0000","last_undegraded":"2023-11-28T04:07:44.115013+0000","last_fullsized":"2023-11-28T04:07:44.115013+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485088+0000","last_change":"2023-11-28T04:07:44.485088+0000","last_active":"2023-11-28T04:07:44.485088+0000","last_peered":"2023-11-28T04:07:44.485088+0000","last_clean":"2023-11-28T04:07:44.485088+0000","last_became_active":"2023-11-28T04:07:44.484826+0000","last_became_peered":"2023-11-28T04:07:44.484826+0000","last_unstale":"2023-11-28T04:07:44.485088+0000","last_undegraded":"2023-11-28T04:07:44.485088+0000","last_fullsized":"2023-11-28T04:07:44.485088+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.116491+0000","last_change":"2023-11-28T04:07:44.110072+0000","last_active":"2023-11-28T04:07:44.116491+0000","last_peered":"2023-11-28T04:07:44.116491+0000","last_clean":"2023-11-28T04:07:44.116491+0000","last_became_active":"2023-11-28T04:07:44.109819+0000","last_became_peered":"2023-11-28T04:07:44.109819+0000","last_unstale":"2023-11-28T04:07:44.116491+0000","last_undegraded":"2023-11-28T04:07:44.116491+0000","last_fullsized":"2023-11-28T04:07:44.116491+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489777+0000","last_change":"2023-11-28T04:07:44.489777+0000","last_active":"2023-11-28T04:07:44.489777+0000","last_peered":"2023-11-28T04:07:44.489777+0000","last_clean":"2023-11-28T04:07:44.489777+0000","last_became_active":"2023-11-28T04:07:44.489547+0000","last_became_peered":"2023-11-28T04:07:44.489547+0000","last_unstale":"2023-11-28T04:07:44.489777+0000","last_undegraded":"2023-11-28T04:07:44.489777+0000","last_fullsized":"2023-11-28T04:07:44.489777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.495253+0000","last_change":"2023-11-28T04:07:44.113703+0000","last_active":"2023-11-28T04:07:44.495253+0000","last_peered":"2023-11-28T04:07:44.495253+0000","last_clean":"2023-11-28T04:07:44.495253+0000","last_became_active":"2023-11-28T04:07:44.113417+0000","last_became_peered":"2023-11-28T04:07:44.113417+0000","last_unstale":"2023-11-28T04:07:44.495253+0000","last_undegraded":"2023-11-28T04:07:44.495253+0000","last_fullsized":"2023-11-28T04:07:44.495253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481988+0000","last_change":"2023-11-28T04:07:44.481988+0000","last_active":"2023-11-28T04:07:44.481988+0000","last_peered":"2023-11-28T04:07:44.481988+0000","last_clean":"2023-11-28T04:07:44.481988+0000","last_became_active":"2023-11-28T04:07:44.475932+0000","last_became_peered":"2023-11-28T04:07:44.475932+0000","last_unstale":"2023-11-28T04:07:44.481988+0000","last_undegraded":"2023-11-28T04:07:44.481988+0000","last_fullsized":"2023-11-28T04:07:44.481988+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.33","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468554+0000","last_change":"2023-11-28T04:07:44.104938+0000","last_active":"2023-11-28T04:07:44.468554+0000","last_peered":"2023-11-28T04:07:44.468554+0000","last_clean":"2023-11-28T04:07:44.468554+0000","last_became_active":"2023-11-28T04:07:44.102437+0000","last_became_peered":"2023-11-28T04:07:44.102437+0000","last_unstale":"2023-11-28T04:07:44.468554+0000","last_undegraded":"2023-11-28T04:07:44.468554+0000","last_fullsized":"2023-11-28T04:07:44.468554+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494657+0000","last_change":"2023-11-28T04:07:44.488430+0000","last_active":"2023-11-28T04:07:44.494657+0000","last_peered":"2023-11-28T04:07:44.494657+0000","last_clean":"2023-11-28T04:07:44.494657+0000","last_became_active":"2023-11-28T04:07:44.488148+0000","last_became_peered":"2023-11-28T04:07:44.488148+0000","last_unstale":"2023-11-28T04:07:44.494657+0000","last_undegraded":"2023-11-28T04:07:44.494657+0000","last_fullsized":"2023-11-28T04:07:44.494657+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114904+0000","last_change":"2023-11-28T04:07:44.110316+0000","last_active":"2023-11-28T04:07:44.114904+0000","last_peered":"2023-11-28T04:07:44.114904+0000","last_clean":"2023-11-28T04:07:44.114904+0000","last_became_active":"2023-11-28T04:07:44.110028+0000","last_became_peered":"2023-11-28T04:07:44.110028+0000","last_unstale":"2023-11-28T04:07:44.114904+0000","last_undegraded":"2023-11-28T04:07:44.114904+0000","last_fullsized":"2023-11-28T04:07:44.114904+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475360+0000","last_change":"2023-11-28T04:07:44.475360+0000","last_active":"2023-11-28T04:07:44.475360+0000","last_peered":"2023-11-28T04:07:44.475360+0000","last_clean":"2023-11-28T04:07:44.475360+0000","last_became_active":"2023-11-28T04:07:44.475098+0000","last_became_peered":"2023-11-28T04:07:44.475098+0000","last_unstale":"2023-11-28T04:07:44.475360+0000","last_undegraded":"2023-11-28T04:07:44.475360+0000","last_fullsized":"2023-11-28T04:07:44.475360+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117436+0000","last_change":"2023-11-28T04:07:44.111087+0000","last_active":"2023-11-28T04:07:44.117436+0000","last_peered":"2023-11-28T04:07:44.117436+0000","last_clean":"2023-11-28T04:07:44.117436+0000","last_became_active":"2023-11-28T04:07:44.110400+0000","last_became_peered":"2023-11-28T04:07:44.110400+0000","last_unstale":"2023-11-28T04:07:44.117436+0000","last_undegraded":"2023-11-28T04:07:44.117436+0000","last_fullsized":"2023-11-28T04:07:44.117436+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481492+0000","last_change":"2023-11-28T04:07:44.481492+0000","last_active":"2023-11-28T04:07:44.481492+0000","last_peered":"2023-11-28T04:07:44.481492+0000","last_clean":"2023-11-28T04:07:44.481492+0000","last_became_active":"2023-11-28T04:07:44.481212+0000","last_became_peered":"2023-11-28T04:07:44.481212+0000","last_unstale":"2023-11-28T04:07:44.481492+0000","last_undegraded":"2023-11-28T04:07:44.481492+0000","last_fullsized":"2023-11-28T04:07:44.481492+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483951+0000","last_change":"2023-11-28T04:07:44.105325+0000","last_active":"2023-11-28T04:07:44.483951+0000","last_peered":"2023-11-28T04:07:44.483951+0000","last_clean":"2023-11-28T04:07:44.483951+0000","last_became_active":"2023-11-28T04:07:44.105081+0000","last_became_peered":"2023-11-28T04:07:44.105081+0000","last_unstale":"2023-11-28T04:07:44.483951+0000","last_undegraded":"2023-11-28T04:07:44.483951+0000","last_fullsized":"2023-11-28T04:07:44.483951+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484640+0000","last_change":"2023-11-28T04:07:44.484640+0000","last_active":"2023-11-28T04:07:44.484640+0000","last_peered":"2023-11-28T04:07:44.484640+0000","last_clean":"2023-11-28T04:07:44.484640+0000","last_became_active":"2023-11-28T04:07:44.484227+0000","last_became_peered":"2023-11-28T04:07:44.484227+0000","last_unstale":"2023-11-28T04:07:44.484640+0000","last_undegraded":"2023-11-28T04:07:44.484640+0000","last_fullsized":"2023-11-28T04:07:44.484640+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469618+0000","last_change":"2023-11-28T04:07:44.104353+0000","last_active":"2023-11-28T04:07:44.469618+0000","last_peered":"2023-11-28T04:07:44.469618+0000","last_clean":"2023-11-28T04:07:44.469618+0000","last_became_active":"2023-11-28T04:07:44.104058+0000","last_became_peered":"2023-11-28T04:07:44.104058+0000","last_unstale":"2023-11-28T04:07:44.469618+0000","last_undegraded":"2023-11-28T04:07:44.469618+0000","last_fullsized":"2023-11-28T04:07:44.469618+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492111+0000","last_change":"2023-11-28T04:07:44.487100+0000","last_active":"2023-11-28T04:07:44.492111+0000","last_peered":"2023-11-28T04:07:44.492111+0000","last_clean":"2023-11-28T04:07:44.492111+0000","last_became_active":"2023-11-28T04:07:44.486847+0000","last_became_peered":"2023-11-28T04:07:44.486847+0000","last_unstale":"2023-11-28T04:07:44.492111+0000","last_undegraded":"2023-11-28T04:07:44.492111+0000","last_fullsized":"2023-11-28T04:07:44.492111+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469178+0000","last_change":"2023-11-28T04:07:44.113850+0000","last_active":"2023-11-28T04:07:44.469178+0000","last_peered":"2023-11-28T04:07:44.469178+0000","last_clean":"2023-11-28T04:07:44.469178+0000","last_became_active":"2023-11-28T04:07:44.113616+0000","last_became_peered":"2023-11-28T04:07:44.113616+0000","last_unstale":"2023-11-28T04:07:44.469178+0000","last_undegraded":"2023-11-28T04:07:44.469178+0000","last_fullsized":"2023-11-28T04:07:44.469178+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483363+0000","last_change":"2023-11-28T04:07:44.483363+0000","last_active":"2023-11-28T04:07:44.483363+0000","last_peered":"2023-11-28T04:07:44.483363+0000","last_clean":"2023-11-28T04:07:44.483363+0000","last_became_active":"2023-11-28T04:07:44.483101+0000","last_became_peered":"2023-11-28T04:07:44.483101+0000","last_unstale":"2023-11-28T04:07:44.483363+0000","last_undegraded":"2023-11-28T04:07:44.483363+0000","last_fullsized":"2023-11-28T04:07:44.483363+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2d","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.462659+0000","last_change":"2023-11-28T04:07:44.104302+0000","last_active":"2023-11-28T04:07:44.462659+0000","last_peered":"2023-11-28T04:07:44.462659+0000","last_clean":"2023-11-28T04:07:44.462659+0000","last_became_active":"2023-11-28T04:07:44.103985+0000","last_became_peered":"2023-11-28T04:07:44.103985+0000","last_unstale":"2023-11-28T04:07:44.462659+0000","last_undegraded":"2023-11-28T04:07:44.462659+0000","last_fullsized":"2023-11-28T04:07:44.462659+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482460+0000","last_change":"2023-11-28T04:07:44.477456+0000","last_active":"2023-11-28T04:07:44.482460+0000","last_peered":"2023-11-28T04:07:44.482460+0000","last_clean":"2023-11-28T04:07:44.482460+0000","last_became_active":"2023-11-28T04:07:44.477166+0000","last_became_peered":"2023-11-28T04:07:44.477166+0000","last_unstale":"2023-11-28T04:07:44.482460+0000","last_undegraded":"2023-11-28T04:07:44.482460+0000","last_fullsized":"2023-11-28T04:07:44.482460+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465599+0000","last_change":"2023-11-28T04:07:44.112401+0000","last_active":"2023-11-28T04:07:44.465599+0000","last_peered":"2023-11-28T04:07:44.465599+0000","last_clean":"2023-11-28T04:07:44.465599+0000","last_became_active":"2023-11-28T04:07:44.112163+0000","last_became_peered":"2023-11-28T04:07:44.112163+0000","last_unstale":"2023-11-28T04:07:44.465599+0000","last_undegraded":"2023-11-28T04:07:44.465599+0000","last_fullsized":"2023-11-28T04:07:44.465599+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479617+0000","last_change":"2023-11-28T04:07:44.478586+0000","last_active":"2023-11-28T04:07:44.479617+0000","last_peered":"2023-11-28T04:07:44.479617+0000","last_clean":"2023-11-28T04:07:44.479617+0000","last_became_active":"2023-11-28T04:07:44.478269+0000","last_became_peered":"2023-11-28T04:07:44.478269+0000","last_unstale":"2023-11-28T04:07:44.479617+0000","last_undegraded":"2023-11-28T04:07:44.479617+0000","last_fullsized":"2023-11-28T04:07:44.479617+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468765+0000","last_change":"2023-11-28T04:07:44.112458+0000","last_active":"2023-11-28T04:07:44.468765+0000","last_peered":"2023-11-28T04:07:44.468765+0000","last_clean":"2023-11-28T04:07:44.468765+0000","last_became_active":"2023-11-28T04:07:44.112028+0000","last_became_peered":"2023-11-28T04:07:44.112028+0000","last_unstale":"2023-11-28T04:07:44.468765+0000","last_undegraded":"2023-11-28T04:07:44.468765+0000","last_fullsized":"2023-11-28T04:07:44.468765+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481030+0000","last_change":"2023-11-28T04:07:44.481030+0000","last_active":"2023-11-28T04:07:44.481030+0000","last_peered":"2023-11-28T04:07:44.481030+0000","last_clean":"2023-11-28T04:07:44.481030+0000","last_became_active":"2023-11-28T04:07:44.480689+0000","last_became_peered":"2023-11-28T04:07:44.480689+0000","last_unstale":"2023-11-28T04:07:44.481030+0000","last_undegraded":"2023-11-28T04:07:44.481030+0000","last_fullsized":"2023-11-28T04:07:44.481030+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.109658+0000","last_change":"2023-11-28T04:07:44.107204+0000","last_active":"2023-11-28T04:07:44.109658+0000","last_peered":"2023-11-28T04:07:44.109658+0000","last_clean":"2023-11-28T04:07:44.109658+0000","last_became_active":"2023-11-28T04:07:44.106921+0000","last_became_peered":"2023-11-28T04:07:44.106921+0000","last_unstale":"2023-11-28T04:07:44.109658+0000","last_undegraded":"2023-11-28T04:07:44.109658+0000","last_fullsized":"2023-11-28T04:07:44.109658+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483491+0000","last_change":"2023-11-28T04:07:44.480913+0000","last_active":"2023-11-28T04:07:44.483491+0000","last_peered":"2023-11-28T04:07:44.483491+0000","last_clean":"2023-11-28T04:07:44.483491+0000","last_became_active":"2023-11-28T04:07:44.480659+0000","last_became_peered":"2023-11-28T04:07:44.480659+0000","last_unstale":"2023-11-28T04:07:44.483491+0000","last_undegraded":"2023-11-28T04:07:44.483491+0000","last_fullsized":"2023-11-28T04:07:44.483491+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466068+0000","last_change":"2023-11-28T04:07:44.114355+0000","last_active":"2023-11-28T04:07:44.466068+0000","last_peered":"2023-11-28T04:07:44.466068+0000","last_clean":"2023-11-28T04:07:44.466068+0000","last_became_active":"2023-11-28T04:07:44.113617+0000","last_became_peered":"2023-11-28T04:07:44.113617+0000","last_unstale":"2023-11-28T04:07:44.466068+0000","last_undegraded":"2023-11-28T04:07:44.466068+0000","last_fullsized":"2023-11-28T04:07:44.466068+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481143+0000","last_change":"2023-11-28T04:07:44.481143+0000","last_active":"2023-11-28T04:07:44.481143+0000","last_peered":"2023-11-28T04:07:44.481143+0000","last_clean":"2023-11-28T04:07:44.481143+0000","last_became_active":"2023-11-28T04:07:44.480821+0000","last_became_peered":"2023-11-28T04:07:44.480821+0000","last_unstale":"2023-11-28T04:07:44.481143+0000","last_undegraded":"2023-11-28T04:07:44.481143+0000","last_fullsized":"2023-11-28T04:07:44.481143+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120644+0000","last_change":"2023-11-28T04:07:44.116709+0000","last_active":"2023-11-28T04:07:44.120644+0000","last_peered":"2023-11-28T04:07:44.120644+0000","last_clean":"2023-11-28T04:07:44.120644+0000","last_became_active":"2023-11-28T04:07:44.116378+0000","last_became_peered":"2023-11-28T04:07:44.116378+0000","last_unstale":"2023-11-28T04:07:44.120644+0000","last_undegraded":"2023-11-28T04:07:44.120644+0000","last_fullsized":"2023-11-28T04:07:44.120644+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488886+0000","last_change":"2023-11-28T04:07:44.488886+0000","last_active":"2023-11-28T04:07:44.488886+0000","last_peered":"2023-11-28T04:07:44.488886+0000","last_clean":"2023-11-28T04:07:44.488886+0000","last_became_active":"2023-11-28T04:07:44.488585+0000","last_became_peered":"2023-11-28T04:07:44.488585+0000","last_unstale":"2023-11-28T04:07:44.488886+0000","last_undegraded":"2023-11-28T04:07:44.488886+0000","last_fullsized":"2023-11-28T04:07:44.488886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114783+0000","last_change":"2023-11-28T04:07:44.109538+0000","last_active":"2023-11-28T04:07:44.114783+0000","last_peered":"2023-11-28T04:07:44.114783+0000","last_clean":"2023-11-28T04:07:44.114783+0000","last_became_active":"2023-11-28T04:07:44.109216+0000","last_became_peered":"2023-11-28T04:07:44.109216+0000","last_unstale":"2023-11-28T04:07:44.114783+0000","last_undegraded":"2023-11-28T04:07:44.114783+0000","last_fullsized":"2023-11-28T04:07:44.114783+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485937+0000","last_change":"2023-11-28T04:07:44.481886+0000","last_active":"2023-11-28T04:07:44.485937+0000","last_peered":"2023-11-28T04:07:44.485937+0000","last_clean":"2023-11-28T04:07:44.485937+0000","last_became_active":"2023-11-28T04:07:44.481610+0000","last_became_peered":"2023-11-28T04:07:44.481610+0000","last_unstale":"2023-11-28T04:07:44.485937+0000","last_undegraded":"2023-11-28T04:07:44.485937+0000","last_fullsized":"2023-11-28T04:07:44.485937+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468972+0000","last_change":"2023-11-28T04:07:44.110589+0000","last_active":"2023-11-28T04:07:44.468972+0000","last_peered":"2023-11-28T04:07:44.468972+0000","last_clean":"2023-11-28T04:07:44.468972+0000","last_became_active":"2023-11-28T04:07:44.110322+0000","last_became_peered":"2023-11-28T04:07:44.110322+0000","last_unstale":"2023-11-28T04:07:44.468972+0000","last_undegraded":"2023-11-28T04:07:44.468972+0000","last_fullsized":"2023-11-28T04:07:44.468972+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490904+0000","last_change":"2023-11-28T04:07:44.486310+0000","last_active":"2023-11-28T04:07:44.490904+0000","last_peered":"2023-11-28T04:07:44.490904+0000","last_clean":"2023-11-28T04:07:44.490904+0000","last_became_active":"2023-11-28T04:07:44.484189+0000","last_became_peered":"2023-11-28T04:07:44.484189+0000","last_unstale":"2023-11-28T04:07:44.490904+0000","last_undegraded":"2023-11-28T04:07:44.490904+0000","last_fullsized":"2023-11-28T04:07:44.490904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.116574+0000","last_change":"2023-11-28T04:07:44.104470+0000","last_active":"2023-11-28T04:07:44.116574+0000","last_peered":"2023-11-28T04:07:44.116574+0000","last_clean":"2023-11-28T04:07:44.116574+0000","last_became_active":"2023-11-28T04:07:44.104217+0000","last_became_peered":"2023-11-28T04:07:44.104217+0000","last_unstale":"2023-11-28T04:07:44.116574+0000","last_undegraded":"2023-11-28T04:07:44.116574+0000","last_fullsized":"2023-11-28T04:07:44.116574+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490105+0000","last_change":"2023-11-28T04:07:44.481275+0000","last_active":"2023-11-28T04:07:44.490105+0000","last_peered":"2023-11-28T04:07:44.490105+0000","last_clean":"2023-11-28T04:07:44.490105+0000","last_became_active":"2023-11-28T04:07:44.481006+0000","last_became_peered":"2023-11-28T04:07:44.481006+0000","last_unstale":"2023-11-28T04:07:44.490105+0000","last_undegraded":"2023-11-28T04:07:44.490105+0000","last_fullsized":"2023-11-28T04:07:44.490105+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470540+0000","last_change":"2023-11-28T04:07:44.111555+0000","last_active":"2023-11-28T04:07:44.470540+0000","last_peered":"2023-11-28T04:07:44.470540+0000","last_clean":"2023-11-28T04:07:44.470540+0000","last_became_active":"2023-11-28T04:07:44.111268+0000","last_became_peered":"2023-11-28T04:07:44.111268+0000","last_unstale":"2023-11-28T04:07:44.470540+0000","last_undegraded":"2023-11-28T04:07:44.470540+0000","last_fullsized":"2023-11-28T04:07:44.470540+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.477017+0000","last_change":"2023-11-28T04:07:44.477017+0000","last_active":"2023-11-28T04:07:44.477017+0000","last_peered":"2023-11-28T04:07:44.477017+0000","last_clean":"2023-11-28T04:07:44.477017+0000","last_became_active":"2023-11-28T04:07:44.476741+0000","last_became_peered":"2023-11-28T04:07:44.476741+0000","last_unstale":"2023-11-28T04:07:44.477017+0000","last_undegraded":"2023-11-28T04:07:44.477017+0000","last_fullsized":"2023-11-28T04:07:44.477017+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117327+0000","last_change":"2023-11-28T04:07:44.110023+0000","last_active":"2023-11-28T04:07:44.117327+0000","last_peered":"2023-11-28T04:07:44.117327+0000","last_clean":"2023-11-28T04:07:44.117327+0000","last_became_active":"2023-11-28T04:07:44.109750+0000","last_became_peered":"2023-11-28T04:07:44.109750+0000","last_unstale":"2023-11-28T04:07:44.117327+0000","last_undegraded":"2023-11-28T04:07:44.117327+0000","last_fullsized":"2023-11-28T04:07:44.117327+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488305+0000","last_change":"2023-11-28T04:07:44.488305+0000","last_active":"2023-11-28T04:07:44.488305+0000","last_peered":"2023-11-28T04:07:44.488305+0000","last_clean":"2023-11-28T04:07:44.488305+0000","last_became_active":"2023-11-28T04:07:44.487406+0000","last_became_peered":"2023-11-28T04:07:44.487406+0000","last_unstale":"2023-11-28T04:07:44.488305+0000","last_undegraded":"2023-11-28T04:07:44.488305+0000","last_fullsized":"2023-11-28T04:07:44.488305+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117388+0000","last_change":"2023-11-28T04:07:44.110617+0000","last_active":"2023-11-28T04:07:44.117388+0000","last_peered":"2023-11-28T04:07:44.117388+0000","last_clean":"2023-11-28T04:07:44.117388+0000","last_became_active":"2023-11-28T04:07:44.110327+0000","last_became_peered":"2023-11-28T04:07:44.110327+0000","last_unstale":"2023-11-28T04:07:44.117388+0000","last_undegraded":"2023-11-28T04:07:44.117388+0000","last_fullsized":"2023-11-28T04:07:44.117388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490379+0000","last_change":"2023-11-28T04:07:44.483626+0000","last_active":"2023-11-28T04:07:44.490379+0000","last_peered":"2023-11-28T04:07:44.490379+0000","last_clean":"2023-11-28T04:07:44.490379+0000","last_became_active":"2023-11-28T04:07:44.483359+0000","last_became_peered":"2023-11-28T04:07:44.483359+0000","last_unstale":"2023-11-28T04:07:44.490379+0000","last_undegraded":"2023-11-28T04:07:44.490379+0000","last_fullsized":"2023-11-28T04:07:44.490379+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121180+0000","last_change":"2023-11-28T04:07:44.118163+0000","last_active":"2023-11-28T04:07:44.121180+0000","last_peered":"2023-11-28T04:07:44.121180+0000","last_clean":"2023-11-28T04:07:44.121180+0000","last_became_active":"2023-11-28T04:07:44.117786+0000","last_became_peered":"2023-11-28T04:07:44.117786+0000","last_unstale":"2023-11-28T04:07:44.121180+0000","last_undegraded":"2023-11-28T04:07:44.121180+0000","last_fullsized":"2023-11-28T04:07:44.121180+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489230+0000","last_change":"2023-11-28T04:07:44.476435+0000","last_active":"2023-11-28T04:07:44.489230+0000","last_peered":"2023-11-28T04:07:44.489230+0000","last_clean":"2023-11-28T04:07:44.489230+0000","last_became_active":"2023-11-28T04:07:44.476148+0000","last_became_peered":"2023-11-28T04:07:44.476148+0000","last_unstale":"2023-11-28T04:07:44.489230+0000","last_undegraded":"2023-11-28T04:07:44.489230+0000","last_fullsized":"2023-11-28T04:07:44.489230+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468541+0000","last_change":"2023-11-28T04:07:44.102476+0000","last_active":"2023-11-28T04:07:44.468541+0000","last_peered":"2023-11-28T04:07:44.468541+0000","last_clean":"2023-11-28T04:07:44.468541+0000","last_became_active":"2023-11-28T04:07:44.102178+0000","last_became_peered":"2023-11-28T04:07:44.102178+0000","last_unstale":"2023-11-28T04:07:44.468541+0000","last_undegraded":"2023-11-28T04:07:44.468541+0000","last_fullsized":"2023-11-28T04:07:44.468541+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487891+0000","last_change":"2023-11-28T04:07:44.483294+0000","last_active":"2023-11-28T04:07:44.487891+0000","last_peered":"2023-11-28T04:07:44.487891+0000","last_clean":"2023-11-28T04:07:44.487891+0000","last_became_active":"2023-11-28T04:07:44.483044+0000","last_became_peered":"2023-11-28T04:07:44.483044+0000","last_unstale":"2023-11-28T04:07:44.487891+0000","last_undegraded":"2023-11-28T04:07:44.487891+0000","last_fullsized":"2023-11-28T04:07:44.487891+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489393+0000","last_change":"2023-11-28T04:07:44.110602+0000","last_active":"2023-11-28T04:07:44.489393+0000","last_peered":"2023-11-28T04:07:44.489393+0000","last_clean":"2023-11-28T04:07:44.489393+0000","last_became_active":"2023-11-28T04:07:44.110301+0000","last_became_peered":"2023-11-28T04:07:44.110301+0000","last_unstale":"2023-11-28T04:07:44.489393+0000","last_undegraded":"2023-11-28T04:07:44.489393+0000","last_fullsized":"2023-11-28T04:07:44.489393+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482085+0000","last_change":"2023-11-28T04:07:44.482085+0000","last_active":"2023-11-28T04:07:44.482085+0000","last_peered":"2023-11-28T04:07:44.482085+0000","last_clean":"2023-11-28T04:07:44.482085+0000","last_became_active":"2023-11-28T04:07:44.481145+0000","last_became_peered":"2023-11-28T04:07:44.481145+0000","last_unstale":"2023-11-28T04:07:44.482085+0000","last_undegraded":"2023-11-28T04:07:44.482085+0000","last_fullsized":"2023-11-28T04:07:44.482085+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479304+0000","last_change":"2023-11-28T04:07:44.113764+0000","last_active":"2023-11-28T04:07:44.479304+0000","last_peered":"2023-11-28T04:07:44.479304+0000","last_clean":"2023-11-28T04:07:44.479304+0000","last_became_active":"2023-11-28T04:07:44.113322+0000","last_became_peered":"2023-11-28T04:07:44.113322+0000","last_unstale":"2023-11-28T04:07:44.479304+0000","last_undegraded":"2023-11-28T04:07:44.479304+0000","last_fullsized":"2023-11-28T04:07:44.479304+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492088+0000","last_change":"2023-11-28T04:07:44.485956+0000","last_active":"2023-11-28T04:07:44.492088+0000","last_peered":"2023-11-28T04:07:44.492088+0000","last_clean":"2023-11-28T04:07:44.492088+0000","last_became_active":"2023-11-28T04:07:44.485694+0000","last_became_peered":"2023-11-28T04:07:44.485694+0000","last_unstale":"2023-11-28T04:07:44.492088+0000","last_undegraded":"2023-11-28T04:07:44.492088+0000","last_fullsized":"2023-11-28T04:07:44.492088+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487587+0000","last_change":"2023-11-28T04:07:44.110797+0000","last_active":"2023-11-28T04:07:44.487587+0000","last_peered":"2023-11-28T04:07:44.487587+0000","last_clean":"2023-11-28T04:07:44.487587+0000","last_became_active":"2023-11-28T04:07:44.110494+0000","last_became_peered":"2023-11-28T04:07:44.110494+0000","last_unstale":"2023-11-28T04:07:44.487587+0000","last_undegraded":"2023-11-28T04:07:44.487587+0000","last_fullsized":"2023-11-28T04:07:44.487587+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483810+0000","last_change":"2023-11-28T04:07:44.483810+0000","last_active":"2023-11-28T04:07:44.483810+0000","last_peered":"2023-11-28T04:07:44.483810+0000","last_clean":"2023-11-28T04:07:44.483810+0000","last_became_active":"2023-11-28T04:07:44.483553+0000","last_became_peered":"2023-11-28T04:07:44.483553+0000","last_unstale":"2023-11-28T04:07:44.483810+0000","last_undegraded":"2023-11-28T04:07:44.483810+0000","last_fullsized":"2023-11-28T04:07:44.483810+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120789+0000","last_change":"2023-11-28T04:07:44.115588+0000","last_active":"2023-11-28T04:07:44.120789+0000","last_peered":"2023-11-28T04:07:44.120789+0000","last_clean":"2023-11-28T04:07:44.120789+0000","last_became_active":"2023-11-28T04:07:44.115367+0000","last_became_peered":"2023-11-28T04:07:44.115367+0000","last_unstale":"2023-11-28T04:07:44.120789+0000","last_undegraded":"2023-11-28T04:07:44.120789+0000","last_fullsized":"2023-11-28T04:07:44.120789+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485918+0000","last_change":"2023-11-28T04:07:44.481858+0000","last_active":"2023-11-28T04:07:44.485918+0000","last_peered":"2023-11-28T04:07:44.485918+0000","last_clean":"2023-11-28T04:07:44.485918+0000","last_became_active":"2023-11-28T04:07:44.481571+0000","last_became_peered":"2023-11-28T04:07:44.481571+0000","last_unstale":"2023-11-28T04:07:44.485918+0000","last_undegraded":"2023-11-28T04:07:44.485918+0000","last_fullsized":"2023-11-28T04:07:44.485918+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.471609+0000","last_change":"2023-11-28T04:07:44.105489+0000","last_active":"2023-11-28T04:07:44.471609+0000","last_peered":"2023-11-28T04:07:44.471609+0000","last_clean":"2023-11-28T04:07:44.471609+0000","last_became_active":"2023-11-28T04:07:44.105080+0000","last_became_peered":"2023-11-28T04:07:44.105080+0000","last_unstale":"2023-11-28T04:07:44.471609+0000","last_undegraded":"2023-11-28T04:07:44.471609+0000","last_fullsized":"2023-11-28T04:07:44.471609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480691+0000","last_change":"2023-11-28T04:07:44.477905+0000","last_active":"2023-11-28T04:07:44.480691+0000","last_peered":"2023-11-28T04:07:44.480691+0000","last_clean":"2023-11-28T04:07:44.480691+0000","last_became_active":"2023-11-28T04:07:44.477545+0000","last_became_peered":"2023-11-28T04:07:44.477545+0000","last_unstale":"2023-11-28T04:07:44.480691+0000","last_undegraded":"2023-11-28T04:07:44.480691+0000","last_fullsized":"2023-11-28T04:07:44.480691+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.464896+0000","last_change":"2023-11-28T04:07:44.113284+0000","last_active":"2023-11-28T04:07:44.464896+0000","last_peered":"2023-11-28T04:07:44.464896+0000","last_clean":"2023-11-28T04:07:44.464896+0000","last_became_active":"2023-11-28T04:07:44.113040+0000","last_became_peered":"2023-11-28T04:07:44.113040+0000","last_unstale":"2023-11-28T04:07:44.464896+0000","last_undegraded":"2023-11-28T04:07:44.464896+0000","last_fullsized":"2023-11-28T04:07:44.464896+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486435+0000","last_change":"2023-11-28T04:07:44.485576+0000","last_active":"2023-11-28T04:07:44.486435+0000","last_peered":"2023-11-28T04:07:44.486435+0000","last_clean":"2023-11-28T04:07:44.486435+0000","last_became_active":"2023-11-28T04:07:44.485296+0000","last_became_peered":"2023-11-28T04:07:44.485296+0000","last_unstale":"2023-11-28T04:07:44.486435+0000","last_undegraded":"2023-11-28T04:07:44.486435+0000","last_fullsized":"2023-11-28T04:07:44.486435+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113386+0000","last_change":"2023-11-28T04:07:44.109213+0000","last_active":"2023-11-28T04:07:44.113386+0000","last_peered":"2023-11-28T04:07:44.113386+0000","last_clean":"2023-11-28T04:07:44.113386+0000","last_became_active":"2023-11-28T04:07:44.108530+0000","last_became_peered":"2023-11-28T04:07:44.108530+0000","last_unstale":"2023-11-28T04:07:44.113386+0000","last_undegraded":"2023-11-28T04:07:44.113386+0000","last_fullsized":"2023-11-28T04:07:44.113386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479590+0000","last_change":"2023-11-28T04:07:44.479590+0000","last_active":"2023-11-28T04:07:44.479590+0000","last_peered":"2023-11-28T04:07:44.479590+0000","last_clean":"2023-11-28T04:07:44.479590+0000","last_became_active":"2023-11-28T04:07:44.479336+0000","last_became_peered":"2023-11-28T04:07:44.479336+0000","last_unstale":"2023-11-28T04:07:44.479590+0000","last_undegraded":"2023-11-28T04:07:44.479590+0000","last_fullsized":"2023-11-28T04:07:44.479590+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466934+0000","last_change":"2023-11-28T04:07:44.108560+0000","last_active":"2023-11-28T04:07:44.466934+0000","last_peered":"2023-11-28T04:07:44.466934+0000","last_clean":"2023-11-28T04:07:44.466934+0000","last_became_active":"2023-11-28T04:07:44.108259+0000","last_became_peered":"2023-11-28T04:07:44.108259+0000","last_unstale":"2023-11-28T04:07:44.466934+0000","last_undegraded":"2023-11-28T04:07:44.466934+0000","last_fullsized":"2023-11-28T04:07:44.466934+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488964+0000","last_change":"2023-11-28T04:07:44.474715+0000","last_active":"2023-11-28T04:07:44.488964+0000","last_peered":"2023-11-28T04:07:44.488964+0000","last_clean":"2023-11-28T04:07:44.488964+0000","last_became_active":"2023-11-28T04:07:44.474434+0000","last_became_peered":"2023-11-28T04:07:44.474434+0000","last_unstale":"2023-11-28T04:07:44.488964+0000","last_undegraded":"2023-11-28T04:07:44.488964+0000","last_fullsized":"2023-11-28T04:07:44.488964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489782+0000","last_change":"2023-11-28T04:07:44.113469+0000","last_active":"2023-11-28T04:07:44.489782+0000","last_peered":"2023-11-28T04:07:44.489782+0000","last_clean":"2023-11-28T04:07:44.489782+0000","last_became_active":"2023-11-28T04:07:44.113235+0000","last_became_peered":"2023-11-28T04:07:44.113235+0000","last_unstale":"2023-11-28T04:07:44.489782+0000","last_undegraded":"2023-11-28T04:07:44.489782+0000","last_fullsized":"2023-11-28T04:07:44.489782+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486375+0000","last_change":"2023-11-28T04:07:44.486375+0000","last_active":"2023-11-28T04:07:44.486375+0000","last_peered":"2023-11-28T04:07:44.486375+0000","last_clean":"2023-11-28T04:07:44.486375+0000","last_became_active":"2023-11-28T04:07:44.485929+0000","last_became_peered":"2023-11-28T04:07:44.485929+0000","last_unstale":"2023-11-28T04:07:44.486375+0000","last_undegraded":"2023-11-28T04:07:44.486375+0000","last_fullsized":"2023-11-28T04:07:44.486375+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490200+0000","last_change":"2023-11-28T04:07:44.110425+0000","last_active":"2023-11-28T04:07:44.490200+0000","last_peered":"2023-11-28T04:07:44.490200+0000","last_clean":"2023-11-28T04:07:44.490200+0000","last_became_active":"2023-11-28T04:07:44.110130+0000","last_became_peered":"2023-11-28T04:07:44.110130+0000","last_unstale":"2023-11-28T04:07:44.490200+0000","last_undegraded":"2023-11-28T04:07:44.490200+0000","last_fullsized":"2023-11-28T04:07:44.490200+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480313+0000","last_change":"2023-11-28T04:07:44.480313+0000","last_active":"2023-11-28T04:07:44.480313+0000","last_peered":"2023-11-28T04:07:44.480313+0000","last_clean":"2023-11-28T04:07:44.480313+0000","last_became_active":"2023-11-28T04:07:44.480057+0000","last_became_peered":"2023-11-28T04:07:44.480057+0000","last_unstale":"2023-11-28T04:07:44.480313+0000","last_undegraded":"2023-11-28T04:07:44.480313+0000","last_fullsized":"2023-11-28T04:07:44.480313+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113510+0000","last_change":"2023-11-28T04:07:44.109074+0000","last_active":"2023-11-28T04:07:44.113510+0000","last_peered":"2023-11-28T04:07:44.113510+0000","last_clean":"2023-11-28T04:07:44.113510+0000","last_became_active":"2023-11-28T04:07:44.108769+0000","last_became_peered":"2023-11-28T04:07:44.108769+0000","last_unstale":"2023-11-28T04:07:44.113510+0000","last_undegraded":"2023-11-28T04:07:44.113510+0000","last_fullsized":"2023-11-28T04:07:44.113510+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475563+0000","last_change":"2023-11-28T04:07:44.475563+0000","last_active":"2023-11-28T04:07:44.475563+0000","last_peered":"2023-11-28T04:07:44.475563+0000","last_clean":"2023-11-28T04:07:44.475563+0000","last_became_active":"2023-11-28T04:07:44.475180+0000","last_became_peered":"2023-11-28T04:07:44.475180+0000","last_unstale":"2023-11-28T04:07:44.475563+0000","last_undegraded":"2023-11-28T04:07:44.475563+0000","last_fullsized":"2023-11-28T04:07:44.475563+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465693+0000","last_change":"2023-11-28T04:07:44.110327+0000","last_active":"2023-11-28T04:07:44.465693+0000","last_peered":"2023-11-28T04:07:44.465693+0000","last_clean":"2023-11-28T04:07:44.465693+0000","last_became_active":"2023-11-28T04:07:44.110082+0000","last_became_peered":"2023-11-28T04:07:44.110082+0000","last_unstale":"2023-11-28T04:07:44.465693+0000","last_undegraded":"2023-11-28T04:07:44.465693+0000","last_fullsized":"2023-11-28T04:07:44.465693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.476765+0000","last_change":"2023-11-28T04:07:44.473362+0000","last_active":"2023-11-28T04:07:44.476765+0000","last_peered":"2023-11-28T04:07:44.476765+0000","last_clean":"2023-11-28T04:07:44.476765+0000","last_became_active":"2023-11-28T04:07:44.473079+0000","last_became_peered":"2023-11-28T04:07:44.473079+0000","last_unstale":"2023-11-28T04:07:44.476765+0000","last_undegraded":"2023-11-28T04:07:44.476765+0000","last_fullsized":"2023-11-28T04:07:44.476765+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489420+0000","last_change":"2023-11-28T04:07:44.102468+0000","last_active":"2023-11-28T04:07:44.489420+0000","last_peered":"2023-11-28T04:07:44.489420+0000","last_clean":"2023-11-28T04:07:44.489420+0000","last_became_active":"2023-11-28T04:07:44.102166+0000","last_became_peered":"2023-11-28T04:07:44.102166+0000","last_unstale":"2023-11-28T04:07:44.489420+0000","last_undegraded":"2023-11-28T04:07:44.489420+0000","last_fullsized":"2023-11-28T04:07:44.489420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482885+0000","last_change":"2023-11-28T04:07:44.482885+0000","last_active":"2023-11-28T04:07:44.482885+0000","last_peered":"2023-11-28T04:07:44.482885+0000","last_clean":"2023-11-28T04:07:44.482885+0000","last_became_active":"2023-11-28T04:07:44.482557+0000","last_became_peered":"2023-11-28T04:07:44.482557+0000","last_unstale":"2023-11-28T04:07:44.482885+0000","last_undegraded":"2023-11-28T04:07:44.482885+0000","last_fullsized":"2023-11-28T04:07:44.482885+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.108281+0000","last_change":"2023-11-28T04:07:44.102694+0000","last_active":"2023-11-28T04:07:44.108281+0000","last_peered":"2023-11-28T04:07:44.108281+0000","last_clean":"2023-11-28T04:07:44.108281+0000","last_became_active":"2023-11-28T04:07:44.102089+0000","last_became_peered":"2023-11-28T04:07:44.102089+0000","last_unstale":"2023-11-28T04:07:44.108281+0000","last_undegraded":"2023-11-28T04:07:44.108281+0000","last_fullsized":"2023-11-28T04:07:44.108281+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489965+0000","last_change":"2023-11-28T04:07:44.489160+0000","last_active":"2023-11-28T04:07:44.489965+0000","last_peered":"2023-11-28T04:07:44.489965+0000","last_clean":"2023-11-28T04:07:44.489965+0000","last_became_active":"2023-11-28T04:07:44.488851+0000","last_became_peered":"2023-11-28T04:07:44.488851+0000","last_unstale":"2023-11-28T04:07:44.489965+0000","last_undegraded":"2023-11-28T04:07:44.489965+0000","last_fullsized":"2023-11-28T04:07:44.489965+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121165+0000","last_change":"2023-11-28T04:07:44.118105+0000","last_active":"2023-11-28T04:07:44.121165+0000","last_peered":"2023-11-28T04:07:44.121165+0000","last_clean":"2023-11-28T04:07:44.121165+0000","last_became_active":"2023-11-28T04:07:44.117793+0000","last_became_peered":"2023-11-28T04:07:44.117793+0000","last_unstale":"2023-11-28T04:07:44.121165+0000","last_undegraded":"2023-11-28T04:07:44.121165+0000","last_fullsized":"2023-11-28T04:07:44.121165+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494822+0000","last_change":"2023-11-28T04:07:44.491014+0000","last_active":"2023-11-28T04:07:44.494822+0000","last_peered":"2023-11-28T04:07:44.494822+0000","last_clean":"2023-11-28T04:07:44.494822+0000","last_became_active":"2023-11-28T04:07:44.490714+0000","last_became_peered":"2023-11-28T04:07:44.490714+0000","last_unstale":"2023-11-28T04:07:44.494822+0000","last_undegraded":"2023-11-28T04:07:44.494822+0000","last_fullsized":"2023-11-28T04:07:44.494822+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114821+0000","last_change":"2023-11-28T04:07:44.110080+0000","last_active":"2023-11-28T04:07:44.114821+0000","last_peered":"2023-11-28T04:07:44.114821+0000","last_clean":"2023-11-28T04:07:44.114821+0000","last_became_active":"2023-11-28T04:07:44.109823+0000","last_became_peered":"2023-11-28T04:07:44.109823+0000","last_unstale":"2023-11-28T04:07:44.114821+0000","last_undegraded":"2023-11-28T04:07:44.114821+0000","last_fullsized":"2023-11-28T04:07:44.114821+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479651+0000","last_change":"2023-11-28T04:07:44.477357+0000","last_active":"2023-11-28T04:07:44.479651+0000","last_peered":"2023-11-28T04:07:44.479651+0000","last_clean":"2023-11-28T04:07:44.479651+0000","last_became_active":"2023-11-28T04:07:44.477078+0000","last_became_peered":"2023-11-28T04:07:44.477078+0000","last_unstale":"2023-11-28T04:07:44.479651+0000","last_undegraded":"2023-11-28T04:07:44.479651+0000","last_fullsized":"2023-11-28T04:07:44.479651+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488609+0000","last_change":"2023-11-28T04:07:44.110131+0000","last_active":"2023-11-28T04:07:44.488609+0000","last_peered":"2023-11-28T04:07:44.488609+0000","last_clean":"2023-11-28T04:07:44.488609+0000","last_became_active":"2023-11-28T04:07:44.109844+0000","last_became_peered":"2023-11-28T04:07:44.109844+0000","last_unstale":"2023-11-28T04:07:44.488609+0000","last_undegraded":"2023-11-28T04:07:44.488609+0000","last_fullsized":"2023-11-28T04:07:44.488609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489248+0000","last_change":"2023-11-28T04:07:44.475348+0000","last_active":"2023-11-28T04:07:44.489248+0000","last_peered":"2023-11-28T04:07:44.489248+0000","last_clean":"2023-11-28T04:07:44.489248+0000","last_became_active":"2023-11-28T04:07:44.475089+0000","last_became_peered":"2023-11-28T04:07:44.475089+0000","last_unstale":"2023-11-28T04:07:44.489248+0000","last_undegraded":"2023-11-28T04:07:44.489248+0000","last_fullsized":"2023-11-28T04:07:44.489248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472753+0000","last_change":"2023-11-28T04:07:44.113266+0000","last_active":"2023-11-28T04:07:44.472753+0000","last_peered":"2023-11-28T04:07:44.472753+0000","last_clean":"2023-11-28T04:07:44.472753+0000","last_became_active":"2023-11-28T04:07:44.113000+0000","last_became_peered":"2023-11-28T04:07:44.113000+0000","last_unstale":"2023-11-28T04:07:44.472753+0000","last_undegraded":"2023-11-28T04:07:44.472753+0000","last_fullsized":"2023-11-28T04:07:44.472753+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489755+0000","last_change":"2023-11-28T04:07:44.489755+0000","last_active":"2023-11-28T04:07:44.489755+0000","last_peered":"2023-11-28T04:07:44.489755+0000","last_clean":"2023-11-28T04:07:44.489755+0000","last_became_active":"2023-11-28T04:07:44.489466+0000","last_became_peered":"2023-11-28T04:07:44.489466+0000","last_unstale":"2023-11-28T04:07:44.489755+0000","last_undegraded":"2023-11-28T04:07:44.489755+0000","last_fullsized":"2023-11-28T04:07:44.489755+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468894+0000","last_change":"2023-11-28T04:07:44.109468+0000","last_active":"2023-11-28T04:07:44.468894+0000","last_peered":"2023-11-28T04:07:44.468894+0000","last_clean":"2023-11-28T04:07:44.468894+0000","last_became_active":"2023-11-28T04:07:44.109138+0000","last_became_peered":"2023-11-28T04:07:44.109138+0000","last_unstale":"2023-11-28T04:07:44.468894+0000","last_undegraded":"2023-11-28T04:07:44.468894+0000","last_fullsized":"2023-11-28T04:07:44.468894+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.477053+0000","last_change":"2023-11-28T04:07:44.477053+0000","last_active":"2023-11-28T04:07:44.477053+0000","last_peered":"2023-11-28T04:07:44.477053+0000","last_clean":"2023-11-28T04:07:44.477053+0000","last_became_active":"2023-11-28T04:07:44.476805+0000","last_became_peered":"2023-11-28T04:07:44.476805+0000","last_unstale":"2023-11-28T04:07:44.477053+0000","last_undegraded":"2023-11-28T04:07:44.477053+0000","last_fullsized":"2023-11-28T04:07:44.477053+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113468+0000","last_change":"2023-11-28T04:07:44.109113+0000","last_active":"2023-11-28T04:07:44.113468+0000","last_peered":"2023-11-28T04:07:44.113468+0000","last_clean":"2023-11-28T04:07:44.113468+0000","last_became_active":"2023-11-28T04:07:44.108854+0000","last_became_peered":"2023-11-28T04:07:44.108854+0000","last_unstale":"2023-11-28T04:07:44.113468+0000","last_undegraded":"2023-11-28T04:07:44.113468+0000","last_fullsized":"2023-11-28T04:07:44.113468+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484405+0000","last_change":"2023-11-28T04:07:44.484405+0000","last_active":"2023-11-28T04:07:44.484405+0000","last_peered":"2023-11-28T04:07:44.484405+0000","last_clean":"2023-11-28T04:07:44.484405+0000","last_became_active":"2023-11-28T04:07:44.484145+0000","last_became_peered":"2023-11-28T04:07:44.484145+0000","last_unstale":"2023-11-28T04:07:44.484405+0000","last_undegraded":"2023-11-28T04:07:44.484405+0000","last_fullsized":"2023-11-28T04:07:44.484405+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470044+0000","last_change":"2023-11-28T04:07:44.117498+0000","last_active":"2023-11-28T04:07:44.470044+0000","last_peered":"2023-11-28T04:07:44.470044+0000","last_clean":"2023-11-28T04:07:44.470044+0000","last_became_active":"2023-11-28T04:07:44.117094+0000","last_became_peered":"2023-11-28T04:07:44.117094+0000","last_unstale":"2023-11-28T04:07:44.470044+0000","last_undegraded":"2023-11-28T04:07:44.470044+0000","last_fullsized":"2023-11-28T04:07:44.470044+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478155+0000","last_change":"2023-11-28T04:07:44.478155+0000","last_active":"2023-11-28T04:07:44.478155+0000","last_peered":"2023-11-28T04:07:44.478155+0000","last_clean":"2023-11-28T04:07:44.478155+0000","last_became_active":"2023-11-28T04:07:44.477831+0000","last_became_peered":"2023-11-28T04:07:44.477831+0000","last_unstale":"2023-11-28T04:07:44.478155+0000","last_undegraded":"2023-11-28T04:07:44.478155+0000","last_fullsized":"2023-11-28T04:07:44.478155+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120782+0000","last_change":"2023-11-28T04:07:44.117447+0000","last_active":"2023-11-28T04:07:44.120782+0000","last_peered":"2023-11-28T04:07:44.120782+0000","last_clean":"2023-11-28T04:07:44.120782+0000","last_became_active":"2023-11-28T04:07:44.117157+0000","last_became_peered":"2023-11-28T04:07:44.117157+0000","last_unstale":"2023-11-28T04:07:44.120782+0000","last_undegraded":"2023-11-28T04:07:44.120782+0000","last_fullsized":"2023-11-28T04:07:44.120782+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486163+0000","last_change":"2023-11-28T04:07:44.486163+0000","last_active":"2023-11-28T04:07:44.486163+0000","last_peered":"2023-11-28T04:07:44.486163+0000","last_clean":"2023-11-28T04:07:44.486163+0000","last_became_active":"2023-11-28T04:07:44.485864+0000","last_became_peered":"2023-11-28T04:07:44.485864+0000","last_unstale":"2023-11-28T04:07:44.486163+0000","last_undegraded":"2023-11-28T04:07:44.486163+0000","last_fullsized":"2023-11-28T04:07:44.486163+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113406+0000","last_change":"2023-11-28T04:07:44.109265+0000","last_active":"2023-11-28T04:07:44.113406+0000","last_peered":"2023-11-28T04:07:44.113406+0000","last_clean":"2023-11-28T04:07:44.113406+0000","last_became_active":"2023-11-28T04:07:44.108960+0000","last_became_peered":"2023-11-28T04:07:44.108960+0000","last_unstale":"2023-11-28T04:07:44.113406+0000","last_undegraded":"2023-11-28T04:07:44.113406+0000","last_fullsized":"2023-11-28T04:07:44.113406+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489939+0000","last_change":"2023-11-28T04:07:44.489939+0000","last_active":"2023-11-28T04:07:44.489939+0000","last_peered":"2023-11-28T04:07:44.489939+0000","last_clean":"2023-11-28T04:07:44.489939+0000","last_became_active":"2023-11-28T04:07:44.489622+0000","last_became_peered":"2023-11-28T04:07:44.489622+0000","last_unstale":"2023-11-28T04:07:44.489939+0000","last_undegraded":"2023-11-28T04:07:44.489939+0000","last_fullsized":"2023-11-28T04:07:44.489939+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465855+0000","last_change":"2023-11-28T04:07:44.108144+0000","last_active":"2023-11-28T04:07:44.465855+0000","last_peered":"2023-11-28T04:07:44.465855+0000","last_clean":"2023-11-28T04:07:44.465855+0000","last_became_active":"2023-11-28T04:07:44.107822+0000","last_became_peered":"2023-11-28T04:07:44.107822+0000","last_unstale":"2023-11-28T04:07:44.465855+0000","last_undegraded":"2023-11-28T04:07:44.465855+0000","last_fullsized":"2023-11-28T04:07:44.465855+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490884+0000","last_change":"2023-11-28T04:07:44.484397+0000","last_active":"2023-11-28T04:07:44.490884+0000","last_peered":"2023-11-28T04:07:44.490884+0000","last_clean":"2023-11-28T04:07:44.490884+0000","last_became_active":"2023-11-28T04:07:44.484150+0000","last_became_peered":"2023-11-28T04:07:44.484150+0000","last_unstale":"2023-11-28T04:07:44.490884+0000","last_undegraded":"2023-11-28T04:07:44.490884+0000","last_fullsized":"2023-11-28T04:07:44.490884+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"164'3","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491610+0000","last_change":"2023-11-28T04:07:44.108445+0000","last_active":"2023-11-28T04:07:44.491610+0000","last_peered":"2023-11-28T04:07:44.491610+0000","last_clean":"2023-11-28T04:07:44.491610+0000","last_became_active":"2023-11-28T04:07:44.108144+0000","last_became_peered":"2023-11-28T04:07:44.108144+0000","last_unstale":"2023-11-28T04:07:44.491610+0000","last_undegraded":"2023-11-28T04:07:44.491610+0000","last_fullsized":"2023-11-28T04:07:44.491610+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.104011+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:07:44.104011+0000","last_peered":"2023-11-28T04:07:44.104011+0000","last_clean":"2023-11-28T04:07:44.104011+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:07:44.104011+0000","last_undegraded":"2023-11-28T04:07:44.104011+0000","last_fullsized":"2023-11-28T04:07:44.104011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470645+0000","last_change":"2023-11-28T04:07:44.470645+0000","last_active":"2023-11-28T04:07:44.470645+0000","last_peered":"2023-11-28T04:07:44.470645+0000","last_clean":"2023-11-28T04:07:44.470645+0000","last_became_active":"2023-11-28T04:07:44.470356+0000","last_became_peered":"2023-11-28T04:07:44.470356+0000","last_unstale":"2023-11-28T04:07:44.470645+0000","last_undegraded":"2023-11-28T04:07:44.470645+0000","last_fullsized":"2023-11-28T04:07:44.470645+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468436+0000","last_change":"2023-11-28T04:07:44.101210+0000","last_active":"2023-11-28T04:07:44.468436+0000","last_peered":"2023-11-28T04:07:44.468436+0000","last_clean":"2023-11-28T04:07:44.468436+0000","last_became_active":"2023-11-28T04:07:44.100914+0000","last_became_peered":"2023-11-28T04:07:44.100914+0000","last_unstale":"2023-11-28T04:07:44.468436+0000","last_undegraded":"2023-11-28T04:07:44.468436+0000","last_fullsized":"2023-11-28T04:07:44.468436+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472943+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:07:44.472943+0000","last_peered":"2023-11-28T04:07:44.472943+0000","last_clean":"2023-11-28T04:07:44.472943+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:07:44.472943+0000","last_undegraded":"2023-11-28T04:07:44.472943+0000","last_fullsized":"2023-11-28T04:07:44.472943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479562+0000","last_change":"2023-11-28T04:07:44.476618+0000","last_active":"2023-11-28T04:07:44.479562+0000","last_peered":"2023-11-28T04:07:44.479562+0000","last_clean":"2023-11-28T04:07:44.479562+0000","last_became_active":"2023-11-28T04:07:44.476363+0000","last_became_peered":"2023-11-28T04:07:44.476363+0000","last_unstale":"2023-11-28T04:07:44.479562+0000","last_undegraded":"2023-11-28T04:07:44.479562+0000","last_fullsized":"2023-11-28T04:07:44.479562+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"164'4","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490874+0000","last_change":"2023-11-28T04:07:44.109291+0000","last_active":"2023-11-28T04:07:44.490874+0000","last_peered":"2023-11-28T04:07:44.490874+0000","last_clean":"2023-11-28T04:07:44.490874+0000","last_became_active":"2023-11-28T04:07:44.108915+0000","last_became_peered":"2023-11-28T04:07:44.108915+0000","last_unstale":"2023-11-28T04:07:44.490874+0000","last_undegraded":"2023-11-28T04:07:44.490874+0000","last_fullsized":"2023-11-28T04:07:44.490874+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.091942+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:07:44.091942+0000","last_peered":"2023-11-28T04:07:44.091942+0000","last_clean":"2023-11-28T04:07:44.091942+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:07:44.091942+0000","last_undegraded":"2023-11-28T04:07:44.091942+0000","last_fullsized":"2023-11-28T04:07:44.091942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489166+0000","last_change":"2023-11-28T04:07:44.476393+0000","last_active":"2023-11-28T04:07:44.489166+0000","last_peered":"2023-11-28T04:07:44.489166+0000","last_clean":"2023-11-28T04:07:44.489166+0000","last_became_active":"2023-11-28T04:07:44.475919+0000","last_became_peered":"2023-11-28T04:07:44.475919+0000","last_unstale":"2023-11-28T04:07:44.489166+0000","last_undegraded":"2023-11-28T04:07:44.489166+0000","last_fullsized":"2023-11-28T04:07:44.489166+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121175+0000","last_change":"2023-11-28T04:07:44.118149+0000","last_active":"2023-11-28T04:07:44.121175+0000","last_peered":"2023-11-28T04:07:44.121175+0000","last_clean":"2023-11-28T04:07:44.121175+0000","last_became_active":"2023-11-28T04:07:44.117774+0000","last_became_peered":"2023-11-28T04:07:44.117774+0000","last_unstale":"2023-11-28T04:07:44.121175+0000","last_undegraded":"2023-11-28T04:07:44.121175+0000","last_fullsized":"2023-11-28T04:07:44.121175+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470495+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:07:44.470495+0000","last_peered":"2023-11-28T04:07:44.470495+0000","last_clean":"2023-11-28T04:07:44.470495+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:07:44.470495+0000","last_undegraded":"2023-11-28T04:07:44.470495+0000","last_fullsized":"2023-11-28T04:07:44.470495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.097165+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:07:44.097165+0000","last_peered":"2023-11-28T04:07:44.097165+0000","last_clean":"2023-11-28T04:07:44.097165+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:07:44.097165+0000","last_undegraded":"2023-11-28T04:07:44.097165+0000","last_fullsized":"2023-11-28T04:07:44.097165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480998+0000","last_change":"2023-11-28T04:07:44.480998+0000","last_active":"2023-11-28T04:07:44.480998+0000","last_peered":"2023-11-28T04:07:44.480998+0000","last_clean":"2023-11-28T04:07:44.480998+0000","last_became_active":"2023-11-28T04:07:44.480742+0000","last_became_peered":"2023-11-28T04:07:44.480742+0000","last_unstale":"2023-11-28T04:07:44.480998+0000","last_undegraded":"2023-11-28T04:07:44.480998+0000","last_fullsized":"2023-11-28T04:07:44.480998+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466623+0000","last_change":"2023-11-28T04:07:44.113995+0000","last_active":"2023-11-28T04:07:44.466623+0000","last_peered":"2023-11-28T04:07:44.466623+0000","last_clean":"2023-11-28T04:07:44.466623+0000","last_became_active":"2023-11-28T04:07:44.113244+0000","last_became_peered":"2023-11-28T04:07:44.113244+0000","last_unstale":"2023-11-28T04:07:44.466623+0000","last_undegraded":"2023-11-28T04:07:44.466623+0000","last_fullsized":"2023-11-28T04:07:44.466623+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.097248+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:07:44.097248+0000","last_peered":"2023-11-28T04:07:44.097248+0000","last_clean":"2023-11-28T04:07:44.097248+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:07:44.097248+0000","last_undegraded":"2023-11-28T04:07:44.097248+0000","last_fullsized":"2023-11-28T04:07:44.097248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489019+0000","last_change":"2023-11-28T04:07:44.475084+0000","last_active":"2023-11-28T04:07:44.489019+0000","last_peered":"2023-11-28T04:07:44.489019+0000","last_clean":"2023-11-28T04:07:44.489019+0000","last_became_active":"2023-11-28T04:07:44.474807+0000","last_became_peered":"2023-11-28T04:07:44.474807+0000","last_unstale":"2023-11-28T04:07:44.489019+0000","last_undegraded":"2023-11-28T04:07:44.489019+0000","last_fullsized":"2023-11-28T04:07:44.489019+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484554+0000","last_change":"2023-11-28T04:07:44.109170+0000","last_active":"2023-11-28T04:07:44.484554+0000","last_peered":"2023-11-28T04:07:44.484554+0000","last_clean":"2023-11-28T04:07:44.484554+0000","last_became_active":"2023-11-28T04:07:44.108884+0000","last_became_peered":"2023-11-28T04:07:44.108884+0000","last_unstale":"2023-11-28T04:07:44.484554+0000","last_undegraded":"2023-11-28T04:07:44.484554+0000","last_fullsized":"2023-11-28T04:07:44.484554+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.100643+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:07:44.100643+0000","last_peered":"2023-11-28T04:07:44.100643+0000","last_clean":"2023-11-28T04:07:44.100643+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:07:44.100643+0000","last_undegraded":"2023-11-28T04:07:44.100643+0000","last_fullsized":"2023-11-28T04:07:44.100643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475198+0000","last_change":"2023-11-28T04:07:44.475198+0000","last_active":"2023-11-28T04:07:44.475198+0000","last_peered":"2023-11-28T04:07:44.475198+0000","last_clean":"2023-11-28T04:07:44.475198+0000","last_became_active":"2023-11-28T04:07:44.474922+0000","last_became_peered":"2023-11-28T04:07:44.474922+0000","last_unstale":"2023-11-28T04:07:44.475198+0000","last_undegraded":"2023-11-28T04:07:44.475198+0000","last_fullsized":"2023-11-28T04:07:44.475198+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475742+0000","last_change":"2023-11-28T04:07:44.112188+0000","last_active":"2023-11-28T04:07:44.475742+0000","last_peered":"2023-11-28T04:07:44.475742+0000","last_clean":"2023-11-28T04:07:44.475742+0000","last_became_active":"2023-11-28T04:07:44.111845+0000","last_became_peered":"2023-11-28T04:07:44.111845+0000","last_unstale":"2023-11-28T04:07:44.475742+0000","last_undegraded":"2023-11-28T04:07:44.475742+0000","last_fullsized":"2023-11-28T04:07:44.475742+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.104097+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:07:44.104097+0000","last_peered":"2023-11-28T04:07:44.104097+0000","last_clean":"2023-11-28T04:07:44.104097+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:07:44.104097+0000","last_undegraded":"2023-11-28T04:07:44.104097+0000","last_fullsized":"2023-11-28T04:07:44.104097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490428+0000","last_change":"2023-11-28T04:07:44.490428+0000","last_active":"2023-11-28T04:07:44.490428+0000","last_peered":"2023-11-28T04:07:44.490428+0000","last_clean":"2023-11-28T04:07:44.490428+0000","last_became_active":"2023-11-28T04:07:44.490238+0000","last_became_peered":"2023-11-28T04:07:44.490238+0000","last_unstale":"2023-11-28T04:07:44.490428+0000","last_undegraded":"2023-11-28T04:07:44.490428+0000","last_fullsized":"2023-11-28T04:07:44.490428+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.107289+0000","last_change":"2023-11-28T04:07:44.102624+0000","last_active":"2023-11-28T04:07:44.107289+0000","last_peered":"2023-11-28T04:07:44.107289+0000","last_clean":"2023-11-28T04:07:44.107289+0000","last_became_active":"2023-11-28T04:07:44.102114+0000","last_became_peered":"2023-11-28T04:07:44.102114+0000","last_unstale":"2023-11-28T04:07:44.107289+0000","last_undegraded":"2023-11-28T04:07:44.107289+0000","last_fullsized":"2023-11-28T04:07:44.107289+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":146,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467499+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:07:44.467499+0000","last_peered":"2023-11-28T04:07:44.467499+0000","last_clean":"2023-11-28T04:07:44.467499+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:07:44.467499+0000","last_undegraded":"2023-11-28T04:07:44.467499+0000","last_fullsized":"2023-11-28T04:07:44.467499+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575746,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":2052096,"data_stored":1027884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.379},"max":{"1min":0.72699999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45900000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.495}]},{"osd":6,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.875,"15min":0.875},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575746,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.81899999999999995},"last":0.46100000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.752,"15min":0.80300000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.33800000000000002},"max":{"1min":0.68600000000000005,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67900000000000005,"5min":0.79500000000000004,"15min":0.79600000000000004},"last":0.498},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.78000000000000003,"15min":0.83099999999999996},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.38600000000000001},"max":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79800000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.79200000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575745,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":2039808,"data_stored":1027150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.32400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.374}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":2052096,"data_stored":1027276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.61199999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.309,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.54600000000000004,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56299999999999994,"5min":0.621,"15min":0.67300000000000004},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.77500000000000002},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78500000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.45800000000000002,"15min":0.42499999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70599999999999996},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.42599999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834575747,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307288,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064552,"statfs":{"total":96636764160,"available":96322101248,"internally_reserved":0,"allocated":2048000,"data_stored":1027738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.625,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.249,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.223},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.626},"last":0.27600000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41599999999999998},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.27100000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":0.93400000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40200000000000002},"max":{"1min":0.85899999999999999,"5min":1.048,"15min":1.048},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575746,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.621,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.39800000000000002},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.25700000000000001},"max":{"1min":0.59999999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.67600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.503,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.94799999999999995,"15min":1.085},"last":0.47499999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":1.1200000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.65900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.41599999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.145},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.434},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.47099999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575745,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.753}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77800000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"last":0.498}]},{"osd":1,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.443},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.89400000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71599999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.378},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.69499999999999995,"15min":0.83799999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.84499999999999997},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.79000000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.752}]},{"osd":7,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.78500000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.81999999999999995},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:07:48.144 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T04:07:48.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.288+0000 7efc6ed27700 1 Processor -- start 2023-11-28T04:07:48.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.288+0000 7efc6ed27700 1 -- start start 2023-11-28T04:07:48.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6ed27700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc68157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6ed27700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc680c1750 0x7efc680c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6ed27700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc680c12c0 con 0x7efc680c0180 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6ed27700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc680c06b0 con 0x7efc680c1750 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6ed27700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc680c0990 con 0x7efc68157660 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc680c1750 0x7efc680c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efc680c1750 0x7efc680c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58660/0 (socket says 172.21.15.73:58660) 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6d524700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7efc680c0180 0x7efc681589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55898/0 (socket says 172.21.15.73:55898) 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 -- 172.21.15.73:0/192546160 learned_addr learned my addr 172.21.15.73:0/192546160 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 -- 172.21.15.73:0/192546160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 msgr2=0x7efc68157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6dd25700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc68157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc68157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6cd23700 1 -- 172.21.15.73:0/192546160 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2713551164 0 0) 0x7efc680c12c0 con 0x7efc680c0180 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.289+0000 7efc6e526700 1 -- 172.21.15.73:0/192546160 >> v1:172.21.15.73:6789/0 conn(0x7efc680c0180 legacy=0x7efc681589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6e526700 1 -- 172.21.15.73:0/192546160 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc680c0c70 con 0x7efc680c1750 2023-11-28T04:07:48.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6e526700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680c1bc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7efc60006fd0 tx=0x7efc600090f0).ready entity=mon.0 client_cookie=4b5fa4233aa4af07 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6cd23700 1 -- 172.21.15.73:0/192546160 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000b600 con 0x7efc680c1750 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6cd23700 1 -- 172.21.15.73:0/192546160 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc6000b760 con 0x7efc680c1750 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6cd23700 1 -- 172.21.15.73:0/192546160 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc6000bae0 con 0x7efc680c1750 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 -- 172.21.15.73:0/192546160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 msgr2=0x7efc680c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680c1bc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 rx=0x7efc60006fd0 tx=0x7efc600090f0).stop 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 -- 172.21.15.73:0/192546160 shutdown_connections 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc68157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 --2- 172.21.15.73:0/192546160 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680c1bc0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 -- 172.21.15.73:0/192546160 >> 172.21.15.73:0/192546160 conn(0x7efc680cb000 msgr2=0x7efc680cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 -- 172.21.15.73:0/192546160 shutdown_connections 2023-11-28T04:07:48.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.290+0000 7efc6ed27700 1 -- 172.21.15.73:0/192546160 wait complete. 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.291+0000 7efc6ed27700 1 Processor -- start 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.291+0000 7efc6ed27700 1 -- start start 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680c0180 0x7efc681649a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc680c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc680b02d0 con 0x7efc680c1750 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc68164e90 con 0x7efc680c0180 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6ed27700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc680af440 con 0x7efc68157660 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58674/0 (socket says 172.21.15.73:58674) 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 -- 172.21.15.73:0/1044084667 learned_addr learned my addr 172.21.15.73:0/1044084667 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 -- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 msgr2=0x7efc680c4100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc680c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 -- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680c0180 msgr2=0x7efc681649a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680c0180 0x7efc681649a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.292+0000 7efc6d524700 1 -- 172.21.15.73:0/1044084667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc60009370 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc6d524700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7efc5800c4a0 tx=0x7efc58007c30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580072e0 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc58007440 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc680af6c0 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc580058f0 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.293+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc680afc80 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.294+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc680c4a90 con 0x7efc680c1750 2023-11-28T04:07:48.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.294+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7efc58005ce0 con 0x7efc680c1750 2023-11-28T04:07:48.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.295+0000 7efc5dffb700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 0x7efc4c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.295+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efc58079ac0 con 0x7efc680c1750 2023-11-28T04:07:48.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.295+0000 7efc6dd25700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 0x7efc4c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.295+0000 7efc6dd25700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 0x7efc4c061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7efc64007eb0 tx=0x7efc640073a0).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.299+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efc5804ad90 con 0x7efc680c1750 2023-11-28T04:07:48.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.525+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7efc6806c7c0 con 0x7efc680c1750 2023-11-28T04:07:48.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.526+0000 7efc5dffb700 1 -- 172.21.15.73:0/1044084667 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+955 (secure 0 0 0) 0x7efc58079e60 con 0x7efc680c1750 2023-11-28T04:07:48.527 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:48.527 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4522,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":165,"num_osds":8,"num_up_osds":8,"osd_up_since":1701139955,"num_in_osds":8,"osd_in_since":1701139948,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":137}],"num_pgs":137,"num_pools":4,"num_objects":21,"data_bytes":2306,"bytes_used":2496380928,"bytes_avail":770597732352,"bytes_total":773094113280},"fsmap":{"epoch":58,"id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"a","status":"up:active","gid":8562}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-11-28T03:57:37.317486+0000","services":{}},"progress_events":{}} 2023-11-28T04:07:48.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 msgr2=0x7efc4c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 0x7efc4c061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7efc64007eb0 tx=0x7efc640073a0).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 msgr2=0x7efc680aef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 rx=0x7efc5800c4a0 tx=0x7efc58007c30).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 shutdown_connections 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efc4c05f150 0x7efc4c061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efc68157660 0x7efc680c4100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efc680c0180 0x7efc681649a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 --2- 172.21.15.73:0/1044084667 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efc680c1750 0x7efc680aef50 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 >> 172.21.15.73:0/1044084667 conn(0x7efc680cb000 msgr2=0x7efc680ca510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:48.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 shutdown_connections 2023-11-28T04:07:48.531 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.528+0000 7efc6ed27700 1 -- 172.21.15.73:0/1044084667 wait complete. 2023-11-28T04:07:48.541 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T04:07:48.541 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:07:48.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.652+0000 7fb2bf1b6700 1 Processor -- start 2023-11-28T04:07:48.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.652+0000 7fb2bf1b6700 1 -- start start 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bf1b6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80caab0 0x7fb2b80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bf1b6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2b80cae90 0x7fb2b80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bf1b6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2b80a9a90 con 0x7fb2b80aa960 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bf1b6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2b8154ce0 con 0x7fb2b80cae90 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bf1b6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80a8e80 con 0x7fb2b80caab0 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2affff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2b80cae90 0x7fb2b80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bd753700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb2b80aa960 0x7fb2b80c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55904/0 (socket says 172.21.15.73:55904) 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bd753700 1 -- 172.21.15.73:0/93785431 learned_addr learned my addr 172.21.15.73:0/93785431 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:48.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2bcf52700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80caab0 0x7fb2b80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2affff700 1 -- 172.21.15.73:0/93785431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80caab0 msgr2=0x7fb2b80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2affff700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80caab0 0x7fb2b80aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2affff700 1 -- 172.21.15.73:0/93785431 >> v1:172.21.15.73:6789/0 conn(0x7fb2b80aa960 legacy=0x7fb2b80c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.653+0000 7fb2af7fe700 1 -- 172.21.15.73:0/93785431 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1389291221 0 0) 0x7fb2b80a9a90 con 0x7fb2b80aa960 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2affff700 1 -- 172.21.15.73:0/93785431 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2b80a9160 con 0x7fb2b80cae90 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2affff700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80cae90 0x7fb2b80c0ab0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fb2a8009760 tx=0x7fb2a8009060).ready entity=mon.0 client_cookie=3b0e8b810398162a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2af7fe700 1 -- 172.21.15.73:0/93785431 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800b580 con 0x7fb2b80cae90 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2af7fe700 1 -- 172.21.15.73:0/93785431 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a800b6e0 con 0x7fb2b80cae90 2023-11-28T04:07:48.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2af7fe700 1 -- 172.21.15.73:0/93785431 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a800ba60 con 0x7fb2b80cae90 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/93785431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80cae90 msgr2=0x7fb2b80c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80cae90 0x7fb2b80c0ab0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 rx=0x7fb2a8009760 tx=0x7fb2a8009060).stop 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/93785431 shutdown_connections 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80caab0 0x7fb2b80aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/93785431 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80cae90 0x7fb2b80c0ab0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/93785431 >> 172.21.15.73:0/93785431 conn(0x7fb2b8094920 msgr2=0x7fb2b806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/93785431 shutdown_connections 2023-11-28T04:07:48.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.654+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/93785431 wait complete. 2023-11-28T04:07:48.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.655+0000 7fb2bf1b6700 1 Processor -- start 2023-11-28T04:07:48.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 -- start start 2023-11-28T04:07:48.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cae90 0x7fb2b8151d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81508b0 0x7fb2b8152230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8155f60 con 0x7fb2b80caab0 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b8152720 con 0x7fb2b81508b0 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bf1b6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2b80c3550 con 0x7fb2b80cae90 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bcf52700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bcf52700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58686/0 (socket says 172.21.15.73:58686) 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.656+0000 7fb2bcf52700 1 -- 172.21.15.73:0/2163605156 learned_addr learned my addr 172.21.15.73:0/2163605156 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2affff700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cae90 0x7fb2b8151d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bd753700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81508b0 0x7fb2b8152230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 -- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cae90 msgr2=0x7fb2b8151d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cae90 0x7fb2b8151d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 -- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81508b0 msgr2=0x7fb2b8152230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81508b0 0x7fb2b8152230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 -- 172.21.15.73:0/2163605156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a80092a0 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bcf52700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fb2b400d890 tx=0x7fb2b4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b40125f0 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2b4012750 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2b80c51f0 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2b4012ad0 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.657+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2b8155c80 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.658+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2b806c860 con 0x7fb2b80caab0 2023-11-28T04:07:48.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.658+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb2b4013480 con 0x7fb2b80caab0 2023-11-28T04:07:48.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.659+0000 7fb2acff9700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 0x7fb2a40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:48.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.659+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb2b407a580 con 0x7fb2b80caab0 2023-11-28T04:07:48.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.659+0000 7fb2affff700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 0x7fb2a40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:48.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.659+0000 7fb2affff700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 0x7fb2a40615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb2b80c6040 tx=0x7fb2a8017000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:48.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.663+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb2b404b7f0 con 0x7fb2b80caab0 2023-11-28T04:07:48.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.821+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2b80a9440 con 0x7fb2a405f120 2023-11-28T04:07:48.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.831+0000 7fb2acff9700 1 -- 172.21.15.73:0/2163605156 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362037 (secure 0 0 0) 0x7fb2b80a9440 con 0x7fb2a405f120 2023-11-28T04:07:48.836 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 msgr2=0x7fb2a40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 0x7fb2a40615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7fb2b80c6040 tx=0x7fb2a8017000).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 msgr2=0x7fb2b814d020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 rx=0x7fb2b400d890 tx=0x7fb2b4008040).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 shutdown_connections 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb2a405f120 0x7fb2a40615e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2b81508b0 0x7fb2b8152230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2b80caab0 0x7fb2b814d020 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 --2- 172.21.15.73:0/2163605156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb2b80cae90 0x7fb2b8151d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 >> 172.21.15.73:0/2163605156 conn(0x7fb2b8094920 msgr2=0x7fb2b80b0520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 shutdown_connections 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:48.834+0000 7fb2bf1b6700 1 -- 172.21.15.73:0/2163605156 wait complete. 2023-11-28T04:07:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:07:48.847 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2023-11-28T04:07:48.057344+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":176,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437872,"kb_used_data":16048,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752536848,"statfs":{"total":773094113280,"available":770597732352,"internally_reserved":0,"allocated":16433152,"data_stored":8248628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5035976,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-521,"num_write_kb":-5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-960,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.995164"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480557+0000","last_change":"2023-11-28T04:07:44.477800+0000","last_active":"2023-11-28T04:07:44.480557+0000","last_peered":"2023-11-28T04:07:44.480557+0000","last_clean":"2023-11-28T04:07:44.480557+0000","last_became_active":"2023-11-28T04:07:44.477512+0000","last_became_peered":"2023-11-28T04:07:44.477512+0000","last_unstale":"2023-11-28T04:07:44.480557+0000","last_undegraded":"2023-11-28T04:07:44.480557+0000","last_fullsized":"2023-11-28T04:07:44.480557+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"164'3","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483535+0000","last_change":"2023-11-28T04:07:44.113370+0000","last_active":"2023-11-28T04:07:44.483535+0000","last_peered":"2023-11-28T04:07:44.483535+0000","last_clean":"2023-11-28T04:07:44.483535+0000","last_became_active":"2023-11-28T04:07:44.113093+0000","last_became_peered":"2023-11-28T04:07:44.113093+0000","last_unstale":"2023-11-28T04:07:44.483535+0000","last_undegraded":"2023-11-28T04:07:44.483535+0000","last_fullsized":"2023-11-28T04:07:44.483535+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491520+0000","last_change":"2023-11-28T04:07:44.491520+0000","last_active":"2023-11-28T04:07:44.491520+0000","last_peered":"2023-11-28T04:07:44.491520+0000","last_clean":"2023-11-28T04:07:44.491520+0000","last_became_active":"2023-11-28T04:07:44.491268+0000","last_became_peered":"2023-11-28T04:07:44.491268+0000","last_unstale":"2023-11-28T04:07:44.491520+0000","last_undegraded":"2023-11-28T04:07:44.491520+0000","last_fullsized":"2023-11-28T04:07:44.491520+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.115033+0000","last_change":"2023-11-28T04:07:44.111634+0000","last_active":"2023-11-28T04:07:44.115033+0000","last_peered":"2023-11-28T04:07:44.115033+0000","last_clean":"2023-11-28T04:07:44.115033+0000","last_became_active":"2023-11-28T04:07:44.111156+0000","last_became_peered":"2023-11-28T04:07:44.111156+0000","last_unstale":"2023-11-28T04:07:44.115033+0000","last_undegraded":"2023-11-28T04:07:44.115033+0000","last_fullsized":"2023-11-28T04:07:44.115033+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489125+0000","last_change":"2023-11-28T04:07:44.489125+0000","last_active":"2023-11-28T04:07:44.489125+0000","last_peered":"2023-11-28T04:07:44.489125+0000","last_clean":"2023-11-28T04:07:44.489125+0000","last_became_active":"2023-11-28T04:07:44.488874+0000","last_became_peered":"2023-11-28T04:07:44.488874+0000","last_unstale":"2023-11-28T04:07:44.489125+0000","last_undegraded":"2023-11-28T04:07:44.489125+0000","last_fullsized":"2023-11-28T04:07:44.489125+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467036+0000","last_change":"2023-11-28T04:07:44.106241+0000","last_active":"2023-11-28T04:07:44.467036+0000","last_peered":"2023-11-28T04:07:44.467036+0000","last_clean":"2023-11-28T04:07:44.467036+0000","last_became_active":"2023-11-28T04:07:44.105742+0000","last_became_peered":"2023-11-28T04:07:44.105742+0000","last_unstale":"2023-11-28T04:07:44.467036+0000","last_undegraded":"2023-11-28T04:07:44.467036+0000","last_fullsized":"2023-11-28T04:07:44.467036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479609+0000","last_change":"2023-11-28T04:07:44.477574+0000","last_active":"2023-11-28T04:07:44.479609+0000","last_peered":"2023-11-28T04:07:44.479609+0000","last_clean":"2023-11-28T04:07:44.479609+0000","last_became_active":"2023-11-28T04:07:44.477313+0000","last_became_peered":"2023-11-28T04:07:44.477313+0000","last_unstale":"2023-11-28T04:07:44.479609+0000","last_undegraded":"2023-11-28T04:07:44.479609+0000","last_fullsized":"2023-11-28T04:07:44.479609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482545+0000","last_change":"2023-11-28T04:07:44.108761+0000","last_active":"2023-11-28T04:07:44.482545+0000","last_peered":"2023-11-28T04:07:44.482545+0000","last_clean":"2023-11-28T04:07:44.482545+0000","last_became_active":"2023-11-28T04:07:44.108511+0000","last_became_peered":"2023-11-28T04:07:44.108511+0000","last_unstale":"2023-11-28T04:07:44.482545+0000","last_undegraded":"2023-11-28T04:07:44.482545+0000","last_fullsized":"2023-11-28T04:07:44.482545+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494783+0000","last_change":"2023-11-28T04:07:44.489084+0000","last_active":"2023-11-28T04:07:44.494783+0000","last_peered":"2023-11-28T04:07:44.494783+0000","last_clean":"2023-11-28T04:07:44.494783+0000","last_became_active":"2023-11-28T04:07:44.488814+0000","last_became_peered":"2023-11-28T04:07:44.488814+0000","last_unstale":"2023-11-28T04:07:44.494783+0000","last_undegraded":"2023-11-28T04:07:44.494783+0000","last_fullsized":"2023-11-28T04:07:44.494783+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470962+0000","last_change":"2023-11-28T04:07:44.112289+0000","last_active":"2023-11-28T04:07:44.470962+0000","last_peered":"2023-11-28T04:07:44.470962+0000","last_clean":"2023-11-28T04:07:44.470962+0000","last_became_active":"2023-11-28T04:07:44.111696+0000","last_became_peered":"2023-11-28T04:07:44.111696+0000","last_unstale":"2023-11-28T04:07:44.470962+0000","last_undegraded":"2023-11-28T04:07:44.470962+0000","last_fullsized":"2023-11-28T04:07:44.470962+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489146+0000","last_change":"2023-11-28T04:07:44.476209+0000","last_active":"2023-11-28T04:07:44.489146+0000","last_peered":"2023-11-28T04:07:44.489146+0000","last_clean":"2023-11-28T04:07:44.489146+0000","last_became_active":"2023-11-28T04:07:44.475591+0000","last_became_peered":"2023-11-28T04:07:44.475591+0000","last_unstale":"2023-11-28T04:07:44.489146+0000","last_undegraded":"2023-11-28T04:07:44.489146+0000","last_fullsized":"2023-11-28T04:07:44.489146+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3a","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478528+0000","last_change":"2023-11-28T04:07:44.109581+0000","last_active":"2023-11-28T04:07:44.478528+0000","last_peered":"2023-11-28T04:07:44.478528+0000","last_clean":"2023-11-28T04:07:44.478528+0000","last_became_active":"2023-11-28T04:07:44.109244+0000","last_became_peered":"2023-11-28T04:07:44.109244+0000","last_unstale":"2023-11-28T04:07:44.478528+0000","last_undegraded":"2023-11-28T04:07:44.478528+0000","last_fullsized":"2023-11-28T04:07:44.478528+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484728+0000","last_change":"2023-11-28T04:07:44.480799+0000","last_active":"2023-11-28T04:07:44.484728+0000","last_peered":"2023-11-28T04:07:44.484728+0000","last_clean":"2023-11-28T04:07:44.484728+0000","last_became_active":"2023-11-28T04:07:44.480511+0000","last_became_peered":"2023-11-28T04:07:44.480511+0000","last_unstale":"2023-11-28T04:07:44.484728+0000","last_undegraded":"2023-11-28T04:07:44.484728+0000","last_fullsized":"2023-11-28T04:07:44.484728+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470212+0000","last_change":"2023-11-28T04:07:44.106504+0000","last_active":"2023-11-28T04:07:44.470212+0000","last_peered":"2023-11-28T04:07:44.470212+0000","last_clean":"2023-11-28T04:07:44.470212+0000","last_became_active":"2023-11-28T04:07:44.106202+0000","last_became_peered":"2023-11-28T04:07:44.106202+0000","last_unstale":"2023-11-28T04:07:44.470212+0000","last_undegraded":"2023-11-28T04:07:44.470212+0000","last_fullsized":"2023-11-28T04:07:44.470212+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482773+0000","last_change":"2023-11-28T04:07:44.482773+0000","last_active":"2023-11-28T04:07:44.482773+0000","last_peered":"2023-11-28T04:07:44.482773+0000","last_clean":"2023-11-28T04:07:44.482773+0000","last_became_active":"2023-11-28T04:07:44.482483+0000","last_became_peered":"2023-11-28T04:07:44.482483+0000","last_unstale":"2023-11-28T04:07:44.482773+0000","last_undegraded":"2023-11-28T04:07:44.482773+0000","last_fullsized":"2023-11-28T04:07:44.482773+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466893+0000","last_change":"2023-11-28T04:07:44.110221+0000","last_active":"2023-11-28T04:07:44.466893+0000","last_peered":"2023-11-28T04:07:44.466893+0000","last_clean":"2023-11-28T04:07:44.466893+0000","last_became_active":"2023-11-28T04:07:44.109739+0000","last_became_peered":"2023-11-28T04:07:44.109739+0000","last_unstale":"2023-11-28T04:07:44.466893+0000","last_undegraded":"2023-11-28T04:07:44.466893+0000","last_fullsized":"2023-11-28T04:07:44.466893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481534+0000","last_change":"2023-11-28T04:07:44.481534+0000","last_active":"2023-11-28T04:07:44.481534+0000","last_peered":"2023-11-28T04:07:44.481534+0000","last_clean":"2023-11-28T04:07:44.481534+0000","last_became_active":"2023-11-28T04:07:44.481272+0000","last_became_peered":"2023-11-28T04:07:44.481272+0000","last_unstale":"2023-11-28T04:07:44.481534+0000","last_undegraded":"2023-11-28T04:07:44.481534+0000","last_fullsized":"2023-11-28T04:07:44.481534+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467360+0000","last_change":"2023-11-28T04:07:44.115088+0000","last_active":"2023-11-28T04:07:44.467360+0000","last_peered":"2023-11-28T04:07:44.467360+0000","last_clean":"2023-11-28T04:07:44.467360+0000","last_became_active":"2023-11-28T04:07:44.114010+0000","last_became_peered":"2023-11-28T04:07:44.114010+0000","last_unstale":"2023-11-28T04:07:44.467360+0000","last_undegraded":"2023-11-28T04:07:44.467360+0000","last_fullsized":"2023-11-28T04:07:44.467360+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487403+0000","last_change":"2023-11-28T04:07:44.487403+0000","last_active":"2023-11-28T04:07:44.487403+0000","last_peered":"2023-11-28T04:07:44.487403+0000","last_clean":"2023-11-28T04:07:44.487403+0000","last_became_active":"2023-11-28T04:07:44.487107+0000","last_became_peered":"2023-11-28T04:07:44.487107+0000","last_unstale":"2023-11-28T04:07:44.487403+0000","last_undegraded":"2023-11-28T04:07:44.487403+0000","last_fullsized":"2023-11-28T04:07:44.487403+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.115013+0000","last_change":"2023-11-28T04:07:44.111442+0000","last_active":"2023-11-28T04:07:44.115013+0000","last_peered":"2023-11-28T04:07:44.115013+0000","last_clean":"2023-11-28T04:07:44.115013+0000","last_became_active":"2023-11-28T04:07:44.110824+0000","last_became_peered":"2023-11-28T04:07:44.110824+0000","last_unstale":"2023-11-28T04:07:44.115013+0000","last_undegraded":"2023-11-28T04:07:44.115013+0000","last_fullsized":"2023-11-28T04:07:44.115013+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485088+0000","last_change":"2023-11-28T04:07:44.485088+0000","last_active":"2023-11-28T04:07:44.485088+0000","last_peered":"2023-11-28T04:07:44.485088+0000","last_clean":"2023-11-28T04:07:44.485088+0000","last_became_active":"2023-11-28T04:07:44.484826+0000","last_became_peered":"2023-11-28T04:07:44.484826+0000","last_unstale":"2023-11-28T04:07:44.485088+0000","last_undegraded":"2023-11-28T04:07:44.485088+0000","last_fullsized":"2023-11-28T04:07:44.485088+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.116491+0000","last_change":"2023-11-28T04:07:44.110072+0000","last_active":"2023-11-28T04:07:44.116491+0000","last_peered":"2023-11-28T04:07:44.116491+0000","last_clean":"2023-11-28T04:07:44.116491+0000","last_became_active":"2023-11-28T04:07:44.109819+0000","last_became_peered":"2023-11-28T04:07:44.109819+0000","last_unstale":"2023-11-28T04:07:44.116491+0000","last_undegraded":"2023-11-28T04:07:44.116491+0000","last_fullsized":"2023-11-28T04:07:44.116491+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489777+0000","last_change":"2023-11-28T04:07:44.489777+0000","last_active":"2023-11-28T04:07:44.489777+0000","last_peered":"2023-11-28T04:07:44.489777+0000","last_clean":"2023-11-28T04:07:44.489777+0000","last_became_active":"2023-11-28T04:07:44.489547+0000","last_became_peered":"2023-11-28T04:07:44.489547+0000","last_unstale":"2023-11-28T04:07:44.489777+0000","last_undegraded":"2023-11-28T04:07:44.489777+0000","last_fullsized":"2023-11-28T04:07:44.489777+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.495253+0000","last_change":"2023-11-28T04:07:44.113703+0000","last_active":"2023-11-28T04:07:44.495253+0000","last_peered":"2023-11-28T04:07:44.495253+0000","last_clean":"2023-11-28T04:07:44.495253+0000","last_became_active":"2023-11-28T04:07:44.113417+0000","last_became_peered":"2023-11-28T04:07:44.113417+0000","last_unstale":"2023-11-28T04:07:44.495253+0000","last_undegraded":"2023-11-28T04:07:44.495253+0000","last_fullsized":"2023-11-28T04:07:44.495253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481988+0000","last_change":"2023-11-28T04:07:44.481988+0000","last_active":"2023-11-28T04:07:44.481988+0000","last_peered":"2023-11-28T04:07:44.481988+0000","last_clean":"2023-11-28T04:07:44.481988+0000","last_became_active":"2023-11-28T04:07:44.475932+0000","last_became_peered":"2023-11-28T04:07:44.475932+0000","last_unstale":"2023-11-28T04:07:44.481988+0000","last_undegraded":"2023-11-28T04:07:44.481988+0000","last_fullsized":"2023-11-28T04:07:44.481988+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.33","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468554+0000","last_change":"2023-11-28T04:07:44.104938+0000","last_active":"2023-11-28T04:07:44.468554+0000","last_peered":"2023-11-28T04:07:44.468554+0000","last_clean":"2023-11-28T04:07:44.468554+0000","last_became_active":"2023-11-28T04:07:44.102437+0000","last_became_peered":"2023-11-28T04:07:44.102437+0000","last_unstale":"2023-11-28T04:07:44.468554+0000","last_undegraded":"2023-11-28T04:07:44.468554+0000","last_fullsized":"2023-11-28T04:07:44.468554+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494657+0000","last_change":"2023-11-28T04:07:44.488430+0000","last_active":"2023-11-28T04:07:44.494657+0000","last_peered":"2023-11-28T04:07:44.494657+0000","last_clean":"2023-11-28T04:07:44.494657+0000","last_became_active":"2023-11-28T04:07:44.488148+0000","last_became_peered":"2023-11-28T04:07:44.488148+0000","last_unstale":"2023-11-28T04:07:44.494657+0000","last_undegraded":"2023-11-28T04:07:44.494657+0000","last_fullsized":"2023-11-28T04:07:44.494657+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114904+0000","last_change":"2023-11-28T04:07:44.110316+0000","last_active":"2023-11-28T04:07:44.114904+0000","last_peered":"2023-11-28T04:07:44.114904+0000","last_clean":"2023-11-28T04:07:44.114904+0000","last_became_active":"2023-11-28T04:07:44.110028+0000","last_became_peered":"2023-11-28T04:07:44.110028+0000","last_unstale":"2023-11-28T04:07:44.114904+0000","last_undegraded":"2023-11-28T04:07:44.114904+0000","last_fullsized":"2023-11-28T04:07:44.114904+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475360+0000","last_change":"2023-11-28T04:07:44.475360+0000","last_active":"2023-11-28T04:07:44.475360+0000","last_peered":"2023-11-28T04:07:44.475360+0000","last_clean":"2023-11-28T04:07:44.475360+0000","last_became_active":"2023-11-28T04:07:44.475098+0000","last_became_peered":"2023-11-28T04:07:44.475098+0000","last_unstale":"2023-11-28T04:07:44.475360+0000","last_undegraded":"2023-11-28T04:07:44.475360+0000","last_fullsized":"2023-11-28T04:07:44.475360+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117436+0000","last_change":"2023-11-28T04:07:44.111087+0000","last_active":"2023-11-28T04:07:44.117436+0000","last_peered":"2023-11-28T04:07:44.117436+0000","last_clean":"2023-11-28T04:07:44.117436+0000","last_became_active":"2023-11-28T04:07:44.110400+0000","last_became_peered":"2023-11-28T04:07:44.110400+0000","last_unstale":"2023-11-28T04:07:44.117436+0000","last_undegraded":"2023-11-28T04:07:44.117436+0000","last_fullsized":"2023-11-28T04:07:44.117436+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481492+0000","last_change":"2023-11-28T04:07:44.481492+0000","last_active":"2023-11-28T04:07:44.481492+0000","last_peered":"2023-11-28T04:07:44.481492+0000","last_clean":"2023-11-28T04:07:44.481492+0000","last_became_active":"2023-11-28T04:07:44.481212+0000","last_became_peered":"2023-11-28T04:07:44.481212+0000","last_unstale":"2023-11-28T04:07:44.481492+0000","last_undegraded":"2023-11-28T04:07:44.481492+0000","last_fullsized":"2023-11-28T04:07:44.481492+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483951+0000","last_change":"2023-11-28T04:07:44.105325+0000","last_active":"2023-11-28T04:07:44.483951+0000","last_peered":"2023-11-28T04:07:44.483951+0000","last_clean":"2023-11-28T04:07:44.483951+0000","last_became_active":"2023-11-28T04:07:44.105081+0000","last_became_peered":"2023-11-28T04:07:44.105081+0000","last_unstale":"2023-11-28T04:07:44.483951+0000","last_undegraded":"2023-11-28T04:07:44.483951+0000","last_fullsized":"2023-11-28T04:07:44.483951+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484640+0000","last_change":"2023-11-28T04:07:44.484640+0000","last_active":"2023-11-28T04:07:44.484640+0000","last_peered":"2023-11-28T04:07:44.484640+0000","last_clean":"2023-11-28T04:07:44.484640+0000","last_became_active":"2023-11-28T04:07:44.484227+0000","last_became_peered":"2023-11-28T04:07:44.484227+0000","last_unstale":"2023-11-28T04:07:44.484640+0000","last_undegraded":"2023-11-28T04:07:44.484640+0000","last_fullsized":"2023-11-28T04:07:44.484640+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469618+0000","last_change":"2023-11-28T04:07:44.104353+0000","last_active":"2023-11-28T04:07:44.469618+0000","last_peered":"2023-11-28T04:07:44.469618+0000","last_clean":"2023-11-28T04:07:44.469618+0000","last_became_active":"2023-11-28T04:07:44.104058+0000","last_became_peered":"2023-11-28T04:07:44.104058+0000","last_unstale":"2023-11-28T04:07:44.469618+0000","last_undegraded":"2023-11-28T04:07:44.469618+0000","last_fullsized":"2023-11-28T04:07:44.469618+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492111+0000","last_change":"2023-11-28T04:07:44.487100+0000","last_active":"2023-11-28T04:07:44.492111+0000","last_peered":"2023-11-28T04:07:44.492111+0000","last_clean":"2023-11-28T04:07:44.492111+0000","last_became_active":"2023-11-28T04:07:44.486847+0000","last_became_peered":"2023-11-28T04:07:44.486847+0000","last_unstale":"2023-11-28T04:07:44.492111+0000","last_undegraded":"2023-11-28T04:07:44.492111+0000","last_fullsized":"2023-11-28T04:07:44.492111+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469178+0000","last_change":"2023-11-28T04:07:44.113850+0000","last_active":"2023-11-28T04:07:44.469178+0000","last_peered":"2023-11-28T04:07:44.469178+0000","last_clean":"2023-11-28T04:07:44.469178+0000","last_became_active":"2023-11-28T04:07:44.113616+0000","last_became_peered":"2023-11-28T04:07:44.113616+0000","last_unstale":"2023-11-28T04:07:44.469178+0000","last_undegraded":"2023-11-28T04:07:44.469178+0000","last_fullsized":"2023-11-28T04:07:44.469178+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483363+0000","last_change":"2023-11-28T04:07:44.483363+0000","last_active":"2023-11-28T04:07:44.483363+0000","last_peered":"2023-11-28T04:07:44.483363+0000","last_clean":"2023-11-28T04:07:44.483363+0000","last_became_active":"2023-11-28T04:07:44.483101+0000","last_became_peered":"2023-11-28T04:07:44.483101+0000","last_unstale":"2023-11-28T04:07:44.483363+0000","last_undegraded":"2023-11-28T04:07:44.483363+0000","last_fullsized":"2023-11-28T04:07:44.483363+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2d","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.462659+0000","last_change":"2023-11-28T04:07:44.104302+0000","last_active":"2023-11-28T04:07:44.462659+0000","last_peered":"2023-11-28T04:07:44.462659+0000","last_clean":"2023-11-28T04:07:44.462659+0000","last_became_active":"2023-11-28T04:07:44.103985+0000","last_became_peered":"2023-11-28T04:07:44.103985+0000","last_unstale":"2023-11-28T04:07:44.462659+0000","last_undegraded":"2023-11-28T04:07:44.462659+0000","last_fullsized":"2023-11-28T04:07:44.462659+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482460+0000","last_change":"2023-11-28T04:07:44.477456+0000","last_active":"2023-11-28T04:07:44.482460+0000","last_peered":"2023-11-28T04:07:44.482460+0000","last_clean":"2023-11-28T04:07:44.482460+0000","last_became_active":"2023-11-28T04:07:44.477166+0000","last_became_peered":"2023-11-28T04:07:44.477166+0000","last_unstale":"2023-11-28T04:07:44.482460+0000","last_undegraded":"2023-11-28T04:07:44.482460+0000","last_fullsized":"2023-11-28T04:07:44.482460+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465599+0000","last_change":"2023-11-28T04:07:44.112401+0000","last_active":"2023-11-28T04:07:44.465599+0000","last_peered":"2023-11-28T04:07:44.465599+0000","last_clean":"2023-11-28T04:07:44.465599+0000","last_became_active":"2023-11-28T04:07:44.112163+0000","last_became_peered":"2023-11-28T04:07:44.112163+0000","last_unstale":"2023-11-28T04:07:44.465599+0000","last_undegraded":"2023-11-28T04:07:44.465599+0000","last_fullsized":"2023-11-28T04:07:44.465599+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479617+0000","last_change":"2023-11-28T04:07:44.478586+0000","last_active":"2023-11-28T04:07:44.479617+0000","last_peered":"2023-11-28T04:07:44.479617+0000","last_clean":"2023-11-28T04:07:44.479617+0000","last_became_active":"2023-11-28T04:07:44.478269+0000","last_became_peered":"2023-11-28T04:07:44.478269+0000","last_unstale":"2023-11-28T04:07:44.479617+0000","last_undegraded":"2023-11-28T04:07:44.479617+0000","last_fullsized":"2023-11-28T04:07:44.479617+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468765+0000","last_change":"2023-11-28T04:07:44.112458+0000","last_active":"2023-11-28T04:07:44.468765+0000","last_peered":"2023-11-28T04:07:44.468765+0000","last_clean":"2023-11-28T04:07:44.468765+0000","last_became_active":"2023-11-28T04:07:44.112028+0000","last_became_peered":"2023-11-28T04:07:44.112028+0000","last_unstale":"2023-11-28T04:07:44.468765+0000","last_undegraded":"2023-11-28T04:07:44.468765+0000","last_fullsized":"2023-11-28T04:07:44.468765+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481030+0000","last_change":"2023-11-28T04:07:44.481030+0000","last_active":"2023-11-28T04:07:44.481030+0000","last_peered":"2023-11-28T04:07:44.481030+0000","last_clean":"2023-11-28T04:07:44.481030+0000","last_became_active":"2023-11-28T04:07:44.480689+0000","last_became_peered":"2023-11-28T04:07:44.480689+0000","last_unstale":"2023-11-28T04:07:44.481030+0000","last_undegraded":"2023-11-28T04:07:44.481030+0000","last_fullsized":"2023-11-28T04:07:44.481030+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.109658+0000","last_change":"2023-11-28T04:07:44.107204+0000","last_active":"2023-11-28T04:07:44.109658+0000","last_peered":"2023-11-28T04:07:44.109658+0000","last_clean":"2023-11-28T04:07:44.109658+0000","last_became_active":"2023-11-28T04:07:44.106921+0000","last_became_peered":"2023-11-28T04:07:44.106921+0000","last_unstale":"2023-11-28T04:07:44.109658+0000","last_undegraded":"2023-11-28T04:07:44.109658+0000","last_fullsized":"2023-11-28T04:07:44.109658+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483491+0000","last_change":"2023-11-28T04:07:44.480913+0000","last_active":"2023-11-28T04:07:44.483491+0000","last_peered":"2023-11-28T04:07:44.483491+0000","last_clean":"2023-11-28T04:07:44.483491+0000","last_became_active":"2023-11-28T04:07:44.480659+0000","last_became_peered":"2023-11-28T04:07:44.480659+0000","last_unstale":"2023-11-28T04:07:44.483491+0000","last_undegraded":"2023-11-28T04:07:44.483491+0000","last_fullsized":"2023-11-28T04:07:44.483491+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466068+0000","last_change":"2023-11-28T04:07:44.114355+0000","last_active":"2023-11-28T04:07:44.466068+0000","last_peered":"2023-11-28T04:07:44.466068+0000","last_clean":"2023-11-28T04:07:44.466068+0000","last_became_active":"2023-11-28T04:07:44.113617+0000","last_became_peered":"2023-11-28T04:07:44.113617+0000","last_unstale":"2023-11-28T04:07:44.466068+0000","last_undegraded":"2023-11-28T04:07:44.466068+0000","last_fullsized":"2023-11-28T04:07:44.466068+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.481143+0000","last_change":"2023-11-28T04:07:44.481143+0000","last_active":"2023-11-28T04:07:44.481143+0000","last_peered":"2023-11-28T04:07:44.481143+0000","last_clean":"2023-11-28T04:07:44.481143+0000","last_became_active":"2023-11-28T04:07:44.480821+0000","last_became_peered":"2023-11-28T04:07:44.480821+0000","last_unstale":"2023-11-28T04:07:44.481143+0000","last_undegraded":"2023-11-28T04:07:44.481143+0000","last_fullsized":"2023-11-28T04:07:44.481143+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120644+0000","last_change":"2023-11-28T04:07:44.116709+0000","last_active":"2023-11-28T04:07:44.120644+0000","last_peered":"2023-11-28T04:07:44.120644+0000","last_clean":"2023-11-28T04:07:44.120644+0000","last_became_active":"2023-11-28T04:07:44.116378+0000","last_became_peered":"2023-11-28T04:07:44.116378+0000","last_unstale":"2023-11-28T04:07:44.120644+0000","last_undegraded":"2023-11-28T04:07:44.120644+0000","last_fullsized":"2023-11-28T04:07:44.120644+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488886+0000","last_change":"2023-11-28T04:07:44.488886+0000","last_active":"2023-11-28T04:07:44.488886+0000","last_peered":"2023-11-28T04:07:44.488886+0000","last_clean":"2023-11-28T04:07:44.488886+0000","last_became_active":"2023-11-28T04:07:44.488585+0000","last_became_peered":"2023-11-28T04:07:44.488585+0000","last_unstale":"2023-11-28T04:07:44.488886+0000","last_undegraded":"2023-11-28T04:07:44.488886+0000","last_fullsized":"2023-11-28T04:07:44.488886+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114783+0000","last_change":"2023-11-28T04:07:44.109538+0000","last_active":"2023-11-28T04:07:44.114783+0000","last_peered":"2023-11-28T04:07:44.114783+0000","last_clean":"2023-11-28T04:07:44.114783+0000","last_became_active":"2023-11-28T04:07:44.109216+0000","last_became_peered":"2023-11-28T04:07:44.109216+0000","last_unstale":"2023-11-28T04:07:44.114783+0000","last_undegraded":"2023-11-28T04:07:44.114783+0000","last_fullsized":"2023-11-28T04:07:44.114783+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485937+0000","last_change":"2023-11-28T04:07:44.481886+0000","last_active":"2023-11-28T04:07:44.485937+0000","last_peered":"2023-11-28T04:07:44.485937+0000","last_clean":"2023-11-28T04:07:44.485937+0000","last_became_active":"2023-11-28T04:07:44.481610+0000","last_became_peered":"2023-11-28T04:07:44.481610+0000","last_unstale":"2023-11-28T04:07:44.485937+0000","last_undegraded":"2023-11-28T04:07:44.485937+0000","last_fullsized":"2023-11-28T04:07:44.485937+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468972+0000","last_change":"2023-11-28T04:07:44.110589+0000","last_active":"2023-11-28T04:07:44.468972+0000","last_peered":"2023-11-28T04:07:44.468972+0000","last_clean":"2023-11-28T04:07:44.468972+0000","last_became_active":"2023-11-28T04:07:44.110322+0000","last_became_peered":"2023-11-28T04:07:44.110322+0000","last_unstale":"2023-11-28T04:07:44.468972+0000","last_undegraded":"2023-11-28T04:07:44.468972+0000","last_fullsized":"2023-11-28T04:07:44.468972+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490904+0000","last_change":"2023-11-28T04:07:44.486310+0000","last_active":"2023-11-28T04:07:44.490904+0000","last_peered":"2023-11-28T04:07:44.490904+0000","last_clean":"2023-11-28T04:07:44.490904+0000","last_became_active":"2023-11-28T04:07:44.484189+0000","last_became_peered":"2023-11-28T04:07:44.484189+0000","last_unstale":"2023-11-28T04:07:44.490904+0000","last_undegraded":"2023-11-28T04:07:44.490904+0000","last_fullsized":"2023-11-28T04:07:44.490904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.116574+0000","last_change":"2023-11-28T04:07:44.104470+0000","last_active":"2023-11-28T04:07:44.116574+0000","last_peered":"2023-11-28T04:07:44.116574+0000","last_clean":"2023-11-28T04:07:44.116574+0000","last_became_active":"2023-11-28T04:07:44.104217+0000","last_became_peered":"2023-11-28T04:07:44.104217+0000","last_unstale":"2023-11-28T04:07:44.116574+0000","last_undegraded":"2023-11-28T04:07:44.116574+0000","last_fullsized":"2023-11-28T04:07:44.116574+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490105+0000","last_change":"2023-11-28T04:07:44.481275+0000","last_active":"2023-11-28T04:07:44.490105+0000","last_peered":"2023-11-28T04:07:44.490105+0000","last_clean":"2023-11-28T04:07:44.490105+0000","last_became_active":"2023-11-28T04:07:44.481006+0000","last_became_peered":"2023-11-28T04:07:44.481006+0000","last_unstale":"2023-11-28T04:07:44.490105+0000","last_undegraded":"2023-11-28T04:07:44.490105+0000","last_fullsized":"2023-11-28T04:07:44.490105+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470540+0000","last_change":"2023-11-28T04:07:44.111555+0000","last_active":"2023-11-28T04:07:44.470540+0000","last_peered":"2023-11-28T04:07:44.470540+0000","last_clean":"2023-11-28T04:07:44.470540+0000","last_became_active":"2023-11-28T04:07:44.111268+0000","last_became_peered":"2023-11-28T04:07:44.111268+0000","last_unstale":"2023-11-28T04:07:44.470540+0000","last_undegraded":"2023-11-28T04:07:44.470540+0000","last_fullsized":"2023-11-28T04:07:44.470540+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.477017+0000","last_change":"2023-11-28T04:07:44.477017+0000","last_active":"2023-11-28T04:07:44.477017+0000","last_peered":"2023-11-28T04:07:44.477017+0000","last_clean":"2023-11-28T04:07:44.477017+0000","last_became_active":"2023-11-28T04:07:44.476741+0000","last_became_peered":"2023-11-28T04:07:44.476741+0000","last_unstale":"2023-11-28T04:07:44.477017+0000","last_undegraded":"2023-11-28T04:07:44.477017+0000","last_fullsized":"2023-11-28T04:07:44.477017+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117327+0000","last_change":"2023-11-28T04:07:44.110023+0000","last_active":"2023-11-28T04:07:44.117327+0000","last_peered":"2023-11-28T04:07:44.117327+0000","last_clean":"2023-11-28T04:07:44.117327+0000","last_became_active":"2023-11-28T04:07:44.109750+0000","last_became_peered":"2023-11-28T04:07:44.109750+0000","last_unstale":"2023-11-28T04:07:44.117327+0000","last_undegraded":"2023-11-28T04:07:44.117327+0000","last_fullsized":"2023-11-28T04:07:44.117327+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488305+0000","last_change":"2023-11-28T04:07:44.488305+0000","last_active":"2023-11-28T04:07:44.488305+0000","last_peered":"2023-11-28T04:07:44.488305+0000","last_clean":"2023-11-28T04:07:44.488305+0000","last_became_active":"2023-11-28T04:07:44.487406+0000","last_became_peered":"2023-11-28T04:07:44.487406+0000","last_unstale":"2023-11-28T04:07:44.488305+0000","last_undegraded":"2023-11-28T04:07:44.488305+0000","last_fullsized":"2023-11-28T04:07:44.488305+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.117388+0000","last_change":"2023-11-28T04:07:44.110617+0000","last_active":"2023-11-28T04:07:44.117388+0000","last_peered":"2023-11-28T04:07:44.117388+0000","last_clean":"2023-11-28T04:07:44.117388+0000","last_became_active":"2023-11-28T04:07:44.110327+0000","last_became_peered":"2023-11-28T04:07:44.110327+0000","last_unstale":"2023-11-28T04:07:44.117388+0000","last_undegraded":"2023-11-28T04:07:44.117388+0000","last_fullsized":"2023-11-28T04:07:44.117388+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490379+0000","last_change":"2023-11-28T04:07:44.483626+0000","last_active":"2023-11-28T04:07:44.490379+0000","last_peered":"2023-11-28T04:07:44.490379+0000","last_clean":"2023-11-28T04:07:44.490379+0000","last_became_active":"2023-11-28T04:07:44.483359+0000","last_became_peered":"2023-11-28T04:07:44.483359+0000","last_unstale":"2023-11-28T04:07:44.490379+0000","last_undegraded":"2023-11-28T04:07:44.490379+0000","last_fullsized":"2023-11-28T04:07:44.490379+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121180+0000","last_change":"2023-11-28T04:07:44.118163+0000","last_active":"2023-11-28T04:07:44.121180+0000","last_peered":"2023-11-28T04:07:44.121180+0000","last_clean":"2023-11-28T04:07:44.121180+0000","last_became_active":"2023-11-28T04:07:44.117786+0000","last_became_peered":"2023-11-28T04:07:44.117786+0000","last_unstale":"2023-11-28T04:07:44.121180+0000","last_undegraded":"2023-11-28T04:07:44.121180+0000","last_fullsized":"2023-11-28T04:07:44.121180+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489230+0000","last_change":"2023-11-28T04:07:44.476435+0000","last_active":"2023-11-28T04:07:44.489230+0000","last_peered":"2023-11-28T04:07:44.489230+0000","last_clean":"2023-11-28T04:07:44.489230+0000","last_became_active":"2023-11-28T04:07:44.476148+0000","last_became_peered":"2023-11-28T04:07:44.476148+0000","last_unstale":"2023-11-28T04:07:44.489230+0000","last_undegraded":"2023-11-28T04:07:44.489230+0000","last_fullsized":"2023-11-28T04:07:44.489230+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468541+0000","last_change":"2023-11-28T04:07:44.102476+0000","last_active":"2023-11-28T04:07:44.468541+0000","last_peered":"2023-11-28T04:07:44.468541+0000","last_clean":"2023-11-28T04:07:44.468541+0000","last_became_active":"2023-11-28T04:07:44.102178+0000","last_became_peered":"2023-11-28T04:07:44.102178+0000","last_unstale":"2023-11-28T04:07:44.468541+0000","last_undegraded":"2023-11-28T04:07:44.468541+0000","last_fullsized":"2023-11-28T04:07:44.468541+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487891+0000","last_change":"2023-11-28T04:07:44.483294+0000","last_active":"2023-11-28T04:07:44.487891+0000","last_peered":"2023-11-28T04:07:44.487891+0000","last_clean":"2023-11-28T04:07:44.487891+0000","last_became_active":"2023-11-28T04:07:44.483044+0000","last_became_peered":"2023-11-28T04:07:44.483044+0000","last_unstale":"2023-11-28T04:07:44.487891+0000","last_undegraded":"2023-11-28T04:07:44.487891+0000","last_fullsized":"2023-11-28T04:07:44.487891+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489393+0000","last_change":"2023-11-28T04:07:44.110602+0000","last_active":"2023-11-28T04:07:44.489393+0000","last_peered":"2023-11-28T04:07:44.489393+0000","last_clean":"2023-11-28T04:07:44.489393+0000","last_became_active":"2023-11-28T04:07:44.110301+0000","last_became_peered":"2023-11-28T04:07:44.110301+0000","last_unstale":"2023-11-28T04:07:44.489393+0000","last_undegraded":"2023-11-28T04:07:44.489393+0000","last_fullsized":"2023-11-28T04:07:44.489393+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482085+0000","last_change":"2023-11-28T04:07:44.482085+0000","last_active":"2023-11-28T04:07:44.482085+0000","last_peered":"2023-11-28T04:07:44.482085+0000","last_clean":"2023-11-28T04:07:44.482085+0000","last_became_active":"2023-11-28T04:07:44.481145+0000","last_became_peered":"2023-11-28T04:07:44.481145+0000","last_unstale":"2023-11-28T04:07:44.482085+0000","last_undegraded":"2023-11-28T04:07:44.482085+0000","last_fullsized":"2023-11-28T04:07:44.482085+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479304+0000","last_change":"2023-11-28T04:07:44.113764+0000","last_active":"2023-11-28T04:07:44.479304+0000","last_peered":"2023-11-28T04:07:44.479304+0000","last_clean":"2023-11-28T04:07:44.479304+0000","last_became_active":"2023-11-28T04:07:44.113322+0000","last_became_peered":"2023-11-28T04:07:44.113322+0000","last_unstale":"2023-11-28T04:07:44.479304+0000","last_undegraded":"2023-11-28T04:07:44.479304+0000","last_fullsized":"2023-11-28T04:07:44.479304+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492088+0000","last_change":"2023-11-28T04:07:44.485956+0000","last_active":"2023-11-28T04:07:44.492088+0000","last_peered":"2023-11-28T04:07:44.492088+0000","last_clean":"2023-11-28T04:07:44.492088+0000","last_became_active":"2023-11-28T04:07:44.485694+0000","last_became_peered":"2023-11-28T04:07:44.485694+0000","last_unstale":"2023-11-28T04:07:44.492088+0000","last_undegraded":"2023-11-28T04:07:44.492088+0000","last_fullsized":"2023-11-28T04:07:44.492088+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487587+0000","last_change":"2023-11-28T04:07:44.110797+0000","last_active":"2023-11-28T04:07:44.487587+0000","last_peered":"2023-11-28T04:07:44.487587+0000","last_clean":"2023-11-28T04:07:44.487587+0000","last_became_active":"2023-11-28T04:07:44.110494+0000","last_became_peered":"2023-11-28T04:07:44.110494+0000","last_unstale":"2023-11-28T04:07:44.487587+0000","last_undegraded":"2023-11-28T04:07:44.487587+0000","last_fullsized":"2023-11-28T04:07:44.487587+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483810+0000","last_change":"2023-11-28T04:07:44.483810+0000","last_active":"2023-11-28T04:07:44.483810+0000","last_peered":"2023-11-28T04:07:44.483810+0000","last_clean":"2023-11-28T04:07:44.483810+0000","last_became_active":"2023-11-28T04:07:44.483553+0000","last_became_peered":"2023-11-28T04:07:44.483553+0000","last_unstale":"2023-11-28T04:07:44.483810+0000","last_undegraded":"2023-11-28T04:07:44.483810+0000","last_fullsized":"2023-11-28T04:07:44.483810+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120789+0000","last_change":"2023-11-28T04:07:44.115588+0000","last_active":"2023-11-28T04:07:44.120789+0000","last_peered":"2023-11-28T04:07:44.120789+0000","last_clean":"2023-11-28T04:07:44.120789+0000","last_became_active":"2023-11-28T04:07:44.115367+0000","last_became_peered":"2023-11-28T04:07:44.115367+0000","last_unstale":"2023-11-28T04:07:44.120789+0000","last_undegraded":"2023-11-28T04:07:44.120789+0000","last_fullsized":"2023-11-28T04:07:44.120789+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485918+0000","last_change":"2023-11-28T04:07:44.481858+0000","last_active":"2023-11-28T04:07:44.485918+0000","last_peered":"2023-11-28T04:07:44.485918+0000","last_clean":"2023-11-28T04:07:44.485918+0000","last_became_active":"2023-11-28T04:07:44.481571+0000","last_became_peered":"2023-11-28T04:07:44.481571+0000","last_unstale":"2023-11-28T04:07:44.485918+0000","last_undegraded":"2023-11-28T04:07:44.485918+0000","last_fullsized":"2023-11-28T04:07:44.485918+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.471609+0000","last_change":"2023-11-28T04:07:44.105489+0000","last_active":"2023-11-28T04:07:44.471609+0000","last_peered":"2023-11-28T04:07:44.471609+0000","last_clean":"2023-11-28T04:07:44.471609+0000","last_became_active":"2023-11-28T04:07:44.105080+0000","last_became_peered":"2023-11-28T04:07:44.105080+0000","last_unstale":"2023-11-28T04:07:44.471609+0000","last_undegraded":"2023-11-28T04:07:44.471609+0000","last_fullsized":"2023-11-28T04:07:44.471609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480691+0000","last_change":"2023-11-28T04:07:44.477905+0000","last_active":"2023-11-28T04:07:44.480691+0000","last_peered":"2023-11-28T04:07:44.480691+0000","last_clean":"2023-11-28T04:07:44.480691+0000","last_became_active":"2023-11-28T04:07:44.477545+0000","last_became_peered":"2023-11-28T04:07:44.477545+0000","last_unstale":"2023-11-28T04:07:44.480691+0000","last_undegraded":"2023-11-28T04:07:44.480691+0000","last_fullsized":"2023-11-28T04:07:44.480691+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.464896+0000","last_change":"2023-11-28T04:07:44.113284+0000","last_active":"2023-11-28T04:07:44.464896+0000","last_peered":"2023-11-28T04:07:44.464896+0000","last_clean":"2023-11-28T04:07:44.464896+0000","last_became_active":"2023-11-28T04:07:44.113040+0000","last_became_peered":"2023-11-28T04:07:44.113040+0000","last_unstale":"2023-11-28T04:07:44.464896+0000","last_undegraded":"2023-11-28T04:07:44.464896+0000","last_fullsized":"2023-11-28T04:07:44.464896+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486435+0000","last_change":"2023-11-28T04:07:44.485576+0000","last_active":"2023-11-28T04:07:44.486435+0000","last_peered":"2023-11-28T04:07:44.486435+0000","last_clean":"2023-11-28T04:07:44.486435+0000","last_became_active":"2023-11-28T04:07:44.485296+0000","last_became_peered":"2023-11-28T04:07:44.485296+0000","last_unstale":"2023-11-28T04:07:44.486435+0000","last_undegraded":"2023-11-28T04:07:44.486435+0000","last_fullsized":"2023-11-28T04:07:44.486435+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113386+0000","last_change":"2023-11-28T04:07:44.109213+0000","last_active":"2023-11-28T04:07:44.113386+0000","last_peered":"2023-11-28T04:07:44.113386+0000","last_clean":"2023-11-28T04:07:44.113386+0000","last_became_active":"2023-11-28T04:07:44.108530+0000","last_became_peered":"2023-11-28T04:07:44.108530+0000","last_unstale":"2023-11-28T04:07:44.113386+0000","last_undegraded":"2023-11-28T04:07:44.113386+0000","last_fullsized":"2023-11-28T04:07:44.113386+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479590+0000","last_change":"2023-11-28T04:07:44.479590+0000","last_active":"2023-11-28T04:07:44.479590+0000","last_peered":"2023-11-28T04:07:44.479590+0000","last_clean":"2023-11-28T04:07:44.479590+0000","last_became_active":"2023-11-28T04:07:44.479336+0000","last_became_peered":"2023-11-28T04:07:44.479336+0000","last_unstale":"2023-11-28T04:07:44.479590+0000","last_undegraded":"2023-11-28T04:07:44.479590+0000","last_fullsized":"2023-11-28T04:07:44.479590+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466934+0000","last_change":"2023-11-28T04:07:44.108560+0000","last_active":"2023-11-28T04:07:44.466934+0000","last_peered":"2023-11-28T04:07:44.466934+0000","last_clean":"2023-11-28T04:07:44.466934+0000","last_became_active":"2023-11-28T04:07:44.108259+0000","last_became_peered":"2023-11-28T04:07:44.108259+0000","last_unstale":"2023-11-28T04:07:44.466934+0000","last_undegraded":"2023-11-28T04:07:44.466934+0000","last_fullsized":"2023-11-28T04:07:44.466934+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488964+0000","last_change":"2023-11-28T04:07:44.474715+0000","last_active":"2023-11-28T04:07:44.488964+0000","last_peered":"2023-11-28T04:07:44.488964+0000","last_clean":"2023-11-28T04:07:44.488964+0000","last_became_active":"2023-11-28T04:07:44.474434+0000","last_became_peered":"2023-11-28T04:07:44.474434+0000","last_unstale":"2023-11-28T04:07:44.488964+0000","last_undegraded":"2023-11-28T04:07:44.488964+0000","last_fullsized":"2023-11-28T04:07:44.488964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489782+0000","last_change":"2023-11-28T04:07:44.113469+0000","last_active":"2023-11-28T04:07:44.489782+0000","last_peered":"2023-11-28T04:07:44.489782+0000","last_clean":"2023-11-28T04:07:44.489782+0000","last_became_active":"2023-11-28T04:07:44.113235+0000","last_became_peered":"2023-11-28T04:07:44.113235+0000","last_unstale":"2023-11-28T04:07:44.489782+0000","last_undegraded":"2023-11-28T04:07:44.489782+0000","last_fullsized":"2023-11-28T04:07:44.489782+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486375+0000","last_change":"2023-11-28T04:07:44.486375+0000","last_active":"2023-11-28T04:07:44.486375+0000","last_peered":"2023-11-28T04:07:44.486375+0000","last_clean":"2023-11-28T04:07:44.486375+0000","last_became_active":"2023-11-28T04:07:44.485929+0000","last_became_peered":"2023-11-28T04:07:44.485929+0000","last_unstale":"2023-11-28T04:07:44.486375+0000","last_undegraded":"2023-11-28T04:07:44.486375+0000","last_fullsized":"2023-11-28T04:07:44.486375+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490200+0000","last_change":"2023-11-28T04:07:44.110425+0000","last_active":"2023-11-28T04:07:44.490200+0000","last_peered":"2023-11-28T04:07:44.490200+0000","last_clean":"2023-11-28T04:07:44.490200+0000","last_became_active":"2023-11-28T04:07:44.110130+0000","last_became_peered":"2023-11-28T04:07:44.110130+0000","last_unstale":"2023-11-28T04:07:44.490200+0000","last_undegraded":"2023-11-28T04:07:44.490200+0000","last_fullsized":"2023-11-28T04:07:44.490200+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480313+0000","last_change":"2023-11-28T04:07:44.480313+0000","last_active":"2023-11-28T04:07:44.480313+0000","last_peered":"2023-11-28T04:07:44.480313+0000","last_clean":"2023-11-28T04:07:44.480313+0000","last_became_active":"2023-11-28T04:07:44.480057+0000","last_became_peered":"2023-11-28T04:07:44.480057+0000","last_unstale":"2023-11-28T04:07:44.480313+0000","last_undegraded":"2023-11-28T04:07:44.480313+0000","last_fullsized":"2023-11-28T04:07:44.480313+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113510+0000","last_change":"2023-11-28T04:07:44.109074+0000","last_active":"2023-11-28T04:07:44.113510+0000","last_peered":"2023-11-28T04:07:44.113510+0000","last_clean":"2023-11-28T04:07:44.113510+0000","last_became_active":"2023-11-28T04:07:44.108769+0000","last_became_peered":"2023-11-28T04:07:44.108769+0000","last_unstale":"2023-11-28T04:07:44.113510+0000","last_undegraded":"2023-11-28T04:07:44.113510+0000","last_fullsized":"2023-11-28T04:07:44.113510+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475563+0000","last_change":"2023-11-28T04:07:44.475563+0000","last_active":"2023-11-28T04:07:44.475563+0000","last_peered":"2023-11-28T04:07:44.475563+0000","last_clean":"2023-11-28T04:07:44.475563+0000","last_became_active":"2023-11-28T04:07:44.475180+0000","last_became_peered":"2023-11-28T04:07:44.475180+0000","last_unstale":"2023-11-28T04:07:44.475563+0000","last_undegraded":"2023-11-28T04:07:44.475563+0000","last_fullsized":"2023-11-28T04:07:44.475563+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465693+0000","last_change":"2023-11-28T04:07:44.110327+0000","last_active":"2023-11-28T04:07:44.465693+0000","last_peered":"2023-11-28T04:07:44.465693+0000","last_clean":"2023-11-28T04:07:44.465693+0000","last_became_active":"2023-11-28T04:07:44.110082+0000","last_became_peered":"2023-11-28T04:07:44.110082+0000","last_unstale":"2023-11-28T04:07:44.465693+0000","last_undegraded":"2023-11-28T04:07:44.465693+0000","last_fullsized":"2023-11-28T04:07:44.465693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.476765+0000","last_change":"2023-11-28T04:07:44.473362+0000","last_active":"2023-11-28T04:07:44.476765+0000","last_peered":"2023-11-28T04:07:44.476765+0000","last_clean":"2023-11-28T04:07:44.476765+0000","last_became_active":"2023-11-28T04:07:44.473079+0000","last_became_peered":"2023-11-28T04:07:44.473079+0000","last_unstale":"2023-11-28T04:07:44.476765+0000","last_undegraded":"2023-11-28T04:07:44.476765+0000","last_fullsized":"2023-11-28T04:07:44.476765+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489420+0000","last_change":"2023-11-28T04:07:44.102468+0000","last_active":"2023-11-28T04:07:44.489420+0000","last_peered":"2023-11-28T04:07:44.489420+0000","last_clean":"2023-11-28T04:07:44.489420+0000","last_became_active":"2023-11-28T04:07:44.102166+0000","last_became_peered":"2023-11-28T04:07:44.102166+0000","last_unstale":"2023-11-28T04:07:44.489420+0000","last_undegraded":"2023-11-28T04:07:44.489420+0000","last_fullsized":"2023-11-28T04:07:44.489420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482885+0000","last_change":"2023-11-28T04:07:44.482885+0000","last_active":"2023-11-28T04:07:44.482885+0000","last_peered":"2023-11-28T04:07:44.482885+0000","last_clean":"2023-11-28T04:07:44.482885+0000","last_became_active":"2023-11-28T04:07:44.482557+0000","last_became_peered":"2023-11-28T04:07:44.482557+0000","last_unstale":"2023-11-28T04:07:44.482885+0000","last_undegraded":"2023-11-28T04:07:44.482885+0000","last_fullsized":"2023-11-28T04:07:44.482885+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.108281+0000","last_change":"2023-11-28T04:07:44.102694+0000","last_active":"2023-11-28T04:07:44.108281+0000","last_peered":"2023-11-28T04:07:44.108281+0000","last_clean":"2023-11-28T04:07:44.108281+0000","last_became_active":"2023-11-28T04:07:44.102089+0000","last_became_peered":"2023-11-28T04:07:44.102089+0000","last_unstale":"2023-11-28T04:07:44.108281+0000","last_undegraded":"2023-11-28T04:07:44.108281+0000","last_fullsized":"2023-11-28T04:07:44.108281+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489965+0000","last_change":"2023-11-28T04:07:44.489160+0000","last_active":"2023-11-28T04:07:44.489965+0000","last_peered":"2023-11-28T04:07:44.489965+0000","last_clean":"2023-11-28T04:07:44.489965+0000","last_became_active":"2023-11-28T04:07:44.488851+0000","last_became_peered":"2023-11-28T04:07:44.488851+0000","last_unstale":"2023-11-28T04:07:44.489965+0000","last_undegraded":"2023-11-28T04:07:44.489965+0000","last_fullsized":"2023-11-28T04:07:44.489965+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121165+0000","last_change":"2023-11-28T04:07:44.118105+0000","last_active":"2023-11-28T04:07:44.121165+0000","last_peered":"2023-11-28T04:07:44.121165+0000","last_clean":"2023-11-28T04:07:44.121165+0000","last_became_active":"2023-11-28T04:07:44.117793+0000","last_became_peered":"2023-11-28T04:07:44.117793+0000","last_unstale":"2023-11-28T04:07:44.121165+0000","last_undegraded":"2023-11-28T04:07:44.121165+0000","last_fullsized":"2023-11-28T04:07:44.121165+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494822+0000","last_change":"2023-11-28T04:07:44.491014+0000","last_active":"2023-11-28T04:07:44.494822+0000","last_peered":"2023-11-28T04:07:44.494822+0000","last_clean":"2023-11-28T04:07:44.494822+0000","last_became_active":"2023-11-28T04:07:44.490714+0000","last_became_peered":"2023-11-28T04:07:44.490714+0000","last_unstale":"2023-11-28T04:07:44.494822+0000","last_undegraded":"2023-11-28T04:07:44.494822+0000","last_fullsized":"2023-11-28T04:07:44.494822+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.114821+0000","last_change":"2023-11-28T04:07:44.110080+0000","last_active":"2023-11-28T04:07:44.114821+0000","last_peered":"2023-11-28T04:07:44.114821+0000","last_clean":"2023-11-28T04:07:44.114821+0000","last_became_active":"2023-11-28T04:07:44.109823+0000","last_became_peered":"2023-11-28T04:07:44.109823+0000","last_unstale":"2023-11-28T04:07:44.114821+0000","last_undegraded":"2023-11-28T04:07:44.114821+0000","last_fullsized":"2023-11-28T04:07:44.114821+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479651+0000","last_change":"2023-11-28T04:07:44.477357+0000","last_active":"2023-11-28T04:07:44.479651+0000","last_peered":"2023-11-28T04:07:44.479651+0000","last_clean":"2023-11-28T04:07:44.479651+0000","last_became_active":"2023-11-28T04:07:44.477078+0000","last_became_peered":"2023-11-28T04:07:44.477078+0000","last_unstale":"2023-11-28T04:07:44.479651+0000","last_undegraded":"2023-11-28T04:07:44.479651+0000","last_fullsized":"2023-11-28T04:07:44.479651+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488609+0000","last_change":"2023-11-28T04:07:44.110131+0000","last_active":"2023-11-28T04:07:44.488609+0000","last_peered":"2023-11-28T04:07:44.488609+0000","last_clean":"2023-11-28T04:07:44.488609+0000","last_became_active":"2023-11-28T04:07:44.109844+0000","last_became_peered":"2023-11-28T04:07:44.109844+0000","last_unstale":"2023-11-28T04:07:44.488609+0000","last_undegraded":"2023-11-28T04:07:44.488609+0000","last_fullsized":"2023-11-28T04:07:44.488609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489248+0000","last_change":"2023-11-28T04:07:44.475348+0000","last_active":"2023-11-28T04:07:44.489248+0000","last_peered":"2023-11-28T04:07:44.489248+0000","last_clean":"2023-11-28T04:07:44.489248+0000","last_became_active":"2023-11-28T04:07:44.475089+0000","last_became_peered":"2023-11-28T04:07:44.475089+0000","last_unstale":"2023-11-28T04:07:44.489248+0000","last_undegraded":"2023-11-28T04:07:44.489248+0000","last_fullsized":"2023-11-28T04:07:44.489248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472753+0000","last_change":"2023-11-28T04:07:44.113266+0000","last_active":"2023-11-28T04:07:44.472753+0000","last_peered":"2023-11-28T04:07:44.472753+0000","last_clean":"2023-11-28T04:07:44.472753+0000","last_became_active":"2023-11-28T04:07:44.113000+0000","last_became_peered":"2023-11-28T04:07:44.113000+0000","last_unstale":"2023-11-28T04:07:44.472753+0000","last_undegraded":"2023-11-28T04:07:44.472753+0000","last_fullsized":"2023-11-28T04:07:44.472753+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489755+0000","last_change":"2023-11-28T04:07:44.489755+0000","last_active":"2023-11-28T04:07:44.489755+0000","last_peered":"2023-11-28T04:07:44.489755+0000","last_clean":"2023-11-28T04:07:44.489755+0000","last_became_active":"2023-11-28T04:07:44.489466+0000","last_became_peered":"2023-11-28T04:07:44.489466+0000","last_unstale":"2023-11-28T04:07:44.489755+0000","last_undegraded":"2023-11-28T04:07:44.489755+0000","last_fullsized":"2023-11-28T04:07:44.489755+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468894+0000","last_change":"2023-11-28T04:07:44.109468+0000","last_active":"2023-11-28T04:07:44.468894+0000","last_peered":"2023-11-28T04:07:44.468894+0000","last_clean":"2023-11-28T04:07:44.468894+0000","last_became_active":"2023-11-28T04:07:44.109138+0000","last_became_peered":"2023-11-28T04:07:44.109138+0000","last_unstale":"2023-11-28T04:07:44.468894+0000","last_undegraded":"2023-11-28T04:07:44.468894+0000","last_fullsized":"2023-11-28T04:07:44.468894+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.477053+0000","last_change":"2023-11-28T04:07:44.477053+0000","last_active":"2023-11-28T04:07:44.477053+0000","last_peered":"2023-11-28T04:07:44.477053+0000","last_clean":"2023-11-28T04:07:44.477053+0000","last_became_active":"2023-11-28T04:07:44.476805+0000","last_became_peered":"2023-11-28T04:07:44.476805+0000","last_unstale":"2023-11-28T04:07:44.477053+0000","last_undegraded":"2023-11-28T04:07:44.477053+0000","last_fullsized":"2023-11-28T04:07:44.477053+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113468+0000","last_change":"2023-11-28T04:07:44.109113+0000","last_active":"2023-11-28T04:07:44.113468+0000","last_peered":"2023-11-28T04:07:44.113468+0000","last_clean":"2023-11-28T04:07:44.113468+0000","last_became_active":"2023-11-28T04:07:44.108854+0000","last_became_peered":"2023-11-28T04:07:44.108854+0000","last_unstale":"2023-11-28T04:07:44.113468+0000","last_undegraded":"2023-11-28T04:07:44.113468+0000","last_fullsized":"2023-11-28T04:07:44.113468+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484405+0000","last_change":"2023-11-28T04:07:44.484405+0000","last_active":"2023-11-28T04:07:44.484405+0000","last_peered":"2023-11-28T04:07:44.484405+0000","last_clean":"2023-11-28T04:07:44.484405+0000","last_became_active":"2023-11-28T04:07:44.484145+0000","last_became_peered":"2023-11-28T04:07:44.484145+0000","last_unstale":"2023-11-28T04:07:44.484405+0000","last_undegraded":"2023-11-28T04:07:44.484405+0000","last_fullsized":"2023-11-28T04:07:44.484405+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470044+0000","last_change":"2023-11-28T04:07:44.117498+0000","last_active":"2023-11-28T04:07:44.470044+0000","last_peered":"2023-11-28T04:07:44.470044+0000","last_clean":"2023-11-28T04:07:44.470044+0000","last_became_active":"2023-11-28T04:07:44.117094+0000","last_became_peered":"2023-11-28T04:07:44.117094+0000","last_unstale":"2023-11-28T04:07:44.470044+0000","last_undegraded":"2023-11-28T04:07:44.470044+0000","last_fullsized":"2023-11-28T04:07:44.470044+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478155+0000","last_change":"2023-11-28T04:07:44.478155+0000","last_active":"2023-11-28T04:07:44.478155+0000","last_peered":"2023-11-28T04:07:44.478155+0000","last_clean":"2023-11-28T04:07:44.478155+0000","last_became_active":"2023-11-28T04:07:44.477831+0000","last_became_peered":"2023-11-28T04:07:44.477831+0000","last_unstale":"2023-11-28T04:07:44.478155+0000","last_undegraded":"2023-11-28T04:07:44.478155+0000","last_fullsized":"2023-11-28T04:07:44.478155+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.120782+0000","last_change":"2023-11-28T04:07:44.117447+0000","last_active":"2023-11-28T04:07:44.120782+0000","last_peered":"2023-11-28T04:07:44.120782+0000","last_clean":"2023-11-28T04:07:44.120782+0000","last_became_active":"2023-11-28T04:07:44.117157+0000","last_became_peered":"2023-11-28T04:07:44.117157+0000","last_unstale":"2023-11-28T04:07:44.120782+0000","last_undegraded":"2023-11-28T04:07:44.120782+0000","last_fullsized":"2023-11-28T04:07:44.120782+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486163+0000","last_change":"2023-11-28T04:07:44.486163+0000","last_active":"2023-11-28T04:07:44.486163+0000","last_peered":"2023-11-28T04:07:44.486163+0000","last_clean":"2023-11-28T04:07:44.486163+0000","last_became_active":"2023-11-28T04:07:44.485864+0000","last_became_peered":"2023-11-28T04:07:44.485864+0000","last_unstale":"2023-11-28T04:07:44.486163+0000","last_undegraded":"2023-11-28T04:07:44.486163+0000","last_fullsized":"2023-11-28T04:07:44.486163+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.113406+0000","last_change":"2023-11-28T04:07:44.109265+0000","last_active":"2023-11-28T04:07:44.113406+0000","last_peered":"2023-11-28T04:07:44.113406+0000","last_clean":"2023-11-28T04:07:44.113406+0000","last_became_active":"2023-11-28T04:07:44.108960+0000","last_became_peered":"2023-11-28T04:07:44.108960+0000","last_unstale":"2023-11-28T04:07:44.113406+0000","last_undegraded":"2023-11-28T04:07:44.113406+0000","last_fullsized":"2023-11-28T04:07:44.113406+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489939+0000","last_change":"2023-11-28T04:07:44.489939+0000","last_active":"2023-11-28T04:07:44.489939+0000","last_peered":"2023-11-28T04:07:44.489939+0000","last_clean":"2023-11-28T04:07:44.489939+0000","last_became_active":"2023-11-28T04:07:44.489622+0000","last_became_peered":"2023-11-28T04:07:44.489622+0000","last_unstale":"2023-11-28T04:07:44.489939+0000","last_undegraded":"2023-11-28T04:07:44.489939+0000","last_fullsized":"2023-11-28T04:07:44.489939+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465855+0000","last_change":"2023-11-28T04:07:44.108144+0000","last_active":"2023-11-28T04:07:44.465855+0000","last_peered":"2023-11-28T04:07:44.465855+0000","last_clean":"2023-11-28T04:07:44.465855+0000","last_became_active":"2023-11-28T04:07:44.107822+0000","last_became_peered":"2023-11-28T04:07:44.107822+0000","last_unstale":"2023-11-28T04:07:44.465855+0000","last_undegraded":"2023-11-28T04:07:44.465855+0000","last_fullsized":"2023-11-28T04:07:44.465855+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490884+0000","last_change":"2023-11-28T04:07:44.484397+0000","last_active":"2023-11-28T04:07:44.490884+0000","last_peered":"2023-11-28T04:07:44.490884+0000","last_clean":"2023-11-28T04:07:44.490884+0000","last_became_active":"2023-11-28T04:07:44.484150+0000","last_became_peered":"2023-11-28T04:07:44.484150+0000","last_unstale":"2023-11-28T04:07:44.490884+0000","last_undegraded":"2023-11-28T04:07:44.490884+0000","last_fullsized":"2023-11-28T04:07:44.490884+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"164'3","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491610+0000","last_change":"2023-11-28T04:07:44.108445+0000","last_active":"2023-11-28T04:07:44.491610+0000","last_peered":"2023-11-28T04:07:44.491610+0000","last_clean":"2023-11-28T04:07:44.491610+0000","last_became_active":"2023-11-28T04:07:44.108144+0000","last_became_peered":"2023-11-28T04:07:44.108144+0000","last_unstale":"2023-11-28T04:07:44.491610+0000","last_undegraded":"2023-11-28T04:07:44.491610+0000","last_fullsized":"2023-11-28T04:07:44.491610+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":132,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.104011+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:07:44.104011+0000","last_peered":"2023-11-28T04:07:44.104011+0000","last_clean":"2023-11-28T04:07:44.104011+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:07:44.104011+0000","last_undegraded":"2023-11-28T04:07:44.104011+0000","last_fullsized":"2023-11-28T04:07:44.104011+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470645+0000","last_change":"2023-11-28T04:07:44.470645+0000","last_active":"2023-11-28T04:07:44.470645+0000","last_peered":"2023-11-28T04:07:44.470645+0000","last_clean":"2023-11-28T04:07:44.470645+0000","last_became_active":"2023-11-28T04:07:44.470356+0000","last_became_peered":"2023-11-28T04:07:44.470356+0000","last_unstale":"2023-11-28T04:07:44.470645+0000","last_undegraded":"2023-11-28T04:07:44.470645+0000","last_fullsized":"2023-11-28T04:07:44.470645+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"164'1","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468436+0000","last_change":"2023-11-28T04:07:44.101210+0000","last_active":"2023-11-28T04:07:44.468436+0000","last_peered":"2023-11-28T04:07:44.468436+0000","last_clean":"2023-11-28T04:07:44.468436+0000","last_became_active":"2023-11-28T04:07:44.100914+0000","last_became_peered":"2023-11-28T04:07:44.100914+0000","last_unstale":"2023-11-28T04:07:44.468436+0000","last_undegraded":"2023-11-28T04:07:44.468436+0000","last_fullsized":"2023-11-28T04:07:44.468436+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472943+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:07:44.472943+0000","last_peered":"2023-11-28T04:07:44.472943+0000","last_clean":"2023-11-28T04:07:44.472943+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:07:44.472943+0000","last_undegraded":"2023-11-28T04:07:44.472943+0000","last_fullsized":"2023-11-28T04:07:44.472943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479562+0000","last_change":"2023-11-28T04:07:44.476618+0000","last_active":"2023-11-28T04:07:44.479562+0000","last_peered":"2023-11-28T04:07:44.479562+0000","last_clean":"2023-11-28T04:07:44.479562+0000","last_became_active":"2023-11-28T04:07:44.476363+0000","last_became_peered":"2023-11-28T04:07:44.476363+0000","last_unstale":"2023-11-28T04:07:44.479562+0000","last_undegraded":"2023-11-28T04:07:44.479562+0000","last_fullsized":"2023-11-28T04:07:44.479562+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"164'4","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490874+0000","last_change":"2023-11-28T04:07:44.109291+0000","last_active":"2023-11-28T04:07:44.490874+0000","last_peered":"2023-11-28T04:07:44.490874+0000","last_clean":"2023-11-28T04:07:44.490874+0000","last_became_active":"2023-11-28T04:07:44.108915+0000","last_became_peered":"2023-11-28T04:07:44.108915+0000","last_unstale":"2023-11-28T04:07:44.490874+0000","last_undegraded":"2023-11-28T04:07:44.490874+0000","last_fullsized":"2023-11-28T04:07:44.490874+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":139,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.091942+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:07:44.091942+0000","last_peered":"2023-11-28T04:07:44.091942+0000","last_clean":"2023-11-28T04:07:44.091942+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:07:44.091942+0000","last_undegraded":"2023-11-28T04:07:44.091942+0000","last_fullsized":"2023-11-28T04:07:44.091942+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489166+0000","last_change":"2023-11-28T04:07:44.476393+0000","last_active":"2023-11-28T04:07:44.489166+0000","last_peered":"2023-11-28T04:07:44.489166+0000","last_clean":"2023-11-28T04:07:44.489166+0000","last_became_active":"2023-11-28T04:07:44.475919+0000","last_became_peered":"2023-11-28T04:07:44.475919+0000","last_unstale":"2023-11-28T04:07:44.489166+0000","last_undegraded":"2023-11-28T04:07:44.489166+0000","last_fullsized":"2023-11-28T04:07:44.489166+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.121175+0000","last_change":"2023-11-28T04:07:44.118149+0000","last_active":"2023-11-28T04:07:44.121175+0000","last_peered":"2023-11-28T04:07:44.121175+0000","last_clean":"2023-11-28T04:07:44.121175+0000","last_became_active":"2023-11-28T04:07:44.117774+0000","last_became_peered":"2023-11-28T04:07:44.117774+0000","last_unstale":"2023-11-28T04:07:44.121175+0000","last_undegraded":"2023-11-28T04:07:44.121175+0000","last_fullsized":"2023-11-28T04:07:44.121175+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470495+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:07:44.470495+0000","last_peered":"2023-11-28T04:07:44.470495+0000","last_clean":"2023-11-28T04:07:44.470495+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:07:44.470495+0000","last_undegraded":"2023-11-28T04:07:44.470495+0000","last_fullsized":"2023-11-28T04:07:44.470495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":133,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.097165+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:07:44.097165+0000","last_peered":"2023-11-28T04:07:44.097165+0000","last_clean":"2023-11-28T04:07:44.097165+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:07:44.097165+0000","last_undegraded":"2023-11-28T04:07:44.097165+0000","last_fullsized":"2023-11-28T04:07:44.097165+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480998+0000","last_change":"2023-11-28T04:07:44.480998+0000","last_active":"2023-11-28T04:07:44.480998+0000","last_peered":"2023-11-28T04:07:44.480998+0000","last_clean":"2023-11-28T04:07:44.480998+0000","last_became_active":"2023-11-28T04:07:44.480742+0000","last_became_peered":"2023-11-28T04:07:44.480742+0000","last_unstale":"2023-11-28T04:07:44.480998+0000","last_undegraded":"2023-11-28T04:07:44.480998+0000","last_fullsized":"2023-11-28T04:07:44.480998+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466623+0000","last_change":"2023-11-28T04:07:44.113995+0000","last_active":"2023-11-28T04:07:44.466623+0000","last_peered":"2023-11-28T04:07:44.466623+0000","last_clean":"2023-11-28T04:07:44.466623+0000","last_became_active":"2023-11-28T04:07:44.113244+0000","last_became_peered":"2023-11-28T04:07:44.113244+0000","last_unstale":"2023-11-28T04:07:44.466623+0000","last_undegraded":"2023-11-28T04:07:44.466623+0000","last_fullsized":"2023-11-28T04:07:44.466623+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.097248+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:07:44.097248+0000","last_peered":"2023-11-28T04:07:44.097248+0000","last_clean":"2023-11-28T04:07:44.097248+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:07:44.097248+0000","last_undegraded":"2023-11-28T04:07:44.097248+0000","last_fullsized":"2023-11-28T04:07:44.097248+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489019+0000","last_change":"2023-11-28T04:07:44.475084+0000","last_active":"2023-11-28T04:07:44.489019+0000","last_peered":"2023-11-28T04:07:44.489019+0000","last_clean":"2023-11-28T04:07:44.489019+0000","last_became_active":"2023-11-28T04:07:44.474807+0000","last_became_peered":"2023-11-28T04:07:44.474807+0000","last_unstale":"2023-11-28T04:07:44.489019+0000","last_undegraded":"2023-11-28T04:07:44.489019+0000","last_fullsized":"2023-11-28T04:07:44.489019+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"164'2","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484554+0000","last_change":"2023-11-28T04:07:44.109170+0000","last_active":"2023-11-28T04:07:44.484554+0000","last_peered":"2023-11-28T04:07:44.484554+0000","last_clean":"2023-11-28T04:07:44.484554+0000","last_became_active":"2023-11-28T04:07:44.108884+0000","last_became_peered":"2023-11-28T04:07:44.108884+0000","last_unstale":"2023-11-28T04:07:44.484554+0000","last_undegraded":"2023-11-28T04:07:44.484554+0000","last_fullsized":"2023-11-28T04:07:44.484554+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":131,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.100643+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:07:44.100643+0000","last_peered":"2023-11-28T04:07:44.100643+0000","last_clean":"2023-11-28T04:07:44.100643+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:07:44.100643+0000","last_undegraded":"2023-11-28T04:07:44.100643+0000","last_fullsized":"2023-11-28T04:07:44.100643+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475198+0000","last_change":"2023-11-28T04:07:44.475198+0000","last_active":"2023-11-28T04:07:44.475198+0000","last_peered":"2023-11-28T04:07:44.475198+0000","last_clean":"2023-11-28T04:07:44.475198+0000","last_became_active":"2023-11-28T04:07:44.474922+0000","last_became_peered":"2023-11-28T04:07:44.474922+0000","last_unstale":"2023-11-28T04:07:44.475198+0000","last_undegraded":"2023-11-28T04:07:44.475198+0000","last_fullsized":"2023-11-28T04:07:44.475198+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475742+0000","last_change":"2023-11-28T04:07:44.112188+0000","last_active":"2023-11-28T04:07:44.475742+0000","last_peered":"2023-11-28T04:07:44.475742+0000","last_clean":"2023-11-28T04:07:44.475742+0000","last_became_active":"2023-11-28T04:07:44.111845+0000","last_became_peered":"2023-11-28T04:07:44.111845+0000","last_unstale":"2023-11-28T04:07:44.475742+0000","last_undegraded":"2023-11-28T04:07:44.475742+0000","last_fullsized":"2023-11-28T04:07:44.475742+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":132,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.104097+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:07:44.104097+0000","last_peered":"2023-11-28T04:07:44.104097+0000","last_clean":"2023-11-28T04:07:44.104097+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:07:44.104097+0000","last_undegraded":"2023-11-28T04:07:44.104097+0000","last_fullsized":"2023-11-28T04:07:44.104097+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":8,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490428+0000","last_change":"2023-11-28T04:07:44.490428+0000","last_active":"2023-11-28T04:07:44.490428+0000","last_peered":"2023-11-28T04:07:44.490428+0000","last_clean":"2023-11-28T04:07:44.490428+0000","last_became_active":"2023-11-28T04:07:44.490238+0000","last_became_peered":"2023-11-28T04:07:44.490238+0000","last_unstale":"2023-11-28T04:07:44.490428+0000","last_undegraded":"2023-11-28T04:07:44.490428+0000","last_fullsized":"2023-11-28T04:07:44.490428+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":163,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.107289+0000","last_change":"2023-11-28T04:07:44.102624+0000","last_active":"2023-11-28T04:07:44.107289+0000","last_peered":"2023-11-28T04:07:44.107289+0000","last_clean":"2023-11-28T04:07:44.107289+0000","last_became_active":"2023-11-28T04:07:44.102114+0000","last_became_peered":"2023-11-28T04:07:44.102114+0000","last_unstale":"2023-11-28T04:07:44.107289+0000","last_undegraded":"2023-11-28T04:07:44.107289+0000","last_fullsized":"2023-11-28T04:07:44.107289+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":146,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467499+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:07:44.467499+0000","last_peered":"2023-11-28T04:07:44.467499+0000","last_clean":"2023-11-28T04:07:44.467499+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:07:44.467499+0000","last_undegraded":"2023-11-28T04:07:44.467499+0000","last_fullsized":"2023-11-28T04:07:44.467499+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575746,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":2052096,"data_stored":1027884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.435},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.379},"max":{"1min":0.72699999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.54800000000000004}]},{"osd":1,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.45900000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.53900000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.51200000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.495}]},{"osd":6,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.875,"15min":0.875},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.52500000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575746,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.81899999999999995},"last":0.46100000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.752,"15min":0.80300000000000005},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.33800000000000002},"max":{"1min":0.68600000000000005,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.57099999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67900000000000005,"5min":0.79500000000000004,"15min":0.79600000000000004},"last":0.498},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.78000000000000003,"15min":0.83099999999999996},"last":0.58699999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.38600000000000001},"max":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79800000000000004},"last":0.55900000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.629},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.79200000000000004},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.67900000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71299999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575745,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303184,"kb_used_data":1992,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068656,"statfs":{"total":96636764160,"available":96326303744,"internally_reserved":0,"allocated":2039808,"data_stored":1027150,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.32400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.36599999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.374}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.60599999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.61799999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.64900000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.59399999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575746,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":2052096,"data_stored":1027276,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.61199999999999999},"last":0.311}]},{"osd":2,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.309,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.54600000000000004,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.34200000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56299999999999994,"5min":0.621,"15min":0.67300000000000004},"last":0.441},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.77500000000000002},"last":0.39800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.74399999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.78500000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.45800000000000002,"15min":0.42499999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.76200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.755},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70599999999999996},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.42599999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834575747,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307288,"kb_used_data":2000,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064552,"statfs":{"total":96636764160,"available":96322101248,"internally_reserved":0,"allocated":2048000,"data_stored":1027738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.376,"15min":0.379},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.33900000000000002,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.625,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.32800000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.39000000000000001,"15min":0.39000000000000001},"min":{"1min":0.308,"5min":0.249,"15min":0.248},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.223},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.41399999999999998,"15min":0.41099999999999998},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26100000000000001},"max":{"1min":0.55500000000000005,"5min":0.58799999999999997,"15min":0.626},"last":0.27600000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.441},"min":{"1min":0.35299999999999998,"5min":0.32100000000000001,"15min":0.30599999999999999},"max":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.435,"5min":0.41299999999999998,"15min":0.41599999999999998},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.27100000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.59799999999999998,"15min":0.60099999999999998},"min":{"1min":0.44900000000000001,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.73499999999999999,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.51200000000000001,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.84299999999999997,"5min":0.93200000000000005,"15min":0.93400000000000005},"last":0.51400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40200000000000002},"max":{"1min":0.85899999999999999,"5min":1.048,"15min":1.048},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.65400000000000003,"15min":0.65600000000000003},"min":{"1min":0.53900000000000003,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.77200000000000002,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.56200000000000006}]},{"osd":6,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.49399999999999999,"5min":0.433,"15min":0.433},"max":{"1min":0.86399999999999999,"5min":0.90800000000000003,"15min":0.997},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.67700000000000005,"15min":0.68100000000000005},"min":{"1min":0.47499999999999998,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.80200000000000005,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:06:44 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.65300000000000002},"min":{"1min":0.505,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.81000000000000005,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65400000000000003,"15min":0.65700000000000003},"min":{"1min":0.51200000000000001,"5min":0.434,"15min":0.434},"max":{"1min":0.83599999999999997,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.47899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575746,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.621,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.35199999999999998},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.36899999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.39800000000000002},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.25700000000000001},"max":{"1min":0.59999999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.26800000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.67600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38200000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.503,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.94799999999999995,"15min":1.085},"last":0.47499999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":1.1200000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.65900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.41599999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.145},"last":0.49399999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.434},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.47099999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.63200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575745,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.71799999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.753}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.78200000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.80600000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.81899999999999995},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.61099999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.76900000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575747,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77800000000000002},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"last":0.498}]},{"osd":1,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.443},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.51100000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.89400000000000002},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.71599999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.378},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.63100000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.69499999999999995,"15min":0.83799999999999997},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.84499999999999997},"last":0.69399999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.79000000000000004},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.752}]},{"osd":7,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.78500000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.81999999999999995},"last":0.70599999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:07:50.383 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:07:50.382+0000 7fe25392b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:07:51.854 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:07:51.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.979+0000 7ff8a9eff700 1 Processor -- start 2023-11-28T04:07:51.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.979+0000 7ff8a9eff700 1 -- start start 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a9eff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40caab0 0x7ff8a40aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a9eff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8a40cae90 0x7ff8a40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a9eff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a40a9a90 con 0x7ff8a40aa960 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a9eff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a4154ce0 con 0x7ff8a40cae90 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a9eff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40a8e80 con 0x7ff8a40caab0 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8a40cae90 0x7ff8a40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40caab0 0x7ff8a40aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff8a40cae90 0x7ff8a40c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:57156/0 (socket says 172.21.15.73:57156) 2023-11-28T04:07:51.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 -- 172.21.15.73:0/643361565 learned_addr learned my addr 172.21.15.73:0/643361565 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 -- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40caab0 msgr2=0x7ff8a40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 --2- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40caab0 0x7ff8a40aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 -- 172.21.15.73:0/643361565 >> v1:172.21.15.150:6789/0 conn(0x7ff8a40aa960 legacy=0x7ff8a40c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.980+0000 7ff8a2ffd700 1 -- 172.21.15.73:0/643361565 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a40a9160 con 0x7ff8a40cae90 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a2ffd700 1 --2- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40cae90 0x7ff8a40c0ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7ff898006fd0 tx=0x7ff8980090f0).ready entity=mon.1 client_cookie=4fcb49b9177e52ec server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a27fc700 1 -- 172.21.15.73:0/643361565 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800b580 con 0x7ff8a40cae90 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a27fc700 1 -- 172.21.15.73:0/643361565 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89800b6e0 con 0x7ff8a40cae90 2023-11-28T04:07:51.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a27fc700 1 -- 172.21.15.73:0/643361565 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89800ba60 con 0x7ff8a40cae90 2023-11-28T04:07:51.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 -- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40cae90 msgr2=0x7ff8a40c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40cae90 0x7ff8a40c0ab0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 rx=0x7ff898006fd0 tx=0x7ff8980090f0).stop 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 -- 172.21.15.73:0/643361565 shutdown_connections 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40caab0 0x7ff8a40aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/643361565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40cae90 0x7ff8a40c0ab0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 -- 172.21.15.73:0/643361565 >> 172.21.15.73:0/643361565 conn(0x7ff8a4094920 msgr2=0x7ff8a406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 -- 172.21.15.73:0/643361565 shutdown_connections 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.981+0000 7ff8a9eff700 1 -- 172.21.15.73:0/643361565 wait complete. 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 Processor -- start 2023-11-28T04:07:51.986 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 -- start start 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40cae90 0x7ff8a40ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40b06a0 0x7ff8a40ad120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40b1180 con 0x7ff8a40caab0 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40b0bc0 con 0x7ff8a40b06a0 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.983+0000 7ff8a9eff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40b0ea0 con 0x7ff8a40cae90 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58696/0 (socket says 172.21.15.73:58696) 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 -- 172.21.15.73:0/3885394738 learned_addr learned my addr 172.21.15.73:0/3885394738 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a2ffd700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40cae90 0x7ff8a40ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a3fff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40b06a0 0x7ff8a40ad120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 -- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40cae90 msgr2=0x7ff8a40ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40cae90 0x7ff8a40ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 -- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40b06a0 msgr2=0x7ff8a40ad120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40b06a0 0x7ff8a40ad120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 -- 172.21.15.73:0/3885394738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff898009370 con 0x7ff8a40caab0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a37fe700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7ff89400e900 tx=0x7ff894006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012600 con 0x7ff8a40caab0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff894012760 con 0x7ff8a40caab0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a4155080 con 0x7ff8a40caab0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff894012ae0 con 0x7ff8a40caab0 2023-11-28T04:07:51.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.984+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a40ae710 con 0x7ff8a40caab0 2023-11-28T04:07:51.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.985+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a406c860 con 0x7ff8a40caab0 2023-11-28T04:07:51.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.986+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff89401b400 con 0x7ff8a40caab0 2023-11-28T04:07:51.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.986+0000 7ff88b7fe700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:51.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.986+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff89407a570 con 0x7ff8a40caab0 2023-11-28T04:07:51.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.986+0000 7ff8a2ffd700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:51.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.987+0000 7ff8a2ffd700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 0x7ff890061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff8a40c6040 tx=0x7ff898017000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:51.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:51.990+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff89404b7e0 con 0x7ff8a40caab0 2023-11-28T04:07:52.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.155+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8a40a9440 con 0x7ff89005f150 2023-11-28T04:07:52.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.165+0000 7ff88b7fe700 1 -- 172.21.15.73:0/3885394738 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361919 (secure 0 0 0) 0x7ff8a40a9440 con 0x7ff89005f150 2023-11-28T04:07:52.170 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 msgr2=0x7ff890061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 0x7ff890061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7ff8a40c6040 tx=0x7ff898017000).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 msgr2=0x7ff8a4151a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 rx=0x7ff89400e900 tx=0x7ff894006f40).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 shutdown_connections 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff89005f150 0x7ff890061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff8a40b06a0 0x7ff8a40ad120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff8a40caab0 0x7ff8a4151a90 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 --2- 172.21.15.73:0/3885394738 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff8a40cae90 0x7ff8a40ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.173 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 >> 172.21.15.73:0/3885394738 conn(0x7ff8a4094920 msgr2=0x7ff8a40ab740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:52.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 shutdown_connections 2023-11-28T04:07:52.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.168+0000 7ff8a9eff700 1 -- 172.21.15.73:0/3885394738 wait complete. 2023-11-28T04:07:52.174 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:07:52.181 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2023-11-28T04:07:52.058368+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2437920,"kb_used_data":16096,"kb_used_omap":0,"kb_used_meta":2421760,"kb_avail":752536800,"statfs":{"total":773094113280,"available":770597683200,"internally_reserved":0,"allocated":16482304,"data_stored":8275444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2479882240},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-5035976,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-520,"num_write_kb":-5311,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":-960,"num_keys_recovered":0,"num_objects_omap":-13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.022463"},"pg_stats":[{"pgid":"20.38","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480557+0000","last_change":"2023-11-28T04:07:44.477800+0000","last_active":"2023-11-28T04:07:44.480557+0000","last_peered":"2023-11-28T04:07:44.480557+0000","last_clean":"2023-11-28T04:07:44.480557+0000","last_became_active":"2023-11-28T04:07:44.477512+0000","last_became_peered":"2023-11-28T04:07:44.477512+0000","last_unstale":"2023-11-28T04:07:44.480557+0000","last_undegraded":"2023-11-28T04:07:44.480557+0000","last_fullsized":"2023-11-28T04:07:44.480557+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3f","version":"164'3","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483636+0000","last_change":"2023-11-28T04:07:44.113370+0000","last_active":"2023-11-28T04:07:44.483636+0000","last_peered":"2023-11-28T04:07:44.483636+0000","last_clean":"2023-11-28T04:07:44.483636+0000","last_became_active":"2023-11-28T04:07:44.113093+0000","last_became_peered":"2023-11-28T04:07:44.113093+0000","last_unstale":"2023-11-28T04:07:44.483636+0000","last_undegraded":"2023-11-28T04:07:44.483636+0000","last_fullsized":"2023-11-28T04:07:44.483636+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.39","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494697+0000","last_change":"2023-11-28T04:07:44.491520+0000","last_active":"2023-11-28T04:07:44.494697+0000","last_peered":"2023-11-28T04:07:44.494697+0000","last_clean":"2023-11-28T04:07:44.494697+0000","last_became_active":"2023-11-28T04:07:44.491268+0000","last_became_peered":"2023-11-28T04:07:44.491268+0000","last_unstale":"2023-11-28T04:07:44.494697+0000","last_undegraded":"2023-11-28T04:07:44.494697+0000","last_fullsized":"2023-11-28T04:07:44.494697+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472554+0000","last_change":"2023-11-28T04:07:44.111634+0000","last_active":"2023-11-28T04:07:44.472554+0000","last_peered":"2023-11-28T04:07:44.472554+0000","last_clean":"2023-11-28T04:07:44.472554+0000","last_became_active":"2023-11-28T04:07:44.111156+0000","last_became_peered":"2023-11-28T04:07:44.111156+0000","last_unstale":"2023-11-28T04:07:44.472554+0000","last_undegraded":"2023-11-28T04:07:44.472554+0000","last_fullsized":"2023-11-28T04:07:44.472554+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.3a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490445+0000","last_change":"2023-11-28T04:07:44.489125+0000","last_active":"2023-11-28T04:07:44.490445+0000","last_peered":"2023-11-28T04:07:44.490445+0000","last_clean":"2023-11-28T04:07:44.490445+0000","last_became_active":"2023-11-28T04:07:44.488874+0000","last_became_peered":"2023-11-28T04:07:44.488874+0000","last_unstale":"2023-11-28T04:07:44.490445+0000","last_undegraded":"2023-11-28T04:07:44.490445+0000","last_fullsized":"2023-11-28T04:07:44.490445+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467036+0000","last_change":"2023-11-28T04:07:44.106241+0000","last_active":"2023-11-28T04:07:44.467036+0000","last_peered":"2023-11-28T04:07:44.467036+0000","last_clean":"2023-11-28T04:07:44.467036+0000","last_became_active":"2023-11-28T04:07:44.105742+0000","last_became_peered":"2023-11-28T04:07:44.105742+0000","last_unstale":"2023-11-28T04:07:44.467036+0000","last_undegraded":"2023-11-28T04:07:44.467036+0000","last_fullsized":"2023-11-28T04:07:44.467036+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.3b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479609+0000","last_change":"2023-11-28T04:07:44.477574+0000","last_active":"2023-11-28T04:07:44.479609+0000","last_peered":"2023-11-28T04:07:44.479609+0000","last_clean":"2023-11-28T04:07:44.479609+0000","last_became_active":"2023-11-28T04:07:44.477313+0000","last_became_peered":"2023-11-28T04:07:44.477313+0000","last_unstale":"2023-11-28T04:07:44.479609+0000","last_undegraded":"2023-11-28T04:07:44.479609+0000","last_fullsized":"2023-11-28T04:07:44.479609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485285+0000","last_change":"2023-11-28T04:07:44.108761+0000","last_active":"2023-11-28T04:07:44.485285+0000","last_peered":"2023-11-28T04:07:44.485285+0000","last_clean":"2023-11-28T04:07:44.485285+0000","last_became_active":"2023-11-28T04:07:44.108511+0000","last_became_peered":"2023-11-28T04:07:44.108511+0000","last_unstale":"2023-11-28T04:07:44.485285+0000","last_undegraded":"2023-11-28T04:07:44.485285+0000","last_fullsized":"2023-11-28T04:07:44.485285+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494783+0000","last_change":"2023-11-28T04:07:44.489084+0000","last_active":"2023-11-28T04:07:44.494783+0000","last_peered":"2023-11-28T04:07:44.494783+0000","last_clean":"2023-11-28T04:07:44.494783+0000","last_became_active":"2023-11-28T04:07:44.488814+0000","last_became_peered":"2023-11-28T04:07:44.488814+0000","last_unstale":"2023-11-28T04:07:44.494783+0000","last_undegraded":"2023-11-28T04:07:44.494783+0000","last_fullsized":"2023-11-28T04:07:44.494783+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480590+0000","last_change":"2023-11-28T04:07:44.112289+0000","last_active":"2023-11-28T04:07:44.480590+0000","last_peered":"2023-11-28T04:07:44.480590+0000","last_clean":"2023-11-28T04:07:44.480590+0000","last_became_active":"2023-11-28T04:07:44.111696+0000","last_became_peered":"2023-11-28T04:07:44.111696+0000","last_unstale":"2023-11-28T04:07:44.480590+0000","last_undegraded":"2023-11-28T04:07:44.480590+0000","last_fullsized":"2023-11-28T04:07:44.480590+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.3d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489146+0000","last_change":"2023-11-28T04:07:44.476209+0000","last_active":"2023-11-28T04:07:44.489146+0000","last_peered":"2023-11-28T04:07:44.489146+0000","last_clean":"2023-11-28T04:07:44.489146+0000","last_became_active":"2023-11-28T04:07:44.475591+0000","last_became_peered":"2023-11-28T04:07:44.475591+0000","last_unstale":"2023-11-28T04:07:44.489146+0000","last_undegraded":"2023-11-28T04:07:44.489146+0000","last_fullsized":"2023-11-28T04:07:44.489146+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3a","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478528+0000","last_change":"2023-11-28T04:07:44.109581+0000","last_active":"2023-11-28T04:07:44.478528+0000","last_peered":"2023-11-28T04:07:44.478528+0000","last_clean":"2023-11-28T04:07:44.478528+0000","last_became_active":"2023-11-28T04:07:44.109244+0000","last_became_peered":"2023-11-28T04:07:44.109244+0000","last_unstale":"2023-11-28T04:07:44.478528+0000","last_undegraded":"2023-11-28T04:07:44.478528+0000","last_fullsized":"2023-11-28T04:07:44.478528+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.484728+0000","last_change":"2023-11-28T04:07:44.480799+0000","last_active":"2023-11-28T04:07:44.484728+0000","last_peered":"2023-11-28T04:07:44.484728+0000","last_clean":"2023-11-28T04:07:44.484728+0000","last_became_active":"2023-11-28T04:07:44.480511+0000","last_became_peered":"2023-11-28T04:07:44.480511+0000","last_unstale":"2023-11-28T04:07:44.484728+0000","last_undegraded":"2023-11-28T04:07:44.484728+0000","last_fullsized":"2023-11-28T04:07:44.484728+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470212+0000","last_change":"2023-11-28T04:07:44.106504+0000","last_active":"2023-11-28T04:07:44.470212+0000","last_peered":"2023-11-28T04:07:44.470212+0000","last_clean":"2023-11-28T04:07:44.470212+0000","last_became_active":"2023-11-28T04:07:44.106202+0000","last_became_peered":"2023-11-28T04:07:44.106202+0000","last_unstale":"2023-11-28T04:07:44.470212+0000","last_undegraded":"2023-11-28T04:07:44.470212+0000","last_fullsized":"2023-11-28T04:07:44.470212+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486485+0000","last_change":"2023-11-28T04:07:44.482773+0000","last_active":"2023-11-28T04:07:44.486485+0000","last_peered":"2023-11-28T04:07:44.486485+0000","last_clean":"2023-11-28T04:07:44.486485+0000","last_became_active":"2023-11-28T04:07:44.482483+0000","last_became_peered":"2023-11-28T04:07:44.482483+0000","last_unstale":"2023-11-28T04:07:44.486485+0000","last_undegraded":"2023-11-28T04:07:44.486485+0000","last_fullsized":"2023-11-28T04:07:44.486485+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466893+0000","last_change":"2023-11-28T04:07:44.110221+0000","last_active":"2023-11-28T04:07:44.466893+0000","last_peered":"2023-11-28T04:07:44.466893+0000","last_clean":"2023-11-28T04:07:44.466893+0000","last_became_active":"2023-11-28T04:07:44.109739+0000","last_became_peered":"2023-11-28T04:07:44.109739+0000","last_unstale":"2023-11-28T04:07:44.466893+0000","last_undegraded":"2023-11-28T04:07:44.466893+0000","last_fullsized":"2023-11-28T04:07:44.466893+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.30","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487227+0000","last_change":"2023-11-28T04:07:44.481534+0000","last_active":"2023-11-28T04:07:44.487227+0000","last_peered":"2023-11-28T04:07:44.487227+0000","last_clean":"2023-11-28T04:07:44.487227+0000","last_became_active":"2023-11-28T04:07:44.481272+0000","last_became_peered":"2023-11-28T04:07:44.481272+0000","last_unstale":"2023-11-28T04:07:44.487227+0000","last_undegraded":"2023-11-28T04:07:44.487227+0000","last_fullsized":"2023-11-28T04:07:44.487227+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467360+0000","last_change":"2023-11-28T04:07:44.115088+0000","last_active":"2023-11-28T04:07:44.467360+0000","last_peered":"2023-11-28T04:07:44.467360+0000","last_clean":"2023-11-28T04:07:44.467360+0000","last_became_active":"2023-11-28T04:07:44.114010+0000","last_became_peered":"2023-11-28T04:07:44.114010+0000","last_unstale":"2023-11-28T04:07:44.467360+0000","last_undegraded":"2023-11-28T04:07:44.467360+0000","last_fullsized":"2023-11-28T04:07:44.467360+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.31","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489782+0000","last_change":"2023-11-28T04:07:44.487403+0000","last_active":"2023-11-28T04:07:44.489782+0000","last_peered":"2023-11-28T04:07:44.489782+0000","last_clean":"2023-11-28T04:07:44.489782+0000","last_became_active":"2023-11-28T04:07:44.487107+0000","last_became_peered":"2023-11-28T04:07:44.487107+0000","last_unstale":"2023-11-28T04:07:44.489782+0000","last_undegraded":"2023-11-28T04:07:44.489782+0000","last_fullsized":"2023-11-28T04:07:44.489782+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472672+0000","last_change":"2023-11-28T04:07:44.111442+0000","last_active":"2023-11-28T04:07:44.472672+0000","last_peered":"2023-11-28T04:07:44.472672+0000","last_clean":"2023-11-28T04:07:44.472672+0000","last_became_active":"2023-11-28T04:07:44.110824+0000","last_became_peered":"2023-11-28T04:07:44.110824+0000","last_unstale":"2023-11-28T04:07:44.472672+0000","last_undegraded":"2023-11-28T04:07:44.472672+0000","last_fullsized":"2023-11-28T04:07:44.472672+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.32","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486478+0000","last_change":"2023-11-28T04:07:44.485088+0000","last_active":"2023-11-28T04:07:44.486478+0000","last_peered":"2023-11-28T04:07:44.486478+0000","last_clean":"2023-11-28T04:07:44.486478+0000","last_became_active":"2023-11-28T04:07:44.484826+0000","last_became_peered":"2023-11-28T04:07:44.484826+0000","last_unstale":"2023-11-28T04:07:44.486478+0000","last_undegraded":"2023-11-28T04:07:44.486478+0000","last_fullsized":"2023-11-28T04:07:44.486478+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.462604+0000","last_change":"2023-11-28T04:07:44.110072+0000","last_active":"2023-11-28T04:07:44.462604+0000","last_peered":"2023-11-28T04:07:44.462604+0000","last_clean":"2023-11-28T04:07:44.462604+0000","last_became_active":"2023-11-28T04:07:44.109819+0000","last_became_peered":"2023-11-28T04:07:44.109819+0000","last_unstale":"2023-11-28T04:07:44.462604+0000","last_undegraded":"2023-11-28T04:07:44.462604+0000","last_fullsized":"2023-11-28T04:07:44.462604+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.33","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490468+0000","last_change":"2023-11-28T04:07:44.489777+0000","last_active":"2023-11-28T04:07:44.490468+0000","last_peered":"2023-11-28T04:07:44.490468+0000","last_clean":"2023-11-28T04:07:44.490468+0000","last_became_active":"2023-11-28T04:07:44.489547+0000","last_became_peered":"2023-11-28T04:07:44.489547+0000","last_unstale":"2023-11-28T04:07:44.490468+0000","last_undegraded":"2023-11-28T04:07:44.490468+0000","last_fullsized":"2023-11-28T04:07:44.490468+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.34","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.495253+0000","last_change":"2023-11-28T04:07:44.113703+0000","last_active":"2023-11-28T04:07:44.495253+0000","last_peered":"2023-11-28T04:07:44.495253+0000","last_clean":"2023-11-28T04:07:44.495253+0000","last_became_active":"2023-11-28T04:07:44.113417+0000","last_became_peered":"2023-11-28T04:07:44.113417+0000","last_unstale":"2023-11-28T04:07:44.495253+0000","last_undegraded":"2023-11-28T04:07:44.495253+0000","last_fullsized":"2023-11-28T04:07:44.495253+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.34","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485788+0000","last_change":"2023-11-28T04:07:44.481988+0000","last_active":"2023-11-28T04:07:44.485788+0000","last_peered":"2023-11-28T04:07:44.485788+0000","last_clean":"2023-11-28T04:07:44.485788+0000","last_became_active":"2023-11-28T04:07:44.475932+0000","last_became_peered":"2023-11-28T04:07:44.475932+0000","last_unstale":"2023-11-28T04:07:44.485788+0000","last_undegraded":"2023-11-28T04:07:44.485788+0000","last_fullsized":"2023-11-28T04:07:44.485788+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.33","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475965+0000","last_change":"2023-11-28T04:07:44.104938+0000","last_active":"2023-11-28T04:07:44.475965+0000","last_peered":"2023-11-28T04:07:44.475965+0000","last_clean":"2023-11-28T04:07:44.475965+0000","last_became_active":"2023-11-28T04:07:44.102437+0000","last_became_peered":"2023-11-28T04:07:44.102437+0000","last_unstale":"2023-11-28T04:07:44.475965+0000","last_undegraded":"2023-11-28T04:07:44.475965+0000","last_fullsized":"2023-11-28T04:07:44.475965+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.35","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494657+0000","last_change":"2023-11-28T04:07:44.488430+0000","last_active":"2023-11-28T04:07:44.494657+0000","last_peered":"2023-11-28T04:07:44.494657+0000","last_clean":"2023-11-28T04:07:44.494657+0000","last_became_active":"2023-11-28T04:07:44.488148+0000","last_became_peered":"2023-11-28T04:07:44.488148+0000","last_unstale":"2023-11-28T04:07:44.494657+0000","last_undegraded":"2023-11-28T04:07:44.494657+0000","last_fullsized":"2023-11-28T04:07:44.494657+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.474816+0000","last_change":"2023-11-28T04:07:44.110316+0000","last_active":"2023-11-28T04:07:44.474816+0000","last_peered":"2023-11-28T04:07:44.474816+0000","last_clean":"2023-11-28T04:07:44.474816+0000","last_became_active":"2023-11-28T04:07:44.110028+0000","last_became_peered":"2023-11-28T04:07:44.110028+0000","last_unstale":"2023-11-28T04:07:44.474816+0000","last_undegraded":"2023-11-28T04:07:44.474816+0000","last_fullsized":"2023-11-28T04:07:44.474816+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.36","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480024+0000","last_change":"2023-11-28T04:07:44.475360+0000","last_active":"2023-11-28T04:07:44.480024+0000","last_peered":"2023-11-28T04:07:44.480024+0000","last_clean":"2023-11-28T04:07:44.480024+0000","last_became_active":"2023-11-28T04:07:44.475098+0000","last_became_peered":"2023-11-28T04:07:44.475098+0000","last_unstale":"2023-11-28T04:07:44.480024+0000","last_undegraded":"2023-11-28T04:07:44.480024+0000","last_fullsized":"2023-11-28T04:07:44.480024+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468504+0000","last_change":"2023-11-28T04:07:44.111087+0000","last_active":"2023-11-28T04:07:44.468504+0000","last_peered":"2023-11-28T04:07:44.468504+0000","last_clean":"2023-11-28T04:07:44.468504+0000","last_became_active":"2023-11-28T04:07:44.110400+0000","last_became_peered":"2023-11-28T04:07:44.110400+0000","last_unstale":"2023-11-28T04:07:44.468504+0000","last_undegraded":"2023-11-28T04:07:44.468504+0000","last_fullsized":"2023-11-28T04:07:44.468504+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.37","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487177+0000","last_change":"2023-11-28T04:07:44.481492+0000","last_active":"2023-11-28T04:07:44.487177+0000","last_peered":"2023-11-28T04:07:44.487177+0000","last_clean":"2023-11-28T04:07:44.487177+0000","last_became_active":"2023-11-28T04:07:44.481212+0000","last_became_peered":"2023-11-28T04:07:44.481212+0000","last_unstale":"2023-11-28T04:07:44.487177+0000","last_undegraded":"2023-11-28T04:07:44.487177+0000","last_fullsized":"2023-11-28T04:07:44.487177+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.30","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483951+0000","last_change":"2023-11-28T04:07:44.105325+0000","last_active":"2023-11-28T04:07:44.483951+0000","last_peered":"2023-11-28T04:07:44.483951+0000","last_clean":"2023-11-28T04:07:44.483951+0000","last_became_active":"2023-11-28T04:07:44.105081+0000","last_became_peered":"2023-11-28T04:07:44.105081+0000","last_unstale":"2023-11-28T04:07:44.483951+0000","last_undegraded":"2023-11-28T04:07:44.483951+0000","last_fullsized":"2023-11-28T04:07:44.483951+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.28","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487271+0000","last_change":"2023-11-28T04:07:44.484640+0000","last_active":"2023-11-28T04:07:44.487271+0000","last_peered":"2023-11-28T04:07:44.487271+0000","last_clean":"2023-11-28T04:07:44.487271+0000","last_became_active":"2023-11-28T04:07:44.484227+0000","last_became_peered":"2023-11-28T04:07:44.484227+0000","last_unstale":"2023-11-28T04:07:44.487271+0000","last_undegraded":"2023-11-28T04:07:44.487271+0000","last_fullsized":"2023-11-28T04:07:44.487271+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469618+0000","last_change":"2023-11-28T04:07:44.104353+0000","last_active":"2023-11-28T04:07:44.469618+0000","last_peered":"2023-11-28T04:07:44.469618+0000","last_clean":"2023-11-28T04:07:44.469618+0000","last_became_active":"2023-11-28T04:07:44.104058+0000","last_became_peered":"2023-11-28T04:07:44.104058+0000","last_unstale":"2023-11-28T04:07:44.469618+0000","last_undegraded":"2023-11-28T04:07:44.469618+0000","last_fullsized":"2023-11-28T04:07:44.469618+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.29","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492111+0000","last_change":"2023-11-28T04:07:44.487100+0000","last_active":"2023-11-28T04:07:44.492111+0000","last_peered":"2023-11-28T04:07:44.492111+0000","last_clean":"2023-11-28T04:07:44.492111+0000","last_became_active":"2023-11-28T04:07:44.486847+0000","last_became_peered":"2023-11-28T04:07:44.486847+0000","last_unstale":"2023-11-28T04:07:44.492111+0000","last_undegraded":"2023-11-28T04:07:44.492111+0000","last_fullsized":"2023-11-28T04:07:44.492111+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469178+0000","last_change":"2023-11-28T04:07:44.113850+0000","last_active":"2023-11-28T04:07:44.469178+0000","last_peered":"2023-11-28T04:07:44.469178+0000","last_clean":"2023-11-28T04:07:44.469178+0000","last_became_active":"2023-11-28T04:07:44.113616+0000","last_became_peered":"2023-11-28T04:07:44.113616+0000","last_unstale":"2023-11-28T04:07:44.469178+0000","last_undegraded":"2023-11-28T04:07:44.469178+0000","last_fullsized":"2023-11-28T04:07:44.469178+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486410+0000","last_change":"2023-11-28T04:07:44.483363+0000","last_active":"2023-11-28T04:07:44.486410+0000","last_peered":"2023-11-28T04:07:44.486410+0000","last_clean":"2023-11-28T04:07:44.486410+0000","last_became_active":"2023-11-28T04:07:44.483101+0000","last_became_peered":"2023-11-28T04:07:44.483101+0000","last_unstale":"2023-11-28T04:07:44.486410+0000","last_undegraded":"2023-11-28T04:07:44.486410+0000","last_fullsized":"2023-11-28T04:07:44.486410+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.2d","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475196+0000","last_change":"2023-11-28T04:07:44.104302+0000","last_active":"2023-11-28T04:07:44.475196+0000","last_peered":"2023-11-28T04:07:44.475196+0000","last_clean":"2023-11-28T04:07:44.475196+0000","last_became_active":"2023-11-28T04:07:44.103985+0000","last_became_peered":"2023-11-28T04:07:44.103985+0000","last_unstale":"2023-11-28T04:07:44.475196+0000","last_undegraded":"2023-11-28T04:07:44.475196+0000","last_fullsized":"2023-11-28T04:07:44.475196+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.482460+0000","last_change":"2023-11-28T04:07:44.477456+0000","last_active":"2023-11-28T04:07:44.482460+0000","last_peered":"2023-11-28T04:07:44.482460+0000","last_clean":"2023-11-28T04:07:44.482460+0000","last_became_active":"2023-11-28T04:07:44.477166+0000","last_became_peered":"2023-11-28T04:07:44.477166+0000","last_unstale":"2023-11-28T04:07:44.482460+0000","last_undegraded":"2023-11-28T04:07:44.482460+0000","last_fullsized":"2023-11-28T04:07:44.482460+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465599+0000","last_change":"2023-11-28T04:07:44.112401+0000","last_active":"2023-11-28T04:07:44.465599+0000","last_peered":"2023-11-28T04:07:44.465599+0000","last_clean":"2023-11-28T04:07:44.465599+0000","last_became_active":"2023-11-28T04:07:44.112163+0000","last_became_peered":"2023-11-28T04:07:44.112163+0000","last_unstale":"2023-11-28T04:07:44.465599+0000","last_undegraded":"2023-11-28T04:07:44.465599+0000","last_fullsized":"2023-11-28T04:07:44.465599+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.2c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479617+0000","last_change":"2023-11-28T04:07:44.478586+0000","last_active":"2023-11-28T04:07:44.479617+0000","last_peered":"2023-11-28T04:07:44.479617+0000","last_clean":"2023-11-28T04:07:44.479617+0000","last_became_active":"2023-11-28T04:07:44.478269+0000","last_became_peered":"2023-11-28T04:07:44.478269+0000","last_unstale":"2023-11-28T04:07:44.479617+0000","last_undegraded":"2023-11-28T04:07:44.479617+0000","last_fullsized":"2023-11-28T04:07:44.479617+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480999+0000","last_change":"2023-11-28T04:07:44.112458+0000","last_active":"2023-11-28T04:07:44.480999+0000","last_peered":"2023-11-28T04:07:44.480999+0000","last_clean":"2023-11-28T04:07:44.480999+0000","last_became_active":"2023-11-28T04:07:44.112028+0000","last_became_peered":"2023-11-28T04:07:44.112028+0000","last_unstale":"2023-11-28T04:07:44.480999+0000","last_undegraded":"2023-11-28T04:07:44.480999+0000","last_fullsized":"2023-11-28T04:07:44.480999+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.2d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486432+0000","last_change":"2023-11-28T04:07:44.481030+0000","last_active":"2023-11-28T04:07:44.486432+0000","last_peered":"2023-11-28T04:07:44.486432+0000","last_clean":"2023-11-28T04:07:44.486432+0000","last_became_active":"2023-11-28T04:07:44.480689+0000","last_became_peered":"2023-11-28T04:07:44.480689+0000","last_unstale":"2023-11-28T04:07:44.486432+0000","last_undegraded":"2023-11-28T04:07:44.486432+0000","last_fullsized":"2023-11-28T04:07:44.486432+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.462163+0000","last_change":"2023-11-28T04:07:44.107204+0000","last_active":"2023-11-28T04:07:44.462163+0000","last_peered":"2023-11-28T04:07:44.462163+0000","last_clean":"2023-11-28T04:07:44.462163+0000","last_became_active":"2023-11-28T04:07:44.106921+0000","last_became_peered":"2023-11-28T04:07:44.106921+0000","last_unstale":"2023-11-28T04:07:44.462163+0000","last_undegraded":"2023-11-28T04:07:44.462163+0000","last_fullsized":"2023-11-28T04:07:44.462163+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.2e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483491+0000","last_change":"2023-11-28T04:07:44.480913+0000","last_active":"2023-11-28T04:07:44.483491+0000","last_peered":"2023-11-28T04:07:44.483491+0000","last_clean":"2023-11-28T04:07:44.483491+0000","last_became_active":"2023-11-28T04:07:44.480659+0000","last_became_peered":"2023-11-28T04:07:44.480659+0000","last_unstale":"2023-11-28T04:07:44.483491+0000","last_undegraded":"2023-11-28T04:07:44.483491+0000","last_fullsized":"2023-11-28T04:07:44.483491+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466068+0000","last_change":"2023-11-28T04:07:44.114355+0000","last_active":"2023-11-28T04:07:44.466068+0000","last_peered":"2023-11-28T04:07:44.466068+0000","last_clean":"2023-11-28T04:07:44.466068+0000","last_became_active":"2023-11-28T04:07:44.113617+0000","last_became_peered":"2023-11-28T04:07:44.113617+0000","last_unstale":"2023-11-28T04:07:44.466068+0000","last_undegraded":"2023-11-28T04:07:44.466068+0000","last_fullsized":"2023-11-28T04:07:44.466068+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.2f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487145+0000","last_change":"2023-11-28T04:07:44.481143+0000","last_active":"2023-11-28T04:07:44.487145+0000","last_peered":"2023-11-28T04:07:44.487145+0000","last_clean":"2023-11-28T04:07:44.487145+0000","last_became_active":"2023-11-28T04:07:44.480821+0000","last_became_peered":"2023-11-28T04:07:44.480821+0000","last_unstale":"2023-11-28T04:07:44.487145+0000","last_undegraded":"2023-11-28T04:07:44.487145+0000","last_fullsized":"2023-11-28T04:07:44.487145+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.462132+0000","last_change":"2023-11-28T04:07:44.116709+0000","last_active":"2023-11-28T04:07:44.462132+0000","last_peered":"2023-11-28T04:07:44.462132+0000","last_clean":"2023-11-28T04:07:44.462132+0000","last_became_active":"2023-11-28T04:07:44.116378+0000","last_became_peered":"2023-11-28T04:07:44.116378+0000","last_unstale":"2023-11-28T04:07:44.462132+0000","last_undegraded":"2023-11-28T04:07:44.462132+0000","last_fullsized":"2023-11-28T04:07:44.462132+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.20","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489915+0000","last_change":"2023-11-28T04:07:44.488886+0000","last_active":"2023-11-28T04:07:44.489915+0000","last_peered":"2023-11-28T04:07:44.489915+0000","last_clean":"2023-11-28T04:07:44.489915+0000","last_became_active":"2023-11-28T04:07:44.488585+0000","last_became_peered":"2023-11-28T04:07:44.488585+0000","last_unstale":"2023-11-28T04:07:44.489915+0000","last_undegraded":"2023-11-28T04:07:44.489915+0000","last_fullsized":"2023-11-28T04:07:44.489915+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.474476+0000","last_change":"2023-11-28T04:07:44.109538+0000","last_active":"2023-11-28T04:07:44.474476+0000","last_peered":"2023-11-28T04:07:44.474476+0000","last_clean":"2023-11-28T04:07:44.474476+0000","last_became_active":"2023-11-28T04:07:44.109216+0000","last_became_peered":"2023-11-28T04:07:44.109216+0000","last_unstale":"2023-11-28T04:07:44.474476+0000","last_undegraded":"2023-11-28T04:07:44.474476+0000","last_fullsized":"2023-11-28T04:07:44.474476+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.21","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485937+0000","last_change":"2023-11-28T04:07:44.481886+0000","last_active":"2023-11-28T04:07:44.485937+0000","last_peered":"2023-11-28T04:07:44.485937+0000","last_clean":"2023-11-28T04:07:44.485937+0000","last_became_active":"2023-11-28T04:07:44.481610+0000","last_became_peered":"2023-11-28T04:07:44.481610+0000","last_unstale":"2023-11-28T04:07:44.485937+0000","last_undegraded":"2023-11-28T04:07:44.485937+0000","last_fullsized":"2023-11-28T04:07:44.485937+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.26","version":"165'1","reported_seq":11,"reported_epoch":165,"state":"active+clean","last_fresh":"2023-11-28T04:07:45.471198+0000","last_change":"2023-11-28T04:07:44.110589+0000","last_active":"2023-11-28T04:07:45.471198+0000","last_peered":"2023-11-28T04:07:45.471198+0000","last_clean":"2023-11-28T04:07:45.471198+0000","last_became_active":"2023-11-28T04:07:44.110322+0000","last_became_peered":"2023-11-28T04:07:44.110322+0000","last_unstale":"2023-11-28T04:07:45.471198+0000","last_undegraded":"2023-11-28T04:07:45.471198+0000","last_fullsized":"2023-11-28T04:07:45.471198+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.22","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490904+0000","last_change":"2023-11-28T04:07:44.486310+0000","last_active":"2023-11-28T04:07:44.490904+0000","last_peered":"2023-11-28T04:07:44.490904+0000","last_clean":"2023-11-28T04:07:44.490904+0000","last_became_active":"2023-11-28T04:07:44.484189+0000","last_became_peered":"2023-11-28T04:07:44.484189+0000","last_unstale":"2023-11-28T04:07:44.490904+0000","last_undegraded":"2023-11-28T04:07:44.490904+0000","last_fullsized":"2023-11-28T04:07:44.490904+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469868+0000","last_change":"2023-11-28T04:07:44.104470+0000","last_active":"2023-11-28T04:07:44.469868+0000","last_peered":"2023-11-28T04:07:44.469868+0000","last_clean":"2023-11-28T04:07:44.469868+0000","last_became_active":"2023-11-28T04:07:44.104217+0000","last_became_peered":"2023-11-28T04:07:44.104217+0000","last_unstale":"2023-11-28T04:07:44.469868+0000","last_undegraded":"2023-11-28T04:07:44.469868+0000","last_fullsized":"2023-11-28T04:07:44.469868+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.23","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490105+0000","last_change":"2023-11-28T04:07:44.481275+0000","last_active":"2023-11-28T04:07:44.490105+0000","last_peered":"2023-11-28T04:07:44.490105+0000","last_clean":"2023-11-28T04:07:44.490105+0000","last_became_active":"2023-11-28T04:07:44.481006+0000","last_became_peered":"2023-11-28T04:07:44.481006+0000","last_unstale":"2023-11-28T04:07:44.490105+0000","last_undegraded":"2023-11-28T04:07:44.490105+0000","last_fullsized":"2023-11-28T04:07:44.490105+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470540+0000","last_change":"2023-11-28T04:07:44.111555+0000","last_active":"2023-11-28T04:07:44.470540+0000","last_peered":"2023-11-28T04:07:44.470540+0000","last_clean":"2023-11-28T04:07:44.470540+0000","last_became_active":"2023-11-28T04:07:44.111268+0000","last_became_peered":"2023-11-28T04:07:44.111268+0000","last_unstale":"2023-11-28T04:07:44.470540+0000","last_undegraded":"2023-11-28T04:07:44.470540+0000","last_fullsized":"2023-11-28T04:07:44.470540+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.24","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489415+0000","last_change":"2023-11-28T04:07:44.477017+0000","last_active":"2023-11-28T04:07:44.489415+0000","last_peered":"2023-11-28T04:07:44.489415+0000","last_clean":"2023-11-28T04:07:44.489415+0000","last_became_active":"2023-11-28T04:07:44.476741+0000","last_became_peered":"2023-11-28T04:07:44.476741+0000","last_unstale":"2023-11-28T04:07:44.489415+0000","last_undegraded":"2023-11-28T04:07:44.489415+0000","last_fullsized":"2023-11-28T04:07:44.489415+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468340+0000","last_change":"2023-11-28T04:07:44.110023+0000","last_active":"2023-11-28T04:07:44.468340+0000","last_peered":"2023-11-28T04:07:44.468340+0000","last_clean":"2023-11-28T04:07:44.468340+0000","last_became_active":"2023-11-28T04:07:44.109750+0000","last_became_peered":"2023-11-28T04:07:44.109750+0000","last_unstale":"2023-11-28T04:07:44.468340+0000","last_undegraded":"2023-11-28T04:07:44.468340+0000","last_fullsized":"2023-11-28T04:07:44.468340+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.25","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494578+0000","last_change":"2023-11-28T04:07:44.488305+0000","last_active":"2023-11-28T04:07:44.494578+0000","last_peered":"2023-11-28T04:07:44.494578+0000","last_clean":"2023-11-28T04:07:44.494578+0000","last_became_active":"2023-11-28T04:07:44.487406+0000","last_became_peered":"2023-11-28T04:07:44.487406+0000","last_unstale":"2023-11-28T04:07:44.494578+0000","last_undegraded":"2023-11-28T04:07:44.494578+0000","last_fullsized":"2023-11-28T04:07:44.494578+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468055+0000","last_change":"2023-11-28T04:07:44.110617+0000","last_active":"2023-11-28T04:07:44.468055+0000","last_peered":"2023-11-28T04:07:44.468055+0000","last_clean":"2023-11-28T04:07:44.468055+0000","last_became_active":"2023-11-28T04:07:44.110327+0000","last_became_peered":"2023-11-28T04:07:44.110327+0000","last_unstale":"2023-11-28T04:07:44.468055+0000","last_undegraded":"2023-11-28T04:07:44.468055+0000","last_fullsized":"2023-11-28T04:07:44.468055+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.26","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490379+0000","last_change":"2023-11-28T04:07:44.483626+0000","last_active":"2023-11-28T04:07:44.490379+0000","last_peered":"2023-11-28T04:07:44.490379+0000","last_clean":"2023-11-28T04:07:44.490379+0000","last_became_active":"2023-11-28T04:07:44.483359+0000","last_became_peered":"2023-11-28T04:07:44.483359+0000","last_unstale":"2023-11-28T04:07:44.490379+0000","last_undegraded":"2023-11-28T04:07:44.490379+0000","last_fullsized":"2023-11-28T04:07:44.490379+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478832+0000","last_change":"2023-11-28T04:07:44.118163+0000","last_active":"2023-11-28T04:07:44.478832+0000","last_peered":"2023-11-28T04:07:44.478832+0000","last_clean":"2023-11-28T04:07:44.478832+0000","last_became_active":"2023-11-28T04:07:44.117786+0000","last_became_peered":"2023-11-28T04:07:44.117786+0000","last_unstale":"2023-11-28T04:07:44.478832+0000","last_undegraded":"2023-11-28T04:07:44.478832+0000","last_fullsized":"2023-11-28T04:07:44.478832+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.27","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489230+0000","last_change":"2023-11-28T04:07:44.476435+0000","last_active":"2023-11-28T04:07:44.489230+0000","last_peered":"2023-11-28T04:07:44.489230+0000","last_clean":"2023-11-28T04:07:44.489230+0000","last_became_active":"2023-11-28T04:07:44.476148+0000","last_became_peered":"2023-11-28T04:07:44.476148+0000","last_unstale":"2023-11-28T04:07:44.489230+0000","last_undegraded":"2023-11-28T04:07:44.489230+0000","last_fullsized":"2023-11-28T04:07:44.489230+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468541+0000","last_change":"2023-11-28T04:07:44.102476+0000","last_active":"2023-11-28T04:07:44.468541+0000","last_peered":"2023-11-28T04:07:44.468541+0000","last_clean":"2023-11-28T04:07:44.468541+0000","last_became_active":"2023-11-28T04:07:44.102178+0000","last_became_peered":"2023-11-28T04:07:44.102178+0000","last_unstale":"2023-11-28T04:07:44.468541+0000","last_undegraded":"2023-11-28T04:07:44.468541+0000","last_fullsized":"2023-11-28T04:07:44.468541+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487891+0000","last_change":"2023-11-28T04:07:44.483294+0000","last_active":"2023-11-28T04:07:44.487891+0000","last_peered":"2023-11-28T04:07:44.487891+0000","last_clean":"2023-11-28T04:07:44.487891+0000","last_became_active":"2023-11-28T04:07:44.483044+0000","last_became_peered":"2023-11-28T04:07:44.483044+0000","last_unstale":"2023-11-28T04:07:44.487891+0000","last_undegraded":"2023-11-28T04:07:44.487891+0000","last_fullsized":"2023-11-28T04:07:44.487891+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489393+0000","last_change":"2023-11-28T04:07:44.110602+0000","last_active":"2023-11-28T04:07:44.489393+0000","last_peered":"2023-11-28T04:07:44.489393+0000","last_clean":"2023-11-28T04:07:44.489393+0000","last_became_active":"2023-11-28T04:07:44.110301+0000","last_became_peered":"2023-11-28T04:07:44.110301+0000","last_unstale":"2023-11-28T04:07:44.489393+0000","last_undegraded":"2023-11-28T04:07:44.489393+0000","last_fullsized":"2023-11-28T04:07:44.489393+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485933+0000","last_change":"2023-11-28T04:07:44.482085+0000","last_active":"2023-11-28T04:07:44.485933+0000","last_peered":"2023-11-28T04:07:44.485933+0000","last_clean":"2023-11-28T04:07:44.485933+0000","last_became_active":"2023-11-28T04:07:44.481145+0000","last_became_peered":"2023-11-28T04:07:44.481145+0000","last_unstale":"2023-11-28T04:07:44.485933+0000","last_undegraded":"2023-11-28T04:07:44.485933+0000","last_fullsized":"2023-11-28T04:07:44.485933+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1e","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479304+0000","last_change":"2023-11-28T04:07:44.113764+0000","last_active":"2023-11-28T04:07:44.479304+0000","last_peered":"2023-11-28T04:07:44.479304+0000","last_clean":"2023-11-28T04:07:44.479304+0000","last_became_active":"2023-11-28T04:07:44.113322+0000","last_became_peered":"2023-11-28T04:07:44.113322+0000","last_unstale":"2023-11-28T04:07:44.479304+0000","last_undegraded":"2023-11-28T04:07:44.479304+0000","last_fullsized":"2023-11-28T04:07:44.479304+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.492088+0000","last_change":"2023-11-28T04:07:44.485956+0000","last_active":"2023-11-28T04:07:44.492088+0000","last_peered":"2023-11-28T04:07:44.492088+0000","last_clean":"2023-11-28T04:07:44.492088+0000","last_became_active":"2023-11-28T04:07:44.485694+0000","last_became_peered":"2023-11-28T04:07:44.485694+0000","last_unstale":"2023-11-28T04:07:44.492088+0000","last_undegraded":"2023-11-28T04:07:44.492088+0000","last_fullsized":"2023-11-28T04:07:44.492088+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487680+0000","last_change":"2023-11-28T04:07:44.110797+0000","last_active":"2023-11-28T04:07:44.487680+0000","last_peered":"2023-11-28T04:07:44.487680+0000","last_clean":"2023-11-28T04:07:44.487680+0000","last_became_active":"2023-11-28T04:07:44.110494+0000","last_became_peered":"2023-11-28T04:07:44.110494+0000","last_unstale":"2023-11-28T04:07:44.487680+0000","last_undegraded":"2023-11-28T04:07:44.487680+0000","last_fullsized":"2023-11-28T04:07:44.487680+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486444+0000","last_change":"2023-11-28T04:07:44.483810+0000","last_active":"2023-11-28T04:07:44.486444+0000","last_peered":"2023-11-28T04:07:44.486444+0000","last_clean":"2023-11-28T04:07:44.486444+0000","last_became_active":"2023-11-28T04:07:44.483553+0000","last_became_peered":"2023-11-28T04:07:44.483553+0000","last_unstale":"2023-11-28T04:07:44.486444+0000","last_undegraded":"2023-11-28T04:07:44.486444+0000","last_fullsized":"2023-11-28T04:07:44.486444+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.476217+0000","last_change":"2023-11-28T04:07:44.115588+0000","last_active":"2023-11-28T04:07:44.476217+0000","last_peered":"2023-11-28T04:07:44.476217+0000","last_clean":"2023-11-28T04:07:44.476217+0000","last_became_active":"2023-11-28T04:07:44.115367+0000","last_became_peered":"2023-11-28T04:07:44.115367+0000","last_unstale":"2023-11-28T04:07:44.476217+0000","last_undegraded":"2023-11-28T04:07:44.476217+0000","last_fullsized":"2023-11-28T04:07:44.476217+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485918+0000","last_change":"2023-11-28T04:07:44.481858+0000","last_active":"2023-11-28T04:07:44.485918+0000","last_peered":"2023-11-28T04:07:44.485918+0000","last_clean":"2023-11-28T04:07:44.485918+0000","last_became_active":"2023-11-28T04:07:44.481571+0000","last_became_peered":"2023-11-28T04:07:44.481571+0000","last_unstale":"2023-11-28T04:07:44.485918+0000","last_undegraded":"2023-11-28T04:07:44.485918+0000","last_fullsized":"2023-11-28T04:07:44.485918+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.471609+0000","last_change":"2023-11-28T04:07:44.105489+0000","last_active":"2023-11-28T04:07:44.471609+0000","last_peered":"2023-11-28T04:07:44.471609+0000","last_clean":"2023-11-28T04:07:44.471609+0000","last_became_active":"2023-11-28T04:07:44.105080+0000","last_became_peered":"2023-11-28T04:07:44.105080+0000","last_unstale":"2023-11-28T04:07:44.471609+0000","last_undegraded":"2023-11-28T04:07:44.471609+0000","last_fullsized":"2023-11-28T04:07:44.471609+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.480691+0000","last_change":"2023-11-28T04:07:44.477905+0000","last_active":"2023-11-28T04:07:44.480691+0000","last_peered":"2023-11-28T04:07:44.480691+0000","last_clean":"2023-11-28T04:07:44.480691+0000","last_became_active":"2023-11-28T04:07:44.477545+0000","last_became_peered":"2023-11-28T04:07:44.477545+0000","last_unstale":"2023-11-28T04:07:44.480691+0000","last_undegraded":"2023-11-28T04:07:44.480691+0000","last_fullsized":"2023-11-28T04:07:44.480691+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.464896+0000","last_change":"2023-11-28T04:07:44.113284+0000","last_active":"2023-11-28T04:07:44.464896+0000","last_peered":"2023-11-28T04:07:44.464896+0000","last_clean":"2023-11-28T04:07:44.464896+0000","last_became_active":"2023-11-28T04:07:44.113040+0000","last_became_peered":"2023-11-28T04:07:44.113040+0000","last_unstale":"2023-11-28T04:07:44.464896+0000","last_undegraded":"2023-11-28T04:07:44.464896+0000","last_fullsized":"2023-11-28T04:07:44.464896+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486435+0000","last_change":"2023-11-28T04:07:44.485576+0000","last_active":"2023-11-28T04:07:44.486435+0000","last_peered":"2023-11-28T04:07:44.486435+0000","last_clean":"2023-11-28T04:07:44.486435+0000","last_became_active":"2023-11-28T04:07:44.485296+0000","last_became_peered":"2023-11-28T04:07:44.485296+0000","last_unstale":"2023-11-28T04:07:44.486435+0000","last_undegraded":"2023-11-28T04:07:44.486435+0000","last_fullsized":"2023-11-28T04:07:44.486435+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475316+0000","last_change":"2023-11-28T04:07:44.109213+0000","last_active":"2023-11-28T04:07:44.475316+0000","last_peered":"2023-11-28T04:07:44.475316+0000","last_clean":"2023-11-28T04:07:44.475316+0000","last_became_active":"2023-11-28T04:07:44.108530+0000","last_became_peered":"2023-11-28T04:07:44.108530+0000","last_unstale":"2023-11-28T04:07:44.475316+0000","last_undegraded":"2023-11-28T04:07:44.475316+0000","last_fullsized":"2023-11-28T04:07:44.475316+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485626+0000","last_change":"2023-11-28T04:07:44.479590+0000","last_active":"2023-11-28T04:07:44.485626+0000","last_peered":"2023-11-28T04:07:44.485626+0000","last_clean":"2023-11-28T04:07:44.485626+0000","last_became_active":"2023-11-28T04:07:44.479336+0000","last_became_peered":"2023-11-28T04:07:44.479336+0000","last_unstale":"2023-11-28T04:07:44.485626+0000","last_undegraded":"2023-11-28T04:07:44.485626+0000","last_fullsized":"2023-11-28T04:07:44.485626+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466934+0000","last_change":"2023-11-28T04:07:44.108560+0000","last_active":"2023-11-28T04:07:44.466934+0000","last_peered":"2023-11-28T04:07:44.466934+0000","last_clean":"2023-11-28T04:07:44.466934+0000","last_became_active":"2023-11-28T04:07:44.108259+0000","last_became_peered":"2023-11-28T04:07:44.108259+0000","last_unstale":"2023-11-28T04:07:44.466934+0000","last_undegraded":"2023-11-28T04:07:44.466934+0000","last_fullsized":"2023-11-28T04:07:44.466934+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.488964+0000","last_change":"2023-11-28T04:07:44.474715+0000","last_active":"2023-11-28T04:07:44.488964+0000","last_peered":"2023-11-28T04:07:44.488964+0000","last_clean":"2023-11-28T04:07:44.488964+0000","last_became_active":"2023-11-28T04:07:44.474434+0000","last_became_peered":"2023-11-28T04:07:44.474434+0000","last_unstale":"2023-11-28T04:07:44.488964+0000","last_undegraded":"2023-11-28T04:07:44.488964+0000","last_fullsized":"2023-11-28T04:07:44.488964+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.f","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489782+0000","last_change":"2023-11-28T04:07:44.113469+0000","last_active":"2023-11-28T04:07:44.489782+0000","last_peered":"2023-11-28T04:07:44.489782+0000","last_clean":"2023-11-28T04:07:44.489782+0000","last_became_active":"2023-11-28T04:07:44.113235+0000","last_became_peered":"2023-11-28T04:07:44.113235+0000","last_unstale":"2023-11-28T04:07:44.489782+0000","last_undegraded":"2023-11-28T04:07:44.489782+0000","last_fullsized":"2023-11-28T04:07:44.489782+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491951+0000","last_change":"2023-11-28T04:07:44.486375+0000","last_active":"2023-11-28T04:07:44.491951+0000","last_peered":"2023-11-28T04:07:44.491951+0000","last_clean":"2023-11-28T04:07:44.491951+0000","last_became_active":"2023-11-28T04:07:44.485929+0000","last_became_peered":"2023-11-28T04:07:44.485929+0000","last_unstale":"2023-11-28T04:07:44.491951+0000","last_undegraded":"2023-11-28T04:07:44.491951+0000","last_fullsized":"2023-11-28T04:07:44.491951+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.e","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490200+0000","last_change":"2023-11-28T04:07:44.110425+0000","last_active":"2023-11-28T04:07:44.490200+0000","last_peered":"2023-11-28T04:07:44.490200+0000","last_clean":"2023-11-28T04:07:44.490200+0000","last_became_active":"2023-11-28T04:07:44.110130+0000","last_became_peered":"2023-11-28T04:07:44.110130+0000","last_unstale":"2023-11-28T04:07:44.490200+0000","last_undegraded":"2023-11-28T04:07:44.490200+0000","last_fullsized":"2023-11-28T04:07:44.490200+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485532+0000","last_change":"2023-11-28T04:07:44.480313+0000","last_active":"2023-11-28T04:07:44.485532+0000","last_peered":"2023-11-28T04:07:44.485532+0000","last_clean":"2023-11-28T04:07:44.485532+0000","last_became_active":"2023-11-28T04:07:44.480057+0000","last_became_peered":"2023-11-28T04:07:44.480057+0000","last_unstale":"2023-11-28T04:07:44.485532+0000","last_undegraded":"2023-11-28T04:07:44.485532+0000","last_fullsized":"2023-11-28T04:07:44.485532+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472584+0000","last_change":"2023-11-28T04:07:44.109074+0000","last_active":"2023-11-28T04:07:44.472584+0000","last_peered":"2023-11-28T04:07:44.472584+0000","last_clean":"2023-11-28T04:07:44.472584+0000","last_became_active":"2023-11-28T04:07:44.108769+0000","last_became_peered":"2023-11-28T04:07:44.108769+0000","last_unstale":"2023-11-28T04:07:44.472584+0000","last_undegraded":"2023-11-28T04:07:44.472584+0000","last_fullsized":"2023-11-28T04:07:44.472584+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.b","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485396+0000","last_change":"2023-11-28T04:07:44.475563+0000","last_active":"2023-11-28T04:07:44.485396+0000","last_peered":"2023-11-28T04:07:44.485396+0000","last_clean":"2023-11-28T04:07:44.485396+0000","last_became_active":"2023-11-28T04:07:44.475180+0000","last_became_peered":"2023-11-28T04:07:44.475180+0000","last_unstale":"2023-11-28T04:07:44.485396+0000","last_undegraded":"2023-11-28T04:07:44.485396+0000","last_fullsized":"2023-11-28T04:07:44.485396+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465693+0000","last_change":"2023-11-28T04:07:44.110327+0000","last_active":"2023-11-28T04:07:44.465693+0000","last_peered":"2023-11-28T04:07:44.465693+0000","last_clean":"2023-11-28T04:07:44.465693+0000","last_became_active":"2023-11-28T04:07:44.110082+0000","last_became_peered":"2023-11-28T04:07:44.110082+0000","last_unstale":"2023-11-28T04:07:44.465693+0000","last_undegraded":"2023-11-28T04:07:44.465693+0000","last_fullsized":"2023-11-28T04:07:44.465693+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.476765+0000","last_change":"2023-11-28T04:07:44.473362+0000","last_active":"2023-11-28T04:07:44.476765+0000","last_peered":"2023-11-28T04:07:44.476765+0000","last_clean":"2023-11-28T04:07:44.476765+0000","last_became_active":"2023-11-28T04:07:44.473079+0000","last_became_peered":"2023-11-28T04:07:44.473079+0000","last_unstale":"2023-11-28T04:07:44.476765+0000","last_undegraded":"2023-11-28T04:07:44.476765+0000","last_fullsized":"2023-11-28T04:07:44.476765+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489420+0000","last_change":"2023-11-28T04:07:44.102468+0000","last_active":"2023-11-28T04:07:44.489420+0000","last_peered":"2023-11-28T04:07:44.489420+0000","last_clean":"2023-11-28T04:07:44.489420+0000","last_became_active":"2023-11-28T04:07:44.102166+0000","last_became_peered":"2023-11-28T04:07:44.102166+0000","last_unstale":"2023-11-28T04:07:44.489420+0000","last_undegraded":"2023-11-28T04:07:44.489420+0000","last_fullsized":"2023-11-28T04:07:44.489420+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.d","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486320+0000","last_change":"2023-11-28T04:07:44.482885+0000","last_active":"2023-11-28T04:07:44.486320+0000","last_peered":"2023-11-28T04:07:44.486320+0000","last_clean":"2023-11-28T04:07:44.486320+0000","last_became_active":"2023-11-28T04:07:44.482557+0000","last_became_peered":"2023-11-28T04:07:44.482557+0000","last_unstale":"2023-11-28T04:07:44.486320+0000","last_undegraded":"2023-11-28T04:07:44.486320+0000","last_fullsized":"2023-11-28T04:07:44.486320+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468160+0000","last_change":"2023-11-28T04:07:44.102694+0000","last_active":"2023-11-28T04:07:44.468160+0000","last_peered":"2023-11-28T04:07:44.468160+0000","last_clean":"2023-11-28T04:07:44.468160+0000","last_became_active":"2023-11-28T04:07:44.102089+0000","last_became_peered":"2023-11-28T04:07:44.102089+0000","last_unstale":"2023-11-28T04:07:44.468160+0000","last_undegraded":"2023-11-28T04:07:44.468160+0000","last_fullsized":"2023-11-28T04:07:44.468160+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489965+0000","last_change":"2023-11-28T04:07:44.489160+0000","last_active":"2023-11-28T04:07:44.489965+0000","last_peered":"2023-11-28T04:07:44.489965+0000","last_clean":"2023-11-28T04:07:44.489965+0000","last_became_active":"2023-11-28T04:07:44.488851+0000","last_became_peered":"2023-11-28T04:07:44.488851+0000","last_unstale":"2023-11-28T04:07:44.489965+0000","last_undegraded":"2023-11-28T04:07:44.489965+0000","last_fullsized":"2023-11-28T04:07:44.489965+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478938+0000","last_change":"2023-11-28T04:07:44.118105+0000","last_active":"2023-11-28T04:07:44.478938+0000","last_peered":"2023-11-28T04:07:44.478938+0000","last_clean":"2023-11-28T04:07:44.478938+0000","last_became_active":"2023-11-28T04:07:44.117793+0000","last_became_peered":"2023-11-28T04:07:44.117793+0000","last_unstale":"2023-11-28T04:07:44.478938+0000","last_undegraded":"2023-11-28T04:07:44.478938+0000","last_fullsized":"2023-11-28T04:07:44.478938+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494822+0000","last_change":"2023-11-28T04:07:44.491014+0000","last_active":"2023-11-28T04:07:44.494822+0000","last_peered":"2023-11-28T04:07:44.494822+0000","last_clean":"2023-11-28T04:07:44.494822+0000","last_became_active":"2023-11-28T04:07:44.490714+0000","last_became_peered":"2023-11-28T04:07:44.490714+0000","last_unstale":"2023-11-28T04:07:44.494822+0000","last_undegraded":"2023-11-28T04:07:44.494822+0000","last_fullsized":"2023-11-28T04:07:44.494822+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.478588+0000","last_change":"2023-11-28T04:07:44.110080+0000","last_active":"2023-11-28T04:07:44.478588+0000","last_peered":"2023-11-28T04:07:44.478588+0000","last_clean":"2023-11-28T04:07:44.478588+0000","last_became_active":"2023-11-28T04:07:44.109823+0000","last_became_peered":"2023-11-28T04:07:44.109823+0000","last_unstale":"2023-11-28T04:07:44.478588+0000","last_undegraded":"2023-11-28T04:07:44.478588+0000","last_fullsized":"2023-11-28T04:07:44.478588+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479651+0000","last_change":"2023-11-28T04:07:44.477357+0000","last_active":"2023-11-28T04:07:44.479651+0000","last_peered":"2023-11-28T04:07:44.479651+0000","last_clean":"2023-11-28T04:07:44.479651+0000","last_became_active":"2023-11-28T04:07:44.477078+0000","last_became_peered":"2023-11-28T04:07:44.477078+0000","last_unstale":"2023-11-28T04:07:44.479651+0000","last_undegraded":"2023-11-28T04:07:44.479651+0000","last_fullsized":"2023-11-28T04:07:44.479651+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.7","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489149+0000","last_change":"2023-11-28T04:07:44.110131+0000","last_active":"2023-11-28T04:07:44.489149+0000","last_peered":"2023-11-28T04:07:44.489149+0000","last_clean":"2023-11-28T04:07:44.489149+0000","last_became_active":"2023-11-28T04:07:44.109844+0000","last_became_peered":"2023-11-28T04:07:44.109844+0000","last_unstale":"2023-11-28T04:07:44.489149+0000","last_undegraded":"2023-11-28T04:07:44.489149+0000","last_fullsized":"2023-11-28T04:07:44.489149+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489248+0000","last_change":"2023-11-28T04:07:44.475348+0000","last_active":"2023-11-28T04:07:44.489248+0000","last_peered":"2023-11-28T04:07:44.489248+0000","last_clean":"2023-11-28T04:07:44.489248+0000","last_became_active":"2023-11-28T04:07:44.475089+0000","last_became_peered":"2023-11-28T04:07:44.475089+0000","last_unstale":"2023-11-28T04:07:44.489248+0000","last_undegraded":"2023-11-28T04:07:44.489248+0000","last_fullsized":"2023-11-28T04:07:44.489248+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472753+0000","last_change":"2023-11-28T04:07:44.113266+0000","last_active":"2023-11-28T04:07:44.472753+0000","last_peered":"2023-11-28T04:07:44.472753+0000","last_clean":"2023-11-28T04:07:44.472753+0000","last_became_active":"2023-11-28T04:07:44.113000+0000","last_became_peered":"2023-11-28T04:07:44.113000+0000","last_unstale":"2023-11-28T04:07:44.472753+0000","last_undegraded":"2023-11-28T04:07:44.472753+0000","last_fullsized":"2023-11-28T04:07:44.472753+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494609+0000","last_change":"2023-11-28T04:07:44.489755+0000","last_active":"2023-11-28T04:07:44.494609+0000","last_peered":"2023-11-28T04:07:44.494609+0000","last_clean":"2023-11-28T04:07:44.494609+0000","last_became_active":"2023-11-28T04:07:44.489466+0000","last_became_peered":"2023-11-28T04:07:44.489466+0000","last_unstale":"2023-11-28T04:07:44.494609+0000","last_undegraded":"2023-11-28T04:07:44.494609+0000","last_fullsized":"2023-11-28T04:07:44.494609+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468894+0000","last_change":"2023-11-28T04:07:44.109468+0000","last_active":"2023-11-28T04:07:44.468894+0000","last_peered":"2023-11-28T04:07:44.468894+0000","last_clean":"2023-11-28T04:07:44.468894+0000","last_became_active":"2023-11-28T04:07:44.109138+0000","last_became_peered":"2023-11-28T04:07:44.109138+0000","last_unstale":"2023-11-28T04:07:44.468894+0000","last_undegraded":"2023-11-28T04:07:44.468894+0000","last_fullsized":"2023-11-28T04:07:44.468894+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489399+0000","last_change":"2023-11-28T04:07:44.477053+0000","last_active":"2023-11-28T04:07:44.489399+0000","last_peered":"2023-11-28T04:07:44.489399+0000","last_clean":"2023-11-28T04:07:44.489399+0000","last_became_active":"2023-11-28T04:07:44.476805+0000","last_became_peered":"2023-11-28T04:07:44.476805+0000","last_unstale":"2023-11-28T04:07:44.489399+0000","last_undegraded":"2023-11-28T04:07:44.489399+0000","last_fullsized":"2023-11-28T04:07:44.489399+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472771+0000","last_change":"2023-11-28T04:07:44.109113+0000","last_active":"2023-11-28T04:07:44.472771+0000","last_peered":"2023-11-28T04:07:44.472771+0000","last_clean":"2023-11-28T04:07:44.472771+0000","last_became_active":"2023-11-28T04:07:44.108854+0000","last_became_peered":"2023-11-28T04:07:44.108854+0000","last_unstale":"2023-11-28T04:07:44.472771+0000","last_undegraded":"2023-11-28T04:07:44.472771+0000","last_fullsized":"2023-11-28T04:07:44.472771+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487309+0000","last_change":"2023-11-28T04:07:44.484405+0000","last_active":"2023-11-28T04:07:44.487309+0000","last_peered":"2023-11-28T04:07:44.487309+0000","last_clean":"2023-11-28T04:07:44.487309+0000","last_became_active":"2023-11-28T04:07:44.484145+0000","last_became_peered":"2023-11-28T04:07:44.484145+0000","last_unstale":"2023-11-28T04:07:44.487309+0000","last_undegraded":"2023-11-28T04:07:44.487309+0000","last_fullsized":"2023-11-28T04:07:44.487309+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470044+0000","last_change":"2023-11-28T04:07:44.117498+0000","last_active":"2023-11-28T04:07:44.470044+0000","last_peered":"2023-11-28T04:07:44.470044+0000","last_clean":"2023-11-28T04:07:44.470044+0000","last_became_active":"2023-11-28T04:07:44.117094+0000","last_became_peered":"2023-11-28T04:07:44.117094+0000","last_unstale":"2023-11-28T04:07:44.470044+0000","last_undegraded":"2023-11-28T04:07:44.470044+0000","last_fullsized":"2023-11-28T04:07:44.470044+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486771+0000","last_change":"2023-11-28T04:07:44.478155+0000","last_active":"2023-11-28T04:07:44.486771+0000","last_peered":"2023-11-28T04:07:44.486771+0000","last_clean":"2023-11-28T04:07:44.486771+0000","last_became_active":"2023-11-28T04:07:44.477831+0000","last_became_peered":"2023-11-28T04:07:44.477831+0000","last_unstale":"2023-11-28T04:07:44.486771+0000","last_undegraded":"2023-11-28T04:07:44.486771+0000","last_fullsized":"2023-11-28T04:07:44.486771+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.474595+0000","last_change":"2023-11-28T04:07:44.117447+0000","last_active":"2023-11-28T04:07:44.474595+0000","last_peered":"2023-11-28T04:07:44.474595+0000","last_clean":"2023-11-28T04:07:44.474595+0000","last_became_active":"2023-11-28T04:07:44.117157+0000","last_became_peered":"2023-11-28T04:07:44.117157+0000","last_unstale":"2023-11-28T04:07:44.474595+0000","last_undegraded":"2023-11-28T04:07:44.474595+0000","last_fullsized":"2023-11-28T04:07:44.474595+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.487297+0000","last_change":"2023-11-28T04:07:44.486163+0000","last_active":"2023-11-28T04:07:44.487297+0000","last_peered":"2023-11-28T04:07:44.487297+0000","last_clean":"2023-11-28T04:07:44.487297+0000","last_became_active":"2023-11-28T04:07:44.485864+0000","last_became_peered":"2023-11-28T04:07:44.485864+0000","last_unstale":"2023-11-28T04:07:44.487297+0000","last_undegraded":"2023-11-28T04:07:44.487297+0000","last_fullsized":"2023-11-28T04:07:44.487297+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475407+0000","last_change":"2023-11-28T04:07:44.109265+0000","last_active":"2023-11-28T04:07:44.475407+0000","last_peered":"2023-11-28T04:07:44.475407+0000","last_clean":"2023-11-28T04:07:44.475407+0000","last_became_active":"2023-11-28T04:07:44.108960+0000","last_became_peered":"2023-11-28T04:07:44.108960+0000","last_unstale":"2023-11-28T04:07:44.475407+0000","last_undegraded":"2023-11-28T04:07:44.475407+0000","last_fullsized":"2023-11-28T04:07:44.475407+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.7","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.494620+0000","last_change":"2023-11-28T04:07:44.489939+0000","last_active":"2023-11-28T04:07:44.494620+0000","last_peered":"2023-11-28T04:07:44.494620+0000","last_clean":"2023-11-28T04:07:44.494620+0000","last_became_active":"2023-11-28T04:07:44.489622+0000","last_became_peered":"2023-11-28T04:07:44.489622+0000","last_unstale":"2023-11-28T04:07:44.494620+0000","last_undegraded":"2023-11-28T04:07:44.494620+0000","last_fullsized":"2023-11-28T04:07:44.494620+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.465855+0000","last_change":"2023-11-28T04:07:44.108144+0000","last_active":"2023-11-28T04:07:44.465855+0000","last_peered":"2023-11-28T04:07:44.465855+0000","last_clean":"2023-11-28T04:07:44.465855+0000","last_became_active":"2023-11-28T04:07:44.107822+0000","last_became_peered":"2023-11-28T04:07:44.107822+0000","last_unstale":"2023-11-28T04:07:44.465855+0000","last_undegraded":"2023-11-28T04:07:44.465855+0000","last_fullsized":"2023-11-28T04:07:44.465855+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490884+0000","last_change":"2023-11-28T04:07:44.484397+0000","last_active":"2023-11-28T04:07:44.490884+0000","last_peered":"2023-11-28T04:07:44.490884+0000","last_clean":"2023-11-28T04:07:44.490884+0000","last_became_active":"2023-11-28T04:07:44.484150+0000","last_became_peered":"2023-11-28T04:07:44.484150+0000","last_unstale":"2023-11-28T04:07:44.490884+0000","last_undegraded":"2023-11-28T04:07:44.490884+0000","last_fullsized":"2023-11-28T04:07:44.490884+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.15","version":"164'3","reported_seq":13,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491701+0000","last_change":"2023-11-28T04:07:44.108445+0000","last_active":"2023-11-28T04:07:44.491701+0000","last_peered":"2023-11-28T04:07:44.491701+0000","last_clean":"2023-11-28T04:07:44.491701+0000","last_became_active":"2023-11-28T04:07:44.108144+0000","last_became_peered":"2023-11-28T04:07:44.108144+0000","last_unstale":"2023-11-28T04:07:44.491701+0000","last_undegraded":"2023-11-28T04:07:44.491701+0000","last_fullsized":"2023-11-28T04:07:44.491701+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472826+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:07:44.472826+0000","last_peered":"2023-11-28T04:07:44.472826+0000","last_clean":"2023-11-28T04:07:44.472826+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:07:44.472826+0000","last_undegraded":"2023-11-28T04:07:44.472826+0000","last_fullsized":"2023-11-28T04:07:44.472826+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.14","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.476487+0000","last_change":"2023-11-28T04:07:44.470645+0000","last_active":"2023-11-28T04:07:44.476487+0000","last_peered":"2023-11-28T04:07:44.476487+0000","last_clean":"2023-11-28T04:07:44.476487+0000","last_became_active":"2023-11-28T04:07:44.470356+0000","last_became_peered":"2023-11-28T04:07:44.470356+0000","last_unstale":"2023-11-28T04:07:44.476487+0000","last_undegraded":"2023-11-28T04:07:44.476487+0000","last_fullsized":"2023-11-28T04:07:44.476487+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.13","version":"164'1","reported_seq":11,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483965+0000","last_change":"2023-11-28T04:07:44.101210+0000","last_active":"2023-11-28T04:07:44.483965+0000","last_peered":"2023-11-28T04:07:44.483965+0000","last_clean":"2023-11-28T04:07:44.483965+0000","last_became_active":"2023-11-28T04:07:44.100914+0000","last_became_peered":"2023-11-28T04:07:44.100914+0000","last_unstale":"2023-11-28T04:07:44.483965+0000","last_undegraded":"2023-11-28T04:07:44.483965+0000","last_fullsized":"2023-11-28T04:07:44.483965+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472943+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:07:44.472943+0000","last_peered":"2023-11-28T04:07:44.472943+0000","last_clean":"2023-11-28T04:07:44.472943+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:07:44.472943+0000","last_undegraded":"2023-11-28T04:07:44.472943+0000","last_fullsized":"2023-11-28T04:07:44.472943+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.479562+0000","last_change":"2023-11-28T04:07:44.476618+0000","last_active":"2023-11-28T04:07:44.479562+0000","last_peered":"2023-11-28T04:07:44.479562+0000","last_clean":"2023-11-28T04:07:44.479562+0000","last_became_active":"2023-11-28T04:07:44.476363+0000","last_became_peered":"2023-11-28T04:07:44.476363+0000","last_unstale":"2023-11-28T04:07:44.479562+0000","last_undegraded":"2023-11-28T04:07:44.479562+0000","last_fullsized":"2023-11-28T04:07:44.479562+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.10","version":"164'4","reported_seq":14,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.490996+0000","last_change":"2023-11-28T04:07:44.109291+0000","last_active":"2023-11-28T04:07:44.490996+0000","last_peered":"2023-11-28T04:07:44.490996+0000","last_clean":"2023-11-28T04:07:44.490996+0000","last_became_active":"2023-11-28T04:07:44.108915+0000","last_became_peered":"2023-11-28T04:07:44.108915+0000","last_unstale":"2023-11-28T04:07:44.490996+0000","last_undegraded":"2023-11-28T04:07:44.490996+0000","last_fullsized":"2023-11-28T04:07:44.490996+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":140,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468583+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:07:44.468583+0000","last_peered":"2023-11-28T04:07:44.468583+0000","last_clean":"2023-11-28T04:07:44.468583+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:07:44.468583+0000","last_undegraded":"2023-11-28T04:07:44.468583+0000","last_fullsized":"2023-11-28T04:07:44.468583+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.15","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489166+0000","last_change":"2023-11-28T04:07:44.476393+0000","last_active":"2023-11-28T04:07:44.489166+0000","last_peered":"2023-11-28T04:07:44.489166+0000","last_clean":"2023-11-28T04:07:44.489166+0000","last_became_active":"2023-11-28T04:07:44.475919+0000","last_became_peered":"2023-11-28T04:07:44.475919+0000","last_unstale":"2023-11-28T04:07:44.489166+0000","last_undegraded":"2023-11-28T04:07:44.489166+0000","last_fullsized":"2023-11-28T04:07:44.489166+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.474486+0000","last_change":"2023-11-28T04:07:44.118149+0000","last_active":"2023-11-28T04:07:44.474486+0000","last_peered":"2023-11-28T04:07:44.474486+0000","last_clean":"2023-11-28T04:07:44.474486+0000","last_became_active":"2023-11-28T04:07:44.117774+0000","last_became_peered":"2023-11-28T04:07:44.117774+0000","last_unstale":"2023-11-28T04:07:44.474486+0000","last_undegraded":"2023-11-28T04:07:44.474486+0000","last_fullsized":"2023-11-28T04:07:44.474486+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.470495+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:07:44.470495+0000","last_peered":"2023-11-28T04:07:44.470495+0000","last_clean":"2023-11-28T04:07:44.470495+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:07:44.470495+0000","last_undegraded":"2023-11-28T04:07:44.470495+0000","last_fullsized":"2023-11-28T04:07:44.470495+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":134,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469257+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:07:44.469257+0000","last_peered":"2023-11-28T04:07:44.469257+0000","last_clean":"2023-11-28T04:07:44.469257+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:07:44.469257+0000","last_undegraded":"2023-11-28T04:07:44.469257+0000","last_fullsized":"2023-11-28T04:07:44.469257+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.485657+0000","last_change":"2023-11-28T04:07:44.480998+0000","last_active":"2023-11-28T04:07:44.485657+0000","last_peered":"2023-11-28T04:07:44.485657+0000","last_clean":"2023-11-28T04:07:44.485657+0000","last_became_active":"2023-11-28T04:07:44.480742+0000","last_became_peered":"2023-11-28T04:07:44.480742+0000","last_unstale":"2023-11-28T04:07:44.485657+0000","last_undegraded":"2023-11-28T04:07:44.485657+0000","last_fullsized":"2023-11-28T04:07:44.485657+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.466623+0000","last_change":"2023-11-28T04:07:44.113995+0000","last_active":"2023-11-28T04:07:44.466623+0000","last_peered":"2023-11-28T04:07:44.466623+0000","last_clean":"2023-11-28T04:07:44.466623+0000","last_became_active":"2023-11-28T04:07:44.113244+0000","last_became_peered":"2023-11-28T04:07:44.113244+0000","last_unstale":"2023-11-28T04:07:44.466623+0000","last_undegraded":"2023-11-28T04:07:44.466623+0000","last_fullsized":"2023-11-28T04:07:44.466623+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.469564+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:07:44.469564+0000","last_peered":"2023-11-28T04:07:44.469564+0000","last_clean":"2023-11-28T04:07:44.469564+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:07:44.469564+0000","last_undegraded":"2023-11-28T04:07:44.469564+0000","last_fullsized":"2023-11-28T04:07:44.469564+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.13","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.489019+0000","last_change":"2023-11-28T04:07:44.475084+0000","last_active":"2023-11-28T04:07:44.489019+0000","last_peered":"2023-11-28T04:07:44.489019+0000","last_clean":"2023-11-28T04:07:44.489019+0000","last_became_active":"2023-11-28T04:07:44.474807+0000","last_became_peered":"2023-11-28T04:07:44.474807+0000","last_unstale":"2023-11-28T04:07:44.489019+0000","last_undegraded":"2023-11-28T04:07:44.489019+0000","last_fullsized":"2023-11-28T04:07:44.489019+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.14","version":"164'2","reported_seq":12,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.486214+0000","last_change":"2023-11-28T04:07:44.109170+0000","last_active":"2023-11-28T04:07:44.486214+0000","last_peered":"2023-11-28T04:07:44.486214+0000","last_clean":"2023-11-28T04:07:44.486214+0000","last_became_active":"2023-11-28T04:07:44.108884+0000","last_became_peered":"2023-11-28T04:07:44.108884+0000","last_unstale":"2023-11-28T04:07:44.486214+0000","last_undegraded":"2023-11-28T04:07:44.486214+0000","last_fullsized":"2023-11-28T04:07:44.486214+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":132,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475283+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:07:44.475283+0000","last_peered":"2023-11-28T04:07:44.475283+0000","last_clean":"2023-11-28T04:07:44.475283+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:07:44.475283+0000","last_undegraded":"2023-11-28T04:07:44.475283+0000","last_fullsized":"2023-11-28T04:07:44.475283+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.10","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.483361+0000","last_change":"2023-11-28T04:07:44.475198+0000","last_active":"2023-11-28T04:07:44.483361+0000","last_peered":"2023-11-28T04:07:44.483361+0000","last_clean":"2023-11-28T04:07:44.483361+0000","last_became_active":"2023-11-28T04:07:44.474922+0000","last_became_peered":"2023-11-28T04:07:44.474922+0000","last_unstale":"2023-11-28T04:07:44.483361+0000","last_undegraded":"2023-11-28T04:07:44.483361+0000","last_fullsized":"2023-11-28T04:07:44.483361+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.475742+0000","last_change":"2023-11-28T04:07:44.112188+0000","last_active":"2023-11-28T04:07:44.475742+0000","last_peered":"2023-11-28T04:07:44.475742+0000","last_clean":"2023-11-28T04:07:44.475742+0000","last_became_active":"2023-11-28T04:07:44.111845+0000","last_became_peered":"2023-11-28T04:07:44.111845+0000","last_unstale":"2023-11-28T04:07:44.475742+0000","last_undegraded":"2023-11-28T04:07:44.475742+0000","last_fullsized":"2023-11-28T04:07:44.475742+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":133,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.472715+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:07:44.472715+0000","last_peered":"2023-11-28T04:07:44.472715+0000","last_clean":"2023-11-28T04:07:44.472715+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:07:44.472715+0000","last_undegraded":"2023-11-28T04:07:44.472715+0000","last_fullsized":"2023-11-28T04:07:44.472715+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":9,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.491484+0000","last_change":"2023-11-28T04:07:44.490428+0000","last_active":"2023-11-28T04:07:44.491484+0000","last_peered":"2023-11-28T04:07:44.491484+0000","last_clean":"2023-11-28T04:07:44.491484+0000","last_became_active":"2023-11-28T04:07:44.490238+0000","last_became_peered":"2023-11-28T04:07:44.490238+0000","last_unstale":"2023-11-28T04:07:44.491484+0000","last_undegraded":"2023-11-28T04:07:44.491484+0000","last_fullsized":"2023-11-28T04:07:44.491484+0000","mapping_epoch":163,"log_start":"0'0","ondisk_log_start":"0'0","created":163,"last_epoch_clean":164,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:44.085647+0000","last_clean_scrub_stamp":"2023-11-28T04:07:44.085647+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":10,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.468682+0000","last_change":"2023-11-28T04:07:44.102624+0000","last_active":"2023-11-28T04:07:44.468682+0000","last_peered":"2023-11-28T04:07:44.468682+0000","last_clean":"2023-11-28T04:07:44.468682+0000","last_became_active":"2023-11-28T04:07:44.102114+0000","last_became_peered":"2023-11-28T04:07:44.102114+0000","last_unstale":"2023-11-28T04:07:44.468682+0000","last_undegraded":"2023-11-28T04:07:44.468682+0000","last_fullsized":"2023-11-28T04:07:44.468682+0000","mapping_epoch":162,"log_start":"0'0","ondisk_log_start":"0'0","created":162,"last_epoch_clean":163,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:07:43.078371+0000","last_clean_scrub_stamp":"2023-11-28T04:07:43.078371+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":146,"reported_epoch":164,"state":"active+clean","last_fresh":"2023-11-28T04:07:44.467499+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:07:44.467499+0000","last_peered":"2023-11-28T04:07:44.467499+0000","last_clean":"2023-11-28T04:07:44.467499+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:07:44.467499+0000","last_undegraded":"2023-11-28T04:07:44.467499+0000","last_fullsized":"2023-11-28T04:07:44.467499+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":20,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575747,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":2064384,"data_stored":1034588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.47399999999999998,"5min":0.32100000000000001,"15min":0.29299999999999998},"max":{"1min":0.755,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.622,"15min":0.61699999999999999},"min":{"1min":0.47799999999999998,"5min":0.46100000000000002,"15min":0.379},"max":{"1min":0.72699999999999998,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.57599999999999996}]},{"osd":1,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56100000000000005,"15min":0.56100000000000005},"min":{"1min":0.443,"5min":0.372,"15min":0.32100000000000001},"max":{"1min":0.78100000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.56999999999999995},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34300000000000003},"max":{"1min":0.72399999999999998,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.61699999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.40999999999999998,"5min":0.39000000000000001,"15min":0.34599999999999997},"max":{"1min":0.746,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58299999999999996,"15min":0.57599999999999996},"min":{"1min":0.437,"5min":0.373,"15min":0.35299999999999998},"max":{"1min":0.76500000000000001,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.71499999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58099999999999996,"15min":0.57599999999999996},"min":{"1min":0.40999999999999998,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.72799999999999998,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.61099999999999999,"15min":0.60299999999999998},"min":{"1min":0.41099999999999998,"5min":0.40699999999999997,"15min":0.39100000000000001},"max":{"1min":0.82399999999999995,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.67500000000000004}]},{"osd":4,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45600000000000002,"15min":0.45900000000000002},"min":{"1min":0.33300000000000002,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":0.77300000000000002,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.496},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48899999999999999,"15min":0.48799999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.74099999999999999,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.60699999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.505,"15min":0.499},"min":{"1min":0.36299999999999999,"5min":0.312,"15min":0.312},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53600000000000003,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.83499999999999996,"5min":0.878,"15min":0.878},"last":0.65700000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:07:16 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.56200000000000006,"15min":0.56000000000000005},"min":{"1min":0.44500000000000001,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.872,"5min":0.875,"15min":0.875},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.56100000000000005,"15min":0.55400000000000005},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78100000000000003,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.58499999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575747,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55100000000000005,"15min":0.54200000000000004},"min":{"1min":0.434,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.71299999999999997,"5min":0.75600000000000001,"15min":0.78400000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56599999999999995,"15min":0.56299999999999994},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.29099999999999998},"max":{"1min":0.70899999999999996,"5min":0.73499999999999999,"15min":0.81899999999999995},"last":0.56899999999999995}]},{"osd":1,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.442,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":0.68799999999999994,"5min":0.752,"15min":0.80300000000000005},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55200000000000005,"15min":0.54700000000000004},"min":{"1min":0.45900000000000002,"5min":0.41699999999999998,"15min":0.33800000000000002},"max":{"1min":0.68600000000000005,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.44700000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.57999999999999996,"15min":0.57299999999999995},"min":{"1min":0.41599999999999998,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.67900000000000005,"5min":0.79500000000000004,"15min":0.79600000000000004},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.57199999999999995,"15min":0.57699999999999996},"min":{"1min":0.42099999999999999,"5min":0.40799999999999997,"15min":0.379},"max":{"1min":0.71899999999999997,"5min":0.78000000000000003,"15min":0.83099999999999996},"last":0.55900000000000005}]},{"osd":3,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.58199999999999996,"15min":0.57999999999999996},"min":{"1min":0.44700000000000001,"5min":0.40500000000000003,"15min":0.38600000000000001},"max":{"1min":0.73599999999999999,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.58699999999999997,"15min":0.59099999999999997},"min":{"1min":0.44900000000000001,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79800000000000004},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.44600000000000001,"15min":0.45000000000000001},"min":{"1min":0.318,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.65500000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34599999999999997,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.61799999999999999,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.502,"15min":0.51800000000000002},"min":{"1min":0.36199999999999999,"5min":0.318,"15min":0.27800000000000002},"max":{"1min":0.65400000000000003,"5min":0.71499999999999997,"15min":0.79200000000000004},"last":0.623},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.68700000000000006,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.63300000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:07:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.54200000000000004},"min":{"1min":0.39300000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.68899999999999995,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.55500000000000005,"15min":0.55000000000000004},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.318},"max":{"1min":0.67800000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.64900000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575746,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303196,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068644,"statfs":{"total":96636764160,"available":96326291456,"internally_reserved":0,"allocated":2052096,"data_stored":1033854,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.27800000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.41499999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.56399999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.68400000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.60699999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.64200000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575747,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303208,"kb_used_data":2016,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068632,"statfs":{"total":96636764160,"available":96326279168,"internally_reserved":0,"allocated":2064384,"data_stored":1033980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.35899999999999999,"15min":0.36099999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.45700000000000002,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38800000000000001},"min":{"1min":0.26900000000000002,"5min":0.26100000000000001,"15min":0.245},"max":{"1min":0.48199999999999998,"5min":0.60099999999999998,"15min":0.61199999999999999},"last":0.22800000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.376,"15min":0.38200000000000001},"min":{"1min":0.309,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.54600000000000004,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.46899999999999997,"15min":0.46800000000000003},"min":{"1min":0.36199999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.54700000000000004,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.32200000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.41299999999999998,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.56299999999999994,"5min":0.621,"15min":0.67300000000000004},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.436,"5min":0.44500000000000001,"15min":0.44800000000000001},"min":{"1min":0.317,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.59299999999999997,"5min":0.65700000000000003,"15min":0.77500000000000002},"last":0.29399999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.65300000000000002,"15min":0.64100000000000001},"min":{"1min":0.46300000000000002,"5min":0.438,"15min":0.39700000000000002},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.62},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.66000000000000003,"15min":0.65400000000000003},"min":{"1min":0.54600000000000004,"5min":0.41199999999999998,"15min":0.39500000000000002},"max":{"1min":0.83799999999999997,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.66200000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65500000000000003,"15min":0.65100000000000002},"min":{"1min":0.48099999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.874,"5min":0.95899999999999996,"15min":0.98899999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66000000000000003,"15min":0.65100000000000002},"min":{"1min":0.51300000000000001,"5min":0.377,"15min":0.377},"max":{"1min":1.248,"5min":1.248,"15min":1.248},"last":0.628}]},{"osd":6,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.64000000000000001,"15min":0.63200000000000001},"min":{"1min":0.46700000000000003,"5min":0.45800000000000002,"15min":0.42499999999999999},"max":{"1min":1.1439999999999999,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.64700000000000002,"15min":0.64400000000000002},"min":{"1min":0.47499999999999998,"5min":0.42499999999999999,"15min":0.40999999999999998},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.53500000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:06:59 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.67100000000000004,"15min":0.65900000000000003},"min":{"1min":0.53600000000000003,"5min":0.439,"15min":0.41799999999999998},"max":{"1min":1.1850000000000001,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.71499999999999997,"15min":0.70599999999999996},"min":{"1min":0.61599999999999999,"5min":0.49399999999999999,"15min":0.42599999999999999},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.47999999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575748,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034442,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.41299999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.38200000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.314}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.61199999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.68000000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.627},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.53700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575747,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40600000000000003,"5min":0.38500000000000001,"15min":0.371},"min":{"1min":0.27200000000000002,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.621,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.39900000000000002,"15min":0.39500000000000002},"min":{"1min":0.28799999999999998,"5min":0.27600000000000002,"15min":0.22900000000000001},"max":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.245}]},{"osd":1,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.40899999999999997,"15min":0.39800000000000002},"min":{"1min":0.30399999999999999,"5min":0.26800000000000002,"15min":0.246},"max":{"1min":0.59699999999999998,"5min":0.59899999999999998,"15min":0.626},"last":0.26000000000000001},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.442,"15min":0.434},"min":{"1min":0.29499999999999998,"5min":0.29199999999999998,"15min":0.25700000000000001},"max":{"1min":0.59999999999999998,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.36399999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.44,"15min":0.436},"min":{"1min":0.35699999999999998,"5min":0.30499999999999999,"15min":0.30399999999999999},"max":{"1min":0.65200000000000002,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.48299999999999998,"15min":0.46899999999999997},"min":{"1min":0.33600000000000002,"5min":0.33000000000000002,"15min":0.30399999999999999},"max":{"1min":0.67600000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.38600000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.627,"15min":0.61799999999999999},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.63600000000000001,"15min":0.64200000000000002},"min":{"1min":0.503,"5min":0.375,"15min":0.375},"max":{"1min":0.88,"5min":0.94799999999999995,"15min":1.085},"last":0.64300000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.63300000000000001,"15min":0.63600000000000001},"min":{"1min":0.44400000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.93600000000000005,"15min":1.1200000000000001},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.67600000000000005,"15min":0.65900000000000003},"min":{"1min":0.436,"5min":0.436,"15min":0.42599999999999999},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.65800000000000003,"15min":0.65300000000000002},"min":{"1min":0.48099999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.0980000000000001,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.70099999999999996,"5min":0.65900000000000003,"15min":0.66200000000000003},"min":{"1min":0.48299999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":1.145},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 04:07:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68799999999999994,"15min":0.67300000000000004},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.434},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.67500000000000004,"15min":0.67000000000000004},"min":{"1min":0.52300000000000002,"5min":0.49199999999999999,"15min":0.47099999999999997},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.61599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575746,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303204,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068636,"statfs":{"total":96636764160,"available":96326283264,"internally_reserved":0,"allocated":2060288,"data_stored":1034478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.499}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.57699999999999996}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48299999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.59099999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.432},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.56499999999999995}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.621},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.629}]}]},{"osd":5,"up_from":13,"seq":55834575748,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307300,"kb_used_data":2012,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064540,"statfs":{"total":96636764160,"available":96322088960,"internally_reserved":0,"allocated":2060288,"data_stored":1034768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.53700000000000003},"min":{"1min":0.32800000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.72599999999999998,"5min":0.755,"15min":0.77800000000000002},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.54900000000000004,"15min":0.55000000000000004},"min":{"1min":0.371,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.67300000000000004,"5min":0.76600000000000001,"15min":0.76800000000000002},"last":0.39600000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.59699999999999998,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.73399999999999999,"5min":0.78900000000000003,"15min":0.79800000000000004},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.60499999999999998,"15min":0.59599999999999997},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.74199999999999999,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.40899999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.60299999999999998,"15min":0.59799999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.80100000000000005,"5min":0.85299999999999998,"15min":0.89400000000000002},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.61099999999999999,"15min":0.60599999999999998},"min":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.376},"max":{"1min":0.77200000000000002,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.34699999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55900000000000005,"15min":0.55600000000000005},"min":{"1min":0.39600000000000002,"5min":0.374,"15min":0.34699999999999998},"max":{"1min":0.69799999999999995,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.372},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58799999999999997,"15min":0.58399999999999996},"min":{"1min":0.42499999999999999,"5min":0.38200000000000001,"15min":0.378},"max":{"1min":0.76000000000000001,"5min":0.85499999999999998,"15min":0.85499999999999998},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.442,"15min":0.44700000000000001},"min":{"1min":0.33400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.63900000000000001,"5min":0.69499999999999995,"15min":0.83799999999999997},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.54700000000000004,"15min":0.53900000000000003},"min":{"1min":0.41199999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":0.73299999999999998,"5min":0.73399999999999999,"15min":0.84499999999999997},"last":0.498}]},{"osd":6,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.48199999999999998,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.79000000000000004},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.55500000000000005,"15min":0.55100000000000005},"min":{"1min":0.40899999999999997,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.56899999999999995}]},{"osd":7,"last update":"Tue Nov 28 04:07:00 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.499,"15min":0.49099999999999999},"min":{"1min":0.34699999999999998,"5min":0.308,"15min":0.26600000000000001},"max":{"1min":0.66300000000000003,"5min":0.69899999999999995,"15min":0.78500000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.372,"5min":0.32000000000000001,"15min":0.313},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.81999999999999995},"last":0.51700000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":588,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":914,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:07:52.187 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T04:07:52.188 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T04:07:52.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.340+0000 7ff5a759e700 1 Processor -- start 2023-11-28T04:07:52.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.340+0000 7ff5a759e700 1 -- start start 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a814e530 0x7ff5a80c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a759e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff5a8154950 0x7ff5a814e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a759e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff5a80c7bc0 con 0x7ff5a8158570 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a759e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff5a8155920 con 0x7ff5a8154950 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a8154dd0 con 0x7ff5a814e530 2023-11-28T04:07:52.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff5a8154950 0x7ff5a814e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff5a8154950 0x7ff5a814e910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58698/0 (socket says 172.21.15.73:58698) 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 -- 172.21.15.73:0/2466626359 learned_addr learned my addr 172.21.15.73:0/2466626359 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 -- 172.21.15.73:0/2466626359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a814e530 msgr2=0x7ff5a80c66f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 --2- 172.21.15.73:0/2466626359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a814e530 0x7ff5a80c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 -- 172.21.15.73:0/2466626359 >> v1:172.21.15.150:6789/0 conn(0x7ff5a8158570 legacy=0x7ff5a80aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 -- 172.21.15.73:0/2466626359 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff5a8155080 con 0x7ff5a8154950 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.341+0000 7ff5a5d9b700 1 --2- 172.21.15.73:0/2466626359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a8154950 0x7ff5a814e910 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7ff59c006fd0 tx=0x7ff59c0090f0).ready entity=mon.0 client_cookie=7107e1e334e030cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a559a700 1 -- 172.21.15.73:0/2466626359 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00b600 con 0x7ff5a8154950 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a559a700 1 -- 172.21.15.73:0/2466626359 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59c00b760 con 0x7ff5a8154950 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a559a700 1 -- 172.21.15.73:0/2466626359 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59c00bae0 con 0x7ff5a8154950 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 -- 172.21.15.73:0/2466626359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a8154950 msgr2=0x7ff5a814e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 --2- 172.21.15.73:0/2466626359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a8154950 0x7ff5a814e910 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 rx=0x7ff59c006fd0 tx=0x7ff59c0090f0).stop 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 -- 172.21.15.73:0/2466626359 shutdown_connections 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 --2- 172.21.15.73:0/2466626359 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a814e530 0x7ff5a80c66f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 --2- 172.21.15.73:0/2466626359 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a8154950 0x7ff5a814e910 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 -- 172.21.15.73:0/2466626359 >> 172.21.15.73:0/2466626359 conn(0x7ff5a8064990 msgr2=0x7ff5a8064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 -- 172.21.15.73:0/2466626359 shutdown_connections 2023-11-28T04:07:52.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.342+0000 7ff5a759e700 1 -- 172.21.15.73:0/2466626359 wait complete. 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.343+0000 7ff5a759e700 1 Processor -- start 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.343+0000 7ff5a759e700 1 -- start start 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a8154950 0x7ff5a8164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a8158570 0x7ff5a814d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80b0e10 con 0x7ff5a814e530 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a80c4e90 con 0x7ff5a8158570 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a759e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff5a81653b0 con 0x7ff5a8154950 2023-11-28T04:07:52.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58712/0 (socket says 172.21.15.73:58712) 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a659c700 1 -- 172.21.15.73:0/1368530230 learned_addr learned my addr 172.21.15.73:0/1368530230 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a5d9b700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a8154950 0x7ff5a8164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.344+0000 7ff5a6d9d700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a8158570 0x7ff5a814d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 -- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a8154950 msgr2=0x7ff5a8164ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a8154950 0x7ff5a8164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 -- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a8158570 msgr2=0x7ff5a814d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a8158570 0x7ff5a814d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 -- 172.21.15.73:0/1368530230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59c009370 con 0x7ff5a814e530 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a659c700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7ff59800b530 tx=0x7ff5980090f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800d4a0 con 0x7ff5a814e530 2023-11-28T04:07:52.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff59800d620 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff59800d9e0 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5a814ed80 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.345+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5a80c53c0 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.346+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff5a80c3050 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.346+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff598012410 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.347+0000 7ff5967fc700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 0x7ff588061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.347+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7ff59807a110 con 0x7ff5a814e530 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.347+0000 7ff5a5d9b700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 0x7ff588061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:52.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.347+0000 7ff5a5d9b700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 0x7ff588061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff59c006fd0 tx=0x7ff59c017000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:52.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.351+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff59804b380 con 0x7ff5a814e530 2023-11-28T04:07:52.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.580+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7ff5a8062ee0 con 0x7ff5a814e530 2023-11-28T04:07:52.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.584+0000 7ff5967fc700 1 -- 172.21.15.73:0/1368530230 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v38) v1 ==== 190+0+0 (secure 0 0 0) 0x7ff59804e9a0 con 0x7ff5a814e530 2023-11-28T04:07:52.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 msgr2=0x7ff588061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 0x7ff588061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7ff59c006fd0 tx=0x7ff59c017000).stop 2023-11-28T04:07:52.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 msgr2=0x7ff5a80c3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:52.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 rx=0x7ff59800b530 tx=0x7ff5980090f0).stop 2023-11-28T04:07:52.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 shutdown_connections 2023-11-28T04:07:52.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff58805f150 0x7ff588061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff5a8158570 0x7ff5a814d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff5a814e530 0x7ff5a80c3200 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 --2- 172.21.15.73:0/1368530230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff5a8154950 0x7ff5a8164ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:52.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 >> 172.21.15.73:0/1368530230 conn(0x7ff5a8064990 msgr2=0x7ff5a8155ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:52.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 shutdown_connections 2023-11-28T04:07:52.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.586+0000 7ff5a759e700 1 -- 172.21.15.73:0/1368530230 wait complete. 2023-11-28T04:07:52.590 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T04:07:52.599 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:07:52.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.715+0000 7f79f7b0c700 1 Processor -- start 2023-11-28T04:07:53.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.715+0000 7f79f7b0c700 1 -- start start 2023-11-28T04:07:53.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f7b0c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00caab0 0x7f79f00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f7b0c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79f00cae90 0x7f79f00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f7b0c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79f00a9a90 con 0x7f79f00aa960 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f7b0c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79f0154ce0 con 0x7f79f00cae90 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f7b0c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00a8e80 con 0x7f79f00caab0 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79f00cae90 0x7f79f00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f60a9700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f79f00aa960 0x7f79f00c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55908/0 (socket says 172.21.15.73:55908) 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f79f00cae90 0x7f79f00c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58714/0 (socket says 172.21.15.73:58714) 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f60a9700 1 -- 172.21.15.73:0/3704098112 learned_addr learned my addr 172.21.15.73:0/3704098112 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 -- 172.21.15.73:0/3704098112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00caab0 msgr2=0x7f79f00aa470 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 --2- 172.21.15.73:0/3704098112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00caab0 0x7f79f00aa470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 -- 172.21.15.73:0/3704098112 >> v1:172.21.15.73:6789/0 conn(0x7f79f00aa960 legacy=0x7f79f00c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.141 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 -- 172.21.15.73:0/3704098112 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79f00a9160 con 0x7f79f00cae90 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f48a6700 1 -- 172.21.15.73:0/3704098112 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 509916513 0 0) 0x7f79f00a9a90 con 0x7f79f00aa960 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f50a7700 1 --2- 172.21.15.73:0/3704098112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00cae90 0x7f79f00c0ab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f79e0009760 tx=0x7f79e0009060).ready entity=mon.0 client_cookie=65d13cf800b0385f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f48a6700 1 -- 172.21.15.73:0/3704098112 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000b580 con 0x7f79f00cae90 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f48a6700 1 -- 172.21.15.73:0/3704098112 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e000b6e0 con 0x7f79f00cae90 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.716+0000 7f79f48a6700 1 -- 172.21.15.73:0/3704098112 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e000ba60 con 0x7f79f00cae90 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3704098112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00cae90 msgr2=0x7f79f00c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3704098112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00cae90 0x7f79f00c0ab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 rx=0x7f79e0009760 tx=0x7f79e0009060).stop 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3704098112 shutdown_connections 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3704098112 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00caab0 0x7f79f00aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3704098112 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00cae90 0x7f79f00c0ab0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3704098112 >> 172.21.15.73:0/3704098112 conn(0x7f79f0094920 msgr2=0x7f79f006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3704098112 shutdown_connections 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.717+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3704098112 wait complete. 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.718+0000 7f79f7b0c700 1 Processor -- start 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.718+0000 7f79f7b0c700 1 -- start start 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00cae90 0x7f79f00c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 0x7f79f00afcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00c2c40 con 0x7f79f00caab0 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00c3a90 con 0x7f79f00c2800 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79f00af840 con 0x7f79f00cae90 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:58728/0 (socket says 172.21.15.73:58728) 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 -- 172.21.15.73:0/3512359976 learned_addr learned my addr 172.21.15.73:0/3512359976 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 -- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00cae90 msgr2=0x7f79f00c2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00cae90 0x7f79f00c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f60a9700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 0x7f79f00afcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 -- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 msgr2=0x7f79f00afcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 0x7f79f00afcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 -- 172.21.15.73:0/3512359976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e00092a0 con 0x7f79f00caab0 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f60a9700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 0x7f79f00afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f58a8700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f79ec00b8e0 tx=0x7f79ec009400).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00c7d0 con 0x7f79f00caab0 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79ec00c040 con 0x7f79f00caab0 2023-11-28T04:07:53.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79f0150e30 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.719+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79ec00c990 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.720+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79f00c3160 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.721+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79f006c860 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.721+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f79ec012020 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.721+0000 7f79e5ffb700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 0x7f79d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.721+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f79ec0795d0 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.721+0000 7f79f50a7700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 0x7f79d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.722+0000 7f79f50a7700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 0x7f79d4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f79f00c6040 tx=0x7f79e0017000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.725+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f79ec04a840 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.927+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f79f00a9440 con 0x7f79f00caab0 2023-11-28T04:07:53.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.928+0000 7f79e5ffb700 1 -- 172.21.15.73:0/3512359976 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f79ec01a070 con 0x7f79f00caab0 2023-11-28T04:07:53.146 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:07:53.146 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:07:53.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 msgr2=0x7f79d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.147 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 0x7f79d4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f79f00c6040 tx=0x7f79e0017000).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 msgr2=0x7f79f0150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 rx=0x7f79ec00b8e0 tx=0x7f79ec009400).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 shutdown_connections 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d405f150 0x7f79d4061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79f00c2800 0x7f79f00afcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79f00caab0 0x7f79f0150940 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 --2- 172.21.15.73:0/3512359976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79f00cae90 0x7f79f00c2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 >> 172.21.15.73:0/3512359976 conn(0x7f79f0094920 msgr2=0x7f79f00ab7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 shutdown_connections 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:52.930+0000 7f79f7b0c700 1 -- 172.21.15.73:0/3512359976 wait complete. 2023-11-28T04:07:53.148 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:07:53.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 58, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T04:07:44.458686+0000', 'modified': '2023-11-28T04:07:46.468974+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8562}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8562': {'gid': 8562, 'name': 'a', 'rank': 0, 'incarnation': 56, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/3981387608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 3981387608}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3981387608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [20], 'metadata_pool': 19, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T04:07:53.552 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T04:07:53.552 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T04:07:53.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.666+0000 7f685c1b0700 1 Processor -- start 2023-11-28T04:07:53.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.666+0000 7f685c1b0700 1 -- start start 2023-11-28T04:07:53.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685c1b0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f68540c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.669 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685c1b0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6854158320 0x7f68541587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685c1b0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6854158cc0 con 0x7f68540c6190 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685c1b0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68540c72d0 con 0x7f6854158320 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685c1b0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854064570 con 0x7f6854098d10 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685a74d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6854158320 0x7f68541587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685a74d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6854158320 0x7f68541587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38072/0 (socket says 172.21.15.73:38072) 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f685a74d700 1 -- 172.21.15.73:0/1034927252 learned_addr learned my addr 172.21.15.73:0/1034927252 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.667+0000 7f6859f4c700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f68540c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685a74d700 1 -- 172.21.15.73:0/1034927252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 msgr2=0x7f68540c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685a74d700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f68540c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685a74d700 1 -- 172.21.15.73:0/1034927252 >> v1:172.21.15.150:6789/0 conn(0x7f68540c6190 legacy=0x7f6854063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685a74d700 1 -- 172.21.15.73:0/1034927252 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854154bc0 con 0x7f6854158320 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f6859f4c700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f68540c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685a74d700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f68541587d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f684c006fd0 tx=0x7f684c0090f0).ready entity=mon.0 client_cookie=a67e3c27b94851e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f6858f4a700 1 -- 172.21.15.73:0/1034927252 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00b600 con 0x7f6854158320 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f6858f4a700 1 -- 172.21.15.73:0/1034927252 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00b760 con 0x7f6854158320 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f6858f4a700 1 -- 172.21.15.73:0/1034927252 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00bae0 con 0x7f6854158320 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 -- 172.21.15.73:0/1034927252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 msgr2=0x7f68541587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f68541587d0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 rx=0x7f684c006fd0 tx=0x7f684c0090f0).stop 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 -- 172.21.15.73:0/1034927252 shutdown_connections 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f68541587d0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 --2- 172.21.15.73:0/1034927252 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f68540c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.668+0000 7f685c1b0700 1 -- 172.21.15.73:0/1034927252 >> 172.21.15.73:0/1034927252 conn(0x7f68540cb310 msgr2=0x7f68540cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.669+0000 7f685c1b0700 1 -- 172.21.15.73:0/1034927252 shutdown_connections 2023-11-28T04:07:53.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.669+0000 7f685c1b0700 1 -- 172.21.15.73:0/1034927252 wait complete. 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 Processor -- start 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 -- start start 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f6854154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68540c6190 0x7f685414a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.674 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854157a50 con 0x7f6854158320 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6854155f70 con 0x7f68540c6190 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685c1b0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6854155760 con 0x7f6854098d10 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.670+0000 7f685a74d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38086/0 (socket says 172.21.15.73:38086) 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 -- 172.21.15.73:0/2544058818 learned_addr learned my addr 172.21.15.73:0/2544058818 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f6854154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685974b700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68540c6190 0x7f685414a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 -- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 msgr2=0x7f6854154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f6854154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 -- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68540c6190 msgr2=0x7f685414a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68540c6190 0x7f685414a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 -- 172.21.15.73:0/2544058818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f684c009370 con 0x7f6854158320 2023-11-28T04:07:53.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685a74d700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f68540c2360 tx=0x7f684c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f6849ffb700 1 -- 172.21.15.73:0/2544058818 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c006d90 con 0x7f6854158320 2023-11-28T04:07:53.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f6849ffb700 1 -- 172.21.15.73:0/2544058818 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f684c00bea0 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68541559b0 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f6849ffb700 1 -- 172.21.15.73:0/2544058818 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c00bae0 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.671+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6854157670 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.673+0000 7f6849ffb700 1 -- 172.21.15.73:0/2544058818 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f684c01b8b0 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.673+0000 7f6849ffb700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 0x7f68380615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.673+0000 7f6849ffb700 1 -- 172.21.15.73:0/2544058818 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f684c07bcc0 con 0x7f6854158320 2023-11-28T04:07:53.677 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.673+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 0x7f68380615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.674+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 0x7f68380615e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68540ca1c0 tx=0x7f68500099d0).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.675+0000 7f686db48b80 1 Processor -- start 2023-11-28T04:07:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.675+0000 7f686db48b80 1 -- start start 2023-11-28T04:07:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e17df5e40 0x560e18061d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.684 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e18034600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e17fff060 con 0x560e18082ea0 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e17f761f0 con 0x560e18039cb0 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e17f8b440 con 0x560e17df5e40 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38092/0 (socket says 172.21.15.73:38092) 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 -- 172.21.15.73:0/1100121646 learned_addr learned my addr 172.21.15.73:0/1100121646 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 -- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e17df5e40 msgr2=0x560e18061d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e17df5e40 0x560e18061d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 -- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 msgr2=0x560e18034600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685a74d700 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e18034600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e18034600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 -- 172.21.15.73:0/1100121646 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e17f84620 con 0x560e18082ea0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f685974b700 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f6844003880 tx=0x7f6844007d40).ready entity=mon.0 client_cookie=fe32345fab541aa1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.676+0000 7f683effd700 1 -- 172.21.15.73:0/1100121646 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6844003040 con 0x560e18082ea0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f683effd700 1 -- 172.21.15.73:0/1100121646 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6844003dc0 con 0x560e18082ea0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f683effd700 1 -- 172.21.15.73:0/1100121646 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684400fa80 con 0x560e18082ea0 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 -- 172.21.15.73:0/1100121646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 msgr2=0x560e180397c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 rx=0x7f6844003880 tx=0x7f6844007d40).stop 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 -- 172.21.15.73:0/1100121646 shutdown_connections 2023-11-28T04:07:53.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e18034600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e17df5e40 0x560e18061d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 --2- 172.21.15.73:0/1100121646 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e180397c0 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 -- 172.21.15.73:0/1100121646 >> 172.21.15.73:0/1100121646 conn(0x560e17ea34a0 msgr2=0x560e18082790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 -- 172.21.15.73:0/1100121646 shutdown_connections 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.677+0000 7f686db48b80 1 -- 172.21.15.73:0/1100121646 wait complete. 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.678+0000 7f686db48b80 1 -- start start 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.678+0000 7f686db48b80 1 Processor -- start 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e17df5e40 0x560e18063c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 0x560e1807d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e17f8b580 con 0x560e18082ea0 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e17f8b800 con 0x560e17df5e40 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e17f56e10 con 0x560e18039cb0 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 10 client.0 ll_register_callbacks cb 0x560e17ff9460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38096/0 (socket says 172.21.15.73:38096) 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 -- 172.21.15.73:0/576276475 learned_addr learned my addr 172.21.15.73:0/576276475 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f6859f4c700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e17df5e40 0x560e18063c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685974b700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 0x560e1807d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 -- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 msgr2=0x560e1807d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 0x560e1807d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 -- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e17df5e40 msgr2=0x560e18063c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e17df5e40 0x560e18063c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 -- 172.21.15.73:0/576276475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6844003500 con 0x560e18082ea0 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685974b700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 0x560e1807d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:53.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f685a74d700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f684c07a140 tx=0x7f684c00e000).ready entity=mon.0 client_cookie=566b1073c45f7cc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f6833fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f684c056250 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f684c031c90 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e17df1ca0 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f684c031e50 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 10 client.8721 fetch_fsmap learned FSMap version 58 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.679+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x560e17f7f9c0 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560e17df1ca0 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f684c050080 con 0x560e18082ea0 2023-11-28T04:07:53.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 10 client.8721 fetch_fsmap finished waiting for FSMap version 58 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 10 client.8721 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 20 client.8721 populate_metadata read hostname 'smithi073' 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e18052850 0x560e18052c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 4 client.8721 mds_command: new command op to 8562 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 --> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560e17e28ca0 con 0x560e18052850 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f6859f4c700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e18052850 0x560e18052c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f6859f4c700 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e18052850 0x560e18052c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=565a7f36c5434a13 server_cookie=f1bca23ae78ee027 in_seq=0 out_seq=0 2023-11-28T04:07:53.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.680+0000 7f6833fff700 10 client.8721 ms_handle_connect on v2:172.21.15.73:6835/3981387608 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f6833fff700 1 -- 172.21.15.73:0/576276475 <== mds.0 v2:172.21.15.73:6835/3981387608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x560e17e28ca0 con 0x560e18052850 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f6833fff700 10 client.8721 handle_command_reply: tid=0 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 client.8721 shutdown 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 msgr2=0x560e18052080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 rx=0x7f684c07a140 tx=0x7f684c00e000).stop 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 shutdown_connections 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e18052850 0x560e18052c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e18082ea0 0x560e18052080 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e17df5e40 0x560e18063c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f6833fff700 0 client.8721 ms_handle_reset on v2:172.21.15.73:6835/3981387608 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 --2- 172.21.15.73:0/576276475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18039cb0 0x560e1807d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 >> 172.21.15.73:0/576276475 conn(0x560e17ea34a0 msgr2=0x560e1803ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 shutdown_connections 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 1 -- 172.21.15.73:0/576276475 wait complete. 2023-11-28T04:07:53.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.681+0000 7f686db48b80 20 client.8721 trim_cache size 0 max 16384 2023-11-28T04:07:53.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.697+0000 7f686db48b80 1 Processor -- start 2023-11-28T04:07:53.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.697+0000 7f686db48b80 1 -- start start 2023-11-28T04:07:53.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e1807e640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e1807d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e180651a0 con 0x560e17df5e40 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e180655e0 con 0x560e18039cb0 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e18038830 con 0x560e18082ea0 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f6859f4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f6859f4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38102/0 (socket says 172.21.15.73:38102) 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.698+0000 7f6859f4c700 1 -- 172.21.15.73:0/2216549045 learned_addr learned my addr 172.21.15.73:0/2216549045 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 -- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 msgr2=0x560e1807d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e1807d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 -- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 msgr2=0x560e1807e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e1807e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 -- 172.21.15.73:0/2216549045 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560e1802d340 con 0x560e17df5e40 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f6859f4c700 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x560e18064cc0 tx=0x7f6850014000).ready entity=mon.0 client_cookie=8adf8da98a1f097d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f683f7fe700 1 -- 172.21.15.73:0/2216549045 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68500250d0 con 0x560e17df5e40 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f683f7fe700 1 -- 172.21.15.73:0/2216549045 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6850029020 con 0x560e17df5e40 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f683f7fe700 1 -- 172.21.15.73:0/2216549045 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850025990 con 0x560e17df5e40 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 -- 172.21.15.73:0/2216549045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 msgr2=0x560e1807e150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 rx=0x560e18064cc0 tx=0x7f6850014000).stop 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 -- 172.21.15.73:0/2216549045 shutdown_connections 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e1807d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e1807e150 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 --2- 172.21.15.73:0/2216549045 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e1807e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 -- 172.21.15.73:0/2216549045 >> 172.21.15.73:0/2216549045 conn(0x560e17ea34a0 msgr2=0x560e18097f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 -- 172.21.15.73:0/2216549045 shutdown_connections 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.699+0000 7f686db48b80 1 -- 172.21.15.73:0/2216549045 wait complete. 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.700+0000 7f686db48b80 1 -- start start 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.700+0000 7f686db48b80 1 Processor -- start 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e180e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e180506c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e180641b0 con 0x560e17df5e40 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x560e18051340 con 0x560e18039cb0 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x560e180e45a0 con 0x560e18082ea0 2023-11-28T04:07:53.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 10 client.0 ll_register_callbacks cb 0x560e17ff9460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38110/0 (socket says 172.21.15.73:38110) 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 -- 172.21.15.73:0/3762960404 learned_addr learned my addr 172.21.15.73:0/3762960404 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f685974b700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e180e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f685a74d700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e180506c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 -- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 msgr2=0x560e180506c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e180506c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 -- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 msgr2=0x560e180e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e180e4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 -- 172.21.15.73:0/3762960404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6850015970 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6859f4c700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x560e18064cf0 tx=0x7f6850027030).ready entity=mon.0 client_cookie=1dd158a0bede61c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6833fff700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68500236a0 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560e18061f70 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.701+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f685002a030 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=58) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6850023a10 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 10 client.8727 fetch_fsmap learned FSMap version 58 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=58,monmap=2+}) v3 -- 0x560e17f7f710 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685003d070 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 58) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f6850023b70 con 0x560e17df5e40 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 10 client.8727 fetch_fsmap finished waiting for FSMap version 58 2023-11-28T04:07:53.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 10 client.8727 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 20 client.8727 populate_metadata read hostname 'smithi073' 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e1807e130 0x560e1807e510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 4 client.8727 mds_command: new command op to 8562 tid=0[{"prefix": "status"}] 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 --> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] -- command(tid 0: {"prefix": "status"}) v1 -- 0x560e1802d720 con 0x560e1807e130 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f685974b700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e1807e130 0x560e1807e510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 10 client.8727 ms_handle_connect on v2:172.21.15.73:6835/3981387608 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f685974b700 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e1807e130 0x560e1807e510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=7dce11845089a717 server_cookie=48acba7ae55c29b3 in_seq=0 out_seq=0 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 1 -- 172.21.15.73:0/3762960404 <== mds.0 v2:172.21.15.73:6835/3981387608 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x560e1802d720 con 0x560e1807e130 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f6833fff700 10 client.8727 handle_command_reply: tid=0 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 client.8727 shutdown 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 msgr2=0x560e180de110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.702+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 rx=0x560e18064cf0 tx=0x7f6850027030).stop 2023-11-28T04:07:53.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 shutdown_connections 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:6835/3981387608,v1:172.21.15.73:6837/3981387608] conn(0x560e1807e130 0x560e1807e510 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x560e18082ea0 0x560e180506c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x560e17df5e40 0x560e180de110 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f6833fff700 0 client.8727 ms_handle_reset on v2:172.21.15.73:6835/3981387608 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 --2- 172.21.15.73:0/3762960404 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x560e18039cb0 0x560e180e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 >> 172.21.15.73:0/3762960404 conn(0x560e17ea34a0 msgr2=0x560e18033670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 shutdown_connections 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 1 -- 172.21.15.73:0/3762960404 wait complete. 2023-11-28T04:07:53.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f686db48b80 20 client.8727 trim_cache size 0 max 16384 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 8562, 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 9.2433577140000001, 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 58, 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 165, 2023-11-28T04:07:53.713 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 165, 2023-11-28T04:07:53.714 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 13.766609299000001 2023-11-28T04:07:53.714 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 msgr2=0x7f68380615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 0x7f68380615e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f68540ca1c0 tx=0x7f68500099d0).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 msgr2=0x7f685414ae70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 rx=0x7f68540c2360 tx=0x7f684c017000).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 shutdown_connections 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f683805f120 0x7f68380615e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6854158320 0x7f685414ae70 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6854098d10 0x7f6854154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 --2- 172.21.15.73:0/2544058818 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68540c6190 0x7f685414a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 >> 172.21.15.73:0/2544058818 conn(0x7f68540cb310 msgr2=0x7f68541532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 shutdown_connections 2023-11-28T04:07:53.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.703+0000 7f685c1b0700 1 -- 172.21.15.73:0/2544058818 wait complete. 2023-11-28T04:07:53.717 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T04:07:53.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.816+0000 7f2110d13700 1 Processor -- start 2023-11-28T04:07:53.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.816+0000 7f2110d13700 1 -- start start 2023-11-28T04:07:53.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2110d13700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 0x7f210c0c7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2110d13700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f210c0c7c50 0x7f210c0c8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2110d13700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f210c0c85f0 con 0x7f210c0c7700 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2110d13700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f210c0c6d80 con 0x7f210c0c7c50 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2110d13700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c154ae0 con 0x7f210c154700 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f210c0c7c50 0x7f210c0c8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2109d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f210c0c7700 0x7f210c0aa1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38434/0 (socket says 172.21.15.73:38434) 2023-11-28T04:07:53.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f2109d9b700 1 -- 172.21.15.73:0/2306251432 learned_addr learned my addr 172.21.15.73:0/2306251432 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210a59c700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 0x7f210c0c7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 -- 172.21.15.73:0/2306251432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 msgr2=0x7f210c0c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 0x7f210c0c7210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 -- 172.21.15.73:0/2306251432 >> v1:172.21.15.73:6789/0 conn(0x7f210c0c7700 legacy=0x7f210c0aa1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210959a700 1 -- 172.21.15.73:0/2306251432 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4125711993 0 0) 0x7f2100001110 con 0x7f210c0c7700 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 -- 172.21.15.73:0/2306251432 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f210c064c30 con 0x7f210c0c7c50 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210a59c700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 0x7f210c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:07:53.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.817+0000 7f210ad9d700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7c50 0x7f210c0c8100 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f20fc009e60 tx=0x7f20fc009780).ready entity=mon.0 client_cookie=99f2858f1a757370 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f210959a700 1 -- 172.21.15.73:0/2306251432 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc00e070 con 0x7f210c0c7c50 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f210959a700 1 -- 172.21.15.73:0/2306251432 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20fc00bc30 con 0x7f210c0c7c50 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f210959a700 1 -- 172.21.15.73:0/2306251432 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20fc012410 con 0x7f210c0c7c50 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 -- 172.21.15.73:0/2306251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7c50 msgr2=0x7f210c0c8100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7c50 0x7f210c0c8100 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 rx=0x7f20fc009e60 tx=0x7f20fc009780).stop 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 -- 172.21.15.73:0/2306251432 shutdown_connections 2023-11-28T04:07:53.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c154700 0x7f210c0c7210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 --2- 172.21.15.73:0/2306251432 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7c50 0x7f210c0c8100 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:53.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 -- 172.21.15.73:0/2306251432 >> 172.21.15.73:0/2306251432 conn(0x7f210c0948b0 msgr2=0x7f210c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:53.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 -- 172.21.15.73:0/2306251432 shutdown_connections 2023-11-28T04:07:53.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.818+0000 7f2110d13700 1 -- 172.21.15.73:0/2306251432 wait complete. 2023-11-28T04:07:53.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.819+0000 7f2110d13700 1 Processor -- start 2023-11-28T04:07:53.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.819+0000 7f2110d13700 1 -- start start 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c0c7c50 0x7f210c14ee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c154700 0x7f210c14f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c14f860 con 0x7f210c0c7700 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0cab60 con 0x7f210c154700 2023-11-28T04:07:53.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f2110d13700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f210c0c9f50 con 0x7f210c0c7c50 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38138/0 (socket says 172.21.15.73:38138) 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 -- 172.21.15.73:0/1729765511 learned_addr learned my addr 172.21.15.73:0/1729765511 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 -- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c0c7c50 msgr2=0x7f210c14ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c0c7c50 0x7f210c14ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 -- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c154700 msgr2=0x7f210c14f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c154700 0x7f210c14f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:53.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 -- 172.21.15.73:0/1729765511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20fc009a20 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.820+0000 7f210a59c700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f210400d860 tx=0x7f2104008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104011590 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21040116f0 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2104011a70 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f210c0ca230 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f210c0ca7f0 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.821+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f210c0caed0 con 0x7f210c0c7700 2023-11-28T04:07:53.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.822+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f210401c460 con 0x7f210c0c7700 2023-11-28T04:07:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.822+0000 7f20fa7fc700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 0x7f20ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.822+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f210407a110 con 0x7f210c0c7700 2023-11-28T04:07:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.822+0000 7f2109d9b700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 0x7f20ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.823+0000 7f2109d9b700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 0x7f20ec061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2100000c00 tx=0x7f2100007a30).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:53.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:53.826+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f210404b380 con 0x7f210c0c7700 2023-11-28T04:07:54.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.032+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f210c0475e0 con 0x7f210c0c7700 2023-11-28T04:07:54.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.033+0000 7f20fa7fc700 1 -- 172.21.15.73:0/1729765511 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v58) v1 ==== 53+0+73 (secure 0 0 0) 0x7f210404e9a0 con 0x7f210c0c7700 2023-11-28T04:07:54.034 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 msgr2=0x7f20ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 0x7f20ec061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f2100000c00 tx=0x7f2100007a30).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 msgr2=0x7f210c14cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 rx=0x7f210400d860 tx=0x7f2104008040).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 shutdown_connections 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f20ec05f150 0x7f20ec061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f210c154700 0x7f210c14f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f210c0c7700 0x7f210c14cfc0 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 --2- 172.21.15.73:0/1729765511 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f210c0c7c50 0x7f210c14ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:54.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 >> 172.21.15.73:0/1729765511 conn(0x7f210c0948b0 msgr2=0x7f210c0c6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:54.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 shutdown_connections 2023-11-28T04:07:54.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.034+0000 7f2110d13700 1 -- 172.21.15.73:0/1729765511 wait complete. 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T04:07:54.047 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T04:07:54.048 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:07:54.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.048 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T04:07:54.063 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T04:07:54.063 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T04:07:54.064 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 38006sec preferred_lft 38006sec 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.065 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:07:54.066 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.067 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T04:07:54.127 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T04:07:54.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.128 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T04:07:54.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T04:07:54.185 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T04:07:54.283 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.283 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:07:54.297 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T04:07:54.298 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T04:07:54.298 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:07:54.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.298 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T04:07:54.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.362 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T04:07:54.417 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T04:07:54.417 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:07:54.496 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:07:54.497 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:07:54.522 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:07:54.522 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:07:54.525 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T04:07:54.525 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:07:54.525 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T04:07:54.590 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T04:07:54.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.743+0000 7f97bd91c700 1 Processor -- start 2023-11-28T04:07:54.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.743+0000 7f97bd91c700 1 -- start start 2023-11-28T04:07:54.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.743+0000 7f97bd91c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f97b813bce0 0x7f97b80c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.743+0000 7f97bd91c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97b80a8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bd91c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97b80c7480 con 0x7f97b80c5380 2023-11-28T04:07:54.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bd91c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97b813c0c0 con 0x7f97b813bce0 2023-11-28T04:07:54.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bd91c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80c76d0 con 0x7f97b80c6050 2023-11-28T04:07:54.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97affff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f97b80c5380 0x7f97b80c5940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:38444/0 (socket says 172.21.15.73:38444) 2023-11-28T04:07:54.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97affff700 1 -- 172.21.15.73:0/2998936771 learned_addr learned my addr 172.21.15.73:0/2998936771 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:07:54.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bc91a700 1 --2- 172.21.15.73:0/2998936771 >> v2:172.21.15.150:3300/0 conn(0x7f97b813bce0 0x7f97b80c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bd11b700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97b80a8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3434343474 0 0) 0x7f97b80c7480 con 0x7f97b80c5380 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f97a00033f0 con 0x7f97b80c5380 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4261699294 0 0) 0x7f97a00033f0 con 0x7f97b80c5380 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 msgr2=0x7f97b80a8ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97b80a8ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 >> v2:172.21.15.150:3300/0 conn(0x7f97b813bce0 msgr2=0x7f97b80c3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 --2- 172.21.15.73:0/2998936771 >> v2:172.21.15.150:3300/0 conn(0x7f97b813bce0 0x7f97b80c3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b80c79b0 con 0x7f97b80c5380 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f97a8001470 con 0x7f97b80c5380 2023-11-28T04:07:54.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 >> v1:172.21.15.73:6789/0 conn(0x7f97b80c5380 legacy=0x7f97b80c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a00038e0 0x7f97a0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97a0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a0004b80 0x7f97a0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b813c0c0 con 0x7f97a00038e0 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80c76d0 con 0x7f97a0004b80 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97a000b8e0 con 0x7f97b80c6050 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.744+0000 7f97bc91a700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a00038e0 0x7f97a0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.751 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd11b700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a0004b80 0x7f97a0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97affff700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97a0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 -- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 msgr2=0x7f97a0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97a0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 -- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a0004b80 msgr2=0x7f97a0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a0004b80 0x7f97a0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 -- 172.21.15.73:0/2998936771 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97a00033f0 con 0x7f97a00038e0 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97affff700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b80c6050 0x7f97a0004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bc91a700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a00038e0 0x7f97a0003d00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f97b400e210 tx=0x7f97b400b040).ready entity=mon.0 client_cookie=b1466536c42729ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4009ac0 con 0x7f97a00038e0 2023-11-28T04:07:54.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b400f040 con 0x7f97a00038e0 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97af7fe700 1 -- 172.21.15.73:0/2998936771 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b400ecd0 con 0x7f97a00038e0 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 -- 172.21.15.73:0/2998936771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a00038e0 msgr2=0x7f97a0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 --2- 172.21.15.73:0/2998936771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97a00038e0 0x7f97a0003d00 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 rx=0x7f97b400e210 tx=0x7f97b400b040).stop 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd11b700 1 -- 172.21.15.73:0/2998936771 reap_dead start 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 -- 172.21.15.73:0/2998936771 shutdown_connections 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 -- 172.21.15.73:0/2998936771 >> 172.21.15.73:0/2998936771 conn(0x7f97b8062700 msgr2=0x7f97b8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 -- 172.21.15.73:0/2998936771 shutdown_connections 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.745+0000 7f97bd91c700 1 -- 172.21.15.73:0/2998936771 wait complete. 2023-11-28T04:07:54.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.747+0000 7f97bd91c700 1 Processor -- start 2023-11-28T04:07:54.754 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.747+0000 7f97bd91c700 1 -- start start 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a00038e0 0x7f97b8138c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b8155f80 0x7f97b81563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b81578c0 con 0x7f97b80c8ac0 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97b8139120 con 0x7f97a00038e0 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97b80c9430 con 0x7f97b8155f80 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38146/0 (socket says 172.21.15.73:38146) 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 -- 172.21.15.73:0/4144794840 learned_addr learned my addr 172.21.15.73:0/4144794840 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:54.757 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bc91a700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a00038e0 0x7f97b8138c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd11b700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b8155f80 0x7f97b81563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 -- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b8155f80 msgr2=0x7f97b81563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b8155f80 0x7f97b81563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 -- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a00038e0 msgr2=0x7f97b8138c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a00038e0 0x7f97b8138c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 -- 172.21.15.73:0/4144794840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97b8156cb0 con 0x7f97b80c8ac0 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97affff700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f97a800d830 tx=0x7f97a8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:54.758 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a8007170 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a80072d0 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97b8156f90 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a80048c0 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.748+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97b8157550 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.749+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97b8157a70 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.750+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f97a8004c30 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.750+0000 7f97acff9700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 0x7f97a4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.750+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f97a807a320 con 0x7f97b80c8ac0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.750+0000 7f97bc91a700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 0x7f97a4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.750+0000 7f97bc91a700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 0x7f97a4061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f97b400afd0 tx=0x7f97b401a000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:54.759 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.754+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f97b8157a70 con 0x7f97b80c8ac0 2023-11-28T04:07:54.947 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.946+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f97b8157a70 con 0x7f97a405f150 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.948+0000 7f97acff9700 1 -- 172.21.15.73:0/4144794840 <== mgr.8379 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f97b8157a70 con 0x7f97a405f150 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:07:54.949 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 msgr2=0x7f97a4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 0x7f97a4061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f97b400afd0 tx=0x7f97b401a000).stop 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 msgr2=0x7f97b80c8f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 rx=0x7f97a800d830 tx=0x7f97a8009040).stop 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 shutdown_connections 2023-11-28T04:07:54.951 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f97a405f150 0x7f97a4061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f97b8155f80 0x7f97b81563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f97a00038e0 0x7f97b8138c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 --2- 172.21.15.73:0/4144794840 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f97b80c8ac0 0x7f97b80c8f40 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 >> 172.21.15.73:0/4144794840 conn(0x7f97b8062700 msgr2=0x7f97b80ab000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 shutdown_connections 2023-11-28T04:07:54.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:54.950+0000 7f97bd91c700 1 -- 172.21.15.73:0/4144794840 wait complete. 2023-11-28T04:07:54.963 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.082+0000 7f01034af700 1 Processor -- start 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.082+0000 7f01034af700 1 -- start start 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f01034af700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00fc0c6b30 0x7f00fc155ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f01034af700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f01034af700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00fc155fd0 con 0x7f00fc0bdc30 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f01034af700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00fc14bff0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f01034af700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc155650 con 0x7f00fc1524b0 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f0100a4a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f010124b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00fc0c6b30 0x7f00fc155ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f0100a4a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:38090/0 (socket says 172.21.15.73:38090) 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f010124b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f00fc0c6b30 0x7f00fc155ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:55316/0 (socket says 172.21.15.73:55316) 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f0100a4a700 1 -- 172.21.15.73:0/1800347719 learned_addr learned my addr 172.21.15.73:0/1800347719 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.083+0000 7f0101a4c700 1 --1- 172.21.15.73:0/1800347719 >> v1:172.21.15.150:6789/0 conn(0x7f00fc0bdc30 0x7f00fc0c2ce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:44054/0 (socket says 172.21.15.73:44054) 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f010124b700 1 -- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 msgr2=0x7f00fc0bd740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f010124b700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f010124b700 1 -- 172.21.15.73:0/1800347719 >> v1:172.21.15.150:6789/0 conn(0x7f00fc0bdc30 legacy=0x7f00fc0c2ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f010124b700 1 -- 172.21.15.73:0/1800347719 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00fc0be0c0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f00f3fff700 1 -- 172.21.15.73:0/1800347719 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 855545509 0 0) 0x7f00fc155fd0 con 0x7f00fc0bdc30 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f0100a4a700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:55.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f010124b700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0c6b30 0x7f00fc155ae0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f00ec006fd0 tx=0x7f00ec006f40).ready entity=mon.1 client_cookie=36881a6be8dda274 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f00f3fff700 1 -- 172.21.15.73:0/1800347719 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec005880 con 0x7f00fc0c6b30 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f00f3fff700 1 -- 172.21.15.73:0/1800347719 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00ec0059e0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.084+0000 7f00f3fff700 1 -- 172.21.15.73:0/1800347719 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00ec005d60 con 0x7f00fc0c6b30 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 -- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0c6b30 msgr2=0x7f00fc155ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0c6b30 0x7f00fc155ae0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 rx=0x7f00ec006fd0 tx=0x7f00ec006f40).stop 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 -- 172.21.15.73:0/1800347719 shutdown_connections 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0c6b30 0x7f00fc155ae0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 --2- 172.21.15.73:0/1800347719 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc0bd740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 -- 172.21.15.73:0/1800347719 >> 172.21.15.73:0/1800347719 conn(0x7f00fc0c8230 msgr2=0x7f00fc0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 -- 172.21.15.73:0/1800347719 shutdown_connections 2023-11-28T04:07:55.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.085+0000 7f01034af700 1 -- 172.21.15.73:0/1800347719 wait complete. 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.086+0000 7f01034af700 1 Processor -- start 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.086+0000 7f01034af700 1 -- start start 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0bdc30 0x7f00fc1483f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc14a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ab6b0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0ab990 con 0x7f00fc0bdc30 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f01034af700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00fc0abc70 con 0x7f00fc1524b0 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38148/0 (socket says 172.21.15.73:38148) 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 -- 172.21.15.73:0/3698041842 learned_addr learned my addr 172.21.15.73:0/3698041842 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:55.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f010124b700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0bdc30 0x7f00fc1483f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0101a4c700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc14a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 -- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 msgr2=0x7f00fc14a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc14a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 -- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0bdc30 msgr2=0x7f00fc1483f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0bdc30 0x7f00fc1483f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.087+0000 7f0100a4a700 1 -- 172.21.15.73:0/3698041842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00f800b5a0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f0100a4a700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f00f8010de0 tx=0x7f00f800b300).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800ed40 con 0x7f00fc0c6b30 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00f8005880 con 0x7f00fc0c6b30 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ec009250 con 0x7f00fc0c6b30 2023-11-28T04:07:55.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00f800eea0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.088+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00fc0abfd0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.089+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00fc06c860 con 0x7f00fc0c6b30 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.089+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 66) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f00f8017070 con 0x7f00fc0c6b30 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.089+0000 7f00f17fa700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 0x7f00e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.089+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f00f807caa0 con 0x7f00fc0c6b30 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.090+0000 7f010124b700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 0x7f00e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.090+0000 7f010124b700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 0x7f00e0061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f00ec005fd0 tx=0x7f00ec015000).ready entity=mgr.8379 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:55.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.093+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f00f804dd70 con 0x7f00fc0c6b30 2023-11-28T04:07:55.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:55.311+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f00fc063b60 con 0x7f00fc0c6b30 2023-11-28T04:07:56.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.053+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 67) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f00f807a870 con 0x7f00fc0c6b30 2023-11-28T04:07:56.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.054+0000 7f00f17fa700 1 -- 172.21.15.73:0/3698041842 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v67) v1 ==== 92+0+0 (secure 0 0 0) 0x7f00f8077f80 con 0x7f00fc0c6b30 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 msgr2=0x7f00e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 0x7f00e0061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f00ec005fd0 tx=0x7f00ec015000).stop 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 msgr2=0x7f00fc0aac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 rx=0x7f00f8010de0 tx=0x7f00f800b300).stop 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 shutdown_connections 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f00e005f150 0x7f00e0061610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f00fc1524b0 0x7f00fc14a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f00fc0bdc30 0x7f00fc1483f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 --2- 172.21.15.73:0/3698041842 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f00fc0c6b30 0x7f00fc0aac60 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.056+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 >> 172.21.15.73:0/3698041842 conn(0x7f00fc0c8230 msgr2=0x7f00fc062700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:56.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.057+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 shutdown_connections 2023-11-28T04:07:56.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.057+0000 7f01034af700 1 -- 172.21.15.73:0/3698041842 wait complete. 2023-11-28T04:07:56.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T04:07:56.199 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:56.198+0000 7f2ace1ee680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:07:56.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.200+0000 7efd82b03700 1 Processor -- start 2023-11-28T04:07:56.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.200+0000 7efd82b03700 1 -- start start 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd82b03700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c152640 0x7efd7c062c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd82b03700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd7c0be5d0 0x7efd7c0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd82b03700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd7c0c5380 con 0x7efd7c0c4f20 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd82b03700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd7c155e20 con 0x7efd7c0be5d0 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd82b03700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0bed70 con 0x7efd7c152640 2023-11-28T04:07:56.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd7bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd7c0be5d0 0x7efd7c0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd7bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7efd7c0be5d0 0x7efd7c0c4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38176/0 (socket says 172.21.15.73:38176) 2023-11-28T04:07:56.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd7bfff700 1 -- 172.21.15.73:0/4257760595 learned_addr learned my addr 172.21.15.73:0/4257760595 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:56.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.201+0000 7efd8089f700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c152640 0x7efd7c062c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7bfff700 1 -- 172.21.15.73:0/4257760595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c152640 msgr2=0x7efd7c062c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:56.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7bfff700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c152640 0x7efd7c062c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7bfff700 1 -- 172.21.15.73:0/4257760595 >> v1:172.21.15.150:6789/0 conn(0x7efd7c0c4f20 legacy=0x7efd7c0658e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7bfff700 1 -- 172.21.15.73:0/4257760595 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd7c158420 con 0x7efd7c0be5d0 2023-11-28T04:07:56.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7bfff700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c0be5d0 0x7efd7c0c4a30 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7efd68006fd0 tx=0x7efd680090f0).ready entity=mon.0 client_cookie=60925f905137faed server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:56.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7b7fe700 1 -- 172.21.15.73:0/4257760595 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6800b580 con 0x7efd7c0be5d0 2023-11-28T04:07:56.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7b7fe700 1 -- 172.21.15.73:0/4257760595 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800b6e0 con 0x7efd7c0be5d0 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd7b7fe700 1 -- 172.21.15.73:0/4257760595 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6800ba60 con 0x7efd7c0be5d0 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd82b03700 1 -- 172.21.15.73:0/4257760595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c0be5d0 msgr2=0x7efd7c0c4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd82b03700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c0be5d0 0x7efd7c0c4a30 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 rx=0x7efd68006fd0 tx=0x7efd680090f0).stop 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd82b03700 1 -- 172.21.15.73:0/4257760595 shutdown_connections 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd82b03700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c152640 0x7efd7c062c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.202+0000 7efd82b03700 1 --2- 172.21.15.73:0/4257760595 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c0be5d0 0x7efd7c0c4a30 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.203+0000 7efd82b03700 1 -- 172.21.15.73:0/4257760595 >> 172.21.15.73:0/4257760595 conn(0x7efd7c063810 msgr2=0x7efd7c063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:56.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.203+0000 7efd82b03700 1 -- 172.21.15.73:0/4257760595 shutdown_connections 2023-11-28T04:07:56.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.203+0000 7efd82b03700 1 -- 172.21.15.73:0/4257760595 wait complete. 2023-11-28T04:07:56.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.204+0000 7efd82b03700 1 Processor -- start 2023-11-28T04:07:56.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.204+0000 7efd82b03700 1 -- start start 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.204+0000 7efd82b03700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 0x7efd7c0ad3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.204+0000 7efd82b03700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 0x7efd7c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd82b03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd82b03700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c15b460 con 0x7efd7c152640 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd82b03700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c14c3b0 con 0x7efd7c0c4f20 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd82b03700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7c0a59d0 con 0x7efd7c0be5d0 2023-11-28T04:07:56.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd810a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 0x7efd7c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd8089f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 0x7efd7c0ad3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd810a0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38182/0 (socket says 172.21.15.73:38182) 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd810a0700 1 -- 172.21.15.73:0/4134022262 learned_addr learned my addr 172.21.15.73:0/4134022262 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 msgr2=0x7efd7c0ad3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 0x7efd7c0ad3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 msgr2=0x7efd7c15af70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 -- 172.21.15.73:0/4134022262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd68009370 con 0x7efd7c0c4f20 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd8089f700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 0x7efd7c0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd810a0700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd7bfff700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 0x7efd7c15aa80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7efd7c0c4950 tx=0x7efd68017000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:56.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6800b850 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd6800b9b0 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd7c0ad8b0 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.205+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd6801c670 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.206+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd7c0a5380 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 67) v1 ==== 75463+0+0 (secure 0 0 0) 0x7efd6801ca20 con 0x7efd7c0c4f20 2023-11-28T04:07:56.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd7c0a5b80 con 0x7efd7c0c4f20 2023-11-28T04:07:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd78ff9700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 0x7efd60061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd8089f700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 msgr2=0x7efd60061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7efd6807bcb0 con 0x7efd7c0c4f20 2023-11-28T04:07:56.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.207+0000 7efd8089f700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 0x7efd60061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:56.217 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:56.210+0000 7fae0a9fa680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:07:56.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.212+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7efd7c0a5b80 con 0x7efd7c0c4f20 2023-11-28T04:07:56.263 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:56.262+0000 7f2ace1ee680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:07:56.295 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:56.294+0000 7fae0a9fa680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:07:56.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.407+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7efd7c0a5b80 con 0x7efd7c0c4f20 2023-11-28T04:07:56.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.408+0000 7efd8089f700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 msgr2=0x7efd60061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:56.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.408+0000 7efd8089f700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 0x7efd60061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:56.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.416+0000 7efd78ff9700 1 -- 172.21.15.73:0/4134022262 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7efd7c0a5b80 con 0x7efd7c0c4f20 2023-11-28T04:07:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 msgr2=0x7efd60061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:07:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 0x7efd60061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 msgr2=0x7efd7c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 0x7efd7c15aa80 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 rx=0x7efd7c0c4950 tx=0x7efd68017000).stop 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 shutdown_connections 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7efd6005f1a0 0x7efd60061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7efd7c152640 0x7efd7c15af70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7efd7c0be5d0 0x7efd7c0ad3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 --2- 172.21.15.73:0/4134022262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7efd7c0c4f20 0x7efd7c15aa80 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 >> 172.21.15.73:0/4134022262 conn(0x7efd7c063810 msgr2=0x7efd7c0c10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 shutdown_connections 2023-11-28T04:07:56.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.418+0000 7efd82b03700 1 -- 172.21.15.73:0/4134022262 wait complete. 2023-11-28T04:07:56.432 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:07:56.432 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T04:07:56.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1h 2023-11-28T04:07:56.541 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:56.541+0000 7f2ace1ee680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:07:56.585 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:56.584+0000 7fae0a9fa680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:07:56.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.609+0000 7fe0a3502700 1 Processor -- start 2023-11-28T04:07:56.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 -- start start 2023-11-28T04:07:56.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe09c0c71b0 0x7fe09c0bc660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe09c0c4d80 0x7fe09c156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c156610 0x7fe09c14fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c0c7590 con 0x7fe09c0c4d80 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe09c063ae0 con 0x7fe09c0c71b0 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.610+0000 7fe0a3502700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c156a80 con 0x7fe09c156610 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe09c0c4d80 0x7fe09c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe09c0c4d80 0x7fe09c156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38194/0 (socket says 172.21.15.73:38194) 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 -- 172.21.15.73:0/109472209 learned_addr learned my addr 172.21.15.73:0/109472209 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a129e700 1 --2- 172.21.15.73:0/109472209 >> v2:172.21.15.150:3300/0 conn(0x7fe09c0c71b0 0x7fe09c0bc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a1a9f700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c156610 0x7fe09c14fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 -- 172.21.15.73:0/109472209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c156610 msgr2=0x7fe09c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c156610 0x7fe09c14fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 -- 172.21.15.73:0/109472209 >> v2:172.21.15.150:3300/0 conn(0x7fe09c0c71b0 msgr2=0x7fe09c0bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 --2- 172.21.15.73:0/109472209 >> v2:172.21.15.150:3300/0 conn(0x7fe09c0c71b0 0x7fe09c0bc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 -- 172.21.15.73:0/109472209 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe09c0631b0 con 0x7fe09c0c4d80 2023-11-28T04:07:56.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe0a0a9d700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c0c4d80 0x7fe09c156120 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fe09c06a370 tx=0x7fe08c0090f0).ready entity=mon.0 client_cookie=cf80c202de830334 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:56.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe093fff700 1 -- 172.21.15.73:0/109472209 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00f510 con 0x7fe09c0c4d80 2023-11-28T04:07:56.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe093fff700 1 -- 172.21.15.73:0/109472209 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08c00f670 con 0x7fe09c0c4d80 2023-11-28T04:07:56.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.611+0000 7fe093fff700 1 -- 172.21.15.73:0/109472209 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08c00fa10 con 0x7fe09c0c4d80 2023-11-28T04:07:56.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 -- 172.21.15.73:0/109472209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c0c4d80 msgr2=0x7fe09c156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c0c4d80 0x7fe09c156120 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 rx=0x7fe09c06a370 tx=0x7fe08c0090f0).stop 2023-11-28T04:07:56.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 -- 172.21.15.73:0/109472209 shutdown_connections 2023-11-28T04:07:56.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c156610 0x7fe09c14fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 --2- 172.21.15.73:0/109472209 >> v2:172.21.15.150:3300/0 conn(0x7fe09c0c71b0 0x7fe09c0bc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 --2- 172.21.15.73:0/109472209 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c0c4d80 0x7fe09c156120 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 -- 172.21.15.73:0/109472209 >> 172.21.15.73:0/109472209 conn(0x7fe09c0c8170 msgr2=0x7fe09c0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 -- 172.21.15.73:0/109472209 shutdown_connections 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.612+0000 7fe0a3502700 1 -- 172.21.15.73:0/109472209 wait complete. 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.613+0000 7fe0a3502700 1 Processor -- start 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.613+0000 7fe0a3502700 1 -- start start 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c0c4d80 0x7fe09c150db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c0c71b0 0x7fe09c0ac5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c162570 con 0x7fe09c156610 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c1512a0 con 0x7fe09c0c4d80 2023-11-28T04:07:56.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a3502700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe09c0ace60 con 0x7fe09c0c71b0 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a1a9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a1a9f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38210/0 (socket says 172.21.15.73:38210) 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a1a9f700 1 -- 172.21.15.73:0/2221926685 learned_addr learned my addr 172.21.15.73:0/2221926685 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c0c4d80 0x7fe09c150db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a0a9d700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c0c71b0 0x7fe09c0ac5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a1a9f700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c0c71b0 msgr2=0x7fe09c0ac5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.614+0000 7fe0a1a9f700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c0c71b0 0x7fe09c0ac5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a1a9f700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c0c4d80 msgr2=0x7fe09c150db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a1a9f700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c0c4d80 0x7fe09c150db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a1a9f700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c009370 con 0x7fe09c156610 2023-11-28T04:07:56.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a1a9f700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fe094009ec0 tx=0x7fe09400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:07:56.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400cb70 con 0x7fe09c156610 2023-11-28T04:07:56.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe09c0ad140 con 0x7fe09c156610 2023-11-28T04:07:56.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe09c0ad700 con 0x7fe09c156610 2023-11-28T04:07:56.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe09400f040 con 0x7fe09c156610 2023-11-28T04:07:56.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.615+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe09400cee0 con 0x7fe09c156610 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.616+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09c1626e0 con 0x7fe09c156610 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.616+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 67) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fe09401d670 con 0x7fe09c156610 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.617+0000 7fe0917fa700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.617+0000 7fe0a129e700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.617+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(165..165 src has 1..165) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fe09407aa30 con 0x7fe09c156610 2023-11-28T04:07:56.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.617+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:07:56.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.624+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe09404bda0 con 0x7fe09c156610 2023-11-28T04:07:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.818+0000 7fe0a129e700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:56.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.818+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:07:56.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:56.821+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe09c06c860 con 0x7fe08005f1a0 2023-11-28T04:07:57.159 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:57.158+0000 7f2ace1ee680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:07:57.214 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:57.213+0000 7fae0a9fa680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:07:57.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:57.219+0000 7fe0a129e700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:57.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:57.219+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T04:07:57.232 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:57.231+0000 7f2ace1ee680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:07:57.286 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:57.285+0000 7fae0a9fa680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:07:57.469 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:57.421+0000 7f2ace1ee680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:07:57.474 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:57.471+0000 7fae0a9fa680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:07:57.845 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:57.844+0000 7f2ace1ee680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:07:57.888 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:57.888+0000 7fae0a9fa680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:07:57.949 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:57.948+0000 7f2ace1ee680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:57.991 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:57.990+0000 7fae0a9fa680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:58.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:58.020+0000 7fe0a129e700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:58.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:58.020+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T04:07:58.103 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:58.102+0000 7f2ace1ee680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:58.144 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:58.143+0000 7fae0a9fa680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:58.178 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:58.177+0000 7f2ace1ee680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:07:58.217 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:58.216+0000 7fae0a9fa680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:07:58.245 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:58.244+0000 7f2ace1ee680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:07:58.284 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:58.283+0000 7fae0a9fa680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:07:58.504 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:58.503+0000 7f2ace1ee680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:07:58.540 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:58.539+0000 7fae0a9fa680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:07:58.688 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:58.687+0000 7f2ace1ee680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:58.721 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:58.720+0000 7fae0a9fa680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:07:59.342 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.341+0000 7f2ace1ee680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:07:59.368 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.367+0000 7fae0a9fa680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:07:59.411 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.410+0000 7f2ace1ee680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:07:59.437 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.436+0000 7fae0a9fa680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:07:59.487 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.486+0000 7f2ace1ee680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:07:59.513 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.512+0000 7fae0a9fa680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:07:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:59.621+0000 7fe0a129e700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:07:59.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:07:59.621+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T04:07:59.636 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.635+0000 7f2ace1ee680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:07:59.661 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.660+0000 7fae0a9fa680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:07:59.708 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.707+0000 7f2ace1ee680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:07:59.732 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.731+0000 7fae0a9fa680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:07:59.825 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.824+0000 7f2ace1ee680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:07:59.848 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.846+0000 7fae0a9fa680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:07:59.934 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:07:59.933+0000 7f2ace1ee680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:07:59.956 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:07:59.955+0000 7fae0a9fa680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:00.228 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:00.228+0000 7f2ace1ee680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:00.245 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.244+0000 7fae0a9fa680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:00.300 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:00.299+0000 7f2ace1ee680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:00.316 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.315+0000 7fae0a9fa680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:00.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:00.335+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 68) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fe0940788a0 con 0x7fe09c156610 2023-11-28T04:08:00.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:00.336+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe080061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:00.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:00.336+0000 7fe0917fa700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe080061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:00.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad7f180700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad7f180700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad7f180700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.399 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad7f180700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad7f180700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad82186700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad82186700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad82186700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad82186700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:00.400 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:00.385+0000 7fad82186700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:01.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.337+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 69) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fe094078580 con 0x7fe09c156610 2023-11-28T04:08:01.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.338+0000 7fe0917fa700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe0800615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.338+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fe09c06c860 con 0x7fe08005f1a0 2023-11-28T04:08:01.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.339+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe0800615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.340+0000 7fe0a129e700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe0800615e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fe098009690 tx=0x7fe098009060).ready entity=mgr.8757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.353+0000 7fe0917fa700 1 -- 172.21.15.73:0/2221926685 <== mgr.8757 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe09c06c860 con 0x7fe08005f1a0 2023-11-28T04:08:01.353 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 msgr2=0x7fe0800615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe0800615e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 rx=0x7fe098009690 tx=0x7fe098009060).stop 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 msgr2=0x7fe09c0c5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 rx=0x7fe094009ec0 tx=0x7fe09400b040).stop 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 shutdown_connections 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fe08005f1a0 0x7fe0800615e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe09c156610 0x7fe09c0c5510 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe09c0c4d80 0x7fe09c150db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 --2- 172.21.15.73:0/2221926685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe09c0c71b0 0x7fe09c0ac5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 >> 172.21.15.73:0/2221926685 conn(0x7fe09c0c8170 msgr2=0x7fe09c153d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:01.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 shutdown_connections 2023-11-28T04:08:01.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.355+0000 7fe0a3502700 1 -- 172.21.15.73:0/2221926685 wait complete. 2023-11-28T04:08:01.368 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T04:08:01.368 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-28T04:08:01.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.483+0000 7fccd333e700 1 Processor -- start 2023-11-28T04:08:01.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 -- start start 2023-11-28T04:08:01.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 0x7fcccc067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcccc13af40 0x7fcccc0bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcccc0bddb0 con 0x7fcccc13ab60 2023-11-28T04:08:01.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcccc067c70 con 0x7fcccc13af40 2023-11-28T04:08:01.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.484+0000 7fccd333e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0c3510 con 0x7fcccc1556f0 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcccc13af40 0x7fcccc0bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcccc13af40 0x7fcccc0bd8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38300/0 (socket says 172.21.15.73:38300) 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 -- 172.21.15.73:0/306684352 learned_addr learned my addr 172.21.15.73:0/306684352 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd08d9700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 0x7fcccc067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 -- 172.21.15.73:0/306684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 msgr2=0x7fcccc067780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 0x7fcccc067780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 -- 172.21.15.73:0/306684352 >> v1:172.21.15.73:6789/0 conn(0x7fcccc13ab60 legacy=0x7fcccc154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 -- 172.21.15.73:0/306684352 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcccc0bdfe0 con 0x7fcccc13af40 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd08d9700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 0x7fcccc067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccd18db700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc13af40 0x7fcccc0bd8c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fccc4009f20 tx=0x7fccc4009840).ready entity=mon.0 client_cookie=471fd1235738acab server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccc3fff700 1 -- 172.21.15.73:0/306684352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400e070 con 0x7fcccc13af40 2023-11-28T04:08:01.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccc3fff700 1 -- 172.21.15.73:0/306684352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc400ba80 con 0x7fcccc13af40 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.485+0000 7fccc3fff700 1 -- 172.21.15.73:0/306684352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc400be00 con 0x7fcccc13af40 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 -- 172.21.15.73:0/306684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc13af40 msgr2=0x7fcccc0bd8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc13af40 0x7fcccc0bd8c0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 rx=0x7fccc4009f20 tx=0x7fccc4009840).stop 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 -- 172.21.15.73:0/306684352 shutdown_connections 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc13af40 0x7fcccc0bd8c0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 --2- 172.21.15.73:0/306684352 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc1556f0 0x7fcccc067780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 -- 172.21.15.73:0/306684352 >> 172.21.15.73:0/306684352 conn(0x7fcccc063810 msgr2=0x7fcccc063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 -- 172.21.15.73:0/306684352 shutdown_connections 2023-11-28T04:08:01.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.486+0000 7fccd333e700 1 -- 172.21.15.73:0/306684352 wait complete. 2023-11-28T04:08:01.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.487+0000 7fccd333e700 1 Processor -- start 2023-11-28T04:08:01.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.487+0000 7fccd333e700 1 -- start start 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 0x7fcccc0a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc13af40 0x7fcccc0bf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0adfe0 con 0x7fcccc1556f0 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0bf6c0 con 0x7fcccc13ab60 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd333e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcccc0bf9a0 con 0x7fcccc13af40 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38302/0 (socket says 172.21.15.73:38302) 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 -- 172.21.15.73:0/2104971517 learned_addr learned my addr 172.21.15.73:0/2104971517 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd08d9700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc13af40 0x7fcccc0bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd10da700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 0x7fcccc0a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 -- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc13af40 msgr2=0x7fcccc0bf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc13af40 0x7fcccc0bf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 -- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 msgr2=0x7fcccc0a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 0x7fcccc0a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 -- 172.21.15.73:0/2104971517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccc4009ae0 con 0x7fcccc1556f0 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd10da700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 0x7fcccc0a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.488+0000 7fccd18db700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fccc400ba50 tx=0x7fccc4009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc40032f0 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccc40092e0 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcccc0bfc80 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccc401e6a0 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcccc0c0240 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.489+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcccc06c860 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.490+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 69) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fccc4003040 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.490+0000 7fccc17fa700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 0x7fccb0061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.490+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fccc407b6b0 con 0x7fcccc1556f0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.490+0000 7fccd10da700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 0x7fccb0061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.490+0000 7fccd10da700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 0x7fccb0061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fcccc154850 tx=0x7fccc8009040).ready entity=mgr.8757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.494+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fccc404caa0 con 0x7fcccc1556f0 2023-11-28T04:08:01.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.682+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fcccc0be2c0 con 0x7fccb005f080 2023-11-28T04:08:01.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.686+0000 7fccc17fa700 1 -- 172.21.15.73:0/2104971517 <== mgr.8757 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcccc0be2c0 con 0x7fccb005f080 2023-11-28T04:08:01.687 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path snap_test_dir1 2023-11-28T04:08:01.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 msgr2=0x7fccb0061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 0x7fccb0061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fcccc154850 tx=0x7fccc8009040).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 msgr2=0x7fcccc0adaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 rx=0x7fccc400ba50 tx=0x7fccc4009840).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 shutdown_connections 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fccb005f080 0x7fccb0061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcccc1556f0 0x7fcccc0adaf0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcccc13ab60 0x7fcccc0a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 --2- 172.21.15.73:0/2104971517 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcccc13af40 0x7fcccc0bf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.687+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 >> 172.21.15.73:0/2104971517 conn(0x7fcccc063810 msgr2=0x7fcccc0c4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.688+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 shutdown_connections 2023-11-28T04:08:01.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.688+0000 7fccd333e700 1 -- 172.21.15.73:0/2104971517 wait complete. 2023-11-28T04:08:01.700 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-28T04:08:01.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs snap_test_dir1 2023-11-28T04:08:01.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.816+0000 7f3ddeb24700 1 Processor -- start 2023-11-28T04:08:01.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.816+0000 7f3ddeb24700 1 -- start start 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd806a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3dd8154900 0x7f3dd806a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 0x7f3dd80bd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd80bd890 con 0x7f3dd8154900 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd8067840 con 0x7f3dd80bcef0 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddeb24700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd8066c30 con 0x7f3dd8154520 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3dd8154900 0x7f3dd806a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3dd8154900 0x7f3dd806a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38308/0 (socket says 172.21.15.73:38308) 2023-11-28T04:08:01.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 -- 172.21.15.73:0/2853657253 learned_addr learned my addr 172.21.15.73:0/2853657253 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddd0c1700 1 --2- 172.21.15.73:0/2853657253 >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 0x7f3dd80bd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddc8c0700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd806a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 -- 172.21.15.73:0/2853657253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 msgr2=0x7f3dd806a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd806a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 -- 172.21.15.73:0/2853657253 >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 msgr2=0x7f3dd80bd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 --2- 172.21.15.73:0/2853657253 >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 0x7f3dd80bd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 -- 172.21.15.73:0/2853657253 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd8066f10 con 0x7f3dd8154900 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddd0c1700 1 --2- 172.21.15.73:0/2853657253 >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 0x7f3dd80bd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3ddc8c0700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd806a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd7fff700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd806a8a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f3dd8069610 tx=0x7f3dc40090f0).ready entity=mon.0 client_cookie=7b76c7dc789d6220 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.817+0000 7f3dd77fe700 1 -- 172.21.15.73:0/2853657253 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400b600 con 0x7f3dd8154900 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3dd77fe700 1 -- 172.21.15.73:0/2853657253 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc400b760 con 0x7f3dd8154900 2023-11-28T04:08:01.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3dd77fe700 1 -- 172.21.15.73:0/2853657253 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc400bb00 con 0x7f3dd8154900 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2853657253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 msgr2=0x7f3dd806a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd806a8a0 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 rx=0x7f3dd8069610 tx=0x7f3dc40090f0).stop 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2853657253 shutdown_connections 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 --2- 172.21.15.73:0/2853657253 >> v2:172.21.15.150:3300/0 conn(0x7f3dd80bcef0 0x7f3dd80bd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd806a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 --2- 172.21.15.73:0/2853657253 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd806a8a0 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2853657253 >> 172.21.15.73:0/2853657253 conn(0x7f3dd8063810 msgr2=0x7f3dd8063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2853657253 shutdown_connections 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.818+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2853657253 wait complete. 2023-11-28T04:08:01.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.819+0000 7f3ddeb24700 1 Processor -- start 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.819+0000 7f3ddeb24700 1 -- start start 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3dd80bcef0 0x7f3dd80c5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd80c0900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd80bf030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c0df0 con 0x7f3dd8154900 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c0470 con 0x7f3dd80bcef0 2023-11-28T04:08:01.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddeb24700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80bf860 con 0x7f3dd8154520 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddd0c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd80bf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddd0c1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd80bf030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38316/0 (socket says 172.21.15.73:38316) 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddd0c1700 1 -- 172.21.15.73:0/2142272338 learned_addr learned my addr 172.21.15.73:0/2142272338 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3dd7fff700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd80c0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.820+0000 7f3ddc8c0700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3dd80bcef0 0x7f3dd80c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 -- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 msgr2=0x7f3dd80c0900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3dd8154520 0x7f3dd80c0900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 -- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3dd80bcef0 msgr2=0x7f3dd80c5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3dd80bcef0 0x7f3dd80c5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 -- 172.21.15.73:0/2142272338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc4009370 con 0x7f3dd8154900 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddd0c1700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3dd8154900 0x7f3dd80bf030 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 rx=0x7f3dd000cef0 tx=0x7f3dd0007860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd000add0 con 0x7f3dd8154900 2023-11-28T04:08:01.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd0016400 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd0016780 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2142272338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd80bfb40 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.821+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2142272338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd80c0100 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.822+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2142272338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd80c0600 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.822+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 69) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f3dd00203f0 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.822+0000 7f3dd4ff9700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3dbc05f080 0x7f3dbc061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.823+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f3dd007aa60 con 0x7f3dd8154900 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.823+0000 7f3ddc8c0700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3dbc05f080 0x7f3dbc061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:01.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.823+0000 7f3ddc8c0700 1 --2- 172.21.15.73:0/2142272338 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f3dbc05f080 0x7f3dbc061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f3dd80adee0 tx=0x7f3dcc008040).ready entity=mgr.8757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:01.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:01.827+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3dd004bdf0 con 0x7f3dd8154900 2023-11-28T04:08:02.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.016+0000 7f3ddeb24700 1 -- 172.21.15.73:0/2142272338 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3dd806c860 con 0x7f3dbc05f080 2023-11-28T04:08:02.019 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:02.018+0000 7fad8a196700 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-11-28T04:08:02.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.018+0000 7f3dd4ff9700 1 -- 172.21.15.73:0/2142272338 <== mgr.8757 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f3dd806c860 con 0x7f3dbc05f080 2023-11-28T04:08:02.020 INFO:teuthology.orchestra.run.smithi073.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-11-28T04:08:02.023 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-11-28T04:08:02.024 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:08:02.024 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T04:08:02.040 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T04:08:02.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.195+0000 7fa9def3a700 1 Processor -- start 2023-11-28T04:08:02.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.195+0000 7fa9def3a700 1 -- start start 2023-11-28T04:08:02.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9def3a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9d80c6190 0x7fa9d80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9def3a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa9d8158320 0x7fa9d81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9def3a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9d8158cc0 con 0x7fa9d8098d10 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9def3a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d80c72d0 con 0x7fa9d80c6190 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9def3a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9d8064570 con 0x7fa9d8158320 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9d7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9d80c6190 0x7fa9d80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9d7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa9d80c6190 0x7fa9d80c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38324/0 (socket says 172.21.15.73:38324) 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9d7fff700 1 -- 172.21.15.73:0/1176993368 learned_addr learned my addr 172.21.15.73:0/1176993368 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.196+0000 7fa9dd4d7700 1 --2- 172.21.15.73:0/1176993368 >> v2:172.21.15.150:3300/0 conn(0x7fa9d8158320 0x7fa9d81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d7fff700 1 -- 172.21.15.73:0/1176993368 >> v2:172.21.15.150:3300/0 conn(0x7fa9d8158320 msgr2=0x7fa9d81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d7fff700 1 --2- 172.21.15.73:0/1176993368 >> v2:172.21.15.150:3300/0 conn(0x7fa9d8158320 0x7fa9d81587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:02.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d7fff700 1 -- 172.21.15.73:0/1176993368 >> v1:172.21.15.150:6789/0 conn(0x7fa9d8098d10 legacy=0x7fa9d8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d7fff700 1 -- 172.21.15.73:0/1176993368 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9d8154bc0 con 0x7fa9d80c6190 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d7fff700 1 --2- 172.21.15.73:0/1176993368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d80c7760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fa9d80657a0 tx=0x7fa9cc0090f0).ready entity=mon.0 client_cookie=c82616e058ba9a8a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d77fe700 1 -- 172.21.15.73:0/1176993368 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00b520 con 0x7fa9d80c6190 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d77fe700 1 -- 172.21.15.73:0/1176993368 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00b680 con 0x7fa9d80c6190 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9d77fe700 1 -- 172.21.15.73:0/1176993368 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc00ba00 con 0x7fa9d80c6190 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 -- 172.21.15.73:0/1176993368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 msgr2=0x7fa9d80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 --2- 172.21.15.73:0/1176993368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d80c7760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 rx=0x7fa9d80657a0 tx=0x7fa9cc0090f0).stop 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 -- 172.21.15.73:0/1176993368 shutdown_connections 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 --2- 172.21.15.73:0/1176993368 >> v2:172.21.15.150:3300/0 conn(0x7fa9d8158320 0x7fa9d81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 --2- 172.21.15.73:0/1176993368 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d80c7760 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:02.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 -- 172.21.15.73:0/1176993368 >> 172.21.15.73:0/1176993368 conn(0x7fa9d80cb310 msgr2=0x7fa9d80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:02.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 -- 172.21.15.73:0/1176993368 shutdown_connections 2023-11-28T04:08:02.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.197+0000 7fa9def3a700 1 -- 172.21.15.73:0/1176993368 wait complete. 2023-11-28T04:08:02.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.198+0000 7fa9def3a700 1 Processor -- start 2023-11-28T04:08:02.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 -- start start 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d8098d10 0x7fa9d81578a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d8158320 0x7fa9d80c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8154d50 con 0x7fa9d80c6190 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8151910 con 0x7fa9d8098d10 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9def3a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9d8168500 con 0x7fa9d8158320 2023-11-28T04:08:02.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9d7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38326/0 (socket says 172.21.15.73:38326) 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.199+0000 7fa9d7fff700 1 -- 172.21.15.73:0/3206201873 learned_addr learned my addr 172.21.15.73:0/3206201873 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9dccd6700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d8098d10 0x7fa9d81578a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9dd4d7700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d8158320 0x7fa9d80c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 -- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d8158320 msgr2=0x7fa9d80c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d8158320 0x7fa9d80c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 -- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d8098d10 msgr2=0x7fa9d81578a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d8098d10 0x7fa9d81578a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 -- 172.21.15.73:0/3206201873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9cc009370 con 0x7fa9d80c6190 2023-11-28T04:08:02.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d7fff700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7fa9cc011000 tx=0x7fa9cc017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc000bc0 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9cc00b7f0 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9d81687e0 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9cc000d20 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.200+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9d8168da0 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.201+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9d8154ec0 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.201+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 69) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fa9cc01c880 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.202+0000 7fa9d4ff9700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 0x7fa9c4061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.202+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fa9cc07bc10 con 0x7fa9d80c6190 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.202+0000 7fa9dccd6700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 0x7fa9c4061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:02.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.202+0000 7fa9dccd6700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 0x7fa9c4061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa9c8004290 tx=0x7fa9c8000f40).ready entity=mgr.8757 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:02.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.206+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa9cc04d000 con 0x7fa9d80c6190 2023-11-28T04:08:02.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:02.418+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa9d806c860 con 0x7fa9d80c6190 2023-11-28T04:08:03.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.345+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fa9cc0aaaf0 con 0x7fa9d80c6190 2023-11-28T04:08:03.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.346+0000 7fa9d4ff9700 1 -- 172.21.15.73:0/3206201873 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v70) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa9cc042940 con 0x7fa9d80c6190 2023-11-28T04:08:03.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 msgr2=0x7fa9c4061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 0x7fa9c4061540 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa9c8004290 tx=0x7fa9c8000f40).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 msgr2=0x7fa9d8151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 rx=0x7fa9cc011000 tx=0x7fa9cc017000).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 shutdown_connections 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa9c405f080 0x7fa9c4061540 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa9d8158320 0x7fa9d80c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa9d8098d10 0x7fa9d81578a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 --2- 172.21.15.73:0/3206201873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa9d80c6190 0x7fa9d8151420 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 >> 172.21.15.73:0/3206201873 conn(0x7fa9d80cb310 msgr2=0x7fa9d80a96a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 shutdown_connections 2023-11-28T04:08:03.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.348+0000 7fa9def3a700 1 -- 172.21.15.73:0/3206201873 wait complete. 2023-11-28T04:08:03.362 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:08:03.363 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:08:03.406 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:08:03.444 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:08:03.445 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T04:08:03.445 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:08:03.445 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T04:08:03.487 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:08:03.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:03.488 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T04:08:03.491 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:03.490+0000 7f0069a99680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:08:03.493 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:03.492+0000 7f38de016680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:08:03.505 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:03.556 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:03.555+0000 7f38de016680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:08:03.557 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:03.556+0000 7f0069a99680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:08:03.662 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.661+0000 7f9a1d31a700 1 Processor -- start 2023-11-28T04:08:03.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.661+0000 7f9a1d31a700 1 -- start start 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a1d31a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a18069900 0x7f9a18069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a1d31a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9a181544f0 0x7f9a18154940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a1d31a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a1806a270 con 0x7f9a18150890 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a1d31a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a180c28c0 con 0x7f9a18069900 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a1d31a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a181552e0 con 0x7f9a181544f0 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.662+0000 7f9a167fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a18069900 0x7f9a18069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a18069900 0x7f9a18069d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37622/0 (socket says 172.21.15.73:37622) 2023-11-28T04:08:03.665 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 -- 172.21.15.73:0/4251078028 learned_addr learned my addr 172.21.15.73:0/4251078028 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a177fe700 1 --2- 172.21.15.73:0/4251078028 >> v2:172.21.15.150:3300/0 conn(0x7f9a181544f0 0x7f9a18154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 -- 172.21.15.73:0/4251078028 >> v2:172.21.15.150:3300/0 conn(0x7f9a181544f0 msgr2=0x7f9a18154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 --2- 172.21.15.73:0/4251078028 >> v2:172.21.15.150:3300/0 conn(0x7f9a181544f0 0x7f9a18154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 -- 172.21.15.73:0/4251078028 >> v1:172.21.15.150:6789/0 conn(0x7f9a18150890 legacy=0x7f9a180bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 -- 172.21.15.73:0/4251078028 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a180bc540 con 0x7f9a18069900 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a167fc700 1 --2- 172.21.15.73:0/4251078028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18069d80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f9a1806acd0 tx=0x7f9a080090f0).ready entity=mon.0 client_cookie=44ef9ec68b84c630 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a15ffb700 1 -- 172.21.15.73:0/4251078028 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800b600 con 0x7f9a18069900 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a15ffb700 1 -- 172.21.15.73:0/4251078028 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0800b760 con 0x7f9a18069900 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.663+0000 7f9a15ffb700 1 -- 172.21.15.73:0/4251078028 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0800bae0 con 0x7f9a18069900 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 -- 172.21.15.73:0/4251078028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 msgr2=0x7f9a18069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/4251078028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18069d80 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 rx=0x7f9a1806acd0 tx=0x7f9a080090f0).stop 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 -- 172.21.15.73:0/4251078028 shutdown_connections 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/4251078028 >> v2:172.21.15.150:3300/0 conn(0x7f9a181544f0 0x7f9a18154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/4251078028 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18069d80 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 -- 172.21.15.73:0/4251078028 >> 172.21.15.73:0/4251078028 conn(0x7f9a180631d0 msgr2=0x7f9a180c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 -- 172.21.15.73:0/4251078028 shutdown_connections 2023-11-28T04:08:03.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.664+0000 7f9a1d31a700 1 -- 172.21.15.73:0/4251078028 wait complete. 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.665+0000 7f9a1d31a700 1 Processor -- start 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.665+0000 7f9a1d31a700 1 -- start start 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a18150890 0x7f9a180942c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a181544f0 0x7f9a1814f410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180ab610 con 0x7f9a18069900 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180ab000 con 0x7f9a18150890 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a1d31a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a180947b0 con 0x7f9a181544f0 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37624/0 (socket says 172.21.15.73:37624) 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 -- 172.21.15.73:0/1024041235 learned_addr learned my addr 172.21.15.73:0/1024041235 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a167fc700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a18150890 0x7f9a180942c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a177fe700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a181544f0 0x7f9a1814f410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a181544f0 msgr2=0x7f9a1814f410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a181544f0 0x7f9a1814f410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a18150890 msgr2=0x7f9a180942c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a18150890 0x7f9a180942c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 -- 172.21.15.73:0/1024041235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a08009370 con 0x7f9a18069900 2023-11-28T04:08:03.671 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.666+0000 7f9a16ffd700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f9a0000c4c0 tx=0x7f9a00007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.667+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0000ce70 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.667+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a18094a80 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.667+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a00001e90 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.667+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a00007250 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.667+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a180ab2a0 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.668+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a18154d60 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.668+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f9a000066a0 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.669+0000 7f9a0effd700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 0x7f99f80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.669+0000 7f9a167fc700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 msgr2=0x7f99f80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.669+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9a00079d60 con 0x7f9a18069900 2023-11-28T04:08:03.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.669+0000 7f9a167fc700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 0x7f99f80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:03.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.673+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a0004ac80 con 0x7f9a18069900 2023-11-28T04:08:03.830 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:03.829+0000 7f38de016680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:08:03.843 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:03.842+0000 7f0069a99680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:08:03.870 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.870+0000 7f9a167fc700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 msgr2=0x7f99f80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:03.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.870+0000 7f9a167fc700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 0x7f99f80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:03.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.884+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a1806a270 con 0x7f9a18069900 2023-11-28T04:08:03.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.885+0000 7f9a0effd700 1 -- 172.21.15.73:0/1024041235 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f9a1806a270 con 0x7f9a18069900 2023-11-28T04:08:03.886 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:03.886 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:08:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 msgr2=0x7f99f80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:03.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 0x7f99f80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 msgr2=0x7f9a18091f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 rx=0x7f9a0000c4c0 tx=0x7f9a00007e00).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 shutdown_connections 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f99f805f300 0x7f99f80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a181544f0 0x7f9a1814f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a18069900 0x7f9a18091f10 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 --2- 172.21.15.73:0/1024041235 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a18150890 0x7f9a180942c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 >> 172.21.15.73:0/1024041235 conn(0x7f9a180631d0 msgr2=0x7f9a180bf350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 shutdown_connections 2023-11-28T04:08:03.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:03.887+0000 7f9a1d31a700 1 -- 172.21.15.73:0/1024041235 wait complete. 2023-11-28T04:08:03.890 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:08:03.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:04.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 Processor -- start 2023-11-28T04:08:04.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 -- start start 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9ab01544d0 0x7f9ab0155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 0x7f9ab0095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab00683d0 con 0x7f9ab00673c0 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ab0067850 con 0x7f9ab01544d0 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.058+0000 7f9ab62e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab0067b30 con 0x7f9ab0066f40 2023-11-28T04:08:04.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9ab4883700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f9ab00673c0 0x7f9ab0063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41058/0 (socket says 172.21.15.73:41058) 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9ab4883700 1 -- 172.21.15.73:0/2381363964 learned_addr learned my addr 172.21.15.73:0/2381363964 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaf7fe700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 0x7f9ab0095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 --2- 172.21.15.73:0/2381363964 >> v2:172.21.15.150:3300/0 conn(0x7f9ab01544d0 0x7f9ab0155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaeffd700 1 -- 172.21.15.73:0/2381363964 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1047323919 0 0) 0x7f9ab00683d0 con 0x7f9ab00673c0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaeffd700 1 -- 172.21.15.73:0/2381363964 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a9c0033f0 con 0x7f9ab00673c0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 -- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 msgr2=0x7f9ab0095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 0x7f9ab0095e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 -- 172.21.15.73:0/2381363964 >> v1:172.21.15.73:6789/0 conn(0x7f9ab00673c0 legacy=0x7f9ab0063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 -- 172.21.15.73:0/2381363964 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab0067e10 con 0x7f9ab01544d0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaf7fe700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 0x7f9ab0095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaffff700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab01544d0 0x7f9ab0155750 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9a98006fd0 tx=0x7f9a98006f40).ready entity=mon.1 client_cookie=2ec9c7ddc79b1377 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaeffd700 1 -- 172.21.15.73:0/2381363964 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98005880 con 0x7f9ab01544d0 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaeffd700 1 -- 172.21.15.73:0/2381363964 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a980059e0 con 0x7f9ab01544d0 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.059+0000 7f9aaeffd700 1 -- 172.21.15.73:0/2381363964 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98005d60 con 0x7f9ab01544d0 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 -- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab01544d0 msgr2=0x7f9ab0155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab01544d0 0x7f9ab0155750 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 rx=0x7f9a98006fd0 tx=0x7f9a98006f40).stop 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 -- 172.21.15.73:0/2381363964 shutdown_connections 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab01544d0 0x7f9ab0155750 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/2381363964 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab0066f40 0x7f9ab0095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 -- 172.21.15.73:0/2381363964 >> 172.21.15.73:0/2381363964 conn(0x7f9ab00631d0 msgr2=0x7f9ab00c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 -- 172.21.15.73:0/2381363964 shutdown_connections 2023-11-28T04:08:04.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.060+0000 7f9ab62e6700 1 -- 172.21.15.73:0/2381363964 wait complete. 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.061+0000 7f9ab62e6700 1 Processor -- start 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.061+0000 7f9ab62e6700 1 -- start start 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab01544d0 0x7f9ab00aea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab00aef50 0x7f9ab00ac0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab0156620 con 0x7f9ab0066f40 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab0155a10 con 0x7f9ab00aef50 2023-11-28T04:08:04.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab62e6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab0155cf0 con 0x7f9ab01544d0 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37634/0 (socket says 172.21.15.73:37634) 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 -- 172.21.15.73:0/4230978655 learned_addr learned my addr 172.21.15.73:0/4230978655 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaf7fe700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab01544d0 0x7f9ab00aea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9ab4883700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab00aef50 0x7f9ab00ac0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 -- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab01544d0 msgr2=0x7f9ab00aea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab01544d0 0x7f9ab00aea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 -- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab00aef50 msgr2=0x7f9ab00ac0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab00aef50 0x7f9ab00ac0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.062+0000 7f9aaffff700 1 -- 172.21.15.73:0/4230978655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aa00092a0 con 0x7f9ab0066f40 2023-11-28T04:08:04.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9aaffff700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f9a98005fd0 tx=0x7f9a98015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a98006bd0 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a9800d6b0 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a98009250 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a9800da70 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.063+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab01562b0 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.064+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab006c860 con 0x7f9ab0066f40 2023-11-28T04:08:04.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.065+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f9a980113f0 con 0x7f9ab0066f40 2023-11-28T04:08:04.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.065+0000 7f9aa7fff700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 0x7f9a900617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.066+0000 7f9aaf7fe700 1 -- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 msgr2=0x7f9a900617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:04.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.066+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9a9807bd00 con 0x7f9ab0066f40 2023-11-28T04:08:04.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.066+0000 7f9aaf7fe700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 0x7f9a900617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:04.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.069+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9a9804cc40 con 0x7f9ab0066f40 2023-11-28T04:08:04.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.240+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ab00680f0 con 0x7f9ab0066f40 2023-11-28T04:08:04.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.241+0000 7f9aa7fff700 1 -- 172.21.15.73:0/4230978655 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+17186 (secure 0 0 0) 0x7f9a98023070 con 0x7f9ab0066f40 2023-11-28T04:08:04.243 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:04.243 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":166,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:00.330191+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/1584723932":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:0/3403509088":"2023-11-29T04:08:00.330152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 msgr2=0x7f9a900617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 0x7f9a900617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 msgr2=0x7f9ab0156ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 rx=0x7f9a98005fd0 tx=0x7f9a98015000).stop 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 shutdown_connections 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f9a9005f300 0x7f9a900617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9ab00aef50 0x7f9ab00ac0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9ab0066f40 0x7f9ab0156ab0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 --2- 172.21.15.73:0/4230978655 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9ab01544d0 0x7f9ab00aea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 >> 172.21.15.73:0/4230978655 conn(0x7f9ab00631d0 msgr2=0x7f9ab00c7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 shutdown_connections 2023-11-28T04:08:04.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.244+0000 7f9ab62e6700 1 -- 172.21.15.73:0/4230978655 wait complete. 2023-11-28T04:08:04.257 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '154', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 19, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T04:07:43.015630+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '164', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 20, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T04:07:43.427278+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '165', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T04:08:04.258 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:08:04.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.378+0000 7fee27458700 1 Processor -- start 2023-11-28T04:08:04.379 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.378+0000 7fee27458700 1 -- start start 2023-11-28T04:08:04.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee27458700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee27458700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 0x7fee20063330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee27458700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee20151f90 con 0x7fee200653d0 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee27458700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee20062790 con 0x7fee20065850 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee27458700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee20065e90 con 0x7fee200bcc10 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26c57700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 0x7fee20063330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:57490/0 (socket says 172.21.15.73:57490) 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 -- 172.21.15.73:0/1429657614 learned_addr learned my addr 172.21.15.73:0/1429657614 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 -- 172.21.15.73:0/1429657614 >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 msgr2=0x7fee20063330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 --2- 172.21.15.73:0/1429657614 >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 0x7fee20063330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 -- 172.21.15.73:0/1429657614 >> v1:172.21.15.150:6789/0 conn(0x7fee200653d0 legacy=0x7fee20062c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 -- 172.21.15.73:0/1429657614 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20150e50 con 0x7fee200bcc10 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26c57700 1 --2- 172.21.15.73:0/1429657614 >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 0x7fee20063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.379+0000 7fee26456700 1 --2- 172.21.15.73:0/1429657614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fee1c006fd0 tx=0x7fee1c0090f0).ready entity=mon.2 client_cookie=1dff6b23dfa8c4d4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee25454700 1 -- 172.21.15.73:0/1429657614 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00b580 con 0x7fee200bcc10 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee25454700 1 -- 172.21.15.73:0/1429657614 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee1c00b6e0 con 0x7fee200bcc10 2023-11-28T04:08:04.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee25454700 1 -- 172.21.15.73:0/1429657614 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1c00ba80 con 0x7fee200bcc10 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 -- 172.21.15.73:0/1429657614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 msgr2=0x7fee200bcff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 --2- 172.21.15.73:0/1429657614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 rx=0x7fee1c006fd0 tx=0x7fee1c0090f0).stop 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 -- 172.21.15.73:0/1429657614 shutdown_connections 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 --2- 172.21.15.73:0/1429657614 >> v2:172.21.15.150:3300/0 conn(0x7fee20065850 0x7fee20063330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 --2- 172.21.15.73:0/1429657614 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee200bcff0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 -- 172.21.15.73:0/1429657614 >> 172.21.15.73:0/1429657614 conn(0x7fee200c4010 msgr2=0x7fee200c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 -- 172.21.15.73:0/1429657614 shutdown_connections 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.380+0000 7fee27458700 1 -- 172.21.15.73:0/1429657614 wait complete. 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.381+0000 7fee27458700 1 Processor -- start 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.381+0000 7fee27458700 1 -- start start 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 0x7fee200c1320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee2014af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200bf4d0 con 0x7fee20065850 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee200bff50 con 0x7fee200653d0 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee27458700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee200abaa0 con 0x7fee200bcc10 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee25c55700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee25c55700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37640/0 (socket says 172.21.15.73:37640) 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee25c55700 1 -- 172.21.15.73:0/3219076328 learned_addr learned my addr 172.21.15.73:0/3219076328 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee26456700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 0x7fee200c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.382+0000 7fee26c57700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee2014af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 msgr2=0x7fee2014af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee2014af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 msgr2=0x7fee200c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 0x7fee200c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 -- 172.21.15.73:0/3219076328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee18009a20 con 0x7fee20065850 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee26456700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 0x7fee200c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:04.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee25c55700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7fee1000c4c0 tx=0x7fee10007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee1000ce60 con 0x7fee20065850 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee10007020 con 0x7fee20065850 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee1c009370 con 0x7fee20065850 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee100073c0 con 0x7fee20065850 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.383+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee200bf730 con 0x7fee20065850 2023-11-28T04:08:04.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.384+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee20066970 con 0x7fee20065850 2023-11-28T04:08:04.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.385+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fee100068b0 con 0x7fee20065850 2023-11-28T04:08:04.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.385+0000 7fee167fc700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 0x7fee040617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.385+0000 7fee26456700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 msgr2=0x7fee040617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:04.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.385+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fee10079e60 con 0x7fee20065850 2023-11-28T04:08:04.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.385+0000 7fee26456700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 0x7fee040617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:04.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.389+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fee100a2af0 con 0x7fee20065850 2023-11-28T04:08:04.445 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:04.443+0000 7f38de016680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:08:04.495 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:04.487+0000 7f0069a99680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:08:04.519 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:04.518+0000 7f38de016680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:08:04.561 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:04.560+0000 7f0069a99680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:08:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.586+0000 7fee26456700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 msgr2=0x7fee040617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:04.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.586+0000 7fee26456700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 0x7fee040617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:04.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.593+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fee201470f0 con 0x7fee20065850 2023-11-28T04:08:04.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.593+0000 7fee167fc700 1 -- 172.21.15.73:0/3219076328 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v166) v1 ==== 110+0+10 (secure 0 0 0) 0x7fee10020030 con 0x7fee20065850 2023-11-28T04:08:04.594 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 msgr2=0x7fee040617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 0x7fee040617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 msgr2=0x7fee200befe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 rx=0x7fee1000c4c0 tx=0x7fee10007dd0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 shutdown_connections 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fee0405f300 0x7fee040617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fee200bcc10 0x7fee2014af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fee200653d0 0x7fee200c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 --2- 172.21.15.73:0/3219076328 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fee20065850 0x7fee200befe0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 >> 172.21.15.73:0/3219076328 conn(0x7fee200c4010 msgr2=0x7fee20151480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 shutdown_connections 2023-11-28T04:08:04.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.595+0000 7fee27458700 1 -- 172.21.15.73:0/3219076328 wait complete. 2023-11-28T04:08:04.609 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:08:04.705 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:04.704+0000 7f38de016680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:08:04.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.727+0000 7f79f09dc700 1 Processor -- start 2023-11-28T04:08:04.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 -- start start 2023-11-28T04:08:04.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79ec063b40 con 0x7f79ec1510d0 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79ec0c3ed0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.728+0000 7f79f09dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0c2ce0 con 0x7f79ec1514b0 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79eb59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:57504/0 (socket says 172.21.15.73:57504) 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 -- 172.21.15.73:0/3539402532 learned_addr learned my addr 172.21.15.73:0/3539402532 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79eb59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:52024/0 (socket says 172.21.15.73:52024) 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 -- 172.21.15.73:0/3539402532 >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 msgr2=0x7f79ec0bd3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 --2- 172.21.15.73:0/3539402532 >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 -- 172.21.15.73:0/3539402532 >> v1:172.21.15.150:6789/0 conn(0x7f79ec1510d0 legacy=0x7f79ec0c37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 -- 172.21.15.73:0/3539402532 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79ec0c2fc0 con 0x7f79ec1514b0 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79eb59e700 1 --2- 172.21.15.73:0/3539402532 >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79ea59c700 1 --2- 172.21.15.73:0/3539402532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f79dc006fd0 tx=0x7f79dc0090f0).ready entity=mon.2 client_cookie=7aa3ab29b3e7be2f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79e9d9b700 1 -- 172.21.15.73:0/3539402532 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00f6b0 con 0x7f79ec1514b0 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79e9d9b700 1 -- 172.21.15.73:0/3539402532 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79dc00f810 con 0x7f79ec1514b0 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.729+0000 7f79e9d9b700 1 -- 172.21.15.73:0/3539402532 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79dc00fb90 con 0x7f79ec1514b0 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 -- 172.21.15.73:0/3539402532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 msgr2=0x7f79ec0a6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 --2- 172.21.15.73:0/3539402532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 rx=0x7f79dc006fd0 tx=0x7f79dc0090f0).stop 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 -- 172.21.15.73:0/3539402532 shutdown_connections 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 --2- 172.21.15.73:0/3539402532 >> v2:172.21.15.150:3300/0 conn(0x7f79ec0bcea0 0x7f79ec0bd3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 --2- 172.21.15.73:0/3539402532 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec1514b0 0x7f79ec0a6b30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 -- 172.21.15.73:0/3539402532 >> 172.21.15.73:0/3539402532 conn(0x7f79ec0c4010 msgr2=0x7f79ec0c4420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 -- 172.21.15.73:0/3539402532 shutdown_connections 2023-11-28T04:08:04.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.730+0000 7f79f09dc700 1 -- 172.21.15.73:0/3539402532 wait complete. 2023-11-28T04:08:04.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.731+0000 7f79f09dc700 1 Processor -- start 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.731+0000 7f79f09dc700 1 -- start start 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 0x7f79ec1463a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79ec1510d0 0x7f79ec160420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 0x7f79ec0ac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0be740 con 0x7f79ec1514b0 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0ad3b0 con 0x7f79ec1510d0 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79f09dc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79ec0acb10 con 0x7f79ec0bcea0 2023-11-28T04:08:04.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79eb59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 0x7f79ec0ac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 0x7f79ec1463a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79eb59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 0x7f79ec0ac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37650/0 (socket says 172.21.15.73:37650) 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79eb59e700 1 -- 172.21.15.73:0/2176953962 learned_addr learned my addr 172.21.15.73:0/2176953962 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ea59c700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79ec1510d0 0x7f79ec160420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 -- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79ec1510d0 msgr2=0x7f79ec160420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79ec1510d0 0x7f79ec160420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 -- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 msgr2=0x7f79ec0ac1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 0x7f79ec0ac1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.732+0000 7f79ead9d700 1 -- 172.21.15.73:0/2176953962 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79e4009a20 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79ead9d700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 0x7f79ec1463a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f79e000c440 tx=0x7f79e0007d40).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007020 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79e0007180 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79e0007520 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79dc009370 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.733+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79ec0ad0d0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.734+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79ec06c7c0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.735+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f79e00068b0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.735+0000 7f79d2ffd700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 0x7f79d80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:04.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.735+0000 7f79ea59c700 1 -- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 msgr2=0x7f79d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:04.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.735+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f79e00796a0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.735+0000 7f79ea59c700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 0x7f79d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:04.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.739+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f79e0047af0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.758 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:04.757+0000 7f0069a99680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:08:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.911+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f79ec0c32a0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.912+0000 7f79d2ffd700 1 -- 172.21.15.73:0/2176953962 <== mon.2 v2:172.21.15.150:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v166) v1 ==== 92+0+10 (secure 0 0 0) 0x7f79e004a6c0 con 0x7f79ec0bcea0 2023-11-28T04:08:04.913 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:08:04.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 msgr2=0x7f79d80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 0x7f79d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 msgr2=0x7f79ec1463a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 0x7f79ec1463a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 rx=0x7f79e000c440 tx=0x7f79e0007d40).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 shutdown_connections 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f79d805f300 0x7f79d80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79ec1514b0 0x7f79ec0ac1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79ec0bcea0 0x7f79ec1463a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 --2- 172.21.15.73:0/2176953962 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f79ec1510d0 0x7f79ec160420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:04.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 >> 172.21.15.73:0/2176953962 conn(0x7f79ec0c4010 msgr2=0x7f79ec150280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:04.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 shutdown_connections 2023-11-28T04:08:04.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:04.914+0000 7f79f09dc700 1 -- 172.21.15.73:0/2176953962 wait complete. 2023-11-28T04:08:04.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T04:08:05.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.049+0000 7f51ea026700 1 Processor -- start 2023-11-28T04:08:05.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.049+0000 7f51ea026700 1 -- start start 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51ea026700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e40a8280 0x7f51e40a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51ea026700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e414e600 0x7f51e414eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51ea026700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51e4067770 con 0x7f51e4158160 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51ea026700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51e4159180 con 0x7f51e414e600 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51ea026700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40be8d0 con 0x7f51e40a8280 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51e3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e414e600 0x7f51e414eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51e3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f51e414e600 0x7f51e414eab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37658/0 (socket says 172.21.15.73:37658) 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51e3fff700 1 -- 172.21.15.73:0/2104975661 learned_addr learned my addr 172.21.15.73:0/2104975661 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:05.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.050+0000 7f51e37fe700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e40a8280 0x7f51e40a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e3fff700 1 -- 172.21.15.73:0/2104975661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e40a8280 msgr2=0x7f51e40a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e3fff700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e40a8280 0x7f51e40a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e3fff700 1 -- 172.21.15.73:0/2104975661 >> v1:172.21.15.150:6789/0 conn(0x7f51e4158160 legacy=0x7f51e4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e3fff700 1 -- 172.21.15.73:0/2104975661 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51e4158600 con 0x7f51e414e600 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e3fff700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e414e600 0x7f51e414eab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f51dc009f20 tx=0x7f51dc009840).ready entity=mon.0 client_cookie=576dbbeaece2f335 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e27fc700 1 -- 172.21.15.73:0/2104975661 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00e070 con 0x7f51e414e600 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e27fc700 1 -- 172.21.15.73:0/2104975661 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00b9d0 con 0x7f51e414e600 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51e27fc700 1 -- 172.21.15.73:0/2104975661 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00bd50 con 0x7f51e414e600 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 -- 172.21.15.73:0/2104975661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e414e600 msgr2=0x7f51e414eab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e414e600 0x7f51e414eab0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 rx=0x7f51dc009f20 tx=0x7f51dc009840).stop 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 -- 172.21.15.73:0/2104975661 shutdown_connections 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e414e600 0x7f51e414eab0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 --2- 172.21.15.73:0/2104975661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e40a8280 0x7f51e40a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.051+0000 7f51ea026700 1 -- 172.21.15.73:0/2104975661 >> 172.21.15.73:0/2104975661 conn(0x7f51e4062700 msgr2=0x7f51e4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.052+0000 7f51ea026700 1 -- 172.21.15.73:0/2104975661 shutdown_connections 2023-11-28T04:08:05.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.052+0000 7f51ea026700 1 -- 172.21.15.73:0/2104975661 wait complete. 2023-11-28T04:08:05.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.053+0000 7f51ea026700 1 Processor -- start 2023-11-28T04:08:05.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.053+0000 7f51ea026700 1 -- start start 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 0x7f51e4157a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e4158160 0x7f51e414cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40ab930 con 0x7f51e40a8280 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51e40aaaa0 con 0x7f51e414e600 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51e4157f50 con 0x7f51e4158160 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37672/0 (socket says 172.21.15.73:37672) 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 -- 172.21.15.73:0/447810916 learned_addr learned my addr 172.21.15.73:0/447810916 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e2ffd700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 0x7f51e4157a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e3fff700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e4158160 0x7f51e414cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 -- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e4158160 msgr2=0x7f51e414cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e4158160 0x7f51e414cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 -- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 msgr2=0x7f51e4157a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 0x7f51e4157a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 -- 172.21.15.73:0/447810916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc009ae0 con 0x7f51e40a8280 2023-11-28T04:08:05.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e2ffd700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 0x7f51e4157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51e37fe700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f51d800e980 tx=0x7f51d8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800c6a0 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d800c800 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d800cb80 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.054+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51e40aad20 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.055+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51e40ab360 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f51d801b400 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51e406c860 con 0x7f51e40a8280 2023-11-28T04:08:05.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51cb7fe700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 0x7f51d00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51e2ffd700 1 -- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 msgr2=0x7f51d00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:05.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f51d807a9e0 con 0x7f51e40a8280 2023-11-28T04:08:05.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.056+0000 7f51e2ffd700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 0x7f51d00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:05.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.060+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51d804ba50 con 0x7f51e40a8280 2023-11-28T04:08:05.130 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.129+0000 7f38de016680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:08:05.200 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.199+0000 7f0069a99680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:08:05.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.229+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f51e40ab680 con 0x7f51e40a8280 2023-11-28T04:08:05.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.230+0000 7f51cb7fe700 1 -- 172.21.15.73:0/447810916 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v166) v1 ==== 104+0+11 (secure 0 0 0) 0x7f51d80160a0 con 0x7f51e40a8280 2023-11-28T04:08:05.231 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:08:05.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 msgr2=0x7f51d00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:05.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 0x7f51d00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 msgr2=0x7f51e40aa5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 rx=0x7f51d800e980 tx=0x7f51d8009060).stop 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 shutdown_connections 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51d005f300 0x7f51d00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51e4158160 0x7f51e414cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51e40a8280 0x7f51e40aa5b0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 --2- 172.21.15.73:0/447810916 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51e414e600 0x7f51e4157a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 >> 172.21.15.73:0/447810916 conn(0x7f51e4062700 msgr2=0x7f51e40bf4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 shutdown_connections 2023-11-28T04:08:05.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.232+0000 7f51ea026700 1 -- 172.21.15.73:0/447810916 wait complete. 2023-11-28T04:08:05.235 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.232+0000 7f38de016680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:05.245 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T04:08:05.309 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.309+0000 7f0069a99680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:05.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.363+0000 7f2591e74700 1 Processor -- start 2023-11-28T04:08:05.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.363+0000 7f2591e74700 1 -- start start 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f2591e74700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f2591e74700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c0634a0 con 0x7f258c062770 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f2591e74700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f258c063780 con 0x7f258c14eb00 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f2591e74700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c137d90 con 0x7f258c0bcdd0 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f258c062770 0x7f258c062c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41072/0 (socket says 172.21.15.73:41072) 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258bfff700 1 -- 172.21.15.73:0/3291018157 learned_addr learned my addr 172.21.15.73:0/3291018157 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258affd700 1 --2- 172.21.15.73:0/3291018157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331881726 0 0) 0x7f258c0634a0 con 0x7f258c062770 2023-11-28T04:08:05.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f25740033f0 con 0x7f258c062770 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174997701 0 0) 0x7f258c063780 con 0x7f258c14eb00 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f258c0634a0 con 0x7f258c14eb00 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258affd700 1 -- 172.21.15.73:0/3291018157 >> v1:172.21.15.150:6789/0 conn(0x7f258c14eb00 legacy=0x7f258c0bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258affd700 1 -- 172.21.15.73:0/3291018157 >> v1:172.21.15.73:6789/0 conn(0x7f258c062770 legacy=0x7f258c062c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.364+0000 7f258affd700 1 -- 172.21.15.73:0/3291018157 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f258c063a60 con 0x7f258c0bcdd0 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f258affd700 1 --2- 172.21.15.73:0/3291018157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c14eee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f2580009760 tx=0x7f2580006f40).ready entity=mon.2 client_cookie=4303aba73fe75e9c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580005880 con 0x7f258c0bcdd0 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25800059e0 con 0x7f258c0bcdd0 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f258a7fc700 1 -- 172.21.15.73:0/3291018157 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2580005d60 con 0x7f258c0bcdd0 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 -- 172.21.15.73:0/3291018157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 msgr2=0x7f258c14eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 --2- 172.21.15.73:0/3291018157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c14eee0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 rx=0x7f2580009760 tx=0x7f2580006f40).stop 2023-11-28T04:08:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 -- 172.21.15.73:0/3291018157 shutdown_connections 2023-11-28T04:08:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 --2- 172.21.15.73:0/3291018157 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c14eee0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 -- 172.21.15.73:0/3291018157 >> 172.21.15.73:0/3291018157 conn(0x7f258c0c8230 msgr2=0x7f258c0c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 -- 172.21.15.73:0/3291018157 shutdown_connections 2023-11-28T04:08:05.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.365+0000 7f2591e74700 1 -- 172.21.15.73:0/3291018157 wait complete. 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 Processor -- start 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 -- start start 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c0be840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f258c14eb00 0x7f258c156fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c13aef0 con 0x7f258c062770 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c156b20 con 0x7f258c14eb00 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.367+0000 7f2591e74700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f258c156510 con 0x7f258c0bcdd0 2023-11-28T04:08:05.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37676/0 (socket says 172.21.15.73:37676) 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 -- 172.21.15.73:0/1246511589 learned_addr learned my addr 172.21.15.73:0/1246511589 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258bfff700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f258c14eb00 0x7f258c156fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258affd700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c0be840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 -- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 msgr2=0x7f258c0be840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c0be840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 -- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f258c14eb00 msgr2=0x7f258c156fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f258c14eb00 0x7f258c156fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 -- 172.21.15.73:0/1246511589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2580009250 con 0x7f258c062770 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f258b7fe700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f257c00c4c0 tx=0x7f257c007c80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00ce30 con 0x7f258c062770 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f257c00f470 con 0x7f258c062770 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f258c13a930 con 0x7f258c062770 2023-11-28T04:08:05.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f257c00f810 con 0x7f258c062770 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.368+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c1567b0 con 0x7f258c062770 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.369+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f257c00fbc0 con 0x7f258c062770 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.369+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c06c860 con 0x7f258c062770 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.370+0000 7f25737fe700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 0x7f25780617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.370+0000 7f258affd700 1 -- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 msgr2=0x7f25780617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:05.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.370+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f257c079920 con 0x7f258c062770 2023-11-28T04:08:05.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.370+0000 7f258affd700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 0x7f25780617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:05.378 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.377+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f257c048af0 con 0x7f258c062770 2023-11-28T04:08:05.386 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.385+0000 7f38de016680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:05.461 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.460+0000 7f38de016680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:08:05.487 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.486+0000 7f0069a99680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:05.527 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.526+0000 7f38de016680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:08:05.564 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.563+0000 7f0069a99680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:08:05.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.567+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f258c0c4ac0 con 0x7f258c062770 2023-11-28T04:08:05.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.568+0000 7f25737fe700 1 -- 172.21.15.73:0/1246511589 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v166) v1 ==== 100+0+11 (secure 0 0 0) 0x7f257c04a990 con 0x7f258c062770 2023-11-28T04:08:05.568 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 msgr2=0x7f25780617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 0x7f25780617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 msgr2=0x7f258c1363c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 rx=0x7f257c00c4c0 tx=0x7f257c007c80).stop 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 shutdown_connections 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f257805f300 0x7f25780617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f258c14eb00 0x7f258c156fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f258c062770 0x7f258c1363c0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 --2- 172.21.15.73:0/1246511589 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f258c0bcdd0 0x7f258c0be840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 >> 172.21.15.73:0/1246511589 conn(0x7f258c0c8230 msgr2=0x7f258c0c4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 shutdown_connections 2023-11-28T04:08:05.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.570+0000 7f2591e74700 1 -- 172.21.15.73:0/1246511589 wait complete. 2023-11-28T04:08:05.583 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:05.637 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.636+0000 7f0069a99680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:08:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.698+0000 7f57c629e700 1 Processor -- start 2023-11-28T04:08:05.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.698+0000 7f57c629e700 1 -- start start 2023-11-28T04:08:05.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c629e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0157660 0x7f57c0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c629e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f57c00c1750 0x7f57c00c1b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c629e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57c00c12c0 con 0x7f57c00c0180 2023-11-28T04:08:05.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c629e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00c06b0 con 0x7f57c0157660 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c629e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57c00c0990 con 0x7f57c00c1750 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0157660 0x7f57c0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f57c0157660 0x7f57c0157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37690/0 (socket says 172.21.15.73:37690) 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 -- 172.21.15.73:0/3526989197 learned_addr learned my addr 172.21.15.73:0/3526989197 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c5a9d700 1 --2- 172.21.15.73:0/3526989197 >> v2:172.21.15.150:3300/0 conn(0x7f57c00c1750 0x7f57c00c1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 -- 172.21.15.73:0/3526989197 >> v2:172.21.15.150:3300/0 conn(0x7f57c00c1750 msgr2=0x7f57c00c1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 --2- 172.21.15.73:0/3526989197 >> v2:172.21.15.150:3300/0 conn(0x7f57c00c1750 0x7f57c00c1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 -- 172.21.15.73:0/3526989197 >> v1:172.21.15.150:6789/0 conn(0x7f57c00c0180 legacy=0x7f57c01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 -- 172.21.15.73:0/3526989197 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57c00c0c70 con 0x7f57c0157660 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.699+0000 7f57c529c700 1 --2- 172.21.15.73:0/3526989197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0157660 0x7f57c0157a40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f57c0084650 tx=0x7f57bc0090f0).ready entity=mon.0 client_cookie=ebda7bb47a78e3bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57b7fff700 1 -- 172.21.15.73:0/3526989197 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00b520 con 0x7f57c0157660 2023-11-28T04:08:05.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57b7fff700 1 -- 172.21.15.73:0/3526989197 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57bc00b680 con 0x7f57c0157660 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57b7fff700 1 -- 172.21.15.73:0/3526989197 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57bc00ba00 con 0x7f57c0157660 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 -- 172.21.15.73:0/3526989197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0157660 msgr2=0x7f57c0157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 --2- 172.21.15.73:0/3526989197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0157660 0x7f57c0157a40 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 rx=0x7f57c0084650 tx=0x7f57bc0090f0).stop 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 -- 172.21.15.73:0/3526989197 shutdown_connections 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 --2- 172.21.15.73:0/3526989197 >> v2:172.21.15.150:3300/0 conn(0x7f57c00c1750 0x7f57c00c1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 --2- 172.21.15.73:0/3526989197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c0157660 0x7f57c0157a40 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 -- 172.21.15.73:0/3526989197 >> 172.21.15.73:0/3526989197 conn(0x7f57c00cb000 msgr2=0x7f57c00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 -- 172.21.15.73:0/3526989197 shutdown_connections 2023-11-28T04:08:05.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.700+0000 7f57c629e700 1 -- 172.21.15.73:0/3526989197 wait complete. 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.701+0000 7f57c629e700 1 Processor -- start 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.701+0000 7f57c629e700 1 -- start start 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57c00c0180 0x7f57c00c4be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57c0157660 0x7f57c00c31f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c3b40 con 0x7f57c00c1750 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c3e20 con 0x7f57c00c0180 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c629e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57c00c4100 con 0x7f57c0157660 2023-11-28T04:08:05.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c4a9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c4a9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37700/0 (socket says 172.21.15.73:37700) 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c4a9b700 1 -- 172.21.15.73:0/16783599 learned_addr learned my addr 172.21.15.73:0/16783599 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.702+0000 7f57c529c700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57c00c0180 0x7f57c00c4be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c5a9d700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57c0157660 0x7f57c00c31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57c0157660 msgr2=0x7f57c00c31f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57c0157660 0x7f57c00c31f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57c00c0180 msgr2=0x7f57c00c4be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57c00c0180 0x7f57c00c4be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 -- 172.21.15.73:0/16783599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57bc009370 con 0x7f57c00c1750 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c4a9b700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f57b000c4c0 tx=0x7f57b0007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b000ce70 con 0x7f57c00c1750 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57b0001e90 con 0x7f57c00c1750 2023-11-28T04:08:05.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57c00c43e0 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57b0007250 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.703+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57c0147520 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.704+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57c00c4900 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.704+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f57b00066a0 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.705+0000 7f57b57fa700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 0x7f57a4061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.705+0000 7f57c529c700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 msgr2=0x7f57a4061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.705+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f57b0079d90 con 0x7f57c00c1750 2023-11-28T04:08:05.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.705+0000 7f57c529c700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 0x7f57a4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:05.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.709+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f57c00c4900 con 0x7f57c00c1750 2023-11-28T04:08:05.785 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.785+0000 7f38de016680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:08:05.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.906+0000 7f57c529c700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 msgr2=0x7f57a4061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:05.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.906+0000 7f57c529c700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 0x7f57a4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:05.907 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:05.906+0000 7f0069a99680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:08:05.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.919+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f57c00c4900 con 0x7f57c00c1750 2023-11-28T04:08:05.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.919+0000 7f57b57fa700 1 -- 172.21.15.73:0/16783599 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f57c00c4900 con 0x7f57c00c1750 2023-11-28T04:08:05.920 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:05.920 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:08:05.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 msgr2=0x7f57a4061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:05.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 0x7f57a4061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 msgr2=0x7f57c0147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 rx=0x7f57b000c4c0 tx=0x7f57b0007e00).stop 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 shutdown_connections 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f57a405f2d0 0x7f57a4061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57c0157660 0x7f57c00c31f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57c00c0180 0x7f57c00c4be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 --2- 172.21.15.73:0/16783599 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f57c00c1750 0x7f57c0147030 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 >> 172.21.15.73:0/16783599 conn(0x7f57c00cb000 msgr2=0x7f57c00aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 shutdown_connections 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:05.921+0000 7f57c629e700 1 -- 172.21.15.73:0/16783599 wait complete. 2023-11-28T04:08:05.923 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:08:05.935 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:05.966 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:05.967+0000 7f38de016680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:06.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.054+0000 7f6ceee64700 1 Processor -- start 2023-11-28T04:08:06.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.054+0000 7f6ceee64700 1 -- start start 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ceee64700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6ce81542d0 0x7f6ce80697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ceee64700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ce8069ce0 0x7f6ce8062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ceee64700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce8065760 con 0x7f6ce8062bf0 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ceee64700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce8063080 con 0x7f6ce8069ce0 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ceee64700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce80c3250 con 0x7f6ce81542d0 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ce7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ce8069ce0 0x7f6ce8062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ce7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6ce8069ce0 0x7f6ce8062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37712/0 (socket says 172.21.15.73:37712) 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6ce7fff700 1 -- 172.21.15.73:0/2019448125 learned_addr learned my addr 172.21.15.73:0/2019448125 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.055+0000 7f6cecc00700 1 --2- 172.21.15.73:0/2019448125 >> v2:172.21.15.150:3300/0 conn(0x7f6ce81542d0 0x7f6ce80697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce7fff700 1 -- 172.21.15.73:0/2019448125 >> v2:172.21.15.150:3300/0 conn(0x7f6ce81542d0 msgr2=0x7f6ce80697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce7fff700 1 --2- 172.21.15.73:0/2019448125 >> v2:172.21.15.150:3300/0 conn(0x7f6ce81542d0 0x7f6ce80697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce7fff700 1 -- 172.21.15.73:0/2019448125 >> v1:172.21.15.73:6789/0 conn(0x7f6ce8062bf0 legacy=0x7f6ce8157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce7fff700 1 -- 172.21.15.73:0/2019448125 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce806a340 con 0x7f6ce8069ce0 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce7fff700 1 --2- 172.21.15.73:0/2019448125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce8062700 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f6ce80961f0 tx=0x7f6cd40090f0).ready entity=mon.0 client_cookie=22ee04536504601e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce77fe700 1 -- 172.21.15.73:0/2019448125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400b520 con 0x7f6ce8069ce0 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce77fe700 1 -- 172.21.15.73:0/2019448125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd400b680 con 0x7f6ce8069ce0 2023-11-28T04:08:06.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.056+0000 7f6ce77fe700 1 -- 172.21.15.73:0/2019448125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd400ba00 con 0x7f6ce8069ce0 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 -- 172.21.15.73:0/2019448125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 msgr2=0x7f6ce8062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 --2- 172.21.15.73:0/2019448125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce8062700 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 rx=0x7f6ce80961f0 tx=0x7f6cd40090f0).stop 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 -- 172.21.15.73:0/2019448125 shutdown_connections 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 --2- 172.21.15.73:0/2019448125 >> v2:172.21.15.150:3300/0 conn(0x7f6ce81542d0 0x7f6ce80697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 --2- 172.21.15.73:0/2019448125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce8062700 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 -- 172.21.15.73:0/2019448125 >> 172.21.15.73:0/2019448125 conn(0x7f6ce80c7fd0 msgr2=0x7f6ce80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 -- 172.21.15.73:0/2019448125 shutdown_connections 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.057+0000 7f6ceee64700 1 -- 172.21.15.73:0/2019448125 wait complete. 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.058+0000 7f6ceee64700 1 Processor -- start 2023-11-28T04:08:06.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.058+0000 7f6ceee64700 1 -- start start 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce8062bf0 0x7f6ce814cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ce81542d0 0x7f6ce80abb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80bfd20 con 0x7f6ce8069ce0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80bf110 con 0x7f6ce8062bf0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce80bf3f0 con 0x7f6ce81542d0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37728/0 (socket says 172.21.15.73:37728) 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 -- 172.21.15.73:0/1364876391 learned_addr learned my addr 172.21.15.73:0/1364876391 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6cecc00700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce8062bf0 0x7f6ce814cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ced401700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ce81542d0 0x7f6ce80abb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 -- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ce81542d0 msgr2=0x7f6ce80abb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ce81542d0 0x7f6ce80abb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 -- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce8062bf0 msgr2=0x7f6ce814cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce8062bf0 0x7f6ce814cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 -- 172.21.15.73:0/1364876391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cd4009370 con 0x7f6ce8069ce0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce7fff700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f6cd4009130 tx=0x7f6cd4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4000bc0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd400b7f0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce80bf6d0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.059+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd4000d20 con 0x7f6ce8069ce0 2023-11-28T04:08:06.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.060+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce80a94c0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.061+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce80a96e0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.061+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f6cd401c8c0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.061+0000 7f6ce4ff9700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 0x7f6ccc0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.062+0000 7f6cecc00700 1 -- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 msgr2=0x7f6ccc0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.062+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f6cd407c1d0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.062+0000 7f6cecc00700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 0x7f6ccc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:06.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.066+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6ce80a96e0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.102 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:06.101+0000 7f0069a99680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:06.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.237+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ce80a96e0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.238+0000 7f6ce4ff9700 1 -- 172.21.15.73:0/1364876391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+17186 (secure 0 0 0) 0x7f6ce80a96e0 con 0x7f6ce8069ce0 2023-11-28T04:08:06.242 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:06.242 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":166,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:00.330191+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/1584723932":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:0/3403509088":"2023-11-29T04:08:00.330152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.239+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 msgr2=0x7f6ccc0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 0x7f6ccc0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 msgr2=0x7f6ce80a98f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 rx=0x7f6cd4009130 tx=0x7f6cd4017000).stop 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 shutdown_connections 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6ccc05f300 0x7f6ccc0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6ce81542d0 0x7f6ce80abb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6ce8062bf0 0x7f6ce814cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 --2- 172.21.15.73:0/1364876391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6ce8069ce0 0x7f6ce80a98f0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 >> 172.21.15.73:0/1364876391 conn(0x7f6ce80c7fd0 msgr2=0x7f6ce80c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 shutdown_connections 2023-11-28T04:08:06.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.240+0000 7f6ceee64700 1 -- 172.21.15.73:0/1364876391 wait complete. 2023-11-28T04:08:06.253 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:08:06.253 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:06.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.370+0000 7f61fc185700 1 Processor -- start 2023-11-28T04:08:06.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.370+0000 7f61fc185700 1 -- start start 2023-11-28T04:08:06.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fc185700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 0x7f61f4062b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fc185700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f61f40661b0 0x7f61f4066640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fc185700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61f4065760 con 0x7f61f41542d0 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fc185700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61f4063070 con 0x7f61f40661b0 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fc185700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f40c3250 con 0x7f61f4062700 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61f9f21700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f61f41542d0 0x7f61f40697f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41096/0 (socket says 172.21.15.73:41096) 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61f9f21700 1 -- 172.21.15.73:0/139142870 learned_addr learned my addr 172.21.15.73:0/139142870 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61fa722700 1 --2- 172.21.15.73:0/139142870 >> v2:172.21.15.73:3300/0 conn(0x7f61f40661b0 0x7f61f4066640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.371+0000 7f61f9720700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 0x7f61f4062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fa722700 1 -- 172.21.15.73:0/139142870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 msgr2=0x7f61f4062b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fa722700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 0x7f61f4062b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fa722700 1 -- 172.21.15.73:0/139142870 >> v1:172.21.15.73:6789/0 conn(0x7f61f41542d0 legacy=0x7f61f40697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fa722700 1 -- 172.21.15.73:0/139142870 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61f406a340 con 0x7f61f40661b0 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61f8f1f700 1 -- 172.21.15.73:0/139142870 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 660188480 0 0) 0x7f61f4065760 con 0x7f61f41542d0 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61f9720700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 0x7f61f4062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fa722700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f40661b0 0x7f61f4066640 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f61ec009e60 tx=0x7f61ec009780).ready entity=mon.0 client_cookie=c9cd6ddb749c7370 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61f8f1f700 1 -- 172.21.15.73:0/139142870 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec00e070 con 0x7f61f40661b0 2023-11-28T04:08:06.375 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61f8f1f700 1 -- 172.21.15.73:0/139142870 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec00bc30 con 0x7f61f40661b0 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61f8f1f700 1 -- 172.21.15.73:0/139142870 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec012410 con 0x7f61f40661b0 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 -- 172.21.15.73:0/139142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f40661b0 msgr2=0x7f61f4066640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f40661b0 0x7f61f4066640 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 rx=0x7f61ec009e60 tx=0x7f61ec009780).stop 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 -- 172.21.15.73:0/139142870 shutdown_connections 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f40661b0 0x7f61f4066640 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 --2- 172.21.15.73:0/139142870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f4062700 0x7f61f4062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 -- 172.21.15.73:0/139142870 >> 172.21.15.73:0/139142870 conn(0x7f61f40c7fd0 msgr2=0x7f61f40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 -- 172.21.15.73:0/139142870 shutdown_connections 2023-11-28T04:08:06.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.372+0000 7f61fc185700 1 -- 172.21.15.73:0/139142870 wait complete. 2023-11-28T04:08:06.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.373+0000 7f61fc185700 1 Processor -- start 2023-11-28T04:08:06.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 -- start start 2023-11-28T04:08:06.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f4062700 0x7f61f40a7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.381 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f40661b0 0x7f61f40a7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f414f410 con 0x7f61f41542d0 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4067ca0 con 0x7f61f4062700 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fc185700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61f4153290 con 0x7f61f40661b0 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fa722700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fa722700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37736/0 (socket says 172.21.15.73:37736) 2023-11-28T04:08:06.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61fa722700 1 -- 172.21.15.73:0/1157530447 learned_addr learned my addr 172.21.15.73:0/1157530447 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61f9720700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f40661b0 0x7f61f40a7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.374+0000 7f61f9f21700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f4062700 0x7f61f40a7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f40661b0 msgr2=0x7f61f40a7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f40661b0 0x7f61f40a7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f4062700 msgr2=0x7f61f40a7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f4062700 0x7f61f40a7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 -- 172.21.15.73:0/1157530447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ec009a20 con 0x7f61f41542d0 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fa722700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f61f40c5b90 tx=0x7f61ec009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec009440 con 0x7f61f41542d0 2023-11-28T04:08:06.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61ec01d630 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61f4152430 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61ec01d9d0 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.375+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61f4152960 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.376+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61f406c860 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.377+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f61ec003040 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.377+0000 7f61e9ffb700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 0x7f61d80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.377+0000 7f61f9f21700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 msgr2=0x7f61d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.377+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f61ec07c350 con 0x7f61f41542d0 2023-11-28T04:08:06.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.377+0000 7f61f9f21700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 0x7f61d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:06.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.385+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61ec04d220 con 0x7f61f41542d0 2023-11-28T04:08:06.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.578+0000 7f61f9f21700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 msgr2=0x7f61d80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:06.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.578+0000 7f61f9f21700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 0x7f61d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:06.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.596+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61f405f660 con 0x7f61f41542d0 2023-11-28T04:08:06.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.596+0000 7f61e9ffb700 1 -- 172.21.15.73:0/1157530447 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7f61ec050840 con 0x7f61f41542d0 2023-11-28T04:08:06.597 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:06.597 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 msgr2=0x7f61d80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 0x7f61d80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 msgr2=0x7f61f40a9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 rx=0x7f61f40c5b90 tx=0x7f61ec009780).stop 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 shutdown_connections 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f61d805f300 0x7f61d80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61f41542d0 0x7f61f40a9790 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61f4062700 0x7f61f40a7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 --2- 172.21.15.73:0/1157530447 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61f40661b0 0x7f61f40a7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 >> 172.21.15.73:0/1157530447 conn(0x7f61f40c7fd0 msgr2=0x7f61f40bcf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 shutdown_connections 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.599+0000 7f61fc185700 1 -- 172.21.15.73:0/1157530447 wait complete. 2023-11-28T04:08:06.601 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:08:06.612 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:06.625 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:06.626+0000 7f38de016680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:08:06.694 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:06.694+0000 7f38de016680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:08:06.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.732+0000 7ff1ece0e700 1 Processor -- start 2023-11-28T04:08:06.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.732+0000 7ff1ece0e700 1 -- start start 2023-11-28T04:08:06.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1ece0e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 0x7ff1e80bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1ece0e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1e806a010 0x7ff1e81544f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1ece0e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1e80c28c0 con 0x7ff1e80bc320 2023-11-28T04:08:06.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1ece0e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e81552e0 con 0x7ff1e806a010 2023-11-28T04:08:06.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1ece0e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1e806a4b0 con 0x7ff1e8150890 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1e806a010 0x7ff1e81544f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff1e806a010 0x7ff1e81544f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37752/0 (socket says 172.21.15.73:37752) 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/2657646363 learned_addr learned my addr 172.21.15.73:0/2657646363 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e659c700 1 --2- 172.21.15.73:0/2657646363 >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 0x7ff1e80bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/2657646363 >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 msgr2=0x7ff1e80bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 --2- 172.21.15.73:0/2657646363 >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 0x7ff1e80bbe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/2657646363 >> v1:172.21.15.150:6789/0 conn(0x7ff1e80bc320 legacy=0x7ff1e8069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/2657646363 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e80c1f90 con 0x7ff1e806a010 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.733+0000 7ff1e659c700 1 --2- 172.21.15.73:0/2657646363 >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 0x7ff1e80bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1e6d9d700 1 --2- 172.21.15.73:0/2657646363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e806a010 0x7ff1e81544f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7ff1e80c7ed0 tx=0x7ff1e0009840).ready entity=mon.0 client_cookie=9678388a03b15bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1e559a700 1 -- 172.21.15.73:0/2657646363 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000e070 con 0x7ff1e806a010 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1e559a700 1 -- 172.21.15.73:0/2657646363 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e000ba50 con 0x7ff1e806a010 2023-11-28T04:08:06.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1e559a700 1 -- 172.21.15.73:0/2657646363 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e000bdd0 con 0x7ff1e806a010 2023-11-28T04:08:06.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 -- 172.21.15.73:0/2657646363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e806a010 msgr2=0x7ff1e81544f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/2657646363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e806a010 0x7ff1e81544f0 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 rx=0x7ff1e80c7ed0 tx=0x7ff1e0009840).stop 2023-11-28T04:08:06.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 -- 172.21.15.73:0/2657646363 shutdown_connections 2023-11-28T04:08:06.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/2657646363 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e806a010 0x7ff1e81544f0 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/2657646363 >> v2:172.21.15.150:3300/0 conn(0x7ff1e8150890 0x7ff1e80bbe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 -- 172.21.15.73:0/2657646363 >> 172.21.15.73:0/2657646363 conn(0x7ff1e80631d0 msgr2=0x7ff1e80c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 -- 172.21.15.73:0/2657646363 shutdown_connections 2023-11-28T04:08:06.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.734+0000 7ff1ece0e700 1 -- 172.21.15.73:0/2657646363 wait complete. 2023-11-28T04:08:06.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.735+0000 7ff1ece0e700 1 Processor -- start 2023-11-28T04:08:06.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.735+0000 7ff1ece0e700 1 -- start start 2023-11-28T04:08:06.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e806a010 0x7ff1e8093d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80bc320 0x7ff1e80c0550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80bea30 con 0x7ff1e8150890 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e80be120 con 0x7ff1e80bc320 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1ece0e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1e8094750 con 0x7ff1e806a010 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1e6d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37756/0 (socket says 172.21.15.73:37756) 2023-11-28T04:08:06.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/275430388 learned_addr learned my addr 172.21.15.73:0/275430388 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1e5d9b700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80bc320 0x7ff1e80c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.736+0000 7ff1e659c700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e806a010 0x7ff1e8093d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e806a010 msgr2=0x7ff1e8093d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e806a010 0x7ff1e8093d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80bc320 msgr2=0x7ff1e80c0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80bc320 0x7ff1e80c0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 -- 172.21.15.73:0/275430388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1e0009ae0 con 0x7ff1e8150890 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1e6d9d700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7ff1e0009fd0 tx=0x7ff1e0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00032d0 con 0x7ff1e8150890 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0013e30 con 0x7ff1e8150890 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1e80be370 con 0x7ff1e8150890 2023-11-28T04:08:06.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e001d620 con 0x7ff1e8150890 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.737+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1e814ab10 con 0x7ff1e8150890 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.738+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1e814ad00 con 0x7ff1e8150890 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.739+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7ff1e001da50 con 0x7ff1e8150890 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.739+0000 7ff1d67fc700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 0x7ff1c80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.739+0000 7ff1e659c700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 msgr2=0x7ff1c80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.739+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7ff1e007cb30 con 0x7ff1e8150890 2023-11-28T04:08:06.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.739+0000 7ff1e659c700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 0x7ff1c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:06.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.743+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff1e004dac0 con 0x7ff1e8150890 2023-11-28T04:08:06.770 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:06.770+0000 7f38de016680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:08:06.791 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:06.790+0000 7f0069a99680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:08:06.862 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:06.862+0000 7f0069a99680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:08:06.919 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:06.919+0000 7f38de016680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:08:06.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.940+0000 7ff1e659c700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 msgr2=0x7ff1c80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:06.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.940+0000 7ff1e659c700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 0x7ff1c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:06.941 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:06.940+0000 7f0069a99680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:08:06.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.954+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1e80beb70 con 0x7ff1e8150890 2023-11-28T04:08:06.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.955+0000 7ff1d67fc700 1 -- 172.21.15.73:0/275430388 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 58 v58) v1 ==== 94+0+4497 (secure 0 0 0) 0x7ff1e80beb70 con 0x7ff1e8150890 2023-11-28T04:08:06.956 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:06.956 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":58,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55}],"filesystems":[{"mdsmap":{"epoch":58,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:07:44.458686+0000","modified":"2023-11-28T04:07:46.468974+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8562},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8562":{"gid":8562,"name":"a","rank":0,"incarnation":56,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3981387608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3981387608},{"type":"v1","addr":"172.21.15.73:6837","nonce":3981387608}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[20],"metadata_pool":19,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":9}]} 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 msgr2=0x7ff1c80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 0x7ff1c80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 msgr2=0x7ff1e80c0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 rx=0x7ff1e0009fd0 tx=0x7ff1e0009840).stop 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 shutdown_connections 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7ff1c805f300 0x7ff1c80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff1e8150890 0x7ff1e80c0a40 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff1e806a010 0x7ff1e8093d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 --2- 172.21.15.73:0/275430388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff1e80bc320 0x7ff1e80c0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:06.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 >> 172.21.15.73:0/275430388 conn(0x7ff1e80631d0 msgr2=0x7ff1e80c6550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:06.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 shutdown_connections 2023-11-28T04:08:06.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:06.957+0000 7ff1ece0e700 1 -- 172.21.15.73:0/275430388 wait complete. 2023-11-28T04:08:06.959 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 58 2023-11-28T04:08:06.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:06.990 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:06.990+0000 7f38de016680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:08:07.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.087+0000 7f2af7324700 1 Processor -- start 2023-11-28T04:08:07.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.087+0000 7f2af7324700 1 -- start start 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af7324700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c0180 0x7f2af0157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af7324700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af00c1750 0x7f2af00c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af7324700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af00c12c0 con 0x7f2af0157660 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af7324700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af00c06b0 con 0x7f2af00c1750 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af7324700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af00c0990 con 0x7f2af00c0180 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af6b23700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af00c1750 0x7f2af00c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af6b23700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2af00c1750 0x7f2af00c1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37760/0 (socket says 172.21.15.73:37760) 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 -- 172.21.15.73:0/4251964390 learned_addr learned my addr 172.21.15.73:0/4251964390 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.088+0000 7f2af6322700 1 --1- 172.21.15.73:0/4251964390 >> v1:172.21.15.73:6789/0 conn(0x7f2af0157660 0x7f2af01589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41102/0 (socket says 172.21.15.73:41102) 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af5b21700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c0180 0x7f2af0157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 -- 172.21.15.73:0/4251964390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c0180 msgr2=0x7f2af0157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c0180 0x7f2af0157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 -- 172.21.15.73:0/4251964390 >> v1:172.21.15.73:6789/0 conn(0x7f2af0157660 legacy=0x7f2af01589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 -- 172.21.15.73:0/4251964390 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af00c0c70 con 0x7f2af00c1750 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af6b23700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c1750 0x7f2af00c1ba0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f2ae8009f20 tx=0x7f2ae8009840).ready entity=mon.0 client_cookie=232e265248f3b116 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af5320700 1 -- 172.21.15.73:0/4251964390 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800e070 con 0x7f2af00c1750 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af5320700 1 -- 172.21.15.73:0/4251964390 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae800b9d0 con 0x7f2af00c1750 2023-11-28T04:08:07.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.089+0000 7f2af5320700 1 -- 172.21.15.73:0/4251964390 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800bd50 con 0x7f2af00c1750 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 -- 172.21.15.73:0/4251964390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c1750 msgr2=0x7f2af00c1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c1750 0x7f2af00c1ba0 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 rx=0x7f2ae8009f20 tx=0x7f2ae8009840).stop 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 -- 172.21.15.73:0/4251964390 shutdown_connections 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c1750 0x7f2af00c1ba0 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 --2- 172.21.15.73:0/4251964390 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c0180 0x7f2af0157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 -- 172.21.15.73:0/4251964390 >> 172.21.15.73:0/4251964390 conn(0x7f2af00cb000 msgr2=0x7f2af00cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 -- 172.21.15.73:0/4251964390 shutdown_connections 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.090+0000 7f2af7324700 1 -- 172.21.15.73:0/4251964390 wait complete. 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.091+0000 7f2af7324700 1 Processor -- start 2023-11-28T04:08:07.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.091+0000 7f2af7324700 1 -- start start 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c1750 0x7f2af00b04b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af0157660 0x7f2af00b09a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0167d20 con 0x7f2af00c0180 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af0167fc0 con 0x7f2af0157660 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01682a0 con 0x7f2af00c1750 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37766/0 (socket says 172.21.15.73:37766) 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 -- 172.21.15.73:0/1242363674 learned_addr learned my addr 172.21.15.73:0/1242363674 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af5b21700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c1750 0x7f2af00b04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6b23700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af0157660 0x7f2af00b09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 -- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c1750 msgr2=0x7f2af00b04b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c1750 0x7f2af00b04b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 -- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af0157660 msgr2=0x7f2af00b09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af0157660 0x7f2af00b09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 -- 172.21.15.73:0/1242363674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae8009ae0 con 0x7f2af00c0180 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af6322700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7f2aec001eb0 tx=0x7f2aec007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0070d0 con 0x7f2af00c0180 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec007230 con 0x7f2af00c0180 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af0168580 con 0x7f2af00c0180 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec0075b0 con 0x7f2af00c0180 2023-11-28T04:08:07.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.092+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af0168aa0 con 0x7f2af00c0180 2023-11-28T04:08:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f2aec0058f0 con 0x7f2af00c0180 2023-11-28T04:08:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af006c7c0 con 0x7f2af00c0180 2023-11-28T04:08:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2ae67fc700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 0x7f2ad40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2af5b21700 1 -- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 msgr2=0x7f2ad40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:07.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2aec079c00 con 0x7f2af00c0180 2023-11-28T04:08:07.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.094+0000 7f2af5b21700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 0x7f2ad40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:07.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.098+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2aec07f020 con 0x7f2af00c0180 2023-11-28T04:08:07.104 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.104+0000 7f0069a99680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:08:07.110 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:07.110+0000 7f38de016680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:08:07.195 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.194+0000 7f0069a99680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:08:07.219 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:07.219+0000 7f38de016680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:07.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.288+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2af0168770 con 0x7f2af00c0180 2023-11-28T04:08:07.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.289+0000 7f2ae67fc700 1 -- 172.21.15.73:0/1242363674 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v166) v1 ==== 74+0+17186 (secure 0 0 0) 0x7f2aec018020 con 0x7f2af00c0180 2023-11-28T04:08:07.290 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:07.290 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":166,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:00.330191+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"154","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":19,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:07:43.015630+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"164","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":20,"pool_name":"cephfs_data","create_time":"2023-11-28T04:07:43.427278+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"165","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/1584723932":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:00.330152+0000","172.21.15.73:0/3403509088":"2023-11-29T04:08:00.330152+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:07.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 msgr2=0x7f2ad40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 0x7f2ad40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 msgr2=0x7f2af00c4130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 rx=0x7f2aec001eb0 tx=0x7f2aec007df0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 shutdown_connections 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad405f300 0x7f2ad40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2af0157660 0x7f2af00b09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af00c0180 0x7f2af00c4130 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 --2- 172.21.15.73:0/1242363674 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af00c1750 0x7f2af00b04b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 >> 172.21.15.73:0/1242363674 conn(0x7f2af00cb000 msgr2=0x7f2af00aa100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 shutdown_connections 2023-11-28T04:08:07.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.291+0000 7f2af7324700 1 -- 172.21.15.73:0/1242363674 wait complete. 2023-11-28T04:08:07.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T04:08:07.314 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.313+0000 7f0069a99680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:08:07.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.421+0000 7fb2893fb700 1 Processor -- start 2023-11-28T04:08:07.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.421+0000 7fb2893fb700 1 -- start start 2023-11-28T04:08:07.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2893fb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2840bcba0 0x7fb284069df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2893fb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840c4df0 con 0x7fb284152480 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2893fb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2840627a0 con 0x7fb284152860 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2893fb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb28406a2e0 con 0x7fb2840bcba0 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2827fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb2840bcba0 0x7fb284069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb282ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb284152480 0x7fb2840bc490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41106/0 (socket says 172.21.15.73:41106) 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb282ffd700 1 -- 172.21.15.73:0/954293009 learned_addr learned my addr 172.21.15.73:0/954293009 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb281ffb700 1 -- 172.21.15.73:0/954293009 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745142137 0 0) 0x7fb2840c4df0 con 0x7fb284152480 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb281ffb700 1 -- 172.21.15.73:0/954293009 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb26c0033f0 con 0x7fb284152480 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2827fc700 1 -- 172.21.15.73:0/954293009 >> v1:172.21.15.150:6789/0 conn(0x7fb284152860 legacy=0x7fb2840aa5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2827fc700 1 -- 172.21.15.73:0/954293009 >> v1:172.21.15.73:6789/0 conn(0x7fb284152480 legacy=0x7fb2840bc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.422+0000 7fb2827fc700 1 -- 172.21.15.73:0/954293009 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb284062a80 con 0x7fb2840bcba0 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2827fc700 1 --2- 172.21.15.73:0/954293009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2840bcba0 0x7fb284069df0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fb274009760 tx=0x7fb274009060).ready entity=mon.0 client_cookie=db2967fa0a306d8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:07.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb281ffb700 1 -- 172.21.15.73:0/954293009 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400f510 con 0x7fb2840bcba0 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb281ffb700 1 -- 172.21.15.73:0/954293009 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27400f670 con 0x7fb2840bcba0 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb281ffb700 1 -- 172.21.15.73:0/954293009 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27400f9f0 con 0x7fb2840bcba0 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 -- 172.21.15.73:0/954293009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2840bcba0 msgr2=0x7fb284069df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 --2- 172.21.15.73:0/954293009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2840bcba0 0x7fb284069df0 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 rx=0x7fb274009760 tx=0x7fb274009060).stop 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 -- 172.21.15.73:0/954293009 shutdown_connections 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 --2- 172.21.15.73:0/954293009 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb2840bcba0 0x7fb284069df0 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 -- 172.21.15.73:0/954293009 >> 172.21.15.73:0/954293009 conn(0x7fb2840c7fb0 msgr2=0x7fb2840c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 -- 172.21.15.73:0/954293009 shutdown_connections 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.423+0000 7fb2893fb700 1 -- 172.21.15.73:0/954293009 wait complete. 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.424+0000 7fb2893fb700 1 Processor -- start 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 -- start start 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2840bcba0 0x7fb2840c0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb284152480 0x7fb284144b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb284166d00 con 0x7fb284152860 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2841660f0 con 0x7fb2840bcba0 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2893fb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2841663d0 con 0x7fb284152480 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb282ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2840bcba0 0x7fb2840c0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2837fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37784/0 (socket says 172.21.15.73:37784) 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2837fe700 1 -- 172.21.15.73:0/2576621560 learned_addr learned my addr 172.21.15.73:0/2576621560 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.425+0000 7fb2827fc700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb284152480 0x7fb284144b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb284152480 msgr2=0x7fb284144b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb284152480 0x7fb284144b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2840bcba0 msgr2=0x7fb2840c0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2840bcba0 0x7fb2840c0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 -- 172.21.15.73:0/2576621560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2740092a0 con 0x7fb284152860 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2837fe700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fb27c00c7c0 tx=0x7fb27c001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00e4a0 con 0x7fb284152860 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb27c00e600 con 0x7fb284152860 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb27c00e980 con 0x7fb284152860 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2841666b0 con 0x7fb284152860 2023-11-28T04:08:07.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.426+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2841456c0 con 0x7fb284152860 2023-11-28T04:08:07.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.427+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb284166eb0 con 0x7fb284152860 2023-11-28T04:08:07.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.427+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 70) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb27c0123f0 con 0x7fb284152860 2023-11-28T04:08:07.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.428+0000 7fb26affd700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 0x7fb2700617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:07.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.428+0000 7fb282ffd700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 msgr2=0x7fb2700617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:07.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.428+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(166..166 src has 1..166) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb27c079d00 con 0x7fb284152860 2023-11-28T04:08:07.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.428+0000 7fb282ffd700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 0x7fb2700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:07.433 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.429+0000 7f0069a99680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:07.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.432+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb27c047af0 con 0x7fb284152860 2023-11-28T04:08:07.526 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:07.526+0000 7f38de016680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:07.597 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:07.597+0000 7f38de016680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:07.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.629+0000 7fb282ffd700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 msgr2=0x7fb2700617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:07.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.629+0000 7fb282ffd700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 0x7fb2700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:07.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.648+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb28406c7c0 con 0x7fb284152860 2023-11-28T04:08:07.731 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.730+0000 7f0069a99680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:07.802 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.801+0000 7f0069a99680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:07.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.820+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 71) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fb27c077820 con 0x7fb284152860 2023-11-28T04:08:07.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.821+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 msgr2=0x7fb2700617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:07.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:07.821+0000 7fb26affd700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 0x7fb2700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:07.824 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:08:07.823+0000 7fe250124700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:08:07.824 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:08:07.823+0000 7fe250124700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:08:07.824 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:08:07.823+0000 7fe250124700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:08:07.829 INFO:tasks.ceph.mds.a.smithi073.stderr: -2174> 2023-11-28T04:07:50.382+0000 7fe25392b700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:08:07.832 INFO:tasks.ceph.mds.a.smithi073.stderr: -5> 2023-11-28T04:08:07.823+0000 7fe250124700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:08:07.832 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T04:08:07.823+0000 7fe250124700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:08:07.832 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T04:08:07.823+0000 7fe250124700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:08:07.851 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe3a2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe3a2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe3a2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe3a2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe3a2a700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.878 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe7231700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.879 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe7231700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.879 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe7231700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.879 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe7231700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:07.879 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:07.876+0000 7effe7231700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:08.036 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T04:08:08.035+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T04:08:08.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.042+0000 7fb26affd700 1 -- 172.21.15.73:0/2576621560 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v59) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb27c0710b0 con 0x7fb284152860 2023-11-28T04:08:08.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 msgr2=0x7fb284150a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 rx=0x7fb27c00c7c0 tx=0x7fb27c001510).stop 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 shutdown_connections 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fb27005f300 0x7fb2700617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb284152860 0x7fb284150a30 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb2840bcba0 0x7fb2840c0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 --2- 172.21.15.73:0/2576621560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb284152480 0x7fb284144b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 >> 172.21.15.73:0/2576621560 conn(0x7fb2840c7fb0 msgr2=0x7fb284068cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 shutdown_connections 2023-11-28T04:08:08.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.044+0000 7fb2893fb700 1 -- 172.21.15.73:0/2576621560 wait complete. 2023-11-28T04:08:08.047 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T04:08:08.058 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T04:08:08.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.213+0000 7fc72f318700 1 Processor -- start 2023-11-28T04:08:08.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.213+0000 7fc72f318700 1 -- start start 2023-11-28T04:08:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.214+0000 7fc72f318700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.214+0000 7fc72f318700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc728158320 0x7fc7281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.214+0000 7fc72f318700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc728158cc0 con 0x7fc728098d10 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.214+0000 7fc72f318700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280c72d0 con 0x7fc728158320 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.214+0000 7fc72f318700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc728064570 con 0x7fc7280c6190 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72c8b3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc728158320 0x7fc7281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d0b4700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc728098d10 0x7fc728063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:48554/0 (socket says 172.21.15.73:48554) 2023-11-28T04:08:08.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d0b4700 1 -- 172.21.15.73:0/3493545121 learned_addr learned my addr 172.21.15.73:0/3493545121 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 msgr2=0x7fc7280c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3493545121 >> v1:172.21.15.150:6789/0 conn(0x7fc728098d10 legacy=0x7fc728063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc727fff700 1 -- 172.21.15.73:0/3493545121 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1199243167 0 0) 0x7fc718001490 con 0x7fc728098d10 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3493545121 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc728154bc0 con 0x7fc728158320 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72c8b3700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc72d8b5700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728158320 0x7fc7281587d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fc720009760 tx=0x7fc720009060).ready entity=mon.1 client_cookie=ede8858cc1914648 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc727fff700 1 -- 172.21.15.73:0/3493545121 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000f510 con 0x7fc728158320 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc727fff700 1 -- 172.21.15.73:0/3493545121 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000f670 con 0x7fc728158320 2023-11-28T04:08:08.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.215+0000 7fc727fff700 1 -- 172.21.15.73:0/3493545121 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000f9f0 con 0x7fc728158320 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 -- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728158320 msgr2=0x7fc7281587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728158320 0x7fc7281587d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 rx=0x7fc720009760 tx=0x7fc720009060).stop 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 -- 172.21.15.73:0/3493545121 shutdown_connections 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728158320 0x7fc7281587d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 --2- 172.21.15.73:0/3493545121 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 -- 172.21.15.73:0/3493545121 >> 172.21.15.73:0/3493545121 conn(0x7fc7280cb310 msgr2=0x7fc7280cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 -- 172.21.15.73:0/3493545121 shutdown_connections 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.216+0000 7fc72f318700 1 -- 172.21.15.73:0/3493545121 wait complete. 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.217+0000 7fc72f318700 1 Processor -- start 2023-11-28T04:08:08.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.217+0000 7fc72f318700 1 -- start start 2023-11-28T04:08:08.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728098d10 0x7fc72814bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c3a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280c4460 con 0x7fc728158320 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280c51e0 con 0x7fc728098d10 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72f318700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280b12a0 con 0x7fc7280c6190 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37900/0 (socket says 172.21.15.73:37900) 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3723028098 learned_addr learned my addr 172.21.15.73:0/3723028098 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d0b4700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728098d10 0x7fc72814bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72c8b3700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c3a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 msgr2=0x7fc7280c3a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c3a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728098d10 msgr2=0x7fc72814bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:08.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728098d10 0x7fc72814bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.218+0000 7fc72d8b5700 1 -- 172.21.15.73:0/3723028098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c0092a0 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc72d8b5700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fc7280c2360 tx=0x7fc720017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72000f7e0 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc72000f940 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7200092a0 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc72001b620 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.219+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7280c49c0 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.220+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc728154de0 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.220+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 71) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fc72001ba10 con 0x7fc728158320 2023-11-28T04:08:08.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.221+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fc72007bc20 con 0x7fc728158320 2023-11-28T04:08:08.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.225+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc728154de0 con 0x7fc728158320 2023-11-28T04:08:08.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.429+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc728154de0 con 0x7fc728158320 2023-11-28T04:08:08.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.822+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fc7200799e0 con 0x7fc728158320 2023-11-28T04:08:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.823+0000 7fc7257fa700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 0x7fc714067a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:08.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.823+0000 7fc72d0b4700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 0x7fc714067a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:08.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:08.825+0000 7fc72d0b4700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 0x7fc714067a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc718007fd0 tx=0x7fc718007d90).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:09.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.043+0000 7fc7257fa700 1 -- 172.21.15.73:0/3723028098 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v60) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc7200549b0 con 0x7fc728158320 2023-11-28T04:08:09.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 msgr2=0x7fc714067a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 0x7fc714067a70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc718007fd0 tx=0x7fc718007d90).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 msgr2=0x7fc7280c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 rx=0x7fc7280c2360 tx=0x7fc720017000).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 shutdown_connections 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc7140302c0 0x7fc714067a70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc728158320 0x7fc7280c3f70 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc728098d10 0x7fc72814bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 --2- 172.21.15.73:0/3723028098 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc7280c6190 0x7fc7280c3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 >> 172.21.15.73:0/3723028098 conn(0x7fc7280cb310 msgr2=0x7fc7280c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:09.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 shutdown_connections 2023-11-28T04:08:09.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.045+0000 7fc72f318700 1 -- 172.21.15.73:0/3723028098 wait complete. 2023-11-28T04:08:09.059 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T04:08:09.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.176+0000 7f534f0d3700 1 Processor -- start 2023-11-28T04:08:09.177 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.176+0000 7f534f0d3700 1 -- start start 2023-11-28T04:08:09.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534f0d3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5348154610 0x7f53481582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.179 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534f0d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53480ab3b0 0x7f53480c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534f0d3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5348159210 con 0x7f53480aa1c0 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534f0d3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53480c0c70 con 0x7f5348154610 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534f0d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53480c2110 con 0x7f53480ab3b0 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5348154610 0x7f53481582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5348154610 0x7f53481582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37914/0 (socket says 172.21.15.73:37914) 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 -- 172.21.15.73:0/2915520092 learned_addr learned my addr 172.21.15.73:0/2915520092 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f5347fff700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53480ab3b0 0x7f53480c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 -- 172.21.15.73:0/2915520092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53480ab3b0 msgr2=0x7f53480c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53480ab3b0 0x7f53480c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 -- 172.21.15.73:0/2915520092 >> v1:172.21.15.73:6789/0 conn(0x7f53480aa1c0 legacy=0x7f5348063550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 -- 172.21.15.73:0/2915520092 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5348158970 con 0x7f5348154610 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f534ce6f700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5348154610 0x7f53481582e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f5340009f20 tx=0x7f5340009840).ready entity=mon.0 client_cookie=3e1ee69c44f181c2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:09.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f53477fe700 1 -- 172.21.15.73:0/2915520092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000e070 con 0x7f5348154610 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f53477fe700 1 -- 172.21.15.73:0/2915520092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f534000ba80 con 0x7f5348154610 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.177+0000 7f53477fe700 1 -- 172.21.15.73:0/2915520092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534000be00 con 0x7f5348154610 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 -- 172.21.15.73:0/2915520092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5348154610 msgr2=0x7f53481582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5348154610 0x7f53481582e0 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 rx=0x7f5340009f20 tx=0x7f5340009840).stop 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 -- 172.21.15.73:0/2915520092 shutdown_connections 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5348154610 0x7f53481582e0 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 --2- 172.21.15.73:0/2915520092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53480ab3b0 0x7f53480c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 -- 172.21.15.73:0/2915520092 >> 172.21.15.73:0/2915520092 conn(0x7f53480655a0 msgr2=0x7f5348069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 -- 172.21.15.73:0/2915520092 shutdown_connections 2023-11-28T04:08:09.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.178+0000 7f534f0d3700 1 -- 172.21.15.73:0/2915520092 wait complete. 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.179+0000 7f534f0d3700 1 Processor -- start 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.179+0000 7f534f0d3700 1 -- start start 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 0x7f5348156ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5348157390 0x7f53480c1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53481559c0 con 0x7f53480aa1c0 2023-11-28T04:08:09.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5348155c10 con 0x7f5348154610 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534f0d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5348155ef0 con 0x7f5348157390 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37916/0 (socket says 172.21.15.73:37916) 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 -- 172.21.15.73:0/1799398079 learned_addr learned my addr 172.21.15.73:0/1799398079 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f5347fff700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 0x7f5348156ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534d670700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5348157390 0x7f53480c1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 -- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5348157390 msgr2=0x7f53480c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5348157390 0x7f53480c1100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 -- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 msgr2=0x7f5348156ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 0x7f5348156ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:09.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f534ce6f700 1 -- 172.21.15.73:0/1799398079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5340009ae0 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.180+0000 7f5347fff700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 0x7f5348156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f534ce6f700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 rx=0x7f534000b6c0 tx=0x7f5340009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5340013cc0 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5340013e20 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f534001e680 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53481561d0 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.181+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5348153e80 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.182+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53481566d0 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.182+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f5340003040 con 0x7f53480aa1c0 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.182+0000 7f5344ff9700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 0x7f53280614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:09.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.183+0000 7f5347fff700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 0x7f53280614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.183+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f534007b660 con 0x7f53480aa1c0 2023-11-28T04:08:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.183+0000 7f5347fff700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 0x7f53280614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f5348157910 tx=0x7f5338008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:09.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.187+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f534004c9d0 con 0x7f53480aa1c0 2023-11-28T04:08:09.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:09.350+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5348158c50 con 0x7f53480aa1c0 2023-11-28T04:08:10.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.049+0000 7f5344ff9700 1 -- 172.21.15.73:0/1799398079 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v169) v1 ==== 181+0+0 (secure 0 0 0) 0x7f5340023070 con 0x7f53480aa1c0 2023-11-28T04:08:10.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 msgr2=0x7f53280614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 0x7f53280614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f5348157910 tx=0x7f5338008040).stop 2023-11-28T04:08:10.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 msgr2=0x7f5348153990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 rx=0x7f534000b6c0 tx=0x7f5340009840).stop 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 shutdown_connections 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f532805f030 0x7f53280614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5348157390 0x7f53480c1100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53480aa1c0 0x7f5348153990 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 --2- 172.21.15.73:0/1799398079 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5348154610 0x7f5348156ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 >> 172.21.15.73:0/1799398079 conn(0x7f53480655a0 msgr2=0x7f53480ad8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 shutdown_connections 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.053+0000 7f534f0d3700 1 -- 172.21.15.73:0/1799398079 wait complete. 2023-11-28T04:08:10.055 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T04:08:10.067 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T04:08:10.183 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.182+0000 7fa840c54700 1 Processor -- start 2023-11-28T04:08:10.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.182+0000 7fa840c54700 1 -- start start 2023-11-28T04:08:10.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa840c54700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa83c0c5f30 0x7fa83c0c63e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa840c54700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa83c063af0 0x7fa83c063f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa840c54700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83c0c68d0 con 0x7fa83c1373a0 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa840c54700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c158e20 con 0x7fa83c063af0 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa840c54700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa83c1582a0 con 0x7fa83c0c5f30 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa83a59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa83c1373a0 0x7fa83c0c72e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41118/0 (socket says 172.21.15.73:41118) 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa83a59c700 1 -- 172.21.15.73:0/234705180 learned_addr learned my addr 172.21.15.73:0/234705180 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.183+0000 7fa83ad9d700 1 --2- 172.21.15.73:0/234705180 >> v2:172.21.15.73:3300/0 conn(0x7fa83c063af0 0x7fa83c063f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:10.187 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa831d9b700 1 --2- 172.21.15.73:0/234705180 >> v2:172.21.15.150:3300/0 conn(0x7fa83c0c5f30 0x7fa83c0c63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa83ad9d700 1 -- 172.21.15.73:0/234705180 >> v2:172.21.15.150:3300/0 conn(0x7fa83c0c5f30 msgr2=0x7fa83c0c63e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa83ad9d700 1 --2- 172.21.15.73:0/234705180 >> v2:172.21.15.150:3300/0 conn(0x7fa83c0c5f30 0x7fa83c0c63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa83ad9d700 1 -- 172.21.15.73:0/234705180 >> v1:172.21.15.73:6789/0 conn(0x7fa83c1373a0 legacy=0x7fa83c0c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa83ad9d700 1 -- 172.21.15.73:0/234705180 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83c158580 con 0x7fa83c063af0 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa839d9b700 1 -- 172.21.15.73:0/234705180 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340075005 0 0) 0x7fa82c001110 con 0x7fa83c1373a0 2023-11-28T04:08:10.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa83ad9d700 1 --2- 172.21.15.73:0/234705180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c063f40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7fa83c069110 tx=0x7fa834009780).ready entity=mon.0 client_cookie=9592c784d889206a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa839d9b700 1 -- 172.21.15.73:0/234705180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa83400e070 con 0x7fa83c063af0 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa839d9b700 1 -- 172.21.15.73:0/234705180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa83400bc30 con 0x7fa83c063af0 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa839d9b700 1 -- 172.21.15.73:0/234705180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa834012410 con 0x7fa83c063af0 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa840c54700 1 -- 172.21.15.73:0/234705180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 msgr2=0x7fa83c063f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.184+0000 7fa840c54700 1 --2- 172.21.15.73:0/234705180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c063f40 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 rx=0x7fa83c069110 tx=0x7fa834009780).stop 2023-11-28T04:08:10.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 -- 172.21.15.73:0/234705180 shutdown_connections 2023-11-28T04:08:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 --2- 172.21.15.73:0/234705180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c063f40 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 --2- 172.21.15.73:0/234705180 >> v2:172.21.15.150:3300/0 conn(0x7fa83c0c5f30 0x7fa83c0c63e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 -- 172.21.15.73:0/234705180 >> 172.21.15.73:0/234705180 conn(0x7fa83c0628f0 msgr2=0x7fa83c062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 -- 172.21.15.73:0/234705180 shutdown_connections 2023-11-28T04:08:10.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.185+0000 7fa840c54700 1 -- 172.21.15.73:0/234705180 wait complete. 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.186+0000 7fa840c54700 1 Processor -- start 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.186+0000 7fa840c54700 1 -- start start 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c0c5f30 0x7fa83c0aeb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa83c1373a0 0x7fa83c13a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c164040 con 0x7fa83c063af0 2023-11-28T04:08:10.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c164f20 con 0x7fa83c0c5f30 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83c0af060 con 0x7fa83c1373a0 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37926/0 (socket says 172.21.15.73:37926) 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 -- 172.21.15.73:0/2206010175 learned_addr learned my addr 172.21.15.73:0/2206010175 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 -- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa83c1373a0 msgr2=0x7fa83c13a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa83c1373a0 0x7fa83c13a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 -- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c0c5f30 msgr2=0x7fa83c0aeb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c0c5f30 0x7fa83c0aeb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 -- 172.21.15.73:0/2206010175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa834009a20 con 0x7fa83c063af0 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa83a59c700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7fa82c00c460 tx=0x7fa82c007e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0072e0 con 0x7fa83c063af0 2023-11-28T04:08:10.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c007440 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa83c1379f0 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0057a0 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.187+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa83c137fb0 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.188+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83c06c860 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.189+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 72) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fa82c005b90 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.189+0000 7fa832ffd700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 0x7fa81c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.189+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 6156+0+0 (secure 0 0 0) 0x7fa82c0796f0 con 0x7fa83c063af0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.189+0000 7fa831d9b700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 0x7fa81c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.189+0000 7fa831d9b700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 0x7fa81c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa82400a880 tx=0x7fa824008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:10.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.193+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa82c04ab70 con 0x7fa83c063af0 2023-11-28T04:08:10.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.357+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa83c13ab30 con 0x7fa83c063af0 2023-11-28T04:08:10.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:10.825+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa82c077670 con 0x7fa83c063af0 2023-11-28T04:08:11.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.054+0000 7fa832ffd700 1 -- 172.21.15.73:0/2206010175 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v170) v1 ==== 169+0+0 (secure 0 0 0) 0x7fa82c076c20 con 0x7fa83c063af0 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.058+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 msgr2=0x7fa81c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.058+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 0x7fa81c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fa82400a880 tx=0x7fa824008040).stop 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.058+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 msgr2=0x7fa83c164a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.058+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 rx=0x7fa82c00c460 tx=0x7fa82c007e90).stop 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 shutdown_connections 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa81c05f030 0x7fa81c0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa83c1373a0 0x7fa83c13a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:11.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa83c063af0 0x7fa83c164a30 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 --2- 172.21.15.73:0/2206010175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa83c0c5f30 0x7fa83c0aeb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:11.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 >> 172.21.15.73:0/2206010175 conn(0x7fa83c0628f0 msgr2=0x7fa83c062cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 shutdown_connections 2023-11-28T04:08:11.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.059+0000 7fa840c54700 1 -- 172.21.15.73:0/2206010175 wait complete. 2023-11-28T04:08:11.062 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T04:08:11.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T04:08:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.190+0000 7effaf59e700 1 Processor -- start 2023-11-28T04:08:11.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.190+0000 7effaf59e700 1 -- start start 2023-11-28T04:08:11.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaf59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7effb0066a30 0x7effb0066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaf59e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb01586a0 con 0x7effb00c6eb0 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaf59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb0157de0 con 0x7effb00c6a30 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaf59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb0158050 con 0x7effb0066a30 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7effb00c6eb0 0x7effb0063810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41128/0 (socket says 172.21.15.73:41128) 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.191+0000 7effaed9d700 1 -- 172.21.15.73:0/303007282 learned_addr learned my addr 172.21.15.73:0/303007282 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effae59c700 1 --2- 172.21.15.73:0/303007282 >> v2:172.21.15.150:3300/0 conn(0x7effb0066a30 0x7effb0066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2620522110 0 0) 0x7effb01586a0 con 0x7effb00c6eb0 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff980033f0 con 0x7effb00c6eb0 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1205634598 0 0) 0x7eff980033f0 con 0x7effb00c6eb0 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 >> v2:172.21.15.150:3300/0 conn(0x7effb0066a30 msgr2=0x7effb0066e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 --2- 172.21.15.73:0/303007282 >> v2:172.21.15.150:3300/0 conn(0x7effb0066a30 0x7effb0066e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 >> v1:172.21.15.150:6789/0 conn(0x7effb00c6a30 legacy=0x7effb013d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0158330 con 0x7effb00c6eb0 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7effa8001470 con 0x7effb00c6eb0 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 >> v1:172.21.15.73:6789/0 conn(0x7effb00c6eb0 legacy=0x7effb0063810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980038e0 0x7eff98003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 0x7eff98004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff98005000 0x7eff9800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb0158050 con 0x7eff98004690 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9800bf60 con 0x7eff98005000 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9800c0f0 con 0x7eff980038e0 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 0x7eff98004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effae59c700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980038e0 0x7eff98003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effaed9d700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff98005000 0x7eff9800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 -- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980038e0 msgr2=0x7eff98003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980038e0 0x7eff98003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 -- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff98005000 msgr2=0x7eff9800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff98005000 0x7eff9800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effaed9d700 1 -- 172.21.15.73:0/303007282 reap_dead start 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effadd9b700 1 -- 172.21.15.73:0/303007282 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980033f0 con 0x7eff98004690 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.192+0000 7effae59c700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff980038e0 0x7eff98003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effadd9b700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 0x7eff98004b10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7effa4001730 tx=0x7effa4007bb0).ready entity=mon.0 client_cookie=bd538b1a7a57948d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4007020 con 0x7eff98004690 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa4007180 con 0x7eff98004690 2023-11-28T04:08:11.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effad59a700 1 -- 172.21.15.73:0/303007282 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4007520 con 0x7eff98004690 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 -- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 msgr2=0x7eff98004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 0x7eff98004b10 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 rx=0x7effa4001730 tx=0x7effa4007bb0).stop 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 -- 172.21.15.73:0/303007282 shutdown_connections 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 --2- 172.21.15.73:0/303007282 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff98004690 0x7eff98004b10 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 -- 172.21.15.73:0/303007282 >> 172.21.15.73:0/303007282 conn(0x7effb00c8ed0 msgr2=0x7effb00c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 -- 172.21.15.73:0/303007282 shutdown_connections 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.193+0000 7effaf59e700 1 -- 172.21.15.73:0/303007282 wait complete. 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.194+0000 7effaf59e700 1 Processor -- start 2023-11-28T04:08:11.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.194+0000 7effaf59e700 1 -- start start 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff98004690 0x7effb0165430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 0x7effb00ab000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb00c4730 con 0x7effb00c4bc0 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb01685b0 con 0x7effb0167e40 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaf59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb0168890 con 0x7eff98004690 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37956/0 (socket says 172.21.15.73:37956) 2023-11-28T04:08:11.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 -- 172.21.15.73:0/782922689 learned_addr learned my addr 172.21.15.73:0/782922689 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effadd9b700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 0x7effb00ab000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 -- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff98004690 msgr2=0x7effb0165430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff98004690 0x7effb0165430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 -- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 msgr2=0x7effb00ab000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 0x7effb00ab000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effaed9d700 1 -- 172.21.15.73:0/782922689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb0168b70 con 0x7effb00c4bc0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.195+0000 7effadd9b700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 0x7effb00ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7effaed9d700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7effb0154770 tx=0x7effa8001010).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8010520 con 0x7effb00c4bc0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa8010680 con 0x7effb00c4bc0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb00c43c0 con 0x7effb00c4bc0 2023-11-28T04:08:11.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa8010a00 con 0x7effb00c4bc0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.196+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb00ab4f0 con 0x7effb00c4bc0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.197+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb00c4a10 con 0x7effb00c4bc0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.197+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7effa80133d0 con 0x7effb00c4bc0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.198+0000 7eff9e7fc700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 0x7eff90061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.198+0000 7effae59c700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 0x7eff90061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.198+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7effa8079b50 con 0x7effb00c4bc0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.198+0000 7effae59c700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 0x7eff90061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7effa0009fd0 tx=0x7effa000b040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.202+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7effa804af40 con 0x7effb00c4bc0 2023-11-28T04:08:11.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.365+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7effb00c4870 con 0x7effb00c4bc0 2023-11-28T04:08:11.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.369+0000 7eff9e7fc700 1 -- 172.21.15.73:0/782922689 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v32) v1 ==== 120+0+0 (secure 0 0 0) 0x7effb00c4870 con 0x7effb00c4bc0 2023-11-28T04:08:11.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 msgr2=0x7eff90061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 0x7eff90061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7effa0009fd0 tx=0x7effa000b040).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 msgr2=0x7effb00c5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 rx=0x7effb0154770 tx=0x7effa8001010).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 shutdown_connections 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff9005f150 0x7eff90061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7effb00c4bc0 0x7effb00c5040 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff98004690 0x7effb0165430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 --2- 172.21.15.73:0/782922689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7effb0167e40 0x7effb00ab000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 >> 172.21.15.73:0/782922689 conn(0x7effb00c8ed0 msgr2=0x7effb00c92b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.373 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 shutdown_connections 2023-11-28T04:08:11.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.371+0000 7effaf59e700 1 -- 172.21.15.73:0/782922689 wait complete. 2023-11-28T04:08:11.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T04:08:11.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.503+0000 7fd45200f700 1 Processor -- start 2023-11-28T04:08:11.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.503+0000 7fd45200f700 1 -- start start 2023-11-28T04:08:11.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd45200f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd44c158a30 0x7fd44c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd45200f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c0c72d0 con 0x7fd44c098d10 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd45200f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd44c064570 con 0x7fd44c0c6190 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd45200f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd44c154bc0 con 0x7fd44c158a30 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd44bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd44c158a30 0x7fd44c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd44bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd44c158a30 0x7fd44c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37970/0 (socket says 172.21.15.73:37970) 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd44bfff700 1 -- 172.21.15.73:0/1043020129 learned_addr learned my addr 172.21.15.73:0/1043020129 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.504+0000 7fd44bfff700 1 -- 172.21.15.73:0/1043020129 >> v1:172.21.15.150:6789/0 conn(0x7fd44c0c6190 legacy=0x7fd44c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44bfff700 1 -- 172.21.15.73:0/1043020129 >> v1:172.21.15.73:6789/0 conn(0x7fd44c098d10 legacy=0x7fd44c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44bfff700 1 -- 172.21.15.73:0/1043020129 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd44c0c66c0 con 0x7fd44c158a30 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44a7fc700 1 -- 172.21.15.73:0/1043020129 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3106948111 0 0) 0x7fd44c0c72d0 con 0x7fd44c098d10 2023-11-28T04:08:11.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44bfff700 1 --2- 172.21.15.73:0/1043020129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0c7760 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7fd444009f20 tx=0x7fd444009840).ready entity=mon.0 client_cookie=4ec97631ddb917de server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44a7fc700 1 -- 172.21.15.73:0/1043020129 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400e070 con 0x7fd44c158a30 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44a7fc700 1 -- 172.21.15.73:0/1043020129 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44400ba40 con 0x7fd44c158a30 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd44a7fc700 1 -- 172.21.15.73:0/1043020129 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44400bdc0 con 0x7fd44c158a30 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 -- 172.21.15.73:0/1043020129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 msgr2=0x7fd44c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 --2- 172.21.15.73:0/1043020129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0c7760 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 rx=0x7fd444009f20 tx=0x7fd444009840).stop 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 -- 172.21.15.73:0/1043020129 shutdown_connections 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 --2- 172.21.15.73:0/1043020129 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0c7760 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 -- 172.21.15.73:0/1043020129 >> 172.21.15.73:0/1043020129 conn(0x7fd44c0cb310 msgr2=0x7fd44c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 -- 172.21.15.73:0/1043020129 shutdown_connections 2023-11-28T04:08:11.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.505+0000 7fd45200f700 1 -- 172.21.15.73:0/1043020129 wait complete. 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 Processor -- start 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 -- start start 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44c098d10 0x7fd44c155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 0x7fd44c0ab350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c164840 con 0x7fd44c158a30 2023-11-28T04:08:11.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c1568a0 con 0x7fd44c0c6190 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd45200f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c156000 con 0x7fd44c098d10 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd44bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd44bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37974/0 (socket says 172.21.15.73:37974) 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.507+0000 7fd44bfff700 1 -- 172.21.15.73:0/4224357119 learned_addr learned my addr 172.21.15.73:0/4224357119 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 -- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44c098d10 msgr2=0x7fd44c155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44affd700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 0x7fd44c0ab350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44b7fe700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44c098d10 0x7fd44c155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44c098d10 0x7fd44c155760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 -- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 msgr2=0x7fd44c0ab350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 0x7fd44c0ab350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 -- 172.21.15.73:0/4224357119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd444009ae0 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44affd700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 0x7fd44c0ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd44bfff700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7fd444010000 tx=0x7fd44c0a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44401d470 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44401d5d0 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd44c1562e0 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44401d950 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.508+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c0ae460 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.509+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c0ae650 con 0x7fd44c158a30 2023-11-28T04:08:11.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.509+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd44401dd40 con 0x7fd44c158a30 2023-11-28T04:08:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.510+0000 7fd4337fe700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 0x7fd4380615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.510+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fd44407c2c0 con 0x7fd44c158a30 2023-11-28T04:08:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.510+0000 7fd44b7fe700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 0x7fd4380615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.510+0000 7fd44b7fe700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 0x7fd4380615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd43c0011e0 tx=0x7fd43c009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.514+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd44404d710 con 0x7fd44c158a30 2023-11-28T04:08:11.683 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.682+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fd44c0ae650 con 0x7fd44c158a30 2023-11-28T04:08:11.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.685+0000 7fd4337fe700 1 -- 172.21.15.73:0/4224357119 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fd444050d30 con 0x7fd44c158a30 2023-11-28T04:08:11.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 msgr2=0x7fd4380615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 0x7fd4380615e0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd43c0011e0 tx=0x7fd43c009040).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 msgr2=0x7fd44c0ab840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 rx=0x7fd444010000 tx=0x7fd44c0a8040).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 shutdown_connections 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fd43805f120 0x7fd4380615e0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd44c158a30 0x7fd44c0ab840 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd44c098d10 0x7fd44c155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 --2- 172.21.15.73:0/4224357119 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd44c0c6190 0x7fd44c0ab350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 >> 172.21.15.73:0/4224357119 conn(0x7fd44c0cb310 msgr2=0x7fd44c0caa70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 shutdown_connections 2023-11-28T04:08:11.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.687+0000 7fd45200f700 1 -- 172.21.15.73:0/4224357119 wait complete. 2023-11-28T04:08:11.700 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-11-28T04:08:11.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.813+0000 7eff3a82c700 1 Processor -- start 2023-11-28T04:08:11.814 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.813+0000 7eff3a82c700 1 -- start start 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff340be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7eff34154500 0x7eff340bea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff340c3fd0 0x7eff340c4400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff340bfc00 con 0x7eff340c3fd0 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff340c48f0 con 0x7eff34154500 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a82c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340beff0 con 0x7eff34154120 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a02b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff340c3fd0 0x7eff340c4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a02b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff340c3fd0 0x7eff340c4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37980/0 (socket says 172.21.15.73:37980) 2023-11-28T04:08:11.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3a02b700 1 -- 172.21.15.73:0/1217237918 learned_addr learned my addr 172.21.15.73:0/1217237918 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff3982a700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff340be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.814+0000 7eff39029700 1 --2- 172.21.15.73:0/1217237918 >> v2:172.21.15.150:3300/0 conn(0x7eff34154500 0x7eff340bea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 -- 172.21.15.73:0/1217237918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 msgr2=0x7eff340be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff340be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 -- 172.21.15.73:0/1217237918 >> v2:172.21.15.150:3300/0 conn(0x7eff34154500 msgr2=0x7eff340bea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 --2- 172.21.15.73:0/1217237918 >> v2:172.21.15.150:3300/0 conn(0x7eff34154500 0x7eff340bea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 -- 172.21.15.73:0/1217237918 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff340bf2d0 con 0x7eff340c3fd0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a02b700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff340c4400 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7eff34083a60 tx=0x7eff2c009840).ready entity=mon.0 client_cookie=69d8f4649dc7c75e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff38828700 1 -- 172.21.15.73:0/1217237918 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00e070 con 0x7eff340c3fd0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff38828700 1 -- 172.21.15.73:0/1217237918 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff2c00ba50 con 0x7eff340c3fd0 2023-11-28T04:08:11.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff38828700 1 -- 172.21.15.73:0/1217237918 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff2c00bdd0 con 0x7eff340c3fd0 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 -- 172.21.15.73:0/1217237918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 msgr2=0x7eff340c4400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff340c4400 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 rx=0x7eff34083a60 tx=0x7eff2c009840).stop 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 -- 172.21.15.73:0/1217237918 shutdown_connections 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff340c4400 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 --2- 172.21.15.73:0/1217237918 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff340be510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 --2- 172.21.15.73:0/1217237918 >> v2:172.21.15.150:3300/0 conn(0x7eff34154500 0x7eff340bea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 -- 172.21.15.73:0/1217237918 >> 172.21.15.73:0/1217237918 conn(0x7eff34083000 msgr2=0x7eff34092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 -- 172.21.15.73:0/1217237918 shutdown_connections 2023-11-28T04:08:11.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.815+0000 7eff3a82c700 1 -- 172.21.15.73:0/1217237918 wait complete. 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.816+0000 7eff3a82c700 1 Processor -- start 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 -- start start 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff341645e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff34154500 0x7eff34164ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340a8d10 con 0x7eff340c3fd0 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff340a8ff0 con 0x7eff34154500 2023-11-28T04:08:11.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a82c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff340a92d0 con 0x7eff34154120 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3982a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3982a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37994/0 (socket says 172.21.15.73:37994) 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3982a700 1 -- 172.21.15.73:0/2116905549 learned_addr learned my addr 172.21.15.73:0/2116905549 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff3a02b700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff34154500 0x7eff34164ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.817+0000 7eff39029700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff341645e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 -- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 msgr2=0x7eff341645e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff341645e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 -- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff34154500 msgr2=0x7eff34164ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff34154500 0x7eff34164ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 -- 172.21.15.73:0/2116905549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff2c009ae0 con 0x7eff340c3fd0 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3982a700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7eff3000d860 tx=0x7eff30008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30011590 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff300116f0 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff340a95b0 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff30011a70 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.818+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff340a9890 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.819+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff3406c7c0 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.819+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff3001c460 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.820+0000 7eff29ffb700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 0x7eff18061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.820+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7eff3007a0b0 con 0x7eff340c3fd0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.820+0000 7eff39029700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 0x7eff18061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:11.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.820+0000 7eff39029700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 0x7eff18061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7eff34096700 tx=0x7eff24008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:11.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:11.824+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff3004b4a0 con 0x7eff340c3fd0 2023-11-28T04:08:12.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:12.054+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7eff340bf5b0 con 0x7eff340c3fd0 2023-11-28T04:08:12.680 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:08:12.683 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:08:13.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.060+0000 7eff29ffb700 1 -- 172.21.15.73:0/2116905549 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2424) v1 ==== 174+0+0 (secure 0 0 0) 0x7eff30011e60 con 0x7eff340c3fd0 2023-11-28T04:08:13.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 msgr2=0x7eff18061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 0x7eff18061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7eff34096700 tx=0x7eff24008040).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 msgr2=0x7eff341524f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 rx=0x7eff3000d860 tx=0x7eff30008040).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 shutdown_connections 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7eff1805f150 0x7eff18061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff34154500 0x7eff34164ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff340c3fd0 0x7eff341524f0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 --2- 172.21.15.73:0/2116905549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff34154120 0x7eff341645e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 >> 172.21.15.73:0/2116905549 conn(0x7eff34083000 msgr2=0x7eff340640b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 shutdown_connections 2023-11-28T04:08:13.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.062+0000 7eff3a82c700 1 -- 172.21.15.73:0/2116905549 wait complete. 2023-11-28T04:08:13.075 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-11-28T04:08:13.075 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-11-28T04:08:13.075 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T04:08:13.076 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-11-28T04:08:13.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.188+0000 7fa6e4b85700 1 Processor -- start 2023-11-28T04:08:13.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.188+0000 7fa6e4b85700 1 -- start start 2023-11-28T04:08:13.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6e4b85700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa6e0154c00 0x7fa6e0069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6e4b85700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e006a270 con 0x7fa6e0150890 2023-11-28T04:08:13.192 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6e4b85700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6e00c28c0 con 0x7fa6e0069900 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6e4b85700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6e01552e0 con 0x7fa6e0154c00 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6de59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa6e0150890 0x7fa6e00bbe30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:41146/0 (socket says 172.21.15.73:41146) 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6de59c700 1 -- 172.21.15.73:0/2880108865 learned_addr learned my addr 172.21.15.73:0/2880108865 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/2880108865 >> v2:172.21.15.150:3300/0 conn(0x7fa6e0154c00 0x7fa6e0069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3175264929 0 0) 0x7fa6d0001110 con 0x7fa6e0150890 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6cc0033f0 con 0x7fa6e0150890 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 567948683 0 0) 0x7fa6cc0033f0 con 0x7fa6e0150890 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 >> v2:172.21.15.150:3300/0 conn(0x7fa6e0154c00 msgr2=0x7fa6e0069d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 --2- 172.21.15.73:0/2880108865 >> v2:172.21.15.150:3300/0 conn(0x7fa6e0154c00 0x7fa6e0069d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 >> v1:172.21.15.150:6789/0 conn(0x7fa6e0069900 legacy=0x7fa6e01544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00bc540 con 0x7fa6e0150890 2023-11-28T04:08:13.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fa6d0001600 con 0x7fa6e0150890 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 >> v1:172.21.15.73:6789/0 conn(0x7fa6e0150890 legacy=0x7fa6e00bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6cc0038e0 0x7fa6cc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6cc004690 0x7fa6cc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6cc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e01552e0 con 0x7fa6cc005000 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc00beb0 con 0x7fa6cc004690 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6cc00d180 con 0x7fa6cc0038e0 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6cc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.189+0000 7fa6d5d9b700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6cc004690 0x7fa6cc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.194 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6de59c700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6cc0038e0 0x7fa6cc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 -- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6cc0038e0 msgr2=0x7fa6cc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6cc0038e0 0x7fa6cc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 -- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6cc004690 msgr2=0x7fa6cc004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6cc004690 0x7fa6cc004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 -- 172.21.15.73:0/2880108865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6cc0033f0 con 0x7fa6cc005000 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 -- 172.21.15.73:0/2880108865 reap_dead start 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6cc00b8e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fa6e00c1330 tx=0x7fa6d800b040).ready entity=mon.0 client_cookie=a36a82dd3dc39fb2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d8009ac0 con 0x7fa6cc005000 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d800f040 con 0x7fa6cc005000 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6ddd9b700 1 -- 172.21.15.73:0/2880108865 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d800ec50 con 0x7fa6cc005000 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 -- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 msgr2=0x7fa6cc00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6cc00b8e0 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 rx=0x7fa6e00c1330 tx=0x7fa6d800b040).stop 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 -- 172.21.15.73:0/2880108865 shutdown_connections 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/2880108865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6cc00b8e0 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 -- 172.21.15.73:0/2880108865 >> 172.21.15.73:0/2880108865 conn(0x7fa6e00631d0 msgr2=0x7fa6e00c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 -- 172.21.15.73:0/2880108865 shutdown_connections 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.190+0000 7fa6e4b85700 1 -- 172.21.15.73:0/2880108865 wait complete. 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 Processor -- start 2023-11-28T04:08:13.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 -- start start 2023-11-28T04:08:13.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6e00c0610 0x7fa6e014c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 0x7fa6e0094100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00945f0 con 0x7fa6cc005000 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00c0d80 con 0x7fa6e0093c80 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.192+0000 7fa6e4b85700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6e00c51d0 con 0x7fa6e00c0610 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38002/0 (socket says 172.21.15.73:38002) 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 -- 172.21.15.73:0/728089816 learned_addr learned my addr 172.21.15.73:0/728089816 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 0x7fa6e0094100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6d5d9b700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6e00c0610 0x7fa6e014c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 -- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6e00c0610 msgr2=0x7fa6e014c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6e00c0610 0x7fa6e014c9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 -- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 msgr2=0x7fa6e0094100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 0x7fa6e0094100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 -- 172.21.15.73:0/728089816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e00c73a0 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6ded9d700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 0x7fa6e0094100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6de59c700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fa6d000e7b0 tx=0x7fa6d0001010).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0010490 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d0010040 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6e01540d0 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0010820 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.193+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6e00c0aa0 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.194+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6e006c860 con 0x7fa6cc005000 2023-11-28T04:08:13.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.194+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa6d0010c10 con 0x7fa6cc005000 2023-11-28T04:08:13.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.195+0000 7fa6d6ffd700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 0x7fa6c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:13.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.195+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fa6d00796d0 con 0x7fa6cc005000 2023-11-28T04:08:13.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.195+0000 7fa6d5d9b700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 0x7fa6c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:13.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.195+0000 7fa6d5d9b700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 0x7fa6c0061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa6c8000f60 tx=0x7fa6c8009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:13.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.199+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa6d004ab20 con 0x7fa6cc005000 2023-11-28T04:08:13.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:13.427+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7fa6e006a270 con 0x7fa6cc005000 2023-11-28T04:08:14.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.067+0000 7fa6d6ffd700 1 -- 172.21.15.73:0/728089816 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2425) v1 ==== 168+0+0 (secure 0 0 0) 0x7fa6e006a270 con 0x7fa6cc005000 2023-11-28T04:08:14.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 msgr2=0x7fa6c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 0x7fa6c0061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7fa6c8000f60 tx=0x7fa6c8009040).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 msgr2=0x7fa6e00c4ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 rx=0x7fa6d000e7b0 tx=0x7fa6d0001010).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 shutdown_connections 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa6c005f150 0x7fa6c0061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa6e0093c80 0x7fa6e0094100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa6cc005000 0x7fa6e00c4ce0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 --2- 172.21.15.73:0/728089816 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa6e00c0610 0x7fa6e014c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 >> 172.21.15.73:0/728089816 conn(0x7fa6e00631d0 msgr2=0x7fa6e00645e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 shutdown_connections 2023-11-28T04:08:14.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.069+0000 7fa6e4b85700 1 -- 172.21.15.73:0/728089816 wait complete. 2023-11-28T04:08:14.083 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T04:08:14.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.199+0000 7f5e4259f700 1 Processor -- start 2023-11-28T04:08:14.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.199+0000 7f5e4259f700 1 -- start start 2023-11-28T04:08:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4259f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c14e530 0x7f5e3c0c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4259f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5e3c158570 0x7f5e3c158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4259f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e3c0c7bc0 con 0x7f5e3c154950 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4259f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c155920 con 0x7f5e3c14e530 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4259f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e3c154e00 con 0x7f5e3c158570 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c14e530 0x7f5e3c0c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5e3c14e530 0x7f5e3c0c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40084/0 (socket says 172.21.15.73:40084) 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 -- 172.21.15.73:0/250379477 learned_addr learned my addr 172.21.15.73:0/250379477 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e41d9e700 1 --2- 172.21.15.73:0/250379477 >> v2:172.21.15.150:3300/0 conn(0x7f5e3c158570 0x7f5e3c158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 -- 172.21.15.73:0/250379477 >> v2:172.21.15.150:3300/0 conn(0x7f5e3c158570 msgr2=0x7f5e3c158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 --2- 172.21.15.73:0/250379477 >> v2:172.21.15.150:3300/0 conn(0x7f5e3c158570 0x7f5e3c158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 -- 172.21.15.73:0/250379477 >> v1:172.21.15.150:6789/0 conn(0x7f5e3c154950 legacy=0x7f5e3c14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 -- 172.21.15.73:0/250379477 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e3c155080 con 0x7f5e3c14e530 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.200+0000 7f5e4159d700 1 --2- 172.21.15.73:0/250379477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c14e530 0x7f5e3c0c66f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f5e3c0aac80 tx=0x7f5e380090f0).ready entity=mon.0 client_cookie=54b216f41d3adc01 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e33fff700 1 -- 172.21.15.73:0/250379477 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800b600 con 0x7f5e3c14e530 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e33fff700 1 -- 172.21.15.73:0/250379477 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e3800b760 con 0x7f5e3c14e530 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e33fff700 1 -- 172.21.15.73:0/250379477 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e3800bae0 con 0x7f5e3c14e530 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 -- 172.21.15.73:0/250379477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c14e530 msgr2=0x7f5e3c0c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 --2- 172.21.15.73:0/250379477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c14e530 0x7f5e3c0c66f0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 rx=0x7f5e3c0aac80 tx=0x7f5e380090f0).stop 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 -- 172.21.15.73:0/250379477 shutdown_connections 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 --2- 172.21.15.73:0/250379477 >> v2:172.21.15.150:3300/0 conn(0x7f5e3c158570 0x7f5e3c158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 --2- 172.21.15.73:0/250379477 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c14e530 0x7f5e3c0c66f0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 -- 172.21.15.73:0/250379477 >> 172.21.15.73:0/250379477 conn(0x7f5e3c064990 msgr2=0x7f5e3c064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 -- 172.21.15.73:0/250379477 shutdown_connections 2023-11-28T04:08:14.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.201+0000 7f5e4259f700 1 -- 172.21.15.73:0/250379477 wait complete. 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.202+0000 7f5e4259f700 1 Processor -- start 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.202+0000 7f5e4259f700 1 -- start start 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c14e530 0x7f5e3c168910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c158570 0x7f5e3c0c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c0b0b80 con 0x7f5e3c154950 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c168e00 con 0x7f5e3c158570 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4259f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3c1537c0 con 0x7f5e3c14e530 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e40d9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e40d9c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40094/0 (socket says 172.21.15.73:40094) 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e40d9c700 1 -- 172.21.15.73:0/1733561415 learned_addr learned my addr 172.21.15.73:0/1733561415 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:14.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e4159d700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c14e530 0x7f5e3c168910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.203+0000 7f5e41d9e700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c158570 0x7f5e3c0c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 -- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c14e530 msgr2=0x7f5e3c168910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c14e530 0x7f5e3c168910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 -- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c158570 msgr2=0x7f5e3c0c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c158570 0x7f5e3c0c3d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 -- 172.21.15.73:0/1733561415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e38009370 con 0x7f5e3c154950 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e40d9c700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f5e2c00c4c0 tx=0x7f5e2c007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c00ce70 con 0x7f5e3c154950 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e2c001e90 con 0x7f5e3c154950 2023-11-28T04:08:14.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3c0b08a0 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e2c007250 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.204+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3c0c2620 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.205+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e3c062ee0 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.205+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5e2c0066a0 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.206+0000 7f5e317fa700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 0x7f5e20061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.206+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f5e2c079930 con 0x7f5e3c154950 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.206+0000 7f5e4159d700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 0x7f5e20061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.206+0000 7f5e4159d700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 0x7f5e20061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5e38009130 tx=0x7f5e38017000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.210+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5e2c04ad80 con 0x7f5e3c154950 2023-11-28T04:08:14.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.373+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5e3c0c2100 con 0x7f5e3c154950 2023-11-28T04:08:14.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.379+0000 7f5e317fa700 1 -- 172.21.15.73:0/1733561415 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5e2c04e3a0 con 0x7f5e3c154950 2023-11-28T04:08:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 msgr2=0x7f5e20061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 0x7f5e20061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f5e38009130 tx=0x7f5e38017000).stop 2023-11-28T04:08:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 msgr2=0x7f5e3c0ae030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 rx=0x7f5e2c00c4c0 tx=0x7f5e2c007e00).stop 2023-11-28T04:08:14.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 shutdown_connections 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5e2005f150 0x7f5e20061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5e3c158570 0x7f5e3c0c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5e3c14e530 0x7f5e3c168910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 --2- 172.21.15.73:0/1733561415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5e3c154950 0x7f5e3c0ae030 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 >> 172.21.15.73:0/1733561415 conn(0x7f5e3c064990 msgr2=0x7f5e3c14ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:14.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 shutdown_connections 2023-11-28T04:08:14.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.381+0000 7f5e4259f700 1 -- 172.21.15.73:0/1733561415 wait complete. 2023-11-28T04:08:14.395 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:08:14.395 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:08:14.440 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:08:14.490 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:08:14.490 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T04:08:14.490 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:14.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.602+0000 7fec0da28700 1 Processor -- start 2023-11-28T04:08:14.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.602+0000 7fec0da28700 1 -- start start 2023-11-28T04:08:14.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec0da28700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec08064e20 0x7fec08155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec0da28700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec080683d0 con 0x7fec081544d0 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec0da28700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec08067850 con 0x7fec08095e80 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec0da28700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec08067650 con 0x7fec08064e20 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec067fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec08064e20 0x7fec08155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec067fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec08064e20 0x7fec08155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40100/0 (socket says 172.21.15.73:40100) 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.603+0000 7fec067fc700 1 -- 172.21.15.73:0/1757969417 learned_addr learned my addr 172.21.15.73:0/1757969417 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec067fc700 1 -- 172.21.15.73:0/1757969417 >> v1:172.21.15.150:6789/0 conn(0x7fec08095e80 legacy=0x7fec0815e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec067fc700 1 -- 172.21.15.73:0/1757969417 >> v1:172.21.15.73:6789/0 conn(0x7fec081544d0 legacy=0x7fec08066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec05ffb700 1 -- 172.21.15.73:0/1757969417 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2069907581 0 0) 0x7fec080683d0 con 0x7fec081544d0 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec067fc700 1 -- 172.21.15.73:0/1757969417 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec08067b30 con 0x7fec08064e20 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec067fc700 1 --2- 172.21.15.73:0/1757969417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08155750 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7febf0006fd0 tx=0x7febf00090f0).ready entity=mon.0 client_cookie=a4838110d9117635 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec05ffb700 1 -- 172.21.15.73:0/1757969417 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf000b600 con 0x7fec08064e20 2023-11-28T04:08:14.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec05ffb700 1 -- 172.21.15.73:0/1757969417 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf000b760 con 0x7fec08064e20 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec05ffb700 1 -- 172.21.15.73:0/1757969417 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf000bae0 con 0x7fec08064e20 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 -- 172.21.15.73:0/1757969417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 msgr2=0x7fec08155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 --2- 172.21.15.73:0/1757969417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08155750 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 rx=0x7febf0006fd0 tx=0x7febf00090f0).stop 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 -- 172.21.15.73:0/1757969417 shutdown_connections 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 --2- 172.21.15.73:0/1757969417 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08155750 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 -- 172.21.15.73:0/1757969417 >> 172.21.15.73:0/1757969417 conn(0x7fec080631d0 msgr2=0x7fec080c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 -- 172.21.15.73:0/1757969417 shutdown_connections 2023-11-28T04:08:14.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.604+0000 7fec0da28700 1 -- 172.21.15.73:0/1757969417 wait complete. 2023-11-28T04:08:14.609 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 Processor -- start 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 -- start start 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec08095e80 0x7fec080a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec081544d0 0x7fec0814d550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080a9760 con 0x7fec08064e20 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec080a9a40 con 0x7fec08095e80 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.606+0000 7fec0da28700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec080a9d20 con 0x7fec081544d0 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40108/0 (socket says 172.21.15.73:40108) 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 -- 172.21.15.73:0/445963193 learned_addr learned my addr 172.21.15.73:0/445963193 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec067fc700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec08095e80 0x7fec080a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec077fe700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec081544d0 0x7fec0814d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 -- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec081544d0 msgr2=0x7fec0814d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec081544d0 0x7fec0814d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 -- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec08095e80 msgr2=0x7fec080a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec08095e80 0x7fec080a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 -- 172.21.15.73:0/445963193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febf0009370 con 0x7fec08064e20 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec06ffd700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7febf800c810 tx=0x7febf8001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800e4a0 con 0x7fec08064e20 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febf800e600 con 0x7fec08064e20 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec080aa000 con 0x7fec08064e20 2023-11-28T04:08:14.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febf800e9c0 con 0x7fec08064e20 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.607+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec08167ac0 con 0x7fec08064e20 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.608+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec080aa520 con 0x7fec08064e20 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.608+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7febf8003070 con 0x7fec08064e20 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.609+0000 7febfeffd700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 0x7febe8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.609+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5742+0+0 (secure 0 0 0) 0x7febf8079660 con 0x7fec08064e20 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.609+0000 7fec067fc700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 0x7febe8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:14.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.609+0000 7fec067fc700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 0x7febe8061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7febf0009130 tx=0x7febf0017000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:14.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.613+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7febf8048af0 con 0x7fec08064e20 2023-11-28T04:08:14.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.816+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fec081658c0 con 0x7fec08064e20 2023-11-28T04:08:14.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.816+0000 7febfeffd700 1 -- 172.21.15.73:0/445963193 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 60 v60) v1 ==== 94+0+3535 (secure 0 0 0) 0x7fec080aa520 con 0x7fec08064e20 2023-11-28T04:08:14.817 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:14.817 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":60,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6887,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/2404646484","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":2404646484},{"type":"v1","addr":"172.21.15.150:6834","nonce":2404646484}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":7810,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/2905490382","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":2905490382},{"type":"v1","addr":"172.21.15.150:6835","nonce":2905490382}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8559,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2631367693","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2631367693},{"type":"v1","addr":"172.21.15.73:6836","nonce":2631367693}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":55},{"gid":8913,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2129690597","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2129690597},{"type":"v1","addr":"172.21.15.73:6837","nonce":2129690597}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":59}],"filesystems":[]} 2023-11-28T04:08:14.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 msgr2=0x7febe8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 0x7febe8061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7febf0009130 tx=0x7febf0017000).stop 2023-11-28T04:08:14.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 msgr2=0x7fec08164840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:14.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 rx=0x7febf800c810 tx=0x7febf8001510).stop 2023-11-28T04:08:14.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 shutdown_connections 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7febe805f150 0x7febe8061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec081544d0 0x7fec0814d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec08064e20 0x7fec08164840 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 --2- 172.21.15.73:0/445963193 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec08095e80 0x7fec080a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 >> 172.21.15.73:0/445963193 conn(0x7fec080631d0 msgr2=0x7fec080c90f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 shutdown_connections 2023-11-28T04:08:14.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:14.818+0000 7fec0da28700 1 -- 172.21.15.73:0/445963193 wait complete. 2023-11-28T04:08:14.822 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 60 2023-11-28T04:08:14.831 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T04:08:14.831 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T04:08:14.832 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T04:08:14.832 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:08:14.832 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T04:08:14.832 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T04:08:14.832 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T04:08:14.832 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:08:14.832 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T04:08:14.833 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T04:08:14.833 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T04:08:14.833 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:08:14.833 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T04:08:14.833 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T04:08:14.833 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T04:08:14.833 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:08:14.842 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T04:08:14.842 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T04:08:14.843 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T04:08:14.843 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T04:08:14.844 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T04:08:14.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T04:08:14.845 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T04:08:14.845 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T04:08:14.885 INFO:tasks.ceph.mds.a:Started 2023-11-28T04:08:14.886 DEBUG:teuthology.parallel:result is None 2023-11-28T04:08:14.887 INFO:tasks.ceph.mds.c:Started 2023-11-28T04:08:14.887 DEBUG:teuthology.parallel:result is None 2023-11-28T04:08:14.888 INFO:tasks.ceph.mds.d:Started 2023-11-28T04:08:14.888 DEBUG:teuthology.parallel:result is None 2023-11-28T04:08:14.888 INFO:tasks.ceph.mds.b:Started 2023-11-28T04:08:14.889 DEBUG:teuthology.parallel:result is None 2023-11-28T04:08:14.889 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T04:08:14.958 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T04:08:14.958 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:08:14.974 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T04:08:14.975 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T04:08:14.975 INFO:tasks.ceph.mds.d.smithi150.stderr:2023-11-28T04:08:14.974+0000 7f09b4dca700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-11-28T04:08:15.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 Processor -- start 2023-11-28T04:08:15.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 -- start start 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f60b40bd6b0 0x7f60b4150d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b4062910 con 0x7f60b4150930 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60b40c6c00 con 0x7f60b40c6620 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.010+0000 7f60bacb0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60b40c2460 con 0x7f60b40bd6b0 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b8a4c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f60b4150930 0x7f60b40a6ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44222/0 (socket says 172.21.15.73:44222) 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b8a4c700 1 -- 172.21.15.73:0/3499229815 learned_addr learned my addr 172.21.15.73:0/3499229815 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> v2:172.21.15.150:3300/0 conn(0x7f60b40bd6b0 0x7f60b4150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2913461980 0 0) 0x7f60b4062910 con 0x7f60b4150930 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f60a40033f0 con 0x7f60b4150930 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 995915248 0 0) 0x7f60a40033f0 con 0x7f60b4150930 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 >> v2:172.21.15.150:3300/0 conn(0x7f60b40bd6b0 msgr2=0x7f60b4150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 --2- 172.21.15.73:0/3499229815 >> v2:172.21.15.150:3300/0 conn(0x7f60b40bd6b0 0x7f60b4150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 >> v1:172.21.15.150:6789/0 conn(0x7f60b40c6620 legacy=0x7f60b40bcfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b40c6ee0 con 0x7f60b4150930 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f60a80015f0 con 0x7f60b4150930 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> v2:172.21.15.150:3300/0 conn(0x7f60b40bd6b0 0x7f60b4150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 >> v1:172.21.15.73:6789/0 conn(0x7f60b4150930 legacy=0x7f60b40a6ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a40038e0 0x7f60a4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a4004690 0x7f60a4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60a400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40c2460 con 0x7f60a4005050 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60a400bfe0 con 0x7f60a40038e0 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60a400c1a0 con 0x7f60a4004690 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60a400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b8a4c700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a40038e0 0x7f60a4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.011+0000 7f60b3fff700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a4004690 0x7f60a4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 -- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a4004690 msgr2=0x7f60a4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a4004690 0x7f60a4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 -- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a40038e0 msgr2=0x7f60a4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60a40038e0 0x7f60a4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 -- 172.21.15.73:0/3499229815 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60a40033f0 con 0x7f60a4005050 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b3fff700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60a4004690 0x7f60a4004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:15.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 -- 172.21.15.73:0/3499229815 reap_dead start 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b924d700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60a400b8e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7f60ac00a5e0 tx=0x7f60ac00c040).ready entity=mon.0 client_cookie=a6d327eb4679833d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00fc90 con 0x7f60a4005050 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60ac010040 con 0x7f60a4005050 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60b37fe700 1 -- 172.21.15.73:0/3499229815 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60ac00a0d0 con 0x7f60a4005050 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 -- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 msgr2=0x7f60a400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60a400b8e0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 rx=0x7f60ac00a5e0 tx=0x7f60ac00c040).stop 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 -- 172.21.15.73:0/3499229815 shutdown_connections 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 --2- 172.21.15.73:0/3499229815 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60a400b8e0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 -- 172.21.15.73:0/3499229815 >> 172.21.15.73:0/3499229815 conn(0x7f60b40c8230 msgr2=0x7f60b40c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:15.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.012+0000 7f60bacb0700 1 -- 172.21.15.73:0/3499229815 shutdown_connections 2023-11-28T04:08:15.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.013+0000 7f60bacb0700 1 -- 172.21.15.73:0/3499229815 wait complete. 2023-11-28T04:08:15.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.014+0000 7f60bacb0700 1 Processor -- start 2023-11-28T04:08:15.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.014+0000 7f60bacb0700 1 -- start start 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60b414e770 0x7f60b414ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60b4155120 0x7f60b41555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b40aa940 con 0x7f60a4005050 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60b414fad0 con 0x7f60b4155120 2023-11-28T04:08:15.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60b4155a90 con 0x7f60b414e770 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40144/0 (socket says 172.21.15.73:40144) 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 -- 172.21.15.73:0/1520109968 learned_addr learned my addr 172.21.15.73:0/1520109968 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b3fff700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60b414e770 0x7f60b414ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b924d700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60b4155120 0x7f60b41555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 -- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60b414e770 msgr2=0x7f60b414ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60b414e770 0x7f60b414ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 -- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60b4155120 msgr2=0x7f60b41555a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60b4155120 0x7f60b41555a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 -- 172.21.15.73:0/1520109968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60b4143ab0 con 0x7f60a4005050 2023-11-28T04:08:15.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b8a4c700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f60a800f880 tx=0x7f60a8009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a80070d0 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f60a8007230 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60b414f230 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60a8005880 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.015+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60b414f7f0 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.016+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60b406c860 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.017+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f60a8005c30 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.017+0000 7f60b0ff9700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 0x7f6098061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.017+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f60a8079790 con 0x7f60a4005050 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.017+0000 7f60b3fff700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 0x7f6098061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:15.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.017+0000 7f60b3fff700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 0x7f6098061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f60a0007fd0 tx=0x7f60a000c040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:15.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.021+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f60a8049af0 con 0x7f60a4005050 2023-11-28T04:08:15.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:15.200+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f60b40a6990 con 0x7f60a4005050 2023-11-28T04:08:16.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.075+0000 7f60b0ff9700 1 -- 172.21.15.73:0/1520109968 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v175) v1 ==== 97+0+0 (secure 0 0 0) 0x7f60a804ac30 con 0x7f60a4005050 2023-11-28T04:08:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 msgr2=0x7f6098061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 0x7f6098061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f60a0007fd0 tx=0x7f60a000c040).stop 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 msgr2=0x7f60b41435c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 rx=0x7f60a800f880 tx=0x7f60a8009040).stop 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 shutdown_connections 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f609805f150 0x7f6098061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60b4155120 0x7f60b41555a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f60a4005050 0x7f60b41435c0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 --2- 172.21.15.73:0/1520109968 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60b414e770 0x7f60b414ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 >> 172.21.15.73:0/1520109968 conn(0x7f60b40c8230 msgr2=0x7f60b40c35f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 shutdown_connections 2023-11-28T04:08:16.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.077+0000 7f60bacb0700 1 -- 172.21.15.73:0/1520109968 wait complete. 2023-11-28T04:08:16.080 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T04:08:16.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T04:08:16.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.205+0000 7f51ca66e700 1 Processor -- start 2023-11-28T04:08:16.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.205+0000 7f51ca66e700 1 -- start start 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51ca66e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 0x7f51c4157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51ca66e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 0x7f51c40c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51ca66e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51c40c12c0 con 0x7f51c40c0180 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51ca66e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51c40c06b0 con 0x7f51c4157660 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51ca66e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40c0990 con 0x7f51c40c1750 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c9e6d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 0x7f51c40c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c966c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 0x7f51c4157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c8e6b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f51c40c0180 0x7f51c41589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39520/0 (socket says 172.21.15.73:39520) 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c8e6b700 1 -- 172.21.15.73:0/2664120886 learned_addr learned my addr 172.21.15.73:0/2664120886 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51bbfff700 1 -- 172.21.15.73:0/2664120886 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4223771754 0 0) 0x7f51c40c12c0 con 0x7f51c40c0180 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51bbfff700 1 -- 172.21.15.73:0/2664120886 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b00033f0 con 0x7f51c40c0180 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c9e6d700 1 -- 172.21.15.73:0/2664120886 >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 msgr2=0x7f51c4157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:16.209 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c9e6d700 1 --2- 172.21.15.73:0/2664120886 >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 0x7f51c4157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c9e6d700 1 -- 172.21.15.73:0/2664120886 >> v1:172.21.15.150:6789/0 conn(0x7f51c40c0180 legacy=0x7f51c41589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c9e6d700 1 -- 172.21.15.73:0/2664120886 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c40c0c70 con 0x7f51c40c1750 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.206+0000 7f51c966c700 1 --2- 172.21.15.73:0/2664120886 >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 0x7f51c4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51c9e6d700 1 --2- 172.21.15.73:0/2664120886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 0x7f51c40c1bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f51bc009e60 tx=0x7f51bc009780).ready entity=mon.2 client_cookie=b6234fcbb80d3cfd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51bbfff700 1 -- 172.21.15.73:0/2664120886 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00e070 con 0x7f51c40c1750 2023-11-28T04:08:16.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51bbfff700 1 -- 172.21.15.73:0/2664120886 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51bc00bab0 con 0x7f51c40c1750 2023-11-28T04:08:16.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51bbfff700 1 -- 172.21.15.73:0/2664120886 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51bc00be30 con 0x7f51c40c1750 2023-11-28T04:08:16.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 -- 172.21.15.73:0/2664120886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 msgr2=0x7f51c40c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 --2- 172.21.15.73:0/2664120886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 0x7f51c40c1bc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 rx=0x7f51bc009e60 tx=0x7f51bc009780).stop 2023-11-28T04:08:16.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 -- 172.21.15.73:0/2664120886 shutdown_connections 2023-11-28T04:08:16.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 --2- 172.21.15.73:0/2664120886 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c1750 0x7f51c40c1bc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 --2- 172.21.15.73:0/2664120886 >> v2:172.21.15.150:3300/0 conn(0x7f51c4157660 0x7f51c4157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 -- 172.21.15.73:0/2664120886 >> 172.21.15.73:0/2664120886 conn(0x7f51c40cb000 msgr2=0x7f51c40cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 -- 172.21.15.73:0/2664120886 shutdown_connections 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.207+0000 7f51ca66e700 1 -- 172.21.15.73:0/2664120886 wait complete. 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.208+0000 7f51ca66e700 1 Processor -- start 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.208+0000 7f51ca66e700 1 -- start start 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c0180 0x7f51c40b0b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4157660 0x7f51c4155d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c4155900 con 0x7f51c40c1750 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40b0ff0 con 0x7f51c4157660 2023-11-28T04:08:16.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51ca66e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51c40aa9b0 con 0x7f51c40c0180 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51c8e6b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51c8e6b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40158/0 (socket says 172.21.15.73:40158) 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51c8e6b700 1 -- 172.21.15.73:0/307756167 learned_addr learned my addr 172.21.15.73:0/307756167 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.209+0000 7f51c966c700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c0180 0x7f51c40b0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c9e6d700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4157660 0x7f51c4155d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 -- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c0180 msgr2=0x7f51c40b0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c0180 0x7f51c40b0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 -- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4157660 msgr2=0x7f51c4155d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4157660 0x7f51c4155d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 -- 172.21.15.73:0/307756167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51c00092a0 con 0x7f51c40c1750 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51c8e6b700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f51b4001ed0 tx=0x7f51b4007c70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b400ce10 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b40072e0 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51bc009a20 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b40057a0 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.210+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51c4154fd0 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.211+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51c40c5980 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.211+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f51b4005b50 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.212+0000 7f51b97fa700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 0x7f51a8061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.212+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f51b4079460 con 0x7f51c40c1750 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.212+0000 7f51c966c700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 0x7f51a8061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.212+0000 7f51c966c700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 0x7f51a8061640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f51c0009d20 tx=0x7f51c0009060).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.216+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51b404aa80 con 0x7f51c40c1750 2023-11-28T04:08:16.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.441+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f51c406c7c0 con 0x7f51c40c1750 2023-11-28T04:08:16.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.442+0000 7f51b97fa700 1 -- 172.21.15.73:0/307756167 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v38) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f51b4018070 con 0x7f51c40c1750 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:08:16.443 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.444 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T04:08:16.445 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.446 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.447 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T04:08:16.448 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.449 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T04:08:16.450 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:08:16.451 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.452 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.453 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:08:16.454 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.455 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T04:08:16.456 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T04:08:16.457 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:08:16.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 msgr2=0x7f51a8061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 0x7f51a8061640 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f51c0009d20 tx=0x7f51c0009060).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 msgr2=0x7f51c40aa4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 rx=0x7f51b4001ed0 tx=0x7f51b4007c70).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 shutdown_connections 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f51a805f180 0x7f51a8061640 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51c4157660 0x7f51c4155d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51c40c0180 0x7f51c40b0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 --2- 172.21.15.73:0/307756167 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f51c40c1750 0x7f51c40aa4c0 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 >> 172.21.15.73:0/307756167 conn(0x7f51c40cb000 msgr2=0x7f51c40ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 shutdown_connections 2023-11-28T04:08:16.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.443+0000 7f51ca66e700 1 -- 172.21.15.73:0/307756167 wait complete. 2023-11-28T04:08:16.460 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:16.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.574+0000 7f508dab7700 1 Processor -- start 2023-11-28T04:08:16.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.574+0000 7f508dab7700 1 -- start start 2023-11-28T04:08:16.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.574+0000 7f508dab7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f508dab7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5088068090 0x7f50880c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f508dab7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5088154d20 con 0x7f50880c2640 2023-11-28T04:08:16.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f508dab7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5088151670 con 0x7f5088068090 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f508dab7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880c48a0 con 0x7f5088067cb0 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f50877fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5088068090 0x7f50880c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5087fff700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f50880c2640 0x7f50880c2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39536/0 (socket says 172.21.15.73:39536) 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f50877fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:43930/0 (socket says 172.21.15.73:43930) 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5087fff700 1 -- 172.21.15.73:0/2483517093 learned_addr learned my addr 172.21.15.73:0/2483517093 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 -- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 msgr2=0x7f50880c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 -- 172.21.15.73:0/2483517093 >> v1:172.21.15.150:6789/0 conn(0x7f50880c2640 legacy=0x7f50880c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 -- 172.21.15.73:0/2483517093 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50880a7490 con 0x7f5088068090 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f50877fe700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:16.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f5086ffd700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088068090 0x7f50880c43b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5070009760 tx=0x7f5070009060).ready entity=mon.1 client_cookie=f3d9814a4a0c4b4d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.575+0000 7f50867fc700 1 -- 172.21.15.73:0/2483517093 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000f510 con 0x7f5088068090 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f50867fc700 1 -- 172.21.15.73:0/2483517093 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000f670 con 0x7f5088068090 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f50867fc700 1 -- 172.21.15.73:0/2483517093 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000f9f0 con 0x7f5088068090 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 -- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088068090 msgr2=0x7f50880c43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088068090 0x7f50880c43b0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 rx=0x7f5070009760 tx=0x7f5070009060).stop 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 -- 172.21.15.73:0/2483517093 shutdown_connections 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5088067cb0 0x7f50880c3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 --2- 172.21.15.73:0/2483517093 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088068090 0x7f50880c43b0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 -- 172.21.15.73:0/2483517093 >> 172.21.15.73:0/2483517093 conn(0x7f50880c7fd0 msgr2=0x7f50880c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 -- 172.21.15.73:0/2483517093 shutdown_connections 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.576+0000 7f508dab7700 1 -- 172.21.15.73:0/2483517093 wait complete. 2023-11-28T04:08:16.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.577+0000 7f508dab7700 1 Processor -- start 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.577+0000 7f508dab7700 1 -- start start 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088067cb0 0x7f50880bf380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50880c2640 0x7f50880bcfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50880bfeb0 con 0x7f5088068090 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5088164cb0 con 0x7f5088067cb0 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f508dab7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50880bfbd0 con 0x7f50880c2640 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f5086ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f5086ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40168/0 (socket says 172.21.15.73:40168) 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f5086ffd700 1 -- 172.21.15.73:0/2091951195 learned_addr learned my addr 172.21.15.73:0/2091951195 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f50877fe700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088067cb0 0x7f50880bf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.578+0000 7f5087fff700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50880c2640 0x7f50880bcfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 -- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50880c2640 msgr2=0x7f50880bcfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50880c2640 0x7f50880bcfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 -- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088067cb0 msgr2=0x7f50880bf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088067cb0 0x7f50880bf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 -- 172.21.15.73:0/2091951195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5078009a20 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f5086ffd700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7f5070011000 tx=0x7f5070017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507000f7e0 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507000f940 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50700092a0 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507001b660 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.579+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50881651a0 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.580+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508806c860 con 0x7f5088068090 2023-11-28T04:08:16.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.580+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f507001ba90 con 0x7f5088068090 2023-11-28T04:08:16.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.580+0000 7f507f7fe700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 0x7f5068061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.581+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f507007b930 con 0x7f5088068090 2023-11-28T04:08:16.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.581+0000 7f50877fe700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 0x7f5068061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.581+0000 7f50877fe700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 0x7f5068061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f50780099f0 tx=0x7f507800b040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.585+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f507004cf50 con 0x7f5088068090 2023-11-28T04:08:16.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.747+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50880a7490 con 0x7f5088068090 2023-11-28T04:08:16.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.748+0000 7f507f7fe700 1 -- 172.21.15.73:0/2091951195 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v175) v1 ==== 74+0+13723 (secure 0 0 0) 0x7f50880a7490 con 0x7f5088068090 2023-11-28T04:08:16.749 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:16.749 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":175,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:16.069045+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":20,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":163,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 msgr2=0x7f5068061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 0x7f5068061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7f50780099f0 tx=0x7f507800b040).stop 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 msgr2=0x7f50880c1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 rx=0x7f5070011000 tx=0x7f5070017000).stop 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 shutdown_connections 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f506805f150 0x7f5068061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50880c2640 0x7f50880bcfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5088067cb0 0x7f50880bf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 --2- 172.21.15.73:0/2091951195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5088068090 0x7f50880c1860 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 >> 172.21.15.73:0/2091951195 conn(0x7f50880c7fd0 msgr2=0x7f50880c83b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 shutdown_connections 2023-11-28T04:08:16.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.750+0000 7f508dab7700 1 -- 172.21.15.73:0/2091951195 wait complete. 2023-11-28T04:08:16.764 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '170', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T04:08:16.764 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:08:16.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.880+0000 7f8503316700 1 Processor -- start 2023-11-28T04:08:16.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.880+0000 7f8503316700 1 -- start start 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8503316700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84fc0c66f0 0x7f84fc154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8503316700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc154e40 con 0x7f84fc14e530 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8503316700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84fc0c7bc0 con 0x7f84fc158570 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8503316700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84fc0c6ba0 con 0x7f84fc0c66f0 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501b13700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f84fc0c66f0 0x7f84fc154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8502314700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f84fc14e530 0x7f84fc14e910 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44228/0 (socket says 172.21.15.73:44228) 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8502314700 1 -- 172.21.15.73:0/1182207199 learned_addr learned my addr 172.21.15.73:0/1182207199 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:16.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501b13700 1 -- 172.21.15.73:0/1182207199 >> v1:172.21.15.150:6789/0 conn(0x7f84fc158570 legacy=0x7f84fc0aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501b13700 1 -- 172.21.15.73:0/1182207199 >> v1:172.21.15.73:6789/0 conn(0x7f84fc14e530 legacy=0x7f84fc14e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501312700 1 -- 172.21.15.73:0/1182207199 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2942930898 0 0) 0x7f84f8001110 con 0x7f84fc14e530 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501b13700 1 -- 172.21.15.73:0/1182207199 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84fc155920 con 0x7f84fc0c66f0 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501b13700 1 --2- 172.21.15.73:0/1182207199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc0c66f0 0x7f84fc154950 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f84ec009760 tx=0x7f84ec009060).ready entity=mon.0 client_cookie=b193dab8fdaca7d1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.881+0000 7f8501312700 1 -- 172.21.15.73:0/1182207199 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00b580 con 0x7f84fc0c66f0 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8501312700 1 -- 172.21.15.73:0/1182207199 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84ec00b6e0 con 0x7f84fc0c66f0 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8501312700 1 -- 172.21.15.73:0/1182207199 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84ec00ba60 con 0x7f84fc0c66f0 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 -- 172.21.15.73:0/1182207199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc0c66f0 msgr2=0x7f84fc154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 --2- 172.21.15.73:0/1182207199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc0c66f0 0x7f84fc154950 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 rx=0x7f84ec009760 tx=0x7f84ec009060).stop 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 -- 172.21.15.73:0/1182207199 shutdown_connections 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 --2- 172.21.15.73:0/1182207199 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc0c66f0 0x7f84fc154950 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 -- 172.21.15.73:0/1182207199 >> 172.21.15.73:0/1182207199 conn(0x7f84fc064990 msgr2=0x7f84fc064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:16.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 -- 172.21.15.73:0/1182207199 shutdown_connections 2023-11-28T04:08:16.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.882+0000 7f8503316700 1 -- 172.21.15.73:0/1182207199 wait complete. 2023-11-28T04:08:16.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.883+0000 7f8503316700 1 Processor -- start 2023-11-28T04:08:16.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.883+0000 7f8503316700 1 -- start start 2023-11-28T04:08:16.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 0x7f84fc0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84fc14e530 0x7f84fc164760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0c53b0 con 0x7f84fc158570 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0af1a0 con 0x7f84fc14e530 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8503316700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84fc0af480 con 0x7f84fc0c66f0 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40184/0 (socket says 172.21.15.73:40184) 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 -- 172.21.15.73:0/3749672759 learned_addr learned my addr 172.21.15.73:0/3749672759 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8501b13700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84fc14e530 0x7f84fc164760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502314700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 0x7f84fc0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 -- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 msgr2=0x7f84fc0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 0x7f84fc0c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 -- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84fc14e530 msgr2=0x7f84fc164760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84fc14e530 0x7f84fc164760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 -- 172.21.15.73:0/3749672759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84ec0092a0 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502314700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 0x7f84fc0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.884+0000 7f8502b15700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f84f400c7c0 tx=0x7f84f4001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.885+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f4010510 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.885+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84f4010670 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.885+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84fc0af760 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.885+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84f40109f0 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.885+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84fc0c5070 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.886+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84fc062ee0 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.886+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f84f401b400 con 0x7f84fc158570 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.886+0000 7f84f27fc700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 0x7f84e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:16.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.886+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f84f4079c10 con 0x7f84fc158570 2023-11-28T04:08:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.886+0000 7f8502314700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 0x7f84e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:16.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.887+0000 7f8502314700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 0x7f84e0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f84f8000c00 tx=0x7f84f8009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:16.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:16.890+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f84f404b1d0 con 0x7f84fc158570 2023-11-28T04:08:17.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.055+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f84fc06c7c0 con 0x7f84fc158570 2023-11-28T04:08:17.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.055+0000 7f84f27fc700 1 -- 172.21.15.73:0/3749672759 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v175) v1 ==== 110+0+10 (secure 0 0 0) 0x7f84f404e7f0 con 0x7f84fc158570 2023-11-28T04:08:17.056 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 msgr2=0x7f84e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 0x7f84e0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f84f8000c00 tx=0x7f84f8009040).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 msgr2=0x7f84fc152ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 rx=0x7f84f400c7c0 tx=0x7f84f4001510).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 shutdown_connections 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f84e005f150 0x7f84e0061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84fc158570 0x7f84fc152ff0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84fc0c66f0 0x7f84fc0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 --2- 172.21.15.73:0/3749672759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f84fc14e530 0x7f84fc164760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.057+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 >> 172.21.15.73:0/3749672759 conn(0x7f84fc064990 msgr2=0x7f84fc0caf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.058+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 shutdown_connections 2023-11-28T04:08:17.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.058+0000 7f8503316700 1 -- 172.21.15.73:0/3749672759 wait complete. 2023-11-28T04:08:17.070 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:08:17.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.185+0000 7f5da2247700 1 Processor -- start 2023-11-28T04:08:17.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.185+0000 7f5da2247700 1 -- start start 2023-11-28T04:08:17.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5da2247700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d9c098d10 0x7f5d9c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5da2247700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5d9c0c6190 0x7f5d9c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5da2247700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d9c0c72d0 con 0x7f5d9c064350 2023-11-28T04:08:17.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5da2247700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c154bc0 con 0x7f5d9c098d10 2023-11-28T04:08:17.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5da2247700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d9c0c66c0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d9c098d10 0x7f5d9c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5d9c098d10 0x7f5d9c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40186/0 (socket says 172.21.15.73:40186) 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 -- 172.21.15.73:0/4233252034 learned_addr learned my addr 172.21.15.73:0/4233252034 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9affd700 1 --2- 172.21.15.73:0/4233252034 >> v2:172.21.15.150:3300/0 conn(0x7f5d9c0c6190 0x7f5d9c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 -- 172.21.15.73:0/4233252034 >> v2:172.21.15.150:3300/0 conn(0x7f5d9c0c6190 msgr2=0x7f5d9c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:17.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 --2- 172.21.15.73:0/4233252034 >> v2:172.21.15.150:3300/0 conn(0x7f5d9c0c6190 0x7f5d9c063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 -- 172.21.15.73:0/4233252034 >> v1:172.21.15.150:6789/0 conn(0x7f5d9c064350 legacy=0x7f5d9c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 -- 172.21.15.73:0/4233252034 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d9c0c69a0 con 0x7f5d9c098d10 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9b7fe700 1 --2- 172.21.15.73:0/4233252034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c098d10 0x7f5d9c0c7760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f5d9c0c5b40 tx=0x7f5d8c0090f0).ready entity=mon.0 client_cookie=17125f4912123f6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9a7fc700 1 -- 172.21.15.73:0/4233252034 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00b600 con 0x7f5d9c098d10 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9a7fc700 1 -- 172.21.15.73:0/4233252034 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d8c00b760 con 0x7f5d9c098d10 2023-11-28T04:08:17.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.186+0000 7f5d9a7fc700 1 -- 172.21.15.73:0/4233252034 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d8c00bae0 con 0x7f5d9c098d10 2023-11-28T04:08:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 -- 172.21.15.73:0/4233252034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c098d10 msgr2=0x7f5d9c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 --2- 172.21.15.73:0/4233252034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c098d10 0x7f5d9c0c7760 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 rx=0x7f5d9c0c5b40 tx=0x7f5d8c0090f0).stop 2023-11-28T04:08:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 -- 172.21.15.73:0/4233252034 shutdown_connections 2023-11-28T04:08:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 --2- 172.21.15.73:0/4233252034 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c098d10 0x7f5d9c0c7760 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 --2- 172.21.15.73:0/4233252034 >> v2:172.21.15.150:3300/0 conn(0x7f5d9c0c6190 0x7f5d9c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 -- 172.21.15.73:0/4233252034 >> 172.21.15.73:0/4233252034 conn(0x7f5d9c0cb310 msgr2=0x7f5d9c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 -- 172.21.15.73:0/4233252034 shutdown_connections 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.187+0000 7f5da2247700 1 -- 172.21.15.73:0/4233252034 wait complete. 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.188+0000 7f5da2247700 1 Processor -- start 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.188+0000 7f5da2247700 1 -- start start 2023-11-28T04:08:17.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d9c064350 0x7f5d9c0ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9c098d10 0x7f5d9c0a9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0ab690 con 0x7f5d9c0c6190 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0a99e0 con 0x7f5d9c064350 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5da2247700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d9c0a9cc0 con 0x7f5d9c098d10 2023-11-28T04:08:17.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40188/0 (socket says 172.21.15.73:40188) 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 -- 172.21.15.73:0/3852827254 learned_addr learned my addr 172.21.15.73:0/3852827254 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9b7fe700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d9c064350 0x7f5d9c0ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9affd700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9c098d10 0x7f5d9c0a9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 -- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9c098d10 msgr2=0x7f5d9c0a9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9c098d10 0x7f5d9c0a9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 -- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d9c064350 msgr2=0x7f5d9c0ad8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d9c064350 0x7f5d9c0ad8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.189+0000 7f5d9bfff700 1 -- 172.21.15.73:0/3852827254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d8c009370 con 0x7f5d9c0c6190 2023-11-28T04:08:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5d9bfff700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f5d94002230 tx=0x7f5d940015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400e4a0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d9400e600 con 0x7f5d9c0c6190 2023-11-28T04:08:17.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d9c0a9fa0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d9400e980 con 0x7f5d9c0c6190 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.190+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d9c155ab0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.191+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d9c0ab840 con 0x7f5d9c0c6190 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.191+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5d9401b3b0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.191+0000 7f5d937fe700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 0x7f5d7c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.192+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f5d94079520 con 0x7f5d9c0c6190 2023-11-28T04:08:17.207 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.192+0000 7f5d9b7fe700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 0x7f5d7c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.192+0000 7f5d9b7fe700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 0x7f5d7c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5d8c009130 tx=0x7f5d8c017000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.196+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5d94048af0 con 0x7f5d9c0c6190 2023-11-28T04:08:17.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.358+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f5d9c06c860 con 0x7f5d9c0c6190 2023-11-28T04:08:17.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.358+0000 7f5d937fe700 1 -- 172.21.15.73:0/3852827254 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v175) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5d9c0ab840 con 0x7f5d9c0c6190 2023-11-28T04:08:17.359 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 msgr2=0x7f5d7c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 0x7f5d7c061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f5d8c009130 tx=0x7f5d8c017000).stop 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 msgr2=0x7f5d9c1555c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 rx=0x7f5d94002230 tx=0x7f5d940015e0).stop 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 shutdown_connections 2023-11-28T04:08:17.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5d7c05f150 0x7f5d7c061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5d9c0c6190 0x7f5d9c1555c0 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5d9c064350 0x7f5d9c0ad8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 --2- 172.21.15.73:0/3852827254 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5d9c098d10 0x7f5d9c0a9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 >> 172.21.15.73:0/3852827254 conn(0x7f5d9c0cb310 msgr2=0x7f5d9c0ca8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 shutdown_connections 2023-11-28T04:08:17.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.360+0000 7f5da2247700 1 -- 172.21.15.73:0/3852827254 wait complete. 2023-11-28T04:08:17.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T04:08:17.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.484+0000 7f1a72aa7700 1 Processor -- start 2023-11-28T04:08:17.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.484+0000 7f1a72aa7700 1 -- start start 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a72aa7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a72aa7700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1a6c0ca180 0x7f1a6c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a72aa7700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a6c0a9a90 con 0x7f1a6c0caab0 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a72aa7700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c1490 con 0x7f1a6c0cae90 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a72aa7700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a6c154ce0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a6bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a6bfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40198/0 (socket says 172.21.15.73:40198) 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a6bfff700 1 -- 172.21.15.73:0/4163799739 learned_addr learned my addr 172.21.15.73:0/4163799739 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.485+0000 7f1a71044700 1 --2- 172.21.15.73:0/4163799739 >> v2:172.21.15.150:3300/0 conn(0x7f1a6c0ca180 0x7f1a6c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6bfff700 1 -- 172.21.15.73:0/4163799739 >> v2:172.21.15.150:3300/0 conn(0x7f1a6c0ca180 msgr2=0x7f1a6c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:17.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6bfff700 1 --2- 172.21.15.73:0/4163799739 >> v2:172.21.15.150:3300/0 conn(0x7f1a6c0ca180 0x7f1a6c0c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6bfff700 1 -- 172.21.15.73:0/4163799739 >> v1:172.21.15.150:6789/0 conn(0x7f1a6c0caab0 legacy=0x7f1a6c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6bfff700 1 -- 172.21.15.73:0/4163799739 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a6c0aab80 con 0x7f1a6c0cae90 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6bfff700 1 --2- 172.21.15.73:0/4163799739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f1a6c0a9a00 tx=0x7f1a54009840).ready entity=mon.0 client_cookie=6c96d4a5926bdff8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6b7fe700 1 -- 172.21.15.73:0/4163799739 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400e070 con 0x7f1a6c0cae90 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6b7fe700 1 -- 172.21.15.73:0/4163799739 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5400ba20 con 0x7f1a6c0cae90 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a6b7fe700 1 -- 172.21.15.73:0/4163799739 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5400bda0 con 0x7f1a6c0cae90 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 -- 172.21.15.73:0/4163799739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0cae90 msgr2=0x7f1a6c0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/4163799739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 rx=0x7f1a6c0a9a00 tx=0x7f1a54009840).stop 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 -- 172.21.15.73:0/4163799739 shutdown_connections 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/4163799739 >> v2:172.21.15.150:3300/0 conn(0x7f1a6c0ca180 0x7f1a6c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/4163799739 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0cae90 0x7f1a6c0c0ab0 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 -- 172.21.15.73:0/4163799739 >> 172.21.15.73:0/4163799739 conn(0x7f1a6c094920 msgr2=0x7f1a6c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 -- 172.21.15.73:0/4163799739 shutdown_connections 2023-11-28T04:08:17.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.486+0000 7f1a72aa7700 1 -- 172.21.15.73:0/4163799739 wait complete. 2023-11-28T04:08:17.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.487+0000 7f1a72aa7700 1 Processor -- start 2023-11-28T04:08:17.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 -- start start 2023-11-28T04:08:17.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6c0caab0 0x7f1a6c064930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6c0cae90 0x7f1a6c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c2cb0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c2ea0 con 0x7f1a6c0caab0 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a72aa7700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a6c0c3180 con 0x7f1a6c0cae90 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.488+0000 7f1a70843700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40202/0 (socket says 172.21.15.73:40202) 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 -- 172.21.15.73:0/2168020586 learned_addr learned my addr 172.21.15.73:0/2168020586 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a6bfff700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6c0caab0 0x7f1a6c064930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a71044700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6c0cae90 0x7f1a6c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 -- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6c0cae90 msgr2=0x7f1a6c152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6c0cae90 0x7f1a6c152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 -- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6c0caab0 msgr2=0x7f1a6c064930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6c0caab0 0x7f1a6c064930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 -- 172.21.15.73:0/2168020586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a54009ae0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a70843700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f1a5c00c4c0 tx=0x7f1a5c007df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c00ce70 con 0x7f1a6c0ca180 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a5c007020 con 0x7f1a6c0ca180 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a6c0c3460 con 0x7f1a6c0ca180 2023-11-28T04:08:17.494 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a5c0073c0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.489+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a6c156030 con 0x7f1a6c0ca180 2023-11-28T04:08:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.490+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a6c0c3650 con 0x7f1a6c0ca180 2023-11-28T04:08:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.490+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1a5c0066f0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.491+0000 7f1a68ff9700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 0x7f1a4c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.495 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.491+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f1a5c0797d0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.491+0000 7f1a6bfff700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 0x7f1a4c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.491+0000 7f1a6bfff700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 0x7f1a4c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f1a6c1581d0 tx=0x7f1a54009840).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.496 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.495+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1a5c04adf0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.660+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f1a6c0991a0 con 0x7f1a6c0ca180 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.660+0000 7f1a68ff9700 1 -- 172.21.15.73:0/2168020586 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v175) v1 ==== 84+0+143 (secure 0 0 0) 0x7f1a5c04e410 con 0x7f1a6c0ca180 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:17.661 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:17.662 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:08:17.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 msgr2=0x7f1a4c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 0x7f1a4c061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f1a6c1581d0 tx=0x7f1a54009840).stop 2023-11-28T04:08:17.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 msgr2=0x7f1a6c0c27c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 rx=0x7f1a5c00c4c0 tx=0x7f1a5c007df0).stop 2023-11-28T04:08:17.663 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 shutdown_connections 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1a4c05f150 0x7f1a4c061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1a6c0cae90 0x7f1a6c152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1a6c0ca180 0x7f1a6c0c27c0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 --2- 172.21.15.73:0/2168020586 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1a6c0caab0 0x7f1a6c064930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 >> 172.21.15.73:0/2168020586 conn(0x7f1a6c094920 msgr2=0x7f1a6c0a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 shutdown_connections 2023-11-28T04:08:17.664 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.662+0000 7f1a72aa7700 1 -- 172.21.15.73:0/2168020586 wait complete. 2023-11-28T04:08:17.675 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T04:08:17.676 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T04:08:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.792+0000 7f7fd21c2700 1 Processor -- start 2023-11-28T04:08:17.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.792+0000 7f7fd21c2700 1 -- start start 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fd21c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc066b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fd21c2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fd21c2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fcc152e10 con 0x7f7fcc14cda0 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fd21c2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fcc1530f0 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fd21c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc1533d0 con 0x7f7fcc14c9c0 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fcbfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fcbfff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40214/0 (socket says 172.21.15.73:40214) 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fcaffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7fcc14cda0 0x7f7fcc0c4ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44234/0 (socket says 172.21.15.73:44234) 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fcbfff700 1 -- 172.21.15.73:0/3790882645 learned_addr learned my addr 172.21.15.73:0/3790882645 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.793+0000 7f7fcb7fe700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc066b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcbfff700 1 -- 172.21.15.73:0/3790882645 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 msgr2=0x7f7fcc066b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:17.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcbfff700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc066b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcbfff700 1 -- 172.21.15.73:0/3790882645 >> v1:172.21.15.73:6789/0 conn(0x7f7fcc14cda0 legacy=0x7f7fcc0c4ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcbfff700 1 -- 172.21.15.73:0/3790882645 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fcc1536b0 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fca7fc700 1 -- 172.21.15.73:0/3790882645 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 871085113 0 0) 0x7f7fcc152e10 con 0x7f7fcc14cda0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcb7fe700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc066b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fcbfff700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f7fc4006fd0 tx=0x7f7fc40090f0).ready entity=mon.0 client_cookie=a9ba29db533e68f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fca7fc700 1 -- 172.21.15.73:0/3790882645 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400b600 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fca7fc700 1 -- 172.21.15.73:0/3790882645 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fc400b760 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fca7fc700 1 -- 172.21.15.73:0/3790882645 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fc400bb00 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 -- 172.21.15.73:0/3790882645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 msgr2=0x7f7fcc0bd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 rx=0x7f7fc4006fd0 tx=0x7f7fc40090f0).stop 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 -- 172.21.15.73:0/3790882645 shutdown_connections 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc066b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/3790882645 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc0bd370 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 -- 172.21.15.73:0/3790882645 >> 172.21.15.73:0/3790882645 conn(0x7f7fcc063810 msgr2=0x7f7fcc063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:17.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 -- 172.21.15.73:0/3790882645 shutdown_connections 2023-11-28T04:08:17.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.794+0000 7f7fd21c2700 1 -- 172.21.15.73:0/3790882645 wait complete. 2023-11-28T04:08:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 Processor -- start 2023-11-28T04:08:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 -- start start 2023-11-28T04:08:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc0ac480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 0x7f7fcc151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c3330 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c3610 con 0x7f7fcc14cda0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fd21c2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fcc0c00a0 con 0x7f7fcc14c9c0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fcb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fcb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40226/0 (socket says 172.21.15.73:40226) 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.796+0000 7f7fcb7fe700 1 -- 172.21.15.73:0/2727191279 learned_addr learned my addr 172.21.15.73:0/2727191279 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcaffd700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc0ac480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcbfff700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 0x7f7fcc151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 -- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 msgr2=0x7f7fcc0ac480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc0ac480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 -- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 msgr2=0x7f7fcc151620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 0x7f7fcc151620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:17.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 -- 172.21.15.73:0/2727191279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fc4009370 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcaffd700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc0ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcbfff700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 0x7f7fcc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fcb7fe700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f7fbc00d1d0 tx=0x7f7fbc009a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc00de70 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fbc00f040 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fcc0c0380 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fbc013600 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.797+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fcc149960 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.798+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fcc0c0820 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.798+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7fbc0139f0 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.799+0000 7f7fb37fe700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 0x7f7fb8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.799+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(175..175 src has 1..175) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f7fbc07a670 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.799+0000 7f7fcaffd700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 0x7f7fb8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:17.803 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.799+0000 7f7fcaffd700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 0x7f7fb8061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7fc0007fd0 tx=0x7f7fc0009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:17.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.803+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7fbc04bc30 con 0x7f7fcc0bcef0 2023-11-28T04:08:17.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:17.969+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f7fcc06c860 con 0x7f7fcc0bcef0 2023-11-28T04:08:18.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.081+0000 7f7fb37fe700 1 -- 172.21.15.73:0/2727191279 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v176) v1 ==== 152+0+0 (secure 0 0 0) 0x7f7fbc04f250 con 0x7f7fcc0bcef0 2023-11-28T04:08:18.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 msgr2=0x7f7fb8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.087 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 0x7f7fb8061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f7fc0007fd0 tx=0x7f7fc0009040).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 msgr2=0x7f7fcc149470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 rx=0x7f7fbc00d1d0 tx=0x7f7fbc009a90).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 shutdown_connections 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7fb805f150 0x7f7fb8061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7fcc14cda0 0x7f7fcc151620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7fcc0bcef0 0x7f7fcc149470 secure :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 rx=0x7f7fbc00d1d0 tx=0x7f7fbc009a90).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 --2- 172.21.15.73:0/2727191279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7fcc14c9c0 0x7f7fcc0ac480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.086+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 >> 172.21.15.73:0/2727191279 conn(0x7f7fcc063810 msgr2=0x7f7fcc0c4e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:18.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.087+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 shutdown_connections 2023-11-28T04:08:18.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.088+0000 7f7fd21c2700 1 -- 172.21.15.73:0/2727191279 wait complete. 2023-11-28T04:08:18.089 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T04:08:18.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T04:08:18.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.221+0000 7f2cae3f3700 1 Processor -- start 2023-11-28T04:08:18.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.221+0000 7f2cae3f3700 1 -- start start 2023-11-28T04:08:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.221+0000 7f2cae3f3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.221+0000 7f2cae3f3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8153390 0x7f2ca80c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.221+0000 7f2cae3f3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca8155f20 con 0x7f2ca8156690 2023-11-28T04:08:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2cae3f3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ca8153990 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2cae3f3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8156200 con 0x7f2ca8153390 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40240/0 (socket says 172.21.15.73:40240) 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 -- 172.21.15.73:0/2257757133 learned_addr learned my addr 172.21.15.73:0/2257757133 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca77fe700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8153390 0x7f2ca80c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 -- 172.21.15.73:0/2257757133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8153390 msgr2=0x7f2ca80c5ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8153390 0x7f2ca80c5ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 -- 172.21.15.73:0/2257757133 >> v1:172.21.15.150:6789/0 conn(0x7f2ca8156690 legacy=0x7f2ca8066a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 -- 172.21.15.73:0/2257757133 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca80bff70 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f2ca8153ad0 tx=0x7f2c980090f0).ready entity=mon.0 client_cookie=c18287b03b2d9fdb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:18.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca6ffd700 1 -- 172.21.15.73:0/2257757133 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800b600 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca6ffd700 1 -- 172.21.15.73:0/2257757133 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9800b760 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.222+0000 7f2ca6ffd700 1 -- 172.21.15.73:0/2257757133 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9800bae0 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 -- 172.21.15.73:0/2257757133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca80c5ac0 msgr2=0x7f2ca80c51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 rx=0x7f2ca8153ad0 tx=0x7f2c980090f0).stop 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 -- 172.21.15.73:0/2257757133 shutdown_connections 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca80c5ac0 0x7f2ca80c51a0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/2257757133 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca8153390 0x7f2ca80c5ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 -- 172.21.15.73:0/2257757133 >> 172.21.15.73:0/2257757133 conn(0x7f2ca8062700 msgr2=0x7f2ca8062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 -- 172.21.15.73:0/2257757133 shutdown_connections 2023-11-28T04:08:18.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.223+0000 7f2cae3f3700 1 -- 172.21.15.73:0/2257757133 wait complete. 2023-11-28T04:08:18.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.224+0000 7f2cae3f3700 1 Processor -- start 2023-11-28T04:08:18.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.224+0000 7f2cae3f3700 1 -- start start 2023-11-28T04:08:18.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 0x7f2ca80c2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 0x7f2ca80c3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80c4260 con 0x7f2ca8153390 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca813e8f0 con 0x7f2ca8156690 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cae3f3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca8162880 con 0x7f2ca80c5ac0 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40256/0 (socket says 172.21.15.73:40256) 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 -- 172.21.15.73:0/826711530 learned_addr learned my addr 172.21.15.73:0/826711530 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 0x7f2ca80c2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 -- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 msgr2=0x7f2ca80c2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 0x7f2ca80c2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cac990700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 0x7f2ca80c3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 -- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 msgr2=0x7f2ca80c3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:18.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 0x7f2ca80c3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 -- 172.21.15.73:0/826711530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c98009370 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 0x7f2ca80c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2cac990700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 0x7f2ca80c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.225+0000 7f2ca77fe700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f2c9c00d890 tx=0x7f2c9c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.226+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0125f0 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.226+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c9c012750 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.226+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca8162b20 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.226+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c012ad0 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.226+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca81630e0 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.227+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca806c860 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.227+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2c9c013480 con 0x7f2ca8153390 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.227+0000 7f2c8ffff700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 0x7f2c94061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:18.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.227+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f2c9c07a1b0 con 0x7f2ca8153390 2023-11-28T04:08:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.227+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 0x7f2c94061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.228+0000 7f2ca7fff700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 0x7f2c94061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2c98009fd0 tx=0x7f2c98017000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:18.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.232+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2c9c04b600 con 0x7f2ca8153390 2023-11-28T04:08:18.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:18.395+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f2ca80c5470 con 0x7f2ca8153390 2023-11-28T04:08:19.086 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.085+0000 7f2c8ffff700 1 -- 172.21.15.73:0/826711530 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v177) v1 ==== 170+0+0 (secure 0 0 0) 0x7f2c9c04ec20 con 0x7f2ca8153390 2023-11-28T04:08:19.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.088+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 msgr2=0x7f2c94061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.088+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 0x7f2c94061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f2c98009fd0 tx=0x7f2c98017000).stop 2023-11-28T04:08:19.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.088+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 msgr2=0x7f2ca80c3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.088+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 rx=0x7f2c9c00d890 tx=0x7f2c9c008040).stop 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 shutdown_connections 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2c9405f150 0x7f2c94061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ca8156690 0x7f2ca80c3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ca80c5ac0 0x7f2ca80c2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 --2- 172.21.15.73:0/826711530 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ca8153390 0x7f2ca80c3880 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.090+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 >> 172.21.15.73:0/826711530 conn(0x7f2ca8062700 msgr2=0x7f2ca8155310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.092+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 shutdown_connections 2023-11-28T04:08:19.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.094+0000 7f2cae3f3700 1 -- 172.21.15.73:0/826711530 wait complete. 2023-11-28T04:08:19.097 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T04:08:19.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T04:08:19.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 Processor -- start 2023-11-28T04:08:19.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 -- start start 2023-11-28T04:08:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82f414e530 0x7f82f40c66f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f4154950 0x7f82f414e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f82f4158570 0x7f82f4158a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f40c7bc0 con 0x7f82f414e530 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82f4155920 con 0x7f82f4158570 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.230+0000 7f82f8cca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f4154dd0 con 0x7f82f4154950 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82f414e530 0x7f82f40c66f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f82f414e530 0x7f82f40c66f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40272/0 (socket says 172.21.15.73:40272) 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 -- 172.21.15.73:0/329128718 learned_addr learned my addr 172.21.15.73:0/329128718 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f3fff700 1 --2- 172.21.15.73:0/329128718 >> v2:172.21.15.150:3300/0 conn(0x7f82f4158570 0x7f82f4158a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f2ffd700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f4154950 0x7f82f414e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 -- 172.21.15.73:0/329128718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f4154950 msgr2=0x7f82f414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:19.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f4154950 0x7f82f414e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 -- 172.21.15.73:0/329128718 >> v2:172.21.15.150:3300/0 conn(0x7f82f4158570 msgr2=0x7f82f4158a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:19.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 --2- 172.21.15.73:0/329128718 >> v2:172.21.15.150:3300/0 conn(0x7f82f4158570 0x7f82f4158a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 -- 172.21.15.73:0/329128718 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f4155080 con 0x7f82f414e530 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f37fe700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f414e530 0x7f82f40c66f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f82f40644d0 tx=0x7f82e4009840).ready entity=mon.0 client_cookie=49b7fc10752ca07c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f27fc700 1 -- 172.21.15.73:0/329128718 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400e070 con 0x7f82f414e530 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f27fc700 1 -- 172.21.15.73:0/329128718 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e400ba50 con 0x7f82f414e530 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.231+0000 7f82f27fc700 1 -- 172.21.15.73:0/329128718 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e400bdd0 con 0x7f82f414e530 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 -- 172.21.15.73:0/329128718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f414e530 msgr2=0x7f82f40c66f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f414e530 0x7f82f40c66f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 rx=0x7f82f40644d0 tx=0x7f82e4009840).stop 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 -- 172.21.15.73:0/329128718 shutdown_connections 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f414e530 0x7f82f40c66f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 --2- 172.21.15.73:0/329128718 >> v2:172.21.15.150:3300/0 conn(0x7f82f4158570 0x7f82f4158a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 --2- 172.21.15.73:0/329128718 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f4154950 0x7f82f414e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 -- 172.21.15.73:0/329128718 >> 172.21.15.73:0/329128718 conn(0x7f82f4064990 msgr2=0x7f82f4064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 -- 172.21.15.73:0/329128718 shutdown_connections 2023-11-28T04:08:19.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.232+0000 7f82f8cca700 1 -- 172.21.15.73:0/329128718 wait complete. 2023-11-28T04:08:19.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.233+0000 7f82f8cca700 1 Processor -- start 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.233+0000 7f82f8cca700 1 -- start start 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f414e530 0x7f82f414dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82f4158570 0x7f82f40c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c4340 con 0x7f82f4154950 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c3730 con 0x7f82f4158570 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f8cca700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f40c3a10 con 0x7f82f414e530 2023-11-28T04:08:19.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40282/0 (socket says 172.21.15.73:40282) 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 -- 172.21.15.73:0/3214391308 learned_addr learned my addr 172.21.15.73:0/3214391308 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f37fe700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f414e530 0x7f82f414dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f3fff700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82f4158570 0x7f82f40c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 -- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f414e530 msgr2=0x7f82f414dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f414e530 0x7f82f414dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 -- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82f4158570 msgr2=0x7f82f40c5220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82f4158570 0x7f82f40c5220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 -- 172.21.15.73:0/3214391308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82e4009ae0 con 0x7f82f4154950 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82f2ffd700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f82e800d890 tx=0x7f82e8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011590 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.234+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82e80116f0 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.235+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82f40c3cf0 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.235+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82e8011a70 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.235+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f82f414e1a0 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.236+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82f40c44f0 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.236+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f82e801c460 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.237+0000 7f82db7fe700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 0x7f82e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.237+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f82e8079e50 con 0x7f82f4154950 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.237+0000 7f82f37fe700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 0x7f82e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.237+0000 7f82f37fe700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 0x7f82e0061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f82e4009930 tx=0x7f82e4009840).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.241+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f82f40c44f0 con 0x7f82f4154950 2023-11-28T04:08:19.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.446+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f82f4062ee0 con 0x7f82f4154950 2023-11-28T04:08:19.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.454+0000 7f82db7fe700 1 -- 172.21.15.73:0/3214391308 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 21 and data pool 22 v62) v1 ==== 174+0+0 (secure 0 0 0) 0x7f82e804b120 con 0x7f82f4154950 2023-11-28T04:08:19.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 msgr2=0x7f82e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 0x7f82e0061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f82e4009930 tx=0x7f82e4009840).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 msgr2=0x7f82f40c2e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 rx=0x7f82e800d890 tx=0x7f82e8008040).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 shutdown_connections 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f82e005f150 0x7f82e0061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f82f4158570 0x7f82f40c5220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f82f414e530 0x7f82f414dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 --2- 172.21.15.73:0/3214391308 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f82f4154950 0x7f82f40c2e70 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.460+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 >> 172.21.15.73:0/3214391308 conn(0x7f82f4064990 msgr2=0x7f82f4155f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.463+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 shutdown_connections 2023-11-28T04:08:19.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.463+0000 7f82f8cca700 1 -- 172.21.15.73:0/3214391308 wait complete. 2023-11-28T04:08:19.467 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 21 and data pool 22 2023-11-28T04:08:19.485 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:19.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.596+0000 7fbb567db700 1 Processor -- start 2023-11-28T04:08:19.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.596+0000 7fbb567db700 1 -- start start 2023-11-28T04:08:19.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb567db700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501524a0 0x7fbb50067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb567db700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb50067f30 0x7fbb500be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb567db700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb500683e0 con 0x7fbb501520c0 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb567db700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb500be110 con 0x7fbb50067f30 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb567db700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50066fc0 con 0x7fbb501524a0 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb54d78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb50067f30 0x7fbb500be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb54d78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbb50067f30 0x7fbb500be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40298/0 (socket says 172.21.15.73:40298) 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.597+0000 7fbb54d78700 1 -- 172.21.15.73:0/2136135397 learned_addr learned my addr 172.21.15.73:0/2136135397 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb4f7fe700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501524a0 0x7fbb50067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb54d78700 1 -- 172.21.15.73:0/2136135397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501524a0 msgr2=0x7fbb50067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb54d78700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501524a0 0x7fbb50067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb54d78700 1 -- 172.21.15.73:0/2136135397 >> v1:172.21.15.150:6789/0 conn(0x7fbb501520c0 legacy=0x7fbb50066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb54d78700 1 -- 172.21.15.73:0/2136135397 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb500672a0 con 0x7fbb50067f30 2023-11-28T04:08:19.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb54d78700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb50067f30 0x7fbb500be8a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7fbb48009f20 tx=0x7fbb48009840).ready entity=mon.0 client_cookie=86b44554af989da4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb4effd700 1 -- 172.21.15.73:0/2136135397 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800e070 con 0x7fbb50067f30 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb4effd700 1 -- 172.21.15.73:0/2136135397 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb4800ba80 con 0x7fbb50067f30 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb4effd700 1 -- 172.21.15.73:0/2136135397 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4800be00 con 0x7fbb50067f30 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 -- 172.21.15.73:0/2136135397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb50067f30 msgr2=0x7fbb500be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb50067f30 0x7fbb500be8a0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 rx=0x7fbb48009f20 tx=0x7fbb48009840).stop 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 -- 172.21.15.73:0/2136135397 shutdown_connections 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb50067f30 0x7fbb500be8a0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 --2- 172.21.15.73:0/2136135397 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501524a0 0x7fbb50067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.598+0000 7fbb567db700 1 -- 172.21.15.73:0/2136135397 >> 172.21.15.73:0/2136135397 conn(0x7fbb500631d0 msgr2=0x7fbb500c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.599+0000 7fbb567db700 1 -- 172.21.15.73:0/2136135397 shutdown_connections 2023-11-28T04:08:19.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.599+0000 7fbb567db700 1 -- 172.21.15.73:0/2136135397 wait complete. 2023-11-28T04:08:19.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 Processor -- start 2023-11-28T04:08:19.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 -- start start 2023-11-28T04:08:19.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb50067f30 0x7fbb50163ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501520c0 0x7fbb500a87b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500a9190 con 0x7fbb501524a0 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.600+0000 7fbb567db700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500a6790 con 0x7fbb50067f30 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb567db700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50163790 con 0x7fbb501520c0 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40312/0 (socket says 172.21.15.73:40312) 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 -- 172.21.15.73:0/3367942388 learned_addr learned my addr 172.21.15.73:0/3367942388 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb4ffff700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb50067f30 0x7fbb50163ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb4f7fe700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501520c0 0x7fbb500a87b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 -- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501520c0 msgr2=0x7fbb500a87b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501520c0 0x7fbb500a87b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 -- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb50067f30 msgr2=0x7fbb50163ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb50067f30 0x7fbb50163ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 -- 172.21.15.73:0/3367942388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb48009ae0 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb54d78700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7fbb50047d20 tx=0x7fbb48009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb48013e90 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb480092e0 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb500a5e60 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb4801e6c0 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.601+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb500a6420 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.602+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb500a6900 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.603+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fbb48003040 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.603+0000 7fbb37fff700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 0x7fbb3c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.603+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fbb4807b700 con 0x7fbb501524a0 2023-11-28T04:08:19.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.603+0000 7fbb4ffff700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 0x7fbb3c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.603+0000 7fbb4ffff700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 0x7fbb3c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbb40007fd0 tx=0x7fbb40007d30).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.612 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.607+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbb4804ca80 con 0x7fbb501524a0 2023-11-28T04:08:19.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.771+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbb5006c860 con 0x7fbb501524a0 2023-11-28T04:08:19.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.772+0000 7fbb37fff700 1 -- 172.21.15.73:0/3367942388 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16956 (secure 0 0 0) 0x7fbb48023070 con 0x7fbb501524a0 2023-11-28T04:08:19.773 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:19.773 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":178,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:19.448444+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:19.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 msgr2=0x7fbb3c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 0x7fbb3c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fbb40007fd0 tx=0x7fbb40007d30).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 msgr2=0x7fbb500a8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 rx=0x7fbb50047d20 tx=0x7fbb48009840).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 shutdown_connections 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fbb3c05f150 0x7fbb3c061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbb501524a0 0x7fbb500a8ca0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbb50067f30 0x7fbb50163ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 --2- 172.21.15.73:0/3367942388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbb501520c0 0x7fbb500a87b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.773+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 >> 172.21.15.73:0/3367942388 conn(0x7fbb500631d0 msgr2=0x7fbb5014ed30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.774+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 shutdown_connections 2023-11-28T04:08:19.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.774+0000 7fbb567db700 1 -- 172.21.15.73:0/3367942388 wait complete. 2023-11-28T04:08:19.788 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:19.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.901+0000 7f52f9512700 1 Processor -- start 2023-11-28T04:08:19.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 -- start start 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 0x7f52f40c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52f40ca180 0x7f52f40c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52f40a9a90 con 0x7f52f40caab0 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52f40c1490 con 0x7f52f40ca180 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.902+0000 7f52f9512700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4154ce0 con 0x7f52f40cae90 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f27fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 0x7f52f40c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f52f40ca180 0x7f52f40c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f2ffd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f52f40caab0 0x7f52f40aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39582/0 (socket says 172.21.15.73:39582) 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f2ffd700 1 -- 172.21.15.73:0/1751807685 learned_addr learned my addr 172.21.15.73:0/1751807685 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 -- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 msgr2=0x7f52f40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 0x7f52f40c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 -- 172.21.15.73:0/1751807685 >> v1:172.21.15.150:6789/0 conn(0x7f52f40caab0 legacy=0x7f52f40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 -- 172.21.15.73:0/1751807685 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40aab80 con 0x7f52f40ca180 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f27fc700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 0x7f52f40c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f37fe700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f40c0fa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f52ec009760 tx=0x7f52ec009060).ready entity=mon.1 client_cookie=636c7cf941513934 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f1ffb700 1 -- 172.21.15.73:0/1751807685 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f510 con 0x7f52f40ca180 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f1ffb700 1 -- 172.21.15.73:0/1751807685 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec00f670 con 0x7f52f40ca180 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.903+0000 7f52f1ffb700 1 -- 172.21.15.73:0/1751807685 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f9f0 con 0x7f52f40ca180 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 -- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 msgr2=0x7f52f40c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f40c0fa0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 rx=0x7f52ec009760 tx=0x7f52ec009060).stop 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 -- 172.21.15.73:0/1751807685 shutdown_connections 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f40c0fa0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 --2- 172.21.15.73:0/1751807685 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40cae90 0x7f52f40c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 -- 172.21.15.73:0/1751807685 >> 172.21.15.73:0/1751807685 conn(0x7f52f4094920 msgr2=0x7f52f406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 -- 172.21.15.73:0/1751807685 shutdown_connections 2023-11-28T04:08:19.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.904+0000 7f52f9512700 1 -- 172.21.15.73:0/1751807685 wait complete. 2023-11-28T04:08:19.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.905+0000 7f52f9512700 1 Processor -- start 2023-11-28T04:08:19.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.905+0000 7f52f9512700 1 -- start start 2023-11-28T04:08:19.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f4153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40caab0 0x7f52f41642c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40c3cb0 con 0x7f52f40cae90 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f4156a30 con 0x7f52f40ca180 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f9512700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f40c3380 con 0x7f52f40caab0 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40326/0 (socket says 172.21.15.73:40326) 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 -- 172.21.15.73:0/3108938661 learned_addr learned my addr 172.21.15.73:0/3108938661 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f2ffd700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f4153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f27fc700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40caab0 0x7f52f41642c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 -- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40caab0 msgr2=0x7f52f41642c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.911 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40caab0 0x7f52f41642c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 -- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 msgr2=0x7f52f4153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f4153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f37fe700 1 -- 172.21.15.73:0/3108938661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80092a0 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.906+0000 7f52f2ffd700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f4153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52f37fe700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f52ec011000 tx=0x7f52ec017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec00f9f0 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52ec01b410 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52ec0092a0 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52ec01b7b0 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.907+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52f40c3940 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.908+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52f40c3e60 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.908+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f52ec01bb60 con 0x7f52f40cae90 2023-11-28T04:08:19.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.908+0000 7f52daffd700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 0x7f52e0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:19.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.908+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f52ec07bed0 con 0x7f52f40cae90 2023-11-28T04:08:19.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.909+0000 7f52f2ffd700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 0x7f52e0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:19.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.909+0000 7f52f2ffd700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 0x7f52e0061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52e4005b20 tx=0x7f52e4009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:19.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:19.912+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52f40c3e60 con 0x7f52f40cae90 2023-11-28T04:08:20.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.075+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f52f40c58f0 con 0x7f52f40cae90 2023-11-28T04:08:20.077 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.076+0000 7f52daffd700 1 -- 172.21.15.73:0/3108938661 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v178) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f52ec028070 con 0x7f52f40cae90 2023-11-28T04:08:20.077 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:20.077 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":178,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:19.448444+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:20.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 msgr2=0x7f52e0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 0x7f52e0061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52e4005b20 tx=0x7f52e4009040).stop 2023-11-28T04:08:20.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 msgr2=0x7f52f4157410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 rx=0x7f52ec011000 tx=0x7f52ec017000).stop 2023-11-28T04:08:20.079 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 shutdown_connections 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f52e005f150 0x7f52e0061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f40cae90 0x7f52f4157410 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f40ca180 0x7f52f4153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 --2- 172.21.15.73:0/3108938661 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f52f40caab0 0x7f52f41642c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 >> 172.21.15.73:0/3108938661 conn(0x7f52f4094920 msgr2=0x7f52f40a7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 shutdown_connections 2023-11-28T04:08:20.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.078+0000 7f52f9512700 1 -- 172.21.15.73:0/3108938661 wait complete. 2023-11-28T04:08:20.091 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T04:08:20.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.207+0000 7fc6ace0b700 1 Processor -- start 2023-11-28T04:08:20.208 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.207+0000 7fc6ace0b700 1 -- start start 2023-11-28T04:08:20.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6ace0b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc6a81547f0 0x7fc6a80a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6ace0b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6ace0b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6a80c7e30 con 0x7fc6a8062700 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6ace0b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80bcf40 con 0x7fc6a80c86f0 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6ace0b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6a80c6b20 con 0x7fc6a81547f0 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40340/0 (socket says 172.21.15.73:40340) 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a6d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc6a8062700 0x7fc6a80a4db0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44242/0 (socket says 172.21.15.73:44242) 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 -- 172.21.15.73:0/2180867344 learned_addr learned my addr 172.21.15.73:0/2180867344 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a659c700 1 --2- 172.21.15.73:0/2180867344 >> v2:172.21.15.150:3300/0 conn(0x7fc6a81547f0 0x7fc6a80a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 -- 172.21.15.73:0/2180867344 >> v2:172.21.15.150:3300/0 conn(0x7fc6a81547f0 msgr2=0x7fc6a80a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/2180867344 >> v2:172.21.15.150:3300/0 conn(0x7fc6a81547f0 0x7fc6a80a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a5d9b700 1 -- 172.21.15.73:0/2180867344 >> v1:172.21.15.73:6789/0 conn(0x7fc6a8062700 legacy=0x7fc6a80a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.208+0000 7fc6a559a700 1 -- 172.21.15.73:0/2180867344 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 847011108 0 0) 0x7fc6a80c7e30 con 0x7fc6a8062700 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6a5d9b700 1 -- 172.21.15.73:0/2180867344 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a80a77d0 con 0x7fc6a80c86f0 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/2180867344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fc6a805f7f0 tx=0x7fc69c0090f0).ready entity=mon.0 client_cookie=7f134b689b2c51f2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6a559a700 1 -- 172.21.15.73:0/2180867344 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00b520 con 0x7fc6a80c86f0 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6a559a700 1 -- 172.21.15.73:0/2180867344 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69c00b680 con 0x7fc6a80c86f0 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6a559a700 1 -- 172.21.15.73:0/2180867344 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69c00ba20 con 0x7fc6a80c86f0 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 -- 172.21.15.73:0/2180867344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a80c86f0 msgr2=0x7fc6a80c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/2180867344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 rx=0x7fc6a805f7f0 tx=0x7fc69c0090f0).stop 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 -- 172.21.15.73:0/2180867344 shutdown_connections 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/2180867344 >> v2:172.21.15.150:3300/0 conn(0x7fc6a81547f0 0x7fc6a80a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/2180867344 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80c6fb0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 -- 172.21.15.73:0/2180867344 >> 172.21.15.73:0/2180867344 conn(0x7fc6a80642d0 msgr2=0x7fc6a80c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:20.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 -- 172.21.15.73:0/2180867344 shutdown_connections 2023-11-28T04:08:20.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.209+0000 7fc6ace0b700 1 -- 172.21.15.73:0/2180867344 wait complete. 2023-11-28T04:08:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.210+0000 7fc6ace0b700 1 Processor -- start 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.210+0000 7fc6ace0b700 1 -- start start 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80bf120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6a81547f0 0x7fc6a80bf610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80bfb00 con 0x7fc6a8062700 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80ac0d0 con 0x7fc6a80c86f0 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6ace0b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6a80aa5d0 con 0x7fc6a81547f0 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6a659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40344/0 (socket says 172.21.15.73:40344) 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6a659c700 1 -- 172.21.15.73:0/1179552334 learned_addr learned my addr 172.21.15.73:0/1179552334 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80bf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.211+0000 7fc6a6d9d700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6a81547f0 0x7fc6a80bf610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 -- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6a81547f0 msgr2=0x7fc6a80bf610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6a81547f0 0x7fc6a80bf610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 -- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 msgr2=0x7fc6a80bf120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80bf120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 -- 172.21.15.73:0/1179552334 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc69c009370 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6a659c700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7fc69800e960 tx=0x7fc698009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800c850 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc69800c9b0 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6a80bfd00 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc69800cd30 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.212+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6a80c02c0 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.213+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6a806c860 con 0x7fc6a8062700 2023-11-28T04:08:20.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.213+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc698012600 con 0x7fc6a8062700 2023-11-28T04:08:20.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.214+0000 7fc6967fc700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 0x7fc6880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.214+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 0x7fc6880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.214+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc69807a710 con 0x7fc6a8062700 2023-11-28T04:08:20.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.214+0000 7fc6a5d9b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 0x7fc6880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc6a805f7f0 tx=0x7fc69c017000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.218+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc69804b9e0 con 0x7fc6a8062700 2023-11-28T04:08:20.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.423+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc6a80a6220 con 0x7fc6a8062700 2023-11-28T04:08:20.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.452+0000 7fc6967fc700 1 -- 172.21.15.73:0/1179552334 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v63) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc69804f000 con 0x7fc6a8062700 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 msgr2=0x7fc6880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 0x7fc6880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fc6a805f7f0 tx=0x7fc69c017000).stop 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 msgr2=0x7fc6a8150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 rx=0x7fc69800e960 tx=0x7fc698009060).stop 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 shutdown_connections 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc68805f120 0x7fc6880615e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc6a81547f0 0x7fc6a80bf610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc6a8062700 0x7fc6a8150e70 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 --2- 172.21.15.73:0/1179552334 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc6a80c86f0 0x7fc6a80bf120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 >> 172.21.15.73:0/1179552334 conn(0x7fc6a80642d0 msgr2=0x7fc6a8153ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:20.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 shutdown_connections 2023-11-28T04:08:20.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.454+0000 7fc6ace0b700 1 -- 172.21.15.73:0/1179552334 wait complete. 2023-11-28T04:08:20.468 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T04:08:20.468 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.582+0000 7fc42cd38700 1 Processor -- start 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.582+0000 7fc42cd38700 1 -- start start 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.583+0000 7fc42cd38700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc4280a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.583+0000 7fc42cd38700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc42814e600 0x7fc42814eab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.583+0000 7fc42cd38700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc428067770 con 0x7fc428158160 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.583+0000 7fc42cd38700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc428159180 con 0x7fc42814e600 2023-11-28T04:08:20.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.583+0000 7fc42cd38700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280be8d0 con 0x7fc4280a8280 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc425d9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc428158160 0x7fc428155500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44256/0 (socket says 172.21.15.73:44256) 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc425d9b700 1 -- 172.21.15.73:0/252347538 learned_addr learned my addr 172.21.15.73:0/252347538 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc426d9d700 1 --2- 172.21.15.73:0/252347538 >> v2:172.21.15.150:3300/0 conn(0x7fc42814e600 0x7fc42814eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42659c700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc4280a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2195275202 0 0) 0x7fc428067770 con 0x7fc428158160 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4140033f0 con 0x7fc428158160 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 56370747 0 0) 0x7fc4140033f0 con 0x7fc428158160 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 msgr2=0x7fc4280a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc4280a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 >> v2:172.21.15.150:3300/0 conn(0x7fc42814e600 msgr2=0x7fc42814eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 --2- 172.21.15.73:0/252347538 >> v2:172.21.15.150:3300/0 conn(0x7fc42814e600 0x7fc42814eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc428158600 con 0x7fc428158160 2023-11-28T04:08:20.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc410001470 con 0x7fc428158160 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 >> v1:172.21.15.73:6789/0 conn(0x7fc428158160 legacy=0x7fc428155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4140038e0 0x7fc414003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414004690 0x7fc414004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc414005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc428159180 con 0x7fc4140038e0 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280be8d0 con 0x7fc414004690 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc41400b8e0 con 0x7fc4280a8280 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc42659c700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4140038e0 0x7fc414003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc425d9b700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414004690 0x7fc414004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.584+0000 7fc426d9d700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc414005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 -- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 msgr2=0x7fc414005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280a8280 0x7fc414005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 -- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414004690 msgr2=0x7fc414004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:20.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414004690 0x7fc414004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 -- 172.21.15.73:0/252347538 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4140033f0 con 0x7fc4140038e0 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc425d9b700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc414004690 0x7fc414004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42659c700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4140038e0 0x7fc414003d00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fc41800e210 tx=0x7fc41800b040).ready entity=mon.0 client_cookie=e450df2fe1998e8f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc418009ac0 con 0x7fc4140038e0 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc41800f040 con 0x7fc4140038e0 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42559a700 1 -- 172.21.15.73:0/252347538 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc41800ecd0 con 0x7fc4140038e0 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 -- 172.21.15.73:0/252347538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4140038e0 msgr2=0x7fc414003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 --2- 172.21.15.73:0/252347538 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4140038e0 0x7fc414003d00 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 rx=0x7fc41800e210 tx=0x7fc41800b040).stop 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc426d9d700 1 -- 172.21.15.73:0/252347538 reap_dead start 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 -- 172.21.15.73:0/252347538 shutdown_connections 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 -- 172.21.15.73:0/252347538 >> 172.21.15.73:0/252347538 conn(0x7fc428062700 msgr2=0x7fc428062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 -- 172.21.15.73:0/252347538 shutdown_connections 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.585+0000 7fc42cd38700 1 -- 172.21.15.73:0/252347538 wait complete. 2023-11-28T04:08:20.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.586+0000 7fc42cd38700 1 Processor -- start 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.586+0000 7fc42cd38700 1 -- start start 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 0x7fc4280c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c6060 0x7fc42814c8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280acc00 con 0x7fc4280adc20 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4280c4680 con 0x7fc4140038e0 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42cd38700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc42814cdc0 con 0x7fc4280c6060 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc426d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc426d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40358/0 (socket says 172.21.15.73:40358) 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc426d9d700 1 -- 172.21.15.73:0/4246835273 learned_addr learned my addr 172.21.15.73:0/4246835273 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc42659c700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 0x7fc4280c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.587+0000 7fc425d9b700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c6060 0x7fc42814c8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 -- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c6060 msgr2=0x7fc42814c8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c6060 0x7fc42814c8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 -- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 msgr2=0x7fc4280c01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 0x7fc4280c01e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 -- 172.21.15.73:0/4246835273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4280ae590 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc42659c700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 0x7fc4280c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc426d9d700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fc428154b90 tx=0x7fc420008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420015440 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4200155a0 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4280c4030 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc420015940 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.588+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4280ae0a0 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.589+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4280ae2f0 con 0x7fc4280adc20 2023-11-28T04:08:20.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.589+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc420015d30 con 0x7fc4280adc20 2023-11-28T04:08:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.589+0000 7fc41e7fc700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 0x7fc408061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.589+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc42007a290 con 0x7fc4280adc20 2023-11-28T04:08:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.590+0000 7fc42659c700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 0x7fc408061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.590+0000 7fc42659c700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 0x7fc408061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc41800afd0 tx=0x7fc41801a000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:20.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.594+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc42004b600 con 0x7fc4280adc20 2023-11-28T04:08:20.800 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:20.799+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc42806c860 con 0x7fc4280adc20 2023-11-28T04:08:21.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.459+0000 7fc41e7fc700 1 -- 172.21.15.73:0/4246835273 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc42004ec20 con 0x7fc4280adc20 2023-11-28T04:08:21.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 msgr2=0x7fc408061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 0x7fc408061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fc41800afd0 tx=0x7fc41801a000).stop 2023-11-28T04:08:21.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 msgr2=0x7fc4280c2460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 rx=0x7fc428154b90 tx=0x7fc420008040).stop 2023-11-28T04:08:21.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 shutdown_connections 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fc40805f150 0x7fc408061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4280adc20 0x7fc4280c2460 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc4140038e0 0x7fc4280c01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 --2- 172.21.15.73:0/4246835273 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc4280c6060 0x7fc42814c8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 >> 172.21.15.73:0/4246835273 conn(0x7fc428062700 msgr2=0x7fc4280650b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:21.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 shutdown_connections 2023-11-28T04:08:21.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.461+0000 7fc42cd38700 1 -- 172.21.15.73:0/4246835273 wait complete. 2023-11-28T04:08:21.474 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T04:08:21.475 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 Processor -- start 2023-11-28T04:08:21.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 -- start start 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a941544d0 0x7f5a94155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5a940673c0 0x7f5a9415e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a940683d0 con 0x7f5a94066f40 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a94067850 con 0x7f5a940673c0 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a9a6a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94067b30 con 0x7f5a941544d0 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a937fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5a94066f40 0x7f5a94095e80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44268/0 (socket says 172.21.15.73:44268) 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a937fe700 1 -- 172.21.15.73:0/3724110937 learned_addr learned my addr 172.21.15.73:0/3724110937 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a93fff700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a941544d0 0x7f5a94155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/3724110937 >> v2:172.21.15.150:3300/0 conn(0x7f5a940673c0 0x7f5a9415e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1472018471 0 0) 0x7f5a940683d0 con 0x7f5a94066f40 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a7c0033f0 con 0x7f5a94066f40 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 850738323 0 0) 0x7f5a7c0033f0 con 0x7f5a94066f40 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a941544d0 msgr2=0x7f5a94155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a941544d0 0x7f5a94155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 >> v2:172.21.15.150:3300/0 conn(0x7f5a940673c0 msgr2=0x7f5a9415e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 --2- 172.21.15.73:0/3724110937 >> v2:172.21.15.150:3300/0 conn(0x7f5a940673c0 0x7f5a9415e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a94067e10 con 0x7f5a94066f40 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f5a88001470 con 0x7f5a94066f40 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/3724110937 >> v2:172.21.15.150:3300/0 conn(0x7f5a940673c0 0x7f5a9415e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:21.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 >> v1:172.21.15.73:6789/0 conn(0x7f5a94066f40 legacy=0x7f5a94095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c0038e0 0x7f5a7c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7c004690 0x7f5a7c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a7c005000 0x7f5a7c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94067850 con 0x7f5a7c004690 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a94067b30 con 0x7f5a7c0038e0 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a7c00bdd0 con 0x7f5a7c005000 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a93fff700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a941544d0 0x7f5a94155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.589+0000 7f5a937fe700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7c004690 0x7f5a7c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a7c005000 0x7f5a7c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a93fff700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c0038e0 0x7f5a7c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 -- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a7c005000 msgr2=0x7f5a7c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a7c005000 0x7f5a7c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 -- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c0038e0 msgr2=0x7f5a7c003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c0038e0 0x7f5a7c003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 -- 172.21.15.73:0/3724110937 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a7c0033f0 con 0x7f5a7c004690 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a7c005000 0x7f5a7c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a93fff700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c0038e0 0x7f5a7c003d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a937fe700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7c004690 0x7f5a7c004b10 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7f5a8800c210 tx=0x7f5a88001e90).ready entity=mon.0 client_cookie=2da1bc1df4046cc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a88007a60 con 0x7f5a7c004690 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8800caa0 con 0x7f5a7c004690 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a92ffd700 1 -- 172.21.15.73:0/3724110937 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8800ce40 con 0x7f5a7c004690 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/3724110937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7c004690 msgr2=0x7f5a7c004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/3724110937 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a7c004690 0x7f5a7c004b10 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 rx=0x7f5a8800c210 tx=0x7f5a88001e90).stop 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a98c3e700 1 -- 172.21.15.73:0/3724110937 reap_dead start 2023-11-28T04:08:21.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/3724110937 shutdown_connections 2023-11-28T04:08:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/3724110937 >> 172.21.15.73:0/3724110937 conn(0x7f5a940631d0 msgr2=0x7f5a940c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/3724110937 shutdown_connections 2023-11-28T04:08:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.590+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/3724110937 wait complete. 2023-11-28T04:08:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 Processor -- start 2023-11-28T04:08:21.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 -- start start 2023-11-28T04:08:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c004690 0x7f5a9414b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a9414d010 0x7f5a9414d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414b3f0 con 0x7f5a940a8df0 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414d980 con 0x7f5a7c004690 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.592+0000 7f5a9a6a1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a9414bc30 con 0x7f5a9414d010 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40384/0 (socket says 172.21.15.73:40384) 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 -- 172.21.15.73:0/265500403 learned_addr learned my addr 172.21.15.73:0/265500403 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a93fff700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c004690 0x7f5a9414b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a937fe700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a9414d010 0x7f5a9414d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 -- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a9414d010 msgr2=0x7f5a9414d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a9414d010 0x7f5a9414d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 -- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c004690 msgr2=0x7f5a9414b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c004690 0x7f5a9414b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 -- 172.21.15.73:0/265500403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a9414bf10 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a98c3e700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f5a940c9040 tx=0x7f5a8c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c00e030 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a8c011db0 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a9414c1f0 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a8c015620 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.593+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a9414c7b0 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.594+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a9406c860 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.594+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5a8c015a10 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.595+0000 7f5a7bfff700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 0x7f5a80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.595+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5a8c00f070 con 0x7f5a940a8df0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.595+0000 7f5a93fff700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 0x7f5a80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.595+0000 7f5a93fff700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 0x7f5a80061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5a84006be0 tx=0x7f5a84008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.600+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5a8c04bf60 con 0x7f5a940a8df0 2023-11-28T04:08:21.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.801+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a940680f0 con 0x7f5a940a8df0 2023-11-28T04:08:21.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.801+0000 7f5a7bfff700 1 -- 172.21.15.73:0/265500403 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f5a8c04f580 con 0x7f5a940a8df0 2023-11-28T04:08:21.802 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:21.802 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:08:21.804 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 msgr2=0x7f5a80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 0x7f5a80061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f5a84006be0 tx=0x7f5a84008040).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 msgr2=0x7f5a940a9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 rx=0x7f5a940c9040 tx=0x7f5a8c00b040).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 shutdown_connections 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f5a8005f150 0x7f5a80061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a940a8df0 0x7f5a940a9270 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a7c004690 0x7f5a9414b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 --2- 172.21.15.73:0/265500403 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a9414d010 0x7f5a9414d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.803+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 >> 172.21.15.73:0/265500403 conn(0x7f5a940631d0 msgr2=0x7f5a9414eb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.804+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 shutdown_connections 2023-11-28T04:08:21.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.804+0000 7f5a9a6a1700 1 -- 172.21.15.73:0/265500403 wait complete. 2023-11-28T04:08:21.806 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:08:21.816 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:08:21.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 Processor -- start 2023-11-28T04:08:21.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 -- start start 2023-11-28T04:08:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a41542d0 0x7f32a40697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32a4062e10 0x7f32a40661b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f32a4065760 con 0x7f32a4069ce0 2023-11-28T04:08:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f32a406a160 con 0x7f32a4062e10 2023-11-28T04:08:21.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32aa4fa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40c3250 con 0x7f32a41542d0 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f32a4062e10 0x7f32a40661b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a37fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f32a4069ce0 0x7f32a4062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44270/0 (socket says 172.21.15.73:44270) 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a37fe700 1 -- 172.21.15.73:0/2160495483 learned_addr learned my addr 172.21.15.73:0/2160495483 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 --2- 172.21.15.73:0/2160495483 >> v2:172.21.15.73:3300/0 conn(0x7f32a4062e10 0x7f32a40661b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40400/0 (socket says 172.21.15.73:40400) 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a3fff700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a41542d0 0x7f32a40697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 -- 172.21.15.73:0/2160495483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a41542d0 msgr2=0x7f32a40697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a41542d0 0x7f32a40697f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 -- 172.21.15.73:0/2160495483 >> v1:172.21.15.73:6789/0 conn(0x7f32a4069ce0 legacy=0x7f32a4062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 -- 172.21.15.73:0/2160495483 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32a406a340 con 0x7f32a4062e10 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a2ffd700 1 -- 172.21.15.73:0/2160495483 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991807405 0 0) 0x7f32a4065760 con 0x7f32a4069ce0 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a8a97700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a40661b0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f329c009e60 tx=0x7f329c009780).ready entity=mon.0 client_cookie=55e7363525942908 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a2ffd700 1 -- 172.21.15.73:0/2160495483 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c00e070 con 0x7f32a4062e10 2023-11-28T04:08:21.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a2ffd700 1 -- 172.21.15.73:0/2160495483 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f329c00bc30 con 0x7f32a4062e10 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.931+0000 7f32a2ffd700 1 -- 172.21.15.73:0/2160495483 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f329c012410 con 0x7f32a4062e10 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 -- 172.21.15.73:0/2160495483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 msgr2=0x7f32a40661b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a40661b0 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 rx=0x7f329c009e60 tx=0x7f329c009780).stop 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 -- 172.21.15.73:0/2160495483 shutdown_connections 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a41542d0 0x7f32a40697f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/2160495483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a40661b0 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 -- 172.21.15.73:0/2160495483 >> 172.21.15.73:0/2160495483 conn(0x7f32a40c7fd0 msgr2=0x7f32a40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 -- 172.21.15.73:0/2160495483 shutdown_connections 2023-11-28T04:08:21.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.932+0000 7f32aa4fa700 1 -- 172.21.15.73:0/2160495483 wait complete. 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.933+0000 7f32aa4fa700 1 Processor -- start 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.933+0000 7f32aa4fa700 1 -- start start 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a4069ce0 0x7f32a40a8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a41542d0 0x7f32a40a9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40bf670 con 0x7f32a4062e10 2023-11-28T04:08:21.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40bf910 con 0x7f32a41542d0 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f32a40bfbf0 con 0x7f32a4069ce0 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40416/0 (socket says 172.21.15.73:40416) 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 -- 172.21.15.73:0/4293887124 learned_addr learned my addr 172.21.15.73:0/4293887124 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a37fe700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a4069ce0 0x7f32a40a8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a8a97700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a41542d0 0x7f32a40a9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 -- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a4069ce0 msgr2=0x7f32a40a8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a4069ce0 0x7f32a40a8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 -- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a41542d0 msgr2=0x7f32a40a9080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a41542d0 0x7f32a40a9080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 -- 172.21.15.73:0/4293887124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f329c009a20 con 0x7f32a4062e10 2023-11-28T04:08:21.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32a3fff700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f329400d890 tx=0x7f3294008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32940125f0 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3294012750 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32a40bfed0 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3294012ad0 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32a40a9570 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.935+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32a40c0520 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.936+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3294013480 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.936+0000 7f328bfff700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 0x7f3290061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.936+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3294079f40 con 0x7f32a4062e10 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.936+0000 7f32a37fe700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 0x7f3290061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:21.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.936+0000 7f32a37fe700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 0x7f3290061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f32a40aa460 tx=0x7f3298009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:21.941 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:21.941+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f329404b1b0 con 0x7f32a4062e10 2023-11-28T04:08:22.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.106+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f32a406a620 con 0x7f32a4062e10 2023-11-28T04:08:22.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.107+0000 7f328bfff700 1 -- 172.21.15.73:0/4293887124 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f329404e7d0 con 0x7f32a4062e10 2023-11-28T04:08:22.107 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:22.107 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":179,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:08:20.450534+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"170","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 msgr2=0x7f3290061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 0x7f3290061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f32a40aa460 tx=0x7f3298009040).stop 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 msgr2=0x7f32a414ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 rx=0x7f329400d890 tx=0x7f3294008040).stop 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 shutdown_connections 2023-11-28T04:08:22.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f329005f150 0x7f3290061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f32a41542d0 0x7f32a40a9080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f32a4062e10 0x7f32a414ce70 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 --2- 172.21.15.73:0/4293887124 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f32a4069ce0 0x7f32a40a8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 >> 172.21.15.73:0/4293887124 conn(0x7f32a40c7fd0 msgr2=0x7f32a414d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 shutdown_connections 2023-11-28T04:08:22.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.109+0000 7f32aa4fa700 1 -- 172.21.15.73:0/4293887124 wait complete. 2023-11-28T04:08:22.121 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T04:08:22.121 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:08:22.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.235+0000 7f6dc2e2c700 1 Processor -- start 2023-11-28T04:08:22.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.235+0000 7f6dc2e2c700 1 -- start start 2023-11-28T04:08:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc2e2c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc2e2c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6dbc0ca180 0x7f6dbc15e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc2e2c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6dbc0a9a90 con 0x7f6dbc0cae90 2023-11-28T04:08:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc2e2c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6dbc154ce0 con 0x7f6dbc0ca180 2023-11-28T04:08:22.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc2e2c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0a8e80 con 0x7f6dbc0caab0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc0bc8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc13c9700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6dbc0ca180 0x7f6dbc15e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc0bc8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:44162/0 (socket says 172.21.15.73:44162) 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6dc0bc8700 1 -- 172.21.15.73:0/558344412 learned_addr learned my addr 172.21.15.73:0/558344412 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.236+0000 7f6db3fff700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f6dbc0cae90 0x7f6dbc0c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39588/0 (socket says 172.21.15.73:39588) 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc0bc8700 1 -- 172.21.15.73:0/558344412 >> v2:172.21.15.150:3300/0 conn(0x7f6dbc0ca180 msgr2=0x7f6dbc15e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc0bc8700 1 --2- 172.21.15.73:0/558344412 >> v2:172.21.15.150:3300/0 conn(0x7f6dbc0ca180 0x7f6dbc15e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc0bc8700 1 -- 172.21.15.73:0/558344412 >> v1:172.21.15.150:6789/0 conn(0x7f6dbc0cae90 legacy=0x7f6dbc0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc0bc8700 1 -- 172.21.15.73:0/558344412 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dbc0a9160 con 0x7f6dbc0caab0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc0bc8700 1 --2- 172.21.15.73:0/558344412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f6db8009f70 tx=0x7f6db8009840).ready entity=mon.2 client_cookie=227313ccd866053f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6db37fe700 1 -- 172.21.15.73:0/558344412 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800e070 con 0x7f6dbc0caab0 2023-11-28T04:08:22.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6db37fe700 1 -- 172.21.15.73:0/558344412 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db800ba20 con 0x7f6dbc0caab0 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6db37fe700 1 -- 172.21.15.73:0/558344412 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db800bda0 con 0x7f6dbc0caab0 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/558344412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 msgr2=0x7f6dbc0aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/558344412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 rx=0x7f6db8009f70 tx=0x7f6db8009840).stop 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/558344412 shutdown_connections 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/558344412 >> v2:172.21.15.150:3300/0 conn(0x7f6dbc0ca180 0x7f6dbc15e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/558344412 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0caab0 0x7f6dbc0aa470 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/558344412 >> 172.21.15.73:0/558344412 conn(0x7f6dbc094920 msgr2=0x7f6dbc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/558344412 shutdown_connections 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.237+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/558344412 wait complete. 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.238+0000 7f6dc2e2c700 1 Processor -- start 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 -- start start 2023-11-28T04:08:22.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0ca180 0x7f6dbc0ad470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6dbc0cae90 0x7f6dbc157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0c3400 con 0x7f6dbc0caab0 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0c5150 con 0x7f6dbc0cae90 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.239+0000 7f6dc2e2c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6dbc0c2df0 con 0x7f6dbc0ca180 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40422/0 (socket says 172.21.15.73:40422) 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 -- 172.21.15.73:0/97329156 learned_addr learned my addr 172.21.15.73:0/97329156 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6dc13c9700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6dbc0cae90 0x7f6dbc157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6dc0bc8700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0ca180 0x7f6dbc0ad470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 -- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0ca180 msgr2=0x7f6dbc0ad470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0ca180 0x7f6dbc0ad470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 -- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6dbc0cae90 msgr2=0x7f6dbc157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6dbc0cae90 0x7f6dbc157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 -- 172.21.15.73:0/97329156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db8009ae0 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db3fff700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f6dac00c440 tx=0x7f6dac007d60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0070d0 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dac007230 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dac0075b0 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6dbc0c4e70 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.240+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6dbc0ad960 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.241+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6dbc06c860 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.241+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6dac0058f0 con 0x7f6dbc0caab0 2023-11-28T04:08:22.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.242+0000 7f6db0ff9700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 0x7f6da8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.242+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6dac079780 con 0x7f6dbc0caab0 2023-11-28T04:08:22.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.242+0000 7f6dc0bc8700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 0x7f6da8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.242+0000 7f6dc0bc8700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 0x7f6da8061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6db800c010 tx=0x7f6db8009840).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.246+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6dac04aa50 con 0x7f6dbc0caab0 2023-11-28T04:08:22.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.406+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6dbc0c3540 con 0x7f6da805f150 2023-11-28T04:08:22.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.416+0000 7f6db0ff9700 1 -- 172.21.15.73:0/97329156 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353070 (secure 0 0 0) 0x7f6dbc0c3540 con 0x7f6da805f150 2023-11-28T04:08:22.420 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 msgr2=0x7f6da8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 0x7f6da8061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f6db800c010 tx=0x7f6db8009840).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 msgr2=0x7f6dbc156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 rx=0x7f6dac00c440 tx=0x7f6dac007d60).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 shutdown_connections 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6da805f150 0x7f6da8061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6dbc0cae90 0x7f6dbc157440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6dbc0ca180 0x7f6dbc0ad470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 --2- 172.21.15.73:0/97329156 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6dbc0caab0 0x7f6dbc156f50 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 >> 172.21.15.73:0/97329156 conn(0x7f6dbc094920 msgr2=0x7f6dbc0b0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 shutdown_connections 2023-11-28T04:08:22.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.420+0000 7f6dc2e2c700 1 -- 172.21.15.73:0/97329156 wait complete. 2023-11-28T04:08:22.424 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:08:22.432 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:08:21.829709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447132,"kb_used_data":17116,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752527588,"statfs":{"total":773094113280,"available":770588250112,"internally_reserved":0,"allocated":17526784,"data_stored":8810065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10874,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.221058"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084377+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085516+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090766+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085468+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088918+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084382+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086833+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083710+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085932+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087817+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086562+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087153+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088501+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087381+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087366+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084907+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087224+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090260+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086202+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083498+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086161+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086964+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084559+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086717+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083889+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085041+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083886+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086849+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084665+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086573+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085474+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083390+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085531+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087284+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086201+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085130+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085384+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085789+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088361+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085471+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083454+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086810+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085856+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090687+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086508+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086055+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086411+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084583+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090939+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087104+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085495+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086113+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087998+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083451+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087427+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086059+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085857+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938971+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:08:16.938971+0000","last_peered":"2023-11-28T04:08:16.938971+0000","last_clean":"2023-11-28T04:08:16.938971+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:08:16.938971+0000","last_undegraded":"2023-11-28T04:08:16.938971+0000","last_fullsized":"2023-11-28T04:08:16.938971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083446+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382553+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:08:17.382553+0000","last_peered":"2023-11-28T04:08:17.382553+0000","last_clean":"2023-11-28T04:08:17.382553+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:08:17.382553+0000","last_undegraded":"2023-11-28T04:08:17.382553+0000","last_fullsized":"2023-11-28T04:08:17.382553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087826+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382000+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:08:17.382000+0000","last_peered":"2023-11-28T04:08:17.382000+0000","last_clean":"2023-11-28T04:08:17.382000+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:08:17.382000+0000","last_undegraded":"2023-11-28T04:08:17.382000+0000","last_fullsized":"2023-11-28T04:08:17.382000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086460+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382551+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:08:17.382551+0000","last_peered":"2023-11-28T04:08:17.382551+0000","last_clean":"2023-11-28T04:08:17.382551+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:08:17.382551+0000","last_undegraded":"2023-11-28T04:08:17.382551+0000","last_fullsized":"2023-11-28T04:08:17.382551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395626+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:08:16.395626+0000","last_peered":"2023-11-28T04:08:16.395626+0000","last_clean":"2023-11-28T04:08:16.395626+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:08:16.395626+0000","last_undegraded":"2023-11-28T04:08:16.395626+0000","last_fullsized":"2023-11-28T04:08:16.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086320+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395614+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:08:16.395614+0000","last_peered":"2023-11-28T04:08:16.395614+0000","last_clean":"2023-11-28T04:08:16.395614+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:08:16.395614+0000","last_undegraded":"2023-11-28T04:08:16.395614+0000","last_fullsized":"2023-11-28T04:08:16.395614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.089557+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395621+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:08:16.395621+0000","last_peered":"2023-11-28T04:08:16.395621+0000","last_clean":"2023-11-28T04:08:16.395621+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:08:16.395621+0000","last_undegraded":"2023-11-28T04:08:16.395621+0000","last_fullsized":"2023-11-28T04:08:16.395621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090112+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938998+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:08:16.938998+0000","last_peered":"2023-11-28T04:08:16.938998+0000","last_clean":"2023-11-28T04:08:16.938998+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:08:16.938998+0000","last_undegraded":"2023-11-28T04:08:16.938998+0000","last_fullsized":"2023-11-28T04:08:16.938998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086959+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.077366+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:08:16.077366+0000","last_peered":"2023-11-28T04:08:16.077366+0000","last_clean":"2023-11-28T04:08:16.077366+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:08:16.077366+0000","last_undegraded":"2023-11-28T04:08:16.077366+0000","last_fullsized":"2023-11-28T04:08:16.077366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.29299999999999998},"max":{"1min":0.85999999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.623},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.35299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71299999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.501},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.878,"15min":0.878},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":0.875,"15min":0.875},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":0.73499999999999999,"15min":0.75},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.83099999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.63700000000000001,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.311},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.434},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303324,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068516,"statfs":{"total":96636764160,"available":96326160384,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.29199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.60299999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.46600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59999999999999998,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.57599999999999996,"15min":0.67300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42899999999999999,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.77500000000000002},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":1.24,"15min":1.24},"last":0.432},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91200000000000003,"15min":0.98899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.248,"15min":1.248},"last":0.439}]},{"osd":6,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.2,"15min":1.2},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.42599999999999999},"max":{"1min":0.81200000000000006,"5min":1.27,"15min":1.27},"last":0.40600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.627}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.371},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.59499999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.39600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58599999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.312},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":1.085},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93400000000000005,"15min":1.1200000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.97099999999999997,"15min":1.145},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.0189999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89100000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.497},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.624}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.77800000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.318,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.26600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:08:22.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:08:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.585+0000 7f205f59e700 1 Processor -- start 2023-11-28T04:08:22.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.585+0000 7f205f59e700 1 -- start start 2023-11-28T04:08:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205f59e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205f59e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20600c12c0 con 0x7f20600c0180 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205f59e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20600c06b0 con 0x7f2060157660 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20600c0990 con 0x7f20600c1750 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205ed9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205e59c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205dd9b700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f20600c0180 0x7f20601589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39600/0 (socket says 172.21.15.73:39600) 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205e59c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:44486/0 (socket says 172.21.15.73:44486) 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205ed9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:44178/0 (socket says 172.21.15.73:44178) 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205dd9b700 1 -- 172.21.15.73:0/2826829489 learned_addr learned my addr 172.21.15.73:0/2826829489 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205ed9d700 1 -- 172.21.15.73:0/2826829489 >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 msgr2=0x7f2060157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205ed9d700 1 --2- 172.21.15.73:0/2826829489 >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205ed9d700 1 -- 172.21.15.73:0/2826829489 >> v1:172.21.15.150:6789/0 conn(0x7f20600c0180 legacy=0x7f20601589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.588 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.586+0000 7f205d59a700 1 -- 172.21.15.73:0/2826829489 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 290113017 0 0) 0x7f20600c12c0 con 0x7f20600c0180 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205ed9d700 1 -- 172.21.15.73:0/2826829489 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20600c0c70 con 0x7f20600c1750 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205e59c700 1 --2- 172.21.15.73:0/2826829489 >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205ed9d700 1 --2- 172.21.15.73:0/2826829489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2058009e60 tx=0x7f2058009780).ready entity=mon.2 client_cookie=e4464256f0456e70 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205d59a700 1 -- 172.21.15.73:0/2826829489 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205800e070 con 0x7f20600c1750 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205d59a700 1 -- 172.21.15.73:0/2826829489 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205800bb60 con 0x7f20600c1750 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205d59a700 1 -- 172.21.15.73:0/2826829489 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2058012410 con 0x7f20600c1750 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 -- 172.21.15.73:0/2826829489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 msgr2=0x7f20600c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 --2- 172.21.15.73:0/2826829489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0x7f2058009e60 tx=0x7f2058009780).stop 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 -- 172.21.15.73:0/2826829489 shutdown_connections 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 --2- 172.21.15.73:0/2826829489 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c1bc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 --2- 172.21.15.73:0/2826829489 >> v2:172.21.15.150:3300/0 conn(0x7f2060157660 0x7f2060157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 -- 172.21.15.73:0/2826829489 >> 172.21.15.73:0/2826829489 conn(0x7f20600cb000 msgr2=0x7f20600cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 -- 172.21.15.73:0/2826829489 shutdown_connections 2023-11-28T04:08:22.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.587+0000 7f205f59e700 1 -- 172.21.15.73:0/2826829489 wait complete. 2023-11-28T04:08:22.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.588+0000 7f205f59e700 1 Processor -- start 2023-11-28T04:08:22.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 -- start start 2023-11-28T04:08:22.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 0x7f2060151ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2060152ef0 con 0x7f2060157660 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20601522e0 con 0x7f20600c0180 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205f59e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20601525c0 con 0x7f20600c1750 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40428/0 (socket says 172.21.15.73:40428) 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205ed9d700 1 -- 172.21.15.73:0/3204445688 learned_addr learned my addr 172.21.15.73:0/3204445688 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.589+0000 7f205dd9b700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205e59c700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 0x7f2060151ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 -- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 msgr2=0x7f20600c44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c44e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 -- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 msgr2=0x7f2060151ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 0x7f2060151ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 -- 172.21.15.73:0/3204445688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20500092a0 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205e59c700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 0x7f2060151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205ed9d700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f205800b6c0 tx=0x7f2058009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205801d630 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f205801d790 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f205801db50 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2058009a20 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.590+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2060152b80 con 0x7f2060157660 2023-11-28T04:08:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.591+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f206006c7c0 con 0x7f2060157660 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.591+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2058003040 con 0x7f2060157660 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.591+0000 7f204e7fc700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 0x7f2040061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.592+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f205807c000 con 0x7f2060157660 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.592+0000 7f205e59c700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 0x7f2040061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.592+0000 7f205e59c700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 0x7f2040061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2050009270 tx=0x7f2050008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.596+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f205804d2d0 con 0x7f2060157660 2023-11-28T04:08:22.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.753+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20600c0f50 con 0x7f204005f150 2023-11-28T04:08:22.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.762+0000 7f204e7fc700 1 -- 172.21.15.73:0/3204445688 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353070 (secure 0 0 0) 0x7f20600c0f50 con 0x7f204005f150 2023-11-28T04:08:22.766 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 msgr2=0x7f2040061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 0x7f2040061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f2050009270 tx=0x7f2050008040).stop 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 msgr2=0x7f2060150a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 rx=0x7f205800b6c0 tx=0x7f2058009780).stop 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 shutdown_connections 2023-11-28T04:08:22.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f204005f150 0x7f2040061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2060157660 0x7f2060150a20 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20600c0180 0x7f2060151ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 --2- 172.21.15.73:0/3204445688 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20600c1750 0x7f20600c44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 >> 172.21.15.73:0/3204445688 conn(0x7f20600cb000 msgr2=0x7f20600aae50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 shutdown_connections 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.766+0000 7f205f59e700 1 -- 172.21.15.73:0/3204445688 wait complete. 2023-11-28T04:08:22.770 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:08:22.778 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:08:21.829709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447132,"kb_used_data":17116,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752527588,"statfs":{"total":773094113280,"available":770588250112,"internally_reserved":0,"allocated":17526784,"data_stored":8810065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10874,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.221058"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084377+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085516+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090766+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085468+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088918+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084382+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086833+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083710+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085932+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087817+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086562+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087153+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088501+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087381+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087366+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084907+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087224+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090260+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086202+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083498+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086161+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086964+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084559+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086717+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083889+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085041+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083886+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086849+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084665+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086573+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085474+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083390+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085531+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087284+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086201+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085130+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085384+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085789+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088361+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085471+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083454+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086810+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085856+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090687+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086508+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086055+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086411+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084583+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090939+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087104+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085495+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086113+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087998+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083451+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087427+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086059+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085857+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938971+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:08:16.938971+0000","last_peered":"2023-11-28T04:08:16.938971+0000","last_clean":"2023-11-28T04:08:16.938971+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:08:16.938971+0000","last_undegraded":"2023-11-28T04:08:16.938971+0000","last_fullsized":"2023-11-28T04:08:16.938971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083446+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382553+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:08:17.382553+0000","last_peered":"2023-11-28T04:08:17.382553+0000","last_clean":"2023-11-28T04:08:17.382553+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:08:17.382553+0000","last_undegraded":"2023-11-28T04:08:17.382553+0000","last_fullsized":"2023-11-28T04:08:17.382553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087826+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382000+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:08:17.382000+0000","last_peered":"2023-11-28T04:08:17.382000+0000","last_clean":"2023-11-28T04:08:17.382000+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:08:17.382000+0000","last_undegraded":"2023-11-28T04:08:17.382000+0000","last_fullsized":"2023-11-28T04:08:17.382000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086460+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382551+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:08:17.382551+0000","last_peered":"2023-11-28T04:08:17.382551+0000","last_clean":"2023-11-28T04:08:17.382551+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:08:17.382551+0000","last_undegraded":"2023-11-28T04:08:17.382551+0000","last_fullsized":"2023-11-28T04:08:17.382551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395626+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:08:16.395626+0000","last_peered":"2023-11-28T04:08:16.395626+0000","last_clean":"2023-11-28T04:08:16.395626+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:08:16.395626+0000","last_undegraded":"2023-11-28T04:08:16.395626+0000","last_fullsized":"2023-11-28T04:08:16.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086320+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395614+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:08:16.395614+0000","last_peered":"2023-11-28T04:08:16.395614+0000","last_clean":"2023-11-28T04:08:16.395614+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:08:16.395614+0000","last_undegraded":"2023-11-28T04:08:16.395614+0000","last_fullsized":"2023-11-28T04:08:16.395614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.089557+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395621+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:08:16.395621+0000","last_peered":"2023-11-28T04:08:16.395621+0000","last_clean":"2023-11-28T04:08:16.395621+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:08:16.395621+0000","last_undegraded":"2023-11-28T04:08:16.395621+0000","last_fullsized":"2023-11-28T04:08:16.395621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090112+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938998+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:08:16.938998+0000","last_peered":"2023-11-28T04:08:16.938998+0000","last_clean":"2023-11-28T04:08:16.938998+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:08:16.938998+0000","last_undegraded":"2023-11-28T04:08:16.938998+0000","last_fullsized":"2023-11-28T04:08:16.938998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086959+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.077366+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:08:16.077366+0000","last_peered":"2023-11-28T04:08:16.077366+0000","last_clean":"2023-11-28T04:08:16.077366+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:08:16.077366+0000","last_undegraded":"2023-11-28T04:08:16.077366+0000","last_fullsized":"2023-11-28T04:08:16.077366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.29299999999999998},"max":{"1min":0.85999999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.623},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.35299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71299999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.501},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.878,"15min":0.878},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":0.875,"15min":0.875},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":0.73499999999999999,"15min":0.75},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.83099999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.63700000000000001,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.311},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.434},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303324,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068516,"statfs":{"total":96636764160,"available":96326160384,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.29199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.60299999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.46600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59999999999999998,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.57599999999999996,"15min":0.67300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42899999999999999,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.77500000000000002},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":1.24,"15min":1.24},"last":0.432},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91200000000000003,"15min":0.98899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.248,"15min":1.248},"last":0.439}]},{"osd":6,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.2,"15min":1.2},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.42599999999999999},"max":{"1min":0.81200000000000006,"5min":1.27,"15min":1.27},"last":0.40600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.627}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.371},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.59499999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.39600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58599999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.312},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":1.085},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93400000000000005,"15min":1.1200000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.97099999999999997,"15min":1.145},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.0189999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89100000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.497},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.624}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.77800000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.318,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.26600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:08:22.783 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:08:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.931+0000 7f0d8e6d5700 1 Processor -- start 2023-11-28T04:08:22.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.931+0000 7f0d8e6d5700 1 -- start start 2023-11-28T04:08:22.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8e6d5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 0x7f0d880aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8e6d5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d880cae90 0x7f0d880c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8e6d5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d880a9a90 con 0x7f0d880aa960 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8e6d5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d88154ce0 con 0x7f0d880cae90 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8e6d5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d880a8e80 con 0x7f0d880caab0 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 0x7f0d880aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d877fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d880cae90 0x7f0d880c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8cc72700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f0d880aa960 0x7f0d880c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:39616/0 (socket says 172.21.15.73:39616) 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.932+0000 7f0d8cc72700 1 -- 172.21.15.73:0/2982028859 learned_addr learned my addr 172.21.15.73:0/2982028859 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d87fff700 1 -- 172.21.15.73:0/2982028859 >> v2:172.21.15.150:3300/0 conn(0x7f0d880cae90 msgr2=0x7f0d880c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d87fff700 1 --2- 172.21.15.73:0/2982028859 >> v2:172.21.15.150:3300/0 conn(0x7f0d880cae90 0x7f0d880c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d87fff700 1 -- 172.21.15.73:0/2982028859 >> v1:172.21.15.150:6789/0 conn(0x7f0d880aa960 legacy=0x7f0d880c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d87fff700 1 -- 172.21.15.73:0/2982028859 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d880a9160 con 0x7f0d880caab0 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d87fff700 1 --2- 172.21.15.73:0/2982028859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 0x7f0d880aa470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f0d78006fd0 tx=0x7f0d78006f40).ready entity=mon.2 client_cookie=48ceeac3edf379f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d86ffd700 1 -- 172.21.15.73:0/2982028859 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78005880 con 0x7f0d880caab0 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d86ffd700 1 -- 172.21.15.73:0/2982028859 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d780059e0 con 0x7f0d880caab0 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d86ffd700 1 -- 172.21.15.73:0/2982028859 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d78005d60 con 0x7f0d880caab0 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/2982028859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 msgr2=0x7f0d880aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/2982028859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 0x7f0d880aa470 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0x7f0d78006fd0 tx=0x7f0d78006f40).stop 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/2982028859 shutdown_connections 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/2982028859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880caab0 0x7f0d880aa470 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/2982028859 >> v2:172.21.15.150:3300/0 conn(0x7f0d880cae90 0x7f0d880c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/2982028859 >> 172.21.15.73:0/2982028859 conn(0x7f0d88094920 msgr2=0x7f0d8806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/2982028859 shutdown_connections 2023-11-28T04:08:22.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.933+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/2982028859 wait complete. 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 Processor -- start 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 -- start start 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d880cae90 0x7f0d88157b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880c50d0 0x7f0d880af210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d881576f0 con 0x7f0d880caab0 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88156ae0 con 0x7f0d880cae90 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.935+0000 7f0d8e6d5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d88156dc0 con 0x7f0d880c50d0 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40444/0 (socket says 172.21.15.73:40444) 2023-11-28T04:08:22.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 -- 172.21.15.73:0/58537443 learned_addr learned my addr 172.21.15.73:0/58537443 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d8cc72700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880c50d0 0x7f0d880af210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d877fe700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d880cae90 0x7f0d88157b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 -- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880c50d0 msgr2=0x7f0d880af210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880c50d0 0x7f0d880af210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 -- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d880cae90 msgr2=0x7f0d88157b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d880cae90 0x7f0d88157b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 -- 172.21.15.73:0/58537443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d78009250 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d87fff700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f0d880a8fc0 tx=0x7f0d78016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800f720 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7800f8a0 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d881570a0 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7800fc60 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.936+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d88158070 con 0x7f0d880caab0 2023-11-28T04:08:22.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.937+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d881579d0 con 0x7f0d880caab0 2023-11-28T04:08:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.938+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0d78011650 con 0x7f0d880caab0 2023-11-28T04:08:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.938+0000 7f0d6ffff700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 0x7f0d74061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.938+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0d7807b5f0 con 0x7f0d880caab0 2023-11-28T04:08:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.938+0000 7f0d877fe700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 0x7f0d74061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:22.940 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.938+0000 7f0d877fe700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 0x7f0d74061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0d880b1330 tx=0x7f0d7c009060).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:22.942 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:22.942+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d7804c8c0 con 0x7f0d880caab0 2023-11-28T04:08:23.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.102+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d88157830 con 0x7f0d7405f150 2023-11-28T04:08:23.110 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.111+0000 7f0d6ffff700 1 -- 172.21.15.73:0/58537443 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353070 (secure 0 0 0) 0x7f0d7c0097f0 con 0x7f0d7405f150 2023-11-28T04:08:23.115 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 msgr2=0x7f0d74061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 0x7f0d74061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f0d880b1330 tx=0x7f0d7c009060).stop 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 msgr2=0x7f0d88152c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 rx=0x7f0d880a8fc0 tx=0x7f0d78016000).stop 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 shutdown_connections 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0d7405f150 0x7f0d74061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d880c50d0 0x7f0d880af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d880caab0 0x7f0d88152c60 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 --2- 172.21.15.73:0/58537443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d880cae90 0x7f0d88157b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 >> 172.21.15.73:0/58537443 conn(0x7f0d88094920 msgr2=0x7f0d880aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 shutdown_connections 2023-11-28T04:08:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.114+0000 7f0d8e6d5700 1 -- 172.21.15.73:0/58537443 wait complete. 2023-11-28T04:08:23.119 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:08:23.126 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:08:21.829709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447132,"kb_used_data":17116,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752527588,"statfs":{"total":773094113280,"available":770588250112,"internally_reserved":0,"allocated":17526784,"data_stored":8810065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10874,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.221058"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084377+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085516+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090766+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085468+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088918+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084382+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086833+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083710+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085932+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087817+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086562+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087153+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088501+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087381+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087366+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084907+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087224+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090260+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086202+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083498+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086161+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086964+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084559+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086717+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083889+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085041+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083886+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086849+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084665+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086573+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085474+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083390+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085531+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087284+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086201+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085130+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085384+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085789+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088361+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085471+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083454+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086810+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085856+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090687+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086508+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086055+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086411+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084583+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090939+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087104+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085495+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086113+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087998+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083451+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087427+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086059+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085857+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938971+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:08:16.938971+0000","last_peered":"2023-11-28T04:08:16.938971+0000","last_clean":"2023-11-28T04:08:16.938971+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:08:16.938971+0000","last_undegraded":"2023-11-28T04:08:16.938971+0000","last_fullsized":"2023-11-28T04:08:16.938971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083446+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382553+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:08:17.382553+0000","last_peered":"2023-11-28T04:08:17.382553+0000","last_clean":"2023-11-28T04:08:17.382553+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:08:17.382553+0000","last_undegraded":"2023-11-28T04:08:17.382553+0000","last_fullsized":"2023-11-28T04:08:17.382553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087826+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382000+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:08:17.382000+0000","last_peered":"2023-11-28T04:08:17.382000+0000","last_clean":"2023-11-28T04:08:17.382000+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:08:17.382000+0000","last_undegraded":"2023-11-28T04:08:17.382000+0000","last_fullsized":"2023-11-28T04:08:17.382000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086460+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382551+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:08:17.382551+0000","last_peered":"2023-11-28T04:08:17.382551+0000","last_clean":"2023-11-28T04:08:17.382551+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:08:17.382551+0000","last_undegraded":"2023-11-28T04:08:17.382551+0000","last_fullsized":"2023-11-28T04:08:17.382551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395626+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:08:16.395626+0000","last_peered":"2023-11-28T04:08:16.395626+0000","last_clean":"2023-11-28T04:08:16.395626+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:08:16.395626+0000","last_undegraded":"2023-11-28T04:08:16.395626+0000","last_fullsized":"2023-11-28T04:08:16.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086320+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395614+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:08:16.395614+0000","last_peered":"2023-11-28T04:08:16.395614+0000","last_clean":"2023-11-28T04:08:16.395614+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:08:16.395614+0000","last_undegraded":"2023-11-28T04:08:16.395614+0000","last_fullsized":"2023-11-28T04:08:16.395614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.089557+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395621+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:08:16.395621+0000","last_peered":"2023-11-28T04:08:16.395621+0000","last_clean":"2023-11-28T04:08:16.395621+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:08:16.395621+0000","last_undegraded":"2023-11-28T04:08:16.395621+0000","last_fullsized":"2023-11-28T04:08:16.395621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090112+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938998+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:08:16.938998+0000","last_peered":"2023-11-28T04:08:16.938998+0000","last_clean":"2023-11-28T04:08:16.938998+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:08:16.938998+0000","last_undegraded":"2023-11-28T04:08:16.938998+0000","last_fullsized":"2023-11-28T04:08:16.938998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086959+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.077366+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:08:16.077366+0000","last_peered":"2023-11-28T04:08:16.077366+0000","last_clean":"2023-11-28T04:08:16.077366+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:08:16.077366+0000","last_undegraded":"2023-11-28T04:08:16.077366+0000","last_fullsized":"2023-11-28T04:08:16.077366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.29299999999999998},"max":{"1min":0.85999999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.623},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.35299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71299999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.501},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.878,"15min":0.878},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":0.875,"15min":0.875},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":0.73499999999999999,"15min":0.75},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.83099999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.63700000000000001,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.311},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.434},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303324,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068516,"statfs":{"total":96636764160,"available":96326160384,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.29199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.60299999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.46600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59999999999999998,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.57599999999999996,"15min":0.67300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42899999999999999,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.77500000000000002},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":1.24,"15min":1.24},"last":0.432},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91200000000000003,"15min":0.98899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.248,"15min":1.248},"last":0.439}]},{"osd":6,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.2,"15min":1.2},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.42599999999999999},"max":{"1min":0.81200000000000006,"5min":1.27,"15min":1.27},"last":0.40600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.627}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.371},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.59499999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.39600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58599999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.312},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":1.085},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93400000000000005,"15min":1.1200000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.97099999999999997,"15min":1.145},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.0189999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89100000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.497},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.624}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.77800000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.318,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.26600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:08:23.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3c is not active+clean 2023-11-28T04:08:23.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3f is not active+clean 2023-11-28T04:08:23.132 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084377+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.132 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3d is not active+clean 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3e is not active+clean 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085516+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3e is not active+clean 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3d is not active+clean 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.090766+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3f is not active+clean 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.133 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3c is not active+clean 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085468+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.38 is not active+clean 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3b is not active+clean 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.088918+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.39 is not active+clean 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3a is not active+clean 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084382+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.134 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3a is not active+clean 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.39 is not active+clean 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086833+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3b is not active+clean 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.38 is not active+clean 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083710+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.34 is not active+clean 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.135 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.37 is not active+clean 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085932+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.35 is not active+clean 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.36 is not active+clean 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087817+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.36 is not active+clean 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.35 is not active+clean 2023-11-28T04:08:23.136 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086562+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.37 is not active+clean 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.34 is not active+clean 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087153+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.30 is not active+clean 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.137 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.33 is not active+clean 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.088501+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.31 is not active+clean 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.32 is not active+clean 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087381+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.32 is not active+clean 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.31 is not active+clean 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087366+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.138 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.33 is not active+clean 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.30 is not active+clean 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084907+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2c is not active+clean 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2f is not active+clean 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087224+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2d is not active+clean 2023-11-28T04:08:23.139 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2e is not active+clean 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.090260+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2e is not active+clean 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2d is not active+clean 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086202+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2f is not active+clean 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2c is not active+clean 2023-11-28T04:08:23.140 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083498+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.28 is not active+clean 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2b is not active+clean 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086161+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.29 is not active+clean 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2a is not active+clean 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086964+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [3], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2a is not active+clean 2023-11-28T04:08:23.141 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.29 is not active+clean 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084559+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2b is not active+clean 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.28 is not active+clean 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086717+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 4], 'acting': [7, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.24 is not active+clean 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.27 is not active+clean 2023-11-28T04:08:23.142 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083889+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.25 is not active+clean 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.26 is not active+clean 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085041+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.26 is not active+clean 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.25 is not active+clean 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083886+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.27 is not active+clean 2023-11-28T04:08:23.143 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.24 is not active+clean 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086849+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.20 is not active+clean 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.23 is not active+clean 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084665+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.21 is not active+clean 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.22 is not active+clean 2023-11-28T04:08:23.144 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086573+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.22 is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.21 is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085474+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.23 is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.20 is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083390+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1c is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1f is not active+clean 2023-11-28T04:08:23.145 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085531+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1d is not active+clean 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1e is not active+clean 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087284+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1e is not active+clean 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1d is not active+clean 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086201+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1f is not active+clean 2023-11-28T04:08:23.146 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1c is not active+clean 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085130+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.18 is not active+clean 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1b is not active+clean 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085384+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.19 is not active+clean 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1a is not active+clean 2023-11-28T04:08:23.147 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085789+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1a is not active+clean 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.19 is not active+clean 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.088361+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1b is not active+clean 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.18 is not active+clean 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085471+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [5], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.c is not active+clean 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.148 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.f is not active+clean 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083454+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.d is not active+clean 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.e is not active+clean 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086810+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 1], 'acting': [7, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.e is not active+clean 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.d is not active+clean 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085856+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.149 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.f is not active+clean 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.c is not active+clean 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.090687+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.8 is not active+clean 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.b is not active+clean 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086508+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.9 is not active+clean 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.150 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.a is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086055+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [1], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.a is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.9 is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086411+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.b is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.8 is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.084583+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.4 is not active+clean 2023-11-28T04:08:23.151 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.7 is not active+clean 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.090939+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.5 is not active+clean 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.6 is not active+clean 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087104+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.6 is not active+clean 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.5 is not active+clean 2023-11-28T04:08:23.152 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085495+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.7 is not active+clean 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.4 is not active+clean 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086113+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.0 is not active+clean 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.3 is not active+clean 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087998+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [6], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.1 is not active+clean 2023-11-28T04:08:23.153 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.2 is not active+clean 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083451+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.2 is not active+clean 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.1 is not active+clean 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087427+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.3 is not active+clean 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.0 is not active+clean 2023-11-28T04:08:23.154 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086059+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.10 is not active+clean 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.13 is not active+clean 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.085857+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 0], 'acting': [3, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.16 is not active+clean 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.15 is not active+clean 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.083446+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.15 is not active+clean 2023-11-28T04:08:23.155 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.16 is not active+clean 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.087826+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 2], 'acting': [5, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.17 is not active+clean 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.14 is not active+clean 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086460+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.14 is not active+clean 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.17 is not active+clean 2023-11-28T04:08:23.156 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086320+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [0], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.11 is not active+clean 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.12 is not active+clean 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.089557+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [4], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.12 is not active+clean 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.11 is not active+clean 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.090112+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 2], 'acting': [6, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:PG 22.13 is not active+clean 2023-11-28T04:08:23.157 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '22.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-11-28T04:08:19.080346+0000', 'last_change': '2023-11-28T04:08:19.080346+0000', 'last_active': '2023-11-28T04:08:19.080346+0000', 'last_peered': '2023-11-28T04:08:19.080346+0000', 'last_clean': '2023-11-28T04:08:19.080346+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:19.080346+0000', 'last_undegraded': '2023-11-28T04:08:19.080346+0000', 'last_fullsized': '2023-11-28T04:08:19.080346+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:19.080346+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-11-28T04:08:23.158 INFO:tasks.cephfs.filesystem.ceph_manager:PG 21.10 is not active+clean 2023-11-28T04:08:23.158 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '21.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'creating+peering', 'last_fresh': '2023-11-28T04:08:18.075747+0000', 'last_change': '2023-11-28T04:08:18.086959+0000', 'last_active': '2023-11-28T04:08:18.075747+0000', 'last_peered': '2023-11-28T04:08:18.075747+0000', 'last_clean': '2023-11-28T04:08:18.075747+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-11-28T04:08:18.075747+0000', 'last_undegraded': '2023-11-28T04:08:18.075747+0000', 'last_fullsized': '2023-11-28T04:08:18.075747+0000', 'mapping_epoch': 176, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 176, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'last_clean_scrub_stamp': '2023-11-28T04:08:18.075747+0000', 'log_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-11-28T04:08:23.158 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-11-28T04:08:23.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.278+0000 7f0fcc2b2700 1 Processor -- start 2023-11-28T04:08:23.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.279+0000 7f0fcc2b2700 1 -- start start 2023-11-28T04:08:23.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.279+0000 7f0fcc2b2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0fc4066e00 0x7f0fc40671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.279+0000 7f0fcc2b2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fc4067df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fcc2b2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0fc40bd590 con 0x7f0fc4155e40 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fcc2b2700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0fc40a6090 con 0x7f0fc4066e00 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fcc2b2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40682e0 con 0x7f0fc4067980 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc984d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f0fc4155e40 0x7f0fc40a5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:44282/0 (socket says 172.21.15.73:44282) 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc984d700 1 -- 172.21.15.73:0/4043409164 learned_addr learned my addr 172.21.15.73:0/4043409164 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> v2:172.21.15.150:3300/0 conn(0x7f0fc4066e00 0x7f0fc40671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fca84f700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fc4067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953974201 0 0) 0x7f0fc40bd590 con 0x7f0fc4155e40 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0fb00033f0 con 0x7f0fc4155e40 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1936031754 0 0) 0x7f0fb00033f0 con 0x7f0fc4155e40 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 msgr2=0x7f0fc4067df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:23.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fc4067df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 >> v2:172.21.15.150:3300/0 conn(0x7f0fc4066e00 msgr2=0x7f0fc40671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 --2- 172.21.15.73:0/4043409164 >> v2:172.21.15.150:3300/0 conn(0x7f0fc4066e00 0x7f0fc40671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc4156370 con 0x7f0fc4155e40 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f0fb4001470 con 0x7f0fc4155e40 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> v2:172.21.15.150:3300/0 conn(0x7f0fc4066e00 0x7f0fc40671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 >> v1:172.21.15.73:6789/0 conn(0x7f0fc4155e40 legacy=0x7f0fc40a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb00038e0 0x7f0fb0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb0004690 0x7f0fb0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fb0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40a6090 con 0x7f0fb00038e0 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40682e0 con 0x7f0fb0004690 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fb000b8e0 con 0x7f0fc4067980 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb00038e0 0x7f0fb0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.280+0000 7f0fc984d700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb0004690 0x7f0fb0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca84f700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fb0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 -- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 msgr2=0x7f0fb0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc4067980 0x7f0fb0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 -- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb0004690 msgr2=0x7f0fb0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb0004690 0x7f0fb0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 -- 172.21.15.73:0/4043409164 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fb00033f0 con 0x7f0fb00038e0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca04e700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb00038e0 0x7f0fb0003d00 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f0fbc009c70 tx=0x7f0fbc009060).ready entity=mon.0 client_cookie=4b204ec6bfa7ac89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0095c0 con 0x7f0fb00038e0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fbc00e6f0 con 0x7f0fb00038e0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fc904c700 1 -- 172.21.15.73:0/4043409164 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc00ea70 con 0x7f0fb00038e0 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/4043409164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb00038e0 msgr2=0x7f0fb0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/4043409164 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fb00038e0 0x7f0fb0003d00 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 rx=0x7f0fbc009c70 tx=0x7f0fbc009060).stop 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fca84f700 1 -- 172.21.15.73:0/4043409164 reap_dead start 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/4043409164 shutdown_connections 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/4043409164 >> 172.21.15.73:0/4043409164 conn(0x7f0fc40c7fd0 msgr2=0x7f0fc40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/4043409164 shutdown_connections 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.281+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/4043409164 wait complete. 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.282+0000 7f0fcc2b2700 1 Processor -- start 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 -- start start 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb00038e0 0x7f0fc40adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 0x7f0fc40c8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40ab090 con 0x7f0fc4154590 2023-11-28T04:08:23.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40adfb0 con 0x7f0fb00038e0 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fcc2b2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0fc40c8d50 con 0x7f0fc40c83e0 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fc984d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fc984d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40462/0 (socket says 172.21.15.73:40462) 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fc984d700 1 -- 172.21.15.73:0/1806748197 learned_addr learned my addr 172.21.15.73:0/1806748197 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fca04e700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb00038e0 0x7f0fc40adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.283+0000 7f0fca84f700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 0x7f0fc40c8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 -- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 msgr2=0x7f0fc40c8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 0x7f0fc40c8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 -- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb00038e0 msgr2=0x7f0fc40adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb00038e0 0x7f0fc40adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 -- 172.21.15.73:0/1806748197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0fc40ad2d0 con 0x7f0fc4154590 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fca84f700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 0x7f0fc40c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:23.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fc984d700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f0fb400e840 tx=0x7f0fb4001df0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4010520 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0fb4010680 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0fb4010a20 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0fc4154cf0 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.284+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0fc41552b0 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.285+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0fc40ab240 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.285+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0fb4014430 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.286+0000 7f0fba7fc700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 0x7f0fa8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.286+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f0fb4079d90 con 0x7f0fc4154590 2023-11-28T04:08:23.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.286+0000 7f0fca04e700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 0x7f0fa8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.286+0000 7f0fca04e700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 0x7f0fa8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f0fbc006d50 tx=0x7f0fbc01a000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.290+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0fb4049af0 con 0x7f0fc4154590 2023-11-28T04:08:23.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.518+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f0fc406c860 con 0x7f0fc4154590 2023-11-28T04:08:23.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.518+0000 7f0fba7fc700 1 -- 172.21.15.73:0/1806748197 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1113 (secure 0 0 0) 0x7f0fc40ab240 con 0x7f0fc4154590 2023-11-28T04:08:23.518 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:23.518 INFO:teuthology.orchestra.run.smithi073.stdout:{"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":4557,"monmap":{"epoch":1,"min_mon_release_name":"pacific","num_mons":3},"osdmap":{"epoch":179,"num_osds":8,"num_up_osds":8,"osd_up_since":1701139955,"num_in_osds":8,"osd_in_since":1701139948,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"unknown","count":64},{"state_name":"active+clean","count":9}],"num_pgs":137,"num_pools":4,"num_objects":0,"data_bytes":0,"bytes_used":2505863168,"bytes_avail":770588250112,"bytes_total":773094113280,"unknown_pgs_ratio":0.46715328097343445,"inactive_pgs_ratio":0.46715328097343445},"fsmap":{"epoch":64,"id":10,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":10,"rank":0,"name":"a","status":"up:active","gid":9006}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":11,"modified":"2023-11-28T03:57:37.317486+0000","services":{}},"progress_events":{}} 2023-11-28T04:08:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 msgr2=0x7f0fa8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 0x7f0fa8061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7f0fbc006d50 tx=0x7f0fbc01a000).stop 2023-11-28T04:08:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 msgr2=0x7f0fc40acde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 rx=0x7f0fb400e840 tx=0x7f0fb4001df0).stop 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 shutdown_connections 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f0fa805f150 0x7f0fa8061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0fc40c83e0 0x7f0fc40c8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0fb00038e0 0x7f0fc40adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 --2- 172.21.15.73:0/1806748197 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0fc4154590 0x7f0fc40acde0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 >> 172.21.15.73:0/1806748197 conn(0x7f0fc40c7fd0 msgr2=0x7f0fc4150c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:23.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 shutdown_connections 2023-11-28T04:08:23.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.520+0000 7f0fcc2b2700 1 -- 172.21.15.73:0/1806748197 wait complete. 2023-11-28T04:08:23.532 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-11-28T04:08:23.532 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:08:23.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.646+0000 7f64d0af5700 1 Processor -- start 2023-11-28T04:08:23.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.646+0000 7f64d0af5700 1 -- start start 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64d0af5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f64cc157660 0x7f64cc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64d0af5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc1589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64d0af5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64cc0c12c0 con 0x7f64cc0c1750 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64d0af5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64cc0c06b0 con 0x7f64cc157660 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64d0af5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0c0990 con 0x7f64cc0c0180 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64cbfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f64cc0c1750 0x7f64cc0648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59746/0 (socket says 172.21.15.73:59746) 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64cbfff700 1 -- 172.21.15.73:0/2387807579 learned_addr learned my addr 172.21.15.73:0/2387807579 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64cb7fe700 1 --2- 172.21.15.73:0/2387807579 >> v2:172.21.15.150:3300/0 conn(0x7f64cc157660 0x7f64cc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64caffd700 1 --2- 172.21.15.73:0/2387807579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc1589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3344148564 0 0) 0x7f64cc0c12c0 con 0x7f64cc0c1750 2023-11-28T04:08:23.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.647+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64b40033f0 con 0x7f64cc0c1750 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64caffd700 1 -- 172.21.15.73:0/2387807579 >> v2:172.21.15.150:3300/0 conn(0x7f64cc157660 msgr2=0x7f64cc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64caffd700 1 --2- 172.21.15.73:0/2387807579 >> v2:172.21.15.150:3300/0 conn(0x7f64cc157660 0x7f64cc157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64caffd700 1 -- 172.21.15.73:0/2387807579 >> v1:172.21.15.73:6789/0 conn(0x7f64cc0c1750 legacy=0x7f64cc0648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1682665716 0 0) 0x7f64b40033f0 con 0x7f64cc0c1750 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64caffd700 1 -- 172.21.15.73:0/2387807579 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64cc0c0c70 con 0x7f64cc0c0180 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64caffd700 1 --2- 172.21.15.73:0/2387807579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc1589c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f64c0009760 tx=0x7f64c0009060).ready entity=mon.2 client_cookie=3ce400d8c6580b7c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4001470 con 0x7f64cc0c0180 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c0011400 con 0x7f64cc0c0180 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64ca7fc700 1 -- 172.21.15.73:0/2387807579 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c0011780 con 0x7f64cc0c0180 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64d0af5700 1 -- 172.21.15.73:0/2387807579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 msgr2=0x7f64cc1589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.648+0000 7f64d0af5700 1 --2- 172.21.15.73:0/2387807579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc1589c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 rx=0x7f64c0009760 tx=0x7f64c0009060).stop 2023-11-28T04:08:23.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 -- 172.21.15.73:0/2387807579 shutdown_connections 2023-11-28T04:08:23.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 --2- 172.21.15.73:0/2387807579 >> v2:172.21.15.150:3300/0 conn(0x7f64cc157660 0x7f64cc157a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 --2- 172.21.15.73:0/2387807579 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc1589c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 -- 172.21.15.73:0/2387807579 >> 172.21.15.73:0/2387807579 conn(0x7f64cc0cb000 msgr2=0x7f64cc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:23.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 -- 172.21.15.73:0/2387807579 shutdown_connections 2023-11-28T04:08:23.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.649+0000 7f64d0af5700 1 -- 172.21.15.73:0/2387807579 wait complete. 2023-11-28T04:08:23.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.650+0000 7f64d0af5700 1 Processor -- start 2023-11-28T04:08:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.650+0000 7f64d0af5700 1 -- start start 2023-11-28T04:08:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc0aa620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64cc0c1750 0x7f64cc156840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0aab10 con 0x7f64cc157660 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc0c4a90 con 0x7f64cc0c1750 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64cc1536f0 con 0x7f64cc0c0180 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40054/0 (socket says 172.21.15.73:40054) 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 -- 172.21.15.73:0/3776089531 learned_addr learned my addr 172.21.15.73:0/3776089531 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cb7fe700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc0aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64caffd700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64cc0c1750 0x7f64cc156840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 -- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 msgr2=0x7f64cc0aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc0aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 -- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64cc0c1750 msgr2=0x7f64cc156840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64cc0c1750 0x7f64cc156840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 -- 172.21.15.73:0/3776089531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c00092a0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64cbfff700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f64c400c810 tx=0x7f64c4001aa0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4007170 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c40072d0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64cc1539d0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c40048c0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.651+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64cc153f90 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.652+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64cc0c1bc0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.653+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f64c4004cf0 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.653+0000 7f64b37fe700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 0x7f64b8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.653+0000 7f64cb7fe700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 0x7f64b8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.653+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f64c4079c20 con 0x7f64cc157660 2023-11-28T04:08:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.653+0000 7f64cb7fe700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 0x7f64b8061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f64bc009fd0 tx=0x7f64bc00b040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.657+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f64cc0c1bc0 con 0x7f64cc157660 2023-11-28T04:08:23.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.817+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f64cc0c1bc0 con 0x7f64b805f150 2023-11-28T04:08:23.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.826+0000 7f64b37fe700 1 -- 172.21.15.73:0/3776089531 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+353070 (secure 0 0 0) 0x7f64cc0c1bc0 con 0x7f64b805f150 2023-11-28T04:08:23.830 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:23.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.829+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 msgr2=0x7f64b8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.829+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 0x7f64b8061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f64bc009fd0 tx=0x7f64bc00b040).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.829+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 msgr2=0x7f64cc154790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.829+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 rx=0x7f64c400c810 tx=0x7f64c4001aa0).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 shutdown_connections 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f64b805f150 0x7f64b8061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f64cc157660 0x7f64cc154790 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f64cc0c0180 0x7f64cc0aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 --2- 172.21.15.73:0/3776089531 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f64cc0c1750 0x7f64cc156840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 >> 172.21.15.73:0/3776089531 conn(0x7f64cc0cb000 msgr2=0x7f64cc0c5400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 shutdown_connections 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:23.830+0000 7f64d0af5700 1 -- 172.21.15.73:0/3776089531 wait complete. 2023-11-28T04:08:23.833 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:08:23.842 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:08:21.829709+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":146,"acting":146,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":31,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447132,"kb_used_data":17116,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752527588,"statfs":{"total":773094113280,"available":770588250112,"internally_reserved":0,"allocated":17526784,"data_stored":8810065,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10874,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.221058"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084377+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085516+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090766+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085468+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088918+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084382+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086833+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083710+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085932+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087817+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086562+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087153+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088501+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087381+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087366+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084907+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087224+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090260+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086202+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083498+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086161+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086964+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[3],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084559+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086717+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083889+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085041+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083886+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086849+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084665+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086573+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085474+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083390+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085531+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087284+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086201+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085130+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085384+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085789+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.088361+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085471+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083454+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086810+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085856+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090687+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086508+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086055+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086411+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.084583+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090939+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087104+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085495+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086113+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087998+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083451+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087427+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086059+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.085857+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938971+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:08:16.938971+0000","last_peered":"2023-11-28T04:08:16.938971+0000","last_clean":"2023-11-28T04:08:16.938971+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:08:16.938971+0000","last_undegraded":"2023-11-28T04:08:16.938971+0000","last_fullsized":"2023-11-28T04:08:16.938971+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.083446+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382553+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:08:17.382553+0000","last_peered":"2023-11-28T04:08:17.382553+0000","last_clean":"2023-11-28T04:08:17.382553+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:08:17.382553+0000","last_undegraded":"2023-11-28T04:08:17.382553+0000","last_fullsized":"2023-11-28T04:08:17.382553+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.087826+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":148,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382000+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:08:17.382000+0000","last_peered":"2023-11-28T04:08:17.382000+0000","last_clean":"2023-11-28T04:08:17.382000+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:08:17.382000+0000","last_undegraded":"2023-11-28T04:08:17.382000+0000","last_fullsized":"2023-11-28T04:08:17.382000+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086460+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:17.382551+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:08:17.382551+0000","last_peered":"2023-11-28T04:08:17.382551+0000","last_clean":"2023-11-28T04:08:17.382551+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:08:17.382551+0000","last_undegraded":"2023-11-28T04:08:17.382551+0000","last_fullsized":"2023-11-28T04:08:17.382551+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":141,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395626+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:08:16.395626+0000","last_peered":"2023-11-28T04:08:16.395626+0000","last_clean":"2023-11-28T04:08:16.395626+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:08:16.395626+0000","last_undegraded":"2023-11-28T04:08:16.395626+0000","last_fullsized":"2023-11-28T04:08:16.395626+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086320+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395614+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:08:16.395614+0000","last_peered":"2023-11-28T04:08:16.395614+0000","last_clean":"2023-11-28T04:08:16.395614+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:08:16.395614+0000","last_undegraded":"2023-11-28T04:08:16.395614+0000","last_fullsized":"2023-11-28T04:08:16.395614+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.089557+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":139,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.395621+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:08:16.395621+0000","last_peered":"2023-11-28T04:08:16.395621+0000","last_clean":"2023-11-28T04:08:16.395621+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:08:16.395621+0000","last_undegraded":"2023-11-28T04:08:16.395621+0000","last_fullsized":"2023-11-28T04:08:16.395621+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.090112+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":140,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.938998+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:08:16.938998+0000","last_peered":"2023-11-28T04:08:16.938998+0000","last_clean":"2023-11-28T04:08:16.938998+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:08:16.938998+0000","last_undegraded":"2023-11-28T04:08:16.938998+0000","last_fullsized":"2023-11-28T04:08:16.938998+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-11-28T04:08:19.080346+0000","last_change":"2023-11-28T04:08:19.080346+0000","last_active":"2023-11-28T04:08:19.080346+0000","last_peered":"2023-11-28T04:08:19.080346+0000","last_clean":"2023-11-28T04:08:19.080346+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:19.080346+0000","last_undegraded":"2023-11-28T04:08:19.080346+0000","last_fullsized":"2023-11-28T04:08:19.080346+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"creating+peering","last_fresh":"2023-11-28T04:08:18.075747+0000","last_change":"2023-11-28T04:08:18.086959+0000","last_active":"2023-11-28T04:08:18.075747+0000","last_peered":"2023-11-28T04:08:18.075747+0000","last_clean":"2023-11-28T04:08:18.075747+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-11-28T04:08:18.075747+0000","last_undegraded":"2023-11-28T04:08:18.075747+0000","last_fullsized":"2023-11-28T04:08:18.075747+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":156,"reported_epoch":175,"state":"active+clean","last_fresh":"2023-11-28T04:08:16.077366+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:08:16.077366+0000","last_peered":"2023-11-28T04:08:16.077366+0000","last_clean":"2023-11-28T04:08:16.077366+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:08:16.077366+0000","last_undegraded":"2023-11-28T04:08:16.077366+0000","last_fullsized":"2023-11-28T04:08:16.077366+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575751,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.29299999999999998},"max":{"1min":0.85999999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.623},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.48799999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.35299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.66800000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.58499999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71299999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.501},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.878,"15min":0.878},"last":0.69799999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":0.875,"15min":0.875},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.63700000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575751,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.379},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":0.73499999999999999,"15min":0.75},"last":0.53200000000000003}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.42299999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.83099999999999996},"last":0.48199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.45500000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.63700000000000001,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.39800000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.311},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.434},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.50700000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.495},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.57599999999999996}]}]},{"osd":1,"up_from":13,"seq":55834575750,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303324,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068516,"statfs":{"total":96636764160,"available":96326160384,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.27400000000000002},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.29199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.34999999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.35699999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.42599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.51900000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.54200000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.61099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575751,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303336,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068504,"statfs":{"total":96636764160,"available":96326148096,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.23599999999999999},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.60299999999999998},"last":0.33200000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.46600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59999999999999998,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.36499999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.57599999999999996,"15min":0.67300000000000004},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42899999999999999,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.77500000000000002},"last":0.27400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":1.24,"15min":1.24},"last":0.432},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.46700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91200000000000003,"15min":0.98899999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.248,"15min":1.248},"last":0.439}]},{"osd":6,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.2,"15min":1.2},"last":0.34399999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.42599999999999999},"max":{"1min":0.81200000000000006,"5min":1.27,"15min":1.27},"last":0.40600000000000003}]}]},{"osd":2,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.38600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.376},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.35699999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.33900000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.84599999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.627}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.82699999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89300000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.77100000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575751,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.371},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.59499999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.27800000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.39600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.29899999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58599999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.312},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.34799999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":1.056,"15min":1.056},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":1.085},"last":0.59399999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93400000000000005,"15min":1.1200000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.97099999999999997,"15min":1.145},"last":0.66300000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.0189999999999999},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89100000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.67500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307432,"kb_used_data":2144,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064408,"statfs":{"total":96636764160,"available":96321953792,"internally_reserved":0,"allocated":2195456,"data_stored":1103591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.57499999999999996,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":0.71699999999999997,"5min":0.83799999999999997,"15min":1.7350000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.40100000000000002,"5min":0.35299999999999998,"15min":0.32100000000000001},"max":{"1min":0.71199999999999997,"5min":0.77000000000000002,"15min":0.84199999999999997},"last":0.42799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.56799999999999995,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.56999999999999995,"15min":0.57499999999999996},"min":{"1min":0.42699999999999999,"5min":0.36299999999999999,"15min":0.317},"max":{"1min":0.71299999999999997,"5min":0.79100000000000004,"15min":1.778},"last":0.56899999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.56999999999999995},"min":{"1min":0.40799999999999997,"5min":0.373,"15min":0.373},"max":{"1min":0.69399999999999995,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.497},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58199999999999996,"15min":0.57899999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":0.77700000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52400000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.61399999999999999,"15min":0.61699999999999999},"min":{"1min":0.41099999999999998,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":0.76000000000000001,"5min":0.872,"15min":1.8160000000000001},"last":0.628},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.51200000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":0.77800000000000002,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.60099999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47199999999999998},"min":{"1min":0.34599999999999997,"5min":0.33400000000000002,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33600000000000002,"5min":0.307,"15min":0.307},"max":{"1min":0.71899999999999997,"5min":0.82399999999999995,"15min":1.6899999999999999},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.497,"15min":0.5},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.76200000000000001,"5min":0.79100000000000004,"15min":0.79700000000000004},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54800000000000004,"15min":0.53500000000000003},"min":{"1min":0.38600000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81799999999999995},"last":0.67400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:19 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55600000000000005},"min":{"1min":0.37,"5min":0.35399999999999998,"15min":0.33600000000000002},"max":{"1min":0.72899999999999998,"5min":0.82199999999999995,"15min":2.069},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.58799999999999997,"15min":0.57299999999999995},"min":{"1min":0.36499999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.753,"5min":0.88100000000000001,"15min":0.88200000000000001},"last":0.624}]}]},{"osd":5,"up_from":13,"seq":55834575752,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307420,"kb_used_data":2132,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064420,"statfs":{"total":96636764160,"available":96321966080,"internally_reserved":0,"allocated":2183168,"data_stored":1097370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.77800000000000002},"last":0.443},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.378}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.307},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.58699999999999997}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.56100000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.318,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.67400000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.54700000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.26600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.61099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:08:25.385 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:08:25.385+0000 7f7f9954d700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:08:26.850 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:08:26.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.975+0000 7f8ad88b1700 1 Processor -- start 2023-11-28T04:08:26.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.975+0000 7f8ad88b1700 1 -- start start 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.976+0000 7f8ad88b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ad40c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.976+0000 7f8ad88b1700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8ad4068090 0x7f8ad40c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.976+0000 7f8ad88b1700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ad4154d20 con 0x7f8ad4067cb0 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad88b1700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ad4151670 con 0x7f8ad4068090 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad88b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40a7490 con 0x7f8ad40c45d0 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8ad4067cb0 0x7f8ad40c3ec0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59748/0 (socket says 172.21.15.73:59748) 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad259c700 1 -- 172.21.15.73:0/2754545442 learned_addr learned my addr 172.21.15.73:0/2754545442 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad2d9d700 1 --2- 172.21.15.73:0/2754545442 >> v2:172.21.15.150:3300/0 conn(0x7f8ad4068090 0x7f8ad40c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1175677050 0 0) 0x7f8ad4154d20 con 0x7f8ad4067cb0 2023-11-28T04:08:26.981 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ad40c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ac00033f0 con 0x7f8ad4067cb0 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 607030512 0 0) 0x7f8ac00033f0 con 0x7f8ad4067cb0 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 msgr2=0x7f8ad40c2640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ad40c2640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 >> v2:172.21.15.150:3300/0 conn(0x7f8ad4068090 msgr2=0x7f8ad40c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 --2- 172.21.15.73:0/2754545442 >> v2:172.21.15.150:3300/0 conn(0x7f8ad4068090 0x7f8ad40c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad41555c0 con 0x7f8ad4067cb0 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8ac40017f0 con 0x7f8ad4067cb0 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 >> v1:172.21.15.73:6789/0 conn(0x7f8ad4067cb0 legacy=0x7f8ad40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ac0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ac0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac0004b80 0x7f8ac0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.982 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40a7490 con 0x7f8ac00038e0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac00044c0 con 0x7f8ac0004b80 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac000b930 con 0x7f8ad40c45d0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.977+0000 7f8ad259c700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ac0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ac0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad2d9d700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac0004b80 0x7f8ac0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 -- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 msgr2=0x7f8ac0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ac0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 -- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac0004b80 msgr2=0x7f8ac0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ac0004b80 0x7f8ac0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 -- 172.21.15.73:0/2754545442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac00033f0 con 0x7f8ac00038e0 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad40c45d0 0x7f8ac0004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:26.983 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad259c700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ac0003d00 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f8ac4002180 tx=0x7f8ad4155530).ready entity=mon.0 client_cookie=8752be26f53b42a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac400ce20 con 0x7f8ac00038e0 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac4005b20 con 0x7f8ac00038e0 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad159a700 1 -- 172.21.15.73:0/2754545442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac40070d0 con 0x7f8ac00038e0 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad88b1700 1 -- 172.21.15.73:0/2754545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 msgr2=0x7f8ac0003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.978+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/2754545442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ac0003d00 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 rx=0x7f8ac4002180 tx=0x7f8ad4155530).stop 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.979+0000 7f8ad2d9d700 1 -- 172.21.15.73:0/2754545442 reap_dead start 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.979+0000 7f8ad88b1700 1 -- 172.21.15.73:0/2754545442 shutdown_connections 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.979+0000 7f8ad88b1700 1 -- 172.21.15.73:0/2754545442 >> 172.21.15.73:0/2754545442 conn(0x7f8ad40c7fd0 msgr2=0x7f8ad40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.979+0000 7f8ad88b1700 1 -- 172.21.15.73:0/2754545442 shutdown_connections 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.979+0000 7f8ad88b1700 1 -- 172.21.15.73:0/2754545442 wait complete. 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.980+0000 7f8ad88b1700 1 Processor -- start 2023-11-28T04:08:26.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.980+0000 7f8ad88b1700 1 -- start start 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4062700 0x7f8ad4062b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad40a8b40 0x7f8ad40a8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40abca0 con 0x7f8ac00038e0 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4063070 con 0x7f8ad40a8b40 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40ab090 con 0x7f8ad4062700 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40062/0 (socket says 172.21.15.73:40062) 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 -- 172.21.15.73:0/1341738560 learned_addr learned my addr 172.21.15.73:0/1341738560 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:26.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4062700 0x7f8ad4062b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad2d9d700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad40a8b40 0x7f8ad40a8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 -- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4062700 msgr2=0x7f8ad4062b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4062700 0x7f8ad4062b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 -- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad40a8b40 msgr2=0x7f8ad40a8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad40a8b40 0x7f8ad40a8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 -- 172.21.15.73:0/1341738560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad40ab370 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad259c700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f8ac4007fd0 tx=0x7f8ac4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007440 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac40075a0 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad40ab650 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac4007940 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.981+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad414d030 con 0x7f8ac00038e0 2023-11-28T04:08:26.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.982+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad406c860 con 0x7f8ac00038e0 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.983+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8ac4011680 con 0x7f8ac00038e0 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.983+0000 7f8aca7fc700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 0x7f8ab4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.983+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8ac407b360 con 0x7f8ac00038e0 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.983+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 0x7f8ab4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.983+0000 7f8ad1d9b700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 0x7f8ab4061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8ad4095c60 tx=0x7f8abc008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:26.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:26.987+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8ac404c630 con 0x7f8ac00038e0 2023-11-28T04:08:27.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.151+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8ad40abde0 con 0x7f8ab405f150 2023-11-28T04:08:27.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.160+0000 7f8aca7fc700 1 -- 172.21.15.73:0/1341738560 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+361709 (secure 0 0 0) 0x7f8abc0099a0 con 0x7f8ab405f150 2023-11-28T04:08:27.165 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:27.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 msgr2=0x7f8ab4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 0x7f8ab4061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f8ad4095c60 tx=0x7f8abc008040).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 msgr2=0x7f8ad414cb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 rx=0x7f8ac4007fd0 tx=0x7f8ac4017000).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 shutdown_connections 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f8ab405f150 0x7f8ab4061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8ad40a8b40 0x7f8ad40a8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8ac00038e0 0x7f8ad414cb40 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 --2- 172.21.15.73:0/1341738560 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8ad4062700 0x7f8ad4062b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 >> 172.21.15.73:0/1341738560 conn(0x7f8ad40c7fd0 msgr2=0x7f8ad4069fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 shutdown_connections 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.164+0000 7f8ad88b1700 1 -- 172.21.15.73:0/1341738560 wait complete. 2023-11-28T04:08:27.169 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:08:27.177 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-11-28T04:08:25.830596+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2447520,"kb_used_data":17504,"kb_used_omap":0,"kb_used_meta":2429952,"kb_avail":752527200,"statfs":{"total":773094113280,"available":770587852800,"internally_reserved":0,"allocated":17924096,"data_stored":8997836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2488270848},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":38,"apply_latency_ms":38,"commit_latency_ns":38000000,"apply_latency_ns":38000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-10874,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-31,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.221657"},"pg_stats":[{"pgid":"22.3c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482322+0000","last_change":"2023-11-28T04:08:19.477365+0000","last_active":"2023-11-28T04:08:19.482322+0000","last_peered":"2023-11-28T04:08:19.482322+0000","last_clean":"2023-11-28T04:08:19.482322+0000","last_became_active":"2023-11-28T04:08:19.476516+0000","last_became_peered":"2023-11-28T04:08:19.476516+0000","last_unstale":"2023-11-28T04:08:19.482322+0000","last_undegraded":"2023-11-28T04:08:19.482322+0000","last_fullsized":"2023-11-28T04:08:19.482322+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3f","version":"178'3","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485143+0000","last_change":"2023-11-28T04:08:19.106763+0000","last_active":"2023-11-28T04:08:19.485143+0000","last_peered":"2023-11-28T04:08:19.485143+0000","last_clean":"2023-11-28T04:08:19.485143+0000","last_became_active":"2023-11-28T04:08:19.106497+0000","last_became_peered":"2023-11-28T04:08:19.106497+0000","last_unstale":"2023-11-28T04:08:19.485143+0000","last_undegraded":"2023-11-28T04:08:19.485143+0000","last_fullsized":"2023-11-28T04:08:19.485143+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485237+0000","last_change":"2023-11-28T04:08:19.478725+0000","last_active":"2023-11-28T04:08:19.485237+0000","last_peered":"2023-11-28T04:08:19.485237+0000","last_clean":"2023-11-28T04:08:19.485237+0000","last_became_active":"2023-11-28T04:08:19.478405+0000","last_became_peered":"2023-11-28T04:08:19.478405+0000","last_unstale":"2023-11-28T04:08:19.485237+0000","last_undegraded":"2023-11-28T04:08:19.485237+0000","last_fullsized":"2023-11-28T04:08:19.485237+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.462117+0000","last_change":"2023-11-28T04:08:19.104186+0000","last_active":"2023-11-28T04:08:19.462117+0000","last_peered":"2023-11-28T04:08:19.462117+0000","last_clean":"2023-11-28T04:08:19.462117+0000","last_became_active":"2023-11-28T04:08:19.103536+0000","last_became_peered":"2023-11-28T04:08:19.103536+0000","last_unstale":"2023-11-28T04:08:19.462117+0000","last_undegraded":"2023-11-28T04:08:19.462117+0000","last_fullsized":"2023-11-28T04:08:19.462117+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3e","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482182+0000","last_change":"2023-11-28T04:08:19.475919+0000","last_active":"2023-11-28T04:08:19.482182+0000","last_peered":"2023-11-28T04:08:19.482182+0000","last_clean":"2023-11-28T04:08:19.482182+0000","last_became_active":"2023-11-28T04:08:19.475658+0000","last_became_peered":"2023-11-28T04:08:19.475658+0000","last_unstale":"2023-11-28T04:08:19.482182+0000","last_undegraded":"2023-11-28T04:08:19.482182+0000","last_fullsized":"2023-11-28T04:08:19.482182+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.472728+0000","last_change":"2023-11-28T04:08:19.100787+0000","last_active":"2023-11-28T04:08:19.472728+0000","last_peered":"2023-11-28T04:08:19.472728+0000","last_clean":"2023-11-28T04:08:19.472728+0000","last_became_active":"2023-11-28T04:08:19.100514+0000","last_became_peered":"2023-11-28T04:08:19.100514+0000","last_unstale":"2023-11-28T04:08:19.472728+0000","last_undegraded":"2023-11-28T04:08:19.472728+0000","last_fullsized":"2023-11-28T04:08:19.472728+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.3f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482219+0000","last_change":"2023-11-28T04:08:19.478296+0000","last_active":"2023-11-28T04:08:19.482219+0000","last_peered":"2023-11-28T04:08:19.482219+0000","last_clean":"2023-11-28T04:08:19.482219+0000","last_became_active":"2023-11-28T04:08:19.478022+0000","last_became_peered":"2023-11-28T04:08:19.478022+0000","last_unstale":"2023-11-28T04:08:19.482219+0000","last_undegraded":"2023-11-28T04:08:19.482219+0000","last_fullsized":"2023-11-28T04:08:19.482219+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3c","version":"178'2","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.478158+0000","last_change":"2023-11-28T04:08:19.102474+0000","last_active":"2023-11-28T04:08:19.478158+0000","last_peered":"2023-11-28T04:08:19.478158+0000","last_clean":"2023-11-28T04:08:19.478158+0000","last_became_active":"2023-11-28T04:08:19.102098+0000","last_became_peered":"2023-11-28T04:08:19.102098+0000","last_unstale":"2023-11-28T04:08:19.478158+0000","last_undegraded":"2023-11-28T04:08:19.478158+0000","last_fullsized":"2023-11-28T04:08:19.478158+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.38","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480070+0000","last_change":"2023-11-28T04:08:19.473627+0000","last_active":"2023-11-28T04:08:19.480070+0000","last_peered":"2023-11-28T04:08:19.480070+0000","last_clean":"2023-11-28T04:08:19.480070+0000","last_became_active":"2023-11-28T04:08:19.471508+0000","last_became_peered":"2023-11-28T04:08:19.471508+0000","last_unstale":"2023-11-28T04:08:19.480070+0000","last_undegraded":"2023-11-28T04:08:19.480070+0000","last_fullsized":"2023-11-28T04:08:19.480070+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480196+0000","last_change":"2023-11-28T04:08:19.103618+0000","last_active":"2023-11-28T04:08:19.480196+0000","last_peered":"2023-11-28T04:08:19.480196+0000","last_clean":"2023-11-28T04:08:19.480196+0000","last_became_active":"2023-11-28T04:08:19.103356+0000","last_became_peered":"2023-11-28T04:08:19.103356+0000","last_unstale":"2023-11-28T04:08:19.480196+0000","last_undegraded":"2023-11-28T04:08:19.480196+0000","last_fullsized":"2023-11-28T04:08:19.480196+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.39","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484793+0000","last_change":"2023-11-28T04:08:19.479666+0000","last_active":"2023-11-28T04:08:19.484793+0000","last_peered":"2023-11-28T04:08:19.484793+0000","last_clean":"2023-11-28T04:08:19.484793+0000","last_became_active":"2023-11-28T04:08:19.479351+0000","last_became_peered":"2023-11-28T04:08:19.479351+0000","last_unstale":"2023-11-28T04:08:19.484793+0000","last_undegraded":"2023-11-28T04:08:19.484793+0000","last_fullsized":"2023-11-28T04:08:19.484793+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473286+0000","last_change":"2023-11-28T04:08:19.109998+0000","last_active":"2023-11-28T04:08:19.473286+0000","last_peered":"2023-11-28T04:08:19.473286+0000","last_clean":"2023-11-28T04:08:19.473286+0000","last_became_active":"2023-11-28T04:08:19.109717+0000","last_became_peered":"2023-11-28T04:08:19.109717+0000","last_unstale":"2023-11-28T04:08:19.473286+0000","last_undegraded":"2023-11-28T04:08:19.473286+0000","last_fullsized":"2023-11-28T04:08:19.473286+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.3a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473934+0000","last_change":"2023-11-28T04:08:19.472074+0000","last_active":"2023-11-28T04:08:19.473934+0000","last_peered":"2023-11-28T04:08:19.473934+0000","last_clean":"2023-11-28T04:08:19.473934+0000","last_became_active":"2023-11-28T04:08:19.471805+0000","last_became_peered":"2023-11-28T04:08:19.471805+0000","last_unstale":"2023-11-28T04:08:19.473934+0000","last_undegraded":"2023-11-28T04:08:19.473934+0000","last_fullsized":"2023-11-28T04:08:19.473934+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.457387+0000","last_change":"2023-11-28T04:08:19.108438+0000","last_active":"2023-11-28T04:08:19.457387+0000","last_peered":"2023-11-28T04:08:19.457387+0000","last_clean":"2023-11-28T04:08:19.457387+0000","last_became_active":"2023-11-28T04:08:19.108137+0000","last_became_peered":"2023-11-28T04:08:19.108137+0000","last_unstale":"2023-11-28T04:08:19.457387+0000","last_undegraded":"2023-11-28T04:08:19.457387+0000","last_fullsized":"2023-11-28T04:08:19.457387+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.3b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473957+0000","last_change":"2023-11-28T04:08:19.472038+0000","last_active":"2023-11-28T04:08:19.473957+0000","last_peered":"2023-11-28T04:08:19.473957+0000","last_clean":"2023-11-28T04:08:19.473957+0000","last_became_active":"2023-11-28T04:08:19.471783+0000","last_became_peered":"2023-11-28T04:08:19.471783+0000","last_unstale":"2023-11-28T04:08:19.473957+0000","last_undegraded":"2023-11-28T04:08:19.473957+0000","last_fullsized":"2023-11-28T04:08:19.473957+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.455286+0000","last_change":"2023-11-28T04:08:19.106535+0000","last_active":"2023-11-28T04:08:19.455286+0000","last_peered":"2023-11-28T04:08:19.455286+0000","last_clean":"2023-11-28T04:08:19.455286+0000","last_became_active":"2023-11-28T04:08:19.106287+0000","last_became_peered":"2023-11-28T04:08:19.106287+0000","last_unstale":"2023-11-28T04:08:19.455286+0000","last_undegraded":"2023-11-28T04:08:19.455286+0000","last_fullsized":"2023-11-28T04:08:19.455286+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.34","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482333+0000","last_change":"2023-11-28T04:08:19.478419+0000","last_active":"2023-11-28T04:08:19.482333+0000","last_peered":"2023-11-28T04:08:19.482333+0000","last_clean":"2023-11-28T04:08:19.482333+0000","last_became_active":"2023-11-28T04:08:19.477631+0000","last_became_peered":"2023-11-28T04:08:19.477631+0000","last_unstale":"2023-11-28T04:08:19.482333+0000","last_undegraded":"2023-11-28T04:08:19.482333+0000","last_fullsized":"2023-11-28T04:08:19.482333+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.462061+0000","last_change":"2023-11-28T04:08:19.109755+0000","last_active":"2023-11-28T04:08:19.462061+0000","last_peered":"2023-11-28T04:08:19.462061+0000","last_clean":"2023-11-28T04:08:19.462061+0000","last_became_active":"2023-11-28T04:08:19.109506+0000","last_became_peered":"2023-11-28T04:08:19.109506+0000","last_unstale":"2023-11-28T04:08:19.462061+0000","last_undegraded":"2023-11-28T04:08:19.462061+0000","last_fullsized":"2023-11-28T04:08:19.462061+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.35","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482436+0000","last_change":"2023-11-28T04:08:19.479076+0000","last_active":"2023-11-28T04:08:19.482436+0000","last_peered":"2023-11-28T04:08:19.482436+0000","last_clean":"2023-11-28T04:08:19.482436+0000","last_became_active":"2023-11-28T04:08:19.478577+0000","last_became_peered":"2023-11-28T04:08:19.478577+0000","last_unstale":"2023-11-28T04:08:19.482436+0000","last_undegraded":"2023-11-28T04:08:19.482436+0000","last_fullsized":"2023-11-28T04:08:19.482436+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469298+0000","last_change":"2023-11-28T04:08:19.111176+0000","last_active":"2023-11-28T04:08:19.469298+0000","last_peered":"2023-11-28T04:08:19.469298+0000","last_clean":"2023-11-28T04:08:19.469298+0000","last_became_active":"2023-11-28T04:08:19.110900+0000","last_became_peered":"2023-11-28T04:08:19.110900+0000","last_unstale":"2023-11-28T04:08:19.469298+0000","last_undegraded":"2023-11-28T04:08:19.469298+0000","last_fullsized":"2023-11-28T04:08:19.469298+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.36","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.467733+0000","last_change":"2023-11-28T04:08:19.466843+0000","last_active":"2023-11-28T04:08:19.467733+0000","last_peered":"2023-11-28T04:08:19.467733+0000","last_clean":"2023-11-28T04:08:19.467733+0000","last_became_active":"2023-11-28T04:08:19.466557+0000","last_became_peered":"2023-11-28T04:08:19.466557+0000","last_unstale":"2023-11-28T04:08:19.467733+0000","last_undegraded":"2023-11-28T04:08:19.467733+0000","last_fullsized":"2023-11-28T04:08:19.467733+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.458379+0000","last_change":"2023-11-28T04:08:19.107017+0000","last_active":"2023-11-28T04:08:19.458379+0000","last_peered":"2023-11-28T04:08:19.458379+0000","last_clean":"2023-11-28T04:08:19.458379+0000","last_became_active":"2023-11-28T04:08:19.106802+0000","last_became_peered":"2023-11-28T04:08:19.106802+0000","last_unstale":"2023-11-28T04:08:19.458379+0000","last_undegraded":"2023-11-28T04:08:19.458379+0000","last_fullsized":"2023-11-28T04:08:19.458379+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.37","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484915+0000","last_change":"2023-11-28T04:08:19.480608+0000","last_active":"2023-11-28T04:08:19.484915+0000","last_peered":"2023-11-28T04:08:19.484915+0000","last_clean":"2023-11-28T04:08:19.484915+0000","last_became_active":"2023-11-28T04:08:19.480336+0000","last_became_peered":"2023-11-28T04:08:19.480336+0000","last_unstale":"2023-11-28T04:08:19.484915+0000","last_undegraded":"2023-11-28T04:08:19.484915+0000","last_fullsized":"2023-11-28T04:08:19.484915+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.34","version":"178'4","reported_seq":14,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.489045+0000","last_change":"2023-11-28T04:08:19.105087+0000","last_active":"2023-11-28T04:08:19.489045+0000","last_peered":"2023-11-28T04:08:19.489045+0000","last_clean":"2023-11-28T04:08:19.489045+0000","last_became_active":"2023-11-28T04:08:19.104810+0000","last_became_peered":"2023-11-28T04:08:19.104810+0000","last_unstale":"2023-11-28T04:08:19.489045+0000","last_undegraded":"2023-11-28T04:08:19.489045+0000","last_fullsized":"2023-11-28T04:08:19.489045+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.30","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.470054+0000","last_change":"2023-11-28T04:08:19.468794+0000","last_active":"2023-11-28T04:08:19.470054+0000","last_peered":"2023-11-28T04:08:19.470054+0000","last_clean":"2023-11-28T04:08:19.470054+0000","last_became_active":"2023-11-28T04:08:19.468559+0000","last_became_peered":"2023-11-28T04:08:19.468559+0000","last_unstale":"2023-11-28T04:08:19.470054+0000","last_undegraded":"2023-11-28T04:08:19.470054+0000","last_fullsized":"2023-11-28T04:08:19.470054+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.33","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480331+0000","last_change":"2023-11-28T04:08:19.107362+0000","last_active":"2023-11-28T04:08:19.480331+0000","last_peered":"2023-11-28T04:08:19.480331+0000","last_clean":"2023-11-28T04:08:19.480331+0000","last_became_active":"2023-11-28T04:08:19.107035+0000","last_became_peered":"2023-11-28T04:08:19.107035+0000","last_unstale":"2023-11-28T04:08:19.480331+0000","last_undegraded":"2023-11-28T04:08:19.480331+0000","last_fullsized":"2023-11-28T04:08:19.480331+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.31","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482285+0000","last_change":"2023-11-28T04:08:19.475810+0000","last_active":"2023-11-28T04:08:19.482285+0000","last_peered":"2023-11-28T04:08:19.482285+0000","last_clean":"2023-11-28T04:08:19.482285+0000","last_became_active":"2023-11-28T04:08:19.475538+0000","last_became_peered":"2023-11-28T04:08:19.475538+0000","last_unstale":"2023-11-28T04:08:19.482285+0000","last_undegraded":"2023-11-28T04:08:19.482285+0000","last_fullsized":"2023-11-28T04:08:19.482285+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459836+0000","last_change":"2023-11-28T04:08:19.106450+0000","last_active":"2023-11-28T04:08:19.459836+0000","last_peered":"2023-11-28T04:08:19.459836+0000","last_clean":"2023-11-28T04:08:19.459836+0000","last_became_active":"2023-11-28T04:08:19.103897+0000","last_became_peered":"2023-11-28T04:08:19.103897+0000","last_unstale":"2023-11-28T04:08:19.459836+0000","last_undegraded":"2023-11-28T04:08:19.459836+0000","last_fullsized":"2023-11-28T04:08:19.459836+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.32","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481837+0000","last_change":"2023-11-28T04:08:19.476941+0000","last_active":"2023-11-28T04:08:19.481837+0000","last_peered":"2023-11-28T04:08:19.481837+0000","last_clean":"2023-11-28T04:08:19.481837+0000","last_became_active":"2023-11-28T04:08:19.476651+0000","last_became_peered":"2023-11-28T04:08:19.476651+0000","last_unstale":"2023-11-28T04:08:19.481837+0000","last_undegraded":"2023-11-28T04:08:19.481837+0000","last_fullsized":"2023-11-28T04:08:19.481837+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469736+0000","last_change":"2023-11-28T04:08:19.106941+0000","last_active":"2023-11-28T04:08:19.469736+0000","last_peered":"2023-11-28T04:08:19.469736+0000","last_clean":"2023-11-28T04:08:19.469736+0000","last_became_active":"2023-11-28T04:08:19.106614+0000","last_became_peered":"2023-11-28T04:08:19.106614+0000","last_unstale":"2023-11-28T04:08:19.469736+0000","last_undegraded":"2023-11-28T04:08:19.469736+0000","last_fullsized":"2023-11-28T04:08:19.469736+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.33","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.467500+0000","last_change":"2023-11-28T04:08:19.466196+0000","last_active":"2023-11-28T04:08:19.467500+0000","last_peered":"2023-11-28T04:08:19.467500+0000","last_clean":"2023-11-28T04:08:19.467500+0000","last_became_active":"2023-11-28T04:08:19.465843+0000","last_became_peered":"2023-11-28T04:08:19.465843+0000","last_unstale":"2023-11-28T04:08:19.467500+0000","last_undegraded":"2023-11-28T04:08:19.467500+0000","last_fullsized":"2023-11-28T04:08:19.467500+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.30","version":"178'4","reported_seq":14,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477749+0000","last_change":"2023-11-28T04:08:19.108819+0000","last_active":"2023-11-28T04:08:19.477749+0000","last_peered":"2023-11-28T04:08:19.477749+0000","last_clean":"2023-11-28T04:08:19.477749+0000","last_became_active":"2023-11-28T04:08:19.107322+0000","last_became_peered":"2023-11-28T04:08:19.107322+0000","last_unstale":"2023-11-28T04:08:19.477749+0000","last_undegraded":"2023-11-28T04:08:19.477749+0000","last_fullsized":"2023-11-28T04:08:19.477749+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.472573+0000","last_change":"2023-11-28T04:08:19.468669+0000","last_active":"2023-11-28T04:08:19.472573+0000","last_peered":"2023-11-28T04:08:19.472573+0000","last_clean":"2023-11-28T04:08:19.472573+0000","last_became_active":"2023-11-28T04:08:19.468407+0000","last_became_peered":"2023-11-28T04:08:19.468407+0000","last_unstale":"2023-11-28T04:08:19.472573+0000","last_undegraded":"2023-11-28T04:08:19.472573+0000","last_fullsized":"2023-11-28T04:08:19.472573+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.474262+0000","last_change":"2023-11-28T04:08:19.105000+0000","last_active":"2023-11-28T04:08:19.474262+0000","last_peered":"2023-11-28T04:08:19.474262+0000","last_clean":"2023-11-28T04:08:19.474262+0000","last_became_active":"2023-11-28T04:08:19.104600+0000","last_became_peered":"2023-11-28T04:08:19.104600+0000","last_unstale":"2023-11-28T04:08:19.474262+0000","last_undegraded":"2023-11-28T04:08:19.474262+0000","last_fullsized":"2023-11-28T04:08:19.474262+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.2d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488842+0000","last_change":"2023-11-28T04:08:19.481412+0000","last_active":"2023-11-28T04:08:19.488842+0000","last_peered":"2023-11-28T04:08:19.488842+0000","last_clean":"2023-11-28T04:08:19.488842+0000","last_became_active":"2023-11-28T04:08:19.480772+0000","last_became_peered":"2023-11-28T04:08:19.480772+0000","last_unstale":"2023-11-28T04:08:19.488842+0000","last_undegraded":"2023-11-28T04:08:19.488842+0000","last_fullsized":"2023-11-28T04:08:19.488842+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.471677+0000","last_change":"2023-11-28T04:08:19.100221+0000","last_active":"2023-11-28T04:08:19.471677+0000","last_peered":"2023-11-28T04:08:19.471677+0000","last_clean":"2023-11-28T04:08:19.471677+0000","last_became_active":"2023-11-28T04:08:19.099951+0000","last_became_peered":"2023-11-28T04:08:19.099951+0000","last_unstale":"2023-11-28T04:08:19.471677+0000","last_undegraded":"2023-11-28T04:08:19.471677+0000","last_fullsized":"2023-11-28T04:08:19.471677+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.2e","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481103+0000","last_change":"2023-11-28T04:08:19.475716+0000","last_active":"2023-11-28T04:08:19.481103+0000","last_peered":"2023-11-28T04:08:19.481103+0000","last_clean":"2023-11-28T04:08:19.481103+0000","last_became_active":"2023-11-28T04:08:19.475076+0000","last_became_peered":"2023-11-28T04:08:19.475076+0000","last_unstale":"2023-11-28T04:08:19.481103+0000","last_undegraded":"2023-11-28T04:08:19.481103+0000","last_fullsized":"2023-11-28T04:08:19.481103+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2d","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473309+0000","last_change":"2023-11-28T04:08:19.108778+0000","last_active":"2023-11-28T04:08:19.473309+0000","last_peered":"2023-11-28T04:08:19.473309+0000","last_clean":"2023-11-28T04:08:19.473309+0000","last_became_active":"2023-11-28T04:08:19.108530+0000","last_became_peered":"2023-11-28T04:08:19.108530+0000","last_unstale":"2023-11-28T04:08:19.473309+0000","last_undegraded":"2023-11-28T04:08:19.473309+0000","last_fullsized":"2023-11-28T04:08:19.473309+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.2f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481500+0000","last_change":"2023-11-28T04:08:19.479696+0000","last_active":"2023-11-28T04:08:19.481500+0000","last_peered":"2023-11-28T04:08:19.481500+0000","last_clean":"2023-11-28T04:08:19.481500+0000","last_became_active":"2023-11-28T04:08:19.478619+0000","last_became_peered":"2023-11-28T04:08:19.478619+0000","last_unstale":"2023-11-28T04:08:19.481500+0000","last_undegraded":"2023-11-28T04:08:19.481500+0000","last_fullsized":"2023-11-28T04:08:19.481500+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.455044+0000","last_change":"2023-11-28T04:08:19.101210+0000","last_active":"2023-11-28T04:08:19.455044+0000","last_peered":"2023-11-28T04:08:19.455044+0000","last_clean":"2023-11-28T04:08:19.455044+0000","last_became_active":"2023-11-28T04:08:19.100353+0000","last_became_peered":"2023-11-28T04:08:19.100353+0000","last_unstale":"2023-11-28T04:08:19.455044+0000","last_undegraded":"2023-11-28T04:08:19.455044+0000","last_fullsized":"2023-11-28T04:08:19.455044+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.28","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.472768+0000","last_change":"2023-11-28T04:08:19.464532+0000","last_active":"2023-11-28T04:08:19.472768+0000","last_peered":"2023-11-28T04:08:19.472768+0000","last_clean":"2023-11-28T04:08:19.472768+0000","last_became_active":"2023-11-28T04:08:19.464265+0000","last_became_peered":"2023-11-28T04:08:19.464265+0000","last_unstale":"2023-11-28T04:08:19.472768+0000","last_undegraded":"2023-11-28T04:08:19.472768+0000","last_fullsized":"2023-11-28T04:08:19.472768+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481064+0000","last_change":"2023-11-28T04:08:19.113600+0000","last_active":"2023-11-28T04:08:19.481064+0000","last_peered":"2023-11-28T04:08:19.481064+0000","last_clean":"2023-11-28T04:08:19.481064+0000","last_became_active":"2023-11-28T04:08:19.113339+0000","last_became_peered":"2023-11-28T04:08:19.113339+0000","last_unstale":"2023-11-28T04:08:19.481064+0000","last_undegraded":"2023-11-28T04:08:19.481064+0000","last_fullsized":"2023-11-28T04:08:19.481064+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.29","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.476270+0000","last_change":"2023-11-28T04:08:19.472208+0000","last_active":"2023-11-28T04:08:19.476270+0000","last_peered":"2023-11-28T04:08:19.476270+0000","last_clean":"2023-11-28T04:08:19.476270+0000","last_became_active":"2023-11-28T04:08:19.471903+0000","last_became_peered":"2023-11-28T04:08:19.471903+0000","last_unstale":"2023-11-28T04:08:19.476270+0000","last_undegraded":"2023-11-28T04:08:19.476270+0000","last_fullsized":"2023-11-28T04:08:19.476270+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.460027+0000","last_change":"2023-11-28T04:08:19.106063+0000","last_active":"2023-11-28T04:08:19.460027+0000","last_peered":"2023-11-28T04:08:19.460027+0000","last_clean":"2023-11-28T04:08:19.460027+0000","last_became_active":"2023-11-28T04:08:19.104125+0000","last_became_peered":"2023-11-28T04:08:19.104125+0000","last_unstale":"2023-11-28T04:08:19.460027+0000","last_undegraded":"2023-11-28T04:08:19.460027+0000","last_fullsized":"2023-11-28T04:08:19.460027+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484742+0000","last_change":"2023-11-28T04:08:19.479543+0000","last_active":"2023-11-28T04:08:19.484742+0000","last_peered":"2023-11-28T04:08:19.484742+0000","last_clean":"2023-11-28T04:08:19.484742+0000","last_became_active":"2023-11-28T04:08:19.479257+0000","last_became_peered":"2023-11-28T04:08:19.479257+0000","last_unstale":"2023-11-28T04:08:19.484742+0000","last_undegraded":"2023-11-28T04:08:19.484742+0000","last_fullsized":"2023-11-28T04:08:19.484742+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459553+0000","last_change":"2023-11-28T04:08:19.104724+0000","last_active":"2023-11-28T04:08:19.459553+0000","last_peered":"2023-11-28T04:08:19.459553+0000","last_clean":"2023-11-28T04:08:19.459553+0000","last_became_active":"2023-11-28T04:08:19.104436+0000","last_became_peered":"2023-11-28T04:08:19.104436+0000","last_unstale":"2023-11-28T04:08:19.459553+0000","last_undegraded":"2023-11-28T04:08:19.459553+0000","last_fullsized":"2023-11-28T04:08:19.459553+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484813+0000","last_change":"2023-11-28T04:08:19.479819+0000","last_active":"2023-11-28T04:08:19.484813+0000","last_peered":"2023-11-28T04:08:19.484813+0000","last_clean":"2023-11-28T04:08:19.484813+0000","last_became_active":"2023-11-28T04:08:19.479529+0000","last_became_peered":"2023-11-28T04:08:19.479529+0000","last_unstale":"2023-11-28T04:08:19.484813+0000","last_undegraded":"2023-11-28T04:08:19.484813+0000","last_fullsized":"2023-11-28T04:08:19.484813+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469033+0000","last_change":"2023-11-28T04:08:19.111234+0000","last_active":"2023-11-28T04:08:19.469033+0000","last_peered":"2023-11-28T04:08:19.469033+0000","last_clean":"2023-11-28T04:08:19.469033+0000","last_became_active":"2023-11-28T04:08:19.110896+0000","last_became_peered":"2023-11-28T04:08:19.110896+0000","last_unstale":"2023-11-28T04:08:19.469033+0000","last_undegraded":"2023-11-28T04:08:19.469033+0000","last_fullsized":"2023-11-28T04:08:19.469033+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.24","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473050+0000","last_change":"2023-11-28T04:08:19.465553+0000","last_active":"2023-11-28T04:08:19.473050+0000","last_peered":"2023-11-28T04:08:19.473050+0000","last_clean":"2023-11-28T04:08:19.473050+0000","last_became_active":"2023-11-28T04:08:19.465169+0000","last_became_peered":"2023-11-28T04:08:19.465169+0000","last_unstale":"2023-11-28T04:08:19.473050+0000","last_undegraded":"2023-11-28T04:08:19.473050+0000","last_fullsized":"2023-11-28T04:08:19.473050+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.455422+0000","last_change":"2023-11-28T04:08:19.107624+0000","last_active":"2023-11-28T04:08:19.455422+0000","last_peered":"2023-11-28T04:08:19.455422+0000","last_clean":"2023-11-28T04:08:19.455422+0000","last_became_active":"2023-11-28T04:08:19.107310+0000","last_became_peered":"2023-11-28T04:08:19.107310+0000","last_unstale":"2023-11-28T04:08:19.455422+0000","last_undegraded":"2023-11-28T04:08:19.455422+0000","last_fullsized":"2023-11-28T04:08:19.455422+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.25","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484885+0000","last_change":"2023-11-28T04:08:19.480175+0000","last_active":"2023-11-28T04:08:19.484885+0000","last_peered":"2023-11-28T04:08:19.484885+0000","last_clean":"2023-11-28T04:08:19.484885+0000","last_became_active":"2023-11-28T04:08:19.479727+0000","last_became_peered":"2023-11-28T04:08:19.479727+0000","last_unstale":"2023-11-28T04:08:19.484885+0000","last_undegraded":"2023-11-28T04:08:19.484885+0000","last_fullsized":"2023-11-28T04:08:19.484885+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.458151+0000","last_change":"2023-11-28T04:08:19.105992+0000","last_active":"2023-11-28T04:08:19.458151+0000","last_peered":"2023-11-28T04:08:19.458151+0000","last_clean":"2023-11-28T04:08:19.458151+0000","last_became_active":"2023-11-28T04:08:19.105716+0000","last_became_peered":"2023-11-28T04:08:19.105716+0000","last_unstale":"2023-11-28T04:08:19.458151+0000","last_undegraded":"2023-11-28T04:08:19.458151+0000","last_fullsized":"2023-11-28T04:08:19.458151+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.26","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477115+0000","last_change":"2023-11-28T04:08:19.472487+0000","last_active":"2023-11-28T04:08:19.477115+0000","last_peered":"2023-11-28T04:08:19.477115+0000","last_clean":"2023-11-28T04:08:19.477115+0000","last_became_active":"2023-11-28T04:08:19.472045+0000","last_became_peered":"2023-11-28T04:08:19.472045+0000","last_unstale":"2023-11-28T04:08:19.477115+0000","last_undegraded":"2023-11-28T04:08:19.477115+0000","last_fullsized":"2023-11-28T04:08:19.477115+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.455124+0000","last_change":"2023-11-28T04:08:19.095896+0000","last_active":"2023-11-28T04:08:19.455124+0000","last_peered":"2023-11-28T04:08:19.455124+0000","last_clean":"2023-11-28T04:08:19.455124+0000","last_became_active":"2023-11-28T04:08:19.095595+0000","last_became_peered":"2023-11-28T04:08:19.095595+0000","last_unstale":"2023-11-28T04:08:19.455124+0000","last_undegraded":"2023-11-28T04:08:19.455124+0000","last_fullsized":"2023-11-28T04:08:19.455124+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.27","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.475978+0000","last_change":"2023-11-28T04:08:19.472520+0000","last_active":"2023-11-28T04:08:19.475978+0000","last_peered":"2023-11-28T04:08:19.475978+0000","last_clean":"2023-11-28T04:08:19.475978+0000","last_became_active":"2023-11-28T04:08:19.472229+0000","last_became_peered":"2023-11-28T04:08:19.472229+0000","last_unstale":"2023-11-28T04:08:19.475978+0000","last_undegraded":"2023-11-28T04:08:19.475978+0000","last_fullsized":"2023-11-28T04:08:19.475978+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459368+0000","last_change":"2023-11-28T04:08:19.105879+0000","last_active":"2023-11-28T04:08:19.459368+0000","last_peered":"2023-11-28T04:08:19.459368+0000","last_clean":"2023-11-28T04:08:19.459368+0000","last_became_active":"2023-11-28T04:08:19.105597+0000","last_became_peered":"2023-11-28T04:08:19.105597+0000","last_unstale":"2023-11-28T04:08:19.459368+0000","last_undegraded":"2023-11-28T04:08:19.459368+0000","last_fullsized":"2023-11-28T04:08:19.459368+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.20","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480047+0000","last_change":"2023-11-28T04:08:19.473839+0000","last_active":"2023-11-28T04:08:19.480047+0000","last_peered":"2023-11-28T04:08:19.480047+0000","last_clean":"2023-11-28T04:08:19.480047+0000","last_became_active":"2023-11-28T04:08:19.471754+0000","last_became_peered":"2023-11-28T04:08:19.471754+0000","last_unstale":"2023-11-28T04:08:19.480047+0000","last_undegraded":"2023-11-28T04:08:19.480047+0000","last_fullsized":"2023-11-28T04:08:19.480047+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.457625+0000","last_change":"2023-11-28T04:08:19.108980+0000","last_active":"2023-11-28T04:08:19.457625+0000","last_peered":"2023-11-28T04:08:19.457625+0000","last_clean":"2023-11-28T04:08:19.457625+0000","last_became_active":"2023-11-28T04:08:19.108697+0000","last_became_peered":"2023-11-28T04:08:19.108697+0000","last_unstale":"2023-11-28T04:08:19.457625+0000","last_undegraded":"2023-11-28T04:08:19.457625+0000","last_fullsized":"2023-11-28T04:08:19.457625+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.21","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.467396+0000","last_change":"2023-11-28T04:08:19.466196+0000","last_active":"2023-11-28T04:08:19.467396+0000","last_peered":"2023-11-28T04:08:19.467396+0000","last_clean":"2023-11-28T04:08:19.467396+0000","last_became_active":"2023-11-28T04:08:19.465923+0000","last_became_peered":"2023-11-28T04:08:19.465923+0000","last_unstale":"2023-11-28T04:08:19.467396+0000","last_undegraded":"2023-11-28T04:08:19.467396+0000","last_fullsized":"2023-11-28T04:08:19.467396+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.460177+0000","last_change":"2023-11-28T04:08:19.103483+0000","last_active":"2023-11-28T04:08:19.460177+0000","last_peered":"2023-11-28T04:08:19.460177+0000","last_clean":"2023-11-28T04:08:19.460177+0000","last_became_active":"2023-11-28T04:08:19.098550+0000","last_became_peered":"2023-11-28T04:08:19.098550+0000","last_unstale":"2023-11-28T04:08:19.460177+0000","last_undegraded":"2023-11-28T04:08:19.460177+0000","last_fullsized":"2023-11-28T04:08:19.460177+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.22","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484987+0000","last_change":"2023-11-28T04:08:19.481053+0000","last_active":"2023-11-28T04:08:19.484987+0000","last_peered":"2023-11-28T04:08:19.484987+0000","last_clean":"2023-11-28T04:08:19.484987+0000","last_became_active":"2023-11-28T04:08:19.480817+0000","last_became_peered":"2023-11-28T04:08:19.480817+0000","last_unstale":"2023-11-28T04:08:19.484987+0000","last_undegraded":"2023-11-28T04:08:19.484987+0000","last_fullsized":"2023-11-28T04:08:19.484987+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459436+0000","last_change":"2023-11-28T04:08:19.099255+0000","last_active":"2023-11-28T04:08:19.459436+0000","last_peered":"2023-11-28T04:08:19.459436+0000","last_clean":"2023-11-28T04:08:19.459436+0000","last_became_active":"2023-11-28T04:08:19.098961+0000","last_became_peered":"2023-11-28T04:08:19.098961+0000","last_unstale":"2023-11-28T04:08:19.459436+0000","last_undegraded":"2023-11-28T04:08:19.459436+0000","last_fullsized":"2023-11-28T04:08:19.459436+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.23","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.476906+0000","last_change":"2023-11-28T04:08:19.468734+0000","last_active":"2023-11-28T04:08:19.476906+0000","last_peered":"2023-11-28T04:08:19.476906+0000","last_clean":"2023-11-28T04:08:19.476906+0000","last_became_active":"2023-11-28T04:08:19.468463+0000","last_became_peered":"2023-11-28T04:08:19.468463+0000","last_unstale":"2023-11-28T04:08:19.476906+0000","last_undegraded":"2023-11-28T04:08:19.476906+0000","last_fullsized":"2023-11-28T04:08:19.476906+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.454543+0000","last_change":"2023-11-28T04:08:19.104427+0000","last_active":"2023-11-28T04:08:19.454543+0000","last_peered":"2023-11-28T04:08:19.454543+0000","last_clean":"2023-11-28T04:08:19.454543+0000","last_became_active":"2023-11-28T04:08:19.103673+0000","last_became_peered":"2023-11-28T04:08:19.103673+0000","last_unstale":"2023-11-28T04:08:19.454543+0000","last_undegraded":"2023-11-28T04:08:19.454543+0000","last_fullsized":"2023-11-28T04:08:19.454543+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485213+0000","last_change":"2023-11-28T04:08:19.478682+0000","last_active":"2023-11-28T04:08:19.485213+0000","last_peered":"2023-11-28T04:08:19.485213+0000","last_clean":"2023-11-28T04:08:19.485213+0000","last_became_active":"2023-11-28T04:08:19.478410+0000","last_became_peered":"2023-11-28T04:08:19.478410+0000","last_unstale":"2023-11-28T04:08:19.485213+0000","last_undegraded":"2023-11-28T04:08:19.485213+0000","last_fullsized":"2023-11-28T04:08:19.485213+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1f","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.466474+0000","last_change":"2023-11-28T04:08:19.104657+0000","last_active":"2023-11-28T04:08:19.466474+0000","last_peered":"2023-11-28T04:08:19.466474+0000","last_clean":"2023-11-28T04:08:19.466474+0000","last_became_active":"2023-11-28T04:08:19.104376+0000","last_became_peered":"2023-11-28T04:08:19.104376+0000","last_unstale":"2023-11-28T04:08:19.466474+0000","last_undegraded":"2023-11-28T04:08:19.466474+0000","last_fullsized":"2023-11-28T04:08:19.466474+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485173+0000","last_change":"2023-11-28T04:08:19.477797+0000","last_active":"2023-11-28T04:08:19.485173+0000","last_peered":"2023-11-28T04:08:19.485173+0000","last_clean":"2023-11-28T04:08:19.485173+0000","last_became_active":"2023-11-28T04:08:19.477532+0000","last_became_peered":"2023-11-28T04:08:19.477532+0000","last_unstale":"2023-11-28T04:08:19.485173+0000","last_undegraded":"2023-11-28T04:08:19.485173+0000","last_fullsized":"2023-11-28T04:08:19.485173+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1e","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.483150+0000","last_change":"2023-11-28T04:08:19.107838+0000","last_active":"2023-11-28T04:08:19.483150+0000","last_peered":"2023-11-28T04:08:19.483150+0000","last_clean":"2023-11-28T04:08:19.483150+0000","last_became_active":"2023-11-28T04:08:19.107239+0000","last_became_peered":"2023-11-28T04:08:19.107239+0000","last_unstale":"2023-11-28T04:08:19.483150+0000","last_undegraded":"2023-11-28T04:08:19.483150+0000","last_fullsized":"2023-11-28T04:08:19.483150+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.479000+0000","last_change":"2023-11-28T04:08:19.474887+0000","last_active":"2023-11-28T04:08:19.479000+0000","last_peered":"2023-11-28T04:08:19.479000+0000","last_clean":"2023-11-28T04:08:19.479000+0000","last_became_active":"2023-11-28T04:08:19.474576+0000","last_became_peered":"2023-11-28T04:08:19.474576+0000","last_unstale":"2023-11-28T04:08:19.479000+0000","last_undegraded":"2023-11-28T04:08:19.479000+0000","last_fullsized":"2023-11-28T04:08:19.479000+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"178'2","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488756+0000","last_change":"2023-11-28T04:08:19.106627+0000","last_active":"2023-11-28T04:08:19.488756+0000","last_peered":"2023-11-28T04:08:19.488756+0000","last_clean":"2023-11-28T04:08:19.488756+0000","last_became_active":"2023-11-28T04:08:19.106349+0000","last_became_peered":"2023-11-28T04:08:19.106349+0000","last_unstale":"2023-11-28T04:08:19.488756+0000","last_undegraded":"2023-11-28T04:08:19.488756+0000","last_fullsized":"2023-11-28T04:08:19.488756+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.489052+0000","last_change":"2023-11-28T04:08:19.485259+0000","last_active":"2023-11-28T04:08:19.489052+0000","last_peered":"2023-11-28T04:08:19.489052+0000","last_clean":"2023-11-28T04:08:19.489052+0000","last_became_active":"2023-11-28T04:08:19.484986+0000","last_became_peered":"2023-11-28T04:08:19.484986+0000","last_unstale":"2023-11-28T04:08:19.489052+0000","last_undegraded":"2023-11-28T04:08:19.489052+0000","last_fullsized":"2023-11-28T04:08:19.489052+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.457930+0000","last_change":"2023-11-28T04:08:19.110891+0000","last_active":"2023-11-28T04:08:19.457930+0000","last_peered":"2023-11-28T04:08:19.457930+0000","last_clean":"2023-11-28T04:08:19.457930+0000","last_became_active":"2023-11-28T04:08:19.110649+0000","last_became_peered":"2023-11-28T04:08:19.110649+0000","last_unstale":"2023-11-28T04:08:19.457930+0000","last_undegraded":"2023-11-28T04:08:19.457930+0000","last_fullsized":"2023-11-28T04:08:19.457930+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.472158+0000","last_change":"2023-11-28T04:08:19.465811+0000","last_active":"2023-11-28T04:08:19.472158+0000","last_peered":"2023-11-28T04:08:19.472158+0000","last_clean":"2023-11-28T04:08:19.472158+0000","last_became_active":"2023-11-28T04:08:19.465556+0000","last_became_peered":"2023-11-28T04:08:19.465556+0000","last_unstale":"2023-11-28T04:08:19.472158+0000","last_undegraded":"2023-11-28T04:08:19.472158+0000","last_fullsized":"2023-11-28T04:08:19.472158+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459744+0000","last_change":"2023-11-28T04:08:19.095101+0000","last_active":"2023-11-28T04:08:19.459744+0000","last_peered":"2023-11-28T04:08:19.459744+0000","last_clean":"2023-11-28T04:08:19.459744+0000","last_became_active":"2023-11-28T04:08:19.094824+0000","last_became_peered":"2023-11-28T04:08:19.094824+0000","last_unstale":"2023-11-28T04:08:19.459744+0000","last_undegraded":"2023-11-28T04:08:19.459744+0000","last_fullsized":"2023-11-28T04:08:19.459744+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481706+0000","last_change":"2023-11-28T04:08:19.477717+0000","last_active":"2023-11-28T04:08:19.481706+0000","last_peered":"2023-11-28T04:08:19.481706+0000","last_clean":"2023-11-28T04:08:19.481706+0000","last_became_active":"2023-11-28T04:08:19.477451+0000","last_became_peered":"2023-11-28T04:08:19.477451+0000","last_unstale":"2023-11-28T04:08:19.481706+0000","last_undegraded":"2023-11-28T04:08:19.481706+0000","last_fullsized":"2023-11-28T04:08:19.481706+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.462352+0000","last_change":"2023-11-28T04:08:19.109577+0000","last_active":"2023-11-28T04:08:19.462352+0000","last_peered":"2023-11-28T04:08:19.462352+0000","last_clean":"2023-11-28T04:08:19.462352+0000","last_became_active":"2023-11-28T04:08:19.109285+0000","last_became_peered":"2023-11-28T04:08:19.109285+0000","last_unstale":"2023-11-28T04:08:19.462352+0000","last_undegraded":"2023-11-28T04:08:19.462352+0000","last_fullsized":"2023-11-28T04:08:19.462352+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.1a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.476844+0000","last_change":"2023-11-28T04:08:19.468602+0000","last_active":"2023-11-28T04:08:19.476844+0000","last_peered":"2023-11-28T04:08:19.476844+0000","last_clean":"2023-11-28T04:08:19.476844+0000","last_became_active":"2023-11-28T04:08:19.468332+0000","last_became_peered":"2023-11-28T04:08:19.468332+0000","last_unstale":"2023-11-28T04:08:19.476844+0000","last_undegraded":"2023-11-28T04:08:19.476844+0000","last_fullsized":"2023-11-28T04:08:19.476844+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.464162+0000","last_change":"2023-11-28T04:08:19.107368+0000","last_active":"2023-11-28T04:08:19.464162+0000","last_peered":"2023-11-28T04:08:19.464162+0000","last_clean":"2023-11-28T04:08:19.464162+0000","last_became_active":"2023-11-28T04:08:19.107127+0000","last_became_peered":"2023-11-28T04:08:19.107127+0000","last_unstale":"2023-11-28T04:08:19.464162+0000","last_undegraded":"2023-11-28T04:08:19.464162+0000","last_fullsized":"2023-11-28T04:08:19.464162+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482059+0000","last_change":"2023-11-28T04:08:19.476555+0000","last_active":"2023-11-28T04:08:19.482059+0000","last_peered":"2023-11-28T04:08:19.482059+0000","last_clean":"2023-11-28T04:08:19.482059+0000","last_became_active":"2023-11-28T04:08:19.475940+0000","last_became_peered":"2023-11-28T04:08:19.475940+0000","last_unstale":"2023-11-28T04:08:19.482059+0000","last_undegraded":"2023-11-28T04:08:19.482059+0000","last_fullsized":"2023-11-28T04:08:19.482059+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459974+0000","last_change":"2023-11-28T04:08:19.103852+0000","last_active":"2023-11-28T04:08:19.459974+0000","last_peered":"2023-11-28T04:08:19.459974+0000","last_clean":"2023-11-28T04:08:19.459974+0000","last_became_active":"2023-11-28T04:08:19.103476+0000","last_became_peered":"2023-11-28T04:08:19.103476+0000","last_unstale":"2023-11-28T04:08:19.459974+0000","last_undegraded":"2023-11-28T04:08:19.459974+0000","last_fullsized":"2023-11-28T04:08:19.459974+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488841+0000","last_change":"2023-11-28T04:08:19.482604+0000","last_active":"2023-11-28T04:08:19.488841+0000","last_peered":"2023-11-28T04:08:19.488841+0000","last_clean":"2023-11-28T04:08:19.488841+0000","last_became_active":"2023-11-28T04:08:19.478643+0000","last_became_peered":"2023-11-28T04:08:19.478643+0000","last_unstale":"2023-11-28T04:08:19.488841+0000","last_undegraded":"2023-11-28T04:08:19.488841+0000","last_fullsized":"2023-11-28T04:08:19.488841+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.f","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485492+0000","last_change":"2023-11-28T04:08:19.105042+0000","last_active":"2023-11-28T04:08:19.485492+0000","last_peered":"2023-11-28T04:08:19.485492+0000","last_clean":"2023-11-28T04:08:19.485492+0000","last_became_active":"2023-11-28T04:08:19.104793+0000","last_became_peered":"2023-11-28T04:08:19.104793+0000","last_unstale":"2023-11-28T04:08:19.485492+0000","last_undegraded":"2023-11-28T04:08:19.485492+0000","last_fullsized":"2023-11-28T04:08:19.485492+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.d","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488819+0000","last_change":"2023-11-28T04:08:19.481196+0000","last_active":"2023-11-28T04:08:19.488819+0000","last_peered":"2023-11-28T04:08:19.488819+0000","last_clean":"2023-11-28T04:08:19.488819+0000","last_became_active":"2023-11-28T04:08:19.478619+0000","last_became_peered":"2023-11-28T04:08:19.478619+0000","last_unstale":"2023-11-28T04:08:19.488819+0000","last_undegraded":"2023-11-28T04:08:19.488819+0000","last_fullsized":"2023-11-28T04:08:19.488819+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.e","version":"178'2","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.483301+0000","last_change":"2023-11-28T04:08:19.107994+0000","last_active":"2023-11-28T04:08:19.483301+0000","last_peered":"2023-11-28T04:08:19.483301+0000","last_clean":"2023-11-28T04:08:19.483301+0000","last_became_active":"2023-11-28T04:08:19.107587+0000","last_became_peered":"2023-11-28T04:08:19.107587+0000","last_unstale":"2023-11-28T04:08:19.483301+0000","last_undegraded":"2023-11-28T04:08:19.483301+0000","last_fullsized":"2023-11-28T04:08:19.483301+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.e","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477142+0000","last_change":"2023-11-28T04:08:19.472670+0000","last_active":"2023-11-28T04:08:19.477142+0000","last_peered":"2023-11-28T04:08:19.477142+0000","last_clean":"2023-11-28T04:08:19.477142+0000","last_became_active":"2023-11-28T04:08:19.472093+0000","last_became_peered":"2023-11-28T04:08:19.472093+0000","last_unstale":"2023-11-28T04:08:19.477142+0000","last_undegraded":"2023-11-28T04:08:19.477142+0000","last_fullsized":"2023-11-28T04:08:19.477142+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.458552+0000","last_change":"2023-11-28T04:08:19.110142+0000","last_active":"2023-11-28T04:08:19.458552+0000","last_peered":"2023-11-28T04:08:19.458552+0000","last_clean":"2023-11-28T04:08:19.458552+0000","last_became_active":"2023-11-28T04:08:19.109878+0000","last_became_peered":"2023-11-28T04:08:19.109878+0000","last_unstale":"2023-11-28T04:08:19.458552+0000","last_undegraded":"2023-11-28T04:08:19.458552+0000","last_fullsized":"2023-11-28T04:08:19.458552+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485069+0000","last_change":"2023-11-28T04:08:19.479782+0000","last_active":"2023-11-28T04:08:19.485069+0000","last_peered":"2023-11-28T04:08:19.485069+0000","last_clean":"2023-11-28T04:08:19.485069+0000","last_became_active":"2023-11-28T04:08:19.479495+0000","last_became_peered":"2023-11-28T04:08:19.479495+0000","last_unstale":"2023-11-28T04:08:19.485069+0000","last_undegraded":"2023-11-28T04:08:19.485069+0000","last_fullsized":"2023-11-28T04:08:19.485069+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473938+0000","last_change":"2023-11-28T04:08:19.097999+0000","last_active":"2023-11-28T04:08:19.473938+0000","last_peered":"2023-11-28T04:08:19.473938+0000","last_clean":"2023-11-28T04:08:19.473938+0000","last_became_active":"2023-11-28T04:08:19.097208+0000","last_became_peered":"2023-11-28T04:08:19.097208+0000","last_unstale":"2023-11-28T04:08:19.473938+0000","last_undegraded":"2023-11-28T04:08:19.473938+0000","last_fullsized":"2023-11-28T04:08:19.473938+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488988+0000","last_change":"2023-11-28T04:08:19.483479+0000","last_active":"2023-11-28T04:08:19.488988+0000","last_peered":"2023-11-28T04:08:19.488988+0000","last_clean":"2023-11-28T04:08:19.488988+0000","last_became_active":"2023-11-28T04:08:19.483176+0000","last_became_peered":"2023-11-28T04:08:19.483176+0000","last_unstale":"2023-11-28T04:08:19.488988+0000","last_undegraded":"2023-11-28T04:08:19.488988+0000","last_fullsized":"2023-11-28T04:08:19.488988+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.b","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.478306+0000","last_change":"2023-11-28T04:08:19.108931+0000","last_active":"2023-11-28T04:08:19.478306+0000","last_peered":"2023-11-28T04:08:19.478306+0000","last_clean":"2023-11-28T04:08:19.478306+0000","last_became_active":"2023-11-28T04:08:19.108087+0000","last_became_peered":"2023-11-28T04:08:19.108087+0000","last_unstale":"2023-11-28T04:08:19.478306+0000","last_undegraded":"2023-11-28T04:08:19.478306+0000","last_fullsized":"2023-11-28T04:08:19.478306+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482265+0000","last_change":"2023-11-28T04:08:19.474722+0000","last_active":"2023-11-28T04:08:19.482265+0000","last_peered":"2023-11-28T04:08:19.482265+0000","last_clean":"2023-11-28T04:08:19.482265+0000","last_became_active":"2023-11-28T04:08:19.474453+0000","last_became_peered":"2023-11-28T04:08:19.474453+0000","last_unstale":"2023-11-28T04:08:19.482265+0000","last_undegraded":"2023-11-28T04:08:19.482265+0000","last_fullsized":"2023-11-28T04:08:19.482265+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.460282+0000","last_change":"2023-11-28T04:08:19.103220+0000","last_active":"2023-11-28T04:08:19.460282+0000","last_peered":"2023-11-28T04:08:19.460282+0000","last_clean":"2023-11-28T04:08:19.460282+0000","last_became_active":"2023-11-28T04:08:19.098166+0000","last_became_peered":"2023-11-28T04:08:19.098166+0000","last_unstale":"2023-11-28T04:08:19.460282+0000","last_undegraded":"2023-11-28T04:08:19.460282+0000","last_fullsized":"2023-11-28T04:08:19.460282+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.488744+0000","last_change":"2023-11-28T04:08:19.478020+0000","last_active":"2023-11-28T04:08:19.488744+0000","last_peered":"2023-11-28T04:08:19.488744+0000","last_clean":"2023-11-28T04:08:19.488744+0000","last_became_active":"2023-11-28T04:08:19.477703+0000","last_became_peered":"2023-11-28T04:08:19.477703+0000","last_unstale":"2023-11-28T04:08:19.488744+0000","last_undegraded":"2023-11-28T04:08:19.488744+0000","last_fullsized":"2023-11-28T04:08:19.488744+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.457485+0000","last_change":"2023-11-28T04:08:19.106346+0000","last_active":"2023-11-28T04:08:19.457485+0000","last_peered":"2023-11-28T04:08:19.457485+0000","last_clean":"2023-11-28T04:08:19.457485+0000","last_became_active":"2023-11-28T04:08:19.106069+0000","last_became_peered":"2023-11-28T04:08:19.106069+0000","last_unstale":"2023-11-28T04:08:19.457485+0000","last_undegraded":"2023-11-28T04:08:19.457485+0000","last_fullsized":"2023-11-28T04:08:19.457485+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.b","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484810+0000","last_change":"2023-11-28T04:08:19.477093+0000","last_active":"2023-11-28T04:08:19.484810+0000","last_peered":"2023-11-28T04:08:19.484810+0000","last_clean":"2023-11-28T04:08:19.484810+0000","last_became_active":"2023-11-28T04:08:19.476790+0000","last_became_peered":"2023-11-28T04:08:19.476790+0000","last_unstale":"2023-11-28T04:08:19.484810+0000","last_undegraded":"2023-11-28T04:08:19.484810+0000","last_fullsized":"2023-11-28T04:08:19.484810+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.459742+0000","last_change":"2023-11-28T04:08:19.104928+0000","last_active":"2023-11-28T04:08:19.459742+0000","last_peered":"2023-11-28T04:08:19.459742+0000","last_clean":"2023-11-28T04:08:19.459742+0000","last_became_active":"2023-11-28T04:08:19.104612+0000","last_became_peered":"2023-11-28T04:08:19.104612+0000","last_unstale":"2023-11-28T04:08:19.459742+0000","last_undegraded":"2023-11-28T04:08:19.459742+0000","last_fullsized":"2023-11-28T04:08:19.459742+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477196+0000","last_change":"2023-11-28T04:08:19.473099+0000","last_active":"2023-11-28T04:08:19.477196+0000","last_peered":"2023-11-28T04:08:19.477196+0000","last_clean":"2023-11-28T04:08:19.477196+0000","last_became_active":"2023-11-28T04:08:19.472877+0000","last_became_peered":"2023-11-28T04:08:19.472877+0000","last_unstale":"2023-11-28T04:08:19.477196+0000","last_undegraded":"2023-11-28T04:08:19.477196+0000","last_fullsized":"2023-11-28T04:08:19.477196+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.7","version":"178'2","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.479936+0000","last_change":"2023-11-28T04:08:19.098526+0000","last_active":"2023-11-28T04:08:19.479936+0000","last_peered":"2023-11-28T04:08:19.479936+0000","last_clean":"2023-11-28T04:08:19.479936+0000","last_became_active":"2023-11-28T04:08:19.098258+0000","last_became_peered":"2023-11-28T04:08:19.098258+0000","last_unstale":"2023-11-28T04:08:19.479936+0000","last_undegraded":"2023-11-28T04:08:19.479936+0000","last_fullsized":"2023-11-28T04:08:19.479936+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480147+0000","last_change":"2023-11-28T04:08:19.474421+0000","last_active":"2023-11-28T04:08:19.480147+0000","last_peered":"2023-11-28T04:08:19.480147+0000","last_clean":"2023-11-28T04:08:19.480147+0000","last_became_active":"2023-11-28T04:08:19.473998+0000","last_became_peered":"2023-11-28T04:08:19.473998+0000","last_unstale":"2023-11-28T04:08:19.480147+0000","last_undegraded":"2023-11-28T04:08:19.480147+0000","last_fullsized":"2023-11-28T04:08:19.480147+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.463897+0000","last_change":"2023-11-28T04:08:19.107048+0000","last_active":"2023-11-28T04:08:19.463897+0000","last_peered":"2023-11-28T04:08:19.463897+0000","last_clean":"2023-11-28T04:08:19.463897+0000","last_became_active":"2023-11-28T04:08:19.106736+0000","last_became_peered":"2023-11-28T04:08:19.106736+0000","last_unstale":"2023-11-28T04:08:19.463897+0000","last_undegraded":"2023-11-28T04:08:19.463897+0000","last_fullsized":"2023-11-28T04:08:19.463897+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482260+0000","last_change":"2023-11-28T04:08:19.478284+0000","last_active":"2023-11-28T04:08:19.482260+0000","last_peered":"2023-11-28T04:08:19.482260+0000","last_clean":"2023-11-28T04:08:19.482260+0000","last_became_active":"2023-11-28T04:08:19.477984+0000","last_became_peered":"2023-11-28T04:08:19.477984+0000","last_unstale":"2023-11-28T04:08:19.482260+0000","last_undegraded":"2023-11-28T04:08:19.482260+0000","last_fullsized":"2023-11-28T04:08:19.482260+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.458645+0000","last_change":"2023-11-28T04:08:19.111559+0000","last_active":"2023-11-28T04:08:19.458645+0000","last_peered":"2023-11-28T04:08:19.458645+0000","last_clean":"2023-11-28T04:08:19.458645+0000","last_became_active":"2023-11-28T04:08:19.111288+0000","last_became_peered":"2023-11-28T04:08:19.111288+0000","last_unstale":"2023-11-28T04:08:19.458645+0000","last_undegraded":"2023-11-28T04:08:19.458645+0000","last_fullsized":"2023-11-28T04:08:19.458645+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.7","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484934+0000","last_change":"2023-11-28T04:08:19.481018+0000","last_active":"2023-11-28T04:08:19.484934+0000","last_peered":"2023-11-28T04:08:19.484934+0000","last_clean":"2023-11-28T04:08:19.484934+0000","last_became_active":"2023-11-28T04:08:19.480861+0000","last_became_peered":"2023-11-28T04:08:19.480861+0000","last_unstale":"2023-11-28T04:08:19.484934+0000","last_undegraded":"2023-11-28T04:08:19.484934+0000","last_fullsized":"2023-11-28T04:08:19.484934+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.462768+0000","last_change":"2023-11-28T04:08:19.104363+0000","last_active":"2023-11-28T04:08:19.462768+0000","last_peered":"2023-11-28T04:08:19.462768+0000","last_clean":"2023-11-28T04:08:19.462768+0000","last_became_active":"2023-11-28T04:08:19.104098+0000","last_became_peered":"2023-11-28T04:08:19.104098+0000","last_unstale":"2023-11-28T04:08:19.462768+0000","last_undegraded":"2023-11-28T04:08:19.462768+0000","last_fullsized":"2023-11-28T04:08:19.462768+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477697+0000","last_change":"2023-11-28T04:08:19.467772+0000","last_active":"2023-11-28T04:08:19.477697+0000","last_peered":"2023-11-28T04:08:19.477697+0000","last_clean":"2023-11-28T04:08:19.477697+0000","last_became_active":"2023-11-28T04:08:19.467467+0000","last_became_peered":"2023-11-28T04:08:19.467467+0000","last_unstale":"2023-11-28T04:08:19.477697+0000","last_undegraded":"2023-11-28T04:08:19.477697+0000","last_fullsized":"2023-11-28T04:08:19.477697+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.464042+0000","last_change":"2023-11-28T04:08:19.105503+0000","last_active":"2023-11-28T04:08:19.464042+0000","last_peered":"2023-11-28T04:08:19.464042+0000","last_clean":"2023-11-28T04:08:19.464042+0000","last_became_active":"2023-11-28T04:08:19.105155+0000","last_became_peered":"2023-11-28T04:08:19.105155+0000","last_unstale":"2023-11-28T04:08:19.464042+0000","last_undegraded":"2023-11-28T04:08:19.464042+0000","last_fullsized":"2023-11-28T04:08:19.464042+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.480326+0000","last_change":"2023-11-28T04:08:19.474923+0000","last_active":"2023-11-28T04:08:19.480326+0000","last_peered":"2023-11-28T04:08:19.480326+0000","last_clean":"2023-11-28T04:08:19.480326+0000","last_became_active":"2023-11-28T04:08:19.474651+0000","last_became_peered":"2023-11-28T04:08:19.474651+0000","last_unstale":"2023-11-28T04:08:19.480326+0000","last_undegraded":"2023-11-28T04:08:19.480326+0000","last_fullsized":"2023-11-28T04:08:19.480326+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.454948+0000","last_change":"2023-11-28T04:08:19.099661+0000","last_active":"2023-11-28T04:08:19.454948+0000","last_peered":"2023-11-28T04:08:19.454948+0000","last_clean":"2023-11-28T04:08:19.454948+0000","last_became_active":"2023-11-28T04:08:19.099369+0000","last_became_peered":"2023-11-28T04:08:19.099369+0000","last_unstale":"2023-11-28T04:08:19.454948+0000","last_undegraded":"2023-11-28T04:08:19.454948+0000","last_fullsized":"2023-11-28T04:08:19.454948+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.477006+0000","last_change":"2023-11-28T04:08:19.468675+0000","last_active":"2023-11-28T04:08:19.477006+0000","last_peered":"2023-11-28T04:08:19.477006+0000","last_clean":"2023-11-28T04:08:19.477006+0000","last_became_active":"2023-11-28T04:08:19.468437+0000","last_became_peered":"2023-11-28T04:08:19.468437+0000","last_unstale":"2023-11-28T04:08:19.477006+0000","last_undegraded":"2023-11-28T04:08:19.477006+0000","last_fullsized":"2023-11-28T04:08:19.477006+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.464243+0000","last_change":"2023-11-28T04:08:19.106281+0000","last_active":"2023-11-28T04:08:19.464243+0000","last_peered":"2023-11-28T04:08:19.464243+0000","last_clean":"2023-11-28T04:08:19.464243+0000","last_became_active":"2023-11-28T04:08:19.105943+0000","last_became_peered":"2023-11-28T04:08:19.105943+0000","last_unstale":"2023-11-28T04:08:19.464243+0000","last_undegraded":"2023-11-28T04:08:19.464243+0000","last_fullsized":"2023-11-28T04:08:19.464243+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.476886+0000","last_change":"2023-11-28T04:08:19.468620+0000","last_active":"2023-11-28T04:08:19.476886+0000","last_peered":"2023-11-28T04:08:19.476886+0000","last_clean":"2023-11-28T04:08:19.476886+0000","last_became_active":"2023-11-28T04:08:19.468080+0000","last_became_peered":"2023-11-28T04:08:19.468080+0000","last_unstale":"2023-11-28T04:08:19.476886+0000","last_undegraded":"2023-11-28T04:08:19.476886+0000","last_fullsized":"2023-11-28T04:08:19.476886+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469159+0000","last_change":"2023-11-28T04:08:19.106972+0000","last_active":"2023-11-28T04:08:19.469159+0000","last_peered":"2023-11-28T04:08:19.469159+0000","last_clean":"2023-11-28T04:08:19.469159+0000","last_became_active":"2023-11-28T04:08:19.106701+0000","last_became_peered":"2023-11-28T04:08:19.106701+0000","last_unstale":"2023-11-28T04:08:19.469159+0000","last_undegraded":"2023-11-28T04:08:19.469159+0000","last_fullsized":"2023-11-28T04:08:19.469159+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482033+0000","last_change":"2023-11-28T04:08:19.477600+0000","last_active":"2023-11-28T04:08:19.482033+0000","last_peered":"2023-11-28T04:08:19.482033+0000","last_clean":"2023-11-28T04:08:19.482033+0000","last_became_active":"2023-11-28T04:08:19.475705+0000","last_became_peered":"2023-11-28T04:08:19.475705+0000","last_unstale":"2023-11-28T04:08:19.482033+0000","last_undegraded":"2023-11-28T04:08:19.482033+0000","last_fullsized":"2023-11-28T04:08:19.482033+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.13","version":"178'1","reported_seq":11,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.471276+0000","last_change":"2023-11-28T04:08:19.103813+0000","last_active":"2023-11-28T04:08:19.471276+0000","last_peered":"2023-11-28T04:08:19.471276+0000","last_clean":"2023-11-28T04:08:19.471276+0000","last_became_active":"2023-11-28T04:08:19.103526+0000","last_became_peered":"2023-11-28T04:08:19.103526+0000","last_unstale":"2023-11-28T04:08:19.471276+0000","last_undegraded":"2023-11-28T04:08:19.471276+0000","last_fullsized":"2023-11-28T04:08:19.471276+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":143,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.457972+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:08:19.457972+0000","last_peered":"2023-11-28T04:08:19.457972+0000","last_clean":"2023-11-28T04:08:19.457972+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:08:19.457972+0000","last_undegraded":"2023-11-28T04:08:19.457972+0000","last_fullsized":"2023-11-28T04:08:19.457972+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.481122+0000","last_change":"2023-11-28T04:08:19.475837+0000","last_active":"2023-11-28T04:08:19.481122+0000","last_peered":"2023-11-28T04:08:19.481122+0000","last_clean":"2023-11-28T04:08:19.481122+0000","last_became_active":"2023-11-28T04:08:19.475378+0000","last_became_peered":"2023-11-28T04:08:19.475378+0000","last_unstale":"2023-11-28T04:08:19.481122+0000","last_undegraded":"2023-11-28T04:08:19.481122+0000","last_fullsized":"2023-11-28T04:08:19.481122+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.15","version":"178'3","reported_seq":13,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.474291+0000","last_change":"2023-11-28T04:08:19.103345+0000","last_active":"2023-11-28T04:08:19.474291+0000","last_peered":"2023-11-28T04:08:19.474291+0000","last_clean":"2023-11-28T04:08:19.474291+0000","last_became_active":"2023-11-28T04:08:19.103084+0000","last_became_peered":"2023-11-28T04:08:19.103084+0000","last_unstale":"2023-11-28T04:08:19.474291+0000","last_undegraded":"2023-11-28T04:08:19.474291+0000","last_fullsized":"2023-11-28T04:08:19.474291+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.463789+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:08:19.463789+0000","last_peered":"2023-11-28T04:08:19.463789+0000","last_clean":"2023-11-28T04:08:19.463789+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:08:19.463789+0000","last_undegraded":"2023-11-28T04:08:19.463789+0000","last_fullsized":"2023-11-28T04:08:19.463789+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.15","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.482459+0000","last_change":"2023-11-28T04:08:19.479229+0000","last_active":"2023-11-28T04:08:19.482459+0000","last_peered":"2023-11-28T04:08:19.482459+0000","last_clean":"2023-11-28T04:08:19.482459+0000","last_became_active":"2023-11-28T04:08:19.478940+0000","last_became_peered":"2023-11-28T04:08:19.478940+0000","last_unstale":"2023-11-28T04:08:19.482459+0000","last_undegraded":"2023-11-28T04:08:19.482459+0000","last_fullsized":"2023-11-28T04:08:19.482459+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.465109+0000","last_change":"2023-11-28T04:08:19.106261+0000","last_active":"2023-11-28T04:08:19.465109+0000","last_peered":"2023-11-28T04:08:19.465109+0000","last_clean":"2023-11-28T04:08:19.465109+0000","last_became_active":"2023-11-28T04:08:19.105950+0000","last_became_peered":"2023-11-28T04:08:19.105950+0000","last_unstale":"2023-11-28T04:08:19.465109+0000","last_undegraded":"2023-11-28T04:08:19.465109+0000","last_fullsized":"2023-11-28T04:08:19.465109+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":151,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.454664+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:08:19.454664+0000","last_peered":"2023-11-28T04:08:19.454664+0000","last_clean":"2023-11-28T04:08:19.454664+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:08:19.454664+0000","last_undegraded":"2023-11-28T04:08:19.454664+0000","last_fullsized":"2023-11-28T04:08:19.454664+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.476040+0000","last_change":"2023-11-28T04:08:19.465921+0000","last_active":"2023-11-28T04:08:19.476040+0000","last_peered":"2023-11-28T04:08:19.476040+0000","last_clean":"2023-11-28T04:08:19.476040+0000","last_became_active":"2023-11-28T04:08:19.465639+0000","last_became_peered":"2023-11-28T04:08:19.465639+0000","last_unstale":"2023-11-28T04:08:19.476040+0000","last_undegraded":"2023-11-28T04:08:19.476040+0000","last_fullsized":"2023-11-28T04:08:19.476040+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.14","version":"178'2","reported_seq":12,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.478850+0000","last_change":"2023-11-28T04:08:19.108174+0000","last_active":"2023-11-28T04:08:19.478850+0000","last_peered":"2023-11-28T04:08:19.478850+0000","last_clean":"2023-11-28T04:08:19.478850+0000","last_became_active":"2023-11-28T04:08:19.107895+0000","last_became_peered":"2023-11-28T04:08:19.107895+0000","last_unstale":"2023-11-28T04:08:19.478850+0000","last_undegraded":"2023-11-28T04:08:19.478850+0000","last_fullsized":"2023-11-28T04:08:19.478850+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":143,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.464099+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:08:19.464099+0000","last_peered":"2023-11-28T04:08:19.464099+0000","last_clean":"2023-11-28T04:08:19.464099+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:08:19.464099+0000","last_undegraded":"2023-11-28T04:08:19.464099+0000","last_fullsized":"2023-11-28T04:08:19.464099+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":144,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469244+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:08:19.469244+0000","last_peered":"2023-11-28T04:08:19.469244+0000","last_clean":"2023-11-28T04:08:19.469244+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:08:19.469244+0000","last_undegraded":"2023-11-28T04:08:19.469244+0000","last_fullsized":"2023-11-28T04:08:19.469244+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.484996+0000","last_change":"2023-11-28T04:08:19.479667+0000","last_active":"2023-11-28T04:08:19.484996+0000","last_peered":"2023-11-28T04:08:19.484996+0000","last_clean":"2023-11-28T04:08:19.484996+0000","last_became_active":"2023-11-28T04:08:19.479397+0000","last_became_peered":"2023-11-28T04:08:19.479397+0000","last_unstale":"2023-11-28T04:08:19.484996+0000","last_undegraded":"2023-11-28T04:08:19.484996+0000","last_fullsized":"2023-11-28T04:08:19.484996+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.474375+0000","last_change":"2023-11-28T04:08:19.105498+0000","last_active":"2023-11-28T04:08:19.474375+0000","last_peered":"2023-11-28T04:08:19.474375+0000","last_clean":"2023-11-28T04:08:19.474375+0000","last_became_active":"2023-11-28T04:08:19.105225+0000","last_became_peered":"2023-11-28T04:08:19.105225+0000","last_unstale":"2023-11-28T04:08:19.474375+0000","last_undegraded":"2023-11-28T04:08:19.474375+0000","last_fullsized":"2023-11-28T04:08:19.474375+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":142,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.469324+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:08:19.469324+0000","last_peered":"2023-11-28T04:08:19.469324+0000","last_clean":"2023-11-28T04:08:19.469324+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:08:19.469324+0000","last_undegraded":"2023-11-28T04:08:19.469324+0000","last_fullsized":"2023-11-28T04:08:19.469324+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.485157+0000","last_change":"2023-11-28T04:08:19.477789+0000","last_active":"2023-11-28T04:08:19.485157+0000","last_peered":"2023-11-28T04:08:19.485157+0000","last_clean":"2023-11-28T04:08:19.485157+0000","last_became_active":"2023-11-28T04:08:19.477506+0000","last_became_peered":"2023-11-28T04:08:19.477506+0000","last_unstale":"2023-11-28T04:08:19.485157+0000","last_undegraded":"2023-11-28T04:08:19.485157+0000","last_fullsized":"2023-11-28T04:08:19.485157+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473132+0000","last_change":"2023-11-28T04:08:19.096931+0000","last_active":"2023-11-28T04:08:19.473132+0000","last_peered":"2023-11-28T04:08:19.473132+0000","last_clean":"2023-11-28T04:08:19.473132+0000","last_became_active":"2023-11-28T04:08:19.096660+0000","last_became_peered":"2023-11-28T04:08:19.096660+0000","last_unstale":"2023-11-28T04:08:19.473132+0000","last_undegraded":"2023-11-28T04:08:19.473132+0000","last_fullsized":"2023-11-28T04:08:19.473132+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":142,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.470658+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:08:19.470658+0000","last_peered":"2023-11-28T04:08:19.470658+0000","last_clean":"2023-11-28T04:08:19.470658+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:08:19.470658+0000","last_undegraded":"2023-11-28T04:08:19.470658+0000","last_fullsized":"2023-11-28T04:08:19.470658+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.473912+0000","last_change":"2023-11-28T04:08:19.471975+0000","last_active":"2023-11-28T04:08:19.473912+0000","last_peered":"2023-11-28T04:08:19.473912+0000","last_clean":"2023-11-28T04:08:19.473912+0000","last_became_active":"2023-11-28T04:08:19.471719+0000","last_became_peered":"2023-11-28T04:08:19.471719+0000","last_unstale":"2023-11-28T04:08:19.473912+0000","last_undegraded":"2023-11-28T04:08:19.473912+0000","last_fullsized":"2023-11-28T04:08:19.473912+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":10,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.471069+0000","last_change":"2023-11-28T04:08:19.101897+0000","last_active":"2023-11-28T04:08:19.471069+0000","last_peered":"2023-11-28T04:08:19.471069+0000","last_clean":"2023-11-28T04:08:19.471069+0000","last_became_active":"2023-11-28T04:08:19.101620+0000","last_became_peered":"2023-11-28T04:08:19.101620+0000","last_unstale":"2023-11-28T04:08:19.471069+0000","last_undegraded":"2023-11-28T04:08:19.471069+0000","last_fullsized":"2023-11-28T04:08:19.471069+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":143,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.458221+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:08:19.458221+0000","last_peered":"2023-11-28T04:08:19.458221+0000","last_clean":"2023-11-28T04:08:19.458221+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:08:19.458221+0000","last_undegraded":"2023-11-28T04:08:19.458221+0000","last_fullsized":"2023-11-28T04:08:19.458221+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.13","version":"0'0","reported_seq":9,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.468329+0000","last_change":"2023-11-28T04:08:19.465447+0000","last_active":"2023-11-28T04:08:19.468329+0000","last_peered":"2023-11-28T04:08:19.468329+0000","last_clean":"2023-11-28T04:08:19.468329+0000","last_became_active":"2023-11-28T04:08:19.465181+0000","last_became_peered":"2023-11-28T04:08:19.465181+0000","last_unstale":"2023-11-28T04:08:19.468329+0000","last_undegraded":"2023-11-28T04:08:19.468329+0000","last_fullsized":"2023-11-28T04:08:19.468329+0000","mapping_epoch":177,"log_start":"0'0","ondisk_log_start":"0'0","created":177,"last_epoch_clean":178,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:19.080346+0000","last_clean_scrub_stamp":"2023-11-28T04:08:19.080346+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.10","version":"178'4","reported_seq":14,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.479895+0000","last_change":"2023-11-28T04:08:19.103116+0000","last_active":"2023-11-28T04:08:19.479895+0000","last_peered":"2023-11-28T04:08:19.479895+0000","last_clean":"2023-11-28T04:08:19.479895+0000","last_became_active":"2023-11-28T04:08:19.102833+0000","last_became_peered":"2023-11-28T04:08:19.102833+0000","last_unstale":"2023-11-28T04:08:19.479895+0000","last_undegraded":"2023-11-28T04:08:19.479895+0000","last_fullsized":"2023-11-28T04:08:19.479895+0000","mapping_epoch":176,"log_start":"0'0","ondisk_log_start":"0'0","created":176,"last_epoch_clean":177,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:08:18.075747+0000","last_clean_scrub_stamp":"2023-11-28T04:08:18.075747+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":159,"reported_epoch":178,"state":"active+clean","last_fresh":"2023-11-28T04:08:19.471990+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:08:19.471990+0000","last_peered":"2023-11-28T04:08:19.471990+0000","last_clean":"2023-11-28T04:08:19.471990+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:08:19.471990+0000","last_undegraded":"2023-11-28T04:08:19.471990+0000","last_fullsized":"2023-11-28T04:08:19.471990+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575752,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303384,"kb_used_data":2192,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068456,"statfs":{"total":96636764160,"available":96326098944,"internally_reserved":0,"allocated":2244608,"data_stored":1125621,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.58599999999999997,"5min":0.61799999999999999,"15min":0.61699999999999999},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.29299999999999998},"max":{"1min":0.85999999999999999,"5min":0.86499999999999999,"15min":0.86499999999999999},"last":0.501},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.61599999999999999,"15min":0.61699999999999999},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.38500000000000001},"max":{"1min":0.81599999999999995,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.46899999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.56200000000000006},"min":{"1min":0.43099999999999999,"5min":0.40400000000000003,"15min":0.32100000000000001},"max":{"1min":0.70199999999999996,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56799999999999995,"15min":0.57099999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.78200000000000003,"5min":0.80400000000000005,"15min":0.80400000000000005},"last":0.41499999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.57799999999999996,"15min":0.57299999999999995},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.34599999999999997},"max":{"1min":0.76000000000000001,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.436},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.372,"5min":0.372,"15min":0.35299999999999998},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.753,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58499999999999996,"5min":0.60499999999999998,"15min":0.60399999999999998},"min":{"1min":0.41599999999999998,"5min":0.41099999999999998,"15min":0.39100000000000001},"max":{"1min":0.78700000000000003,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.40100000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.46999999999999997,"15min":0.46400000000000002},"min":{"1min":0.34100000000000003,"5min":0.317,"15min":0.28699999999999998},"max":{"1min":1.0840000000000001,"5min":1.0840000000000001,"15min":1.0840000000000001},"last":0.371},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.49199999999999999,"15min":0.49099999999999999},"min":{"1min":0.33000000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71299999999999997,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48699999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.498,"15min":0.501},"min":{"1min":0.318,"5min":0.312,"15min":0.312},"max":{"1min":0.73599999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53300000000000003,"15min":0.53400000000000003},"min":{"1min":0.38300000000000001,"5min":0.34899999999999998,"15min":0.32400000000000001},"max":{"1min":0.70299999999999996,"5min":0.878,"15min":0.878},"last":0.51400000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:08:17 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55800000000000005,"15min":0.56100000000000005},"min":{"1min":0.41499999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78900000000000003,"5min":0.875,"15min":0.875},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55800000000000005,"15min":0.55600000000000005},"min":{"1min":0.42599999999999999,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.84299999999999997,"5min":0.95499999999999996,"15min":0.95499999999999996},"last":0.55500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575752,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307468,"kb_used_data":2180,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064372,"statfs":{"total":96636764160,"available":96321916928,"internally_reserved":0,"allocated":2232320,"data_stored":1124153,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54600000000000004,"15min":0.54300000000000004},"min":{"1min":0.32700000000000001,"5min":0.32300000000000001,"15min":0.28299999999999997},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55900000000000005,"15min":0.56299999999999994},"min":{"1min":0.39000000000000001,"5min":0.375,"15min":0.29099999999999998},"max":{"1min":0.69699999999999995,"5min":0.73499999999999999,"15min":0.75},"last":0.59199999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.41399999999999998,"5min":0.36199999999999999,"15min":0.32100000000000001},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.495},{"interface":"front","average":{"1min":0.51500000000000001,"5min":0.54600000000000004,"15min":0.54700000000000004},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.70899999999999996,"5min":0.73299999999999998,"15min":0.81899999999999995},"last":0.66400000000000003}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.56899999999999995,"15min":0.57299999999999995},"min":{"1min":0.36699999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":1.075,"5min":1.075,"15min":1.075},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.57899999999999996},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.379},"max":{"1min":0.751,"5min":0.76000000000000001,"15min":0.83099999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58299999999999996},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":1.123,"5min":1.123,"15min":1.123},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58299999999999996,"15min":0.59199999999999997},"min":{"1min":0.41499999999999998,"5min":0.40000000000000002,"15min":0.374},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64500000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45300000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.33100000000000002,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.629,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.35199999999999998,"5min":0.32200000000000001,"15min":0.32000000000000001},"max":{"1min":0.63700000000000001,"5min":0.69099999999999995,"15min":0.78300000000000003},"last":0.72699999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.51000000000000001,"15min":0.52100000000000002},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.311},"max":{"1min":0.89100000000000001,"5min":0.89100000000000001,"15min":0.89100000000000001},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56000000000000005,"15min":0.55300000000000005},"min":{"1min":0.40699999999999997,"5min":0.34999999999999998,"15min":0.33300000000000002},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.73399999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.76900000000000002,"5min":0.77200000000000002,"15min":0.79600000000000004},"last":0.75},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.55500000000000005,"15min":0.55200000000000005},"min":{"1min":0.379,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.79300000000000004,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.63200000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575751,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303380,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068460,"statfs":{"total":96636764160,"available":96326103040,"internally_reserved":0,"allocated":2240512,"data_stored":1124777,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.36199999999999999,"15min":0.35899999999999999},"min":{"1min":0.253,"5min":0.223,"15min":0.21199999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.376,"15min":0.38300000000000001},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.23799999999999999},"max":{"1min":0.53000000000000003,"5min":0.65700000000000003,"15min":0.67000000000000004},"last":0.253}]},{"osd":2,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.40000000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.313,"5min":0.26600000000000001,"15min":0.25800000000000001},"max":{"1min":0.52000000000000002,"5min":0.60399999999999998,"15min":0.60399999999999998},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.41599999999999998,"5min":0.42399999999999999,"15min":0.42099999999999999},"min":{"1min":0.29999999999999999,"5min":0.28999999999999998,"15min":0.23799999999999999},"max":{"1min":0.59199999999999997,"5min":0.67900000000000005,"15min":0.68500000000000005},"last":0.31}]},{"osd":3,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.47199999999999998},"min":{"1min":0.35299999999999998,"5min":0.33700000000000002,"15min":0.307},"max":{"1min":0.56499999999999995,"5min":0.65200000000000002,"15min":0.69799999999999995},"last":0.376},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.434,"15min":0.433},"min":{"1min":0.312,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.56100000000000005,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.60399999999999998,"15min":0.60599999999999998},"min":{"1min":0.374,"5min":0.374,"15min":0.373},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.61799999999999999,"15min":0.61899999999999999},"min":{"1min":0.432,"5min":0.432,"15min":0.36699999999999999},"max":{"1min":0.80400000000000005,"5min":0.85399999999999998,"15min":0.98999999999999999},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.64400000000000002,"15min":0.64900000000000002},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":1.0249999999999999},"last":0.625},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.66500000000000004,"15min":0.67300000000000004},"min":{"1min":0.48099999999999998,"5min":0.46700000000000003,"15min":0.42399999999999999},"max":{"1min":0.86899999999999999,"5min":0.88700000000000001,"15min":1.016},"last":0.59899999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61599999999999999,"15min":0.623},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.38600000000000001},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.92600000000000005},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.64700000000000002,"15min":0.64900000000000002},"min":{"1min":0.496,"5min":0.40600000000000003,"15min":0.37},"max":{"1min":0.81100000000000005,"5min":0.84599999999999997,"15min":0.93100000000000005},"last":0.66800000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:07:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.63900000000000001,"15min":0.64200000000000002},"min":{"1min":0.45800000000000002,"5min":0.443,"15min":0.34499999999999997},"max":{"1min":0.88,"5min":0.88,"15min":0.91700000000000004},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.65600000000000003,"15min":0.64800000000000002},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.40799999999999997},"max":{"1min":0.84899999999999998,"5min":0.86399999999999999,"15min":0.95599999999999996},"last":0.54900000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575752,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303388,"kb_used_data":2196,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068452,"statfs":{"total":96636764160,"available":96326094848,"internally_reserved":0,"allocated":2248704,"data_stored":1124921,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33700000000000002,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.255,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.40999999999999998,"5min":0.52300000000000002,"15min":0.56799999999999995},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.34100000000000003,"5min":0.373,"15min":0.38600000000000001},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.48299999999999998,"5min":0.54200000000000004,"15min":0.60299999999999998},"last":0.30299999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.33800000000000002,"5min":0.36699999999999999,"15min":0.38100000000000001},"min":{"1min":0.27600000000000002,"5min":0.24399999999999999,"15min":0.23200000000000001},"max":{"1min":0.48699999999999999,"5min":0.60299999999999998,"15min":0.61399999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.40100000000000002,"5min":0.45400000000000001,"15min":0.46600000000000003},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.59999999999999998,"5min":0.68000000000000005,"15min":0.70999999999999996},"last":0.36199999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.39700000000000002,"15min":0.41299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.48899999999999999,"5min":0.57599999999999996,"15min":0.67300000000000004},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.42899999999999999,"15min":0.44500000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.52000000000000002,"5min":0.61899999999999999,"15min":0.77500000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.64100000000000001,"15min":0.63800000000000001},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.76400000000000001,"5min":1.24,"15min":1.24},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.64900000000000002,"15min":0.65000000000000002},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.754,"5min":0.97599999999999998,"15min":0.97599999999999998},"last":0.49299999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.64200000000000002,"15min":0.64800000000000002},"min":{"1min":0.436,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.83899999999999997,"5min":0.91200000000000003,"15min":0.98899999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.65300000000000002,"15min":0.64900000000000002},"min":{"1min":0.39900000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.78500000000000003,"5min":1.248,"15min":1.248},"last":0.53400000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.63100000000000001,"15min":0.63},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.77700000000000002,"5min":1.1439999999999999,"15min":1.1439999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.63600000000000001,"15min":0.63900000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.76200000000000001,"5min":1.2,"15min":1.2},"last":0.43099999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:08:02 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.41799999999999998},"max":{"1min":0.80000000000000004,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.627,"5min":0.70199999999999996,"15min":0.70099999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.42599999999999999},"max":{"1min":0.81200000000000006,"5min":1.27,"15min":1.27},"last":0.52100000000000002}]}]},{"osd":2,"up_from":13,"seq":55834575753,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307480,"kb_used_data":2192,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064360,"statfs":{"total":96636764160,"available":96321904640,"internally_reserved":0,"allocated":2244608,"data_stored":1124335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36899999999999999,"15min":0.379},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.52400000000000002,"5min":0.626,"15min":0.626},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.41399999999999998,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.19700000000000001,"5min":0.19700000000000001,"15min":0.19700000000000001},"max":{"1min":0.54900000000000004,"5min":0.63900000000000001,"15min":0.78300000000000003},"last":0.34999999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.35199999999999998,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.45700000000000002,"5min":0.63900000000000001,"15min":0.65400000000000003},"last":0.23699999999999999},{"interface":"front","average":{"1min":0.375,"5min":0.40100000000000002,"15min":0.40899999999999997},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.502,"5min":0.55500000000000005,"15min":0.626},"last":0.30399999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.39300000000000002,"5min":0.42099999999999999,"15min":0.439},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":0.53300000000000003,"5min":0.56799999999999995,"15min":0.80200000000000005},"last":0.317},{"interface":"front","average":{"1min":0.38600000000000001,"5min":0.40200000000000002,"15min":0.41499999999999998},"min":{"1min":0.27100000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.56999999999999995,"5min":0.59899999999999998,"15min":0.71599999999999997},"last":0.33700000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59299999999999997,"15min":0.60499999999999998},"min":{"1min":0.41899999999999998,"5min":0.38900000000000001,"15min":0.38100000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60799999999999998,"15min":0.624},"min":{"1min":0.46100000000000002,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.93400000000000005},"last":0.50900000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.64500000000000002,"15min":0.65800000000000003},"min":{"1min":0.45400000000000001,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79200000000000004,"5min":1.048,"15min":1.048},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.65700000000000003},"min":{"1min":0.45700000000000002,"5min":0.45100000000000001,"15min":0.41899999999999998},"max":{"1min":0.80300000000000005,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76100000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63500000000000001,"15min":0.65600000000000003},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.433},"max":{"1min":0.77800000000000002,"5min":0.90800000000000003,"15min":0.997},"last":0.751},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.66200000000000003,"15min":0.68200000000000005},"min":{"1min":0.50700000000000001,"5min":0.46200000000000002,"15min":0.42299999999999999},"max":{"1min":0.81699999999999995,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.71099999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:07:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.63600000000000001,"15min":0.65200000000000002},"min":{"1min":0.42999999999999999,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.79100000000000004,"5min":0.92800000000000005,"15min":0.97799999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.627,"5min":0.64000000000000001,"15min":0.65600000000000003},"min":{"1min":0.47899999999999998,"5min":0.434,"15min":0.434},"max":{"1min":0.79100000000000004,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.60199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575752,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307472,"kb_used_data":2184,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064368,"statfs":{"total":96636764160,"available":96321912832,"internally_reserved":0,"allocated":2236416,"data_stored":1124187,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.374,"15min":0.371},"min":{"1min":0.254,"5min":0.23999999999999999,"15min":0.22},"max":{"1min":0.59499999999999997,"5min":0.70399999999999996,"15min":0.70399999999999996},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.35499999999999998,"5min":0.38800000000000001,"15min":0.39200000000000002},"min":{"1min":0.24299999999999999,"5min":0.24299999999999999,"15min":0.23999999999999999},"max":{"1min":0.56799999999999995,"5min":0.63300000000000001,"15min":0.63300000000000001},"last":0.315}]},{"osd":1,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.39900000000000002,"15min":0.39600000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.54600000000000004,"5min":0.59899999999999998,"15min":0.626},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.36899999999999999,"5min":0.42899999999999999,"15min":0.42999999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.53900000000000003,"5min":0.68999999999999995,"15min":0.68999999999999995},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.42799999999999999,"15min":0.433},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.58599999999999997,"5min":0.66200000000000003,"15min":0.67400000000000004},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.46600000000000003,"15min":0.46700000000000003},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.68200000000000005,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.28699999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.60699999999999998,"15min":0.61399999999999999},"min":{"1min":0.42499999999999999,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.83499999999999996,"5min":1.056,"15min":1.056},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60999999999999999,"15min":0.63700000000000001},"min":{"1min":0.45000000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.755,"5min":0.93400000000000005,"15min":1.085},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61299999999999999,"15min":0.63500000000000001},"min":{"1min":0.44500000000000001,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.79200000000000004,"5min":0.93400000000000005,"15min":1.1200000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.64200000000000002,"15min":0.65400000000000003},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.88,"5min":1.0329999999999999,"15min":1.0329999999999999},"last":0.44}]},{"osd":6,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.64200000000000002,"15min":0.65100000000000002},"min":{"1min":0.41699999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.91200000000000003,"5min":1.0980000000000001,"15min":1.0980000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.64100000000000001,"15min":0.65900000000000003},"min":{"1min":0.45200000000000001,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86599999999999999,"5min":0.97099999999999997,"15min":1.145},"last":0.5}]},{"osd":7,"last update":"Tue Nov 28 04:08:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.66100000000000003,"15min":0.67000000000000004},"min":{"1min":0.45500000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.006,"15min":1.0189999999999999},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.65600000000000003,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.47099999999999997},"max":{"1min":0.89100000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.50800000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":41,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307476,"kb_used_data":2188,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064364,"statfs":{"total":96636764160,"available":96321908736,"internally_reserved":0,"allocated":2240512,"data_stored":1124797,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.57099999999999995,"15min":0.57199999999999995},"min":{"1min":0.371,"5min":0.34899999999999998,"15min":0.33800000000000002},"max":{"1min":1.472,"5min":1.472,"15min":1.472},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.55700000000000005,"15min":0.56000000000000005},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.32100000000000001},"max":{"1min":1.3,"5min":1.3,"15min":1.3},"last":0.48599999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.64000000000000001,"15min":0.59099999999999997},"min":{"1min":0.376,"5min":0.35699999999999998,"15min":0.34000000000000002},"max":{"1min":8.2300000000000004,"5min":8.2300000000000004,"15min":8.2300000000000004},"last":0.505},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.317},"max":{"1min":1.4470000000000001,"5min":1.4470000000000001,"15min":1.4470000000000001},"last":0.68100000000000005}]},{"osd":2,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.95299999999999996,"5min":0.65600000000000003,"15min":0.60399999999999998},"min":{"1min":0.42699999999999999,"5min":0.39100000000000001,"15min":0.38800000000000001},"max":{"1min":8.2520000000000007,"5min":8.2520000000000007,"15min":8.2520000000000007},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.61599999999999999,"15min":0.61499999999999999},"min":{"1min":0.39000000000000001,"5min":0.38800000000000001,"15min":0.376},"max":{"1min":1.4930000000000001,"5min":1.4930000000000001,"15min":1.4930000000000001},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63500000000000001,"15min":0.63300000000000001},"min":{"1min":0.44400000000000001,"5min":0.433,"15min":0.39700000000000002},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.625}]},{"osd":5,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.56999999999999995,"15min":0.503},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":8.1720000000000006,"5min":8.1720000000000006,"15min":8.1720000000000006},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.53100000000000003,"15min":0.50700000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.512,"5min":1.512,"15min":1.512},"last":0.434}]},{"osd":6,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.51800000000000002,"15min":0.505},"min":{"1min":0.32200000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.365,"5min":1.365,"15min":1.365},"last":0.74099999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":0.64500000000000002,"15min":0.56799999999999995},"min":{"1min":0.33300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":8.202,"5min":8.202,"15min":8.202},"last":0.71399999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:08:21 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.59599999999999997,"15min":0.56000000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.33600000000000002},"max":{"1min":2.2530000000000001,"5min":2.2530000000000001,"15min":2.2530000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.75,"5min":0.627,"15min":0.58599999999999997},"min":{"1min":0.41699999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":2.6459999999999999,"5min":2.6459999999999999,"15min":2.6459999999999999},"last":0.69699999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575753,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307472,"kb_used_data":2184,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064368,"statfs":{"total":96636764160,"available":96321912832,"internally_reserved":0,"allocated":2236416,"data_stored":1125045,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.54300000000000004,"15min":0.53300000000000003},"min":{"1min":0.35599999999999998,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.755,"15min":0.77800000000000002},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.45300000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.59499999999999997,"15min":0.57999999999999996},"min":{"1min":0.36799999999999999,"5min":0.34999999999999998,"15min":0.33900000000000002},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.40899999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":1.631,"5min":1.631,"15min":1.631},"last":0.54200000000000004}]},{"osd":2,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60799999999999998,"15min":0.59699999999999998},"min":{"1min":0.38500000000000001,"5min":0.35699999999999998,"15min":0.33900000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.309},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.61699999999999999,"15min":0.60499999999999998},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"last":0.47599999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.57399999999999995,"15min":0.55700000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.40999999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.45700000000000002,"15min":0.45100000000000001},"min":{"1min":0.318,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.56899999999999995,"15min":0.54400000000000004},"min":{"1min":0.34899999999999998,"5min":0.34599999999999997,"15min":0.32700000000000001},"max":{"1min":1.5780000000000001,"5min":1.5780000000000001,"15min":1.5780000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.498,"15min":0.49099999999999999},"min":{"1min":0.30399999999999999,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.56799999999999995,"15min":0.55400000000000005},"min":{"1min":0.39400000000000002,"5min":0.34100000000000003,"15min":0.33400000000000002},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.52900000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:08:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.52000000000000002,"15min":0.498},"min":{"1min":0.34699999999999998,"5min":0.33000000000000002,"15min":0.26600000000000001},"max":{"1min":1.5409999999999999,"5min":1.5409999999999999,"15min":1.5409999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.54900000000000004,"15min":0.53200000000000003},"min":{"1min":0.40999999999999998,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.871,"5min":0.871,"15min":0.871},"last":0.60299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:08:27.183 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T04:08:27.183 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T04:08:27.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.335+0000 7f6bd5c90700 1 Processor -- start 2023-11-28T04:08:27.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.335+0000 7f6bd5c90700 1 -- start start 2023-11-28T04:08:27.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd0098d10 0x7f6bd00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 0x7f6bd0063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bd0064350 0x7f6bd0158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd00c72d0 con 0x7f6bd0064350 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd0154bc0 con 0x7f6bd00c6190 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bd5c90700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd00c66c0 con 0x7f6bd0098d10 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bd0064350 0x7f6bd0158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6bd0064350 0x7f6bd0158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40078/0 (socket says 172.21.15.73:40078) 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 -- 172.21.15.73:0/2641468092 learned_addr learned my addr 172.21.15.73:0/2641468092 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcf7fe700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd0098d10 0x7f6bd00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bceffd700 1 --2- 172.21.15.73:0/2641468092 >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 0x7f6bd0063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 -- 172.21.15.73:0/2641468092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd0098d10 msgr2=0x7f6bd00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd0098d10 0x7f6bd00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 -- 172.21.15.73:0/2641468092 >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 msgr2=0x7f6bd0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 --2- 172.21.15.73:0/2641468092 >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 0x7f6bd0063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 -- 172.21.15.73:0/2641468092 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd00c69a0 con 0x7f6bd0064350 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bceffd700 1 --2- 172.21.15.73:0/2641468092 >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 0x7f6bd0063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.336+0000 7f6bcffff700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0158320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f6bd00c8e20 tx=0x7f6bc8009840).ready entity=mon.0 client_cookie=a6def15d51f7143d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bce7fc700 1 -- 172.21.15.73:0/2641468092 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800e070 con 0x7f6bd0064350 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bce7fc700 1 -- 172.21.15.73:0/2641468092 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc800ba50 con 0x7f6bd0064350 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bce7fc700 1 -- 172.21.15.73:0/2641468092 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc800bdd0 con 0x7f6bd0064350 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2641468092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 msgr2=0x7f6bd0158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0158320 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 rx=0x7f6bd00c8e20 tx=0x7f6bc8009840).stop 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2641468092 shutdown_connections 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0158320 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2641468092 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd0098d10 0x7f6bd00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2641468092 >> v2:172.21.15.150:3300/0 conn(0x7f6bd00c6190 0x7f6bd0063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2641468092 >> 172.21.15.73:0/2641468092 conn(0x7f6bd00cb310 msgr2=0x7f6bd00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:27.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2641468092 shutdown_connections 2023-11-28T04:08:27.341 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.337+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2641468092 wait complete. 2023-11-28T04:08:27.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.338+0000 7f6bd5c90700 1 Processor -- start 2023-11-28T04:08:27.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.338+0000 7f6bd5c90700 1 -- start start 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd0098d10 0x7f6bd00a9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd00c6190 0x7f6bd0154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd014ec80 con 0x7f6bd0064350 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd014ef60 con 0x7f6bd0098d10 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bd5c90700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd014e1d0 con 0x7f6bd00c6190 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bcf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bcf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40086/0 (socket says 172.21.15.73:40086) 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bcf7fe700 1 -- 172.21.15.73:0/2507201016 learned_addr learned my addr 172.21.15.73:0/2507201016 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bcffff700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd00c6190 0x7f6bd0154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.339+0000 7f6bceffd700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd0098d10 0x7f6bd00a9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 -- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd00c6190 msgr2=0x7f6bd0154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd00c6190 0x7f6bd0154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 -- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd0098d10 msgr2=0x7f6bd00a9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd0098d10 0x7f6bd00a9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 -- 172.21.15.73:0/2507201016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bc8009ae0 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bcf7fe700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7f6bc000d890 tx=0x7f6bc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0011590 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc00116f0 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd00aa1b0 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0011a70 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.340+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd00aa770 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.341+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd006c860 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.341+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6bc0012420 con 0x7f6bd0064350 2023-11-28T04:08:27.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.341+0000 7f6bb77fe700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 0x7f6bbc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.342+0000 7f6bceffd700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 0x7f6bbc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.342+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f6bc0079ba0 con 0x7f6bd0064350 2023-11-28T04:08:27.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.342+0000 7f6bceffd700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 0x7f6bbc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6bd00ad7a0 tx=0x7f6bc4008040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.346+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6bc004ae10 con 0x7f6bd0064350 2023-11-28T04:08:27.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.581+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f6bd00c6c80 con 0x7f6bd0064350 2023-11-28T04:08:27.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.585+0000 7f6bb77fe700 1 -- 172.21.15.73:0/2507201016 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v40) v1 ==== 190+0+0 (secure 0 0 0) 0x7f6bc004e430 con 0x7f6bd0064350 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.587+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 msgr2=0x7f6bbc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.587+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 0x7f6bbc061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f6bd00ad7a0 tx=0x7f6bc4008040).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 msgr2=0x7f6bd0156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 rx=0x7f6bc000d890 tx=0x7f6bc0008040).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 shutdown_connections 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f6bbc05f150 0x7f6bbc061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd00c6190 0x7f6bd0154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd0064350 0x7f6bd0156230 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 --2- 172.21.15.73:0/2507201016 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd0098d10 0x7f6bd00a9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 >> 172.21.15.73:0/2507201016 conn(0x7f6bd00cb310 msgr2=0x7f6bd0152290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 shutdown_connections 2023-11-28T04:08:27.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.588+0000 7f6bd5c90700 1 -- 172.21.15.73:0/2507201016 wait complete. 2023-11-28T04:08:27.590 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T04:08:27.600 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:08:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.712+0000 7f796ed8f700 1 Processor -- start 2023-11-28T04:08:27.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.712+0000 7f796ed8f700 1 -- start start 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796ed8f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f79681544e0 0x7f79681548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796ed8f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f79680c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796ed8f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79680c69a0 con 0x7f796806a610 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796ed8f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79680c13a0 con 0x7f79681544e0 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796ed8f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79680c1680 con 0x7f79680c6e30 2023-11-28T04:08:27.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f7967fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f796806a610 0x7f79680c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59754/0 (socket says 172.21.15.73:59754) 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f7967fff700 1 -- 172.21.15.73:0/1580634472 learned_addr learned my addr 172.21.15.73:0/1580634472 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1580634472 >> v2:172.21.15.150:3300/0 conn(0x7f79681544e0 0x7f79681548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f796d32c700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f79680c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3567404734 0 0) 0x7f79680c69a0 con 0x7f796806a610 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.713+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f79500033f0 con 0x7f796806a610 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155795986 0 0) 0x7f79500033f0 con 0x7f796806a610 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 msgr2=0x7f79680c72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f79680c72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 >> v2:172.21.15.150:3300/0 conn(0x7f79681544e0 msgr2=0x7f79681548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 --2- 172.21.15.73:0/1580634472 >> v2:172.21.15.150:3300/0 conn(0x7f79681544e0 0x7f79681548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680c6630 con 0x7f796806a610 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f795c001470 con 0x7f796806a610 2023-11-28T04:08:27.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 >> v1:172.21.15.73:6789/0 conn(0x7f796806a610 legacy=0x7f79680c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79500038e0 0x7f7950003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7950004690 0x7f7950004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f7950005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680c13a0 con 0x7f79500038e0 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79680c1680 con 0x7f7950004690 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f795000b8e0 con 0x7f79680c6e30 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79500038e0 0x7f7950003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f7967fff700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7950004690 0x7f7950004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796d32c700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f7950005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 -- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 msgr2=0x7f7950005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f7950005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 -- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7950004690 msgr2=0x7f7950004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7950004690 0x7f7950004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 -- 172.21.15.73:0/1580634472 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79500033f0 con 0x7f79500038e0 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796d32c700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79680c6e30 0x7f7950005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79500038e0 0x7f7950003d00 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f795800e2d0 tx=0x7f795800b040).ready entity=mon.0 client_cookie=432d61c924296a9b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7958009b10 con 0x7f79500038e0 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f795800f040 con 0x7f79500038e0 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.714+0000 7f79677fe700 1 -- 172.21.15.73:0/1580634472 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f795800ed30 con 0x7f79500038e0 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 -- 172.21.15.73:0/1580634472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79500038e0 msgr2=0x7f7950003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1580634472 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f79500038e0 0x7f7950003d00 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 rx=0x7f795800e2d0 tx=0x7f795800b040).stop 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796d32c700 1 -- 172.21.15.73:0/1580634472 reap_dead start 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 -- 172.21.15.73:0/1580634472 shutdown_connections 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 -- 172.21.15.73:0/1580634472 >> 172.21.15.73:0/1580634472 conn(0x7f7968085020 msgr2=0x7f7968094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:27.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 -- 172.21.15.73:0/1580634472 shutdown_connections 2023-11-28T04:08:27.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.715+0000 7f796ed8f700 1 -- 172.21.15.73:0/1580634472 wait complete. 2023-11-28T04:08:27.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.716+0000 7f796ed8f700 1 Processor -- start 2023-11-28T04:08:27.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.716+0000 7f796ed8f700 1 -- start start 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79500038e0 0x7f79680955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968142f90 0x7f7968096230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968096f90 con 0x7f7968096720 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7968095130 con 0x7f7968142f90 2023-11-28T04:08:27.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796ed8f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7968095ab0 con 0x7f79500038e0 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40096/0 (socket says 172.21.15.73:40096) 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 -- 172.21.15.73:0/1476799437 learned_addr learned my addr 172.21.15.73:0/1476799437 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 -- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79500038e0 msgr2=0x7f79680955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79500038e0 0x7f79680955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 -- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968142f90 msgr2=0x7f7968096230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f7967fff700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968142f90 0x7f7968096230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968142f90 0x7f7968096230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.717+0000 7f796d32c700 1 -- 172.21.15.73:0/1476799437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7968096cb0 con 0x7f7968096720 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f796d32c700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f7968062ec0 tx=0x7f7960008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960009410 con 0x7f7968096720 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79600125b0 con 0x7f7968096720 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7968094ae0 con 0x7f7968096720 2023-11-28T04:08:27.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7960012950 con 0x7f7968096720 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.718+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7968143410 con 0x7f7968096720 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.719+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7968094940 con 0x7f7968096720 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.719+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7960012d00 con 0x7f7968096720 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.719+0000 7f7964ff9700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.719+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f796007a2d0 con 0x7f7968096720 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.720+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.720+0000 7f796cb2b700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 0x7f7954061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7958009f20 tx=0x7f795801a000).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:27.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.724+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7968094940 con 0x7f7968096720 2023-11-28T04:08:27.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.927+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7968094940 con 0x7f7968096720 2023-11-28T04:08:27.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.927+0000 7f7964ff9700 1 -- 172.21.15.73:0/1476799437 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f7968094940 con 0x7f7968096720 2023-11-28T04:08:27.928 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:08:27.928 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:08:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 msgr2=0x7f7954061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 0x7f7954061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f7958009f20 tx=0x7f795801a000).stop 2023-11-28T04:08:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 msgr2=0x7f79680b0220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:27.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 rx=0x7f7968062ec0 tx=0x7f7960008040).stop 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 shutdown_connections 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f795405f150 0x7f7954061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7968096720 0x7f79680b0220 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f79500038e0 0x7f79680955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 --2- 172.21.15.73:0/1476799437 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7968142f90 0x7f7968096230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 >> 172.21.15.73:0/1476799437 conn(0x7f7968085020 msgr2=0x7f79680aead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 shutdown_connections 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:27.929+0000 7f796ed8f700 1 -- 172.21.15.73:0/1476799437 wait complete. 2023-11-28T04:08:27.931 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:08:28.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 64, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T04:08:19.448808+0000', 'modified': '2023-11-28T04:08:21.457494+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9006}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9006': {'gid': 9006, 'name': 'a', 'rank': 0, 'incarnation': 62, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/3312350001', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 3312350001}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 3312350001}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [22], 'metadata_pool': 21, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T04:08:28.345 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T04:08:28.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T04:08:28.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.462+0000 7f7cb6b97700 1 Processor -- start 2023-11-28T04:08:28.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.462+0000 7f7cb6b97700 1 -- start start 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb6b97700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 0x7f7cb00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb6b97700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7cb00c6190 0x7f7cb0063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb6b97700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cb00c72d0 con 0x7f7cb0064350 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb6b97700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cb0154bc0 con 0x7f7cb00c6190 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb6b97700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb00c66c0 con 0x7f7cb0098d10 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7caffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7cb00c6190 0x7f7cb0063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7caffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f7cb00c6190 0x7f7cb0063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40110/0 (socket says 172.21.15.73:40110) 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7caffff700 1 -- 172.21.15.73:0/3151910168 learned_addr learned my addr 172.21.15.73:0/3151910168 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb5134700 1 --1- 172.21.15.73:0/3151910168 >> v1:172.21.15.73:6789/0 conn(0x7f7cb0064350 0x7f7cb0158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59756/0 (socket says 172.21.15.73:59756) 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.463+0000 7f7cb4933700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 0x7f7cb00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caffff700 1 -- 172.21.15.73:0/3151910168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 msgr2=0x7f7cb00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caffff700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 0x7f7cb00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caffff700 1 -- 172.21.15.73:0/3151910168 >> v1:172.21.15.73:6789/0 conn(0x7f7cb0064350 legacy=0x7f7cb0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caffff700 1 -- 172.21.15.73:0/3151910168 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cb00c69a0 con 0x7f7cb00c6190 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caf7fe700 1 -- 172.21.15.73:0/3151910168 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4227830373 0 0) 0x7f7cb00c72d0 con 0x7f7cb0064350 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb4933700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 0x7f7cb00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caffff700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb00c6190 0x7f7cb0063e60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7f7ca4006fd0 tx=0x7f7ca40090f0).ready entity=mon.0 client_cookie=b482034e3548e68a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caf7fe700 1 -- 172.21.15.73:0/3151910168 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400b600 con 0x7f7cb00c6190 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caf7fe700 1 -- 172.21.15.73:0/3151910168 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca400b760 con 0x7f7cb00c6190 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7caf7fe700 1 -- 172.21.15.73:0/3151910168 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400bae0 con 0x7f7cb00c6190 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 -- 172.21.15.73:0/3151910168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb00c6190 msgr2=0x7f7cb0063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb00c6190 0x7f7cb0063e60 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 rx=0x7f7ca4006fd0 tx=0x7f7ca40090f0).stop 2023-11-28T04:08:28.467 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 -- 172.21.15.73:0/3151910168 shutdown_connections 2023-11-28T04:08:28.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb0098d10 0x7f7cb00c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/3151910168 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb00c6190 0x7f7cb0063e60 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 -- 172.21.15.73:0/3151910168 >> 172.21.15.73:0/3151910168 conn(0x7f7cb00cb310 msgr2=0x7f7cb00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 -- 172.21.15.73:0/3151910168 shutdown_connections 2023-11-28T04:08:28.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.464+0000 7f7cb6b97700 1 -- 172.21.15.73:0/3151910168 wait complete. 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.465+0000 7f7cb6b97700 1 Processor -- start 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 -- start start 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0098d10 0x7f7cb01539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb00c6190 0x7f7cb01576e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0157bd0 con 0x7f7cb0064350 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0152930 con 0x7f7cb0098d10 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.466+0000 7f7cb6b97700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cb0152c10 con 0x7f7cb00c6190 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40126/0 (socket says 172.21.15.73:40126) 2023-11-28T04:08:28.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 -- 172.21.15.73:0/1524246881 learned_addr learned my addr 172.21.15.73:0/1524246881 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 -- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb00c6190 msgr2=0x7f7cb01576e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb5134700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb00c6190 0x7f7cb01576e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7caffff700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0098d10 0x7f7cb01539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb00c6190 0x7f7cb01576e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 -- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0098d10 msgr2=0x7f7cb01539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0098d10 0x7f7cb01539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 -- 172.21.15.73:0/1524246881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ca4009370 con 0x7f7cb0064350 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb4933700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f7ca000d1d0 tx=0x7f7ca0009a90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cacff9700 1 -- 172.21.15.73:0/1524246881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca000de70 con 0x7f7cb0064350 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cacff9700 1 -- 172.21.15.73:0/1524246881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca000f040 con 0x7f7cb0064350 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb0152ef0 con 0x7f7cb0064350 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cacff9700 1 -- 172.21.15.73:0/1524246881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca0013600 con 0x7f7cb0064350 2023-11-28T04:08:28.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.467+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb01534b0 con 0x7f7cb0064350 2023-11-28T04:08:28.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.471+0000 7f7cc852fb80 1 Processor -- start 2023-11-28T04:08:28.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.471+0000 7f7cc852fb80 1 -- start start 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573ad2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aaa7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b573a70060 con 0x55b573aaacb0 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5739e71f0 con 0x55b573866e40 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5739fc440 con 0x55b573af3ea0 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40142/0 (socket says 172.21.15.73:40142) 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 -- 172.21.15.73:0/4214363307 learned_addr learned my addr 172.21.15.73:0/4214363307 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cacff9700 1 -- 172.21.15.73:0/1524246881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7ca00139f0 con 0x7f7cb0064350 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 -- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 msgr2=0x55b573aaa7c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-11-28T04:08:28.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aaa7c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cb5134700 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573ad2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 -- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 msgr2=0x55b573ad2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573ad2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 -- 172.21.15.73:0/4214363307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b5739f5620 con 0x55b573aaacb0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cb5134700 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573ad2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7caffff700 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f7cb00c2360 tx=0x7f7ca4017000).ready entity=mon.0 client_cookie=3ddbf4cc6fda2b5d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cacff9700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 0x7f7c9c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7c95ffb700 1 -- 172.21.15.73:0/4214363307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca400bea0 con 0x55b573aaacb0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7c95ffb700 1 -- 172.21.15.73:0/4214363307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4000bc0 con 0x55b573aaacb0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7c95ffb700 1 -- 172.21.15.73:0/4214363307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca401b410 con 0x55b573aaacb0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cacff9700 1 -- 172.21.15.73:0/1524246881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ca007a9d0 con 0x7f7cb0064350 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.472+0000 7f7cb5134700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 0x7f7c9c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4214363307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 msgr2=0x55b573aa5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 rx=0x7f7cb00c2360 tx=0x7f7ca4017000).stop 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cb5134700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 0x7f7c9c061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7ca8000c00 tx=0x7f7ca8009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4214363307 shutdown_connections 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aa5600 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573ad2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4214363307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aaa7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4214363307 >> 172.21.15.73:0/4214363307 conn(0x55b5739144a0 msgr2=0x55b573af3790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4214363307 shutdown_connections 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.473+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4214363307 wait complete. 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.474+0000 7f7cc852fb80 1 -- start start 2023-11-28T04:08:28.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.474+0000 7f7cc852fb80 1 Processor -- start 2023-11-28T04:08:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.474+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573aecb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573ad79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5739fc580 con 0x55b573aaacb0 2023-11-28T04:08:28.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b5739fc800 con 0x55b573866e40 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b5739c7e10 con 0x55b573af3ea0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 10 client.0 ll_register_callbacks cb 0x55b573a6a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40146/0 (socket says 172.21.15.73:40146) 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 -- 172.21.15.73:0/3028466154 learned_addr learned my addr 172.21.15.73:0/3028466154 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cb4933700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573aecb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cb5134700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573ad79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 -- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 msgr2=0x55b573ad79c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573ad79c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 -- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 msgr2=0x55b573aecb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573aecb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 -- 172.21.15.73:0/3028466154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cb00c69a0 con 0x55b573aaacb0 2023-11-28T04:08:28.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cb5134700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573ad79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7caffff700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f7ca4008000 tx=0x7f7ca401c000).ready entity=mon.0 client_cookie=4aaa16351e88dd81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4029070 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ca401bbe0 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b573a9e720 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca4024450 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 10 client.9159 fetch_fsmap learned FSMap version 64 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55b5739f09c0 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.475+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b573a9e720 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7ca40098e0 con 0x55b573aaacb0 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 10 client.9159 fetch_fsmap finished waiting for FSMap version 64 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 10 client.9159 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 20 client.9159 populate_metadata read hostname 'smithi073' 2023-11-28T04:08:28.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573ad0000 0x55b573ad83a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 4 client.9159 mds_command: new command op to 9006 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 --> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b573862ca0 con 0x55b573ad0000 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cb4933700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573ad0000 0x55b573ad83a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7c8effd700 10 client.9159 ms_handle_connect on v2:172.21.15.73:6835/3312350001 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7cb4933700 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573ad0000 0x55b573ad83a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=514500804d35409b server_cookie=b292bc9934ef9396 in_seq=0 out_seq=0 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7c8effd700 1 -- 172.21.15.73:0/3028466154 <== mds.0 v2:172.21.15.73:6835/3312350001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x55b573862ca0 con 0x55b573ad0000 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.476+0000 7f7c8effd700 10 client.9159 handle_command_reply: tid=0 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 client.9159 shutdown 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 msgr2=0x55b573aecff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 rx=0x7f7ca4008000 tx=0x7f7ca401c000).stop 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 shutdown_connections 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573ad79c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573aecb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573ad0000 0x55b573ad83a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/3028466154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aecff0 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7c8effd700 0 client.9159 ms_handle_reset on v2:172.21.15.73:6835/3312350001 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 >> 172.21.15.73:0/3028466154 conn(0x55b5739144a0 msgr2=0x55b573aab0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 shutdown_connections 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 1 -- 172.21.15.73:0/3028466154 wait complete. 2023-11-28T04:08:28.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.477+0000 7f7cc852fb80 20 client.9159 trim_cache size 0 max 16384 2023-11-28T04:08:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.493+0000 7f7cc852fb80 1 Processor -- start 2023-11-28T04:08:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.493+0000 7f7cc852fb80 1 -- start start 2023-11-28T04:08:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573b088f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aebd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b573aec220 con 0x55b573aaacb0 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b573aced30 con 0x55b573866e40 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b573aac2c0 con 0x55b573af3ea0 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40160/0 (socket says 172.21.15.73:40160) 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 -- 172.21.15.73:0/1537261745 learned_addr learned my addr 172.21.15.73:0/1537261745 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 -- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 msgr2=0x55b573aebd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aebd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 -- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 msgr2=0x55b573b088f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573b088f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 -- 172.21.15.73:0/1537261745 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b573a9e340 con 0x55b573aaacb0 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7caffff700 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f7ca400b210 tx=0x7f7ca4029000).ready entity=mon.0 client_cookie=56f656b13195928 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.494+0000 7f7c967fc700 1 -- 172.21.15.73:0/1537261745 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca402e450 con 0x55b573aaacb0 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7c967fc700 1 -- 172.21.15.73:0/1537261745 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca401bbe0 con 0x55b573aaacb0 2023-11-28T04:08:28.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7c967fc700 1 -- 172.21.15.73:0/1537261745 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca401b570 con 0x55b573aaacb0 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 -- 172.21.15.73:0/1537261745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 msgr2=0x55b573aeb840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 rx=0x7f7ca400b210 tx=0x7f7ca4029000).stop 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 -- 172.21.15.73:0/1537261745 shutdown_connections 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573af3ea0 0x55b573aebd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573866e40 0x55b573b088f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:28.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/1537261745 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573aeb840 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 -- 172.21.15.73:0/1537261745 >> 172.21.15.73:0/1537261745 conn(0x55b5739144a0 msgr2=0x55b573aa4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 -- 172.21.15.73:0/1537261745 shutdown_connections 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.495+0000 7f7cc852fb80 1 -- 172.21.15.73:0/1537261745 wait complete. 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.496+0000 7f7cc852fb80 1 -- start start 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.496+0000 7f7cc852fb80 1 Processor -- start 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573866e40 0x55b573aeba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 0x55b573b3d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b573b3dad0 con 0x55b573aaacb0 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b573b55710 con 0x55b573af3ea0 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b573af2660 con 0x55b573866e40 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 10 client.0 ll_register_callbacks cb 0x55b573a6a460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:08:28.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40170/0 (socket says 172.21.15.73:40170) 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 -- 172.21.15.73:0/4289654563 learned_addr learned my addr 172.21.15.73:0/4289654563 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cb5134700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 0x55b573b3d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 -- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573866e40 msgr2=0x55b573aeba50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cb4933700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573866e40 0x55b573aeba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573866e40 0x55b573aeba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 -- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 msgr2=0x55b573b3d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 0x55b573b3d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 -- 172.21.15.73:0/4289654563 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b5739f09c0 con 0x55b573aaacb0 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cb5134700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 0x55b573b3d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7caffff700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f7ca400bc60 tx=0x7f7ca401c000).ready entity=mon.0 client_cookie=bf1a6fa8c4ca9311 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7c8effd700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ca4029040 con 0x55b573aaacb0 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f7ca40098e0 con 0x55b573aaacb0 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7ca4000bc0 con 0x55b573aaacb0 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=64) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7ca4041e70 con 0x55b573aaacb0 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 10 client.9165 fetch_fsmap learned FSMap version 64 2023-11-28T04:08:28.511 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.497+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=64,monmap=2+}) v3 -- 0x55b573ad7d20 con 0x55b573aaacb0 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ca4000bc0 con 0x55b573aaacb0 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 64) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f7ca4024450 con 0x55b573aaacb0 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 10 client.9165 fetch_fsmap finished waiting for FSMap version 64 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 10 client.9165 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 20 client.9165 populate_metadata read hostname 'smithi073' 2023-11-28T04:08:28.512 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573aa5230 0x55b573aa5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 4 client.9165 mds_command: new command op to 9006 tid=0[{"prefix": "status"}] 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 --> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b573899ca0 con 0x55b573aa5230 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cb4933700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573aa5230 0x55b573aa5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 10 client.9165 ms_handle_connect on v2:172.21.15.73:6835/3312350001 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cb4933700 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573aa5230 0x55b573aa5610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=a86fb659da9e84a server_cookie=f044a9721692198b in_seq=0 out_seq=0 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 1 -- 172.21.15.73:0/4289654563 <== mds.0 v2:172.21.15.73:6835/3312350001 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55b573899ca0 con 0x55b573aa5230 2023-11-28T04:08:28.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 10 client.9165 handle_command_reply: tid=0 2023-11-28T04:08:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 client.9165 shutdown 2023-11-28T04:08:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 msgr2=0x55b573b3d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 rx=0x7f7ca400bc60 tx=0x7f7ca401c000).stop 2023-11-28T04:08:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 shutdown_connections 2023-11-28T04:08:28.514 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x55b573af3ea0 0x55b573b3d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x55b573866e40 0x55b573aeba50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:6835/3312350001,v1:172.21.15.73:6837/3312350001] conn(0x55b573aa5230 0x55b573aa5610 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 --2- 172.21.15.73:0/4289654563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x55b573aaacb0 0x55b573b3d0f0 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7c8effd700 0 client.9165 ms_handle_reset on v2:172.21.15.73:6835/3312350001 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.498+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 >> 172.21.15.73:0/4289654563 conn(0x55b5739144a0 msgr2=0x55b573b084d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 shutdown_connections 2023-11-28T04:08:28.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cc852fb80 1 -- 172.21.15.73:0/4289654563 wait complete. 2023-11-28T04:08:28.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cc852fb80 20 client.9165 trim_cache size 0 max 16384 2023-11-28T04:08:28.516 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 9006, 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T04:08:28.517 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 9.0490753880000003, 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 64, 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 179, 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 179, 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 13.539915625000001 2023-11-28T04:08:28.518 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:08:28.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 msgr2=0x7f7c9c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 0x7f7c9c061610 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 rx=0x7f7ca8000c00 tx=0x7f7ca8009040).stop 2023-11-28T04:08:28.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 msgr2=0x7f7cb0152370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 rx=0x7f7ca000d1d0 tx=0x7f7ca0009a90).stop 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 shutdown_connections 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f7c9c05f150 0x7f7c9c061610 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7cb00c6190 0x7f7cb01576e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7cb0064350 0x7f7cb0152370 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 --2- 172.21.15.73:0/1524246881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7cb0098d10 0x7f7cb01539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 >> 172.21.15.73:0/1524246881 conn(0x7f7cb00cb310 msgr2=0x7f7cb00ca4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 shutdown_connections 2023-11-28T04:08:28.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.499+0000 7f7cb6b97700 1 -- 172.21.15.73:0/1524246881 wait complete. 2023-11-28T04:08:28.523 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T04:08:28.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.625+0000 7f1ca4efa700 1 Processor -- start 2023-11-28T04:08:28.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.625+0000 7f1ca4efa700 1 -- start start 2023-11-28T04:08:28.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1ca4efa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 0x7f1ca00aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1ca4efa700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca00c6d80 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1ca4efa700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ca0154ae0 con 0x7f1ca0154700 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1ca4efa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0064c30 con 0x7f1ca00c8360 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9dd9b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1ca00aa1a0 0x7f1ca00c7c50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59760/0 (socket says 172.21.15.73:59760) 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9dd9b700 1 -- 172.21.15.73:0/3721318010 learned_addr learned my addr 172.21.15.73:0/3721318010 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9ed9d700 1 --2- 172.21.15.73:0/3721318010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 0x7f1ca00aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 662442425 0 0) 0x7f1ca00c6d80 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c880033f0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1334809345 0 0) 0x7f1ca0154ae0 con 0x7f1ca0154700 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ca00c6d80 con 0x7f1ca0154700 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9ed9d700 1 -- 172.21.15.73:0/3721318010 >> v1:172.21.15.150:6789/0 conn(0x7f1ca0154700 legacy=0x7f1ca00c7210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2603952020 0 0) 0x7f1c880033f0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9ed9d700 1 -- 172.21.15.73:0/3721318010 >> v1:172.21.15.73:6789/0 conn(0x7f1ca00aa1a0 legacy=0x7f1ca00c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9ed9d700 1 -- 172.21.15.73:0/3721318010 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ca00c6450 con 0x7f1ca00c8360 2023-11-28T04:08:28.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.626+0000 7f1c9ed9d700 1 --2- 172.21.15.73:0/3721318010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 0x7f1ca00aa620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f1c98009e90 tx=0x7f1c98009780).ready entity=mon.2 client_cookie=11afcfeb1437aeb9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9800e070 con 0x7f1ca00c8360 2023-11-28T04:08:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9800b7a0 con 0x7f1ca00c8360 2023-11-28T04:08:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1c9d59a700 1 -- 172.21.15.73:0/3721318010 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c94001470 con 0x7f1ca00c8360 2023-11-28T04:08:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3721318010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 msgr2=0x7f1ca00aa620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3721318010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 0x7f1ca00aa620 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0x7f1c98009e90 tx=0x7f1c98009780).stop 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3721318010 shutdown_connections 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3721318010 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca00c8360 0x7f1ca00aa620 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3721318010 >> 172.21.15.73:0/3721318010 conn(0x7f1ca00948b0 msgr2=0x7f1ca006a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3721318010 shutdown_connections 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.627+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3721318010 wait complete. 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.628+0000 7f1ca4efa700 1 Processor -- start 2023-11-28T04:08:28.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.628+0000 7f1ca4efa700 1 -- start start 2023-11-28T04:08:28.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 0x7f1ca0142f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 0x7f1ca014e770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca0144ff0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca014f5c0 con 0x7f1ca00c8360 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1ca4efa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ca01683d0 con 0x7f1ca0154700 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1c9e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1c9e59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40172/0 (socket says 172.21.15.73:40172) 2023-11-28T04:08:28.638 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.629+0000 7f1c9e59c700 1 -- 172.21.15.73:0/3440591031 learned_addr learned my addr 172.21.15.73:0/3440591031 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:28.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9ed9d700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 0x7f1ca014e770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9dd9b700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 0x7f1ca0142f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 -- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 msgr2=0x7f1ca014e770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 0x7f1ca014e770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 -- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 msgr2=0x7f1ca0142f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 0x7f1ca0142f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 -- 172.21.15.73:0/3440591031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c98009a20 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9ed9d700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 0x7f1ca014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9dd9b700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 0x7f1ca0142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c9e59c700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7f1c9000c4d0 tx=0x7f1c90007c50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000f470 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9000f5d0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ca014f2e0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9000f970 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.630+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ca0168060 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.631+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ca01686b0 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.631+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1c9000fd20 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.632+0000 7f1c8e7fc700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 0x7f1c80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.632+0000 7f1c9dd9b700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 0x7f1c80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.632+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f1c90079840 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.632+0000 7f1c9dd9b700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 0x7f1c80061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1ca0158090 tx=0x7f1c94009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:28.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.636+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c9004ab10 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.842+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1ca014f080 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.842+0000 7f1c8e7fc700 1 -- 172.21.15.73:0/3440591031 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v64) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c9004e130 con 0x7f1ca00aa1a0 2023-11-28T04:08:28.843 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 msgr2=0x7f1c80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 0x7f1c80061610 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 rx=0x7f1ca0158090 tx=0x7f1c94009040).stop 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 msgr2=0x7f1ca0168860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 rx=0x7f1c9000c4d0 tx=0x7f1c90007c50).stop 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 shutdown_connections 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f1c8005f150 0x7f1c80061610 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1ca0154700 0x7f1ca014e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1ca00aa1a0 0x7f1ca0168860 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 --2- 172.21.15.73:0/3440591031 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1ca00c8360 0x7f1ca0142f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:28.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 >> 172.21.15.73:0/3440591031 conn(0x7f1ca00948b0 msgr2=0x7f1ca00ca250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:28.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 shutdown_connections 2023-11-28T04:08:28.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:28.844+0000 7f1ca4efa700 1 -- 172.21.15.73:0/3440591031 wait complete. 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T04:08:28.856 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T04:08:28.857 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:08:28.857 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:28.857 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.872 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 37971sec preferred_lft 37971sec 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.873 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T04:08:28.874 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:08:28.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:28.875 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T04:08:28.936 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T04:08:28.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:28.937 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T04:08:28.994 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T04:08:28.994 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T04:08:28.994 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T04:08:28.994 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T04:08:28.995 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T04:08:28.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T04:08:28.995 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T04:08:29.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:29.095 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:08:29.109 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T04:08:29.110 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T04:08:29.110 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:08:29.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:29.110 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T04:08:29.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:29.174 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T04:08:29.230 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T04:08:29.230 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:08:29.309 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:08:29.323 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:08:29.336 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:08:29.336 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:08:29.339 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T04:08:29.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:08:29.339 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T04:08:29.404 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T04:08:29.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.556+0000 7f59782d3700 1 Processor -- start 2023-11-28T04:08:29.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.556+0000 7f59782d3700 1 -- start start 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f59782d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f59700c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f59782d3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59700c72d0 con 0x7f5970098d10 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f59782d3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5970064570 con 0x7f59700c6190 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f59782d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5970154bc0 con 0x7f5970158a30 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f597606f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f5970098d10 0x7f5970063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59762/0 (socket says 172.21.15.73:59762) 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f597606f700 1 -- 172.21.15.73:0/1329142048 learned_addr learned my addr 172.21.15.73:0/1329142048 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.557+0000 7f5976870700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f59700c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1575158197 0 0) 0x7f59700c72d0 con 0x7f5970098d10 2023-11-28T04:08:29.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59580033f0 con 0x7f5970098d10 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894952498 0 0) 0x7f5970064570 con 0x7f59700c6190 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59700c72d0 con 0x7f59700c6190 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3182336099 0 0) 0x7f59580033f0 con 0x7f5970098d10 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 msgr2=0x7f59700c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f59700c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 >> v1:172.21.15.150:6789/0 conn(0x7f59700c6190 legacy=0x7f5970158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700c66c0 con 0x7f5970098d10 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f59600017f0 con 0x7f5970098d10 2023-11-28T04:08:29.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 >> v1:172.21.15.73:6789/0 conn(0x7f5970098d10 legacy=0x7f5970063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958003aa0 0x7f5958003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5958004690 0x7f5958004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f5958005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970154bc0 con 0x7f5958004690 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595800b990 con 0x7f5958003aa0 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595800bb20 con 0x7f5970158a30 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5958004690 0x7f5958004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597606f700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958003aa0 0x7f5958003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f5976870700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f5958005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 -- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 msgr2=0x7f5958005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5970158a30 0x7f5958005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 -- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958003aa0 msgr2=0x7f5958003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958003aa0 0x7f5958003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 -- 172.21.15.73:0/1329142048 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580033f0 con 0x7f5958004690 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.558+0000 7f597586e700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5958004690 0x7f5958004b10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f59640018d0 tx=0x7f5964007bb0).ready entity=mon.0 client_cookie=4641fd9e13dc7856 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964007020 con 0x7f5958004690 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964007180 con 0x7f5958004690 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f597506d700 1 -- 172.21.15.73:0/1329142048 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964007500 con 0x7f5958004690 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 -- 172.21.15.73:0/1329142048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5958004690 msgr2=0x7f5958004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 --2- 172.21.15.73:0/1329142048 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5958004690 0x7f5958004b10 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 rx=0x7f59640018d0 tx=0x7f5964007bb0).stop 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f5976870700 1 -- 172.21.15.73:0/1329142048 reap_dead start 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 -- 172.21.15.73:0/1329142048 shutdown_connections 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 -- 172.21.15.73:0/1329142048 >> 172.21.15.73:0/1329142048 conn(0x7f59700cb310 msgr2=0x7f59700cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 -- 172.21.15.73:0/1329142048 shutdown_connections 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.559+0000 7f59782d3700 1 -- 172.21.15.73:0/1329142048 wait complete. 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.560+0000 7f59782d3700 1 Processor -- start 2023-11-28T04:08:29.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.560+0000 7f59782d3700 1 -- start start 2023-11-28T04:08:29.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 0x7f59701529e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 0x7f5970153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59700c35f0 con 0x7f59701531b0 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5970152ed0 con 0x7f5958004690 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f59782d3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59700ade00 con 0x7f59700ad2c0 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40196/0 (socket says 172.21.15.73:40196) 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 -- 172.21.15.73:0/3482270195 learned_addr learned my addr 172.21.15.73:0/3482270195 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f5976870700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 0x7f5970153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597606f700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 0x7f59701529e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 -- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 msgr2=0x7f5970153bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 0x7f5970153bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 -- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 msgr2=0x7f59701529e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 0x7f59701529e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 -- 172.21.15.73:0/3482270195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59700ad840 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f5976870700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 0x7f5970153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597606f700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 0x7f59701529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.561+0000 7f597586e700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7f5958005630 tx=0x7f5964016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.562+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964007020 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.562+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5964012610 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.562+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59700adb20 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.562+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5964012990 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.562+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5970164690 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.563+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f597006c860 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.563+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5964012d00 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.563+0000 7f596e7fc700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 0x7f595c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.563+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f596407b090 con 0x7f59701531b0 2023-11-28T04:08:29.570 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.563+0000 7f597606f700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 0x7f595c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.564+0000 7f597606f700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 0x7f595c061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f5960007fd0 tx=0x7f5960009040).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.567+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f596404c410 con 0x7f59701531b0 2023-11-28T04:08:29.762 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.761+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f59700c3730 con 0x7f595c05f150 2023-11-28T04:08:29.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.763+0000 7f596e7fc700 1 -- 172.21.15.73:0/3482270195 <== mgr.8829 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f59700c3730 con 0x7f595c05f150 2023-11-28T04:08:29.764 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:08:29.764 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:08:29.764 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T04:08:29.764 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:08:29.764 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:08:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 msgr2=0x7f595c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 0x7f595c061610 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 rx=0x7f5960007fd0 tx=0x7f5960009040).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 msgr2=0x7f59700acdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 rx=0x7f5958005630 tx=0x7f5964016000).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 shutdown_connections 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f595c05f150 0x7f595c061610 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f59700ad2c0 0x7f5970153bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5958004690 0x7f59701529e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 --2- 172.21.15.73:0/3482270195 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f59701531b0 0x7f59700acdd0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 >> 172.21.15.73:0/3482270195 conn(0x7f59700cb310 msgr2=0x7f5970156070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 shutdown_connections 2023-11-28T04:08:29.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.765+0000 7f59782d3700 1 -- 172.21.15.73:0/3482270195 wait complete. 2023-11-28T04:08:29.779 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T04:08:29.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.894+0000 7f633ea1c700 1 Processor -- start 2023-11-28T04:08:29.896 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.894+0000 7f633ea1c700 1 -- start start 2023-11-28T04:08:29.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f633ea1c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f63380c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f633ea1c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63380c72d0 con 0x7f6338098d10 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f633ea1c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6338064570 con 0x7f6338158320 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f633ea1c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6338154bc0 con 0x7f63380c6190 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f6337fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6338098d10 0x7f6338063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59770/0 (socket says 172.21.15.73:59770) 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f6337fff700 1 -- 172.21.15.73:0/3388778024 learned_addr learned my addr 172.21.15.73:0/3388778024 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f63377fe700 1 --2- 172.21.15.73:0/3388778024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f63380c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 545965068 0 0) 0x7f63380c72d0 con 0x7f6338098d10 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.895+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63200033f0 con 0x7f6338098d10 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2989910923 0 0) 0x7f6338064570 con 0x7f6338158320 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63380c72d0 con 0x7f6338158320 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f63377fe700 1 -- 172.21.15.73:0/3388778024 >> v1:172.21.15.150:6789/0 conn(0x7f6338158320 legacy=0x7f63380a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f63377fe700 1 -- 172.21.15.73:0/3388778024 >> v1:172.21.15.73:6789/0 conn(0x7f6338098d10 legacy=0x7f6338063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4219576852 0 0) 0x7f63200033f0 con 0x7f6338098d10 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f63377fe700 1 -- 172.21.15.73:0/3388778024 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380c66c0 con 0x7f63380c6190 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f63377fe700 1 --2- 172.21.15.73:0/3388778024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f63380c7760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f6328009790 tx=0x7f6328009060).ready entity=mon.2 client_cookie=4f225798f2c1f4ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c001470 con 0x7f63380c6190 2023-11-28T04:08:29.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328011400 con 0x7f63380c6190 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f6336ffd700 1 -- 172.21.15.73:0/3388778024 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328011780 con 0x7f63380c6190 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f633ea1c700 1 -- 172.21.15.73:0/3388778024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 msgr2=0x7f63380c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f633ea1c700 1 --2- 172.21.15.73:0/3388778024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f63380c7760 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 rx=0x7f6328009790 tx=0x7f6328009060).stop 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f633ea1c700 1 -- 172.21.15.73:0/3388778024 shutdown_connections 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f633ea1c700 1 --2- 172.21.15.73:0/3388778024 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f63380c7760 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.896+0000 7f633ea1c700 1 -- 172.21.15.73:0/3388778024 >> 172.21.15.73:0/3388778024 conn(0x7f63380cb310 msgr2=0x7f63380cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.897+0000 7f633ea1c700 1 -- 172.21.15.73:0/3388778024 shutdown_connections 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.897+0000 7f633ea1c700 1 -- 172.21.15.73:0/3388778024 wait complete. 2023-11-28T04:08:29.907 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.898+0000 7f633ea1c700 1 Processor -- start 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.898+0000 7f633ea1c700 1 -- start start 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.898+0000 7f633ea1c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 0x7f63380ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.898+0000 7f633ea1c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f6338165560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.898+0000 7f633ea1c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633ea1c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63380ae950 con 0x7f6338158320 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633ea1c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6338165a50 con 0x7f6338098d10 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633ea1c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6338168300 con 0x7f63380c6190 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40204/0 (socket says 172.21.15.73:40204) 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 -- 172.21.15.73:0/297074537 learned_addr learned my addr 172.21.15.73:0/297074537 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f63377fe700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f6338165560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f6337fff700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 0x7f63380ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 -- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 msgr2=0x7f6338165560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f6338165560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 -- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 msgr2=0x7f63380ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 0x7f63380ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 -- 172.21.15.73:0/297074537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280092a0 con 0x7f6338158320 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f6337fff700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 0x7f63380ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633cfb9700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f633000c7c0 tx=0x7f633000cad0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330002020 con 0x7f6338158320 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6330002180 con 0x7f6338158320 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6330001590 con 0x7f6338158320 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63380c2790 con 0x7f6338158320 2023-11-28T04:08:29.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.899+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6338147dd0 con 0x7f6338158320 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.900+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6338148050 con 0x7f6338158320 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.901+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 73) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6330001980 con 0x7f6338158320 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.901+0000 7f631ffff700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 0x7f6324061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.901+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f63300797b0 con 0x7f6338158320 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.901+0000 7f6337fff700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 0x7f6324061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.901+0000 7f6337fff700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 0x7f6324061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f632c0011e0 tx=0x7f632c000bc0).ready entity=mgr.8829 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:29.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:29.905+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6330048af0 con 0x7f6338158320 2023-11-28T04:08:30.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:30.112+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f63381680d0 con 0x7f6338158320 2023-11-28T04:08:31.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.047+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 74) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f6330077580 con 0x7f6338158320 2023-11-28T04:08:31.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.048+0000 7f631ffff700 1 -- 172.21.15.73:0/297074537 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v74) v1 ==== 92+0+0 (secure 0 0 0) 0x7f6330074470 con 0x7f6338158320 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 msgr2=0x7f6324061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 0x7f6324061610 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 rx=0x7f632c0011e0 tx=0x7f632c000bc0).stop 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 msgr2=0x7f63381478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 rx=0x7f633000c7c0 tx=0x7f633000cad0).stop 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 shutdown_connections 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f632405f150 0x7f6324061610 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6338158320 0x7f63381478e0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6338098d10 0x7f63380ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 --2- 172.21.15.73:0/297074537 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f63380c6190 0x7f6338165560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 >> 172.21.15.73:0/297074537 conn(0x7f63380cb310 msgr2=0x7f6338148b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 shutdown_connections 2023-11-28T04:08:31.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.051+0000 7f633ea1c700 1 -- 172.21.15.73:0/297074537 wait complete. 2023-11-28T04:08:31.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T04:08:31.192 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:31.191+0000 7f81fe176680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.191+0000 7f1586130700 1 Processor -- start 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 -- start start 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f15801566e0 0x7f15800c9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 0x7f1580067870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15800be960 con 0x7f158013bd10 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1580066f60 con 0x7f15801566e0 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.192+0000 7f1586130700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15800c9d60 con 0x7f15800673f0 2023-11-28T04:08:31.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157f7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f158013bd10 0x7f1580155fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59794/0 (socket says 172.21.15.73:59794) 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157f7fe700 1 -- 172.21.15.73:0/4230605782 learned_addr learned my addr 172.21.15.73:0/4230605782 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 --2- 172.21.15.73:0/4230605782 >> v2:172.21.15.150:3300/0 conn(0x7f15801566e0 0x7f15800c9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157ffff700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 0x7f1580067870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157effd700 1 -- 172.21.15.73:0/4230605782 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1578247441 0 0) 0x7f1570001110 con 0x7f158013bd10 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157effd700 1 -- 172.21.15.73:0/4230605782 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f156c0033f0 con 0x7f158013bd10 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 -- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 msgr2=0x7f1580067870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 0x7f1580067870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 -- 172.21.15.73:0/4230605782 >> v1:172.21.15.73:6789/0 conn(0x7f158013bd10 legacy=0x7f1580155fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 -- 172.21.15.73:0/4230605782 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15800be0c0 con 0x7f15801566e0 2023-11-28T04:08:31.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f1577fff700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15801566e0 0x7f15800c9870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f1568009760 tx=0x7f1568009060).ready entity=mon.1 client_cookie=6e006caa83bc61dd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157ffff700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 0x7f1580067870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157effd700 1 -- 172.21.15.73:0/4230605782 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800d4a0 con 0x7f15801566e0 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157effd700 1 -- 172.21.15.73:0/4230605782 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f156800d600 con 0x7f15801566e0 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.193+0000 7f157effd700 1 -- 172.21.15.73:0/4230605782 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f156800d980 con 0x7f15801566e0 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 -- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15801566e0 msgr2=0x7f15800c9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15801566e0 0x7f15800c9870 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 rx=0x7f1568009760 tx=0x7f1568009060).stop 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 -- 172.21.15.73:0/4230605782 shutdown_connections 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15800673f0 0x7f1580067870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 --2- 172.21.15.73:0/4230605782 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f15801566e0 0x7f15800c9870 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 -- 172.21.15.73:0/4230605782 >> 172.21.15.73:0/4230605782 conn(0x7f15800631d0 msgr2=0x7f15800c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:31.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 -- 172.21.15.73:0/4230605782 shutdown_connections 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.194+0000 7f1586130700 1 -- 172.21.15.73:0/4230605782 wait complete. 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.195+0000 7f1586130700 1 Processor -- start 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.195+0000 7f1586130700 1 -- start start 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158013bd10 0x7f15800ab780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15801566e0 0x7f1580138cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800abc70 con 0x7f15800673f0 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15800ae070 con 0x7f158013bd10 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1586130700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15800ae350 con 0x7f15801566e0 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40224/0 (socket says 172.21.15.73:40224) 2023-11-28T04:08:31.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 -- 172.21.15.73:0/3130576140 learned_addr learned my addr 172.21.15.73:0/3130576140 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f1577fff700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158013bd10 0x7f15800ab780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157ffff700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15801566e0 0x7f1580138cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 -- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15801566e0 msgr2=0x7f1580138cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15801566e0 0x7f1580138cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 -- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158013bd10 msgr2=0x7f15800ab780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158013bd10 0x7f15800ab780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 -- 172.21.15.73:0/3130576140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1578009a20 con 0x7f15800673f0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f157f7fe700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f157000c4a0 tx=0x7f1570007af0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.196+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700072e0 con 0x7f15800673f0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.197+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1570007440 con 0x7f15800673f0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.197+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15680092a0 con 0x7f15800673f0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.197+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15700057a0 con 0x7f15800673f0 2023-11-28T04:08:31.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.197+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15800ae910 con 0x7f15800673f0 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f158006c860 con 0x7f15800673f0 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 74) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f1570005bd0 con 0x7f15800673f0 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f15777fe700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 0x7f1560061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f1577fff700 1 -- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 msgr2=0x7f1560061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f15700798a0 con 0x7f15800673f0 2023-11-28T04:08:31.205 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.198+0000 7f1577fff700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 0x7f1560061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:31.205 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:31.200+0000 7f5962fc9680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:08:31.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.202+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f157004ab70 con 0x7f15800673f0 2023-11-28T04:08:31.256 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:31.255+0000 7f81fe176680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:08:31.266 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:31.265+0000 7f5962fc9680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:08:31.377 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.376+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f15800be960 con 0x7f15800673f0 2023-11-28T04:08:31.380 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.380+0000 7f15777fe700 1 -- 172.21.15.73:0/3130576140 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f15800be960 con 0x7f15800673f0 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 msgr2=0x7f1560061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 0x7f1560061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 msgr2=0x7f15800ada80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 rx=0x7f157000c4a0 tx=0x7f1570007af0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 shutdown_connections 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f156005f1a0 0x7f1560061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f15801566e0 0x7f1580138cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f15800673f0 0x7f15800ada80 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 --2- 172.21.15.73:0/3130576140 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f158013bd10 0x7f15800ab780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 >> 172.21.15.73:0/3130576140 conn(0x7f15800631d0 msgr2=0x7f15800a5eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 shutdown_connections 2023-11-28T04:08:31.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.381+0000 7f1586130700 1 -- 172.21.15.73:0/3130576140 wait complete. 2023-11-28T04:08:31.395 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:08:31.395 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T04:08:31.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:08:31.413 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-11-28T04:08:31.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-28T04:08:31.532 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:31.532+0000 7f81fe176680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:08:31.553 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:31.552+0000 7f5962fc9680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:08:31.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.631+0000 7f572fb3e700 1 Processor -- start 2023-11-28T04:08:31.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.631+0000 7f572fb3e700 1 -- start start 2023-11-28T04:08:31.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572fb3e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f572813ab60 0x7f5728067780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572fb3e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57281556f0 0x7f57280bd8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572fb3e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57280bddb0 con 0x7f572813af40 2023-11-28T04:08:31.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572fb3e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5728067c70 con 0x7f572813ab60 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572fb3e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57280c3510 con 0x7f57281556f0 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572d8da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f572813ab60 0x7f5728067780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572d0d9700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f572813af40 0x7f5728154fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:59796/0 (socket says 172.21.15.73:59796) 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.632+0000 7f572d0d9700 1 -- 172.21.15.73:0/2764187201 learned_addr learned my addr 172.21.15.73:0/2764187201 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572e0db700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57281556f0 0x7f57280bd8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572d8da700 1 -- 172.21.15.73:0/2764187201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57281556f0 msgr2=0x7f57280bd8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572d8da700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57281556f0 0x7f57280bd8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572d8da700 1 -- 172.21.15.73:0/2764187201 >> v1:172.21.15.73:6789/0 conn(0x7f572813af40 legacy=0x7f5728154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572c8d8700 1 -- 172.21.15.73:0/2764187201 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3693281273 0 0) 0x7f57280bddb0 con 0x7f572813af40 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572d8da700 1 -- 172.21.15.73:0/2764187201 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57280bdfe0 con 0x7f572813ab60 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572d8da700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728067780 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f5718009760 tx=0x7f5718009060).ready entity=mon.0 client_cookie=d2638093bb924f14 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572c8d8700 1 -- 172.21.15.73:0/2764187201 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800f510 con 0x7f572813ab60 2023-11-28T04:08:31.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572c8d8700 1 -- 172.21.15.73:0/2764187201 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571800f670 con 0x7f572813ab60 2023-11-28T04:08:31.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.633+0000 7f572c8d8700 1 -- 172.21.15.73:0/2764187201 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800f9f0 con 0x7f572813ab60 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 -- 172.21.15.73:0/2764187201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 msgr2=0x7f5728067780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728067780 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 rx=0x7f5718009760 tx=0x7f5718009060).stop 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 -- 172.21.15.73:0/2764187201 shutdown_connections 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f57281556f0 0x7f57280bd8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2764187201 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728067780 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 -- 172.21.15.73:0/2764187201 >> 172.21.15.73:0/2764187201 conn(0x7f5728063810 msgr2=0x7f5728063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 -- 172.21.15.73:0/2764187201 shutdown_connections 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.634+0000 7f572fb3e700 1 -- 172.21.15.73:0/2764187201 wait complete. 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.636+0000 7f572fb3e700 1 Processor -- start 2023-11-28T04:08:31.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.636+0000 7f572fb3e700 1 -- start start 2023-11-28T04:08:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f572813af40 0x7f57280a9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57281556f0 0x7f57280aa210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280aa700 con 0x7f572813ab60 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57280ac080 con 0x7f57281556f0 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572fb3e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5728151380 con 0x7f572813af40 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40242/0 (socket says 172.21.15.73:40242) 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 -- 172.21.15.73:0/2807477380 learned_addr learned my addr 172.21.15.73:0/2807477380 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f572813af40 0x7f57280a9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572e0db700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57281556f0 0x7f57280aa210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f572813af40 msgr2=0x7f57280a9d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f572813af40 0x7f57280a9d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57281556f0 msgr2=0x7f57280aa210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:31.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57281556f0 0x7f57280aa210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57180092a0 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.637+0000 7f572d8da700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f571800f610 tx=0x7f5718017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.638+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571800fe80 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.638+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f571800f7e0 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.638+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57280ab470 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.638+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f571801b640 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.638+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57280aba30 con 0x7f572813ab60 2023-11-28T04:08:31.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.639+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f572806c860 con 0x7f572813ab60 2023-11-28T04:08:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.640+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 74) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f571801ba70 con 0x7f572813ab60 2023-11-28T04:08:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.640+0000 7f571dffb700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.640+0000 7f572d0d9700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.640+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f571807bd00 con 0x7f572813ab60 2023-11-28T04:08:31.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.640+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:08:31.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.644+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f571804cfd0 con 0x7f572813ab60 2023-11-28T04:08:31.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.838+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5728067560 con 0x7f570c05f1a0 2023-11-28T04:08:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.841+0000 7f572d0d9700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:31.841+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:08:32.163 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:32.153+0000 7f81fe176680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:08:32.178 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:32.177+0000 7f5962fc9680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:08:32.227 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:32.226+0000 7f81fe176680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:08:32.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:32.242+0000 7f572d0d9700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:32.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:32.242+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T04:08:32.251 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:32.250+0000 7f5962fc9680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:08:32.414 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:32.413+0000 7f81fe176680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:08:32.438 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:32.436+0000 7f5962fc9680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:08:32.837 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:32.836+0000 7f81fe176680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:08:32.856 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:32.855+0000 7f5962fc9680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:08:32.940 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:32.939+0000 7f81fe176680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:32.959 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:32.958+0000 7f5962fc9680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:33.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:33.043+0000 7f572d0d9700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:33.044 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:33.043+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T04:08:33.094 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:33.093+0000 7f81fe176680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:33.112 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:33.111+0000 7f5962fc9680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:33.168 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:33.168+0000 7f81fe176680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:08:33.186 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:33.185+0000 7f5962fc9680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:08:33.236 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:33.235+0000 7f81fe176680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:08:33.254 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:33.252+0000 7f5962fc9680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:08:33.496 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:33.495+0000 7f81fe176680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:08:33.512 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:33.511+0000 7f5962fc9680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:08:33.680 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:33.679+0000 7f81fe176680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:33.699 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:33.694+0000 7f5962fc9680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:08:34.337 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.335+0000 7f81fe176680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:08:34.343 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.342+0000 7f5962fc9680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:08:34.405 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.404+0000 7f81fe176680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:08:34.412 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.411+0000 7f5962fc9680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:08:34.481 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.480+0000 7f81fe176680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:08:34.487 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.486+0000 7f5962fc9680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:08:34.627 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.626+0000 7f81fe176680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:08:34.635 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.634+0000 7f5962fc9680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:08:34.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:34.644+0000 7f572d0d9700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/90017 2023-11-28T04:08:34.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:34.644+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T04:08:34.699 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.697+0000 7f81fe176680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:08:34.707 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.706+0000 7f5962fc9680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:08:34.813 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.813+0000 7f81fe176680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:08:34.823 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.822+0000 7f5962fc9680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:08:34.921 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:34.921+0000 7f81fe176680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:34.931 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:34.930+0000 7f5962fc9680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:08:35.215 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:35.215+0000 7f81fe176680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:35.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.222+0000 7f5962fc9680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:08:35.287 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:08:35.286+0000 7f81fe176680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:35.294 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.293+0000 7f5962fc9680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:08:35.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:35.312+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 75) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f571801bd20 con 0x7f572813ab60 2023-11-28T04:08:35.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:35.313+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:08:35.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:35.313+0000 7f571dffb700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:08:35.382 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58da755700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.382 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58da755700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.382 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58da755700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.382 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58da755700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.382 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58da755700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.383 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58d774f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.383 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58d774f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.383 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58d774f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.383 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58d774f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:35.383 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:08:35.381+0000 7f58d774f700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:08:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.316+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 76) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5718079e50 con 0x7f572813ab60 2023-11-28T04:08:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.316+0000 7f571dffb700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.316+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5728067560 con 0x7f570c05f1a0 2023-11-28T04:08:36.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.317+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.318+0000 7f572d0d9700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c0615e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f5728165bf0 tx=0x7f5724007bb0).ready entity=mgr.8350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:36.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.334+0000 7f571dffb700 1 -- 172.21.15.73:0/2807477380 <== mgr.8350 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f5728067560 con 0x7f570c05f1a0 2023-11-28T04:08:36.338 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 msgr2=0x7f570c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c0615e0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 rx=0x7f5728165bf0 tx=0x7f5724007bb0).stop 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 msgr2=0x7f5728150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 rx=0x7f571800f610 tx=0x7f5718017000).stop 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 shutdown_connections 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f570c05f1a0 0x7f570c0615e0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f57281556f0 0x7f57280aa210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f572813ab60 0x7f5728150e90 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 --2- 172.21.15.73:0/2807477380 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f572813af40 0x7f57280a9d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 >> 172.21.15.73:0/2807477380 conn(0x7f5728063810 msgr2=0x7f5728137b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:36.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 shutdown_connections 2023-11-28T04:08:36.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.337+0000 7f572fb3e700 1 -- 172.21.15.73:0/2807477380 wait complete. 2023-11-28T04:08:36.352 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-11-28T04:08:36.352 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-28T04:08:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.516+0000 7f2af094c700 1 Processor -- start 2023-11-28T04:08:36.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.516+0000 7f2af094c700 1 -- start start 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2af094c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2aec1542d0 0x7f2aec0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2af094c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2aec0aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2af094c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec158d50 con 0x7f2aec1546b0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2af094c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec0c6b60 con 0x7f2aec1542d0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2af094c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0aa950 con 0x7f2aec0c66d0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aea59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2aec1546b0 0x7f2aec0c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:60144/0 (socket says 172.21.15.73:60144) 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aea59c700 1 -- 172.21.15.73:0/2629813504 learned_addr learned my addr 172.21.15.73:0/2629813504 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3809689442 0 0) 0x7f2aec158d50 con 0x7f2aec1546b0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aeb59e700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2aec0aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ad40033f0 con 0x7f2aec1546b0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aead9d700 1 --2- 172.21.15.73:0/2629813504 >> v2:172.21.15.150:3300/0 conn(0x7f2aec1542d0 0x7f2aec0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1582689294 0 0) 0x7f2ad40033f0 con 0x7f2aec1546b0 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 msgr2=0x7f2aec0aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2aec0aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 >> v2:172.21.15.150:3300/0 conn(0x7f2aec1542d0 msgr2=0x7f2aec0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 --2- 172.21.15.73:0/2629813504 >> v2:172.21.15.150:3300/0 conn(0x7f2aec1542d0 0x7f2aec0a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec158140 con 0x7f2aec1546b0 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2ae0001470 con 0x7f2aec1546b0 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 >> v1:172.21.15.73:6789/0 conn(0x7f2aec1546b0 legacy=0x7f2aec0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2ad40038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2ad4004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4004b80 0x7f2ad4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec0aa950 con 0x7f2ad4003ec0 2023-11-28T04:08:36.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad400b910 con 0x7f2ad4004b80 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ad400bad0 con 0x7f2aec0c66d0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aea59c700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2ad4004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.517+0000 7f2aead9d700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2ad40038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aeb59e700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4004b80 0x7f2ad4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 -- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 msgr2=0x7f2ad40038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec0c66d0 0x7f2ad40038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 -- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4004b80 msgr2=0x7f2ad4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ad4004b80 0x7f2ad4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 -- 172.21.15.73:0/2629813504 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ad40033f0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aea59c700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2ad4004690 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f2ae000c210 tx=0x7f2ae0001e90).ready entity=mon.0 client_cookie=c2fa5389751edabc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae0007a60 con 0x7f2ad4003ec0 2023-11-28T04:08:36.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae000cae0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2ae9d9b700 1 -- 172.21.15.73:0/2629813504 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae000ce80 con 0x7f2ad4003ec0 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 -- 172.21.15.73:0/2629813504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 msgr2=0x7f2ad4004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 --2- 172.21.15.73:0/2629813504 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2ad4004690 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 rx=0x7f2ae000c210 tx=0x7f2ae0001e90).stop 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2aeb59e700 1 -- 172.21.15.73:0/2629813504 reap_dead start 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 -- 172.21.15.73:0/2629813504 shutdown_connections 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 -- 172.21.15.73:0/2629813504 >> 172.21.15.73:0/2629813504 conn(0x7f2aec0cb010 msgr2=0x7f2aec0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 -- 172.21.15.73:0/2629813504 shutdown_connections 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.518+0000 7f2af094c700 1 -- 172.21.15.73:0/2629813504 wait complete. 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 Processor -- start 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 -- start start 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.526 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec157130 0x7f2aec1575b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 0x7f2aec0c99a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec156c10 con 0x7f2ad4003ec0 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec167a40 con 0x7f2aec0c9540 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2af094c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec1682e0 con 0x7f2aec157130 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.520+0000 7f2aead9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:60576/0 (socket says 172.21.15.73:60576) 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 -- 172.21.15.73:0/3944916562 learned_addr learned my addr 172.21.15.73:0/3944916562 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aea59c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec157130 0x7f2aec1575b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aeb59e700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 0x7f2aec0c99a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 -- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec157130 msgr2=0x7f2aec1575b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec157130 0x7f2aec1575b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.527 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 -- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 msgr2=0x7f2aec0c99a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 0x7f2aec0c99a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 -- 172.21.15.73:0/3944916562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0c9e90 con 0x7f2ad4003ec0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aeb59e700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 0x7f2aec0c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2aead9d700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f2adc0109e0 tx=0x7f2adc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc010ed0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc00e8d0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec157c10 con 0x7f2ad4003ec0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc00ec90 con 0x7f2ad4003ec0 2023-11-28T04:08:36.528 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.521+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec167f70 con 0x7f2ad4003ec0 2023-11-28T04:08:36.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.522+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec06c7c0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.522+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 76) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2adc014680 con 0x7f2ad4003ec0 2023-11-28T04:08:36.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.523+0000 7f2ad2ffd700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:08:36.529 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.523+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2adc07a250 con 0x7f2ad4003ec0 2023-11-28T04:08:36.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.523+0000 7f2aea59c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:08:36.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.523+0000 7f2aea59c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 0x7f2ad8061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2ae0001ed0 tx=0x7f2ae0015000).ready entity=mgr.8350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:08:36.530 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.527+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2adc04b5e0 con 0x7f2ad4003ec0 2023-11-28T04:08:36.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.717+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2aec168420 con 0x7f2ad805f080 2023-11-28T04:08:36.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.722+0000 7f2ad2ffd700 1 -- 172.21.15.73:0/3944916562 <== mgr.8350 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f2ae000cc50 con 0x7f2ad805f080 2023-11-28T04:08:36.723 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-11-28T04:08:36.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 msgr2=0x7f2ad8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 0x7f2ad8061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f2ae0001ed0 tx=0x7f2ae0015000).stop 2023-11-28T04:08:36.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 msgr2=0x7f2aec14c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:08:36.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 rx=0x7f2adc0109e0 tx=0x7f2adc008040).stop 2023-11-28T04:08:36.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 shutdown_connections 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2aec0c9540 0x7f2aec0c99a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ad4003ec0 0x7f2aec14c480 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 --2- 172.21.15.73:0/3944916562 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2aec157130 0x7f2aec1575b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 >> 172.21.15.73:0/3944916562 conn(0x7f2aec0cb010 msgr2=0x7f2aec14ee70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:08:36.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 shutdown_connections 2023-11-28T04:08:36.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:08:36.724+0000 7f2af094c700 1 -- 172.21.15.73:0/3944916562 wait complete. 2023-11-28T04:08:36.739 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-11-28T04:08:42.811 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:08:42.814 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:09:00.728 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:09:00.727+0000 7f58c9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:09:00.728 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:09:00.727+0000 7f58c9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:09:00.728 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:09:00.727+0000 7f58c9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:09:00.728 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:09:00.727+0000 7f58c9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:09:00.728 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:09:00.727+0000 7f58c9733700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:09:12.973 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:09:12.976 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:09:43.120 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:09:43.124 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:10:13.196 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:10:13.199 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:10:43.351 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:10:43.355 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:11:13.477 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:11:13.481 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:11:43.588 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:11:43.592 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:12:13.713 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:12:13.720 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:12:43.858 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:12:43.861 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:13:14.013 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:13:14.016 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:13:44.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:13:44.170 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:14:14.271 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:14:16.105 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:14:46.243 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:14:46.247 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:15:16.390 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:15:16.394 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:15:46.469 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:15:46.472 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:16:16.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:16:16.611 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:16:46.772 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:16:46.775 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:17:17.077 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:17:17.180 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:17:47.234 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:17:47.237 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:18:17.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:18:17.368 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:18:47.511 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:18:47.515 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:19:15.740 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:19:15.740 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-28T04:19:15.760 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_09_00_UTC 2023-11-28T04:19:15.760 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_10_00_UTC 2023-11-28T04:19:15.760 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_11_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_12_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_13_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_14_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_15_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_16_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_17_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_18_00_UTC 2023-11-28T04:19:15.761 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_19_00_UTC 2023-11-28T04:19:15.761 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-04_09_00_UTC', 'scheduled-2023-11-28-04_10_00_UTC', 'scheduled-2023-11-28-04_11_00_UTC', 'scheduled-2023-11-28-04_12_00_UTC', 'scheduled-2023-11-28-04_13_00_UTC', 'scheduled-2023-11-28-04_14_00_UTC', 'scheduled-2023-11-28-04_15_00_UTC', 'scheduled-2023-11-28-04_16_00_UTC', 'scheduled-2023-11-28-04_17_00_UTC', 'scheduled-2023-11-28-04_18_00_UTC', 'scheduled-2023-11-28-04_19_00_UTC'] 2023-11-28T04:19:15.762 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-11-28T04:19:15.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.925+0000 7f404372e700 1 Processor -- start 2023-11-28T04:19:15.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.925+0000 7f404372e700 1 -- start start 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f404372e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f403c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f404372e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f403c0aa220 0x7f403c0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f404372e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f403c0c6630 con 0x7f403c154530 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f404372e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f403c0c6390 con 0x7f403c0aa220 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f404372e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a84e0 con 0x7f403c154910 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f40414ca700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f403c154530 0x7f403c0c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37446/0 (socket says 172.21.15.73:37446) 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.926+0000 7f40414ca700 1 -- 172.21.15.73:0/4031657026 learned_addr learned my addr 172.21.15.73:0/4031657026 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f4040cc9700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f403c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f4041ccb700 1 --2- 172.21.15.73:0/4031657026 >> v2:172.21.15.150:3300/0 conn(0x7f403c0aa220 0x7f403c0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2560138940 0 0) 0x7f403c0c6630 con 0x7f403c154530 2023-11-28T04:19:15.931 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40240033f0 con 0x7f403c154530 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3485305028 0 0) 0x7f40240033f0 con 0x7f403c154530 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 msgr2=0x7f403c0a7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f403c0a7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 >> v2:172.21.15.150:3300/0 conn(0x7f403c0aa220 msgr2=0x7f403c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 --2- 172.21.15.73:0/4031657026 >> v2:172.21.15.150:3300/0 conn(0x7f403c0aa220 0x7f403c0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0aaba0 con 0x7f403c154530 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f402c001970 con 0x7f403c154530 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 >> v1:172.21.15.73:6789/0 conn(0x7f403c154530 legacy=0x7f403c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40240038e0 0x7f4024003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f4024004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0c6390 con 0x7f40240038e0 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0a84e0 con 0x7f4024004b80 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f402400b8e0 con 0x7f403c154910 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40240038e0 0x7f4024003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f4040cc9700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f4024004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f4041ccb700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 -- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 msgr2=0x7f4024004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c154910 0x7f4024004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 -- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 msgr2=0x7f4024005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 -- 172.21.15.73:0/4031657026 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40240033f0 con 0x7f40240038e0 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f4041ccb700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4024004b80 0x7f4024005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:19:15.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.927+0000 7f40414ca700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40240038e0 0x7f4024003d00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7f402c00c5f0 tx=0x7f402c007b20).ready entity=mon.0 client_cookie=5e08b34622424cef server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c007d50 con 0x7f40240038e0 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f402c0059f0 con 0x7f40240038e0 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f403bfff700 1 -- 172.21.15.73:0/4031657026 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f402c005d70 con 0x7f40240038e0 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 -- 172.21.15.73:0/4031657026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40240038e0 msgr2=0x7f4024003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 --2- 172.21.15.73:0/4031657026 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f40240038e0 0x7f4024003d00 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 rx=0x7f402c00c5f0 tx=0x7f402c007b20).stop 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f4041ccb700 1 -- 172.21.15.73:0/4031657026 reap_dead start 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 -- 172.21.15.73:0/4031657026 shutdown_connections 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 -- 172.21.15.73:0/4031657026 >> 172.21.15.73:0/4031657026 conn(0x7f403c094930 msgr2=0x7f403c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 -- 172.21.15.73:0/4031657026 shutdown_connections 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.928+0000 7f404372e700 1 -- 172.21.15.73:0/4031657026 wait complete. 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.929+0000 7f404372e700 1 Processor -- start 2023-11-28T04:19:15.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.929+0000 7f404372e700 1 -- start start 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40240038e0 0x7f403c150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0ab8d0 0x7f403c0abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0c50f0 con 0x7f403c0c3790 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f403c0ac240 con 0x7f40240038e0 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f404372e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f403c150880 con 0x7f403c0ab8d0 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f4041ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f4041ccb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40906/0 (socket says 172.21.15.73:40906) 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f4041ccb700 1 -- 172.21.15.73:0/3434661314 learned_addr learned my addr 172.21.15.73:0/3434661314 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:19:15.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f4040cc9700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0ab8d0 0x7f403c0abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.930+0000 7f40414ca700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40240038e0 0x7f403c150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 -- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0ab8d0 msgr2=0x7f403c0abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0ab8d0 0x7f403c0abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 -- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40240038e0 msgr2=0x7f403c150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40240038e0 0x7f403c150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 -- 172.21.15.73:0/3434661314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f403c0c44e0 con 0x7f403c0c3790 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f4041ccb700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f403c0856c0 tx=0x7f4034008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034015480 con 0x7f403c0c3790 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40340155e0 con 0x7f403c0c3790 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f403c0c47c0 con 0x7f403c0c3790 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4034015980 con 0x7f403c0c3790 2023-11-28T04:19:15.938 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.931+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f403c0c4d80 con 0x7f403c0c3790 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.932+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f403c06c860 con 0x7f403c0c3790 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.932+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 79) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f403401f460 con 0x7f403c0c3790 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.933+0000 7f40397fa700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 0x7f4028061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.933+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6380+0+0 (secure 0 0 0) 0x7f403407a650 con 0x7f403c0c3790 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.933+0000 7f40414ca700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 0x7f4028061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.933+0000 7f40414ca700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 0x7f4028061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f402c001270 tx=0x7f402c015000).ready entity=mgr.8350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:15.939 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:15.938+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f403404b640 con 0x7f403c0c3790 2023-11-28T04:19:16.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.121+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 --> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f403c05f760 con 0x7f402805f2c0 2023-11-28T04:19:16.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.123+0000 7f40397fa700 1 -- 172.21.15.73:0/3434661314 <== mgr.8350 v2:172.21.15.73:6832/90017 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f403c05f760 con 0x7f402805f2c0 2023-11-28T04:19:16.124 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:19:16.124 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T04:08:36", "first": "2023-11-28T04:09:00", "last": "2023-11-28T04:19:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-11-28T04:19:16.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.124+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 msgr2=0x7f4028061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.124+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 0x7f4028061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f402c001270 tx=0x7f402c015000).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 msgr2=0x7f403c0c3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 rx=0x7f403c0856c0 tx=0x7f4034008040).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 shutdown_connections 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7f402805f2c0 0x7f4028061780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f403c0c3790 0x7f403c0c3c10 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f40240038e0 0x7f403c150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 --2- 172.21.15.73:0/3434661314 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f403c0ab8d0 0x7f403c0abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 >> 172.21.15.73:0/3434661314 conn(0x7f403c094930 msgr2=0x7f403c0ca9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:19:16.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 shutdown_connections 2023-11-28T04:19:16.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.125+0000 7f404372e700 1 -- 172.21.15.73:0/3434661314 wait complete. 2023-11-28T04:19:16.137 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T04:08:36", "first": "2023-11-28T04:09:00", "last": "2023-11-28T04:19:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-11-28T04:19:16.137 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T04:08:36', 'first': '2023-11-28T04:09:00', 'last': '2023-11-28T04:19:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-11-28T04:19:16.137 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-11-28T04:19:16.137 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-11-28T04:19:16.138 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-11-28T04:19:16.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 Processor -- start 2023-11-28T04:19:16.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 -- start start 2023-11-28T04:19:16.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8f00c32b0 0x7fa8f015bea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:16.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0066e60 con 0x7fa8f013a8d0 2023-11-28T04:19:16.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0067140 con 0x7fa8f0066880 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f6796700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f0067420 con 0x7fa8f00c32b0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5f95700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8f00c32b0 0x7fa8f015bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5794700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fa8f013a8d0 0x7fa8f0155510 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:37458/0 (socket says 172.21.15.73:37458) 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5f95700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa8f00c32b0 0x7fa8f015bea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40916/0 (socket says 172.21.15.73:40916) 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5794700 1 -- 172.21.15.73:0/2886713352 learned_addr learned my addr 172.21.15.73:0/2886713352 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5f95700 1 -- 172.21.15.73:0/2886713352 >> v1:172.21.15.150:6789/0 conn(0x7fa8f0066880 legacy=0x7fa8f00694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5f95700 1 -- 172.21.15.73:0/2886713352 >> v1:172.21.15.73:6789/0 conn(0x7fa8f013a8d0 legacy=0x7fa8f0155510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8e7fff700 1 -- 172.21.15.73:0/2886713352 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 858712069 0 0) 0x7fa8f0066e60 con 0x7fa8f013a8d0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.251+0000 7fa8f5f95700 1 -- 172.21.15.73:0/2886713352 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f0067700 con 0x7fa8f00c32b0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f5f95700 1 --2- 172.21.15.73:0/2886713352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f015bea0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fa8e8009e60 tx=0x7fa8e8009780).ready entity=mon.0 client_cookie=f13dd08cdd9a3959 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8e7fff700 1 -- 172.21.15.73:0/2886713352 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e800e070 con 0x7fa8f00c32b0 2023-11-28T04:19:16.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8e7fff700 1 -- 172.21.15.73:0/2886713352 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e800bbf0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8e7fff700 1 -- 172.21.15.73:0/2886713352 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e8012410 con 0x7fa8f00c32b0 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 -- 172.21.15.73:0/2886713352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 msgr2=0x7fa8f015bea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2886713352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f015bea0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 rx=0x7fa8e8009e60 tx=0x7fa8e8009780).stop 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 -- 172.21.15.73:0/2886713352 shutdown_connections 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2886713352 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f015bea0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 -- 172.21.15.73:0/2886713352 >> 172.21.15.73:0/2886713352 conn(0x7fa8f00c7ca0 msgr2=0x7fa8f00c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 -- 172.21.15.73:0/2886713352 shutdown_connections 2023-11-28T04:19:16.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.252+0000 7fa8f6796700 1 -- 172.21.15.73:0/2886713352 wait complete. 2023-11-28T04:19:16.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.253+0000 7fa8f6796700 1 Processor -- start 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.253+0000 7fa8f6796700 1 -- start start 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f0066880 0x7fa8f00a9db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 0x7fa8f0153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f01546d0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0153ac0 con 0x7fa8f013a8d0 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f6796700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f0153da0 con 0x7fa8f0066880 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f4f93700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f4f93700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40926/0 (socket says 172.21.15.73:40926) 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.254+0000 7fa8f4f93700 1 -- 172.21.15.73:0/2182900290 learned_addr learned my addr 172.21.15.73:0/2182900290 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 -- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f0066880 msgr2=0x7fa8f00a9db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f0066880 0x7fa8f00a9db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f5f95700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 0x7fa8f0153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 -- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 msgr2=0x7fa8f0153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 0x7fa8f0153220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 -- 172.21.15.73:0/2182900290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e8009a20 con 0x7fa8f00c32b0 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f5f95700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 0x7fa8f0153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:19:16.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f4f93700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fa8e000b490 tx=0x7fa8e0006f00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:16.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0002320 con 0x7fa8f00c32b0 2023-11-28T04:19:16.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e00024c0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f0154080 con 0x7fa8f00c32b0 2023-11-28T04:19:16.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0002880 con 0x7fa8f00c32b0 2023-11-28T04:19:16.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.255+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f0155050 con 0x7fa8f00c32b0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.256+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f00c6db0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.256+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 79) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fa8e0002bf0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.257+0000 7fa8e57fa700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 0x7fa8d4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.257+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6380+0+0 (secure 0 0 0) 0x7fa8e007a330 con 0x7fa8f00c32b0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.257+0000 7fa8f5794700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 0x7fa8d4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.257+0000 7fa8f5794700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 0x7fa8d4061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa8ec0011e0 tx=0x7fa8ec007850).ready entity=mgr.8350 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:19:16.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.261+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa8e004b380 con 0x7fa8f00c32b0 2023-11-28T04:19:16.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.468+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fa8f006c7c0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.470+0000 7fa8e57fa700 1 -- 172.21.15.73:0/2182900290 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v79) v1 ==== 74+0+149155 (secure 0 0 0) 0x7fa8e004e9a0 con 0x7fa8f00c32b0 2023-11-28T04:19:16.473 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:19:16.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 msgr2=0x7fa8d4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 0x7fa8d4061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fa8ec0011e0 tx=0x7fa8ec007850).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 msgr2=0x7fa8f0154b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 rx=0x7fa8e000b490 tx=0x7fa8e0006f00).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 shutdown_connections 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:6832/90017,v1:172.21.15.73:6833/90017] conn(0x7fa8d405f2c0 0x7fa8d4061780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa8f013a8d0 0x7fa8f0153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa8f0066880 0x7fa8f00a9db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 --2- 172.21.15.73:0/2182900290 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa8f00c32b0 0x7fa8f0154b60 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 >> 172.21.15.73:0/2182900290 conn(0x7fa8f00c7ca0 msgr2=0x7fa8f00aba90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 shutdown_connections 2023-11-28T04:19:16.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:19:16.473+0000 7fa8f6796700 1 -- 172.21.15.73:0/2182900290 wait complete. 2023-11-28T04:19:16.486 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":79,"active_gid":8350,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":90017},{"type":"v1","addr":"172.21.15.73:6833","nonce":90017}]},"active_addr":"172.21.15.73:6833/90017","active_change":"2023-11-28T04:08:35.307885+0000","active_mgr_features":4540138314316775423,"available":true,"standbys":[{"gid":8344,"name":"x","mgr_features":4540138314316775423,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.23.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:8.3.5","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/library/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"docker.io/arcts/keepalived","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.3.1","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.33.4","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":180,"active_clients":[{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":436795956}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":314665802}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":3165037877}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.73:0","nonce":1627000602}]}]} 2023-11-28T04:19:16.492 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-11-28T04:19:16.492 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-11-28T04:19:16.492 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-11-28T04:19:16.492 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T04:19:16.492 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:19:16.518 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T04:19:16.518 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-11-28T04:19:16.560 INFO:tasks.ceph.mgr.y:Started 2023-11-28T04:19:16.734 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:16.734+0000 7ff556ea9680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:19:16.796 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:16.796+0000 7ff556ea9680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:19:17.067 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:17.067+0000 7ff556ea9680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:19:17.650 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:19:17.653 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:19:17.681 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:17.676+0000 7ff556ea9680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:19:17.749 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:17.749+0000 7ff556ea9680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:19:17.933 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:17.933+0000 7ff556ea9680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:19:18.350 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:18.350+0000 7ff556ea9680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:19:18.453 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:18.453+0000 7ff556ea9680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:19:18.605 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:18.606+0000 7ff556ea9680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:19:18.679 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:18.679+0000 7ff556ea9680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:19:18.745 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:18.746+0000 7ff556ea9680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:19:19.001 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:19.001+0000 7ff556ea9680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:19:19.184 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:19.184+0000 7ff556ea9680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:19:19.834 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:19.834+0000 7ff556ea9680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:19:19.902 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:19.902+0000 7ff556ea9680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:19:19.978 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:19.978+0000 7ff556ea9680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:19:20.125 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.125+0000 7ff556ea9680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:19:20.195 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.196+0000 7ff556ea9680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:19:20.312 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.312+0000 7ff556ea9680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:19:20.419 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.419+0000 7ff556ea9680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:19:20.708 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.709+0000 7ff556ea9680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:19:20.781 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.781+0000 7ff556ea9680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:19:20.867 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cde34700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.867 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cde34700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.867 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cde34700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cde34700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cde34700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cae2e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cae2e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cae2e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cae2e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:20.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:19:20.867+0000 7ff4cae2e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:19:47.790 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:19:47.794 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:20:00.859 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:20:00.859+0000 7ff4d5e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:20:00.859 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:20:00.859+0000 7ff4d5e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:20:00.859 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:20:00.859+0000 7ff4d5e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:20:00.859 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:20:00.859+0000 7ff4d5e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:20:00.859 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:20:00.859+0000 7ff4d5e44700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:20:17.921 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:20:17.925 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:20:48.061 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:20:48.064 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:21:18.208 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:21:18.213 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:21:48.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:21:48.384 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:22:18.528 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:22:18.532 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:22:48.668 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:22:48.672 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:23:18.824 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:23:18.827 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:23:48.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:23:48.973 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:24:16.562 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-28T04:24:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.693+0000 7fbf03e4c700 1 Processor -- start 2023-11-28T04:24:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.693+0000 7fbf03e4c700 1 -- start start 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc154530 0x7fbefc154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbefc0a7ff0 0x7fbefc0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 0x7fbefc0aa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbefc0c6390 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbefc0aabc0 con 0x7fbefc0aa220 2023-11-28T04:24:16.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.694+0000 7fbf03e4c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc0c6630 con 0x7fbefc154530 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbefc0a7ff0 0x7fbefc0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fbefc0a7ff0 0x7fbefc0c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35324/0 (socket says 172.21.15.73:35324) 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 -- 172.21.15.73:0/750834152 learned_addr learned my addr 172.21.15.73:0/750834152 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf023e9700 1 --2- 172.21.15.73:0/750834152 >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 0x7fbefc0aa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf01be8700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc154530 0x7fbefc154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 -- 172.21.15.73:0/750834152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc154530 msgr2=0x7fbefc154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc154530 0x7fbefc154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 -- 172.21.15.73:0/750834152 >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 msgr2=0x7fbefc0aa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 --2- 172.21.15.73:0/750834152 >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 0x7fbefc0aa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 -- 172.21.15.73:0/750834152 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbefc0a8470 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf023e9700 1 --2- 172.21.15.73:0/750834152 >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 0x7fbefc0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf013e7700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0a7ff0 0x7fbefc0c8580 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7fbefc0a9fc0 tx=0x7fbeec0090f0).ready entity=mon.0 client_cookie=25e9471054cc64cf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf00be6700 1 -- 172.21.15.73:0/750834152 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00b600 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf00be6700 1 -- 172.21.15.73:0/750834152 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00b760 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.695+0000 7fbf00be6700 1 -- 172.21.15.73:0/750834152 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00bae0 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 -- 172.21.15.73:0/750834152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0a7ff0 msgr2=0x7fbefc0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0a7ff0 0x7fbefc0c8580 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 rx=0x7fbefc0a9fc0 tx=0x7fbeec0090f0).stop 2023-11-28T04:24:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 -- 172.21.15.73:0/750834152 shutdown_connections 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/750834152 >> v2:172.21.15.150:3300/0 conn(0x7fbefc0aa220 0x7fbefc0aa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc154530 0x7fbefc154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/750834152 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0a7ff0 0x7fbefc0c8580 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 -- 172.21.15.73:0/750834152 >> 172.21.15.73:0/750834152 conn(0x7fbefc094930 msgr2=0x7fbefc06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 -- 172.21.15.73:0/750834152 shutdown_connections 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.696+0000 7fbf03e4c700 1 -- 172.21.15.73:0/750834152 wait complete. 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.697+0000 7fbf03e4c700 1 Processor -- start 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.697+0000 7fbf03e4c700 1 -- start start 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc0a7ff0 0x7fbefc141a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbefc154530 0x7fbefc0b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc0aad60 con 0x7fbefc0aa220 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc0ab320 con 0x7fbefc154530 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbefc0afca0 con 0x7fbefc0a7ff0 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35332/0 (socket says 172.21.15.73:35332) 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 -- 172.21.15.73:0/2153742972 learned_addr learned my addr 172.21.15.73:0/2153742972 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf023e9700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbefc154530 0x7fbefc0b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf01be8700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc0a7ff0 0x7fbefc141a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 -- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc0a7ff0 msgr2=0x7fbefc141a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc0a7ff0 0x7fbefc141a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 -- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbefc154530 msgr2=0x7fbefc0b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbefc154530 0x7fbefc0b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 -- 172.21.15.73:0/2153742972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeec009370 con 0x7fbefc0aa220 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf013e7700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7fbeec006fd0 tx=0x7fbeec017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec00b8d0 con 0x7fbefc0aa220 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeec00ba30 con 0x7fbefc0aa220 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeec01b640 con 0x7fbefc0aa220 2023-11-28T04:24:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.698+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbefc142440 con 0x7fbefc0aa220 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.699+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbefc0b1050 con 0x7fbefc0aa220 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.700+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbefc0ab4b0 con 0x7fbefc0aa220 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.700+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fbeec01ba30 con 0x7fbefc0aa220 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.700+0000 7fbef1ffb700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 0x7fbee0061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.700+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fbeec07bfe0 con 0x7fbefc0aa220 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.700+0000 7fbf01be8700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 0x7fbee0061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:16.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.701+0000 7fbf01be8700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 0x7fbee0061780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbef8007470 tx=0x7fbef800b040).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:16.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.705+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fbeec04cf80 con 0x7fbefc0aa220 2023-11-28T04:24:16.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.896+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbefc06c860 con 0x7fbee005f2c0 2023-11-28T04:24:16.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.900+0000 7fbef1ffb700 1 -- 172.21.15.73:0/2153742972 <== mgr.9273 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fbefc06c860 con 0x7fbee005f2c0 2023-11-28T04:24:16.901 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-11-28T04:24:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 msgr2=0x7fbee0061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 0x7fbee0061780 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7fbef8007470 tx=0x7fbef800b040).stop 2023-11-28T04:24:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 msgr2=0x7fbefc0b0200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:16.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 rx=0x7fbeec006fd0 tx=0x7fbeec017000).stop 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 shutdown_connections 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fbee005f2c0 0x7fbee0061780 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fbefc154530 0x7fbefc0b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fbefc0a7ff0 0x7fbefc141a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 --2- 172.21.15.73:0/2153742972 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fbefc0aa220 0x7fbefc0b0200 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 >> 172.21.15.73:0/2153742972 conn(0x7fbefc094930 msgr2=0x7fbefc0c9980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 shutdown_connections 2023-11-28T04:24:16.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:16.902+0000 7fbf03e4c700 1 -- 172.21.15.73:0/2153742972 wait complete. 2023-11-28T04:24:16.915 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-11-28T04:24:16.915 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:16.915 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_09_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_10_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_11_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_12_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_13_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_14_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_15_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_16_00_UTC 2023-11-28T04:24:16.932 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_17_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_18_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_19_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_20_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_21_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_22_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_23_00_UTC 2023-11-28T04:24:16.933 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_24_00_UTC 2023-11-28T04:24:16.934 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-11-28-04_09_00_UTC', 'scheduled-2023-11-28-04_10_00_UTC', 'scheduled-2023-11-28-04_11_00_UTC', 'scheduled-2023-11-28-04_12_00_UTC', 'scheduled-2023-11-28-04_13_00_UTC', 'scheduled-2023-11-28-04_14_00_UTC', 'scheduled-2023-11-28-04_15_00_UTC', 'scheduled-2023-11-28-04_16_00_UTC', 'scheduled-2023-11-28-04_17_00_UTC', 'scheduled-2023-11-28-04_18_00_UTC', 'scheduled-2023-11-28-04_19_00_UTC', 'scheduled-2023-11-28-04_20_00_UTC', 'scheduled-2023-11-28-04_21_00_UTC', 'scheduled-2023-11-28-04_22_00_UTC', 'scheduled-2023-11-28-04_23_00_UTC', 'scheduled-2023-11-28-04_24_00_UTC'] 2023-11-28T04:24:16.934 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --fs cephfs --format json /snap_test_dir1/test_restart 2023-11-28T04:24:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.087+0000 7f1081823700 1 Processor -- start 2023-11-28T04:24:17.088 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.087+0000 7f1081823700 1 -- start start 2023-11-28T04:24:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f1081823700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f107c154530 0x7f107c154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f1081823700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c0c6630 con 0x7f107c0aa220 2023-11-28T04:24:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f1081823700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f107c0a8470 con 0x7f107c0a7ff0 2023-11-28T04:24:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f1081823700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f107c0c6390 con 0x7f107c154530 2023-11-28T04:24:17.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f107b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f107c0aa220 0x7f107c0c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36920/0 (socket says 172.21.15.73:36920) 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f107b7fe700 1 -- 172.21.15.73:0/3049896648 learned_addr learned my addr 172.21.15.73:0/3049896648 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f107affd700 1 --2- 172.21.15.73:0/3049896648 >> v2:172.21.15.150:3300/0 conn(0x7f107c154530 0x7f107c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.088+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2601762940 0 0) 0x7f107c0c6630 con 0x7f107c0aa220 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10680033f0 con 0x7f107c0aa220 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4002762255 0 0) 0x7f10680033f0 con 0x7f107c0aa220 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 >> v2:172.21.15.150:3300/0 conn(0x7f107c154530 msgr2=0x7f107c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 --2- 172.21.15.73:0/3049896648 >> v2:172.21.15.150:3300/0 conn(0x7f107c154530 0x7f107c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 >> v1:172.21.15.150:6789/0 conn(0x7f107c0a7ff0 legacy=0x7f107c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c064230 con 0x7f107c0aa220 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f10740017f0 con 0x7f107c0aa220 2023-11-28T04:24:17.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 >> v1:172.21.15.73:6789/0 conn(0x7f107c0aa220 legacy=0x7f107c0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10680038e0 0x7f1068003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f1068004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1068005000 0x7f106800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0c6390 con 0x7f1068004690 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f106800bf60 con 0x7f10680038e0 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f106800c0f0 con 0x7f1068005000 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f1068004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107affd700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10680038e0 0x7f1068003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107b7fe700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1068005000 0x7f106800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 -- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1068005000 msgr2=0x7f106800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1068005000 0x7f106800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 -- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10680038e0 msgr2=0x7f1068003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f10680038e0 0x7f1068003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 -- 172.21.15.73:0/3049896648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10680033f0 con 0x7f1068004690 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107b7fe700 1 -- 172.21.15.73:0/3049896648 reap_dead start 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f10727fc700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f1068004b10 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f1064001730 tx=0x7f1064007bb0).ready entity=mon.0 client_cookie=dac4d4a0f5b74a6e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007020 con 0x7f1068004690 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.089+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1064007180 con 0x7f1068004690 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f107a7fc700 1 -- 172.21.15.73:0/3049896648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1064007520 con 0x7f1068004690 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 -- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 msgr2=0x7f1068004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f1068004b10 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 rx=0x7f1064001730 tx=0x7f1064007bb0).stop 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 -- 172.21.15.73:0/3049896648 shutdown_connections 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 --2- 172.21.15.73:0/3049896648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f1068004b10 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 -- 172.21.15.73:0/3049896648 >> 172.21.15.73:0/3049896648 conn(0x7f107c094930 msgr2=0x7f107c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:17.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 -- 172.21.15.73:0/3049896648 shutdown_connections 2023-11-28T04:24:17.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.090+0000 7f1081823700 1 -- 172.21.15.73:0/3049896648 wait complete. 2023-11-28T04:24:17.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.091+0000 7f1081823700 1 Processor -- start 2023-11-28T04:24:17.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.091+0000 7f1081823700 1 -- start start 2023-11-28T04:24:17.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f107c0aa650 0x7f107c0acdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f107c0ac070 0x7f107c0ac4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0b04e0 con 0x7f1068004690 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0c23e0 con 0x7f107c0aa650 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f1081823700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f107c0ac9e0 con 0x7f107c0ac070 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f107affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f107affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35352/0 (socket says 172.21.15.73:35352) 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f107affd700 1 -- 172.21.15.73:0/133000749 learned_addr learned my addr 172.21.15.73:0/133000749 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f10727fc700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f107c0aa650 0x7f107c0acdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.092+0000 7f107b7fe700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f107c0ac070 0x7f107c0ac4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 -- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f107c0ac070 msgr2=0x7f107c0ac4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f107c0ac070 0x7f107c0ac4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 -- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f107c0aa650 msgr2=0x7f107c0acdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f107c0aa650 0x7f107c0acdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 -- 172.21.15.73:0/133000749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f107c0b0730 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f107affd700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7f106c011a50 tx=0x7f106c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c005a90 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f106c005bf0 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f106c006030 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f107c0b0a10 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.093+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f107c0ad2e0 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.094+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f107c06c860 con 0x7f1068004690 2023-11-28T04:24:17.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.094+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f106c006420 con 0x7f1068004690 2023-11-28T04:24:17.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.094+0000 7f10737fe700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 0x7f105c061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.095+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f106c07b0b0 con 0x7f1068004690 2023-11-28T04:24:17.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.095+0000 7f10727fc700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 0x7f105c061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.095+0000 7f10727fc700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 0x7f105c061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1064001010 tx=0x7f1064015000).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.098+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f106c04c050 con 0x7f1068004690 2023-11-28T04:24:17.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.286+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f107c0c2520 con 0x7f105c05f2c0 2023-11-28T04:24:17.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.288+0000 7f10737fe700 1 -- 172.21.15.73:0/133000749 <== mgr.9273 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f1064007400 con 0x7f105c05f2c0 2023-11-28T04:24:17.289 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:17.289 INFO:teuthology.orchestra.run.smithi073.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T04:08:36", "first": "2023-11-28T04:09:00", "last": "2023-11-28T04:24:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 msgr2=0x7f105c061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 0x7f105c061780 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f1064001010 tx=0x7f1064015000).stop 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 msgr2=0x7f107c1482b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 rx=0x7f106c011a50 tx=0x7f106c006f40).stop 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 shutdown_connections 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f105c05f2c0 0x7f105c061780 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f107c0ac070 0x7f107c0ac4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1068004690 0x7f107c1482b0 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 --2- 172.21.15.73:0/133000749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f107c0aa650 0x7f107c0acdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 >> 172.21.15.73:0/133000749 conn(0x7f107c094930 msgr2=0x7f107c148bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:17.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 shutdown_connections 2023-11-28T04:24:17.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.290+0000 7f1081823700 1 -- 172.21.15.73:0/133000749 wait complete. 2023-11-28T04:24:17.302 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-11-28T00:00:00", "created": "2023-11-28T04:08:36", "first": "2023-11-28T04:09:00", "last": "2023-11-28T04:24:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-11-28T04:24:17.302 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-11-28T00:00:00', 'created': '2023-11-28T04:08:36', 'first': '2023-11-28T04:09:00', 'last': '2023-11-28T04:24:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-11-28T04:24:17.302 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-11-28T04:24:17.303 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-11-28T04:24:17.303 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs /snap_test_dir1/test_restart 1M 2023-11-28T04:24:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.419+0000 7f44d8dee700 1 Processor -- start 2023-11-28T04:24:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.419+0000 7f44d8dee700 1 -- start start 2023-11-28T04:24:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d8dee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44d4151f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d8dee700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f44d414b700 0x7f44d40bbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d8dee700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44d41523f0 con 0x7f44d414b320 2023-11-28T04:24:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d8dee700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44d40c3040 con 0x7f44d414b700 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d8dee700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40bd140 con 0x7f44d4064db0 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f44d414b320 0x7f44d40646a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36922/0 (socket says 172.21.15.73:36922) 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d259c700 1 -- 172.21.15.73:0/483576356 learned_addr learned my addr 172.21.15.73:0/483576356 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44d4151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/483576356 >> v2:172.21.15.150:3300/0 conn(0x7f44d414b700 0x7f44d40bbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3416964528 0 0) 0x7f44d41523f0 con 0x7f44d414b320 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.420+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f44c00033f0 con 0x7f44d414b320 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 86179528 0 0) 0x7f44c00033f0 con 0x7f44d414b320 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 msgr2=0x7f44d4151f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44d4151f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 >> v2:172.21.15.150:3300/0 conn(0x7f44d414b700 msgr2=0x7f44d40bbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 --2- 172.21.15.73:0/483576356 >> v2:172.21.15.150:3300/0 conn(0x7f44d414b700 0x7f44d40bbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40bc2d0 con 0x7f44d414b320 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f44c4001470 con 0x7f44d414b320 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/483576356 >> v2:172.21.15.150:3300/0 conn(0x7f44d414b700 0x7f44d40bbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44d4151f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 >> v1:172.21.15.73:6789/0 conn(0x7f44d414b320 legacy=0x7f44d40646a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44c00038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44c0003dd0 0x7f44c0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44c0004b80 0x7f44c0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c3040 con 0x7f44c0003dd0 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40bd140 con 0x7f44c0004b80 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44c000b8e0 con 0x7f44d4064db0 2023-11-28T04:24:17.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44c0003dd0 0x7f44c0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d259c700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44c00038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44c0004b80 0x7f44c0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 -- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 msgr2=0x7f44c00038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44d4064db0 0x7f44c00038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 -- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44c0004b80 msgr2=0x7f44c0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44c0004b80 0x7f44c0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 -- 172.21.15.73:0/483576356 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44bc0092a0 con 0x7f44c0003dd0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44c0003dd0 0x7f44c0004690 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f44d40651b0 tx=0x7f44bc009060).ready entity=mon.0 client_cookie=229be64c47ea1cc4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc0095c0 con 0x7f44c0003dd0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44bc010920 con 0x7f44c0003dd0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.421+0000 7f44d1d9b700 1 -- 172.21.15.73:0/483576356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44bc010ca0 con 0x7f44c0003dd0 2023-11-28T04:24:17.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 -- 172.21.15.73:0/483576356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44c0003dd0 msgr2=0x7f44c0004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 --2- 172.21.15.73:0/483576356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44c0003dd0 0x7f44c0004690 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 rx=0x7f44d40651b0 tx=0x7f44bc009060).stop 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d2d9d700 1 -- 172.21.15.73:0/483576356 reap_dead start 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 -- 172.21.15.73:0/483576356 shutdown_connections 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 -- 172.21.15.73:0/483576356 >> 172.21.15.73:0/483576356 conn(0x7f44d4062700 msgr2=0x7f44d4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 -- 172.21.15.73:0/483576356 shutdown_connections 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.422+0000 7f44d8dee700 1 -- 172.21.15.73:0/483576356 wait complete. 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.423+0000 7f44d8dee700 1 Processor -- start 2023-11-28T04:24:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.423+0000 7f44d8dee700 1 -- start start 2023-11-28T04:24:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 0x7f44d413bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d4148f80 0x7f44d4149400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40a78e0 con 0x7f44d40c6bd0 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40c7540 con 0x7f44d4148f80 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d8dee700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44d40a6cd0 con 0x7f44c0003dd0 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35376/0 (socket says 172.21.15.73:35376) 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 -- 172.21.15.73:0/3656747142 learned_addr learned my addr 172.21.15.73:0/3656747142 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44c9d9b700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d4148f80 0x7f44d4149400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d259c700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 0x7f44d413bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 -- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 msgr2=0x7f44d413bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 0x7f44d413bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 -- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d4148f80 msgr2=0x7f44d4149400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d4148f80 0x7f44d4149400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 -- 172.21.15.73:0/3656747142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44d40a6fb0 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d259c700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 0x7f44d413bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.424+0000 7f44d2d9d700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7f44d414bb30 tx=0x7f44cc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.425+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00e040 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.425+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f44cc00f040 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.425+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f44cc00fe50 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.425+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44d40a7290 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.425+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44d413bfc0 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.426+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44d406c860 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.426+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7f44cc015660 con 0x7f44d40c6bd0 2023-11-28T04:24:17.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.426+0000 7f44caffd700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 0x7f44b4061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.426+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f44cc07b2a0 con 0x7f44d40c6bd0 2023-11-28T04:24:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.426+0000 7f44d259c700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 0x7f44b4061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.427+0000 7f44d259c700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 0x7f44b4061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f44c4007d80 tx=0x7f44c4009040).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:17.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.430+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f44cc04c240 con 0x7f44d40c6bd0 2023-11-28T04:24:17.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.614+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f44d40bc530 con 0x7f44b405f2c0 2023-11-28T04:24:17.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.618+0000 7f44caffd700 1 -- 172.21.15.73:0/3656747142 <== mgr.9273 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7f44d40bc530 con 0x7f44b405f2c0 2023-11-28T04:24:17.620 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-11-28T04:24:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 msgr2=0x7f44b4061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 0x7f44b4061780 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f44c4007d80 tx=0x7f44c4009040).stop 2023-11-28T04:24:17.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 msgr2=0x7f44d40c7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 rx=0x7f44d414bb30 tx=0x7f44cc00b040).stop 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 shutdown_connections 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44b405f2c0 0x7f44b4061780 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44d40c6bd0 0x7f44d40c7050 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44c0003dd0 0x7f44d413bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 --2- 172.21.15.73:0/3656747142 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44d4148f80 0x7f44d4149400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 >> 172.21.15.73:0/3656747142 conn(0x7f44d4062700 msgr2=0x7f44d40c07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 shutdown_connections 2023-11-28T04:24:17.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:17.620+0000 7f44d8dee700 1 -- 172.21.15.73:0/3656747142 wait complete. 2023-11-28T04:24:17.633 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-11-28T04:24:17.633 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.633 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-11-28T04:24:17.649 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_09_00_UTC 2023-11-28T04:24:17.649 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_10_00_UTC 2023-11-28T04:24:17.649 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_11_00_UTC 2023-11-28T04:24:17.649 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_12_00_UTC 2023-11-28T04:24:17.649 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_13_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_14_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_15_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_16_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_17_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_18_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_19_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_20_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_21_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_22_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_23_00_UTC 2023-11-28T04:24:17.650 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_24_00_UTC 2023-11-28T04:24:17.651 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_09_00_UTC 2023-11-28T04:24:17.651 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.651 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_09_00_UTC) 2023-11-28T04:24:17.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_10_00_UTC 2023-11-28T04:24:17.714 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.714 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_10_00_UTC) 2023-11-28T04:24:17.775 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_11_00_UTC 2023-11-28T04:24:17.775 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.775 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_11_00_UTC) 2023-11-28T04:24:17.835 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_12_00_UTC 2023-11-28T04:24:17.835 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.835 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_12_00_UTC) 2023-11-28T04:24:17.895 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_13_00_UTC 2023-11-28T04:24:17.895 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.895 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_13_00_UTC) 2023-11-28T04:24:17.954 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_14_00_UTC 2023-11-28T04:24:17.955 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:17.955 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_14_00_UTC) 2023-11-28T04:24:18.014 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_15_00_UTC 2023-11-28T04:24:18.015 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.015 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_15_00_UTC) 2023-11-28T04:24:18.075 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_16_00_UTC 2023-11-28T04:24:18.075 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.075 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_16_00_UTC) 2023-11-28T04:24:18.135 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_17_00_UTC 2023-11-28T04:24:18.135 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.135 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_17_00_UTC) 2023-11-28T04:24:18.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_18_00_UTC 2023-11-28T04:24:18.195 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.195 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_18_00_UTC) 2023-11-28T04:24:18.254 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_19_00_UTC 2023-11-28T04:24:18.254 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.254 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_19_00_UTC) 2023-11-28T04:24:18.314 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_20_00_UTC 2023-11-28T04:24:18.315 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.315 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_20_00_UTC) 2023-11-28T04:24:18.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_21_00_UTC 2023-11-28T04:24:18.375 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.375 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_21_00_UTC) 2023-11-28T04:24:18.434 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_22_00_UTC 2023-11-28T04:24:18.434 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.435 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_22_00_UTC) 2023-11-28T04:24:18.495 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_23_00_UTC 2023-11-28T04:24:18.495 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.495 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_23_00_UTC) 2023-11-28T04:24:18.554 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_24_00_UTC 2023-11-28T04:24:18.554 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.555 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-11-28-04_24_00_UTC) 2023-11-28T04:24:18.615 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:18.615 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-11-28T04:24:18.672 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T04:24:18.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.825+0000 7fd72c040700 1 Processor -- start 2023-11-28T04:24:18.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.825+0000 7fd72c040700 1 -- start start 2023-11-28T04:24:18.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72c040700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd724098d10 0x7fd7240c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72c040700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7240c6190 0x7fd724063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72c040700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7240c72d0 con 0x7fd724064350 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72c040700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd724154bc0 con 0x7fd7240c6190 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72c040700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7240c66c0 con 0x7fd724098d10 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd7295db700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7240c6190 0x7fd724063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd7295db700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd7240c6190 0x7fd724063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35386/0 (socket says 172.21.15.73:35386) 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd7295db700 1 -- 172.21.15.73:0/3034979391 learned_addr learned my addr 172.21.15.73:0/3034979391 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd72a5dd700 1 --1- 172.21.15.73:0/3034979391 >> v1:172.21.15.73:6789/0 conn(0x7fd724064350 0x7fd724158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36934/0 (socket says 172.21.15.73:36934) 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.826+0000 7fd729ddc700 1 --2- 172.21.15.73:0/3034979391 >> v2:172.21.15.150:3300/0 conn(0x7fd724098d10 0x7fd7240c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd7295db700 1 -- 172.21.15.73:0/3034979391 >> v2:172.21.15.150:3300/0 conn(0x7fd724098d10 msgr2=0x7fd7240c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:18.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd7295db700 1 --2- 172.21.15.73:0/3034979391 >> v2:172.21.15.150:3300/0 conn(0x7fd724098d10 0x7fd7240c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd7295db700 1 -- 172.21.15.73:0/3034979391 >> v1:172.21.15.73:6789/0 conn(0x7fd724064350 legacy=0x7fd724158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd7295db700 1 -- 172.21.15.73:0/3034979391 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7240c69a0 con 0x7fd7240c6190 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd7295db700 1 --2- 172.21.15.73:0/3034979391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724063e60 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fd724083f80 tx=0x7fd7140090f0).ready entity=mon.0 client_cookie=b0d0d1c677b7f970 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd728dda700 1 -- 172.21.15.73:0/3034979391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400b600 con 0x7fd7240c6190 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd728dda700 1 -- 172.21.15.73:0/3034979391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71400b760 con 0x7fd7240c6190 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd728dda700 1 -- 172.21.15.73:0/3034979391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71400bae0 con 0x7fd7240c6190 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 -- 172.21.15.73:0/3034979391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 msgr2=0x7fd724063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 --2- 172.21.15.73:0/3034979391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724063e60 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 rx=0x7fd724083f80 tx=0x7fd7140090f0).stop 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 -- 172.21.15.73:0/3034979391 shutdown_connections 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 --2- 172.21.15.73:0/3034979391 >> v2:172.21.15.150:3300/0 conn(0x7fd724098d10 0x7fd7240c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:18.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 --2- 172.21.15.73:0/3034979391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724063e60 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 -- 172.21.15.73:0/3034979391 >> 172.21.15.73:0/3034979391 conn(0x7fd7240cb310 msgr2=0x7fd7240cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 -- 172.21.15.73:0/3034979391 shutdown_connections 2023-11-28T04:24:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.827+0000 7fd72c040700 1 -- 172.21.15.73:0/3034979391 wait complete. 2023-11-28T04:24:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.828+0000 7fd72c040700 1 Processor -- start 2023-11-28T04:24:18.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 -- start start 2023-11-28T04:24:18.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd724064350 0x7fd7241572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 0x7fd724155d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240c5650 con 0x7fd7240c6190 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7240af180 con 0x7fd724064350 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72c040700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd72414b850 con 0x7fd724098d10 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72a5dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72a5dd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35390/0 (socket says 172.21.15.73:35390) 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd72a5dd700 1 -- 172.21.15.73:0/3322153243 learned_addr learned my addr 172.21.15.73:0/3322153243 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:18.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.829+0000 7fd729ddc700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd724064350 0x7fd7241572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd7295db700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 0x7fd724155d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 -- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 msgr2=0x7fd724155d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 0x7fd724155d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 -- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd724064350 msgr2=0x7fd7241572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd724064350 0x7fd7241572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 -- 172.21.15.73:0/3322153243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd714009370 con 0x7fd7240c6190 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd7295db700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 0x7fd724155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72a5dd700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fd71c00c7c0 tx=0x7fd71c0015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c005c70 con 0x7fd7240c6190 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd71c005dd0 con 0x7fd7240c6190 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd724157790 con 0x7fd7240c6190 2023-11-28T04:24:18.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd71c007320 con 0x7fd7240c6190 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.830+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7240aeb30 con 0x7fd7240c6190 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.831+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd72406c860 con 0x7fd7240c6190 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.831+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 84) v1 ==== 75577+0+0 (secure 0 0 0) 0x7fd71c007710 con 0x7fd7240c6190 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.831+0000 7fd719ffb700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 0x7fd708061780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.832+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fd71c07a4a0 con 0x7fd7240c6190 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.832+0000 7fd729ddc700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 0x7fd708061780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.832+0000 7fd729ddc700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 0x7fd708061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd72000bbc0 tx=0x7fd720008040).ready entity=mgr.9273 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:18.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:18.835+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd71c04b440 con 0x7fd7240c6190 2023-11-28T04:24:19.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.042+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd7240c6c80 con 0x7fd7240c6190 2023-11-28T04:24:19.101 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:24:19.104 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:24:19.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.671+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fd71c077f40 con 0x7fd7240c6190 2023-11-28T04:24:19.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.672+0000 7fd719ffb700 1 -- 172.21.15.73:0/3322153243 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v85) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd71c072b70 con 0x7fd7240c6190 2023-11-28T04:24:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 msgr2=0x7fd708061780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 0x7fd708061780 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fd72000bbc0 tx=0x7fd720008040).stop 2023-11-28T04:24:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 msgr2=0x7fd724156220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 rx=0x7fd71c00c7c0 tx=0x7fd71c0015e0).stop 2023-11-28T04:24:19.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 shutdown_connections 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd70805f2c0 0x7fd708061780 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd7240c6190 0x7fd724156220 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd724064350 0x7fd7241572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 --2- 172.21.15.73:0/3322153243 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd724098d10 0x7fd724155d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 >> 172.21.15.73:0/3322153243 conn(0x7fd7240cb310 msgr2=0x7fd72414c0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 shutdown_connections 2023-11-28T04:24:19.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.674+0000 7fd72c040700 1 -- 172.21.15.73:0/3322153243 wait complete. 2023-11-28T04:24:19.687 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:24:19.688 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:24:19.729 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:24:19.773 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:24:19.773 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T04:24:19.774 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:19.774 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T04:24:19.814 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:24:19.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:19.818 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T04:24:19.819 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:19.816+0000 7fb51f3a6680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:24:19.823 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:19.822+0000 7fec53d20680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:24:19.834 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:19.882 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:19.882+0000 7fb51f3a6680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:24:19.886 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:19.885+0000 7fec53d20680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:24:19.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.991+0000 7fc45baa5700 1 Processor -- start 2023-11-28T04:24:19.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.991+0000 7fc45baa5700 1 -- start start 2023-11-28T04:24:19.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45baa5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc454150890 0x7fc4540bbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45baa5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540c28c0 con 0x7fc45406a010 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45baa5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4541552e0 con 0x7fc4540bc320 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45baa5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc45406a4b0 con 0x7fc454150890 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc454150890 0x7fc4540bbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc454150890 0x7fc4540bbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35416/0 (socket says 172.21.15.73:35416) 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45a042700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc45406a010 0x7fc4541544f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36938/0 (socket says 172.21.15.73:36938) 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 -- 172.21.15.73:0/1981501944 learned_addr learned my addr 172.21.15.73:0/1981501944 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 -- 172.21.15.73:0/1981501944 >> v1:172.21.15.150:6789/0 conn(0x7fc4540bc320 legacy=0x7fc454069900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 -- 172.21.15.73:0/1981501944 >> v1:172.21.15.73:6789/0 conn(0x7fc45406a010 legacy=0x7fc4541544f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 -- 172.21.15.73:0/1981501944 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4540c1f90 con 0x7fc454150890 2023-11-28T04:24:19.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc45883f700 1 -- 172.21.15.73:0/1981501944 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1795195285 0 0) 0x7fc4540c28c0 con 0x7fc45406a010 2023-11-28T04:24:19.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.992+0000 7fc459841700 1 --2- 172.21.15.73:0/1981501944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc454150890 0x7fc4540bbe30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7fc45000bac0 tx=0x7fc45000b370).ready entity=mon.0 client_cookie=6090b744779e70f3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:19.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45883f700 1 -- 172.21.15.73:0/1981501944 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc45000dc40 con 0x7fc454150890 2023-11-28T04:24:19.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45883f700 1 -- 172.21.15.73:0/1981501944 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc45000dda0 con 0x7fc454150890 2023-11-28T04:24:19.995 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45883f700 1 -- 172.21.15.73:0/1981501944 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc450015600 con 0x7fc454150890 2023-11-28T04:24:19.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 -- 172.21.15.73:0/1981501944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc454150890 msgr2=0x7fc4540bbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 --2- 172.21.15.73:0/1981501944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc454150890 0x7fc4540bbe30 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 rx=0x7fc45000bac0 tx=0x7fc45000b370).stop 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 -- 172.21.15.73:0/1981501944 shutdown_connections 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 --2- 172.21.15.73:0/1981501944 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc454150890 0x7fc4540bbe30 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 -- 172.21.15.73:0/1981501944 >> 172.21.15.73:0/1981501944 conn(0x7fc4540631d0 msgr2=0x7fc4540c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 -- 172.21.15.73:0/1981501944 shutdown_connections 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.993+0000 7fc45baa5700 1 -- 172.21.15.73:0/1981501944 wait complete. 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.994+0000 7fc45baa5700 1 Processor -- start 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.994+0000 7fc45baa5700 1 -- start start 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 0x7fc4541601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:19.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc454150890 0x7fc454161340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454093420 con 0x7fc4540bc320 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4540bfe00 con 0x7fc45406a010 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45baa5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc454160830 con 0x7fc454150890 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc459040700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc459040700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35428/0 (socket says 172.21.15.73:35428) 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc459040700 1 -- 172.21.15.73:0/531996503 learned_addr learned my addr 172.21.15.73:0/531996503 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc459841700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 0x7fc4541601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.995+0000 7fc45a042700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc454150890 0x7fc454161340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:19.999 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc454150890 msgr2=0x7fc454161340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc454150890 0x7fc454161340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 msgr2=0x7fc4541601a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 0x7fc4541601a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 -- 172.21.15.73:0/531996503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc45000b6a0 con 0x7fc4540bc320 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459841700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 0x7fc4541601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc459040700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fc44400c490 tx=0x7fc444007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400f470 con 0x7fc4540bc320 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc44400f5d0 con 0x7fc4540bc320 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454160b10 con 0x7fc4540bc320 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44400f950 con 0x7fc4540bc320 2023-11-28T04:24:20.002 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.996+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4541610d0 con 0x7fc4540bc320 2023-11-28T04:24:20.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.997+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc45406c860 con 0x7fc4540bc320 2023-11-28T04:24:20.003 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.997+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fc44400fd40 con 0x7fc4540bc320 2023-11-28T04:24:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.998+0000 7fc449ffb700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 0x7fc438061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.998+0000 7fc459841700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 msgr2=0x7fc438061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.998+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7fc444079a60 con 0x7fc4540bc320 2023-11-28T04:24:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:19.998+0000 7fc459841700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 0x7fc438061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:20.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.002+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc444048af0 con 0x7fc4540bc320 2023-11-28T04:24:20.162 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:20.162+0000 7fec53d20680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:24:20.165 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:20.164+0000 7fb51f3a6680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:24:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.199+0000 7fc459841700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 msgr2=0x7fc438061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:20.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.199+0000 7fc459841700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 0x7fc438061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.210+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4540c1cd0 con 0x7fc4540bc320 2023-11-28T04:24:20.211 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.210+0000 7fc449ffb700 1 -- 172.21.15.73:0/531996503 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fc44404aab0 con 0x7fc4540bc320 2023-11-28T04:24:20.212 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:20.212 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 msgr2=0x7fc438061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 0x7fc438061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 msgr2=0x7fc4540bf910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 rx=0x7fc44400c490 tx=0x7fc444007db0).stop 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 shutdown_connections 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc43805f270 0x7fc438061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc454150890 0x7fc454161340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc45406a010 0x7fc4541601a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 --2- 172.21.15.73:0/531996503 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc4540bc320 0x7fc4540bf910 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 >> 172.21.15.73:0/531996503 conn(0x7fc4540631d0 msgr2=0x7fc454065250 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 shutdown_connections 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.213+0000 7fc45baa5700 1 -- 172.21.15.73:0/531996503 wait complete. 2023-11-28T04:24:20.215 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:24:20.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:20.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.382+0000 7f450d797700 1 Processor -- start 2023-11-28T04:24:20.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.382+0000 7f450d797700 1 -- start start 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f450d797700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f45080c0d10 0x7f45080c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f450d797700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45081546f0 0x7f4508158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f450d797700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4508063380 con 0x7f450806b510 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f450d797700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45080c19e0 con 0x7f45081546f0 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f450d797700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4508159060 con 0x7f45080c0d10 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f45077fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45081546f0 0x7f4508158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f45077fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45081546f0 0x7f4508158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35444/0 (socket says 172.21.15.73:35444) 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.383+0000 7f45077fe700 1 -- 172.21.15.73:0/2965133661 learned_addr learned my addr 172.21.15.73:0/2965133661 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f4506ffd700 1 --2- 172.21.15.73:0/2965133661 >> v2:172.21.15.150:3300/0 conn(0x7f45080c0d10 0x7f45080c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f45077fe700 1 -- 172.21.15.73:0/2965133661 >> v2:172.21.15.150:3300/0 conn(0x7f45080c0d10 msgr2=0x7f45080c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f45077fe700 1 --2- 172.21.15.73:0/2965133661 >> v2:172.21.15.150:3300/0 conn(0x7f45080c0d10 0x7f45080c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f45077fe700 1 -- 172.21.15.73:0/2965133661 >> v1:172.21.15.150:6789/0 conn(0x7f450806b510 legacy=0x7f4508153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f45077fe700 1 -- 172.21.15.73:0/2965133661 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4508062770 con 0x7f45081546f0 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f45077fe700 1 --2- 172.21.15.73:0/2965133661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f4508158b70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f4508064940 tx=0x7f4500009840).ready entity=mon.0 client_cookie=c5c146d0111e4376 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f4505ffb700 1 -- 172.21.15.73:0/2965133661 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000e070 con 0x7f45081546f0 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f4505ffb700 1 -- 172.21.15.73:0/2965133661 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f450000ba20 con 0x7f45081546f0 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.384+0000 7f4505ffb700 1 -- 172.21.15.73:0/2965133661 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f450000bda0 con 0x7f45081546f0 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 -- 172.21.15.73:0/2965133661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 msgr2=0x7f4508158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 --2- 172.21.15.73:0/2965133661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f4508158b70 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 rx=0x7f4508064940 tx=0x7f4500009840).stop 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 -- 172.21.15.73:0/2965133661 shutdown_connections 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 --2- 172.21.15.73:0/2965133661 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f4508158b70 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 --2- 172.21.15.73:0/2965133661 >> v2:172.21.15.150:3300/0 conn(0x7f45080c0d10 0x7f45080c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 -- 172.21.15.73:0/2965133661 >> 172.21.15.73:0/2965133661 conn(0x7f45080c7760 msgr2=0x7f45080c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 -- 172.21.15.73:0/2965133661 shutdown_connections 2023-11-28T04:24:20.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.385+0000 7f450d797700 1 -- 172.21.15.73:0/2965133661 wait complete. 2023-11-28T04:24:20.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.386+0000 7f450d797700 1 Processor -- start 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.386+0000 7f450d797700 1 -- start start 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f450806b510 0x7f4508136a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45080c0d10 0x7f450814bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f450813a290 con 0x7f45081546f0 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45081372c0 con 0x7f450806b510 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f450d797700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45081375a0 con 0x7f45080c0d10 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35448/0 (socket says 172.21.15.73:35448) 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 -- 172.21.15.73:0/1584008635 learned_addr learned my addr 172.21.15.73:0/1584008635 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f4506ffd700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f450806b510 0x7f4508136a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45067fc700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45080c0d10 0x7f450814bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 -- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45080c0d10 msgr2=0x7f450814bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45080c0d10 0x7f450814bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 -- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f450806b510 msgr2=0x7f4508136a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f450806b510 0x7f4508136a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 -- 172.21.15.73:0/1584008635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4500009ae0 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f45077fe700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 rx=0x7f450000b6c0 tx=0x7f4500009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.387+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000092e0 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.388+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4500007610 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.388+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508137880 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.388+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45000079d0 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.388+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4508137e40 con 0x7f45081546f0 2023-11-28T04:24:20.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45081381b0 con 0x7f45081546f0 2023-11-28T04:24:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f4500003040 con 0x7f45081546f0 2023-11-28T04:24:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f44eeffd700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 0x7f44f4061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f4506ffd700 1 -- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 msgr2=0x7f44f4061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 6552+0+0 (secure 0 0 0) 0x7f450007be30 con 0x7f45081546f0 2023-11-28T04:24:20.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.389+0000 7f4506ffd700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 0x7f44f4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:20.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.393+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f450004cf40 con 0x7f45081546f0 2023-11-28T04:24:20.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.561+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4508137fd0 con 0x7f45081546f0 2023-11-28T04:24:20.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.562+0000 7f44eeffd700 1 -- 172.21.15.73:0/1584008635 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v183) v1 ==== 74+0+17575 (secure 0 0 0) 0x7f4500023030 con 0x7f45081546f0 2023-11-28T04:24:20.564 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:20.564 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":183,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:19:20.794193+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":4,"to":3}]},{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.5","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3165037877":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/436795956":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1627000602":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1597394357":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:0/314665802":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1917929308":"2023-11-29T04:08:35.307681+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:20.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 msgr2=0x7f44f4061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:20.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 0x7f44f4061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 msgr2=0x7f45080c3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 rx=0x7f450000b6c0 tx=0x7f4500009840).stop 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 shutdown_connections 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f44f405f270 0x7f44f4061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45081546f0 0x7f45080c3c30 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f450806b510 0x7f4508136a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 --2- 172.21.15.73:0/1584008635 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f45080c0d10 0x7f450814bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 >> 172.21.15.73:0/1584008635 conn(0x7f45080c7760 msgr2=0x7f450814b570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 shutdown_connections 2023-11-28T04:24:20.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.564+0000 7f450d797700 1 -- 172.21.15.73:0/1584008635 wait complete. 2023-11-28T04:24:20.578 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '182', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 21, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T04:08:17.997851+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '178', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 22, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T04:08:18.423274+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T04:24:20.578 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:24:20.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.714+0000 7f81f720c700 1 Processor -- start 2023-11-28T04:24:20.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.715+0000 7f81f720c700 1 -- start start 2023-11-28T04:24:20.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81f0158ac0 0x7f81f00c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f81f0065640 0x7f81f00bd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f00c2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00be560 con 0x7f81f0158ac0 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00c2330 con 0x7f81f0065640 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f720c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00c30e0 con 0x7f81f00c2740 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f4fa8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81f0158ac0 0x7f81f00c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f4fa8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f81f0158ac0 0x7f81f00c1e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35464/0 (socket says 172.21.15.73:35464) 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3114056291 learned_addr learned my addr 172.21.15.73:0/3114056291 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81f57a9700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f00c2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.716+0000 7f81effff700 1 --2- 172.21.15.73:0/3114056291 >> v2:172.21.15.150:3300/0 conn(0x7f81f0065640 0x7f81f00bd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3114056291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 msgr2=0x7f81f00c2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:20.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f00c2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3114056291 >> v2:172.21.15.150:3300/0 conn(0x7f81f0065640 msgr2=0x7f81f00bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:20.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3114056291 >> v2:172.21.15.150:3300/0 conn(0x7f81f0065640 0x7f81f00bd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3114056291 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00bdd50 con 0x7f81f0158ac0 2023-11-28T04:24:20.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0158ac0 0x7f81f00c1e40 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f81f006b770 tx=0x7f81e8009840).ready entity=mon.0 client_cookie=fb3a85df91716ee server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:20.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81ef7fe700 1 -- 172.21.15.73:0/3114056291 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e800e070 con 0x7f81f0158ac0 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81ef7fe700 1 -- 172.21.15.73:0/3114056291 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e800b970 con 0x7f81f0158ac0 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.717+0000 7f81ef7fe700 1 -- 172.21.15.73:0/3114056291 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e800bcf0 con 0x7f81f0158ac0 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 -- 172.21.15.73:0/3114056291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0158ac0 msgr2=0x7f81f00c1e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0158ac0 0x7f81f00c1e40 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 rx=0x7f81f006b770 tx=0x7f81e8009840).stop 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 -- 172.21.15.73:0/3114056291 shutdown_connections 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0158ac0 0x7f81f00c1e40 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 --2- 172.21.15.73:0/3114056291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f00c2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 --2- 172.21.15.73:0/3114056291 >> v2:172.21.15.150:3300/0 conn(0x7f81f0065640 0x7f81f00bd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 -- 172.21.15.73:0/3114056291 >> 172.21.15.73:0/3114056291 conn(0x7f81f00c4200 msgr2=0x7f81f00c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.718+0000 7f81f720c700 1 -- 172.21.15.73:0/3114056291 shutdown_connections 2023-11-28T04:24:20.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.719+0000 7f81f720c700 1 -- 172.21.15.73:0/3114056291 wait complete. 2023-11-28T04:24:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.720+0000 7f81f720c700 1 Processor -- start 2023-11-28T04:24:20.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.720+0000 7f81f720c700 1 -- start start 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.721+0000 7f81f720c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f720c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f0157f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f720c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81f0158ac0 0x7f81f00a1000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f720c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f016d740 con 0x7f81f0065640 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f720c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f016ccb0 con 0x7f81f0158ac0 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f720c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0158400 con 0x7f81f00c2740 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35476/0 (socket says 172.21.15.73:35476) 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3828676884 learned_addr learned my addr 172.21.15.73:0/3828676884 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:20.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f57a9700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81f0158ac0 0x7f81f00a1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81effff700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f0157f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:20.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 msgr2=0x7f81f0157f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f0157f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81f0158ac0 msgr2=0x7f81f00a1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81f0158ac0 0x7f81f00a1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 -- 172.21.15.73:0/3828676884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009ae0 con 0x7f81f0065640 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81effff700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f0157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.722+0000 7f81f4fa8700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f81f006a0b0 tx=0x7f81e8009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.723+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8013910 con 0x7f81f0065640 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.723+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e8013a90 con 0x7f81f0065640 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.723+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e8013e50 con 0x7f81f0065640 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.724+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f016cea0 con 0x7f81f0065640 2023-11-28T04:24:20.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.724+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f016d460 con 0x7f81f0065640 2023-11-28T04:24:20.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.725+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f006c860 con 0x7f81f0065640 2023-11-28T04:24:20.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.725+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f81e8003040 con 0x7f81f0065640 2023-11-28T04:24:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.726+0000 7f81ecff9700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 0x7f81d0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.726+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f81e807b5a0 con 0x7f81f0065640 2023-11-28T04:24:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.726+0000 7f81effff700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 msgr2=0x7f81d0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:20.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.726+0000 7f81effff700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 0x7f81d0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:20.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.731+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f81e804c520 con 0x7f81f0065640 2023-11-28T04:24:20.851 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:20.850+0000 7fec53d20680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:24:20.873 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:20.872+0000 7fb51f3a6680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:24:20.923 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:20.922+0000 7fec53d20680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:24:20.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.927+0000 7f81effff700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 msgr2=0x7f81d0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:20.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.927+0000 7f81effff700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 0x7f81d0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:20.941 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:20.940+0000 7fb51f3a6680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:24:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.959+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f81f00bca40 con 0x7f81f0065640 2023-11-28T04:24:20.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.959+0000 7f81ecff9700 1 -- 172.21.15.73:0/3828676884 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v184) v1 ==== 110+0+10 (secure 0 0 0) 0x7f81e8022020 con 0x7f81f0065640 2023-11-28T04:24:20.961 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:24:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 msgr2=0x7f81d0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 0x7f81d0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 msgr2=0x7f81f016c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 rx=0x7f81f006a0b0 tx=0x7f81e8009840).stop 2023-11-28T04:24:20.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 shutdown_connections 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f81d005f270 0x7f81d0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f81f0158ac0 0x7f81f00a1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f81f0065640 0x7f81f016c7c0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 --2- 172.21.15.73:0/3828676884 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f81f00c2740 0x7f81f0157f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 >> 172.21.15.73:0/3828676884 conn(0x7f81f00c4200 msgr2=0x7f81f00ac700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:20.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 shutdown_connections 2023-11-28T04:24:20.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:20.961+0000 7f81f720c700 1 -- 172.21.15.73:0/3828676884 wait complete. 2023-11-28T04:24:20.975 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:24:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.096+0000 7f0b92917700 1 Processor -- start 2023-11-28T04:24:21.097 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.096+0000 7f0b92917700 1 -- start start 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b92917700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c1547f0 0x7f0b8c0a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b92917700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b92917700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b8c0c7e30 con 0x7f0b8c062700 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b92917700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b8c0bcf40 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b92917700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c6b20 con 0x7f0b8c1547f0 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35480/0 (socket says 172.21.15.73:35480) 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 -- 172.21.15.73:0/1067900990 learned_addr learned my addr 172.21.15.73:0/1067900990 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8bfff700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c1547f0 0x7f0b8c0a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 -- 172.21.15.73:0/1067900990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c1547f0 msgr2=0x7f0b8c0a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c1547f0 0x7f0b8c0a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 -- 172.21.15.73:0/1067900990 >> v1:172.21.15.150:6789/0 conn(0x7f0b8c062700 legacy=0x7f0b8c0a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 -- 172.21.15.73:0/1067900990 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b8c0a77d0 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.097+0000 7f0b8b7fe700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f0b80006fd0 tx=0x7f0b800090f0).ready entity=mon.0 client_cookie=95575dd6e04b5c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b8affd700 1 -- 172.21.15.73:0/1067900990 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000b580 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b8affd700 1 -- 172.21.15.73:0/1067900990 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8000b6e0 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b8affd700 1 -- 172.21.15.73:0/1067900990 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8000ba80 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 -- 172.21.15.73:0/1067900990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c0c86f0 msgr2=0x7f0b8c0c6fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 rx=0x7f0b80006fd0 tx=0x7f0b800090f0).stop 2023-11-28T04:24:21.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 -- 172.21.15.73:0/1067900990 shutdown_connections 2023-11-28T04:24:21.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c1547f0 0x7f0b8c0a72e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 --2- 172.21.15.73:0/1067900990 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c6fb0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 -- 172.21.15.73:0/1067900990 >> 172.21.15.73:0/1067900990 conn(0x7f0b8c0642d0 msgr2=0x7f0b8c0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 -- 172.21.15.73:0/1067900990 shutdown_connections 2023-11-28T04:24:21.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.098+0000 7f0b92917700 1 -- 172.21.15.73:0/1067900990 wait complete. 2023-11-28T04:24:21.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.099+0000 7f0b92917700 1 Processor -- start 2023-11-28T04:24:21.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 -- start start 2023-11-28T04:24:21.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c062700 0x7f0b8c144d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c0c3040 con 0x7f0b8c1547f0 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c142e70 con 0x7f0b8c0c86f0 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b92917700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b8c168ca0 con 0x7f0b8c062700 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b90eb4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b90eb4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35488/0 (socket says 172.21.15.73:35488) 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b8b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b8bfff700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c062700 0x7f0b8c144d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.100+0000 7f0b90eb4700 1 -- 172.21.15.73:0/3312815409 learned_addr learned my addr 172.21.15.73:0/3312815409 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 -- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c062700 msgr2=0x7f0b8c144d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c062700 0x7f0b8c144d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 -- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c0c86f0 msgr2=0x7f0b8c0c2660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c2660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 -- 172.21.15.73:0/3312815409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b80009370 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b90eb4700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f0b84002230 tx=0x7f0b840015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400e4a0 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b8400e600 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b8c142540 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b8400e980 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.101+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b8c142b00 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f0b8401b3b0 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b8c142fe0 con 0x7f0b8c1547f0 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b73fff700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 0x7f0b78061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b8bfff700 1 -- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 msgr2=0x7f0b78061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:21.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f0b84079bc0 con 0x7f0b8c1547f0 2023-11-28T04:24:21.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.103+0000 7f0b8bfff700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 0x7f0b78061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:21.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.107+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0b84048af0 con 0x7f0b8c1547f0 2023-11-28T04:24:21.111 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.111+0000 7fec53d20680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:24:21.139 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.138+0000 7fb51f3a6680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:24:21.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.278+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f0b8c06c860 con 0x7f0b8c1547f0 2023-11-28T04:24:21.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.278+0000 7f0b73fff700 1 -- 172.21.15.73:0/3312815409 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v184) v1 ==== 92+0+10 (secure 0 0 0) 0x7f0b8404ab80 con 0x7f0b8c1547f0 2023-11-28T04:24:21.279 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 msgr2=0x7f0b78061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 0x7f0b78061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 msgr2=0x7f0b8c0c2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 rx=0x7f0b84002230 tx=0x7f0b840015e0).stop 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 shutdown_connections 2023-11-28T04:24:21.282 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0b7805f270 0x7f0b78061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0b8c1547f0 0x7f0b8c0c2b50 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0b8c062700 0x7f0b8c144d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 --2- 172.21.15.73:0/3312815409 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0b8c0c86f0 0x7f0b8c0c2660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 >> 172.21.15.73:0/3312815409 conn(0x7f0b8c0642d0 msgr2=0x7f0b8c153a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 shutdown_connections 2023-11-28T04:24:21.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.280+0000 7f0b92917700 1 -- 172.21.15.73:0/3312815409 wait complete. 2023-11-28T04:24:21.293 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T04:24:21.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.414+0000 7fdfac40c700 1 Processor -- start 2023-11-28T04:24:21.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.414+0000 7fdfac40c700 1 -- start start 2023-11-28T04:24:21.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfac40c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfa40a8280 0x7fdfa40a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfac40c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa4155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfac40c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfa4067770 con 0x7fdfa414e600 2023-11-28T04:24:21.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfac40c700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfa4159180 con 0x7fdfa40a8280 2023-11-28T04:24:21.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfac40c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40be8d0 con 0x7fdfa4158160 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfa40a8280 0x7fdfa40a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfa40a8280 0x7fdfa40a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35498/0 (socket says 172.21.15.73:35498) 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 -- 172.21.15.73:0/2020421821 learned_addr learned my addr 172.21.15.73:0/2020421821 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfa99a7700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 -- 172.21.15.73:0/2020421821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 msgr2=0x7fdfa4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa4155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 -- 172.21.15.73:0/2020421821 >> v1:172.21.15.150:6789/0 conn(0x7fdfa414e600 legacy=0x7fdfa414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 -- 172.21.15.73:0/2020421821 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa4158600 con 0x7fdfa40a8280 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfa99a7700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.415+0000 7fdfaa1a8700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa40a8280 0x7fdfa40a8660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fdfa0006fd0 tx=0x7fdfa00090f0).ready entity=mon.0 client_cookie=23051502b9390171 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfa91a6700 1 -- 172.21.15.73:0/2020421821 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000b600 con 0x7fdfa40a8280 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfa91a6700 1 -- 172.21.15.73:0/2020421821 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfa000b760 con 0x7fdfa40a8280 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfa91a6700 1 -- 172.21.15.73:0/2020421821 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfa000bae0 con 0x7fdfa40a8280 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 -- 172.21.15.73:0/2020421821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa40a8280 msgr2=0x7fdfa40a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa40a8280 0x7fdfa40a8660 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 rx=0x7fdfa0006fd0 tx=0x7fdfa00090f0).stop 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 -- 172.21.15.73:0/2020421821 shutdown_connections 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa40a8280 0x7fdfa40a8660 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 --2- 172.21.15.73:0/2020421821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 -- 172.21.15.73:0/2020421821 >> 172.21.15.73:0/2020421821 conn(0x7fdfa4062700 msgr2=0x7fdfa4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 -- 172.21.15.73:0/2020421821 shutdown_connections 2023-11-28T04:24:21.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.416+0000 7fdfac40c700 1 -- 172.21.15.73:0/2020421821 wait complete. 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.417+0000 7fdfac40c700 1 Processor -- start 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.417+0000 7fdfac40c700 1 -- start start 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfa40a8280 0x7fdfa4165d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa414d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40c0ba0 con 0x7fdfa414e600 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40c0e80 con 0x7fdfa40a8280 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfac40c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfa40c1160 con 0x7fdfa4158160 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfa99a7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfa99a7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35508/0 (socket says 172.21.15.73:35508) 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfa99a7700 1 -- 172.21.15.73:0/3682767408 learned_addr learned my addr 172.21.15.73:0/3682767408 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfaa1a8700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfa40a8280 0x7fdfa4165d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.418+0000 7fdfaa9a9700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa414d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 -- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 msgr2=0x7fdfa414d130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa414d130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 -- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfa40a8280 msgr2=0x7fdfa4165d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfa40a8280 0x7fdfa4165d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 -- 172.21.15.73:0/3682767408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfa0009370 con 0x7fdfa414e600 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfa99a7700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7fdf9400d890 tx=0x7fdf94008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011590 con 0x7fdfa414e600 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf940116f0 con 0x7fdfa414e600 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfa40c3f50 con 0x7fdfa414e600 2023-11-28T04:24:21.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf94011a70 con 0x7fdfa414e600 2023-11-28T04:24:21.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.419+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfa40c56a0 con 0x7fdfa414e600 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.420+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfa40c5890 con 0x7fdfa414e600 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.421+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fdf94012420 con 0x7fdfa414e600 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.421+0000 7fdf9a7fc700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 0x7fdf88061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.421+0000 7fdfaa1a8700 1 -- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 msgr2=0x7fdf88061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.421+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fdf94079fa0 con 0x7fdfa414e600 2023-11-28T04:24:21.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.421+0000 7fdfaa1a8700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 0x7fdf88061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:21.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.425+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdfa40c5890 con 0x7fdfa414e600 2023-11-28T04:24:21.535 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.534+0000 7fec53d20680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:24:21.577 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.576+0000 7fb51f3a6680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:24:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.594+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdfa40c5890 con 0x7fdfa414e600 2023-11-28T04:24:21.595 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.594+0000 7fdf9a7fc700 1 -- 172.21.15.73:0/3682767408 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v184) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdfa40c5890 con 0x7fdfa414e600 2023-11-28T04:24:21.595 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:24:21.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 msgr2=0x7fdf88061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 0x7fdf88061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 msgr2=0x7fdfa40c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 rx=0x7fdf9400d890 tx=0x7fdf94008040).stop 2023-11-28T04:24:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 shutdown_connections 2023-11-28T04:24:21.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdf8805f270 0x7fdf88061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfa4158160 0x7fdfa414d130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfa40a8280 0x7fdfa4165d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 --2- 172.21.15.73:0/3682767408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfa414e600 0x7fdfa40c01e0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 >> 172.21.15.73:0/3682767408 conn(0x7fdfa4062700 msgr2=0x7fdfa414dbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 shutdown_connections 2023-11-28T04:24:21.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.597+0000 7fdfac40c700 1 -- 172.21.15.73:0/3682767408 wait complete. 2023-11-28T04:24:21.611 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T04:24:21.639 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.638+0000 7fec53d20680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:21.690 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.687+0000 7fb51f3a6680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.731+0000 7fda6a0ba700 1 Processor -- start 2023-11-28T04:24:21.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 -- start start 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda641509b0 0x7fda64150d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fda640c4db0 0x7fda64069830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda64068e00 con 0x7fda64069d20 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640693a0 con 0x7fda641509b0 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.732+0000 7fda6a0ba700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda640c2490 con 0x7fda640c4db0 2023-11-28T04:24:21.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda641509b0 0x7fda64150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fda641509b0 0x7fda64150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35520/0 (socket says 172.21.15.73:35520) 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 -- 172.21.15.73:0/3228323213 learned_addr learned my addr 172.21.15.73:0/3228323213 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda63fff700 1 --1- 172.21.15.73:0/3228323213 >> v1:172.21.15.73:6789/0 conn(0x7fda64069d20 0x7fda640bc670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36946/0 (socket says 172.21.15.73:36946) 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda62ffd700 1 --2- 172.21.15.73:0/3228323213 >> v2:172.21.15.150:3300/0 conn(0x7fda640c4db0 0x7fda64069830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 -- 172.21.15.73:0/3228323213 >> v2:172.21.15.150:3300/0 conn(0x7fda640c4db0 msgr2=0x7fda64069830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:21.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 --2- 172.21.15.73:0/3228323213 >> v2:172.21.15.150:3300/0 conn(0x7fda640c4db0 0x7fda64069830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 -- 172.21.15.73:0/3228323213 >> v1:172.21.15.73:6789/0 conn(0x7fda64069d20 legacy=0x7fda640bc670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 -- 172.21.15.73:0/3228323213 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda640c2770 con 0x7fda641509b0 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda637fe700 1 --2- 172.21.15.73:0/3228323213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda641509b0 0x7fda64150d90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7fda640c45f0 tx=0x7fda5c009840).ready entity=mon.0 client_cookie=96ff3cf1c1f9da62 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda627fc700 1 -- 172.21.15.73:0/3228323213 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00e070 con 0x7fda641509b0 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda627fc700 1 -- 172.21.15.73:0/3228323213 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c00ba20 con 0x7fda641509b0 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.733+0000 7fda627fc700 1 -- 172.21.15.73:0/3228323213 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c00bda0 con 0x7fda641509b0 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3228323213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda641509b0 msgr2=0x7fda64150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3228323213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda641509b0 0x7fda64150d90 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 rx=0x7fda640c45f0 tx=0x7fda5c009840).stop 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3228323213 shutdown_connections 2023-11-28T04:24:21.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3228323213 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda641509b0 0x7fda64150d90 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3228323213 >> v2:172.21.15.150:3300/0 conn(0x7fda640c4db0 0x7fda64069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3228323213 >> 172.21.15.73:0/3228323213 conn(0x7fda64062700 msgr2=0x7fda64062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3228323213 shutdown_connections 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.734+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3228323213 wait complete. 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.735+0000 7fda6a0ba700 1 Processor -- start 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.735+0000 7fda6a0ba700 1 -- start start 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda640c4db0 0x7fda640c0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda641509b0 0x7fda640c0860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640c0d50 con 0x7fda64069d20 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda640a79a0 con 0x7fda641509b0 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda6a0ba700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda64164440 con 0x7fda640c4db0 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35522/0 (socket says 172.21.15.73:35522) 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 -- 172.21.15.73:0/3191479801 learned_addr learned my addr 172.21.15.73:0/3191479801 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:21.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda62ffd700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda640c4db0 0x7fda640c0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda63fff700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda641509b0 0x7fda640c0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 -- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda640c4db0 msgr2=0x7fda640c0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda640c4db0 0x7fda640c0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 -- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda641509b0 msgr2=0x7fda640c0860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda641509b0 0x7fda640c0860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.736+0000 7fda637fe700 1 -- 172.21.15.73:0/3191479801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda5c009ae0 con 0x7fda64069d20 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda637fe700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7fda5c009580 tx=0x7fda5c009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:21.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c0092e0 con 0x7fda64069d20 2023-11-28T04:24:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda5c00bb90 con 0x7fda64069d20 2023-11-28T04:24:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda640a76c0 con 0x7fda64069d20 2023-11-28T04:24:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda5c01d460 con 0x7fda64069d20 2023-11-28T04:24:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.737+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda641640d0 con 0x7fda64069d20 2023-11-28T04:24:21.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.738+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda6406c860 con 0x7fda64069d20 2023-11-28T04:24:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.738+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fda5c003040 con 0x7fda64069d20 2023-11-28T04:24:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.739+0000 7fda5b7fe700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 0x7fda44061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.739+0000 7fda62ffd700 1 -- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 msgr2=0x7fda44061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.739+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fda5c07b6b0 con 0x7fda64069d20 2023-11-28T04:24:21.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.739+0000 7fda62ffd700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 0x7fda44061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:21.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.746+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fda5c04c6b0 con 0x7fda64069d20 2023-11-28T04:24:21.796 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.795+0000 7fec53d20680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:21.849 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.848+0000 7fb51f3a6680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:21.872 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.871+0000 7fec53d20680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:24:21.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.917+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fda640bc190 con 0x7fda64069d20 2023-11-28T04:24:21.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.918+0000 7fda5b7fe700 1 -- 172.21.15.73:0/3191479801 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v184) v1 ==== 100+0+11 (secure 0 0 0) 0x7fda5c022020 con 0x7fda64069d20 2023-11-28T04:24:21.919 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:24:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 msgr2=0x7fda44061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 0x7fda44061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 msgr2=0x7fda6414e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:21.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 rx=0x7fda5c009580 tx=0x7fda5c009840).stop 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 shutdown_connections 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fda4405f270 0x7fda44061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fda641509b0 0x7fda640c0860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fda64069d20 0x7fda6414e800 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 --2- 172.21.15.73:0/3191479801 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fda640c4db0 0x7fda640c0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 >> 172.21.15.73:0/3191479801 conn(0x7fda64062700 msgr2=0x7fda64154730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 shutdown_connections 2023-11-28T04:24:21.922 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:21.920+0000 7fda6a0ba700 1 -- 172.21.15.73:0/3191479801 wait complete. 2023-11-28T04:24:21.927 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.926+0000 7fb51f3a6680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:24:21.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:21.938 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:21.938+0000 7fec53d20680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:24:22.000 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:21.999+0000 7fb51f3a6680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:24:22.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.048+0000 7f9d1cb9d700 1 Processor -- start 2023-11-28T04:24:22.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.048+0000 7f9d1cb9d700 1 -- start start 2023-11-28T04:24:22.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1cb9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d18066e00 0x7f9d180671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1cb9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 0x7f9d180a5980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1cb9d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d180bd590 con 0x7f9d180a5e70 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1cb9d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d18156370 con 0x7f9d18066e00 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1cb9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d18156650 con 0x7f9d18155e40 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d18066e00 0x7f9d180671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.049+0000 7f9d1659c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9d18066e00 0x7f9d180671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35532/0 (socket says 172.21.15.73:35532) 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 -- 172.21.15.73:0/1074142588 learned_addr learned my addr 172.21.15.73:0/1074142588 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d15d9b700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 0x7f9d180a5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 -- 172.21.15.73:0/1074142588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 msgr2=0x7f9d180a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 0x7f9d180a5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 -- 172.21.15.73:0/1074142588 >> v1:172.21.15.150:6789/0 conn(0x7f9d180a5e70 legacy=0x7f9d18067980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 -- 172.21.15.73:0/1074142588 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d18156930 con 0x7f9d18066e00 2023-11-28T04:24:22.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d15d9b700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 0x7f9d180a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1659c700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18066e00 0x7f9d180671e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f9d08006fd0 tx=0x7f9d080090f0).ready entity=mon.0 client_cookie=2ea93c0c16a632a1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1559a700 1 -- 172.21.15.73:0/1074142588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800b600 con 0x7f9d18066e00 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1559a700 1 -- 172.21.15.73:0/1074142588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d0800b760 con 0x7f9d18066e00 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.050+0000 7f9d1559a700 1 -- 172.21.15.73:0/1074142588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d0800bae0 con 0x7f9d18066e00 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/1074142588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18066e00 msgr2=0x7f9d180671e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18066e00 0x7f9d180671e0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 rx=0x7f9d08006fd0 tx=0x7f9d080090f0).stop 2023-11-28T04:24:22.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/1074142588 shutdown_connections 2023-11-28T04:24:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18066e00 0x7f9d180671e0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/1074142588 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18155e40 0x7f9d180a5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/1074142588 >> 172.21.15.73:0/1074142588 conn(0x7f9d180c7fd0 msgr2=0x7f9d180c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/1074142588 shutdown_connections 2023-11-28T04:24:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.051+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/1074142588 wait complete. 2023-11-28T04:24:22.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.052+0000 7f9d1cb9d700 1 Processor -- start 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.052+0000 7f9d1cb9d700 1 -- start start 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18066e00 0x7f9d180a9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d181626c0 0x7f9d18165f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180aaf60 con 0x7f9d18155e40 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180a9f20 con 0x7f9d181626c0 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1cb9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d180ab1b0 con 0x7f9d18066e00 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35540/0 (socket says 172.21.15.73:35540) 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 -- 172.21.15.73:0/2195438828 learned_addr learned my addr 172.21.15.73:0/2195438828 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:22.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d1659c700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18066e00 0x7f9d180a9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d16d9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d181626c0 0x7f9d18165f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18066e00 msgr2=0x7f9d180a9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18066e00 0x7f9d180a9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d181626c0 msgr2=0x7f9d18165f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d181626c0 0x7f9d18165f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 -- 172.21.15.73:0/2195438828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d08009370 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d15d9b700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f9d0000d860 tx=0x7f9d00008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.053+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011590 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.054+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d000116f0 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.054+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d180ab490 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.054+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d00011a70 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.054+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d180a9510 con 0x7f9d18155e40 2023-11-28T04:24:22.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.054+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d180a9880 con 0x7f9d18155e40 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.055+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f9d00012420 con 0x7f9d18155e40 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.055+0000 7f9d0e7fc700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 0x7f9cf8061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.055+0000 7f9d1659c700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 msgr2=0x7f9cf8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.055+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f9d00079ed0 con 0x7f9d18155e40 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.055+0000 7f9d1659c700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 0x7f9cf8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:22.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.063+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d0004ade0 con 0x7f9d18155e40 2023-11-28T04:24:22.203 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:22.202+0000 7fec53d20680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:24:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.256+0000 7f9d1659c700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 msgr2=0x7f9cf8061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:22.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.256+0000 7f9d1659c700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 0x7f9cf8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:22.273 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:22.272+0000 7fb51f3a6680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:24:22.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.274+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9d1806c860 con 0x7f9d18155e40 2023-11-28T04:24:22.275 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.274+0000 7f9d0e7fc700 1 -- 172.21.15.73:0/2195438828 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f9d0004e400 con 0x7f9d18155e40 2023-11-28T04:24:22.275 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:22.275 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:24:22.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 msgr2=0x7f9cf8061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:22.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 0x7f9cf8061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 msgr2=0x7f9d181621d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 rx=0x7f9d0000d860 tx=0x7f9d00008040).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 shutdown_connections 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9cf805f270 0x7f9cf8061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d181626c0 0x7f9d18165f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d18066e00 0x7f9d180a9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 --2- 172.21.15.73:0/2195438828 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d18155e40 0x7f9d181621d0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 >> 172.21.15.73:0/2195438828 conn(0x7f9d180c7fd0 msgr2=0x7f9d18150f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 shutdown_connections 2023-11-28T04:24:22.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.276+0000 7f9d1cb9d700 1 -- 172.21.15.73:0/2195438828 wait complete. 2023-11-28T04:24:22.279 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:24:22.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:22.388 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:22.387+0000 7fec53d20680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:22.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.409+0000 7f2b30498700 1 Processor -- start 2023-11-28T04:24:22.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.409+0000 7f2b30498700 1 -- start start 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b30498700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 0x7f2b280c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b30498700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 0x7f2b280c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b30498700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b28154d20 con 0x7f2b280c2640 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b30498700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b28151670 con 0x7f2b28068090 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b30498700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280c48a0 con 0x7f2b28067cb0 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b2ea35700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2b280c2640 0x7f2b280c2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:36956/0 (socket says 172.21.15.73:36956) 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b2ea35700 1 -- 172.21.15.73:0/3634033607 learned_addr learned my addr 172.21.15.73:0/3634033607 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3634033607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 0x7f2b280c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.410+0000 7f2b2da33700 1 --2- 172.21.15.73:0/3634033607 >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 0x7f2b280c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2d232700 1 -- 172.21.15.73:0/3634033607 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4182988320 0 0) 0x7f2b28154d20 con 0x7f2b280c2640 2023-11-28T04:24:22.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2d232700 1 -- 172.21.15.73:0/3634033607 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b140033f0 con 0x7f2b280c2640 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2e234700 1 -- 172.21.15.73:0/3634033607 >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 msgr2=0x7f2b280c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3634033607 >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 0x7f2b280c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2e234700 1 -- 172.21.15.73:0/3634033607 >> v1:172.21.15.73:6789/0 conn(0x7f2b280c2640 legacy=0x7f2b280c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2e234700 1 -- 172.21.15.73:0/3634033607 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b280a7490 con 0x7f2b28067cb0 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2da33700 1 --2- 172.21.15.73:0/3634033607 >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 0x7f2b280c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3634033607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 0x7f2b280c3ec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2b18009760 tx=0x7f2b18009060).ready entity=mon.2 client_cookie=d88ad512232d7d75 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2d232700 1 -- 172.21.15.73:0/3634033607 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800d4a0 con 0x7f2b28067cb0 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2d232700 1 -- 172.21.15.73:0/3634033607 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b1800d600 con 0x7f2b28067cb0 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.411+0000 7f2b2d232700 1 -- 172.21.15.73:0/3634033607 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800d980 con 0x7f2b28067cb0 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 -- 172.21.15.73:0/3634033607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 msgr2=0x7f2b280c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 --2- 172.21.15.73:0/3634033607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 0x7f2b280c3ec0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 rx=0x7f2b18009760 tx=0x7f2b18009060).stop 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 -- 172.21.15.73:0/3634033607 shutdown_connections 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 --2- 172.21.15.73:0/3634033607 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b28067cb0 0x7f2b280c3ec0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 --2- 172.21.15.73:0/3634033607 >> v2:172.21.15.150:3300/0 conn(0x7f2b28068090 0x7f2b280c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 -- 172.21.15.73:0/3634033607 >> 172.21.15.73:0/3634033607 conn(0x7f2b280c7fd0 msgr2=0x7f2b280c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 -- 172.21.15.73:0/3634033607 shutdown_connections 2023-11-28T04:24:22.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.412+0000 7f2b30498700 1 -- 172.21.15.73:0/3634033607 wait complete. 2023-11-28T04:24:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.413+0000 7f2b30498700 1 Processor -- start 2023-11-28T04:24:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.413+0000 7f2b30498700 1 -- start start 2023-11-28T04:24:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b28068090 0x7f2b28062700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b280c2640 0x7f2b28062bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b280630e0 con 0x7f2b28067cb0 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28065b00 con 0x7f2b28068090 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b30498700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b28065de0 con 0x7f2b280c2640 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35550/0 (socket says 172.21.15.73:35550) 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 -- 172.21.15.73:0/3162947279 learned_addr learned my addr 172.21.15.73:0/3162947279 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2da33700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b28068090 0x7f2b28062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2ea35700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b280c2640 0x7f2b28062bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 -- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b280c2640 msgr2=0x7f2b28062bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.418 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b280c2640 0x7f2b28062bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 -- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b28068090 msgr2=0x7f2b28062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b28068090 0x7f2b28062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 -- 172.21.15.73:0/3162947279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240092a0 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b2e234700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7f2b18009d50 tx=0x7f2b18016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.414+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18021070 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.415+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b18000bc0 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.415+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800d980 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.415+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b180092a0 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.415+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b280663a0 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b28066560 con 0x7f2b28067cb0 2023-11-28T04:24:22.419 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f2b1801d640 con 0x7f2b28067cb0 2023-11-28T04:24:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b1e7fc700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 0x7f2b0c061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b2da33700 1 -- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 msgr2=0x7f2b0c061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f2b1807b9b0 con 0x7f2b28067cb0 2023-11-28T04:24:22.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.416+0000 7f2b2da33700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 0x7f2b0c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:22.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.420+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2b1804c880 con 0x7f2b28067cb0 2023-11-28T04:24:22.466 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:22.464+0000 7fb51f3a6680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:22.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.592+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b280a7490 con 0x7f2b28067cb0 2023-11-28T04:24:22.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.593+0000 7f2b1e7fc700 1 -- 172.21.15.73:0/3162947279 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17689 (secure 0 0 0) 0x7f2b280a7490 con 0x7f2b28067cb0 2023-11-28T04:24:22.594 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:22.594 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":184,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:20.668564+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":184,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":4,"to":3}]},{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.5","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3165037877":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/436795956":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1627000602":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1597394357":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:0/314665802":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1917929308":"2023-11-29T04:08:35.307681+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:22.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 msgr2=0x7f2b0c061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 0x7f2b0c061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 msgr2=0x7f2b280671c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 rx=0x7f2b18009d50 tx=0x7f2b18016000).stop 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 shutdown_connections 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2b0c05f270 0x7f2b0c061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2b280c2640 0x7f2b28062bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2b28067cb0 0x7f2b280671c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 --2- 172.21.15.73:0/3162947279 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2b28068090 0x7f2b28062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 >> 172.21.15.73:0/3162947279 conn(0x7f2b280c7fd0 msgr2=0x7f2b28154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 shutdown_connections 2023-11-28T04:24:22.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.595+0000 7f2b30498700 1 -- 172.21.15.73:0/3162947279 wait complete. 2023-11-28T04:24:22.608 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:24:22.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:22.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.724+0000 7f9375e32700 1 Processor -- start 2023-11-28T04:24:22.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.724+0000 7f9375e32700 1 -- start start 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9375e32700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f9370155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9375e32700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9370095e80 0x7f9370096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9375e32700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9370096800 con 0x7f93701544d0 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9375e32700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9370067650 con 0x7f9370095e80 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9375e32700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93700683d0 con 0x7f9370064e20 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f936ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9370095e80 0x7f9370096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f936ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9370095e80 0x7f9370096310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35566/0 (socket says 172.21.15.73:35566) 2023-11-28T04:24:22.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f936ffff700 1 -- 172.21.15.73:0/2368422513 learned_addr learned my addr 172.21.15.73:0/2368422513 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.725+0000 7f9366ffd700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f9370155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936ffff700 1 -- 172.21.15.73:0/2368422513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 msgr2=0x7f9370155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936ffff700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f9370155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936ffff700 1 -- 172.21.15.73:0/2368422513 >> v1:172.21.15.73:6789/0 conn(0x7f93701544d0 legacy=0x7f9370066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936effd700 1 -- 172.21.15.73:0/2368422513 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3353671117 0 0) 0x7f9370096800 con 0x7f93701544d0 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936ffff700 1 -- 172.21.15.73:0/2368422513 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370067850 con 0x7f9370095e80 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936ffff700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370096310 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f9368009f20 tx=0x7f9368009840).ready entity=mon.0 client_cookie=3377e412ba5e5e96 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936effd700 1 -- 172.21.15.73:0/2368422513 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800e070 con 0x7f9370095e80 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936effd700 1 -- 172.21.15.73:0/2368422513 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f936800b9d0 con 0x7f9370095e80 2023-11-28T04:24:22.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f936effd700 1 -- 172.21.15.73:0/2368422513 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f936800bd50 con 0x7f9370095e80 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 -- 172.21.15.73:0/2368422513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 msgr2=0x7f9370096310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370096310 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 rx=0x7f9368009f20 tx=0x7f9368009840).stop 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 -- 172.21.15.73:0/2368422513 shutdown_connections 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370096310 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 --2- 172.21.15.73:0/2368422513 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f9370155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 -- 172.21.15.73:0/2368422513 >> 172.21.15.73:0/2368422513 conn(0x7f93700631d0 msgr2=0x7f93700c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 -- 172.21.15.73:0/2368422513 shutdown_connections 2023-11-28T04:24:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.726+0000 7f9375e32700 1 -- 172.21.15.73:0/2368422513 wait complete. 2023-11-28T04:24:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.727+0000 7f9375e32700 1 Processor -- start 2023-11-28T04:24:22.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 -- start start 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f937014de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93701544d0 0x7f93700c6f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9370168a40 con 0x7f9370095e80 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937014f880 con 0x7f93701544d0 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.728+0000 7f9375e32700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f937014fe20 con 0x7f9370064e20 2023-11-28T04:24:22.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35578/0 (socket says 172.21.15.73:35578) 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 -- 172.21.15.73:0/3688682700 learned_addr learned my addr 172.21.15.73:0/3688682700 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f936f7fe700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f937014de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f936ffff700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93701544d0 0x7f93700c6f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 msgr2=0x7f937014de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f937014de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93701544d0 msgr2=0x7f93700c6f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93701544d0 0x7f93700c6f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 -- 172.21.15.73:0/3688682700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9368009ae0 con 0x7f9370095e80 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9366ffd700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f935800e950 tx=0x7f9358009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:22.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358012600 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9358012760 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9370168480 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358012ae0 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.729+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937014fab0 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.730+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93700c33f0 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.731+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f935801b400 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.731+0000 7f9367fff700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 0x7f9350061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.731+0000 7f936f7fe700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 msgr2=0x7f9350061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.731+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f935807aa90 con 0x7f9370095e80 2023-11-28T04:24:22.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.731+0000 7f936f7fe700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 0x7f9350061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:22.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.739+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f935804ba00 con 0x7f9370095e80 2023-11-28T04:24:22.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.932+0000 7f936f7fe700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 msgr2=0x7f9350061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:22.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.932+0000 7f936f7fe700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 0x7f9350061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.954+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f937006c860 con 0x7f9370095e80 2023-11-28T04:24:22.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.954+0000 7f9367fff700 1 -- 172.21.15.73:0/3688682700 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f9358016020 con 0x7f9370095e80 2023-11-28T04:24:22.955 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:22.956 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:24:22.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 msgr2=0x7f9350061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:22.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 0x7f9350061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 msgr2=0x7f9370167ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 rx=0x7f935800e950 tx=0x7f9358009060).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 shutdown_connections 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f935005f270 0x7f9350061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f93701544d0 0x7f93700c6f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9370064e20 0x7f937014de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 --2- 172.21.15.73:0/3688682700 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9370095e80 0x7f9370167ac0 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:22.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 >> 172.21.15.73:0/3688682700 conn(0x7f93700631d0 msgr2=0x7f93700c3860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:22.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 shutdown_connections 2023-11-28T04:24:22.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:22.956+0000 7f9375e32700 1 -- 172.21.15.73:0/3688682700 wait complete. 2023-11-28T04:24:22.960 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:24:22.970 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:23.048 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.048+0000 7fec53d20680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:24:23.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.090+0000 7f4b5e4ac700 1 Processor -- start 2023-11-28T04:24:23.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.090+0000 7f4b5e4ac700 1 -- start start 2023-11-28T04:24:23.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5e4ac700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4b5806b510 0x7f4b581546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5e4ac700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b580c10f0 0x7f4b58158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5e4ac700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4b58063380 con 0x7f4b580c0d10 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5e4ac700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b580c19e0 con 0x7f4b580c10f0 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5e4ac700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4b58159060 con 0x7f4b5806b510 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b580c10f0 0x7f4b58158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4b580c10f0 0x7f4b58158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35580/0 (socket says 172.21.15.73:35580) 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 -- 172.21.15.73:0/3950391686 learned_addr learned my addr 172.21.15.73:0/3950391686 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b577fe700 1 --2- 172.21.15.73:0/3950391686 >> v2:172.21.15.150:3300/0 conn(0x7f4b5806b510 0x7f4b581546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 -- 172.21.15.73:0/3950391686 >> v2:172.21.15.150:3300/0 conn(0x7f4b5806b510 msgr2=0x7f4b581546f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:23.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/3950391686 >> v2:172.21.15.150:3300/0 conn(0x7f4b5806b510 0x7f4b581546f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 -- 172.21.15.73:0/3950391686 >> v1:172.21.15.150:6789/0 conn(0x7f4b580c0d10 legacy=0x7f4b58153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.091+0000 7f4b5ca49700 1 -- 172.21.15.73:0/3950391686 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b58062770 con 0x7f4b580c10f0 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/3950391686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b580c10f0 0x7f4b58158b70 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f4b50009f20 tx=0x7f4b50009840).ready entity=mon.0 client_cookie=33ca34dc7180d5a9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b56ffd700 1 -- 172.21.15.73:0/3950391686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000e070 con 0x7f4b580c10f0 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b56ffd700 1 -- 172.21.15.73:0/3950391686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b5000b990 con 0x7f4b580c10f0 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b56ffd700 1 -- 172.21.15.73:0/3950391686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000bd10 con 0x7f4b580c10f0 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/3950391686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b580c10f0 msgr2=0x7f4b58158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/3950391686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b580c10f0 0x7f4b58158b70 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 rx=0x7f4b50009f20 tx=0x7f4b50009840).stop 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/3950391686 shutdown_connections 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/3950391686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b580c10f0 0x7f4b58158b70 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/3950391686 >> v2:172.21.15.150:3300/0 conn(0x7f4b5806b510 0x7f4b581546f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/3950391686 >> 172.21.15.73:0/3950391686 conn(0x7f4b580c7760 msgr2=0x7f4b580c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:23.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/3950391686 shutdown_connections 2023-11-28T04:24:23.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.092+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/3950391686 wait complete. 2023-11-28T04:24:23.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.093+0000 7f4b5e4ac700 1 Processor -- start 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 -- start start 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b580c0d10 0x7f4b58152ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 0x7f4b58152fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b581534b0 con 0x7f4b5806b510 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4b580a95a0 con 0x7f4b580c10f0 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.094+0000 7f4b5e4ac700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4b58155bf0 con 0x7f4b580c0d10 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b57fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b57fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35592/0 (socket says 172.21.15.73:35592) 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b57fff700 1 -- 172.21.15.73:0/922976806 learned_addr learned my addr 172.21.15.73:0/922976806 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 0x7f4b58152fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b577fe700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b580c0d10 0x7f4b58152ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.099 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b580c0d10 msgr2=0x7f4b58152ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b580c0d10 0x7f4b58152ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 msgr2=0x7f4b580a90b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 -- 172.21.15.73:0/922976806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4b50009ae0 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b57fff700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5ca49700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 0x7f4b58152fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f4b50013000 tx=0x7f4b50009840).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5000bd10 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4b500092e0 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4b58155070 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4b5001c630 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.095+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4b58155630 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.096+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4b580ac260 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.097+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f4b5001c9a0 con 0x7f4b580c10f0 2023-11-28T04:24:23.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.097+0000 7f4b3ffff700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 0x7f4b44061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.097+0000 7f4b57fff700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 msgr2=0x7f4b44061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:23.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.097+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f4b5007beb0 con 0x7f4b580c10f0 2023-11-28T04:24:23.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.097+0000 7f4b57fff700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 0x7f4b44061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:23.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.101+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4b580ac260 con 0x7f4b580c10f0 2023-11-28T04:24:23.120 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.119+0000 7fec53d20680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:24:23.153 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.152+0000 7fb51f3a6680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:24:23.195 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.194+0000 7fec53d20680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:24:23.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.223+0000 7fb51f3a6680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:24:23.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.298+0000 7f4b57fff700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 msgr2=0x7f4b44061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:23.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.298+0000 7f4b57fff700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 0x7f4b44061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:23.300 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.300+0000 7fb51f3a6680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:24:23.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.313+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4b580ac260 con 0x7f4b580c10f0 2023-11-28T04:24:23.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.314+0000 7f4b3ffff700 1 -- 172.21.15.73:0/922976806 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f4b580ac260 con 0x7f4b580c10f0 2023-11-28T04:24:23.315 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:23.315 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":64,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":64,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:08:19.448808+0000","modified":"2023-11-28T04:08:21.457494+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9006},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9006":{"gid":9006,"name":"a","rank":0,"incarnation":62,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/3312350001","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3312350001},{"type":"v1","addr":"172.21.15.73:6837","nonce":3312350001}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[22],"metadata_pool":21,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":10}]} 2023-11-28T04:24:23.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 msgr2=0x7f4b44061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:23.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 0x7f4b44061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:23.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 msgr2=0x7f4b58152fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 0x7f4b58152fc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 rx=0x7f4b50013000 tx=0x7f4b50009840).stop 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 shutdown_connections 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4b4405f270 0x7f4b44061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4b580c10f0 0x7f4b58152fc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4b5806b510 0x7f4b580a90b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 --2- 172.21.15.73:0/922976806 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4b580c0d10 0x7f4b58152ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 >> 172.21.15.73:0/922976806 conn(0x7f4b580c7760 msgr2=0x7f4b580c56b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 shutdown_connections 2023-11-28T04:24:23.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.316+0000 7f4b5e4ac700 1 -- 172.21.15.73:0/922976806 wait complete. 2023-11-28T04:24:23.319 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 64 2023-11-28T04:24:23.330 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:23.343 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.343+0000 7fec53d20680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:24:23.415 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.414+0000 7fec53d20680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:24:23.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 Processor -- start 2023-11-28T04:24:23.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 -- start start 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 0x7f50c406a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50c40c69a0 con 0x7f50c41544e0 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50c40c13a0 con 0x7f50c41548c0 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.445+0000 7f50c9799700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40c1680 con 0x7f50c40c6e30 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c27fc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 0x7f50c406a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c2ffd700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f50c41544e0 0x7f50c40c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45688/0 (socket says 172.21.15.73:45688) 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:39270/0 (socket says 172.21.15.73:39270) 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c2ffd700 1 -- 172.21.15.73:0/3656892164 learned_addr learned my addr 172.21.15.73:0/3656892164 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 -- 172.21.15.73:0/3656892164 >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 msgr2=0x7f50c406a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 --2- 172.21.15.73:0/3656892164 >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 0x7f50c406a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 -- 172.21.15.73:0/3656892164 >> v1:172.21.15.150:6789/0 conn(0x7f50c41544e0 legacy=0x7f50c40c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c1ffb700 1 -- 172.21.15.73:0/3656892164 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4032305699 0 0) 0x7f50c40c69a0 con 0x7f50c41544e0 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 -- 172.21.15.73:0/3656892164 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50c40c6630 con 0x7f50c40c6e30 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c27fc700 1 --2- 172.21.15.73:0/3656892164 >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 0x7f50c406a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c37fe700 1 --2- 172.21.15.73:0/3656892164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f50bc009e60 tx=0x7f50bc009780).ready entity=mon.2 client_cookie=7253557730d456e9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c1ffb700 1 -- 172.21.15.73:0/3656892164 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc00e070 con 0x7f50c40c6e30 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c1ffb700 1 -- 172.21.15.73:0/3656892164 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50bc00bb60 con 0x7f50c40c6e30 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.446+0000 7f50c1ffb700 1 -- 172.21.15.73:0/3656892164 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50bc012410 con 0x7f50c40c6e30 2023-11-28T04:24:23.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 -- 172.21.15.73:0/3656892164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 msgr2=0x7f50c40c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 --2- 172.21.15.73:0/3656892164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 rx=0x7f50bc009e60 tx=0x7f50bc009780).stop 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 -- 172.21.15.73:0/3656892164 shutdown_connections 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 --2- 172.21.15.73:0/3656892164 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c40c6e30 0x7f50c40c72e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 --2- 172.21.15.73:0/3656892164 >> v2:172.21.15.150:3300/0 conn(0x7f50c41548c0 0x7f50c406a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 -- 172.21.15.73:0/3656892164 >> 172.21.15.73:0/3656892164 conn(0x7f50c4085020 msgr2=0x7f50c4094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 -- 172.21.15.73:0/3656892164 shutdown_connections 2023-11-28T04:24:23.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.447+0000 7f50c9799700 1 -- 172.21.15.73:0/3656892164 wait complete. 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.448+0000 7f50c9799700 1 Processor -- start 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.448+0000 7f50c9799700 1 -- start start 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 0x7f50c40c4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 0x7f50c414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c41684d0 con 0x7f50c40c6e30 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50c40ad040 con 0x7f50c41548c0 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c9799700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50c4167ec0 con 0x7f50c41544e0 2023-11-28T04:24:23.453 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53226/0 (socket says 172.21.15.73:53226) 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 -- 172.21.15.73:0/1073857240 learned_addr learned my addr 172.21.15.73:0/1073857240 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c27fc700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 0x7f50c40c4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c37fe700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 0x7f50c414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 -- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 msgr2=0x7f50c40c4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 0x7f50c40c4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 -- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 msgr2=0x7f50c414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.449+0000 7f50c2ffd700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 0x7f50c414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c2ffd700 1 -- 172.21.15.73:0/1073857240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50b80092a0 con 0x7f50c40c6e30 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c37fe700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 0x7f50c414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c27fc700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 0x7f50c40c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:23.454 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c2ffd700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f50b400c490 tx=0x7f50b4007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b40070d0 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50b4007230 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50b40075f0 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50bc009a20 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.450+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50c40acd60 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.451+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50c41687b0 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.451+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7f50b4005c10 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.451+0000 7f50aaffd700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 0x7f50b0061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.451+0000 7f50c27fc700 1 -- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 msgr2=0x7f50b0061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.452+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7f50b4079d30 con 0x7f50c40c6e30 2023-11-28T04:24:23.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.452+0000 7f50c27fc700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 0x7f50b0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:23.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.455+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f50b404aca0 con 0x7f50c40c6e30 2023-11-28T04:24:23.458 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.457+0000 7fb51f3a6680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:24:23.532 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.532+0000 7fec53d20680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:24:23.534 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.533+0000 7fb51f3a6680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:24:23.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.630+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50c406c860 con 0x7f50c40c6e30 2023-11-28T04:24:23.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.631+0000 7f50aaffd700 1 -- 172.21.15.73:0/1073857240 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v184) v1 ==== 74+0+17689 (secure 0 0 0) 0x7f50b4018030 con 0x7f50c40c6e30 2023-11-28T04:24:23.633 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:23.633 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":184,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:20.668564+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"182","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":21,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:08:17.997851+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"178","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":22,"pool_name":"cephfs_data","create_time":"2023-11-28T04:08:18.423274+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"184","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":184,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.7","mappings":[{"from":4,"to":3}]},{"pgid":"22.3","mappings":[{"from":3,"to":0}]},{"pgid":"22.5","mappings":[{"from":3,"to":0}]}],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/3165037877":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/436795956":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1627000602":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1597394357":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6833/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:6832/90017":"2023-11-29T04:08:35.307681+0000","172.21.15.73:0/314665802":"2023-11-29T04:19:20.794146+0000","172.21.15.73:0/1917929308":"2023-11-29T04:08:35.307681+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":22,"snaps":[{"begin":2,"length":32}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:23.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.633+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 msgr2=0x7f50b0061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:23.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.633+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 0x7f50b0061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:23.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.633+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 msgr2=0x7f50c4168960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.633+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 rx=0x7f50b400c490 tx=0x7f50b4007e80).stop 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 shutdown_connections 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f50b005f270 0x7f50b0061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f50c41548c0 0x7f50c414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f50c40c6e30 0x7f50c4168960 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 --2- 172.21.15.73:0/1073857240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f50c41544e0 0x7f50c40c4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 >> 172.21.15.73:0/1073857240 conn(0x7f50c4085020 msgr2=0x7f50c40caaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 shutdown_connections 2023-11-28T04:24:23.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.634+0000 7f50c9799700 1 -- 172.21.15.73:0/1073857240 wait complete. 2023-11-28T04:24:23.642 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.641+0000 7fec53d20680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:23.648 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T04:24:23.655 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.654+0000 7fb51f3a6680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:24:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.766+0000 7fba48f3e700 1 Processor -- start 2023-11-28T04:24:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.766+0000 7fba48f3e700 1 -- start start 2023-11-28T04:24:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba48f3e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fba440c6190 0x7fba440c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba48f3e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba44158320 0x7fba441587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.767 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba48f3e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba44158cc0 con 0x7fba44098d10 2023-11-28T04:24:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba48f3e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba440c72d0 con 0x7fba440c6190 2023-11-28T04:24:23.768 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba48f3e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba44064570 con 0x7fba44158320 2023-11-28T04:24:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fba440c6190 0x7fba440c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba42d9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba44158320 0x7fba441587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba4259c700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fba44098d10 0x7fba44063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45694/0 (socket says 172.21.15.73:45694) 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba4259c700 1 -- 172.21.15.73:0/3591254322 learned_addr learned my addr 172.21.15.73:0/3591254322 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 -- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba44158320 msgr2=0x7fba441587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 --2- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba44158320 0x7fba441587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 -- 172.21.15.73:0/3591254322 >> v1:172.21.15.150:6789/0 conn(0x7fba44098d10 legacy=0x7fba44063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 -- 172.21.15.73:0/3591254322 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba44154bc0 con 0x7fba440c6190 2023-11-28T04:24:23.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba4159a700 1 -- 172.21.15.73:0/3591254322 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3732927657 0 0) 0x7fba34001110 con 0x7fba44098d10 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.767+0000 7fba41d9b700 1 --2- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba440c6190 0x7fba440c7760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fba2c009760 tx=0x7fba2c009060).ready entity=mon.1 client_cookie=d748239ca5170fb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba4159a700 1 -- 172.21.15.73:0/3591254322 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00f510 con 0x7fba440c6190 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba4159a700 1 -- 172.21.15.73:0/3591254322 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba2c00f670 con 0x7fba440c6190 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba4159a700 1 -- 172.21.15.73:0/3591254322 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba2c00f9f0 con 0x7fba440c6190 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 -- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba440c6190 msgr2=0x7fba440c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 --2- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba440c6190 0x7fba440c7760 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 rx=0x7fba2c009760 tx=0x7fba2c009060).stop 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 -- 172.21.15.73:0/3591254322 shutdown_connections 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 --2- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba44158320 0x7fba441587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 --2- 172.21.15.73:0/3591254322 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba440c6190 0x7fba440c7760 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 -- 172.21.15.73:0/3591254322 >> 172.21.15.73:0/3591254322 conn(0x7fba440cb310 msgr2=0x7fba440cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 -- 172.21.15.73:0/3591254322 shutdown_connections 2023-11-28T04:24:23.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.768+0000 7fba48f3e700 1 -- 172.21.15.73:0/3591254322 wait complete. 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.769+0000 7fba48f3e700 1 Processor -- start 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.769+0000 7fba48f3e700 1 -- start start 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba44098d10 0x7fba440ad560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba440c6190 0x7fba440aae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba4414c770 con 0x7fba44158320 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba44156ba0 con 0x7fba44098d10 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba48f3e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba44156e80 con 0x7fba440c6190 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53230/0 (socket says 172.21.15.73:53230) 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 -- 172.21.15.73:0/2678088520 learned_addr learned my addr 172.21.15.73:0/2678088520 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba41d9b700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba440c6190 0x7fba440aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba4259c700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba44098d10 0x7fba440ad560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba440c6190 msgr2=0x7fba440aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba440c6190 0x7fba440aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba44098d10 msgr2=0x7fba440ad560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba44098d10 0x7fba440ad560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.770+0000 7fba42d9d700 1 -- 172.21.15.73:0/2678088520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba2c0092a0 con 0x7fba44158320 2023-11-28T04:24:23.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba42d9d700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7fba3c009df0 tx=0x7fba3c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00caa0 con 0x7fba44158320 2023-11-28T04:24:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba3c00f040 con 0x7fba44158320 2023-11-28T04:24:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba44157160 con 0x7fba44158320 2023-11-28T04:24:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba3c00ce10 con 0x7fba44158320 2023-11-28T04:24:23.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba440a8d10 con 0x7fba44158320 2023-11-28T04:24:23.776 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:23.771+0000 7fb51f3a6680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.772+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba4406c860 con 0x7fba44158320 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.772+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 85) v1 ==== 75560+0+0 (secure 0 0 0) 0x7fba3c013610 con 0x7fba44158320 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.772+0000 7fba3a7fc700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 0x7fba24061730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.773+0000 7fba4259c700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 msgr2=0x7fba24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.773+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(184..184 src has 1..184) v4 ==== 6608+0+0 (secure 0 0 0) 0x7fba3c07acc0 con 0x7fba44158320 2023-11-28T04:24:23.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.773+0000 7fba4259c700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 0x7fba24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:23.778 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.777+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fba3c04bc30 con 0x7fba44158320 2023-11-28T04:24:23.936 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:23.935+0000 7fec53d20680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:24:23.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.974+0000 7fba4259c700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 msgr2=0x7fba24061730 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:23.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.974+0000 7fba4259c700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 0x7fba24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:23.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:23.989+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fba440c66c0 con 0x7fba44158320 2023-11-28T04:24:24.008 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:24.007+0000 7fec53d20680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:24:24.071 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.070+0000 7fb51f3a6680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:24:24.143 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.142+0000 7fb51f3a6680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:24:24.165 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:24:24.166 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:24:24.166 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:24:24.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.165+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 86) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fba3c07a030 con 0x7fba44158320 2023-11-28T04:24:24.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.165+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 msgr2=0x7fba24061730 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:24.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.165+0000 7fba3a7fc700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 0x7fba24061730 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:24.181 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:24:24.182 INFO:tasks.ceph.mds.a.smithi073.stderr: -3> 2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:24:24.182 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T04:24:24.164+0000 7f7f95d46700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:24:24.202 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:24:24.222 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb49cb3e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb499b38700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb49cb3e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb49cb3e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb499b38700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb49cb3e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb499b38700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb49cb3e700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb499b38700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.223 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:24.221+0000 7fb499b38700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:24.680 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T04:24:24.679+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T04:24:24.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.685+0000 7fba3a7fc700 1 -- 172.21.15.73:0/2678088520 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v65) v1 ==== 161+0+0 (secure 0 0 0) 0x7fba3c06e0e0 con 0x7fba44158320 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 msgr2=0x7fba440ab310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 rx=0x7fba3c009df0 tx=0x7fba3c00b040).stop 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 shutdown_connections 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fba2405f270 0x7fba24061730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fba44158320 0x7fba440ab310 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fba44098d10 0x7fba440ad560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 --2- 172.21.15.73:0/2678088520 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fba440c6190 0x7fba440aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 >> 172.21.15.73:0/2678088520 conn(0x7fba440cb310 msgr2=0x7fba44063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 shutdown_connections 2023-11-28T04:24:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.687+0000 7fba48f3e700 1 -- 172.21.15.73:0/2678088520 wait complete. 2023-11-28T04:24:24.690 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T04:24:24.701 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T04:24:24.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.855+0000 7fed00b7d700 1 Processor -- start 2023-11-28T04:24:24.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.855+0000 7fed00b7d700 1 -- start start 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fed00b7d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecfc158a30 0x7fecfc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fed00b7d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecfc0c72d0 con 0x7fecfc098d10 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fed00b7d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecfc064570 con 0x7fecfc0c6190 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fed00b7d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecfc154bc0 con 0x7fecfc158a30 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fecfad9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fecfc158a30 0x7fecfc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fecfa59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fecfc098d10 0x7fecfc063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55764/0 (socket says 172.21.15.73:55764) 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fecfa59c700 1 -- 172.21.15.73:0/3506818337 learned_addr learned my addr 172.21.15.73:0/3506818337 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fecf959a700 1 -- 172.21.15.73:0/3506818337 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3792238410 0 0) 0x7fecfc0c72d0 con 0x7fecfc098d10 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.856+0000 7fecf959a700 1 -- 172.21.15.73:0/3506818337 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece40033f0 con 0x7fecfc098d10 2023-11-28T04:24:24.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecfad9d700 1 -- 172.21.15.73:0/3506818337 >> v1:172.21.15.150:6789/0 conn(0x7fecfc0c6190 legacy=0x7fecfc158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecfad9d700 1 -- 172.21.15.73:0/3506818337 >> v1:172.21.15.73:6789/0 conn(0x7fecfc098d10 legacy=0x7fecfc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecfad9d700 1 -- 172.21.15.73:0/3506818337 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0c66c0 con 0x7fecfc158a30 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecfad9d700 1 --2- 172.21.15.73:0/3506818337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc158a30 0x7fecfc0c7760 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fecf4009760 tx=0x7fecf4009060).ready entity=mon.0 client_cookie=9af1e65012c9ed9a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecf959a700 1 -- 172.21.15.73:0/3506818337 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf400f510 con 0x7fecfc158a30 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecf959a700 1 -- 172.21.15.73:0/3506818337 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf400f670 con 0x7fecfc158a30 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fecf959a700 1 -- 172.21.15.73:0/3506818337 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf400f9f0 con 0x7fecfc158a30 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 -- 172.21.15.73:0/3506818337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc158a30 msgr2=0x7fecfc0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 --2- 172.21.15.73:0/3506818337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc158a30 0x7fecfc0c7760 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 rx=0x7fecf4009760 tx=0x7fecf4009060).stop 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 -- 172.21.15.73:0/3506818337 shutdown_connections 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 --2- 172.21.15.73:0/3506818337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc158a30 0x7fecfc0c7760 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 -- 172.21.15.73:0/3506818337 >> 172.21.15.73:0/3506818337 conn(0x7fecfc0cb310 msgr2=0x7fecfc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:24.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 -- 172.21.15.73:0/3506818337 shutdown_connections 2023-11-28T04:24:24.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.857+0000 7fed00b7d700 1 -- 172.21.15.73:0/3506818337 wait complete. 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.858+0000 7fed00b7d700 1 Processor -- start 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.858+0000 7fed00b7d700 1 -- start start 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 0x7fecfc1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 0x7fecfc0b0370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:24.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0c2ce0 con 0x7fecfc0c6190 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc1529c0 con 0x7fecfc098d10 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fed00b7d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0afbc0 con 0x7fecfc158a30 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fecf9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fecf9d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53336/0 (socket says 172.21.15.73:53336) 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fecf9d9b700 1 -- 172.21.15.73:0/1006686995 learned_addr learned my addr 172.21.15.73:0/1006686995 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fecfa59c700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 0x7fecfc1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.859+0000 7fecfad9d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 0x7fecfc0b0370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 -- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 msgr2=0x7fecfc0b0370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 0x7fecfc0b0370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 -- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 msgr2=0x7fecfc1524d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 0x7fecfc1524d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 -- 172.21.15.73:0/1006686995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecf40092a0 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecfa59c700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 0x7fecfc1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecfad9d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 0x7fecfc0b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecf9d9b700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fecf000c4c0 tx=0x7fecf0007dd0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0007020 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf0007180 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf0007520 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc0c23b0 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.860+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc0c2970 con 0x7fecfc0c6190 2023-11-28T04:24:24.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.861+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc06c860 con 0x7fecfc0c6190 2023-11-28T04:24:24.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.861+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 86) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fecf00068b0 con 0x7fecfc0c6190 2023-11-28T04:24:24.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.861+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6924+0+0 (secure 0 0 0) 0x7fecf0079740 con 0x7fecfc0c6190 2023-11-28T04:24:24.866 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:24.865+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fecf0047af0 con 0x7fecfc0c6190 2023-11-28T04:24:25.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.069+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fecfc0c69a0 con 0x7fecfc0c6190 2023-11-28T04:24:25.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.165+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 87) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fecf00773c0 con 0x7fecfc0c6190 2023-11-28T04:24:25.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.165+0000 7fecea7fc700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 0x7fecdc067e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.166+0000 7fecfa59c700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 0x7fecdc067e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.167+0000 7fecfa59c700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 0x7fecdc067e30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fecec005ad0 tx=0x7fecec009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:25.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.688+0000 7fecea7fc700 1 -- 172.21.15.73:0/1006686995 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v66) v1 ==== 104+0+0 (secure 0 0 0) 0x7fecf0071ba0 con 0x7fecfc0c6190 2023-11-28T04:24:25.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 msgr2=0x7fecdc067e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 0x7fecdc067e30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fecec005ad0 tx=0x7fecec009040).stop 2023-11-28T04:24:25.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 msgr2=0x7fecfc0af1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 rx=0x7fecf000c4c0 tx=0x7fecf0007dd0).stop 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 shutdown_connections 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fecdc0302f0 0x7fecdc067e30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fecfc158a30 0x7fecfc0b0370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fecfc098d10 0x7fecfc1524d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 --2- 172.21.15.73:0/1006686995 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fecfc0c6190 0x7fecfc0af1e0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.690+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 >> 172.21.15.73:0/1006686995 conn(0x7fecfc0cb310 msgr2=0x7fecfc0a8860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.691+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 shutdown_connections 2023-11-28T04:24:25.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.691+0000 7fed00b7d700 1 -- 172.21.15.73:0/1006686995 wait complete. 2023-11-28T04:24:25.704 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T04:24:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.819+0000 7f71f98bf700 1 Processor -- start 2023-11-28T04:24:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.819+0000 7f71f98bf700 1 -- start start 2023-11-28T04:24:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.820+0000 7f71f98bf700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f71f4154610 0x7f71f41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.820+0000 7f71f98bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.820+0000 7f71f98bf700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f4159210 con 0x7f71f40aa1c0 2023-11-28T04:24:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.820+0000 7f71f98bf700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f40c0c70 con 0x7f71f4154610 2023-11-28T04:24:25.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.820+0000 7f71f98bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c2110 con 0x7f71f40ab3b0 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f71f4154610 0x7f71f41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f27fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f71f4154610 0x7f71f41582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:54032/0 (socket says 172.21.15.73:54032) 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f27fc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:39390/0 (socket says 172.21.15.73:39390) 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 -- 172.21.15.73:0/3694206841 learned_addr learned my addr 172.21.15.73:0/3694206841 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f37fe700 1 --1- 172.21.15.73:0/3694206841 >> v1:172.21.15.150:6789/0 conn(0x7f71f40aa1c0 0x7f71f4063550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45710/0 (socket says 172.21.15.73:45710) 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 -- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 msgr2=0x7f71f40c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 -- 172.21.15.73:0/3694206841 >> v1:172.21.15.150:6789/0 conn(0x7f71f40aa1c0 legacy=0x7f71f4063550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 -- 172.21.15.73:0/3694206841 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f4158970 con 0x7f71f4154610 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f27fc700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:25.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f2ffd700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f4154610 0x7f71f41582e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f71e4006fd0 tx=0x7f71e40090f0).ready entity=mon.1 client_cookie=9ddbae878bd278c8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f1ffb700 1 -- 172.21.15.73:0/3694206841 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400b580 con 0x7f71f4154610 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f1ffb700 1 -- 172.21.15.73:0/3694206841 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400b6e0 con 0x7f71f4154610 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.821+0000 7f71f1ffb700 1 -- 172.21.15.73:0/3694206841 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400ba60 con 0x7f71f4154610 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 -- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f4154610 msgr2=0x7f71f41582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f4154610 0x7f71f41582e0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 rx=0x7f71e4006fd0 tx=0x7f71e40090f0).stop 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 -- 172.21.15.73:0/3694206841 shutdown_connections 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f4154610 0x7f71f41582e0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 --2- 172.21.15.73:0/3694206841 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f40ab3b0 0x7f71f40c80c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 -- 172.21.15.73:0/3694206841 >> 172.21.15.73:0/3694206841 conn(0x7f71f40655a0 msgr2=0x7f71f4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 -- 172.21.15.73:0/3694206841 shutdown_connections 2023-11-28T04:24:25.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.822+0000 7f71f98bf700 1 -- 172.21.15.73:0/3694206841 wait complete. 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.823+0000 7f71f98bf700 1 Processor -- start 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.823+0000 7f71f98bf700 1 -- start start 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f4154610 0x7f71f40c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 0x7f71f40ad040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4168810 con 0x7f71f40aa1c0 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c5390 con 0x7f71f40acbe0 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f98bf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f4164c20 con 0x7f71f4154610 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53340/0 (socket says 172.21.15.73:53340) 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 -- 172.21.15.73:0/1654478040 learned_addr learned my addr 172.21.15.73:0/1654478040 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f37fe700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 0x7f71f40ad040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f27fc700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f4154610 0x7f71f40c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 -- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f4154610 msgr2=0x7f71f40c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f4154610 0x7f71f40c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 -- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 msgr2=0x7f71f40ad040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 0x7f71f40ad040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:25.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 -- 172.21.15.73:0/1654478040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e80092a0 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f37fe700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 0x7f71f40ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71f2ffd700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f71e4009fd0 tx=0x7f71e4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.824+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e400b850 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.825+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e400b9b0 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.825+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e4009370 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.825+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e401c670 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.825+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f4168d00 con 0x7f71f40aa1c0 2023-11-28T04:24:25.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f406c860 con 0x7f71f40aa1c0 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 87) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f71e401ca60 con 0x7f71f40aa1c0 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71daffd700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 0x7f71e00614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(186..186 src has 1..186) v4 ==== 6924+0+0 (secure 0 0 0) 0x7f71e407bf30 con 0x7f71f40aa1c0 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71f27fc700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 0x7f71e00614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.826+0000 7f71f27fc700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 0x7f71e00614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f71e8009270 tx=0x7f71e8008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:25.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.830+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f71e40aaaf0 con 0x7f71f40aa1c0 2023-11-28T04:24:25.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:25.992+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f71f4157ff0 con 0x7f71f40aa1c0 2023-11-28T04:24:26.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.694+0000 7f71daffd700 1 -- 172.21.15.73:0/1654478040 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v187) v1 ==== 181+0+0 (secure 0 0 0) 0x7f71e402a030 con 0x7f71f40aa1c0 2023-11-28T04:24:26.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.697+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 msgr2=0x7f71e00614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:26.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.697+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 0x7f71e00614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f71e8009270 tx=0x7f71e8008040).stop 2023-11-28T04:24:26.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.697+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 msgr2=0x7f71f4168320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.697+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 rx=0x7f71e4009fd0 tx=0x7f71e4017000).stop 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 shutdown_connections 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f71e005f030 0x7f71e00614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f71f40acbe0 0x7f71f40ad040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f71f40aa1c0 0x7f71f4168320 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 --2- 172.21.15.73:0/1654478040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f71f4154610 0x7f71f40c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 >> 172.21.15.73:0/1654478040 conn(0x7f71f40655a0 msgr2=0x7f71f40ca610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:26.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 shutdown_connections 2023-11-28T04:24:26.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.698+0000 7f71f98bf700 1 -- 172.21.15.73:0/1654478040 wait complete. 2023-11-28T04:24:26.700 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T04:24:26.712 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T04:24:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.828+0000 7fc2188e4700 1 Processor -- start 2023-11-28T04:24:26.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.828+0000 7fc2188e4700 1 -- start start 2023-11-28T04:24:26.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2188e4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 0x7fc2140640b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2188e4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2140c57e0 0x7fc2140be880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2188e4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2140bf220 con 0x7fc2140bed70 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2188e4700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc214065680 con 0x7fc2140c57e0 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2188e4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2140c6910 con 0x7fc2140c6530 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2140c57e0 0x7fc2140be880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2140c57e0 0x7fc2140be880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53344/0 (socket says 172.21.15.73:53344) 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 -- 172.21.15.73:0/1000203588 learned_addr learned my addr 172.21.15.73:0/1000203588 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2137fe700 1 --2- 172.21.15.73:0/1000203588 >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 0x7fc2140640b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 -- 172.21.15.73:0/1000203588 >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 msgr2=0x7fc2140640b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 --2- 172.21.15.73:0/1000203588 >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 0x7fc2140640b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 -- 172.21.15.73:0/1000203588 >> v1:172.21.15.150:6789/0 conn(0x7fc2140bed70 legacy=0x7fc2141549a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:26.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 -- 172.21.15.73:0/1000203588 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc214064a00 con 0x7fc2140c57e0 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2137fe700 1 --2- 172.21.15.73:0/1000203588 >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 0x7fc2140640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc212ffd700 1 --2- 172.21.15.73:0/1000203588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c57e0 0x7fc2140be880 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7fc214063670 tx=0x7fc2080090f0).ready entity=mon.0 client_cookie=edd4716f75789ed2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2127fc700 1 -- 172.21.15.73:0/1000203588 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800b5a0 con 0x7fc2140c57e0 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2127fc700 1 -- 172.21.15.73:0/1000203588 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20800b700 con 0x7fc2140c57e0 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.829+0000 7fc2127fc700 1 -- 172.21.15.73:0/1000203588 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20800ba80 con 0x7fc2140c57e0 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 -- 172.21.15.73:0/1000203588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c57e0 msgr2=0x7fc2140be880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 --2- 172.21.15.73:0/1000203588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c57e0 0x7fc2140be880 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 rx=0x7fc214063670 tx=0x7fc2080090f0).stop 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 -- 172.21.15.73:0/1000203588 shutdown_connections 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 --2- 172.21.15.73:0/1000203588 >> v2:172.21.15.150:3300/0 conn(0x7fc2140c6530 0x7fc2140640b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 --2- 172.21.15.73:0/1000203588 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c57e0 0x7fc2140be880 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:26.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 -- 172.21.15.73:0/1000203588 >> 172.21.15.73:0/1000203588 conn(0x7fc214092520 msgr2=0x7fc214062ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 -- 172.21.15.73:0/1000203588 shutdown_connections 2023-11-28T04:24:26.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.830+0000 7fc2188e4700 1 -- 172.21.15.73:0/1000203588 wait complete. 2023-11-28T04:24:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.831+0000 7fc2188e4700 1 Processor -- start 2023-11-28T04:24:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.831+0000 7fc2188e4700 1 -- start start 2023-11-28T04:24:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2140bed70 0x7fc214153110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2140c57e0 0x7fc2140c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140c0c30 con 0x7fc2140c6530 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc214159d30 con 0x7fc2140c57e0 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc2188e4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2140c0340 con 0x7fc2140bed70 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53356/0 (socket says 172.21.15.73:53356) 2023-11-28T04:24:26.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 -- 172.21.15.73:0/3142554224 learned_addr learned my addr 172.21.15.73:0/3142554224 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 -- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2140bed70 msgr2=0x7fc214153110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2140bed70 0x7fc214153110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 -- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2140c57e0 msgr2=0x7fc2140c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2140c57e0 0x7fc2140c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 -- 172.21.15.73:0/3142554224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc208009370 con 0x7fc2140c6530 2023-11-28T04:24:26.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc213fff700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7fc20400c490 tx=0x7fc204007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:26.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.832+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc204002ea0 con 0x7fc2140c6530 2023-11-28T04:24:26.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.833+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc20400f470 con 0x7fc2140c6530 2023-11-28T04:24:26.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.833+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2140c05e0 con 0x7fc2140c6530 2023-11-28T04:24:26.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.833+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc20400f810 con 0x7fc2140c6530 2023-11-28T04:24:26.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.833+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc214156f00 con 0x7fc2140c6530 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.834+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc21406c7c0 con 0x7fc2140c6530 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.834+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 87) v1 ==== 75340+0+0 (secure 0 0 0) 0x7fc20400fc40 con 0x7fc2140c6530 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.834+0000 7fc1fb7fe700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 0x7fc2000614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.834+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 6457+0+0 (secure 0 0 0) 0x7fc204079590 con 0x7fc2140c6530 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.834+0000 7fc2137fe700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 0x7fc2000614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:26.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.835+0000 7fc2137fe700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 0x7fc2000614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc20c007f40 tx=0x7fc20c007430).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:26.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:26.838+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc204047af0 con 0x7fc2140c6530 2023-11-28T04:24:27.001 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.000+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc2140c0d70 con 0x7fc2140c6530 2023-11-28T04:24:27.174 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.173+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc2040773e0 con 0x7fc2140c6530 2023-11-28T04:24:27.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.250+0000 7fc1fb7fe700 1 -- 172.21.15.73:0/3142554224 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v188) v1 ==== 169+0+0 (secure 0 0 0) 0x7fc2040751a0 con 0x7fc2140c6530 2023-11-28T04:24:27.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 msgr2=0x7fc2000614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 0x7fc2000614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fc20c007f40 tx=0x7fc20c007430).stop 2023-11-28T04:24:27.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 msgr2=0x7fc214159250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 rx=0x7fc20400c490 tx=0x7fc204007de0).stop 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 shutdown_connections 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc20005f030 0x7fc2000614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2140c6530 0x7fc214159250 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2140bed70 0x7fc214153110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 --2- 172.21.15.73:0/3142554224 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2140c57e0 0x7fc2140c7ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 >> 172.21.15.73:0/3142554224 conn(0x7fc214092520 msgr2=0x7fc2140abe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 shutdown_connections 2023-11-28T04:24:27.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.253+0000 7fc2188e4700 1 -- 172.21.15.73:0/3142554224 wait complete. 2023-11-28T04:24:27.256 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T04:24:27.267 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T04:24:27.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.380+0000 7f303e608700 1 Processor -- start 2023-11-28T04:24:27.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.380+0000 7f303e608700 1 -- start start 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f303e608700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3038098d10 0x7f30380c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f303e608700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30380c72d0 con 0x7f30380c6190 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f303e608700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3038064570 con 0x7f3038158320 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f303e608700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3038154bc0 con 0x7f3038098d10 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f30377fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f30380c6190 0x7f3038063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55778/0 (socket says 172.21.15.73:55778) 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.381+0000 7f30377fe700 1 -- 172.21.15.73:0/2738402609 learned_addr learned my addr 172.21.15.73:0/2738402609 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3037fff700 1 --2- 172.21.15.73:0/2738402609 >> v2:172.21.15.150:3300/0 conn(0x7f3038098d10 0x7f30380c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038195213 0 0) 0x7f30380c72d0 con 0x7f30380c6190 2023-11-28T04:24:27.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30200033f0 con 0x7f30380c6190 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1096994132 0 0) 0x7f30200033f0 con 0x7f30380c6190 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 >> v2:172.21.15.150:3300/0 conn(0x7f3038098d10 msgr2=0x7f30380c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 --2- 172.21.15.73:0/2738402609 >> v2:172.21.15.150:3300/0 conn(0x7f3038098d10 0x7f30380c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 >> v1:172.21.15.150:6789/0 conn(0x7f3038158320 legacy=0x7f30380a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380c66c0 con 0x7f30380c6190 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f302c001470 con 0x7f30380c6190 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 >> v1:172.21.15.73:6789/0 conn(0x7f30380c6190 legacy=0x7f3038063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30200038e0 0x7f3020003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3020004690 0x7f3020004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f302000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3038154bc0 con 0x7f3020005000 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302000beb0 con 0x7f3020004690 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302000c070 con 0x7f30200038e0 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f302000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f30377fe700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3020004690 0x7f3020004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f3037fff700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30200038e0 0x7f3020003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.393 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 -- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30200038e0 msgr2=0x7f3020003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:27.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f30200038e0 0x7f3020003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 -- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3020004690 msgr2=0x7f3020004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:27.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3020004690 0x7f3020004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.382+0000 7f303cba5700 1 -- 172.21.15.73:0/2738402609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30200033f0 con 0x7f3020005000 2023-11-28T04:24:27.394 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303cba5700 1 -- 172.21.15.73:0/2738402609 reap_dead start 2023-11-28T04:24:27.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303cba5700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f302000b8e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7f3030001e30 tx=0x7f3030007f30).ready entity=mon.0 client_cookie=cfa357649af2512 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3030006a20 con 0x7f3020005000 2023-11-28T04:24:27.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3030006b80 con 0x7f3020005000 2023-11-28T04:24:27.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f3036ffd700 1 -- 172.21.15.73:0/2738402609 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f303000c5d0 con 0x7f3020005000 2023-11-28T04:24:27.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 -- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 msgr2=0x7f302000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f302000b8e0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 rx=0x7f3030001e30 tx=0x7f3030007f30).stop 2023-11-28T04:24:27.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 -- 172.21.15.73:0/2738402609 shutdown_connections 2023-11-28T04:24:27.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 --2- 172.21.15.73:0/2738402609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f302000b8e0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 -- 172.21.15.73:0/2738402609 >> 172.21.15.73:0/2738402609 conn(0x7f30380cb310 msgr2=0x7f30380cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 -- 172.21.15.73:0/2738402609 shutdown_connections 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.383+0000 7f303e608700 1 -- 172.21.15.73:0/2738402609 wait complete. 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.384+0000 7f303e608700 1 Processor -- start 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 -- start start 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 0x7f30381588f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3038158de0 0x7f30380ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380ab560 con 0x7f3020005000 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30380ab840 con 0x7f30380aad70 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303e608700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30380abb20 con 0x7f3038158de0 2023-11-28T04:24:27.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f3037fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f3037fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53380/0 (socket says 172.21.15.73:53380) 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f3037fff700 1 -- 172.21.15.73:0/776328340 learned_addr learned my addr 172.21.15.73:0/776328340 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f30377fe700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 0x7f30381588f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.385+0000 7f303cba5700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3038158de0 0x7f30380ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 -- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3038158de0 msgr2=0x7f30380ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3038158de0 0x7f30380ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 -- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 msgr2=0x7f30381588f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 0x7f30381588f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 -- 172.21.15.73:0/776328340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30380abe00 con 0x7f3020005000 2023-11-28T04:24:27.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f30377fe700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 0x7f30381588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f3037fff700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7f30280119c0 tx=0x7f3028009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800f9d0 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302800fb30 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302800feb0 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30380ac0e0 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.386+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30380acc00 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.387+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f303806c860 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.387+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3028014650 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.388+0000 7f301ffff700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 0x7f3024061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.388+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f302807a7b0 con 0x7f3020005000 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.388+0000 7f30377fe700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 0x7f3024061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.388+0000 7f30377fe700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 0x7f3024061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f302c0015a0 tx=0x7f302c009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.392+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f302804ba40 con 0x7f3020005000 2023-11-28T04:24:27.564 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.563+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f30380c69a0 con 0x7f3020005000 2023-11-28T04:24:27.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.566+0000 7f301ffff700 1 -- 172.21.15.73:0/776328340 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v36) v1 ==== 120+0+0 (secure 0 0 0) 0x7f302804f060 con 0x7f3020005000 2023-11-28T04:24:27.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.568+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 msgr2=0x7f3024061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.568+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 0x7f3024061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f302c0015a0 tx=0x7f302c009040).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.568+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 msgr2=0x7f30380c8690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.568+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 rx=0x7f30280119c0 tx=0x7f3028009060).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 shutdown_connections 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f302405f150 0x7f3024061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3038158de0 0x7f30380ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3020005000 0x7f30380c8690 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 --2- 172.21.15.73:0/776328340 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f30380aad70 0x7f30381588f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 >> 172.21.15.73:0/776328340 conn(0x7f30380cb310 msgr2=0x7f30381543c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 shutdown_connections 2023-11-28T04:24:27.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.569+0000 7f303e608700 1 -- 172.21.15.73:0/776328340 wait complete. 2023-11-28T04:24:27.587 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T04:24:27.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.700+0000 7f8aadbaf700 1 Processor -- start 2023-11-28T04:24:27.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.700+0000 7f8aadbaf700 1 -- start start 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aa8069d10 0x7f8aa806a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8aa8151a20 0x7f8aa80be1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa80bd1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa80c3470 con 0x7f8aa8069d10 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa8151ea0 con 0x7f8aa8151a20 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aadbaf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8152180 con 0x7f8aa80be6a0 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aa77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aa8069d10 0x7f8aa806a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aa77fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8aa8069d10 0x7f8aa806a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53382/0 (socket says 172.21.15.73:53382) 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.701+0000 7f8aa77fe700 1 -- 172.21.15.73:0/3826997027 learned_addr learned my addr 172.21.15.73:0/3826997027 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 -- 172.21.15.73:0/3826997027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 msgr2=0x7f8aa80bd1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/3826997027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa80bd1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 -- 172.21.15.73:0/3826997027 >> v2:172.21.15.150:3300/0 conn(0x7f8aa8151a20 msgr2=0x7f8aa80be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/3826997027 >> v2:172.21.15.150:3300/0 conn(0x7f8aa8151a20 0x7f8aa80be1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 -- 172.21.15.73:0/3826997027 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8152460 con 0x7f8aa8069d10 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/3826997027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8069d10 0x7f8aa806a0f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f8aa8069370 tx=0x7f8a980090f0).ready entity=mon.0 client_cookie=3872d27b12e685c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa6ffd700 1 -- 172.21.15.73:0/3826997027 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800b600 con 0x7f8aa8069d10 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa6ffd700 1 -- 172.21.15.73:0/3826997027 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a9800b760 con 0x7f8aa8069d10 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aa6ffd700 1 -- 172.21.15.73:0/3826997027 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9800bae0 con 0x7f8aa8069d10 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 -- 172.21.15.73:0/3826997027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8069d10 msgr2=0x7f8aa806a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/3826997027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8069d10 0x7f8aa806a0f0 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 rx=0x7f8aa8069370 tx=0x7f8a980090f0).stop 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 -- 172.21.15.73:0/3826997027 shutdown_connections 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/3826997027 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8069d10 0x7f8aa806a0f0 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/3826997027 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa80bd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/3826997027 >> v2:172.21.15.150:3300/0 conn(0x7f8aa8151a20 0x7f8aa80be1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 -- 172.21.15.73:0/3826997027 >> 172.21.15.73:0/3826997027 conn(0x7f8aa80642d0 msgr2=0x7f8aa80c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:27.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.702+0000 7f8aadbaf700 1 -- 172.21.15.73:0/3826997027 shutdown_connections 2023-11-28T04:24:27.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.703+0000 7f8aadbaf700 1 -- 172.21.15.73:0/3826997027 wait complete. 2023-11-28T04:24:27.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 Processor -- start 2023-11-28T04:24:27.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 -- start start 2023-11-28T04:24:27.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aa8069d10 0x7f8aa8145590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa813fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa81441c0 con 0x7f8aa8151a20 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.704+0000 7f8aadbaf700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa80c42f0 con 0x7f8aa8069d10 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aadbaf700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8146800 con 0x7f8aa80be6a0 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53392/0 (socket says 172.21.15.73:53392) 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 -- 172.21.15.73:0/571591291 learned_addr learned my addr 172.21.15.73:0/571591291 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aa8069d10 0x7f8aa8145590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8a9effd700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa813fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 -- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 msgr2=0x7f8aa813fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa813fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 -- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aa8069d10 msgr2=0x7f8aa8145590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aa8069d10 0x7f8aa8145590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 -- 172.21.15.73:0/571591291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a98009370 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8a9effd700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aa7fff700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7f8aa000bb60 tx=0x7f8aa0009810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000c8c0 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aa000ca40 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aa000cbf0 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa8145a80 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.705+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa80c4580 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.706+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa806c860 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.706+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8aa0012020 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.707+0000 7f8a9ffff700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 0x7f8a88061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.707+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f8aa007a070 con 0x7f8aa8151a20 2023-11-28T04:24:27.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.707+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 0x7f8a88061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:27.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.707+0000 7f8aa77fe700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 0x7f8a88061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8a98009fd0 tx=0x7f8a98017000).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:27.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.711+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8aa004b3c0 con 0x7f8aa8151a20 2023-11-28T04:24:27.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.878+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f8aa806a620 con 0x7f8aa8151a20 2023-11-28T04:24:27.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.882+0000 7f8a9ffff700 1 -- 172.21.15.73:0/571591291 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v37) v1 ==== 104+0+0 (secure 0 0 0) 0x7f8aa0009910 con 0x7f8aa8151a20 2023-11-28T04:24:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 msgr2=0x7f8a88061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 0x7f8a88061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f8a98009fd0 tx=0x7f8a98017000).stop 2023-11-28T04:24:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 msgr2=0x7f8aa8146f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:27.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 rx=0x7f8aa000bb60 tx=0x7f8aa0009810).stop 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 shutdown_connections 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8a8805f150 0x7f8a88061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8aa8151a20 0x7f8aa8146f10 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8aa8069d10 0x7f8aa8145590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 --2- 172.21.15.73:0/571591291 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8aa80be6a0 0x7f8aa813fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:27.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 >> 172.21.15.73:0/571591291 conn(0x7f8aa80642d0 msgr2=0x7f8aa8065c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 shutdown_connections 2023-11-28T04:24:27.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:27.884+0000 7f8aadbaf700 1 -- 172.21.15.73:0/571591291 wait complete. 2023-11-28T04:24:27.897 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-11-28T04:24:28.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.009+0000 7f4685826700 1 Processor -- start 2023-11-28T04:24:28.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.009+0000 7f4685826700 1 -- start start 2023-11-28T04:24:28.013 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f4685826700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46800caab0 0x7f46800aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f4685826700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f46800ca180 0x7f468015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f4685826700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46800a9a90 con 0x7f46800cae90 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f4685826700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4680154ce0 con 0x7f46800caab0 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f4685826700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46800a8e80 con 0x7f46800ca180 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46800caab0 0x7f46800aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f46800caab0 0x7f46800aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53394/0 (socket says 172.21.15.73:53394) 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 -- 172.21.15.73:0/939115135 learned_addr learned my addr 172.21.15.73:0/939115135 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467f7fe700 1 --2- 172.21.15.73:0/939115135 >> v2:172.21.15.150:3300/0 conn(0x7f46800ca180 0x7f468015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.014 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 -- 172.21.15.73:0/939115135 >> v2:172.21.15.150:3300/0 conn(0x7f46800ca180 msgr2=0x7f468015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 --2- 172.21.15.73:0/939115135 >> v2:172.21.15.150:3300/0 conn(0x7f46800ca180 0x7f468015e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 -- 172.21.15.73:0/939115135 >> v1:172.21.15.73:6789/0 conn(0x7f46800cae90 legacy=0x7f46800c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 -- 172.21.15.73:0/939115135 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46800a9160 con 0x7f46800caab0 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467effd700 1 --2- 172.21.15.73:0/939115135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f46800aa470 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7f46800a9a00 tx=0x7f4678009840).ready entity=mon.0 client_cookie=99ec89df08ef60e4 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.010+0000 7f467dffb700 1 -- 172.21.15.73:0/939115135 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800e070 con 0x7f46800caab0 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f467dffb700 1 -- 172.21.15.73:0/939115135 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f467800bad0 con 0x7f46800caab0 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f467dffb700 1 -- 172.21.15.73:0/939115135 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f467800be50 con 0x7f46800caab0 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 -- 172.21.15.73:0/939115135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 msgr2=0x7f46800aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 --2- 172.21.15.73:0/939115135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f46800aa470 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 rx=0x7f46800a9a00 tx=0x7f4678009840).stop 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 -- 172.21.15.73:0/939115135 shutdown_connections 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 --2- 172.21.15.73:0/939115135 >> v2:172.21.15.150:3300/0 conn(0x7f46800ca180 0x7f468015e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.015 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 --2- 172.21.15.73:0/939115135 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f46800aa470 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 -- 172.21.15.73:0/939115135 >> 172.21.15.73:0/939115135 conn(0x7f4680094920 msgr2=0x7f468006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:28.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 -- 172.21.15.73:0/939115135 shutdown_connections 2023-11-28T04:24:28.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.011+0000 7f4685826700 1 -- 172.21.15.73:0/939115135 wait complete. 2023-11-28T04:24:28.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.012+0000 7f4685826700 1 Processor -- start 2023-11-28T04:24:28.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.012+0000 7f4685826700 1 -- start start 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46800ca180 0x7f4680155070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46800cae90 0x7f4680168a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4680168f10 con 0x7f46800caab0 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46800aa920 con 0x7f46800ca180 2023-11-28T04:24:28.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f4685826700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46800aaee0 con 0x7f46800cae90 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467e7fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53408/0 (socket says 172.21.15.73:53408) 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467e7fc700 1 -- 172.21.15.73:0/1047122983 learned_addr learned my addr 172.21.15.73:0/1047122983 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467effd700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46800ca180 0x7f4680155070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467f7fe700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46800cae90 0x7f4680168a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.013+0000 7f467e7fc700 1 -- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46800cae90 msgr2=0x7f4680168a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f467e7fc700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46800cae90 0x7f4680168a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f467e7fc700 1 -- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46800ca180 msgr2=0x7f4680155070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f467e7fc700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46800ca180 0x7f4680155070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f467e7fc700 1 -- 172.21.15.73:0/1047122983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4678009ae0 con 0x7f46800caab0 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f467e7fc700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 rx=0x7f467400c440 tx=0x7f4674007db0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46740123f0 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4674001e90 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4680157c80 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4674012780 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.014+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46800aab70 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.015+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f468006c860 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.015+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4674012af0 con 0x7f46800caab0 2023-11-28T04:24:28.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.016+0000 7f4666ffd700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 0x7f466c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.016+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f4674079390 con 0x7f46800caab0 2023-11-28T04:24:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.016+0000 7f467effd700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 0x7f466c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.016+0000 7f467effd700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 0x7f466c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f46801569f0 tx=0x7f4678009840).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.020+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4674047af0 con 0x7f46800caab0 2023-11-28T04:24:28.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.247+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f46800a9440 con 0x7f46800caab0 2023-11-28T04:24:28.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.695+0000 7f4666ffd700 1 -- 172.21.15.73:0/1047122983 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2929) v1 ==== 165+0+0 (secure 0 0 0) 0x7f467404e110 con 0x7f46800caab0 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 msgr2=0x7f466c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 0x7f466c061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f46801569f0 tx=0x7f4678009840).stop 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 msgr2=0x7f4680168530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 rx=0x7f467400c440 tx=0x7f4674007db0).stop 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 shutdown_connections 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f466c05f150 0x7f466c061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f46800cae90 0x7f4680168a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f46800ca180 0x7f4680155070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 --2- 172.21.15.73:0/1047122983 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f46800caab0 0x7f4680168530 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 >> 172.21.15.73:0/1047122983 conn(0x7f4680094920 msgr2=0x7f46800ae5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:28.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 shutdown_connections 2023-11-28T04:24:28.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.697+0000 7f4685826700 1 -- 172.21.15.73:0/1047122983 wait complete. 2023-11-28T04:24:28.710 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-11-28T04:24:28.711 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-11-28T04:24:28.711 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-11-28T04:24:28.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-11-28T04:24:28.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.824+0000 7f1523efc700 1 Processor -- start 2023-11-28T04:24:28.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 -- start start 2023-11-28T04:24:28.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 0x7f151c0671e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f151c067980 0x7f151c067df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c0bd590 con 0x7f151c066e00 2023-11-28T04:24:28.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0a6090 con 0x7f151c067980 2023-11-28T04:24:28.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.825+0000 7f1523efc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0682e0 con 0x7f151c155e40 2023-11-28T04:24:28.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f151c067980 0x7f151c067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1521497700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 0x7f151c0671e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1521497700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 0x7f151c0671e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:39446/0 (socket says 172.21.15.73:39446) 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1521c98700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f151c066e00 0x7f151c0a5980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45752/0 (socket says 172.21.15.73:45752) 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f151c067980 0x7f151c067df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:54134/0 (socket says 172.21.15.73:54134) 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1521497700 1 -- 172.21.15.73:0/122278329 learned_addr learned my addr 172.21.15.73:0/122278329 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 -- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 msgr2=0x7f151c0671e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 --2- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 0x7f151c0671e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 -- 172.21.15.73:0/122278329 >> v1:172.21.15.150:6789/0 conn(0x7f151c066e00 legacy=0x7f151c0a5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1520c96700 1 -- 172.21.15.73:0/122278329 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 633152958 0 0) 0x7f151c0bd590 con 0x7f151c066e00 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 -- 172.21.15.73:0/122278329 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c156370 con 0x7f151c067980 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.826+0000 7f1522499700 1 --2- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c067980 0x7f151c067df0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f1514009e60 tx=0x7f1514009780).ready entity=mon.1 client_cookie=4c2f9ec0e401fb89 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1520c96700 1 -- 172.21.15.73:0/122278329 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f151400e070 con 0x7f151c067980 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1520c96700 1 -- 172.21.15.73:0/122278329 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f151400bb60 con 0x7f151c067980 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1520c96700 1 -- 172.21.15.73:0/122278329 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1514012410 con 0x7f151c067980 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 -- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c067980 msgr2=0x7f151c067df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 --2- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c067980 0x7f151c067df0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 rx=0x7f1514009e60 tx=0x7f1514009780).stop 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 -- 172.21.15.73:0/122278329 shutdown_connections 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 --2- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c067980 0x7f151c067df0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 --2- 172.21.15.73:0/122278329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c155e40 0x7f151c0671e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 -- 172.21.15.73:0/122278329 >> 172.21.15.73:0/122278329 conn(0x7f151c0c7fd0 msgr2=0x7f151c0c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:28.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 -- 172.21.15.73:0/122278329 shutdown_connections 2023-11-28T04:24:28.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.827+0000 7f1523efc700 1 -- 172.21.15.73:0/122278329 wait complete. 2023-11-28T04:24:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.828+0000 7f1523efc700 1 Processor -- start 2023-11-28T04:24:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.828+0000 7f1523efc700 1 -- start start 2023-11-28T04:24:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c067980 0x7f151c149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 0x7f151c0a7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0a8000 con 0x7f151c066e00 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0a82e0 con 0x7f151c155e40 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1523efc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0a85c0 con 0x7f151c067980 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1521c98700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1521c98700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53422/0 (socket says 172.21.15.73:53422) 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1521c98700 1 -- 172.21.15.73:0/3442503835 learned_addr learned my addr 172.21.15.73:0/3442503835 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:28.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.829+0000 7f1521497700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c067980 0x7f151c149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1522499700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 0x7f151c0a7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 -- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c067980 msgr2=0x7f151c149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c067980 0x7f151c149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 -- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 msgr2=0x7f151c0a7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 0x7f151c0a7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 -- 172.21.15.73:0/3442503835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009a20 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1522499700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 0x7f151c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1521c98700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f151800c560 tx=0x7f1518007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180072e0 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c0a88a0 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1518007440 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15180058f0 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.830+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c0aa230 con 0x7f151c066e00 2023-11-28T04:24:28.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.831+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c0a8dc0 con 0x7f151c066e00 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.831+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1518005ce0 con 0x7f151c066e00 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.832+0000 7f1511ffb700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 0x7f1500061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.832+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f15180799a0 con 0x7f151c066e00 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.832+0000 7f1521497700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 0x7f1500061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.832+0000 7f1521497700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 0x7f1500061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f151c068260 tx=0x7f150c0090f0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:28.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:28.836+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f151c0a8dc0 con 0x7f151c066e00 2023-11-28T04:24:29.066 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.065+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f151c06c860 con 0x7f151c066e00 2023-11-28T04:24:29.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.696+0000 7f1511ffb700 1 -- 172.21.15.73:0/3442503835 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2930) v1 ==== 166+0+0 (secure 0 0 0) 0x7f151804ad90 con 0x7f151c066e00 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 msgr2=0x7f1500061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 0x7f1500061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f151c068260 tx=0x7f150c0090f0).stop 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 msgr2=0x7f151c0aaf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 rx=0x7f151800c560 tx=0x7f1518007c90).stop 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 shutdown_connections 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f150005f150 0x7f1500061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f151c155e40 0x7f151c0a7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f151c066e00 0x7f151c0aaf60 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 --2- 172.21.15.73:0/3442503835 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f151c067980 0x7f151c149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 >> 172.21.15.73:0/3442503835 conn(0x7f151c0c7fd0 msgr2=0x7f151c0be6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 shutdown_connections 2023-11-28T04:24:29.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.699+0000 7f1523efc700 1 -- 172.21.15.73:0/3442503835 wait complete. 2023-11-28T04:24:29.712 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T04:24:29.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.827+0000 7f27525ab700 1 Processor -- start 2023-11-28T04:24:29.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.827+0000 7f27525ab700 1 -- start start 2023-11-28T04:24:29.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f27525ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f274c1553e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f27525ab700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c0c18b0 con 0x7f274c0c7170 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f27525ab700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274c0c0ca0 con 0x7f274c064220 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f27525ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c0f80 con 0x7f274c0c1d40 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f274bfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f274c0c7170 0x7f274c063b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55796/0 (socket says 172.21.15.73:55796) 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f274bfff700 1 -- 172.21.15.73:0/854976938 learned_addr learned my addr 172.21.15.73:0/854976938 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1015034902 0 0) 0x7f274c0c18b0 con 0x7f274c0c7170 2023-11-28T04:24:29.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f27340033f0 con 0x7f274c0c7170 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.828+0000 7f2750b48700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f274c1553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 362239176 0 0) 0x7f27340033f0 con 0x7f274c0c7170 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 msgr2=0x7f274c1553e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f274c1553e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 >> v1:172.21.15.150:6789/0 conn(0x7f274c064220 legacy=0x7f274c154cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c0c1260 con 0x7f274c0c7170 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f273c001470 con 0x7f274c0c7170 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 >> v1:172.21.15.73:6789/0 conn(0x7f274c0c7170 legacy=0x7f274c063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f2734003910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2734003ef0 0x7f2734004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2734004b80 0x7f2734005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0c0f80 con 0x7f2734004b80 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273400bb20 con 0x7f2734003ef0 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273400bce0 con 0x7f274c0c1d40 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2734004b80 0x7f2734005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274bfff700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f2734003910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274b7fe700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2734003ef0 0x7f2734004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 -- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 msgr2=0x7f2734003910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f274c0c1d40 0x7f2734003910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 -- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2734003ef0 msgr2=0x7f2734004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2734003ef0 0x7f2734004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 -- 172.21.15.73:0/854976938 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27340033f0 con 0x7f2734004b80 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f2750b48700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2734004b80 0x7f2734005000 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7f2744009fd0 tx=0x7f2744009730).ready entity=mon.0 client_cookie=d0f1b6ea967c4619 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:29.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744009a40 con 0x7f2734004b80 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2744007cc0 con 0x7f2734004b80 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.829+0000 7f274affd700 1 -- 172.21.15.73:0/854976938 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2744015660 con 0x7f2734004b80 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 -- 172.21.15.73:0/854976938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2734004b80 msgr2=0x7f2734005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 --2- 172.21.15.73:0/854976938 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2734004b80 0x7f2734005000 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 rx=0x7f2744009fd0 tx=0x7f2744009730).stop 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f2750b48700 1 -- 172.21.15.73:0/854976938 reap_dead start 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 -- 172.21.15.73:0/854976938 shutdown_connections 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 -- 172.21.15.73:0/854976938 >> 172.21.15.73:0/854976938 conn(0x7f274c0cb260 msgr2=0x7f274c0cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 -- 172.21.15.73:0/854976938 shutdown_connections 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.830+0000 7f27525ab700 1 -- 172.21.15.73:0/854976938 wait complete. 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.831+0000 7f27525ab700 1 Processor -- start 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.831+0000 7f27525ab700 1 -- start start 2023-11-28T04:24:29.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 0x7f274c0ca2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 0x7f274c1675e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c167ad0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f274c0b1230 con 0x7f274c152f40 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f27525ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f274c167d70 con 0x7f2734004b80 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53442/0 (socket says 172.21.15.73:53442) 2023-11-28T04:24:29.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 -- 172.21.15.73:0/2389659812 learned_addr learned my addr 172.21.15.73:0/2389659812 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:29.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 -- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 msgr2=0x7f274c0ca2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274bfff700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 0x7f274c0ca2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f2750b48700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 0x7f274c1675e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 0x7f274c0ca2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 -- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 msgr2=0x7f274c1675e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:29.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 0x7f274c1675e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:29.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274b7fe700 1 -- 172.21.15.73:0/2389659812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274c167fe0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f274bfff700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 0x7f274c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:29.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.832+0000 7f2750b48700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 0x7f274c1675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:29.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f274b7fe700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7f2740000f20 tx=0x7f2740007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2740007240 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27400073a0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f274c1682c0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27400066a0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.833+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f274c0af1c0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.834+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274c1688d0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.834+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2740006a50 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.834+0000 7f2733fff700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.834+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7f274007a3b0 con 0x7f274c0ca7c0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.834+0000 7f274bfff700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:29.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.835+0000 7f274bfff700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 0x7f2738061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f273c001ce0 tx=0x7f273c009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:29.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:29.838+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f274004b700 con 0x7f274c0ca7c0 2023-11-28T04:24:30.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.003+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f274c06c830 con 0x7f274c0ca7c0 2023-11-28T04:24:30.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.006+0000 7f2733fff700 1 -- 172.21.15.73:0/2389659812 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) v1 ==== 122+0+0 (secure 0 0 0) 0x7f274c06c830 con 0x7f274c0ca7c0 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 msgr2=0x7f2738061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 0x7f2738061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f273c001ce0 tx=0x7f273c009040).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 msgr2=0x7f274c1670f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 rx=0x7f2740000f20 tx=0x7f2740007bb0).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 shutdown_connections 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f273805f150 0x7f2738061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f274c152f40 0x7f274c1675e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2734004b80 0x7f274c0ca2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 --2- 172.21.15.73:0/2389659812 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f274c0ca7c0 0x7f274c1670f0 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 >> 172.21.15.73:0/2389659812 conn(0x7f274c0cb260 msgr2=0x7f274c0aa600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:30.010 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.008+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 shutdown_connections 2023-11-28T04:24:30.011 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.009+0000 7f27525ab700 1 -- 172.21.15.73:0/2389659812 wait complete. 2023-11-28T04:24:30.022 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:24:30.022 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:24:30.057 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:24:30.101 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:24:30.101 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T04:24:30.102 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:30.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.214+0000 7ff3ee98d700 1 Processor -- start 2023-11-28T04:24:30.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.214+0000 7ff3ee98d700 1 -- start start 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ee98d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff3e81524a0 0x7ff3e8067a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ee98d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3e8067f30 0x7ff3e80be8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ee98d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3e80683e0 con 0x7ff3e81520c0 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ee98d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e80be110 con 0x7ff3e8067f30 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ee98d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3e8066fc0 con 0x7ff3e81524a0 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ecf2a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3e8067f30 0x7ff3e80be8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ecf2a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff3e8067f30 0x7ff3e80be8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53454/0 (socket says 172.21.15.73:53454) 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.215+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/720159520 learned_addr learned my addr 172.21.15.73:0/720159520 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3e77fe700 1 --2- 172.21.15.73:0/720159520 >> v2:172.21.15.150:3300/0 conn(0x7ff3e81524a0 0x7ff3e8067a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/720159520 >> v2:172.21.15.150:3300/0 conn(0x7ff3e81524a0 msgr2=0x7ff3e8067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3ecf2a700 1 --2- 172.21.15.73:0/720159520 >> v2:172.21.15.150:3300/0 conn(0x7ff3e81524a0 0x7ff3e8067a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/720159520 >> v1:172.21.15.150:6789/0 conn(0x7ff3e81520c0 legacy=0x7ff3e8066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/720159520 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e80672a0 con 0x7ff3e8067f30 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3ecf2a700 1 --2- 172.21.15.73:0/720159520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e8067f30 0x7ff3e80be8a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7ff3e806c7c0 tx=0x7ff3e0009840).ready entity=mon.0 client_cookie=96a26dc62922b4f0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3e6ffd700 1 -- 172.21.15.73:0/720159520 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000e070 con 0x7ff3e8067f30 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3e6ffd700 1 -- 172.21.15.73:0/720159520 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e000ba50 con 0x7ff3e8067f30 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.216+0000 7ff3e6ffd700 1 -- 172.21.15.73:0/720159520 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e000bdd0 con 0x7ff3e8067f30 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 -- 172.21.15.73:0/720159520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e8067f30 msgr2=0x7ff3e80be8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/720159520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e8067f30 0x7ff3e80be8a0 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 rx=0x7ff3e806c7c0 tx=0x7ff3e0009840).stop 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 -- 172.21.15.73:0/720159520 shutdown_connections 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/720159520 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e8067f30 0x7ff3e80be8a0 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/720159520 >> v2:172.21.15.150:3300/0 conn(0x7ff3e81524a0 0x7ff3e8067a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 -- 172.21.15.73:0/720159520 >> 172.21.15.73:0/720159520 conn(0x7ff3e80631d0 msgr2=0x7ff3e80c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:30.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 -- 172.21.15.73:0/720159520 shutdown_connections 2023-11-28T04:24:30.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.217+0000 7ff3ee98d700 1 -- 172.21.15.73:0/720159520 wait complete. 2023-11-28T04:24:30.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.218+0000 7ff3ee98d700 1 Processor -- start 2023-11-28T04:24:30.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.218+0000 7ff3ee98d700 1 -- start start 2023-11-28T04:24:30.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8067f30 0x7ff3e80a9050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 0x7ff3e814d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80a8bc0 con 0x7ff3e81524a0 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c2950 con 0x7ff3e8067f30 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3e80c2c30 con 0x7ff3e81520c0 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53462/0 (socket says 172.21.15.73:53462) 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/780382382 learned_addr learned my addr 172.21.15.73:0/780382382 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3e7fff700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8067f30 0x7ff3e80a9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3e77fe700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 0x7ff3e814d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 msgr2=0x7ff3e814d780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 0x7ff3e814d780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8067f30 msgr2=0x7ff3e80a9050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8067f30 0x7ff3e80a9050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 -- 172.21.15.73:0/780382382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3e0009ae0 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3e77fe700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 0x7ff3e814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ecf2a700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7ff3e0009fd0 tx=0x7ff3e0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e00092e0 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3e001d410 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3e80c2f10 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3e001d7d0 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.219+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3e814e170 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.220+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3e814e360 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.221+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff3e001dc00 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.221+0000 7ff3cffff700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 0x7ff3d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.221+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 5985+0+0 (secure 0 0 0) 0x7ff3e007bd80 con 0x7ff3e81524a0 2023-11-28T04:24:30.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.221+0000 7ff3e7fff700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 0x7ff3d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.221+0000 7ff3e7fff700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 0x7ff3d4061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff3d8007fd0 tx=0x7ff3d8007d30).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.225+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff3e814e360 con 0x7ff3e81524a0 2023-11-28T04:24:30.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.431+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3e814e360 con 0x7ff3e81524a0 2023-11-28T04:24:30.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.431+0000 7ff3cffff700 1 -- 172.21.15.73:0/780382382 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 66 v66) v1 ==== 94+0+3532 (secure 0 0 0) 0x7ff3e814e360 con 0x7ff3e81524a0 2023-11-28T04:24:30.435 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:30.435 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":66,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7187,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/3682670242","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":3682670242},{"type":"v1","addr":"172.21.15.150:6834","nonce":3682670242}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":8170,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/4129100830","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":4129100830},{"type":"v1","addr":"172.21.15.150:6835","nonce":4129100830}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9003,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/152954123","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":152954123},{"type":"v1","addr":"172.21.15.73:6836","nonce":152954123}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":61},{"gid":9435,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2122107740","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2122107740},{"type":"v1","addr":"172.21.15.73:6837","nonce":2122107740}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[]} 2023-11-28T04:24:30.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 msgr2=0x7ff3d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 0x7ff3d4061610 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7ff3d8007fd0 tx=0x7ff3d8007d30).stop 2023-11-28T04:24:30.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 msgr2=0x7ff3e80c0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 rx=0x7ff3e0009fd0 tx=0x7ff3e0009840).stop 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 shutdown_connections 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff3d405f150 0x7ff3d4061610 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff3e81524a0 0x7ff3e80c0930 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff3e8067f30 0x7ff3e80a9050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 --2- 172.21.15.73:0/780382382 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff3e81520c0 0x7ff3e814d780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 >> 172.21.15.73:0/780382382 conn(0x7ff3e80631d0 msgr2=0x7ff3e80a7800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:30.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 shutdown_connections 2023-11-28T04:24:30.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.433+0000 7ff3ee98d700 1 -- 172.21.15.73:0/780382382 wait complete. 2023-11-28T04:24:30.439 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 66 2023-11-28T04:24:30.447 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T04:24:30.447 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T04:24:30.447 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T04:24:30.447 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:24:30.447 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T04:24:30.447 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T04:24:30.447 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T04:24:30.448 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:24:30.448 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T04:24:30.448 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T04:24:30.448 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T04:24:30.449 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:24:30.449 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T04:24:30.449 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T04:24:30.449 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T04:24:30.449 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:24:30.456 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T04:24:30.456 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T04:24:30.457 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T04:24:30.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T04:24:30.460 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T04:24:30.460 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T04:24:30.461 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T04:24:30.461 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T04:24:30.498 INFO:tasks.ceph.mds.a:Started 2023-11-28T04:24:30.498 DEBUG:teuthology.parallel:result is None 2023-11-28T04:24:30.499 INFO:tasks.ceph.mds.c:Started 2023-11-28T04:24:30.499 DEBUG:teuthology.parallel:result is None 2023-11-28T04:24:30.503 INFO:tasks.ceph.mds.d:Started 2023-11-28T04:24:30.503 DEBUG:teuthology.parallel:result is None 2023-11-28T04:24:30.504 INFO:tasks.ceph.mds.b:Started 2023-11-28T04:24:30.504 DEBUG:teuthology.parallel:result is None 2023-11-28T04:24:30.504 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T04:24:30.570 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T04:24:30.571 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:24:30.583 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T04:24:30.585 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T04:24:30.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.628+0000 7f241fb3d700 1 Processor -- start 2023-11-28T04:24:30.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.628+0000 7f241fb3d700 1 -- start start 2023-11-28T04:24:30.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241fb3d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 0x7f24180c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241fb3d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2418158320 0x7f24181587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241fb3d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2418158cc0 con 0x7f24180c6190 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241fb3d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24180c72d0 con 0x7f2418158320 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241fb3d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2418064570 con 0x7f2418098d10 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241e0da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2418158320 0x7f24181587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241e0da700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2418158320 0x7f24181587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53510/0 (socket says 172.21.15.73:53510) 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241e0da700 1 -- 172.21.15.73:0/1755148230 learned_addr learned my addr 172.21.15.73:0/1755148230 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241e0da700 1 -- 172.21.15.73:0/1755148230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 msgr2=0x7f24180c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:30.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.629+0000 7f241d8d9700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 0x7f24180c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241e0da700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 0x7f24180c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241e0da700 1 -- 172.21.15.73:0/1755148230 >> v1:172.21.15.150:6789/0 conn(0x7f24180c6190 legacy=0x7f2418063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241e0da700 1 -- 172.21.15.73:0/1755148230 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2418154bc0 con 0x7f2418158320 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241d8d9700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 0x7f24180c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241e0da700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418158320 0x7f24181587d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7f2410009f20 tx=0x7f2410009840).ready entity=mon.0 client_cookie=72aa01ec24221014 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241c8d7700 1 -- 172.21.15.73:0/1755148230 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241000e070 con 0x7f2418158320 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241c8d7700 1 -- 172.21.15.73:0/1755148230 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f241000ba40 con 0x7f2418158320 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241c8d7700 1 -- 172.21.15.73:0/1755148230 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f241000bdc0 con 0x7f2418158320 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 -- 172.21.15.73:0/1755148230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418158320 msgr2=0x7f24181587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418158320 0x7f24181587d0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 rx=0x7f2410009f20 tx=0x7f2410009840).stop 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 -- 172.21.15.73:0/1755148230 shutdown_connections 2023-11-28T04:24:30.634 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418158320 0x7f24181587d0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 --2- 172.21.15.73:0/1755148230 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2418098d10 0x7f24180c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 -- 172.21.15.73:0/1755148230 >> 172.21.15.73:0/1755148230 conn(0x7f24180cb310 msgr2=0x7f24180cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 -- 172.21.15.73:0/1755148230 shutdown_connections 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.630+0000 7f241fb3d700 1 -- 172.21.15.73:0/1755148230 wait complete. 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.631+0000 7f241fb3d700 1 Processor -- start 2023-11-28T04:24:30.635 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.631+0000 7f241fb3d700 1 -- start start 2023-11-28T04:24:30.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24180c6190 0x7f2418152150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2418158320 0x7f2418152640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2418152b30 con 0x7f2418098d10 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24180c5340 con 0x7f2418158320 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241fb3d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24180c4eb0 con 0x7f24180c6190 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241d8d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241d8d9700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53524/0 (socket says 172.21.15.73:53524) 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241d8d9700 1 -- 172.21.15.73:0/3307028255 learned_addr learned my addr 172.21.15.73:0/3307028255 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241e0da700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2418158320 0x7f2418152640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.632+0000 7f241d0d8700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24180c6190 0x7f2418152150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 -- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24180c6190 msgr2=0x7f2418152150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24180c6190 0x7f2418152150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 -- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2418158320 msgr2=0x7f2418152640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2418158320 0x7f2418152640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 -- 172.21.15.73:0/3307028255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2410009ae0 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241d8d9700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f240800c8b0 tx=0x7f2408008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408010580 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24080106e0 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2408010a60 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24180c42a0 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.633+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24180c4b40 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.634+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24180c1a20 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.634+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f240801b460 con 0x7f2418098d10 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.634+0000 7f240dffb700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 0x7f23fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:30.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.634+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5957+0+0 (secure 0 0 0) 0x7f240807a060 con 0x7f2418098d10 2023-11-28T04:24:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.635+0000 7f241d0d8700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 0x7f23fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.635+0000 7f241d0d8700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 0x7f23fc061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2418084960 tx=0x7f2414007d20).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:30.643 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.639+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f240804b370 con 0x7f2418098d10 2023-11-28T04:24:30.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:30.817+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f24180c5190 con 0x7f2418098d10 2023-11-28T04:24:31.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.699+0000 7f240dffb700 1 -- 172.21.15.73:0/3307028255 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v193) v1 ==== 97+0+0 (secure 0 0 0) 0x7f240804e990 con 0x7f2418098d10 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 msgr2=0x7f23fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 0x7f23fc061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f2418084960 tx=0x7f2414007d20).stop 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 msgr2=0x7f241814e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 rx=0x7f240800c8b0 tx=0x7f2408008040).stop 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 shutdown_connections 2023-11-28T04:24:31.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f23fc05f150 0x7f23fc061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2418158320 0x7f2418152640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2418098d10 0x7f241814e3a0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 --2- 172.21.15.73:0/3307028255 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f24180c6190 0x7f2418152150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 >> 172.21.15.73:0/3307028255 conn(0x7f24180cb310 msgr2=0x7f241814f290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 shutdown_connections 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.702+0000 7f241fb3d700 1 -- 172.21.15.73:0/3307028255 wait complete. 2023-11-28T04:24:31.704 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T04:24:31.716 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T04:24:31.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.829+0000 7f6378be5700 1 Processor -- start 2023-11-28T04:24:31.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.829+0000 7f6378be5700 1 -- start start 2023-11-28T04:24:31.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6378be5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f63740a8380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6378be5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63740bf0e0 con 0x7f63740c5ff0 2023-11-28T04:24:31.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6378be5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63740be510 con 0x7f63740a5c00 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6378be5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6374152570 con 0x7f6374152190 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6372ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f63740c5ff0 0x7f63740bf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55814/0 (socket says 172.21.15.73:55814) 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f6372ffd700 1 -- 172.21.15.73:0/2634259462 learned_addr learned my addr 172.21.15.73:0/2634259462 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2078789544 0 0) 0x7f63740bf0e0 con 0x7f63740c5ff0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f635c0033f0 con 0x7f63740c5ff0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63737fe700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f63740a8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2588039975 0 0) 0x7f635c0033f0 con 0x7f63740c5ff0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 msgr2=0x7f63740a8380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f63740a8380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 >> v1:172.21.15.150:6789/0 conn(0x7f63740a5c00 legacy=0x7f63740c40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63740be7b0 con 0x7f63740c5ff0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6368001470 con 0x7f63740c5ff0 2023-11-28T04:24:31.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.830+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 >> v1:172.21.15.73:6789/0 conn(0x7f63740c5ff0 legacy=0x7f63740bf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f635c003910 0x7f635c003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f635c004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c004b80 0x7f635c005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374152570 con 0x7f635c004b80 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f635c00b910 con 0x7f635c003910 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f635c00bad0 con 0x7f6374152190 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63737fe700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f635c003910 0x7f635c003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c004b80 0x7f635c005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6372ffd700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f635c004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 -- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 msgr2=0x7f635c004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6374152190 0x7f635c004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 -- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f635c003910 msgr2=0x7f635c003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f635c003910 0x7f635c003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 -- 172.21.15.73:0/2634259462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f635c0033f0 con 0x7f635c004b80 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f6373fff700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c004b80 0x7f635c005000 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 rx=0x7f636c001730 tx=0x7f636c007bb0).ready entity=mon.0 client_cookie=a46f085a59612ddf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c007020 con 0x7f635c004b80 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c007180 con 0x7f635c004b80 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.831+0000 7f63727fc700 1 -- 172.21.15.73:0/2634259462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c007500 con 0x7f635c004b80 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 -- 172.21.15.73:0/2634259462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c004b80 msgr2=0x7f635c005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 --2- 172.21.15.73:0/2634259462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f635c004b80 0x7f635c005000 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 rx=0x7f636c001730 tx=0x7f636c007bb0).stop 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6373fff700 1 -- 172.21.15.73:0/2634259462 reap_dead start 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 -- 172.21.15.73:0/2634259462 shutdown_connections 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 -- 172.21.15.73:0/2634259462 >> 172.21.15.73:0/2634259462 conn(0x7f6374083000 msgr2=0x7f6374092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 -- 172.21.15.73:0/2634259462 shutdown_connections 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.832+0000 7f6378be5700 1 -- 172.21.15.73:0/2634259462 wait complete. 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.833+0000 7f6378be5700 1 Processor -- start 2023-11-28T04:24:31.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.833+0000 7f6378be5700 1 -- start start 2023-11-28T04:24:31.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f635c004b80 0x7f63740c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 0x7f6374165cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374154db0 con 0x7f63740c7150 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741661a0 con 0x7f637414c8e0 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6378be5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6374154130 con 0x7f635c004b80 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53546/0 (socket says 172.21.15.73:53546) 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 -- 172.21.15.73:0/3140825880 learned_addr learned my addr 172.21.15.73:0/3140825880 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f63737fe700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f635c004b80 0x7f63740c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6373fff700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 0x7f6374165cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 -- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f635c004b80 msgr2=0x7f63740c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f635c004b80 0x7f63740c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 -- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 msgr2=0x7f6374165cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 0x7f6374165cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 -- 172.21.15.73:0/3140825880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6374154410 con 0x7f63740c7150 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6373fff700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 0x7f6374165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:31.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.834+0000 7f6372ffd700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f6368004730 tx=0x7f6368001e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.835+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368011590 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.835+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63680116f0 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.835+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6368011a70 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.835+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63741546f0 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.835+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6374161ee0 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.836+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6374154ef0 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.836+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f636801c360 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.836+0000 7f635b7fe700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 0x7f6360061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.836+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f63680795d0 con 0x7f63740c7150 2023-11-28T04:24:31.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.836+0000 7f63737fe700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 0x7f6360061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.837+0000 7f63737fe700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 0x7f6360061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f63640094d0 tx=0x7f636400b040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:31.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:31.840+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6374154ef0 con 0x7f63740c7150 2023-11-28T04:24:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.067+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f6374154ef0 con 0x7f63740c7150 2023-11-28T04:24:32.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.067+0000 7f635b7fe700 1 -- 172.21.15.73:0/3140825880 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v46) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f6374154ef0 con 0x7f63740c7150 2023-11-28T04:24:32.071 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.072 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.073 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:24:32.074 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.075 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:24:32.076 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:24:32.076 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.076 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.077 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.078 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T04:24:32.079 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.080 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T04:24:32.081 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.082 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T04:24:32.083 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:24:32.084 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:24:32.085 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.086 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.087 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T04:24:32.088 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T04:24:32.089 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:24:32.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.069+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 msgr2=0x7f6360061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.069+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 0x7f6360061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f63640094d0 tx=0x7f636400b040).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 msgr2=0x7f63741657c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 rx=0x7f6368004730 tx=0x7f6368001e50).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 shutdown_connections 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f636005f150 0x7f6360061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f637414c8e0 0x7f6374165cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f635c004b80 0x7f63740c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 --2- 172.21.15.73:0/3140825880 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f63740c7150 0x7f63741657c0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 >> 172.21.15.73:0/3140825880 conn(0x7f6374083000 msgr2=0x7f63741573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.091 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 shutdown_connections 2023-11-28T04:24:32.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.070+0000 7f6378be5700 1 -- 172.21.15.73:0/3140825880 wait complete. 2023-11-28T04:24:32.092 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:32.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.204+0000 7fde4152a700 1 Processor -- start 2023-11-28T04:24:32.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde4152a700 1 -- start start 2023-11-28T04:24:32.214 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde4152a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde3c0be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde4152a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c0bfc00 con 0x7fde3c154500 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde4152a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde3c0beff0 con 0x7fde3c0c46e0 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde4152a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0bf2d0 con 0x7fde3c154120 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde3b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fde3c154500 0x7fde3c0c3fd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55826/0 (socket says 172.21.15.73:55826) 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.205+0000 7fde3b7fe700 1 -- 172.21.15.73:0/66360565 learned_addr learned my addr 172.21.15.73:0/66360565 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3bfff700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde3c0be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.215 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2618033737 0 0) 0x7fde3c0bfc00 con 0x7fde3c154500 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fde240033f0 con 0x7fde3c154500 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935136695 0 0) 0x7fde240033f0 con 0x7fde3c154500 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 msgr2=0x7fde3c0be510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde3c0be510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 >> v1:172.21.15.150:6789/0 conn(0x7fde3c0c46e0 legacy=0x7fde3c15def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c0bf5b0 con 0x7fde3c154500 2023-11-28T04:24:32.216 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fde30001470 con 0x7fde3c154500 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 >> v1:172.21.15.73:6789/0 conn(0x7fde3c154500 legacy=0x7fde3c0c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde24003910 0x7fde24003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde24004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde24004b80 0x7fde24005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c0bf2d0 con 0x7fde24004b80 2023-11-28T04:24:32.217 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2400b910 con 0x7fde24003910 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2400bad0 con 0x7fde3c154120 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde24004b80 0x7fde24005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3bfff700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde24003910 0x7fde24003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3b7fe700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde24004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 -- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 msgr2=0x7fde24004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.218 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde24004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 -- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde24003910 msgr2=0x7fde24003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde24003910 0x7fde24003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde40d29700 1 -- 172.21.15.73:0/66360565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde240033f0 con 0x7fde24004b80 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.206+0000 7fde3b7fe700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c154120 0x7fde24004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde40d29700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde24004b80 0x7fde24005000 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7fde34001e30 tx=0x7fde34007f30).ready entity=mon.0 client_cookie=1586b99629ea00bd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde34006a20 con 0x7fde24004b80 2023-11-28T04:24:32.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde34006b80 con 0x7fde24004b80 2023-11-28T04:24:32.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde3affd700 1 -- 172.21.15.73:0/66360565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde3400c5d0 con 0x7fde24004b80 2023-11-28T04:24:32.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 -- 172.21.15.73:0/66360565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde24004b80 msgr2=0x7fde24005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 --2- 172.21.15.73:0/66360565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde24004b80 0x7fde24005000 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 rx=0x7fde34001e30 tx=0x7fde34007f30).stop 2023-11-28T04:24:32.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde40d29700 1 -- 172.21.15.73:0/66360565 reap_dead start 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 -- 172.21.15.73:0/66360565 shutdown_connections 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 -- 172.21.15.73:0/66360565 >> 172.21.15.73:0/66360565 conn(0x7fde3c083000 msgr2=0x7fde3c092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 -- 172.21.15.73:0/66360565 shutdown_connections 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.207+0000 7fde4152a700 1 -- 172.21.15.73:0/66360565 wait complete. 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.208+0000 7fde4152a700 1 Processor -- start 2023-11-28T04:24:32.221 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.208+0000 7fde4152a700 1 -- start start 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c164550 0x7fde3c0aabc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde3c0ab0b0 0x7fde3c1665e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c154960 con 0x7fde3c161610 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c154f20 con 0x7fde3c0ab0b0 2023-11-28T04:24:32.222 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde4152a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde3c164b70 con 0x7fde3c164550 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde3bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53550/0 (socket says 172.21.15.73:53550) 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde3bfff700 1 -- 172.21.15.73:0/2910685272 learned_addr learned my addr 172.21.15.73:0/2910685272 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde40d29700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde3c0ab0b0 0x7fde3c1665e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.209+0000 7fde3b7fe700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c164550 0x7fde3c0aabc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 -- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c164550 msgr2=0x7fde3c0aabc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c164550 0x7fde3c0aabc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 -- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde3c0ab0b0 msgr2=0x7fde3c1665e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde3c0ab0b0 0x7fde3c1665e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 -- 172.21.15.73:0/2910685272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde3c164e50 con 0x7fde3c161610 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde3bfff700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7fde2c011a20 tx=0x7fde2c006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c011eb0 con 0x7fde3c161610 2023-11-28T04:24:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde2c005a90 con 0x7fde3c161610 2023-11-28T04:24:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde3c154bb0 con 0x7fde3c161610 2023-11-28T04:24:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde2c005e10 con 0x7fde3c161610 2023-11-28T04:24:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.210+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde3c1619f0 con 0x7fde3c161610 2023-11-28T04:24:32.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.211+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde3c164930 con 0x7fde3c161610 2023-11-28T04:24:32.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.211+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fde2c006240 con 0x7fde3c161610 2023-11-28T04:24:32.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.211+0000 7fde23fff700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 0x7fde28061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.212+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fde2c07a290 con 0x7fde3c161610 2023-11-28T04:24:32.226 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.212+0000 7fde3b7fe700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 0x7fde28061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.212+0000 7fde3b7fe700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 0x7fde28061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fde30000e10 tx=0x7fde30009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.216+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fde3c164930 con 0x7fde3c161610 2023-11-28T04:24:32.382 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.381+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fde3c164930 con 0x7fde3c161610 2023-11-28T04:24:32.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.382+0000 7fde23fff700 1 -- 172.21.15.73:0/2910685272 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+13723 (secure 0 0 0) 0x7fde3c164930 con 0x7fde3c161610 2023-11-28T04:24:32.383 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:32.383 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":193,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:31.697518+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":22,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":181,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:32.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 msgr2=0x7fde28061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 0x7fde28061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fde30000e10 tx=0x7fde30009040).stop 2023-11-28T04:24:32.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 msgr2=0x7fde3c164060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 rx=0x7fde2c011a20 tx=0x7fde2c006f40).stop 2023-11-28T04:24:32.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 shutdown_connections 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fde2805f150 0x7fde28061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fde3c0ab0b0 0x7fde3c1665e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fde3c161610 0x7fde3c164060 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 --2- 172.21.15.73:0/2910685272 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fde3c164550 0x7fde3c0aabc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.383+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 >> 172.21.15.73:0/2910685272 conn(0x7fde3c083000 msgr2=0x7fde3c0c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.384+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 shutdown_connections 2023-11-28T04:24:32.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.384+0000 7fde4152a700 1 -- 172.21.15.73:0/2910685272 wait complete. 2023-11-28T04:24:32.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T04:24:32.397 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:24:32.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.512+0000 7fd8a2339700 1 Processor -- start 2023-11-28T04:24:32.513 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.512+0000 7fd8a2339700 1 -- start start 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a2339700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 0x7fd89c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a2339700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd89c158320 0x7fd89c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a2339700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd89c158cc0 con 0x7fd89c0c6190 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a2339700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c0c72d0 con 0x7fd89c158320 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a2339700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd89c064570 con 0x7fd89c098d10 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a08d6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd89c158320 0x7fd89c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.516 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd89b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd89c0c6190 0x7fd89c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55842/0 (socket says 172.21.15.73:55842) 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd89b7fe700 1 -- 172.21.15.73:0/3947477136 learned_addr learned my addr 172.21.15.73:0/3947477136 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd89bfff700 1 --2- 172.21.15.73:0/3947477136 >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 0x7fd89c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a08d6700 1 -- 172.21.15.73:0/3947477136 >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 msgr2=0x7fd89c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a08d6700 1 --2- 172.21.15.73:0/3947477136 >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 0x7fd89c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd89affd700 1 -- 172.21.15.73:0/3947477136 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 67898049 0 0) 0x7fd890001110 con 0x7fd89c0c6190 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a08d6700 1 -- 172.21.15.73:0/3947477136 >> v1:172.21.15.73:6789/0 conn(0x7fd89c0c6190 legacy=0x7fd89c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd8a08d6700 1 -- 172.21.15.73:0/3947477136 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd89c154bc0 con 0x7fd89c158320 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.513+0000 7fd89bfff700 1 --2- 172.21.15.73:0/3947477136 >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 0x7fd89c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a08d6700 1 --2- 172.21.15.73:0/3947477136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c1587d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7fd89c083f80 tx=0x7fd894009780).ready entity=mon.0 client_cookie=fc78ef89dd8cb100 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd89affd700 1 -- 172.21.15.73:0/3947477136 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89400e070 con 0x7fd89c158320 2023-11-28T04:24:32.517 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd89affd700 1 -- 172.21.15.73:0/3947477136 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89400bc30 con 0x7fd89c158320 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd89affd700 1 -- 172.21.15.73:0/3947477136 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd894012410 con 0x7fd89c158320 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 -- 172.21.15.73:0/3947477136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 msgr2=0x7fd89c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 --2- 172.21.15.73:0/3947477136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c1587d0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 rx=0x7fd89c083f80 tx=0x7fd894009780).stop 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 -- 172.21.15.73:0/3947477136 shutdown_connections 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 --2- 172.21.15.73:0/3947477136 >> v2:172.21.15.150:3300/0 conn(0x7fd89c098d10 0x7fd89c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 --2- 172.21.15.73:0/3947477136 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c1587d0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 -- 172.21.15.73:0/3947477136 >> 172.21.15.73:0/3947477136 conn(0x7fd89c0cb310 msgr2=0x7fd89c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 -- 172.21.15.73:0/3947477136 shutdown_connections 2023-11-28T04:24:32.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.514+0000 7fd8a2339700 1 -- 172.21.15.73:0/3947477136 wait complete. 2023-11-28T04:24:32.519 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.515+0000 7fd8a2339700 1 Processor -- start 2023-11-28T04:24:32.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 -- start start 2023-11-28T04:24:32.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 0x7fd89c0c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd89c0c6190 0x7fd89c0c4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0c5310 con 0x7fd89c158320 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c155360 con 0x7fd89c0c6190 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a2339700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd89c0c4990 con 0x7fd89c098d10 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a08d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a08d6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53564/0 (socket says 172.21.15.73:53564) 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd8a08d6700 1 -- 172.21.15.73:0/4113654698 learned_addr learned my addr 172.21.15.73:0/4113654698 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.516+0000 7fd89b7fe700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd89c0c6190 0x7fd89c0c4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd89bfff700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 0x7fd89c0c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 -- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 msgr2=0x7fd89c0c36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 0x7fd89c0c36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 -- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd89c0c6190 msgr2=0x7fd89c0c4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd89c0c6190 0x7fd89c0c4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.522 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 -- 172.21.15.73:0/4113654698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd894009a20 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd89bfff700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 0x7fd89c0c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a08d6700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7fd89c0c2360 tx=0x7fd89c0a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89401d630 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd89401d790 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd89401db50 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd89c0c3d80 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.517+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd89c0c4340 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.518+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd89c0c4b00 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.518+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd894027460 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.518+0000 7fd883fff700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 0x7fd8880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.519+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fd89407bf80 con 0x7fd89c158320 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.519+0000 7fd89bfff700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 0x7fd8880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.523 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.519+0000 7fd89bfff700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 0x7fd8880615e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd88c00aa90 tx=0x7fd88c008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.523+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd8940aaaf0 con 0x7fd89c158320 2023-11-28T04:24:32.687 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.686+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fd89c06c860 con 0x7fd89c158320 2023-11-28T04:24:32.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.687+0000 7fd883fff700 1 -- 172.21.15.73:0/4113654698 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v193) v1 ==== 110+0+10 (secure 0 0 0) 0x7fd89404d540 con 0x7fd89c158320 2023-11-28T04:24:32.689 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 msgr2=0x7fd8880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 0x7fd8880615e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7fd88c00aa90 tx=0x7fd88c008040).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 msgr2=0x7fd89c154e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 rx=0x7fd89c0c2360 tx=0x7fd89c0a8040).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 shutdown_connections 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd88805f120 0x7fd8880615e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd89c158320 0x7fd89c154e70 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd89c098d10 0x7fd89c0c36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 --2- 172.21.15.73:0/4113654698 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd89c0c6190 0x7fd89c0c4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.688+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 >> 172.21.15.73:0/4113654698 conn(0x7fd89c0cb310 msgr2=0x7fd89c0c9030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.689+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 shutdown_connections 2023-11-28T04:24:32.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.689+0000 7fd8a2339700 1 -- 172.21.15.73:0/4113654698 wait complete. 2023-11-28T04:24:32.702 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:24:32.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.815+0000 7f70e2466700 1 Processor -- start 2023-11-28T04:24:32.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.815+0000 7f70e2466700 1 -- start start 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70e2466700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f70dc098d10 0x7f70dc0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70e2466700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc0c72d0 con 0x7f70dc0c6190 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70e2466700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70dc064570 con 0x7f70dc158320 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70e2466700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70dc154bc0 con 0x7f70dc098d10 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70db7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f70dc0c6190 0x7f70dc063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55856/0 (socket says 172.21.15.73:55856) 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.816+0000 7f70db7fe700 1 -- 172.21.15.73:0/1431266915 learned_addr learned my addr 172.21.15.73:0/1431266915 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:32.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70dbfff700 1 --2- 172.21.15.73:0/1431266915 >> v2:172.21.15.150:3300/0 conn(0x7f70dc098d10 0x7f70dc0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1612861679 0 0) 0x7f70dc0c72d0 con 0x7f70dc0c6190 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f70c40033f0 con 0x7f70dc0c6190 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 127009845 0 0) 0x7f70c40033f0 con 0x7f70dc0c6190 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 >> v2:172.21.15.150:3300/0 conn(0x7f70dc098d10 msgr2=0x7f70dc0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 --2- 172.21.15.73:0/1431266915 >> v2:172.21.15.150:3300/0 conn(0x7f70dc098d10 0x7f70dc0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 >> v1:172.21.15.150:6789/0 conn(0x7f70dc158320 legacy=0x7f70dc0a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0c66c0 con 0x7f70dc0c6190 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f70d0001470 con 0x7f70dc0c6190 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 >> v1:172.21.15.73:6789/0 conn(0x7f70dc0c6190 legacy=0x7f70dc063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c40038e0 0x7f70c4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4004690 0x7f70c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 0x7f70c400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc154bc0 con 0x7f70c4005000 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70c400beb0 con 0x7f70c40038e0 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70c400c070 con 0x7f70c4004690 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70e0a03700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 0x7f70c400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70dbfff700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c40038e0 0x7f70c4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.817+0000 7f70db7fe700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4004690 0x7f70c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 -- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4004690 msgr2=0x7f70c4004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4004690 0x7f70c4004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 -- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c40038e0 msgr2=0x7f70c4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70c40038e0 0x7f70c4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 -- 172.21.15.73:0/1431266915 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c40033f0 con 0x7f70c4005000 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 -- 172.21.15.73:0/1431266915 reap_dead start 2023-11-28T04:24:32.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e0a03700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 0x7f70c400b8e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f70d4001e30 tx=0x7f70d4007f30).ready entity=mon.0 client_cookie=d937c776553709f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4006a20 con 0x7f70c4005000 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d4006b80 con 0x7f70c4005000 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70daffd700 1 -- 172.21.15.73:0/1431266915 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d400c5d0 con 0x7f70c4005000 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 -- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 msgr2=0x7f70c400b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 0x7f70c400b8e0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 rx=0x7f70d4001e30 tx=0x7f70d4007f30).stop 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 -- 172.21.15.73:0/1431266915 shutdown_connections 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 --2- 172.21.15.73:0/1431266915 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70c4005000 0x7f70c400b8e0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 -- 172.21.15.73:0/1431266915 >> 172.21.15.73:0/1431266915 conn(0x7f70dc0cb310 msgr2=0x7f70dc0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 -- 172.21.15.73:0/1431266915 shutdown_connections 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.818+0000 7f70e2466700 1 -- 172.21.15.73:0/1431266915 wait complete. 2023-11-28T04:24:32.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.819+0000 7f70e2466700 1 Processor -- start 2023-11-28T04:24:32.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 -- start start 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4005000 0x7f70dc0ad100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70dc156610 0x7f70dc156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0aad70 con 0x7f70dc0c8310 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc0ab050 con 0x7f70dc156610 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e2466700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70dc156f80 con 0x7f70c4005000 2023-11-28T04:24:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e0a03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e0a03700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53592/0 (socket says 172.21.15.73:53592) 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.820+0000 7f70e0a03700 1 -- 172.21.15.73:0/4079387894 learned_addr learned my addr 172.21.15.73:0/4079387894 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70dbfff700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4005000 0x7f70dc0ad100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70db7fe700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70dc156610 0x7f70dc156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 -- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4005000 msgr2=0x7f70dc0ad100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4005000 0x7f70dc0ad100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 -- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70dc156610 msgr2=0x7f70dc156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70dc156610 0x7f70dc156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 -- 172.21.15.73:0/4079387894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70dc0c8f60 con 0x7f70dc0c8310 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e0a03700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f70d4013740 tx=0x7f70d4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4006a20 con 0x7f70dc0c8310 2023-11-28T04:24:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70d400c5d0 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70dc0c9240 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70d4013d80 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.821+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70dc0c8c80 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.822+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70dc06c860 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.822+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f70d401d680 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.823+0000 7f70c3fff700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 0x7f70c80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.823+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f70d407b6f0 con 0x7f70dc0c8310 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.823+0000 7f70dbfff700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 0x7f70c80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.823+0000 7f70dbfff700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 0x7f70c80615e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f70cc006d80 tx=0x7f70cc009060).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:32.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.827+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f70d404ccb0 con 0x7f70dc0c8310 2023-11-28T04:24:32.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.989+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f70dc0ab190 con 0x7f70dc0c8310 2023-11-28T04:24:32.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.989+0000 7f70c3fff700 1 -- 172.21.15.73:0/4079387894 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v193) v1 ==== 92+0+10 (secure 0 0 0) 0x7f70d40502d0 con 0x7f70dc0c8310 2023-11-28T04:24:32.991 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:24:32.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 msgr2=0x7f70c80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 0x7f70c80615e0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f70cc006d80 tx=0x7f70cc009060).stop 2023-11-28T04:24:32.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 msgr2=0x7f70dc0c8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 rx=0x7f70d4013740 tx=0x7f70d4016000).stop 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 shutdown_connections 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f70c805f120 0x7f70c80615e0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f70dc0c8310 0x7f70dc0c8790 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f70c4005000 0x7f70dc0ad100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 --2- 172.21.15.73:0/4079387894 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f70dc156610 0x7f70dc156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 >> 172.21.15.73:0/4079387894 conn(0x7f70dc0cb310 msgr2=0x7f70dc153fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 shutdown_connections 2023-11-28T04:24:32.993 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:32.991+0000 7f70e2466700 1 -- 172.21.15.73:0/4079387894 wait complete. 2023-11-28T04:24:33.005 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T04:24:33.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.117+0000 7f8f8d587700 1 Processor -- start 2023-11-28T04:24:33.118 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.117+0000 7f8f8d587700 1 -- start start 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8d587700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8f880c0180 0x7f8f88157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8d587700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f880c1750 0x7f8f880c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8d587700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f880c12c0 con 0x7f8f88157660 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8d587700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f880c06b0 con 0x7f8f880c0180 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8d587700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880c0990 con 0x7f8f880c1750 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f87fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8f88157660 0x7f8f881589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55868/0 (socket says 172.21.15.73:55868) 2023-11-28T04:24:33.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f87fff700 1 -- 172.21.15.73:0/127861938 learned_addr learned my addr 172.21.15.73:0/127861938 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f877fe700 1 --2- 172.21.15.73:0/127861938 >> v2:172.21.15.150:3300/0 conn(0x7f8f880c0180 0x7f8f88157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f8cd86700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f880c1750 0x7f8f880c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2100009187 0 0) 0x7f8f880c12c0 con 0x7f8f88157660 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f700033f0 con 0x7f8f88157660 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3471977723 0 0) 0x7f8f700033f0 con 0x7f8f88157660 2023-11-28T04:24:33.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f877fe700 1 -- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f880c1750 msgr2=0x7f8f880c1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f877fe700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f880c1750 0x7f8f880c1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f877fe700 1 -- 172.21.15.73:0/127861938 >> v1:172.21.15.73:6789/0 conn(0x7f8f88157660 legacy=0x7f8f881589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.118+0000 7f8f877fe700 1 -- 172.21.15.73:0/127861938 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f880c0c70 con 0x7f8f880c0180 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f877fe700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c0180 0x7f8f88157a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f8f7c009760 tx=0x7f8f7c009060).ready entity=mon.1 client_cookie=6d40644687eec830 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78001470 con 0x7f8f880c0180 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f7c011400 con 0x7f8f880c0180 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f86ffd700 1 -- 172.21.15.73:0/127861938 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f7c011780 con 0x7f8f880c0180 2023-11-28T04:24:33.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 -- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c0180 msgr2=0x7f8f88157a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c0180 0x7f8f88157a40 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 rx=0x7f8f7c009760 tx=0x7f8f7c009060).stop 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 -- 172.21.15.73:0/127861938 shutdown_connections 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f880c1750 0x7f8f880c1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 --2- 172.21.15.73:0/127861938 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c0180 0x7f8f88157a40 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 -- 172.21.15.73:0/127861938 >> 172.21.15.73:0/127861938 conn(0x7f8f880cb000 msgr2=0x7f8f880cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 -- 172.21.15.73:0/127861938 shutdown_connections 2023-11-28T04:24:33.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.119+0000 7f8f8d587700 1 -- 172.21.15.73:0/127861938 wait complete. 2023-11-28T04:24:33.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.120+0000 7f8f8d587700 1 Processor -- start 2023-11-28T04:24:33.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 -- start start 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c1750 0x7f8f880c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f88157660 0x7f8f881544a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880c4a40 con 0x7f8f880c0180 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f880c9570 con 0x7f8f880c1750 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f8d587700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f88141760 con 0x7f8f88157660 2023-11-28T04:24:33.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f87fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:53594/0 (socket says 172.21.15.73:53594) 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.121+0000 7f8f87fff700 1 -- 172.21.15.73:0/746810464 learned_addr learned my addr 172.21.15.73:0/746810464 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f8cd86700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f88157660 0x7f8f881544a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f877fe700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c1750 0x7f8f880c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 -- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f88157660 msgr2=0x7f8f881544a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f88157660 0x7f8f881544a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 -- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c1750 msgr2=0x7f8f880c2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c1750 0x7f8f880c2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 -- 172.21.15.73:0/746810464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f7c0092a0 con 0x7f8f880c0180 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f87fff700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f8f7800c810 tx=0x7f8f78001a00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78001a90 con 0x7f8f880c0180 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f880c4480 con 0x7f8f880c0180 2023-11-28T04:24:33.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f78007170 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f78007530 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.122+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f880c9200 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.123+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f8806c7c0 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.123+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8f78004ad0 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.124+0000 7f8f6ffff700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 0x7f8f740615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.124+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f8f78079810 con 0x7f8f880c0180 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.124+0000 7f8f877fe700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 0x7f8f740615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.131 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.124+0000 7f8f877fe700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 0x7f8f740615e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f8f881676e0 tx=0x7f8f7c016000).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.132 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.128+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8f7804ae30 con 0x7f8f880c0180 2023-11-28T04:24:33.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.295+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8f880c0f50 con 0x7f8f880c0180 2023-11-28T04:24:33.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.295+0000 7f8f6ffff700 1 -- 172.21.15.73:0/746810464 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v193) v1 ==== 84+0+143 (secure 0 0 0) 0x7f8f7804e450 con 0x7f8f880c0180 2023-11-28T04:24:33.296 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:33.296 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:24:33.296 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:33.297 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:24:33.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 msgr2=0x7f8f740615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 0x7f8f740615e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f8f881676e0 tx=0x7f8f7c016000).stop 2023-11-28T04:24:33.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 msgr2=0x7f8f88144090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 rx=0x7f8f7800c810 tx=0x7f8f78001a00).stop 2023-11-28T04:24:33.298 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 shutdown_connections 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8f7405f120 0x7f8f740615e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8f88157660 0x7f8f881544a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8f880c0180 0x7f8f88144090 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 --2- 172.21.15.73:0/746810464 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8f880c1750 0x7f8f880c2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 >> 172.21.15.73:0/746810464 conn(0x7f8f880cb000 msgr2=0x7f8f880ab010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 shutdown_connections 2023-11-28T04:24:33.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.297+0000 7f8f8d587700 1 -- 172.21.15.73:0/746810464 wait complete. 2023-11-28T04:24:33.310 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T04:24:33.310 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T04:24:33.427 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.426+0000 7ff64bfff700 1 Processor -- start 2023-11-28T04:24:33.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.426+0000 7ff64bfff700 1 -- start start 2023-11-28T04:24:33.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff64bfff700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff64c1542d0 0x7ff64c0a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff64bfff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c158d50 con 0x7ff64c1546b0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff64bfff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff64c158140 con 0x7ff64c0c66d0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff64bfff700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff64c0c6b60 con 0x7ff64c1542d0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff6427fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7ff64c1546b0 0x7ff64c0c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40734/0 (socket says 172.21.15.73:40734) 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.427+0000 7ff6427fc700 1 -- 172.21.15.73:0/3042205834 learned_addr learned my addr 172.21.15.73:0/3042205834 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64affd700 1 --2- 172.21.15.73:0/3042205834 >> v2:172.21.15.150:3300/0 conn(0x7ff64c1542d0 0x7ff64c0a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3325498829 0 0) 0x7ff64c158d50 con 0x7ff64c1546b0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6380033f0 con 0x7ff64c1546b0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2162786303 0 0) 0x7ff64c158140 con 0x7ff64c0c66d0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff64c158d50 con 0x7ff64c0c66d0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3797605215 0 0) 0x7ff6380033f0 con 0x7ff64c1546b0 2023-11-28T04:24:33.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 >> v2:172.21.15.150:3300/0 conn(0x7ff64c1542d0 msgr2=0x7ff64c0a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 --2- 172.21.15.73:0/3042205834 >> v2:172.21.15.150:3300/0 conn(0x7ff64c1542d0 0x7ff64c0a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 >> v1:172.21.15.150:6789/0 conn(0x7ff64c0c66d0 legacy=0x7ff64c0c0890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c158420 con 0x7ff64c1546b0 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7ff634001470 con 0x7ff64c1546b0 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 >> v1:172.21.15.73:6789/0 conn(0x7ff64c1546b0 legacy=0x7ff64c0c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638003a70 0x7ff638003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff638004690 0x7ff638004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff63800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800bf60 con 0x7ff638005000 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800c0a0 con 0x7ff638003a70 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63800c210 con 0x7ff638004690 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64b7fe700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff63800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64affd700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638003a70 0x7ff638003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff6427fc700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff638004690 0x7ff638004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64b7fe700 1 -- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff638004690 msgr2=0x7ff638004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.428+0000 7ff64b7fe700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff638004690 0x7ff638004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64b7fe700 1 -- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638003a70 msgr2=0x7ff638003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64b7fe700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff638003a70 0x7ff638003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64b7fe700 1 -- 172.21.15.73:0/3042205834 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380033f0 con 0x7ff638005000 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64b7fe700 1 -- 172.21.15.73:0/3042205834 reap_dead start 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64b7fe700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff63800b8e0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7ff644001f90 tx=0x7ff644007f30).ready entity=mon.0 client_cookie=41cc189ef5998367 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff644006a20 con 0x7ff638005000 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff644006b80 con 0x7ff638005000 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64a7fc700 1 -- 172.21.15.73:0/3042205834 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff64400c8b0 con 0x7ff638005000 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 -- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 msgr2=0x7ff63800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff63800b8e0 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 rx=0x7ff644001f90 tx=0x7ff644007f30).stop 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 -- 172.21.15.73:0/3042205834 shutdown_connections 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 --2- 172.21.15.73:0/3042205834 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff63800b8e0 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 -- 172.21.15.73:0/3042205834 >> 172.21.15.73:0/3042205834 conn(0x7ff64c0cb010 msgr2=0x7ff64c0cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 -- 172.21.15.73:0/3042205834 shutdown_connections 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.429+0000 7ff64bfff700 1 -- 172.21.15.73:0/3042205834 wait complete. 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 Processor -- start 2023-11-28T04:24:33.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 -- start start 2023-11-28T04:24:33.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 0x7ff64c0af030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c148280 0x7ff64c0af520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c3820 con 0x7ff638005000 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c3b00 con 0x7ff64c147e00 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64bfff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff64c0c3de0 con 0x7ff64c148280 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64affd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41696/0 (socket says 172.21.15.73:41696) 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.431+0000 7ff64affd700 1 -- 172.21.15.73:0/2777758547 learned_addr learned my addr 172.21.15.73:0/2777758547 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff6427fc700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 0x7ff64c0af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64b7fe700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c148280 0x7ff64c0af520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 -- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c148280 msgr2=0x7ff64c0af520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c148280 0x7ff64c0af520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 -- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 msgr2=0x7ff64c0af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 0x7ff64c0af030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 -- 172.21.15.73:0/2777758547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff64c0c40c0 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff6427fc700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 0x7ff64c0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64affd700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 rx=0x7ff63c0119c0 tx=0x7ff63c009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00f940 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff63c00faa0 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff64c0c43a0 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff63c00fe20 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.432+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff64c0c34b0 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.433+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff64c06c7c0 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.433+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff63c014650 con 0x7ff638005000 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.434+0000 7ff6437fe700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 0x7ff62c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.434+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 5269+0+0 (secure 0 0 0) 0x7ff63c07aa60 con 0x7ff638005000 2023-11-28T04:24:33.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.434+0000 7ff6427fc700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 0x7ff62c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.434+0000 7ff6427fc700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 0x7ff62c061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff634001200 tx=0x7ff634009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.438+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7ff63c04c080 con 0x7ff638005000 2023-11-28T04:24:33.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.600+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7ff64c0c0120 con 0x7ff638005000 2023-11-28T04:24:33.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.713+0000 7ff6437fe700 1 -- 172.21.15.73:0/2777758547 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v194) v1 ==== 152+0+0 (secure 0 0 0) 0x7ff63c04f6a0 con 0x7ff638005000 2023-11-28T04:24:33.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.716+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 msgr2=0x7ff62c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.716+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 0x7ff62c061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7ff634001200 tx=0x7ff634009040).stop 2023-11-28T04:24:33.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.716+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 msgr2=0x7ff64c0c48c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.716+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 rx=0x7ff63c0119c0 tx=0x7ff63c009060).stop 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 shutdown_connections 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7ff62c05f150 0x7ff62c061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff64c148280 0x7ff64c0af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff638005000 0x7ff64c0c48c0 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 --2- 172.21.15.73:0/2777758547 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff64c147e00 0x7ff64c0af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.718+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 >> 172.21.15.73:0/2777758547 conn(0x7ff64c0cb010 msgr2=0x7ff64c14e7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:33.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.719+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 shutdown_connections 2023-11-28T04:24:33.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.720+0000 7ff64bfff700 1 -- 172.21.15.73:0/2777758547 wait complete. 2023-11-28T04:24:33.725 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T04:24:33.735 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T04:24:33.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.852+0000 7f41c44d0700 1 Processor -- start 2023-11-28T04:24:33.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 -- start start 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f41bc0c5ac0 0x7f41bc0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41bc153390 0x7f41bc0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 0x7f41bc156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc153990 con 0x7f41bc153390 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc155f20 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.853+0000 7f41c44d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc156200 con 0x7f41bc156690 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41bc153390 0x7f41bc0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f41bc153390 0x7f41bc0c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41706/0 (socket says 172.21.15.73:41706) 2023-11-28T04:24:33.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 -- 172.21.15.73:0/2891470425 learned_addr learned my addr 172.21.15.73:0/2891470425 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c2a6d700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 0x7f41bc156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c226c700 1 --2- 172.21.15.73:0/2891470425 >> v2:172.21.15.150:3300/0 conn(0x7f41bc0c5ac0 0x7f41bc0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 -- 172.21.15.73:0/2891470425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 msgr2=0x7f41bc156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 0x7f41bc156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 -- 172.21.15.73:0/2891470425 >> v2:172.21.15.150:3300/0 conn(0x7f41bc0c5ac0 msgr2=0x7f41bc0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/2891470425 >> v2:172.21.15.150:3300/0 conn(0x7f41bc0c5ac0 0x7f41bc0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 -- 172.21.15.73:0/2891470425 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc0bff70 con 0x7f41bc153390 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c2a6d700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 0x7f41bc156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc153390 0x7f41bc0c5ea0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7f41bc0c9050 tx=0x7f41ac0090f0).ready entity=mon.0 client_cookie=3410162bc7ffa56c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c126a700 1 -- 172.21.15.73:0/2891470425 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00b600 con 0x7f41bc153390 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c126a700 1 -- 172.21.15.73:0/2891470425 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41ac00b760 con 0x7f41bc153390 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.854+0000 7f41c126a700 1 -- 172.21.15.73:0/2891470425 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41ac00bb00 con 0x7f41bc153390 2023-11-28T04:24:33.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 -- 172.21.15.73:0/2891470425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc153390 msgr2=0x7f41bc0c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc153390 0x7f41bc0c5ea0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 rx=0x7f41bc0c9050 tx=0x7f41ac0090f0).stop 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 -- 172.21.15.73:0/2891470425 shutdown_connections 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc156690 0x7f41bc156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 --2- 172.21.15.73:0/2891470425 >> v2:172.21.15.150:3300/0 conn(0x7f41bc0c5ac0 0x7f41bc0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 --2- 172.21.15.73:0/2891470425 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc153390 0x7f41bc0c5ea0 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 -- 172.21.15.73:0/2891470425 >> 172.21.15.73:0/2891470425 conn(0x7f41bc062700 msgr2=0x7f41bc062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 -- 172.21.15.73:0/2891470425 shutdown_connections 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.855+0000 7f41c44d0700 1 -- 172.21.15.73:0/2891470425 wait complete. 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.856+0000 7f41c44d0700 1 Processor -- start 2023-11-28T04:24:33.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.856+0000 7f41c44d0700 1 -- start start 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc153390 0x7f41bc0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 0x7f41bc162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0ae540 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0ad930 con 0x7f41bc156690 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0adc10 con 0x7f41bc153390 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41708/0 (socket says 172.21.15.73:41708) 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 -- 172.21.15.73:0/3266451796 learned_addr learned my addr 172.21.15.73:0/3266451796 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:33.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc153390 0x7f41bc0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c2a6d700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 0x7f41bc162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 -- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc153390 msgr2=0x7f41bc0ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc153390 0x7f41bc0ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 -- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 msgr2=0x7f41bc162140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 0x7f41bc162140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 -- 172.21.15.73:0/3266451796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac009370 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c2a6d700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 0x7f41bc162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c226c700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f41b8009fa0 tx=0x7f41b8009880).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b800ed20 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41b800ee80 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc0adef0 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41b8014610 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.857+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc158b80 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.858+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc067e10 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.859+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f41b801d3b0 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.859+0000 7f41b27fc700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 0x7f41a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.859+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 5639+0+0 (secure 0 0 0) 0x7f41b807a460 con 0x7f41bc0c5ac0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.859+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 0x7f41a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.860+0000 7f41c1a6b700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 0x7f41a0061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f41ac009fd0 tx=0x7f41ac017000).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:33.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:33.863+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f41b804b8b0 con 0x7f41bc0c5ac0 2023-11-28T04:24:34.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.025+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f41bc06c860 con 0x7f41bc0c5ac0 2023-11-28T04:24:34.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.720+0000 7f41b27fc700 1 -- 172.21.15.73:0/3266451796 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v195) v1 ==== 170+0+0 (secure 0 0 0) 0x7f41b804eed0 con 0x7f41bc0c5ac0 2023-11-28T04:24:34.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.725+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 msgr2=0x7f41a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.725+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 0x7f41a0061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7f41ac009fd0 tx=0x7f41ac017000).stop 2023-11-28T04:24:34.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.725+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 msgr2=0x7f41bc158690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.725+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 rx=0x7f41b8009fa0 tx=0x7f41b8009880).stop 2023-11-28T04:24:34.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 shutdown_connections 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f41a005f150 0x7f41a0061610 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f41bc156690 0x7f41bc162140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f41bc0c5ac0 0x7f41bc158690 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 --2- 172.21.15.73:0/3266451796 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f41bc153390 0x7f41bc0ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.729+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 >> 172.21.15.73:0/3266451796 conn(0x7f41bc062700 msgr2=0x7f41bc0c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.731+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 shutdown_connections 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.732+0000 7f41c44d0700 1 -- 172.21.15.73:0/3266451796 wait complete. 2023-11-28T04:24:34.734 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T04:24:34.752 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T04:24:34.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.868+0000 7fe482a54700 1 Processor -- start 2023-11-28T04:24:34.869 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.868+0000 7fe482a54700 1 -- start start 2023-11-28T04:24:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe482a54700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 0x7fe47c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe482a54700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe47c158320 0x7fe47c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe482a54700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe47c158cc0 con 0x7fe47c0c6190 2023-11-28T04:24:34.871 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe482a54700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c0c72d0 con 0x7fe47c158320 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe482a54700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe47c064570 con 0x7fe47c098d10 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fe47c158320 0x7fe47c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe47b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe47c0c6190 0x7fe47c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40744/0 (socket says 172.21.15.73:40744) 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe47b7fe700 1 -- 172.21.15.73:0/3191479969 learned_addr learned my addr 172.21.15.73:0/3191479969 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe47bfff700 1 --2- 172.21.15.73:0/3191479969 >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 0x7fe47c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 -- 172.21.15.73:0/3191479969 >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 msgr2=0x7fe47c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 --2- 172.21.15.73:0/3191479969 >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 0x7fe47c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 -- 172.21.15.73:0/3191479969 >> v1:172.21.15.73:6789/0 conn(0x7fe47c0c6190 legacy=0x7fe47c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe47affd700 1 -- 172.21.15.73:0/3191479969 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3231310532 0 0) 0x7fe470001110 con 0x7fe47c0c6190 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 -- 172.21.15.73:0/3191479969 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe47c154bc0 con 0x7fe47c158320 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe47bfff700 1 --2- 172.21.15.73:0/3191479969 >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 0x7fe47c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.869+0000 7fe480ff1700 1 --2- 172.21.15.73:0/3191479969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c158320 0x7fe47c1587d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 rx=0x7fe47c083f80 tx=0x7fe474009780).ready entity=mon.0 client_cookie=e1ed8200d8318f86 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe47affd700 1 -- 172.21.15.73:0/3191479969 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47400e070 con 0x7fe47c158320 2023-11-28T04:24:34.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe47affd700 1 -- 172.21.15.73:0/3191479969 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe47400bc30 con 0x7fe47c158320 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe47affd700 1 -- 172.21.15.73:0/3191479969 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe474012410 con 0x7fe47c158320 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 -- 172.21.15.73:0/3191479969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c158320 msgr2=0x7fe47c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 --2- 172.21.15.73:0/3191479969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c158320 0x7fe47c1587d0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 rx=0x7fe47c083f80 tx=0x7fe474009780).stop 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 -- 172.21.15.73:0/3191479969 shutdown_connections 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 --2- 172.21.15.73:0/3191479969 >> v2:172.21.15.150:3300/0 conn(0x7fe47c098d10 0x7fe47c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 --2- 172.21.15.73:0/3191479969 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c158320 0x7fe47c1587d0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 -- 172.21.15.73:0/3191479969 >> 172.21.15.73:0/3191479969 conn(0x7fe47c0cb310 msgr2=0x7fe47c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 -- 172.21.15.73:0/3191479969 shutdown_connections 2023-11-28T04:24:34.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.870+0000 7fe482a54700 1 -- 172.21.15.73:0/3191479969 wait complete. 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.871+0000 7fe482a54700 1 Processor -- start 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 -- start start 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe47c098d10 0x7fe47c0aecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe47c158320 0x7fe47c0a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0b0110 con 0x7fe47c0c6190 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0af500 con 0x7fe47c098d10 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe482a54700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe47c0af7e0 con 0x7fe47c158320 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe47b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe47b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41722/0 (socket says 172.21.15.73:41722) 2023-11-28T04:24:34.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe47b7fe700 1 -- 172.21.15.73:0/4221958008 learned_addr learned my addr 172.21.15.73:0/4221958008 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.872+0000 7fe47bfff700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe47c098d10 0x7fe47c0aecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe480ff1700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe47c158320 0x7fe47c0a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 -- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe47c158320 msgr2=0x7fe47c0a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe47c158320 0x7fe47c0a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 -- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe47c098d10 msgr2=0x7fe47c0aecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe47c098d10 0x7fe47c0aecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 -- 172.21.15.73:0/4221958008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe474009a20 con 0x7fe47c0c6190 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe47b7fe700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7fe47000c470 tx=0x7fe470007a50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe47000ce20 con 0x7fe47c0c6190 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4700072e0 con 0x7fe47c0c6190 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe47c0afac0 con 0x7fe47c0c6190 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe470007660 con 0x7fe47c0c6190 2023-11-28T04:24:34.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.873+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe47c0af1a0 con 0x7fe47c0c6190 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.874+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe47c06c860 con 0x7fe47c0c6190 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.874+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe4700059b0 con 0x7fe47c0c6190 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.875+0000 7fe463fff700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 0x7fe4680615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.875+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6021+0+0 (secure 0 0 0) 0x7fe470079760 con 0x7fe47c0c6190 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.875+0000 7fe47bfff700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 0x7fe4680615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:34.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.875+0000 7fe47bfff700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 0x7fe4680615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe46c006d80 tx=0x7fe46c008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:34.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:34.879+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe47004aa90 con 0x7fe47c0c6190 2023-11-28T04:24:35.085 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.084+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe47c0c66c0 con 0x7fe47c0c6190 2023-11-28T04:24:35.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.089+0000 7fe463fff700 1 -- 172.21.15.73:0/4221958008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 23 and data pool 24 v68) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe47004e0b0 con 0x7fe47c0c6190 2023-11-28T04:24:35.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.092+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 msgr2=0x7fe4680615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.092+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 0x7fe4680615e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7fe46c006d80 tx=0x7fe46c008040).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.092+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 msgr2=0x7fe47c148960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.092+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 rx=0x7fe47000c470 tx=0x7fe470007a50).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 shutdown_connections 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe46805f120 0x7fe4680615e0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0x7fe46c006d80 tx=0x7fe46c008040).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe47c158320 0x7fe47c0a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe47c098d10 0x7fe47c0aecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 --2- 172.21.15.73:0/4221958008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe47c0c6190 0x7fe47c148960 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 >> 172.21.15.73:0/4221958008 conn(0x7fe47c0cb310 msgr2=0x7fe47c0ca6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.093+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 shutdown_connections 2023-11-28T04:24:35.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.095+0000 7fe482a54700 1 -- 172.21.15.73:0/4221958008 wait complete. 2023-11-28T04:24:35.102 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 23 and data pool 24 2023-11-28T04:24:35.122 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:35.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.234+0000 7f6686baa700 1 Processor -- start 2023-11-28T04:24:35.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.234+0000 7f6686baa700 1 -- start start 2023-11-28T04:24:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6686baa700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f66800caab0 0x7f66800aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6686baa700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66800a9a90 con 0x7f66800ca180 2023-11-28T04:24:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6686baa700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6680154ce0 con 0x7f66800cae90 2023-11-28T04:24:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6686baa700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66800a8e80 con 0x7f66800caab0 2023-11-28T04:24:35.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6685147700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f66800ca180 0x7f668015e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40750/0 (socket says 172.21.15.73:40750) 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.235+0000 7f6685147700 1 -- 172.21.15.73:0/2552492874 learned_addr learned my addr 172.21.15.73:0/2552492874 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f6684946700 1 --2- 172.21.15.73:0/2552492874 >> v2:172.21.15.150:3300/0 conn(0x7f66800caab0 0x7f66800aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3654212761 0 0) 0x7f66800a9a90 con 0x7f66800ca180 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66680033f0 con 0x7f66800ca180 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3055667529 0 0) 0x7f66680033f0 con 0x7f66800ca180 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 >> v2:172.21.15.150:3300/0 conn(0x7f66800caab0 msgr2=0x7f66800aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 --2- 172.21.15.73:0/2552492874 >> v2:172.21.15.150:3300/0 conn(0x7f66800caab0 0x7f66800aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 >> v1:172.21.15.150:6789/0 conn(0x7f66800cae90 legacy=0x7f66800c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800a9160 con 0x7f66800ca180 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6678001470 con 0x7f66800ca180 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 >> v1:172.21.15.73:6789/0 conn(0x7f66800ca180 legacy=0x7f668015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66680038e0 0x7f6668003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 0x7f6668004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668005000 0x7f666800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66800a8e80 con 0x7f6668004690 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f666800bf60 con 0x7f66680038e0 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f666800c0f0 con 0x7f6668005000 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 0x7f6668004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f6684946700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66680038e0 0x7f6668003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f6685147700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668005000 0x7f666800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 -- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668005000 msgr2=0x7f666800b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668005000 0x7f666800b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 -- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66680038e0 msgr2=0x7f6668003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66680038e0 0x7f6668003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 -- 172.21.15.73:0/2552492874 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66680033f0 con 0x7f6668004690 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f6685147700 1 -- 172.21.15.73:0/2552492874 reap_dead start 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.236+0000 7f667ffff700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 0x7f6668004b10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7f6674001730 tx=0x7f6674007bb0).ready entity=mon.0 client_cookie=241dd28333588b36 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674007020 con 0x7f6668004690 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6674007180 con 0x7f6668004690 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f667f7fe700 1 -- 172.21.15.73:0/2552492874 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6674007500 con 0x7f6668004690 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 -- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 msgr2=0x7f6668004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 0x7f6668004b10 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 rx=0x7f6674001730 tx=0x7f6674007bb0).stop 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 -- 172.21.15.73:0/2552492874 shutdown_connections 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 --2- 172.21.15.73:0/2552492874 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6668004690 0x7f6668004b10 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 -- 172.21.15.73:0/2552492874 >> 172.21.15.73:0/2552492874 conn(0x7f6680094920 msgr2=0x7f668006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 -- 172.21.15.73:0/2552492874 shutdown_connections 2023-11-28T04:24:35.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.237+0000 7f6686baa700 1 -- 172.21.15.73:0/2552492874 wait complete. 2023-11-28T04:24:35.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.238+0000 7f6686baa700 1 Processor -- start 2023-11-28T04:24:35.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.238+0000 7f6686baa700 1 -- start start 2023-11-28T04:24:35.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668004690 0x7f66800c62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 0x7f6680154180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66800c67d0 con 0x7f66800ab1a0 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6680156750 con 0x7f66800ac070 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6686baa700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66800ac5a0 con 0x7f6668004690 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41744/0 (socket says 172.21.15.73:41744) 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 -- 172.21.15.73:0/2512682783 learned_addr learned my addr 172.21.15.73:0/2512682783 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f667ffff700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 0x7f6680154180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6684946700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668004690 0x7f66800c62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 -- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668004690 msgr2=0x7f66800c62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668004690 0x7f66800c62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 -- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 msgr2=0x7f6680154180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.239+0000 7f6685147700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 0x7f6680154180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f6685147700 1 -- 172.21.15.73:0/2512682783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6680155e20 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f667ffff700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 0x7f6680154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f6685147700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7f6680084860 tx=0x7f6678001010).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678010520 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66780106a0 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6680156100 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6678010a60 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.240+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66800c56f0 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.241+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66800c5910 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.241+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6678014430 con 0x7f66800ab1a0 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.241+0000 7f667cff9700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 0x7f666c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.242+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f66780799f0 con 0x7f66800ab1a0 2023-11-28T04:24:35.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.242+0000 7f6684946700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 0x7f666c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.242+0000 7f6684946700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 0x7f666c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f66700094d0 tx=0x7f667000b040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.246+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f66800c5910 con 0x7f66800ab1a0 2023-11-28T04:24:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.409+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f66800c5910 con 0x7f66800ab1a0 2023-11-28T04:24:35.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.410+0000 7f667cff9700 1 -- 172.21.15.73:0/2512682783 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f66800c5910 con 0x7f66800ab1a0 2023-11-28T04:24:35.411 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:35.411 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":196,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:35.085684+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 msgr2=0x7f666c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 0x7f666c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7f66700094d0 tx=0x7f667000b040).stop 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 msgr2=0x7f66800c2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 rx=0x7f6680084860 tx=0x7f6678001010).stop 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 shutdown_connections 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f666c05f150 0x7f666c061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f66800ab1a0 0x7f66800c2310 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6668004690 0x7f66800c62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 --2- 172.21.15.73:0/2512682783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f66800ac070 0x7f6680154180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 >> 172.21.15.73:0/2512682783 conn(0x7f6680094920 msgr2=0x7f66800650f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 shutdown_connections 2023-11-28T04:24:35.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.412+0000 7f6686baa700 1 -- 172.21.15.73:0/2512682783 wait complete. 2023-11-28T04:24:35.425 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:35.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.538+0000 7fb1761cc700 1 Processor -- start 2023-11-28T04:24:35.539 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.538+0000 7fb1761cc700 1 -- start start 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1761cc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1700a8380 0x7fb1700bf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1761cc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb1700a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1761cc700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1700bf0e0 con 0x7fb170152190 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1761cc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1700be510 con 0x7fb1700a8380 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1761cc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170152570 con 0x7fb1700bfa60 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1749c9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb1700a8380 0x7fb1700bf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1751ca700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb170152190 0x7fb1700c5ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40756/0 (socket says 172.21.15.73:40756) 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1751ca700 1 -- 172.21.15.73:0/2406405420 learned_addr learned my addr 172.21.15.73:0/2406405420 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1759cb700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb1700a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.542 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1749c9700 1 -- 172.21.15.73:0/2406405420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 msgr2=0x7fb1700a5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1749c9700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb1700a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1749c9700 1 -- 172.21.15.73:0/2406405420 >> v1:172.21.15.73:6789/0 conn(0x7fb170152190 legacy=0x7fb1700c5ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb167fff700 1 -- 172.21.15.73:0/2406405420 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 391282202 0 0) 0x7fb1700bf0e0 con 0x7fb170152190 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.539+0000 7fb1749c9700 1 -- 172.21.15.73:0/2406405420 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1700be7b0 con 0x7fb1700a8380 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1749c9700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700a8380 0x7fb1700bf570 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7fb160009760 tx=0x7fb160009060).ready entity=mon.0 client_cookie=c3db513357f5eb21 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb167fff700 1 -- 172.21.15.73:0/2406405420 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000b580 con 0x7fb1700a8380 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb167fff700 1 -- 172.21.15.73:0/2406405420 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb16000b6e0 con 0x7fb1700a8380 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb167fff700 1 -- 172.21.15.73:0/2406405420 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb16000ba60 con 0x7fb1700a8380 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 -- 172.21.15.73:0/2406405420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700a8380 msgr2=0x7fb1700bf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700a8380 0x7fb1700bf570 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 rx=0x7fb160009760 tx=0x7fb160009060).stop 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 -- 172.21.15.73:0/2406405420 shutdown_connections 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb1700a5c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.543 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 --2- 172.21.15.73:0/2406405420 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb1700a8380 0x7fb1700bf570 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 -- 172.21.15.73:0/2406405420 >> 172.21.15.73:0/2406405420 conn(0x7fb170083000 msgr2=0x7fb170092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 -- 172.21.15.73:0/2406405420 shutdown_connections 2023-11-28T04:24:35.544 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.540+0000 7fb1761cc700 1 -- 172.21.15.73:0/2406405420 wait complete. 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.541+0000 7fb1761cc700 1 Processor -- start 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.541+0000 7fb1761cc700 1 -- start start 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700a8380 0x7fb17014b480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb17014c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb170064da0 con 0x7fb170152190 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1700641e0 con 0x7fb1700a8380 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1761cc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb170064470 con 0x7fb1700bfa60 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1759cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.542+0000 7fb1759cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41766/0 (socket says 172.21.15.73:41766) 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 -- 172.21.15.73:0/3101306318 learned_addr learned my addr 172.21.15.73:0/3101306318 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:35.547 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1749c9700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb17014c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1751ca700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700a8380 0x7fb17014b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 -- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 msgr2=0x7fb17014c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb17014c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 -- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700a8380 msgr2=0x7fb17014b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700a8380 0x7fb17014b480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 -- 172.21.15.73:0/3101306318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1600092a0 con 0x7fb170152190 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1759cb700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7fb16800c550 tx=0x7fb168008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168010580 con 0x7fb170152190 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1680106e0 con 0x7fb170152190 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb170064750 con 0x7fb170152190 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb168010a60 con 0x7fb170152190 2023-11-28T04:24:35.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.543+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb170065230 con 0x7fb170152190 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.544+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb170064f10 con 0x7fb170152190 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.544+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb16801b460 con 0x7fb170152190 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.545+0000 7fb1657fa700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 0x7fb154061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.545+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb168079f60 con 0x7fb170152190 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.545+0000 7fb1751ca700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 0x7fb154061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.545+0000 7fb1751ca700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 0x7fb154061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb16c0016e0 tx=0x7fb16c009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.550 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.549+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb16804b1d0 con 0x7fb170152190 2023-11-28T04:24:35.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.711+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1700bea90 con 0x7fb170152190 2023-11-28T04:24:35.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.712+0000 7fb1657fa700 1 -- 172.21.15.73:0/3101306318 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+16956 (secure 0 0 0) 0x7fb16804e7f0 con 0x7fb170152190 2023-11-28T04:24:35.713 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:35.713 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":196,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:35.085684+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 msgr2=0x7fb154061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 0x7fb154061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7fb16c0016e0 tx=0x7fb16c009040).stop 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 msgr2=0x7fb1700ac090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 rx=0x7fb16800c550 tx=0x7fb168008040).stop 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 shutdown_connections 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb15405f150 0x7fb154061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb170152190 0x7fb1700ac090 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb1700a8380 0x7fb17014b480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 --2- 172.21.15.73:0/3101306318 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb1700bfa60 0x7fb17014c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 >> 172.21.15.73:0/3101306318 conn(0x7fb170083000 msgr2=0x7fb1700667f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 shutdown_connections 2023-11-28T04:24:35.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.714+0000 7fb1761cc700 1 -- 172.21.15.73:0/3101306318 wait complete. 2023-11-28T04:24:35.728 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T04:24:35.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.842+0000 7f51314e3700 1 Processor -- start 2023-11-28T04:24:35.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.842+0000 7f51314e3700 1 -- start start 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f512c0c5ac0 0x7f512c0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f512c153390 0x7f512c0c5ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 0x7f512c156ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c153990 con 0x7f512c153390 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c155f20 con 0x7f512c0c5ac0 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f51314e3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c156200 con 0x7f512c156690 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f512a7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f512c153390 0x7f512c0c5ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f512a7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f512c153390 0x7f512c0c5ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41772/0 (socket says 172.21.15.73:41772) 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f512a7fc700 1 -- 172.21.15.73:0/4187255585 learned_addr learned my addr 172.21.15.73:0/4187255585 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f512b7fe700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 0x7f512c156ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.843+0000 7f512affd700 1 --2- 172.21.15.73:0/4187255585 >> v2:172.21.15.150:3300/0 conn(0x7f512c0c5ac0 0x7f512c0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 -- 172.21.15.73:0/4187255585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 msgr2=0x7f512c156ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 0x7f512c156ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 -- 172.21.15.73:0/4187255585 >> v2:172.21.15.150:3300/0 conn(0x7f512c0c5ac0 msgr2=0x7f512c0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 --2- 172.21.15.73:0/4187255585 >> v2:172.21.15.150:3300/0 conn(0x7f512c0c5ac0 0x7f512c0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 -- 172.21.15.73:0/4187255585 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c0bff70 con 0x7f512c153390 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512b7fe700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 0x7f512c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f512a7fc700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c153390 0x7f512c0c5ea0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7f512c0c9050 tx=0x7f51140090f0).ready entity=mon.0 client_cookie=7d9f9d1dfd8b8dc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f5129ffb700 1 -- 172.21.15.73:0/4187255585 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400b600 con 0x7f512c153390 2023-11-28T04:24:35.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f5129ffb700 1 -- 172.21.15.73:0/4187255585 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511400b760 con 0x7f512c153390 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f5129ffb700 1 -- 172.21.15.73:0/4187255585 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400bb00 con 0x7f512c153390 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 -- 172.21.15.73:0/4187255585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c153390 msgr2=0x7f512c0c5ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c153390 0x7f512c0c5ea0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 rx=0x7f512c0c9050 tx=0x7f51140090f0).stop 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 -- 172.21.15.73:0/4187255585 shutdown_connections 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c156690 0x7f512c156ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 --2- 172.21.15.73:0/4187255585 >> v2:172.21.15.150:3300/0 conn(0x7f512c0c5ac0 0x7f512c0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 --2- 172.21.15.73:0/4187255585 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c153390 0x7f512c0c5ea0 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 -- 172.21.15.73:0/4187255585 >> 172.21.15.73:0/4187255585 conn(0x7f512c062700 msgr2=0x7f512c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 -- 172.21.15.73:0/4187255585 shutdown_connections 2023-11-28T04:24:35.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.844+0000 7f51314e3700 1 -- 172.21.15.73:0/4187255585 wait complete. 2023-11-28T04:24:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 Processor -- start 2023-11-28T04:24:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 -- start start 2023-11-28T04:24:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f512c0c5ac0 0x7f512c158690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c153390 0x7f512c0ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0ae540 con 0x7f512c156690 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0ad930 con 0x7f512c0c5ac0 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f51314e3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0adc10 con 0x7f512c153390 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f512b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41786/0 (socket says 172.21.15.73:41786) 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.846+0000 7f512b7fe700 1 -- 172.21.15.73:0/2421588648 learned_addr learned my addr 172.21.15.73:0/2421588648 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512affd700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f512c0c5ac0 0x7f512c158690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512a7fc700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c153390 0x7f512c0ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 -- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c153390 msgr2=0x7f512c0ae9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c153390 0x7f512c0ae9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 -- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f512c0c5ac0 msgr2=0x7f512c158690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f512c0c5ac0 0x7f512c158690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:35.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 -- 172.21.15.73:0/2421588648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114009370 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f512b7fe700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7f512400d860 tx=0x7f5124008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011590 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51240116f0 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0adef0 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5124011a90 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.847+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c158b80 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.848+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c067e10 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.848+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f5124012420 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.849+0000 7f5122ffd700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 0x7f510c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.849+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5124079b50 con 0x7f512c156690 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.849+0000 7f512affd700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 0x7f510c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:35.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.849+0000 7f512affd700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 0x7f510c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f511c0075a0 tx=0x7f511c0097e0).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:35.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:35.853+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f512404adc0 con 0x7f512c156690 2023-11-28T04:24:36.059 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.058+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f512c06c860 con 0x7f512c156690 2023-11-28T04:24:36.090 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.089+0000 7f5122ffd700 1 -- 172.21.15.73:0/2421588648 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) v1 ==== 118+0+0 (secure 0 0 0) 0x7f512404e3e0 con 0x7f512c156690 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 msgr2=0x7f510c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 0x7f510c061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f511c0075a0 tx=0x7f511c0097e0).stop 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 msgr2=0x7f512c162140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 rx=0x7f512400d860 tx=0x7f5124008040).stop 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 shutdown_connections 2023-11-28T04:24:36.093 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f510c05f150 0x7f510c061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f512c156690 0x7f512c162140 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f512c0c5ac0 0x7f512c158690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 --2- 172.21.15.73:0/2421588648 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f512c153390 0x7f512c0ae9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 >> 172.21.15.73:0/2421588648 conn(0x7f512c062700 msgr2=0x7f512c0c3830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 shutdown_connections 2023-11-28T04:24:36.094 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.091+0000 7f51314e3700 1 -- 172.21.15.73:0/2421588648 wait complete. 2023-11-28T04:24:36.107 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T04:24:36.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T04:24:36.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.222+0000 7f4d077de700 1 Processor -- start 2023-11-28T04:24:36.224 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 -- start start 2023-11-28T04:24:36.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d000a8280 0x7f4d000a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4d00155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d00067770 con 0x7f4d0014e600 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d00159180 con 0x7f4d000a8280 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.223+0000 7f4d077de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000be8d0 con 0x7f4d00158160 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d05d7b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4d0014e600 0x7f4d0014ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40766/0 (socket says 172.21.15.73:40766) 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d05d7b700 1 -- 172.21.15.73:0/3088667671 learned_addr learned my addr 172.21.15.73:0/3088667671 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723164410 0 0) 0x7f4d00067770 con 0x7f4d0014e600 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d04d79700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4d00155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d0557a700 1 --2- 172.21.15.73:0/3088667671 >> v2:172.21.15.150:3300/0 conn(0x7f4d000a8280 0x7f4d000a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4cec0033f0 con 0x7f4d0014e600 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314084971 0 0) 0x7f4cec0033f0 con 0x7f4d0014e600 2023-11-28T04:24:36.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 msgr2=0x7f4d00155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4d00155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 >> v2:172.21.15.150:3300/0 conn(0x7f4d000a8280 msgr2=0x7f4d000a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 --2- 172.21.15.73:0/3088667671 >> v2:172.21.15.150:3300/0 conn(0x7f4d000a8280 0x7f4d000a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d00158600 con 0x7f4d0014e600 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f4cf80017f0 con 0x7f4d0014e600 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 >> v1:172.21.15.73:6789/0 conn(0x7f4d0014e600 legacy=0x7f4d0014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cec0038e0 0x7f4cec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cec004690 0x7f4cec004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4cec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d00159180 con 0x7f4cec004690 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000be8d0 con 0x7f4cec0038e0 2023-11-28T04:24:36.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4cec00b8e0 con 0x7f4d00158160 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d04d79700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cec004690 0x7f4cec004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d0557a700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cec0038e0 0x7f4cec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.224+0000 7f4d05d7b700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4cec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 -- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 msgr2=0x7f4cec005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4cec005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 -- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cec0038e0 msgr2=0x7f4cec003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4cec0038e0 0x7f4cec003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 -- 172.21.15.73:0/3088667671 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cec0033f0 con 0x7f4cec004690 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d05d7b700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d00158160 0x7f4cec005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d04d79700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cec004690 0x7f4cec004b10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7f4cf0009a60 tx=0x7f4cf0009060).ready entity=mon.0 client_cookie=5a2f8a4665816efc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0009ea0 con 0x7f4cec004690 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf0009430 con 0x7f4cec004690 2023-11-28T04:24:36.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4cf7fff700 1 -- 172.21.15.73:0/3088667671 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf000e870 con 0x7f4cec004690 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 -- 172.21.15.73:0/3088667671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cec004690 msgr2=0x7f4cec004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 --2- 172.21.15.73:0/3088667671 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4cec004690 0x7f4cec004b10 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 rx=0x7f4cf0009a60 tx=0x7f4cf0009060).stop 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d05d7b700 1 -- 172.21.15.73:0/3088667671 reap_dead start 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 -- 172.21.15.73:0/3088667671 shutdown_connections 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 -- 172.21.15.73:0/3088667671 >> 172.21.15.73:0/3088667671 conn(0x7f4d00062700 msgr2=0x7f4d00062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 -- 172.21.15.73:0/3088667671 shutdown_connections 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.225+0000 7f4d077de700 1 -- 172.21.15.73:0/3088667671 wait complete. 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 Processor -- start 2023-11-28T04:24:36.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 -- start start 2023-11-28T04:24:36.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cec004690 0x7f4d000a95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0014d8f0 0x7f4d0014dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000aae00 con 0x7f4d000bee70 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d0014e240 con 0x7f4d0014d8f0 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.227+0000 7f4d077de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d000aa1f0 con 0x7f4cec004690 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41804/0 (socket says 172.21.15.73:41804) 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 -- 172.21.15.73:0/2455185125 learned_addr learned my addr 172.21.15.73:0/2455185125 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d0557a700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cec004690 0x7f4d000a95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d05d7b700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0014d8f0 0x7f4d0014dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 -- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cec004690 msgr2=0x7f4d000a95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cec004690 0x7f4d000a95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 -- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0014d8f0 msgr2=0x7f4d0014dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0014d8f0 0x7f4d0014dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 -- 172.21.15.73:0/2455185125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d000aa4d0 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d04d79700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7f4cf000e4b0 tx=0x7f4cf001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf0020070 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4cf001b960 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cf001bd20 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d000aa7b0 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.228+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d000a9ae0 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.229+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d0006c860 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.230+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4cf0024620 con 0x7f4d000bee70 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.230+0000 7f4cf57fa700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 0x7f4ce4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:36.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.230+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4cf007b450 con 0x7f4d000bee70 2023-11-28T04:24:36.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.230+0000 7f4d0557a700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 0x7f4ce4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:36.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.230+0000 7f4d0557a700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 0x7f4ce4061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4cfc006d80 tx=0x7f4cfc009060).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:36.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.234+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4cf004c720 con 0x7f4d000bee70 2023-11-28T04:24:36.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:36.438+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4d001588e0 con 0x7f4d000bee70 2023-11-28T04:24:37.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.099+0000 7f4cf57fa700 1 -- 172.21.15.73:0/2455185125 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4cf0016070 con 0x7f4d000bee70 2023-11-28T04:24:37.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 msgr2=0x7f4ce4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 0x7f4ce4061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f4cfc006d80 tx=0x7f4cfc009060).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 msgr2=0x7f4d000bf2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 rx=0x7f4cf000e4b0 tx=0x7f4cf001a000).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 shutdown_connections 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4ce405f150 0x7f4ce4061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d0014d8f0 0x7f4d0014dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4cec004690 0x7f4d000a95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 --2- 172.21.15.73:0/2455185125 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d000bee70 0x7f4d000bf2f0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 >> 172.21.15.73:0/2455185125 conn(0x7f4d00062700 msgr2=0x7f4d000a62f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 shutdown_connections 2023-11-28T04:24:37.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.101+0000 7f4d077de700 1 -- 172.21.15.73:0/2455185125 wait complete. 2023-11-28T04:24:37.115 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T04:24:37.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:37.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.228+0000 7f9a0a986700 1 Processor -- start 2023-11-28T04:24:37.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.228+0000 7f9a0a986700 1 -- start start 2023-11-28T04:24:37.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a040caab0 0x7f9a040aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9a040cae90 0x7f9a040c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a040aa960 0x7f9a040c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040c1490 con 0x7f9a040caab0 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a040a9a90 con 0x7f9a040cae90 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a0a986700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a04154ce0 con 0x7f9a040aa960 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a040caab0 0x7f9a040aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9a040caab0 0x7f9a040aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41814/0 (socket says 172.21.15.73:41814) 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 -- 172.21.15.73:0/2381148788 learned_addr learned my addr 172.21.15.73:0/2381148788 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 -- 172.21.15.73:0/2381148788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a040aa960 msgr2=0x7f9a040c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 --2- 172.21.15.73:0/2381148788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a040aa960 0x7f9a040c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 -- 172.21.15.73:0/2381148788 >> v2:172.21.15.150:3300/0 conn(0x7f9a040cae90 msgr2=0x7f9a040c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a037fe700 1 --2- 172.21.15.73:0/2381148788 >> v2:172.21.15.150:3300/0 conn(0x7f9a040cae90 0x7f9a040c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 --2- 172.21.15.73:0/2381148788 >> v2:172.21.15.150:3300/0 conn(0x7f9a040cae90 0x7f9a040c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 -- 172.21.15.73:0/2381148788 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a040a8e80 con 0x7f9a040caab0 2023-11-28T04:24:37.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a03fff700 1 --2- 172.21.15.73:0/2381148788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040aa470 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f9a040b1440 tx=0x7f99f4009060).ready entity=mon.0 client_cookie=11fd64d15dd6ad2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a02ffd700 1 -- 172.21.15.73:0/2381148788 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4005880 con 0x7f9a040caab0 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a02ffd700 1 -- 172.21.15.73:0/2381148788 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f40059e0 con 0x7f9a040caab0 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.229+0000 7f9a02ffd700 1 -- 172.21.15.73:0/2381148788 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f4005d60 con 0x7f9a040caab0 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 -- 172.21.15.73:0/2381148788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 msgr2=0x7f9a040aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 --2- 172.21.15.73:0/2381148788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040aa470 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 rx=0x7f9a040b1440 tx=0x7f99f4009060).stop 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 -- 172.21.15.73:0/2381148788 shutdown_connections 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 --2- 172.21.15.73:0/2381148788 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040aa470 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 --2- 172.21.15.73:0/2381148788 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a040aa960 0x7f9a040c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 --2- 172.21.15.73:0/2381148788 >> v2:172.21.15.150:3300/0 conn(0x7f9a040cae90 0x7f9a040c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 -- 172.21.15.73:0/2381148788 >> 172.21.15.73:0/2381148788 conn(0x7f9a04094920 msgr2=0x7f9a0406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 -- 172.21.15.73:0/2381148788 shutdown_connections 2023-11-28T04:24:37.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.230+0000 7f9a0a986700 1 -- 172.21.15.73:0/2381148788 wait complete. 2023-11-28T04:24:37.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.231+0000 7f9a0a986700 1 Processor -- start 2023-11-28T04:24:37.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.231+0000 7f9a0a986700 1 -- start start 2023-11-28T04:24:37.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a040cae90 0x7f9a04153280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a04155820 0x7f9a040acea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040c4880 con 0x7f9a040caab0 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040c3c70 con 0x7f9a040cae90 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a0a986700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a040c3f50 con 0x7f9a04155820 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41828/0 (socket says 172.21.15.73:41828) 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 -- 172.21.15.73:0/1802585600 learned_addr learned my addr 172.21.15.73:0/1802585600 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a037fe700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a040cae90 0x7f9a04153280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 -- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a04155820 msgr2=0x7f9a040acea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a04155820 0x7f9a040acea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:37.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 -- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a040cae90 msgr2=0x7f9a04153280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a040cae90 0x7f9a04153280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 -- 172.21.15.73:0/1802585600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f4009300 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f9a03fff700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 rx=0x7f9a04083c30 tx=0x7f99f4016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.232+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400b810 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.233+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99f400b970 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.233+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a040c4230 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.233+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f400bd30 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.233+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a04153770 con 0x7f9a040caab0 2023-11-28T04:24:37.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.233+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0406c860 con 0x7f9a040caab0 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.234+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f99f4018030 con 0x7f9a040caab0 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.234+0000 7f99ebfff700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 0x7f99f0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.234+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f99f407b210 con 0x7f9a040caab0 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.234+0000 7f9a037fe700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 0x7f99f0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.235+0000 7f9a037fe700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 0x7f99f0061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f99f8006fd0 tx=0x7f99f8008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.238+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f99f404c4e0 con 0x7f9a040caab0 2023-11-28T04:24:37.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.441+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f9a040a9160 con 0x7f9a040caab0 2023-11-28T04:24:37.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.442+0000 7f99ebfff700 1 -- 172.21.15.73:0/1802585600 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f99f4028020 con 0x7f9a040caab0 2023-11-28T04:24:37.443 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:37.443 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 msgr2=0x7f99f0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 0x7f99f0061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f99f8006fd0 tx=0x7f99f8008040).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 msgr2=0x7f9a040c4d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 rx=0x7f9a04083c30 tx=0x7f99f4016000).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 shutdown_connections 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f99f005f150 0x7f99f0061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9a04155820 0x7f9a040acea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9a040caab0 0x7f9a040c4d10 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 --2- 172.21.15.73:0/1802585600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9a040cae90 0x7f9a04153280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.443+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 >> 172.21.15.73:0/1802585600 conn(0x7f9a04094920 msgr2=0x7f9a041598b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.444+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 shutdown_connections 2023-11-28T04:24:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.444+0000 7f9a0a986700 1 -- 172.21.15.73:0/1802585600 wait complete. 2023-11-28T04:24:37.446 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:24:37.457 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:24:37.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.568+0000 7fb890214700 1 Processor -- start 2023-11-28T04:24:37.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.568+0000 7fb890214700 1 -- start start 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb890214700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb8880cae90 0x7fb8880c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb890214700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8880c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb890214700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8880a9a90 con 0x7fb8880caab0 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb890214700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8880c1490 con 0x7fb8880cae90 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb890214700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb888154ce0 con 0x7fb8880ca180 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb88dfb0700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb8880caab0 0x7fb8880aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40768/0 (socket says 172.21.15.73:40768) 2023-11-28T04:24:37.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb88dfb0700 1 -- 172.21.15.73:0/2419946850 learned_addr learned my addr 172.21.15.73:0/2419946850 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb88d7af700 1 --2- 172.21.15.73:0/2419946850 >> v2:172.21.15.150:3300/0 conn(0x7fb8880cae90 0x7fb8880c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.569+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8880c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4101660587 0 0) 0x7fb8880a9a90 con 0x7fb8880caab0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb8740033f0 con 0x7fb8880caab0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 320781549 0 0) 0x7fb8740033f0 con 0x7fb8880caab0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 msgr2=0x7fb8880c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8880c0fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 >> v2:172.21.15.150:3300/0 conn(0x7fb8880cae90 msgr2=0x7fb8880c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 --2- 172.21.15.73:0/2419946850 >> v2:172.21.15.150:3300/0 conn(0x7fb8880cae90 0x7fb8880c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880aab80 con 0x7fb8880caab0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb8840015f0 con 0x7fb8880caab0 2023-11-28T04:24:37.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 >> v1:172.21.15.73:6789/0 conn(0x7fb8880caab0 legacy=0x7fb8880aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8740038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874003e10 0x7fb874004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb874004b80 0x7fb874005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c1490 con 0x7fb874004b80 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb888154ce0 con 0x7fb874003e10 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb87400b8e0 con 0x7fb8880ca180 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb874004b80 0x7fb874005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88dfb0700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8740038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88d7af700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874003e10 0x7fb874004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 msgr2=0x7fb8740038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880ca180 0x7fb8740038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874003e10 msgr2=0x7fb874004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb874003e10 0x7fb874004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2419946850 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8740033f0 con 0x7fb874004b80 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.570+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb874004b80 0x7fb874005000 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7fb88000e1c0 tx=0x7fb88000b040).ready entity=mon.0 client_cookie=44c17e17637584c5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880009ac0 con 0x7fb874004b80 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb88000f040 con 0x7fb874004b80 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb88cfae700 1 -- 172.21.15.73:0/2419946850 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb88000eca0 con 0x7fb874004b80 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 -- 172.21.15.73:0/2419946850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb874004b80 msgr2=0x7fb874005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 --2- 172.21.15.73:0/2419946850 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb874004b80 0x7fb874005000 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 rx=0x7fb88000e1c0 tx=0x7fb88000b040).stop 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2419946850 reap_dead start 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 -- 172.21.15.73:0/2419946850 shutdown_connections 2023-11-28T04:24:37.576 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 -- 172.21.15.73:0/2419946850 >> 172.21.15.73:0/2419946850 conn(0x7fb888094920 msgr2=0x7fb88806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 -- 172.21.15.73:0/2419946850 shutdown_connections 2023-11-28T04:24:37.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.571+0000 7fb890214700 1 -- 172.21.15.73:0/2419946850 wait complete. 2023-11-28T04:24:37.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.572+0000 7fb890214700 1 Processor -- start 2023-11-28T04:24:37.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.572+0000 7fb890214700 1 -- start start 2023-11-28T04:24:37.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb888153180 0x7fb888153560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880aeaa0 0x7fb8880c89a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c7670 con 0x7fb8880c8e90 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c7950 con 0x7fb888153180 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb890214700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8880c7c30 con 0x7fb8880aeaa0 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41848/0 (socket says 172.21.15.73:41848) 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2050902536 learned_addr learned my addr 172.21.15.73:0/2050902536 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88dfb0700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb888153180 0x7fb888153560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88d7af700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880aeaa0 0x7fb8880c89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880aeaa0 msgr2=0x7fb8880c89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880aeaa0 0x7fb8880c89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb888153180 msgr2=0x7fb888153560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb888153180 0x7fb888153560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.573+0000 7fb88e7b1700 1 -- 172.21.15.73:0/2050902536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8880c7f10 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb88e7b1700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fb880014000 tx=0x7fb88001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880021030 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb880013790 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb880013b50 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8880c72b0 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.574+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8880af0e0 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.575+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb888167930 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.576+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb880024070 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.576+0000 7fb87dffb700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 0x7fb86c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.576+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb880002020 con 0x7fb8880c8e90 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.576+0000 7fb88dfb0700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 0x7fb86c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.576+0000 7fb88dfb0700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 0x7fb86c061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb8840078b0 tx=0x7fb884009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.580+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb88004cff0 con 0x7fb8880c8e90 2023-11-28T04:24:37.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.744+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8880c7440 con 0x7fb8880c8e90 2023-11-28T04:24:37.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.745+0000 7fb87dffb700 1 -- 172.21.15.73:0/2050902536 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v197) v1 ==== 74+0+16943 (secure 0 0 0) 0x7fb8800505d0 con 0x7fb8880c8e90 2023-11-28T04:24:37.746 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:37.746 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":197,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:36.087823+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 msgr2=0x7fb86c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 0x7fb86c061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fb8840078b0 tx=0x7fb884009040).stop 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 msgr2=0x7fb8880ad130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 rx=0x7fb880014000 tx=0x7fb88001a000).stop 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 shutdown_connections 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb86c05f150 0x7fb86c061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb8880c8e90 0x7fb8880ad130 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb888153180 0x7fb888153560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 --2- 172.21.15.73:0/2050902536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb8880aeaa0 0x7fb8880c89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 >> 172.21.15.73:0/2050902536 conn(0x7fb888094920 msgr2=0x7fb8880a9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 shutdown_connections 2023-11-28T04:24:37.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.747+0000 7fb890214700 1 -- 172.21.15.73:0/2050902536 wait complete. 2023-11-28T04:24:37.760 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T04:24:37.760 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:24:37.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.872+0000 7f38de343700 1 Processor -- start 2023-11-28T04:24:37.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.872+0000 7f38de343700 1 -- start start 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f38d80caab0 0x7f38d80aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80cae90 0x7f38d80c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38d80aa960 0x7f38d80c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80c1490 con 0x7f38d80aa960 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38d80a9a90 con 0x7f38d80caab0 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38de343700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8154ce0 con 0x7f38d80cae90 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38dc8e0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38d80aa960 0x7f38d80c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38dc8e0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f38d80aa960 0x7f38d80c0fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41854/0 (socket says 172.21.15.73:41854) 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38dc8e0700 1 -- 172.21.15.73:0/1010076624 learned_addr learned my addr 172.21.15.73:0/1010076624 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:37.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38d77fe700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80cae90 0x7f38d80c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.873+0000 7f38d7fff700 1 --2- 172.21.15.73:0/1010076624 >> v2:172.21.15.150:3300/0 conn(0x7f38d80caab0 0x7f38d80aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 -- 172.21.15.73:0/1010076624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80cae90 msgr2=0x7f38d80c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80cae90 0x7f38d80c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 -- 172.21.15.73:0/1010076624 >> v2:172.21.15.150:3300/0 conn(0x7f38d80caab0 msgr2=0x7f38d80aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/1010076624 >> v2:172.21.15.150:3300/0 conn(0x7f38d80caab0 0x7f38d80aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 -- 172.21.15.73:0/1010076624 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d80a8e80 con 0x7f38d80aa960 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d80aa960 0x7f38d80c0fa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f38d80848c0 tx=0x7f38d0009840).ready entity=mon.0 client_cookie=f59d49487bfb1688 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38d6ffd700 1 -- 172.21.15.73:0/1010076624 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000e070 con 0x7f38d80aa960 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38d6ffd700 1 -- 172.21.15.73:0/1010076624 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d000b970 con 0x7f38d80aa960 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38d6ffd700 1 -- 172.21.15.73:0/1010076624 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000bcf0 con 0x7f38d80aa960 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 -- 172.21.15.73:0/1010076624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d80aa960 msgr2=0x7f38d80c0fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d80aa960 0x7f38d80c0fa0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 rx=0x7f38d80848c0 tx=0x7f38d0009840).stop 2023-11-28T04:24:37.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 -- 172.21.15.73:0/1010076624 shutdown_connections 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d80aa960 0x7f38d80c0fa0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 --2- 172.21.15.73:0/1010076624 >> v2:172.21.15.150:3300/0 conn(0x7f38d80caab0 0x7f38d80aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 --2- 172.21.15.73:0/1010076624 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80cae90 0x7f38d80c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 -- 172.21.15.73:0/1010076624 >> 172.21.15.73:0/1010076624 conn(0x7f38d8094920 msgr2=0x7f38d806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 -- 172.21.15.73:0/1010076624 shutdown_connections 2023-11-28T04:24:37.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.874+0000 7f38de343700 1 -- 172.21.15.73:0/1010076624 wait complete. 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.875+0000 7f38de343700 1 Processor -- start 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 -- start start 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80caab0 0x7f38d81516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d80cae90 0x7f38d80c3910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8099220 con 0x7f38d8151bc0 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d80c42f0 con 0x7f38d80cae90 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.876+0000 7f38de343700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d8098c60 con 0x7f38d80caab0 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41856/0 (socket says 172.21.15.73:41856) 2023-11-28T04:24:37.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 -- 172.21.15.73:0/827982720 learned_addr learned my addr 172.21.15.73:0/827982720 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38d7fff700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80caab0 0x7f38d81516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38d77fe700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d80cae90 0x7f38d80c3910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 -- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80caab0 msgr2=0x7f38d81516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80caab0 0x7f38d81516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 -- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d80cae90 msgr2=0x7f38d80c3910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d80cae90 0x7f38d80c3910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 -- 172.21.15.73:0/827982720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d0009ae0 con 0x7f38d8151bc0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38dc8e0700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f38d8083c30 tx=0x7f38d0009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d000bee0 con 0x7f38d8151bc0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38d0013e60 con 0x7f38d8151bc0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d8098f40 con 0x7f38d8151bc0 2023-11-28T04:24:37.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d001e6c0 con 0x7f38d8151bc0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.877+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d80ac5a0 con 0x7f38d8151bc0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.878+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d80ac890 con 0x7f38d8151bc0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.878+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f38d0003040 con 0x7f38d8151bc0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.879+0000 7f38bffff700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 0x7f38c4061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.879+0000 7f38d7fff700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 0x7f38c4061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.879+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f38d007b420 con 0x7f38d8151bc0 2023-11-28T04:24:37.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.879+0000 7f38d7fff700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 0x7f38c4061640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f38c8006fd0 tx=0x7f38c8006f40).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:37.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:37.883+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f38d004c7a0 con 0x7f38d8151bc0 2023-11-28T04:24:38.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.041+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f38d80ac0a0 con 0x7f38c405f180 2023-11-28T04:24:38.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.051+0000 7f38bffff700 1 -- 172.21.15.73:0/827982720 <== mgr.9345 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362381 (secure 0 0 0) 0x7f38d80ac0a0 con 0x7f38c405f180 2023-11-28T04:24:38.057 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 msgr2=0x7f38c4061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 0x7f38c4061640 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f38c8006fd0 tx=0x7f38c8006f40).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 msgr2=0x7f38d80c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 rx=0x7f38d8083c30 tx=0x7f38d0009840).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 shutdown_connections 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f38c405f180 0x7f38c4061640 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38d8151bc0 0x7f38d80c3e00 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38d80caab0 0x7f38d81516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 --2- 172.21.15.73:0/827982720 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f38d80cae90 0x7f38d80c3910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 >> 172.21.15.73:0/827982720 conn(0x7f38d8094920 msgr2=0x7f38d8065010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 shutdown_connections 2023-11-28T04:24:38.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.055+0000 7f38de343700 1 -- 172.21.15.73:0/827982720 wait complete. 2023-11-28T04:24:38.061 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:24:38.068 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T04:24:36.174618+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453104,"kb_used_data":18992,"kb_used_omap":0,"kb_used_meta":2434048,"kb_avail":752521616,"statfs":{"total":773094113280,"available":770582134784,"internally_reserved":0,"allocated":19447808,"data_stored":9818500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2492465152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-592408,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-207,"num_write_kb":-717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.603857"},"pg_stats":[{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.120034+0000","last_change":"2023-11-28T04:24:35.120034+0000","last_active":"2023-11-28T04:24:35.120034+0000","last_peered":"2023-11-28T04:24:35.120034+0000","last_clean":"2023-11-28T04:24:35.120034+0000","last_became_active":"2023-11-28T04:24:35.119751+0000","last_became_peered":"2023-11-28T04:24:35.119751+0000","last_unstale":"2023-11-28T04:24:35.120034+0000","last_undegraded":"2023-11-28T04:24:35.120034+0000","last_fullsized":"2023-11-28T04:24:35.120034+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3f","version":"196'3","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113644+0000","last_change":"2023-11-28T04:24:34.740428+0000","last_active":"2023-11-28T04:24:35.113644+0000","last_peered":"2023-11-28T04:24:35.113644+0000","last_clean":"2023-11-28T04:24:35.113644+0000","last_became_active":"2023-11-28T04:24:34.740101+0000","last_became_peered":"2023-11-28T04:24:34.740101+0000","last_unstale":"2023-11-28T04:24:35.113644+0000","last_undegraded":"2023-11-28T04:24:35.113644+0000","last_fullsized":"2023-11-28T04:24:35.113644+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110763+0000","last_change":"2023-11-28T04:24:35.110763+0000","last_active":"2023-11-28T04:24:35.110763+0000","last_peered":"2023-11-28T04:24:35.110763+0000","last_clean":"2023-11-28T04:24:35.110763+0000","last_became_active":"2023-11-28T04:24:35.110481+0000","last_became_peered":"2023-11-28T04:24:35.110481+0000","last_unstale":"2023-11-28T04:24:35.110763+0000","last_undegraded":"2023-11-28T04:24:35.110763+0000","last_fullsized":"2023-11-28T04:24:35.110763+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741706+0000","last_change":"2023-11-28T04:24:34.738931+0000","last_active":"2023-11-28T04:24:34.741706+0000","last_peered":"2023-11-28T04:24:34.741706+0000","last_clean":"2023-11-28T04:24:34.741706+0000","last_became_active":"2023-11-28T04:24:34.738635+0000","last_became_peered":"2023-11-28T04:24:34.738635+0000","last_unstale":"2023-11-28T04:24:34.741706+0000","last_undegraded":"2023-11-28T04:24:34.741706+0000","last_fullsized":"2023-11-28T04:24:34.741706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101200+0000","last_change":"2023-11-28T04:24:35.101200+0000","last_active":"2023-11-28T04:24:35.101200+0000","last_peered":"2023-11-28T04:24:35.101200+0000","last_clean":"2023-11-28T04:24:35.101200+0000","last_became_active":"2023-11-28T04:24:35.100809+0000","last_became_peered":"2023-11-28T04:24:35.100809+0000","last_unstale":"2023-11-28T04:24:35.101200+0000","last_undegraded":"2023-11-28T04:24:35.101200+0000","last_fullsized":"2023-11-28T04:24:35.101200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750237+0000","last_change":"2023-11-28T04:24:34.746059+0000","last_active":"2023-11-28T04:24:34.750237+0000","last_peered":"2023-11-28T04:24:34.750237+0000","last_clean":"2023-11-28T04:24:34.750237+0000","last_became_active":"2023-11-28T04:24:34.745417+0000","last_became_peered":"2023-11-28T04:24:34.745417+0000","last_unstale":"2023-11-28T04:24:34.750237+0000","last_undegraded":"2023-11-28T04:24:34.750237+0000","last_fullsized":"2023-11-28T04:24:34.750237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116254+0000","last_change":"2023-11-28T04:24:35.116254+0000","last_active":"2023-11-28T04:24:35.116254+0000","last_peered":"2023-11-28T04:24:35.116254+0000","last_clean":"2023-11-28T04:24:35.116254+0000","last_became_active":"2023-11-28T04:24:35.115839+0000","last_became_peered":"2023-11-28T04:24:35.115839+0000","last_unstale":"2023-11-28T04:24:35.116254+0000","last_undegraded":"2023-11-28T04:24:35.116254+0000","last_fullsized":"2023-11-28T04:24:35.116254+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3c","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112720+0000","last_change":"2023-11-28T04:24:34.742092+0000","last_active":"2023-11-28T04:24:35.112720+0000","last_peered":"2023-11-28T04:24:35.112720+0000","last_clean":"2023-11-28T04:24:35.112720+0000","last_became_active":"2023-11-28T04:24:34.741803+0000","last_became_peered":"2023-11-28T04:24:34.741803+0000","last_unstale":"2023-11-28T04:24:35.112720+0000","last_undegraded":"2023-11-28T04:24:35.112720+0000","last_fullsized":"2023-11-28T04:24:35.112720+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114602+0000","last_change":"2023-11-28T04:24:35.114602+0000","last_active":"2023-11-28T04:24:35.114602+0000","last_peered":"2023-11-28T04:24:35.114602+0000","last_clean":"2023-11-28T04:24:35.114602+0000","last_became_active":"2023-11-28T04:24:35.114226+0000","last_became_peered":"2023-11-28T04:24:35.114226+0000","last_unstale":"2023-11-28T04:24:35.114602+0000","last_undegraded":"2023-11-28T04:24:35.114602+0000","last_fullsized":"2023-11-28T04:24:35.114602+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093477+0000","last_change":"2023-11-28T04:24:34.735064+0000","last_active":"2023-11-28T04:24:35.093477+0000","last_peered":"2023-11-28T04:24:35.093477+0000","last_clean":"2023-11-28T04:24:35.093477+0000","last_became_active":"2023-11-28T04:24:34.734836+0000","last_became_peered":"2023-11-28T04:24:34.734836+0000","last_unstale":"2023-11-28T04:24:35.093477+0000","last_undegraded":"2023-11-28T04:24:35.093477+0000","last_fullsized":"2023-11-28T04:24:35.093477+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104438+0000","last_change":"2023-11-28T04:24:35.104438+0000","last_active":"2023-11-28T04:24:35.104438+0000","last_peered":"2023-11-28T04:24:35.104438+0000","last_clean":"2023-11-28T04:24:35.104438+0000","last_became_active":"2023-11-28T04:24:35.104243+0000","last_became_peered":"2023-11-28T04:24:35.104243+0000","last_unstale":"2023-11-28T04:24:35.104438+0000","last_undegraded":"2023-11-28T04:24:35.104438+0000","last_fullsized":"2023-11-28T04:24:35.104438+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3a","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093920+0000","last_change":"2023-11-28T04:24:34.738025+0000","last_active":"2023-11-28T04:24:35.093920+0000","last_peered":"2023-11-28T04:24:35.093920+0000","last_clean":"2023-11-28T04:24:35.093920+0000","last_became_active":"2023-11-28T04:24:34.737763+0000","last_became_peered":"2023-11-28T04:24:34.737763+0000","last_unstale":"2023-11-28T04:24:35.093920+0000","last_undegraded":"2023-11-28T04:24:35.093920+0000","last_fullsized":"2023-11-28T04:24:35.093920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111410+0000","last_change":"2023-11-28T04:24:35.111410+0000","last_active":"2023-11-28T04:24:35.111410+0000","last_peered":"2023-11-28T04:24:35.111410+0000","last_clean":"2023-11-28T04:24:35.111410+0000","last_became_active":"2023-11-28T04:24:35.111149+0000","last_became_peered":"2023-11-28T04:24:35.111149+0000","last_unstale":"2023-11-28T04:24:35.111410+0000","last_undegraded":"2023-11-28T04:24:35.111410+0000","last_fullsized":"2023-11-28T04:24:35.111410+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.744180+0000","last_change":"2023-11-28T04:24:34.736450+0000","last_active":"2023-11-28T04:24:34.744180+0000","last_peered":"2023-11-28T04:24:34.744180+0000","last_clean":"2023-11-28T04:24:34.744180+0000","last_became_active":"2023-11-28T04:24:34.736210+0000","last_became_peered":"2023-11-28T04:24:34.736210+0000","last_unstale":"2023-11-28T04:24:34.744180+0000","last_undegraded":"2023-11-28T04:24:34.744180+0000","last_fullsized":"2023-11-28T04:24:34.744180+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107926+0000","last_change":"2023-11-28T04:24:35.107926+0000","last_active":"2023-11-28T04:24:35.107926+0000","last_peered":"2023-11-28T04:24:35.107926+0000","last_clean":"2023-11-28T04:24:35.107926+0000","last_became_active":"2023-11-28T04:24:35.107697+0000","last_became_peered":"2023-11-28T04:24:35.107697+0000","last_unstale":"2023-11-28T04:24:35.107926+0000","last_undegraded":"2023-11-28T04:24:35.107926+0000","last_fullsized":"2023-11-28T04:24:35.107926+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741855+0000","last_change":"2023-11-28T04:24:34.740311+0000","last_active":"2023-11-28T04:24:34.741855+0000","last_peered":"2023-11-28T04:24:34.741855+0000","last_clean":"2023-11-28T04:24:34.741855+0000","last_became_active":"2023-11-28T04:24:34.739733+0000","last_became_peered":"2023-11-28T04:24:34.739733+0000","last_unstale":"2023-11-28T04:24:34.741855+0000","last_undegraded":"2023-11-28T04:24:34.741855+0000","last_fullsized":"2023-11-28T04:24:34.741855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111348+0000","last_change":"2023-11-28T04:24:35.111348+0000","last_active":"2023-11-28T04:24:35.111348+0000","last_peered":"2023-11-28T04:24:35.111348+0000","last_clean":"2023-11-28T04:24:35.111348+0000","last_became_active":"2023-11-28T04:24:35.111072+0000","last_became_peered":"2023-11-28T04:24:35.111072+0000","last_unstale":"2023-11-28T04:24:35.111348+0000","last_undegraded":"2023-11-28T04:24:35.111348+0000","last_fullsized":"2023-11-28T04:24:35.111348+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741675+0000","last_change":"2023-11-28T04:24:34.738385+0000","last_active":"2023-11-28T04:24:34.741675+0000","last_peered":"2023-11-28T04:24:34.741675+0000","last_clean":"2023-11-28T04:24:34.741675+0000","last_became_active":"2023-11-28T04:24:34.738044+0000","last_became_peered":"2023-11-28T04:24:34.738044+0000","last_unstale":"2023-11-28T04:24:34.741675+0000","last_undegraded":"2023-11-28T04:24:34.741675+0000","last_fullsized":"2023-11-28T04:24:34.741675+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107728+0000","last_change":"2023-11-28T04:24:35.107728+0000","last_active":"2023-11-28T04:24:35.107728+0000","last_peered":"2023-11-28T04:24:35.107728+0000","last_clean":"2023-11-28T04:24:35.107728+0000","last_became_active":"2023-11-28T04:24:35.107343+0000","last_became_peered":"2023-11-28T04:24:35.107343+0000","last_unstale":"2023-11-28T04:24:35.107728+0000","last_undegraded":"2023-11-28T04:24:35.107728+0000","last_fullsized":"2023-11-28T04:24:35.107728+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747547+0000","last_change":"2023-11-28T04:24:34.745745+0000","last_active":"2023-11-28T04:24:34.747547+0000","last_peered":"2023-11-28T04:24:34.747547+0000","last_clean":"2023-11-28T04:24:34.747547+0000","last_became_active":"2023-11-28T04:24:34.745289+0000","last_became_peered":"2023-11-28T04:24:34.745289+0000","last_unstale":"2023-11-28T04:24:34.747547+0000","last_undegraded":"2023-11-28T04:24:34.747547+0000","last_fullsized":"2023-11-28T04:24:34.747547+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114486+0000","last_change":"2023-11-28T04:24:35.114486+0000","last_active":"2023-11-28T04:24:35.114486+0000","last_peered":"2023-11-28T04:24:35.114486+0000","last_clean":"2023-11-28T04:24:35.114486+0000","last_became_active":"2023-11-28T04:24:35.114148+0000","last_became_peered":"2023-11-28T04:24:35.114148+0000","last_unstale":"2023-11-28T04:24:35.114486+0000","last_undegraded":"2023-11-28T04:24:35.114486+0000","last_fullsized":"2023-11-28T04:24:35.114486+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.740820+0000","last_change":"2023-11-28T04:24:34.734570+0000","last_active":"2023-11-28T04:24:34.740820+0000","last_peered":"2023-11-28T04:24:34.740820+0000","last_clean":"2023-11-28T04:24:34.740820+0000","last_became_active":"2023-11-28T04:24:34.734260+0000","last_became_peered":"2023-11-28T04:24:34.734260+0000","last_unstale":"2023-11-28T04:24:34.740820+0000","last_undegraded":"2023-11-28T04:24:34.740820+0000","last_fullsized":"2023-11-28T04:24:34.740820+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114073+0000","last_change":"2023-11-28T04:24:35.114073+0000","last_active":"2023-11-28T04:24:35.114073+0000","last_peered":"2023-11-28T04:24:35.114073+0000","last_clean":"2023-11-28T04:24:35.114073+0000","last_became_active":"2023-11-28T04:24:35.113782+0000","last_became_peered":"2023-11-28T04:24:35.113782+0000","last_unstale":"2023-11-28T04:24:35.114073+0000","last_undegraded":"2023-11-28T04:24:35.114073+0000","last_fullsized":"2023-11-28T04:24:35.114073+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.34","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111142+0000","last_change":"2023-11-28T04:24:34.732938+0000","last_active":"2023-11-28T04:24:35.111142+0000","last_peered":"2023-11-28T04:24:35.111142+0000","last_clean":"2023-11-28T04:24:35.111142+0000","last_became_active":"2023-11-28T04:24:34.732634+0000","last_became_peered":"2023-11-28T04:24:34.732634+0000","last_unstale":"2023-11-28T04:24:35.111142+0000","last_undegraded":"2023-11-28T04:24:35.111142+0000","last_fullsized":"2023-11-28T04:24:35.111142+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.119088+0000","last_change":"2023-11-28T04:24:35.119088+0000","last_active":"2023-11-28T04:24:35.119088+0000","last_peered":"2023-11-28T04:24:35.119088+0000","last_clean":"2023-11-28T04:24:35.119088+0000","last_became_active":"2023-11-28T04:24:35.118793+0000","last_became_peered":"2023-11-28T04:24:35.118793+0000","last_unstale":"2023-11-28T04:24:35.119088+0000","last_undegraded":"2023-11-28T04:24:35.119088+0000","last_fullsized":"2023-11-28T04:24:35.119088+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.33","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095572+0000","last_change":"2023-11-28T04:24:34.744404+0000","last_active":"2023-11-28T04:24:35.095572+0000","last_peered":"2023-11-28T04:24:35.095572+0000","last_clean":"2023-11-28T04:24:35.095572+0000","last_became_active":"2023-11-28T04:24:34.744092+0000","last_became_peered":"2023-11-28T04:24:34.744092+0000","last_unstale":"2023-11-28T04:24:35.095572+0000","last_undegraded":"2023-11-28T04:24:35.095572+0000","last_fullsized":"2023-11-28T04:24:35.095572+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118241+0000","last_change":"2023-11-28T04:24:35.118241+0000","last_active":"2023-11-28T04:24:35.118241+0000","last_peered":"2023-11-28T04:24:35.118241+0000","last_clean":"2023-11-28T04:24:35.118241+0000","last_became_active":"2023-11-28T04:24:35.117560+0000","last_became_peered":"2023-11-28T04:24:35.117560+0000","last_unstale":"2023-11-28T04:24:35.118241+0000","last_undegraded":"2023-11-28T04:24:35.118241+0000","last_fullsized":"2023-11-28T04:24:35.118241+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742088+0000","last_change":"2023-11-28T04:24:34.738138+0000","last_active":"2023-11-28T04:24:34.742088+0000","last_peered":"2023-11-28T04:24:34.742088+0000","last_clean":"2023-11-28T04:24:34.742088+0000","last_became_active":"2023-11-28T04:24:34.737851+0000","last_became_peered":"2023-11-28T04:24:34.737851+0000","last_unstale":"2023-11-28T04:24:34.742088+0000","last_undegraded":"2023-11-28T04:24:34.742088+0000","last_fullsized":"2023-11-28T04:24:34.742088+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112108+0000","last_change":"2023-11-28T04:24:35.112108+0000","last_active":"2023-11-28T04:24:35.112108+0000","last_peered":"2023-11-28T04:24:35.112108+0000","last_clean":"2023-11-28T04:24:35.112108+0000","last_became_active":"2023-11-28T04:24:35.111823+0000","last_became_peered":"2023-11-28T04:24:35.111823+0000","last_unstale":"2023-11-28T04:24:35.112108+0000","last_undegraded":"2023-11-28T04:24:35.112108+0000","last_fullsized":"2023-11-28T04:24:35.112108+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750016+0000","last_change":"2023-11-28T04:24:34.743775+0000","last_active":"2023-11-28T04:24:34.750016+0000","last_peered":"2023-11-28T04:24:34.750016+0000","last_clean":"2023-11-28T04:24:34.750016+0000","last_became_active":"2023-11-28T04:24:34.743498+0000","last_became_peered":"2023-11-28T04:24:34.743498+0000","last_unstale":"2023-11-28T04:24:34.750016+0000","last_undegraded":"2023-11-28T04:24:34.750016+0000","last_fullsized":"2023-11-28T04:24:34.750016+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113413+0000","last_change":"2023-11-28T04:24:35.113413+0000","last_active":"2023-11-28T04:24:35.113413+0000","last_peered":"2023-11-28T04:24:35.113413+0000","last_clean":"2023-11-28T04:24:35.113413+0000","last_became_active":"2023-11-28T04:24:35.113070+0000","last_became_peered":"2023-11-28T04:24:35.113070+0000","last_unstale":"2023-11-28T04:24:35.113413+0000","last_undegraded":"2023-11-28T04:24:35.113413+0000","last_fullsized":"2023-11-28T04:24:35.113413+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.30","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109889+0000","last_change":"2023-11-28T04:24:34.736610+0000","last_active":"2023-11-28T04:24:35.109889+0000","last_peered":"2023-11-28T04:24:35.109889+0000","last_clean":"2023-11-28T04:24:35.109889+0000","last_became_active":"2023-11-28T04:24:34.736347+0000","last_became_peered":"2023-11-28T04:24:34.736347+0000","last_unstale":"2023-11-28T04:24:35.109889+0000","last_undegraded":"2023-11-28T04:24:35.109889+0000","last_fullsized":"2023-11-28T04:24:35.109889+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107936+0000","last_change":"2023-11-28T04:24:35.107936+0000","last_active":"2023-11-28T04:24:35.107936+0000","last_peered":"2023-11-28T04:24:35.107936+0000","last_clean":"2023-11-28T04:24:35.107936+0000","last_became_active":"2023-11-28T04:24:35.107676+0000","last_became_peered":"2023-11-28T04:24:35.107676+0000","last_unstale":"2023-11-28T04:24:35.107936+0000","last_undegraded":"2023-11-28T04:24:35.107936+0000","last_fullsized":"2023-11-28T04:24:35.107936+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742243+0000","last_change":"2023-11-28T04:24:34.740319+0000","last_active":"2023-11-28T04:24:34.742243+0000","last_peered":"2023-11-28T04:24:34.742243+0000","last_clean":"2023-11-28T04:24:34.742243+0000","last_became_active":"2023-11-28T04:24:34.740049+0000","last_became_peered":"2023-11-28T04:24:34.740049+0000","last_unstale":"2023-11-28T04:24:34.742243+0000","last_undegraded":"2023-11-28T04:24:34.742243+0000","last_fullsized":"2023-11-28T04:24:34.742243+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.108591+0000","last_change":"2023-11-28T04:24:35.108591+0000","last_active":"2023-11-28T04:24:35.108591+0000","last_peered":"2023-11-28T04:24:35.108591+0000","last_clean":"2023-11-28T04:24:35.108591+0000","last_became_active":"2023-11-28T04:24:35.108300+0000","last_became_peered":"2023-11-28T04:24:35.108300+0000","last_unstale":"2023-11-28T04:24:35.108591+0000","last_undegraded":"2023-11-28T04:24:35.108591+0000","last_fullsized":"2023-11-28T04:24:35.108591+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748235+0000","last_change":"2023-11-28T04:24:34.743482+0000","last_active":"2023-11-28T04:24:34.748235+0000","last_peered":"2023-11-28T04:24:34.748235+0000","last_clean":"2023-11-28T04:24:34.748235+0000","last_became_active":"2023-11-28T04:24:34.743228+0000","last_became_peered":"2023-11-28T04:24:34.743228+0000","last_unstale":"2023-11-28T04:24:34.748235+0000","last_undegraded":"2023-11-28T04:24:34.748235+0000","last_fullsized":"2023-11-28T04:24:34.748235+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101830+0000","last_change":"2023-11-28T04:24:35.101830+0000","last_active":"2023-11-28T04:24:35.101830+0000","last_peered":"2023-11-28T04:24:35.101830+0000","last_clean":"2023-11-28T04:24:35.101830+0000","last_became_active":"2023-11-28T04:24:35.101565+0000","last_became_peered":"2023-11-28T04:24:35.101565+0000","last_unstale":"2023-11-28T04:24:35.101830+0000","last_undegraded":"2023-11-28T04:24:35.101830+0000","last_fullsized":"2023-11-28T04:24:35.101830+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2d","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095148+0000","last_change":"2023-11-28T04:24:34.735844+0000","last_active":"2023-11-28T04:24:35.095148+0000","last_peered":"2023-11-28T04:24:35.095148+0000","last_clean":"2023-11-28T04:24:35.095148+0000","last_became_active":"2023-11-28T04:24:34.735554+0000","last_became_peered":"2023-11-28T04:24:34.735554+0000","last_unstale":"2023-11-28T04:24:35.095148+0000","last_undegraded":"2023-11-28T04:24:35.095148+0000","last_fullsized":"2023-11-28T04:24:35.095148+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110387+0000","last_change":"2023-11-28T04:24:35.110387+0000","last_active":"2023-11-28T04:24:35.110387+0000","last_peered":"2023-11-28T04:24:35.110387+0000","last_clean":"2023-11-28T04:24:35.110387+0000","last_became_active":"2023-11-28T04:24:35.110070+0000","last_became_peered":"2023-11-28T04:24:35.110070+0000","last_unstale":"2023-11-28T04:24:35.110387+0000","last_undegraded":"2023-11-28T04:24:35.110387+0000","last_fullsized":"2023-11-28T04:24:35.110387+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742069+0000","last_change":"2023-11-28T04:24:34.737473+0000","last_active":"2023-11-28T04:24:34.742069+0000","last_peered":"2023-11-28T04:24:34.742069+0000","last_clean":"2023-11-28T04:24:34.742069+0000","last_became_active":"2023-11-28T04:24:34.737237+0000","last_became_peered":"2023-11-28T04:24:34.737237+0000","last_unstale":"2023-11-28T04:24:34.742069+0000","last_undegraded":"2023-11-28T04:24:34.742069+0000","last_fullsized":"2023-11-28T04:24:34.742069+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116869+0000","last_change":"2023-11-28T04:24:35.116869+0000","last_active":"2023-11-28T04:24:35.116869+0000","last_peered":"2023-11-28T04:24:35.116869+0000","last_clean":"2023-11-28T04:24:35.116869+0000","last_became_active":"2023-11-28T04:24:35.116288+0000","last_became_peered":"2023-11-28T04:24:35.116288+0000","last_unstale":"2023-11-28T04:24:35.116869+0000","last_undegraded":"2023-11-28T04:24:35.116869+0000","last_fullsized":"2023-11-28T04:24:35.116869+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093920+0000","last_change":"2023-11-28T04:24:34.740392+0000","last_active":"2023-11-28T04:24:35.093920+0000","last_peered":"2023-11-28T04:24:35.093920+0000","last_clean":"2023-11-28T04:24:35.093920+0000","last_became_active":"2023-11-28T04:24:34.740126+0000","last_became_peered":"2023-11-28T04:24:34.740126+0000","last_unstale":"2023-11-28T04:24:35.093920+0000","last_undegraded":"2023-11-28T04:24:35.093920+0000","last_fullsized":"2023-11-28T04:24:35.093920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109567+0000","last_change":"2023-11-28T04:24:35.109567+0000","last_active":"2023-11-28T04:24:35.109567+0000","last_peered":"2023-11-28T04:24:35.109567+0000","last_clean":"2023-11-28T04:24:35.109567+0000","last_became_active":"2023-11-28T04:24:35.109194+0000","last_became_peered":"2023-11-28T04:24:35.109194+0000","last_unstale":"2023-11-28T04:24:35.109567+0000","last_undegraded":"2023-11-28T04:24:35.109567+0000","last_fullsized":"2023-11-28T04:24:35.109567+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741732+0000","last_change":"2023-11-28T04:24:34.738279+0000","last_active":"2023-11-28T04:24:34.741732+0000","last_peered":"2023-11-28T04:24:34.741732+0000","last_clean":"2023-11-28T04:24:34.741732+0000","last_became_active":"2023-11-28T04:24:34.737992+0000","last_became_peered":"2023-11-28T04:24:34.737992+0000","last_unstale":"2023-11-28T04:24:34.741732+0000","last_undegraded":"2023-11-28T04:24:34.741732+0000","last_fullsized":"2023-11-28T04:24:34.741732+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111319+0000","last_change":"2023-11-28T04:24:35.111319+0000","last_active":"2023-11-28T04:24:35.111319+0000","last_peered":"2023-11-28T04:24:35.111319+0000","last_clean":"2023-11-28T04:24:35.111319+0000","last_became_active":"2023-11-28T04:24:35.111064+0000","last_became_peered":"2023-11-28T04:24:35.111064+0000","last_unstale":"2023-11-28T04:24:35.111319+0000","last_undegraded":"2023-11-28T04:24:35.111319+0000","last_fullsized":"2023-11-28T04:24:35.111319+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741586+0000","last_change":"2023-11-28T04:24:34.732186+0000","last_active":"2023-11-28T04:24:34.741586+0000","last_peered":"2023-11-28T04:24:34.741586+0000","last_clean":"2023-11-28T04:24:34.741586+0000","last_became_active":"2023-11-28T04:24:34.731975+0000","last_became_peered":"2023-11-28T04:24:34.731975+0000","last_unstale":"2023-11-28T04:24:34.741586+0000","last_undegraded":"2023-11-28T04:24:34.741586+0000","last_fullsized":"2023-11-28T04:24:34.741586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106021+0000","last_change":"2023-11-28T04:24:35.106021+0000","last_active":"2023-11-28T04:24:35.106021+0000","last_peered":"2023-11-28T04:24:35.106021+0000","last_clean":"2023-11-28T04:24:35.106021+0000","last_became_active":"2023-11-28T04:24:35.105753+0000","last_became_peered":"2023-11-28T04:24:35.105753+0000","last_unstale":"2023-11-28T04:24:35.106021+0000","last_undegraded":"2023-11-28T04:24:35.106021+0000","last_fullsized":"2023-11-28T04:24:35.106021+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741874+0000","last_change":"2023-11-28T04:24:34.740479+0000","last_active":"2023-11-28T04:24:34.741874+0000","last_peered":"2023-11-28T04:24:34.741874+0000","last_clean":"2023-11-28T04:24:34.741874+0000","last_became_active":"2023-11-28T04:24:34.740053+0000","last_became_peered":"2023-11-28T04:24:34.740053+0000","last_unstale":"2023-11-28T04:24:34.741874+0000","last_undegraded":"2023-11-28T04:24:34.741874+0000","last_fullsized":"2023-11-28T04:24:34.741874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118118+0000","last_change":"2023-11-28T04:24:35.118118+0000","last_active":"2023-11-28T04:24:35.118118+0000","last_peered":"2023-11-28T04:24:35.118118+0000","last_clean":"2023-11-28T04:24:35.118118+0000","last_became_active":"2023-11-28T04:24:35.117779+0000","last_became_peered":"2023-11-28T04:24:35.117779+0000","last_unstale":"2023-11-28T04:24:35.118118+0000","last_undegraded":"2023-11-28T04:24:35.118118+0000","last_fullsized":"2023-11-28T04:24:35.118118+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750228+0000","last_change":"2023-11-28T04:24:34.746015+0000","last_active":"2023-11-28T04:24:34.750228+0000","last_peered":"2023-11-28T04:24:34.750228+0000","last_clean":"2023-11-28T04:24:34.750228+0000","last_became_active":"2023-11-28T04:24:34.745687+0000","last_became_peered":"2023-11-28T04:24:34.745687+0000","last_unstale":"2023-11-28T04:24:34.750228+0000","last_undegraded":"2023-11-28T04:24:34.750228+0000","last_fullsized":"2023-11-28T04:24:34.750228+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105824+0000","last_change":"2023-11-28T04:24:35.105824+0000","last_active":"2023-11-28T04:24:35.105824+0000","last_peered":"2023-11-28T04:24:35.105824+0000","last_clean":"2023-11-28T04:24:35.105824+0000","last_became_active":"2023-11-28T04:24:35.105582+0000","last_became_peered":"2023-11-28T04:24:35.105582+0000","last_unstale":"2023-11-28T04:24:35.105824+0000","last_undegraded":"2023-11-28T04:24:35.105824+0000","last_fullsized":"2023-11-28T04:24:35.105824+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750120+0000","last_change":"2023-11-28T04:24:34.745797+0000","last_active":"2023-11-28T04:24:34.750120+0000","last_peered":"2023-11-28T04:24:34.750120+0000","last_clean":"2023-11-28T04:24:34.750120+0000","last_became_active":"2023-11-28T04:24:34.745498+0000","last_became_peered":"2023-11-28T04:24:34.745498+0000","last_unstale":"2023-11-28T04:24:34.750120+0000","last_undegraded":"2023-11-28T04:24:34.750120+0000","last_fullsized":"2023-11-28T04:24:34.750120+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110230+0000","last_change":"2023-11-28T04:24:35.110230+0000","last_active":"2023-11-28T04:24:35.110230+0000","last_peered":"2023-11-28T04:24:35.110230+0000","last_clean":"2023-11-28T04:24:35.110230+0000","last_became_active":"2023-11-28T04:24:35.109391+0000","last_became_peered":"2023-11-28T04:24:35.109391+0000","last_unstale":"2023-11-28T04:24:35.110230+0000","last_undegraded":"2023-11-28T04:24:35.110230+0000","last_fullsized":"2023-11-28T04:24:35.110230+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750252+0000","last_change":"2023-11-28T04:24:34.746207+0000","last_active":"2023-11-28T04:24:34.750252+0000","last_peered":"2023-11-28T04:24:34.750252+0000","last_clean":"2023-11-28T04:24:34.750252+0000","last_became_active":"2023-11-28T04:24:34.745767+0000","last_became_peered":"2023-11-28T04:24:34.745767+0000","last_unstale":"2023-11-28T04:24:34.750252+0000","last_undegraded":"2023-11-28T04:24:34.750252+0000","last_fullsized":"2023-11-28T04:24:34.750252+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116341+0000","last_change":"2023-11-28T04:24:35.116341+0000","last_active":"2023-11-28T04:24:35.116341+0000","last_peered":"2023-11-28T04:24:35.116341+0000","last_clean":"2023-11-28T04:24:35.116341+0000","last_became_active":"2023-11-28T04:24:35.116103+0000","last_became_peered":"2023-11-28T04:24:35.116103+0000","last_unstale":"2023-11-28T04:24:35.116341+0000","last_undegraded":"2023-11-28T04:24:35.116341+0000","last_fullsized":"2023-11-28T04:24:35.116341+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749339+0000","last_change":"2023-11-28T04:24:34.741276+0000","last_active":"2023-11-28T04:24:34.749339+0000","last_peered":"2023-11-28T04:24:34.749339+0000","last_clean":"2023-11-28T04:24:34.749339+0000","last_became_active":"2023-11-28T04:24:34.740833+0000","last_became_peered":"2023-11-28T04:24:34.740833+0000","last_unstale":"2023-11-28T04:24:34.749339+0000","last_undegraded":"2023-11-28T04:24:34.749339+0000","last_fullsized":"2023-11-28T04:24:34.749339+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104996+0000","last_change":"2023-11-28T04:24:35.104996+0000","last_active":"2023-11-28T04:24:35.104996+0000","last_peered":"2023-11-28T04:24:35.104996+0000","last_clean":"2023-11-28T04:24:35.104996+0000","last_became_active":"2023-11-28T04:24:35.104732+0000","last_became_peered":"2023-11-28T04:24:35.104732+0000","last_unstale":"2023-11-28T04:24:35.104996+0000","last_undegraded":"2023-11-28T04:24:35.104996+0000","last_fullsized":"2023-11-28T04:24:35.104996+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750214+0000","last_change":"2023-11-28T04:24:34.746012+0000","last_active":"2023-11-28T04:24:34.750214+0000","last_peered":"2023-11-28T04:24:34.750214+0000","last_clean":"2023-11-28T04:24:34.750214+0000","last_became_active":"2023-11-28T04:24:34.745502+0000","last_became_peered":"2023-11-28T04:24:34.745502+0000","last_unstale":"2023-11-28T04:24:34.750214+0000","last_undegraded":"2023-11-28T04:24:34.750214+0000","last_fullsized":"2023-11-28T04:24:34.750214+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110468+0000","last_change":"2023-11-28T04:24:35.110468+0000","last_active":"2023-11-28T04:24:35.110468+0000","last_peered":"2023-11-28T04:24:35.110468+0000","last_clean":"2023-11-28T04:24:35.110468+0000","last_became_active":"2023-11-28T04:24:35.110189+0000","last_became_peered":"2023-11-28T04:24:35.110189+0000","last_unstale":"2023-11-28T04:24:35.110468+0000","last_undegraded":"2023-11-28T04:24:35.110468+0000","last_fullsized":"2023-11-28T04:24:35.110468+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747569+0000","last_change":"2023-11-28T04:24:34.743048+0000","last_active":"2023-11-28T04:24:34.747569+0000","last_peered":"2023-11-28T04:24:34.747569+0000","last_clean":"2023-11-28T04:24:34.747569+0000","last_became_active":"2023-11-28T04:24:34.742757+0000","last_became_peered":"2023-11-28T04:24:34.742757+0000","last_unstale":"2023-11-28T04:24:34.747569+0000","last_undegraded":"2023-11-28T04:24:34.747569+0000","last_fullsized":"2023-11-28T04:24:34.747569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106921+0000","last_change":"2023-11-28T04:24:35.106921+0000","last_active":"2023-11-28T04:24:35.106921+0000","last_peered":"2023-11-28T04:24:35.106921+0000","last_clean":"2023-11-28T04:24:35.106921+0000","last_became_active":"2023-11-28T04:24:35.106659+0000","last_became_peered":"2023-11-28T04:24:35.106659+0000","last_unstale":"2023-11-28T04:24:35.106921+0000","last_undegraded":"2023-11-28T04:24:35.106921+0000","last_fullsized":"2023-11-28T04:24:35.106921+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750129+0000","last_change":"2023-11-28T04:24:34.746184+0000","last_active":"2023-11-28T04:24:34.750129+0000","last_peered":"2023-11-28T04:24:34.750129+0000","last_clean":"2023-11-28T04:24:34.750129+0000","last_became_active":"2023-11-28T04:24:34.745948+0000","last_became_peered":"2023-11-28T04:24:34.745948+0000","last_unstale":"2023-11-28T04:24:34.750129+0000","last_undegraded":"2023-11-28T04:24:34.750129+0000","last_fullsized":"2023-11-28T04:24:34.750129+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107383+0000","last_change":"2023-11-28T04:24:35.107383+0000","last_active":"2023-11-28T04:24:35.107383+0000","last_peered":"2023-11-28T04:24:35.107383+0000","last_clean":"2023-11-28T04:24:35.107383+0000","last_became_active":"2023-11-28T04:24:35.107114+0000","last_became_peered":"2023-11-28T04:24:35.107114+0000","last_unstale":"2023-11-28T04:24:35.107383+0000","last_undegraded":"2023-11-28T04:24:35.107383+0000","last_fullsized":"2023-11-28T04:24:35.107383+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748141+0000","last_change":"2023-11-28T04:24:34.743069+0000","last_active":"2023-11-28T04:24:34.748141+0000","last_peered":"2023-11-28T04:24:34.748141+0000","last_clean":"2023-11-28T04:24:34.748141+0000","last_became_active":"2023-11-28T04:24:34.742766+0000","last_became_peered":"2023-11-28T04:24:34.742766+0000","last_unstale":"2023-11-28T04:24:34.748141+0000","last_undegraded":"2023-11-28T04:24:34.748141+0000","last_fullsized":"2023-11-28T04:24:34.748141+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118811+0000","last_change":"2023-11-28T04:24:35.118811+0000","last_active":"2023-11-28T04:24:35.118811+0000","last_peered":"2023-11-28T04:24:35.118811+0000","last_clean":"2023-11-28T04:24:35.118811+0000","last_became_active":"2023-11-28T04:24:35.118070+0000","last_became_peered":"2023-11-28T04:24:35.118070+0000","last_unstale":"2023-11-28T04:24:35.118811+0000","last_undegraded":"2023-11-28T04:24:35.118811+0000","last_fullsized":"2023-11-28T04:24:35.118811+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106962+0000","last_change":"2023-11-28T04:24:34.743724+0000","last_active":"2023-11-28T04:24:35.106962+0000","last_peered":"2023-11-28T04:24:35.106962+0000","last_clean":"2023-11-28T04:24:35.106962+0000","last_became_active":"2023-11-28T04:24:34.743430+0000","last_became_peered":"2023-11-28T04:24:34.743430+0000","last_unstale":"2023-11-28T04:24:35.106962+0000","last_undegraded":"2023-11-28T04:24:35.106962+0000","last_fullsized":"2023-11-28T04:24:35.106962+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101773+0000","last_change":"2023-11-28T04:24:35.101773+0000","last_active":"2023-11-28T04:24:35.101773+0000","last_peered":"2023-11-28T04:24:35.101773+0000","last_clean":"2023-11-28T04:24:35.101773+0000","last_became_active":"2023-11-28T04:24:35.101503+0000","last_became_peered":"2023-11-28T04:24:35.101503+0000","last_unstale":"2023-11-28T04:24:35.101773+0000","last_undegraded":"2023-11-28T04:24:35.101773+0000","last_fullsized":"2023-11-28T04:24:35.101773+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095928+0000","last_change":"2023-11-28T04:24:34.746192+0000","last_active":"2023-11-28T04:24:35.095928+0000","last_peered":"2023-11-28T04:24:35.095928+0000","last_clean":"2023-11-28T04:24:35.095928+0000","last_became_active":"2023-11-28T04:24:34.745885+0000","last_became_peered":"2023-11-28T04:24:34.745885+0000","last_unstale":"2023-11-28T04:24:35.095928+0000","last_undegraded":"2023-11-28T04:24:35.095928+0000","last_fullsized":"2023-11-28T04:24:35.095928+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101062+0000","last_change":"2023-11-28T04:24:35.101062+0000","last_active":"2023-11-28T04:24:35.101062+0000","last_peered":"2023-11-28T04:24:35.101062+0000","last_clean":"2023-11-28T04:24:35.101062+0000","last_became_active":"2023-11-28T04:24:35.100534+0000","last_became_peered":"2023-11-28T04:24:35.100534+0000","last_unstale":"2023-11-28T04:24:35.101062+0000","last_undegraded":"2023-11-28T04:24:35.101062+0000","last_fullsized":"2023-11-28T04:24:35.101062+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.108820+0000","last_change":"2023-11-28T04:24:34.743046+0000","last_active":"2023-11-28T04:24:35.108820+0000","last_peered":"2023-11-28T04:24:35.108820+0000","last_clean":"2023-11-28T04:24:35.108820+0000","last_became_active":"2023-11-28T04:24:34.742717+0000","last_became_peered":"2023-11-28T04:24:34.742717+0000","last_unstale":"2023-11-28T04:24:35.108820+0000","last_undegraded":"2023-11-28T04:24:35.108820+0000","last_fullsized":"2023-11-28T04:24:35.108820+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116507+0000","last_change":"2023-11-28T04:24:35.116507+0000","last_active":"2023-11-28T04:24:35.116507+0000","last_peered":"2023-11-28T04:24:35.116507+0000","last_clean":"2023-11-28T04:24:35.116507+0000","last_became_active":"2023-11-28T04:24:35.116187+0000","last_became_peered":"2023-11-28T04:24:35.116187+0000","last_unstale":"2023-11-28T04:24:35.116507+0000","last_undegraded":"2023-11-28T04:24:35.116507+0000","last_fullsized":"2023-11-28T04:24:35.116507+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742189+0000","last_change":"2023-11-28T04:24:34.739740+0000","last_active":"2023-11-28T04:24:34.742189+0000","last_peered":"2023-11-28T04:24:34.742189+0000","last_clean":"2023-11-28T04:24:34.742189+0000","last_became_active":"2023-11-28T04:24:34.739431+0000","last_became_peered":"2023-11-28T04:24:34.739431+0000","last_unstale":"2023-11-28T04:24:34.742189+0000","last_undegraded":"2023-11-28T04:24:34.742189+0000","last_fullsized":"2023-11-28T04:24:34.742189+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113793+0000","last_change":"2023-11-28T04:24:35.113793+0000","last_active":"2023-11-28T04:24:35.113793+0000","last_peered":"2023-11-28T04:24:35.113793+0000","last_clean":"2023-11-28T04:24:35.113793+0000","last_became_active":"2023-11-28T04:24:35.113501+0000","last_became_peered":"2023-11-28T04:24:35.113501+0000","last_unstale":"2023-11-28T04:24:35.113793+0000","last_undegraded":"2023-11-28T04:24:35.113793+0000","last_fullsized":"2023-11-28T04:24:35.113793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749370+0000","last_change":"2023-11-28T04:24:34.743481+0000","last_active":"2023-11-28T04:24:34.749370+0000","last_peered":"2023-11-28T04:24:34.749370+0000","last_clean":"2023-11-28T04:24:34.749370+0000","last_became_active":"2023-11-28T04:24:34.743247+0000","last_became_peered":"2023-11-28T04:24:34.743247+0000","last_unstale":"2023-11-28T04:24:34.749370+0000","last_undegraded":"2023-11-28T04:24:34.749370+0000","last_fullsized":"2023-11-28T04:24:34.749370+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.102799+0000","last_change":"2023-11-28T04:24:35.102799+0000","last_active":"2023-11-28T04:24:35.102799+0000","last_peered":"2023-11-28T04:24:35.102799+0000","last_clean":"2023-11-28T04:24:35.102799+0000","last_became_active":"2023-11-28T04:24:35.102354+0000","last_became_peered":"2023-11-28T04:24:35.102354+0000","last_unstale":"2023-11-28T04:24:35.102799+0000","last_undegraded":"2023-11-28T04:24:35.102799+0000","last_fullsized":"2023-11-28T04:24:35.102799+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742530+0000","last_change":"2023-11-28T04:24:34.741653+0000","last_active":"2023-11-28T04:24:34.742530+0000","last_peered":"2023-11-28T04:24:34.742530+0000","last_clean":"2023-11-28T04:24:34.742530+0000","last_became_active":"2023-11-28T04:24:34.741389+0000","last_became_peered":"2023-11-28T04:24:34.741389+0000","last_unstale":"2023-11-28T04:24:34.742530+0000","last_undegraded":"2023-11-28T04:24:34.742530+0000","last_fullsized":"2023-11-28T04:24:34.742530+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103440+0000","last_change":"2023-11-28T04:24:35.103440+0000","last_active":"2023-11-28T04:24:35.103440+0000","last_peered":"2023-11-28T04:24:35.103440+0000","last_clean":"2023-11-28T04:24:35.103440+0000","last_became_active":"2023-11-28T04:24:35.103198+0000","last_became_peered":"2023-11-28T04:24:35.103198+0000","last_unstale":"2023-11-28T04:24:35.103440+0000","last_undegraded":"2023-11-28T04:24:35.103440+0000","last_fullsized":"2023-11-28T04:24:35.103440+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742097+0000","last_change":"2023-11-28T04:24:34.738490+0000","last_active":"2023-11-28T04:24:34.742097+0000","last_peered":"2023-11-28T04:24:34.742097+0000","last_clean":"2023-11-28T04:24:34.742097+0000","last_became_active":"2023-11-28T04:24:34.738225+0000","last_became_peered":"2023-11-28T04:24:34.738225+0000","last_unstale":"2023-11-28T04:24:34.742097+0000","last_undegraded":"2023-11-28T04:24:34.742097+0000","last_fullsized":"2023-11-28T04:24:34.742097+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115885+0000","last_change":"2023-11-28T04:24:35.115885+0000","last_active":"2023-11-28T04:24:35.115885+0000","last_peered":"2023-11-28T04:24:35.115885+0000","last_clean":"2023-11-28T04:24:35.115885+0000","last_became_active":"2023-11-28T04:24:35.115572+0000","last_became_peered":"2023-11-28T04:24:35.115572+0000","last_unstale":"2023-11-28T04:24:35.115885+0000","last_undegraded":"2023-11-28T04:24:35.115885+0000","last_fullsized":"2023-11-28T04:24:35.115885+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.745931+0000","last_change":"2023-11-28T04:24:34.738309+0000","last_active":"2023-11-28T04:24:34.745931+0000","last_peered":"2023-11-28T04:24:34.745931+0000","last_clean":"2023-11-28T04:24:34.745931+0000","last_became_active":"2023-11-28T04:24:34.737352+0000","last_became_peered":"2023-11-28T04:24:34.737352+0000","last_unstale":"2023-11-28T04:24:34.745931+0000","last_undegraded":"2023-11-28T04:24:34.745931+0000","last_fullsized":"2023-11-28T04:24:34.745931+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107176+0000","last_change":"2023-11-28T04:24:35.107176+0000","last_active":"2023-11-28T04:24:35.107176+0000","last_peered":"2023-11-28T04:24:35.107176+0000","last_clean":"2023-11-28T04:24:35.107176+0000","last_became_active":"2023-11-28T04:24:35.106878+0000","last_became_peered":"2023-11-28T04:24:35.106878+0000","last_unstale":"2023-11-28T04:24:35.107176+0000","last_undegraded":"2023-11-28T04:24:35.107176+0000","last_fullsized":"2023-11-28T04:24:35.107176+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.091495+0000","last_change":"2023-11-28T04:24:34.745386+0000","last_active":"2023-11-28T04:24:35.091495+0000","last_peered":"2023-11-28T04:24:35.091495+0000","last_clean":"2023-11-28T04:24:35.091495+0000","last_became_active":"2023-11-28T04:24:34.745123+0000","last_became_peered":"2023-11-28T04:24:34.745123+0000","last_unstale":"2023-11-28T04:24:35.091495+0000","last_undegraded":"2023-11-28T04:24:35.091495+0000","last_fullsized":"2023-11-28T04:24:35.091495+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.100696+0000","last_change":"2023-11-28T04:24:35.100696+0000","last_active":"2023-11-28T04:24:35.100696+0000","last_peered":"2023-11-28T04:24:35.100696+0000","last_clean":"2023-11-28T04:24:35.100696+0000","last_became_active":"2023-11-28T04:24:35.100370+0000","last_became_peered":"2023-11-28T04:24:35.100370+0000","last_unstale":"2023-11-28T04:24:35.100696+0000","last_undegraded":"2023-11-28T04:24:35.100696+0000","last_fullsized":"2023-11-28T04:24:35.100696+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.117427+0000","last_change":"2023-11-28T04:24:34.741248+0000","last_active":"2023-11-28T04:24:35.117427+0000","last_peered":"2023-11-28T04:24:35.117427+0000","last_clean":"2023-11-28T04:24:35.117427+0000","last_became_active":"2023-11-28T04:24:34.740942+0000","last_became_peered":"2023-11-28T04:24:34.740942+0000","last_unstale":"2023-11-28T04:24:35.117427+0000","last_undegraded":"2023-11-28T04:24:35.117427+0000","last_fullsized":"2023-11-28T04:24:35.117427+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112261+0000","last_change":"2023-11-28T04:24:35.112261+0000","last_active":"2023-11-28T04:24:35.112261+0000","last_peered":"2023-11-28T04:24:35.112261+0000","last_clean":"2023-11-28T04:24:35.112261+0000","last_became_active":"2023-11-28T04:24:35.111924+0000","last_became_peered":"2023-11-28T04:24:35.111924+0000","last_unstale":"2023-11-28T04:24:35.112261+0000","last_undegraded":"2023-11-28T04:24:35.112261+0000","last_fullsized":"2023-11-28T04:24:35.112261+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742315+0000","last_change":"2023-11-28T04:24:34.735718+0000","last_active":"2023-11-28T04:24:34.742315+0000","last_peered":"2023-11-28T04:24:34.742315+0000","last_clean":"2023-11-28T04:24:34.742315+0000","last_became_active":"2023-11-28T04:24:34.735416+0000","last_became_peered":"2023-11-28T04:24:34.735416+0000","last_unstale":"2023-11-28T04:24:34.742315+0000","last_undegraded":"2023-11-28T04:24:34.742315+0000","last_fullsized":"2023-11-28T04:24:34.742315+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105168+0000","last_change":"2023-11-28T04:24:35.105168+0000","last_active":"2023-11-28T04:24:35.105168+0000","last_peered":"2023-11-28T04:24:35.105168+0000","last_clean":"2023-11-28T04:24:35.105168+0000","last_became_active":"2023-11-28T04:24:35.104876+0000","last_became_peered":"2023-11-28T04:24:35.104876+0000","last_unstale":"2023-11-28T04:24:35.105168+0000","last_undegraded":"2023-11-28T04:24:35.105168+0000","last_fullsized":"2023-11-28T04:24:35.105168+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749400+0000","last_change":"2023-11-28T04:24:34.744046+0000","last_active":"2023-11-28T04:24:34.749400+0000","last_peered":"2023-11-28T04:24:34.749400+0000","last_clean":"2023-11-28T04:24:34.749400+0000","last_became_active":"2023-11-28T04:24:34.743758+0000","last_became_peered":"2023-11-28T04:24:34.743758+0000","last_unstale":"2023-11-28T04:24:34.749400+0000","last_undegraded":"2023-11-28T04:24:34.749400+0000","last_fullsized":"2023-11-28T04:24:34.749400+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116918+0000","last_change":"2023-11-28T04:24:35.116918+0000","last_active":"2023-11-28T04:24:35.116918+0000","last_peered":"2023-11-28T04:24:35.116918+0000","last_clean":"2023-11-28T04:24:35.116918+0000","last_became_active":"2023-11-28T04:24:35.116627+0000","last_became_peered":"2023-11-28T04:24:35.116627+0000","last_unstale":"2023-11-28T04:24:35.116918+0000","last_undegraded":"2023-11-28T04:24:35.116918+0000","last_fullsized":"2023-11-28T04:24:35.116918+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.092975+0000","last_change":"2023-11-28T04:24:34.738909+0000","last_active":"2023-11-28T04:24:35.092975+0000","last_peered":"2023-11-28T04:24:35.092975+0000","last_clean":"2023-11-28T04:24:35.092975+0000","last_became_active":"2023-11-28T04:24:34.738673+0000","last_became_peered":"2023-11-28T04:24:34.738673+0000","last_unstale":"2023-11-28T04:24:35.092975+0000","last_undegraded":"2023-11-28T04:24:35.092975+0000","last_fullsized":"2023-11-28T04:24:35.092975+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104195+0000","last_change":"2023-11-28T04:24:35.104195+0000","last_active":"2023-11-28T04:24:35.104195+0000","last_peered":"2023-11-28T04:24:35.104195+0000","last_clean":"2023-11-28T04:24:35.104195+0000","last_became_active":"2023-11-28T04:24:35.103883+0000","last_became_peered":"2023-11-28T04:24:35.103883+0000","last_unstale":"2023-11-28T04:24:35.104195+0000","last_undegraded":"2023-11-28T04:24:35.104195+0000","last_fullsized":"2023-11-28T04:24:35.104195+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747501+0000","last_change":"2023-11-28T04:24:34.744419+0000","last_active":"2023-11-28T04:24:34.747501+0000","last_peered":"2023-11-28T04:24:34.747501+0000","last_clean":"2023-11-28T04:24:34.747501+0000","last_became_active":"2023-11-28T04:24:34.744146+0000","last_became_peered":"2023-11-28T04:24:34.744146+0000","last_unstale":"2023-11-28T04:24:34.747501+0000","last_undegraded":"2023-11-28T04:24:34.747501+0000","last_fullsized":"2023-11-28T04:24:34.747501+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107194+0000","last_change":"2023-11-28T04:24:35.107194+0000","last_active":"2023-11-28T04:24:35.107194+0000","last_peered":"2023-11-28T04:24:35.107194+0000","last_clean":"2023-11-28T04:24:35.107194+0000","last_became_active":"2023-11-28T04:24:35.106932+0000","last_became_peered":"2023-11-28T04:24:35.106932+0000","last_unstale":"2023-11-28T04:24:35.107194+0000","last_undegraded":"2023-11-28T04:24:35.107194+0000","last_fullsized":"2023-11-28T04:24:35.107194+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741566+0000","last_change":"2023-11-28T04:24:34.732015+0000","last_active":"2023-11-28T04:24:34.741566+0000","last_peered":"2023-11-28T04:24:34.741566+0000","last_clean":"2023-11-28T04:24:34.741566+0000","last_became_active":"2023-11-28T04:24:34.731857+0000","last_became_peered":"2023-11-28T04:24:34.731857+0000","last_unstale":"2023-11-28T04:24:34.741566+0000","last_undegraded":"2023-11-28T04:24:34.741566+0000","last_fullsized":"2023-11-28T04:24:34.741566+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113482+0000","last_change":"2023-11-28T04:24:35.113482+0000","last_active":"2023-11-28T04:24:35.113482+0000","last_peered":"2023-11-28T04:24:35.113482+0000","last_clean":"2023-11-28T04:24:35.113482+0000","last_became_active":"2023-11-28T04:24:35.113209+0000","last_became_peered":"2023-11-28T04:24:35.113209+0000","last_unstale":"2023-11-28T04:24:35.113482+0000","last_undegraded":"2023-11-28T04:24:35.113482+0000","last_fullsized":"2023-11-28T04:24:35.113482+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.745112+0000","last_change":"2023-11-28T04:24:34.740814+0000","last_active":"2023-11-28T04:24:34.745112+0000","last_peered":"2023-11-28T04:24:34.745112+0000","last_clean":"2023-11-28T04:24:34.745112+0000","last_became_active":"2023-11-28T04:24:34.740557+0000","last_became_peered":"2023-11-28T04:24:34.740557+0000","last_unstale":"2023-11-28T04:24:34.745112+0000","last_undegraded":"2023-11-28T04:24:34.745112+0000","last_fullsized":"2023-11-28T04:24:34.745112+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103281+0000","last_change":"2023-11-28T04:24:35.103281+0000","last_active":"2023-11-28T04:24:35.103281+0000","last_peered":"2023-11-28T04:24:35.103281+0000","last_clean":"2023-11-28T04:24:35.103281+0000","last_became_active":"2023-11-28T04:24:35.103014+0000","last_became_peered":"2023-11-28T04:24:35.103014+0000","last_unstale":"2023-11-28T04:24:35.103281+0000","last_undegraded":"2023-11-28T04:24:35.103281+0000","last_fullsized":"2023-11-28T04:24:35.103281+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107906+0000","last_change":"2023-11-28T04:24:34.731547+0000","last_active":"2023-11-28T04:24:35.107906+0000","last_peered":"2023-11-28T04:24:35.107906+0000","last_clean":"2023-11-28T04:24:35.107906+0000","last_became_active":"2023-11-28T04:24:34.731254+0000","last_became_peered":"2023-11-28T04:24:34.731254+0000","last_unstale":"2023-11-28T04:24:35.107906+0000","last_undegraded":"2023-11-28T04:24:35.107906+0000","last_fullsized":"2023-11-28T04:24:35.107906+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107573+0000","last_change":"2023-11-28T04:24:35.107573+0000","last_active":"2023-11-28T04:24:35.107573+0000","last_peered":"2023-11-28T04:24:35.107573+0000","last_clean":"2023-11-28T04:24:35.107573+0000","last_became_active":"2023-11-28T04:24:35.106948+0000","last_became_peered":"2023-11-28T04:24:35.106948+0000","last_unstale":"2023-11-28T04:24:35.107573+0000","last_undegraded":"2023-11-28T04:24:35.107573+0000","last_fullsized":"2023-11-28T04:24:35.107573+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750058+0000","last_change":"2023-11-28T04:24:34.745571+0000","last_active":"2023-11-28T04:24:34.750058+0000","last_peered":"2023-11-28T04:24:34.750058+0000","last_clean":"2023-11-28T04:24:34.750058+0000","last_became_active":"2023-11-28T04:24:34.745306+0000","last_became_peered":"2023-11-28T04:24:34.745306+0000","last_unstale":"2023-11-28T04:24:34.750058+0000","last_undegraded":"2023-11-28T04:24:34.750058+0000","last_fullsized":"2023-11-28T04:24:34.750058+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103341+0000","last_change":"2023-11-28T04:24:35.103341+0000","last_active":"2023-11-28T04:24:35.103341+0000","last_peered":"2023-11-28T04:24:35.103341+0000","last_clean":"2023-11-28T04:24:35.103341+0000","last_became_active":"2023-11-28T04:24:35.103063+0000","last_became_peered":"2023-11-28T04:24:35.103063+0000","last_unstale":"2023-11-28T04:24:35.103341+0000","last_undegraded":"2023-11-28T04:24:35.103341+0000","last_fullsized":"2023-11-28T04:24:35.103341+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742755+0000","last_change":"2023-11-28T04:24:34.738406+0000","last_active":"2023-11-28T04:24:34.742755+0000","last_peered":"2023-11-28T04:24:34.742755+0000","last_clean":"2023-11-28T04:24:34.742755+0000","last_became_active":"2023-11-28T04:24:34.738126+0000","last_became_peered":"2023-11-28T04:24:34.738126+0000","last_unstale":"2023-11-28T04:24:34.742755+0000","last_undegraded":"2023-11-28T04:24:34.742755+0000","last_fullsized":"2023-11-28T04:24:34.742755+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113570+0000","last_change":"2023-11-28T04:24:35.113570+0000","last_active":"2023-11-28T04:24:35.113570+0000","last_peered":"2023-11-28T04:24:35.113570+0000","last_clean":"2023-11-28T04:24:35.113570+0000","last_became_active":"2023-11-28T04:24:35.113321+0000","last_became_peered":"2023-11-28T04:24:35.113321+0000","last_unstale":"2023-11-28T04:24:35.113570+0000","last_undegraded":"2023-11-28T04:24:35.113570+0000","last_fullsized":"2023-11-28T04:24:35.113570+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750233+0000","last_change":"2023-11-28T04:24:34.748547+0000","last_active":"2023-11-28T04:24:34.750233+0000","last_peered":"2023-11-28T04:24:34.750233+0000","last_clean":"2023-11-28T04:24:34.750233+0000","last_became_active":"2023-11-28T04:24:34.748278+0000","last_became_peered":"2023-11-28T04:24:34.748278+0000","last_unstale":"2023-11-28T04:24:34.750233+0000","last_undegraded":"2023-11-28T04:24:34.750233+0000","last_fullsized":"2023-11-28T04:24:34.750233+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113149+0000","last_change":"2023-11-28T04:24:35.113149+0000","last_active":"2023-11-28T04:24:35.113149+0000","last_peered":"2023-11-28T04:24:35.113149+0000","last_clean":"2023-11-28T04:24:35.113149+0000","last_became_active":"2023-11-28T04:24:35.112802+0000","last_became_peered":"2023-11-28T04:24:35.112802+0000","last_unstale":"2023-11-28T04:24:35.113149+0000","last_undegraded":"2023-11-28T04:24:35.113149+0000","last_fullsized":"2023-11-28T04:24:35.113149+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747558+0000","last_change":"2023-11-28T04:24:34.743755+0000","last_active":"2023-11-28T04:24:34.747558+0000","last_peered":"2023-11-28T04:24:34.747558+0000","last_clean":"2023-11-28T04:24:34.747558+0000","last_became_active":"2023-11-28T04:24:34.743485+0000","last_became_peered":"2023-11-28T04:24:34.743485+0000","last_unstale":"2023-11-28T04:24:34.747558+0000","last_undegraded":"2023-11-28T04:24:34.747558+0000","last_fullsized":"2023-11-28T04:24:34.747558+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116495+0000","last_change":"2023-11-28T04:24:35.116495+0000","last_active":"2023-11-28T04:24:35.116495+0000","last_peered":"2023-11-28T04:24:35.116495+0000","last_clean":"2023-11-28T04:24:35.116495+0000","last_became_active":"2023-11-28T04:24:35.116203+0000","last_became_peered":"2023-11-28T04:24:35.116203+0000","last_unstale":"2023-11-28T04:24:35.116495+0000","last_undegraded":"2023-11-28T04:24:35.116495+0000","last_fullsized":"2023-11-28T04:24:35.116495+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748207+0000","last_change":"2023-11-28T04:24:34.743493+0000","last_active":"2023-11-28T04:24:34.748207+0000","last_peered":"2023-11-28T04:24:34.748207+0000","last_clean":"2023-11-28T04:24:34.748207+0000","last_became_active":"2023-11-28T04:24:34.742776+0000","last_became_peered":"2023-11-28T04:24:34.742776+0000","last_unstale":"2023-11-28T04:24:34.748207+0000","last_undegraded":"2023-11-28T04:24:34.748207+0000","last_fullsized":"2023-11-28T04:24:34.748207+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115808+0000","last_change":"2023-11-28T04:24:35.115808+0000","last_active":"2023-11-28T04:24:35.115808+0000","last_peered":"2023-11-28T04:24:35.115808+0000","last_clean":"2023-11-28T04:24:35.115808+0000","last_became_active":"2023-11-28T04:24:35.115534+0000","last_became_peered":"2023-11-28T04:24:35.115534+0000","last_unstale":"2023-11-28T04:24:35.115808+0000","last_undegraded":"2023-11-28T04:24:35.115808+0000","last_fullsized":"2023-11-28T04:24:35.115808+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747461+0000","last_change":"2023-11-28T04:24:34.743250+0000","last_active":"2023-11-28T04:24:34.747461+0000","last_peered":"2023-11-28T04:24:34.747461+0000","last_clean":"2023-11-28T04:24:34.747461+0000","last_became_active":"2023-11-28T04:24:34.742734+0000","last_became_peered":"2023-11-28T04:24:34.742734+0000","last_unstale":"2023-11-28T04:24:34.747461+0000","last_undegraded":"2023-11-28T04:24:34.747461+0000","last_fullsized":"2023-11-28T04:24:34.747461+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105298+0000","last_change":"2023-11-28T04:24:35.105298+0000","last_active":"2023-11-28T04:24:35.105298+0000","last_peered":"2023-11-28T04:24:35.105298+0000","last_clean":"2023-11-28T04:24:35.105298+0000","last_became_active":"2023-11-28T04:24:35.105013+0000","last_became_peered":"2023-11-28T04:24:35.105013+0000","last_unstale":"2023-11-28T04:24:35.105298+0000","last_undegraded":"2023-11-28T04:24:35.105298+0000","last_fullsized":"2023-11-28T04:24:35.105298+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742230+0000","last_change":"2023-11-28T04:24:34.736589+0000","last_active":"2023-11-28T04:24:34.742230+0000","last_peered":"2023-11-28T04:24:34.742230+0000","last_clean":"2023-11-28T04:24:34.742230+0000","last_became_active":"2023-11-28T04:24:34.736331+0000","last_became_peered":"2023-11-28T04:24:34.736331+0000","last_unstale":"2023-11-28T04:24:34.742230+0000","last_undegraded":"2023-11-28T04:24:34.742230+0000","last_fullsized":"2023-11-28T04:24:34.742230+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104048+0000","last_change":"2023-11-28T04:24:35.104048+0000","last_active":"2023-11-28T04:24:35.104048+0000","last_peered":"2023-11-28T04:24:35.104048+0000","last_clean":"2023-11-28T04:24:35.104048+0000","last_became_active":"2023-11-28T04:24:35.103782+0000","last_became_peered":"2023-11-28T04:24:35.103782+0000","last_unstale":"2023-11-28T04:24:35.104048+0000","last_undegraded":"2023-11-28T04:24:35.104048+0000","last_fullsized":"2023-11-28T04:24:35.104048+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750266+0000","last_change":"2023-11-28T04:24:34.746313+0000","last_active":"2023-11-28T04:24:34.750266+0000","last_peered":"2023-11-28T04:24:34.750266+0000","last_clean":"2023-11-28T04:24:34.750266+0000","last_became_active":"2023-11-28T04:24:34.745826+0000","last_became_peered":"2023-11-28T04:24:34.745826+0000","last_unstale":"2023-11-28T04:24:34.750266+0000","last_undegraded":"2023-11-28T04:24:34.750266+0000","last_fullsized":"2023-11-28T04:24:34.750266+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728575+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:24:34.728575+0000","last_peered":"2023-11-28T04:24:34.728575+0000","last_clean":"2023-11-28T04:24:34.728575+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:24:34.728575+0000","last_undegraded":"2023-11-28T04:24:34.728575+0000","last_fullsized":"2023-11-28T04:24:34.728575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118109+0000","last_change":"2023-11-28T04:24:35.118109+0000","last_active":"2023-11-28T04:24:35.118109+0000","last_peered":"2023-11-28T04:24:35.118109+0000","last_clean":"2023-11-28T04:24:35.118109+0000","last_became_active":"2023-11-28T04:24:35.117658+0000","last_became_peered":"2023-11-28T04:24:35.117658+0000","last_unstale":"2023-11-28T04:24:35.118109+0000","last_undegraded":"2023-11-28T04:24:35.118109+0000","last_fullsized":"2023-11-28T04:24:35.118109+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750135+0000","last_change":"2023-11-28T04:24:34.747233+0000","last_active":"2023-11-28T04:24:34.750135+0000","last_peered":"2023-11-28T04:24:34.750135+0000","last_clean":"2023-11-28T04:24:34.750135+0000","last_became_active":"2023-11-28T04:24:34.746948+0000","last_became_peered":"2023-11-28T04:24:34.746948+0000","last_unstale":"2023-11-28T04:24:34.750135+0000","last_undegraded":"2023-11-28T04:24:34.750135+0000","last_fullsized":"2023-11-28T04:24:34.750135+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.731095+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:24:34.731095+0000","last_peered":"2023-11-28T04:24:34.731095+0000","last_clean":"2023-11-28T04:24:34.731095+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:24:34.731095+0000","last_undegraded":"2023-11-28T04:24:34.731095+0000","last_fullsized":"2023-11-28T04:24:34.731095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106535+0000","last_change":"2023-11-28T04:24:35.106535+0000","last_active":"2023-11-28T04:24:35.106535+0000","last_peered":"2023-11-28T04:24:35.106535+0000","last_clean":"2023-11-28T04:24:35.106535+0000","last_became_active":"2023-11-28T04:24:35.105171+0000","last_became_peered":"2023-11-28T04:24:35.105171+0000","last_unstale":"2023-11-28T04:24:35.106535+0000","last_undegraded":"2023-11-28T04:24:35.106535+0000","last_fullsized":"2023-11-28T04:24:35.106535+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110642+0000","last_change":"2023-11-28T04:24:34.746128+0000","last_active":"2023-11-28T04:24:35.110642+0000","last_peered":"2023-11-28T04:24:35.110642+0000","last_clean":"2023-11-28T04:24:35.110642+0000","last_became_active":"2023-11-28T04:24:34.745853+0000","last_became_peered":"2023-11-28T04:24:34.745853+0000","last_unstale":"2023-11-28T04:24:35.110642+0000","last_undegraded":"2023-11-28T04:24:35.110642+0000","last_fullsized":"2023-11-28T04:24:35.110642+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.724501+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:24:34.724501+0000","last_peered":"2023-11-28T04:24:34.724501+0000","last_clean":"2023-11-28T04:24:34.724501+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:24:34.724501+0000","last_undegraded":"2023-11-28T04:24:34.724501+0000","last_fullsized":"2023-11-28T04:24:34.724501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109989+0000","last_change":"2023-11-28T04:24:35.109989+0000","last_active":"2023-11-28T04:24:35.109989+0000","last_peered":"2023-11-28T04:24:35.109989+0000","last_clean":"2023-11-28T04:24:35.109989+0000","last_became_active":"2023-11-28T04:24:35.109692+0000","last_became_peered":"2023-11-28T04:24:35.109692+0000","last_unstale":"2023-11-28T04:24:35.109989+0000","last_undegraded":"2023-11-28T04:24:35.109989+0000","last_fullsized":"2023-11-28T04:24:35.109989+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750079+0000","last_change":"2023-11-28T04:24:34.745644+0000","last_active":"2023-11-28T04:24:34.750079+0000","last_peered":"2023-11-28T04:24:34.750079+0000","last_clean":"2023-11-28T04:24:34.750079+0000","last_became_active":"2023-11-28T04:24:34.745383+0000","last_became_peered":"2023-11-28T04:24:34.745383+0000","last_unstale":"2023-11-28T04:24:34.750079+0000","last_undegraded":"2023-11-28T04:24:34.750079+0000","last_fullsized":"2023-11-28T04:24:34.750079+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.731689+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:24:34.731689+0000","last_peered":"2023-11-28T04:24:34.731689+0000","last_clean":"2023-11-28T04:24:34.731689+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:24:34.731689+0000","last_undegraded":"2023-11-28T04:24:34.731689+0000","last_fullsized":"2023-11-28T04:24:34.731689+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":155,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728516+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:24:34.728516+0000","last_peered":"2023-11-28T04:24:34.728516+0000","last_clean":"2023-11-28T04:24:34.728516+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:24:34.728516+0000","last_undegraded":"2023-11-28T04:24:34.728516+0000","last_fullsized":"2023-11-28T04:24:34.728516+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103284+0000","last_change":"2023-11-28T04:24:35.103284+0000","last_active":"2023-11-28T04:24:35.103284+0000","last_peered":"2023-11-28T04:24:35.103284+0000","last_clean":"2023-11-28T04:24:35.103284+0000","last_became_active":"2023-11-28T04:24:35.102999+0000","last_became_peered":"2023-11-28T04:24:35.102999+0000","last_unstale":"2023-11-28T04:24:35.103284+0000","last_undegraded":"2023-11-28T04:24:35.103284+0000","last_fullsized":"2023-11-28T04:24:35.103284+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"196'3","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115537+0000","last_change":"2023-11-28T04:24:34.741114+0000","last_active":"2023-11-28T04:24:35.115537+0000","last_peered":"2023-11-28T04:24:35.115537+0000","last_clean":"2023-11-28T04:24:35.115537+0000","last_became_active":"2023-11-28T04:24:34.740810+0000","last_became_peered":"2023-11-28T04:24:34.740810+0000","last_unstale":"2023-11-28T04:24:35.115537+0000","last_undegraded":"2023-11-28T04:24:35.115537+0000","last_fullsized":"2023-11-28T04:24:35.115537+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728571+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:24:34.728571+0000","last_peered":"2023-11-28T04:24:34.728571+0000","last_clean":"2023-11-28T04:24:34.728571+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:24:34.728571+0000","last_undegraded":"2023-11-28T04:24:34.728571+0000","last_fullsized":"2023-11-28T04:24:34.728571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114219+0000","last_change":"2023-11-28T04:24:35.114219+0000","last_active":"2023-11-28T04:24:35.114219+0000","last_peered":"2023-11-28T04:24:35.114219+0000","last_clean":"2023-11-28T04:24:35.114219+0000","last_became_active":"2023-11-28T04:24:35.113936+0000","last_became_peered":"2023-11-28T04:24:35.113936+0000","last_unstale":"2023-11-28T04:24:35.114219+0000","last_undegraded":"2023-11-28T04:24:35.114219+0000","last_fullsized":"2023-11-28T04:24:35.114219+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.119839+0000","last_change":"2023-11-28T04:24:34.740421+0000","last_active":"2023-11-28T04:24:35.119839+0000","last_peered":"2023-11-28T04:24:35.119839+0000","last_clean":"2023-11-28T04:24:35.119839+0000","last_became_active":"2023-11-28T04:24:34.740193+0000","last_became_peered":"2023-11-28T04:24:34.740193+0000","last_unstale":"2023-11-28T04:24:35.119839+0000","last_undegraded":"2023-11-28T04:24:35.119839+0000","last_fullsized":"2023-11-28T04:24:35.119839+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728562+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:24:34.728562+0000","last_peered":"2023-11-28T04:24:34.728562+0000","last_clean":"2023-11-28T04:24:34.728562+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:24:34.728562+0000","last_undegraded":"2023-11-28T04:24:34.728562+0000","last_fullsized":"2023-11-28T04:24:34.728562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110020+0000","last_change":"2023-11-28T04:24:35.110020+0000","last_active":"2023-11-28T04:24:35.110020+0000","last_peered":"2023-11-28T04:24:35.110020+0000","last_clean":"2023-11-28T04:24:35.110020+0000","last_became_active":"2023-11-28T04:24:35.108591+0000","last_became_peered":"2023-11-28T04:24:35.108591+0000","last_unstale":"2023-11-28T04:24:35.110020+0000","last_undegraded":"2023-11-28T04:24:35.110020+0000","last_fullsized":"2023-11-28T04:24:35.110020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.092879+0000","last_change":"2023-11-28T04:24:34.739859+0000","last_active":"2023-11-28T04:24:35.092879+0000","last_peered":"2023-11-28T04:24:35.092879+0000","last_clean":"2023-11-28T04:24:35.092879+0000","last_became_active":"2023-11-28T04:24:34.739589+0000","last_became_peered":"2023-11-28T04:24:34.739589+0000","last_unstale":"2023-11-28T04:24:35.092879+0000","last_undegraded":"2023-11-28T04:24:35.092879+0000","last_fullsized":"2023-11-28T04:24:35.092879+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.730721+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:24:34.730721+0000","last_peered":"2023-11-28T04:24:34.730721+0000","last_clean":"2023-11-28T04:24:34.730721+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:24:34.730721+0000","last_undegraded":"2023-11-28T04:24:34.730721+0000","last_fullsized":"2023-11-28T04:24:34.730721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109750+0000","last_change":"2023-11-28T04:24:35.109750+0000","last_active":"2023-11-28T04:24:35.109750+0000","last_peered":"2023-11-28T04:24:35.109750+0000","last_clean":"2023-11-28T04:24:35.109750+0000","last_became_active":"2023-11-28T04:24:35.109465+0000","last_became_peered":"2023-11-28T04:24:35.109465+0000","last_unstale":"2023-11-28T04:24:35.109750+0000","last_undegraded":"2023-11-28T04:24:35.109750+0000","last_fullsized":"2023-11-28T04:24:35.109750+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.743569+0000","last_change":"2023-11-28T04:24:34.740418+0000","last_active":"2023-11-28T04:24:34.743569+0000","last_peered":"2023-11-28T04:24:34.743569+0000","last_clean":"2023-11-28T04:24:34.743569+0000","last_became_active":"2023-11-28T04:24:34.740140+0000","last_became_peered":"2023-11-28T04:24:34.740140+0000","last_unstale":"2023-11-28T04:24:34.743569+0000","last_undegraded":"2023-11-28T04:24:34.743569+0000","last_fullsized":"2023-11-28T04:24:34.743569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":173,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.726681+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:24:34.726681+0000","last_peered":"2023-11-28T04:24:34.726681+0000","last_clean":"2023-11-28T04:24:34.726681+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:24:34.726681+0000","last_undegraded":"2023-11-28T04:24:34.726681+0000","last_fullsized":"2023-11-28T04:24:34.726681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575945,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303572,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068268,"statfs":{"total":96636764160,"available":96325906432,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.84499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.622},"min":{"1min":0.496,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":0.77600000000000002,"5min":0.80400000000000005,"15min":0.872},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.496},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73399999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.89000000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36099999999999999},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.35999999999999999},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.83599999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.39700000000000002},"max":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.90300000000000002},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46400000000000002,"15min":0.46700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.65900000000000003,"5min":0.71699999999999997,"15min":0.73399999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.496,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80200000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.79900000000000004,"15min":0.81899999999999995},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575945,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.39000000000000001,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.68400000000000005,"5min":0.77500000000000002,"15min":0.77600000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.28499999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.71999999999999997,"5min":0.81999999999999995,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.46200000000000002,"5min":0.379,"15min":0.29899999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80900000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.502}]},{"osd":3,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.84999999999999998},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.60899999999999999,"5min":0.78800000000000003,"15min":0.92400000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47399999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76500000000000001,"15min":0.96899999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.77700000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.315},"max":{"1min":0.749,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.752,"15min":0.81399999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73699999999999999,"5min":0.82999999999999996,"15min":0.83299999999999996},"last":0.71799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303560,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068280,"statfs":{"total":96636764160,"available":96325918720,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.372},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.74299999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":0.626,"15min":0.67000000000000004},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41199999999999998},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.24299999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.81200000000000006},"last":0.432},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65500000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.47199999999999998,"5min":0.42299999999999999,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.99099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.98599999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64500000000000002,"15min":0.65000000000000002},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.0349999999999999},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.51600000000000001,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.999},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.44600000000000001},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":1.02},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.52300000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.95499999999999996},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575945,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.36399999999999999},"min":{"1min":0.25800000000000001,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.97799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.61699999999999999,"15min":0.95099999999999996},"last":0.38}]},{"osd":2,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.85999999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61299999999999999,"5min":0.66000000000000003,"15min":0.86799999999999999},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.64600000000000002},"last":0.442},{"interface":"front","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.59399999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61799999999999999,"15min":0.64100000000000001},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":1.0409999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.038},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":1.0800000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.878,"15min":1.0549999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.95099999999999996,"15min":1.0449999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.42199999999999999},"max":{"1min":0.84499999999999997,"5min":0.876,"15min":1.036},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.97299999999999998,"15min":1.0189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.69899999999999995},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.0669999999999999},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834575946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":0.45800000000000002,"5min":0.57199999999999995,"15min":0.61899999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.31900000000000001,"5min":0.314,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.67000000000000004},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.25900000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.622},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41099999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44400000000000001,"15min":0.443},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65900000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41899999999999998,"15min":0.42199999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38}]},{"osd":4,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.82599999999999996,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.622},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79800000000000004,"5min":0.87,"15min":0.96199999999999997},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.44500000000000001},"max":{"1min":0.82199999999999995,"5min":0.92000000000000004,"15min":1.0760000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.92500000000000004,"15min":1.016},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.93999999999999995,"15min":0.99099999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84599999999999997,"5min":0.93400000000000005,"15min":1.0529999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.95599999999999996},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":1.0129999999999999},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575945,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":0.47999999999999998,"5min":0.57399999999999995,"15min":0.58899999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.501,"5min":0.54500000000000004,"15min":0.66500000000000004},"last":0.26400000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.39100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.52100000000000002,"5min":0.53800000000000003,"15min":0.65500000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.43099999999999999,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63900000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.746},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27400000000000002},"max":{"1min":0.57099999999999995,"5min":0.63300000000000001,"15min":0.68000000000000005},"last":0.27700000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.92500000000000004,"15min":0.96099999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.42699999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0409999999999999},"last":0.437},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.71099999999999997,"15min":0.68000000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.3049999999999997,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.378}]},{"osd":6,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65900000000000003,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85599999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":1.028},"last":0.40999999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.82499999999999996,"5min":1.008,"15min":1.008},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.89900000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.49299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575944,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65600000000000003,"5min":0.80600000000000005,"15min":0.83699999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.78800000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.38900000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.874},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.67900000000000005,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67100000000000004,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":0.78700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.52500000000000002,"5min":0.51900000000000002,"15min":0.42499999999999999},"max":{"1min":0.77900000000000003,"5min":0.79900000000000004,"15min":0.872},"last":0.61799999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.83499999999999996},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.504,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71999999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.79100000000000004,"15min":0.85099999999999998},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.82499999999999996,"15min":0.86099999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.35099999999999998,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":0.82499999999999996},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.80800000000000005,"15min":0.82199999999999995},"last":0.628},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.48299999999999998,"5min":0.38400000000000001,"15min":0.36099999999999999},"max":{"1min":0.745,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.84999999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70399999999999996,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":0.85199999999999998},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.88200000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.495,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64200000000000002,"5min":0.77300000000000002,"15min":0.872},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":0.82699999999999996,"15min":0.92100000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.90300000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.90700000000000003},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:24:38.075 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:24:38.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.222+0000 7f39e5ec4700 1 Processor -- start 2023-11-28T04:24:38.223 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.222+0000 7f39e5ec4700 1 -- start start 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39e5ec4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f39e01520c0 0x7f39e0066520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39e5ec4700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e00be110 con 0x7f39e01524a0 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39e5ec4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39e0066fc0 con 0x7f39e0068150 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39e5ec4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39e00672a0 con 0x7f39e01520c0 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39d6ffd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f39e01524a0 0x7f39e0067a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40784/0 (socket says 172.21.15.73:40784) 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39d6ffd700 1 -- 172.21.15.73:0/2287412438 learned_addr learned my addr 172.21.15.73:0/2287412438 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:38.227 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 118542884 0 0) 0x7f39e00be110 con 0x7f39e01524a0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39cc0033f0 con 0x7f39e01524a0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.223+0000 7f39df7fe700 1 --2- 172.21.15.73:0/2287412438 >> v2:172.21.15.150:3300/0 conn(0x7f39e01520c0 0x7f39e0066520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3427976780 0 0) 0x7f39cc0033f0 con 0x7f39e01524a0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 >> v2:172.21.15.150:3300/0 conn(0x7f39e01520c0 msgr2=0x7f39e0066520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 --2- 172.21.15.73:0/2287412438 >> v2:172.21.15.150:3300/0 conn(0x7f39e01520c0 0x7f39e0066520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 >> v1:172.21.15.150:6789/0 conn(0x7f39e0068150 legacy=0x7f39e00be8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00be380 con 0x7f39e01524a0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f39c8001470 con 0x7f39e01524a0 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 >> v1:172.21.15.73:6789/0 conn(0x7f39e01524a0 legacy=0x7f39e0067a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39cc0038e0 0x7f39cc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 0x7f39cc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc005050 0x7f39cc00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00672a0 con 0x7f39cc004690 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc00bf30 con 0x7f39cc005050 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc00c0f0 con 0x7f39cc0038e0 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 0x7f39cc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39df7fe700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39cc0038e0 0x7f39cc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39dffff700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc005050 0x7f39cc00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 -- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39cc0038e0 msgr2=0x7f39cc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39cc0038e0 0x7f39cc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 -- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc005050 msgr2=0x7f39cc00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc005050 0x7f39cc00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39dffff700 1 -- 172.21.15.73:0/2287412438 reap_dead start 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 -- 172.21.15.73:0/2287412438 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc0033f0 con 0x7f39cc004690 2023-11-28T04:24:38.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39d6ffd700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 0x7f39cc004b10 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 rx=0x7f39c800c160 tx=0x7f39c8001e50).ready entity=mon.0 client_cookie=f64d546cc491fd4a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8007a20 con 0x7f39cc004690 2023-11-28T04:24:38.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c800ca90 con 0x7f39cc004690 2023-11-28T04:24:38.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.224+0000 7f39deffd700 1 -- 172.21.15.73:0/2287412438 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c800ce10 con 0x7f39cc004690 2023-11-28T04:24:38.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 -- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 msgr2=0x7f39cc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 0x7f39cc004b10 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 rx=0x7f39c800c160 tx=0x7f39c8001e50).stop 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 -- 172.21.15.73:0/2287412438 shutdown_connections 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/2287412438 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39cc004690 0x7f39cc004b10 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 -- 172.21.15.73:0/2287412438 >> 172.21.15.73:0/2287412438 conn(0x7f39e00631d0 msgr2=0x7f39e00c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 -- 172.21.15.73:0/2287412438 shutdown_connections 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.225+0000 7f39e5ec4700 1 -- 172.21.15.73:0/2287412438 wait complete. 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.226+0000 7f39e5ec4700 1 Processor -- start 2023-11-28T04:24:38.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.226+0000 7f39e5ec4700 1 -- start start 2023-11-28T04:24:38.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 0x7f39e00c2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39e01505e0 0x7f39e014e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e014ea20 con 0x7f39e00c1440 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00af160 con 0x7f39cc004690 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39e5ec4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39e00c2750 con 0x7f39e01505e0 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41880/0 (socket says 172.21.15.73:41880) 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 -- 172.21.15.73:0/1351716316 learned_addr learned my addr 172.21.15.73:0/1351716316 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 -- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39e01505e0 msgr2=0x7f39e014e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39e01505e0 0x7f39e014e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:38.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 -- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 msgr2=0x7f39e00c2be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39df7fe700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 0x7f39e00c2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 0x7f39e00c2be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 -- 172.21.15.73:0/1351716316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39e00c18c0 con 0x7f39e00c1440 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39df7fe700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 0x7f39e00c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.227+0000 7f39dffff700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7f39e006a980 tx=0x7f39d8007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.228+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d8006c30 con 0x7f39e00c1440 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.228+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39d8006d90 con 0x7f39e00c1440 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.228+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39d80072a0 con 0x7f39e00c1440 2023-11-28T04:24:38.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.228+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39e00c1b40 con 0x7f39e00c1440 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.228+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39e00c2100 con 0x7f39e00c1440 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.229+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39e006c860 con 0x7f39e00c1440 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.229+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f39d8007610 con 0x7f39e00c1440 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.229+0000 7f39d7fff700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 0x7f39c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.229+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f39d80799d0 con 0x7f39e00c1440 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.229+0000 7f39df7fe700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 0x7f39c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.236 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.230+0000 7f39df7fe700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 0x7f39c0061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f39d0006d20 tx=0x7f39d0008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.237 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.233+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f39d8049af0 con 0x7f39e00c1440 2023-11-28T04:24:38.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.394+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f39e00c2890 con 0x7f39c005f150 2023-11-28T04:24:38.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.404+0000 7f39d7fff700 1 -- 172.21.15.73:0/1351716316 <== mgr.9345 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362381 (secure 0 0 0) 0x7f39e00c2890 con 0x7f39c005f150 2023-11-28T04:24:38.409 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 msgr2=0x7f39c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 0x7f39c0061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f39d0006d20 tx=0x7f39d0008040).stop 2023-11-28T04:24:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 msgr2=0x7f39e00acd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.412 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 rx=0x7f39e006a980 tx=0x7f39d8007f30).stop 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 shutdown_connections 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f39c005f150 0x7f39c0061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f39e00c1440 0x7f39e00acd00 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f39cc004690 0x7f39e00c2be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 --2- 172.21.15.73:0/1351716316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f39e01505e0 0x7f39e014e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:38.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 >> 172.21.15.73:0/1351716316 conn(0x7f39e00631d0 msgr2=0x7f39e00a5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 shutdown_connections 2023-11-28T04:24:38.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.407+0000 7f39e5ec4700 1 -- 172.21.15.73:0/1351716316 wait complete. 2023-11-28T04:24:38.414 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:24:38.420 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:24:38.174928+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":55,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2453104,"kb_used_data":18992,"kb_used_omap":0,"kb_used_meta":2434048,"kb_avail":752521616,"statfs":{"total":773094113280,"available":770582134784,"internally_reserved":0,"allocated":19447808,"data_stored":9818500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2492465152},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-592408,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-2,"num_read_kb":-1,"num_write":-207,"num_write_kb":-717,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":-1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.604167"},"pg_stats":[{"pgid":"24.30","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.120034+0000","last_change":"2023-11-28T04:24:35.120034+0000","last_active":"2023-11-28T04:24:35.120034+0000","last_peered":"2023-11-28T04:24:35.120034+0000","last_clean":"2023-11-28T04:24:35.120034+0000","last_became_active":"2023-11-28T04:24:35.119751+0000","last_became_peered":"2023-11-28T04:24:35.119751+0000","last_unstale":"2023-11-28T04:24:35.120034+0000","last_undegraded":"2023-11-28T04:24:35.120034+0000","last_fullsized":"2023-11-28T04:24:35.120034+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3f","version":"196'3","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113644+0000","last_change":"2023-11-28T04:24:34.740428+0000","last_active":"2023-11-28T04:24:35.113644+0000","last_peered":"2023-11-28T04:24:35.113644+0000","last_clean":"2023-11-28T04:24:35.113644+0000","last_became_active":"2023-11-28T04:24:34.740101+0000","last_became_peered":"2023-11-28T04:24:34.740101+0000","last_unstale":"2023-11-28T04:24:35.113644+0000","last_undegraded":"2023-11-28T04:24:35.113644+0000","last_fullsized":"2023-11-28T04:24:35.113644+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.31","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110763+0000","last_change":"2023-11-28T04:24:35.110763+0000","last_active":"2023-11-28T04:24:35.110763+0000","last_peered":"2023-11-28T04:24:35.110763+0000","last_clean":"2023-11-28T04:24:35.110763+0000","last_became_active":"2023-11-28T04:24:35.110481+0000","last_became_peered":"2023-11-28T04:24:35.110481+0000","last_unstale":"2023-11-28T04:24:35.110763+0000","last_undegraded":"2023-11-28T04:24:35.110763+0000","last_fullsized":"2023-11-28T04:24:35.110763+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741706+0000","last_change":"2023-11-28T04:24:34.738931+0000","last_active":"2023-11-28T04:24:34.741706+0000","last_peered":"2023-11-28T04:24:34.741706+0000","last_clean":"2023-11-28T04:24:34.741706+0000","last_became_active":"2023-11-28T04:24:34.738635+0000","last_became_peered":"2023-11-28T04:24:34.738635+0000","last_unstale":"2023-11-28T04:24:34.741706+0000","last_undegraded":"2023-11-28T04:24:34.741706+0000","last_fullsized":"2023-11-28T04:24:34.741706+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.32","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101200+0000","last_change":"2023-11-28T04:24:35.101200+0000","last_active":"2023-11-28T04:24:35.101200+0000","last_peered":"2023-11-28T04:24:35.101200+0000","last_clean":"2023-11-28T04:24:35.101200+0000","last_became_active":"2023-11-28T04:24:35.100809+0000","last_became_peered":"2023-11-28T04:24:35.100809+0000","last_unstale":"2023-11-28T04:24:35.101200+0000","last_undegraded":"2023-11-28T04:24:35.101200+0000","last_fullsized":"2023-11-28T04:24:35.101200+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750237+0000","last_change":"2023-11-28T04:24:34.746059+0000","last_active":"2023-11-28T04:24:34.750237+0000","last_peered":"2023-11-28T04:24:34.750237+0000","last_clean":"2023-11-28T04:24:34.750237+0000","last_became_active":"2023-11-28T04:24:34.745417+0000","last_became_peered":"2023-11-28T04:24:34.745417+0000","last_unstale":"2023-11-28T04:24:34.750237+0000","last_undegraded":"2023-11-28T04:24:34.750237+0000","last_fullsized":"2023-11-28T04:24:34.750237+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.33","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116254+0000","last_change":"2023-11-28T04:24:35.116254+0000","last_active":"2023-11-28T04:24:35.116254+0000","last_peered":"2023-11-28T04:24:35.116254+0000","last_clean":"2023-11-28T04:24:35.116254+0000","last_became_active":"2023-11-28T04:24:35.115839+0000","last_became_peered":"2023-11-28T04:24:35.115839+0000","last_unstale":"2023-11-28T04:24:35.116254+0000","last_undegraded":"2023-11-28T04:24:35.116254+0000","last_fullsized":"2023-11-28T04:24:35.116254+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.3c","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112720+0000","last_change":"2023-11-28T04:24:34.742092+0000","last_active":"2023-11-28T04:24:35.112720+0000","last_peered":"2023-11-28T04:24:35.112720+0000","last_clean":"2023-11-28T04:24:35.112720+0000","last_became_active":"2023-11-28T04:24:34.741803+0000","last_became_peered":"2023-11-28T04:24:34.741803+0000","last_unstale":"2023-11-28T04:24:35.112720+0000","last_undegraded":"2023-11-28T04:24:35.112720+0000","last_fullsized":"2023-11-28T04:24:35.112720+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.34","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114602+0000","last_change":"2023-11-28T04:24:35.114602+0000","last_active":"2023-11-28T04:24:35.114602+0000","last_peered":"2023-11-28T04:24:35.114602+0000","last_clean":"2023-11-28T04:24:35.114602+0000","last_became_active":"2023-11-28T04:24:35.114226+0000","last_became_peered":"2023-11-28T04:24:35.114226+0000","last_unstale":"2023-11-28T04:24:35.114602+0000","last_undegraded":"2023-11-28T04:24:35.114602+0000","last_fullsized":"2023-11-28T04:24:35.114602+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093477+0000","last_change":"2023-11-28T04:24:34.735064+0000","last_active":"2023-11-28T04:24:35.093477+0000","last_peered":"2023-11-28T04:24:35.093477+0000","last_clean":"2023-11-28T04:24:35.093477+0000","last_became_active":"2023-11-28T04:24:34.734836+0000","last_became_peered":"2023-11-28T04:24:34.734836+0000","last_unstale":"2023-11-28T04:24:35.093477+0000","last_undegraded":"2023-11-28T04:24:35.093477+0000","last_fullsized":"2023-11-28T04:24:35.093477+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.35","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104438+0000","last_change":"2023-11-28T04:24:35.104438+0000","last_active":"2023-11-28T04:24:35.104438+0000","last_peered":"2023-11-28T04:24:35.104438+0000","last_clean":"2023-11-28T04:24:35.104438+0000","last_became_active":"2023-11-28T04:24:35.104243+0000","last_became_peered":"2023-11-28T04:24:35.104243+0000","last_unstale":"2023-11-28T04:24:35.104438+0000","last_undegraded":"2023-11-28T04:24:35.104438+0000","last_fullsized":"2023-11-28T04:24:35.104438+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3a","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093920+0000","last_change":"2023-11-28T04:24:34.738025+0000","last_active":"2023-11-28T04:24:35.093920+0000","last_peered":"2023-11-28T04:24:35.093920+0000","last_clean":"2023-11-28T04:24:35.093920+0000","last_became_active":"2023-11-28T04:24:34.737763+0000","last_became_peered":"2023-11-28T04:24:34.737763+0000","last_unstale":"2023-11-28T04:24:35.093920+0000","last_undegraded":"2023-11-28T04:24:35.093920+0000","last_fullsized":"2023-11-28T04:24:35.093920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.36","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111410+0000","last_change":"2023-11-28T04:24:35.111410+0000","last_active":"2023-11-28T04:24:35.111410+0000","last_peered":"2023-11-28T04:24:35.111410+0000","last_clean":"2023-11-28T04:24:35.111410+0000","last_became_active":"2023-11-28T04:24:35.111149+0000","last_became_peered":"2023-11-28T04:24:35.111149+0000","last_unstale":"2023-11-28T04:24:35.111410+0000","last_undegraded":"2023-11-28T04:24:35.111410+0000","last_fullsized":"2023-11-28T04:24:35.111410+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.744180+0000","last_change":"2023-11-28T04:24:34.736450+0000","last_active":"2023-11-28T04:24:34.744180+0000","last_peered":"2023-11-28T04:24:34.744180+0000","last_clean":"2023-11-28T04:24:34.744180+0000","last_became_active":"2023-11-28T04:24:34.736210+0000","last_became_peered":"2023-11-28T04:24:34.736210+0000","last_unstale":"2023-11-28T04:24:34.744180+0000","last_undegraded":"2023-11-28T04:24:34.744180+0000","last_fullsized":"2023-11-28T04:24:34.744180+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.37","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107926+0000","last_change":"2023-11-28T04:24:35.107926+0000","last_active":"2023-11-28T04:24:35.107926+0000","last_peered":"2023-11-28T04:24:35.107926+0000","last_clean":"2023-11-28T04:24:35.107926+0000","last_became_active":"2023-11-28T04:24:35.107697+0000","last_became_peered":"2023-11-28T04:24:35.107697+0000","last_unstale":"2023-11-28T04:24:35.107926+0000","last_undegraded":"2023-11-28T04:24:35.107926+0000","last_fullsized":"2023-11-28T04:24:35.107926+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741855+0000","last_change":"2023-11-28T04:24:34.740311+0000","last_active":"2023-11-28T04:24:34.741855+0000","last_peered":"2023-11-28T04:24:34.741855+0000","last_clean":"2023-11-28T04:24:34.741855+0000","last_became_active":"2023-11-28T04:24:34.739733+0000","last_became_peered":"2023-11-28T04:24:34.739733+0000","last_unstale":"2023-11-28T04:24:34.741855+0000","last_undegraded":"2023-11-28T04:24:34.741855+0000","last_fullsized":"2023-11-28T04:24:34.741855+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.38","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111348+0000","last_change":"2023-11-28T04:24:35.111348+0000","last_active":"2023-11-28T04:24:35.111348+0000","last_peered":"2023-11-28T04:24:35.111348+0000","last_clean":"2023-11-28T04:24:35.111348+0000","last_became_active":"2023-11-28T04:24:35.111072+0000","last_became_peered":"2023-11-28T04:24:35.111072+0000","last_unstale":"2023-11-28T04:24:35.111348+0000","last_undegraded":"2023-11-28T04:24:35.111348+0000","last_fullsized":"2023-11-28T04:24:35.111348+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741675+0000","last_change":"2023-11-28T04:24:34.738385+0000","last_active":"2023-11-28T04:24:34.741675+0000","last_peered":"2023-11-28T04:24:34.741675+0000","last_clean":"2023-11-28T04:24:34.741675+0000","last_became_active":"2023-11-28T04:24:34.738044+0000","last_became_peered":"2023-11-28T04:24:34.738044+0000","last_unstale":"2023-11-28T04:24:34.741675+0000","last_undegraded":"2023-11-28T04:24:34.741675+0000","last_fullsized":"2023-11-28T04:24:34.741675+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.39","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107728+0000","last_change":"2023-11-28T04:24:35.107728+0000","last_active":"2023-11-28T04:24:35.107728+0000","last_peered":"2023-11-28T04:24:35.107728+0000","last_clean":"2023-11-28T04:24:35.107728+0000","last_became_active":"2023-11-28T04:24:35.107343+0000","last_became_peered":"2023-11-28T04:24:35.107343+0000","last_unstale":"2023-11-28T04:24:35.107728+0000","last_undegraded":"2023-11-28T04:24:35.107728+0000","last_fullsized":"2023-11-28T04:24:35.107728+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747547+0000","last_change":"2023-11-28T04:24:34.745745+0000","last_active":"2023-11-28T04:24:34.747547+0000","last_peered":"2023-11-28T04:24:34.747547+0000","last_clean":"2023-11-28T04:24:34.747547+0000","last_became_active":"2023-11-28T04:24:34.745289+0000","last_became_peered":"2023-11-28T04:24:34.745289+0000","last_unstale":"2023-11-28T04:24:34.747547+0000","last_undegraded":"2023-11-28T04:24:34.747547+0000","last_fullsized":"2023-11-28T04:24:34.747547+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114486+0000","last_change":"2023-11-28T04:24:35.114486+0000","last_active":"2023-11-28T04:24:35.114486+0000","last_peered":"2023-11-28T04:24:35.114486+0000","last_clean":"2023-11-28T04:24:35.114486+0000","last_became_active":"2023-11-28T04:24:35.114148+0000","last_became_peered":"2023-11-28T04:24:35.114148+0000","last_unstale":"2023-11-28T04:24:35.114486+0000","last_undegraded":"2023-11-28T04:24:35.114486+0000","last_fullsized":"2023-11-28T04:24:35.114486+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.740820+0000","last_change":"2023-11-28T04:24:34.734570+0000","last_active":"2023-11-28T04:24:34.740820+0000","last_peered":"2023-11-28T04:24:34.740820+0000","last_clean":"2023-11-28T04:24:34.740820+0000","last_became_active":"2023-11-28T04:24:34.734260+0000","last_became_peered":"2023-11-28T04:24:34.734260+0000","last_unstale":"2023-11-28T04:24:34.740820+0000","last_undegraded":"2023-11-28T04:24:34.740820+0000","last_fullsized":"2023-11-28T04:24:34.740820+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114073+0000","last_change":"2023-11-28T04:24:35.114073+0000","last_active":"2023-11-28T04:24:35.114073+0000","last_peered":"2023-11-28T04:24:35.114073+0000","last_clean":"2023-11-28T04:24:35.114073+0000","last_became_active":"2023-11-28T04:24:35.113782+0000","last_became_peered":"2023-11-28T04:24:35.113782+0000","last_unstale":"2023-11-28T04:24:35.114073+0000","last_undegraded":"2023-11-28T04:24:35.114073+0000","last_fullsized":"2023-11-28T04:24:35.114073+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.34","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111142+0000","last_change":"2023-11-28T04:24:34.732938+0000","last_active":"2023-11-28T04:24:35.111142+0000","last_peered":"2023-11-28T04:24:35.111142+0000","last_clean":"2023-11-28T04:24:35.111142+0000","last_became_active":"2023-11-28T04:24:34.732634+0000","last_became_peered":"2023-11-28T04:24:34.732634+0000","last_unstale":"2023-11-28T04:24:35.111142+0000","last_undegraded":"2023-11-28T04:24:35.111142+0000","last_fullsized":"2023-11-28T04:24:35.111142+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.3c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.119088+0000","last_change":"2023-11-28T04:24:35.119088+0000","last_active":"2023-11-28T04:24:35.119088+0000","last_peered":"2023-11-28T04:24:35.119088+0000","last_clean":"2023-11-28T04:24:35.119088+0000","last_became_active":"2023-11-28T04:24:35.118793+0000","last_became_peered":"2023-11-28T04:24:35.118793+0000","last_unstale":"2023-11-28T04:24:35.119088+0000","last_undegraded":"2023-11-28T04:24:35.119088+0000","last_fullsized":"2023-11-28T04:24:35.119088+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.33","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095572+0000","last_change":"2023-11-28T04:24:34.744404+0000","last_active":"2023-11-28T04:24:35.095572+0000","last_peered":"2023-11-28T04:24:35.095572+0000","last_clean":"2023-11-28T04:24:35.095572+0000","last_became_active":"2023-11-28T04:24:34.744092+0000","last_became_peered":"2023-11-28T04:24:34.744092+0000","last_unstale":"2023-11-28T04:24:35.095572+0000","last_undegraded":"2023-11-28T04:24:35.095572+0000","last_fullsized":"2023-11-28T04:24:35.095572+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118241+0000","last_change":"2023-11-28T04:24:35.118241+0000","last_active":"2023-11-28T04:24:35.118241+0000","last_peered":"2023-11-28T04:24:35.118241+0000","last_clean":"2023-11-28T04:24:35.118241+0000","last_became_active":"2023-11-28T04:24:35.117560+0000","last_became_peered":"2023-11-28T04:24:35.117560+0000","last_unstale":"2023-11-28T04:24:35.118241+0000","last_undegraded":"2023-11-28T04:24:35.118241+0000","last_fullsized":"2023-11-28T04:24:35.118241+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742088+0000","last_change":"2023-11-28T04:24:34.738138+0000","last_active":"2023-11-28T04:24:34.742088+0000","last_peered":"2023-11-28T04:24:34.742088+0000","last_clean":"2023-11-28T04:24:34.742088+0000","last_became_active":"2023-11-28T04:24:34.737851+0000","last_became_peered":"2023-11-28T04:24:34.737851+0000","last_unstale":"2023-11-28T04:24:34.742088+0000","last_undegraded":"2023-11-28T04:24:34.742088+0000","last_fullsized":"2023-11-28T04:24:34.742088+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.3e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112108+0000","last_change":"2023-11-28T04:24:35.112108+0000","last_active":"2023-11-28T04:24:35.112108+0000","last_peered":"2023-11-28T04:24:35.112108+0000","last_clean":"2023-11-28T04:24:35.112108+0000","last_became_active":"2023-11-28T04:24:35.111823+0000","last_became_peered":"2023-11-28T04:24:35.111823+0000","last_unstale":"2023-11-28T04:24:35.112108+0000","last_undegraded":"2023-11-28T04:24:35.112108+0000","last_fullsized":"2023-11-28T04:24:35.112108+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750016+0000","last_change":"2023-11-28T04:24:34.743775+0000","last_active":"2023-11-28T04:24:34.750016+0000","last_peered":"2023-11-28T04:24:34.750016+0000","last_clean":"2023-11-28T04:24:34.750016+0000","last_became_active":"2023-11-28T04:24:34.743498+0000","last_became_peered":"2023-11-28T04:24:34.743498+0000","last_unstale":"2023-11-28T04:24:34.750016+0000","last_undegraded":"2023-11-28T04:24:34.750016+0000","last_fullsized":"2023-11-28T04:24:34.750016+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113413+0000","last_change":"2023-11-28T04:24:35.113413+0000","last_active":"2023-11-28T04:24:35.113413+0000","last_peered":"2023-11-28T04:24:35.113413+0000","last_clean":"2023-11-28T04:24:35.113413+0000","last_became_active":"2023-11-28T04:24:35.113070+0000","last_became_peered":"2023-11-28T04:24:35.113070+0000","last_unstale":"2023-11-28T04:24:35.113413+0000","last_undegraded":"2023-11-28T04:24:35.113413+0000","last_fullsized":"2023-11-28T04:24:35.113413+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.30","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109889+0000","last_change":"2023-11-28T04:24:34.736610+0000","last_active":"2023-11-28T04:24:35.109889+0000","last_peered":"2023-11-28T04:24:35.109889+0000","last_clean":"2023-11-28T04:24:35.109889+0000","last_became_active":"2023-11-28T04:24:34.736347+0000","last_became_peered":"2023-11-28T04:24:34.736347+0000","last_unstale":"2023-11-28T04:24:35.109889+0000","last_undegraded":"2023-11-28T04:24:35.109889+0000","last_fullsized":"2023-11-28T04:24:35.109889+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.20","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107936+0000","last_change":"2023-11-28T04:24:35.107936+0000","last_active":"2023-11-28T04:24:35.107936+0000","last_peered":"2023-11-28T04:24:35.107936+0000","last_clean":"2023-11-28T04:24:35.107936+0000","last_became_active":"2023-11-28T04:24:35.107676+0000","last_became_peered":"2023-11-28T04:24:35.107676+0000","last_unstale":"2023-11-28T04:24:35.107936+0000","last_undegraded":"2023-11-28T04:24:35.107936+0000","last_fullsized":"2023-11-28T04:24:35.107936+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742243+0000","last_change":"2023-11-28T04:24:34.740319+0000","last_active":"2023-11-28T04:24:34.742243+0000","last_peered":"2023-11-28T04:24:34.742243+0000","last_clean":"2023-11-28T04:24:34.742243+0000","last_became_active":"2023-11-28T04:24:34.740049+0000","last_became_peered":"2023-11-28T04:24:34.740049+0000","last_unstale":"2023-11-28T04:24:34.742243+0000","last_undegraded":"2023-11-28T04:24:34.742243+0000","last_fullsized":"2023-11-28T04:24:34.742243+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.21","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.108591+0000","last_change":"2023-11-28T04:24:35.108591+0000","last_active":"2023-11-28T04:24:35.108591+0000","last_peered":"2023-11-28T04:24:35.108591+0000","last_clean":"2023-11-28T04:24:35.108591+0000","last_became_active":"2023-11-28T04:24:35.108300+0000","last_became_peered":"2023-11-28T04:24:35.108300+0000","last_unstale":"2023-11-28T04:24:35.108591+0000","last_undegraded":"2023-11-28T04:24:35.108591+0000","last_fullsized":"2023-11-28T04:24:35.108591+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748235+0000","last_change":"2023-11-28T04:24:34.743482+0000","last_active":"2023-11-28T04:24:34.748235+0000","last_peered":"2023-11-28T04:24:34.748235+0000","last_clean":"2023-11-28T04:24:34.748235+0000","last_became_active":"2023-11-28T04:24:34.743228+0000","last_became_peered":"2023-11-28T04:24:34.743228+0000","last_unstale":"2023-11-28T04:24:34.748235+0000","last_undegraded":"2023-11-28T04:24:34.748235+0000","last_fullsized":"2023-11-28T04:24:34.748235+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.22","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101830+0000","last_change":"2023-11-28T04:24:35.101830+0000","last_active":"2023-11-28T04:24:35.101830+0000","last_peered":"2023-11-28T04:24:35.101830+0000","last_clean":"2023-11-28T04:24:35.101830+0000","last_became_active":"2023-11-28T04:24:35.101565+0000","last_became_peered":"2023-11-28T04:24:35.101565+0000","last_unstale":"2023-11-28T04:24:35.101830+0000","last_undegraded":"2023-11-28T04:24:35.101830+0000","last_fullsized":"2023-11-28T04:24:35.101830+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2d","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095148+0000","last_change":"2023-11-28T04:24:34.735844+0000","last_active":"2023-11-28T04:24:35.095148+0000","last_peered":"2023-11-28T04:24:35.095148+0000","last_clean":"2023-11-28T04:24:35.095148+0000","last_became_active":"2023-11-28T04:24:34.735554+0000","last_became_peered":"2023-11-28T04:24:34.735554+0000","last_unstale":"2023-11-28T04:24:35.095148+0000","last_undegraded":"2023-11-28T04:24:35.095148+0000","last_fullsized":"2023-11-28T04:24:35.095148+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.23","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110387+0000","last_change":"2023-11-28T04:24:35.110387+0000","last_active":"2023-11-28T04:24:35.110387+0000","last_peered":"2023-11-28T04:24:35.110387+0000","last_clean":"2023-11-28T04:24:35.110387+0000","last_became_active":"2023-11-28T04:24:35.110070+0000","last_became_peered":"2023-11-28T04:24:35.110070+0000","last_unstale":"2023-11-28T04:24:35.110387+0000","last_undegraded":"2023-11-28T04:24:35.110387+0000","last_fullsized":"2023-11-28T04:24:35.110387+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742069+0000","last_change":"2023-11-28T04:24:34.737473+0000","last_active":"2023-11-28T04:24:34.742069+0000","last_peered":"2023-11-28T04:24:34.742069+0000","last_clean":"2023-11-28T04:24:34.742069+0000","last_became_active":"2023-11-28T04:24:34.737237+0000","last_became_peered":"2023-11-28T04:24:34.737237+0000","last_unstale":"2023-11-28T04:24:34.742069+0000","last_undegraded":"2023-11-28T04:24:34.742069+0000","last_fullsized":"2023-11-28T04:24:34.742069+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.24","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116869+0000","last_change":"2023-11-28T04:24:35.116869+0000","last_active":"2023-11-28T04:24:35.116869+0000","last_peered":"2023-11-28T04:24:35.116869+0000","last_clean":"2023-11-28T04:24:35.116869+0000","last_became_active":"2023-11-28T04:24:35.116288+0000","last_became_peered":"2023-11-28T04:24:35.116288+0000","last_unstale":"2023-11-28T04:24:35.116869+0000","last_undegraded":"2023-11-28T04:24:35.116869+0000","last_fullsized":"2023-11-28T04:24:35.116869+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.093920+0000","last_change":"2023-11-28T04:24:34.740392+0000","last_active":"2023-11-28T04:24:35.093920+0000","last_peered":"2023-11-28T04:24:35.093920+0000","last_clean":"2023-11-28T04:24:35.093920+0000","last_became_active":"2023-11-28T04:24:34.740126+0000","last_became_peered":"2023-11-28T04:24:34.740126+0000","last_unstale":"2023-11-28T04:24:35.093920+0000","last_undegraded":"2023-11-28T04:24:35.093920+0000","last_fullsized":"2023-11-28T04:24:35.093920+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.25","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109567+0000","last_change":"2023-11-28T04:24:35.109567+0000","last_active":"2023-11-28T04:24:35.109567+0000","last_peered":"2023-11-28T04:24:35.109567+0000","last_clean":"2023-11-28T04:24:35.109567+0000","last_became_active":"2023-11-28T04:24:35.109194+0000","last_became_peered":"2023-11-28T04:24:35.109194+0000","last_unstale":"2023-11-28T04:24:35.109567+0000","last_undegraded":"2023-11-28T04:24:35.109567+0000","last_fullsized":"2023-11-28T04:24:35.109567+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741732+0000","last_change":"2023-11-28T04:24:34.738279+0000","last_active":"2023-11-28T04:24:34.741732+0000","last_peered":"2023-11-28T04:24:34.741732+0000","last_clean":"2023-11-28T04:24:34.741732+0000","last_became_active":"2023-11-28T04:24:34.737992+0000","last_became_peered":"2023-11-28T04:24:34.737992+0000","last_unstale":"2023-11-28T04:24:34.741732+0000","last_undegraded":"2023-11-28T04:24:34.741732+0000","last_fullsized":"2023-11-28T04:24:34.741732+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.26","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.111319+0000","last_change":"2023-11-28T04:24:35.111319+0000","last_active":"2023-11-28T04:24:35.111319+0000","last_peered":"2023-11-28T04:24:35.111319+0000","last_clean":"2023-11-28T04:24:35.111319+0000","last_became_active":"2023-11-28T04:24:35.111064+0000","last_became_peered":"2023-11-28T04:24:35.111064+0000","last_unstale":"2023-11-28T04:24:35.111319+0000","last_undegraded":"2023-11-28T04:24:35.111319+0000","last_fullsized":"2023-11-28T04:24:35.111319+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741586+0000","last_change":"2023-11-28T04:24:34.732186+0000","last_active":"2023-11-28T04:24:34.741586+0000","last_peered":"2023-11-28T04:24:34.741586+0000","last_clean":"2023-11-28T04:24:34.741586+0000","last_became_active":"2023-11-28T04:24:34.731975+0000","last_became_peered":"2023-11-28T04:24:34.731975+0000","last_unstale":"2023-11-28T04:24:34.741586+0000","last_undegraded":"2023-11-28T04:24:34.741586+0000","last_fullsized":"2023-11-28T04:24:34.741586+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.27","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106021+0000","last_change":"2023-11-28T04:24:35.106021+0000","last_active":"2023-11-28T04:24:35.106021+0000","last_peered":"2023-11-28T04:24:35.106021+0000","last_clean":"2023-11-28T04:24:35.106021+0000","last_became_active":"2023-11-28T04:24:35.105753+0000","last_became_peered":"2023-11-28T04:24:35.105753+0000","last_unstale":"2023-11-28T04:24:35.106021+0000","last_undegraded":"2023-11-28T04:24:35.106021+0000","last_fullsized":"2023-11-28T04:24:35.106021+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741874+0000","last_change":"2023-11-28T04:24:34.740479+0000","last_active":"2023-11-28T04:24:34.741874+0000","last_peered":"2023-11-28T04:24:34.741874+0000","last_clean":"2023-11-28T04:24:34.741874+0000","last_became_active":"2023-11-28T04:24:34.740053+0000","last_became_peered":"2023-11-28T04:24:34.740053+0000","last_unstale":"2023-11-28T04:24:34.741874+0000","last_undegraded":"2023-11-28T04:24:34.741874+0000","last_fullsized":"2023-11-28T04:24:34.741874+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.28","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118118+0000","last_change":"2023-11-28T04:24:35.118118+0000","last_active":"2023-11-28T04:24:35.118118+0000","last_peered":"2023-11-28T04:24:35.118118+0000","last_clean":"2023-11-28T04:24:35.118118+0000","last_became_active":"2023-11-28T04:24:35.117779+0000","last_became_peered":"2023-11-28T04:24:35.117779+0000","last_unstale":"2023-11-28T04:24:35.118118+0000","last_undegraded":"2023-11-28T04:24:35.118118+0000","last_fullsized":"2023-11-28T04:24:35.118118+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750228+0000","last_change":"2023-11-28T04:24:34.746015+0000","last_active":"2023-11-28T04:24:34.750228+0000","last_peered":"2023-11-28T04:24:34.750228+0000","last_clean":"2023-11-28T04:24:34.750228+0000","last_became_active":"2023-11-28T04:24:34.745687+0000","last_became_peered":"2023-11-28T04:24:34.745687+0000","last_unstale":"2023-11-28T04:24:34.750228+0000","last_undegraded":"2023-11-28T04:24:34.750228+0000","last_fullsized":"2023-11-28T04:24:34.750228+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.29","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105824+0000","last_change":"2023-11-28T04:24:35.105824+0000","last_active":"2023-11-28T04:24:35.105824+0000","last_peered":"2023-11-28T04:24:35.105824+0000","last_clean":"2023-11-28T04:24:35.105824+0000","last_became_active":"2023-11-28T04:24:35.105582+0000","last_became_peered":"2023-11-28T04:24:35.105582+0000","last_unstale":"2023-11-28T04:24:35.105824+0000","last_undegraded":"2023-11-28T04:24:35.105824+0000","last_fullsized":"2023-11-28T04:24:35.105824+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750120+0000","last_change":"2023-11-28T04:24:34.745797+0000","last_active":"2023-11-28T04:24:34.750120+0000","last_peered":"2023-11-28T04:24:34.750120+0000","last_clean":"2023-11-28T04:24:34.750120+0000","last_became_active":"2023-11-28T04:24:34.745498+0000","last_became_peered":"2023-11-28T04:24:34.745498+0000","last_unstale":"2023-11-28T04:24:34.750120+0000","last_undegraded":"2023-11-28T04:24:34.750120+0000","last_fullsized":"2023-11-28T04:24:34.750120+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.2a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110230+0000","last_change":"2023-11-28T04:24:35.110230+0000","last_active":"2023-11-28T04:24:35.110230+0000","last_peered":"2023-11-28T04:24:35.110230+0000","last_clean":"2023-11-28T04:24:35.110230+0000","last_became_active":"2023-11-28T04:24:35.109391+0000","last_became_peered":"2023-11-28T04:24:35.109391+0000","last_unstale":"2023-11-28T04:24:35.110230+0000","last_undegraded":"2023-11-28T04:24:35.110230+0000","last_fullsized":"2023-11-28T04:24:35.110230+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750252+0000","last_change":"2023-11-28T04:24:34.746207+0000","last_active":"2023-11-28T04:24:34.750252+0000","last_peered":"2023-11-28T04:24:34.750252+0000","last_clean":"2023-11-28T04:24:34.750252+0000","last_became_active":"2023-11-28T04:24:34.745767+0000","last_became_peered":"2023-11-28T04:24:34.745767+0000","last_unstale":"2023-11-28T04:24:34.750252+0000","last_undegraded":"2023-11-28T04:24:34.750252+0000","last_fullsized":"2023-11-28T04:24:34.750252+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116341+0000","last_change":"2023-11-28T04:24:35.116341+0000","last_active":"2023-11-28T04:24:35.116341+0000","last_peered":"2023-11-28T04:24:35.116341+0000","last_clean":"2023-11-28T04:24:35.116341+0000","last_became_active":"2023-11-28T04:24:35.116103+0000","last_became_peered":"2023-11-28T04:24:35.116103+0000","last_unstale":"2023-11-28T04:24:35.116341+0000","last_undegraded":"2023-11-28T04:24:35.116341+0000","last_fullsized":"2023-11-28T04:24:35.116341+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749339+0000","last_change":"2023-11-28T04:24:34.741276+0000","last_active":"2023-11-28T04:24:34.749339+0000","last_peered":"2023-11-28T04:24:34.749339+0000","last_clean":"2023-11-28T04:24:34.749339+0000","last_became_active":"2023-11-28T04:24:34.740833+0000","last_became_peered":"2023-11-28T04:24:34.740833+0000","last_unstale":"2023-11-28T04:24:34.749339+0000","last_undegraded":"2023-11-28T04:24:34.749339+0000","last_fullsized":"2023-11-28T04:24:34.749339+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104996+0000","last_change":"2023-11-28T04:24:35.104996+0000","last_active":"2023-11-28T04:24:35.104996+0000","last_peered":"2023-11-28T04:24:35.104996+0000","last_clean":"2023-11-28T04:24:35.104996+0000","last_became_active":"2023-11-28T04:24:35.104732+0000","last_became_peered":"2023-11-28T04:24:35.104732+0000","last_unstale":"2023-11-28T04:24:35.104996+0000","last_undegraded":"2023-11-28T04:24:35.104996+0000","last_fullsized":"2023-11-28T04:24:35.104996+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750214+0000","last_change":"2023-11-28T04:24:34.746012+0000","last_active":"2023-11-28T04:24:34.750214+0000","last_peered":"2023-11-28T04:24:34.750214+0000","last_clean":"2023-11-28T04:24:34.750214+0000","last_became_active":"2023-11-28T04:24:34.745502+0000","last_became_peered":"2023-11-28T04:24:34.745502+0000","last_unstale":"2023-11-28T04:24:34.750214+0000","last_undegraded":"2023-11-28T04:24:34.750214+0000","last_fullsized":"2023-11-28T04:24:34.750214+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110468+0000","last_change":"2023-11-28T04:24:35.110468+0000","last_active":"2023-11-28T04:24:35.110468+0000","last_peered":"2023-11-28T04:24:35.110468+0000","last_clean":"2023-11-28T04:24:35.110468+0000","last_became_active":"2023-11-28T04:24:35.110189+0000","last_became_peered":"2023-11-28T04:24:35.110189+0000","last_unstale":"2023-11-28T04:24:35.110468+0000","last_undegraded":"2023-11-28T04:24:35.110468+0000","last_fullsized":"2023-11-28T04:24:35.110468+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747569+0000","last_change":"2023-11-28T04:24:34.743048+0000","last_active":"2023-11-28T04:24:34.747569+0000","last_peered":"2023-11-28T04:24:34.747569+0000","last_clean":"2023-11-28T04:24:34.747569+0000","last_became_active":"2023-11-28T04:24:34.742757+0000","last_became_peered":"2023-11-28T04:24:34.742757+0000","last_unstale":"2023-11-28T04:24:34.747569+0000","last_undegraded":"2023-11-28T04:24:34.747569+0000","last_fullsized":"2023-11-28T04:24:34.747569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106921+0000","last_change":"2023-11-28T04:24:35.106921+0000","last_active":"2023-11-28T04:24:35.106921+0000","last_peered":"2023-11-28T04:24:35.106921+0000","last_clean":"2023-11-28T04:24:35.106921+0000","last_became_active":"2023-11-28T04:24:35.106659+0000","last_became_peered":"2023-11-28T04:24:35.106659+0000","last_unstale":"2023-11-28T04:24:35.106921+0000","last_undegraded":"2023-11-28T04:24:35.106921+0000","last_fullsized":"2023-11-28T04:24:35.106921+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750129+0000","last_change":"2023-11-28T04:24:34.746184+0000","last_active":"2023-11-28T04:24:34.750129+0000","last_peered":"2023-11-28T04:24:34.750129+0000","last_clean":"2023-11-28T04:24:34.750129+0000","last_became_active":"2023-11-28T04:24:34.745948+0000","last_became_peered":"2023-11-28T04:24:34.745948+0000","last_unstale":"2023-11-28T04:24:34.750129+0000","last_undegraded":"2023-11-28T04:24:34.750129+0000","last_fullsized":"2023-11-28T04:24:34.750129+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.2f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107383+0000","last_change":"2023-11-28T04:24:35.107383+0000","last_active":"2023-11-28T04:24:35.107383+0000","last_peered":"2023-11-28T04:24:35.107383+0000","last_clean":"2023-11-28T04:24:35.107383+0000","last_became_active":"2023-11-28T04:24:35.107114+0000","last_became_peered":"2023-11-28T04:24:35.107114+0000","last_unstale":"2023-11-28T04:24:35.107383+0000","last_undegraded":"2023-11-28T04:24:35.107383+0000","last_fullsized":"2023-11-28T04:24:35.107383+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748141+0000","last_change":"2023-11-28T04:24:34.743069+0000","last_active":"2023-11-28T04:24:34.748141+0000","last_peered":"2023-11-28T04:24:34.748141+0000","last_clean":"2023-11-28T04:24:34.748141+0000","last_became_active":"2023-11-28T04:24:34.742766+0000","last_became_peered":"2023-11-28T04:24:34.742766+0000","last_unstale":"2023-11-28T04:24:34.748141+0000","last_undegraded":"2023-11-28T04:24:34.748141+0000","last_fullsized":"2023-11-28T04:24:34.748141+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118811+0000","last_change":"2023-11-28T04:24:35.118811+0000","last_active":"2023-11-28T04:24:35.118811+0000","last_peered":"2023-11-28T04:24:35.118811+0000","last_clean":"2023-11-28T04:24:35.118811+0000","last_became_active":"2023-11-28T04:24:35.118070+0000","last_became_peered":"2023-11-28T04:24:35.118070+0000","last_unstale":"2023-11-28T04:24:35.118811+0000","last_undegraded":"2023-11-28T04:24:35.118811+0000","last_fullsized":"2023-11-28T04:24:35.118811+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106962+0000","last_change":"2023-11-28T04:24:34.743724+0000","last_active":"2023-11-28T04:24:35.106962+0000","last_peered":"2023-11-28T04:24:35.106962+0000","last_clean":"2023-11-28T04:24:35.106962+0000","last_became_active":"2023-11-28T04:24:34.743430+0000","last_became_peered":"2023-11-28T04:24:34.743430+0000","last_unstale":"2023-11-28T04:24:35.106962+0000","last_undegraded":"2023-11-28T04:24:35.106962+0000","last_fullsized":"2023-11-28T04:24:35.106962+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101773+0000","last_change":"2023-11-28T04:24:35.101773+0000","last_active":"2023-11-28T04:24:35.101773+0000","last_peered":"2023-11-28T04:24:35.101773+0000","last_clean":"2023-11-28T04:24:35.101773+0000","last_became_active":"2023-11-28T04:24:35.101503+0000","last_became_peered":"2023-11-28T04:24:35.101503+0000","last_unstale":"2023-11-28T04:24:35.101773+0000","last_undegraded":"2023-11-28T04:24:35.101773+0000","last_fullsized":"2023-11-28T04:24:35.101773+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.095928+0000","last_change":"2023-11-28T04:24:34.746192+0000","last_active":"2023-11-28T04:24:35.095928+0000","last_peered":"2023-11-28T04:24:35.095928+0000","last_clean":"2023-11-28T04:24:35.095928+0000","last_became_active":"2023-11-28T04:24:34.745885+0000","last_became_peered":"2023-11-28T04:24:34.745885+0000","last_unstale":"2023-11-28T04:24:35.095928+0000","last_undegraded":"2023-11-28T04:24:35.095928+0000","last_fullsized":"2023-11-28T04:24:35.095928+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.101062+0000","last_change":"2023-11-28T04:24:35.101062+0000","last_active":"2023-11-28T04:24:35.101062+0000","last_peered":"2023-11-28T04:24:35.101062+0000","last_clean":"2023-11-28T04:24:35.101062+0000","last_became_active":"2023-11-28T04:24:35.100534+0000","last_became_peered":"2023-11-28T04:24:35.100534+0000","last_unstale":"2023-11-28T04:24:35.101062+0000","last_undegraded":"2023-11-28T04:24:35.101062+0000","last_fullsized":"2023-11-28T04:24:35.101062+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1d","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.108820+0000","last_change":"2023-11-28T04:24:34.743046+0000","last_active":"2023-11-28T04:24:35.108820+0000","last_peered":"2023-11-28T04:24:35.108820+0000","last_clean":"2023-11-28T04:24:35.108820+0000","last_became_active":"2023-11-28T04:24:34.742717+0000","last_became_peered":"2023-11-28T04:24:34.742717+0000","last_unstale":"2023-11-28T04:24:35.108820+0000","last_undegraded":"2023-11-28T04:24:35.108820+0000","last_fullsized":"2023-11-28T04:24:35.108820+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.13","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116507+0000","last_change":"2023-11-28T04:24:35.116507+0000","last_active":"2023-11-28T04:24:35.116507+0000","last_peered":"2023-11-28T04:24:35.116507+0000","last_clean":"2023-11-28T04:24:35.116507+0000","last_became_active":"2023-11-28T04:24:35.116187+0000","last_became_peered":"2023-11-28T04:24:35.116187+0000","last_unstale":"2023-11-28T04:24:35.116507+0000","last_undegraded":"2023-11-28T04:24:35.116507+0000","last_fullsized":"2023-11-28T04:24:35.116507+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742189+0000","last_change":"2023-11-28T04:24:34.739740+0000","last_active":"2023-11-28T04:24:34.742189+0000","last_peered":"2023-11-28T04:24:34.742189+0000","last_clean":"2023-11-28T04:24:34.742189+0000","last_became_active":"2023-11-28T04:24:34.739431+0000","last_became_peered":"2023-11-28T04:24:34.739431+0000","last_unstale":"2023-11-28T04:24:34.742189+0000","last_undegraded":"2023-11-28T04:24:34.742189+0000","last_fullsized":"2023-11-28T04:24:34.742189+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113793+0000","last_change":"2023-11-28T04:24:35.113793+0000","last_active":"2023-11-28T04:24:35.113793+0000","last_peered":"2023-11-28T04:24:35.113793+0000","last_clean":"2023-11-28T04:24:35.113793+0000","last_became_active":"2023-11-28T04:24:35.113501+0000","last_became_peered":"2023-11-28T04:24:35.113501+0000","last_unstale":"2023-11-28T04:24:35.113793+0000","last_undegraded":"2023-11-28T04:24:35.113793+0000","last_fullsized":"2023-11-28T04:24:35.113793+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749370+0000","last_change":"2023-11-28T04:24:34.743481+0000","last_active":"2023-11-28T04:24:34.749370+0000","last_peered":"2023-11-28T04:24:34.749370+0000","last_clean":"2023-11-28T04:24:34.749370+0000","last_became_active":"2023-11-28T04:24:34.743247+0000","last_became_peered":"2023-11-28T04:24:34.743247+0000","last_unstale":"2023-11-28T04:24:34.749370+0000","last_undegraded":"2023-11-28T04:24:34.749370+0000","last_fullsized":"2023-11-28T04:24:34.749370+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.15","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.102799+0000","last_change":"2023-11-28T04:24:35.102799+0000","last_active":"2023-11-28T04:24:35.102799+0000","last_peered":"2023-11-28T04:24:35.102799+0000","last_clean":"2023-11-28T04:24:35.102799+0000","last_became_active":"2023-11-28T04:24:35.102354+0000","last_became_peered":"2023-11-28T04:24:35.102354+0000","last_unstale":"2023-11-28T04:24:35.102799+0000","last_undegraded":"2023-11-28T04:24:35.102799+0000","last_fullsized":"2023-11-28T04:24:35.102799+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742530+0000","last_change":"2023-11-28T04:24:34.741653+0000","last_active":"2023-11-28T04:24:34.742530+0000","last_peered":"2023-11-28T04:24:34.742530+0000","last_clean":"2023-11-28T04:24:34.742530+0000","last_became_active":"2023-11-28T04:24:34.741389+0000","last_became_peered":"2023-11-28T04:24:34.741389+0000","last_unstale":"2023-11-28T04:24:34.742530+0000","last_undegraded":"2023-11-28T04:24:34.742530+0000","last_fullsized":"2023-11-28T04:24:34.742530+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103440+0000","last_change":"2023-11-28T04:24:35.103440+0000","last_active":"2023-11-28T04:24:35.103440+0000","last_peered":"2023-11-28T04:24:35.103440+0000","last_clean":"2023-11-28T04:24:35.103440+0000","last_became_active":"2023-11-28T04:24:35.103198+0000","last_became_peered":"2023-11-28T04:24:35.103198+0000","last_unstale":"2023-11-28T04:24:35.103440+0000","last_undegraded":"2023-11-28T04:24:35.103440+0000","last_fullsized":"2023-11-28T04:24:35.103440+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742097+0000","last_change":"2023-11-28T04:24:34.738490+0000","last_active":"2023-11-28T04:24:34.742097+0000","last_peered":"2023-11-28T04:24:34.742097+0000","last_clean":"2023-11-28T04:24:34.742097+0000","last_became_active":"2023-11-28T04:24:34.738225+0000","last_became_peered":"2023-11-28T04:24:34.738225+0000","last_unstale":"2023-11-28T04:24:34.742097+0000","last_undegraded":"2023-11-28T04:24:34.742097+0000","last_fullsized":"2023-11-28T04:24:34.742097+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115885+0000","last_change":"2023-11-28T04:24:35.115885+0000","last_active":"2023-11-28T04:24:35.115885+0000","last_peered":"2023-11-28T04:24:35.115885+0000","last_clean":"2023-11-28T04:24:35.115885+0000","last_became_active":"2023-11-28T04:24:35.115572+0000","last_became_peered":"2023-11-28T04:24:35.115572+0000","last_unstale":"2023-11-28T04:24:35.115885+0000","last_undegraded":"2023-11-28T04:24:35.115885+0000","last_fullsized":"2023-11-28T04:24:35.115885+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.745931+0000","last_change":"2023-11-28T04:24:34.738309+0000","last_active":"2023-11-28T04:24:34.745931+0000","last_peered":"2023-11-28T04:24:34.745931+0000","last_clean":"2023-11-28T04:24:34.745931+0000","last_became_active":"2023-11-28T04:24:34.737352+0000","last_became_peered":"2023-11-28T04:24:34.737352+0000","last_unstale":"2023-11-28T04:24:34.745931+0000","last_undegraded":"2023-11-28T04:24:34.745931+0000","last_fullsized":"2023-11-28T04:24:34.745931+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107176+0000","last_change":"2023-11-28T04:24:35.107176+0000","last_active":"2023-11-28T04:24:35.107176+0000","last_peered":"2023-11-28T04:24:35.107176+0000","last_clean":"2023-11-28T04:24:35.107176+0000","last_became_active":"2023-11-28T04:24:35.106878+0000","last_became_peered":"2023-11-28T04:24:35.106878+0000","last_unstale":"2023-11-28T04:24:35.107176+0000","last_undegraded":"2023-11-28T04:24:35.107176+0000","last_fullsized":"2023-11-28T04:24:35.107176+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.091495+0000","last_change":"2023-11-28T04:24:34.745386+0000","last_active":"2023-11-28T04:24:35.091495+0000","last_peered":"2023-11-28T04:24:35.091495+0000","last_clean":"2023-11-28T04:24:35.091495+0000","last_became_active":"2023-11-28T04:24:34.745123+0000","last_became_peered":"2023-11-28T04:24:34.745123+0000","last_unstale":"2023-11-28T04:24:35.091495+0000","last_undegraded":"2023-11-28T04:24:35.091495+0000","last_fullsized":"2023-11-28T04:24:35.091495+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.100696+0000","last_change":"2023-11-28T04:24:35.100696+0000","last_active":"2023-11-28T04:24:35.100696+0000","last_peered":"2023-11-28T04:24:35.100696+0000","last_clean":"2023-11-28T04:24:35.100696+0000","last_became_active":"2023-11-28T04:24:35.100370+0000","last_became_peered":"2023-11-28T04:24:35.100370+0000","last_unstale":"2023-11-28T04:24:35.100696+0000","last_undegraded":"2023-11-28T04:24:35.100696+0000","last_fullsized":"2023-11-28T04:24:35.100696+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.e","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.117427+0000","last_change":"2023-11-28T04:24:34.741248+0000","last_active":"2023-11-28T04:24:35.117427+0000","last_peered":"2023-11-28T04:24:35.117427+0000","last_clean":"2023-11-28T04:24:35.117427+0000","last_became_active":"2023-11-28T04:24:34.740942+0000","last_became_peered":"2023-11-28T04:24:34.740942+0000","last_unstale":"2023-11-28T04:24:35.117427+0000","last_undegraded":"2023-11-28T04:24:35.117427+0000","last_fullsized":"2023-11-28T04:24:35.117427+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.112261+0000","last_change":"2023-11-28T04:24:35.112261+0000","last_active":"2023-11-28T04:24:35.112261+0000","last_peered":"2023-11-28T04:24:35.112261+0000","last_clean":"2023-11-28T04:24:35.112261+0000","last_became_active":"2023-11-28T04:24:35.111924+0000","last_became_peered":"2023-11-28T04:24:35.111924+0000","last_unstale":"2023-11-28T04:24:35.112261+0000","last_undegraded":"2023-11-28T04:24:35.112261+0000","last_fullsized":"2023-11-28T04:24:35.112261+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742315+0000","last_change":"2023-11-28T04:24:34.735718+0000","last_active":"2023-11-28T04:24:34.742315+0000","last_peered":"2023-11-28T04:24:34.742315+0000","last_clean":"2023-11-28T04:24:34.742315+0000","last_became_active":"2023-11-28T04:24:34.735416+0000","last_became_peered":"2023-11-28T04:24:34.735416+0000","last_unstale":"2023-11-28T04:24:34.742315+0000","last_undegraded":"2023-11-28T04:24:34.742315+0000","last_fullsized":"2023-11-28T04:24:34.742315+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105168+0000","last_change":"2023-11-28T04:24:35.105168+0000","last_active":"2023-11-28T04:24:35.105168+0000","last_peered":"2023-11-28T04:24:35.105168+0000","last_clean":"2023-11-28T04:24:35.105168+0000","last_became_active":"2023-11-28T04:24:35.104876+0000","last_became_peered":"2023-11-28T04:24:35.104876+0000","last_unstale":"2023-11-28T04:24:35.105168+0000","last_undegraded":"2023-11-28T04:24:35.105168+0000","last_fullsized":"2023-11-28T04:24:35.105168+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.749400+0000","last_change":"2023-11-28T04:24:34.744046+0000","last_active":"2023-11-28T04:24:34.749400+0000","last_peered":"2023-11-28T04:24:34.749400+0000","last_clean":"2023-11-28T04:24:34.749400+0000","last_became_active":"2023-11-28T04:24:34.743758+0000","last_became_peered":"2023-11-28T04:24:34.743758+0000","last_unstale":"2023-11-28T04:24:34.749400+0000","last_undegraded":"2023-11-28T04:24:34.749400+0000","last_fullsized":"2023-11-28T04:24:34.749400+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116918+0000","last_change":"2023-11-28T04:24:35.116918+0000","last_active":"2023-11-28T04:24:35.116918+0000","last_peered":"2023-11-28T04:24:35.116918+0000","last_clean":"2023-11-28T04:24:35.116918+0000","last_became_active":"2023-11-28T04:24:35.116627+0000","last_became_peered":"2023-11-28T04:24:35.116627+0000","last_unstale":"2023-11-28T04:24:35.116918+0000","last_undegraded":"2023-11-28T04:24:35.116918+0000","last_fullsized":"2023-11-28T04:24:35.116918+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.b","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.092975+0000","last_change":"2023-11-28T04:24:34.738909+0000","last_active":"2023-11-28T04:24:35.092975+0000","last_peered":"2023-11-28T04:24:35.092975+0000","last_clean":"2023-11-28T04:24:35.092975+0000","last_became_active":"2023-11-28T04:24:34.738673+0000","last_became_peered":"2023-11-28T04:24:34.738673+0000","last_unstale":"2023-11-28T04:24:35.092975+0000","last_undegraded":"2023-11-28T04:24:35.092975+0000","last_fullsized":"2023-11-28T04:24:35.092975+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104195+0000","last_change":"2023-11-28T04:24:35.104195+0000","last_active":"2023-11-28T04:24:35.104195+0000","last_peered":"2023-11-28T04:24:35.104195+0000","last_clean":"2023-11-28T04:24:35.104195+0000","last_became_active":"2023-11-28T04:24:35.103883+0000","last_became_peered":"2023-11-28T04:24:35.103883+0000","last_unstale":"2023-11-28T04:24:35.104195+0000","last_undegraded":"2023-11-28T04:24:35.104195+0000","last_fullsized":"2023-11-28T04:24:35.104195+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747501+0000","last_change":"2023-11-28T04:24:34.744419+0000","last_active":"2023-11-28T04:24:34.747501+0000","last_peered":"2023-11-28T04:24:34.747501+0000","last_clean":"2023-11-28T04:24:34.747501+0000","last_became_active":"2023-11-28T04:24:34.744146+0000","last_became_peered":"2023-11-28T04:24:34.744146+0000","last_unstale":"2023-11-28T04:24:34.747501+0000","last_undegraded":"2023-11-28T04:24:34.747501+0000","last_fullsized":"2023-11-28T04:24:34.747501+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107194+0000","last_change":"2023-11-28T04:24:35.107194+0000","last_active":"2023-11-28T04:24:35.107194+0000","last_peered":"2023-11-28T04:24:35.107194+0000","last_clean":"2023-11-28T04:24:35.107194+0000","last_became_active":"2023-11-28T04:24:35.106932+0000","last_became_peered":"2023-11-28T04:24:35.106932+0000","last_unstale":"2023-11-28T04:24:35.107194+0000","last_undegraded":"2023-11-28T04:24:35.107194+0000","last_fullsized":"2023-11-28T04:24:35.107194+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.741566+0000","last_change":"2023-11-28T04:24:34.732015+0000","last_active":"2023-11-28T04:24:34.741566+0000","last_peered":"2023-11-28T04:24:34.741566+0000","last_clean":"2023-11-28T04:24:34.741566+0000","last_became_active":"2023-11-28T04:24:34.731857+0000","last_became_peered":"2023-11-28T04:24:34.731857+0000","last_unstale":"2023-11-28T04:24:34.741566+0000","last_undegraded":"2023-11-28T04:24:34.741566+0000","last_fullsized":"2023-11-28T04:24:34.741566+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113482+0000","last_change":"2023-11-28T04:24:35.113482+0000","last_active":"2023-11-28T04:24:35.113482+0000","last_peered":"2023-11-28T04:24:35.113482+0000","last_clean":"2023-11-28T04:24:35.113482+0000","last_became_active":"2023-11-28T04:24:35.113209+0000","last_became_peered":"2023-11-28T04:24:35.113209+0000","last_unstale":"2023-11-28T04:24:35.113482+0000","last_undegraded":"2023-11-28T04:24:35.113482+0000","last_fullsized":"2023-11-28T04:24:35.113482+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.745112+0000","last_change":"2023-11-28T04:24:34.740814+0000","last_active":"2023-11-28T04:24:34.745112+0000","last_peered":"2023-11-28T04:24:34.745112+0000","last_clean":"2023-11-28T04:24:34.745112+0000","last_became_active":"2023-11-28T04:24:34.740557+0000","last_became_peered":"2023-11-28T04:24:34.740557+0000","last_unstale":"2023-11-28T04:24:34.745112+0000","last_undegraded":"2023-11-28T04:24:34.745112+0000","last_fullsized":"2023-11-28T04:24:34.745112+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103281+0000","last_change":"2023-11-28T04:24:35.103281+0000","last_active":"2023-11-28T04:24:35.103281+0000","last_peered":"2023-11-28T04:24:35.103281+0000","last_clean":"2023-11-28T04:24:35.103281+0000","last_became_active":"2023-11-28T04:24:35.103014+0000","last_became_peered":"2023-11-28T04:24:35.103014+0000","last_unstale":"2023-11-28T04:24:35.103281+0000","last_undegraded":"2023-11-28T04:24:35.103281+0000","last_fullsized":"2023-11-28T04:24:35.103281+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.7","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107906+0000","last_change":"2023-11-28T04:24:34.731547+0000","last_active":"2023-11-28T04:24:35.107906+0000","last_peered":"2023-11-28T04:24:35.107906+0000","last_clean":"2023-11-28T04:24:35.107906+0000","last_became_active":"2023-11-28T04:24:34.731254+0000","last_became_peered":"2023-11-28T04:24:34.731254+0000","last_unstale":"2023-11-28T04:24:35.107906+0000","last_undegraded":"2023-11-28T04:24:35.107906+0000","last_fullsized":"2023-11-28T04:24:35.107906+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.107573+0000","last_change":"2023-11-28T04:24:35.107573+0000","last_active":"2023-11-28T04:24:35.107573+0000","last_peered":"2023-11-28T04:24:35.107573+0000","last_clean":"2023-11-28T04:24:35.107573+0000","last_became_active":"2023-11-28T04:24:35.106948+0000","last_became_peered":"2023-11-28T04:24:35.106948+0000","last_unstale":"2023-11-28T04:24:35.107573+0000","last_undegraded":"2023-11-28T04:24:35.107573+0000","last_fullsized":"2023-11-28T04:24:35.107573+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750058+0000","last_change":"2023-11-28T04:24:34.745571+0000","last_active":"2023-11-28T04:24:34.750058+0000","last_peered":"2023-11-28T04:24:34.750058+0000","last_clean":"2023-11-28T04:24:34.750058+0000","last_became_active":"2023-11-28T04:24:34.745306+0000","last_became_peered":"2023-11-28T04:24:34.745306+0000","last_unstale":"2023-11-28T04:24:34.750058+0000","last_undegraded":"2023-11-28T04:24:34.750058+0000","last_fullsized":"2023-11-28T04:24:34.750058+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103341+0000","last_change":"2023-11-28T04:24:35.103341+0000","last_active":"2023-11-28T04:24:35.103341+0000","last_peered":"2023-11-28T04:24:35.103341+0000","last_clean":"2023-11-28T04:24:35.103341+0000","last_became_active":"2023-11-28T04:24:35.103063+0000","last_became_peered":"2023-11-28T04:24:35.103063+0000","last_unstale":"2023-11-28T04:24:35.103341+0000","last_undegraded":"2023-11-28T04:24:35.103341+0000","last_fullsized":"2023-11-28T04:24:35.103341+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742755+0000","last_change":"2023-11-28T04:24:34.738406+0000","last_active":"2023-11-28T04:24:34.742755+0000","last_peered":"2023-11-28T04:24:34.742755+0000","last_clean":"2023-11-28T04:24:34.742755+0000","last_became_active":"2023-11-28T04:24:34.738126+0000","last_became_peered":"2023-11-28T04:24:34.738126+0000","last_unstale":"2023-11-28T04:24:34.742755+0000","last_undegraded":"2023-11-28T04:24:34.742755+0000","last_fullsized":"2023-11-28T04:24:34.742755+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113570+0000","last_change":"2023-11-28T04:24:35.113570+0000","last_active":"2023-11-28T04:24:35.113570+0000","last_peered":"2023-11-28T04:24:35.113570+0000","last_clean":"2023-11-28T04:24:35.113570+0000","last_became_active":"2023-11-28T04:24:35.113321+0000","last_became_peered":"2023-11-28T04:24:35.113321+0000","last_unstale":"2023-11-28T04:24:35.113570+0000","last_undegraded":"2023-11-28T04:24:35.113570+0000","last_fullsized":"2023-11-28T04:24:35.113570+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750233+0000","last_change":"2023-11-28T04:24:34.748547+0000","last_active":"2023-11-28T04:24:34.750233+0000","last_peered":"2023-11-28T04:24:34.750233+0000","last_clean":"2023-11-28T04:24:34.750233+0000","last_became_active":"2023-11-28T04:24:34.748278+0000","last_became_peered":"2023-11-28T04:24:34.748278+0000","last_unstale":"2023-11-28T04:24:34.750233+0000","last_undegraded":"2023-11-28T04:24:34.750233+0000","last_fullsized":"2023-11-28T04:24:34.750233+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.113149+0000","last_change":"2023-11-28T04:24:35.113149+0000","last_active":"2023-11-28T04:24:35.113149+0000","last_peered":"2023-11-28T04:24:35.113149+0000","last_clean":"2023-11-28T04:24:35.113149+0000","last_became_active":"2023-11-28T04:24:35.112802+0000","last_became_peered":"2023-11-28T04:24:35.112802+0000","last_unstale":"2023-11-28T04:24:35.113149+0000","last_undegraded":"2023-11-28T04:24:35.113149+0000","last_fullsized":"2023-11-28T04:24:35.113149+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747558+0000","last_change":"2023-11-28T04:24:34.743755+0000","last_active":"2023-11-28T04:24:34.747558+0000","last_peered":"2023-11-28T04:24:34.747558+0000","last_clean":"2023-11-28T04:24:34.747558+0000","last_became_active":"2023-11-28T04:24:34.743485+0000","last_became_peered":"2023-11-28T04:24:34.743485+0000","last_unstale":"2023-11-28T04:24:34.747558+0000","last_undegraded":"2023-11-28T04:24:34.747558+0000","last_fullsized":"2023-11-28T04:24:34.747558+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.116495+0000","last_change":"2023-11-28T04:24:35.116495+0000","last_active":"2023-11-28T04:24:35.116495+0000","last_peered":"2023-11-28T04:24:35.116495+0000","last_clean":"2023-11-28T04:24:35.116495+0000","last_became_active":"2023-11-28T04:24:35.116203+0000","last_became_peered":"2023-11-28T04:24:35.116203+0000","last_unstale":"2023-11-28T04:24:35.116495+0000","last_undegraded":"2023-11-28T04:24:35.116495+0000","last_fullsized":"2023-11-28T04:24:35.116495+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.748207+0000","last_change":"2023-11-28T04:24:34.743493+0000","last_active":"2023-11-28T04:24:34.748207+0000","last_peered":"2023-11-28T04:24:34.748207+0000","last_clean":"2023-11-28T04:24:34.748207+0000","last_became_active":"2023-11-28T04:24:34.742776+0000","last_became_peered":"2023-11-28T04:24:34.742776+0000","last_unstale":"2023-11-28T04:24:34.748207+0000","last_undegraded":"2023-11-28T04:24:34.748207+0000","last_fullsized":"2023-11-28T04:24:34.748207+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115808+0000","last_change":"2023-11-28T04:24:35.115808+0000","last_active":"2023-11-28T04:24:35.115808+0000","last_peered":"2023-11-28T04:24:35.115808+0000","last_clean":"2023-11-28T04:24:35.115808+0000","last_became_active":"2023-11-28T04:24:35.115534+0000","last_became_peered":"2023-11-28T04:24:35.115534+0000","last_unstale":"2023-11-28T04:24:35.115808+0000","last_undegraded":"2023-11-28T04:24:35.115808+0000","last_fullsized":"2023-11-28T04:24:35.115808+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.747461+0000","last_change":"2023-11-28T04:24:34.743250+0000","last_active":"2023-11-28T04:24:34.747461+0000","last_peered":"2023-11-28T04:24:34.747461+0000","last_clean":"2023-11-28T04:24:34.747461+0000","last_became_active":"2023-11-28T04:24:34.742734+0000","last_became_peered":"2023-11-28T04:24:34.742734+0000","last_unstale":"2023-11-28T04:24:34.747461+0000","last_undegraded":"2023-11-28T04:24:34.747461+0000","last_fullsized":"2023-11-28T04:24:34.747461+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.105298+0000","last_change":"2023-11-28T04:24:35.105298+0000","last_active":"2023-11-28T04:24:35.105298+0000","last_peered":"2023-11-28T04:24:35.105298+0000","last_clean":"2023-11-28T04:24:35.105298+0000","last_became_active":"2023-11-28T04:24:35.105013+0000","last_became_peered":"2023-11-28T04:24:35.105013+0000","last_unstale":"2023-11-28T04:24:35.105298+0000","last_undegraded":"2023-11-28T04:24:35.105298+0000","last_fullsized":"2023-11-28T04:24:35.105298+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.742230+0000","last_change":"2023-11-28T04:24:34.736589+0000","last_active":"2023-11-28T04:24:34.742230+0000","last_peered":"2023-11-28T04:24:34.742230+0000","last_clean":"2023-11-28T04:24:34.742230+0000","last_became_active":"2023-11-28T04:24:34.736331+0000","last_became_peered":"2023-11-28T04:24:34.736331+0000","last_unstale":"2023-11-28T04:24:34.742230+0000","last_undegraded":"2023-11-28T04:24:34.742230+0000","last_fullsized":"2023-11-28T04:24:34.742230+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.104048+0000","last_change":"2023-11-28T04:24:35.104048+0000","last_active":"2023-11-28T04:24:35.104048+0000","last_peered":"2023-11-28T04:24:35.104048+0000","last_clean":"2023-11-28T04:24:35.104048+0000","last_became_active":"2023-11-28T04:24:35.103782+0000","last_became_peered":"2023-11-28T04:24:35.103782+0000","last_unstale":"2023-11-28T04:24:35.104048+0000","last_undegraded":"2023-11-28T04:24:35.104048+0000","last_fullsized":"2023-11-28T04:24:35.104048+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750266+0000","last_change":"2023-11-28T04:24:34.746313+0000","last_active":"2023-11-28T04:24:34.750266+0000","last_peered":"2023-11-28T04:24:34.750266+0000","last_clean":"2023-11-28T04:24:34.750266+0000","last_became_active":"2023-11-28T04:24:34.745826+0000","last_became_peered":"2023-11-28T04:24:34.745826+0000","last_unstale":"2023-11-28T04:24:34.750266+0000","last_undegraded":"2023-11-28T04:24:34.750266+0000","last_fullsized":"2023-11-28T04:24:34.750266+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":157,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728575+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:24:34.728575+0000","last_peered":"2023-11-28T04:24:34.728575+0000","last_clean":"2023-11-28T04:24:34.728575+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:24:34.728575+0000","last_undegraded":"2023-11-28T04:24:34.728575+0000","last_fullsized":"2023-11-28T04:24:34.728575+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.118109+0000","last_change":"2023-11-28T04:24:35.118109+0000","last_active":"2023-11-28T04:24:35.118109+0000","last_peered":"2023-11-28T04:24:35.118109+0000","last_clean":"2023-11-28T04:24:35.118109+0000","last_became_active":"2023-11-28T04:24:35.117658+0000","last_became_peered":"2023-11-28T04:24:35.117658+0000","last_unstale":"2023-11-28T04:24:35.118109+0000","last_undegraded":"2023-11-28T04:24:35.118109+0000","last_fullsized":"2023-11-28T04:24:35.118109+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750135+0000","last_change":"2023-11-28T04:24:34.747233+0000","last_active":"2023-11-28T04:24:34.750135+0000","last_peered":"2023-11-28T04:24:34.750135+0000","last_clean":"2023-11-28T04:24:34.750135+0000","last_became_active":"2023-11-28T04:24:34.746948+0000","last_became_peered":"2023-11-28T04:24:34.746948+0000","last_unstale":"2023-11-28T04:24:34.750135+0000","last_undegraded":"2023-11-28T04:24:34.750135+0000","last_fullsized":"2023-11-28T04:24:34.750135+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":156,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.731095+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:24:34.731095+0000","last_peered":"2023-11-28T04:24:34.731095+0000","last_clean":"2023-11-28T04:24:34.731095+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:24:34.731095+0000","last_undegraded":"2023-11-28T04:24:34.731095+0000","last_fullsized":"2023-11-28T04:24:34.731095+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1b","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.106535+0000","last_change":"2023-11-28T04:24:35.106535+0000","last_active":"2023-11-28T04:24:35.106535+0000","last_peered":"2023-11-28T04:24:35.106535+0000","last_clean":"2023-11-28T04:24:35.106535+0000","last_became_active":"2023-11-28T04:24:35.105171+0000","last_became_peered":"2023-11-28T04:24:35.105171+0000","last_unstale":"2023-11-28T04:24:35.106535+0000","last_undegraded":"2023-11-28T04:24:35.106535+0000","last_fullsized":"2023-11-28T04:24:35.106535+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"196'2","reported_seq":11,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110642+0000","last_change":"2023-11-28T04:24:34.746128+0000","last_active":"2023-11-28T04:24:35.110642+0000","last_peered":"2023-11-28T04:24:35.110642+0000","last_clean":"2023-11-28T04:24:35.110642+0000","last_became_active":"2023-11-28T04:24:34.745853+0000","last_became_peered":"2023-11-28T04:24:34.745853+0000","last_unstale":"2023-11-28T04:24:35.110642+0000","last_undegraded":"2023-11-28T04:24:35.110642+0000","last_fullsized":"2023-11-28T04:24:35.110642+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":163,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.724501+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:24:34.724501+0000","last_peered":"2023-11-28T04:24:34.724501+0000","last_clean":"2023-11-28T04:24:34.724501+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:24:34.724501+0000","last_undegraded":"2023-11-28T04:24:34.724501+0000","last_fullsized":"2023-11-28T04:24:34.724501+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109989+0000","last_change":"2023-11-28T04:24:35.109989+0000","last_active":"2023-11-28T04:24:35.109989+0000","last_peered":"2023-11-28T04:24:35.109989+0000","last_clean":"2023-11-28T04:24:35.109989+0000","last_became_active":"2023-11-28T04:24:35.109692+0000","last_became_peered":"2023-11-28T04:24:35.109692+0000","last_unstale":"2023-11-28T04:24:35.109989+0000","last_undegraded":"2023-11-28T04:24:35.109989+0000","last_fullsized":"2023-11-28T04:24:35.109989+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.750079+0000","last_change":"2023-11-28T04:24:34.745644+0000","last_active":"2023-11-28T04:24:34.750079+0000","last_peered":"2023-11-28T04:24:34.750079+0000","last_clean":"2023-11-28T04:24:34.750079+0000","last_became_active":"2023-11-28T04:24:34.745383+0000","last_became_peered":"2023-11-28T04:24:34.745383+0000","last_unstale":"2023-11-28T04:24:34.750079+0000","last_undegraded":"2023-11-28T04:24:34.750079+0000","last_fullsized":"2023-11-28T04:24:34.750079+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":156,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.731689+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:24:34.731689+0000","last_peered":"2023-11-28T04:24:34.731689+0000","last_clean":"2023-11-28T04:24:34.731689+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:24:34.731689+0000","last_undegraded":"2023-11-28T04:24:34.731689+0000","last_fullsized":"2023-11-28T04:24:34.731689+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":155,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728516+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:24:34.728516+0000","last_peered":"2023-11-28T04:24:34.728516+0000","last_clean":"2023-11-28T04:24:34.728516+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:24:34.728516+0000","last_undegraded":"2023-11-28T04:24:34.728516+0000","last_fullsized":"2023-11-28T04:24:34.728516+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.103284+0000","last_change":"2023-11-28T04:24:35.103284+0000","last_active":"2023-11-28T04:24:35.103284+0000","last_peered":"2023-11-28T04:24:35.103284+0000","last_clean":"2023-11-28T04:24:35.103284+0000","last_became_active":"2023-11-28T04:24:35.102999+0000","last_became_peered":"2023-11-28T04:24:35.102999+0000","last_unstale":"2023-11-28T04:24:35.103284+0000","last_undegraded":"2023-11-28T04:24:35.103284+0000","last_fullsized":"2023-11-28T04:24:35.103284+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.15","version":"196'3","reported_seq":12,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.115537+0000","last_change":"2023-11-28T04:24:34.741114+0000","last_active":"2023-11-28T04:24:35.115537+0000","last_peered":"2023-11-28T04:24:35.115537+0000","last_clean":"2023-11-28T04:24:35.115537+0000","last_became_active":"2023-11-28T04:24:34.740810+0000","last_became_peered":"2023-11-28T04:24:34.740810+0000","last_unstale":"2023-11-28T04:24:35.115537+0000","last_undegraded":"2023-11-28T04:24:35.115537+0000","last_fullsized":"2023-11-28T04:24:35.115537+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":153,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728571+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:24:34.728571+0000","last_peered":"2023-11-28T04:24:34.728571+0000","last_clean":"2023-11-28T04:24:34.728571+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:24:34.728571+0000","last_undegraded":"2023-11-28T04:24:34.728571+0000","last_fullsized":"2023-11-28T04:24:34.728571+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.114219+0000","last_change":"2023-11-28T04:24:35.114219+0000","last_active":"2023-11-28T04:24:35.114219+0000","last_peered":"2023-11-28T04:24:35.114219+0000","last_clean":"2023-11-28T04:24:35.114219+0000","last_became_active":"2023-11-28T04:24:35.113936+0000","last_became_peered":"2023-11-28T04:24:35.113936+0000","last_unstale":"2023-11-28T04:24:35.114219+0000","last_undegraded":"2023-11-28T04:24:35.114219+0000","last_fullsized":"2023-11-28T04:24:35.114219+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.10","version":"196'4","reported_seq":13,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.119839+0000","last_change":"2023-11-28T04:24:34.740421+0000","last_active":"2023-11-28T04:24:35.119839+0000","last_peered":"2023-11-28T04:24:35.119839+0000","last_clean":"2023-11-28T04:24:35.119839+0000","last_became_active":"2023-11-28T04:24:34.740193+0000","last_became_peered":"2023-11-28T04:24:34.740193+0000","last_unstale":"2023-11-28T04:24:35.119839+0000","last_undegraded":"2023-11-28T04:24:35.119839+0000","last_fullsized":"2023-11-28T04:24:35.119839+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":153,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.728562+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:24:34.728562+0000","last_peered":"2023-11-28T04:24:34.728562+0000","last_clean":"2023-11-28T04:24:34.728562+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:24:34.728562+0000","last_undegraded":"2023-11-28T04:24:34.728562+0000","last_fullsized":"2023-11-28T04:24:34.728562+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1c","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.110020+0000","last_change":"2023-11-28T04:24:35.110020+0000","last_active":"2023-11-28T04:24:35.110020+0000","last_peered":"2023-11-28T04:24:35.110020+0000","last_clean":"2023-11-28T04:24:35.110020+0000","last_became_active":"2023-11-28T04:24:35.108591+0000","last_became_peered":"2023-11-28T04:24:35.108591+0000","last_unstale":"2023-11-28T04:24:35.110020+0000","last_undegraded":"2023-11-28T04:24:35.110020+0000","last_fullsized":"2023-11-28T04:24:35.110020+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.13","version":"196'1","reported_seq":10,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.092879+0000","last_change":"2023-11-28T04:24:34.739859+0000","last_active":"2023-11-28T04:24:35.092879+0000","last_peered":"2023-11-28T04:24:35.092879+0000","last_clean":"2023-11-28T04:24:35.092879+0000","last_became_active":"2023-11-28T04:24:34.739589+0000","last_became_peered":"2023-11-28T04:24:34.739589+0000","last_unstale":"2023-11-28T04:24:35.092879+0000","last_undegraded":"2023-11-28T04:24:35.092879+0000","last_fullsized":"2023-11-28T04:24:35.092879+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":157,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.730721+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:24:34.730721+0000","last_peered":"2023-11-28T04:24:34.730721+0000","last_clean":"2023-11-28T04:24:34.730721+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:24:34.730721+0000","last_undegraded":"2023-11-28T04:24:34.730721+0000","last_fullsized":"2023-11-28T04:24:34.730721+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"0'0","reported_seq":8,"reported_epoch":196,"state":"active+clean","last_fresh":"2023-11-28T04:24:35.109750+0000","last_change":"2023-11-28T04:24:35.109750+0000","last_active":"2023-11-28T04:24:35.109750+0000","last_peered":"2023-11-28T04:24:35.109750+0000","last_clean":"2023-11-28T04:24:35.109750+0000","last_became_active":"2023-11-28T04:24:35.109465+0000","last_became_peered":"2023-11-28T04:24:35.109465+0000","last_unstale":"2023-11-28T04:24:35.109750+0000","last_undegraded":"2023-11-28T04:24:35.109750+0000","last_fullsized":"2023-11-28T04:24:35.109750+0000","mapping_epoch":195,"log_start":"0'0","ondisk_log_start":"0'0","created":195,"last_epoch_clean":196,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:34.713789+0000","last_clean_scrub_stamp":"2023-11-28T04:24:34.713789+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":9,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.743569+0000","last_change":"2023-11-28T04:24:34.740418+0000","last_active":"2023-11-28T04:24:34.743569+0000","last_peered":"2023-11-28T04:24:34.743569+0000","last_clean":"2023-11-28T04:24:34.743569+0000","last_became_active":"2023-11-28T04:24:34.740140+0000","last_became_peered":"2023-11-28T04:24:34.740140+0000","last_unstale":"2023-11-28T04:24:34.743569+0000","last_undegraded":"2023-11-28T04:24:34.743569+0000","last_fullsized":"2023-11-28T04:24:34.743569+0000","mapping_epoch":194,"log_start":"0'0","ondisk_log_start":"0'0","created":194,"last_epoch_clean":195,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:24:33.707869+0000","last_clean_scrub_stamp":"2023-11-28T04:24:33.707869+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":173,"reported_epoch":195,"state":"active+clean","last_fresh":"2023-11-28T04:24:34.726681+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:24:34.726681+0000","last_peered":"2023-11-28T04:24:34.726681+0000","last_clean":"2023-11-28T04:24:34.726681+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:24:34.726681+0000","last_undegraded":"2023-11-28T04:24:34.726681+0000","last_fullsized":"2023-11-28T04:24:34.726681+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":24,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575945,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303572,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068268,"statfs":{"total":96636764160,"available":96325906432,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.60599999999999998,"15min":0.61699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29399999999999998},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.84499999999999997},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.61199999999999999,"15min":0.622},"min":{"1min":0.496,"5min":0.443,"15min":0.40999999999999998},"max":{"1min":0.77600000000000002,"5min":0.80400000000000005,"15min":0.872},"last":0.60199999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.57299999999999995,"15min":0.57099999999999995},"min":{"1min":0.379,"5min":0.309,"15min":0.309},"max":{"1min":0.69299999999999995,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.496},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.40799999999999997,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.73399999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.56699999999999995}]},{"osd":2,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.58299999999999996,"15min":0.58599999999999997},"min":{"1min":0.48999999999999999,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.77700000000000002,"5min":0.77700000000000002,"15min":0.89000000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.36099999999999999},"max":{"1min":0.71499999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.55000000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.58099999999999996,"15min":0.58699999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.35999999999999999},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.83599999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.59499999999999997,"15min":0.60699999999999998},"min":{"1min":0.433,"5min":0.42399999999999999,"15min":0.39700000000000002},"max":{"1min":0.76200000000000001,"5min":0.81999999999999995,"15min":0.90300000000000002},"last":0.59199999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.46400000000000002,"15min":0.46700000000000003},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.29499999999999998},"max":{"1min":0.65900000000000003,"5min":0.71699999999999997,"15min":0.73399999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.496,"15min":0.498},"min":{"1min":0.36099999999999999,"5min":0.35599999999999998,"15min":0.31900000000000001},"max":{"1min":0.70199999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.57799999999999996}]},{"osd":5,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.504,"15min":0.50600000000000001},"min":{"1min":0.38100000000000001,"5min":0.33100000000000002,"15min":0.26000000000000001},"max":{"1min":0.71099999999999997,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.53200000000000003,"15min":0.54100000000000004},"min":{"1min":0.41999999999999998,"5min":0.35999999999999999,"15min":0.33500000000000002},"max":{"1min":0.71299999999999997,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.629}]},{"osd":6,"last update":"Tue Nov 28 04:23:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.80200000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.56499999999999995,"15min":0.56699999999999995},"min":{"1min":0.441,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.749,"5min":0.79900000000000004,"15min":0.81899999999999995},"last":0.61899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575945,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.54600000000000004,"15min":0.55200000000000005},"min":{"1min":0.39000000000000001,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.68400000000000005,"5min":0.77500000000000002,"15min":0.77600000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.57799999999999996,"15min":0.57599999999999996},"min":{"1min":0.46100000000000002,"5min":0.32900000000000001,"15min":0.28499999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.36799999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.45400000000000001,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.71999999999999997,"5min":0.81999999999999995,"15min":0.86199999999999999},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.46200000000000002,"5min":0.379,"15min":0.29899999999999999},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.44400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.56999999999999995,"15min":0.57599999999999996},"min":{"1min":0.41099999999999998,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.80900000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.58299999999999996,"15min":0.59099999999999997},"min":{"1min":0.436,"5min":0.36099999999999999,"15min":0.34899999999999998},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.502}]},{"osd":3,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57899999999999996,"15min":0.58299999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.32900000000000001},"max":{"1min":0.751,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.58999999999999997,"15min":0.59199999999999997},"min":{"1min":0.49199999999999999,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.84999999999999998},"last":0.52400000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.439,"15min":0.439},"min":{"1min":0.311,"5min":0.27700000000000002,"15min":0.26400000000000001},"max":{"1min":0.60899999999999999,"5min":0.78800000000000003,"15min":0.92400000000000004},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.47399999999999998,"15min":0.48099999999999998},"min":{"1min":0.316,"5min":0.316,"15min":0.28599999999999998},"max":{"1min":0.68600000000000005,"5min":0.76500000000000001,"15min":0.96899999999999997},"last":0.55600000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.503,"15min":0.501},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.30099999999999999},"max":{"1min":0.77700000000000002,"5min":0.80800000000000005,"15min":0.80800000000000005},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.53500000000000003,"5min":0.54600000000000004,"15min":0.54400000000000004},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.315},"max":{"1min":0.749,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.624}]},{"osd":7,"last update":"Tue Nov 28 04:23:36 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.71899999999999997,"5min":0.752,"15min":0.81399999999999995},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.53500000000000003,"15min":0.54100000000000004},"min":{"1min":0.35899999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.73699999999999999,"5min":0.82999999999999996,"15min":0.83299999999999996},"last":0.71799999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575944,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303560,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068280,"statfs":{"total":96636764160,"available":96325918720,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41699999999999998,"5min":0.36699999999999999,"15min":0.372},"min":{"1min":0.24299999999999999,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.74299999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.39800000000000002,"15min":0.39700000000000002},"min":{"1min":0.29899999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.60699999999999998,"5min":0.626,"15min":0.67000000000000004},"last":0.50700000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.438,"5min":0.41999999999999998,"15min":0.41199999999999998},"min":{"1min":0.315,"5min":0.29299999999999998,"15min":0.24299999999999999},"max":{"1min":0.623,"5min":0.623,"15min":0.81200000000000006},"last":0.432},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.44900000000000001,"15min":0.441},"min":{"1min":0.38,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.65500000000000003},"last":0.47299999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49199999999999999,"15min":0.48599999999999999},"min":{"1min":0.36799999999999999,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.67500000000000004,"5min":0.72999999999999998,"15min":0.72999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.45300000000000001},"min":{"1min":0.35699999999999998,"5min":0.28999999999999998,"15min":0.27100000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.496}]},{"osd":4,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60699999999999998,"15min":0.61499999999999999},"min":{"1min":0.47199999999999998,"5min":0.42299999999999999,"15min":0.375},"max":{"1min":0.77400000000000002,"5min":0.93100000000000005,"15min":0.99099999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.63,"15min":0.63300000000000001},"min":{"1min":0.49099999999999999,"5min":0.42699999999999999,"15min":0.39900000000000002},"max":{"1min":0.83899999999999997,"5min":0.92500000000000004,"15min":0.98599999999999999},"last":0.68000000000000005}]},{"osd":5,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64500000000000002,"15min":0.65000000000000002},"min":{"1min":0.48299999999999998,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.82599999999999996,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.68400000000000005},"min":{"1min":0.54900000000000004,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.94599999999999995,"5min":0.99199999999999999,"15min":1.0349999999999999},"last":0.75}]},{"osd":6,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.626,"15min":0.625},"min":{"1min":0.496,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.90100000000000002,"5min":0.95299999999999996,"15min":0.95299999999999996},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.65200000000000002,"15min":0.66300000000000003},"min":{"1min":0.51600000000000001,"5min":0.46500000000000002,"15min":0.41799999999999998},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.999},"last":0.67000000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.65600000000000003,"15min":0.65700000000000003},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.44600000000000001},"max":{"1min":0.88900000000000001,"5min":0.91600000000000004,"15min":1.02},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.65500000000000003,"15min":0.65700000000000003},"min":{"1min":0.52300000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.86699999999999999,"5min":0.90500000000000003,"15min":0.95499999999999996},"last":0.77900000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575945,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.34899999999999998,"5min":0.34999999999999998,"15min":0.36399999999999999},"min":{"1min":0.25800000000000001,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.53100000000000003,"5min":0.56799999999999995,"15min":0.97799999999999998},"last":0.371},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.378,"15min":0.39200000000000002},"min":{"1min":0.249,"5min":0.223,"15min":0.223},"max":{"1min":0.53100000000000003,"5min":0.61699999999999999,"15min":0.95099999999999996},"last":0.38}]},{"osd":2,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.378,"5min":0.38300000000000001,"15min":0.38400000000000001},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.85999999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.46400000000000002},"min":{"1min":0.32500000000000001,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.61299999999999999,"5min":0.66000000000000003,"15min":0.86799999999999999},"last":0.43099999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.40100000000000002,"5min":0.40999999999999998,"15min":0.41299999999999998},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":0.52700000000000002,"5min":0.60199999999999998,"15min":0.64600000000000002},"last":0.442},{"interface":"front","average":{"1min":0.437,"5min":0.44700000000000001,"15min":0.44600000000000001},"min":{"1min":0.33500000000000002,"5min":0.315,"15min":0.30199999999999999},"max":{"1min":0.59399999999999997,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.46999999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59199999999999997,"5min":0.61799999999999999,"15min":0.64100000000000001},"min":{"1min":0.39700000000000002,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.84699999999999998,"5min":0.85799999999999998,"15min":1.0409999999999999},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.63800000000000001,"15min":0.65800000000000003},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":1.038},"last":0.50800000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.65700000000000003,"15min":0.66700000000000004},"min":{"1min":0.42799999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":1.0800000000000001},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.64100000000000001,"15min":0.66000000000000003},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.79400000000000004,"5min":0.878,"15min":1.0549999999999999},"last":0.55500000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.61499999999999999,"15min":0.63100000000000001},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.81599999999999995,"5min":0.95099999999999996,"15min":1.0449999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.64800000000000002},"min":{"1min":0.46700000000000003,"5min":0.433,"15min":0.42199999999999999},"max":{"1min":0.84499999999999997,"5min":0.876,"15min":1.036},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:24:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.65300000000000002,"15min":0.67000000000000004},"min":{"1min":0.46300000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.86199999999999999,"5min":0.97299999999999998,"15min":1.0189999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.69899999999999995},"min":{"1min":0.498,"5min":0.47299999999999998,"15min":0.40300000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":1.0669999999999999},"last":0.624}]}]},{"osd":2,"up_from":13,"seq":55834575946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.34599999999999997,"5min":0.371,"15min":0.376},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":0.45800000000000002,"5min":0.57199999999999995,"15min":0.61899999999999999},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.31900000000000001,"5min":0.314,"15min":0.30599999999999999},"max":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.67000000000000004},"last":0.40699999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.38200000000000001,"15min":0.38400000000000001},"min":{"1min":0.27600000000000002,"5min":0.27100000000000002,"15min":0.25900000000000001},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.622},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.41099999999999998,"15min":0.41199999999999998},"min":{"1min":0.28599999999999998,"5min":0.27900000000000003,"15min":0.26200000000000001},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.35099999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40999999999999998,"5min":0.44400000000000001,"15min":0.443},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65900000000000003,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.41899999999999998,"15min":0.42199999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.59999999999999998,"5min":0.65300000000000002,"15min":0.65300000000000002},"last":0.38}]},{"osd":4,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.60499999999999998,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.82599999999999996,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.622},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.79800000000000004,"5min":0.87,"15min":0.96199999999999997},"last":0.69199999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.66200000000000003,"15min":0.66600000000000004},"min":{"1min":0.46800000000000003,"5min":0.46200000000000002,"15min":0.44500000000000001},"max":{"1min":0.82199999999999995,"5min":0.92000000000000004,"15min":1.0760000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.67400000000000004,"15min":0.67600000000000005},"min":{"1min":0.46800000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.84699999999999998,"5min":0.92500000000000004,"15min":1.016},"last":0.70899999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.65700000000000003,"15min":0.66100000000000003},"min":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.443},"max":{"1min":0.86699999999999999,"5min":0.93999999999999995,"15min":0.99099999999999999},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.68300000000000005,"15min":0.68999999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.84599999999999997,"5min":0.93400000000000005,"15min":1.0529999999999999},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.65000000000000002,"15min":0.65500000000000003},"min":{"1min":0.435,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.83599999999999997,"5min":0.91300000000000003,"15min":0.95599999999999996},"last":0.63},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64900000000000002,"15min":0.66500000000000004},"min":{"1min":0.433,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.81299999999999994,"5min":0.876,"15min":1.0129999999999999},"last":0.72099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834575945,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307668,"kb_used_data":2380,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064172,"statfs":{"total":96636764160,"available":96321712128,"internally_reserved":0,"allocated":2437120,"data_stored":1230652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.36499999999999999,"15min":0.371},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":0.47999999999999998,"5min":0.57399999999999995,"15min":0.58899999999999997},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.36199999999999999,"5min":0.38600000000000001,"15min":0.39600000000000002},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.501,"5min":0.54500000000000004,"15min":0.66500000000000004},"last":0.26400000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.375,"15min":0.39100000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.52100000000000002,"5min":0.53800000000000003,"15min":0.65500000000000003},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.38300000000000001,"5min":0.43099999999999999,"15min":0.434},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63900000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.42899999999999999,"15min":0.439},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.56499999999999995,"5min":0.60099999999999998,"15min":0.746},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.432,"5min":0.45900000000000002,"15min":0.46500000000000002},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.27400000000000002},"max":{"1min":0.57099999999999995,"5min":0.63300000000000001,"15min":0.68000000000000005},"last":0.27700000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.623,"15min":0.624},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90500000000000003,"5min":0.92500000000000004,"15min":0.96099999999999997},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.625,"5min":0.63200000000000001,"15min":0.63500000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.83999999999999997,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.46400000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.63900000000000001,"15min":0.63900000000000001},"min":{"1min":0.45200000000000001,"5min":0.44700000000000001,"15min":0.42699999999999999},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":1.0409999999999999},"last":0.437},{"interface":"front","average":{"1min":0.88600000000000001,"5min":0.71099999999999997,"15min":0.68000000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":4.3049999999999997,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.378}]},{"osd":6,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.65900000000000003,"15min":0.65400000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.85599999999999998,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.66700000000000004,"15min":0.67100000000000004},"min":{"1min":0.47599999999999998,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.88200000000000001,"5min":1.028,"15min":1.028},"last":0.40999999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:24:33 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.66900000000000004,"15min":0.67400000000000004},"min":{"1min":0.47399999999999998,"5min":0.46100000000000002,"15min":0.44},"max":{"1min":0.82499999999999996,"5min":1.008,"15min":1.008},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.67900000000000005,"15min":0.67500000000000004},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.89900000000000002,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.49299999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575944,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.34399999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.65600000000000003,"5min":0.80600000000000005,"15min":0.83699999999999997},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.56699999999999995,"15min":0.56999999999999995},"min":{"1min":0.35899999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.746,"5min":0.746,"15min":0.78800000000000003},"last":0.44500000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57699999999999996,"5min":0.57399999999999995,"15min":0.57999999999999996},"min":{"1min":0.39400000000000002,"5min":0.39300000000000002,"15min":0.38900000000000001},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.874},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.57499999999999996,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.67900000000000005,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.63400000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.55800000000000005,"15min":0.57399999999999995},"min":{"1min":0.33600000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.67100000000000004,"5min":0.76400000000000001,"15min":0.83299999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57799999999999996,"15min":0.58499999999999996},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.71299999999999997,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42999999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.60899999999999999,"15min":0.621},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.42499999999999999},"max":{"1min":0.78700000000000003,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.63,"5min":0.64100000000000001,"15min":0.64500000000000002},"min":{"1min":0.52500000000000002,"5min":0.51900000000000002,"15min":0.42499999999999999},"max":{"1min":0.77900000000000003,"5min":0.79900000000000004,"15min":0.872},"last":0.61799999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.47699999999999998,"15min":0.48199999999999998},"min":{"1min":0.314,"5min":0.314,"15min":0.308},"max":{"1min":0.65100000000000002,"5min":0.75800000000000001,"15min":0.78300000000000003},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.497,"5min":0.51800000000000002,"15min":0.51700000000000002},"min":{"1min":0.35099999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.69599999999999995,"5min":0.79300000000000004,"15min":0.83499999999999996},"last":0.67900000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48599999999999999,"5min":0.504,"15min":0.51300000000000001},"min":{"1min":0.378,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.71999999999999997,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54800000000000004},"min":{"1min":0.41299999999999998,"5min":0.36799999999999999,"15min":0.33200000000000002},"max":{"1min":0.72799999999999998,"5min":0.79100000000000004,"15min":0.85099999999999998},"last":0.60499999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:23:54 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56299999999999994,"15min":0.56899999999999995},"min":{"1min":0.39500000000000002,"5min":0.36299999999999999,"15min":0.33000000000000002},"max":{"1min":0.73599999999999999,"5min":0.82499999999999996,"15min":0.86099999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.41299999999999998,"5min":0.39400000000000002,"15min":0.34200000000000003},"max":{"1min":0.78300000000000003,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.70399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575946,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307656,"kb_used_data":2368,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94064184,"statfs":{"total":96636764160,"available":96321724416,"internally_reserved":0,"allocated":2424832,"data_stored":1223973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.54700000000000004,"15min":0.54600000000000004},"min":{"1min":0.41799999999999998,"5min":0.35099999999999998,"15min":0.32500000000000001},"max":{"1min":0.70599999999999996,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.40799999999999997,"5min":0.36399999999999999,"15min":0.33700000000000002},"max":{"1min":0.70199999999999996,"5min":0.80000000000000004,"15min":0.82499999999999996},"last":0.51600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59699999999999998,"15min":0.59199999999999997},"min":{"1min":0.47799999999999998,"5min":0.40600000000000003,"15min":0.372},"max":{"1min":0.74299999999999999,"5min":0.80800000000000005,"15min":0.82199999999999995},"last":0.628},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.623,"15min":0.61599999999999999},"min":{"1min":0.48299999999999998,"5min":0.38400000000000001,"15min":0.36099999999999999},"max":{"1min":0.745,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.55700000000000005}]},{"osd":2,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.60499999999999998},"min":{"1min":0.44900000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.76700000000000002,"5min":0.78100000000000003,"15min":0.84999999999999998},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63900000000000001,"15min":0.63500000000000001},"min":{"1min":0.53400000000000003,"5min":0.45000000000000001,"15min":0.40400000000000003},"max":{"1min":0.79300000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58899999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.57399999999999995,"15min":0.57199999999999995},"min":{"1min":0.46500000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.70399999999999996,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.58999999999999997,"15min":0.59799999999999998},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73999999999999999,"5min":0.81499999999999995,"15min":0.85199999999999998},"last":0.434}]},{"osd":4,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.45600000000000002,"15min":0.46100000000000002},"min":{"1min":0.32500000000000001,"5min":0.32000000000000001,"15min":0.308},"max":{"1min":0.68400000000000005,"5min":0.70099999999999996,"15min":0.88200000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.55600000000000005,"15min":0.56499999999999995},"min":{"1min":0.41299999999999998,"5min":0.318,"15min":0.318},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.495,"15min":0.50600000000000001},"min":{"1min":0.34200000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.64200000000000002,"5min":0.77300000000000002,"15min":0.872},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.58299999999999996,"15min":0.58199999999999996},"min":{"1min":0.41599999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.75900000000000001,"5min":0.82699999999999996,"15min":0.92100000000000004},"last":0.66900000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:24:29 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.52700000000000002,"15min":0.52600000000000002},"min":{"1min":0.374,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.71899999999999997,"5min":0.80600000000000005,"15min":0.90300000000000002},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55800000000000005,"15min":0.56399999999999995},"min":{"1min":0.42299999999999999,"5min":0.35399999999999998,"15min":0.313},"max":{"1min":0.78100000000000003,"5min":0.85099999999999998,"15min":0.90700000000000003},"last":0.64100000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":644,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":666,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:24:38.427 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T04:24:38.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T04:24:38.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.580+0000 7f616337f700 1 Processor -- start 2023-11-28T04:24:38.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.580+0000 7f616337f700 1 -- start start 2023-11-28T04:24:38.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616337f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f615c098d10 0x7f615c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616337f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f615c0c6190 0x7f615c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616337f700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f615c0c72d0 con 0x7f615c064350 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616337f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c154bc0 con 0x7f615c0c6190 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616337f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f615c0c66c0 con 0x7f615c098d10 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616091a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f615c0c6190 0x7f615c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616091a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f615c0c6190 0x7f615c063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41894/0 (socket says 172.21.15.73:41894) 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616091a700 1 -- 172.21.15.73:0/809444564 learned_addr learned my addr 172.21.15.73:0/809444564 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.581+0000 7f616111b700 1 --2- 172.21.15.73:0/809444564 >> v2:172.21.15.150:3300/0 conn(0x7f615c098d10 0x7f615c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616091a700 1 -- 172.21.15.73:0/809444564 >> v2:172.21.15.150:3300/0 conn(0x7f615c098d10 msgr2=0x7f615c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616091a700 1 --2- 172.21.15.73:0/809444564 >> v2:172.21.15.150:3300/0 conn(0x7f615c098d10 0x7f615c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616091a700 1 -- 172.21.15.73:0/809444564 >> v1:172.21.15.150:6789/0 conn(0x7f615c064350 legacy=0x7f615c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616091a700 1 -- 172.21.15.73:0/809444564 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f615c0c69a0 con 0x7f615c0c6190 2023-11-28T04:24:38.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616091a700 1 --2- 172.21.15.73:0/809444564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c0c6190 0x7f615c063e60 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7f615c083f80 tx=0x7f614c0090f0).ready entity=mon.0 client_cookie=58e4558b09c83cf5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f6153fff700 1 -- 172.21.15.73:0/809444564 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00b520 con 0x7f615c0c6190 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f6153fff700 1 -- 172.21.15.73:0/809444564 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00b680 con 0x7f615c0c6190 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f6153fff700 1 -- 172.21.15.73:0/809444564 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00ba00 con 0x7f615c0c6190 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 -- 172.21.15.73:0/809444564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c0c6190 msgr2=0x7f615c063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 --2- 172.21.15.73:0/809444564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c0c6190 0x7f615c063e60 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 rx=0x7f615c083f80 tx=0x7f614c0090f0).stop 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 -- 172.21.15.73:0/809444564 shutdown_connections 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 --2- 172.21.15.73:0/809444564 >> v2:172.21.15.150:3300/0 conn(0x7f615c098d10 0x7f615c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 --2- 172.21.15.73:0/809444564 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c0c6190 0x7f615c063e60 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 -- 172.21.15.73:0/809444564 >> 172.21.15.73:0/809444564 conn(0x7f615c0cb310 msgr2=0x7f615c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.582+0000 7f616337f700 1 -- 172.21.15.73:0/809444564 shutdown_connections 2023-11-28T04:24:38.585 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.583+0000 7f616337f700 1 -- 172.21.15.73:0/809444564 wait complete. 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.584+0000 7f616337f700 1 Processor -- start 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.584+0000 7f616337f700 1 -- start start 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f615c064350 0x7f615c0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f615c0c6190 0x7f615c14f330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c157470 con 0x7f615c098d10 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f615c156860 con 0x7f615c064350 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f615c156b40 con 0x7f615c0c6190 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41900/0 (socket says 172.21.15.73:41900) 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 -- 172.21.15.73:0/148235534 learned_addr learned my addr 172.21.15.73:0/148235534 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616111b700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f615c064350 0x7f615c0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616191c700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f615c0c6190 0x7f615c14f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 -- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f615c0c6190 msgr2=0x7f615c14f330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f615c0c6190 0x7f615c14f330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 -- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f615c064350 msgr2=0x7f615c0af680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f615c064350 0x7f615c0af680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 -- 172.21.15.73:0/148235534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c009370 con 0x7f615c098d10 2023-11-28T04:24:38.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616091a700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 rx=0x7f614c009130 tx=0x7f614c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c00be90 con 0x7f615c098d10 2023-11-28T04:24:38.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f614c00b7f0 con 0x7f615c098d10 2023-11-28T04:24:38.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614c01b660 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f615c156e20 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.585+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f615c156500 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.586+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f615c0c1f20 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.586+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f614c01b9d0 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.587+0000 7f61517fa700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 0x7f6140061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.587+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f614c07bc20 con 0x7f615c098d10 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.587+0000 7f616111b700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 0x7f6140061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.587+0000 7f616111b700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 0x7f6140061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6158009c30 tx=0x7f6158009060).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.591+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f614c04cef0 con 0x7f615c098d10 2023-11-28T04:24:38.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.818+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f615c06c860 con 0x7f615c098d10 2023-11-28T04:24:38.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.821+0000 7f61517fa700 1 -- 172.21.15.73:0/148235534 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v48) v1 ==== 190+0+0 (secure 0 0 0) 0x7f614c029070 con 0x7f615c098d10 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 msgr2=0x7f6140061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 0x7f6140061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f6158009c30 tx=0x7f6158009060).stop 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 msgr2=0x7f615c156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 rx=0x7f614c009130 tx=0x7f614c017000).stop 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 shutdown_connections 2023-11-28T04:24:38.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f614005f150 0x7f6140061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f615c0c6190 0x7f615c14f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f615c064350 0x7f615c0af680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 --2- 172.21.15.73:0/148235534 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f615c098d10 0x7f615c156010 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 >> 172.21.15.73:0/148235534 conn(0x7f615c0cb310 msgr2=0x7f615c158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 shutdown_connections 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.823+0000 7f616337f700 1 -- 172.21.15.73:0/148235534 wait complete. 2023-11-28T04:24:38.825 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T04:24:38.836 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:24:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.948+0000 7fef5d2bc700 1 Processor -- start 2023-11-28T04:24:38.949 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.948+0000 7fef5d2bc700 1 -- start start 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef5d2bc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef580caab0 0x7fef580aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef5d2bc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580ca180 0x7fef5815e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef5d2bc700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fef580a9a90 con 0x7fef580cae90 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef5d2bc700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fef58154ce0 con 0x7fef580caab0 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef5d2bc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef580a8e80 con 0x7fef580ca180 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef56ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef580caab0 0x7fef580aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef56ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fef580caab0 0x7fef580aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41910/0 (socket says 172.21.15.73:41910) 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef56ffd700 1 -- 172.21.15.73:0/2894833204 learned_addr learned my addr 172.21.15.73:0/2894833204 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.949+0000 7fef577fe700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580ca180 0x7fef5815e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.952 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef56ffd700 1 -- 172.21.15.73:0/2894833204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580ca180 msgr2=0x7fef5815e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef56ffd700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580ca180 0x7fef5815e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef56ffd700 1 -- 172.21.15.73:0/2894833204 >> v1:172.21.15.150:6789/0 conn(0x7fef580cae90 legacy=0x7fef580c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef56ffd700 1 -- 172.21.15.73:0/2894833204 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef580a9160 con 0x7fef580caab0 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef56ffd700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580caab0 0x7fef580aa470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7fef48009810 tx=0x7fef48009060).ready entity=mon.0 client_cookie=fe4f050b72702ed1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef567fc700 1 -- 172.21.15.73:0/2894833204 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48005880 con 0x7fef580caab0 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef567fc700 1 -- 172.21.15.73:0/2894833204 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef480059e0 con 0x7fef580caab0 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef567fc700 1 -- 172.21.15.73:0/2894833204 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef48005d60 con 0x7fef580caab0 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 -- 172.21.15.73:0/2894833204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580caab0 msgr2=0x7fef580aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580caab0 0x7fef580aa470 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 rx=0x7fef48009810 tx=0x7fef48009060).stop 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 -- 172.21.15.73:0/2894833204 shutdown_connections 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580ca180 0x7fef5815e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/2894833204 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580caab0 0x7fef580aa470 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 -- 172.21.15.73:0/2894833204 >> 172.21.15.73:0/2894833204 conn(0x7fef58094920 msgr2=0x7fef5806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:38.953 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 -- 172.21.15.73:0/2894833204 shutdown_connections 2023-11-28T04:24:38.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.950+0000 7fef5d2bc700 1 -- 172.21.15.73:0/2894833204 wait complete. 2023-11-28T04:24:38.956 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 Processor -- start 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 -- start start 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef580caab0 0x7fef58154080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580cae90 0x7fef58157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580c6610 con 0x7fef580ca180 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef580c62e0 con 0x7fef580caab0 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.952+0000 7fef5d2bc700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fef58164f40 con 0x7fef580cae90 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41914/0 (socket says 172.21.15.73:41914) 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 -- 172.21.15.73:0/1020528292 learned_addr learned my addr 172.21.15.73:0/1020528292 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef577fe700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580cae90 0x7fef58157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef4e7fc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef580caab0 0x7fef58154080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 -- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580cae90 msgr2=0x7fef58157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580cae90 0x7fef58157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 -- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef580caab0 msgr2=0x7fef58154080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef580caab0 0x7fef58154080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 -- 172.21.15.73:0/1020528292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef48009300 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef56ffd700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7fef58083c30 tx=0x7fef48016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800b640 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fef4800b7a0 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef580c85a0 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef4800bb60 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.953+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef580c8b60 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.954+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef580ca560 con 0x7fef580ca180 2023-11-28T04:24:38.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.954+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fef48018070 con 0x7fef580ca180 2023-11-28T04:24:38.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.955+0000 7fef4f7fe700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 0x7fef38061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:38.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.955+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fef4807b1b0 con 0x7fef580ca180 2023-11-28T04:24:38.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.955+0000 7fef4e7fc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 0x7fef38061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:38.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.955+0000 7fef4e7fc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 0x7fef38061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fef580c9000 tx=0x7fef40007d40).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:38.960 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:38.959+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fef4804c480 con 0x7fef580ca180 2023-11-28T04:24:39.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.164+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fef5806c860 con 0x7fef580ca180 2023-11-28T04:24:39.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.164+0000 7fef4f7fe700 1 -- 172.21.15.73:0/1020528292 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fef4804faa0 con 0x7fef580ca180 2023-11-28T04:24:39.166 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:39.166 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:24:39.167 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 msgr2=0x7fef38061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 0x7fef38061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fef580c9000 tx=0x7fef40007d40).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 msgr2=0x7fef58164a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 rx=0x7fef58083c30 tx=0x7fef48016000).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 shutdown_connections 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fef3805f150 0x7fef38061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fef580cae90 0x7fef58157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fef580ca180 0x7fef58164a50 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 --2- 172.21.15.73:0/1020528292 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fef580caab0 0x7fef58154080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 >> 172.21.15.73:0/1020528292 conn(0x7fef58094920 msgr2=0x7fef580c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 shutdown_connections 2023-11-28T04:24:39.168 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.166+0000 7fef5d2bc700 1 -- 172.21.15.73:0/1020528292 wait complete. 2023-11-28T04:24:39.169 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:24:39.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T04:24:35.086025+0000', 'modified': '2023-11-28T04:24:37.094342+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9522}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9522': {'gid': 9522, 'name': 'a', 'rank': 0, 'incarnation': 68, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6837/1535793255', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6835', 'nonce': 1535793255}, {'type': 'v1', 'addr': '172.21.15.73:6837', 'nonce': 1535793255}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [24], 'metadata_pool': 23, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T04:24:39.582 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T04:24:39.582 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T04:24:39.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.697+0000 7f8415e5f700 1 Processor -- start 2023-11-28T04:24:39.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.697+0000 7f8415e5f700 1 -- start start 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f8415e5f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8410098d10 0x7f84100c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f8415e5f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84100c6190 0x7f8410063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f8415e5f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84100c72d0 con 0x7f8410064350 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f8415e5f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8410154bc0 con 0x7f8410098d10 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f8415e5f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100c66c0 con 0x7f84100c6190 2023-11-28T04:24:39.701 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f840f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8410098d10 0x7f84100c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f840ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8410064350 0x7f8410158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40786/0 (socket says 172.21.15.73:40786) 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.698+0000 7f840ffff700 1 -- 172.21.15.73:0/4270743852 learned_addr learned my addr 172.21.15.73:0/4270743852 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840effd700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84100c6190 0x7f8410063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840f7fe700 1 -- 172.21.15.73:0/4270743852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84100c6190 msgr2=0x7f8410063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840f7fe700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84100c6190 0x7f8410063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840f7fe700 1 -- 172.21.15.73:0/4270743852 >> v1:172.21.15.73:6789/0 conn(0x7f8410064350 legacy=0x7f8410158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840e7fc700 1 -- 172.21.15.73:0/4270743852 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 406355244 0 0) 0x7f84100c72d0 con 0x7f8410064350 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840f7fe700 1 -- 172.21.15.73:0/4270743852 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100c69a0 con 0x7f8410098d10 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840f7fe700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8410098d10 0x7f84100c7760 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 rx=0x7f8400009e60 tx=0x7f8400009780).ready entity=mon.0 client_cookie=21e4bdd9d948a5a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840e7fc700 1 -- 172.21.15.73:0/4270743852 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840000e070 con 0x7f8410098d10 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840e7fc700 1 -- 172.21.15.73:0/4270743852 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840000bc30 con 0x7f8410098d10 2023-11-28T04:24:39.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f840e7fc700 1 -- 172.21.15.73:0/4270743852 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8400012410 con 0x7f8410098d10 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 -- 172.21.15.73:0/4270743852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8410098d10 msgr2=0x7f84100c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8410098d10 0x7f84100c7760 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 rx=0x7f8400009e60 tx=0x7f8400009780).stop 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 -- 172.21.15.73:0/4270743852 shutdown_connections 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8410098d10 0x7f84100c7760 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 --2- 172.21.15.73:0/4270743852 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f84100c6190 0x7f8410063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.699+0000 7f8415e5f700 1 -- 172.21.15.73:0/4270743852 >> 172.21.15.73:0/4270743852 conn(0x7f84100cb310 msgr2=0x7f84100cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.700+0000 7f8415e5f700 1 -- 172.21.15.73:0/4270743852 shutdown_connections 2023-11-28T04:24:39.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.700+0000 7f8415e5f700 1 -- 172.21.15.73:0/4270743852 wait complete. 2023-11-28T04:24:39.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.701+0000 7f8415e5f700 1 Processor -- start 2023-11-28T04:24:39.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.701+0000 7f8415e5f700 1 -- start start 2023-11-28T04:24:39.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.701+0000 7f8415e5f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8410064350 0x7f8410154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8410098d10 0x7f84101552b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84101557a0 con 0x7f84100c6190 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84100acd70 con 0x7f8410098d10 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84100ac160 con 0x7f8410064350 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41924/0 (socket says 172.21.15.73:41924) 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 -- 172.21.15.73:0/77527025 learned_addr learned my addr 172.21.15.73:0/77527025 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840effd700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8410098d10 0x7f84101552b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840f7fe700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8410064350 0x7f8410154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 -- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8410064350 msgr2=0x7f8410154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8410064350 0x7f8410154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 -- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8410098d10 msgr2=0x7f84101552b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8410098d10 0x7f84101552b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 -- 172.21.15.73:0/77527025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8400009a20 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f840ffff700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f840800c440 tx=0x7f8408007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f83f77fe700 1 -- 172.21.15.73:0/77527025 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080070d0 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f83f77fe700 1 -- 172.21.15.73:0/77527025 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8408007230 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f83f77fe700 1 -- 172.21.15.73:0/77527025 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84080075f0 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84100ac440 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.702+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84100aca00 con 0x7f84100c6190 2023-11-28T04:24:39.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.704+0000 7f83f77fe700 1 -- 172.21.15.73:0/77527025 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f84080059b0 con 0x7f84100c6190 2023-11-28T04:24:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.705+0000 7f83f77fe700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.705+0000 7f83f77fe700 1 -- 172.21.15.73:0/77527025 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8408079860 con 0x7f84100c6190 2023-11-28T04:24:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.705+0000 7f840f7fe700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.705+0000 7f840f7fe700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 0x7f83fc061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f84100c53b0 tx=0x7f8400009780).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.706+0000 7f84277f7b80 1 Processor -- start 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.706+0000 7f84277f7b80 1 -- start start 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b3feed90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3fc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3f8c060 con 0x5588b400fea0 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3f031f0 con 0x5588b3d82e40 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588b3f18440 con 0x5588b3fc6cb0 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f840effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f840effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41940/0 (socket says 172.21.15.73:41940) 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.707+0000 7f840effd700 1 -- 172.21.15.73:0/1032470342 learned_addr learned my addr 172.21.15.73:0/1032470342 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 -- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 msgr2=0x5588b3fc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3fc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840f7fe700 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b3feed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 -- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 msgr2=0x5588b3feed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b3feed90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 -- 172.21.15.73:0/1032470342 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588b3f11620 con 0x5588b400fea0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f840effd700 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f840400b920 tx=0x7f8404008040).ready entity=mon.0 client_cookie=35ece117cda9138d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f83f4ff9700 1 -- 172.21.15.73:0/1032470342 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400be60 con 0x5588b400fea0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f83f4ff9700 1 -- 172.21.15.73:0/1032470342 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400b530 con 0x5588b400fea0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f83f4ff9700 1 -- 172.21.15.73:0/1032470342 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404010b00 con 0x5588b400fea0 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f84277f7b80 1 -- 172.21.15.73:0/1032470342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 msgr2=0x5588b3fc67c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.708+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 rx=0x7f840400b920 tx=0x7f8404008040).stop 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 -- 172.21.15.73:0/1032470342 shutdown_connections 2023-11-28T04:24:39.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3fc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b3feed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1032470342 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fc67c0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 -- 172.21.15.73:0/1032470342 >> 172.21.15.73:0/1032470342 conn(0x5588b3e304a0 msgr2=0x5588b400f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 -- 172.21.15.73:0/1032470342 shutdown_connections 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 -- 172.21.15.73:0/1032470342 wait complete. 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 -- start start 2023-11-28T04:24:39.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.709+0000 7f84277f7b80 1 Processor -- start 2023-11-28T04:24:39.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3d82e40 0x5588b40068b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 0x5588b3ff3150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3f18580 con 0x5588b400fea0 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3f18800 con 0x5588b3fc6cb0 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588b3ee3e10 con 0x5588b3d82e40 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f84277f7b80 10 client.0 ll_register_callbacks cb 0x5588b3f86460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41942/0 (socket says 172.21.15.73:41942) 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 -- 172.21.15.73:0/1911462783 learned_addr learned my addr 172.21.15.73:0/1911462783 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840f7fe700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3d82e40 0x5588b40068b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.717 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840effd700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 0x5588b3ff3150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 -- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3d82e40 msgr2=0x5588b40068b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3d82e40 0x5588b40068b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 -- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 msgr2=0x5588b3ff3150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 0x5588b3ff3150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 -- 172.21.15.73:0/1911462783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f840400e040 con 0x5588b400fea0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840effd700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 0x5588b3ff3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f840ffff700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 rx=0x7f840804ac30 tx=0x7f840800f000).ready entity=mon.0 client_cookie=13e6bd7a53a8cafa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.710+0000 7f83edffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8408007d10 con 0x5588b400fea0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f8408005e10 con 0x5588b400fea0 2023-11-28T04:24:39.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5588b3fba720 con 0x5588b400fea0 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f840804a520 con 0x5588b400fea0 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 10 client.9675 fetch_fsmap learned FSMap version 70 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5588b3f0c9c0 con 0x5588b400fea0 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5588b3fba720 con 0x5588b400fea0 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f840804a680 con 0x5588b400fea0 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 10 client.9675 fetch_fsmap finished waiting for FSMap version 70 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 10 client.9675 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 20 client.9675 populate_metadata read hostname 'smithi073' 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b3fc2560 0x5588b3ff4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 4 client.9675 mds_command: new command op to 9522 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 --> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5588b3d7eca0 con 0x5588b3fc2560 2023-11-28T04:24:39.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f840f7fe700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b3fc2560 0x5588b3ff4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f83edffb700 10 client.9675 ms_handle_connect on v2:172.21.15.73:6835/1535793255 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.711+0000 7f840f7fe700 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b3fc2560 0x5588b3ff4200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=1dc0399a770b1654 server_cookie=7cee11eb6ffe78f7 in_seq=0 out_seq=0 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f83edffb700 1 -- 172.21.15.73:0/1911462783 <== mds.0 v2:172.21.15.73:6835/1535793255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5588b3d7eca0 con 0x5588b3fc2560 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f83edffb700 10 client.9675 handle_command_reply: tid=0 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 client.9675 shutdown 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 msgr2=0x5588b3fec2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 rx=0x7f840804ac30 tx=0x7f840800f000).stop 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 shutdown_connections 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b400fea0 0x5588b3fec2f0 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b3fc2560 0x5588b3ff4200 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3d82e40 0x5588b40068b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1911462783 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3fc6cb0 0x5588b3ff3150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 >> 172.21.15.73:0/1911462783 conn(0x5588b3e304a0 msgr2=0x5588b3fc7aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f83edffb700 0 client.9675 ms_handle_reset on v2:172.21.15.73:6835/1535793255 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 shutdown_connections 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 1 -- 172.21.15.73:0/1911462783 wait complete. 2023-11-28T04:24:39.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.712+0000 7f84277f7b80 20 client.9675 trim_cache size 0 max 16384 2023-11-28T04:24:39.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 Processor -- start 2023-11-28T04:24:39.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 -- start start 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b400b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b400fea0 0x5588b400c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3fee180 con 0x5588b3fc6cb0 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b40090e0 con 0x5588b3d82e40 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588b400cd10 con 0x5588b400fea0 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.729+0000 7f840effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41946/0 (socket says 172.21.15.73:41946) 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 -- 172.21.15.73:0/1456960389 learned_addr learned my addr 172.21.15.73:0/1456960389 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840ffff700 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b400fea0 0x5588b400c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840f7fe700 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b400b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 -- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b400fea0 msgr2=0x5588b400c820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b400fea0 0x5588b400c820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 -- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 msgr2=0x5588b400b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b400b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 -- 172.21.15.73:0/1456960389 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5588b3fba340 con 0x5588b3fc6cb0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f840effd700 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f8404010530 tx=0x7f8404008040).ready entity=mon.0 client_cookie=6723da08a8f111ff server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f83f57fa700 1 -- 172.21.15.73:0/1456960389 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8404024070 con 0x5588b3fc6cb0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f83f57fa700 1 -- 172.21.15.73:0/1456960389 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8404010eb0 con 0x5588b3fc6cb0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f83f57fa700 1 -- 172.21.15.73:0/1456960389 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840400bd90 con 0x5588b3fc6cb0 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 -- 172.21.15.73:0/1456960389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 msgr2=0x5588b400c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 rx=0x7f8404010530 tx=0x7f8404008040).stop 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 -- 172.21.15.73:0/1456960389 shutdown_connections 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b400fea0 0x5588b400c820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b3d82e40 0x5588b400b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 --2- 172.21.15.73:0/1456960389 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3fc6cb0 0x5588b400c330 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.730+0000 7f84277f7b80 1 -- 172.21.15.73:0/1456960389 >> 172.21.15.73:0/1456960389 conn(0x5588b3e304a0 msgr2=0x5588b3fc46f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.731+0000 7f84277f7b80 1 -- 172.21.15.73:0/1456960389 shutdown_connections 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.731+0000 7f84277f7b80 1 -- 172.21.15.73:0/1456960389 wait complete. 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.731+0000 7f84277f7b80 1 -- start start 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.731+0000 7f84277f7b80 1 Processor -- start 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3ff2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 0x5588b4009da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3ff3100 con 0x5588b3d82e40 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5588b3ff3e60 con 0x5588b400fea0 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5588b401d970 con 0x5588b3fc6cb0 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f84277f7b80 10 client.0 ll_register_callbacks cb 0x5588b3f86460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41958/0 (socket says 172.21.15.73:41958) 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 -- 172.21.15.73:0/2958629750 learned_addr learned my addr 172.21.15.73:0/2958629750 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840ffff700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 0x5588b4009da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.740 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840effd700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3ff2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 -- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 msgr2=0x5588b3ff2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3ff2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 -- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 msgr2=0x5588b4009da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 0x5588b4009da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 -- 172.21.15.73:0/2958629750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5588b3f11620 con 0x5588b3d82e40 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840ffff700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 0x5588b4009da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f840f7fe700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7f8400011000 tx=0x7f8400017000).ready entity=mon.0 client_cookie=360a38e6da13629 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f83edffb700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.732+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f840002ee50 con 0x5588b3d82e40 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f840002a420 con 0x5588b3d82e40 2023-11-28T04:24:39.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f840400b530 con 0x5588b3d82e40 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) v2 ==== 24+0+0 (secure 0 0 0) 0x7f840002a7f0 con 0x5588b3d82e40 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 10 client.9681 fetch_fsmap learned FSMap version 70 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) v3 -- 0x5588b3f0c710 con 0x5588b3d82e40 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f840400b530 con 0x5588b3d82e40 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 70) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f840002a950 con 0x5588b3d82e40 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 10 client.9681 fetch_fsmap finished waiting for FSMap version 70 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 10 client.9681 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 20 client.9681 populate_metadata read hostname 'smithi073' 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b400a290 0x5588b400a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 4 client.9681 mds_command: new command op to 9522 tid=0[{"prefix": "status"}] 2023-11-28T04:24:39.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 --> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5588b3db5ca0 con 0x5588b400a290 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f840effd700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b400a290 0x5588b400a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 10 client.9681 ms_handle_connect on v2:172.21.15.73:6835/1535793255 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f840effd700 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b400a290 0x5588b400a670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=e020b1fc848b6086 server_cookie=aa57ba59a1ba664a in_seq=0 out_seq=0 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 1 -- 172.21.15.73:0/2958629750 <== mds.0 v2:172.21.15.73:6835/1535793255 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5588b3db5ca0 con 0x5588b400a290 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f83edffb700 10 client.9681 handle_command_reply: tid=0 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 client.9681 shutdown 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 msgr2=0x5588b3ff2720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.733+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 rx=0x7f8400011000 tx=0x7f8400017000).stop 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 shutdown_connections 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5588b400fea0 0x5588b4009da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5588b3d82e40 0x5588b3ff2720 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.73:6835/1535793255,v1:172.21.15.73:6837/1535793255] conn(0x5588b400a290 0x5588b400a670 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 --2- 172.21.15.73:0/2958629750 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5588b3fc6cb0 0x5588b3ff2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 >> 172.21.15.73:0/2958629750 conn(0x5588b3e304a0 msgr2=0x5588b400f6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f83edffb700 0 client.9681 ms_handle_reset on v2:172.21.15.73:6835/1535793255 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 shutdown_connections 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 1 -- 172.21.15.73:0/2958629750 wait complete. 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f84277f7b80 20 client.9681 trim_cache size 0 max 16384 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T04:24:39.744 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 9522, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 4.6469568470000002, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 70, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 197, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 197, 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 9.1624582350000008 2023-11-28T04:24:39.745 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 msgr2=0x7f83fc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 0x7f83fc061610 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f84100c53b0 tx=0x7f8400009780).stop 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 msgr2=0x7f8410164870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 rx=0x7f840800c440 tx=0x7f8408007bb0).stop 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 shutdown_connections 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f83fc05f150 0x7f83fc061610 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f84100c6190 0x7f8410164870 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8410064350 0x7f8410154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 --2- 172.21.15.73:0/77527025 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8410098d10 0x7f84101552b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 >> 172.21.15.73:0/77527025 conn(0x7f84100cb310 msgr2=0x7f84100aa4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 shutdown_connections 2023-11-28T04:24:39.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.734+0000 7f8415e5f700 1 -- 172.21.15.73:0/77527025 wait complete. 2023-11-28T04:24:39.748 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T04:24:39.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.847+0000 7f3c2308f700 1 Processor -- start 2023-11-28T04:24:39.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.847+0000 7f3c2308f700 1 -- start start 2023-11-28T04:24:39.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2308f700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3c1c0ca740 0x7f3c1c0cab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2308f700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c1c0c1040 0x7f3c1c0aa540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2308f700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c1c0a9700 con 0x7f3c1c0aa0d0 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2308f700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c154ae0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2308f700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c1c0a9b90 con 0x7f3c1c0ca740 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c1c0c1040 0x7f3c1c0aa540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3c1c0c1040 0x7f3c1c0aa540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41960/0 (socket says 172.21.15.73:41960) 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2188c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3c1c0aa0d0 0x7f3c1c0c0930 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40798/0 (socket says 172.21.15.73:40798) 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 -- 172.21.15.73:0/683420257 learned_addr learned my addr 172.21.15.73:0/683420257 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2208d700 1 --2- 172.21.15.73:0/683420257 >> v2:172.21.15.150:3300/0 conn(0x7f3c1c0ca740 0x7f3c1c0cab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 -- 172.21.15.73:0/683420257 >> v2:172.21.15.150:3300/0 conn(0x7f3c1c0ca740 msgr2=0x7f3c1c0cab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:39.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 --2- 172.21.15.73:0/683420257 >> v2:172.21.15.150:3300/0 conn(0x7f3c1c0ca740 0x7f3c1c0cab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 -- 172.21.15.73:0/683420257 >> v1:172.21.15.73:6789/0 conn(0x7f3c1c0aa0d0 legacy=0x7f3c1c0c0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2288e700 1 -- 172.21.15.73:0/683420257 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c1c0a8af0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.848+0000 7f3c2108b700 1 -- 172.21.15.73:0/683420257 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1760780327 0 0) 0x7f3c1c0a9700 con 0x7f3c1c0aa0d0 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2288e700 1 --2- 172.21.15.73:0/683420257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aa540 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 rx=0x7f3c1c084520 tx=0x7f3c18009840).ready entity=mon.0 client_cookie=186aec5345d76556 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2108b700 1 -- 172.21.15.73:0/683420257 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800e070 con 0x7f3c1c0c1040 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2108b700 1 -- 172.21.15.73:0/683420257 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c1800bae0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2108b700 1 -- 172.21.15.73:0/683420257 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c1800be60 con 0x7f3c1c0c1040 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 -- 172.21.15.73:0/683420257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 msgr2=0x7f3c1c0aa540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 --2- 172.21.15.73:0/683420257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aa540 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 rx=0x7f3c1c084520 tx=0x7f3c18009840).stop 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 -- 172.21.15.73:0/683420257 shutdown_connections 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 --2- 172.21.15.73:0/683420257 >> v2:172.21.15.150:3300/0 conn(0x7f3c1c0ca740 0x7f3c1c0cab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 --2- 172.21.15.73:0/683420257 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aa540 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 -- 172.21.15.73:0/683420257 >> 172.21.15.73:0/683420257 conn(0x7f3c1c0946d0 msgr2=0x7f3c1c06a690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:39.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 -- 172.21.15.73:0/683420257 shutdown_connections 2023-11-28T04:24:39.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.849+0000 7f3c2308f700 1 -- 172.21.15.73:0/683420257 wait complete. 2023-11-28T04:24:39.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.850+0000 7f3c2308f700 1 Processor -- start 2023-11-28T04:24:39.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.850+0000 7f3c2308f700 1 -- start start 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c0aa0d0 0x7f3c1c151f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c1c0ca740 0x7f3c1c0ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c0c2b20 con 0x7f3c1c0c1040 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c0c2230 con 0x7f3c1c0aa0d0 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2308f700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c1c0c24d0 con 0x7f3c1c0ca740 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2188c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2188c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41966/0 (socket says 172.21.15.73:41966) 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.851+0000 7f3c2188c700 1 -- 172.21.15.73:0/1835375647 learned_addr learned my addr 172.21.15.73:0/1835375647 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2288e700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c1c0ca740 0x7f3c1c0ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2208d700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c0aa0d0 0x7f3c1c151f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.859 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 -- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c1c0ca740 msgr2=0x7f3c1c0ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c1c0ca740 0x7f3c1c0ae9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 -- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c0aa0d0 msgr2=0x7f3c1c151f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c0aa0d0 0x7f3c1c151f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 -- 172.21.15.73:0/1835375647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c18009ae0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2188c700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 rx=0x7f3c0c00c560 tx=0x7f3c0c007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0072e0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c0c007440 con 0x7f3c1c0c1040 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c0c0058f0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c1c0c27b0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.860 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.852+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c1c0af4f0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.853+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c1c0a8dd0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.853+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3c0c005ce0 con 0x7f3c1c0c1040 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.854+0000 7f3c127fc700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 0x7f3c00061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.854+0000 7f3c2208d700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 0x7f3c00061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.854+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f3c0c079a20 con 0x7f3c1c0c1040 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.854+0000 7f3c2208d700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 0x7f3c00061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3c14006fd0 tx=0x7f3c14008040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:39.861 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:39.858+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3c0c04acf0 con 0x7f3c1c0c1040 2023-11-28T04:24:40.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.063+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f3c1c0c0640 con 0x7f3c1c0c1040 2023-11-28T04:24:40.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.064+0000 7f3c127fc700 1 -- 172.21.15.73:0/1835375647 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) v1 ==== 53+0+73 (secure 0 0 0) 0x7f3c0c04e310 con 0x7f3c1c0c1040 2023-11-28T04:24:40.065 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.065+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 msgr2=0x7f3c00061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.065+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 0x7f3c00061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7f3c14006fd0 tx=0x7f3c14008040).stop 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.065+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 msgr2=0x7f3c1c0aba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.065+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 rx=0x7f3c0c00c560 tx=0x7f3c0c007c90).stop 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 shutdown_connections 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3c0005f150 0x7f3c00061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3c1c0ca740 0x7f3c1c0ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3c1c0aa0d0 0x7f3c1c151f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.067 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 --2- 172.21.15.73:0/1835375647 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3c1c0c1040 0x7f3c1c0aba60 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 >> 172.21.15.73:0/1835375647 conn(0x7f3c1c0946d0 msgr2=0x7f3c1c0c6f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 shutdown_connections 2023-11-28T04:24:40.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.066+0000 7f3c2308f700 1 -- 172.21.15.73:0/1835375647 wait complete. 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T04:24:40.078 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:24:40.079 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.079 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.095 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 37000sec preferred_lft 37000sec 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T04:24:40.096 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T04:24:40.097 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:24:40.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.098 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T04:24:40.158 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T04:24:40.160 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.160 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T04:24:40.216 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T04:24:40.216 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T04:24:40.217 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T04:24:40.217 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T04:24:40.217 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T04:24:40.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.217 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T04:24:40.217 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T04:24:40.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T04:24:40.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T04:24:40.217 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T04:24:40.218 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T04:24:40.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.317 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:24:40.331 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T04:24:40.331 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T04:24:40.332 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:24:40.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.332 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T04:24:40.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.396 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T04:24:40.450 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T04:24:40.451 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:24:40.529 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:24:40.529 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:24:40.554 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:24:40.555 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:24:40.557 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T04:24:40.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:24:40.557 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T04:24:40.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T04:24:40.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.776+0000 7f292f411700 1 Processor -- start 2023-11-28T04:24:40.777 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.776+0000 7f292f411700 1 -- start start 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292f411700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2928098d10 0x7f29280c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292f411700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2928158320 0x7f29281587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292f411700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2928158cc0 con 0x7f29280c6190 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292f411700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29280c72d0 con 0x7f2928158320 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292f411700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2928064570 con 0x7f2928098d10 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292d9ae700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2928158320 0x7f29281587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292c9ac700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f29280c6190 0x7f2928063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40802/0 (socket says 172.21.15.73:40802) 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292c9ac700 1 -- 172.21.15.73:0/1889820117 learned_addr learned my addr 172.21.15.73:0/1889820117 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.777+0000 7f292d1ad700 1 --2- 172.21.15.73:0/1889820117 >> v2:172.21.15.150:3300/0 conn(0x7f2928098d10 0x7f29280c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292d9ae700 1 -- 172.21.15.73:0/1889820117 >> v2:172.21.15.150:3300/0 conn(0x7f2928098d10 msgr2=0x7f29280c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:40.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292d9ae700 1 --2- 172.21.15.73:0/1889820117 >> v2:172.21.15.150:3300/0 conn(0x7f2928098d10 0x7f29280c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f291ffff700 1 -- 172.21.15.73:0/1889820117 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2010694063 0 0) 0x7f2918001110 con 0x7f29280c6190 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292d9ae700 1 -- 172.21.15.73:0/1889820117 >> v1:172.21.15.73:6789/0 conn(0x7f29280c6190 legacy=0x7f2928063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292d9ae700 1 -- 172.21.15.73:0/1889820117 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2928154bc0 con 0x7f2928158320 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292d9ae700 1 --2- 172.21.15.73:0/1889820117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928158320 0x7f29281587d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7f2928083f80 tx=0x7f2920009060).ready entity=mon.0 client_cookie=21ddef065b78532 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f291ffff700 1 -- 172.21.15.73:0/1889820117 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000f510 con 0x7f2928158320 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f291ffff700 1 -- 172.21.15.73:0/1889820117 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292000f670 con 0x7f2928158320 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f291ffff700 1 -- 172.21.15.73:0/1889820117 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292000f9f0 con 0x7f2928158320 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 -- 172.21.15.73:0/1889820117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928158320 msgr2=0x7f29281587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 --2- 172.21.15.73:0/1889820117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928158320 0x7f29281587d0 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 rx=0x7f2928083f80 tx=0x7f2920009060).stop 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 -- 172.21.15.73:0/1889820117 shutdown_connections 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 --2- 172.21.15.73:0/1889820117 >> v2:172.21.15.150:3300/0 conn(0x7f2928098d10 0x7f29280c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.781 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 --2- 172.21.15.73:0/1889820117 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928158320 0x7f29281587d0 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.778+0000 7f292f411700 1 -- 172.21.15.73:0/1889820117 >> 172.21.15.73:0/1889820117 conn(0x7f29280cb310 msgr2=0x7f29280cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:40.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.779+0000 7f292f411700 1 -- 172.21.15.73:0/1889820117 shutdown_connections 2023-11-28T04:24:40.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.779+0000 7f292f411700 1 -- 172.21.15.73:0/1889820117 wait complete. 2023-11-28T04:24:40.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 Processor -- start 2023-11-28T04:24:40.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 -- start start 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 0x7f29280aec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2928158320 0x7f2928164b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29280b0160 con 0x7f2928098d10 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2928098360 con 0x7f29280c6190 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.780+0000 7f292f411700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29280935b0 con 0x7f2928158320 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:41994/0 (socket says 172.21.15.73:41994) 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 -- 172.21.15.73:0/1804717707 learned_addr learned my addr 172.21.15.73:0/1804717707 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292c9ac700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 0x7f29280aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d9ae700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2928158320 0x7f2928164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 -- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2928158320 msgr2=0x7f2928164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2928158320 0x7f2928164b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 -- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 msgr2=0x7f29280aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 0x7f29280aec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 -- 172.21.15.73:0/1804717707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29200092a0 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292c9ac700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 0x7f29280aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292d1ad700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 rx=0x7f2924009ef0 tx=0x7f292400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f292400cc50 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f292400f040 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29280af8c0 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2924013400 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.781+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29280afe80 con 0x7f2928098d10 2023-11-28T04:24:40.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.782+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29280af640 con 0x7f2928098d10 2023-11-28T04:24:40.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.783+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2924013770 con 0x7f2928098d10 2023-11-28T04:24:40.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.783+0000 7f291d7fa700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 0x7f290c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:40.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.783+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f292407ab60 con 0x7f2928098d10 2023-11-28T04:24:40.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.783+0000 7f292c9ac700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 0x7f290c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:40.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.783+0000 7f292c9ac700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 0x7f290c0615e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2918007fd0 tx=0x7f2918009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:40.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.787+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f292404be30 con 0x7f2928098d10 2023-11-28T04:24:40.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.974+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f292806c860 con 0x7f290c05f120 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.975+0000 7f291d7fa700 1 -- 172.21.15.73:0/1804717707 <== mgr.9345 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f292806c860 con 0x7f290c05f120 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:24:40.976 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 msgr2=0x7f290c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 0x7f290c0615e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f2918007fd0 tx=0x7f2918009040).stop 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 msgr2=0x7f2928148b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 rx=0x7f2924009ef0 tx=0x7f292400b040).stop 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 shutdown_connections 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f290c05f120 0x7f290c0615e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2928158320 0x7f2928164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2928098d10 0x7f2928148b20 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 --2- 172.21.15.73:0/1804717707 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29280c6190 0x7f29280aec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:40.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 >> 172.21.15.73:0/1804717707 conn(0x7f29280cb310 msgr2=0x7f2928063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:40.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 shutdown_connections 2023-11-28T04:24:40.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:40.977+0000 7f292f411700 1 -- 172.21.15.73:0/1804717707 wait complete. 2023-11-28T04:24:40.990 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T04:24:41.014 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:24:41.013+0000 7f1eb84a5700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:24:41.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.104+0000 7eff9a7a6700 1 Processor -- start 2023-11-28T04:24:41.105 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.104+0000 7eff9a7a6700 1 -- start start 2023-11-28T04:24:41.106 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff9a7a6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7eff94154950 0x7eff940ca540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff9a7a6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff940c7bc0 con 0x7eff940c66f0 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff9a7a6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff94155920 con 0x7eff9414e530 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff9a7a6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff940c6ba0 con 0x7eff94154950 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff98fa3700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7eff940c66f0 0x7eff94158570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40804/0 (socket says 172.21.15.73:40804) 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff98fa3700 1 -- 172.21.15.73:0/1193728419 learned_addr learned my addr 172.21.15.73:0/1193728419 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff99fa5700 1 --2- 172.21.15.73:0/1193728419 >> v2:172.21.15.73:3300/0 conn(0x7eff94154950 0x7eff940ca540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff93fff700 1 -- 172.21.15.73:0/1193728419 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863038297 0 0) 0x7eff940c7bc0 con 0x7eff940c66f0 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff93fff700 1 -- 172.21.15.73:0/1193728419 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff7c0033f0 con 0x7eff940c66f0 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff99fa5700 1 -- 172.21.15.73:0/1193728419 >> v1:172.21.15.150:6789/0 conn(0x7eff9414e530 legacy=0x7eff9414e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff99fa5700 1 -- 172.21.15.73:0/1193728419 >> v1:172.21.15.73:6789/0 conn(0x7eff940c66f0 legacy=0x7eff94158570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff99fa5700 1 -- 172.21.15.73:0/1193728419 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94154de0 con 0x7eff94154950 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.105+0000 7eff99fa5700 1 --2- 172.21.15.73:0/1193728419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940ca540 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7eff8c009760 tx=0x7eff8c009060).ready entity=mon.0 client_cookie=a832bc669515a2d9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:41.107 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff93fff700 1 -- 172.21.15.73:0/1193728419 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00f510 con 0x7eff94154950 2023-11-28T04:24:41.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff93fff700 1 -- 172.21.15.73:0/1193728419 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00f670 con 0x7eff94154950 2023-11-28T04:24:41.108 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff93fff700 1 -- 172.21.15.73:0/1193728419 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00f9f0 con 0x7eff94154950 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 -- 172.21.15.73:0/1193728419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 msgr2=0x7eff940ca540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/1193728419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940ca540 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 rx=0x7eff8c009760 tx=0x7eff8c009060).stop 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 -- 172.21.15.73:0/1193728419 shutdown_connections 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/1193728419 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940ca540 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 -- 172.21.15.73:0/1193728419 >> 172.21.15.73:0/1193728419 conn(0x7eff94064990 msgr2=0x7eff94064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 -- 172.21.15.73:0/1193728419 shutdown_connections 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.106+0000 7eff9a7a6700 1 -- 172.21.15.73:0/1193728419 wait complete. 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.107+0000 7eff9a7a6700 1 Processor -- start 2023-11-28T04:24:41.109 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.107+0000 7eff9a7a6700 1 -- start start 2023-11-28T04:24:41.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 0x7eff940c2630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:41.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9414e530 0x7eff94163980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:41.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:41.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940c47c0 con 0x7eff94154950 2023-11-28T04:24:41.112 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff940c4aa0 con 0x7eff9414e530 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff9a7a6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff940c4d80 con 0x7eff940c66f0 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff99fa5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff99fa5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42012/0 (socket says 172.21.15.73:42012) 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.108+0000 7eff99fa5700 1 -- 172.21.15.73:0/826881414 learned_addr learned my addr 172.21.15.73:0/826881414 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff98fa3700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9414e530 0x7eff94163980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff997a4700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 0x7eff940c2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 -- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 msgr2=0x7eff940c2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 0x7eff940c2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 -- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9414e530 msgr2=0x7eff94163980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9414e530 0x7eff94163980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 -- 172.21.15.73:0/826881414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff8c0092a0 con 0x7eff94154950 2023-11-28T04:24:41.113 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff997a4700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 0x7eff940c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff99fa5700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7eff8c009e00 tx=0x7eff8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00f9f0 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c01b440 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff940c5060 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c01b800 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.109+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff94163e70 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.110+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff94062ee0 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.111+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 88) v1 ==== 75446+0+0 (secure 0 0 0) 0x7eff8c01bbf0 con 0x7eff94154950 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.111+0000 7eff917fa700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 0x7eff80061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:41.114 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.111+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7eff8c07ba00 con 0x7eff94154950 2023-11-28T04:24:41.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.111+0000 7eff997a4700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 0x7eff80061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:41.115 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.111+0000 7eff997a4700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 0x7eff80061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7eff84001e30 tx=0x7eff84009040).ready entity=mgr.9345 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:41.117 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.116+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7eff8c04ccd0 con 0x7eff94154950 2023-11-28T04:24:41.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:41.323+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7eff9406c7c0 con 0x7eff94154950 2023-11-28T04:24:42.122 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.121+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 89) v1 ==== 75463+0+0 (secure 0 0 0) 0x7eff8c0797d0 con 0x7eff94154950 2023-11-28T04:24:42.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.122+0000 7eff917fa700 1 -- 172.21.15.73:0/826881414 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v89) v1 ==== 92+0+0 (secure 0 0 0) 0x7eff8c042940 con 0x7eff94154950 2023-11-28T04:24:42.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 msgr2=0x7eff80061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 0x7eff80061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7eff84001e30 tx=0x7eff84009040).stop 2023-11-28T04:24:42.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 msgr2=0x7eff940c3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 rx=0x7eff8c009e00 tx=0x7eff8c017000).stop 2023-11-28T04:24:42.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 shutdown_connections 2023-11-28T04:24:42.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7eff8005f150 0x7eff80061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7eff94154950 0x7eff940c3e00 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7eff940c66f0 0x7eff940c2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 --2- 172.21.15.73:0/826881414 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7eff9414e530 0x7eff94163980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 >> 172.21.15.73:0/826881414 conn(0x7eff94064990 msgr2=0x7eff94157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:42.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 shutdown_connections 2023-11-28T04:24:42.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.124+0000 7eff9a7a6700 1 -- 172.21.15.73:0/826881414 wait complete. 2023-11-28T04:24:42.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T04:24:42.268 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:42.267+0000 7f0501232680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:24:42.272 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:42.271+0000 7fcdec259680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:24:42.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.301+0000 7f4c3ac9d700 1 Processor -- start 2023-11-28T04:24:42.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.301+0000 7f4c3ac9d700 1 -- start start 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3ac9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3ac9d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c340c72d0 con 0x7f4c34158320 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3ac9d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c34064570 con 0x7f4c34098d10 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3ac9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34154bc0 con 0x7f4c340c6190 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3923a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f4c34158320 0x7f4c340a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40820/0 (socket says 172.21.15.73:40820) 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c3923a700 1 -- 172.21.15.73:0/1736591346 learned_addr learned my addr 172.21.15.73:0/1736591346 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.302+0000 7f4c33fff700 1 --2- 172.21.15.73:0/1736591346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 683482367 0 0) 0x7f4c340c72d0 con 0x7f4c34158320 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c240033f0 con 0x7f4c34158320 2023-11-28T04:24:42.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2836305898 0 0) 0x7f4c34064570 con 0x7f4c34098d10 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c340c72d0 con 0x7f4c34098d10 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c33fff700 1 -- 172.21.15.73:0/1736591346 >> v1:172.21.15.150:6789/0 conn(0x7f4c34098d10 legacy=0x7f4c34063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c33fff700 1 -- 172.21.15.73:0/1736591346 >> v1:172.21.15.73:6789/0 conn(0x7f4c34158320 legacy=0x7f4c340a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c33fff700 1 -- 172.21.15.73:0/1736591346 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c340c66c0 con 0x7f4c340c6190 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c33fff700 1 --2- 172.21.15.73:0/1736591346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c340c7760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4c20009790 tx=0x7f4c20009060).ready entity=mon.2 client_cookie=eb0a44f2b243c8d3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000f510 con 0x7f4c340c6190 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2000f670 con 0x7f4c340c6190 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.303+0000 7f4c337fe700 1 -- 172.21.15.73:0/1736591346 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2000f9f0 con 0x7f4c340c6190 2023-11-28T04:24:42.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/1736591346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 msgr2=0x7f4c340c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/1736591346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c340c7760 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 rx=0x7f4c20009790 tx=0x7f4c20009060).stop 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/1736591346 shutdown_connections 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/1736591346 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/1736591346 >> 172.21.15.73:0/1736591346 conn(0x7f4c340cb310 msgr2=0x7f4c340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/1736591346 shutdown_connections 2023-11-28T04:24:42.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.304+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/1736591346 wait complete. 2023-11-28T04:24:42.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.305+0000 7f4c3ac9d700 1 Processor -- start 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.305+0000 7f4c3ac9d700 1 -- start start 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c341484a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34148990 con 0x7f4c34158320 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34156c30 con 0x7f4c34098d10 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3ac9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34156020 con 0x7f4c340c6190 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42038/0 (socket says 172.21.15.73:42038) 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c33fff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 -- 172.21.15.73:0/2060259749 learned_addr learned my addr 172.21.15.73:0/2060259749 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c38a39700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c341484a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 -- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 msgr2=0x7f4c34149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 -- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 msgr2=0x7f4c341484a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c341484a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 -- 172.21.15.73:0/2060259749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c200092a0 con 0x7f4c34158320 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c33fff700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c3923a700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 rx=0x7f4c2c001560 tx=0x7f4c2c0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c005d30 con 0x7f4c34158320 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.306+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c005e90 con 0x7f4c34158320 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.307+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34156300 con 0x7f4c34158320 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.307+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c007300 con 0x7f4c34158320 2023-11-28T04:24:42.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.307+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c341568c0 con 0x7f4c34158320 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.307+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c3406c860 con 0x7f4c34158320 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.308+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 89) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f4c2c0076b0 con 0x7f4c34158320 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.308+0000 7f4c30ff9700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 0x7f4c18061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.308+0000 7f4c38a39700 1 -- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 msgr2=0x7f4c18061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.308+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4c2c079be0 con 0x7f4c34158320 2023-11-28T04:24:42.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.308+0000 7f4c38a39700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 0x7f4c18061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:42.313 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.312+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c2c04aeb0 con 0x7f4c34158320 2023-11-28T04:24:42.335 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:42.334+0000 7f0501232680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:24:42.335 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:42.335+0000 7fcdec259680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:24:42.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.484+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f4c34156d70 con 0x7f4c34158320 2023-11-28T04:24:42.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.488+0000 7f4c30ff9700 1 -- 172.21.15.73:0/2060259749 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v39)=0 v39) v1 ==== 138+0+0 (secure 0 0 0) 0x7f4c2c04e4d0 con 0x7f4c34158320 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 msgr2=0x7f4c18061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 0x7f4c18061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 msgr2=0x7f4c34153ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 rx=0x7f4c2c001560 tx=0x7f4c2c0015b0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 shutdown_connections 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c1805f1a0 0x7f4c18061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c34153ab0 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c341484a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 --2- 172.21.15.73:0/2060259749 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 >> 172.21.15.73:0/2060259749 conn(0x7f4c340cb310 msgr2=0x7f4c34148d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 shutdown_connections 2023-11-28T04:24:42.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.490+0000 7f4c3ac9d700 1 -- 172.21.15.73:0/2060259749 wait complete. 2023-11-28T04:24:42.503 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:42.503 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T04:24:42.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-28T04:24:42.611 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:42.610+0000 7fcdec259680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:24:42.623 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:42.622+0000 7f0501232680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:24:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.685+0000 7fc26c302700 1 Processor -- start 2023-11-28T04:24:42.686 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.685+0000 7fc26c302700 1 -- start start 2023-11-28T04:24:42.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26c302700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc2641373a0 0x7fc2640c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26c302700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc264158e20 con 0x7fc264064b00 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26c302700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2640677c0 con 0x7fc2640c5f30 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26c302700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2641582a0 con 0x7fc2641373a0 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26a89f700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc264064b00 0x7fc2640670b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:40822/0 (socket says 172.21.15.73:40822) 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26a89f700 1 -- 172.21.15.73:0/3233773371 learned_addr learned my addr 172.21.15.73:0/3233773371 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3233773371 >> v2:172.21.15.150:3300/0 conn(0x7fc2641373a0 0x7fc2640c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 749141822 0 0) 0x7fc264158e20 con 0x7fc264064b00 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.686+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2500033f0 con 0x7fc264064b00 2023-11-28T04:24:42.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3191933629 0 0) 0x7fc2500033f0 con 0x7fc264064b00 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26a09e700 1 -- 172.21.15.73:0/3233773371 >> v1:172.21.15.150:6789/0 conn(0x7fc2640c5f30 legacy=0x7fc2640669a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26a09e700 1 -- 172.21.15.73:0/3233773371 >> v1:172.21.15.73:6789/0 conn(0x7fc264064b00 legacy=0x7fc2640670b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26a09e700 1 -- 172.21.15.73:0/3233773371 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc264158580 con 0x7fc2641373a0 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3233773371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640c72e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fc254009760 tx=0x7fc254009060).ready entity=mon.1 client_cookie=f7ced35f77f6c86e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.687+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc25c0017f0 con 0x7fc2641373a0 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc254011400 con 0x7fc2641373a0 2023-11-28T04:24:42.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26909c700 1 -- 172.21.15.73:0/3233773371 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc254011780 con 0x7fc2641373a0 2023-11-28T04:24:42.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 -- 172.21.15.73:0/3233773371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 msgr2=0x7fc2640c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 --2- 172.21.15.73:0/3233773371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640c72e0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 rx=0x7fc254009760 tx=0x7fc254009060).stop 2023-11-28T04:24:42.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 -- 172.21.15.73:0/3233773371 shutdown_connections 2023-11-28T04:24:42.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 --2- 172.21.15.73:0/3233773371 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640c72e0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 -- 172.21.15.73:0/3233773371 >> 172.21.15.73:0/3233773371 conn(0x7fc2640628f0 msgr2=0x7fc264062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 -- 172.21.15.73:0/3233773371 shutdown_connections 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.688+0000 7fc26c302700 1 -- 172.21.15.73:0/3233773371 wait complete. 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.689+0000 7fc26c302700 1 Processor -- start 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.689+0000 7fc26c302700 1 -- start start 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc264064b00 0x7fc264135910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640a8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640acef0 con 0x7fc2640c5f30 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc264168b50 con 0x7fc2641373a0 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26c302700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2640ac040 con 0x7fc264064b00 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26989d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26989d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:42040/0 (socket says 172.21.15.73:42040) 2023-11-28T04:24:42.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26989d700 1 -- 172.21.15.73:0/3411310376 learned_addr learned my addr 172.21.15.73:0/3411310376 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26a89f700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640a8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.690+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc264064b00 0x7fc264135910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc264064b00 msgr2=0x7fc264135910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc264064b00 0x7fc264135910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 msgr2=0x7fc2640a8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640a8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2540092a0 con 0x7fc2640c5f30 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26989d700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7fc26000c4c0 tx=0x7fc260007e00).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc26000ce70 con 0x7fc2640c5f30 2023-11-28T04:24:42.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc260001e90 con 0x7fc2640c5f30 2023-11-28T04:24:42.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc260007250 con 0x7fc2640c5f30 2023-11-28T04:24:42.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2640ac2e0 con 0x7fc2640c5f30 2023-11-28T04:24:42.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.691+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2640ac8a0 con 0x7fc2640c5f30 2023-11-28T04:24:42.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.692+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc26406c860 con 0x7fc2640c5f30 2023-11-28T04:24:42.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.693+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 89) v1 ==== 75463+0+0 (secure 0 0 0) 0x7fc2600066a0 con 0x7fc2640c5f30 2023-11-28T04:24:42.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.693+0000 7fc25a7fc700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:42.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.693+0000 7fc26a09e700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:42.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.693+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fc260079b10 con 0x7fc2640c5f30 2023-11-28T04:24:42.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.693+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:24:42.700 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.697+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc26004ade0 con 0x7fc2640c5f30 2023-11-28T04:24:42.894 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.894+0000 7fc26a09e700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:42.895 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.894+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:24:42.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:42.895+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc2640ad030 con 0x7fc24805f170 2023-11-28T04:24:43.231 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:43.230+0000 7fcdec259680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:24:43.253 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:43.254+0000 7f0501232680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:24:43.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:43.295+0000 7fc26a09e700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:43.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:43.295+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T04:24:43.303 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:43.302+0000 7fcdec259680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:24:43.326 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:43.326+0000 7f0501232680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:24:43.490 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:43.489+0000 7fcdec259680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:24:43.512 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:43.512+0000 7f0501232680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:24:43.909 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:43.908+0000 7fcdec259680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:24:43.931 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:43.931+0000 7f0501232680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:24:44.013 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.012+0000 7fcdec259680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:44.034 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.034+0000 7f0501232680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:44.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:44.096+0000 7fc26a09e700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:44.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:44.096+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T04:24:44.166 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.166+0000 7fcdec259680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:44.187 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.187+0000 7f0501232680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:44.243 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.240+0000 7fcdec259680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:24:44.262 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.262+0000 7f0501232680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:24:44.309 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.308+0000 7fcdec259680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:24:44.329 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.329+0000 7f0501232680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:24:44.567 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.566+0000 7fcdec259680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:24:44.586 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.586+0000 7f0501232680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:24:44.751 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:44.750+0000 7fcdec259680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:44.771 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:44.771+0000 7f0501232680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:24:45.404 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.404+0000 7fcdec259680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:24:45.421 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.422+0000 7f0501232680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:24:45.473 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.472+0000 7fcdec259680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:24:45.490 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.490+0000 7f0501232680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:24:45.549 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.548+0000 7fcdec259680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:24:45.566 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.566+0000 7f0501232680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:24:45.695 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.694+0000 7fcdec259680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:24:45.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:45.697+0000 7fc26a09e700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:24:45.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:45.697+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T04:24:45.715 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.715+0000 7f0501232680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:24:45.764 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.763+0000 7fcdec259680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:24:45.787 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.786+0000 7f0501232680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:24:45.880 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.878+0000 7fcdec259680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:24:45.902 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:45.902+0000 7f0501232680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:24:45.989 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:45.987+0000 7fcdec259680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:46.010 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.010+0000 7f0501232680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:24:46.281 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:46.280+0000 7fcdec259680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:24:46.301 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.301+0000 7f0501232680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:24:46.353 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:24:46.353+0000 7fcdec259680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:24:46.373 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.373+0000 7f0501232680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:24:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:46.392+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 90) v1 ==== 75287+0+0 (secure 0 0 0) 0x7fc2600778e0 con 0x7fc2640c5f30 2023-11-28T04:24:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:46.392+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc248061630 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:24:46.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:46.392+0000 7fc25a7fc700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc248061630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:46.454 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04751b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.454 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04751b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04751b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04751b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04751b7700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04789be700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04789be700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04789be700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04789be700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:46.455 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:24:46.454+0000 7f04789be700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:24:47.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.395+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fc260077c60 con 0x7fc2640c5f30 2023-11-28T04:24:47.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.395+0000 7fc25a7fc700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc2480615b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.395 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.395+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc24802fed0 con 0x7fc24805f170 2023-11-28T04:24:47.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.397+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc2480615b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.398+0000 7fc26a09e700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc2480615b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc254009760 tx=0x7fc254014000).ready entity=mgr.9717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.411 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.411+0000 7fc25a7fc700 1 -- 172.21.15.73:0/3411310376 <== mgr.9717 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc24802fed0 con 0x7fc24805f170 2023-11-28T04:24:47.411 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T04:24:47.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 msgr2=0x7fc2480615b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc2480615b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 rx=0x7fc254009760 tx=0x7fc254014000).stop 2023-11-28T04:24:47.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 msgr2=0x7fc2640ae890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 rx=0x7fc26000c4c0 tx=0x7fc260007e00).stop 2023-11-28T04:24:47.413 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 shutdown_connections 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc24805f170 0x7fc2480615b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2641373a0 0x7fc2640a8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc264064b00 0x7fc264135910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 --2- 172.21.15.73:0/3411310376 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2640c5f30 0x7fc2640ae890 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 >> 172.21.15.73:0/3411310376 conn(0x7fc2640628f0 msgr2=0x7fc2640be7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 shutdown_connections 2023-11-28T04:24:47.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.413+0000 7fc26c302700 1 -- 172.21.15.73:0/3411310376 wait complete. 2023-11-28T04:24:47.426 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T04:24:47.427 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-11-28T04:24:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.545+0000 7f9464129700 1 Processor -- start 2023-11-28T04:24:47.546 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.545+0000 7f9464129700 1 -- start start 2023-11-28T04:24:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9464129700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 0x7f945c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9464129700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 0x7f945c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9464129700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f945c158cc0 con 0x7f945c098d10 2023-11-28T04:24:47.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9464129700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f945c0c72d0 con 0x7f945c0c6190 2023-11-28T04:24:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9464129700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c064570 con 0x7f945c158320 2023-11-28T04:24:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9461ec5700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f945c098d10 0x7f945c063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:55802/0 (socket says 172.21.15.73:55802) 2023-11-28T04:24:47.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9461ec5700 1 -- 172.21.15.73:0/2227025721 learned_addr learned my addr 172.21.15.73:0/2227025721 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:24:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94616c4700 1 --2- 172.21.15.73:0/2227025721 >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 0x7f945c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94626c6700 1 --2- 172.21.15.73:0/2227025721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 0x7f945c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 79869151 0 0) 0x7f9458001490 con 0x7f945c098d10 2023-11-28T04:24:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f94480033f0 con 0x7f945c098d10 2023-11-28T04:24:47.553 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 470400267 0 0) 0x7f94480033f0 con 0x7f945c098d10 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94626c6700 1 -- 172.21.15.73:0/2227025721 >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 msgr2=0x7f945c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94626c6700 1 --2- 172.21.15.73:0/2227025721 >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 0x7f945c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94626c6700 1 -- 172.21.15.73:0/2227025721 >> v1:172.21.15.73:6789/0 conn(0x7f945c098d10 legacy=0x7f945c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94626c6700 1 -- 172.21.15.73:0/2227025721 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f945c154bc0 con 0x7f945c158320 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.546+0000 7f94616c4700 1 --2- 172.21.15.73:0/2227025721 >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 0x7f945c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f94626c6700 1 --2- 172.21.15.73:0/2227025721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 0x7f945c1587d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f9454009760 tx=0x7f9454009060).ready entity=mon.2 client_cookie=983db46f152d5cca server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9458001980 con 0x7f945c158320 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454011400 con 0x7f945c158320 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9460ec3700 1 -- 172.21.15.73:0/2227025721 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454011780 con 0x7f945c158320 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 -- 172.21.15.73:0/2227025721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 msgr2=0x7f945c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 --2- 172.21.15.73:0/2227025721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 0x7f945c1587d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 rx=0x7f9454009760 tx=0x7f9454009060).stop 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 -- 172.21.15.73:0/2227025721 shutdown_connections 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 --2- 172.21.15.73:0/2227025721 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c158320 0x7f945c1587d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 --2- 172.21.15.73:0/2227025721 >> v2:172.21.15.150:3300/0 conn(0x7f945c0c6190 0x7f945c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 -- 172.21.15.73:0/2227025721 >> 172.21.15.73:0/2227025721 conn(0x7f945c0cb310 msgr2=0x7f945c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 -- 172.21.15.73:0/2227025721 shutdown_connections 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.547+0000 7f9464129700 1 -- 172.21.15.73:0/2227025721 wait complete. 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.548+0000 7f9464129700 1 Processor -- start 2023-11-28T04:24:47.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.548+0000 7f9464129700 1 -- start start 2023-11-28T04:24:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f945c098d10 0x7f945c14d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c0c6190 0x7f945c159480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c159e60 con 0x7f945c158320 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f945c156360 con 0x7f945c098d10 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f9464129700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f945c14da10 con 0x7f945c0c6190 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f94626c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f94626c6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34470/0 (socket says 172.21.15.73:34470) 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.549+0000 7f94626c6700 1 -- 172.21.15.73:0/303165926 learned_addr learned my addr 172.21.15.73:0/303165926 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 -- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c0c6190 msgr2=0x7f945c159480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c0c6190 0x7f945c159480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 -- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f945c098d10 msgr2=0x7f945c14d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f945c098d10 0x7f945c14d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 -- 172.21.15.73:0/303165926 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f944c0092a0 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f94626c6700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7f9454010690 tx=0x7f9454016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9454010be0 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9454010d40 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94540092a0 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f945401d660 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.550+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f945c155a30 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.551+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f945c0635f0 con 0x7f945c158320 2023-11-28T04:24:47.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.552+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f94540263b0 con 0x7f945c158320 2023-11-28T04:24:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.552+0000 7f9451ffb700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 0x7f9440061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.552+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f945407b050 con 0x7f945c158320 2023-11-28T04:24:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.552+0000 7f9461ec5700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 0x7f9440061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.552+0000 7f9461ec5700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 0x7f9440061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9458001b30 tx=0x7f9458001670).ready entity=mgr.9717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.556+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f945404c3e0 con 0x7f945c158320 2023-11-28T04:24:47.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.745+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f945c1564d0 con 0x7f944005f080 2023-11-28T04:24:47.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.749+0000 7f9451ffb700 1 -- 172.21.15.73:0/303165926 <== mgr.9717 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f945c1564d0 con 0x7f944005f080 2023-11-28T04:24:47.750 INFO:teuthology.orchestra.run.smithi073.stdout:Retention added to path snap_test_dir1 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 msgr2=0x7f9440061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 0x7f9440061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f9458001b30 tx=0x7f9458001670).stop 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 msgr2=0x7f945c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 rx=0x7f9454010690 tx=0x7f9454016000).stop 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 shutdown_connections 2023-11-28T04:24:47.752 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f944005f080 0x7f9440061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f945c158320 0x7f945c159970 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f945c098d10 0x7f945c14d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 --2- 172.21.15.73:0/303165926 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f945c0c6190 0x7f945c159480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 >> 172.21.15.73:0/303165926 conn(0x7f945c0cb310 msgr2=0x7f945c0c17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 shutdown_connections 2023-11-28T04:24:47.753 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.751+0000 7f9464129700 1 -- 172.21.15.73:0/303165926 wait complete. 2023-11-28T04:24:47.764 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-11-28T04:24:47.764 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-28-04_25_00 in ~13s... 2023-11-28T04:24:47.764 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-11-28T04:24:47.765 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-28T04:24:47.880 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.879+0000 7fd45e06b700 1 Processor -- start 2023-11-28T04:24:47.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.879+0000 7fd45e06b700 1 -- start start 2023-11-28T04:24:47.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd458154530 0x7fd458154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd4580c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fd4580aa220 0x7fd4580aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4580c6390 con 0x7fd458154530 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4580c6630 con 0x7fd4580aa220 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd45e06b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580aaba0 con 0x7fd4580a7ff0 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd4577fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd458154530 0x7fd458154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd4577fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd458154530 0x7fd458154910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34480/0 (socket says 172.21.15.73:34480) 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd4577fe700 1 -- 172.21.15.73:0/3975838549 learned_addr learned my addr 172.21.15.73:0/3975838549 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:47.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd457fff700 1 --2- 172.21.15.73:0/3975838549 >> v2:172.21.15.150:3300/0 conn(0x7fd4580aa220 0x7fd4580aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.880+0000 7fd456ffd700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd4580c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 -- 172.21.15.73:0/3975838549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 msgr2=0x7fd4580c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd4580c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 -- 172.21.15.73:0/3975838549 >> v2:172.21.15.150:3300/0 conn(0x7fd4580aa220 msgr2=0x7fd4580aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3975838549 >> v2:172.21.15.150:3300/0 conn(0x7fd4580aa220 0x7fd4580aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.887 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 -- 172.21.15.73:0/3975838549 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4580a8470 con 0x7fd458154530 2023-11-28T04:24:47.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd456ffd700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd4580c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:24:47.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd458154530 0x7fd458154910 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 rx=0x7fd4580a9fc0 tx=0x7fd448009840).ready entity=mon.0 client_cookie=13104389d5437919 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4567fc700 1 -- 172.21.15.73:0/3975838549 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800e070 con 0x7fd458154530 2023-11-28T04:24:47.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4567fc700 1 -- 172.21.15.73:0/3975838549 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44800ba20 con 0x7fd458154530 2023-11-28T04:24:47.888 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd4567fc700 1 -- 172.21.15.73:0/3975838549 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44800bda0 con 0x7fd458154530 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 -- 172.21.15.73:0/3975838549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd458154530 msgr2=0x7fd458154910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd458154530 0x7fd458154910 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 rx=0x7fd4580a9fc0 tx=0x7fd448009840).stop 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 -- 172.21.15.73:0/3975838549 shutdown_connections 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd458154530 0x7fd458154910 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3975838549 >> v2:172.21.15.150:3300/0 conn(0x7fd4580aa220 0x7fd4580aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3975838549 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd4580c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 -- 172.21.15.73:0/3975838549 >> 172.21.15.73:0/3975838549 conn(0x7fd458094930 msgr2=0x7fd45806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:47.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.881+0000 7fd45e06b700 1 -- 172.21.15.73:0/3975838549 shutdown_connections 2023-11-28T04:24:47.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.882+0000 7fd45e06b700 1 -- 172.21.15.73:0/3975838549 wait complete. 2023-11-28T04:24:47.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 Processor -- start 2023-11-28T04:24:47.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 -- start start 2023-11-28T04:24:47.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd458168af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd458154530 0x7fd4580b08b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.891 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580c4120 con 0x7fd4580aa220 2023-11-28T04:24:47.892 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd45814fae0 con 0x7fd458154530 2023-11-28T04:24:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.883+0000 7fd45e06b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4580abec0 con 0x7fd4580a7ff0 2023-11-28T04:24:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:34486/0 (socket says 172.21.15.73:34486) 2023-11-28T04:24:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 -- 172.21.15.73:0/3319499565 learned_addr learned my addr 172.21.15.73:0/3319499565 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:24:47.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd457fff700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd458154530 0x7fd4580b08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd458168af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 -- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 msgr2=0x7fd458168af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd458168af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 -- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd458154530 msgr2=0x7fd4580b08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd458154530 0x7fd4580b08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 -- 172.21.15.73:0/3319499565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448009ae0 con 0x7fd4580aa220 2023-11-28T04:24:47.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd456ffd700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7fd44c00d860 tx=0x7fd44c008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00f520 con 0x7fd4580aa220 2023-11-28T04:24:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd44c00f680 con 0x7fd4580aa220 2023-11-28T04:24:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4580ac350 con 0x7fd4580aa220 2023-11-28T04:24:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd44c00fa00 con 0x7fd4580aa220 2023-11-28T04:24:47.904 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.884+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4580ab870 con 0x7fd4580aa220 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.885+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd45806c860 con 0x7fd4580aa220 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.886+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 91) v1 ==== 75357+0+0 (secure 0 0 0) 0x7fd44c0123e0 con 0x7fd4580aa220 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.886+0000 7fd43f7fe700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 0x7fd444061510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.886+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fd44c079c70 con 0x7fd4580aa220 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.886+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 0x7fd444061510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.886+0000 7fd4577fe700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 0x7fd444061510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd44800c010 tx=0x7fd448009840).ready entity=mgr.9717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:24:47.906 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:47.890+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd44c04b000 con 0x7fd4580aa220 2023-11-28T04:24:48.078 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.077+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd458064230 con 0x7fd44405f050 2023-11-28T04:24:48.080 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.079+0000 7fd43f7fe700 1 -- 172.21.15.73:0/3319499565 <== mgr.9717 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7fd458064230 con 0x7fd44405f050 2023-11-28T04:24:48.080 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:24:48.080 INFO:teuthology.orchestra.run.smithi073.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-28T04:24:48.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 msgr2=0x7fd444061510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:48.082 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 0x7fd444061510 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fd44800c010 tx=0x7fd448009840).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 msgr2=0x7fd458152cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 rx=0x7fd44c00d860 tx=0x7fd44c008040).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 shutdown_connections 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd44405f050 0x7fd444061510 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd458154530 0x7fd4580b08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4580a7ff0 0x7fd458168af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 --2- 172.21.15.73:0/3319499565 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4580aa220 0x7fd458152cb0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 >> 172.21.15.73:0/3319499565 conn(0x7fd458094930 msgr2=0x7fd45814fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 shutdown_connections 2023-11-28T04:24:48.083 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:24:48.081+0000 7fd45e06b700 1 -- 172.21.15.73:0/3319499565 wait complete. 2023-11-28T04:24:48.094 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-28T04:24:48.094 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-11-28T04:24:48.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:48.095 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:49.111 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:49.111 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:49.239 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:24:49.242 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:24:50.130 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:50.131 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:51.152 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:51.152 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:52.177 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:52.178 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:53.199 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:53.200 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:54.220 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:54.220 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:55.239 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:55.240 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:56.260 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:56.260 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:57.279 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:57.279 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:58.300 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:58.300 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:24:59.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:24:59.320 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:00.339 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:00.340 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:00.754 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:25:00.753+0000 7f046799c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:25:00.754 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:25:00.753+0000 7f046799c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:25:00.754 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:25:00.753+0000 7f046799c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:25:00.754 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:25:00.753+0000 7f046799c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:25:00.755 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:25:00.753+0000 7f046799c700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:25:01.359 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:01.359 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:01.378 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:01.378 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-11-28-04_25_00_UTC'] 2023-11-28T04:25:01.378 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701145487.764677, now = 1701145501.37878, timo = 13 2023-11-28T04:25:02.379 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-11-28-04_25_00 in ~73s... 2023-11-28T04:25:02.379 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:02.379 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:02.399 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:03.400 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:03.400 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:03.418 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:04.419 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:04.419 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:04.438 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:05.439 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:05.439 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:05.458 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:06.459 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:06.459 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:06.478 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:07.480 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:07.480 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:07.499 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:08.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:08.500 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:08.520 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:09.521 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:09.521 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:09.541 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:10.541 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:10.542 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:10.561 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:11.562 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:11.562 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:11.582 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:12.583 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:12.583 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:12.602 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:13.605 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:13.605 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:13.624 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:14.625 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:14.625 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:14.644 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:15.645 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:15.645 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:15.665 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:16.666 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:16.667 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:16.687 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:17.687 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:17.687 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:17.705 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:18.706 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:18.706 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:18.725 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:19.385 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:25:19.388 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:25:19.726 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:19.727 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:19.745 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:20.746 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:20.747 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:20.767 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:21.767 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:21.768 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:21.786 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:22.787 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:22.787 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:22.805 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:23.806 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:23.806 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:23.826 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:24.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:24.827 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:24.846 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:25.848 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:25.848 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:25.867 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:26.868 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:26.868 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:26.887 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:27.888 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:27.888 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:27.907 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:28.908 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:28.908 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:28.927 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:29.927 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:29.928 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:29.947 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:30.948 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:30.948 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:30.966 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:31.967 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:37.774 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:37.795 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:38.796 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:38.797 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:38.817 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:39.818 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:39.819 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:39.838 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:40.840 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:40.840 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:40.859 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:41.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:41.860 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:41.881 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:42.881 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:42.882 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:42.900 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:43.902 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:43.902 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:43.922 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:44.923 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:44.924 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:44.943 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:45.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:45.945 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:45.963 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:46.964 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:46.965 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:46.983 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:47.985 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:47.985 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:48.005 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:49.006 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:49.006 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:49.026 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:49.705 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:25:49.708 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:25:50.026 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:50.027 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:50.047 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:51.048 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:51.048 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:51.071 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:52.072 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:52.072 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:52.092 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:53.095 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:53.218 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:53.236 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:54.238 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:54.238 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:54.257 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:55.258 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:55.258 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:55.277 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:56.278 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:56.279 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:56.297 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:57.298 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:57.299 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:57.318 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:58.319 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:58.426 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:58.446 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:25:59.446 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:25:59.447 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:25:59.466 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:26:00.467 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:00.467 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:26:00.486 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_25_00_UTC 2023-11-28T04:26:01.487 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:01.487 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:26:01.506 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_26_00_UTC 2023-11-28T04:26:01.506 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-11-28-04_25_00_UTC'] 2023-11-28T04:26:01.507 DEBUG:tasks.cephfs.test_snap_schedules:exec=1701145487.764677, now = 1701145561.5070987, timo = 73 2023-11-28T04:26:02.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-28T04:26:02.636 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.635+0000 7fe1f3ac8700 1 Processor -- start 2023-11-28T04:26:02.637 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 -- start start 2023-11-28T04:26:02.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 0x7fe1ec154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.639 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1ec0c6630 con 0x7fe1ec0aa220 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1ec0c6390 con 0x7fe1ec154530 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.636+0000 7fe1f3ac8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0a8470 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1864700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 0x7fe1ec154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:35920/0 (socket says 172.21.15.73:35920) 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 -- 172.21.15.73:0/3030246584 learned_addr learned my addr 172.21.15.73:0/3030246584 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 -- 172.21.15.73:0/3030246584 >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 msgr2=0x7fe1ec154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 --2- 172.21.15.73:0/3030246584 >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 0x7fe1ec154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 -- 172.21.15.73:0/3030246584 >> v1:172.21.15.150:6789/0 conn(0x7fe1ec0aa220 legacy=0x7fe1ec0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 -- 172.21.15.73:0/3030246584 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1ec064230 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1864700 1 --2- 172.21.15.73:0/3030246584 >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 0x7fe1ec154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.637+0000 7fe1f1063700 1 --2- 172.21.15.73:0/3030246584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe1e0006fd0 tx=0x7fe1e00090f0).ready entity=mon.2 client_cookie=4bb3f285f72000fc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f0862700 1 -- 172.21.15.73:0/3030246584 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000b580 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f0862700 1 -- 172.21.15.73:0/3030246584 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e000b6e0 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f0862700 1 -- 172.21.15.73:0/3030246584 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000ba60 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/3030246584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 msgr2=0x7fe1ec0c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/3030246584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 rx=0x7fe1e0006fd0 tx=0x7fe1e00090f0).stop 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/3030246584 shutdown_connections 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/3030246584 >> v2:172.21.15.150:3300/0 conn(0x7fe1ec154530 0x7fe1ec154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/3030246584 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec0a7ff0 0x7fe1ec0c8580 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/3030246584 >> 172.21.15.73:0/3030246584 conn(0x7fe1ec094930 msgr2=0x7fe1ec06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:02.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/3030246584 shutdown_connections 2023-11-28T04:26:02.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.638+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/3030246584 wait complete. 2023-11-28T04:26:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.639+0000 7fe1f3ac8700 1 Processor -- start 2023-11-28T04:26:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.639+0000 7fe1f3ac8700 1 -- start start 2023-11-28T04:26:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec0a7ff0 0x7fe1ec168120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec154530 0x7fe1ec0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec168b00 con 0x7fe1ec0aa220 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec149870 con 0x7fe1ec0a7ff0 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f3ac8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1ec0abb30 con 0x7fe1ec154530 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f1063700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f1063700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37980/0 (socket says 172.21.15.73:37980) 2023-11-28T04:26:02.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f1063700 1 -- 172.21.15.73:0/883254724 learned_addr learned my addr 172.21.15.73:0/883254724 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f1864700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec0a7ff0 0x7fe1ec168120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.640+0000 7fe1f2065700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec154530 0x7fe1ec0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 -- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec154530 msgr2=0x7fe1ec0c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec154530 0x7fe1ec0c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 -- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec0a7ff0 msgr2=0x7fe1ec168120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec0a7ff0 0x7fe1ec168120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 -- 172.21.15.73:0/883254724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc009a20 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f1063700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 rx=0x7fe1e0009130 tx=0x7fe1e0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0000bc0 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e000b850 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0000d20 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1e0009370 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.641+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1ec0ac0f0 con 0x7fe1ec0aa220 2023-11-28T04:26:02.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.642+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1ec0add40 con 0x7fe1ec0aa220 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.643+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 93) v1 ==== 75679+0+0 (secure 0 0 0) 0x7fe1e001ca90 con 0x7fe1ec0aa220 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.643+0000 7fe1e9ffb700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 0x7fe1d80618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.643+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fe1e007c3b0 con 0x7fe1ec0aa220 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.643+0000 7fe1f1864700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 0x7fe1d80618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.643+0000 7fe1f1864700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 0x7fe1d80618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe1dc00f6a0 tx=0x7fe1dc00b040).ready entity=mgr.9717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:02.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.647+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1e004d2d0 con 0x7fe1ec0aa220 2023-11-28T04:26:02.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.837+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1ec0c2140 con 0x7fe1d805f3e0 2023-11-28T04:26:02.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.841+0000 7fe1e9ffb700 1 -- 172.21.15.73:0/883254724 <== mgr.9717 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe1ec0c2140 con 0x7fe1d805f3e0 2023-11-28T04:26:02.842 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path snap_test_dir1 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.842+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 msgr2=0x7fe1d80618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.842+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 0x7fe1d80618a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7fe1dc00f6a0 tx=0x7fe1dc00b040).stop 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.842+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 msgr2=0x7fe1ec168610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.842+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 rx=0x7fe1e0009130 tx=0x7fe1e0017000).stop 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 shutdown_connections 2023-11-28T04:26:02.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe1d805f3e0 0x7fe1d80618a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1ec154530 0x7fe1ec0c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1ec0a7ff0 0x7fe1ec168120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 --2- 172.21.15.73:0/883254724 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1ec0aa220 0x7fe1ec168610 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 >> 172.21.15.73:0/883254724 conn(0x7fe1ec094930 msgr2=0x7fe1ec0c7d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 shutdown_connections 2023-11-28T04:26:02.845 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:02.843+0000 7fe1f3ac8700 1 -- 172.21.15.73:0/883254724 wait complete. 2023-11-28T04:26:02.857 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-28T04:26:02.857 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:02.857 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:26:02.872 INFO:teuthology.orchestra.run.smithi073.stdout:scheduled-2023-11-28-04_26_00_UTC 2023-11-28T04:26:02.872 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-11-28-04_26_00_UTC 2023-11-28T04:26:02.872 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:02.873 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-11-28-04_26_00_UTC) 2023-11-28T04:26:02.932 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:02.933 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T04:26:02.990 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T04:26:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.146+0000 7f457ffff700 1 Processor -- start 2023-11-28T04:26:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.146+0000 7f457ffff700 1 -- start start 2023-11-28T04:26:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457ffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45800c66f0 0x7f4580154950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457ffff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4580154e40 con 0x7f458014e530 2023-11-28T04:26:03.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457ffff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45800c7bc0 con 0x7f4580158570 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457ffff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45800c6ba0 con 0x7f45800c66f0 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45800c66f0 0x7f4580154950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f45800c66f0 0x7f4580154950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:37996/0 (socket says 172.21.15.73:37996) 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 -- 172.21.15.73:0/2016693107 learned_addr learned my addr 172.21.15.73:0/2016693107 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 -- 172.21.15.73:0/2016693107 >> v1:172.21.15.150:6789/0 conn(0x7f4580158570 legacy=0x7f45800aa030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:03.151 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 -- 172.21.15.73:0/2016693107 >> v1:172.21.15.73:6789/0 conn(0x7f458014e530 legacy=0x7f458014e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457dffb700 1 -- 172.21.15.73:0/2016693107 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778030847 0 0) 0x7f4570001490 con 0x7f458014e530 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 -- 172.21.15.73:0/2016693107 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4580155920 con 0x7f45800c66f0 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.147+0000 7f457e7fc700 1 --2- 172.21.15.73:0/2016693107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f4580154950 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 rx=0x7f4568006fd0 tx=0x7f45680090f0).ready entity=mon.0 client_cookie=388655209464fa2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457dffb700 1 -- 172.21.15.73:0/2016693107 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800b600 con 0x7f45800c66f0 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457dffb700 1 -- 172.21.15.73:0/2016693107 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f456800b760 con 0x7f45800c66f0 2023-11-28T04:26:03.152 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457dffb700 1 -- 172.21.15.73:0/2016693107 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f456800bae0 con 0x7f45800c66f0 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 -- 172.21.15.73:0/2016693107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 msgr2=0x7f4580154950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 --2- 172.21.15.73:0/2016693107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f4580154950 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 rx=0x7f4568006fd0 tx=0x7f45680090f0).stop 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 -- 172.21.15.73:0/2016693107 shutdown_connections 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 --2- 172.21.15.73:0/2016693107 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f4580154950 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 -- 172.21.15.73:0/2016693107 >> 172.21.15.73:0/2016693107 conn(0x7f4580064990 msgr2=0x7f4580064d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 -- 172.21.15.73:0/2016693107 shutdown_connections 2023-11-28T04:26:03.153 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.148+0000 7f457ffff700 1 -- 172.21.15.73:0/2016693107 wait complete. 2023-11-28T04:26:03.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.149+0000 7f457ffff700 1 Processor -- start 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.149+0000 7f457ffff700 1 -- start start 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458014e530 0x7f45800ca100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4580158570 0x7f45800ca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f458013d0a0 con 0x7f45800c66f0 2023-11-28T04:26:03.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45800caae0 con 0x7f458014e530 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457ffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45801672e0 con 0x7f4580158570 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38002/0 (socket says 172.21.15.73:38002) 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 -- 172.21.15.73:0/1275606073 learned_addr learned my addr 172.21.15.73:0/1275606073 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 -- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4580158570 msgr2=0x7f45800ca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4580158570 0x7f45800ca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 -- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458014e530 msgr2=0x7f45800ca100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.150+0000 7f457effd700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458014e530 0x7f45800ca100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f457effd700 1 -- 172.21.15.73:0/1275606073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009370 con 0x7f45800c66f0 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f457effd700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 rx=0x7f4570002230 tx=0x7f4570007e60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000a5c0 con 0x7f45800c66f0 2023-11-28T04:26:03.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f457000a740 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45801666d0 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f457000ab00 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.151+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4580166c90 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.152+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45800c5ce0 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.152+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 93) v1 ==== 75679+0+0 (secure 0 0 0) 0x7f457001b400 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.152+0000 7f4576ffd700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 0x7f45600618a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.153+0000 7f457e7fc700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 0x7f45600618a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.153+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f457007a550 con 0x7f45800c66f0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.153+0000 7f457e7fc700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 0x7f45600618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4580150060 tx=0x7f4568017000).ready entity=mgr.9717 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:03.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.157+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f457004b410 con 0x7f45800c66f0 2023-11-28T04:26:03.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:03.365+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f4580154e40 con 0x7f45800c66f0 2023-11-28T04:26:04.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.282+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f4570077f70 con 0x7f45800c66f0 2023-11-28T04:26:04.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.283+0000 7f4576ffd700 1 -- 172.21.15.73:0/1275606073 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v94) v1 ==== 93+0+0 (secure 0 0 0) 0x7f4570073170 con 0x7f45800c66f0 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 msgr2=0x7f45600618a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 0x7f45600618a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f4580150060 tx=0x7f4568017000).stop 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 msgr2=0x7f458013cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 rx=0x7f4570002230 tx=0x7f4570007e60).stop 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 shutdown_connections 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f456005f3e0 0x7f45600618a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4580158570 0x7f45800ca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f45800c66f0 0x7f458013cbb0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 --2- 172.21.15.73:0/1275606073 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f458014e530 0x7f45800ca100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.285+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 >> 172.21.15.73:0/1275606073 conn(0x7f4580064990 msgr2=0x7f4580157170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.286+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 shutdown_connections 2023-11-28T04:26:04.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.286+0000 7f457ffff700 1 -- 172.21.15.73:0/1275606073 wait complete. 2023-11-28T04:26:04.303 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:26:04.304 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:04.348 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:04.388 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:26:04.388 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T04:26:04.389 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:04.389 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T04:26:04.431 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:04.430+0000 7f80c38a4680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:04.432 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:04.430+0000 7fd083f24680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:04.436 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:26:04.436 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:04.436 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T04:26:04.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:04.495 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:04.493+0000 7f80c38a4680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:04.496 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:04.496+0000 7fd083f24680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:04.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.613+0000 7f1e46ef5700 1 Processor -- start 2023-11-28T04:26:04.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.613+0000 7f1e46ef5700 1 -- start start 2023-11-28T04:26:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e46ef5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e400c39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e46ef5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e400c60c0 0x7f1e40062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e46ef5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e400639a0 con 0x7f1e400c35c0 2023-11-28T04:26:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e46ef5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e400c2fa0 con 0x7f1e400c60c0 2023-11-28T04:26:04.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e46ef5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40150d00 con 0x7f1e40063e30 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e466f4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e400c60c0 0x7f1e40062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e466f4700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1e400c60c0 0x7f1e40062700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46714/0 (socket says 172.21.15.73:46714) 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e466f4700 1 -- 172.21.15.73:0/552546682 learned_addr learned my addr 172.21.15.73:0/552546682 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.614+0000 7f1e456f2700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e400c39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e466f4700 1 -- 172.21.15.73:0/552546682 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 msgr2=0x7f1e400c39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e466f4700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e400c39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e44ef1700 1 -- 172.21.15.73:0/552546682 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085481643 0 0) 0x7f1e400639a0 con 0x7f1e400c35c0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e466f4700 1 -- 172.21.15.73:0/552546682 >> v1:172.21.15.150:6789/0 conn(0x7f1e400c35c0 legacy=0x7f1e400c59b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e466f4700 1 -- 172.21.15.73:0/552546682 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e40062d90 con 0x7f1e400c60c0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e466f4700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e40062700 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 rx=0x7f1e3800d1a0 tx=0x7f1e3800ba50).ready entity=mon.0 client_cookie=6f8100b63ea9113a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e44ef1700 1 -- 172.21.15.73:0/552546682 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38010070 con 0x7f1e400c60c0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e44ef1700 1 -- 172.21.15.73:0/552546682 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3800de10 con 0x7f1e400c60c0 2023-11-28T04:26:04.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e44ef1700 1 -- 172.21.15.73:0/552546682 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38015680 con 0x7f1e400c60c0 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e46ef5700 1 -- 172.21.15.73:0/552546682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 msgr2=0x7f1e40062700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.615+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e40062700 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 rx=0x7f1e3800d1a0 tx=0x7f1e3800ba50).stop 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 -- 172.21.15.73:0/552546682 shutdown_connections 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e40062700 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/552546682 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e400c39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 -- 172.21.15.73:0/552546682 >> 172.21.15.73:0/552546682 conn(0x7f1e400c7eb0 msgr2=0x7f1e400c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 -- 172.21.15.73:0/552546682 shutdown_connections 2023-11-28T04:26:04.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.616+0000 7f1e46ef5700 1 -- 172.21.15.73:0/552546682 wait complete. 2023-11-28T04:26:04.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 Processor -- start 2023-11-28T04:26:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 -- start start 2023-11-28T04:26:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e401527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400c35c0 0x7f1e40150f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40152360 con 0x7f1e400c60c0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40151750 con 0x7f1e400c35c0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.617+0000 7f1e46ef5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e40151a30 con 0x7f1e40063e30 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46722/0 (socket says 172.21.15.73:46722) 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 -- 172.21.15.73:0/3254501240 learned_addr learned my addr 172.21.15.73:0/3254501240 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e45ef3700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e401527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e456f2700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400c35c0 0x7f1e40150f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 msgr2=0x7f1e401527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e401527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400c35c0 msgr2=0x7f1e40150f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400c35c0 0x7f1e40150f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 -- 172.21.15.73:0/3254501240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3800bcf0 con 0x7f1e400c60c0 2023-11-28T04:26:04.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e466f4700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f1e4006c470 tx=0x7f1e3800ba50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800d870 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e38014ad0 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e40151d10 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e38014e70 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.618+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e401513f0 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.619+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e401524d0 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.620+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1e3801c040 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.620+0000 7f1e35ffb700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 0x7f1e24061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.620+0000 7f1e45ef3700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 msgr2=0x7f1e24061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.620+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f1e3807dd80 con 0x7f1e400c60c0 2023-11-28T04:26:04.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.620+0000 7f1e45ef3700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 0x7f1e24061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:04.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.624+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e3804eb30 con 0x7f1e400c60c0 2023-11-28T04:26:04.771 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:04.770+0000 7f80c38a4680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:04.781 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:04.781+0000 7fd083f24680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:04.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.821+0000 7f1e45ef3700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 msgr2=0x7f1e24061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:04.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.822+0000 7f1e45ef3700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 0x7f1e24061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:04.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.833+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e4006c7c0 con 0x7f1e400c60c0 2023-11-28T04:26:04.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.834+0000 7f1e35ffb700 1 -- 172.21.15.73:0/3254501240 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f1e38022020 con 0x7f1e400c60c0 2023-11-28T04:26:04.834 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:04.834 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:26:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 msgr2=0x7f1e24061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 0x7f1e24061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 msgr2=0x7f1e4014baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 rx=0x7f1e4006c470 tx=0x7f1e3800ba50).stop 2023-11-28T04:26:04.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 shutdown_connections 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1e2405f390 0x7f1e24061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e400c60c0 0x7f1e4014baf0 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e40063e30 0x7f1e401527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 --2- 172.21.15.73:0/3254501240 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e400c35c0 0x7f1e40150f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 >> 172.21.15.73:0/3254501240 conn(0x7f1e400c7eb0 msgr2=0x7f1e40068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 shutdown_connections 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.836+0000 7f1e46ef5700 1 -- 172.21.15.73:0/3254501240 wait complete. 2023-11-28T04:26:04.837 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:26:04.849 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.967+0000 7f499b060700 1 Processor -- start 2023-11-28T04:26:04.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.967+0000 7f499b060700 1 -- start start 2023-11-28T04:26:04.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f499b060700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f49941548c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f499b060700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f499406a610 0x7f49940c0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f499b060700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49940c69a0 con 0x7f49940c6e30 2023-11-28T04:26:04.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f499b060700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49940c13a0 con 0x7f499406a610 2023-11-28T04:26:04.970 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f499b060700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940c1680 con 0x7f49941544e0 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f4993fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f499406a610 0x7f49940c0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f49995fd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f49940c6e30 0x7f49940a8000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34712/0 (socket says 172.21.15.73:34712) 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f49995fd700 1 -- 172.21.15.73:0/795577415 learned_addr learned my addr 172.21.15.73:0/795577415 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f4998dfc700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f49941548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f4993fff700 1 -- 172.21.15.73:0/795577415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 msgr2=0x7f49941548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f4993fff700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f49941548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f4993fff700 1 -- 172.21.15.73:0/795577415 >> v1:172.21.15.73:6789/0 conn(0x7f49940c6e30 legacy=0x7f49940a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.968+0000 7f49937fe700 1 -- 172.21.15.73:0/795577415 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1960019065 0 0) 0x7f49940c69a0 con 0x7f49940c6e30 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f4993fff700 1 -- 172.21.15.73:0/795577415 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49940c6630 con 0x7f499406a610 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f4998dfc700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f49941548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:04.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f4993fff700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499406a610 0x7f49940c0bc0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f4980009760 tx=0x7f4980009060).ready entity=mon.0 client_cookie=4653f5914af20e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f49937fe700 1 -- 172.21.15.73:0/795577415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000b580 con 0x7f499406a610 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f49937fe700 1 -- 172.21.15.73:0/795577415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498000b6e0 con 0x7f499406a610 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f49937fe700 1 -- 172.21.15.73:0/795577415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498000ba60 con 0x7f499406a610 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 -- 172.21.15.73:0/795577415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499406a610 msgr2=0x7f49940c0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499406a610 0x7f49940c0bc0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 rx=0x7f4980009760 tx=0x7f4980009060).stop 2023-11-28T04:26:04.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 -- 172.21.15.73:0/795577415 shutdown_connections 2023-11-28T04:26:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f49941548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 --2- 172.21.15.73:0/795577415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f499406a610 0x7f49940c0bc0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 -- 172.21.15.73:0/795577415 >> 172.21.15.73:0/795577415 conn(0x7f4994085020 msgr2=0x7f4994094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 -- 172.21.15.73:0/795577415 shutdown_connections 2023-11-28T04:26:04.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.969+0000 7f499b060700 1 -- 172.21.15.73:0/795577415 wait complete. 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 Processor -- start 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 -- start start 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f499406a610 0x7f499413ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f4994098b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4994064350 con 0x7f49940c6e30 2023-11-28T04:26:04.977 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49940645f0 con 0x7f499406a610 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.971+0000 7f499b060700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49940648d0 con 0x7f49941544e0 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46744/0 (socket says 172.21.15.73:46744) 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 -- 172.21.15.73:0/2607345909 learned_addr learned my addr 172.21.15.73:0/2607345909 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4998dfc700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f499406a610 0x7f499413ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f49995fd700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f4994098b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 -- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 msgr2=0x7f4994098b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f4994098b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 -- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f499406a610 msgr2=0x7f499413ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:04.978 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f499406a610 0x7f499413ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 -- 172.21.15.73:0/2607345909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49800092a0 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4993fff700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 rx=0x7f499405fb30 tx=0x7f4980017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980000bc0 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498000b850 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4994064bb0 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4980000d20 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.972+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4994151710 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.973+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49940650d0 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.973+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f498001b920 con 0x7f49940c6e30 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.974+0000 7f4990ff9700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:04.979 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.974+0000 7f4998dfc700 1 -- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 msgr2=0x7f4978061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.974+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f498007bbd0 con 0x7f49940c6e30 2023-11-28T04:26:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.974+0000 7f4998dfc700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:04.980 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:04.978+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f49940650d0 con 0x7f49940c6e30 2023-11-28T04:26:05.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.152+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f499406c860 con 0x7f49940c6e30 2023-11-28T04:26:05.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.153+0000 7f4990ff9700 1 -- 172.21.15.73:0/2607345909 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v198) v1 ==== 74+0+17188 (secure 0 0 0) 0x7f4980028020 con 0x7f49940c6e30 2023-11-28T04:26:05.157 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:05.157 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":198,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:24:46.386086+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"197","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2520251340":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6833/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6832/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:0/1497049717":"2023-11-29T04:24:46.386058+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 msgr2=0x7f4978061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 msgr2=0x7f49940986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 rx=0x7f499405fb30 tx=0x7f4980017000).stop 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 shutdown_connections 2023-11-28T04:26:05.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f497805f360 0x7f4978061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f49941544e0 0x7f4994098b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f499406a610 0x7f499413ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 --2- 172.21.15.73:0/2607345909 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f49940c6e30 0x7f49940986a0 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 >> 172.21.15.73:0/2607345909 conn(0x7f4994085020 msgr2=0x7f49941532e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 shutdown_connections 2023-11-28T04:26:05.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.155+0000 7f499b060700 1 -- 172.21.15.73:0/2607345909 wait complete. 2023-11-28T04:26:05.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '188', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 23, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T04:24:33.630693+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '196', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 24, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T04:24:34.055195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '197', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T04:26:05.169 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:26:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.291+0000 7fb10150b700 1 Processor -- start 2023-11-28T04:26:05.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.291+0000 7fb10150b700 1 -- start start 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb10150b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb0fc152480 0x7fb0fc069df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb10150b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc0bcba0 0x7fb0fc0c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb10150b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0fc0c4df0 con 0x7fb0fc152860 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb10150b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0fc06a2e0 con 0x7fb0fc152480 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb10150b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0c4bd0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0fa7fc700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb0fc152860 0x7fb0fc0bc490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34726/0 (socket says 172.21.15.73:34726) 2023-11-28T04:26:05.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0fa7fc700 1 -- 172.21.15.73:0/1638752416 learned_addr learned my addr 172.21.15.73:0/1638752416 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0fb7fe700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc0bcba0 0x7fb0fc0c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1638752416 >> v2:172.21.15.150:3300/0 conn(0x7fb0fc152480 0x7fb0fc069df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0f9ffb700 1 -- 172.21.15.73:0/1638752416 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038091470 0 0) 0x7fb0fc0c4df0 con 0x7fb0fc152860 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0f9ffb700 1 -- 172.21.15.73:0/1638752416 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0e40033f0 con 0x7fb0fc152860 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0faffd700 1 -- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc0bcba0 msgr2=0x7fb0fc0c46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc0bcba0 0x7fb0fc0c46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0faffd700 1 -- 172.21.15.73:0/1638752416 >> v1:172.21.15.73:6789/0 conn(0x7fb0fc152860 legacy=0x7fb0fc0bc490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.292+0000 7fb0faffd700 1 -- 172.21.15.73:0/1638752416 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0fc0627a0 con 0x7fb0fc152480 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc152480 0x7fb0fc069df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fb0f0009790 tx=0x7fb0f0009060).ready entity=mon.1 client_cookie=26c4bc496138732a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb0f9ffb700 1 -- 172.21.15.73:0/1638752416 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f00090d0 con 0x7fb0fc152480 2023-11-28T04:26:05.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb0f9ffb700 1 -- 172.21.15.73:0/1638752416 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f000d4a0 con 0x7fb0fc152480 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb0f9ffb700 1 -- 172.21.15.73:0/1638752416 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f000d820 con 0x7fb0fc152480 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 -- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc152480 msgr2=0x7fb0fc069df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc152480 0x7fb0fc069df0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 rx=0x7fb0f0009790 tx=0x7fb0f0009060).stop 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 -- 172.21.15.73:0/1638752416 shutdown_connections 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc0bcba0 0x7fb0fc0c46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 --2- 172.21.15.73:0/1638752416 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc152480 0x7fb0fc069df0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 -- 172.21.15.73:0/1638752416 >> 172.21.15.73:0/1638752416 conn(0x7fb0fc0c7fb0 msgr2=0x7fb0fc0c3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 -- 172.21.15.73:0/1638752416 shutdown_connections 2023-11-28T04:26:05.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.293+0000 7fb10150b700 1 -- 172.21.15.73:0/1638752416 wait complete. 2023-11-28T04:26:05.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.294+0000 7fb10150b700 1 Processor -- start 2023-11-28T04:26:05.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 -- start start 2023-11-28T04:26:05.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 0x7fb0fc153a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc152480 0x7fb0fc14f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc152d30 con 0x7fb0fc152860 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc1535d0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb10150b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0fc0bfbf0 con 0x7fb0fc152480 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb0fb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46750/0 (socket says 172.21.15.73:46750) 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb0fb7fe700 1 -- 172.21.15.73:0/1002230483 learned_addr learned my addr 172.21.15.73:0/1002230483 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 0x7fb0fc153a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.295+0000 7fb0fa7fc700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc152480 0x7fb0fc14f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc152480 msgr2=0x7fb0fc14f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc152480 0x7fb0fc14f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 msgr2=0x7fb0fc0bf2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 -- 172.21.15.73:0/1002230483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0f00092a0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0fb7fe700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0faffd700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 0x7fb0fc153a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fb0f0013000 tx=0x7fb0f0016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0020070 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0f00127f0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0fc152f80 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0f0012bb0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.296+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0fc14f7f0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.297+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0fc1538b0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.297+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fb0f001c420 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.298+0000 7fb0e2ffd700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 0x7fb0e8061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.298+0000 7fb0fa7fc700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 msgr2=0x7fb0e8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.298+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb0f007aef0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.298+0000 7fb0fa7fc700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 0x7fb0e8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:05.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.302+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb0f004bdb0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.386 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:05.385+0000 7f80c38a4680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:05.429 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:05.427+0000 7fd083f24680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:05.459 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:05.458+0000 7f80c38a4680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:05.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.499+0000 7fb0fa7fc700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 msgr2=0x7fb0e8061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:05.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.499+0000 7fb0fa7fc700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 0x7fb0e8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:05.500 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:05.500+0000 7fd083f24680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:05.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.506+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7fb0fc06c7c0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.506+0000 7fb0e2ffd700 1 -- 172.21.15.73:0/1002230483 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v198) v1 ==== 110+0+10 (secure 0 0 0) 0x7fb0f004f3d0 con 0x7fb0fc0bcba0 2023-11-28T04:26:05.506 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 msgr2=0x7fb0e8061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 0x7fb0e8061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 msgr2=0x7fb0fc153a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 0x7fb0fc153a60 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 rx=0x7fb0f0013000 tx=0x7fb0f0016000).stop 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 shutdown_connections 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb0e805f390 0x7fb0e8061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb0fc152860 0x7fb0fc0bf2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.509 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb0fc0bcba0 0x7fb0fc153a60 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 --2- 172.21.15.73:0/1002230483 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb0fc152480 0x7fb0fc14f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 >> 172.21.15.73:0/1002230483 conn(0x7fb0fc0c7fb0 msgr2=0x7fb0fc0c68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 shutdown_connections 2023-11-28T04:26:05.510 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.508+0000 7fb10150b700 1 -- 172.21.15.73:0/1002230483 wait complete. 2023-11-28T04:26:05.521 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:26:05.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.642+0000 7f08b97cb700 1 Processor -- start 2023-11-28T04:26:05.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.642+0000 7f08b97cb700 1 -- start start 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b97cb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08b40c6190 0x7f08b40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b97cb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f08b4158320 0x7f08b41587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b97cb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08b4158cc0 con 0x7f08b4098d10 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b97cb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b40c72d0 con 0x7f08b40c6190 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b97cb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08b4064570 con 0x7f08b4158320 2023-11-28T04:26:05.646 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08b40c6190 0x7f08b40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b27fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f08b40c6190 0x7f08b40c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46754/0 (socket says 172.21.15.73:46754) 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b27fc700 1 -- 172.21.15.73:0/1775878680 learned_addr learned my addr 172.21.15.73:0/1775878680 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.643+0000 7f08b37fe700 1 --2- 172.21.15.73:0/1775878680 >> v2:172.21.15.150:3300/0 conn(0x7f08b4158320 0x7f08b41587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b27fc700 1 -- 172.21.15.73:0/1775878680 >> v2:172.21.15.150:3300/0 conn(0x7f08b4158320 msgr2=0x7f08b41587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b27fc700 1 --2- 172.21.15.73:0/1775878680 >> v2:172.21.15.150:3300/0 conn(0x7f08b4158320 0x7f08b41587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b27fc700 1 -- 172.21.15.73:0/1775878680 >> v1:172.21.15.150:6789/0 conn(0x7f08b4098d10 legacy=0x7f08b4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b27fc700 1 -- 172.21.15.73:0/1775878680 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b4154bc0 con 0x7f08b40c6190 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b27fc700 1 --2- 172.21.15.73:0/1775878680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b40c7760 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 rx=0x7f08b40657a0 tx=0x7f08a40090f0).ready entity=mon.0 client_cookie=e9c3cb2ac3933bc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b1ffb700 1 -- 172.21.15.73:0/1775878680 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400b520 con 0x7f08b40c6190 2023-11-28T04:26:05.647 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b1ffb700 1 -- 172.21.15.73:0/1775878680 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400b680 con 0x7f08b40c6190 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b1ffb700 1 -- 172.21.15.73:0/1775878680 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a400ba00 con 0x7f08b40c6190 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 -- 172.21.15.73:0/1775878680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 msgr2=0x7f08b40c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 --2- 172.21.15.73:0/1775878680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b40c7760 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 rx=0x7f08b40657a0 tx=0x7f08a40090f0).stop 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 -- 172.21.15.73:0/1775878680 shutdown_connections 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 --2- 172.21.15.73:0/1775878680 >> v2:172.21.15.150:3300/0 conn(0x7f08b4158320 0x7f08b41587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 --2- 172.21.15.73:0/1775878680 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b40c7760 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.644+0000 7f08b97cb700 1 -- 172.21.15.73:0/1775878680 >> 172.21.15.73:0/1775878680 conn(0x7f08b40cb310 msgr2=0x7f08b40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.645+0000 7f08b97cb700 1 -- 172.21.15.73:0/1775878680 shutdown_connections 2023-11-28T04:26:05.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.645+0000 7f08b97cb700 1 -- 172.21.15.73:0/1775878680 wait complete. 2023-11-28T04:26:05.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.646+0000 7f08b97cb700 1 Processor -- start 2023-11-28T04:26:05.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.646+0000 7f08b97cb700 1 -- start start 2023-11-28T04:26:05.649 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:05.646+0000 7f80c38a4680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:05.653 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08b4098d10 0x7f08b4154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4158320 0x7f08b40c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4164af0 con 0x7f08b40c6190 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4156520 con 0x7f08b4158320 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08b4155560 con 0x7f08b4098d10 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46768/0 (socket says 172.21.15.73:46768) 2023-11-28T04:26:05.654 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 -- 172.21.15.73:0/4061864516 learned_addr learned my addr 172.21.15.73:0/4061864516 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b37fe700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4158320 0x7f08b40c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b2ffd700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08b4098d10 0x7f08b4154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 -- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08b4098d10 msgr2=0x7f08b4154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08b4098d10 0x7f08b4154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 -- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4158320 msgr2=0x7f08b40c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4158320 0x7f08b40c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 -- 172.21.15.73:0/4061864516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009370 con 0x7f08b40c6190 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b27fc700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 rx=0x7f08a4009130 tx=0x7f08a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4000bc0 con 0x7f08b40c6190 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a400b7f0 con 0x7f08b40c6190 2023-11-28T04:26:05.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08b4155840 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a4000d20 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.647+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08b4155240 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.648+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08b4155d60 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.649+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f08a401c880 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.649+0000 7f089affd700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 0x7f08a0061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.649+0000 7f08b2ffd700 1 -- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 msgr2=0x7f08a0061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.649+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f08a407c2c0 con 0x7f08b40c6190 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.649+0000 7f08b2ffd700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 0x7f08a0061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:05.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.653+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f08a404d180 con 0x7f08b40c6190 2023-11-28T04:26:05.698 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:05.698+0000 7fd083f24680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.823+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f08b406c860 con 0x7f08b40c6190 2023-11-28T04:26:05.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.823+0000 7f089affd700 1 -- 172.21.15.73:0/4061864516 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v198) v1 ==== 92+0+10 (secure 0 0 0) 0x7f08a402a020 con 0x7f08b40c6190 2023-11-28T04:26:05.824 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 msgr2=0x7f08a0061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 0x7f08a0061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 msgr2=0x7f08b4147030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 rx=0x7f08a4009130 tx=0x7f08a4017000).stop 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 shutdown_connections 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f08a005f360 0x7f08a0061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:05.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f08b4158320 0x7f08b40c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f08b4098d10 0x7f08b4154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 --2- 172.21.15.73:0/4061864516 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f08b40c6190 0x7f08b4147030 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 >> 172.21.15.73:0/4061864516 conn(0x7f08b40cb310 msgr2=0x7f08b4153000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 shutdown_connections 2023-11-28T04:26:05.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.826+0000 7f08b97cb700 1 -- 172.21.15.73:0/4061864516 wait complete. 2023-11-28T04:26:05.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T04:26:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.962+0000 7f6884c68700 1 Processor -- start 2023-11-28T04:26:05.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.962+0000 7f6884c68700 1 -- start start 2023-11-28T04:26:05.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f6884c68700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f68800a8280 0x7f68800a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f6884c68700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6880158160 0x7f6880155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f6884c68700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6880067770 con 0x7f688014e600 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f6884c68700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6880159180 con 0x7f6880158160 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f6884c68700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68800be8d0 con 0x7f68800a8280 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687ed9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f688014e600 0x7f688014ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34730/0 (socket says 172.21.15.73:34730) 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687ed9d700 1 -- 172.21.15.73:0/3408976426 learned_addr learned my addr 172.21.15.73:0/3408976426 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687dd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f6880158160 0x7f6880155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687e59c700 1 --2- 172.21.15.73:0/3408976426 >> v2:172.21.15.150:3300/0 conn(0x7f68800a8280 0x7f68800a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687d59a700 1 -- 172.21.15.73:0/3408976426 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1019773529 0 0) 0x7f6880067770 con 0x7f688014e600 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.963+0000 7f687d59a700 1 -- 172.21.15.73:0/3408976426 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68680033f0 con 0x7f688014e600 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687dd9b700 1 -- 172.21.15.73:0/3408976426 >> v2:172.21.15.150:3300/0 conn(0x7f68800a8280 msgr2=0x7f68800a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:05.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687dd9b700 1 --2- 172.21.15.73:0/3408976426 >> v2:172.21.15.150:3300/0 conn(0x7f68800a8280 0x7f68800a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687dd9b700 1 -- 172.21.15.73:0/3408976426 >> v1:172.21.15.73:6789/0 conn(0x7f688014e600 legacy=0x7f688014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687dd9b700 1 -- 172.21.15.73:0/3408976426 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6880158600 con 0x7f6880158160 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687dd9b700 1 --2- 172.21.15.73:0/3408976426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880155500 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 rx=0x7f68800c9050 tx=0x7f6870009060).ready entity=mon.0 client_cookie=9dace9850f7d3e5f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687d59a700 1 -- 172.21.15.73:0/3408976426 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000d4a0 con 0x7f6880158160 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687d59a700 1 -- 172.21.15.73:0/3408976426 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f687000d600 con 0x7f6880158160 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f687d59a700 1 -- 172.21.15.73:0/3408976426 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f687000d980 con 0x7f6880158160 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 -- 172.21.15.73:0/3408976426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 msgr2=0x7f6880155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 --2- 172.21.15.73:0/3408976426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880155500 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 rx=0x7f68800c9050 tx=0x7f6870009060).stop 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 -- 172.21.15.73:0/3408976426 shutdown_connections 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 --2- 172.21.15.73:0/3408976426 >> v2:172.21.15.150:3300/0 conn(0x7f68800a8280 0x7f68800a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 --2- 172.21.15.73:0/3408976426 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880155500 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.966 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.964+0000 7f6884c68700 1 -- 172.21.15.73:0/3408976426 >> 172.21.15.73:0/3408976426 conn(0x7f6880062700 msgr2=0x7f6880062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:05.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.965+0000 7f6884c68700 1 -- 172.21.15.73:0/3408976426 shutdown_connections 2023-11-28T04:26:05.967 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.965+0000 7f6884c68700 1 -- 172.21.15.73:0/3408976426 wait complete. 2023-11-28T04:26:05.971 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 Processor -- start 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 -- start start 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68800a8280 0x7f688014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688014e600 0x7f68800aeb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68800af010 con 0x7f6880158160 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6880168090 con 0x7f68800a8280 2023-11-28T04:26:05.972 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f6884c68700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6880165350 con 0x7f688014e600 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f687ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.966+0000 7f687ed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46788/0 (socket says 172.21.15.73:46788) 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 -- 172.21.15.73:0/358521600 learned_addr learned my addr 172.21.15.73:0/358521600 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687e59c700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68800a8280 0x7f688014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687dd9b700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688014e600 0x7f68800aeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 -- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688014e600 msgr2=0x7f68800aeb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688014e600 0x7f68800aeb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 -- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68800a8280 msgr2=0x7f688014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68800a8280 0x7f688014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 -- 172.21.15.73:0/358521600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68700092a0 con 0x7f6880158160 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f687ed9d700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 rx=0x7f6878005ad0 tx=0x7f6878001650).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:05.973 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878005d30 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6878005e90 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6880167d20 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6878007320 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.967+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6880168a10 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.968+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6880167b80 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.968+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f6878007710 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.969+0000 7f686e7fc700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 0x7f6860061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.969+0000 7f687e59c700 1 -- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 msgr2=0x7f6860061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.969+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f687807a580 con 0x7f6880158160 2023-11-28T04:26:05.974 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.969+0000 7f687e59c700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 0x7f6860061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:05.975 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:05.973+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6880167b80 con 0x7f6880158160 2023-11-28T04:26:06.075 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.072+0000 7f80c38a4680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:06.136 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.136+0000 7fd083f24680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:06.142 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.142+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6880167b80 con 0x7f6880158160 2023-11-28T04:26:06.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.143+0000 7f686e7fc700 1 -- 172.21.15.73:0/358521600 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v198) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6880167b80 con 0x7f6880158160 2023-11-28T04:26:06.143 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.144+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 msgr2=0x7f6860061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.144+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 0x7f6860061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.144+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 msgr2=0x7f6880168520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.144+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 rx=0x7f6878005ad0 tx=0x7f6878001650).stop 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 shutdown_connections 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f686005f390 0x7f6860061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6880158160 0x7f6880168520 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68800a8280 0x7f688014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 --2- 172.21.15.73:0/358521600 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f688014e600 0x7f68800aeb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 >> 172.21.15.73:0/358521600 conn(0x7f6880062700 msgr2=0x7f68800c6100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 shutdown_connections 2023-11-28T04:26:06.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.145+0000 7f6884c68700 1 -- 172.21.15.73:0/358521600 wait complete. 2023-11-28T04:26:06.157 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T04:26:06.176 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.175+0000 7f80c38a4680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:06.250 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.247+0000 7fd083f24680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.276+0000 7f3bd0d41700 1 Processor -- start 2023-11-28T04:26:06.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.276+0000 7f3bd0d41700 1 -- start start 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 0x7f3bcc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c2c50 0x7f3bcc0c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c4460 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3bcc1561c0 con 0x7f3bcc1516b0 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bd0d41700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c35d0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bc9d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bc9d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46796/0 (socket says 172.21.15.73:46796) 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.277+0000 7f3bc9d9b700 1 -- 172.21.15.73:0/913224782 learned_addr learned my addr 172.21.15.73:0/913224782 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:06.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bca59c700 1 --2- 172.21.15.73:0/913224782 >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 0x7f3bcc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bcad9d700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c2c50 0x7f3bcc0c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 -- 172.21.15.73:0/913224782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c2c50 msgr2=0x7f3bcc0c30e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c2c50 0x7f3bcc0c30e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 -- 172.21.15.73:0/913224782 >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 msgr2=0x7f3bcc069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 --2- 172.21.15.73:0/913224782 >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 0x7f3bcc069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 -- 172.21.15.73:0/913224782 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bcc1555b0 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bca59c700 1 --2- 172.21.15.73:0/913224782 >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 0x7f3bcc069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc9d9b700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 rx=0x7f3bcc0c9010 tx=0x7f3bbc0090f0).ready entity=mon.0 client_cookie=fab71ec02daf3874 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc959a700 1 -- 172.21.15.73:0/913224782 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00b600 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc959a700 1 -- 172.21.15.73:0/913224782 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bbc00b760 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bc959a700 1 -- 172.21.15.73:0/913224782 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bbc00bae0 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bd0d41700 1 -- 172.21.15.73:0/913224782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c3af0 msgr2=0x7f3bcc0c3f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.278+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 rx=0x7f3bcc0c9010 tx=0x7f3bbc0090f0).stop 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 -- 172.21.15.73:0/913224782 shutdown_connections 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c2c50 0x7f3bcc0c30e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/913224782 >> v2:172.21.15.150:3300/0 conn(0x7f3bcc1516b0 0x7f3bcc069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/913224782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c3af0 0x7f3bcc0c3f70 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 -- 172.21.15.73:0/913224782 >> 172.21.15.73:0/913224782 conn(0x7f3bcc063810 msgr2=0x7f3bcc063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 -- 172.21.15.73:0/913224782 shutdown_connections 2023-11-28T04:26:06.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.279+0000 7f3bd0d41700 1 -- 172.21.15.73:0/913224782 wait complete. 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.280+0000 7f3bd0d41700 1 Processor -- start 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.280+0000 7f3bd0d41700 1 -- start start 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c3af0 0x7f3bcc0bf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 0x7f3bcc0bf590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc154db0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0c0560 con 0x7f3bcc1516b0 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bd0d41700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3bcc0bfb60 con 0x7f3bcc0c3af0 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46812/0 (socket says 172.21.15.73:46812) 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 -- 172.21.15.73:0/2948029084 learned_addr learned my addr 172.21.15.73:0/2948029084 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bcad9d700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 0x7f3bcc0bf590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.283 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bc9d9b700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c3af0 0x7f3bcc0bf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 -- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c3af0 msgr2=0x7f3bcc0bf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c3af0 0x7f3bcc0bf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 -- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 msgr2=0x7f3bcc0bf590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 0x7f3bcc0bf590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 -- 172.21.15.73:0/2948029084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bbc009370 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bcad9d700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 0x7f3bcc0bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:06.284 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.281+0000 7f3bca59c700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7f3bc000d890 tx=0x7f3bc0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:06.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.285+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011590 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.285 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.286+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3bc00116f0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.286 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.286+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bc0011a70 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.290+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3bcc0bfe40 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.290+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3bcc0c0120 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.292+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f3bc001c460 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.292+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3bcc0c09c0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.293+0000 7f3bba7fc700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 0x7f3bac061870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.293+0000 7f3bc9d9b700 1 -- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 msgr2=0x7f3bac061870 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:06.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.293+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7f3bc007a350 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.293+0000 7f3bc9d9b700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 0x7f3bac061870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:06.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.297+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3bc004b2a0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.332 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.331+0000 7f80c38a4680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:06.411 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.410+0000 7f80c38a4680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:06.414 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.414+0000 7fd083f24680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:06.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.478+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3bcc06c860 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.479+0000 7f3bba7fc700 1 -- 172.21.15.73:0/2948029084 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v199) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3bc004e8c0 con 0x7f3bcc0c2c50 2023-11-28T04:26:06.479 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 msgr2=0x7f3bac061870 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 0x7f3bac061870 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 msgr2=0x7f3bcc0a9e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 rx=0x7f3bc000d890 tx=0x7f3bc0008040).stop 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 shutdown_connections 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f3bac05f3b0 0x7f3bac061870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3bcc1516b0 0x7f3bcc0bf590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3bcc0c2c50 0x7f3bcc0a9e50 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 --2- 172.21.15.73:0/2948029084 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3bcc0c3af0 0x7f3bcc0bf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 >> 172.21.15.73:0/2948029084 conn(0x7f3bcc063810 msgr2=0x7f3bcc14ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:06.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 shutdown_connections 2023-11-28T04:26:06.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.481+0000 7f3bd0d41700 1 -- 172.21.15.73:0/2948029084 wait complete. 2023-11-28T04:26:06.483 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.482+0000 7f80c38a4680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:06.493 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:06.496 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.496+0000 7fd083f24680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:06.568 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.568+0000 7fd083f24680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:06.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.609+0000 7fc74b661700 1 Processor -- start 2023-11-28T04:26:06.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.610+0000 7fc74b661700 1 -- start start 2023-11-28T04:26:06.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc74b661700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7440c3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc74b661700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc744068090 0x7fc7440c43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc74b661700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc744154d20 con 0x7fc7440c2640 2023-11-28T04:26:06.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc74b661700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc744151670 con 0x7fc744068090 2023-11-28T04:26:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc74b661700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7440c48a0 con 0x7fc744067cb0 2023-11-28T04:26:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc748bfc700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc744068090 0x7fc7440c43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7440c3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc749bfe700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc7440c2640 0x7fc7440c2a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:37174/0 (socket says 172.21.15.73:37174) 2023-11-28T04:26:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc749bfe700 1 -- 172.21.15.73:0/1828105961 learned_addr learned my addr 172.21.15.73:0/1828105961 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc73bfff700 1 -- 172.21.15.73:0/1828105961 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3805821781 0 0) 0x7fc744154d20 con 0x7fc7440c2640 2023-11-28T04:26:06.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc73bfff700 1 -- 172.21.15.73:0/1828105961 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7300033f0 con 0x7fc7440c2640 2023-11-28T04:26:06.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 -- 172.21.15.73:0/1828105961 >> v2:172.21.15.150:3300/0 conn(0x7fc744068090 msgr2=0x7fc7440c43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:06.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 --2- 172.21.15.73:0/1828105961 >> v2:172.21.15.150:3300/0 conn(0x7fc744068090 0x7fc7440c43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 -- 172.21.15.73:0/1828105961 >> v1:172.21.15.150:6789/0 conn(0x7fc7440c2640 legacy=0x7fc7440c2a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 -- 172.21.15.73:0/1828105961 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7440a7490 con 0x7fc744067cb0 2023-11-28T04:26:06.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc7493fd700 1 --2- 172.21.15.73:0/1828105961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7440c3ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc740009760 tx=0x7fc740009060).ready entity=mon.2 client_cookie=22dab142fbed2d6b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:06.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc73bfff700 1 -- 172.21.15.73:0/1828105961 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000d4a0 con 0x7fc744067cb0 2023-11-28T04:26:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc73bfff700 1 -- 172.21.15.73:0/1828105961 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc74000d600 con 0x7fc744067cb0 2023-11-28T04:26:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.611+0000 7fc73bfff700 1 -- 172.21.15.73:0/1828105961 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc74000d980 con 0x7fc744067cb0 2023-11-28T04:26:06.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 -- 172.21.15.73:0/1828105961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 msgr2=0x7fc7440c3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 --2- 172.21.15.73:0/1828105961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7440c3ec0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 rx=0x7fc740009760 tx=0x7fc740009060).stop 2023-11-28T04:26:06.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 -- 172.21.15.73:0/1828105961 shutdown_connections 2023-11-28T04:26:06.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 --2- 172.21.15.73:0/1828105961 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7440c3ec0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 --2- 172.21.15.73:0/1828105961 >> v2:172.21.15.150:3300/0 conn(0x7fc744068090 0x7fc7440c43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 -- 172.21.15.73:0/1828105961 >> 172.21.15.73:0/1828105961 conn(0x7fc7440c7fd0 msgr2=0x7fc7440c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:06.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 -- 172.21.15.73:0/1828105961 shutdown_connections 2023-11-28T04:26:06.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.612+0000 7fc74b661700 1 -- 172.21.15.73:0/1828105961 wait complete. 2023-11-28T04:26:06.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.613+0000 7fc74b661700 1 Processor -- start 2023-11-28T04:26:06.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.613+0000 7fc74b661700 1 -- start start 2023-11-28T04:26:06.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7441455d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc744068090 0x7fc744066eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc744067890 con 0x7fc7440c2640 2023-11-28T04:26:06.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc744166b00 con 0x7fc744068090 2023-11-28T04:26:06.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc74b661700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc744165ef0 con 0x7fc744067cb0 2023-11-28T04:26:06.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc749bfe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc749bfe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46828/0 (socket says 172.21.15.73:46828) 2023-11-28T04:26:06.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc749bfe700 1 -- 172.21.15.73:0/1456528978 learned_addr learned my addr 172.21.15.73:0/1456528978 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:06.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc7493fd700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7441455d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.614+0000 7fc748bfc700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc744068090 0x7fc744066eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:06.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 msgr2=0x7fc7441455d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7441455d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.628 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc744068090 msgr2=0x7fc744066eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc744068090 0x7fc744066eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.629 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 -- 172.21.15.73:0/1456528978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7400092a0 con 0x7fc7440c2640 2023-11-28T04:26:06.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc749bfe700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7fc73c005ad0 tx=0x7fc73c0015b0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:06.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c005d30 con 0x7fc7440c2640 2023-11-28T04:26:06.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc73c005e90 con 0x7fc7440c2640 2023-11-28T04:26:06.630 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7441661d0 con 0x7fc7440c2640 2023-11-28T04:26:06.631 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc73c0072e0 con 0x7fc7440c2640 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.615+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc744166790 con 0x7fc7440c2640 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.616+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fc73c0076d0 con 0x7fc7440c2640 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.616+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc74406c860 con 0x7fc7440c2640 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.617+0000 7fc7397fa700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 0x7fc728061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.617+0000 7fc7493fd700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 msgr2=0x7fc728061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:06.632 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.617+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7fc73c079f00 con 0x7fc7440c2640 2023-11-28T04:26:06.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.617+0000 7fc7493fd700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 0x7fc728061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:06.633 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.621+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc73c04aef0 con 0x7fc7440c2640 2023-11-28T04:26:06.746 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.745+0000 7f80c38a4680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.818+0000 7fc7493fd700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 msgr2=0x7fc728061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:06.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.818+0000 7fc7493fd700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 0x7fc728061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:06.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.833+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc7440be1c0 con 0x7fc7440c2640 2023-11-28T04:26:06.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.834+0000 7fc7397fa700 1 -- 172.21.15.73:0/1456528978 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7fc73c04e510 con 0x7fc7440c2640 2023-11-28T04:26:06.834 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:06.834 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 msgr2=0x7fc728061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 0x7fc728061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 msgr2=0x7fc7440673a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 rx=0x7fc73c005ad0 tx=0x7fc73c0015b0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 shutdown_connections 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc72805f390 0x7fc728061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc7440c2640 0x7fc7440673a0 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc744067cb0 0x7fc7441455d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.835+0000 7fc74b661700 1 --2- 172.21.15.73:0/1456528978 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc744068090 0x7fc744066eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:06.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.836+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 >> 172.21.15.73:0/1456528978 conn(0x7fc7440c7fd0 msgr2=0x7fc74405e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:06.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.836+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 shutdown_connections 2023-11-28T04:26:06.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:06.836+0000 7fc74b661700 1 -- 172.21.15.73:0/1456528978 wait complete. 2023-11-28T04:26:06.837 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:26:06.840 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:06.840+0000 7fd083f24680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:06.848 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:06.932 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:06.928+0000 7f80c38a4680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.010+0000 7f95b5b46700 1 Processor -- start 2023-11-28T04:26:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.010+0000 7f95b5b46700 1 -- start start 2023-11-28T04:26:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b5b46700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95b0154120 0x7f95b00be510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b5b46700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00bfc00 con 0x7f95b00c46e0 2023-11-28T04:26:07.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b5b46700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95b00beff0 con 0x7f95b0154500 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b5b46700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95b00bf2d0 con 0x7f95b0154120 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95b0154120 0x7f95b00be510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f95b0154120 0x7f95b00be510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46834/0 (socket says 172.21.15.73:46834) 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 -- 172.21.15.73:0/2424390879 learned_addr learned my addr 172.21.15.73:0/2424390879 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b5345700 1 --1- 172.21.15.73:0/2424390879 >> v1:172.21.15.73:6789/0 conn(0x7f95b00c46e0 0x7f95b015def0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34738/0 (socket says 172.21.15.73:34738) 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 -- 172.21.15.73:0/2424390879 >> v1:172.21.15.150:6789/0 conn(0x7f95b0154500 legacy=0x7f95b00c3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 -- 172.21.15.73:0/2424390879 >> v1:172.21.15.73:6789/0 conn(0x7f95b00c46e0 legacy=0x7f95b015def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95af7fe700 1 -- 172.21.15.73:0/2424390879 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 138563203 0 0) 0x7f95b00bfc00 con 0x7f95b00c46e0 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.011+0000 7f95b4b44700 1 -- 172.21.15.73:0/2424390879 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95b00bf5b0 con 0x7f95b0154120 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b4b44700 1 --2- 172.21.15.73:0/2424390879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154120 0x7f95b00be510 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7f95a4009810 tx=0x7f95a4009060).ready entity=mon.0 client_cookie=57c7aab00c10c783 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95af7fe700 1 -- 172.21.15.73:0/2424390879 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4005880 con 0x7f95b0154120 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95af7fe700 1 -- 172.21.15.73:0/2424390879 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a40059e0 con 0x7f95b0154120 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95af7fe700 1 -- 172.21.15.73:0/2424390879 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a4005d60 con 0x7f95b0154120 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 -- 172.21.15.73:0/2424390879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154120 msgr2=0x7f95b00be510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 --2- 172.21.15.73:0/2424390879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154120 0x7f95b00be510 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 rx=0x7f95a4009810 tx=0x7f95a4009060).stop 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 -- 172.21.15.73:0/2424390879 shutdown_connections 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 --2- 172.21.15.73:0/2424390879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154120 0x7f95b00be510 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 -- 172.21.15.73:0/2424390879 >> 172.21.15.73:0/2424390879 conn(0x7f95b0083000 msgr2=0x7f95b0092540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 -- 172.21.15.73:0/2424390879 shutdown_connections 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.012+0000 7f95b5b46700 1 -- 172.21.15.73:0/2424390879 wait complete. 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.013+0000 7f95b5b46700 1 Processor -- start 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 -- start start 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00c46e0 0x7f95b0153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 0x7f95b00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00ac450 con 0x7f95b0154500 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95b00aaf50 con 0x7f95b0154120 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.014+0000 7f95b5b46700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95b0168610 con 0x7f95b00c46e0 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46844/0 (socket says 172.21.15.73:46844) 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 -- 172.21.15.73:0/4121912298 learned_addr learned my addr 172.21.15.73:0/4121912298 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b4b44700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00c46e0 0x7f95b0153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95affff700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 0x7f95b00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 -- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00c46e0 msgr2=0x7f95b0153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00c46e0 0x7f95b0153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 -- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 msgr2=0x7f95b00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 0x7f95b00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 -- 172.21.15.73:0/4121912298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95a4009300 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95affff700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 0x7f95b00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5345700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 rx=0x7f95a800c7e0 tx=0x7f95a8007ef0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8005c70 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a8005dd0 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95b01688f0 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a8007320 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.015+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95b00ac0e0 con 0x7f95b0154500 2023-11-28T04:26:07.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.016+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95b00c36f0 con 0x7f95b0154500 2023-11-28T04:26:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.017+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f95a8007710 con 0x7f95b0154500 2023-11-28T04:26:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.017+0000 7f95acff9700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 0x7f9594061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.017+0000 7f95b4b44700 1 -- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 msgr2=0x7f9594061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.017+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7f95a807a0e0 con 0x7f95b0154500 2023-11-28T04:26:07.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.017+0000 7f95b4b44700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 0x7f9594061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:07.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.025+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f95a804afd0 con 0x7f95b0154500 2023-11-28T04:26:07.035 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:07.035+0000 7fd083f24680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:07.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.195+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95b006c7c0 con 0x7f95b0154500 2023-11-28T04:26:07.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.196+0000 7f95acff9700 1 -- 172.21.15.73:0/4121912298 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+17299 (secure 0 0 0) 0x7f95a804e5f0 con 0x7f95b0154500 2023-11-28T04:26:07.197 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:07.197 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":199,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:06.279937+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2520251340":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6833/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6832/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:0/1497049717":"2023-11-29T04:24:46.386058+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 msgr2=0x7f9594061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 0x7f9594061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 msgr2=0x7f95b0167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 rx=0x7f95a800c7e0 tx=0x7f95a8007ef0).stop 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 shutdown_connections 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f959405f390 0x7f9594061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f95b0154500 0x7f95b0167aa0 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f95b00c46e0 0x7f95b0153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 --2- 172.21.15.73:0/4121912298 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f95b0154120 0x7f95b00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 >> 172.21.15.73:0/4121912298 conn(0x7f95b0083000 msgr2=0x7f95b00a7f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 shutdown_connections 2023-11-28T04:26:07.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.198+0000 7f95b5b46700 1 -- 172.21.15.73:0/4121912298 wait complete. 2023-11-28T04:26:07.211 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:26:07.212 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:07.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.329+0000 7f1c6cfa5700 1 Processor -- start 2023-11-28T04:26:07.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.329+0000 7f1c6cfa5700 1 -- start start 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c6cfa5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c6813a8d0 0x7f1c68066880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c6cfa5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 0x7f1c680694e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c6cfa5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c68069e40 con 0x7f1c680699d0 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c6cfa5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c68066e60 con 0x7f1c6813a8d0 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c6cfa5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68067140 con 0x7f1c68155510 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c677fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c6813a8d0 0x7f1c68066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c5ffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 0x7f1c680694e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c67fff700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f1c680699d0 0x7f1c680c5800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:37182/0 (socket says 172.21.15.73:37182) 2023-11-28T04:26:07.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.330+0000 7f1c67fff700 1 -- 172.21.15.73:0/3088967536 learned_addr learned my addr 172.21.15.73:0/3088967536 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c677fe700 1 -- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 msgr2=0x7f1c680694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c677fe700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 0x7f1c680694e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c677fe700 1 -- 172.21.15.73:0/3088967536 >> v1:172.21.15.150:6789/0 conn(0x7f1c680699d0 legacy=0x7f1c680c5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c66ffd700 1 -- 172.21.15.73:0/3088967536 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3653750492 0 0) 0x7f1c68069e40 con 0x7f1c680699d0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c677fe700 1 -- 172.21.15.73:0/3088967536 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c68067420 con 0x7f1c6813a8d0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c5ffff700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 0x7f1c680694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c677fe700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c68066880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f1c58009760 tx=0x7f1c58009060).ready entity=mon.1 client_cookie=43facd55fd93a930 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c66ffd700 1 -- 172.21.15.73:0/3088967536 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800f510 con 0x7f1c6813a8d0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c66ffd700 1 -- 172.21.15.73:0/3088967536 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5800f670 con 0x7f1c6813a8d0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.331+0000 7f1c66ffd700 1 -- 172.21.15.73:0/3088967536 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5800f9f0 con 0x7f1c6813a8d0 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 msgr2=0x7f1c68066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c68066880 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 rx=0x7f1c58009760 tx=0x7f1c58009060).stop 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/3088967536 shutdown_connections 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c68066880 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/3088967536 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c68155510 0x7f1c680694e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/3088967536 >> 172.21.15.73:0/3088967536 conn(0x7f1c680c7ca0 msgr2=0x7f1c680c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/3088967536 shutdown_connections 2023-11-28T04:26:07.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.332+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/3088967536 wait complete. 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.333+0000 7f1c6cfa5700 1 Processor -- start 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.333+0000 7f1c6cfa5700 1 -- start start 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c680699d0 0x7f1c680c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c681507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c680c2540 con 0x7f1c68155510 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c68165ff0 con 0x7f1c6813a8d0 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c6cfa5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c681662d0 con 0x7f1c680699d0 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46858/0 (socket says 172.21.15.73:46858) 2023-11-28T04:26:07.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 -- 172.21.15.73:0/2720553858 learned_addr learned my addr 172.21.15.73:0/2720553858 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c677fe700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c680699d0 0x7f1c680c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c5ffff700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c681507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c680699d0 msgr2=0x7f1c680c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c680699d0 0x7f1c680c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 msgr2=0x7f1c681507c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c681507c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 -- 172.21.15.73:0/2720553858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c500092a0 con 0x7f1c68155510 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c67fff700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f1c6000c730 tx=0x7f1c60001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.334+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60005d30 con 0x7f1c68155510 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.335+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c60005e90 con 0x7f1c68155510 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.335+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c60007300 con 0x7f1c68155510 2023-11-28T04:26:07.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.335+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c580092a0 con 0x7f1c68155510 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.335+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c68166890 con 0x7f1c68155510 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c680c6db0 con 0x7f1c68155510 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f1c600076b0 con 0x7f1c68155510 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c5f7fe700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 0x7f1c48061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c677fe700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 msgr2=0x7f1c48061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7f1c6007a0d0 con 0x7f1c68155510 2023-11-28T04:26:07.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.336+0000 7f1c677fe700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 0x7f1c48061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:07.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.344+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c6004afc0 con 0x7f1c68155510 2023-11-28T04:26:07.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.537+0000 7f1c677fe700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 msgr2=0x7f1c48061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:07.538 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.537+0000 7f1c677fe700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 0x7f1c48061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:07.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.559+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1c6806c7c0 con 0x7f1c68155510 2023-11-28T04:26:07.559 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.559+0000 7f1c5f7fe700 1 -- 172.21.15.73:0/2720553858 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f1c6004e5e0 con 0x7f1c68155510 2023-11-28T04:26:07.559 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:07.560 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 msgr2=0x7f1c48061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 0x7f1c48061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 msgr2=0x7f1c681537f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 rx=0x7f1c6000c730 tx=0x7f1c60001520).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 shutdown_connections 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c4805f390 0x7f1c48061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c68155510 0x7f1c681537f0 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c680699d0 0x7f1c680c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 --2- 172.21.15.73:0/2720553858 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c6813a8d0 0x7f1c681507c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 >> 172.21.15.73:0/2720553858 conn(0x7f1c680c7ca0 msgr2=0x7f1c680ab540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.561+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 shutdown_connections 2023-11-28T04:26:07.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.562+0000 7f1c6cfa5700 1 -- 172.21.15.73:0/2720553858 wait complete. 2023-11-28T04:26:07.563 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:26:07.575 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:07.589 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:07.588+0000 7f80c38a4680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:07.659 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:07.658+0000 7f80c38a4680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:07.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.694+0000 7f5a73278700 1 Processor -- start 2023-11-28T04:26:07.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.694+0000 7f5a73278700 1 -- start start 2023-11-28T04:26:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.695+0000 7f5a73278700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5a6c06b510 0x7f5a6c1546f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.695+0000 7f5a73278700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.695+0000 7f5a73278700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a6c063380 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.695+0000 7f5a73278700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a6c0c19e0 con 0x7f5a6c06b510 2023-11-28T04:26:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.695+0000 7f5a73278700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c159060 con 0x7f5a6c0c10f0 2023-11-28T04:26:07.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5a6c06b510 0x7f5a6c1546f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a71815700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a71014700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f5a6c0c0d10 0x7f5a6c153fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:37186/0 (socket says 172.21.15.73:37186) 2023-11-28T04:26:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a71014700 1 -- 172.21.15.73:0/1336248942 learned_addr learned my addr 172.21.15.73:0/1336248942 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 -- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 msgr2=0x7f5a6c158b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:07.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c158b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 -- 172.21.15.73:0/1336248942 >> v1:172.21.15.150:6789/0 conn(0x7f5a6c0c0d10 legacy=0x7f5a6c153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 -- 172.21.15.73:0/1336248942 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a6c062770 con 0x7f5a6c06b510 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a63fff700 1 -- 172.21.15.73:0/1336248942 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4078294265 0 0) 0x7f5a6c063380 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a71815700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a70813700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c1546f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f5a68009760 tx=0x7f5a68009060).ready entity=mon.1 client_cookie=a8e01862fd447008 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a63fff700 1 -- 172.21.15.73:0/1336248942 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f510 con 0x7f5a6c06b510 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a63fff700 1 -- 172.21.15.73:0/1336248942 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800f670 con 0x7f5a6c06b510 2023-11-28T04:26:07.699 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.696+0000 7f5a63fff700 1 -- 172.21.15.73:0/1336248942 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f9f0 con 0x7f5a6c06b510 2023-11-28T04:26:07.702 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 -- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 msgr2=0x7f5a6c1546f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c1546f0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 rx=0x7f5a68009760 tx=0x7f5a68009060).stop 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 -- 172.21.15.73:0/1336248942 shutdown_connections 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c158b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 --2- 172.21.15.73:0/1336248942 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c1546f0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 -- 172.21.15.73:0/1336248942 >> 172.21.15.73:0/1336248942 conn(0x7f5a6c0c7760 msgr2=0x7f5a6c0c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 -- 172.21.15.73:0/1336248942 shutdown_connections 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.697+0000 7f5a73278700 1 -- 172.21.15.73:0/1336248942 wait complete. 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.698+0000 7f5a73278700 1 Processor -- start 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.698+0000 7f5a73278700 1 -- start start 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c148260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c1490b0 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0ab140 con 0x7f5a6c06b510 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a73278700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a6c0ab420 con 0x7f5a6c0c10f0 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46870/0 (socket says 172.21.15.73:46870) 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 -- 172.21.15.73:0/1234063640 learned_addr learned my addr 172.21.15.73:0/1234063640 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a71014700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c148260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a71815700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 msgr2=0x7f5a6c0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c0c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.704 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 msgr2=0x7f5a6c148260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c148260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.699+0000 7f5a70813700 1 -- 172.21.15.73:0/1234063640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a64009a20 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a70813700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f5a6c064b00 tx=0x7f5a68017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6800f7e0 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a6800f940 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a680092a0 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a6801b660 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.700+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a6c0ab9e0 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7f5a6801ba50 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a6c148e80 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a617fa700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 0x7f5a50061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:07.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a71014700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 msgr2=0x7f5a50061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7f5a6807c0f0 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.701+0000 7f5a71014700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 0x7f5a50061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:07.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.706+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5a6c148e80 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.724 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:07.724+0000 7fd083f24680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:07.736 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:07.734+0000 7f80c38a4680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:07.795 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:07.795+0000 7fd083f24680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:07.873 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:07.873+0000 7fd083f24680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:07.882 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:07.881+0000 7f80c38a4680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:07.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.902+0000 7f5a71014700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 msgr2=0x7f5a50061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:07.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.902+0000 7f5a71014700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 0x7f5a50061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:07.917 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.917+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a6c149220 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.918+0000 7f5a617fa700 1 -- 172.21.15.73:0/1234063640 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) v1 ==== 94+0+4496 (secure 0 0 0) 0x7f5a6c149220 con 0x7f5a6c0c0d10 2023-11-28T04:26:07.918 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:07.918 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":70,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:24:35.086025+0000","modified":"2023-11-28T04:24:37.094342+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9522},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9522":{"gid":9522,"name":"a","rank":0,"incarnation":68,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6837/1535793255","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":1535793255},{"type":"v1","addr":"172.21.15.73:6837","nonce":1535793255}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[24],"metadata_pool":23,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":11}]} 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 msgr2=0x7f5a50061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 0x7f5a50061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 msgr2=0x7f5a6c0aa7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 rx=0x7f5a6c064b00 tx=0x7f5a68017000).stop 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 shutdown_connections 2023-11-28T04:26:07.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5a5005f390 0x7f5a50061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5a6c0c10f0 0x7f5a6c0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5a6c06b510 0x7f5a6c148260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 --2- 172.21.15.73:0/1234063640 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5a6c0c0d10 0x7f5a6c0aa7f0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 >> 172.21.15.73:0/1234063640 conn(0x7f5a6c0c7760 msgr2=0x7f5a6c0c5640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 shutdown_connections 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:07.920+0000 7f5a73278700 1 -- 172.21.15.73:0/1234063640 wait complete. 2023-11-28T04:26:07.921 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 70 2023-11-28T04:26:07.933 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:07.954 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:07.953+0000 7f80c38a4680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:08.030 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.030+0000 7fd083f24680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.050+0000 7fa09b847700 1 Processor -- start 2023-11-28T04:26:08.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.050+0000 7fa09b847700 1 -- start start 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa09b847700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa094150720 0x7fa094150b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa09b847700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 0x7fa0940c2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa09b847700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa094062910 con 0x7fa0941545c0 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa09b847700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0940ab1d0 con 0x7fa094150720 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa09b847700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940c3460 con 0x7fa0940c2b10 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa0995e3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa094150720 0x7fa094150b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 0x7fa0940c2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa098de2700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa0941545c0 0x7fa0940a9c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:37188/0 (socket says 172.21.15.73:37188) 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa098de2700 1 -- 172.21.15.73:0/3094055781 learned_addr learned my addr 172.21.15.73:0/3094055781 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:08.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa08bfff700 1 -- 172.21.15.73:0/3094055781 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26102154 0 0) 0x7fa094062910 con 0x7fa0941545c0 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa08bfff700 1 -- 172.21.15.73:0/3094055781 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa0800033f0 con 0x7fa0941545c0 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 -- 172.21.15.73:0/3094055781 >> v2:172.21.15.150:3300/0 conn(0x7fa094150720 msgr2=0x7fa094150b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 --2- 172.21.15.73:0/3094055781 >> v2:172.21.15.150:3300/0 conn(0x7fa094150720 0x7fa094150b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 -- 172.21.15.73:0/3094055781 >> v1:172.21.15.150:6789/0 conn(0x7fa0941545c0 legacy=0x7fa0940a9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 -- 172.21.15.73:0/3094055781 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa094154ac0 con 0x7fa0940c2b10 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.051+0000 7fa099de4700 1 --2- 172.21.15.73:0/3094055781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 0x7fa0940c2f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fa08c009ee0 tx=0x7fa08c009780).ready entity=mon.2 client_cookie=d8ba79f512530e9e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa08bfff700 1 -- 172.21.15.73:0/3094055781 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c00dd90 con 0x7fa0940c2b10 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa08bfff700 1 -- 172.21.15.73:0/3094055781 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa08c011480 con 0x7fa0940c2b10 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa08bfff700 1 -- 172.21.15.73:0/3094055781 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08c011800 con 0x7fa0940c2b10 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 -- 172.21.15.73:0/3094055781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 msgr2=0x7fa0940c2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 --2- 172.21.15.73:0/3094055781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 0x7fa0940c2f70 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 rx=0x7fa08c009ee0 tx=0x7fa08c009780).stop 2023-11-28T04:26:08.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 -- 172.21.15.73:0/3094055781 shutdown_connections 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 --2- 172.21.15.73:0/3094055781 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0940c2b10 0x7fa0940c2f70 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 --2- 172.21.15.73:0/3094055781 >> v2:172.21.15.150:3300/0 conn(0x7fa094150720 0x7fa094150b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 -- 172.21.15.73:0/3094055781 >> 172.21.15.73:0/3094055781 conn(0x7fa0940c7760 msgr2=0x7fa0940c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 -- 172.21.15.73:0/3094055781 shutdown_connections 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.052+0000 7fa09b847700 1 -- 172.21.15.73:0/3094055781 wait complete. 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 Processor -- start 2023-11-28T04:26:08.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 -- start start 2023-11-28T04:26:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa094150720 0x7fa0940c1a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.060 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0941545c0 0x7fa09414f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.054+0000 7fa09b847700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa094164070 con 0x7fa0940c2b10 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa09b847700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa094163780 con 0x7fa0941545c0 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa09b847700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0940c1ef0 con 0x7fa094150720 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46878/0 (socket says 172.21.15.73:46878) 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 -- 172.21.15.73:0/5740037 learned_addr learned my addr 172.21.15.73:0/5740037 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa098de2700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa094150720 0x7fa0940c1a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa099de4700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0941545c0 0x7fa09414f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.061 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 -- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa094150720 msgr2=0x7fa0940c1a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa094150720 0x7fa0940c1a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 -- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0941545c0 msgr2=0x7fa09414f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0941545c0 0x7fa09414f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 -- 172.21.15.73:0/5740037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08c009a20 con 0x7fa0940c2b10 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0995e3700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 rx=0x7fa08400fa10 tx=0x7fa0840080e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa08400a5b0 con 0x7fa0940c2b10 2023-11-28T04:26:08.062 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0840116a0 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0941532f0 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa084011a20 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.055+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa094163d00 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.056+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09406c860 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.057+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fa084012460 con 0x7fa0940c2b10 2023-11-28T04:26:08.063 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.057+0000 7fa0897fa700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 0x7fa078061850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.057+0000 7fa098de2700 1 -- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 msgr2=0x7fa078061850 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:08.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.057+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7fa08407aca0 con 0x7fa0940c2b10 2023-11-28T04:26:08.064 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.057+0000 7fa098de2700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 0x7fa078061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:08.065 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.061+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa08404ba20 con 0x7fa0940c2b10 2023-11-28T04:26:08.071 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:08.071+0000 7f80c38a4680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:08.106 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.106+0000 7fd083f24680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:08.181 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:08.180+0000 7f80c38a4680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:08.224 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.224+0000 7fd083f24680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:08.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.229+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa0940c5300 con 0x7fa0940c2b10 2023-11-28T04:26:08.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.230+0000 7fa0897fa700 1 -- 172.21.15.73:0/5740037 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v199) v1 ==== 74+0+17299 (secure 0 0 0) 0x7fa084016070 con 0x7fa0940c2b10 2023-11-28T04:26:08.230 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:08.230 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":199,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:06.279937+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"188","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":23,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:24:33.630693+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"196","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":24,"pool_name":"cephfs_data","create_time":"2023-11-28T04:24:34.055195+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"199","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":199,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2520251340":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6833/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:6832/136129":"2023-11-29T04:24:46.386058+0000","172.21.15.73:0/1497049717":"2023-11-29T04:24:46.386058+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":24,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.231+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 msgr2=0x7fa078061850 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.231+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 0x7fa078061850 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.231+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 msgr2=0x7fa094152ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.231+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 rx=0x7fa08400fa10 tx=0x7fa0840080e0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 shutdown_connections 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fa07805f390 0x7fa078061850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0941545c0 0x7fa09414f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0940c2b10 0x7fa094152ca0 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 --2- 172.21.15.73:0/5740037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa094150720 0x7fa0940c1a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 >> 172.21.15.73:0/5740037 conn(0x7fa0940c7760 msgr2=0x7fa0940ad350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:08.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 shutdown_connections 2023-11-28T04:26:08.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.232+0000 7fa09b847700 1 -- 172.21.15.73:0/5740037 wait complete. 2023-11-28T04:26:08.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T04:26:08.342 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.342+0000 7fd083f24680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:08.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.362+0000 7fea18927700 1 Processor -- start 2023-11-28T04:26:08.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.362+0000 7fea18927700 1 -- start start 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea18927700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fea14098d10 0x7fea140c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea18927700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea140c6190 0x7fea14063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea18927700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea140c72d0 con 0x7fea14064350 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea18927700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea14154bc0 con 0x7fea140c6190 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea18927700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea140c66c0 con 0x7fea14098d10 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea11d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea140c6190 0x7fea14063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea11d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fea140c6190 0x7fea14063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46886/0 (socket says 172.21.15.73:46886) 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea11d9b700 1 -- 172.21.15.73:0/2535307782 learned_addr learned my addr 172.21.15.73:0/2535307782 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea12d9d700 1 --1- 172.21.15.73:0/2535307782 >> v1:172.21.15.73:6789/0 conn(0x7fea14064350 0x7fea14158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34746/0 (socket says 172.21.15.73:34746) 2023-11-28T04:26:08.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.363+0000 7fea1259c700 1 --2- 172.21.15.73:0/2535307782 >> v2:172.21.15.150:3300/0 conn(0x7fea14098d10 0x7fea140c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea11d9b700 1 -- 172.21.15.73:0/2535307782 >> v2:172.21.15.150:3300/0 conn(0x7fea14098d10 msgr2=0x7fea140c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea11d9b700 1 --2- 172.21.15.73:0/2535307782 >> v2:172.21.15.150:3300/0 conn(0x7fea14098d10 0x7fea140c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea11d9b700 1 -- 172.21.15.73:0/2535307782 >> v1:172.21.15.73:6789/0 conn(0x7fea14064350 legacy=0x7fea14158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea1159a700 1 -- 172.21.15.73:0/2535307782 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1202227842 0 0) 0x7fea140c72d0 con 0x7fea14064350 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea11d9b700 1 -- 172.21.15.73:0/2535307782 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea140c69a0 con 0x7fea140c6190 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea11d9b700 1 --2- 172.21.15.73:0/2535307782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea140c6190 0x7fea14063e60 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 rx=0x7fea14083f80 tx=0x7fea040090f0).ready entity=mon.0 client_cookie=6043b83bfadb5eaf server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea1159a700 1 -- 172.21.15.73:0/2535307782 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0400b520 con 0x7fea140c6190 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea1159a700 1 -- 172.21.15.73:0/2535307782 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0400b680 con 0x7fea140c6190 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.364+0000 7fea1159a700 1 -- 172.21.15.73:0/2535307782 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea0400ba00 con 0x7fea140c6190 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 -- 172.21.15.73:0/2535307782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea140c6190 msgr2=0x7fea14063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 --2- 172.21.15.73:0/2535307782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea140c6190 0x7fea14063e60 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 rx=0x7fea14083f80 tx=0x7fea040090f0).stop 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 -- 172.21.15.73:0/2535307782 shutdown_connections 2023-11-28T04:26:08.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 --2- 172.21.15.73:0/2535307782 >> v2:172.21.15.150:3300/0 conn(0x7fea14098d10 0x7fea140c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 --2- 172.21.15.73:0/2535307782 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea140c6190 0x7fea14063e60 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 -- 172.21.15.73:0/2535307782 >> 172.21.15.73:0/2535307782 conn(0x7fea140cb310 msgr2=0x7fea140cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:08.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 -- 172.21.15.73:0/2535307782 shutdown_connections 2023-11-28T04:26:08.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.365+0000 7fea18927700 1 -- 172.21.15.73:0/2535307782 wait complete. 2023-11-28T04:26:08.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.366+0000 7fea18927700 1 Processor -- start 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.366+0000 7fea18927700 1 -- start start 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 0x7fea1414eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea140c6190 0x7fea141564d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14155b50 con 0x7fea14098d10 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea14155830 con 0x7fea14064350 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea140af370 con 0x7fea140c6190 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46892/0 (socket says 172.21.15.73:46892) 2023-11-28T04:26:08.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 -- 172.21.15.73:0/413278525 learned_addr learned my addr 172.21.15.73:0/413278525 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea1259c700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 0x7fea1414eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea12d9d700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea140c6190 0x7fea141564d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea140c6190 msgr2=0x7fea141564d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea140c6190 0x7fea141564d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 msgr2=0x7fea1414eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 0x7fea1414eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 -- 172.21.15.73:0/413278525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea04009370 con 0x7fea14098d10 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea1259c700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 0x7fea1414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea11d9b700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7fea04009130 tx=0x7fea04017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04000bc0 con 0x7fea14098d10 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea0400b7f0 con 0x7fea14098d10 2023-11-28T04:26:08.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.367+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea140af650 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.368+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea04000d20 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.368+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea1414eff0 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.369+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea1406c860 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.369+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 94) v1 ==== 75662+0+0 (secure 0 0 0) 0x7fea0401b8b0 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.370+0000 7fea027fc700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 0x7fe9f4061820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.370+0000 7fea1259c700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 msgr2=0x7fe9f4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.370+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6349+0+0 (secure 0 0 0) 0x7fea0407bff0 con 0x7fea14098d10 2023-11-28T04:26:08.372 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.370+0000 7fea1259c700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 0x7fe9f4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:08.374 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.374+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fea0404ce80 con 0x7fea14098d10 2023-11-28T04:26:08.476 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:08.475+0000 7f80c38a4680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:08.548 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:08.547+0000 7f80c38a4680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.571+0000 7fea1259c700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 msgr2=0x7fe9f4061820 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:08.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.571+0000 7fea1259c700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 0x7fe9f4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:08.589 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.589+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fea140afaf0 con 0x7fea14098d10 2023-11-28T04:26:08.644 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.644+0000 7fd083f24680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:08.716 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.716+0000 7fd083f24680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:08.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.737+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 95) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fea040799e0 con 0x7fea14098d10 2023-11-28T04:26:08.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.738+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 msgr2=0x7fe9f4061820 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:08.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:08.738+0000 7fea027fc700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 0x7fe9f4061820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:08.740 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.740 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.740 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.740 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.741 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T04:26:08.741 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.741 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.741 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T04:26:08.741 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -10> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -9> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -8> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -7> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -6> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -5> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) _finish_flush got (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.756 INFO:tasks.ceph.mds.a.smithi073.stderr: -4> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error (108) Cannot send after transport endpoint shutdown 2023-11-28T04:26:08.757 INFO:tasks.ceph.mds.a.smithi073.stderr: -3> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 mds.0.journaler.mdlog(rw) handle_write_error: multiple write errors, handler already called 2023-11-28T04:26:08.757 INFO:tasks.ceph.mds.a.smithi073.stderr: -1> 2023-11-28T04:26:08.740+0000 7f1eb4c9e700 -1 MDSIOContextBase: failed with -108, restarting... 2023-11-28T04:26:08.776 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:26:08.793 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fcffe6b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.793 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fcffe6b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.793 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fcffe6b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.793 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fcffe6b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.793 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fcffe6b6700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.794 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fd0016bc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.794 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fd0016bc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.794 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fd0016bc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.795 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fd0016bc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:08.795 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:08.793+0000 7fd0016bc700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:09.289 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T04:26:09.288+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T04:26:09.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.295+0000 7fea027fc700 1 -- 172.21.15.73:0/413278525 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v71) v1 ==== 161+0+0 (secure 0 0 0) 0x7fea040543f0 con 0x7fea14098d10 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 msgr2=0x7fea14155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 rx=0x7fea04009130 tx=0x7fea04017000).stop 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 shutdown_connections 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe9f405f360 0x7fe9f4061820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fea140c6190 0x7fea141564d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fea14064350 0x7fea1414eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 --2- 172.21.15.73:0/413278525 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fea14098d10 0x7fea14155fe0 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 >> 172.21.15.73:0/413278525 conn(0x7fea140cb310 msgr2=0x7fea140cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 shutdown_connections 2023-11-28T04:26:09.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.298+0000 7fea18927700 1 -- 172.21.15.73:0/413278525 wait complete. 2023-11-28T04:26:09.300 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T04:26:09.312 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T04:26:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 Processor -- start 2023-11-28T04:26:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 -- start start 2023-11-28T04:26:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7ab4062e10 0x7f7ab40a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab40c7e30 con 0x7f7ab41547f0 2023-11-28T04:26:09.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ab40bcf40 con 0x7f7ab40c86f0 2023-11-28T04:26:09.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.427+0000 7f7aba10a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ab40c6b20 con 0x7f7ab4062e10 2023-11-28T04:26:09.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab37fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7ab41547f0 0x7f7ab40c6fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34760/0 (socket says 172.21.15.73:34760) 2023-11-28T04:26:09.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab37fe700 1 -- 172.21.15.73:0/1413949443 learned_addr learned my addr 172.21.15.73:0/1413949443 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/1413949443 >> v2:172.21.15.150:3300/0 conn(0x7f7ab4062e10 0x7f7ab40a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2906215340 0 0) 0x7f7ab40c7e30 con 0x7f7ab41547f0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a9c0033f0 con 0x7f7ab41547f0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 640726944 0 0) 0x7f7ab40bcf40 con 0x7f7ab40c86f0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ab40c7e30 con 0x7f7ab40c86f0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2694831302 0 0) 0x7f7a9c0033f0 con 0x7f7ab41547f0 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 >> v2:172.21.15.150:3300/0 conn(0x7f7ab4062e10 msgr2=0x7f7ab40a72e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 --2- 172.21.15.73:0/1413949443 >> v2:172.21.15.150:3300/0 conn(0x7f7ab4062e10 0x7f7ab40a72e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 >> v1:172.21.15.150:6789/0 conn(0x7f7ab40c86f0 legacy=0x7f7ab4062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab40a77d0 con 0x7f7ab41547f0 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7aa40017f0 con 0x7f7ab41547f0 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 >> v1:172.21.15.73:6789/0 conn(0x7f7ab41547f0 legacy=0x7f7ab40c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a9c004690 0x7f7a9c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 0x7f7a9c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab40c6b20 con 0x7f7a9c005050 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00bfc0 con 0x7f7a9c004690 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c00c180 con 0x7f7a9c003a70 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab37fe700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 0x7f7a9c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.428+0000 7f7ab2ffd700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a9c004690 0x7f7a9c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 -- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c003a70 msgr2=0x7f7a9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:09.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c003a70 0x7f7a9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 -- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a9c004690 msgr2=0x7f7a9c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7a9c004690 0x7f7a9c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 -- 172.21.15.73:0/1413949443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0033f0 con 0x7f7a9c005050 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 -- 172.21.15.73:0/1413949443 reap_dead start 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 0x7f7a9c00b8e0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f7ab40c2630 tx=0x7f7aac009060).ready entity=mon.0 client_cookie=63a91ba6d809e019 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac009e90 con 0x7f7a9c005050 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aac009430 con 0x7f7a9c005050 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7ab27fc700 1 -- 172.21.15.73:0/1413949443 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aac00e840 con 0x7f7a9c005050 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7aba10a700 1 -- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 msgr2=0x7f7a9c00b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7aba10a700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 0x7f7a9c00b8e0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 rx=0x7f7ab40c2630 tx=0x7f7aac009060).stop 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7aba10a700 1 -- 172.21.15.73:0/1413949443 shutdown_connections 2023-11-28T04:26:09.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7aba10a700 1 --2- 172.21.15.73:0/1413949443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7a9c005050 0x7f7a9c00b8e0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.429+0000 7f7aba10a700 1 -- 172.21.15.73:0/1413949443 >> 172.21.15.73:0/1413949443 conn(0x7f7ab40642d0 msgr2=0x7f7ab40c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.430+0000 7f7aba10a700 1 -- 172.21.15.73:0/1413949443 shutdown_connections 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.430+0000 7f7aba10a700 1 -- 172.21.15.73:0/1413949443 wait complete. 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.431+0000 7f7aba10a700 1 Processor -- start 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.431+0000 7f7aba10a700 1 -- start start 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.431+0000 7f7aba10a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c005050 0x7f7ab41507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ab414e5f0 0x7f7ab414f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab414f700 con 0x7f7ab414e170 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab414f9e0 con 0x7f7ab414e5f0 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ab414fcc0 con 0x7f7a9c005050 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46992/0 (socket says 172.21.15.73:46992) 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 -- 172.21.15.73:0/117721180 learned_addr learned my addr 172.21.15.73:0/117721180 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab37fe700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c005050 0x7f7ab41507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab3fff700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ab414e5f0 0x7f7ab414f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 -- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c005050 msgr2=0x7f7ab41507a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c005050 0x7f7ab41507a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 -- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ab414e5f0 msgr2=0x7f7ab414f0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ab414e5f0 0x7f7ab414f0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 -- 172.21.15.73:0/117721180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ab414ffa0 con 0x7f7ab414e170 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7ab2ffd700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 rx=0x7f7aa80010e0 tx=0x7f7aa8007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8007240 con 0x7f7ab414e170 2023-11-28T04:26:09.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7aa80073a0 con 0x7f7ab414e170 2023-11-28T04:26:09.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ab4150280 con 0x7f7ab414e170 2023-11-28T04:26:09.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7aa80066a0 con 0x7f7ab414e170 2023-11-28T04:26:09.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.432+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ab4150c90 con 0x7f7ab414e170 2023-11-28T04:26:09.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.433+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ab406c860 con 0x7f7ab414e170 2023-11-28T04:26:09.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.433+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 95) v1 ==== 75270+0+0 (secure 0 0 0) 0x7f7aa8006a90 con 0x7f7ab414e170 2023-11-28T04:26:09.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.434+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f7aa8079b00 con 0x7f7ab414e170 2023-11-28T04:26:09.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.438+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7aa8049af0 con 0x7f7ab414e170 2023-11-28T04:26:09.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.644+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f7ab4065290 con 0x7f7ab414e170 2023-11-28T04:26:09.741 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.740+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f7aa8006d40 con 0x7f7ab414e170 2023-11-28T04:26:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.740+0000 7f7a9b7fe700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 0x7f7aa0067b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.740+0000 7f7ab37fe700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 0x7f7aa0067b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:09.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:09.741+0000 7f7ab37fe700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 0x7f7aa0067b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7aa40019e0 tx=0x7f7aa4007f40).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:10.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.296+0000 7f7a9b7fe700 1 -- 172.21.15.73:0/117721180 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v72) v1 ==== 104+0+0 (secure 0 0 0) 0x7f7aa8070090 con 0x7f7ab414e170 2023-11-28T04:26:10.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 msgr2=0x7f7aa0067b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 0x7f7aa0067b40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 rx=0x7f7aa40019e0 tx=0x7f7aa4007f40).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 msgr2=0x7f7ab414ec00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 rx=0x7f7aa80010e0 tx=0x7f7aa8007bb0).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 shutdown_connections 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f7aa00302c0 0x7f7aa0067b40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7ab414e5f0 0x7f7ab414f0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7a9c005050 0x7f7ab41507a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 --2- 172.21.15.73:0/117721180 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7ab414e170 0x7f7ab414ec00 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 >> 172.21.15.73:0/117721180 conn(0x7f7ab40642d0 msgr2=0x7f7ab4151020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:10.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 shutdown_connections 2023-11-28T04:26:10.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.298+0000 7f7aba10a700 1 -- 172.21.15.73:0/117721180 wait complete. 2023-11-28T04:26:10.313 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T04:26:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.430+0000 7f60559f6700 1 Processor -- start 2023-11-28T04:26:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.430+0000 7f60559f6700 1 -- start start 2023-11-28T04:26:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f60559f6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f605013b0b0 0x7f60500c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f60559f6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60500c6550 0x7f60500c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f60559f6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6050156d50 con 0x7f605013acd0 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f60559f6700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f60500c69a0 con 0x7f605013b0b0 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f60559f6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050156140 con 0x7f60500c6550 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f604e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f605013b0b0 0x7f60500c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f604effd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f605013acd0 0x7f605013b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34762/0 (socket says 172.21.15.73:34762) 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f604e7fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f605013b0b0 0x7f60500c6060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:46996/0 (socket says 172.21.15.73:46996) 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.431+0000 7f604effd700 1 -- 172.21.15.73:0/351805775 learned_addr learned my addr 172.21.15.73:0/351805775 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:10.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604f7fe700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60500c6550 0x7f60500c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604e7fc700 1 -- 172.21.15.73:0/351805775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60500c6550 msgr2=0x7f60500c7100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604e7fc700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60500c6550 0x7f60500c7100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604e7fc700 1 -- 172.21.15.73:0/351805775 >> v1:172.21.15.73:6789/0 conn(0x7f605013acd0 legacy=0x7f605013b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604e7fc700 1 -- 172.21.15.73:0/351805775 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6050156420 con 0x7f605013b0b0 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604dffb700 1 -- 172.21.15.73:0/351805775 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2401872961 0 0) 0x7f6050156d50 con 0x7f605013acd0 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604e7fc700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f60500c6060 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 rx=0x7f6044009760 tx=0x7f6044009060).ready entity=mon.0 client_cookie=d7301eebd3be9e3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:10.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604dffb700 1 -- 172.21.15.73:0/351805775 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60440090d0 con 0x7f605013b0b0 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604dffb700 1 -- 172.21.15.73:0/351805775 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f604400b580 con 0x7f605013b0b0 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f604dffb700 1 -- 172.21.15.73:0/351805775 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f604400b900 con 0x7f605013b0b0 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 -- 172.21.15.73:0/351805775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 msgr2=0x7f60500c6060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f60500c6060 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 rx=0x7f6044009760 tx=0x7f6044009060).stop 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 -- 172.21.15.73:0/351805775 shutdown_connections 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f60500c6550 0x7f60500c7100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 --2- 172.21.15.73:0/351805775 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f60500c6060 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 -- 172.21.15.73:0/351805775 >> 172.21.15.73:0/351805775 conn(0x7f60500628f0 msgr2=0x7f6050062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 -- 172.21.15.73:0/351805775 shutdown_connections 2023-11-28T04:26:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.432+0000 7f60559f6700 1 -- 172.21.15.73:0/351805775 wait complete. 2023-11-28T04:26:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 Processor -- start 2023-11-28T04:26:10.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 -- start start 2023-11-28T04:26:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60500c6550 0x7f60501664a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f605013acd0 0x7f605014ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050166010 con 0x7f605013b0b0 2023-11-28T04:26:10.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6050166990 con 0x7f60500c6550 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.434+0000 7f60559f6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6050162fa0 con 0x7f605013acd0 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47010/0 (socket says 172.21.15.73:47010) 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 -- 172.21.15.73:0/1989478466 learned_addr learned my addr 172.21.15.73:0/1989478466 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604effd700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60500c6550 0x7f60501664a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604e7fc700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f605013acd0 0x7f605014ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 -- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f605013acd0 msgr2=0x7f605014ec60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f605013acd0 0x7f605014ec60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 -- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60500c6550 msgr2=0x7f60501664a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60500c6550 0x7f60501664a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 -- 172.21.15.73:0/1989478466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60440092a0 con 0x7f605013b0b0 2023-11-28T04:26:10.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f604f7fe700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 rx=0x7f604800d890 tx=0x7f6048008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f60480125f0 con 0x7f605013b0b0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6048012750 con 0x7f605013b0b0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6050163280 con 0x7f605013b0b0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6048012ad0 con 0x7f605013b0b0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.435+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f605014f150 con 0x7f605013b0b0 2023-11-28T04:26:10.446 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.436+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6050166180 con 0x7f605013b0b0 2023-11-28T04:26:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.436+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f6048013480 con 0x7f605013b0b0 2023-11-28T04:26:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.437+0000 7f6036ffd700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 0x7f603c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.437+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(201..201 src has 1..201) v4 ==== 6665+0+0 (secure 0 0 0) 0x7f6048079f20 con 0x7f605013b0b0 2023-11-28T04:26:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.437+0000 7f604effd700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 0x7f603c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:10.447 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.437+0000 7f604effd700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 0x7f603c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6040001560 tx=0x7f6040009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:10.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.441+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f60480a8af0 con 0x7f605013b0b0 2023-11-28T04:26:10.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:10.605+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f605006c860 con 0x7f605013b0b0 2023-11-28T04:26:11.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.298+0000 7f6036ffd700 1 -- 172.21.15.73:0/1989478466 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v202) v1 ==== 181+0+0 (secure 0 0 0) 0x7f604804eb70 con 0x7f605013b0b0 2023-11-28T04:26:11.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 msgr2=0x7f603c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 0x7f603c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f6040001560 tx=0x7f6040009040).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 msgr2=0x7f6050162500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 rx=0x7f604800d890 tx=0x7f6048008040).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 shutdown_connections 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f603c05f030 0x7f603c0614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f605013b0b0 0x7f6050162500 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f60500c6550 0x7f60501664a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 --2- 172.21.15.73:0/1989478466 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f605013acd0 0x7f605014ec60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 >> 172.21.15.73:0/1989478466 conn(0x7f60500628f0 msgr2=0x7f6050155410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:11.302 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.300+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 shutdown_connections 2023-11-28T04:26:11.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.301+0000 7f60559f6700 1 -- 172.21.15.73:0/1989478466 wait complete. 2023-11-28T04:26:11.303 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T04:26:11.314 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T04:26:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.431+0000 7f8d91784700 1 Processor -- start 2023-11-28T04:26:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.431+0000 7f8d91784700 1 -- start start 2023-11-28T04:26:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d91784700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c13b0b0 0x7f8d8c0c6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d91784700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d8c0c6550 0x7f8d8c0c7100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d91784700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d8c156d50 con 0x7f8d8c13acd0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d91784700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8c0c6970 con 0x7f8d8c0c6550 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d91784700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c156140 con 0x7f8d8c13b0b0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8d8c0c6550 0x7f8d8c0c7100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8d8c13acd0 0x7f8d8c13b660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34776/0 (socket says 172.21.15.73:34776) 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8affd700 1 -- 172.21.15.73:0/2544884696 learned_addr learned my addr 172.21.15.73:0/2544884696 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d83fff700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c13b0b0 0x7f8d8c0c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 -- 172.21.15.73:0/2544884696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c13b0b0 msgr2=0x7f8d8c0c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c13b0b0 0x7f8d8c0c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 -- 172.21.15.73:0/2544884696 >> v1:172.21.15.73:6789/0 conn(0x7f8d8c13acd0 legacy=0x7f8d8c13b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 -- 172.21.15.73:0/2544884696 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d8c156420 con 0x7f8d8c0c6550 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8a7fc700 1 -- 172.21.15.73:0/2544884696 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1447432033 0 0) 0x7f8d8c156d50 con 0x7f8d8c13acd0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8b7fe700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c0c6550 0x7f8d8c0c7100 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7f8d84009e60 tx=0x7f8d84009780).ready entity=mon.0 client_cookie=939a895ebef9b569 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8a7fc700 1 -- 172.21.15.73:0/2544884696 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d8400e070 con 0x7f8d8c0c6550 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8a7fc700 1 -- 172.21.15.73:0/2544884696 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d8400bc30 con 0x7f8d8c0c6550 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.432+0000 7f8d8a7fc700 1 -- 172.21.15.73:0/2544884696 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d84012410 con 0x7f8d8c0c6550 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 -- 172.21.15.73:0/2544884696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c0c6550 msgr2=0x7f8d8c0c7100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c0c6550 0x7f8d8c0c7100 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 rx=0x7f8d84009e60 tx=0x7f8d84009780).stop 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 -- 172.21.15.73:0/2544884696 shutdown_connections 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c0c6550 0x7f8d8c0c7100 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 --2- 172.21.15.73:0/2544884696 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c13b0b0 0x7f8d8c0c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 -- 172.21.15.73:0/2544884696 >> 172.21.15.73:0/2544884696 conn(0x7f8d8c0628f0 msgr2=0x7f8d8c062d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 -- 172.21.15.73:0/2544884696 shutdown_connections 2023-11-28T04:26:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.433+0000 7f8d91784700 1 -- 172.21.15.73:0/2544884696 wait complete. 2023-11-28T04:26:11.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.434+0000 7f8d91784700 1 Processor -- start 2023-11-28T04:26:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.434+0000 7f8d91784700 1 -- start start 2023-11-28T04:26:11.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 0x7f8d8c0c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 0x7f8d8c151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0c14d0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0c0640 con 0x7f8d8c13b0b0 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d8c0c08c0 con 0x7f8d8c0c6550 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47026/0 (socket says 172.21.15.73:47026) 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 -- 172.21.15.73:0/770153448 learned_addr learned my addr 172.21.15.73:0/770153448 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d8b7fe700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 0x7f8d8c151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 -- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 msgr2=0x7f8d8c0c3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d8affd700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 0x7f8d8c0c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 0x7f8d8c0c3c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 -- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 msgr2=0x7f8d8c151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 0x7f8d8c151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 -- 172.21.15.73:0/770153448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d84009a20 con 0x7f8d8c13acd0 2023-11-28T04:26:11.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d8b7fe700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 0x7f8d8c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d8affd700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 0x7f8d8c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d83fff700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 rx=0x7f8d7400d890 tx=0x7f8d74008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d740125f0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d74012750 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d8c0c0ba0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.435+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d74012ad0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.436+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d8c0c1160 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.437+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8c0c1640 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.437+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 96) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f8d74013480 con 0x7f8d8c13acd0 2023-11-28T04:26:11.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.437+0000 7f8d82ffd700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 0x7f8d6c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:11.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.437+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(202..202 src has 1..202) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f8d74079ce0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.437+0000 7f8d8affd700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 0x7f8d6c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:11.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.438+0000 7f8d8affd700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 0x7f8d6c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8d8c138460 tx=0x7f8d7c009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:11.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.441+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8d7404b0b0 con 0x7f8d8c13acd0 2023-11-28T04:26:11.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.606+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8d8c06c860 con 0x7f8d8c13acd0 2023-11-28T04:26:11.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:11.745+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8d74077c10 con 0x7f8d8c13acd0 2023-11-28T04:26:12.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.273+0000 7f8d82ffd700 1 -- 172.21.15.73:0/770153448 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v203) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8d74072f10 con 0x7f8d8c13acd0 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 msgr2=0x7f8d6c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 0x7f8d6c0614f0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f8d8c138460 tx=0x7f8d7c009040).stop 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 msgr2=0x7f8d8c0c0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 rx=0x7f8d7400d890 tx=0x7f8d74008040).stop 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 shutdown_connections 2023-11-28T04:26:12.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8d6c05f030 0x7f8d6c0614f0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8d8c13b0b0 0x7f8d8c151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8d8c0c6550 0x7f8d8c0c3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 --2- 172.21.15.73:0/770153448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8d8c13acd0 0x7f8d8c0c0150 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 >> 172.21.15.73:0/770153448 conn(0x7f8d8c0628f0 msgr2=0x7f8d8c0c97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 shutdown_connections 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.276+0000 7f8d91784700 1 -- 172.21.15.73:0/770153448 wait complete. 2023-11-28T04:26:12.278 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T04:26:12.291 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T04:26:12.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.405+0000 7febcf326700 1 Processor -- start 2023-11-28T04:26:12.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.405+0000 7febcf326700 1 -- start start 2023-11-28T04:26:12.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcf326700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcf326700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7febc8158320 0x7febc81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcf326700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febc8158cc0 con 0x7febc8098d10 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcf326700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febc80c72d0 con 0x7febc8158320 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcf326700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8064570 con 0x7febc80c6190 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcd8c3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7febc8158320 0x7febc81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcd8c3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7febc8158320 0x7febc81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47032/0 (socket says 172.21.15.73:47032) 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.406+0000 7febcd8c3700 1 -- 172.21.15.73:0/1182872401 learned_addr learned my addr 172.21.15.73:0/1182872401 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcd8c3700 1 -- 172.21.15.73:0/1182872401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 msgr2=0x7febc80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcd8c3700 1 --2- 172.21.15.73:0/1182872401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcd8c3700 1 -- 172.21.15.73:0/1182872401 >> v1:172.21.15.150:6789/0 conn(0x7febc8098d10 legacy=0x7febc8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:12.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcd8c3700 1 -- 172.21.15.73:0/1182872401 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc8154bc0 con 0x7febc8158320 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcd8c3700 1 --2- 172.21.15.73:0/1182872401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8158320 0x7febc81587d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7febc0006fd0 tx=0x7febc00090f0).ready entity=mon.0 client_cookie=6459d8e5fbf813aa server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febbffff700 1 -- 172.21.15.73:0/1182872401 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc000b600 con 0x7febc8158320 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febbffff700 1 -- 172.21.15.73:0/1182872401 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc000b760 con 0x7febc8158320 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febbffff700 1 -- 172.21.15.73:0/1182872401 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc000bae0 con 0x7febc8158320 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 -- 172.21.15.73:0/1182872401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8158320 msgr2=0x7febc81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 --2- 172.21.15.73:0/1182872401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8158320 0x7febc81587d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 rx=0x7febc0006fd0 tx=0x7febc00090f0).stop 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 -- 172.21.15.73:0/1182872401 shutdown_connections 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 --2- 172.21.15.73:0/1182872401 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8158320 0x7febc81587d0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 --2- 172.21.15.73:0/1182872401 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.407+0000 7febcf326700 1 -- 172.21.15.73:0/1182872401 >> 172.21.15.73:0/1182872401 conn(0x7febc80cb310 msgr2=0x7febc80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.408+0000 7febcf326700 1 -- 172.21.15.73:0/1182872401 shutdown_connections 2023-11-28T04:26:12.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.408+0000 7febcf326700 1 -- 172.21.15.73:0/1182872401 wait complete. 2023-11-28T04:26:12.414 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.409+0000 7febcf326700 1 Processor -- start 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.409+0000 7febcf326700 1 -- start start 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.409+0000 7febcf326700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.409+0000 7febcf326700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.409+0000 7febcf326700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febc8158320 0x7febc8154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcf326700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc8156140 con 0x7febc8098d10 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcf326700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febc80c53b0 con 0x7febc8158320 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcf326700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febc8155530 con 0x7febc80c6190 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47046/0 (socket says 172.21.15.73:47046) 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 -- 172.21.15.73:0/728292702 learned_addr learned my addr 172.21.15.73:0/728292702 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd8c3700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febc8158320 0x7febc8154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcc8c1700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 -- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 msgr2=0x7febc80c4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 -- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febc8158320 msgr2=0x7febc8154d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febc8158320 0x7febc8154d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 -- 172.21.15.73:0/728292702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febc0009370 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcd0c2700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7febc400c810 tx=0x7febc40025e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4010510 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febc40106b0 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febc8155810 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febc4010af0 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.410+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febc8155dd0 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.411+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febc8156420 con 0x7febc8098d10 2023-11-28T04:26:12.416 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.412+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7febc401b400 con 0x7febc8098d10 2023-11-28T04:26:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.412+0000 7febbd7fa700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 0x7febac061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.412+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7febc4079f30 con 0x7febc8098d10 2023-11-28T04:26:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.412+0000 7febcc8c1700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 0x7febac061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.412+0000 7febcc8c1700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 0x7febac061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7febc8165a80 tx=0x7febb8009060).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.417 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.416+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7febc404b2d0 con 0x7febc8098d10 2023-11-28T04:26:12.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.586+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7febc806c860 con 0x7febc8098d10 2023-11-28T04:26:12.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.589+0000 7febbd7fa700 1 -- 172.21.15.73:0/728292702 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v40) v1 ==== 120+0+0 (secure 0 0 0) 0x7febc404e8f0 con 0x7febc8098d10 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 msgr2=0x7febac061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 0x7febac061610 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7febc8165a80 tx=0x7febb8009060).stop 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 msgr2=0x7febc8165590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 rx=0x7febc400c810 tx=0x7febc40025e0).stop 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 shutdown_connections 2023-11-28T04:26:12.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7febac05f150 0x7febac061610 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7febc8158320 0x7febc8154d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7febc8098d10 0x7febc8165590 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 --2- 172.21.15.73:0/728292702 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7febc80c6190 0x7febc80c4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 >> 172.21.15.73:0/728292702 conn(0x7febc80cb310 msgr2=0x7febc80ca930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 shutdown_connections 2023-11-28T04:26:12.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.592+0000 7febcf326700 1 -- 172.21.15.73:0/728292702 wait complete. 2023-11-28T04:26:12.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T04:26:12.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.718+0000 7f44a79c5700 1 Processor -- start 2023-11-28T04:26:12.719 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.718+0000 7f44a79c5700 1 -- start start 2023-11-28T04:26:12.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a79c5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f44a0098d10 0x7f44a00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a79c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a01587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.721 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a79c5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f44a0158cc0 con 0x7f44a00c6190 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a79c5700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f44a00c72d0 con 0x7f44a0098d10 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a79c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0064570 con 0x7f44a0158320 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a5761700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f44a0098d10 0x7f44a00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a4f60700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f44a00c6190 0x7f44a0063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34792/0 (socket says 172.21.15.73:34792) 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a4f60700 1 -- 172.21.15.73:0/2254569991 learned_addr learned my addr 172.21.15.73:0/2254569991 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a5761700 1 -- 172.21.15.73:0/2254569991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 msgr2=0x7f44a01587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a5761700 1 --2- 172.21.15.73:0/2254569991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a01587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a5761700 1 -- 172.21.15.73:0/2254569991 >> v1:172.21.15.73:6789/0 conn(0x7f44a00c6190 legacy=0x7f44a0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f4497fff700 1 -- 172.21.15.73:0/2254569991 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1656978088 0 0) 0x7f4490001110 con 0x7f44a00c6190 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.719+0000 7f44a5761700 1 -- 172.21.15.73:0/2254569991 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f44a0154bc0 con 0x7f44a0098d10 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a5761700 1 --2- 172.21.15.73:0/2254569991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00c7760 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f449c009e60 tx=0x7f449c009780).ready entity=mon.0 client_cookie=9db7c6e521e95c2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f4497fff700 1 -- 172.21.15.73:0/2254569991 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c00e070 con 0x7f44a0098d10 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f4497fff700 1 -- 172.21.15.73:0/2254569991 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c00bc30 con 0x7f44a0098d10 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f4497fff700 1 -- 172.21.15.73:0/2254569991 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c012410 con 0x7f44a0098d10 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 -- 172.21.15.73:0/2254569991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 msgr2=0x7f44a00c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2254569991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00c7760 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 rx=0x7f449c009e60 tx=0x7f449c009780).stop 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 -- 172.21.15.73:0/2254569991 shutdown_connections 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2254569991 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a01587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2254569991 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00c7760 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 -- 172.21.15.73:0/2254569991 >> 172.21.15.73:0/2254569991 conn(0x7f44a00cb310 msgr2=0x7f44a00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 -- 172.21.15.73:0/2254569991 shutdown_connections 2023-11-28T04:26:12.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.720+0000 7f44a79c5700 1 -- 172.21.15.73:0/2254569991 wait complete. 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.721+0000 7f44a79c5700 1 Processor -- start 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 -- start start 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00c6190 0x7f44a014b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a00abeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00aa150 con 0x7f44a0098d10 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f44a0154f80 con 0x7f44a00c6190 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a79c5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f44a00a98b0 con 0x7f44a0158320 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.722+0000 7f44a5761700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47066/0 (socket says 172.21.15.73:47066) 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 -- 172.21.15.73:0/2515929377 learned_addr learned my addr 172.21.15.73:0/2515929377 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5f62700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a00abeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a4f60700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00c6190 0x7f44a014b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 -- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 msgr2=0x7f44a00abeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a00abeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 -- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00c6190 msgr2=0x7f44a014b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00c6190 0x7f44a014b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 -- 172.21.15.73:0/2515929377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f449c009a20 con 0x7f44a0098d10 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a5761700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 rx=0x7f44a00c2360 tx=0x7f44a00a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c01d630 con 0x7f44a0098d10 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f449c01d7b0 con 0x7f44a0098d10 2023-11-28T04:26:12.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f44a00a9b90 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f449c01db70 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.723+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f44a014c120 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.724+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f44a0155260 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.724+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f449c027460 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.725+0000 7f44957fa700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 0x7f4484061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.725+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f449c07bf90 con 0x7f44a0098d10 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.725+0000 7f44a4f60700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 0x7f4484061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:12.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.725+0000 7f44a4f60700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 0x7f4484061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f44a00cb190 tx=0x7f4490009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:12.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.729+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f449c0aaaf0 con 0x7f44a0098d10 2023-11-28T04:26:12.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.901+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f44a0154d50 con 0x7f44a0098d10 2023-11-28T04:26:12.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.904+0000 7f44957fa700 1 -- 172.21.15.73:0/2515929377 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f44a0154d50 con 0x7f44a0098d10 2023-11-28T04:26:12.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 msgr2=0x7f4484061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 0x7f4484061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f44a00cb190 tx=0x7f4490009040).stop 2023-11-28T04:26:12.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 msgr2=0x7f44a00aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:12.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 rx=0x7f44a00c2360 tx=0x7f44a00a8040).stop 2023-11-28T04:26:12.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 shutdown_connections 2023-11-28T04:26:12.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f448405f150 0x7f4484061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f44a0158320 0x7f44a00abeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f44a0098d10 0x7f44a00aa300 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 --2- 172.21.15.73:0/2515929377 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f44a00c6190 0x7f44a014b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:12.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 >> 172.21.15.73:0/2515929377 conn(0x7f44a00cb310 msgr2=0x7f44a0152380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:12.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 shutdown_connections 2023-11-28T04:26:12.910 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:12.906+0000 7f44a79c5700 1 -- 172.21.15.73:0/2515929377 wait complete. 2023-11-28T04:26:12.919 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-11-28T04:26:13.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.031+0000 7f053ada2700 1 Processor -- start 2023-11-28T04:26:13.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.032+0000 7f053ada2700 1 -- start start 2023-11-28T04:26:13.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.032+0000 7f053ada2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05341548c0 0x7f053406a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.032+0000 7f053ada2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340c69a0 con 0x7f05341544e0 2023-11-28T04:26:13.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.032+0000 7f053ada2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05340c13a0 con 0x7f05340c6e30 2023-11-28T04:26:13.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.032+0000 7f053ada2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05340c1680 con 0x7f05341548c0 2023-11-28T04:26:13.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0533fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f05341548c0 0x7f053406a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0538b3e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f05341544e0 0x7f05340c0bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34796/0 (socket says 172.21.15.73:34796) 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0538b3e700 1 -- 172.21.15.73:0/1758254430 learned_addr learned my addr 172.21.15.73:0/1758254430 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0533fff700 1 -- 172.21.15.73:0/1758254430 >> v1:172.21.15.150:6789/0 conn(0x7f05340c6e30 legacy=0x7f05340a8000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0533fff700 1 -- 172.21.15.73:0/1758254430 >> v1:172.21.15.73:6789/0 conn(0x7f05341544e0 legacy=0x7f05340c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f05337fe700 1 -- 172.21.15.73:0/1758254430 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220025714 0 0) 0x7f05340c69a0 con 0x7f05341544e0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0533fff700 1 -- 172.21.15.73:0/1758254430 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05340c6630 con 0x7f05341548c0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f0533fff700 1 --2- 172.21.15.73:0/1758254430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341548c0 0x7f053406a610 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 rx=0x7f051c009e60 tx=0x7f051c009780).ready entity=mon.0 client_cookie=ca0537a89100afeb server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f05337fe700 1 -- 172.21.15.73:0/1758254430 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c00e070 con 0x7f05341548c0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f05337fe700 1 -- 172.21.15.73:0/1758254430 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c00bbf0 con 0x7f05341548c0 2023-11-28T04:26:13.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.033+0000 7f05337fe700 1 -- 172.21.15.73:0/1758254430 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c012410 con 0x7f05341548c0 2023-11-28T04:26:13.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 -- 172.21.15.73:0/1758254430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341548c0 msgr2=0x7f053406a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 --2- 172.21.15.73:0/1758254430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341548c0 0x7f053406a610 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 rx=0x7f051c009e60 tx=0x7f051c009780).stop 2023-11-28T04:26:13.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 -- 172.21.15.73:0/1758254430 shutdown_connections 2023-11-28T04:26:13.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 --2- 172.21.15.73:0/1758254430 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341548c0 0x7f053406a610 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 -- 172.21.15.73:0/1758254430 >> 172.21.15.73:0/1758254430 conn(0x7f0534085020 msgr2=0x7f0534094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:13.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 -- 172.21.15.73:0/1758254430 shutdown_connections 2023-11-28T04:26:13.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.034+0000 7f053ada2700 1 -- 172.21.15.73:0/1758254430 wait complete. 2023-11-28T04:26:13.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.035+0000 7f053ada2700 1 Processor -- start 2023-11-28T04:26:13.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.035+0000 7f053ada2700 1 -- start start 2023-11-28T04:26:13.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 0x7f0534098be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 0x7f0534152a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0534098750 con 0x7f05341544e0 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05340990d0 con 0x7f05341548c0 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0534168bd0 con 0x7f05340c6e30 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:47080/0 (socket says 172.21.15.73:47080) 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 -- 172.21.15.73:0/3799620037 learned_addr learned my addr 172.21.15.73:0/3799620037 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0538b3e700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 0x7f0534098be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053933f700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 0x7f0534152a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 -- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 msgr2=0x7f0534098be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 0x7f0534098be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 -- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 msgr2=0x7f0534152a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 0x7f0534152a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 -- 172.21.15.73:0/3799620037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f051c009a20 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0538b3e700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 0x7f0534098be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053933f700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 0x7f0534152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0533fff700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 rx=0x7f051c009980 tx=0x7f051c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c01d630 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f051c01d790 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0534098100 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.036+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f051c01db30 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.037+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0534097db0 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.037+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05340988c0 con 0x7f05341544e0 2023-11-28T04:26:13.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.038+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f051c026410 con 0x7f05341544e0 2023-11-28T04:26:13.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.038+0000 7f0530ff9700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 0x7f0514061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.038+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f051c07c0d0 con 0x7f05341544e0 2023-11-28T04:26:13.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.038+0000 7f0538b3e700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 0x7f0514061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.038+0000 7f0538b3e700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 0x7f0514061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0524001360 tx=0x7f0524009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.042+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f051c04d470 con 0x7f05341544e0 2023-11-28T04:26:13.268 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.267+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f05340c6250 con 0x7f05341544e0 2023-11-28T04:26:13.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.300+0000 7f0530ff9700 1 -- 172.21.15.73:0/3799620037 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2995) v1 ==== 163+0+0 (secure 0 0 0) 0x7f051c050a90 con 0x7f05341544e0 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 msgr2=0x7f0514061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 0x7f0514061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f0524001360 tx=0x7f0524009040).stop 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 msgr2=0x7f05340c4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 rx=0x7f051c009980 tx=0x7f051c009780).stop 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 shutdown_connections 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f051405f150 0x7f0514061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f05341548c0 0x7f0534152a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f05340c6e30 0x7f0534098be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 --2- 172.21.15.73:0/3799620037 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f05341544e0 0x7f05340c4f30 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.302+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 >> 172.21.15.73:0/3799620037 conn(0x7f0534085020 msgr2=0x7f05340ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.303+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 shutdown_connections 2023-11-28T04:26:13.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.303+0000 7f053ada2700 1 -- 172.21.15.73:0/3799620037 wait complete. 2023-11-28T04:26:13.315 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-11-28T04:26:13.316 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-11-28T04:26:13.316 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-11-28T04:26:13.316 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-11-28T04:26:13.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.428+0000 7f2a2d876700 1 Processor -- start 2023-11-28T04:26:13.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.428+0000 7f2a2d876700 1 -- start start 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d876700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a28157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d876700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2a280c0180 0x7f2a281589c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d876700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a280c12c0 con 0x7f2a280c1750 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d876700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a280c06b0 con 0x7f2a280c0180 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d876700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c0990 con 0x7f2a28157660 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d075700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2a280c1750 0x7f2a280648d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48212/0 (socket says 172.21.15.73:48212) 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2d075700 1 -- 172.21.15.73:0/2960955325 learned_addr learned my addr 172.21.15.73:0/2960955325 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:13.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a2c874700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a28157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2960955325 >> v2:172.21.15.150:3300/0 conn(0x7f2a280c0180 0x7f2a281589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2959557365 0 0) 0x7f2a280c12c0 con 0x7f2a280c1750 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.429+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a100033f0 con 0x7f2a280c1750 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1339884244 0 0) 0x7f2a100033f0 con 0x7f2a280c1750 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 msgr2=0x7f2a28157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a28157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 >> v2:172.21.15.150:3300/0 conn(0x7f2a280c0180 msgr2=0x7f2a281589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 --2- 172.21.15.73:0/2960955325 >> v2:172.21.15.150:3300/0 conn(0x7f2a280c0180 0x7f2a281589c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280c0c70 con 0x7f2a280c1750 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2a20001470 con 0x7f2a280c1750 2023-11-28T04:26:13.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 >> v1:172.21.15.73:6789/0 conn(0x7f2a280c1750 legacy=0x7f2a280648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a100038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a10004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a10004b80 0x7f2a10005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c06b0 con 0x7f2a10003e10 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280c0990 con 0x7f2a10004b80 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a1000b8e0 con 0x7f2a28157660 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a10004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a2c874700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a100038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a2d075700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a10004b80 0x7f2a10005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 -- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 msgr2=0x7f2a100038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a28157660 0x7f2a100038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 -- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a10004b80 msgr2=0x7f2a10005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a10004b80 0x7f2a10005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 -- 172.21.15.73:0/2960955325 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a100033f0 con 0x7f2a10003e10 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a2d075700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a10004b80 0x7f2a10005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a10004690 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 rx=0x7f2a280c5820 tx=0x7f2a18009060).ready entity=mon.0 client_cookie=4ecc43d5cdc37549 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a18009e90 con 0x7f2a10003e10 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a18009430 con 0x7f2a10003e10 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.430+0000 7f2a1f7fe700 1 -- 172.21.15.73:0/2960955325 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a1800e850 con 0x7f2a10003e10 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 -- 172.21.15.73:0/2960955325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 msgr2=0x7f2a10004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2960955325 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a10004690 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 rx=0x7f2a280c5820 tx=0x7f2a18009060).stop 2023-11-28T04:26:13.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d075700 1 -- 172.21.15.73:0/2960955325 reap_dead start 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 -- 172.21.15.73:0/2960955325 shutdown_connections 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 -- 172.21.15.73:0/2960955325 >> 172.21.15.73:0/2960955325 conn(0x7f2a280cb000 msgr2=0x7f2a280cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 -- 172.21.15.73:0/2960955325 shutdown_connections 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.431+0000 7f2a2d876700 1 -- 172.21.15.73:0/2960955325 wait complete. 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.432+0000 7f2a2d876700 1 Processor -- start 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.432+0000 7f2a2d876700 1 -- start start 2023-11-28T04:26:13.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a281529f0 0x7f2a28152e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a280c1ca0 0x7f2a280c2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280adfb0 con 0x7f2a10003e10 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a280aedd0 con 0x7f2a280c1ca0 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d876700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a28153360 con 0x7f2a281529f0 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38688/0 (socket says 172.21.15.73:38688) 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 -- 172.21.15.73:0/2281673609 learned_addr learned my addr 172.21.15.73:0/2281673609 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a281529f0 0x7f2a28152e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 -- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a281529f0 msgr2=0x7f2a28152e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2d075700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a280c1ca0 0x7f2a280c2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a281529f0 0x7f2a28152e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 -- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a280c1ca0 msgr2=0x7f2a280c2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a280c1ca0 0x7f2a280c2120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 -- 172.21.15.73:0/2281673609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a280ae1c0 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a2c874700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 rx=0x7f2a2400f1b0 tx=0x7f2a2400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.433+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a2400e040 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.434+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a2400f040 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.434+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a280ae4a0 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.434+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a24015450 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.434+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a280aea60 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.435+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a280aebf0 con 0x7f2a10003e10 2023-11-28T04:26:13.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.435+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2a24015800 con 0x7f2a10003e10 2023-11-28T04:26:13.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.435+0000 7f2a1cff9700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 0x7f2a14061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:13.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.435+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f2a2407b0a0 con 0x7f2a10003e10 2023-11-28T04:26:13.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.435+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 0x7f2a14061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:13.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.436+0000 7f2a1ffff700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 0x7f2a14061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2a2815ae10 tx=0x7f2a1801a000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:13.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.439+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2a2404c4a0 con 0x7f2a10003e10 2023-11-28T04:26:13.667 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:13.666+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f2a2806c7c0 con 0x7f2a10003e10 2023-11-28T04:26:14.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.302+0000 7f2a1cff9700 1 -- 172.21.15.73:0/2281673609 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v2996) v1 ==== 158+0+0 (secure 0 0 0) 0x7f2a2404fac0 con 0x7f2a10003e10 2023-11-28T04:26:14.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 msgr2=0x7f2a14061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 0x7f2a14061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f2a2815ae10 tx=0x7f2a1801a000).stop 2023-11-28T04:26:14.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 msgr2=0x7f2a280c4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 rx=0x7f2a2400f1b0 tx=0x7f2a2400b040).stop 2023-11-28T04:26:14.305 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 shutdown_connections 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2a1405f150 0x7f2a14061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2a280c1ca0 0x7f2a280c2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2a10003e10 0x7f2a280c4610 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 --2- 172.21.15.73:0/2281673609 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2a281529f0 0x7f2a28152e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 >> 172.21.15.73:0/2281673609 conn(0x7f2a280cb000 msgr2=0x7f2a280ad2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 shutdown_connections 2023-11-28T04:26:14.306 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.304+0000 7f2a2d876700 1 -- 172.21.15.73:0/2281673609 wait complete. 2023-11-28T04:26:14.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-11-28T04:26:14.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.432+0000 7f1ca2b5d700 1 Processor -- start 2023-11-28T04:26:14.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.432+0000 7f1ca2b5d700 1 -- start start 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca2b5d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 0x7f1c9c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca2b5d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c9c158320 0x7f1c9c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca2b5d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c9c158cc0 con 0x7f1c9c098d10 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca2b5d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c0c72d0 con 0x7f1c9c158320 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca2b5d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c9c064570 con 0x7f1c9c0c6190 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c9c158320 0x7f1c9c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1c9c158320 0x7f1c9c1587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38704/0 (socket says 172.21.15.73:38704) 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 -- 172.21.15.73:0/2531526962 learned_addr learned my addr 172.21.15.73:0/2531526962 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1c9bfff700 1 --2- 172.21.15.73:0/2531526962 >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 0x7f1c9c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 -- 172.21.15.73:0/2531526962 >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 msgr2=0x7f1c9c0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 --2- 172.21.15.73:0/2531526962 >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 0x7f1c9c0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 -- 172.21.15.73:0/2531526962 >> v1:172.21.15.150:6789/0 conn(0x7f1c9c098d10 legacy=0x7f1c9c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1ca10fa700 1 -- 172.21.15.73:0/2531526962 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c9c154bc0 con 0x7f1c9c158320 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.433+0000 7f1c9bfff700 1 --2- 172.21.15.73:0/2531526962 >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 0x7f1c9c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca10fa700 1 --2- 172.21.15.73:0/2531526962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c158320 0x7f1c9c1587d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f1c9c0c5b40 tx=0x7f1c940090f0).ready entity=mon.0 client_cookie=fc2b9b63b522ebb5 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1c9b7fe700 1 -- 172.21.15.73:0/2531526962 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400b5a0 con 0x7f1c9c158320 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1c9b7fe700 1 -- 172.21.15.73:0/2531526962 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c9400b700 con 0x7f1c9c158320 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1c9b7fe700 1 -- 172.21.15.73:0/2531526962 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c9400ba80 con 0x7f1c9c158320 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/2531526962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c158320 msgr2=0x7f1c9c1587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/2531526962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c158320 0x7f1c9c1587d0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 rx=0x7f1c9c0c5b40 tx=0x7f1c940090f0).stop 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/2531526962 shutdown_connections 2023-11-28T04:26:14.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/2531526962 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c158320 0x7f1c9c1587d0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/2531526962 >> v2:172.21.15.150:3300/0 conn(0x7f1c9c0c6190 0x7f1c9c0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/2531526962 >> 172.21.15.73:0/2531526962 conn(0x7f1c9c0cb310 msgr2=0x7f1c9c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:14.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/2531526962 shutdown_connections 2023-11-28T04:26:14.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.434+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/2531526962 wait complete. 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.435+0000 7f1ca2b5d700 1 Processor -- start 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 -- start start 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c9c098d10 0x7f1c9c0ae3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c9c158320 0x7f1c9c0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c155d20 con 0x7f1c9c0c6190 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0b0fa0 con 0x7f1c9c098d10 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1ca2b5d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c9c0b1280 con 0x7f1c9c158320 2023-11-28T04:26:14.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1c9bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38720/0 (socket says 172.21.15.73:38720) 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.436+0000 7f1c9bfff700 1 -- 172.21.15.73:0/502063866 learned_addr learned my addr 172.21.15.73:0/502063866 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1ca08f9700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c9c098d10 0x7f1c9c0ae3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1ca10fa700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c9c158320 0x7f1c9c0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 -- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c9c158320 msgr2=0x7f1c9c0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c9c158320 0x7f1c9c0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 -- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c9c098d10 msgr2=0x7f1c9c0ae3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c9c098d10 0x7f1c9c0ae3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 -- 172.21.15.73:0/502063866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c94009370 con 0x7f1c9c0c6190 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c9bfff700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7f1c8c00a480 tx=0x7f1c8c007930).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c016470 con 0x7f1c9c0c6190 2023-11-28T04:26:14.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c8c0165d0 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c9c155f70 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c8c016950 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.437+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c9c0b0c70 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.438+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c9c156410 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.438+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1c8c013070 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.439+0000 7f1c98ff9700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 0x7f1c7c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.439+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f1c8c07acc0 con 0x7f1c9c0c6190 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.439+0000 7f1ca08f9700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 0x7f1c7c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.439+0000 7f1ca08f9700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 0x7f1c7c061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1c840038b0 tx=0x7f1c84007d20).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.443+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1c8c04c0c0 con 0x7f1c9c0c6190 2023-11-28T04:26:14.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.608+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f1c9c0b1440 con 0x7f1c9c0c6190 2023-11-28T04:26:14.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.614+0000 7f1c98ff9700 1 -- 172.21.15.73:0/502063866 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) v1 ==== 122+0+0 (secure 0 0 0) 0x7f1c8c04f6e0 con 0x7f1c9c0c6190 2023-11-28T04:26:14.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 msgr2=0x7f1c7c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 0x7f1c7c061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f1c840038b0 tx=0x7f1c84007d20).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 msgr2=0x7f1c9c0b0780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 rx=0x7f1c8c00a480 tx=0x7f1c8c007930).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 shutdown_connections 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1c7c05f150 0x7f1c7c061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1c9c158320 0x7f1c9c0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1c9c098d10 0x7f1c9c0ae3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 --2- 172.21.15.73:0/502063866 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1c9c0c6190 0x7f1c9c0b0780 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 >> 172.21.15.73:0/502063866 conn(0x7f1c9c0cb310 msgr2=0x7f1c9c0c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 shutdown_connections 2023-11-28T04:26:14.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.616+0000 7f1ca2b5d700 1 -- 172.21.15.73:0/502063866 wait complete. 2023-11-28T04:26:14.630 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:26:14.630 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:14.665 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:14.710 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:26:14.711 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-11-28T04:26:14.711 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:14.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.823+0000 7f8dcc146700 1 Processor -- start 2023-11-28T04:26:14.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.823+0000 7f8dcc146700 1 -- start start 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dcc146700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8dc415f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dcc146700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc40a9a90 con 0x7f8dc40cae90 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dcc146700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dc4154ce0 con 0x7f8dc40caab0 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dcc146700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40aab80 con 0x7f8dc40ca180 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dc96e1700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8dc40cae90 0x7f8dc40c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48228/0 (socket says 172.21.15.73:48228) 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.824+0000 7f8dc96e1700 1 -- 172.21.15.73:0/749332130 learned_addr learned my addr 172.21.15.73:0/749332130 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:14.829 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dca6e3700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8dc415f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3767594107 0 0) 0x7f8dc40a9a90 con 0x7f8dc40cae90 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db00033f0 con 0x7f8dc40cae90 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2192106193 0 0) 0x7f8db00033f0 con 0x7f8dc40cae90 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 msgr2=0x7f8dc415f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8dc415f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 >> v1:172.21.15.150:6789/0 conn(0x7f8dc40caab0 legacy=0x7f8dc40aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc40a8e80 con 0x7f8dc40cae90 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f8db4001470 con 0x7f8dc40cae90 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 >> v1:172.21.15.73:6789/0 conn(0x7f8dc40cae90 legacy=0x7f8dc40c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8db0003910 0x7f8db0003cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.830 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8db0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0004b80 0x7f8db0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc40aab80 con 0x7f8db0003910 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db000b8e0 con 0x7f8db0004b80 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db000ba70 con 0x7f8dc40ca180 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8db0003910 0x7f8db0003cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc96e1700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8db0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dca6e3700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0004b80 0x7f8db0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 -- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 msgr2=0x7f8db0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:14.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc40ca180 0x7f8db0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 -- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0004b80 msgr2=0x7f8db0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0004b80 0x7f8db0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.825+0000 7f8dc9ee2700 1 -- 172.21.15.73:0/749332130 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db00033f0 con 0x7f8db0003910 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8db0003910 0x7f8db0003cf0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 rx=0x7f8dc0001fb0 tx=0x7f8dc000c3c0).ready entity=mon.0 client_cookie=ba70d6b7a281059f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc00019a0 con 0x7f8db0003910 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc0003170 con 0x7f8db0003910 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dc8ee0700 1 -- 172.21.15.73:0/749332130 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000cb10 con 0x7f8db0003910 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 -- 172.21.15.73:0/749332130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8db0003910 msgr2=0x7f8db0003cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 --2- 172.21.15.73:0/749332130 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8db0003910 0x7f8db0003cf0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 rx=0x7f8dc0001fb0 tx=0x7f8dc000c3c0).stop 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dca6e3700 1 -- 172.21.15.73:0/749332130 reap_dead start 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 -- 172.21.15.73:0/749332130 shutdown_connections 2023-11-28T04:26:14.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 -- 172.21.15.73:0/749332130 >> 172.21.15.73:0/749332130 conn(0x7f8dc4094920 msgr2=0x7f8dc406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:14.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 -- 172.21.15.73:0/749332130 shutdown_connections 2023-11-28T04:26:14.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.826+0000 7f8dcc146700 1 -- 172.21.15.73:0/749332130 wait complete. 2023-11-28T04:26:14.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.827+0000 7f8dcc146700 1 Processor -- start 2023-11-28T04:26:14.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.827+0000 7f8dcc146700 1 -- start start 2023-11-28T04:26:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0003910 0x7f8dc40ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc4155840 0x7f8dc4157040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4157950 con 0x7f8dc40c8880 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4157c30 con 0x7f8db0003910 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dcc146700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dc4063840 con 0x7f8dc4155840 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dca6e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dca6e3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38728/0 (socket says 172.21.15.73:38728) 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dca6e3700 1 -- 172.21.15.73:0/2052012223 learned_addr learned my addr 172.21.15.73:0/2052012223 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0003910 0x7f8dc40ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.828+0000 7f8dc96e1700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc4155840 0x7f8dc4157040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 -- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc4155840 msgr2=0x7f8dc4157040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc4155840 0x7f8dc4157040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 -- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0003910 msgr2=0x7f8dc40ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0003910 0x7f8dc40ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 -- 172.21.15.73:0/2052012223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc4063b20 con 0x7f8dc40c8880 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dca6e3700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 rx=0x7f8dbc006d80 tx=0x7f8dbc009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc013680 con 0x7f8dc40c8880 2023-11-28T04:26:14.837 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0137e0 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc4063e00 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc013b80 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.829+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dc4150230 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.830+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dc4063ff0 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.830+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8dbc014440 con 0x7f8dc40c8880 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.831+0000 7f8db9ffb700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 0x7f8da8061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:14.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.831+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(203..203 src has 1..203) v4 ==== 5813+0+0 (secure 0 0 0) 0x7f8dbc07a830 con 0x7f8dc40c8880 2023-11-28T04:26:14.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.831+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 0x7f8da8061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:14.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.831+0000 7f8dc9ee2700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 0x7f8da8061640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f8dc000c400 tx=0x7f8dc0016000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:14.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:14.835+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8dbc04bb50 con 0x7f8dc40c8880 2023-11-28T04:26:15.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.037+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8dc406c860 con 0x7f8dc40c8880 2023-11-28T04:26:15.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.038+0000 7f8db9ffb700 1 -- 172.21.15.73:0/2052012223 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 72 v72) v1 ==== 94+0+3532 (secure 0 0 0) 0x7f8dbc018070 con 0x7f8dc40c8880 2023-11-28T04:26:15.040 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:15.040 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":72,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8650,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6833/773637045","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":773637045},{"type":"v1","addr":"172.21.15.150:6833","nonce":773637045}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":8653,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/1163812558","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6834","nonce":1163812558},{"type":"v1","addr":"172.21.15.150:6835","nonce":1163812558}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9519,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6836/2138310173","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":2138310173},{"type":"v1","addr":"172.21.15.73:6836","nonce":2138310173}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":67},{"gid":9891,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/3914839730","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":3914839730},{"type":"v1","addr":"172.21.15.73:6837","nonce":3914839730}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":71}],"filesystems":[]} 2023-11-28T04:26:15.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 msgr2=0x7f8da8061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 0x7f8da8061640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7f8dc000c400 tx=0x7f8dc0016000).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 msgr2=0x7f8dc40c8ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 rx=0x7f8dbc006d80 tx=0x7f8dbc009060).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 shutdown_connections 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8da805f180 0x7f8da8061640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8dc40c8880 0x7f8dc40c8ce0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8db0003910 0x7f8dc40ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 --2- 172.21.15.73:0/2052012223 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8dc4155840 0x7f8dc4157040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 >> 172.21.15.73:0/2052012223 conn(0x7f8dc4094920 msgr2=0x7f8dc40a9fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 shutdown_connections 2023-11-28T04:26:15.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.040+0000 7f8dcc146700 1 -- 172.21.15.73:0/2052012223 wait complete. 2023-11-28T04:26:15.043 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 72 2023-11-28T04:26:15.053 INFO:tasks.ceph.mds.a:Restarting daemon 2023-11-28T04:26:15.054 INFO:tasks.ceph.mds.a:Stopping old one... 2023-11-28T04:26:15.054 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T04:26:15.054 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:26:15.054 INFO:tasks.ceph.mds.c:Restarting daemon 2023-11-28T04:26:15.054 INFO:tasks.ceph.mds.c:Stopping old one... 2023-11-28T04:26:15.054 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T04:26:15.055 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:26:15.055 INFO:tasks.ceph.mds.b:Restarting daemon 2023-11-28T04:26:15.055 INFO:tasks.ceph.mds.b:Stopping old one... 2023-11-28T04:26:15.055 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T04:26:15.055 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:26:15.055 INFO:tasks.ceph.mds.d:Restarting daemon 2023-11-28T04:26:15.055 INFO:tasks.ceph.mds.d:Stopping old one... 2023-11-28T04:26:15.056 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T04:26:15.056 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:26:15.063 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T04:26:15.063 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-11-28T04:26:15.064 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T04:26:15.064 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-11-28T04:26:15.066 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T04:26:15.066 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-11-28T04:26:15.067 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T04:26:15.067 DEBUG:teuthology.orchestra.run.smithi150:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-11-28T04:26:15.106 INFO:tasks.ceph.mds.a:Started 2023-11-28T04:26:15.106 DEBUG:teuthology.parallel:result is None 2023-11-28T04:26:15.108 INFO:tasks.ceph.mds.c:Started 2023-11-28T04:26:15.108 DEBUG:teuthology.parallel:result is None 2023-11-28T04:26:15.109 INFO:tasks.ceph.mds.d:Started 2023-11-28T04:26:15.109 DEBUG:teuthology.parallel:result is None 2023-11-28T04:26:15.110 INFO:tasks.ceph.mds.b:Started 2023-11-28T04:26:15.110 DEBUG:teuthology.parallel:result is None 2023-11-28T04:26:15.111 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-11-28T04:26:15.178 INFO:tasks.ceph.mds.c.smithi073.stdout:starting mds.c at 2023-11-28T04:26:15.178 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:26:15.179 INFO:tasks.ceph.mds.c.smithi073.stderr:2023-11-28T04:26:15.177+0000 7f59cd65c700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6834/0: (98) Address already in use 2023-11-28T04:26:15.189 INFO:tasks.ceph.mds.d.smithi150.stdout:starting mds.d at 2023-11-28T04:26:15.189 INFO:tasks.ceph.mds.b.smithi150.stdout:starting mds.b at 2023-11-28T04:26:15.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.233+0000 7f6bf1d7c700 1 Processor -- start 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.233+0000 7f6bf1d7c700 1 -- start start 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6bf1d7c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6bec158a30 0x7f6bec0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6bf1d7c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec0c72d0 con 0x7f6bec098d10 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6bf1d7c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bec064570 con 0x7f6bec0c6190 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6bf1d7c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bec154bc0 con 0x7f6bec158a30 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6beb7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6bec098d10 0x7f6bec063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48238/0 (socket says 172.21.15.73:48238) 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.234+0000 7f6beb7fe700 1 -- 172.21.15.73:0/422049976 learned_addr learned my addr 172.21.15.73:0/422049976 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bebfff700 1 --2- 172.21.15.73:0/422049976 >> v2:172.21.15.150:3300/0 conn(0x7f6bec158a30 0x7f6bec0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2702597214 0 0) 0x7f6bec0c72d0 con 0x7f6bec098d10 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bd40033f0 con 0x7f6bec098d10 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3502833190 0 0) 0x7f6bd40033f0 con 0x7f6bec098d10 2023-11-28T04:26:15.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 >> v2:172.21.15.150:3300/0 conn(0x7f6bec158a30 msgr2=0x7f6bec0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 --2- 172.21.15.73:0/422049976 >> v2:172.21.15.150:3300/0 conn(0x7f6bec158a30 0x7f6bec0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 >> v1:172.21.15.150:6789/0 conn(0x7f6bec0c6190 legacy=0x7f6bec158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec0c66c0 con 0x7f6bec098d10 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f6bdc001470 con 0x7f6bec098d10 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 >> v1:172.21.15.73:6789/0 conn(0x7f6bec098d10 legacy=0x7f6bec063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bd4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd4005000 0x7f6bd400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec154bc0 con 0x7f6bd4004690 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd400be50 con 0x7f6bd40038e0 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd400c010 con 0x7f6bd4005000 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bd4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.242 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beb7fe700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bebfff700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd4005000 0x7f6bd400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 -- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd4005000 msgr2=0x7f6bd400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bd4005000 0x7f6bd400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 -- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd40038e0 msgr2=0x7f6bd4003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bd40038e0 0x7f6bd4003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 -- 172.21.15.73:0/422049976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd40033f0 con 0x7f6bd4004690 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6bebfff700 1 -- 172.21.15.73:0/422049976 reap_dead start 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.235+0000 7f6beaffd700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bd4004b10 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7f6be00017f0 tx=0x7f6be0007b90).ready entity=mon.0 client_cookie=cb211493a51acdfc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be0007ea0 con 0x7f6bd4004690 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6be0012400 con 0x7f6bd4004690 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bea7fc700 1 -- 172.21.15.73:0/422049976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6be00127a0 con 0x7f6bd4004690 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 msgr2=0x7f6bd4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.243 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bd4004b10 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 rx=0x7f6be00017f0 tx=0x7f6be0007b90).stop 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/422049976 shutdown_connections 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/422049976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bd4004b10 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/422049976 >> 172.21.15.73:0/422049976 conn(0x7f6bec0cb310 msgr2=0x7f6bec0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/422049976 shutdown_connections 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.236+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/422049976 wait complete. 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.237+0000 7f6bf1d7c700 1 Processor -- start 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.237+0000 7f6bf1d7c700 1 -- start start 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 0x7f6bec0ab660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bec0ad870 0x7f6bec0abb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec151d10 con 0x7f6bd4004690 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec1572b0 con 0x7f6bec0ad3f0 2023-11-28T04:26:15.244 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bf1d7c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bec0aaea0 con 0x7f6bec0ad870 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6beb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6beb7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38792/0 (socket says 172.21.15.73:38792) 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6beb7fe700 1 -- 172.21.15.73:0/638743550 learned_addr learned my addr 172.21.15.73:0/638743550 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6beaffd700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 0x7f6bec0ab660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.238+0000 7f6bebfff700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bec0ad870 0x7f6bec0abb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 -- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bec0ad870 msgr2=0x7f6bec0abb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bec0ad870 0x7f6bec0abb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 -- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 msgr2=0x7f6bec0ab660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 0x7f6bec0ab660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 -- 172.21.15.73:0/638743550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bec156cf0 con 0x7f6bd4004690 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beaffd700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 0x7f6bec0ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6beb7fe700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7f6bdc007690 tx=0x7f6bec0a8040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:15.245 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc004e80 con 0x7f6bd4004690 2023-11-28T04:26:15.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bdc0114c0 con 0x7f6bd4004690 2023-11-28T04:26:15.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bec156fd0 con 0x7f6bd4004690 2023-11-28T04:26:15.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bdc011840 con 0x7f6bd4004690 2023-11-28T04:26:15.246 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.239+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bec155c50 con 0x7f6bd4004690 2023-11-28T04:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.240+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bec06c860 con 0x7f6bd4004690 2023-11-28T04:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.240+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f6bdc011c30 con 0x7f6bd4004690 2023-11-28T04:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.241+0000 7f6bd37fe700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 0x7f6bd8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.241+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5785+0+0 (secure 0 0 0) 0x7f6bdc079630 con 0x7f6bd4004690 2023-11-28T04:26:15.247 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.241+0000 7f6beaffd700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 0x7f6bd8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:15.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.241+0000 7f6beaffd700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 0x7f6bd8061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6be0001050 tx=0x7f6be0015000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:15.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.245+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f6bdc04aaa0 con 0x7f6bd4004690 2023-11-28T04:26:15.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:15.424+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6bec0c69a0 con 0x7f6bd4004690 2023-11-28T04:26:16.308 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.307+0000 7f6bd37fe700 1 -- 172.21.15.73:0/638743550 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v208) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6bdc019030 con 0x7f6bd4004690 2023-11-28T04:26:16.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 msgr2=0x7f6bd8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 0x7f6bd8061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f6be0001050 tx=0x7f6be0015000).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 msgr2=0x7f6bec1482a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 rx=0x7f6bdc007690 tx=0x7f6bec0a8040).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 shutdown_connections 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6bd805f150 0x7f6bd8061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f6bec0ad870 0x7f6bec0abb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6bd4004690 0x7f6bec1482a0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 --2- 172.21.15.73:0/638743550 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6bec0ad3f0 0x7f6bec0ab660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 >> 172.21.15.73:0/638743550 conn(0x7f6bec0cb310 msgr2=0x7f6bec152d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 shutdown_connections 2023-11-28T04:26:16.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.310+0000 7f6bf1d7c700 1 -- 172.21.15.73:0/638743550 wait complete. 2023-11-28T04:26:16.313 INFO:teuthology.orchestra.run.smithi073.stderr: removed all blocklist entries 2023-11-28T04:26:16.325 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-11-28T04:26:16.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.439+0000 7f9b003de700 1 Processor -- start 2023-11-28T04:26:16.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.439+0000 7f9b003de700 1 -- start start 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9b003de700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9af80c45d0 0x7f9af80c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9b003de700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9af8068090 0x7f9af80c2b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9b003de700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9af8154d20 con 0x7f9af8067cb0 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9b003de700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af8151670 con 0x7f9af80c45d0 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9b003de700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9af80a7490 con 0x7f9af8068090 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.440+0000 7f9afd979700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9af80c45d0 0x7f9af80c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9af80c45d0 0x7f9af80c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38796/0 (socket says 172.21.15.73:38796) 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 -- 172.21.15.73:0/1839689485 learned_addr learned my addr 172.21.15.73:0/1839689485 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:16.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afe97b700 1 --2- 172.21.15.73:0/1839689485 >> v2:172.21.15.150:3300/0 conn(0x7f9af8068090 0x7f9af80c2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 -- 172.21.15.73:0/1839689485 >> v2:172.21.15.150:3300/0 conn(0x7f9af8068090 msgr2=0x7f9af80c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 --2- 172.21.15.73:0/1839689485 >> v2:172.21.15.150:3300/0 conn(0x7f9af8068090 0x7f9af80c2b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 -- 172.21.15.73:0/1839689485 >> v1:172.21.15.150:6789/0 conn(0x7f9af8067cb0 legacy=0x7f9af80c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 -- 172.21.15.73:0/1839689485 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9af81555c0 con 0x7f9af80c45d0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd979700 1 --2- 172.21.15.73:0/1839689485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af80c2640 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7f9af8154c50 tx=0x7f9aec0090f0).ready entity=mon.0 client_cookie=aac61fb4bf814409 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd178700 1 -- 172.21.15.73:0/1839689485 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00b5a0 con 0x7f9af80c45d0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd178700 1 -- 172.21.15.73:0/1839689485 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec00b700 con 0x7f9af80c45d0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9afd178700 1 -- 172.21.15.73:0/1839689485 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00ba80 con 0x7f9af80c45d0 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9b003de700 1 -- 172.21.15.73:0/1839689485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 msgr2=0x7f9af80c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.441+0000 7f9b003de700 1 --2- 172.21.15.73:0/1839689485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af80c2640 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 rx=0x7f9af8154c50 tx=0x7f9aec0090f0).stop 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 -- 172.21.15.73:0/1839689485 shutdown_connections 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 --2- 172.21.15.73:0/1839689485 >> v2:172.21.15.150:3300/0 conn(0x7f9af8068090 0x7f9af80c2b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.444 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 --2- 172.21.15.73:0/1839689485 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af80c2640 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 -- 172.21.15.73:0/1839689485 >> 172.21.15.73:0/1839689485 conn(0x7f9af80c7fd0 msgr2=0x7f9af80c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 -- 172.21.15.73:0/1839689485 shutdown_connections 2023-11-28T04:26:16.445 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.442+0000 7f9b003de700 1 -- 172.21.15.73:0/1839689485 wait complete. 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 Processor -- start 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 -- start start 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af8067cb0 0x7f9af81540f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9af8068090 0x7f9af81517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8152b50 con 0x7f9af80c45d0 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9af81545e0 con 0x7f9af8067cb0 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9b003de700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9af8151cb0 con 0x7f9af8068090 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.443+0000 7f9afe97b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.448 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38800/0 (socket says 172.21.15.73:38800) 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 -- 172.21.15.73:0/1065552463 learned_addr learned my addr 172.21.15.73:0/1065552463 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afd979700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9af8068090 0x7f9af81517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe17a700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af8067cb0 0x7f9af81540f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 -- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9af8068090 msgr2=0x7f9af81517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9af8068090 0x7f9af81517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 -- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af8067cb0 msgr2=0x7f9af81540f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af8067cb0 0x7f9af81540f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 -- 172.21.15.73:0/1065552463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aec009370 con 0x7f9af80c45d0 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9afe97b700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7f9af000d890 tx=0x7f9af0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011590 con 0x7f9af80c45d0 2023-11-28T04:26:16.449 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00116f0 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af8151f40 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0011a70 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.444+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9af8152580 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.445+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9af806c860 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.445+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f9af0012420 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.446+0000 7f9af67fc700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 0x7f9ae4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.446+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f9af0079840 con 0x7f9af80c45d0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.446+0000 7f9afe17a700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 0x7f9ae4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.446+0000 7f9afe17a700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 0x7f9ae4061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9ae8003fb0 tx=0x7f9ae8002530).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.451 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.450+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9af004ae00 con 0x7f9af80c45d0 2023-11-28T04:26:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.674+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9af80c3390 con 0x7f9af80c45d0 2023-11-28T04:26:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.675+0000 7f9af67fc700 1 -- 172.21.15.73:0/1065552463 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v48) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9af004e420 con 0x7f9af80c45d0 2023-11-28T04:26:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:16.676 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:16.676 INFO:teuthology.orchestra.run.smithi073.stdout: "auth_dump": [ 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.a", 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVl9p6tOBAAcliR1S0g/QmSuuMVAvB7GA==", 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.677 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.b", 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlgvMqCBAAXWIZj5UTFWa52okENmEoOQ==", 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.678 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.c", 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlFnXRAxAA+/XfubZVOqjEYeoEc6PSgw==", 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.679 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mds.d", 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVl2Db2DhAAIvREFVRVL+rI6gAQvmsH9A==", 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.680 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.0", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDMVWVlzk6FKhAAjCaZ7TBStIMR9F2V36A/gA==", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.1", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDPVWVlVHmvJxAAvyC+1NDdm1g9s2SNUjlYzA==", 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.681 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.2", 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDSVWVldOcFLBAAW+tCrJl/sINLslxP7xB60w==", 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.682 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.3", 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDVVWVlxgP8KBAAwqPeTDHmiVvQobGKFfbXpA==", 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.683 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.4", 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDbVWVl9tL0KhAAH3iTy1auChjuejYgan0lFQ==", 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.684 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.5", 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDeVWVlsZLZKBAAX6peooecoqopNT8nq5t7fw==", 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.685 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.6", 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDhVWVlwNdxJRAAP7kJaYPQaESBmjpF9GeY5A==", 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.686 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "osd.7", 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDkVWVlY+PkIhAAYBMz32m5VKhXzvlAOijKsw==", 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow profile osd", 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile osd", 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.687 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.0", 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlksi5EhAAlfqDIpt14aYyE/MRKhGwhQ==", 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow r", 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.688 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.1", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlXpnzGhAAxhz53Sei6/xoVbCovTACsQ==", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.2", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlAhYBGBAAuCZHDnHTi9Y5bLj/sCAILw==", 2023-11-28T04:26:16.689 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.3", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDJVWVlVKBPIBAAK7ZxFt1+LGc3Pd2btgALBA==", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow", 2023-11-28T04:26:16.690 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow r", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow rw", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow rwx" 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "client.admin", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlCr4THBAAFdfaYaIlzj2xmIVf7HECsg==", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mgr": "allow *", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow *", 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.691 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.x", 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlh7O/NRAAlP382kN/wjwxssJ+rZeoaA==", 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "entity": "mgr.y", 2023-11-28T04:26:16.692 INFO:teuthology.orchestra.run.smithi073.stdout: "key": "AQDIVWVlHHkRMBAAEfAP+wFZ8M0uUPizY4HEdg==", 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: "caps": { 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: "mds": "allow *", 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: "mon": "allow profile mgr", 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: "osd": "allow *" 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout: ] 2023-11-28T04:26:16.693 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.676+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 msgr2=0x7f9ae4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.676+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 0x7f9ae4061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7f9ae8003fb0 tx=0x7f9ae8002530).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.676+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 msgr2=0x7f9af814ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.676+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 rx=0x7f9af000d890 tx=0x7f9af0008040).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 shutdown_connections 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9ae405f150 0x7f9ae4061610 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9af80c45d0 0x7f9af814ef80 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9af8067cb0 0x7f9af81540f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 --2- 172.21.15.73:0/1065552463 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9af8068090 0x7f9af81517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 >> 172.21.15.73:0/1065552463 conn(0x7f9af80c7fd0 msgr2=0x7f9af81508f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 shutdown_connections 2023-11-28T04:26:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.677+0000 7f9b003de700 1 -- 172.21.15.73:0/1065552463 wait complete. 2023-11-28T04:26:16.696 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:16.810 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.809+0000 7fc07eaa8700 1 Processor -- start 2023-11-28T04:26:16.811 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.809+0000 7fc07eaa8700 1 -- start start 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07eaa8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc078157a40 0x7fc0780648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07eaa8700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0780c12c0 con 0x7fc078157660 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07eaa8700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0780c06b0 con 0x7fc0780c0180 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07eaa8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0780c0990 con 0x7fc078157a40 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07daa6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc078157660 0x7fc0781589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48242/0 (socket says 172.21.15.73:48242) 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07daa6700 1 -- 172.21.15.73:0/1464369641 learned_addr learned my addr 172.21.15.73:0/1464369641 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07e2a7700 1 --2- 172.21.15.73:0/1464369641 >> v2:172.21.15.150:3300/0 conn(0x7fc078157a40 0x7fc0780648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181851101 0 0) 0x7fc0780c12c0 con 0x7fc078157660 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc0640033f0 con 0x7fc078157660 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1421211259 0 0) 0x7fc0640033f0 con 0x7fc078157660 2023-11-28T04:26:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 >> v2:172.21.15.150:3300/0 conn(0x7fc078157a40 msgr2=0x7fc0780648d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 --2- 172.21.15.73:0/1464369641 >> v2:172.21.15.150:3300/0 conn(0x7fc078157a40 0x7fc0780648d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 >> v1:172.21.15.150:6789/0 conn(0x7fc0780c0180 legacy=0x7fc0780c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780c0c70 con 0x7fc078157660 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fc074001470 con 0x7fc078157660 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.810+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 >> v1:172.21.15.73:6789/0 conn(0x7fc078157660 legacy=0x7fc0781589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0640038e0 0x7fc064003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc064004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc064005000 0x7fc06400b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c0990 con 0x7fc064004690 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc06400beb0 con 0x7fc064005000 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc06400c070 con 0x7fc0640038e0 2023-11-28T04:26:16.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc064004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07daa6700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0640038e0 0x7fc064003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07e2a7700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc064005000 0x7fc06400b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 -- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0640038e0 msgr2=0x7fc064003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0640038e0 0x7fc064003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 -- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc064005000 msgr2=0x7fc06400b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc064005000 0x7fc06400b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07e2a7700 1 -- 172.21.15.73:0/1464369641 reap_dead start 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 -- 172.21.15.73:0/1464369641 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0640033f0 con 0x7fc064004690 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07daa6700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc0640038e0 0x7fc064003cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc064004b10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 rx=0x7fc068001820 tx=0x7fc068007b90).ready entity=mon.0 client_cookie=148bff5814222e2a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc068007ea0 con 0x7fc064004690 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc068012400 con 0x7fc064004690 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.811+0000 7fc07caa4700 1 -- 172.21.15.73:0/1464369641 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0680127c0 con 0x7fc064004690 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 msgr2=0x7fc064004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc064004b10 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 rx=0x7fc068001820 tx=0x7fc068007b90).stop 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1464369641 shutdown_connections 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1464369641 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc064004b10 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1464369641 >> 172.21.15.73:0/1464369641 conn(0x7fc0780cb000 msgr2=0x7fc0780cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1464369641 shutdown_connections 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.812+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1464369641 wait complete. 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.813+0000 7fc07eaa8700 1 Processor -- start 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.813+0000 7fc07eaa8700 1 -- start start 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 0x7fc078150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc078150f30 0x7fc078164050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c2a90 con 0x7fc064004690 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c1e80 con 0x7fc0780c1650 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0780c2160 con 0x7fc078150f30 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38822/0 (socket says 172.21.15.73:38822) 2023-11-28T04:26:16.820 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 -- 172.21.15.73:0/1909584338 learned_addr learned my addr 172.21.15.73:0/1909584338 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07e2a7700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc078150f30 0x7fc078164050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 0x7fc078150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 -- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc078150f30 msgr2=0x7fc078164050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc078150f30 0x7fc078164050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 -- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 msgr2=0x7fc078150a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 0x7fc078150a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 -- 172.21.15.73:0/1909584338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0780c2440 con 0x7fc064004690 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 0x7fc078150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07daa6700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7fc074001c20 tx=0x7fc0740014c0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074010450 con 0x7fc064004690 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0740105b0 con 0x7fc064004690 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0780c2720 con 0x7fc064004690 2023-11-28T04:26:16.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc074010950 con 0x7fc064004690 2023-11-28T04:26:16.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.814+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc078164540 con 0x7fc064004690 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.815+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc07806c7c0 con 0x7fc064004690 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.816+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc074010d40 con 0x7fc064004690 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.816+0000 7fc06dffb700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 0x7fc05c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.816+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc074079d60 con 0x7fc064004690 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.816+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 0x7fc05c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.816+0000 7fc07d2a5700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 0x7fc05c061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc0780c3d90 tx=0x7fc068016000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:16.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.820+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc07404b430 con 0x7fc064004690 2023-11-28T04:26:16.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.984+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0780bff60 con 0x7fc064004690 2023-11-28T04:26:16.985 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.984+0000 7fc06dffb700 1 -- 172.21.15.73:0/1909584338 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+13723 (secure 0 0 0) 0x7fc07404ea50 con 0x7fc064004690 2023-11-28T04:26:16.985 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:16.986 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":208,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:16.302744+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":24,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":195,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:16.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.986+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 msgr2=0x7fc05c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.986+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 0x7fc05c061610 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc0780c3d90 tx=0x7fc068016000).stop 2023-11-28T04:26:16.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.986+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 msgr2=0x7fc0780c38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:16.988 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.986+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 rx=0x7fc074001c20 tx=0x7fc0740014c0).stop 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 shutdown_connections 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc05c05f150 0x7fc05c061610 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc078150f30 0x7fc078164050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc064004690 0x7fc0780c38a0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 --2- 172.21.15.73:0/1909584338 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc0780c1650 0x7fc078150a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 >> 172.21.15.73:0/1909584338 conn(0x7fc0780cb000 msgr2=0x7fc0780abdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 shutdown_connections 2023-11-28T04:26:16.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:16.987+0000 7fc07eaa8700 1 -- 172.21.15.73:0/1909584338 wait complete. 2023-11-28T04:26:17.000 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '203', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-11-28T04:26:17.000 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:26:17.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.115+0000 7f11ba328700 1 Processor -- start 2023-11-28T04:26:17.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.115+0000 7f11ba328700 1 -- start start 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11ba328700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11b40a8280 0x7f11b40a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11ba328700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b4158160 0x7f11b4155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11ba328700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11b4067770 con 0x7f11b414e600 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11ba328700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11b4159180 con 0x7f11b40a8280 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11ba328700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40be8d0 con 0x7f11b4158160 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11b40a8280 0x7f11b40a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f11b40a8280 0x7f11b40a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38832/0 (socket says 172.21.15.73:38832) 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 -- 172.21.15.73:0/119102040 learned_addr learned my addr 172.21.15.73:0/119102040 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b37fe700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b4158160 0x7f11b4155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 -- 172.21.15.73:0/119102040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b4158160 msgr2=0x7f11b4155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b4158160 0x7f11b4155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 -- 172.21.15.73:0/119102040 >> v1:172.21.15.150:6789/0 conn(0x7f11b414e600 legacy=0x7f11b414ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.116+0000 7f11b3fff700 1 -- 172.21.15.73:0/119102040 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11b4158600 con 0x7f11b40a8280 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11b3fff700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b40a8280 0x7f11b40a8660 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7f11a4006fd0 tx=0x7f11a40090f0).ready entity=mon.0 client_cookie=66558c25db73648d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11b2ffd700 1 -- 172.21.15.73:0/119102040 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400b600 con 0x7f11b40a8280 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11b2ffd700 1 -- 172.21.15.73:0/119102040 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a400b760 con 0x7f11b40a8280 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11b2ffd700 1 -- 172.21.15.73:0/119102040 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a400bae0 con 0x7f11b40a8280 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 -- 172.21.15.73:0/119102040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b40a8280 msgr2=0x7f11b40a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b40a8280 0x7f11b40a8660 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 rx=0x7f11a4006fd0 tx=0x7f11a40090f0).stop 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 -- 172.21.15.73:0/119102040 shutdown_connections 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b40a8280 0x7f11b40a8660 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 --2- 172.21.15.73:0/119102040 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b4158160 0x7f11b4155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 -- 172.21.15.73:0/119102040 >> 172.21.15.73:0/119102040 conn(0x7f11b4062700 msgr2=0x7f11b4062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 -- 172.21.15.73:0/119102040 shutdown_connections 2023-11-28T04:26:17.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.117+0000 7f11ba328700 1 -- 172.21.15.73:0/119102040 wait complete. 2023-11-28T04:26:17.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.118+0000 7f11ba328700 1 Processor -- start 2023-11-28T04:26:17.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 -- start start 2023-11-28T04:26:17.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 0x7f11b414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 0x7f11b40c5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b40a92d0 con 0x7f11b414e600 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4064ee0 con 0x7f11b4158160 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11ba328700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11b4064c00 con 0x7f11b40a8280 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11b37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11b37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38846/0 (socket says 172.21.15.73:38846) 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.119+0000 7f11b37fe700 1 -- 172.21.15.73:0/1567257677 learned_addr learned my addr 172.21.15.73:0/1567257677 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b3fff700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 0x7f11b414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b88c5700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 0x7f11b40c5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 -- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 msgr2=0x7f11b414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 0x7f11b414c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 -- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 msgr2=0x7f11b40c5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 0x7f11b40c5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.126 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 -- 172.21.15.73:0/1567257677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11a4009370 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b3fff700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 0x7f11b414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b88c5700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 0x7f11b40c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11b37fe700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 rx=0x7f11a800d860 tx=0x7f11a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8011590 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11a80116f0 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11b40a8c80 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11a8011a70 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.120+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11b414ce90 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.121+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11b406c860 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.121+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f11a8012420 con 0x7f11b414e600 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.122+0000 7f119bfff700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 0x7f11a0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.122+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f11a8079900 con 0x7f11b414e600 2023-11-28T04:26:17.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.122+0000 7f11b3fff700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 0x7f11a0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.122+0000 7f11b3fff700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 0x7f11a0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f11a4009130 tx=0x7f11a4017000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.128 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.126+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f11a804aec0 con 0x7f11b414e600 2023-11-28T04:26:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.290+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f11b41588e0 con 0x7f11b414e600 2023-11-28T04:26:17.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.290+0000 7f119bfff700 1 -- 172.21.15.73:0/1567257677 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v208) v1 ==== 110+0+10 (secure 0 0 0) 0x7f11a8011e60 con 0x7f11b414e600 2023-11-28T04:26:17.292 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:26:17.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 msgr2=0x7f11a0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 0x7f11a0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f11a4009130 tx=0x7f11a4017000).stop 2023-11-28T04:26:17.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 msgr2=0x7f11b40641e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 rx=0x7f11a800d860 tx=0x7f11a8008040).stop 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 shutdown_connections 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f11a005f150 0x7f11a0061610 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f11b4158160 0x7f11b40c5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f11b40a8280 0x7f11b414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 --2- 172.21.15.73:0/1567257677 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f11b414e600 0x7f11b40641e0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 >> 172.21.15.73:0/1567257677 conn(0x7f11b4062700 msgr2=0x7f11b414ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 shutdown_connections 2023-11-28T04:26:17.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.292+0000 7f11ba328700 1 -- 172.21.15.73:0/1567257677 wait complete. 2023-11-28T04:26:17.305 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:26:17.420 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.419+0000 7f55b4a82700 1 Processor -- start 2023-11-28T04:26:17.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.419+0000 7f55b4a82700 1 -- start start 2023-11-28T04:26:17.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.420+0000 7f55b4a82700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 0x7f55b00a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.420+0000 7f55b4a82700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.421 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.420+0000 7f55b4a82700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55b0067770 con 0x7f55b014e600 2023-11-28T04:26:17.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.420+0000 7f55b4a82700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55b0159180 con 0x7f55b00a8280 2023-11-28T04:26:17.422 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.420+0000 7f55b4a82700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00be8d0 con 0x7f55b0158160 2023-11-28T04:26:17.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.423 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55ae59c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 0x7f55b00a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55aed9d700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f55b014e600 0x7f55b014ea90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:47324/0 (socket says 172.21.15.73:47324) 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55aed9d700 1 -- 172.21.15.73:0/3298068475 learned_addr learned my addr 172.21.15.73:0/3298068475 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:47594/0 (socket says 172.21.15.73:47594) 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 -- 172.21.15.73:0/3298068475 >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 msgr2=0x7f55b00a8660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 --2- 172.21.15.73:0/3298068475 >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 0x7f55b00a8660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 -- 172.21.15.73:0/3298068475 >> v1:172.21.15.150:6789/0 conn(0x7f55b014e600 legacy=0x7f55b014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55ad59a700 1 -- 172.21.15.73:0/3298068475 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3760038714 0 0) 0x7f55b0067770 con 0x7f55b014e600 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 -- 172.21.15.73:0/3298068475 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b0158600 con 0x7f55b0158160 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55ae59c700 1 --2- 172.21.15.73:0/3298068475 >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 0x7f55b00a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.421+0000 7f55add9b700 1 --2- 172.21.15.73:0/3298068475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f55a4009760 tx=0x7f55a4009060).ready entity=mon.2 client_cookie=2a1a162b84e3598f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55ad59a700 1 -- 172.21.15.73:0/3298068475 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f510 con 0x7f55b0158160 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55ad59a700 1 -- 172.21.15.73:0/3298068475 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400f670 con 0x7f55b0158160 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55ad59a700 1 -- 172.21.15.73:0/3298068475 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f9f0 con 0x7f55b0158160 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 -- 172.21.15.73:0/3298068475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 msgr2=0x7f55b0155500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 --2- 172.21.15.73:0/3298068475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 rx=0x7f55a4009760 tx=0x7f55a4009060).stop 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 -- 172.21.15.73:0/3298068475 shutdown_connections 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 --2- 172.21.15.73:0/3298068475 >> v2:172.21.15.150:3300/0 conn(0x7f55b00a8280 0x7f55b00a8660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 --2- 172.21.15.73:0/3298068475 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b0155500 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 -- 172.21.15.73:0/3298068475 >> 172.21.15.73:0/3298068475 conn(0x7f55b0062700 msgr2=0x7f55b0062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 -- 172.21.15.73:0/3298068475 shutdown_connections 2023-11-28T04:26:17.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.422+0000 7f55b4a82700 1 -- 172.21.15.73:0/3298068475 wait complete. 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.423+0000 7f55b4a82700 1 Processor -- start 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.423+0000 7f55b4a82700 1 -- start start 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00a8280 0x7f55b014cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b00ac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00c4e30 con 0x7f55b014e600 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00c4220 con 0x7f55b00a8280 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55b4a82700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b00c4500 con 0x7f55b0158160 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38860/0 (socket says 172.21.15.73:38860) 2023-11-28T04:26:17.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 -- 172.21.15.73:0/1638904910 learned_addr learned my addr 172.21.15.73:0/1638904910 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55ae59c700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00a8280 0x7f55b014cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55aed9d700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b00ac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 -- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 msgr2=0x7f55b00ac0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b00ac0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 -- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00a8280 msgr2=0x7f55b014cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.424+0000 7f55add9b700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00a8280 0x7f55b014cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f55add9b700 1 -- 172.21.15.73:0/1638904910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009a20 con 0x7f55b014e600 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f55add9b700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f55a4009e00 tx=0x7f55a4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a400f7e0 con 0x7f55b014e600 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a400f940 con 0x7f55b014e600 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55a40092a0 con 0x7f55b014e600 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a401b660 con 0x7f55b014e600 2023-11-28T04:26:17.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.425+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b00c4ac0 con 0x7f55b014e600 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.426+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55b006c860 con 0x7f55b014e600 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.426+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f55a401ba50 con 0x7f55b014e600 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.426+0000 7f559e7fc700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 0x7f5590061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.426+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f55a407b8e0 con 0x7f55b014e600 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.426+0000 7f55ae59c700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 0x7f5590061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.427+0000 7f55ae59c700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 0x7f5590061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f55a0009fd0 tx=0x7f55a000b040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.430+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55a404cfb0 con 0x7f55b014e600 2023-11-28T04:26:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.592+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f55b01588e0 con 0x7f55b014e600 2023-11-28T04:26:17.594 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.593+0000 7f559e7fc700 1 -- 172.21.15.73:0/1638904910 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v208) v1 ==== 92+0+10 (secure 0 0 0) 0x7f55a401f020 con 0x7f55b014e600 2023-11-28T04:26:17.594 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:26:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 msgr2=0x7f5590061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 0x7f5590061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f55a0009fd0 tx=0x7f55a000b040).stop 2023-11-28T04:26:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 msgr2=0x7f55b00c52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 rx=0x7f55a4009e00 tx=0x7f55a4017000).stop 2023-11-28T04:26:17.596 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 shutdown_connections 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f559005f150 0x7f5590061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55b0158160 0x7f55b00ac0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55b00a8280 0x7f55b014cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 --2- 172.21.15.73:0/1638904910 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55b014e600 0x7f55b00c52c0 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 >> 172.21.15.73:0/1638904910 conn(0x7f55b0062700 msgr2=0x7f55b00bf880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 shutdown_connections 2023-11-28T04:26:17.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.595+0000 7f55b4a82700 1 -- 172.21.15.73:0/1638904910 wait complete. 2023-11-28T04:26:17.608 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-11-28T04:26:17.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.719+0000 7fc2f8248700 1 Processor -- start 2023-11-28T04:26:17.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.719+0000 7fc2f8248700 1 -- start start 2023-11-28T04:26:17.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f8248700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2f00cae90 0x7fc2f00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f8248700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f00a9a90 con 0x7fc2f00caab0 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f8248700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2f0154ce0 con 0x7fc2f00ca180 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f8248700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2f00aab80 con 0x7fc2f00cae90 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f57e3700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc2f00cae90 0x7fc2f00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f5fe4700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc2f00caab0 0x7fc2f00aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48248/0 (socket says 172.21.15.73:48248) 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.720+0000 7fc2f5fe4700 1 -- 172.21.15.73:0/723959144 learned_addr learned my addr 172.21.15.73:0/723959144 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f4fe2700 1 -- 172.21.15.73:0/723959144 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2464263616 0 0) 0x7fc2f00a9a90 con 0x7fc2f00caab0 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f4fe2700 1 -- 172.21.15.73:0/723959144 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2dc0033f0 con 0x7fc2f00caab0 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f57e3700 1 -- 172.21.15.73:0/723959144 >> v1:172.21.15.150:6789/0 conn(0x7fc2f00ca180 legacy=0x7fc2f015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f57e3700 1 -- 172.21.15.73:0/723959144 >> v1:172.21.15.73:6789/0 conn(0x7fc2f00caab0 legacy=0x7fc2f00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f57e3700 1 -- 172.21.15.73:0/723959144 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2f00a8e80 con 0x7fc2f00cae90 2023-11-28T04:26:17.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f57e3700 1 --2- 172.21.15.73:0/723959144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00cae90 0x7fc2f00c0ab0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 rx=0x7fc2e0009760 tx=0x7fc2e0009060).ready entity=mon.0 client_cookie=174e604457eb3adc server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f4fe2700 1 -- 172.21.15.73:0/723959144 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000f510 con 0x7fc2f00cae90 2023-11-28T04:26:17.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f4fe2700 1 -- 172.21.15.73:0/723959144 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2e000f670 con 0x7fc2f00cae90 2023-11-28T04:26:17.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f4fe2700 1 -- 172.21.15.73:0/723959144 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2e000f9f0 con 0x7fc2f00cae90 2023-11-28T04:26:17.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f8248700 1 -- 172.21.15.73:0/723959144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00cae90 msgr2=0x7fc2f00c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f8248700 1 --2- 172.21.15.73:0/723959144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00cae90 0x7fc2f00c0ab0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 rx=0x7fc2e0009760 tx=0x7fc2e0009060).stop 2023-11-28T04:26:17.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f8248700 1 -- 172.21.15.73:0/723959144 shutdown_connections 2023-11-28T04:26:17.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f8248700 1 --2- 172.21.15.73:0/723959144 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00cae90 0x7fc2f00c0ab0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.721+0000 7fc2f8248700 1 -- 172.21.15.73:0/723959144 >> 172.21.15.73:0/723959144 conn(0x7fc2f0094920 msgr2=0x7fc2f006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.722+0000 7fc2f8248700 1 -- 172.21.15.73:0/723959144 shutdown_connections 2023-11-28T04:26:17.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.722+0000 7fc2f8248700 1 -- 172.21.15.73:0/723959144 wait complete. 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 Processor -- start 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 -- start start 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 0x7fc2f00c6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2f00cae90 0x7fc2f00a7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c8230 con 0x7fc2f00ca180 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f0098bb0 con 0x7fc2f00caab0 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.723+0000 7fc2f8248700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2f00c7620 con 0x7fc2f00cae90 2023-11-28T04:26:17.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38882/0 (socket says 172.21.15.73:38882) 2023-11-28T04:26:17.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 -- 172.21.15.73:0/4131931686 learned_addr learned my addr 172.21.15.73:0/4131931686 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:17.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f57e3700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 0x7fc2f00c6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f67e5700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2f00cae90 0x7fc2f00a7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 -- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2f00cae90 msgr2=0x7fc2f00a7fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2f00cae90 0x7fc2f00a7fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 -- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 msgr2=0x7fc2f00c6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 0x7fc2f00c6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 -- 172.21.15.73:0/4131931686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e00092a0 con 0x7fc2f00ca180 2023-11-28T04:26:17.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f57e3700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 0x7fc2f00c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f5fe4700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7fc2ec00c840 tx=0x7fc2ec007e80).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec005c00 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2ec005d60 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2f00c7900 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec007320 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.724+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2f00c7ec0 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.725+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2f006c860 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.726+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc2ec0076d0 con 0x7fc2f00ca180 2023-11-28T04:26:17.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.726+0000 7fc2e5ffb700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 0x7fc2d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:17.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.726+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7fc2ec079f90 con 0x7fc2f00ca180 2023-11-28T04:26:17.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.726+0000 7fc2f57e3700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 0x7fc2d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:17.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.726+0000 7fc2f57e3700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 0x7fc2d4061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc2f0147ae0 tx=0x7fc2e0017000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:17.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.730+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc2ec04b5b0 con 0x7fc2f00ca180 2023-11-28T04:26:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.895+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fc2f00a9160 con 0x7fc2f00ca180 2023-11-28T04:26:17.897 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.896+0000 7fc2e5ffb700 1 -- 172.21.15.73:0/4131931686 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v208) v1 ==== 84+0+143 (secure 0 0 0) 0x7fc2ec04ebd0 con 0x7fc2f00ca180 2023-11-28T04:26:17.897 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:17.897 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:26:17.897 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 1, 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "device_health_metrics" 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: }, 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: "poolnum": 2, 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: "poolname": "rbd" 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:17.898 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 msgr2=0x7fc2d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 0x7fc2d4061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7fc2f0147ae0 tx=0x7fc2e0017000).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 msgr2=0x7fc2f00986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 rx=0x7fc2ec00c840 tx=0x7fc2ec007e80).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 shutdown_connections 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc2d405f150 0x7fc2d4061610 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc2f00cae90 0x7fc2f00a7fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc2f00ca180 0x7fc2f00986c0 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 --2- 172.21.15.73:0/4131931686 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc2f00caab0 0x7fc2f00c6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 >> 172.21.15.73:0/4131931686 conn(0x7fc2f0094920 msgr2=0x7fc2f0065100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:17.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 shutdown_connections 2023-11-28T04:26:17.901 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:17.898+0000 7fc2f8248700 1 -- 172.21.15.73:0/4131931686 wait complete. 2023-11-28T04:26:17.911 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-11-28T04:26:17.911 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata 64 --pg_num_min 64 2023-11-28T04:26:18.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.027+0000 7f0d0359e700 1 Processor -- start 2023-11-28T04:26:18.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.027+0000 7f0d0359e700 1 -- start start 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d041542d0 0x7f0d040a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 0x7f0d040c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d040aa950 con 0x7f0d041542d0 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d04158d50 con 0x7f0d040c5fc0 2023-11-28T04:26:18.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04158140 con 0x7f0d041546b0 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d041542d0 0x7f0d040a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d041542d0 0x7f0d040a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38892/0 (socket says 172.21.15.73:38892) 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.028+0000 7f0d0259c700 1 -- 172.21.15.73:0/1212789907 learned_addr learned my addr 172.21.15.73:0/1212789907 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d02d9d700 1 --2- 172.21.15.73:0/1212789907 >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 0x7f0d040c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0cf9d9b700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 -- 172.21.15.73:0/1212789907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 msgr2=0x7f0d040aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 -- 172.21.15.73:0/1212789907 >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 msgr2=0x7f0d040c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:18.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 --2- 172.21.15.73:0/1212789907 >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 0x7f0d040c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 -- 172.21.15.73:0/1212789907 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d04158420 con 0x7f0d041542d0 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0cf9d9b700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d02d9d700 1 --2- 172.21.15.73:0/1212789907 >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 0x7f0d040c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0259c700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d041542d0 0x7f0d040a9f70 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 rx=0x7f0d040b11e0 tx=0x7f0cf40090f0).ready entity=mon.0 client_cookie=94a1bc9d3a478208 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d01d9b700 1 -- 172.21.15.73:0/1212789907 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400b600 con 0x7f0d041542d0 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d01d9b700 1 -- 172.21.15.73:0/1212789907 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400b760 con 0x7f0d041542d0 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d01d9b700 1 -- 172.21.15.73:0/1212789907 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400bae0 con 0x7f0d041542d0 2023-11-28T04:26:18.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 -- 172.21.15.73:0/1212789907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d041542d0 msgr2=0x7f0d040a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d041542d0 0x7f0d040a9f70 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 rx=0x7f0d040b11e0 tx=0x7f0cf40090f0).stop 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 -- 172.21.15.73:0/1212789907 shutdown_connections 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d041542d0 0x7f0d040a9f70 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 --2- 172.21.15.73:0/1212789907 >> v2:172.21.15.150:3300/0 conn(0x7f0d040c5fc0 0x7f0d040c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 --2- 172.21.15.73:0/1212789907 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.029+0000 7f0d0359e700 1 -- 172.21.15.73:0/1212789907 >> 172.21.15.73:0/1212789907 conn(0x7f0d040cb010 msgr2=0x7f0d040cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:18.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.030+0000 7f0d0359e700 1 -- 172.21.15.73:0/1212789907 shutdown_connections 2023-11-28T04:26:18.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.030+0000 7f0d0359e700 1 -- 172.21.15.73:0/1212789907 wait complete. 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 Processor -- start 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 -- start start 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d041542d0 0x7f0d04147f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040ab620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c4f20 con 0x7f0d040c5fc0 2023-11-28T04:26:18.037 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d04156310 con 0x7f0d041542d0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.031+0000 7f0d0359e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d040c36f0 con 0x7f0d041546b0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38894/0 (socket says 172.21.15.73:38894) 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 -- 172.21.15.73:0/3555358415 learned_addr learned my addr 172.21.15.73:0/3555358415 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0cf9d9b700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d041542d0 0x7f0d04147f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d02d9d700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040ab620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 -- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 msgr2=0x7f0d040ab620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040ab620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 -- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d041542d0 msgr2=0x7f0d04147f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d041542d0 0x7f0d04147f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 -- 172.21.15.73:0/3555358415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf4009370 con 0x7f0d040c5fc0 2023-11-28T04:26:18.038 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0259c700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 rx=0x7f0cf4006fd0 tx=0x7f0cf4017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf400b8d0 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf400ba30 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d040c4c40 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf401b660 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.032+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d041565f0 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.033+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d04156e70 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.033+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0cf401ba10 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.034+0000 7f0cfaffd700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 0x7f0ce4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.034+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5269+0+0 (secure 0 0 0) 0x7f0cf407b300 con 0x7f0d040c5fc0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.034+0000 7f0cf9d9b700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 0x7f0ce4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.039 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.034+0000 7f0cf9d9b700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 0x7f0ce4061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0d040aa1b0 tx=0x7f0cec008040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.038+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0cf404c9c0 con 0x7f0d040c5fc0 2023-11-28T04:26:18.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.201+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64} v 0) v1 -- 0x7f0d0406c7c0 con 0x7f0d040c5fc0 2023-11-28T04:26:18.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.324+0000 7f0cfaffd700 1 -- 172.21.15.73:0/3555358415 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num": 64, "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v209) v1 ==== 152+0+0 (secure 0 0 0) 0x7f0cf401f020 con 0x7f0d040c5fc0 2023-11-28T04:26:18.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.326+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 msgr2=0x7f0ce4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.326+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 0x7f0ce4061610 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0d040aa1b0 tx=0x7f0cec008040).stop 2023-11-28T04:26:18.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.326+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 msgr2=0x7f0d040c20b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.326+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 rx=0x7f0cf4006fd0 tx=0x7f0cf4017000).stop 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 shutdown_connections 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0ce405f150 0x7f0ce4061610 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d041546b0 0x7f0d040ab620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d040c5fc0 0x7f0d040c20b0 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 --2- 172.21.15.73:0/3555358415 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d041542d0 0x7f0d04147f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 >> 172.21.15.73:0/3555358415 conn(0x7f0d040cb010 msgr2=0x7f0d040c6650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 shutdown_connections 2023-11-28T04:26:18.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.327+0000 7f0d0359e700 1 -- 172.21.15.73:0/3555358415 wait complete. 2023-11-28T04:26:18.333 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' created 2023-11-28T04:26:18.345 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-11-28T04:26:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.465+0000 7fe352661700 1 Processor -- start 2023-11-28T04:26:18.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.465+0000 7fe352661700 1 -- start start 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.465+0000 7fe352661700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe34c154820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.465+0000 7fe352661700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c0bdf70 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe352661700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe34c0627a0 con 0x7fe34c154440 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe352661700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0be1a0 con 0x7fe34c0c2e50 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34b7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe34c0c1dd0 0x7fe34c0c2740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48252/0 (socket says 172.21.15.73:48252) 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34b7fe700 1 -- 172.21.15.73:0/2925565949 learned_addr learned my addr 172.21.15.73:0/2925565949 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe34c154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1705805888 0 0) 0x7fe34c0bdf70 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3340033f0 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.470 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 798536665 0 0) 0x7fe34c0627a0 con 0x7fe34c154440 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe34c0bdf70 con 0x7fe34c154440 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2832963656 0 0) 0x7fe3340033f0 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 msgr2=0x7fe34c154820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe34c154820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 >> v1:172.21.15.150:6789/0 conn(0x7fe34c154440 legacy=0x7fe34c0bd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c062a40 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fe340001470 con 0x7fe34c0c1dd0 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 >> v1:172.21.15.73:6789/0 conn(0x7fe34c0c1dd0 legacy=0x7fe34c0c2740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe334003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe334003fb0 0x7fe334004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe334004b80 0x7fe334005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0be1a0 con 0x7fe334004b80 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe33400b990 con 0x7fe334003fb0 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe33400bb50 con 0x7fe34c0c2e50 2023-11-28T04:26:18.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe334004b80 0x7fe334005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.466+0000 7fe34bfff700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe334003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe34b7fe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe334003fb0 0x7fe334004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 -- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 msgr2=0x7fe334003aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0c2e50 0x7fe334003aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 -- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe334003fb0 msgr2=0x7fe334004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe334003fb0 0x7fe334004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 -- 172.21.15.73:0/2925565949 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3340033f0 con 0x7fe334004b80 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe334004b80 0x7fe334005000 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 rx=0x7fe34400e210 tx=0x7fe34400b040).ready entity=mon.0 client_cookie=56cb47749c54a4cd server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344009ac0 con 0x7fe334004b80 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe34400f040 con 0x7fe334004b80 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.467+0000 7fe34affd700 1 -- 172.21.15.73:0/2925565949 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe34400ecd0 con 0x7fe334004b80 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 -- 172.21.15.73:0/2925565949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe334004b80 msgr2=0x7fe334005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 --2- 172.21.15.73:0/2925565949 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe334004b80 0x7fe334005000 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 rx=0x7fe34400e210 tx=0x7fe34400b040).stop 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe350bfe700 1 -- 172.21.15.73:0/2925565949 reap_dead start 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 -- 172.21.15.73:0/2925565949 shutdown_connections 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 -- 172.21.15.73:0/2925565949 >> 172.21.15.73:0/2925565949 conn(0x7fe34c0c4200 msgr2=0x7fe34c0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 -- 172.21.15.73:0/2925565949 shutdown_connections 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.468+0000 7fe352661700 1 -- 172.21.15.73:0/2925565949 wait complete. 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.469+0000 7fe352661700 1 Processor -- start 2023-11-28T04:26:18.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.469+0000 7fe352661700 1 -- start start 2023-11-28T04:26:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c065720 0x7fe34c065b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0a8cf0 0x7fe34c0a9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0aac70 con 0x7fe34c0c5570 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0a9a40 con 0x7fe34c065720 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe34c0a9d20 con 0x7fe34c0a8cf0 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38918/0 (socket says 172.21.15.73:38918) 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 -- 172.21.15.73:0/2833041444 learned_addr learned my addr 172.21.15.73:0/2833041444 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe34b7fe700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0a8cf0 0x7fe34c0a9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe34bfff700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c065720 0x7fe34c065b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 -- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0a8cf0 msgr2=0x7fe34c0a9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0a8cf0 0x7fe34c0a9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 -- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c065720 msgr2=0x7fe34c065b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:18.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c065720 0x7fe34c065b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 -- 172.21.15.73:0/2833041444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe34c0aa000 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe350bfe700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 rx=0x7fe344014000 tx=0x7fe34401a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344027420 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe344027580 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe34c0aa900 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe344027730 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.470+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34c0be800 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.471+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34c06c860 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.472+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe344024020 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.472+0000 7fe333fff700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 0x7fe338061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.472+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5639+0+0 (secure 0 0 0) 0x7fe34407bec0 con 0x7fe34c0c5570 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.472+0000 7fe34bfff700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 0x7fe338061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:18.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.472+0000 7fe34bfff700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 0x7fe338061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe33c000fb0 tx=0x7fe33c009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:18.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.476+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe34404d370 con 0x7fe34c0c5570 2023-11-28T04:26:18.640 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:18.639+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fe34c062d20 con 0x7fe34c0c5570 2023-11-28T04:26:19.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.329+0000 7fe333fff700 1 -- 172.21.15.73:0/2833041444 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v210) v1 ==== 170+0+0 (secure 0 0 0) 0x7fe344050990 con 0x7fe34c0c5570 2023-11-28T04:26:19.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.333+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 msgr2=0x7fe338061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.333+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 0x7fe338061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fe33c000fb0 tx=0x7fe33c009040).stop 2023-11-28T04:26:19.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.333+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 msgr2=0x7fe34c0c59f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.333+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 rx=0x7fe344014000 tx=0x7fe34401a000).stop 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 shutdown_connections 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fe33805f150 0x7fe338061610 secure :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 rx=0x7fe33c000fb0 tx=0x7fe33c009040).stop 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe34c0c5570 0x7fe34c0c59f0 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe34c065720 0x7fe34c065b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 --2- 172.21.15.73:0/2833041444 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe34c0a8cf0 0x7fe34c0a9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.334+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 >> 172.21.15.73:0/2833041444 conn(0x7fe34c0c4200 msgr2=0x7fe34c15a510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:19.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.336+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 shutdown_connections 2023-11-28T04:26:19.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.337+0000 7fe352661700 1 -- 172.21.15.73:0/2833041444 wait complete. 2023-11-28T04:26:19.343 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' created 2023-11-28T04:26:19.365 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-11-28T04:26:19.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.480+0000 7f4d98940700 1 Processor -- start 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.480+0000 7f4d98940700 1 -- start start 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d98940700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4d94098d10 0x7f4d940c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d98940700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d94158320 0x7f4d941587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d98940700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d94158cc0 con 0x7f4d940c6190 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d98940700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d940c72d0 con 0x7f4d94158320 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d98940700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d94064570 con 0x7f4d94098d10 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d92d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d94158320 0x7f4d941587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d92d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4d94158320 0x7f4d941587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38928/0 (socket says 172.21.15.73:38928) 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d92d9d700 1 -- 172.21.15.73:0/1636113548 learned_addr learned my addr 172.21.15.73:0/1636113548 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:19.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.481+0000 7f4d9259c700 1 --2- 172.21.15.73:0/1636113548 >> v2:172.21.15.150:3300/0 conn(0x7f4d94098d10 0x7f4d940c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d92d9d700 1 -- 172.21.15.73:0/1636113548 >> v2:172.21.15.150:3300/0 conn(0x7f4d94098d10 msgr2=0x7f4d940c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d92d9d700 1 --2- 172.21.15.73:0/1636113548 >> v2:172.21.15.150:3300/0 conn(0x7f4d94098d10 0x7f4d940c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d92d9d700 1 -- 172.21.15.73:0/1636113548 >> v1:172.21.15.150:6789/0 conn(0x7f4d940c6190 legacy=0x7f4d94063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d92d9d700 1 -- 172.21.15.73:0/1636113548 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d94154bc0 con 0x7f4d94158320 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d92d9d700 1 --2- 172.21.15.73:0/1636113548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d941587d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 rx=0x7f4d94083f80 tx=0x7f4d8c0090f0).ready entity=mon.0 client_cookie=d7d7999137eb4673 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d9159a700 1 -- 172.21.15.73:0/1636113548 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00b520 con 0x7f4d94158320 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d9159a700 1 -- 172.21.15.73:0/1636113548 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00b680 con 0x7f4d94158320 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d9159a700 1 -- 172.21.15.73:0/1636113548 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00ba00 con 0x7f4d94158320 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 -- 172.21.15.73:0/1636113548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 msgr2=0x7f4d941587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 --2- 172.21.15.73:0/1636113548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d941587d0 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 rx=0x7f4d94083f80 tx=0x7f4d8c0090f0).stop 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 -- 172.21.15.73:0/1636113548 shutdown_connections 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 --2- 172.21.15.73:0/1636113548 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d941587d0 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 --2- 172.21.15.73:0/1636113548 >> v2:172.21.15.150:3300/0 conn(0x7f4d94098d10 0x7f4d940c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.482+0000 7f4d98940700 1 -- 172.21.15.73:0/1636113548 >> 172.21.15.73:0/1636113548 conn(0x7f4d940cb310 msgr2=0x7f4d940cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.483+0000 7f4d98940700 1 -- 172.21.15.73:0/1636113548 shutdown_connections 2023-11-28T04:26:19.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.483+0000 7f4d98940700 1 -- 172.21.15.73:0/1636113548 wait complete. 2023-11-28T04:26:19.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 Processor -- start 2023-11-28T04:26:19.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 -- start start 2023-11-28T04:26:19.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d94098d10 0x7f4d9414b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940c6190 0x7f4d941480b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d9414c260 con 0x7f4d94158320 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d940ae2c0 con 0x7f4d940c6190 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d98940700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d94155ec0 con 0x7f4d94098d10 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d92d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38938/0 (socket says 172.21.15.73:38938) 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d92d9d700 1 -- 172.21.15.73:0/2159776821 learned_addr learned my addr 172.21.15.73:0/2159776821 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d9259c700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d94098d10 0x7f4d9414b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.484+0000 7f4d91d9b700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940c6190 0x7f4d941480b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 -- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d94098d10 msgr2=0x7f4d9414b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d94098d10 0x7f4d9414b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 -- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940c6190 msgr2=0x7f4d941480b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940c6190 0x7f4d941480b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 -- 172.21.15.73:0/2159776821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d8c009370 con 0x7f4d94158320 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d92d9d700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 rx=0x7f4d8c009130 tx=0x7f4d8c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c00b520 con 0x7f4d94158320 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d8c00b7f0 con 0x7f4d94158320 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d9414bf80 con 0x7f4d94158320 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d8c01c670 con 0x7f4d94158320 2023-11-28T04:26:19.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.485+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d941486c0 con 0x7f4d94158320 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.486+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d941488e0 con 0x7f4d94158320 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.487+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4d8c01c9e0 con 0x7f4d94158320 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.487+0000 7f4d8a7fc700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 0x7f4d74061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.487+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6021+0+0 (secure 0 0 0) 0x7f4d8c07c2f0 con 0x7f4d94158320 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.487+0000 7f4d9259c700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 0x7f4d74061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.488+0000 7f4d9259c700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 0x7f4d74061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4d84009fd0 tx=0x7f4d8400b040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.491+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4d8c04d5e0 con 0x7f4d94158320 2023-11-28T04:26:19.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.697+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f4d9406c860 con 0x7f4d94158320 2023-11-28T04:26:19.705 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.703+0000 7f4d8a7fc700 1 -- 172.21.15.73:0/2159776821 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 25 and data pool 26 v74) v1 ==== 174+0+0 (secure 0 0 0) 0x7f4d8c020070 con 0x7f4d94158320 2023-11-28T04:26:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.708+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 msgr2=0x7f4d74061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.708+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 0x7f4d74061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f4d84009fd0 tx=0x7f4d8400b040).stop 2023-11-28T04:26:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.708+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 msgr2=0x7f4d94147720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.708+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 rx=0x7f4d8c009130 tx=0x7f4d8c017000).stop 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 shutdown_connections 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4d7405f150 0x7f4d74061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4d94158320 0x7f4d94147720 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4d94098d10 0x7f4d9414b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 --2- 172.21.15.73:0/2159776821 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4d940c6190 0x7f4d941480b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 >> 172.21.15.73:0/2159776821 conn(0x7f4d940cb310 msgr2=0x7f4d941530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:19.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.709+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 shutdown_connections 2023-11-28T04:26:19.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.710+0000 7f4d98940700 1 -- 172.21.15.73:0/2159776821 wait complete. 2023-11-28T04:26:19.712 INFO:teuthology.orchestra.run.smithi073.stderr:new fs with metadata pool 25 and data pool 26 2023-11-28T04:26:19.731 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:19.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.843+0000 7fd41b1ab700 1 Processor -- start 2023-11-28T04:26:19.844 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.843+0000 7fd41b1ab700 1 -- start start 2023-11-28T04:26:19.846 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd41b1ab700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4140caab0 0x7fd4140aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd41b1ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd4140c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd41b1ab700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4140a9a90 con 0x7fd4140aa960 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd41b1ab700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd414154ce0 con 0x7fd4140caab0 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd41b1ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140a8e80 con 0x7fd4140cae90 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd418f47700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4140caab0 0x7fd4140aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd419748700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fd4140aa960 0x7fd4140c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48262/0 (socket says 172.21.15.73:48262) 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd418f47700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fd4140caab0 0x7fd4140aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38942/0 (socket says 172.21.15.73:38942) 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd419748700 1 -- 172.21.15.73:0/1264294356 learned_addr learned my addr 172.21.15.73:0/1264294356 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.844+0000 7fd413fff700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd4140c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd418f47700 1 -- 172.21.15.73:0/1264294356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 msgr2=0x7fd4140c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd418f47700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd4140c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd418f47700 1 -- 172.21.15.73:0/1264294356 >> v1:172.21.15.73:6789/0 conn(0x7fd4140aa960 legacy=0x7fd4140c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd418f47700 1 -- 172.21.15.73:0/1264294356 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4140a9160 con 0x7fd4140caab0 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd4137fe700 1 -- 172.21.15.73:0/1264294356 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436640912 0 0) 0x7fd4140a9a90 con 0x7fd4140aa960 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd418f47700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd4140aa470 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 rx=0x7fd404006fd0 tx=0x7fd404006f40).ready entity=mon.0 client_cookie=2213cb727ad50cda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd4137fe700 1 -- 172.21.15.73:0/1264294356 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404005880 con 0x7fd4140caab0 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd4137fe700 1 -- 172.21.15.73:0/1264294356 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4040059e0 con 0x7fd4140caab0 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd4137fe700 1 -- 172.21.15.73:0/1264294356 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd404005d60 con 0x7fd4140caab0 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 -- 172.21.15.73:0/1264294356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 msgr2=0x7fd4140aa470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd4140aa470 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 rx=0x7fd404006fd0 tx=0x7fd404006f40).stop 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 -- 172.21.15.73:0/1264294356 shutdown_connections 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd4140aa470 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/1264294356 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd4140c0ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 -- 172.21.15.73:0/1264294356 >> 172.21.15.73:0/1264294356 conn(0x7fd414094920 msgr2=0x7fd41406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:19.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 -- 172.21.15.73:0/1264294356 shutdown_connections 2023-11-28T04:26:19.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.845+0000 7fd41b1ab700 1 -- 172.21.15.73:0/1264294356 wait complete. 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 Processor -- start 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 -- start start 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd414155a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd414155f20 0x7fd4140c3550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140c3130 con 0x7fd4140caab0 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140c24b0 con 0x7fd414155f20 2023-11-28T04:26:19.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.847+0000 7fd41b1ab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4140c2790 con 0x7fd4140cae90 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38952/0 (socket says 172.21.15.73:38952) 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 -- 172.21.15.73:0/550578559 learned_addr learned my addr 172.21.15.73:0/550578559 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd413fff700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd414155a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd419748700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd414155f20 0x7fd4140c3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 -- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 msgr2=0x7fd414155a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd414155a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 -- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd414155f20 msgr2=0x7fd4140c3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd414155f20 0x7fd4140c3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 -- 172.21.15.73:0/550578559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd404009250 con 0x7fd4140caab0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd418f47700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7fd414083c30 tx=0x7fd404016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400f720 con 0x7fd4140caab0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd40400f8a0 con 0x7fd4140caab0 2023-11-28T04:26:19.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4140c2a70 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd40400fc60 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.848+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd41414e530 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.849+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd41406c860 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.849+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fd404011650 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.850+0000 7fd410ff9700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 0x7fd400061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.850+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fd40407b570 con 0x7fd4140caab0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.850+0000 7fd413fff700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 0x7fd400061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:19.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.850+0000 7fd413fff700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 0x7fd400061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd408009fd0 tx=0x7fd40800b040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:19.853 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:26:19.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:19.854+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fd40404c840 con 0x7fd4140caab0 2023-11-28T04:26:19.855 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:26:20.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.020+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd4140a9440 con 0x7fd4140caab0 2023-11-28T04:26:20.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.021+0000 7fd410ff9700 1 -- 172.21.15.73:0/550578559 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16956 (secure 0 0 0) 0x7fd40404fe60 con 0x7fd4140caab0 2023-11-28T04:26:20.022 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:20.022 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":211,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:19.698728+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 msgr2=0x7fd400061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 0x7fd400061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fd408009fd0 tx=0x7fd40800b040).stop 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 msgr2=0x7fd41414e040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 rx=0x7fd414083c30 tx=0x7fd404016000).stop 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 shutdown_connections 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fd40005f150 0x7fd400061610 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fd414155f20 0x7fd4140c3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fd4140caab0 0x7fd41414e040 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 --2- 172.21.15.73:0/550578559 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fd4140cae90 0x7fd414155a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 >> 172.21.15.73:0/550578559 conn(0x7fd414094920 msgr2=0x7fd4140af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 shutdown_connections 2023-11-28T04:26:20.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.023+0000 7fd41b1ab700 1 -- 172.21.15.73:0/550578559 wait complete. 2023-11-28T04:26:20.036 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.148+0000 7f5310a64700 1 Processor -- start 2023-11-28T04:26:20.149 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.148+0000 7f5310a64700 1 -- start start 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f5310a64700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f530c15f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f5310a64700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c0a9a90 con 0x7f530c0caab0 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f5310a64700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530c154ce0 con 0x7f530c0cae90 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f5310a64700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0aab80 con 0x7f530c0ca180 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f530a59c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f530c0caab0 0x7f530c0aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48272/0 (socket says 172.21.15.73:48272) 2023-11-28T04:26:20.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f530a59c700 1 -- 172.21.15.73:0/406851307 learned_addr learned my addr 172.21.15.73:0/406851307 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f530ad9d700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f530c15f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2392010450 0 0) 0x7f530c0a9a90 con 0x7f530c0caab0 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.149+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f52f80033f0 con 0x7f530c0caab0 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414640449 0 0) 0x7f530c154ce0 con 0x7f530c0cae90 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f530c0a9a90 con 0x7f530c0cae90 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4119419964 0 0) 0x7f52f80033f0 con 0x7f530c0caab0 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 msgr2=0x7f530c15f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f530c15f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 >> v1:172.21.15.150:6789/0 conn(0x7f530c0cae90 legacy=0x7f530c0c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0a8e80 con 0x7f530c0caab0 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f52fc001970 con 0x7f530c0caab0 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 >> v1:172.21.15.73:6789/0 conn(0x7f530c0caab0 legacy=0x7f530c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f8003aa0 0x7f52f8003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f52f8004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f52f8005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0aab80 con 0x7f52f8004b80 2023-11-28T04:26:20.159 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52f800b990 con 0x7f52f8003aa0 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52f800bb50 con 0x7f530c0ca180 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f52f8005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530a59c700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f8003aa0 0x7f52f8003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f5309d9b700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f52f8004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 -- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 msgr2=0x7f52f8004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f52f8004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 -- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f8003aa0 msgr2=0x7f52f8003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f52f8003aa0 0x7f52f8003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 -- 172.21.15.73:0/406851307 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f80033f0 con 0x7f52f8004b80 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f5309d9b700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0ca180 0x7f52f8004690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530ad9d700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f52f8005000 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7f530c0a9a40 tx=0x7f5304009060).ready entity=mon.0 client_cookie=98516e1db3db1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5304009430 con 0x7f52f8004b80 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5304009e70 con 0x7f52f8004b80 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.150+0000 7f530959a700 1 -- 172.21.15.73:0/406851307 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f530400e850 con 0x7f52f8004b80 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 -- 172.21.15.73:0/406851307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 msgr2=0x7f52f8005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 --2- 172.21.15.73:0/406851307 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f52f8005000 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 rx=0x7f530c0a9a40 tx=0x7f5304009060).stop 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f530ad9d700 1 -- 172.21.15.73:0/406851307 reap_dead start 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 -- 172.21.15.73:0/406851307 shutdown_connections 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 -- 172.21.15.73:0/406851307 >> 172.21.15.73:0/406851307 conn(0x7f530c094920 msgr2=0x7f530c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 -- 172.21.15.73:0/406851307 shutdown_connections 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.151+0000 7f5310a64700 1 -- 172.21.15.73:0/406851307 wait complete. 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.152+0000 7f5310a64700 1 Processor -- start 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.152+0000 7f5310a64700 1 -- start start 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 0x7f530c0ad4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 0x7f530c064d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0ac440 con 0x7f52f8004b80 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0ac720 con 0x7f530c064930 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5310a64700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f530c0aca00 con 0x7f530c0c63c0 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38968/0 (socket says 172.21.15.73:38968) 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 -- 172.21.15.73:0/3435000673 learned_addr learned my addr 172.21.15.73:0/3435000673 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:20.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530ad9d700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 0x7f530c064d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5309d9b700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 0x7f530c0ad4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 -- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 msgr2=0x7f530c0ad4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 0x7f530c0ad4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 -- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 msgr2=0x7f530c064d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 0x7f530c064d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 -- 172.21.15.73:0/3435000673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f530c0acce0 con 0x7f52f8004b80 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530ad9d700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 0x7f530c064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f5309d9b700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 0x7f530c0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.153+0000 7f530a59c700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7f52fc001620 tx=0x7f52fc009040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.164 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.154+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc005c00 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.154+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52fc005d60 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.154+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f530c0acfc0 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.154+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52fc007300 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.154+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f530c0ac210 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.155+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f530c0ad200 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.155+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f52fc0076b0 con 0x7f52f8004b80 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.155+0000 7f53027fc700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 0x7f52ec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.165 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.155+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f52fc079d20 con 0x7f52f8004b80 2023-11-28T04:26:20.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.155+0000 7f5309d9b700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 0x7f52ec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.156+0000 7f5309d9b700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 0x7f52ec061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52f40011e0 tx=0x7f52f4009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.166 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.159+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f52fc049af0 con 0x7f52f8004b80 2023-11-28T04:26:20.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.323+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f530c06c860 con 0x7f52f8004b80 2023-11-28T04:26:20.325 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.324+0000 7f53027fc700 1 -- 172.21.15.73:0/3435000673 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v211) v1 ==== 74+0+16956 (secure 0 0 0) 0x7f52fc04b0a0 con 0x7f52f8004b80 2023-11-28T04:26:20.325 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:20.326 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":211,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:19.698728+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:20.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 msgr2=0x7f52ec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 0x7f52ec061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f52f40011e0 tx=0x7f52f4009040).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 msgr2=0x7f530c0c7200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 rx=0x7f52fc001620 tx=0x7f52fc009040).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 shutdown_connections 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f52ec05f150 0x7f52ec061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f530c064930 0x7f530c064d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f52f8004b80 0x7f530c0c7200 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 --2- 172.21.15.73:0/3435000673 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f530c0c63c0 0x7f530c0ad4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 >> 172.21.15.73:0/3435000673 conn(0x7f530c094920 msgr2=0x7f530c0a8690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 shutdown_connections 2023-11-28T04:26:20.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.326+0000 7f5310a64700 1 -- 172.21.15.73:0/3435000673 wait complete. 2023-11-28T04:26:20.340 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-11-28T04:26:20.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.455+0000 7f2dec3a8700 1 Processor -- start 2023-11-28T04:26:20.456 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.455+0000 7f2dec3a8700 1 -- start start 2023-11-28T04:26:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 0x7f2de40c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.458 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2de4064350 0x7f2de4158320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de40c72d0 con 0x7f2de4064350 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2de4154bc0 con 0x7f2de4098d10 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dec3a8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40c66c0 con 0x7f2de40c6190 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2de4064350 0x7f2de4158320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2de4064350 0x7f2de4158320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38976/0 (socket says 172.21.15.73:38976) 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 -- 172.21.15.73:0/1567851605 learned_addr learned my addr 172.21.15.73:0/1567851605 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2de9943700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea144700 1 --2- 172.21.15.73:0/1567851605 >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 0x7f2de40c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 -- 172.21.15.73:0/1567851605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 msgr2=0x7f2de4063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 -- 172.21.15.73:0/1567851605 >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 msgr2=0x7f2de40c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 --2- 172.21.15.73:0/1567851605 >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 0x7f2de40c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 -- 172.21.15.73:0/1567851605 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2de40c69a0 con 0x7f2de4064350 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea144700 1 --2- 172.21.15.73:0/1567851605 >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 0x7f2de40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2de9943700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.456+0000 7f2dea945700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4064350 0x7f2de4158320 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 rx=0x7f2de4083f80 tx=0x7f2ddc009840).ready entity=mon.0 client_cookie=3e5ca81070c9c989 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2de9142700 1 -- 172.21.15.73:0/1567851605 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00e070 con 0x7f2de4064350 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2de9142700 1 -- 172.21.15.73:0/1567851605 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ddc00ba50 con 0x7f2de4064350 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2de9142700 1 -- 172.21.15.73:0/1567851605 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ddc00bdd0 con 0x7f2de4064350 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 -- 172.21.15.73:0/1567851605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4064350 msgr2=0x7f2de4158320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4064350 0x7f2de4158320 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 rx=0x7f2de4083f80 tx=0x7f2ddc009840).stop 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 -- 172.21.15.73:0/1567851605 shutdown_connections 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4064350 0x7f2de4158320 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/1567851605 >> v2:172.21.15.150:3300/0 conn(0x7f2de4098d10 0x7f2de40c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/1567851605 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 -- 172.21.15.73:0/1567851605 >> 172.21.15.73:0/1567851605 conn(0x7f2de40cb310 msgr2=0x7f2de40cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 -- 172.21.15.73:0/1567851605 shutdown_connections 2023-11-28T04:26:20.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.457+0000 7f2dec3a8700 1 -- 172.21.15.73:0/1567851605 wait complete. 2023-11-28T04:26:20.463 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.458+0000 7f2dec3a8700 1 Processor -- start 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.458+0000 7f2dec3a8700 1 -- start start 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de4064350 0x7f2de40accf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40ad1e0 con 0x7f2de4098d10 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2de4167da0 con 0x7f2de4064350 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dec3a8700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2de40a8810 con 0x7f2de40c6190 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:38978/0 (socket says 172.21.15.73:38978) 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 -- 172.21.15.73:0/875413008 learned_addr learned my addr 172.21.15.73:0/875413008 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dea144700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de4064350 0x7f2de40accf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.464 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2dea945700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 -- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 msgr2=0x7f2de4155940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4155940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 -- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de4064350 msgr2=0x7f2de40accf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de4064350 0x7f2de40accf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.459+0000 7f2de9943700 1 -- 172.21.15.73:0/875413008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ddc009ae0 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2de9943700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 rx=0x7f2dd400d890 tx=0x7f2dd4008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4016070 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2dd4011590 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2dd4011910 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2de40a8af0 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.460+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2de40a90b0 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.461+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2de406c860 con 0x7f2de4098d10 2023-11-28T04:26:20.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.461+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2dd4011c80 con 0x7f2de4098d10 2023-11-28T04:26:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.461+0000 7f2dda7fc700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 0x7f2dc8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.461+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2dd4079a00 con 0x7f2de4098d10 2023-11-28T04:26:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.461+0000 7f2dea144700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 0x7f2dc8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.462+0000 7f2dea144700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 0x7f2dc8061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2de0006d80 tx=0x7f2de0008040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.466 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.465+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2dd404acd0 con 0x7f2de4098d10 2023-11-28T04:26:20.672 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.671+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2de40c6c80 con 0x7f2de4098d10 2023-11-28T04:26:20.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.702+0000 7f2dda7fc700 1 -- 172.21.15.73:0/875413008 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v75) v1 ==== 118+0+0 (secure 0 0 0) 0x7f2dd404e2f0 con 0x7f2de4098d10 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 msgr2=0x7f2dc8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 0x7f2dc8061610 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7f2de0006d80 tx=0x7f2de0008040).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 msgr2=0x7f2de4150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 rx=0x7f2dd400d890 tx=0x7f2dd4008040).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 shutdown_connections 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2dc805f150 0x7f2dc8061610 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2de40c6190 0x7f2de4155940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2de4064350 0x7f2de40accf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 --2- 172.21.15.73:0/875413008 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2de4098d10 0x7f2de4150280 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 >> 172.21.15.73:0/875413008 conn(0x7f2de40cb310 msgr2=0x7f2de40c9fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 shutdown_connections 2023-11-28T04:26:20.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.704+0000 7f2dec3a8700 1 -- 172.21.15.73:0/875413008 wait complete. 2023-11-28T04:26:20.718 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-11-28T04:26:20.718 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-11-28T04:26:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.833+0000 7f51b4790700 1 Processor -- start 2023-11-28T04:26:20.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.833+0000 7f51b4790700 1 -- start start 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b4790700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51ac0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b4790700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f51ac158320 0x7f51ac1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b4790700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ac158cc0 con 0x7f51ac0c6190 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b4790700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ac0c72d0 con 0x7f51ac158320 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b4790700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac064570 con 0x7f51ac098d10 2023-11-28T04:26:20.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b1d2b700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f51ac0c6190 0x7f51ac063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48280/0 (socket says 172.21.15.73:48280) 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b1d2b700 1 -- 172.21.15.73:0/1205065329 learned_addr learned my addr 172.21.15.73:0/1205065329 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b252c700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51ac0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b2d2d700 1 --2- 172.21.15.73:0/1205065329 >> v2:172.21.15.150:3300/0 conn(0x7f51ac158320 0x7f51ac1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828955406 0 0) 0x7f519c001110 con 0x7f51ac0c6190 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.834+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51980033f0 con 0x7f51ac0c6190 2023-11-28T04:26:20.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4100540538 0 0) 0x7f51980033f0 con 0x7f51ac0c6190 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 msgr2=0x7f51ac0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51ac0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 >> v2:172.21.15.150:3300/0 conn(0x7f51ac158320 msgr2=0x7f51ac1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 --2- 172.21.15.73:0/1205065329 >> v2:172.21.15.150:3300/0 conn(0x7f51ac158320 0x7f51ac1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac154bc0 con 0x7f51ac0c6190 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f519c001600 con 0x7f51ac0c6190 2023-11-28T04:26:20.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 >> v1:172.21.15.73:6789/0 conn(0x7f51ac0c6190 legacy=0x7f51ac063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51980038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f5198004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5198004b80 0x7f5198005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac0c72d0 con 0x7f5198003e10 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac064570 con 0x7f5198004b80 2023-11-28T04:26:20.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519800b8e0 con 0x7f51ac098d10 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f5198004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b252c700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51980038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b2d2d700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5198004b80 0x7f5198005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 -- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 msgr2=0x7f51980038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51980038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 -- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5198004b80 msgr2=0x7f5198005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5198004b80 0x7f5198005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 -- 172.21.15.73:0/1205065329 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51980033f0 con 0x7f5198003e10 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b252c700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac098d10 0x7f51980038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b2d2d700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5198004b80 0x7f5198005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f5198004690 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 rx=0x7f519c00c3c0 tx=0x7f51ac0a8040).ready entity=mon.0 client_cookie=2d6cdd68d6c56d3f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c007be0 con 0x7f5198003e10 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519c00cc50 con 0x7f5198003e10 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.835+0000 7f51b152a700 1 -- 172.21.15.73:0/1205065329 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519c0070d0 con 0x7f5198003e10 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 -- 172.21.15.73:0/1205065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 msgr2=0x7f5198004690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 --2- 172.21.15.73:0/1205065329 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f5198004690 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 rx=0x7f519c00c3c0 tx=0x7f51ac0a8040).stop 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b2d2d700 1 -- 172.21.15.73:0/1205065329 reap_dead start 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 -- 172.21.15.73:0/1205065329 shutdown_connections 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 -- 172.21.15.73:0/1205065329 >> 172.21.15.73:0/1205065329 conn(0x7f51ac0cb310 msgr2=0x7f51ac0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 -- 172.21.15.73:0/1205065329 shutdown_connections 2023-11-28T04:26:20.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.836+0000 7f51b4790700 1 -- 172.21.15.73:0/1205065329 wait complete. 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.837+0000 7f51b4790700 1 Processor -- start 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.837+0000 7f51b4790700 1 -- start start 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 0x7f51ac14b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 0x7f51ac063d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac164dd0 con 0x7f5198003e10 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac165bf0 con 0x7f51ac0638c0 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b4790700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ac064230 con 0x7f51ac14c1e0 2023-11-28T04:26:20.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39002/0 (socket says 172.21.15.73:39002) 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 -- 172.21.15.73:0/3227981780 learned_addr learned my addr 172.21.15.73:0/3227981780 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 0x7f51ac14b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b2d2d700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 0x7f51ac063d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 -- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 msgr2=0x7f51ac14b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 0x7f51ac14b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 -- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 msgr2=0x7f51ac063d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 0x7f51ac063d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:20.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 -- 172.21.15.73:0/3227981780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ac165000 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b2d2d700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 0x7f51ac063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 0x7f51ac14b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51b252c700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f51a8011990 tx=0x7f51a8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.838+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800f6d0 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.839+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a800f830 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.839+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ac169260 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.839+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a800fbb0 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.839+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ac1655a0 con 0x7f5198003e10 2023-11-28T04:26:20.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.840+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ac165f40 con 0x7f5198003e10 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.840+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f51a8015480 con 0x7f5198003e10 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.840+0000 7f51a27fc700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 0x7f5190061640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.840+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f51a807a690 con 0x7f5198003e10 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.840+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 0x7f5190061640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.841+0000 7f51b1d2b700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 0x7f5190061640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f51ac0c8dc0 tx=0x7f519c015000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:20.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:20.844+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f51a804b900 con 0x7f5198003e10 2023-11-28T04:26:21.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.049+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f51ac06c860 con 0x7f5198003e10 2023-11-28T04:26:21.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.711+0000 7f51a27fc700 1 -- 172.21.15.73:0/3227981780 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7f51a804ef20 con 0x7f5198003e10 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 msgr2=0x7f5190061640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 0x7f5190061640 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7f51ac0c8dc0 tx=0x7f519c015000).stop 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 msgr2=0x7f51ac168d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 rx=0x7f51a8011990 tx=0x7f51a8008040).stop 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 shutdown_connections 2023-11-28T04:26:21.715 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f519005f180 0x7f5190061640 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f51ac0638c0 0x7f51ac063d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5198003e10 0x7f51ac168d70 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 --2- 172.21.15.73:0/3227981780 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f51ac14c1e0 0x7f51ac14b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 >> 172.21.15.73:0/3227981780 conn(0x7f51ac0cb310 msgr2=0x7f51ac154fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 shutdown_connections 2023-11-28T04:26:21.716 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.713+0000 7f51b4790700 1 -- 172.21.15.73:0/3227981780 wait complete. 2023-11-28T04:26:21.727 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-11-28T04:26:21.727 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.841+0000 7f1355744700 1 Processor -- start 2023-11-28T04:26:21.843 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.841+0000 7f1355744700 1 -- start start 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f1355744700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f1350155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f1355744700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1350095e80 0x7f1350096310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f1355744700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1350096800 con 0x7f13501544d0 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f1355744700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1350067650 con 0x7f1350095e80 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f1355744700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13500683d0 con 0x7f1350064e20 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f134f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1350095e80 0x7f1350096310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f134f7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1350095e80 0x7f1350096310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39010/0 (socket says 172.21.15.73:39010) 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f134f7fe700 1 -- 172.21.15.73:0/1886340771 learned_addr learned my addr 172.21.15.73:0/1886340771 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:21.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.842+0000 7f134e7fc700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f1350155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:21.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134f7fe700 1 -- 172.21.15.73:0/1886340771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 msgr2=0x7f1350155750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:21.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134f7fe700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f1350155750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134f7fe700 1 -- 172.21.15.73:0/1886340771 >> v1:172.21.15.150:6789/0 conn(0x7f13501544d0 legacy=0x7f1350066f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:21.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134f7fe700 1 -- 172.21.15.73:0/1886340771 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1350067850 con 0x7f1350095e80 2023-11-28T04:26:21.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134f7fe700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1350095e80 0x7f1350096310 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 rx=0x7f1348006fd0 tx=0x7f13480090f0).ready entity=mon.0 client_cookie=c5612ec8c90b7a68 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134dffb700 1 -- 172.21.15.73:0/1886340771 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800b600 con 0x7f1350095e80 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134dffb700 1 -- 172.21.15.73:0/1886340771 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134800b760 con 0x7f1350095e80 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f134dffb700 1 -- 172.21.15.73:0/1886340771 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800bae0 con 0x7f1350095e80 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 -- 172.21.15.73:0/1886340771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1350095e80 msgr2=0x7f1350096310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1350095e80 0x7f1350096310 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 rx=0x7f1348006fd0 tx=0x7f13480090f0).stop 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 -- 172.21.15.73:0/1886340771 shutdown_connections 2023-11-28T04:26:21.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1350095e80 0x7f1350096310 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 --2- 172.21.15.73:0/1886340771 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f1350155750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 -- 172.21.15.73:0/1886340771 >> 172.21.15.73:0/1886340771 conn(0x7f13500631d0 msgr2=0x7f13500c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:21.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 -- 172.21.15.73:0/1886340771 shutdown_connections 2023-11-28T04:26:21.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.843+0000 7f1355744700 1 -- 172.21.15.73:0/1886340771 wait complete. 2023-11-28T04:26:21.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 Processor -- start 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 -- start start 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f13500a7740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1350095e80 0x7f13500acac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13500a72b0 con 0x7f13501544d0 2023-11-28T04:26:21.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13500a66a0 con 0x7f1350095e80 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.845+0000 7f1355744700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13500a6980 con 0x7f1350064e20 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39016/0 (socket says 172.21.15.73:39016) 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 -- 172.21.15.73:0/3189075865 learned_addr learned my addr 172.21.15.73:0/3189075865 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134effd700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f13500a7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:21.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 -- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 msgr2=0x7f13500a7740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f13500a7740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 -- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1350095e80 msgr2=0x7f13500acac0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1350095e80 0x7f13500acac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 -- 172.21.15.73:0/3189075865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1348009370 con 0x7f13501544d0 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f134f7fe700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 rx=0x7f135006c7f0 tx=0x7f1348017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f134800bae0 con 0x7f13501544d0 2023-11-28T04:26:21.856 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f134800bea0 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13500a6c60 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1348024680 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.846+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13500acfb0 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.847+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1350063810 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.847+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1348021070 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.848+0000 7f1346ffd700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 0x7f1330061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.848+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f134807bf00 con 0x7f13501544d0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.848+0000 7f134effd700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 0x7f1330061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:21.857 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.848+0000 7f134effd700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 0x7f1330061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1340007fd0 tx=0x7f1340001530).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:21.858 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:21.852+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f134804d1d0 con 0x7f13501544d0 2023-11-28T04:26:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.053+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f135006c860 con 0x7f13501544d0 2023-11-28T04:26:22.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.053+0000 7f1346ffd700 1 -- 172.21.15.73:0/3189075865 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f13480507f0 con 0x7f13501544d0 2023-11-28T04:26:22.055 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:22.055 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:22.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 msgr2=0x7f1330061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 0x7f1330061610 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f1340007fd0 tx=0x7f1340001530).stop 2023-11-28T04:26:22.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 msgr2=0x7f135014eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 rx=0x7f135006c7f0 tx=0x7f1348017000).stop 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 shutdown_connections 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f133005f150 0x7f1330061610 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f13501544d0 0x7f135014eb80 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1350064e20 0x7f13500a7740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 --2- 172.21.15.73:0/3189075865 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1350095e80 0x7f13500acac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 >> 172.21.15.73:0/3189075865 conn(0x7f13500631d0 msgr2=0x7f13500bf590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 shutdown_connections 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.055+0000 7f1355744700 1 -- 172.21.15.73:0/3189075865 wait complete. 2023-11-28T04:26:22.057 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:22.068 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:22.180 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.179+0000 7f22733ef700 1 Processor -- start 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.179+0000 7f22733ef700 1 -- start start 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.180+0000 7f22733ef700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.180+0000 7f22733ef700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.180+0000 7f22733ef700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f226c0c69a0 con 0x7f226c1544e0 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.180+0000 7f22733ef700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f226c0c13a0 con 0x7f226c1548c0 2023-11-28T04:26:22.181 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.180+0000 7f22733ef700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c0c1680 con 0x7f226c0c6e30 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227098a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227098a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:36138/0 (socket says 172.21.15.73:36138) 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:47810/0 (socket says 172.21.15.73:47810) 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227098a700 1 -- 172.21.15.73:0/901429170 learned_addr learned my addr 172.21.15.73:0/901429170 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 -- 172.21.15.73:0/901429170 >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 msgr2=0x7f226c06a610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 --2- 172.21.15.73:0/901429170 >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.184 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 -- 172.21.15.73:0/901429170 >> v1:172.21.15.150:6789/0 conn(0x7f226c1544e0 legacy=0x7f226c0c0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 -- 172.21.15.73:0/901429170 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f226c0c6630 con 0x7f226c0c6e30 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227098a700 1 --2- 172.21.15.73:0/901429170 >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f227198c700 1 --2- 172.21.15.73:0/901429170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f2264009e60 tx=0x7f2264009780).ready entity=mon.2 client_cookie=e59ba4fb846e20db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f2263fff700 1 -- 172.21.15.73:0/901429170 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f226400e070 con 0x7f226c0c6e30 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f2263fff700 1 -- 172.21.15.73:0/901429170 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f226400bb60 con 0x7f226c0c6e30 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.181+0000 7f2263fff700 1 -- 172.21.15.73:0/901429170 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2264012410 con 0x7f226c0c6e30 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 -- 172.21.15.73:0/901429170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 msgr2=0x7f226c0c72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 --2- 172.21.15.73:0/901429170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 rx=0x7f2264009e60 tx=0x7f2264009780).stop 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 -- 172.21.15.73:0/901429170 shutdown_connections 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 --2- 172.21.15.73:0/901429170 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c0c6e30 0x7f226c0c72e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 --2- 172.21.15.73:0/901429170 >> v2:172.21.15.150:3300/0 conn(0x7f226c1548c0 0x7f226c06a610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 -- 172.21.15.73:0/901429170 >> 172.21.15.73:0/901429170 conn(0x7f226c085020 msgr2=0x7f226c094940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.185 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 -- 172.21.15.73:0/901429170 shutdown_connections 2023-11-28T04:26:22.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.182+0000 7f22733ef700 1 -- 172.21.15.73:0/901429170 wait complete. 2023-11-28T04:26:22.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.183+0000 7f22733ef700 1 Processor -- start 2023-11-28T04:26:22.186 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.183+0000 7f22733ef700 1 -- start start 2023-11-28T04:26:22.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 0x7f226c0ac660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.188 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c1548c0 0x7f226c167f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c153c80 con 0x7f226c0c6e30 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f226c1523b0 con 0x7f226c1544e0 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f22733ef700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f226c168dd0 con 0x7f226c1548c0 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39030/0 (socket says 172.21.15.73:39030) 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 -- 172.21.15.73:0/2342641411 learned_addr learned my addr 172.21.15.73:0/2342641411 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227198c700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c1548c0 0x7f226c167f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227098a700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 0x7f226c0ac660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 -- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c1548c0 msgr2=0x7f226c167f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c1548c0 0x7f226c167f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 -- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 msgr2=0x7f226c0ac660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 0x7f226c0ac660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.189 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227118b700 1 -- 172.21.15.73:0/2342641411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c009370 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.184+0000 7f227098a700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 0x7f226c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f227118b700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 rx=0x7f226800c440 tx=0x7f2268007e50).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268007020 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2268007180 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2268007520 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2264009a20 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.185+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f226c151d60 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.186+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f226c06c860 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.186+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2268006900 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.186+0000 7f22617fa700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 0x7f22500615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.187+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f2268079640 con 0x7f226c0c6e30 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.187+0000 7f227098a700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 0x7f22500615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.190 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.187+0000 7f227098a700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 0x7f22500615e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f225c006bc0 tx=0x7f225c008040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.191 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.190+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2268047af0 con 0x7f226c0c6e30 2023-11-28T04:26:22.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.357+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f226c0c6250 con 0x7f226c0c6e30 2023-11-28T04:26:22.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.357+0000 7f22617fa700 1 -- 172.21.15.73:0/2342641411 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+16943 (secure 0 0 0) 0x7f226804a910 con 0x7f226c0c6e30 2023-11-28T04:26:22.359 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:22.359 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":212,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:20.700535+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 msgr2=0x7f22500615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 0x7f22500615e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7f225c006bc0 tx=0x7f225c008040).stop 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 msgr2=0x7f226c153790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 rx=0x7f226800c440 tx=0x7f2268007e50).stop 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 shutdown_connections 2023-11-28T04:26:22.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f225005f120 0x7f22500615e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f226c1548c0 0x7f226c167f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f226c0c6e30 0x7f226c153790 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 --2- 172.21.15.73:0/2342641411 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f226c1544e0 0x7f226c0ac660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 >> 172.21.15.73:0/2342641411 conn(0x7f226c085020 msgr2=0x7f226c0ae240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 shutdown_connections 2023-11-28T04:26:22.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.359+0000 7f22733ef700 1 -- 172.21.15.73:0/2342641411 wait complete. 2023-11-28T04:26:22.373 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-11-28T04:26:22.373 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:22.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 Processor -- start 2023-11-28T04:26:22.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 -- start start 2023-11-28T04:26:22.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f289c1544d0 0x7f289c155750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c095e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f289c0683d0 con 0x7f289c0673c0 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f289c067850 con 0x7f289c1544d0 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.486+0000 7f28a2c1e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c067b30 con 0x7f289c066f40 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f289c1544d0 0x7f289c155750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f289c1544d0 0x7f289c155750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39040/0 (socket says 172.21.15.73:39040) 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 -- 172.21.15.73:0/1171259977 learned_addr learned my addr 172.21.15.73:0/1171259977 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a11bb700 1 --1- 172.21.15.73:0/1171259977 >> v1:172.21.15.73:6789/0 conn(0x7f289c0673c0 0x7f289c063e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48296/0 (socket says 172.21.15.73:48296) 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f289bfff700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c095e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 -- 172.21.15.73:0/1171259977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 msgr2=0x7f289c095e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c095e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 -- 172.21.15.73:0/1171259977 >> v1:172.21.15.73:6789/0 conn(0x7f289c0673c0 legacy=0x7f289c063e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f289b7fe700 1 -- 172.21.15.73:0/1171259977 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3940788350 0 0) 0x7f289c0683d0 con 0x7f289c0673c0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 -- 172.21.15.73:0/1171259977 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f289c067e10 con 0x7f289c1544d0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f28a09ba700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c155750 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7f288c009f70 tx=0x7f288c009840).ready entity=mon.0 client_cookie=34746549397a9cda server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f289b7fe700 1 -- 172.21.15.73:0/1171259977 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00e070 con 0x7f289c1544d0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f289b7fe700 1 -- 172.21.15.73:0/1171259977 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f288c00bad0 con 0x7f289c1544d0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.487+0000 7f289b7fe700 1 -- 172.21.15.73:0/1171259977 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c00be50 con 0x7f289c1544d0 2023-11-28T04:26:22.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1171259977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 msgr2=0x7f289c155750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c155750 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 rx=0x7f288c009f70 tx=0x7f288c009840).stop 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1171259977 shutdown_connections 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c155750 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1171259977 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c095e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1171259977 >> 172.21.15.73:0/1171259977 conn(0x7f289c0631d0 msgr2=0x7f289c0c9110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1171259977 shutdown_connections 2023-11-28T04:26:22.492 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.488+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1171259977 wait complete. 2023-11-28T04:26:22.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.489+0000 7f28a2c1e700 1 Processor -- start 2023-11-28T04:26:22.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.489+0000 7f28a2c1e700 1 -- start start 2023-11-28T04:26:22.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c0aab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c15a790 0x7f289c15ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0aa6c0 con 0x7f289c1544d0 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289c15b100 con 0x7f289c15a790 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f289c0a9b10 con 0x7f289c066f40 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39048/0 (socket says 172.21.15.73:39048) 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 -- 172.21.15.73:0/1014391859 learned_addr learned my addr 172.21.15.73:0/1014391859 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a09ba700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c0aab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a11bb700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c15a790 0x7f289c15ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.498 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 -- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 msgr2=0x7f289c0aab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c0aab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 -- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c15a790 msgr2=0x7f289c15ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c15a790 0x7f289c15ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 -- 172.21.15.73:0/1014391859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f288c009ae0 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f289bfff700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 rx=0x7f288400c8b0 tx=0x7f2884008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884010580 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28840106e0 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2884010a60 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.490+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f289c0aee00 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.491+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f289c0aa070 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.492+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f289c0aa870 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.492+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f288401b460 con 0x7f289c1544d0 2023-11-28T04:26:22.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.492+0000 7f2898ff9700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 0x7f287c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.492+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f288407a1b0 con 0x7f289c1544d0 2023-11-28T04:26:22.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.492+0000 7f28a09ba700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 0x7f287c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.493+0000 7f28a09ba700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 0x7f287c061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f288c00baa0 tx=0x7f288c009840).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.496+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f289c0aa870 con 0x7f289c1544d0 2023-11-28T04:26:22.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.655+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f289c0aa870 con 0x7f287c05f150 2023-11-28T04:26:22.666 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.665+0000 7f2898ff9700 1 -- 172.21.15.73:0/1014391859 <== mgr.9807 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362538 (secure 0 0 0) 0x7f289c0aa870 con 0x7f287c05f150 2023-11-28T04:26:22.672 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:22.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.668+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 msgr2=0x7f287c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.668+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 0x7f287c061610 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7f288c00baa0 tx=0x7f288c009840).stop 2023-11-28T04:26:22.675 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.668+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 msgr2=0x7f289c14ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.668+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 rx=0x7f288400c8b0 tx=0x7f2884008040).stop 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 shutdown_connections 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f287c05f150 0x7f287c061610 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f289c15a790 0x7f289c15ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f289c066f40 0x7f289c0aab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 --2- 172.21.15.73:0/1014391859 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f289c1544d0 0x7f289c14ef50 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 >> 172.21.15.73:0/1014391859 conn(0x7f289c0631d0 msgr2=0x7f289c0be070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 shutdown_connections 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.669+0000 7f28a2c1e700 1 -- 172.21.15.73:0/1014391859 wait complete. 2023-11-28T04:26:22.676 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:22.682 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":20,"stamp":"2023-11-28T04:26:20.747104+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2458704,"kb_used_data":20496,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752516016,"statfs":{"total":773094113280,"available":770576400384,"internally_reserved":0,"allocated":20987904,"data_stored":10600724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27323,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-75,"num_write_kb":-57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.594620"},"pg_stats":[{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719068+0000","last_change":"2023-11-28T04:26:19.719068+0000","last_active":"2023-11-28T04:26:19.719068+0000","last_peered":"2023-11-28T04:26:19.719068+0000","last_clean":"2023-11-28T04:26:19.719068+0000","last_became_active":"2023-11-28T04:26:19.718797+0000","last_became_peered":"2023-11-28T04:26:19.718797+0000","last_unstale":"2023-11-28T04:26:19.719068+0000","last_undegraded":"2023-11-28T04:26:19.719068+0000","last_fullsized":"2023-11-28T04:26:19.719068+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3f","version":"211'3","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724230+0000","last_change":"2023-11-28T04:26:19.356666+0000","last_active":"2023-11-28T04:26:19.724230+0000","last_peered":"2023-11-28T04:26:19.724230+0000","last_clean":"2023-11-28T04:26:19.724230+0000","last_became_active":"2023-11-28T04:26:19.356013+0000","last_became_peered":"2023-11-28T04:26:19.356013+0000","last_unstale":"2023-11-28T04:26:19.724230+0000","last_undegraded":"2023-11-28T04:26:19.724230+0000","last_fullsized":"2023-11-28T04:26:19.724230+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722320+0000","last_change":"2023-11-28T04:26:19.722320+0000","last_active":"2023-11-28T04:26:19.722320+0000","last_peered":"2023-11-28T04:26:19.722320+0000","last_clean":"2023-11-28T04:26:19.722320+0000","last_became_active":"2023-11-28T04:26:19.721980+0000","last_became_peered":"2023-11-28T04:26:19.721980+0000","last_unstale":"2023-11-28T04:26:19.722320+0000","last_undegraded":"2023-11-28T04:26:19.722320+0000","last_fullsized":"2023-11-28T04:26:19.722320+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.363552+0000","last_change":"2023-11-28T04:26:19.358959+0000","last_active":"2023-11-28T04:26:19.363552+0000","last_peered":"2023-11-28T04:26:19.363552+0000","last_clean":"2023-11-28T04:26:19.363552+0000","last_became_active":"2023-11-28T04:26:19.358727+0000","last_became_peered":"2023-11-28T04:26:19.358727+0000","last_unstale":"2023-11-28T04:26:19.363552+0000","last_undegraded":"2023-11-28T04:26:19.363552+0000","last_fullsized":"2023-11-28T04:26:19.363552+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730141+0000","last_change":"2023-11-28T04:26:19.730141+0000","last_active":"2023-11-28T04:26:19.730141+0000","last_peered":"2023-11-28T04:26:19.730141+0000","last_clean":"2023-11-28T04:26:19.730141+0000","last_became_active":"2023-11-28T04:26:19.729806+0000","last_became_peered":"2023-11-28T04:26:19.729806+0000","last_unstale":"2023-11-28T04:26:19.730141+0000","last_undegraded":"2023-11-28T04:26:19.730141+0000","last_fullsized":"2023-11-28T04:26:19.730141+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351318+0000","last_change":"2023-11-28T04:26:19.348496+0000","last_active":"2023-11-28T04:26:19.351318+0000","last_peered":"2023-11-28T04:26:19.351318+0000","last_clean":"2023-11-28T04:26:19.351318+0000","last_became_active":"2023-11-28T04:26:19.346230+0000","last_became_peered":"2023-11-28T04:26:19.346230+0000","last_unstale":"2023-11-28T04:26:19.351318+0000","last_undegraded":"2023-11-28T04:26:19.351318+0000","last_fullsized":"2023-11-28T04:26:19.351318+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725688+0000","last_change":"2023-11-28T04:26:19.725688+0000","last_active":"2023-11-28T04:26:19.725688+0000","last_peered":"2023-11-28T04:26:19.725688+0000","last_clean":"2023-11-28T04:26:19.725688+0000","last_became_active":"2023-11-28T04:26:19.725405+0000","last_became_peered":"2023-11-28T04:26:19.725405+0000","last_unstale":"2023-11-28T04:26:19.725688+0000","last_undegraded":"2023-11-28T04:26:19.725688+0000","last_fullsized":"2023-11-28T04:26:19.725688+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722930+0000","last_change":"2023-11-28T04:26:19.360551+0000","last_active":"2023-11-28T04:26:19.722930+0000","last_peered":"2023-11-28T04:26:19.722930+0000","last_clean":"2023-11-28T04:26:19.722930+0000","last_became_active":"2023-11-28T04:26:19.360282+0000","last_became_peered":"2023-11-28T04:26:19.360282+0000","last_unstale":"2023-11-28T04:26:19.722930+0000","last_undegraded":"2023-11-28T04:26:19.722930+0000","last_fullsized":"2023-11-28T04:26:19.722930+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.729724+0000","last_change":"2023-11-28T04:26:19.729724+0000","last_active":"2023-11-28T04:26:19.729724+0000","last_peered":"2023-11-28T04:26:19.729724+0000","last_clean":"2023-11-28T04:26:19.729724+0000","last_became_active":"2023-11-28T04:26:19.729445+0000","last_became_peered":"2023-11-28T04:26:19.729445+0000","last_unstale":"2023-11-28T04:26:19.729724+0000","last_undegraded":"2023-11-28T04:26:19.729724+0000","last_fullsized":"2023-11-28T04:26:19.729724+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.702799+0000","last_change":"2023-11-28T04:26:19.346844+0000","last_active":"2023-11-28T04:26:19.702799+0000","last_peered":"2023-11-28T04:26:19.702799+0000","last_clean":"2023-11-28T04:26:19.702799+0000","last_became_active":"2023-11-28T04:26:19.346569+0000","last_became_peered":"2023-11-28T04:26:19.346569+0000","last_unstale":"2023-11-28T04:26:19.702799+0000","last_undegraded":"2023-11-28T04:26:19.702799+0000","last_fullsized":"2023-11-28T04:26:19.702799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.720105+0000","last_change":"2023-11-28T04:26:19.720105+0000","last_active":"2023-11-28T04:26:19.720105+0000","last_peered":"2023-11-28T04:26:19.720105+0000","last_clean":"2023-11-28T04:26:19.720105+0000","last_became_active":"2023-11-28T04:26:19.719393+0000","last_became_peered":"2023-11-28T04:26:19.719393+0000","last_unstale":"2023-11-28T04:26:19.720105+0000","last_undegraded":"2023-11-28T04:26:19.720105+0000","last_fullsized":"2023-11-28T04:26:19.720105+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715687+0000","last_change":"2023-11-28T04:26:19.349820+0000","last_active":"2023-11-28T04:26:19.715687+0000","last_peered":"2023-11-28T04:26:19.715687+0000","last_clean":"2023-11-28T04:26:19.715687+0000","last_became_active":"2023-11-28T04:26:19.348974+0000","last_became_peered":"2023-11-28T04:26:19.348974+0000","last_unstale":"2023-11-28T04:26:19.715687+0000","last_undegraded":"2023-11-28T04:26:19.715687+0000","last_fullsized":"2023-11-28T04:26:19.715687+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725736+0000","last_change":"2023-11-28T04:26:19.725736+0000","last_active":"2023-11-28T04:26:19.725736+0000","last_peered":"2023-11-28T04:26:19.725736+0000","last_clean":"2023-11-28T04:26:19.725736+0000","last_became_active":"2023-11-28T04:26:19.725393+0000","last_became_peered":"2023-11-28T04:26:19.725393+0000","last_unstale":"2023-11-28T04:26:19.725736+0000","last_undegraded":"2023-11-28T04:26:19.725736+0000","last_fullsized":"2023-11-28T04:26:19.725736+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353095+0000","last_change":"2023-11-28T04:26:19.350280+0000","last_active":"2023-11-28T04:26:19.353095+0000","last_peered":"2023-11-28T04:26:19.353095+0000","last_clean":"2023-11-28T04:26:19.353095+0000","last_became_active":"2023-11-28T04:26:19.349991+0000","last_became_peered":"2023-11-28T04:26:19.349991+0000","last_unstale":"2023-11-28T04:26:19.353095+0000","last_undegraded":"2023-11-28T04:26:19.353095+0000","last_fullsized":"2023-11-28T04:26:19.353095+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727023+0000","last_change":"2023-11-28T04:26:19.727023+0000","last_active":"2023-11-28T04:26:19.727023+0000","last_peered":"2023-11-28T04:26:19.727023+0000","last_clean":"2023-11-28T04:26:19.727023+0000","last_became_active":"2023-11-28T04:26:19.726763+0000","last_became_peered":"2023-11-28T04:26:19.726763+0000","last_unstale":"2023-11-28T04:26:19.727023+0000","last_undegraded":"2023-11-28T04:26:19.727023+0000","last_fullsized":"2023-11-28T04:26:19.727023+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355505+0000","last_change":"2023-11-28T04:26:19.351840+0000","last_active":"2023-11-28T04:26:19.355505+0000","last_peered":"2023-11-28T04:26:19.355505+0000","last_clean":"2023-11-28T04:26:19.355505+0000","last_became_active":"2023-11-28T04:26:19.351394+0000","last_became_peered":"2023-11-28T04:26:19.351394+0000","last_unstale":"2023-11-28T04:26:19.355505+0000","last_undegraded":"2023-11-28T04:26:19.355505+0000","last_fullsized":"2023-11-28T04:26:19.355505+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721107+0000","last_change":"2023-11-28T04:26:19.721107+0000","last_active":"2023-11-28T04:26:19.721107+0000","last_peered":"2023-11-28T04:26:19.721107+0000","last_clean":"2023-11-28T04:26:19.721107+0000","last_became_active":"2023-11-28T04:26:19.720834+0000","last_became_peered":"2023-11-28T04:26:19.720834+0000","last_unstale":"2023-11-28T04:26:19.721107+0000","last_undegraded":"2023-11-28T04:26:19.721107+0000","last_fullsized":"2023-11-28T04:26:19.721107+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353178+0000","last_change":"2023-11-28T04:26:19.351910+0000","last_active":"2023-11-28T04:26:19.353178+0000","last_peered":"2023-11-28T04:26:19.353178+0000","last_clean":"2023-11-28T04:26:19.353178+0000","last_became_active":"2023-11-28T04:26:19.351518+0000","last_became_peered":"2023-11-28T04:26:19.351518+0000","last_unstale":"2023-11-28T04:26:19.353178+0000","last_undegraded":"2023-11-28T04:26:19.353178+0000","last_fullsized":"2023-11-28T04:26:19.353178+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723933+0000","last_change":"2023-11-28T04:26:19.723933+0000","last_active":"2023-11-28T04:26:19.723933+0000","last_peered":"2023-11-28T04:26:19.723933+0000","last_clean":"2023-11-28T04:26:19.723933+0000","last_became_active":"2023-11-28T04:26:19.723666+0000","last_became_peered":"2023-11-28T04:26:19.723666+0000","last_unstale":"2023-11-28T04:26:19.723933+0000","last_undegraded":"2023-11-28T04:26:19.723933+0000","last_fullsized":"2023-11-28T04:26:19.723933+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355398+0000","last_change":"2023-11-28T04:26:19.350321+0000","last_active":"2023-11-28T04:26:19.355398+0000","last_peered":"2023-11-28T04:26:19.355398+0000","last_clean":"2023-11-28T04:26:19.355398+0000","last_became_active":"2023-11-28T04:26:19.350037+0000","last_became_peered":"2023-11-28T04:26:19.350037+0000","last_unstale":"2023-11-28T04:26:19.355398+0000","last_undegraded":"2023-11-28T04:26:19.355398+0000","last_fullsized":"2023-11-28T04:26:19.355398+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715712+0000","last_change":"2023-11-28T04:26:19.715712+0000","last_active":"2023-11-28T04:26:19.715712+0000","last_peered":"2023-11-28T04:26:19.715712+0000","last_clean":"2023-11-28T04:26:19.715712+0000","last_became_active":"2023-11-28T04:26:19.715462+0000","last_became_peered":"2023-11-28T04:26:19.715462+0000","last_unstale":"2023-11-28T04:26:19.715712+0000","last_undegraded":"2023-11-28T04:26:19.715712+0000","last_fullsized":"2023-11-28T04:26:19.715712+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351524+0000","last_change":"2023-11-28T04:26:19.348176+0000","last_active":"2023-11-28T04:26:19.351524+0000","last_peered":"2023-11-28T04:26:19.351524+0000","last_clean":"2023-11-28T04:26:19.351524+0000","last_became_active":"2023-11-28T04:26:19.347459+0000","last_became_peered":"2023-11-28T04:26:19.347459+0000","last_unstale":"2023-11-28T04:26:19.351524+0000","last_undegraded":"2023-11-28T04:26:19.351524+0000","last_fullsized":"2023-11-28T04:26:19.351524+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724485+0000","last_change":"2023-11-28T04:26:19.724485+0000","last_active":"2023-11-28T04:26:19.724485+0000","last_peered":"2023-11-28T04:26:19.724485+0000","last_clean":"2023-11-28T04:26:19.724485+0000","last_became_active":"2023-11-28T04:26:19.723996+0000","last_became_peered":"2023-11-28T04:26:19.723996+0000","last_unstale":"2023-11-28T04:26:19.724485+0000","last_undegraded":"2023-11-28T04:26:19.724485+0000","last_fullsized":"2023-11-28T04:26:19.724485+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.34","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728784+0000","last_change":"2023-11-28T04:26:19.354583+0000","last_active":"2023-11-28T04:26:19.728784+0000","last_peered":"2023-11-28T04:26:19.728784+0000","last_clean":"2023-11-28T04:26:19.728784+0000","last_became_active":"2023-11-28T04:26:19.354331+0000","last_became_peered":"2023-11-28T04:26:19.354331+0000","last_unstale":"2023-11-28T04:26:19.728784+0000","last_undegraded":"2023-11-28T04:26:19.728784+0000","last_fullsized":"2023-11-28T04:26:19.728784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726720+0000","last_change":"2023-11-28T04:26:19.726720+0000","last_active":"2023-11-28T04:26:19.726720+0000","last_peered":"2023-11-28T04:26:19.726720+0000","last_clean":"2023-11-28T04:26:19.726720+0000","last_became_active":"2023-11-28T04:26:19.726451+0000","last_became_peered":"2023-11-28T04:26:19.726451+0000","last_unstale":"2023-11-28T04:26:19.726720+0000","last_undegraded":"2023-11-28T04:26:19.726720+0000","last_fullsized":"2023-11-28T04:26:19.726720+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.33","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718280+0000","last_change":"2023-11-28T04:26:19.351917+0000","last_active":"2023-11-28T04:26:19.718280+0000","last_peered":"2023-11-28T04:26:19.718280+0000","last_clean":"2023-11-28T04:26:19.718280+0000","last_became_active":"2023-11-28T04:26:19.351626+0000","last_became_peered":"2023-11-28T04:26:19.351626+0000","last_unstale":"2023-11-28T04:26:19.718280+0000","last_undegraded":"2023-11-28T04:26:19.718280+0000","last_fullsized":"2023-11-28T04:26:19.718280+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724077+0000","last_change":"2023-11-28T04:26:19.724077+0000","last_active":"2023-11-28T04:26:19.724077+0000","last_peered":"2023-11-28T04:26:19.724077+0000","last_clean":"2023-11-28T04:26:19.724077+0000","last_became_active":"2023-11-28T04:26:19.723799+0000","last_became_peered":"2023-11-28T04:26:19.723799+0000","last_unstale":"2023-11-28T04:26:19.724077+0000","last_undegraded":"2023-11-28T04:26:19.724077+0000","last_fullsized":"2023-11-28T04:26:19.724077+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353162+0000","last_change":"2023-11-28T04:26:19.352478+0000","last_active":"2023-11-28T04:26:19.353162+0000","last_peered":"2023-11-28T04:26:19.353162+0000","last_clean":"2023-11-28T04:26:19.353162+0000","last_became_active":"2023-11-28T04:26:19.350726+0000","last_became_peered":"2023-11-28T04:26:19.350726+0000","last_unstale":"2023-11-28T04:26:19.353162+0000","last_undegraded":"2023-11-28T04:26:19.353162+0000","last_fullsized":"2023-11-28T04:26:19.353162+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716009+0000","last_change":"2023-11-28T04:26:19.716009+0000","last_active":"2023-11-28T04:26:19.716009+0000","last_peered":"2023-11-28T04:26:19.716009+0000","last_clean":"2023-11-28T04:26:19.716009+0000","last_became_active":"2023-11-28T04:26:19.715751+0000","last_became_peered":"2023-11-28T04:26:19.715751+0000","last_unstale":"2023-11-28T04:26:19.716009+0000","last_undegraded":"2023-11-28T04:26:19.716009+0000","last_fullsized":"2023-11-28T04:26:19.716009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357175+0000","last_change":"2023-11-28T04:26:19.354019+0000","last_active":"2023-11-28T04:26:19.357175+0000","last_peered":"2023-11-28T04:26:19.357175+0000","last_clean":"2023-11-28T04:26:19.357175+0000","last_became_active":"2023-11-28T04:26:19.353730+0000","last_became_peered":"2023-11-28T04:26:19.353730+0000","last_unstale":"2023-11-28T04:26:19.357175+0000","last_undegraded":"2023-11-28T04:26:19.357175+0000","last_fullsized":"2023-11-28T04:26:19.357175+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723092+0000","last_change":"2023-11-28T04:26:19.723092+0000","last_active":"2023-11-28T04:26:19.723092+0000","last_peered":"2023-11-28T04:26:19.723092+0000","last_clean":"2023-11-28T04:26:19.723092+0000","last_became_active":"2023-11-28T04:26:19.722668+0000","last_became_peered":"2023-11-28T04:26:19.722668+0000","last_unstale":"2023-11-28T04:26:19.723092+0000","last_undegraded":"2023-11-28T04:26:19.723092+0000","last_fullsized":"2023-11-28T04:26:19.723092+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.30","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727092+0000","last_change":"2023-11-28T04:26:19.353476+0000","last_active":"2023-11-28T04:26:19.727092+0000","last_peered":"2023-11-28T04:26:19.727092+0000","last_clean":"2023-11-28T04:26:19.727092+0000","last_became_active":"2023-11-28T04:26:19.353241+0000","last_became_peered":"2023-11-28T04:26:19.353241+0000","last_unstale":"2023-11-28T04:26:19.727092+0000","last_undegraded":"2023-11-28T04:26:19.727092+0000","last_fullsized":"2023-11-28T04:26:19.727092+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719012+0000","last_change":"2023-11-28T04:26:19.719012+0000","last_active":"2023-11-28T04:26:19.719012+0000","last_peered":"2023-11-28T04:26:19.719012+0000","last_clean":"2023-11-28T04:26:19.719012+0000","last_became_active":"2023-11-28T04:26:19.718733+0000","last_became_peered":"2023-11-28T04:26:19.718733+0000","last_unstale":"2023-11-28T04:26:19.719012+0000","last_undegraded":"2023-11-28T04:26:19.719012+0000","last_fullsized":"2023-11-28T04:26:19.719012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353592+0000","last_change":"2023-11-28T04:26:19.352413+0000","last_active":"2023-11-28T04:26:19.353592+0000","last_peered":"2023-11-28T04:26:19.353592+0000","last_clean":"2023-11-28T04:26:19.353592+0000","last_became_active":"2023-11-28T04:26:19.352122+0000","last_became_peered":"2023-11-28T04:26:19.352122+0000","last_unstale":"2023-11-28T04:26:19.353592+0000","last_undegraded":"2023-11-28T04:26:19.353592+0000","last_fullsized":"2023-11-28T04:26:19.353592+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719497+0000","last_change":"2023-11-28T04:26:19.719497+0000","last_active":"2023-11-28T04:26:19.719497+0000","last_peered":"2023-11-28T04:26:19.719497+0000","last_clean":"2023-11-28T04:26:19.719497+0000","last_became_active":"2023-11-28T04:26:19.719182+0000","last_became_peered":"2023-11-28T04:26:19.719182+0000","last_unstale":"2023-11-28T04:26:19.719497+0000","last_undegraded":"2023-11-28T04:26:19.719497+0000","last_fullsized":"2023-11-28T04:26:19.719497+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357463+0000","last_change":"2023-11-28T04:26:19.353859+0000","last_active":"2023-11-28T04:26:19.357463+0000","last_peered":"2023-11-28T04:26:19.357463+0000","last_clean":"2023-11-28T04:26:19.357463+0000","last_became_active":"2023-11-28T04:26:19.353577+0000","last_became_peered":"2023-11-28T04:26:19.353577+0000","last_unstale":"2023-11-28T04:26:19.357463+0000","last_undegraded":"2023-11-28T04:26:19.357463+0000","last_fullsized":"2023-11-28T04:26:19.357463+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721937+0000","last_change":"2023-11-28T04:26:19.721937+0000","last_active":"2023-11-28T04:26:19.721937+0000","last_peered":"2023-11-28T04:26:19.721937+0000","last_clean":"2023-11-28T04:26:19.721937+0000","last_became_active":"2023-11-28T04:26:19.721569+0000","last_became_peered":"2023-11-28T04:26:19.721569+0000","last_unstale":"2023-11-28T04:26:19.721937+0000","last_undegraded":"2023-11-28T04:26:19.721937+0000","last_fullsized":"2023-11-28T04:26:19.721937+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2d","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.712336+0000","last_change":"2023-11-28T04:26:19.346464+0000","last_active":"2023-11-28T04:26:19.712336+0000","last_peered":"2023-11-28T04:26:19.712336+0000","last_clean":"2023-11-28T04:26:19.712336+0000","last_became_active":"2023-11-28T04:26:19.346153+0000","last_became_peered":"2023-11-28T04:26:19.346153+0000","last_unstale":"2023-11-28T04:26:19.712336+0000","last_undegraded":"2023-11-28T04:26:19.712336+0000","last_fullsized":"2023-11-28T04:26:19.712336+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719934+0000","last_change":"2023-11-28T04:26:19.719934+0000","last_active":"2023-11-28T04:26:19.719934+0000","last_peered":"2023-11-28T04:26:19.719934+0000","last_clean":"2023-11-28T04:26:19.719934+0000","last_became_active":"2023-11-28T04:26:19.719677+0000","last_became_peered":"2023-11-28T04:26:19.719677+0000","last_unstale":"2023-11-28T04:26:19.719934+0000","last_undegraded":"2023-11-28T04:26:19.719934+0000","last_fullsized":"2023-11-28T04:26:19.719934+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350683+0000","last_change":"2023-11-28T04:26:19.348599+0000","last_active":"2023-11-28T04:26:19.350683+0000","last_peered":"2023-11-28T04:26:19.350683+0000","last_clean":"2023-11-28T04:26:19.350683+0000","last_became_active":"2023-11-28T04:26:19.348285+0000","last_became_peered":"2023-11-28T04:26:19.348285+0000","last_unstale":"2023-11-28T04:26:19.350683+0000","last_undegraded":"2023-11-28T04:26:19.350683+0000","last_fullsized":"2023-11-28T04:26:19.350683+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725695+0000","last_change":"2023-11-28T04:26:19.725695+0000","last_active":"2023-11-28T04:26:19.725695+0000","last_peered":"2023-11-28T04:26:19.725695+0000","last_clean":"2023-11-28T04:26:19.725695+0000","last_became_active":"2023-11-28T04:26:19.725420+0000","last_became_peered":"2023-11-28T04:26:19.725420+0000","last_unstale":"2023-11-28T04:26:19.725695+0000","last_undegraded":"2023-11-28T04:26:19.725695+0000","last_fullsized":"2023-11-28T04:26:19.725695+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.702889+0000","last_change":"2023-11-28T04:26:19.346871+0000","last_active":"2023-11-28T04:26:19.702889+0000","last_peered":"2023-11-28T04:26:19.702889+0000","last_clean":"2023-11-28T04:26:19.702889+0000","last_became_active":"2023-11-28T04:26:19.346601+0000","last_became_peered":"2023-11-28T04:26:19.346601+0000","last_unstale":"2023-11-28T04:26:19.702889+0000","last_undegraded":"2023-11-28T04:26:19.702889+0000","last_fullsized":"2023-11-28T04:26:19.702889+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725224+0000","last_change":"2023-11-28T04:26:19.725224+0000","last_active":"2023-11-28T04:26:19.725224+0000","last_peered":"2023-11-28T04:26:19.725224+0000","last_clean":"2023-11-28T04:26:19.725224+0000","last_became_active":"2023-11-28T04:26:19.724735+0000","last_became_peered":"2023-11-28T04:26:19.724735+0000","last_unstale":"2023-11-28T04:26:19.725224+0000","last_undegraded":"2023-11-28T04:26:19.725224+0000","last_fullsized":"2023-11-28T04:26:19.725224+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355822+0000","last_change":"2023-11-28T04:26:19.354593+0000","last_active":"2023-11-28T04:26:19.355822+0000","last_peered":"2023-11-28T04:26:19.355822+0000","last_clean":"2023-11-28T04:26:19.355822+0000","last_became_active":"2023-11-28T04:26:19.354253+0000","last_became_peered":"2023-11-28T04:26:19.354253+0000","last_unstale":"2023-11-28T04:26:19.355822+0000","last_undegraded":"2023-11-28T04:26:19.355822+0000","last_fullsized":"2023-11-28T04:26:19.355822+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725680+0000","last_change":"2023-11-28T04:26:19.725680+0000","last_active":"2023-11-28T04:26:19.725680+0000","last_peered":"2023-11-28T04:26:19.725680+0000","last_clean":"2023-11-28T04:26:19.725680+0000","last_became_active":"2023-11-28T04:26:19.725390+0000","last_became_peered":"2023-11-28T04:26:19.725390+0000","last_unstale":"2023-11-28T04:26:19.725680+0000","last_undegraded":"2023-11-28T04:26:19.725680+0000","last_fullsized":"2023-11-28T04:26:19.725680+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357674+0000","last_change":"2023-11-28T04:26:19.355138+0000","last_active":"2023-11-28T04:26:19.357674+0000","last_peered":"2023-11-28T04:26:19.357674+0000","last_clean":"2023-11-28T04:26:19.357674+0000","last_became_active":"2023-11-28T04:26:19.354847+0000","last_became_peered":"2023-11-28T04:26:19.354847+0000","last_unstale":"2023-11-28T04:26:19.357674+0000","last_undegraded":"2023-11-28T04:26:19.357674+0000","last_fullsized":"2023-11-28T04:26:19.357674+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724054+0000","last_change":"2023-11-28T04:26:19.724054+0000","last_active":"2023-11-28T04:26:19.724054+0000","last_peered":"2023-11-28T04:26:19.724054+0000","last_clean":"2023-11-28T04:26:19.724054+0000","last_became_active":"2023-11-28T04:26:19.723458+0000","last_became_peered":"2023-11-28T04:26:19.723458+0000","last_unstale":"2023-11-28T04:26:19.724054+0000","last_undegraded":"2023-11-28T04:26:19.724054+0000","last_fullsized":"2023-11-28T04:26:19.724054+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.358084+0000","last_change":"2023-11-28T04:26:19.353062+0000","last_active":"2023-11-28T04:26:19.358084+0000","last_peered":"2023-11-28T04:26:19.358084+0000","last_clean":"2023-11-28T04:26:19.358084+0000","last_became_active":"2023-11-28T04:26:19.352793+0000","last_became_peered":"2023-11-28T04:26:19.352793+0000","last_unstale":"2023-11-28T04:26:19.358084+0000","last_undegraded":"2023-11-28T04:26:19.358084+0000","last_fullsized":"2023-11-28T04:26:19.358084+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726127+0000","last_change":"2023-11-28T04:26:19.726127+0000","last_active":"2023-11-28T04:26:19.726127+0000","last_peered":"2023-11-28T04:26:19.726127+0000","last_clean":"2023-11-28T04:26:19.726127+0000","last_became_active":"2023-11-28T04:26:19.725855+0000","last_became_peered":"2023-11-28T04:26:19.725855+0000","last_unstale":"2023-11-28T04:26:19.726127+0000","last_undegraded":"2023-11-28T04:26:19.726127+0000","last_fullsized":"2023-11-28T04:26:19.726127+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.360952+0000","last_change":"2023-11-28T04:26:19.357248+0000","last_active":"2023-11-28T04:26:19.360952+0000","last_peered":"2023-11-28T04:26:19.360952+0000","last_clean":"2023-11-28T04:26:19.360952+0000","last_became_active":"2023-11-28T04:26:19.356918+0000","last_became_peered":"2023-11-28T04:26:19.356918+0000","last_unstale":"2023-11-28T04:26:19.360952+0000","last_undegraded":"2023-11-28T04:26:19.360952+0000","last_fullsized":"2023-11-28T04:26:19.360952+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721165+0000","last_change":"2023-11-28T04:26:19.721165+0000","last_active":"2023-11-28T04:26:19.721165+0000","last_peered":"2023-11-28T04:26:19.721165+0000","last_clean":"2023-11-28T04:26:19.721165+0000","last_became_active":"2023-11-28T04:26:19.720877+0000","last_became_peered":"2023-11-28T04:26:19.720877+0000","last_unstale":"2023-11-28T04:26:19.721165+0000","last_undegraded":"2023-11-28T04:26:19.721165+0000","last_fullsized":"2023-11-28T04:26:19.721165+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354999+0000","last_change":"2023-11-28T04:26:19.350529+0000","last_active":"2023-11-28T04:26:19.354999+0000","last_peered":"2023-11-28T04:26:19.354999+0000","last_clean":"2023-11-28T04:26:19.354999+0000","last_became_active":"2023-11-28T04:26:19.350089+0000","last_became_peered":"2023-11-28T04:26:19.350089+0000","last_unstale":"2023-11-28T04:26:19.354999+0000","last_undegraded":"2023-11-28T04:26:19.354999+0000","last_fullsized":"2023-11-28T04:26:19.354999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727472+0000","last_change":"2023-11-28T04:26:19.727472+0000","last_active":"2023-11-28T04:26:19.727472+0000","last_peered":"2023-11-28T04:26:19.727472+0000","last_clean":"2023-11-28T04:26:19.727472+0000","last_became_active":"2023-11-28T04:26:19.725452+0000","last_became_peered":"2023-11-28T04:26:19.725452+0000","last_unstale":"2023-11-28T04:26:19.727472+0000","last_undegraded":"2023-11-28T04:26:19.727472+0000","last_fullsized":"2023-11-28T04:26:19.727472+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.349904+0000","last_change":"2023-11-28T04:26:19.345619+0000","last_active":"2023-11-28T04:26:19.349904+0000","last_peered":"2023-11-28T04:26:19.349904+0000","last_clean":"2023-11-28T04:26:19.349904+0000","last_became_active":"2023-11-28T04:26:19.345399+0000","last_became_peered":"2023-11-28T04:26:19.345399+0000","last_unstale":"2023-11-28T04:26:19.349904+0000","last_undegraded":"2023-11-28T04:26:19.349904+0000","last_fullsized":"2023-11-28T04:26:19.349904+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716885+0000","last_change":"2023-11-28T04:26:19.716885+0000","last_active":"2023-11-28T04:26:19.716885+0000","last_peered":"2023-11-28T04:26:19.716885+0000","last_clean":"2023-11-28T04:26:19.716885+0000","last_became_active":"2023-11-28T04:26:19.716596+0000","last_became_peered":"2023-11-28T04:26:19.716596+0000","last_unstale":"2023-11-28T04:26:19.716885+0000","last_undegraded":"2023-11-28T04:26:19.716885+0000","last_fullsized":"2023-11-28T04:26:19.716885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.358745+0000","last_change":"2023-11-28T04:26:19.357899+0000","last_active":"2023-11-28T04:26:19.358745+0000","last_peered":"2023-11-28T04:26:19.358745+0000","last_clean":"2023-11-28T04:26:19.358745+0000","last_became_active":"2023-11-28T04:26:19.357588+0000","last_became_peered":"2023-11-28T04:26:19.357588+0000","last_unstale":"2023-11-28T04:26:19.358745+0000","last_undegraded":"2023-11-28T04:26:19.358745+0000","last_fullsized":"2023-11-28T04:26:19.358745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716064+0000","last_change":"2023-11-28T04:26:19.716064+0000","last_active":"2023-11-28T04:26:19.716064+0000","last_peered":"2023-11-28T04:26:19.716064+0000","last_clean":"2023-11-28T04:26:19.716064+0000","last_became_active":"2023-11-28T04:26:19.715788+0000","last_became_peered":"2023-11-28T04:26:19.715788+0000","last_unstale":"2023-11-28T04:26:19.716064+0000","last_undegraded":"2023-11-28T04:26:19.716064+0000","last_fullsized":"2023-11-28T04:26:19.716064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357274+0000","last_change":"2023-11-28T04:26:19.355766+0000","last_active":"2023-11-28T04:26:19.357274+0000","last_peered":"2023-11-28T04:26:19.357274+0000","last_clean":"2023-11-28T04:26:19.357274+0000","last_became_active":"2023-11-28T04:26:19.355478+0000","last_became_peered":"2023-11-28T04:26:19.355478+0000","last_unstale":"2023-11-28T04:26:19.357274+0000","last_undegraded":"2023-11-28T04:26:19.357274+0000","last_fullsized":"2023-11-28T04:26:19.357274+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725695+0000","last_change":"2023-11-28T04:26:19.725695+0000","last_active":"2023-11-28T04:26:19.725695+0000","last_peered":"2023-11-28T04:26:19.725695+0000","last_clean":"2023-11-28T04:26:19.725695+0000","last_became_active":"2023-11-28T04:26:19.725344+0000","last_became_peered":"2023-11-28T04:26:19.725344+0000","last_unstale":"2023-11-28T04:26:19.725695+0000","last_undegraded":"2023-11-28T04:26:19.725695+0000","last_fullsized":"2023-11-28T04:26:19.725695+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356784+0000","last_change":"2023-11-28T04:26:19.354796+0000","last_active":"2023-11-28T04:26:19.356784+0000","last_peered":"2023-11-28T04:26:19.356784+0000","last_clean":"2023-11-28T04:26:19.356784+0000","last_became_active":"2023-11-28T04:26:19.354503+0000","last_became_peered":"2023-11-28T04:26:19.354503+0000","last_unstale":"2023-11-28T04:26:19.356784+0000","last_undegraded":"2023-11-28T04:26:19.356784+0000","last_fullsized":"2023-11-28T04:26:19.356784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730535+0000","last_change":"2023-11-28T04:26:19.730535+0000","last_active":"2023-11-28T04:26:19.730535+0000","last_peered":"2023-11-28T04:26:19.730535+0000","last_clean":"2023-11-28T04:26:19.730535+0000","last_became_active":"2023-11-28T04:26:19.730271+0000","last_became_peered":"2023-11-28T04:26:19.730271+0000","last_unstale":"2023-11-28T04:26:19.730535+0000","last_undegraded":"2023-11-28T04:26:19.730535+0000","last_fullsized":"2023-11-28T04:26:19.730535+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353412+0000","last_change":"2023-11-28T04:26:19.351892+0000","last_active":"2023-11-28T04:26:19.353412+0000","last_peered":"2023-11-28T04:26:19.353412+0000","last_clean":"2023-11-28T04:26:19.353412+0000","last_became_active":"2023-11-28T04:26:19.351614+0000","last_became_peered":"2023-11-28T04:26:19.351614+0000","last_unstale":"2023-11-28T04:26:19.353412+0000","last_undegraded":"2023-11-28T04:26:19.353412+0000","last_fullsized":"2023-11-28T04:26:19.353412+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726326+0000","last_change":"2023-11-28T04:26:19.726326+0000","last_active":"2023-11-28T04:26:19.726326+0000","last_peered":"2023-11-28T04:26:19.726326+0000","last_clean":"2023-11-28T04:26:19.726326+0000","last_became_active":"2023-11-28T04:26:19.725441+0000","last_became_peered":"2023-11-28T04:26:19.725441+0000","last_unstale":"2023-11-28T04:26:19.726326+0000","last_undegraded":"2023-11-28T04:26:19.726326+0000","last_fullsized":"2023-11-28T04:26:19.726326+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.345811+0000","last_change":"2023-11-28T04:26:19.344437+0000","last_active":"2023-11-28T04:26:19.345811+0000","last_peered":"2023-11-28T04:26:19.345811+0000","last_clean":"2023-11-28T04:26:19.345811+0000","last_became_active":"2023-11-28T04:26:19.344146+0000","last_became_peered":"2023-11-28T04:26:19.344146+0000","last_unstale":"2023-11-28T04:26:19.345811+0000","last_undegraded":"2023-11-28T04:26:19.345811+0000","last_fullsized":"2023-11-28T04:26:19.345811+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721015+0000","last_change":"2023-11-28T04:26:19.721015+0000","last_active":"2023-11-28T04:26:19.721015+0000","last_peered":"2023-11-28T04:26:19.721015+0000","last_clean":"2023-11-28T04:26:19.721015+0000","last_became_active":"2023-11-28T04:26:19.720445+0000","last_became_peered":"2023-11-28T04:26:19.720445+0000","last_unstale":"2023-11-28T04:26:19.721015+0000","last_undegraded":"2023-11-28T04:26:19.721015+0000","last_fullsized":"2023-11-28T04:26:19.721015+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351764+0000","last_change":"2023-11-28T04:26:19.342750+0000","last_active":"2023-11-28T04:26:19.351764+0000","last_peered":"2023-11-28T04:26:19.351764+0000","last_clean":"2023-11-28T04:26:19.351764+0000","last_became_active":"2023-11-28T04:26:19.342458+0000","last_became_peered":"2023-11-28T04:26:19.342458+0000","last_unstale":"2023-11-28T04:26:19.351764+0000","last_undegraded":"2023-11-28T04:26:19.351764+0000","last_fullsized":"2023-11-28T04:26:19.351764+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.717716+0000","last_change":"2023-11-28T04:26:19.717716+0000","last_active":"2023-11-28T04:26:19.717716+0000","last_peered":"2023-11-28T04:26:19.717716+0000","last_clean":"2023-11-28T04:26:19.717716+0000","last_became_active":"2023-11-28T04:26:19.717469+0000","last_became_peered":"2023-11-28T04:26:19.717469+0000","last_unstale":"2023-11-28T04:26:19.717716+0000","last_undegraded":"2023-11-28T04:26:19.717716+0000","last_fullsized":"2023-11-28T04:26:19.717716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.359747+0000","last_change":"2023-11-28T04:26:19.359164+0000","last_active":"2023-11-28T04:26:19.359747+0000","last_peered":"2023-11-28T04:26:19.359747+0000","last_clean":"2023-11-28T04:26:19.359747+0000","last_became_active":"2023-11-28T04:26:19.358977+0000","last_became_peered":"2023-11-28T04:26:19.358977+0000","last_unstale":"2023-11-28T04:26:19.359747+0000","last_undegraded":"2023-11-28T04:26:19.359747+0000","last_fullsized":"2023-11-28T04:26:19.359747+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.729199+0000","last_change":"2023-11-28T04:26:19.729199+0000","last_active":"2023-11-28T04:26:19.729199+0000","last_peered":"2023-11-28T04:26:19.729199+0000","last_clean":"2023-11-28T04:26:19.729199+0000","last_became_active":"2023-11-28T04:26:19.728934+0000","last_became_peered":"2023-11-28T04:26:19.728934+0000","last_unstale":"2023-11-28T04:26:19.729199+0000","last_undegraded":"2023-11-28T04:26:19.729199+0000","last_fullsized":"2023-11-28T04:26:19.729199+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"211'3","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727344+0000","last_change":"2023-11-28T04:26:19.357491+0000","last_active":"2023-11-28T04:26:19.727344+0000","last_peered":"2023-11-28T04:26:19.727344+0000","last_clean":"2023-11-28T04:26:19.727344+0000","last_became_active":"2023-11-28T04:26:19.357223+0000","last_became_peered":"2023-11-28T04:26:19.357223+0000","last_unstale":"2023-11-28T04:26:19.727344+0000","last_undegraded":"2023-11-28T04:26:19.727344+0000","last_fullsized":"2023-11-28T04:26:19.727344+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730075+0000","last_change":"2023-11-28T04:26:19.730075+0000","last_active":"2023-11-28T04:26:19.730075+0000","last_peered":"2023-11-28T04:26:19.730075+0000","last_clean":"2023-11-28T04:26:19.730075+0000","last_became_active":"2023-11-28T04:26:19.729774+0000","last_became_peered":"2023-11-28T04:26:19.729774+0000","last_unstale":"2023-11-28T04:26:19.730075+0000","last_undegraded":"2023-11-28T04:26:19.730075+0000","last_fullsized":"2023-11-28T04:26:19.730075+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.713515+0000","last_change":"2023-11-28T04:26:19.348064+0000","last_active":"2023-11-28T04:26:19.713515+0000","last_peered":"2023-11-28T04:26:19.713515+0000","last_clean":"2023-11-28T04:26:19.713515+0000","last_became_active":"2023-11-28T04:26:19.347733+0000","last_became_peered":"2023-11-28T04:26:19.347733+0000","last_unstale":"2023-11-28T04:26:19.713515+0000","last_undegraded":"2023-11-28T04:26:19.713515+0000","last_fullsized":"2023-11-28T04:26:19.713515+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718462+0000","last_change":"2023-11-28T04:26:19.718462+0000","last_active":"2023-11-28T04:26:19.718462+0000","last_peered":"2023-11-28T04:26:19.718462+0000","last_clean":"2023-11-28T04:26:19.718462+0000","last_became_active":"2023-11-28T04:26:19.718206+0000","last_became_peered":"2023-11-28T04:26:19.718206+0000","last_unstale":"2023-11-28T04:26:19.718462+0000","last_undegraded":"2023-11-28T04:26:19.718462+0000","last_fullsized":"2023-11-28T04:26:19.718462+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707905+0000","last_change":"2023-11-28T04:26:19.349824+0000","last_active":"2023-11-28T04:26:19.707905+0000","last_peered":"2023-11-28T04:26:19.707905+0000","last_clean":"2023-11-28T04:26:19.707905+0000","last_became_active":"2023-11-28T04:26:19.349438+0000","last_became_peered":"2023-11-28T04:26:19.349438+0000","last_unstale":"2023-11-28T04:26:19.707905+0000","last_undegraded":"2023-11-28T04:26:19.707905+0000","last_fullsized":"2023-11-28T04:26:19.707905+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721739+0000","last_change":"2023-11-28T04:26:19.721739+0000","last_active":"2023-11-28T04:26:19.721739+0000","last_peered":"2023-11-28T04:26:19.721739+0000","last_clean":"2023-11-28T04:26:19.721739+0000","last_became_active":"2023-11-28T04:26:19.721489+0000","last_became_peered":"2023-11-28T04:26:19.721489+0000","last_unstale":"2023-11-28T04:26:19.721739+0000","last_undegraded":"2023-11-28T04:26:19.721739+0000","last_fullsized":"2023-11-28T04:26:19.721739+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357629+0000","last_change":"2023-11-28T04:26:19.355248+0000","last_active":"2023-11-28T04:26:19.357629+0000","last_peered":"2023-11-28T04:26:19.357629+0000","last_clean":"2023-11-28T04:26:19.357629+0000","last_became_active":"2023-11-28T04:26:19.354958+0000","last_became_peered":"2023-11-28T04:26:19.354958+0000","last_unstale":"2023-11-28T04:26:19.357629+0000","last_undegraded":"2023-11-28T04:26:19.357629+0000","last_fullsized":"2023-11-28T04:26:19.357629+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718326+0000","last_change":"2023-11-28T04:26:19.718326+0000","last_active":"2023-11-28T04:26:19.718326+0000","last_peered":"2023-11-28T04:26:19.718326+0000","last_clean":"2023-11-28T04:26:19.718326+0000","last_became_active":"2023-11-28T04:26:19.718044+0000","last_became_peered":"2023-11-28T04:26:19.718044+0000","last_unstale":"2023-11-28T04:26:19.718326+0000","last_undegraded":"2023-11-28T04:26:19.718326+0000","last_fullsized":"2023-11-28T04:26:19.718326+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355490+0000","last_change":"2023-11-28T04:26:19.351002+0000","last_active":"2023-11-28T04:26:19.355490+0000","last_peered":"2023-11-28T04:26:19.355490+0000","last_clean":"2023-11-28T04:26:19.355490+0000","last_became_active":"2023-11-28T04:26:19.350722+0000","last_became_peered":"2023-11-28T04:26:19.350722+0000","last_unstale":"2023-11-28T04:26:19.355490+0000","last_undegraded":"2023-11-28T04:26:19.355490+0000","last_fullsized":"2023-11-28T04:26:19.355490+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726518+0000","last_change":"2023-11-28T04:26:19.726518+0000","last_active":"2023-11-28T04:26:19.726518+0000","last_peered":"2023-11-28T04:26:19.726518+0000","last_clean":"2023-11-28T04:26:19.726518+0000","last_became_active":"2023-11-28T04:26:19.724068+0000","last_became_peered":"2023-11-28T04:26:19.724068+0000","last_unstale":"2023-11-28T04:26:19.726518+0000","last_undegraded":"2023-11-28T04:26:19.726518+0000","last_fullsized":"2023-11-28T04:26:19.726518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725664+0000","last_change":"2023-11-28T04:26:19.352144+0000","last_active":"2023-11-28T04:26:19.725664+0000","last_peered":"2023-11-28T04:26:19.725664+0000","last_clean":"2023-11-28T04:26:19.725664+0000","last_became_active":"2023-11-28T04:26:19.351806+0000","last_became_peered":"2023-11-28T04:26:19.351806+0000","last_unstale":"2023-11-28T04:26:19.725664+0000","last_undegraded":"2023-11-28T04:26:19.725664+0000","last_fullsized":"2023-11-28T04:26:19.725664+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725518+0000","last_change":"2023-11-28T04:26:19.725518+0000","last_active":"2023-11-28T04:26:19.725518+0000","last_peered":"2023-11-28T04:26:19.725518+0000","last_clean":"2023-11-28T04:26:19.725518+0000","last_became_active":"2023-11-28T04:26:19.725195+0000","last_became_peered":"2023-11-28T04:26:19.725195+0000","last_unstale":"2023-11-28T04:26:19.725518+0000","last_undegraded":"2023-11-28T04:26:19.725518+0000","last_fullsized":"2023-11-28T04:26:19.725518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715131+0000","last_change":"2023-11-28T04:26:19.355528+0000","last_active":"2023-11-28T04:26:19.715131+0000","last_peered":"2023-11-28T04:26:19.715131+0000","last_clean":"2023-11-28T04:26:19.715131+0000","last_became_active":"2023-11-28T04:26:19.355175+0000","last_became_peered":"2023-11-28T04:26:19.355175+0000","last_unstale":"2023-11-28T04:26:19.715131+0000","last_undegraded":"2023-11-28T04:26:19.715131+0000","last_fullsized":"2023-11-28T04:26:19.715131+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.714579+0000","last_change":"2023-11-28T04:26:19.714579+0000","last_active":"2023-11-28T04:26:19.714579+0000","last_peered":"2023-11-28T04:26:19.714579+0000","last_clean":"2023-11-28T04:26:19.714579+0000","last_became_active":"2023-11-28T04:26:19.714332+0000","last_became_peered":"2023-11-28T04:26:19.714332+0000","last_unstale":"2023-11-28T04:26:19.714579+0000","last_undegraded":"2023-11-28T04:26:19.714579+0000","last_fullsized":"2023-11-28T04:26:19.714579+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718296+0000","last_change":"2023-11-28T04:26:19.350570+0000","last_active":"2023-11-28T04:26:19.718296+0000","last_peered":"2023-11-28T04:26:19.718296+0000","last_clean":"2023-11-28T04:26:19.718296+0000","last_became_active":"2023-11-28T04:26:19.350292+0000","last_became_peered":"2023-11-28T04:26:19.350292+0000","last_unstale":"2023-11-28T04:26:19.718296+0000","last_undegraded":"2023-11-28T04:26:19.718296+0000","last_fullsized":"2023-11-28T04:26:19.718296+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727814+0000","last_change":"2023-11-28T04:26:19.727814+0000","last_active":"2023-11-28T04:26:19.727814+0000","last_peered":"2023-11-28T04:26:19.727814+0000","last_clean":"2023-11-28T04:26:19.727814+0000","last_became_active":"2023-11-28T04:26:19.727547+0000","last_became_peered":"2023-11-28T04:26:19.727547+0000","last_unstale":"2023-11-28T04:26:19.727814+0000","last_undegraded":"2023-11-28T04:26:19.727814+0000","last_fullsized":"2023-11-28T04:26:19.727814+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.363569+0000","last_change":"2023-11-28T04:26:19.358993+0000","last_active":"2023-11-28T04:26:19.363569+0000","last_peered":"2023-11-28T04:26:19.363569+0000","last_clean":"2023-11-28T04:26:19.363569+0000","last_became_active":"2023-11-28T04:26:19.358679+0000","last_became_peered":"2023-11-28T04:26:19.358679+0000","last_unstale":"2023-11-28T04:26:19.363569+0000","last_undegraded":"2023-11-28T04:26:19.363569+0000","last_fullsized":"2023-11-28T04:26:19.363569+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724320+0000","last_change":"2023-11-28T04:26:19.724320+0000","last_active":"2023-11-28T04:26:19.724320+0000","last_peered":"2023-11-28T04:26:19.724320+0000","last_clean":"2023-11-28T04:26:19.724320+0000","last_became_active":"2023-11-28T04:26:19.723931+0000","last_became_peered":"2023-11-28T04:26:19.723931+0000","last_unstale":"2023-11-28T04:26:19.724320+0000","last_undegraded":"2023-11-28T04:26:19.724320+0000","last_fullsized":"2023-11-28T04:26:19.724320+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350554+0000","last_change":"2023-11-28T04:26:19.349597+0000","last_active":"2023-11-28T04:26:19.350554+0000","last_peered":"2023-11-28T04:26:19.350554+0000","last_clean":"2023-11-28T04:26:19.350554+0000","last_became_active":"2023-11-28T04:26:19.349311+0000","last_became_peered":"2023-11-28T04:26:19.349311+0000","last_unstale":"2023-11-28T04:26:19.350554+0000","last_undegraded":"2023-11-28T04:26:19.350554+0000","last_fullsized":"2023-11-28T04:26:19.350554+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723573+0000","last_change":"2023-11-28T04:26:19.723573+0000","last_active":"2023-11-28T04:26:19.723573+0000","last_peered":"2023-11-28T04:26:19.723573+0000","last_clean":"2023-11-28T04:26:19.723573+0000","last_became_active":"2023-11-28T04:26:19.723297+0000","last_became_peered":"2023-11-28T04:26:19.723297+0000","last_unstale":"2023-11-28T04:26:19.723573+0000","last_undegraded":"2023-11-28T04:26:19.723573+0000","last_fullsized":"2023-11-28T04:26:19.723573+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707929+0000","last_change":"2023-11-28T04:26:19.358540+0000","last_active":"2023-11-28T04:26:19.707929+0000","last_peered":"2023-11-28T04:26:19.707929+0000","last_clean":"2023-11-28T04:26:19.707929+0000","last_became_active":"2023-11-28T04:26:19.358286+0000","last_became_peered":"2023-11-28T04:26:19.358286+0000","last_unstale":"2023-11-28T04:26:19.707929+0000","last_undegraded":"2023-11-28T04:26:19.707929+0000","last_fullsized":"2023-11-28T04:26:19.707929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718880+0000","last_change":"2023-11-28T04:26:19.718880+0000","last_active":"2023-11-28T04:26:19.718880+0000","last_peered":"2023-11-28T04:26:19.718880+0000","last_clean":"2023-11-28T04:26:19.718880+0000","last_became_active":"2023-11-28T04:26:19.718612+0000","last_became_peered":"2023-11-28T04:26:19.718612+0000","last_unstale":"2023-11-28T04:26:19.718880+0000","last_undegraded":"2023-11-28T04:26:19.718880+0000","last_fullsized":"2023-11-28T04:26:19.718880+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354642+0000","last_change":"2023-11-28T04:26:19.343708+0000","last_active":"2023-11-28T04:26:19.354642+0000","last_peered":"2023-11-28T04:26:19.354642+0000","last_clean":"2023-11-28T04:26:19.354642+0000","last_became_active":"2023-11-28T04:26:19.343447+0000","last_became_peered":"2023-11-28T04:26:19.343447+0000","last_unstale":"2023-11-28T04:26:19.354642+0000","last_undegraded":"2023-11-28T04:26:19.354642+0000","last_fullsized":"2023-11-28T04:26:19.354642+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728032+0000","last_change":"2023-11-28T04:26:19.728032+0000","last_active":"2023-11-28T04:26:19.728032+0000","last_peered":"2023-11-28T04:26:19.728032+0000","last_clean":"2023-11-28T04:26:19.728032+0000","last_became_active":"2023-11-28T04:26:19.727822+0000","last_became_peered":"2023-11-28T04:26:19.727822+0000","last_unstale":"2023-11-28T04:26:19.728032+0000","last_undegraded":"2023-11-28T04:26:19.728032+0000","last_fullsized":"2023-11-28T04:26:19.728032+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.345887+0000","last_change":"2023-11-28T04:26:19.344644+0000","last_active":"2023-11-28T04:26:19.345887+0000","last_peered":"2023-11-28T04:26:19.345887+0000","last_clean":"2023-11-28T04:26:19.345887+0000","last_became_active":"2023-11-28T04:26:19.344159+0000","last_became_peered":"2023-11-28T04:26:19.344159+0000","last_unstale":"2023-11-28T04:26:19.345887+0000","last_undegraded":"2023-11-28T04:26:19.345887+0000","last_fullsized":"2023-11-28T04:26:19.345887+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718534+0000","last_change":"2023-11-28T04:26:19.718534+0000","last_active":"2023-11-28T04:26:19.718534+0000","last_peered":"2023-11-28T04:26:19.718534+0000","last_clean":"2023-11-28T04:26:19.718534+0000","last_became_active":"2023-11-28T04:26:19.718247+0000","last_became_peered":"2023-11-28T04:26:19.718247+0000","last_unstale":"2023-11-28T04:26:19.718534+0000","last_undegraded":"2023-11-28T04:26:19.718534+0000","last_fullsized":"2023-11-28T04:26:19.718534+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357227+0000","last_change":"2023-11-28T04:26:19.355471+0000","last_active":"2023-11-28T04:26:19.357227+0000","last_peered":"2023-11-28T04:26:19.357227+0000","last_clean":"2023-11-28T04:26:19.357227+0000","last_became_active":"2023-11-28T04:26:19.355186+0000","last_became_peered":"2023-11-28T04:26:19.355186+0000","last_unstale":"2023-11-28T04:26:19.357227+0000","last_undegraded":"2023-11-28T04:26:19.357227+0000","last_fullsized":"2023-11-28T04:26:19.357227+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724246+0000","last_change":"2023-11-28T04:26:19.724246+0000","last_active":"2023-11-28T04:26:19.724246+0000","last_peered":"2023-11-28T04:26:19.724246+0000","last_clean":"2023-11-28T04:26:19.724246+0000","last_became_active":"2023-11-28T04:26:19.723984+0000","last_became_peered":"2023-11-28T04:26:19.723984+0000","last_unstale":"2023-11-28T04:26:19.724246+0000","last_undegraded":"2023-11-28T04:26:19.724246+0000","last_fullsized":"2023-11-28T04:26:19.724246+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721279+0000","last_change":"2023-11-28T04:26:19.347878+0000","last_active":"2023-11-28T04:26:19.721279+0000","last_peered":"2023-11-28T04:26:19.721279+0000","last_clean":"2023-11-28T04:26:19.721279+0000","last_became_active":"2023-11-28T04:26:19.347157+0000","last_became_peered":"2023-11-28T04:26:19.347157+0000","last_unstale":"2023-11-28T04:26:19.721279+0000","last_undegraded":"2023-11-28T04:26:19.721279+0000","last_fullsized":"2023-11-28T04:26:19.721279+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724844+0000","last_change":"2023-11-28T04:26:19.724844+0000","last_active":"2023-11-28T04:26:19.724844+0000","last_peered":"2023-11-28T04:26:19.724844+0000","last_clean":"2023-11-28T04:26:19.724844+0000","last_became_active":"2023-11-28T04:26:19.724587+0000","last_became_peered":"2023-11-28T04:26:19.724587+0000","last_unstale":"2023-11-28T04:26:19.724844+0000","last_undegraded":"2023-11-28T04:26:19.724844+0000","last_fullsized":"2023-11-28T04:26:19.724844+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351553+0000","last_change":"2023-11-28T04:26:19.349269+0000","last_active":"2023-11-28T04:26:19.351553+0000","last_peered":"2023-11-28T04:26:19.351553+0000","last_clean":"2023-11-28T04:26:19.351553+0000","last_became_active":"2023-11-28T04:26:19.349013+0000","last_became_peered":"2023-11-28T04:26:19.349013+0000","last_unstale":"2023-11-28T04:26:19.351553+0000","last_undegraded":"2023-11-28T04:26:19.351553+0000","last_fullsized":"2023-11-28T04:26:19.351553+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723187+0000","last_change":"2023-11-28T04:26:19.723187+0000","last_active":"2023-11-28T04:26:19.723187+0000","last_peered":"2023-11-28T04:26:19.723187+0000","last_clean":"2023-11-28T04:26:19.723187+0000","last_became_active":"2023-11-28T04:26:19.722784+0000","last_became_peered":"2023-11-28T04:26:19.722784+0000","last_unstale":"2023-11-28T04:26:19.723187+0000","last_undegraded":"2023-11-28T04:26:19.723187+0000","last_fullsized":"2023-11-28T04:26:19.723187+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.352811+0000","last_change":"2023-11-28T04:26:19.346261+0000","last_active":"2023-11-28T04:26:19.352811+0000","last_peered":"2023-11-28T04:26:19.352811+0000","last_clean":"2023-11-28T04:26:19.352811+0000","last_became_active":"2023-11-28T04:26:19.345988+0000","last_became_peered":"2023-11-28T04:26:19.345988+0000","last_unstale":"2023-11-28T04:26:19.352811+0000","last_undegraded":"2023-11-28T04:26:19.352811+0000","last_fullsized":"2023-11-28T04:26:19.352811+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726412+0000","last_change":"2023-11-28T04:26:19.726412+0000","last_active":"2023-11-28T04:26:19.726412+0000","last_peered":"2023-11-28T04:26:19.726412+0000","last_clean":"2023-11-28T04:26:19.726412+0000","last_became_active":"2023-11-28T04:26:19.726161+0000","last_became_peered":"2023-11-28T04:26:19.726161+0000","last_unstale":"2023-11-28T04:26:19.726412+0000","last_undegraded":"2023-11-28T04:26:19.726412+0000","last_fullsized":"2023-11-28T04:26:19.726412+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356813+0000","last_change":"2023-11-28T04:26:19.342853+0000","last_active":"2023-11-28T04:26:19.356813+0000","last_peered":"2023-11-28T04:26:19.356813+0000","last_clean":"2023-11-28T04:26:19.356813+0000","last_became_active":"2023-11-28T04:26:19.342586+0000","last_became_peered":"2023-11-28T04:26:19.342586+0000","last_unstale":"2023-11-28T04:26:19.356813+0000","last_undegraded":"2023-11-28T04:26:19.356813+0000","last_fullsized":"2023-11-28T04:26:19.356813+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724406+0000","last_change":"2023-11-28T04:26:19.724406+0000","last_active":"2023-11-28T04:26:19.724406+0000","last_peered":"2023-11-28T04:26:19.724406+0000","last_clean":"2023-11-28T04:26:19.724406+0000","last_became_active":"2023-11-28T04:26:19.724102+0000","last_became_peered":"2023-11-28T04:26:19.724102+0000","last_unstale":"2023-11-28T04:26:19.724406+0000","last_undegraded":"2023-11-28T04:26:19.724406+0000","last_fullsized":"2023-11-28T04:26:19.724406+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354965+0000","last_change":"2023-11-28T04:26:19.352655+0000","last_active":"2023-11-28T04:26:19.354965+0000","last_peered":"2023-11-28T04:26:19.354965+0000","last_clean":"2023-11-28T04:26:19.354965+0000","last_became_active":"2023-11-28T04:26:19.352362+0000","last_became_peered":"2023-11-28T04:26:19.352362+0000","last_unstale":"2023-11-28T04:26:19.354965+0000","last_undegraded":"2023-11-28T04:26:19.354965+0000","last_fullsized":"2023-11-28T04:26:19.354965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726230+0000","last_change":"2023-11-28T04:26:19.726230+0000","last_active":"2023-11-28T04:26:19.726230+0000","last_peered":"2023-11-28T04:26:19.726230+0000","last_clean":"2023-11-28T04:26:19.726230+0000","last_became_active":"2023-11-28T04:26:19.723839+0000","last_became_peered":"2023-11-28T04:26:19.723839+0000","last_unstale":"2023-11-28T04:26:19.726230+0000","last_undegraded":"2023-11-28T04:26:19.726230+0000","last_fullsized":"2023-11-28T04:26:19.726230+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357449+0000","last_change":"2023-11-28T04:26:19.353391+0000","last_active":"2023-11-28T04:26:19.357449+0000","last_peered":"2023-11-28T04:26:19.357449+0000","last_clean":"2023-11-28T04:26:19.357449+0000","last_became_active":"2023-11-28T04:26:19.352964+0000","last_became_peered":"2023-11-28T04:26:19.352964+0000","last_unstale":"2023-11-28T04:26:19.357449+0000","last_undegraded":"2023-11-28T04:26:19.357449+0000","last_fullsized":"2023-11-28T04:26:19.357449+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725246+0000","last_change":"2023-11-28T04:26:19.725246+0000","last_active":"2023-11-28T04:26:19.725246+0000","last_peered":"2023-11-28T04:26:19.725246+0000","last_clean":"2023-11-28T04:26:19.725246+0000","last_became_active":"2023-11-28T04:26:19.724719+0000","last_became_peered":"2023-11-28T04:26:19.724719+0000","last_unstale":"2023-11-28T04:26:19.725246+0000","last_undegraded":"2023-11-28T04:26:19.725246+0000","last_fullsized":"2023-11-28T04:26:19.725246+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353631+0000","last_change":"2023-11-28T04:26:19.352948+0000","last_active":"2023-11-28T04:26:19.353631+0000","last_peered":"2023-11-28T04:26:19.353631+0000","last_clean":"2023-11-28T04:26:19.353631+0000","last_became_active":"2023-11-28T04:26:19.352689+0000","last_became_peered":"2023-11-28T04:26:19.352689+0000","last_unstale":"2023-11-28T04:26:19.353631+0000","last_undegraded":"2023-11-28T04:26:19.353631+0000","last_fullsized":"2023-11-28T04:26:19.353631+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724801+0000","last_change":"2023-11-28T04:26:19.724801+0000","last_active":"2023-11-28T04:26:19.724801+0000","last_peered":"2023-11-28T04:26:19.724801+0000","last_clean":"2023-11-28T04:26:19.724801+0000","last_became_active":"2023-11-28T04:26:19.724489+0000","last_became_peered":"2023-11-28T04:26:19.724489+0000","last_unstale":"2023-11-28T04:26:19.724801+0000","last_undegraded":"2023-11-28T04:26:19.724801+0000","last_fullsized":"2023-11-28T04:26:19.724801+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356997+0000","last_change":"2023-11-28T04:26:19.352441+0000","last_active":"2023-11-28T04:26:19.356997+0000","last_peered":"2023-11-28T04:26:19.356997+0000","last_clean":"2023-11-28T04:26:19.356997+0000","last_became_active":"2023-11-28T04:26:19.351976+0000","last_became_peered":"2023-11-28T04:26:19.351976+0000","last_unstale":"2023-11-28T04:26:19.356997+0000","last_undegraded":"2023-11-28T04:26:19.356997+0000","last_fullsized":"2023-11-28T04:26:19.356997+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723772+0000","last_change":"2023-11-28T04:26:19.723772+0000","last_active":"2023-11-28T04:26:19.723772+0000","last_peered":"2023-11-28T04:26:19.723772+0000","last_clean":"2023-11-28T04:26:19.723772+0000","last_became_active":"2023-11-28T04:26:19.723485+0000","last_became_peered":"2023-11-28T04:26:19.723485+0000","last_unstale":"2023-11-28T04:26:19.723772+0000","last_undegraded":"2023-11-28T04:26:19.723772+0000","last_fullsized":"2023-11-28T04:26:19.723772+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.711496+0000","last_change":"2023-11-28T04:26:19.345691+0000","last_active":"2023-11-28T04:26:19.711496+0000","last_peered":"2023-11-28T04:26:19.711496+0000","last_clean":"2023-11-28T04:26:19.711496+0000","last_became_active":"2023-11-28T04:26:19.345420+0000","last_became_peered":"2023-11-28T04:26:19.345420+0000","last_unstale":"2023-11-28T04:26:19.711496+0000","last_undegraded":"2023-11-28T04:26:19.711496+0000","last_fullsized":"2023-11-28T04:26:19.711496+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.342206+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:26:19.342206+0000","last_peered":"2023-11-28T04:26:19.342206+0000","last_clean":"2023-11-28T04:26:19.342206+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:19.342206+0000","last_undegraded":"2023-11-28T04:26:19.342206+0000","last_fullsized":"2023-11-28T04:26:19.342206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725036+0000","last_change":"2023-11-28T04:26:19.725036+0000","last_active":"2023-11-28T04:26:19.725036+0000","last_peered":"2023-11-28T04:26:19.725036+0000","last_clean":"2023-11-28T04:26:19.725036+0000","last_became_active":"2023-11-28T04:26:19.724416+0000","last_became_peered":"2023-11-28T04:26:19.724416+0000","last_unstale":"2023-11-28T04:26:19.725036+0000","last_undegraded":"2023-11-28T04:26:19.725036+0000","last_fullsized":"2023-11-28T04:26:19.725036+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350961+0000","last_change":"2023-11-28T04:26:19.344397+0000","last_active":"2023-11-28T04:26:19.350961+0000","last_peered":"2023-11-28T04:26:19.350961+0000","last_clean":"2023-11-28T04:26:19.350961+0000","last_became_active":"2023-11-28T04:26:19.344118+0000","last_became_peered":"2023-11-28T04:26:19.344118+0000","last_unstale":"2023-11-28T04:26:19.350961+0000","last_undegraded":"2023-11-28T04:26:19.350961+0000","last_fullsized":"2023-11-28T04:26:19.350961+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340702+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:19.340702+0000","last_peered":"2023-11-28T04:26:19.340702+0000","last_clean":"2023-11-28T04:26:19.340702+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:19.340702+0000","last_undegraded":"2023-11-28T04:26:19.340702+0000","last_fullsized":"2023-11-28T04:26:19.340702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723615+0000","last_change":"2023-11-28T04:26:19.723615+0000","last_active":"2023-11-28T04:26:19.723615+0000","last_peered":"2023-11-28T04:26:19.723615+0000","last_clean":"2023-11-28T04:26:19.723615+0000","last_became_active":"2023-11-28T04:26:19.723391+0000","last_became_peered":"2023-11-28T04:26:19.723391+0000","last_unstale":"2023-11-28T04:26:19.723615+0000","last_undegraded":"2023-11-28T04:26:19.723615+0000","last_fullsized":"2023-11-28T04:26:19.723615+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355803+0000","last_change":"2023-11-28T04:26:19.354540+0000","last_active":"2023-11-28T04:26:19.355803+0000","last_peered":"2023-11-28T04:26:19.355803+0000","last_clean":"2023-11-28T04:26:19.355803+0000","last_became_active":"2023-11-28T04:26:19.354241+0000","last_became_peered":"2023-11-28T04:26:19.354241+0000","last_unstale":"2023-11-28T04:26:19.355803+0000","last_undegraded":"2023-11-28T04:26:19.355803+0000","last_fullsized":"2023-11-28T04:26:19.355803+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.333827+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:26:19.333827+0000","last_peered":"2023-11-28T04:26:19.333827+0000","last_clean":"2023-11-28T04:26:19.333827+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:19.333827+0000","last_undegraded":"2023-11-28T04:26:19.333827+0000","last_fullsized":"2023-11-28T04:26:19.333827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728914+0000","last_change":"2023-11-28T04:26:19.728914+0000","last_active":"2023-11-28T04:26:19.728914+0000","last_peered":"2023-11-28T04:26:19.728914+0000","last_clean":"2023-11-28T04:26:19.728914+0000","last_became_active":"2023-11-28T04:26:19.728648+0000","last_became_peered":"2023-11-28T04:26:19.728648+0000","last_unstale":"2023-11-28T04:26:19.728914+0000","last_undegraded":"2023-11-28T04:26:19.728914+0000","last_fullsized":"2023-11-28T04:26:19.728914+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355722+0000","last_change":"2023-11-28T04:26:19.351948+0000","last_active":"2023-11-28T04:26:19.355722+0000","last_peered":"2023-11-28T04:26:19.355722+0000","last_clean":"2023-11-28T04:26:19.355722+0000","last_became_active":"2023-11-28T04:26:19.351696+0000","last_became_peered":"2023-11-28T04:26:19.351696+0000","last_unstale":"2023-11-28T04:26:19.355722+0000","last_undegraded":"2023-11-28T04:26:19.355722+0000","last_fullsized":"2023-11-28T04:26:19.355722+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.341073+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:19.341073+0000","last_peered":"2023-11-28T04:26:19.341073+0000","last_clean":"2023-11-28T04:26:19.341073+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:19.341073+0000","last_undegraded":"2023-11-28T04:26:19.341073+0000","last_fullsized":"2023-11-28T04:26:19.341073+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":166,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340715+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:19.340715+0000","last_peered":"2023-11-28T04:26:19.340715+0000","last_clean":"2023-11-28T04:26:19.340715+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:19.340715+0000","last_undegraded":"2023-11-28T04:26:19.340715+0000","last_fullsized":"2023-11-28T04:26:19.340715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722010+0000","last_change":"2023-11-28T04:26:19.722010+0000","last_active":"2023-11-28T04:26:19.722010+0000","last_peered":"2023-11-28T04:26:19.722010+0000","last_clean":"2023-11-28T04:26:19.722010+0000","last_became_active":"2023-11-28T04:26:19.721747+0000","last_became_peered":"2023-11-28T04:26:19.721747+0000","last_unstale":"2023-11-28T04:26:19.722010+0000","last_undegraded":"2023-11-28T04:26:19.722010+0000","last_fullsized":"2023-11-28T04:26:19.722010+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356215+0000","last_change":"2023-11-28T04:26:19.347219+0000","last_active":"2023-11-28T04:26:19.356215+0000","last_peered":"2023-11-28T04:26:19.356215+0000","last_clean":"2023-11-28T04:26:19.356215+0000","last_became_active":"2023-11-28T04:26:19.346928+0000","last_became_peered":"2023-11-28T04:26:19.346928+0000","last_unstale":"2023-11-28T04:26:19.356215+0000","last_undegraded":"2023-11-28T04:26:19.356215+0000","last_fullsized":"2023-11-28T04:26:19.356215+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340797+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:19.340797+0000","last_peered":"2023-11-28T04:26:19.340797+0000","last_clean":"2023-11-28T04:26:19.340797+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:19.340797+0000","last_undegraded":"2023-11-28T04:26:19.340797+0000","last_fullsized":"2023-11-28T04:26:19.340797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.717441+0000","last_change":"2023-11-28T04:26:19.717441+0000","last_active":"2023-11-28T04:26:19.717441+0000","last_peered":"2023-11-28T04:26:19.717441+0000","last_clean":"2023-11-28T04:26:19.717441+0000","last_became_active":"2023-11-28T04:26:19.717192+0000","last_became_peered":"2023-11-28T04:26:19.717192+0000","last_unstale":"2023-11-28T04:26:19.717441+0000","last_undegraded":"2023-11-28T04:26:19.717441+0000","last_fullsized":"2023-11-28T04:26:19.717441+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707980+0000","last_change":"2023-11-28T04:26:19.350127+0000","last_active":"2023-11-28T04:26:19.707980+0000","last_peered":"2023-11-28T04:26:19.707980+0000","last_clean":"2023-11-28T04:26:19.707980+0000","last_became_active":"2023-11-28T04:26:19.349834+0000","last_became_peered":"2023-11-28T04:26:19.349834+0000","last_unstale":"2023-11-28T04:26:19.707980+0000","last_undegraded":"2023-11-28T04:26:19.707980+0000","last_fullsized":"2023-11-28T04:26:19.707980+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.341455+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:19.341455+0000","last_peered":"2023-11-28T04:26:19.341455+0000","last_clean":"2023-11-28T04:26:19.341455+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:19.341455+0000","last_undegraded":"2023-11-28T04:26:19.341455+0000","last_fullsized":"2023-11-28T04:26:19.341455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727306+0000","last_change":"2023-11-28T04:26:19.727306+0000","last_active":"2023-11-28T04:26:19.727306+0000","last_peered":"2023-11-28T04:26:19.727306+0000","last_clean":"2023-11-28T04:26:19.727306+0000","last_became_active":"2023-11-28T04:26:19.725116+0000","last_became_peered":"2023-11-28T04:26:19.725116+0000","last_unstale":"2023-11-28T04:26:19.727306+0000","last_undegraded":"2023-11-28T04:26:19.727306+0000","last_fullsized":"2023-11-28T04:26:19.727306+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725432+0000","last_change":"2023-11-28T04:26:19.350360+0000","last_active":"2023-11-28T04:26:19.725432+0000","last_peered":"2023-11-28T04:26:19.725432+0000","last_clean":"2023-11-28T04:26:19.725432+0000","last_became_active":"2023-11-28T04:26:19.350083+0000","last_became_peered":"2023-11-28T04:26:19.350083+0000","last_unstale":"2023-11-28T04:26:19.725432+0000","last_undegraded":"2023-11-28T04:26:19.725432+0000","last_fullsized":"2023-11-28T04:26:19.725432+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":170,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.344753+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:19.344753+0000","last_peered":"2023-11-28T04:26:19.344753+0000","last_clean":"2023-11-28T04:26:19.344753+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:19.344753+0000","last_undegraded":"2023-11-28T04:26:19.344753+0000","last_fullsized":"2023-11-28T04:26:19.344753+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726220+0000","last_change":"2023-11-28T04:26:19.726220+0000","last_active":"2023-11-28T04:26:19.726220+0000","last_peered":"2023-11-28T04:26:19.726220+0000","last_clean":"2023-11-28T04:26:19.726220+0000","last_became_active":"2023-11-28T04:26:19.725925+0000","last_became_peered":"2023-11-28T04:26:19.725925+0000","last_unstale":"2023-11-28T04:26:19.726220+0000","last_undegraded":"2023-11-28T04:26:19.726220+0000","last_fullsized":"2023-11-28T04:26:19.726220+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350917+0000","last_change":"2023-11-28T04:26:19.345898+0000","last_active":"2023-11-28T04:26:19.350917+0000","last_peered":"2023-11-28T04:26:19.350917+0000","last_clean":"2023-11-28T04:26:19.350917+0000","last_became_active":"2023-11-28T04:26:19.345632+0000","last_became_peered":"2023-11-28T04:26:19.345632+0000","last_unstale":"2023-11-28T04:26:19.350917+0000","last_undegraded":"2023-11-28T04:26:19.350917+0000","last_fullsized":"2023-11-28T04:26:19.350917+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":187,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.346829+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:19.346829+0000","last_peered":"2023-11-28T04:26:19.346829+0000","last_clean":"2023-11-28T04:26:19.346829+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:19.346829+0000","last_undegraded":"2023-11-28T04:26:19.346829+0000","last_fullsized":"2023-11-28T04:26:19.346829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.438},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75,"5min":0.80400000000000005,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":0.79400000000000004,"15min":0.92800000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61799999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.67100000000000004,"5min":0.77700000000000002,"15min":0.89000000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67800000000000005,"5min":0.79500000000000004,"15min":0.83599999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.65900000000000003,"5min":0.81999999999999995,"15min":0.90300000000000002},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75800000000000001,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.68500000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68200000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575965,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.754,"15min":0.77600000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.499}]},{"osd":1,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.67200000000000004,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.84999999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.438},"min":{"1min":0.28100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.78800000000000003},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.80700000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.496,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.82199999999999995},"last":0.378}]},{"osd":7,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.80100000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.82999999999999996},"last":0.35099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575964,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.255},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.433},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.38100000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.48599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575965,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.28000000000000003,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.51000000000000001,"5min":0.56799999999999995,"15min":0.97799999999999998},"last":2.25},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.95099999999999996},"last":2.1669999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.28999999999999998,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":0.54800000000000004,"5min":0.65800000000000003,"15min":0.85999999999999999},"last":2.0129999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.86799999999999999},"last":1.929}]},{"osd":3,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.64600000000000002},"last":2.379},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.30199999999999999},"max":{"1min":0.628,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":2.1520000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.49299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92400000000000004},"last":2.2709999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.038},"last":2.0800000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53000000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":2.2890000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.878,"15min":1.014},"last":2.0510000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.49199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.95099999999999996,"15min":1.0449999999999999},"last":2.1360000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.51900000000000002,"5min":0.433,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.036},"last":2.4460000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":1.0189999999999999},"last":2.1120000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.54300000000000004,"5min":0.47499999999999998,"15min":0.40300000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0669999999999999},"last":2.4100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.374,"15min":0.378},"min":{"1min":0.30399999999999999,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30599999999999999},"max":{"1min":0.626,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.377,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.25900000000000001},"max":{"1min":0.51900000000000002,"5min":0.57599999999999996,"15min":0.622},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.29199999999999998,"5min":0.28199999999999997,"15min":0.26200000000000001},"max":{"1min":0.52500000000000002,"5min":0.754,"15min":0.754},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.29999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.65300000000000002},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.84599999999999997,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.96199999999999997},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.016},"last":0.64100000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.443},"max":{"1min":0.89500000000000002,"5min":0.93999999999999995,"15min":0.99099999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.625}]},{"osd":7,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.58299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36499999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":0.432,"5min":0.57399999999999995,"15min":0.58899999999999997},"last":0.376},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.66500000000000004},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48099999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53900000000000003,"5min":0.60099999999999998,"15min":0.746},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.68000000000000005},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66400000000000003,"5min":0.92500000000000004,"15min":0.95999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.42699999999999999},"max":{"1min":0.77200000000000002,"5min":0.92100000000000004,"15min":1.0409999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70199999999999996,"15min":0.67400000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.745}]},{"osd":6,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":1.028,"15min":1.028},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.78600000000000003,"5min":1.008,"15min":1.008},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.40799999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.57199999999999995}]},{"osd":1,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.372},"max":{"1min":0.65200000000000002,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.38400000000000001,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.66800000000000004}]},{"osd":2,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.375},"max":{"1min":0.67400000000000004,"5min":0.78100000000000003,"15min":0.79900000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.40400000000000003},"max":{"1min":0.73999999999999999,"5min":0.83099999999999996,"15min":0.86399999999999999},"last":0.69499999999999995}]},{"osd":3,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72599999999999998,"5min":0.81200000000000006,"15min":0.83599999999999997},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60399999999999998,"5min":0.70099999999999996,"15min":0.88200000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.318},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.435}]},{"osd":6,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.317,"5min":0.317,"15min":0.30599999999999999},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":0.86699999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.78100000000000003,"15min":0.92100000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.90300000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76000000000000001,"5min":0.80600000000000005,"15min":0.85799999999999998},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:22.689 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:22.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.834+0000 7f55e1552700 1 Processor -- start 2023-11-28T04:26:22.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.834+0000 7f55e1552700 1 -- start start 2023-11-28T04:26:22.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e1552700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55dc157a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e1552700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c1750 0x7f55dc0c1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.847 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e1552700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55dc0c12c0 con 0x7f55dc157660 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e1552700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55dc0c06b0 con 0x7f55dc0c1750 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e1552700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c0990 con 0x7f55dc0c0180 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55dbfff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f55dc157660 0x7f55dc1589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:48310/0 (socket says 172.21.15.73:48310) 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55dbfff700 1 -- 172.21.15.73:0/809042410 learned_addr learned my addr 172.21.15.73:0/809042410 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55e0d51700 1 --2- 172.21.15.73:0/809042410 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c1750 0x7f55dc0c1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.835+0000 7f55db7fe700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55dc157a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.848 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1400709168 0 0) 0x7f55dc0c12c0 con 0x7f55dc157660 2023-11-28T04:26:22.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55c40033f0 con 0x7f55dc157660 2023-11-28T04:26:22.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1843844293 0 0) 0x7f55c40033f0 con 0x7f55dc157660 2023-11-28T04:26:22.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 msgr2=0x7f55dc157a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.849 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55dc157a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c1750 msgr2=0x7f55dc0c1ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 --2- 172.21.15.73:0/809042410 >> v2:172.21.15.150:3300/0 conn(0x7f55dc0c1750 0x7f55dc0c1ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc0c0c70 con 0x7f55dc157660 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f55cc001470 con 0x7f55dc157660 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 >> v1:172.21.15.73:6789/0 conn(0x7f55dc157660 legacy=0x7f55dc1589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.850 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c40038e0 0x7f55c4003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55c4004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55c4005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c06b0 con 0x7f55c4004690 2023-11-28T04:26:22.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c0990 con 0x7f55c40038e0 2023-11-28T04:26:22.851 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55c400b8e0 con 0x7f55dc0c0180 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55c4004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55dbfff700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c40038e0 0x7f55c4003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55e0d51700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55c4005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 -- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 msgr2=0x7f55c4005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc0c0180 0x7f55c4005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 -- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c40038e0 msgr2=0x7f55c4003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:22.852 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55c40038e0 0x7f55c4003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 -- 172.21.15.73:0/809042410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55c40033f0 con 0x7f55c4004690 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.836+0000 7f55db7fe700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55c4004b10 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 rx=0x7f55dc0c05e0 tx=0x7f55d0009060).ready entity=mon.0 client_cookie=44144c3310dd6c2c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d0009e90 con 0x7f55c4004690 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55d0009430 con 0x7f55c4004690 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55daffd700 1 -- 172.21.15.73:0/809042410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55d000e850 con 0x7f55c4004690 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 -- 172.21.15.73:0/809042410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 msgr2=0x7f55c4004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.853 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 --2- 172.21.15.73:0/809042410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55c4004b10 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 rx=0x7f55dc0c05e0 tx=0x7f55d0009060).stop 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e0d51700 1 -- 172.21.15.73:0/809042410 reap_dead start 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 -- 172.21.15.73:0/809042410 shutdown_connections 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 -- 172.21.15.73:0/809042410 >> 172.21.15.73:0/809042410 conn(0x7f55dc0cb000 msgr2=0x7f55dc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 -- 172.21.15.73:0/809042410 shutdown_connections 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.837+0000 7f55e1552700 1 -- 172.21.15.73:0/809042410 wait complete. 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.838+0000 7f55e1552700 1 Processor -- start 2023-11-28T04:26:22.854 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.838+0000 7f55e1552700 1 -- start start 2023-11-28T04:26:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 0x7f55dc152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc1531c0 0x7f55dc167900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc1504e0 con 0x7f55c4004690 2023-11-28T04:26:22.855 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc0c3cc0 con 0x7f55dc1531c0 2023-11-28T04:26:22.862 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e1552700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55dc153640 con 0x7f55dc153b40 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55dbfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39068/0 (socket says 172.21.15.73:39068) 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55dbfff700 1 -- 172.21.15.73:0/786592186 learned_addr learned my addr 172.21.15.73:0/786592186 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55e0d51700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc1531c0 0x7f55dc167900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.839+0000 7f55db7fe700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 0x7f55dc152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 -- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 msgr2=0x7f55dc152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 0x7f55dc152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 -- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc1531c0 msgr2=0x7f55dc167900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc1531c0 0x7f55dc167900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:22.863 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 -- 172.21.15.73:0/786592186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55dc1685f0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55db7fe700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 0x7f55dc152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55dbfff700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 rx=0x7f55cc004730 tx=0x7f55cc001e90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0114c0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55cc011620 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55dc0c3670 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55cc0119a0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.840+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55dc167df0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.841+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55dc06c7c0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.841+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f55cc01d3b0 con 0x7f55c4004690 2023-11-28T04:26:22.864 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.841+0000 7f55c3fff700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 0x7f55c8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:22.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.841+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f55cc0799a0 con 0x7f55c4004690 2023-11-28T04:26:22.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.841+0000 7f55db7fe700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 0x7f55c8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:22.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.842+0000 7f55db7fe700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 0x7f55c8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f55dc0aa130 tx=0x7f55d001a000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:22.865 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:22.846+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f55cc04acc0 con 0x7f55c4004690 2023-11-28T04:26:23.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.007+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f55dc0c34f0 con 0x7f55c805f150 2023-11-28T04:26:23.018 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.017+0000 7f55c3fff700 1 -- 172.21.15.73:0/786592186 <== mgr.9807 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+362538 (secure 0 0 0) 0x7f55d000e640 con 0x7f55c805f150 2023-11-28T04:26:23.022 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 msgr2=0x7f55c8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 0x7f55c8061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f55dc0aa130 tx=0x7f55d001a000).stop 2023-11-28T04:26:23.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 msgr2=0x7f55dc14fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 rx=0x7f55cc004730 tx=0x7f55cc001e90).stop 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 shutdown_connections 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f55c805f150 0x7f55c8061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f55dc1531c0 0x7f55dc167900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f55c4004690 0x7f55dc14fff0 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 --2- 172.21.15.73:0/786592186 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f55dc153b40 0x7f55dc152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 >> 172.21.15.73:0/786592186 conn(0x7f55dc0cb000 msgr2=0x7f55dc0cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 shutdown_connections 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.020+0000 7f55e1552700 1 -- 172.21.15.73:0/786592186 wait complete. 2023-11-28T04:26:23.026 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:23.034 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:26:22.747406+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":94,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2458704,"kb_used_data":20496,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752516016,"statfs":{"total":773094113280,"available":770576400384,"internally_reserved":0,"allocated":20987904,"data_stored":10600724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-27323,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":-75,"num_write_kb":-57,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"3.594922"},"pg_stats":[{"pgid":"26.3c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719068+0000","last_change":"2023-11-28T04:26:19.719068+0000","last_active":"2023-11-28T04:26:19.719068+0000","last_peered":"2023-11-28T04:26:19.719068+0000","last_clean":"2023-11-28T04:26:19.719068+0000","last_became_active":"2023-11-28T04:26:19.718797+0000","last_became_peered":"2023-11-28T04:26:19.718797+0000","last_unstale":"2023-11-28T04:26:19.719068+0000","last_undegraded":"2023-11-28T04:26:19.719068+0000","last_fullsized":"2023-11-28T04:26:19.719068+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3f","version":"211'3","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724230+0000","last_change":"2023-11-28T04:26:19.356666+0000","last_active":"2023-11-28T04:26:19.724230+0000","last_peered":"2023-11-28T04:26:19.724230+0000","last_clean":"2023-11-28T04:26:19.724230+0000","last_became_active":"2023-11-28T04:26:19.356013+0000","last_became_peered":"2023-11-28T04:26:19.356013+0000","last_unstale":"2023-11-28T04:26:19.724230+0000","last_undegraded":"2023-11-28T04:26:19.724230+0000","last_fullsized":"2023-11-28T04:26:19.724230+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.3d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722320+0000","last_change":"2023-11-28T04:26:19.722320+0000","last_active":"2023-11-28T04:26:19.722320+0000","last_peered":"2023-11-28T04:26:19.722320+0000","last_clean":"2023-11-28T04:26:19.722320+0000","last_became_active":"2023-11-28T04:26:19.721980+0000","last_became_peered":"2023-11-28T04:26:19.721980+0000","last_unstale":"2023-11-28T04:26:19.722320+0000","last_undegraded":"2023-11-28T04:26:19.722320+0000","last_fullsized":"2023-11-28T04:26:19.722320+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.363552+0000","last_change":"2023-11-28T04:26:19.358959+0000","last_active":"2023-11-28T04:26:19.363552+0000","last_peered":"2023-11-28T04:26:19.363552+0000","last_clean":"2023-11-28T04:26:19.363552+0000","last_became_active":"2023-11-28T04:26:19.358727+0000","last_became_peered":"2023-11-28T04:26:19.358727+0000","last_unstale":"2023-11-28T04:26:19.363552+0000","last_undegraded":"2023-11-28T04:26:19.363552+0000","last_fullsized":"2023-11-28T04:26:19.363552+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730141+0000","last_change":"2023-11-28T04:26:19.730141+0000","last_active":"2023-11-28T04:26:19.730141+0000","last_peered":"2023-11-28T04:26:19.730141+0000","last_clean":"2023-11-28T04:26:19.730141+0000","last_became_active":"2023-11-28T04:26:19.729806+0000","last_became_peered":"2023-11-28T04:26:19.729806+0000","last_unstale":"2023-11-28T04:26:19.730141+0000","last_undegraded":"2023-11-28T04:26:19.730141+0000","last_fullsized":"2023-11-28T04:26:19.730141+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351318+0000","last_change":"2023-11-28T04:26:19.348496+0000","last_active":"2023-11-28T04:26:19.351318+0000","last_peered":"2023-11-28T04:26:19.351318+0000","last_clean":"2023-11-28T04:26:19.351318+0000","last_became_active":"2023-11-28T04:26:19.346230+0000","last_became_peered":"2023-11-28T04:26:19.346230+0000","last_unstale":"2023-11-28T04:26:19.351318+0000","last_undegraded":"2023-11-28T04:26:19.351318+0000","last_fullsized":"2023-11-28T04:26:19.351318+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725688+0000","last_change":"2023-11-28T04:26:19.725688+0000","last_active":"2023-11-28T04:26:19.725688+0000","last_peered":"2023-11-28T04:26:19.725688+0000","last_clean":"2023-11-28T04:26:19.725688+0000","last_became_active":"2023-11-28T04:26:19.725405+0000","last_became_peered":"2023-11-28T04:26:19.725405+0000","last_unstale":"2023-11-28T04:26:19.725688+0000","last_undegraded":"2023-11-28T04:26:19.725688+0000","last_fullsized":"2023-11-28T04:26:19.725688+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722930+0000","last_change":"2023-11-28T04:26:19.360551+0000","last_active":"2023-11-28T04:26:19.722930+0000","last_peered":"2023-11-28T04:26:19.722930+0000","last_clean":"2023-11-28T04:26:19.722930+0000","last_became_active":"2023-11-28T04:26:19.360282+0000","last_became_peered":"2023-11-28T04:26:19.360282+0000","last_unstale":"2023-11-28T04:26:19.722930+0000","last_undegraded":"2023-11-28T04:26:19.722930+0000","last_fullsized":"2023-11-28T04:26:19.722930+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.38","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.729724+0000","last_change":"2023-11-28T04:26:19.729724+0000","last_active":"2023-11-28T04:26:19.729724+0000","last_peered":"2023-11-28T04:26:19.729724+0000","last_clean":"2023-11-28T04:26:19.729724+0000","last_became_active":"2023-11-28T04:26:19.729445+0000","last_became_peered":"2023-11-28T04:26:19.729445+0000","last_unstale":"2023-11-28T04:26:19.729724+0000","last_undegraded":"2023-11-28T04:26:19.729724+0000","last_fullsized":"2023-11-28T04:26:19.729724+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.702799+0000","last_change":"2023-11-28T04:26:19.346844+0000","last_active":"2023-11-28T04:26:19.702799+0000","last_peered":"2023-11-28T04:26:19.702799+0000","last_clean":"2023-11-28T04:26:19.702799+0000","last_became_active":"2023-11-28T04:26:19.346569+0000","last_became_peered":"2023-11-28T04:26:19.346569+0000","last_unstale":"2023-11-28T04:26:19.702799+0000","last_undegraded":"2023-11-28T04:26:19.702799+0000","last_fullsized":"2023-11-28T04:26:19.702799+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.39","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.720105+0000","last_change":"2023-11-28T04:26:19.720105+0000","last_active":"2023-11-28T04:26:19.720105+0000","last_peered":"2023-11-28T04:26:19.720105+0000","last_clean":"2023-11-28T04:26:19.720105+0000","last_became_active":"2023-11-28T04:26:19.719393+0000","last_became_peered":"2023-11-28T04:26:19.719393+0000","last_unstale":"2023-11-28T04:26:19.720105+0000","last_undegraded":"2023-11-28T04:26:19.720105+0000","last_fullsized":"2023-11-28T04:26:19.720105+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715687+0000","last_change":"2023-11-28T04:26:19.349820+0000","last_active":"2023-11-28T04:26:19.715687+0000","last_peered":"2023-11-28T04:26:19.715687+0000","last_clean":"2023-11-28T04:26:19.715687+0000","last_became_active":"2023-11-28T04:26:19.348974+0000","last_became_peered":"2023-11-28T04:26:19.348974+0000","last_unstale":"2023-11-28T04:26:19.715687+0000","last_undegraded":"2023-11-28T04:26:19.715687+0000","last_fullsized":"2023-11-28T04:26:19.715687+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.3a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725736+0000","last_change":"2023-11-28T04:26:19.725736+0000","last_active":"2023-11-28T04:26:19.725736+0000","last_peered":"2023-11-28T04:26:19.725736+0000","last_clean":"2023-11-28T04:26:19.725736+0000","last_became_active":"2023-11-28T04:26:19.725393+0000","last_became_peered":"2023-11-28T04:26:19.725393+0000","last_unstale":"2023-11-28T04:26:19.725736+0000","last_undegraded":"2023-11-28T04:26:19.725736+0000","last_fullsized":"2023-11-28T04:26:19.725736+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353095+0000","last_change":"2023-11-28T04:26:19.350280+0000","last_active":"2023-11-28T04:26:19.353095+0000","last_peered":"2023-11-28T04:26:19.353095+0000","last_clean":"2023-11-28T04:26:19.353095+0000","last_became_active":"2023-11-28T04:26:19.349991+0000","last_became_peered":"2023-11-28T04:26:19.349991+0000","last_unstale":"2023-11-28T04:26:19.353095+0000","last_undegraded":"2023-11-28T04:26:19.353095+0000","last_fullsized":"2023-11-28T04:26:19.353095+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.3b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727023+0000","last_change":"2023-11-28T04:26:19.727023+0000","last_active":"2023-11-28T04:26:19.727023+0000","last_peered":"2023-11-28T04:26:19.727023+0000","last_clean":"2023-11-28T04:26:19.727023+0000","last_became_active":"2023-11-28T04:26:19.726763+0000","last_became_peered":"2023-11-28T04:26:19.726763+0000","last_unstale":"2023-11-28T04:26:19.727023+0000","last_undegraded":"2023-11-28T04:26:19.727023+0000","last_fullsized":"2023-11-28T04:26:19.727023+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355505+0000","last_change":"2023-11-28T04:26:19.351840+0000","last_active":"2023-11-28T04:26:19.355505+0000","last_peered":"2023-11-28T04:26:19.355505+0000","last_clean":"2023-11-28T04:26:19.355505+0000","last_became_active":"2023-11-28T04:26:19.351394+0000","last_became_peered":"2023-11-28T04:26:19.351394+0000","last_unstale":"2023-11-28T04:26:19.355505+0000","last_undegraded":"2023-11-28T04:26:19.355505+0000","last_fullsized":"2023-11-28T04:26:19.355505+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.34","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721107+0000","last_change":"2023-11-28T04:26:19.721107+0000","last_active":"2023-11-28T04:26:19.721107+0000","last_peered":"2023-11-28T04:26:19.721107+0000","last_clean":"2023-11-28T04:26:19.721107+0000","last_became_active":"2023-11-28T04:26:19.720834+0000","last_became_peered":"2023-11-28T04:26:19.720834+0000","last_unstale":"2023-11-28T04:26:19.721107+0000","last_undegraded":"2023-11-28T04:26:19.721107+0000","last_fullsized":"2023-11-28T04:26:19.721107+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353178+0000","last_change":"2023-11-28T04:26:19.351910+0000","last_active":"2023-11-28T04:26:19.353178+0000","last_peered":"2023-11-28T04:26:19.353178+0000","last_clean":"2023-11-28T04:26:19.353178+0000","last_became_active":"2023-11-28T04:26:19.351518+0000","last_became_peered":"2023-11-28T04:26:19.351518+0000","last_unstale":"2023-11-28T04:26:19.353178+0000","last_undegraded":"2023-11-28T04:26:19.353178+0000","last_fullsized":"2023-11-28T04:26:19.353178+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.35","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723933+0000","last_change":"2023-11-28T04:26:19.723933+0000","last_active":"2023-11-28T04:26:19.723933+0000","last_peered":"2023-11-28T04:26:19.723933+0000","last_clean":"2023-11-28T04:26:19.723933+0000","last_became_active":"2023-11-28T04:26:19.723666+0000","last_became_peered":"2023-11-28T04:26:19.723666+0000","last_unstale":"2023-11-28T04:26:19.723933+0000","last_undegraded":"2023-11-28T04:26:19.723933+0000","last_fullsized":"2023-11-28T04:26:19.723933+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355398+0000","last_change":"2023-11-28T04:26:19.350321+0000","last_active":"2023-11-28T04:26:19.355398+0000","last_peered":"2023-11-28T04:26:19.355398+0000","last_clean":"2023-11-28T04:26:19.355398+0000","last_became_active":"2023-11-28T04:26:19.350037+0000","last_became_peered":"2023-11-28T04:26:19.350037+0000","last_unstale":"2023-11-28T04:26:19.355398+0000","last_undegraded":"2023-11-28T04:26:19.355398+0000","last_fullsized":"2023-11-28T04:26:19.355398+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.36","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715712+0000","last_change":"2023-11-28T04:26:19.715712+0000","last_active":"2023-11-28T04:26:19.715712+0000","last_peered":"2023-11-28T04:26:19.715712+0000","last_clean":"2023-11-28T04:26:19.715712+0000","last_became_active":"2023-11-28T04:26:19.715462+0000","last_became_peered":"2023-11-28T04:26:19.715462+0000","last_unstale":"2023-11-28T04:26:19.715712+0000","last_undegraded":"2023-11-28T04:26:19.715712+0000","last_fullsized":"2023-11-28T04:26:19.715712+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351524+0000","last_change":"2023-11-28T04:26:19.348176+0000","last_active":"2023-11-28T04:26:19.351524+0000","last_peered":"2023-11-28T04:26:19.351524+0000","last_clean":"2023-11-28T04:26:19.351524+0000","last_became_active":"2023-11-28T04:26:19.347459+0000","last_became_peered":"2023-11-28T04:26:19.347459+0000","last_unstale":"2023-11-28T04:26:19.351524+0000","last_undegraded":"2023-11-28T04:26:19.351524+0000","last_fullsized":"2023-11-28T04:26:19.351524+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.37","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724485+0000","last_change":"2023-11-28T04:26:19.724485+0000","last_active":"2023-11-28T04:26:19.724485+0000","last_peered":"2023-11-28T04:26:19.724485+0000","last_clean":"2023-11-28T04:26:19.724485+0000","last_became_active":"2023-11-28T04:26:19.723996+0000","last_became_peered":"2023-11-28T04:26:19.723996+0000","last_unstale":"2023-11-28T04:26:19.724485+0000","last_undegraded":"2023-11-28T04:26:19.724485+0000","last_fullsized":"2023-11-28T04:26:19.724485+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.34","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728784+0000","last_change":"2023-11-28T04:26:19.354583+0000","last_active":"2023-11-28T04:26:19.728784+0000","last_peered":"2023-11-28T04:26:19.728784+0000","last_clean":"2023-11-28T04:26:19.728784+0000","last_became_active":"2023-11-28T04:26:19.354331+0000","last_became_peered":"2023-11-28T04:26:19.354331+0000","last_unstale":"2023-11-28T04:26:19.728784+0000","last_undegraded":"2023-11-28T04:26:19.728784+0000","last_fullsized":"2023-11-28T04:26:19.728784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":892,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.30","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726720+0000","last_change":"2023-11-28T04:26:19.726720+0000","last_active":"2023-11-28T04:26:19.726720+0000","last_peered":"2023-11-28T04:26:19.726720+0000","last_clean":"2023-11-28T04:26:19.726720+0000","last_became_active":"2023-11-28T04:26:19.726451+0000","last_became_peered":"2023-11-28T04:26:19.726451+0000","last_unstale":"2023-11-28T04:26:19.726720+0000","last_undegraded":"2023-11-28T04:26:19.726720+0000","last_fullsized":"2023-11-28T04:26:19.726720+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.33","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718280+0000","last_change":"2023-11-28T04:26:19.351917+0000","last_active":"2023-11-28T04:26:19.718280+0000","last_peered":"2023-11-28T04:26:19.718280+0000","last_clean":"2023-11-28T04:26:19.718280+0000","last_became_active":"2023-11-28T04:26:19.351626+0000","last_became_peered":"2023-11-28T04:26:19.351626+0000","last_unstale":"2023-11-28T04:26:19.718280+0000","last_undegraded":"2023-11-28T04:26:19.718280+0000","last_fullsized":"2023-11-28T04:26:19.718280+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.31","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724077+0000","last_change":"2023-11-28T04:26:19.724077+0000","last_active":"2023-11-28T04:26:19.724077+0000","last_peered":"2023-11-28T04:26:19.724077+0000","last_clean":"2023-11-28T04:26:19.724077+0000","last_became_active":"2023-11-28T04:26:19.723799+0000","last_became_peered":"2023-11-28T04:26:19.723799+0000","last_unstale":"2023-11-28T04:26:19.724077+0000","last_undegraded":"2023-11-28T04:26:19.724077+0000","last_fullsized":"2023-11-28T04:26:19.724077+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353162+0000","last_change":"2023-11-28T04:26:19.352478+0000","last_active":"2023-11-28T04:26:19.353162+0000","last_peered":"2023-11-28T04:26:19.353162+0000","last_clean":"2023-11-28T04:26:19.353162+0000","last_became_active":"2023-11-28T04:26:19.350726+0000","last_became_peered":"2023-11-28T04:26:19.350726+0000","last_unstale":"2023-11-28T04:26:19.353162+0000","last_undegraded":"2023-11-28T04:26:19.353162+0000","last_fullsized":"2023-11-28T04:26:19.353162+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.32","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716009+0000","last_change":"2023-11-28T04:26:19.716009+0000","last_active":"2023-11-28T04:26:19.716009+0000","last_peered":"2023-11-28T04:26:19.716009+0000","last_clean":"2023-11-28T04:26:19.716009+0000","last_became_active":"2023-11-28T04:26:19.715751+0000","last_became_peered":"2023-11-28T04:26:19.715751+0000","last_unstale":"2023-11-28T04:26:19.716009+0000","last_undegraded":"2023-11-28T04:26:19.716009+0000","last_fullsized":"2023-11-28T04:26:19.716009+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357175+0000","last_change":"2023-11-28T04:26:19.354019+0000","last_active":"2023-11-28T04:26:19.357175+0000","last_peered":"2023-11-28T04:26:19.357175+0000","last_clean":"2023-11-28T04:26:19.357175+0000","last_became_active":"2023-11-28T04:26:19.353730+0000","last_became_peered":"2023-11-28T04:26:19.353730+0000","last_unstale":"2023-11-28T04:26:19.357175+0000","last_undegraded":"2023-11-28T04:26:19.357175+0000","last_fullsized":"2023-11-28T04:26:19.357175+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.33","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723092+0000","last_change":"2023-11-28T04:26:19.723092+0000","last_active":"2023-11-28T04:26:19.723092+0000","last_peered":"2023-11-28T04:26:19.723092+0000","last_clean":"2023-11-28T04:26:19.723092+0000","last_became_active":"2023-11-28T04:26:19.722668+0000","last_became_peered":"2023-11-28T04:26:19.722668+0000","last_unstale":"2023-11-28T04:26:19.723092+0000","last_undegraded":"2023-11-28T04:26:19.723092+0000","last_fullsized":"2023-11-28T04:26:19.723092+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.30","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727092+0000","last_change":"2023-11-28T04:26:19.353476+0000","last_active":"2023-11-28T04:26:19.727092+0000","last_peered":"2023-11-28T04:26:19.727092+0000","last_clean":"2023-11-28T04:26:19.727092+0000","last_became_active":"2023-11-28T04:26:19.353241+0000","last_became_peered":"2023-11-28T04:26:19.353241+0000","last_unstale":"2023-11-28T04:26:19.727092+0000","last_undegraded":"2023-11-28T04:26:19.727092+0000","last_fullsized":"2023-11-28T04:26:19.727092+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719012+0000","last_change":"2023-11-28T04:26:19.719012+0000","last_active":"2023-11-28T04:26:19.719012+0000","last_peered":"2023-11-28T04:26:19.719012+0000","last_clean":"2023-11-28T04:26:19.719012+0000","last_became_active":"2023-11-28T04:26:19.718733+0000","last_became_peered":"2023-11-28T04:26:19.718733+0000","last_unstale":"2023-11-28T04:26:19.719012+0000","last_undegraded":"2023-11-28T04:26:19.719012+0000","last_fullsized":"2023-11-28T04:26:19.719012+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353592+0000","last_change":"2023-11-28T04:26:19.352413+0000","last_active":"2023-11-28T04:26:19.353592+0000","last_peered":"2023-11-28T04:26:19.353592+0000","last_clean":"2023-11-28T04:26:19.353592+0000","last_became_active":"2023-11-28T04:26:19.352122+0000","last_became_peered":"2023-11-28T04:26:19.352122+0000","last_unstale":"2023-11-28T04:26:19.353592+0000","last_undegraded":"2023-11-28T04:26:19.353592+0000","last_fullsized":"2023-11-28T04:26:19.353592+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719497+0000","last_change":"2023-11-28T04:26:19.719497+0000","last_active":"2023-11-28T04:26:19.719497+0000","last_peered":"2023-11-28T04:26:19.719497+0000","last_clean":"2023-11-28T04:26:19.719497+0000","last_became_active":"2023-11-28T04:26:19.719182+0000","last_became_peered":"2023-11-28T04:26:19.719182+0000","last_unstale":"2023-11-28T04:26:19.719497+0000","last_undegraded":"2023-11-28T04:26:19.719497+0000","last_fullsized":"2023-11-28T04:26:19.719497+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357463+0000","last_change":"2023-11-28T04:26:19.353859+0000","last_active":"2023-11-28T04:26:19.357463+0000","last_peered":"2023-11-28T04:26:19.357463+0000","last_clean":"2023-11-28T04:26:19.357463+0000","last_became_active":"2023-11-28T04:26:19.353577+0000","last_became_peered":"2023-11-28T04:26:19.353577+0000","last_unstale":"2023-11-28T04:26:19.357463+0000","last_undegraded":"2023-11-28T04:26:19.357463+0000","last_fullsized":"2023-11-28T04:26:19.357463+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721937+0000","last_change":"2023-11-28T04:26:19.721937+0000","last_active":"2023-11-28T04:26:19.721937+0000","last_peered":"2023-11-28T04:26:19.721937+0000","last_clean":"2023-11-28T04:26:19.721937+0000","last_became_active":"2023-11-28T04:26:19.721569+0000","last_became_peered":"2023-11-28T04:26:19.721569+0000","last_unstale":"2023-11-28T04:26:19.721937+0000","last_undegraded":"2023-11-28T04:26:19.721937+0000","last_fullsized":"2023-11-28T04:26:19.721937+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2d","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.712336+0000","last_change":"2023-11-28T04:26:19.346464+0000","last_active":"2023-11-28T04:26:19.712336+0000","last_peered":"2023-11-28T04:26:19.712336+0000","last_clean":"2023-11-28T04:26:19.712336+0000","last_became_active":"2023-11-28T04:26:19.346153+0000","last_became_peered":"2023-11-28T04:26:19.346153+0000","last_unstale":"2023-11-28T04:26:19.712336+0000","last_undegraded":"2023-11-28T04:26:19.712336+0000","last_fullsized":"2023-11-28T04:26:19.712336+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.719934+0000","last_change":"2023-11-28T04:26:19.719934+0000","last_active":"2023-11-28T04:26:19.719934+0000","last_peered":"2023-11-28T04:26:19.719934+0000","last_clean":"2023-11-28T04:26:19.719934+0000","last_became_active":"2023-11-28T04:26:19.719677+0000","last_became_peered":"2023-11-28T04:26:19.719677+0000","last_unstale":"2023-11-28T04:26:19.719934+0000","last_undegraded":"2023-11-28T04:26:19.719934+0000","last_fullsized":"2023-11-28T04:26:19.719934+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350683+0000","last_change":"2023-11-28T04:26:19.348599+0000","last_active":"2023-11-28T04:26:19.350683+0000","last_peered":"2023-11-28T04:26:19.350683+0000","last_clean":"2023-11-28T04:26:19.350683+0000","last_became_active":"2023-11-28T04:26:19.348285+0000","last_became_peered":"2023-11-28T04:26:19.348285+0000","last_unstale":"2023-11-28T04:26:19.350683+0000","last_undegraded":"2023-11-28T04:26:19.350683+0000","last_fullsized":"2023-11-28T04:26:19.350683+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.28","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725695+0000","last_change":"2023-11-28T04:26:19.725695+0000","last_active":"2023-11-28T04:26:19.725695+0000","last_peered":"2023-11-28T04:26:19.725695+0000","last_clean":"2023-11-28T04:26:19.725695+0000","last_became_active":"2023-11-28T04:26:19.725420+0000","last_became_peered":"2023-11-28T04:26:19.725420+0000","last_unstale":"2023-11-28T04:26:19.725695+0000","last_undegraded":"2023-11-28T04:26:19.725695+0000","last_fullsized":"2023-11-28T04:26:19.725695+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.702889+0000","last_change":"2023-11-28T04:26:19.346871+0000","last_active":"2023-11-28T04:26:19.702889+0000","last_peered":"2023-11-28T04:26:19.702889+0000","last_clean":"2023-11-28T04:26:19.702889+0000","last_became_active":"2023-11-28T04:26:19.346601+0000","last_became_peered":"2023-11-28T04:26:19.346601+0000","last_unstale":"2023-11-28T04:26:19.702889+0000","last_undegraded":"2023-11-28T04:26:19.702889+0000","last_fullsized":"2023-11-28T04:26:19.702889+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.29","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725224+0000","last_change":"2023-11-28T04:26:19.725224+0000","last_active":"2023-11-28T04:26:19.725224+0000","last_peered":"2023-11-28T04:26:19.725224+0000","last_clean":"2023-11-28T04:26:19.725224+0000","last_became_active":"2023-11-28T04:26:19.724735+0000","last_became_peered":"2023-11-28T04:26:19.724735+0000","last_unstale":"2023-11-28T04:26:19.725224+0000","last_undegraded":"2023-11-28T04:26:19.725224+0000","last_fullsized":"2023-11-28T04:26:19.725224+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355822+0000","last_change":"2023-11-28T04:26:19.354593+0000","last_active":"2023-11-28T04:26:19.355822+0000","last_peered":"2023-11-28T04:26:19.355822+0000","last_clean":"2023-11-28T04:26:19.355822+0000","last_became_active":"2023-11-28T04:26:19.354253+0000","last_became_peered":"2023-11-28T04:26:19.354253+0000","last_unstale":"2023-11-28T04:26:19.355822+0000","last_undegraded":"2023-11-28T04:26:19.355822+0000","last_fullsized":"2023-11-28T04:26:19.355822+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.2a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725680+0000","last_change":"2023-11-28T04:26:19.725680+0000","last_active":"2023-11-28T04:26:19.725680+0000","last_peered":"2023-11-28T04:26:19.725680+0000","last_clean":"2023-11-28T04:26:19.725680+0000","last_became_active":"2023-11-28T04:26:19.725390+0000","last_became_peered":"2023-11-28T04:26:19.725390+0000","last_unstale":"2023-11-28T04:26:19.725680+0000","last_undegraded":"2023-11-28T04:26:19.725680+0000","last_fullsized":"2023-11-28T04:26:19.725680+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357674+0000","last_change":"2023-11-28T04:26:19.355138+0000","last_active":"2023-11-28T04:26:19.357674+0000","last_peered":"2023-11-28T04:26:19.357674+0000","last_clean":"2023-11-28T04:26:19.357674+0000","last_became_active":"2023-11-28T04:26:19.354847+0000","last_became_peered":"2023-11-28T04:26:19.354847+0000","last_unstale":"2023-11-28T04:26:19.357674+0000","last_undegraded":"2023-11-28T04:26:19.357674+0000","last_fullsized":"2023-11-28T04:26:19.357674+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.2b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724054+0000","last_change":"2023-11-28T04:26:19.724054+0000","last_active":"2023-11-28T04:26:19.724054+0000","last_peered":"2023-11-28T04:26:19.724054+0000","last_clean":"2023-11-28T04:26:19.724054+0000","last_became_active":"2023-11-28T04:26:19.723458+0000","last_became_peered":"2023-11-28T04:26:19.723458+0000","last_unstale":"2023-11-28T04:26:19.724054+0000","last_undegraded":"2023-11-28T04:26:19.724054+0000","last_fullsized":"2023-11-28T04:26:19.724054+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.358084+0000","last_change":"2023-11-28T04:26:19.353062+0000","last_active":"2023-11-28T04:26:19.358084+0000","last_peered":"2023-11-28T04:26:19.358084+0000","last_clean":"2023-11-28T04:26:19.358084+0000","last_became_active":"2023-11-28T04:26:19.352793+0000","last_became_peered":"2023-11-28T04:26:19.352793+0000","last_unstale":"2023-11-28T04:26:19.358084+0000","last_undegraded":"2023-11-28T04:26:19.358084+0000","last_fullsized":"2023-11-28T04:26:19.358084+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.24","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726127+0000","last_change":"2023-11-28T04:26:19.726127+0000","last_active":"2023-11-28T04:26:19.726127+0000","last_peered":"2023-11-28T04:26:19.726127+0000","last_clean":"2023-11-28T04:26:19.726127+0000","last_became_active":"2023-11-28T04:26:19.725855+0000","last_became_peered":"2023-11-28T04:26:19.725855+0000","last_unstale":"2023-11-28T04:26:19.726127+0000","last_undegraded":"2023-11-28T04:26:19.726127+0000","last_fullsized":"2023-11-28T04:26:19.726127+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.360952+0000","last_change":"2023-11-28T04:26:19.357248+0000","last_active":"2023-11-28T04:26:19.360952+0000","last_peered":"2023-11-28T04:26:19.360952+0000","last_clean":"2023-11-28T04:26:19.360952+0000","last_became_active":"2023-11-28T04:26:19.356918+0000","last_became_peered":"2023-11-28T04:26:19.356918+0000","last_unstale":"2023-11-28T04:26:19.360952+0000","last_undegraded":"2023-11-28T04:26:19.360952+0000","last_fullsized":"2023-11-28T04:26:19.360952+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.25","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721165+0000","last_change":"2023-11-28T04:26:19.721165+0000","last_active":"2023-11-28T04:26:19.721165+0000","last_peered":"2023-11-28T04:26:19.721165+0000","last_clean":"2023-11-28T04:26:19.721165+0000","last_became_active":"2023-11-28T04:26:19.720877+0000","last_became_peered":"2023-11-28T04:26:19.720877+0000","last_unstale":"2023-11-28T04:26:19.721165+0000","last_undegraded":"2023-11-28T04:26:19.721165+0000","last_fullsized":"2023-11-28T04:26:19.721165+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354999+0000","last_change":"2023-11-28T04:26:19.350529+0000","last_active":"2023-11-28T04:26:19.354999+0000","last_peered":"2023-11-28T04:26:19.354999+0000","last_clean":"2023-11-28T04:26:19.354999+0000","last_became_active":"2023-11-28T04:26:19.350089+0000","last_became_peered":"2023-11-28T04:26:19.350089+0000","last_unstale":"2023-11-28T04:26:19.354999+0000","last_undegraded":"2023-11-28T04:26:19.354999+0000","last_fullsized":"2023-11-28T04:26:19.354999+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.26","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727472+0000","last_change":"2023-11-28T04:26:19.727472+0000","last_active":"2023-11-28T04:26:19.727472+0000","last_peered":"2023-11-28T04:26:19.727472+0000","last_clean":"2023-11-28T04:26:19.727472+0000","last_became_active":"2023-11-28T04:26:19.725452+0000","last_became_peered":"2023-11-28T04:26:19.725452+0000","last_unstale":"2023-11-28T04:26:19.727472+0000","last_undegraded":"2023-11-28T04:26:19.727472+0000","last_fullsized":"2023-11-28T04:26:19.727472+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.349904+0000","last_change":"2023-11-28T04:26:19.345619+0000","last_active":"2023-11-28T04:26:19.349904+0000","last_peered":"2023-11-28T04:26:19.349904+0000","last_clean":"2023-11-28T04:26:19.349904+0000","last_became_active":"2023-11-28T04:26:19.345399+0000","last_became_peered":"2023-11-28T04:26:19.345399+0000","last_unstale":"2023-11-28T04:26:19.349904+0000","last_undegraded":"2023-11-28T04:26:19.349904+0000","last_fullsized":"2023-11-28T04:26:19.349904+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.27","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716885+0000","last_change":"2023-11-28T04:26:19.716885+0000","last_active":"2023-11-28T04:26:19.716885+0000","last_peered":"2023-11-28T04:26:19.716885+0000","last_clean":"2023-11-28T04:26:19.716885+0000","last_became_active":"2023-11-28T04:26:19.716596+0000","last_became_peered":"2023-11-28T04:26:19.716596+0000","last_unstale":"2023-11-28T04:26:19.716885+0000","last_undegraded":"2023-11-28T04:26:19.716885+0000","last_fullsized":"2023-11-28T04:26:19.716885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.358745+0000","last_change":"2023-11-28T04:26:19.357899+0000","last_active":"2023-11-28T04:26:19.358745+0000","last_peered":"2023-11-28T04:26:19.358745+0000","last_clean":"2023-11-28T04:26:19.358745+0000","last_became_active":"2023-11-28T04:26:19.357588+0000","last_became_peered":"2023-11-28T04:26:19.357588+0000","last_unstale":"2023-11-28T04:26:19.358745+0000","last_undegraded":"2023-11-28T04:26:19.358745+0000","last_fullsized":"2023-11-28T04:26:19.358745+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.20","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.716064+0000","last_change":"2023-11-28T04:26:19.716064+0000","last_active":"2023-11-28T04:26:19.716064+0000","last_peered":"2023-11-28T04:26:19.716064+0000","last_clean":"2023-11-28T04:26:19.716064+0000","last_became_active":"2023-11-28T04:26:19.715788+0000","last_became_peered":"2023-11-28T04:26:19.715788+0000","last_unstale":"2023-11-28T04:26:19.716064+0000","last_undegraded":"2023-11-28T04:26:19.716064+0000","last_fullsized":"2023-11-28T04:26:19.716064+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357274+0000","last_change":"2023-11-28T04:26:19.355766+0000","last_active":"2023-11-28T04:26:19.357274+0000","last_peered":"2023-11-28T04:26:19.357274+0000","last_clean":"2023-11-28T04:26:19.357274+0000","last_became_active":"2023-11-28T04:26:19.355478+0000","last_became_peered":"2023-11-28T04:26:19.355478+0000","last_unstale":"2023-11-28T04:26:19.357274+0000","last_undegraded":"2023-11-28T04:26:19.357274+0000","last_fullsized":"2023-11-28T04:26:19.357274+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.21","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725695+0000","last_change":"2023-11-28T04:26:19.725695+0000","last_active":"2023-11-28T04:26:19.725695+0000","last_peered":"2023-11-28T04:26:19.725695+0000","last_clean":"2023-11-28T04:26:19.725695+0000","last_became_active":"2023-11-28T04:26:19.725344+0000","last_became_peered":"2023-11-28T04:26:19.725344+0000","last_unstale":"2023-11-28T04:26:19.725695+0000","last_undegraded":"2023-11-28T04:26:19.725695+0000","last_fullsized":"2023-11-28T04:26:19.725695+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356784+0000","last_change":"2023-11-28T04:26:19.354796+0000","last_active":"2023-11-28T04:26:19.356784+0000","last_peered":"2023-11-28T04:26:19.356784+0000","last_clean":"2023-11-28T04:26:19.356784+0000","last_became_active":"2023-11-28T04:26:19.354503+0000","last_became_peered":"2023-11-28T04:26:19.354503+0000","last_unstale":"2023-11-28T04:26:19.356784+0000","last_undegraded":"2023-11-28T04:26:19.356784+0000","last_fullsized":"2023-11-28T04:26:19.356784+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.22","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730535+0000","last_change":"2023-11-28T04:26:19.730535+0000","last_active":"2023-11-28T04:26:19.730535+0000","last_peered":"2023-11-28T04:26:19.730535+0000","last_clean":"2023-11-28T04:26:19.730535+0000","last_became_active":"2023-11-28T04:26:19.730271+0000","last_became_peered":"2023-11-28T04:26:19.730271+0000","last_unstale":"2023-11-28T04:26:19.730535+0000","last_undegraded":"2023-11-28T04:26:19.730535+0000","last_fullsized":"2023-11-28T04:26:19.730535+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353412+0000","last_change":"2023-11-28T04:26:19.351892+0000","last_active":"2023-11-28T04:26:19.353412+0000","last_peered":"2023-11-28T04:26:19.353412+0000","last_clean":"2023-11-28T04:26:19.353412+0000","last_became_active":"2023-11-28T04:26:19.351614+0000","last_became_peered":"2023-11-28T04:26:19.351614+0000","last_unstale":"2023-11-28T04:26:19.353412+0000","last_undegraded":"2023-11-28T04:26:19.353412+0000","last_fullsized":"2023-11-28T04:26:19.353412+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.23","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726326+0000","last_change":"2023-11-28T04:26:19.726326+0000","last_active":"2023-11-28T04:26:19.726326+0000","last_peered":"2023-11-28T04:26:19.726326+0000","last_clean":"2023-11-28T04:26:19.726326+0000","last_became_active":"2023-11-28T04:26:19.725441+0000","last_became_peered":"2023-11-28T04:26:19.725441+0000","last_unstale":"2023-11-28T04:26:19.726326+0000","last_undegraded":"2023-11-28T04:26:19.726326+0000","last_fullsized":"2023-11-28T04:26:19.726326+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.345811+0000","last_change":"2023-11-28T04:26:19.344437+0000","last_active":"2023-11-28T04:26:19.345811+0000","last_peered":"2023-11-28T04:26:19.345811+0000","last_clean":"2023-11-28T04:26:19.345811+0000","last_became_active":"2023-11-28T04:26:19.344146+0000","last_became_peered":"2023-11-28T04:26:19.344146+0000","last_unstale":"2023-11-28T04:26:19.345811+0000","last_undegraded":"2023-11-28T04:26:19.345811+0000","last_fullsized":"2023-11-28T04:26:19.345811+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721015+0000","last_change":"2023-11-28T04:26:19.721015+0000","last_active":"2023-11-28T04:26:19.721015+0000","last_peered":"2023-11-28T04:26:19.721015+0000","last_clean":"2023-11-28T04:26:19.721015+0000","last_became_active":"2023-11-28T04:26:19.720445+0000","last_became_peered":"2023-11-28T04:26:19.720445+0000","last_unstale":"2023-11-28T04:26:19.721015+0000","last_undegraded":"2023-11-28T04:26:19.721015+0000","last_fullsized":"2023-11-28T04:26:19.721015+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351764+0000","last_change":"2023-11-28T04:26:19.342750+0000","last_active":"2023-11-28T04:26:19.351764+0000","last_peered":"2023-11-28T04:26:19.351764+0000","last_clean":"2023-11-28T04:26:19.351764+0000","last_became_active":"2023-11-28T04:26:19.342458+0000","last_became_peered":"2023-11-28T04:26:19.342458+0000","last_unstale":"2023-11-28T04:26:19.351764+0000","last_undegraded":"2023-11-28T04:26:19.351764+0000","last_fullsized":"2023-11-28T04:26:19.351764+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.717716+0000","last_change":"2023-11-28T04:26:19.717716+0000","last_active":"2023-11-28T04:26:19.717716+0000","last_peered":"2023-11-28T04:26:19.717716+0000","last_clean":"2023-11-28T04:26:19.717716+0000","last_became_active":"2023-11-28T04:26:19.717469+0000","last_became_peered":"2023-11-28T04:26:19.717469+0000","last_unstale":"2023-11-28T04:26:19.717716+0000","last_undegraded":"2023-11-28T04:26:19.717716+0000","last_fullsized":"2023-11-28T04:26:19.717716+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.359747+0000","last_change":"2023-11-28T04:26:19.359164+0000","last_active":"2023-11-28T04:26:19.359747+0000","last_peered":"2023-11-28T04:26:19.359747+0000","last_clean":"2023-11-28T04:26:19.359747+0000","last_became_active":"2023-11-28T04:26:19.358977+0000","last_became_peered":"2023-11-28T04:26:19.358977+0000","last_unstale":"2023-11-28T04:26:19.359747+0000","last_undegraded":"2023-11-28T04:26:19.359747+0000","last_fullsized":"2023-11-28T04:26:19.359747+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.729199+0000","last_change":"2023-11-28T04:26:19.729199+0000","last_active":"2023-11-28T04:26:19.729199+0000","last_peered":"2023-11-28T04:26:19.729199+0000","last_clean":"2023-11-28T04:26:19.729199+0000","last_became_active":"2023-11-28T04:26:19.728934+0000","last_became_peered":"2023-11-28T04:26:19.728934+0000","last_unstale":"2023-11-28T04:26:19.729199+0000","last_undegraded":"2023-11-28T04:26:19.729199+0000","last_fullsized":"2023-11-28T04:26:19.729199+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.15","version":"211'3","reported_seq":12,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727344+0000","last_change":"2023-11-28T04:26:19.357491+0000","last_active":"2023-11-28T04:26:19.727344+0000","last_peered":"2023-11-28T04:26:19.727344+0000","last_clean":"2023-11-28T04:26:19.727344+0000","last_became_active":"2023-11-28T04:26:19.357223+0000","last_became_peered":"2023-11-28T04:26:19.357223+0000","last_unstale":"2023-11-28T04:26:19.727344+0000","last_undegraded":"2023-11-28T04:26:19.727344+0000","last_fullsized":"2023-11-28T04:26:19.727344+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":3,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.730075+0000","last_change":"2023-11-28T04:26:19.730075+0000","last_active":"2023-11-28T04:26:19.730075+0000","last_peered":"2023-11-28T04:26:19.730075+0000","last_clean":"2023-11-28T04:26:19.730075+0000","last_became_active":"2023-11-28T04:26:19.729774+0000","last_became_peered":"2023-11-28T04:26:19.729774+0000","last_unstale":"2023-11-28T04:26:19.730075+0000","last_undegraded":"2023-11-28T04:26:19.730075+0000","last_fullsized":"2023-11-28T04:26:19.730075+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.14","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.713515+0000","last_change":"2023-11-28T04:26:19.348064+0000","last_active":"2023-11-28T04:26:19.713515+0000","last_peered":"2023-11-28T04:26:19.713515+0000","last_clean":"2023-11-28T04:26:19.713515+0000","last_became_active":"2023-11-28T04:26:19.347733+0000","last_became_peered":"2023-11-28T04:26:19.347733+0000","last_unstale":"2023-11-28T04:26:19.713515+0000","last_undegraded":"2023-11-28T04:26:19.713515+0000","last_fullsized":"2023-11-28T04:26:19.713515+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":644,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718462+0000","last_change":"2023-11-28T04:26:19.718462+0000","last_active":"2023-11-28T04:26:19.718462+0000","last_peered":"2023-11-28T04:26:19.718462+0000","last_clean":"2023-11-28T04:26:19.718462+0000","last_became_active":"2023-11-28T04:26:19.718206+0000","last_became_peered":"2023-11-28T04:26:19.718206+0000","last_unstale":"2023-11-28T04:26:19.718462+0000","last_undegraded":"2023-11-28T04:26:19.718462+0000","last_fullsized":"2023-11-28T04:26:19.718462+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.13","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707905+0000","last_change":"2023-11-28T04:26:19.349824+0000","last_active":"2023-11-28T04:26:19.707905+0000","last_peered":"2023-11-28T04:26:19.707905+0000","last_clean":"2023-11-28T04:26:19.707905+0000","last_became_active":"2023-11-28T04:26:19.349438+0000","last_became_peered":"2023-11-28T04:26:19.349438+0000","last_unstale":"2023-11-28T04:26:19.707905+0000","last_undegraded":"2023-11-28T04:26:19.707905+0000","last_fullsized":"2023-11-28T04:26:19.707905+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":34,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721739+0000","last_change":"2023-11-28T04:26:19.721739+0000","last_active":"2023-11-28T04:26:19.721739+0000","last_peered":"2023-11-28T04:26:19.721739+0000","last_clean":"2023-11-28T04:26:19.721739+0000","last_became_active":"2023-11-28T04:26:19.721489+0000","last_became_peered":"2023-11-28T04:26:19.721489+0000","last_unstale":"2023-11-28T04:26:19.721739+0000","last_undegraded":"2023-11-28T04:26:19.721739+0000","last_fullsized":"2023-11-28T04:26:19.721739+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357629+0000","last_change":"2023-11-28T04:26:19.355248+0000","last_active":"2023-11-28T04:26:19.357629+0000","last_peered":"2023-11-28T04:26:19.357629+0000","last_clean":"2023-11-28T04:26:19.357629+0000","last_became_active":"2023-11-28T04:26:19.354958+0000","last_became_peered":"2023-11-28T04:26:19.354958+0000","last_unstale":"2023-11-28T04:26:19.357629+0000","last_undegraded":"2023-11-28T04:26:19.357629+0000","last_fullsized":"2023-11-28T04:26:19.357629+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718326+0000","last_change":"2023-11-28T04:26:19.718326+0000","last_active":"2023-11-28T04:26:19.718326+0000","last_peered":"2023-11-28T04:26:19.718326+0000","last_clean":"2023-11-28T04:26:19.718326+0000","last_became_active":"2023-11-28T04:26:19.718044+0000","last_became_peered":"2023-11-28T04:26:19.718044+0000","last_unstale":"2023-11-28T04:26:19.718326+0000","last_undegraded":"2023-11-28T04:26:19.718326+0000","last_fullsized":"2023-11-28T04:26:19.718326+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355490+0000","last_change":"2023-11-28T04:26:19.351002+0000","last_active":"2023-11-28T04:26:19.355490+0000","last_peered":"2023-11-28T04:26:19.355490+0000","last_clean":"2023-11-28T04:26:19.355490+0000","last_became_active":"2023-11-28T04:26:19.350722+0000","last_became_peered":"2023-11-28T04:26:19.350722+0000","last_unstale":"2023-11-28T04:26:19.355490+0000","last_undegraded":"2023-11-28T04:26:19.355490+0000","last_fullsized":"2023-11-28T04:26:19.355490+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726518+0000","last_change":"2023-11-28T04:26:19.726518+0000","last_active":"2023-11-28T04:26:19.726518+0000","last_peered":"2023-11-28T04:26:19.726518+0000","last_clean":"2023-11-28T04:26:19.726518+0000","last_became_active":"2023-11-28T04:26:19.724068+0000","last_became_peered":"2023-11-28T04:26:19.724068+0000","last_unstale":"2023-11-28T04:26:19.726518+0000","last_undegraded":"2023-11-28T04:26:19.726518+0000","last_fullsized":"2023-11-28T04:26:19.726518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.10","version":"211'4","reported_seq":13,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725664+0000","last_change":"2023-11-28T04:26:19.352144+0000","last_active":"2023-11-28T04:26:19.725664+0000","last_peered":"2023-11-28T04:26:19.725664+0000","last_clean":"2023-11-28T04:26:19.725664+0000","last_became_active":"2023-11-28T04:26:19.351806+0000","last_became_peered":"2023-11-28T04:26:19.351806+0000","last_unstale":"2023-11-28T04:26:19.725664+0000","last_undegraded":"2023-11-28T04:26:19.725664+0000","last_fullsized":"2023-11-28T04:26:19.725664+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":4,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725518+0000","last_change":"2023-11-28T04:26:19.725518+0000","last_active":"2023-11-28T04:26:19.725518+0000","last_peered":"2023-11-28T04:26:19.725518+0000","last_clean":"2023-11-28T04:26:19.725518+0000","last_became_active":"2023-11-28T04:26:19.725195+0000","last_became_peered":"2023-11-28T04:26:19.725195+0000","last_unstale":"2023-11-28T04:26:19.725518+0000","last_undegraded":"2023-11-28T04:26:19.725518+0000","last_fullsized":"2023-11-28T04:26:19.725518+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.715131+0000","last_change":"2023-11-28T04:26:19.355528+0000","last_active":"2023-11-28T04:26:19.715131+0000","last_peered":"2023-11-28T04:26:19.715131+0000","last_clean":"2023-11-28T04:26:19.715131+0000","last_became_active":"2023-11-28T04:26:19.355175+0000","last_became_peered":"2023-11-28T04:26:19.355175+0000","last_unstale":"2023-11-28T04:26:19.715131+0000","last_undegraded":"2023-11-28T04:26:19.715131+0000","last_fullsized":"2023-11-28T04:26:19.715131+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.714579+0000","last_change":"2023-11-28T04:26:19.714579+0000","last_active":"2023-11-28T04:26:19.714579+0000","last_peered":"2023-11-28T04:26:19.714579+0000","last_clean":"2023-11-28T04:26:19.714579+0000","last_became_active":"2023-11-28T04:26:19.714332+0000","last_became_peered":"2023-11-28T04:26:19.714332+0000","last_unstale":"2023-11-28T04:26:19.714579+0000","last_undegraded":"2023-11-28T04:26:19.714579+0000","last_fullsized":"2023-11-28T04:26:19.714579+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.e","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718296+0000","last_change":"2023-11-28T04:26:19.350570+0000","last_active":"2023-11-28T04:26:19.718296+0000","last_peered":"2023-11-28T04:26:19.718296+0000","last_clean":"2023-11-28T04:26:19.718296+0000","last_became_active":"2023-11-28T04:26:19.350292+0000","last_became_peered":"2023-11-28T04:26:19.350292+0000","last_unstale":"2023-11-28T04:26:19.718296+0000","last_undegraded":"2023-11-28T04:26:19.718296+0000","last_fullsized":"2023-11-28T04:26:19.718296+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727814+0000","last_change":"2023-11-28T04:26:19.727814+0000","last_active":"2023-11-28T04:26:19.727814+0000","last_peered":"2023-11-28T04:26:19.727814+0000","last_clean":"2023-11-28T04:26:19.727814+0000","last_became_active":"2023-11-28T04:26:19.727547+0000","last_became_peered":"2023-11-28T04:26:19.727547+0000","last_unstale":"2023-11-28T04:26:19.727814+0000","last_undegraded":"2023-11-28T04:26:19.727814+0000","last_fullsized":"2023-11-28T04:26:19.727814+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.363569+0000","last_change":"2023-11-28T04:26:19.358993+0000","last_active":"2023-11-28T04:26:19.363569+0000","last_peered":"2023-11-28T04:26:19.363569+0000","last_clean":"2023-11-28T04:26:19.363569+0000","last_became_active":"2023-11-28T04:26:19.358679+0000","last_became_peered":"2023-11-28T04:26:19.358679+0000","last_unstale":"2023-11-28T04:26:19.363569+0000","last_undegraded":"2023-11-28T04:26:19.363569+0000","last_fullsized":"2023-11-28T04:26:19.363569+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724320+0000","last_change":"2023-11-28T04:26:19.724320+0000","last_active":"2023-11-28T04:26:19.724320+0000","last_peered":"2023-11-28T04:26:19.724320+0000","last_clean":"2023-11-28T04:26:19.724320+0000","last_became_active":"2023-11-28T04:26:19.723931+0000","last_became_peered":"2023-11-28T04:26:19.723931+0000","last_unstale":"2023-11-28T04:26:19.724320+0000","last_undegraded":"2023-11-28T04:26:19.724320+0000","last_fullsized":"2023-11-28T04:26:19.724320+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350554+0000","last_change":"2023-11-28T04:26:19.349597+0000","last_active":"2023-11-28T04:26:19.350554+0000","last_peered":"2023-11-28T04:26:19.350554+0000","last_clean":"2023-11-28T04:26:19.350554+0000","last_became_active":"2023-11-28T04:26:19.349311+0000","last_became_peered":"2023-11-28T04:26:19.349311+0000","last_unstale":"2023-11-28T04:26:19.350554+0000","last_undegraded":"2023-11-28T04:26:19.350554+0000","last_fullsized":"2023-11-28T04:26:19.350554+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723573+0000","last_change":"2023-11-28T04:26:19.723573+0000","last_active":"2023-11-28T04:26:19.723573+0000","last_peered":"2023-11-28T04:26:19.723573+0000","last_clean":"2023-11-28T04:26:19.723573+0000","last_became_active":"2023-11-28T04:26:19.723297+0000","last_became_peered":"2023-11-28T04:26:19.723297+0000","last_unstale":"2023-11-28T04:26:19.723573+0000","last_undegraded":"2023-11-28T04:26:19.723573+0000","last_fullsized":"2023-11-28T04:26:19.723573+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707929+0000","last_change":"2023-11-28T04:26:19.358540+0000","last_active":"2023-11-28T04:26:19.707929+0000","last_peered":"2023-11-28T04:26:19.707929+0000","last_clean":"2023-11-28T04:26:19.707929+0000","last_became_active":"2023-11-28T04:26:19.358286+0000","last_became_peered":"2023-11-28T04:26:19.358286+0000","last_unstale":"2023-11-28T04:26:19.707929+0000","last_undegraded":"2023-11-28T04:26:19.707929+0000","last_fullsized":"2023-11-28T04:26:19.707929+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718880+0000","last_change":"2023-11-28T04:26:19.718880+0000","last_active":"2023-11-28T04:26:19.718880+0000","last_peered":"2023-11-28T04:26:19.718880+0000","last_clean":"2023-11-28T04:26:19.718880+0000","last_became_active":"2023-11-28T04:26:19.718612+0000","last_became_peered":"2023-11-28T04:26:19.718612+0000","last_unstale":"2023-11-28T04:26:19.718880+0000","last_undegraded":"2023-11-28T04:26:19.718880+0000","last_fullsized":"2023-11-28T04:26:19.718880+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354642+0000","last_change":"2023-11-28T04:26:19.343708+0000","last_active":"2023-11-28T04:26:19.354642+0000","last_peered":"2023-11-28T04:26:19.354642+0000","last_clean":"2023-11-28T04:26:19.354642+0000","last_became_active":"2023-11-28T04:26:19.343447+0000","last_became_peered":"2023-11-28T04:26:19.343447+0000","last_unstale":"2023-11-28T04:26:19.354642+0000","last_undegraded":"2023-11-28T04:26:19.354642+0000","last_fullsized":"2023-11-28T04:26:19.354642+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728032+0000","last_change":"2023-11-28T04:26:19.728032+0000","last_active":"2023-11-28T04:26:19.728032+0000","last_peered":"2023-11-28T04:26:19.728032+0000","last_clean":"2023-11-28T04:26:19.728032+0000","last_became_active":"2023-11-28T04:26:19.727822+0000","last_became_peered":"2023-11-28T04:26:19.727822+0000","last_unstale":"2023-11-28T04:26:19.728032+0000","last_undegraded":"2023-11-28T04:26:19.728032+0000","last_fullsized":"2023-11-28T04:26:19.728032+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.345887+0000","last_change":"2023-11-28T04:26:19.344644+0000","last_active":"2023-11-28T04:26:19.345887+0000","last_peered":"2023-11-28T04:26:19.345887+0000","last_clean":"2023-11-28T04:26:19.345887+0000","last_became_active":"2023-11-28T04:26:19.344159+0000","last_became_peered":"2023-11-28T04:26:19.344159+0000","last_unstale":"2023-11-28T04:26:19.345887+0000","last_undegraded":"2023-11-28T04:26:19.345887+0000","last_fullsized":"2023-11-28T04:26:19.345887+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.718534+0000","last_change":"2023-11-28T04:26:19.718534+0000","last_active":"2023-11-28T04:26:19.718534+0000","last_peered":"2023-11-28T04:26:19.718534+0000","last_clean":"2023-11-28T04:26:19.718534+0000","last_became_active":"2023-11-28T04:26:19.718247+0000","last_became_peered":"2023-11-28T04:26:19.718247+0000","last_unstale":"2023-11-28T04:26:19.718534+0000","last_undegraded":"2023-11-28T04:26:19.718534+0000","last_fullsized":"2023-11-28T04:26:19.718534+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357227+0000","last_change":"2023-11-28T04:26:19.355471+0000","last_active":"2023-11-28T04:26:19.357227+0000","last_peered":"2023-11-28T04:26:19.357227+0000","last_clean":"2023-11-28T04:26:19.357227+0000","last_became_active":"2023-11-28T04:26:19.355186+0000","last_became_peered":"2023-11-28T04:26:19.355186+0000","last_unstale":"2023-11-28T04:26:19.357227+0000","last_undegraded":"2023-11-28T04:26:19.357227+0000","last_fullsized":"2023-11-28T04:26:19.357227+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724246+0000","last_change":"2023-11-28T04:26:19.724246+0000","last_active":"2023-11-28T04:26:19.724246+0000","last_peered":"2023-11-28T04:26:19.724246+0000","last_clean":"2023-11-28T04:26:19.724246+0000","last_became_active":"2023-11-28T04:26:19.723984+0000","last_became_peered":"2023-11-28T04:26:19.723984+0000","last_unstale":"2023-11-28T04:26:19.724246+0000","last_undegraded":"2023-11-28T04:26:19.724246+0000","last_fullsized":"2023-11-28T04:26:19.724246+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.7","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.721279+0000","last_change":"2023-11-28T04:26:19.347878+0000","last_active":"2023-11-28T04:26:19.721279+0000","last_peered":"2023-11-28T04:26:19.721279+0000","last_clean":"2023-11-28T04:26:19.721279+0000","last_became_active":"2023-11-28T04:26:19.347157+0000","last_became_peered":"2023-11-28T04:26:19.347157+0000","last_unstale":"2023-11-28T04:26:19.721279+0000","last_undegraded":"2023-11-28T04:26:19.721279+0000","last_fullsized":"2023-11-28T04:26:19.721279+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724844+0000","last_change":"2023-11-28T04:26:19.724844+0000","last_active":"2023-11-28T04:26:19.724844+0000","last_peered":"2023-11-28T04:26:19.724844+0000","last_clean":"2023-11-28T04:26:19.724844+0000","last_became_active":"2023-11-28T04:26:19.724587+0000","last_became_peered":"2023-11-28T04:26:19.724587+0000","last_unstale":"2023-11-28T04:26:19.724844+0000","last_undegraded":"2023-11-28T04:26:19.724844+0000","last_fullsized":"2023-11-28T04:26:19.724844+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.351553+0000","last_change":"2023-11-28T04:26:19.349269+0000","last_active":"2023-11-28T04:26:19.351553+0000","last_peered":"2023-11-28T04:26:19.351553+0000","last_clean":"2023-11-28T04:26:19.351553+0000","last_became_active":"2023-11-28T04:26:19.349013+0000","last_became_peered":"2023-11-28T04:26:19.349013+0000","last_unstale":"2023-11-28T04:26:19.351553+0000","last_undegraded":"2023-11-28T04:26:19.351553+0000","last_fullsized":"2023-11-28T04:26:19.351553+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723187+0000","last_change":"2023-11-28T04:26:19.723187+0000","last_active":"2023-11-28T04:26:19.723187+0000","last_peered":"2023-11-28T04:26:19.723187+0000","last_clean":"2023-11-28T04:26:19.723187+0000","last_became_active":"2023-11-28T04:26:19.722784+0000","last_became_peered":"2023-11-28T04:26:19.722784+0000","last_unstale":"2023-11-28T04:26:19.723187+0000","last_undegraded":"2023-11-28T04:26:19.723187+0000","last_fullsized":"2023-11-28T04:26:19.723187+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.352811+0000","last_change":"2023-11-28T04:26:19.346261+0000","last_active":"2023-11-28T04:26:19.352811+0000","last_peered":"2023-11-28T04:26:19.352811+0000","last_clean":"2023-11-28T04:26:19.352811+0000","last_became_active":"2023-11-28T04:26:19.345988+0000","last_became_peered":"2023-11-28T04:26:19.345988+0000","last_unstale":"2023-11-28T04:26:19.352811+0000","last_undegraded":"2023-11-28T04:26:19.352811+0000","last_fullsized":"2023-11-28T04:26:19.352811+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726412+0000","last_change":"2023-11-28T04:26:19.726412+0000","last_active":"2023-11-28T04:26:19.726412+0000","last_peered":"2023-11-28T04:26:19.726412+0000","last_clean":"2023-11-28T04:26:19.726412+0000","last_became_active":"2023-11-28T04:26:19.726161+0000","last_became_peered":"2023-11-28T04:26:19.726161+0000","last_unstale":"2023-11-28T04:26:19.726412+0000","last_undegraded":"2023-11-28T04:26:19.726412+0000","last_fullsized":"2023-11-28T04:26:19.726412+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356813+0000","last_change":"2023-11-28T04:26:19.342853+0000","last_active":"2023-11-28T04:26:19.356813+0000","last_peered":"2023-11-28T04:26:19.356813+0000","last_clean":"2023-11-28T04:26:19.356813+0000","last_became_active":"2023-11-28T04:26:19.342586+0000","last_became_peered":"2023-11-28T04:26:19.342586+0000","last_unstale":"2023-11-28T04:26:19.356813+0000","last_undegraded":"2023-11-28T04:26:19.356813+0000","last_fullsized":"2023-11-28T04:26:19.356813+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724406+0000","last_change":"2023-11-28T04:26:19.724406+0000","last_active":"2023-11-28T04:26:19.724406+0000","last_peered":"2023-11-28T04:26:19.724406+0000","last_clean":"2023-11-28T04:26:19.724406+0000","last_became_active":"2023-11-28T04:26:19.724102+0000","last_became_peered":"2023-11-28T04:26:19.724102+0000","last_unstale":"2023-11-28T04:26:19.724406+0000","last_undegraded":"2023-11-28T04:26:19.724406+0000","last_fullsized":"2023-11-28T04:26:19.724406+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.354965+0000","last_change":"2023-11-28T04:26:19.352655+0000","last_active":"2023-11-28T04:26:19.354965+0000","last_peered":"2023-11-28T04:26:19.354965+0000","last_clean":"2023-11-28T04:26:19.354965+0000","last_became_active":"2023-11-28T04:26:19.352362+0000","last_became_peered":"2023-11-28T04:26:19.352362+0000","last_unstale":"2023-11-28T04:26:19.354965+0000","last_undegraded":"2023-11-28T04:26:19.354965+0000","last_fullsized":"2023-11-28T04:26:19.354965+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726230+0000","last_change":"2023-11-28T04:26:19.726230+0000","last_active":"2023-11-28T04:26:19.726230+0000","last_peered":"2023-11-28T04:26:19.726230+0000","last_clean":"2023-11-28T04:26:19.726230+0000","last_became_active":"2023-11-28T04:26:19.723839+0000","last_became_peered":"2023-11-28T04:26:19.723839+0000","last_unstale":"2023-11-28T04:26:19.726230+0000","last_undegraded":"2023-11-28T04:26:19.726230+0000","last_fullsized":"2023-11-28T04:26:19.726230+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.357449+0000","last_change":"2023-11-28T04:26:19.353391+0000","last_active":"2023-11-28T04:26:19.357449+0000","last_peered":"2023-11-28T04:26:19.357449+0000","last_clean":"2023-11-28T04:26:19.357449+0000","last_became_active":"2023-11-28T04:26:19.352964+0000","last_became_peered":"2023-11-28T04:26:19.352964+0000","last_unstale":"2023-11-28T04:26:19.357449+0000","last_undegraded":"2023-11-28T04:26:19.357449+0000","last_fullsized":"2023-11-28T04:26:19.357449+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725246+0000","last_change":"2023-11-28T04:26:19.725246+0000","last_active":"2023-11-28T04:26:19.725246+0000","last_peered":"2023-11-28T04:26:19.725246+0000","last_clean":"2023-11-28T04:26:19.725246+0000","last_became_active":"2023-11-28T04:26:19.724719+0000","last_became_peered":"2023-11-28T04:26:19.724719+0000","last_unstale":"2023-11-28T04:26:19.725246+0000","last_undegraded":"2023-11-28T04:26:19.725246+0000","last_fullsized":"2023-11-28T04:26:19.725246+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.353631+0000","last_change":"2023-11-28T04:26:19.352948+0000","last_active":"2023-11-28T04:26:19.353631+0000","last_peered":"2023-11-28T04:26:19.353631+0000","last_clean":"2023-11-28T04:26:19.353631+0000","last_became_active":"2023-11-28T04:26:19.352689+0000","last_became_peered":"2023-11-28T04:26:19.352689+0000","last_unstale":"2023-11-28T04:26:19.353631+0000","last_undegraded":"2023-11-28T04:26:19.353631+0000","last_fullsized":"2023-11-28T04:26:19.353631+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.724801+0000","last_change":"2023-11-28T04:26:19.724801+0000","last_active":"2023-11-28T04:26:19.724801+0000","last_peered":"2023-11-28T04:26:19.724801+0000","last_clean":"2023-11-28T04:26:19.724801+0000","last_became_active":"2023-11-28T04:26:19.724489+0000","last_became_peered":"2023-11-28T04:26:19.724489+0000","last_unstale":"2023-11-28T04:26:19.724801+0000","last_undegraded":"2023-11-28T04:26:19.724801+0000","last_fullsized":"2023-11-28T04:26:19.724801+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356997+0000","last_change":"2023-11-28T04:26:19.352441+0000","last_active":"2023-11-28T04:26:19.356997+0000","last_peered":"2023-11-28T04:26:19.356997+0000","last_clean":"2023-11-28T04:26:19.356997+0000","last_became_active":"2023-11-28T04:26:19.351976+0000","last_became_peered":"2023-11-28T04:26:19.351976+0000","last_unstale":"2023-11-28T04:26:19.356997+0000","last_undegraded":"2023-11-28T04:26:19.356997+0000","last_fullsized":"2023-11-28T04:26:19.356997+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723772+0000","last_change":"2023-11-28T04:26:19.723772+0000","last_active":"2023-11-28T04:26:19.723772+0000","last_peered":"2023-11-28T04:26:19.723772+0000","last_clean":"2023-11-28T04:26:19.723772+0000","last_became_active":"2023-11-28T04:26:19.723485+0000","last_became_peered":"2023-11-28T04:26:19.723485+0000","last_unstale":"2023-11-28T04:26:19.723772+0000","last_undegraded":"2023-11-28T04:26:19.723772+0000","last_fullsized":"2023-11-28T04:26:19.723772+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1f","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.711496+0000","last_change":"2023-11-28T04:26:19.345691+0000","last_active":"2023-11-28T04:26:19.711496+0000","last_peered":"2023-11-28T04:26:19.711496+0000","last_clean":"2023-11-28T04:26:19.711496+0000","last_became_active":"2023-11-28T04:26:19.345420+0000","last_became_peered":"2023-11-28T04:26:19.345420+0000","last_unstale":"2023-11-28T04:26:19.711496+0000","last_undegraded":"2023-11-28T04:26:19.711496+0000","last_fullsized":"2023-11-28T04:26:19.711496+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":170,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.342206+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:26:19.342206+0000","last_peered":"2023-11-28T04:26:19.342206+0000","last_clean":"2023-11-28T04:26:19.342206+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:19.342206+0000","last_undegraded":"2023-11-28T04:26:19.342206+0000","last_fullsized":"2023-11-28T04:26:19.342206+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725036+0000","last_change":"2023-11-28T04:26:19.725036+0000","last_active":"2023-11-28T04:26:19.725036+0000","last_peered":"2023-11-28T04:26:19.725036+0000","last_clean":"2023-11-28T04:26:19.725036+0000","last_became_active":"2023-11-28T04:26:19.724416+0000","last_became_peered":"2023-11-28T04:26:19.724416+0000","last_unstale":"2023-11-28T04:26:19.725036+0000","last_undegraded":"2023-11-28T04:26:19.725036+0000","last_fullsized":"2023-11-28T04:26:19.725036+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350961+0000","last_change":"2023-11-28T04:26:19.344397+0000","last_active":"2023-11-28T04:26:19.350961+0000","last_peered":"2023-11-28T04:26:19.350961+0000","last_clean":"2023-11-28T04:26:19.350961+0000","last_became_active":"2023-11-28T04:26:19.344118+0000","last_became_peered":"2023-11-28T04:26:19.344118+0000","last_unstale":"2023-11-28T04:26:19.350961+0000","last_undegraded":"2023-11-28T04:26:19.350961+0000","last_fullsized":"2023-11-28T04:26:19.350961+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":167,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340702+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:19.340702+0000","last_peered":"2023-11-28T04:26:19.340702+0000","last_clean":"2023-11-28T04:26:19.340702+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:19.340702+0000","last_undegraded":"2023-11-28T04:26:19.340702+0000","last_fullsized":"2023-11-28T04:26:19.340702+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.723615+0000","last_change":"2023-11-28T04:26:19.723615+0000","last_active":"2023-11-28T04:26:19.723615+0000","last_peered":"2023-11-28T04:26:19.723615+0000","last_clean":"2023-11-28T04:26:19.723615+0000","last_became_active":"2023-11-28T04:26:19.723391+0000","last_became_peered":"2023-11-28T04:26:19.723391+0000","last_unstale":"2023-11-28T04:26:19.723615+0000","last_undegraded":"2023-11-28T04:26:19.723615+0000","last_fullsized":"2023-11-28T04:26:19.723615+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355803+0000","last_change":"2023-11-28T04:26:19.354540+0000","last_active":"2023-11-28T04:26:19.355803+0000","last_peered":"2023-11-28T04:26:19.355803+0000","last_clean":"2023-11-28T04:26:19.355803+0000","last_became_active":"2023-11-28T04:26:19.354241+0000","last_became_peered":"2023-11-28T04:26:19.354241+0000","last_unstale":"2023-11-28T04:26:19.355803+0000","last_undegraded":"2023-11-28T04:26:19.355803+0000","last_fullsized":"2023-11-28T04:26:19.355803+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":177,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.333827+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:26:19.333827+0000","last_peered":"2023-11-28T04:26:19.333827+0000","last_clean":"2023-11-28T04:26:19.333827+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:19.333827+0000","last_undegraded":"2023-11-28T04:26:19.333827+0000","last_fullsized":"2023-11-28T04:26:19.333827+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.728914+0000","last_change":"2023-11-28T04:26:19.728914+0000","last_active":"2023-11-28T04:26:19.728914+0000","last_peered":"2023-11-28T04:26:19.728914+0000","last_clean":"2023-11-28T04:26:19.728914+0000","last_became_active":"2023-11-28T04:26:19.728648+0000","last_became_peered":"2023-11-28T04:26:19.728648+0000","last_unstale":"2023-11-28T04:26:19.728914+0000","last_undegraded":"2023-11-28T04:26:19.728914+0000","last_fullsized":"2023-11-28T04:26:19.728914+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.355722+0000","last_change":"2023-11-28T04:26:19.351948+0000","last_active":"2023-11-28T04:26:19.355722+0000","last_peered":"2023-11-28T04:26:19.355722+0000","last_clean":"2023-11-28T04:26:19.355722+0000","last_became_active":"2023-11-28T04:26:19.351696+0000","last_became_peered":"2023-11-28T04:26:19.351696+0000","last_unstale":"2023-11-28T04:26:19.355722+0000","last_undegraded":"2023-11-28T04:26:19.355722+0000","last_fullsized":"2023-11-28T04:26:19.355722+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":167,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.341073+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:19.341073+0000","last_peered":"2023-11-28T04:26:19.341073+0000","last_clean":"2023-11-28T04:26:19.341073+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:19.341073+0000","last_undegraded":"2023-11-28T04:26:19.341073+0000","last_fullsized":"2023-11-28T04:26:19.341073+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":166,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340715+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:19.340715+0000","last_peered":"2023-11-28T04:26:19.340715+0000","last_clean":"2023-11-28T04:26:19.340715+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:19.340715+0000","last_undegraded":"2023-11-28T04:26:19.340715+0000","last_fullsized":"2023-11-28T04:26:19.340715+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.722010+0000","last_change":"2023-11-28T04:26:19.722010+0000","last_active":"2023-11-28T04:26:19.722010+0000","last_peered":"2023-11-28T04:26:19.722010+0000","last_clean":"2023-11-28T04:26:19.722010+0000","last_became_active":"2023-11-28T04:26:19.721747+0000","last_became_peered":"2023-11-28T04:26:19.721747+0000","last_unstale":"2023-11-28T04:26:19.722010+0000","last_undegraded":"2023-11-28T04:26:19.722010+0000","last_fullsized":"2023-11-28T04:26:19.722010+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.356215+0000","last_change":"2023-11-28T04:26:19.347219+0000","last_active":"2023-11-28T04:26:19.356215+0000","last_peered":"2023-11-28T04:26:19.356215+0000","last_clean":"2023-11-28T04:26:19.356215+0000","last_became_active":"2023-11-28T04:26:19.346928+0000","last_became_peered":"2023-11-28T04:26:19.346928+0000","last_unstale":"2023-11-28T04:26:19.356215+0000","last_undegraded":"2023-11-28T04:26:19.356215+0000","last_fullsized":"2023-11-28T04:26:19.356215+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":164,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.340797+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:19.340797+0000","last_peered":"2023-11-28T04:26:19.340797+0000","last_clean":"2023-11-28T04:26:19.340797+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:19.340797+0000","last_undegraded":"2023-11-28T04:26:19.340797+0000","last_fullsized":"2023-11-28T04:26:19.340797+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.717441+0000","last_change":"2023-11-28T04:26:19.717441+0000","last_active":"2023-11-28T04:26:19.717441+0000","last_peered":"2023-11-28T04:26:19.717441+0000","last_clean":"2023-11-28T04:26:19.717441+0000","last_became_active":"2023-11-28T04:26:19.717192+0000","last_became_peered":"2023-11-28T04:26:19.717192+0000","last_unstale":"2023-11-28T04:26:19.717441+0000","last_undegraded":"2023-11-28T04:26:19.717441+0000","last_fullsized":"2023-11-28T04:26:19.717441+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"211'1","reported_seq":10,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.707980+0000","last_change":"2023-11-28T04:26:19.350127+0000","last_active":"2023-11-28T04:26:19.707980+0000","last_peered":"2023-11-28T04:26:19.707980+0000","last_clean":"2023-11-28T04:26:19.707980+0000","last_became_active":"2023-11-28T04:26:19.349834+0000","last_became_peered":"2023-11-28T04:26:19.349834+0000","last_unstale":"2023-11-28T04:26:19.707980+0000","last_undegraded":"2023-11-28T04:26:19.707980+0000","last_fullsized":"2023-11-28T04:26:19.707980+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":1,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":165,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.341455+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:19.341455+0000","last_peered":"2023-11-28T04:26:19.341455+0000","last_clean":"2023-11-28T04:26:19.341455+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:19.341455+0000","last_undegraded":"2023-11-28T04:26:19.341455+0000","last_fullsized":"2023-11-28T04:26:19.341455+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.727306+0000","last_change":"2023-11-28T04:26:19.727306+0000","last_active":"2023-11-28T04:26:19.727306+0000","last_peered":"2023-11-28T04:26:19.727306+0000","last_clean":"2023-11-28T04:26:19.727306+0000","last_became_active":"2023-11-28T04:26:19.725116+0000","last_became_peered":"2023-11-28T04:26:19.725116+0000","last_unstale":"2023-11-28T04:26:19.727306+0000","last_undegraded":"2023-11-28T04:26:19.727306+0000","last_fullsized":"2023-11-28T04:26:19.727306+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1d","version":"211'2","reported_seq":11,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.725432+0000","last_change":"2023-11-28T04:26:19.350360+0000","last_active":"2023-11-28T04:26:19.725432+0000","last_peered":"2023-11-28T04:26:19.725432+0000","last_clean":"2023-11-28T04:26:19.725432+0000","last_became_active":"2023-11-28T04:26:19.350083+0000","last_became_peered":"2023-11-28T04:26:19.350083+0000","last_unstale":"2023-11-28T04:26:19.725432+0000","last_undegraded":"2023-11-28T04:26:19.725432+0000","last_fullsized":"2023-11-28T04:26:19.725432+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":2,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":170,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.344753+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:19.344753+0000","last_peered":"2023-11-28T04:26:19.344753+0000","last_clean":"2023-11-28T04:26:19.344753+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:19.344753+0000","last_undegraded":"2023-11-28T04:26:19.344753+0000","last_fullsized":"2023-11-28T04:26:19.344753+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":8,"reported_epoch":211,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.726220+0000","last_change":"2023-11-28T04:26:19.726220+0000","last_active":"2023-11-28T04:26:19.726220+0000","last_peered":"2023-11-28T04:26:19.726220+0000","last_clean":"2023-11-28T04:26:19.726220+0000","last_became_active":"2023-11-28T04:26:19.725925+0000","last_became_peered":"2023-11-28T04:26:19.725925+0000","last_unstale":"2023-11-28T04:26:19.726220+0000","last_undegraded":"2023-11-28T04:26:19.726220+0000","last_fullsized":"2023-11-28T04:26:19.726220+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:19.323914+0000","last_clean_scrub_stamp":"2023-11-28T04:26:19.323914+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.350917+0000","last_change":"2023-11-28T04:26:19.345898+0000","last_active":"2023-11-28T04:26:19.350917+0000","last_peered":"2023-11-28T04:26:19.350917+0000","last_clean":"2023-11-28T04:26:19.350917+0000","last_became_active":"2023-11-28T04:26:19.345632+0000","last_became_peered":"2023-11-28T04:26:19.345632+0000","last_unstale":"2023-11-28T04:26:19.350917+0000","last_undegraded":"2023-11-28T04:26:19.350917+0000","last_fullsized":"2023-11-28T04:26:19.350917+0000","mapping_epoch":209,"log_start":"0'0","ondisk_log_start":"0'0","created":209,"last_epoch_clean":210,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:18.317057+0000","last_clean_scrub_stamp":"2023-11-28T04:26:18.317057+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":187,"reported_epoch":210,"state":"active+clean","last_fresh":"2023-11-28T04:26:19.346829+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:19.346829+0000","last_peered":"2023-11-28T04:26:19.346829+0000","last_clean":"2023-11-28T04:26:19.346829+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:19.346829+0000","last_undegraded":"2023-11-28T04:26:19.346829+0000","last_fullsized":"2023-11-28T04:26:19.346829+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575965,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.60099999999999998,"15min":0.61399999999999999},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.81100000000000005,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.438},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.60399999999999998,"15min":0.62},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.75,"5min":0.80400000000000005,"15min":0.872},"last":0.34799999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.33500000000000002,"5min":0.309,"15min":0.309},"max":{"1min":0.77500000000000002,"5min":0.79400000000000004,"15min":0.92800000000000005},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.56100000000000005,"15min":0.56699999999999995},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.61799999999999999,"5min":0.79600000000000004,"15min":0.82599999999999996},"last":0.54300000000000004}]},{"osd":2,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.57799999999999996,"15min":0.58099999999999996},"min":{"1min":0.39000000000000001,"5min":0.38100000000000001,"15min":0.379},"max":{"1min":0.67100000000000004,"5min":0.77700000000000002,"15min":0.89000000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.56799999999999995,"15min":0.57299999999999995},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.73299999999999998,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.38900000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.34799999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.67800000000000005,"5min":0.79500000000000004,"15min":0.83599999999999997},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.59099999999999997,"15min":0.60099999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.65900000000000003,"5min":0.81999999999999995,"15min":0.90300000000000002},"last":0.48799999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46800000000000003,"15min":0.46700000000000003},"min":{"1min":0.35199999999999998,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.50800000000000001,"15min":0.5},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.70599999999999996,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.56299999999999994}]},{"osd":5,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52800000000000002,"15min":0.50900000000000001},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.54900000000000004,"15min":0.54200000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.75800000000000001,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.68500000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.57199999999999995,"15min":0.56499999999999995},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.68200000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57399999999999995,"15min":0.56899999999999995},"min":{"1min":0.40200000000000002,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.748,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60799999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575965,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.754,"15min":0.77600000000000002},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.499}]},{"osd":1,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.41899999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.67200000000000004,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.51100000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.84999999999999998},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.438},"min":{"1min":0.28100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.78800000000000003},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.80700000000000005},"last":0.40400000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.496,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.82199999999999995},"last":0.378}]},{"osd":7,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.80100000000000005},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.82999999999999996},"last":0.35099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575964,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.20499999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.34999999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.255},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.40000000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.433},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.38100000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.42099999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.48599999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575965,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.35299999999999998,"15min":0.35999999999999999},"min":{"1min":0.28000000000000003,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":0.51000000000000001,"5min":0.56799999999999995,"15min":0.97799999999999998},"last":2.25},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.38300000000000001,"15min":0.39200000000000002},"min":{"1min":0.23400000000000001,"5min":0.223,"15min":0.223},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.95099999999999996},"last":2.1669999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40300000000000002,"5min":0.38800000000000001,"15min":0.38400000000000001},"min":{"1min":0.28999999999999998,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":0.54800000000000004,"5min":0.65800000000000003,"15min":0.85999999999999999},"last":2.0129999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.34300000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.65600000000000003,"5min":0.66000000000000003,"15min":0.86799999999999999},"last":1.929}]},{"osd":3,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.439,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.32700000000000001,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":0.58699999999999997,"5min":0.59399999999999997,"15min":0.64600000000000002},"last":2.379},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.45000000000000001,"15min":0.44600000000000001},"min":{"1min":0.314,"5min":0.314,"15min":0.30199999999999999},"max":{"1min":0.628,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":2.1520000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60899999999999999,"15min":0.63200000000000001},"min":{"1min":0.49299999999999999,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.92100000000000004,"5min":0.92100000000000004,"15min":0.92400000000000004},"last":2.2709999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.63700000000000001,"15min":0.65100000000000002},"min":{"1min":0.46100000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":1.038},"last":2.0800000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.65100000000000002,"15min":0.65700000000000003},"min":{"1min":0.53000000000000003,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":1.0249999999999999},"last":2.2890000000000001},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.63900000000000001,"15min":0.65300000000000002},"min":{"1min":0.47699999999999998,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.85399999999999998,"5min":0.878,"15min":1.014},"last":2.0510000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.61499999999999999,"15min":0.626},"min":{"1min":0.49199999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.90400000000000003,"5min":0.95099999999999996,"15min":1.0449999999999999},"last":2.1360000000000001},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.63300000000000001,"15min":0.64300000000000002},"min":{"1min":0.51900000000000002,"5min":0.433,"15min":0.42199999999999999},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":1.036},"last":2.4460000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.65500000000000003,"15min":0.66400000000000003},"min":{"1min":0.52700000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.95099999999999996,"5min":0.97299999999999998,"15min":1.0189999999999999},"last":2.1120000000000001},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.69199999999999995},"min":{"1min":0.54300000000000004,"5min":0.47499999999999998,"15min":0.40300000000000002},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":1.0669999999999999},"last":2.4100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575966,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.374,"15min":0.378},"min":{"1min":0.30399999999999999,"5min":0.24099999999999999,"15min":0.22600000000000001},"max":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.442,"5min":0.442,"15min":0.44400000000000001},"min":{"1min":0.32500000000000001,"5min":0.314,"15min":0.30599999999999999},"max":{"1min":0.626,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.34599999999999997}]},{"osd":1,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.377,"15min":0.38500000000000001},"min":{"1min":0.28499999999999998,"5min":0.27100000000000002,"15min":0.25900000000000001},"max":{"1min":0.51900000000000002,"5min":0.57599999999999996,"15min":0.622},"last":0.28299999999999997},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.40899999999999997,"15min":0.41299999999999998},"min":{"1min":0.29199999999999998,"5min":0.28199999999999997,"15min":0.26200000000000001},"max":{"1min":0.52500000000000002,"5min":0.754,"15min":0.754},"last":0.29799999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.437,"15min":0.442},"min":{"1min":0.30499999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.64600000000000002,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.41599999999999998,"15min":0.42199999999999999},"min":{"1min":0.29999999999999999,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.65300000000000002},"last":0.32600000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.60299999999999998,"15min":0.61099999999999999},"min":{"1min":0.44900000000000001,"5min":0.42399999999999999,"15min":0.40100000000000002},"max":{"1min":0.84599999999999997,"5min":0.86299999999999999,"15min":0.94399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63500000000000001,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.876,"5min":0.876,"15min":0.96199999999999997},"last":0.53700000000000003}]},{"osd":5,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.98399999999999999,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67000000000000004,"15min":0.67500000000000004},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":1.016},"last":0.64100000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.65600000000000003,"15min":0.65900000000000003},"min":{"1min":0.48099999999999998,"5min":0.46899999999999997,"15min":0.443},"max":{"1min":0.89500000000000002,"5min":0.93999999999999995,"15min":0.99099999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68200000000000005,"15min":0.68999999999999995},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.625}]},{"osd":7,"last update":"Tue Nov 28 04:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.65000000000000002,"15min":0.65600000000000003},"min":{"1min":0.45900000000000002,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.64500000000000002,"15min":0.66300000000000003},"min":{"1min":0.434,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.58299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575965,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.36499999999999999,"15min":0.37},"min":{"1min":0.27100000000000002,"5min":0.23899999999999999,"15min":0.22500000000000001},"max":{"1min":0.432,"5min":0.57399999999999995,"15min":0.58899999999999997},"last":0.376},{"interface":"front","average":{"1min":0.38,"5min":0.38300000000000001,"15min":0.39400000000000002},"min":{"1min":0.26100000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.51600000000000001,"5min":0.54500000000000004,"15min":0.66500000000000004},"last":0.33600000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.38900000000000001,"5min":0.378,"15min":0.38900000000000001},"min":{"1min":0.27400000000000002,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.42799999999999999,"15min":0.42999999999999999},"min":{"1min":0.32500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.48099999999999998,"5min":0.63400000000000001,"15min":0.63900000000000001},"last":0.48199999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.42699999999999999,"15min":0.437},"min":{"1min":0.33200000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.53900000000000003,"5min":0.60099999999999998,"15min":0.746},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.43099999999999999,"5min":0.45300000000000001,"15min":0.46300000000000002},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27400000000000002},"max":{"1min":0.56100000000000005,"5min":0.63300000000000001,"15min":0.68000000000000005},"last":0.51700000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.61099999999999999,"15min":0.61799999999999999},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.66400000000000003,"5min":0.92500000000000004,"15min":0.95999999999999996},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.626,"15min":0.63},"min":{"1min":0.46400000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.80800000000000005,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.63400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.63,"15min":0.63400000000000001},"min":{"1min":0.437,"5min":0.437,"15min":0.42699999999999999},"max":{"1min":0.77200000000000002,"5min":0.92100000000000004,"15min":1.0409999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.70199999999999996,"15min":0.67400000000000004},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.82199999999999995,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.745}]},{"osd":6,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45200000000000001,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79700000000000004,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.65800000000000003,"15min":0.66500000000000004},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.89300000000000002,"5min":1.028,"15min":1.028},"last":0.58299999999999996}]},{"osd":7,"last update":"Tue Nov 28 04:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.66300000000000003,"15min":0.66900000000000004},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.78600000000000003,"5min":1.008,"15min":1.008},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.46899999999999997,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.84599999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.75700000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575964,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063996,"statfs":{"total":96636764160,"available":96321531904,"internally_reserved":0,"allocated":2617344,"data_stored":1321751,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.55600000000000005}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.44600000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.59199999999999997}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.40799999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.52300000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575966,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2568,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2629632,"data_stored":1328430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.53300000000000003,"15min":0.54400000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.65400000000000003,"5min":0.75900000000000001,"15min":0.75900000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55100000000000005,"15min":0.56299999999999994},"min":{"1min":0.40600000000000003,"5min":0.36499999999999999,"15min":0.33700000000000002},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.57199999999999995}]},{"osd":1,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.58399999999999996,"15min":0.59199999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.372},"max":{"1min":0.65200000000000002,"5min":0.79900000000000004,"15min":0.82199999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.45800000000000002,"5min":0.38400000000000001,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.66800000000000004}]},{"osd":2,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60699999999999998,"15min":0.60399999999999998},"min":{"1min":0.40400000000000003,"5min":0.39300000000000002,"15min":0.375},"max":{"1min":0.67400000000000004,"5min":0.78100000000000003,"15min":0.79900000000000004},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63100000000000001,"15min":0.63400000000000001},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.40400000000000003},"max":{"1min":0.73999999999999999,"5min":0.83099999999999996,"15min":0.86399999999999999},"last":0.69499999999999995}]},{"osd":3,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.56799999999999995,"15min":0.57199999999999995},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58199999999999996,"15min":0.59399999999999997},"min":{"1min":0.40000000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.72599999999999998,"5min":0.81200000000000006,"15min":0.83599999999999997},"last":0.67900000000000005}]},{"osd":4,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.45400000000000001,"15min":0.45900000000000002},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.60399999999999998,"5min":0.70099999999999996,"15min":0.88200000000000001},"last":0.33600000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.55500000000000005,"15min":0.56200000000000006},"min":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.318},"max":{"1min":0.68999999999999995,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.435}]},{"osd":6,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.498,"15min":0.505},"min":{"1min":0.317,"5min":0.317,"15min":0.30599999999999999},"max":{"1min":0.71599999999999997,"5min":0.77300000000000002,"15min":0.86699999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57599999999999996,"15min":0.58099999999999996},"min":{"1min":0.41999999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.746,"5min":0.78100000000000003,"15min":0.92100000000000004},"last":0.53800000000000003}]},{"osd":7,"last update":"Tue Nov 28 04:25:32 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.35199999999999998,"5min":0.33500000000000002,"15min":0.32000000000000001},"max":{"1min":0.71699999999999997,"5min":0.77300000000000002,"15min":0.90300000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55600000000000005,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.76000000000000001,"5min":0.80600000000000005,"15min":0.85799999999999998},"last":0.44900000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":770,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":892,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1446,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:23.042 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-11-28T04:26:23.042 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-11-28T04:26:23.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 Processor -- start 2023-11-28T04:26:23.193 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 -- start start 2023-11-28T04:26:23.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c34158320 0x7f4c341587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c34158cc0 con 0x7f4c34098d10 2023-11-28T04:26:23.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c340c72d0 con 0x7f4c34158320 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.192+0000 7f4c39e24700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c34064570 con 0x7f4c340c6190 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c34158320 0x7f4c341587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f4c34158320 0x7f4c341587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39080/0 (socket says 172.21.15.73:39080) 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 -- 172.21.15.73:0/626012427 learned_addr learned my addr 172.21.15.73:0/626012427 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 -- 172.21.15.73:0/626012427 >> v2:172.21.15.150:3300/0 conn(0x7f4c340c6190 msgr2=0x7f4c340c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c32ffd700 1 --2- 172.21.15.73:0/626012427 >> v2:172.21.15.150:3300/0 conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 --2- 172.21.15.73:0/626012427 >> v2:172.21.15.150:3300/0 conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 -- 172.21.15.73:0/626012427 >> v1:172.21.15.150:6789/0 conn(0x7f4c34098d10 legacy=0x7f4c34063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 -- 172.21.15.73:0/626012427 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c34154bc0 con 0x7f4c34158320 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c33fff700 1 --2- 172.21.15.73:0/626012427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341587d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 rx=0x7f4c340c5b40 tx=0x7f4c2c0090f0).ready entity=mon.0 client_cookie=c589b29c903711da server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c327fc700 1 -- 172.21.15.73:0/626012427 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00b5a0 con 0x7f4c34158320 2023-11-28T04:26:23.196 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c327fc700 1 -- 172.21.15.73:0/626012427 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c00b700 con 0x7f4c34158320 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.193+0000 7f4c327fc700 1 -- 172.21.15.73:0/626012427 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00ba80 con 0x7f4c34158320 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 -- 172.21.15.73:0/626012427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 msgr2=0x7f4c341587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 --2- 172.21.15.73:0/626012427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341587d0 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 rx=0x7f4c340c5b40 tx=0x7f4c2c0090f0).stop 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 -- 172.21.15.73:0/626012427 shutdown_connections 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 --2- 172.21.15.73:0/626012427 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341587d0 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 --2- 172.21.15.73:0/626012427 >> v2:172.21.15.150:3300/0 conn(0x7f4c340c6190 0x7f4c340c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 -- 172.21.15.73:0/626012427 >> 172.21.15.73:0/626012427 conn(0x7f4c340cb310 msgr2=0x7f4c340cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 -- 172.21.15.73:0/626012427 shutdown_connections 2023-11-28T04:26:23.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.194+0000 7f4c39e24700 1 -- 172.21.15.73:0/626012427 wait complete. 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.195+0000 7f4c39e24700 1 Processor -- start 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.195+0000 7f4c39e24700 1 -- start start 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c340b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34155d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34157160 con 0x7f4c34158320 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c341575f0 con 0x7f4c34098d10 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c34156550 con 0x7f4c340c6190 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:39090/0 (socket says 172.21.15.73:39090) 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 -- 172.21.15.73:0/2127737281 learned_addr learned my addr 172.21.15.73:0/2127737281 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c32ffd700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34155d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 -- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 msgr2=0x7f4c34155d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c337fe700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c340b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34155d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 -- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 msgr2=0x7f4c340b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c340b1050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 -- 172.21.15.73:0/2127737281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c2c009370 con 0x7f4c34158320 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c33fff700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 rx=0x7f4c2c006df0 tx=0x7f4c2c015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c00ba80 con 0x7f4c34158320 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c2c01c420 con 0x7f4c34158320 2023-11-28T04:26:23.201 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c2c01c7e0 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c34156830 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.196+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c34156df0 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.197+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c341572d0 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.198+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f4c2c01cbd0 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.198+0000 7f4c1b7fe700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 0x7f4c20061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.198+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f4c2c07c090 con 0x7f4c34158320 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.198+0000 7f4c337fe700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 0x7f4c20061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.199+0000 7f4c337fe700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 0x7f4c20061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4c24002d00 tx=0x7f4c24009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.202+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4c2c04d400 con 0x7f4c34158320 2023-11-28T04:26:23.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.432+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f4c34158cc0 con 0x7f4c34158320 2023-11-28T04:26:23.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.435+0000 7f4c1b7fe700 1 -- 172.21.15.73:0/2127737281 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v49) v1 ==== 190+0+0 (secure 0 0 0) 0x7f4c34158cc0 con 0x7f4c34158320 2023-11-28T04:26:23.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 msgr2=0x7f4c20061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 0x7f4c20061610 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 rx=0x7f4c24002d00 tx=0x7f4c24009040).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 msgr2=0x7f4c341478e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 rx=0x7f4c2c006df0 tx=0x7f4c2c015000).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 shutdown_connections 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f4c2005f150 0x7f4c20061610 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f4c34158320 0x7f4c341478e0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4c34098d10 0x7f4c340b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 --2- 172.21.15.73:0/2127737281 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4c340c6190 0x7f4c34155d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 >> 172.21.15.73:0/2127737281 conn(0x7f4c340cb310 msgr2=0x7f4c340c8e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 shutdown_connections 2023-11-28T04:26:23.439 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.437+0000 7f4c39e24700 1 -- 172.21.15.73:0/2127737281 wait complete. 2023-11-28T04:26:23.440 INFO:teuthology.orchestra.run.smithi073.stderr:updated caps for client.0 2023-11-28T04:26:23.450 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:23.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.562+0000 7fb735fab700 1 Processor -- start 2023-11-28T04:26:23.563 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.562+0000 7fb735fab700 1 -- start start 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb735fab700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fb7300cae90 0x7fb7300c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb735fab700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7300a9a90 con 0x7fb7300ca180 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb735fab700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb730154ce0 con 0x7fb7300caab0 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb735fab700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7300aab80 con 0x7fb7300cae90 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb72ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb7300ca180 0x7fb73015e3c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47122/0 (socket says 172.21.15.73:47122) 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb72ffff700 1 -- 172.21.15.73:0/1603892388 learned_addr learned my addr 172.21.15.73:0/1603892388 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb72effd700 1 --2- 172.21.15.73:0/1603892388 >> v2:172.21.15.150:3300/0 conn(0x7fb7300cae90 0x7fb7300c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.563+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3912662092 0 0) 0x7fb7300a9a90 con 0x7fb7300ca180 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7180033f0 con 0x7fb7300ca180 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4285156948 0 0) 0x7fb730154ce0 con 0x7fb7300caab0 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7300a9a90 con 0x7fb7300caab0 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2240634225 0 0) 0x7fb7180033f0 con 0x7fb7300ca180 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 >> v2:172.21.15.150:3300/0 conn(0x7fb7300cae90 msgr2=0x7fb7300c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 --2- 172.21.15.73:0/1603892388 >> v2:172.21.15.150:3300/0 conn(0x7fb7300cae90 0x7fb7300c0ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 >> v1:172.21.15.150:6789/0 conn(0x7fb7300caab0 legacy=0x7fb7300aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7300a8e80 con 0x7fb7300ca180 2023-11-28T04:26:23.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fb7280017f0 con 0x7fb7300ca180 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 >> v1:172.21.15.73:6789/0 conn(0x7fb7300ca180 legacy=0x7fb73015e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb718003a70 0x7fb718003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718004690 0x7fb718004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 0x7fb71800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300aab80 con 0x7fb718005000 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb71800bf60 con 0x7fb718004690 2023-11-28T04:26:23.573 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb71800c0d0 con 0x7fb718003a70 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 0x7fb71800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72effd700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718004690 0x7fb718004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb718003a70 0x7fb718003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 -- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb718003a70 msgr2=0x7fb718003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb718003a70 0x7fb718003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 -- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718004690 msgr2=0x7fb718004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718004690 0x7fb718004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 -- 172.21.15.73:0/1603892388 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7180033f0 con 0x7fb718005000 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb718003a70 0x7fb718003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 -- 172.21.15.73:0/1603892388 reap_dead start 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.564+0000 7fb72ffff700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 0x7fb71800b8e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 rx=0x7fb72800c5c0 tx=0x7fb728007f40).ready entity=mon.0 client_cookie=8a3b414347887fb3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.574 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb72800cdc0 con 0x7fb718005000 2023-11-28T04:26:23.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb728005b20 con 0x7fb718005000 2023-11-28T04:26:23.575 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb72e7fc700 1 -- 172.21.15.73:0/1603892388 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280070d0 con 0x7fb718005000 2023-11-28T04:26:23.577 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 -- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 msgr2=0x7fb71800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 0x7fb71800b8e0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 rx=0x7fb72800c5c0 tx=0x7fb728007f40).stop 2023-11-28T04:26:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 -- 172.21.15.73:0/1603892388 shutdown_connections 2023-11-28T04:26:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 --2- 172.21.15.73:0/1603892388 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb718005000 0x7fb71800b8e0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 -- 172.21.15.73:0/1603892388 >> 172.21.15.73:0/1603892388 conn(0x7fb730094920 msgr2=0x7fb73006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:23.578 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 -- 172.21.15.73:0/1603892388 shutdown_connections 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.565+0000 7fb735fab700 1 -- 172.21.15.73:0/1603892388 wait complete. 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.566+0000 7fb735fab700 1 Processor -- start 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.566+0000 7fb735fab700 1 -- start start 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 0x7fb7300ae220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 0x7fb7300ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300aec00 con 0x7fb730064600 2023-11-28T04:26:23.579 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c6e70 con 0x7fb718005000 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb735fab700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7300c7710 con 0x7fb7301645b0 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56592/0 (socket says 172.21.15.73:56592) 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 -- 172.21.15.73:0/144459752 learned_addr learned my addr 172.21.15.73:0/144459752 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 0x7fb7300ae220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72effd700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 0x7fb7300ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.580 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 -- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 msgr2=0x7fb7300ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 0x7fb7300ae710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 -- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 msgr2=0x7fb7300ae220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 0x7fb7300ae220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72ffff700 1 -- 172.21.15.73:0/144459752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb730065370 con 0x7fb730064600 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 0x7fb7300ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.567+0000 7fb72effd700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 0x7fb7300ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb72ffff700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7fb72800c5c0 tx=0x7fb728015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.581 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7280070d0 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7280079d0 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7300c70c0 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb728013450 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.568+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb730064f70 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.569+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb73006c860 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.569+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb7280137c0 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.569+0000 7fb7177fe700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 0x7fb71c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.569+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fb72807b590 con 0x7fb730064600 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.569+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 0x7fb71c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:23.582 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.570+0000 7fb72f7fe700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 0x7fb71c061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb720000c00 tx=0x7fb720009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:23.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.573+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb72804c900 con 0x7fb730064600 2023-11-28T04:26:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.778+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb7300c7850 con 0x7fb730064600 2023-11-28T04:26:23.779 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.778+0000 7fb7177fe700 1 -- 172.21.15.73:0/144459752 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fb728025020 con 0x7fb730064600 2023-11-28T04:26:23.780 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:23.780 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 msgr2=0x7fb71c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 0x7fb71c061610 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 rx=0x7fb720000c00 tx=0x7fb720009040).stop 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 msgr2=0x7fb730064a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 rx=0x7fb72800c5c0 tx=0x7fb728015000).stop 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 shutdown_connections 2023-11-28T04:26:23.782 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb71c05f150 0x7fb71c061610 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb730064600 0x7fb730064a80 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb718005000 0x7fb7300ae220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 --2- 172.21.15.73:0/144459752 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb7301645b0 0x7fb7300ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:23.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.780+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 >> 172.21.15.73:0/144459752 conn(0x7fb730094920 msgr2=0x7fb7300ac9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.781+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 shutdown_connections 2023-11-28T04:26:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:23.781+0000 7fb735fab700 1 -- 172.21.15.73:0/144459752 wait complete. 2023-11-28T04:26:23.787 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:24.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 76, 'flags': 18, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-11-28T04:26:19.699059+0000', 'modified': '2023-11-28T04:26:21.704836+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9990}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9990': {'gid': 9990, 'name': 'a', 'rank': 0, 'incarnation': 74, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.73:6836/447432356', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.73:6834', 'nonce': 447432356}, {'type': 'v1', 'addr': '172.21.15.73:6836', 'nonce': 447432356}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138314316775423, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [26], 'metadata_pool': 25, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-11-28T04:26:24.196 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-11-28T04:26:24.197 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-11-28T04:26:24.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.311+0000 7f68ecb9e700 1 Processor -- start 2023-11-28T04:26:24.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.311+0000 7f68ecb9e700 1 -- start start 2023-11-28T04:26:24.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68ecb9e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68e80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68ecb9e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e80c72d0 con 0x7f68e8098d10 2023-11-28T04:26:24.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68ecb9e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68e8064570 con 0x7f68e80c6190 2023-11-28T04:26:24.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68ecb9e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8154bc0 con 0x7f68e8158a30 2023-11-28T04:26:24.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68e659c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f68e8098d10 0x7f68e8063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47128/0 (socket says 172.21.15.73:47128) 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68e659c700 1 -- 172.21.15.73:0/2367295393 learned_addr learned my addr 172.21.15.73:0/2367295393 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68e80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174262168 0 0) 0x7f68e80c72d0 con 0x7f68e8098d10 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.312+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68d00033f0 con 0x7f68e8098d10 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 750659599 0 0) 0x7f68d00033f0 con 0x7f68e8098d10 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 msgr2=0x7f68e80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68e80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 >> v1:172.21.15.150:6789/0 conn(0x7f68e80c6190 legacy=0x7f68e8158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80c66c0 con 0x7f68e8098d10 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f68d8001470 con 0x7f68e8098d10 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 >> v1:172.21.15.73:6789/0 conn(0x7f68e8098d10 legacy=0x7f68e8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.317 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68d0003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68d0004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d0004b80 0x7f68d0005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e8154bc0 con 0x7f68d0003910 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68d000b990 con 0x7f68d0004b80 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68d000bb50 con 0x7f68e8158a30 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68d0003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68d0004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d0004b80 0x7f68d0005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 -- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 msgr2=0x7f68d0004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e8158a30 0x7f68d0004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 -- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d0004b80 msgr2=0x7f68d0005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d0004b80 0x7f68d0005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 -- 172.21.15.73:0/2367295393 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68d00033f0 con 0x7f68d0003910 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68d0004b80 0x7f68d0005000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e659c700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68d0003d30 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f68d800c190 tx=0x7f68d8001e50).ready entity=mon.0 client_cookie=f49a885bc8764ba2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8007a20 con 0x7f68d0003910 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d800cac0 con 0x7f68d0003910 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.313+0000 7f68e559a700 1 -- 172.21.15.73:0/2367295393 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d800ce40 con 0x7f68d0003910 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 -- 172.21.15.73:0/2367295393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 msgr2=0x7f68d0003d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/2367295393 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68d0003d30 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 rx=0x7f68d800c190 tx=0x7f68d8001e50).stop 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68e6d9d700 1 -- 172.21.15.73:0/2367295393 reap_dead start 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 -- 172.21.15.73:0/2367295393 shutdown_connections 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 -- 172.21.15.73:0/2367295393 >> 172.21.15.73:0/2367295393 conn(0x7f68e80cb310 msgr2=0x7f68e80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 -- 172.21.15.73:0/2367295393 shutdown_connections 2023-11-28T04:26:24.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.314+0000 7f68ecb9e700 1 -- 172.21.15.73:0/2367295393 wait complete. 2023-11-28T04:26:24.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.315+0000 7f68ecb9e700 1 Processor -- start 2023-11-28T04:26:24.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.315+0000 7f68ecb9e700 1 -- start start 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e80ae2c0 0x7f68e80ae740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e80affa0 0x7f68e80b0420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80c8a10 con 0x7f68d0003910 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80b0ec0 con 0x7f68e80ae2c0 2023-11-28T04:26:24.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68e80b11a0 con 0x7f68e80affa0 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56610/0 (socket says 172.21.15.73:56610) 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 -- 172.21.15.73:0/1450962104 learned_addr learned my addr 172.21.15.73:0/1450962104 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e80affa0 0x7f68e80b0420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 -- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e80affa0 msgr2=0x7f68e80b0420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e80affa0 0x7f68e80b0420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e80ae2c0 0x7f68e80ae740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 -- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e80ae2c0 msgr2=0x7f68e80ae740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e80ae2c0 0x7f68e80ae740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 -- 172.21.15.73:0/1450962104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68e80c86a0 con 0x7f68d0003910 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68e659c700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f68d0005650 tx=0x7f68d8015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.329 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68d67fc700 1 -- 172.21.15.73:0/1450962104 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d80075c0 con 0x7f68d0003910 2023-11-28T04:26:24.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68d67fc700 1 -- 172.21.15.73:0/1450962104 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d80046b0 con 0x7f68d0003910 2023-11-28T04:26:24.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68d67fc700 1 -- 172.21.15.73:0/1450962104 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004a50 con 0x7f68d0003910 2023-11-28T04:26:24.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68e814c500 con 0x7f68d0003910 2023-11-28T04:26:24.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.316+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e80b0ad0 con 0x7f68d0003910 2023-11-28T04:26:24.330 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.318+0000 7f68d67fc700 1 -- 172.21.15.73:0/1450962104 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f68d8020480 con 0x7f68d0003910 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.318+0000 7f68d67fc700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 0x7f68c80615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.318+0000 7f68d67fc700 1 -- 172.21.15.73:0/1450962104 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68d807b100 con 0x7f68d0003910 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.318+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 0x7f68c80615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.319+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 0x7f68c80615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f68dc007a40 tx=0x7f68dc009040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.320+0000 7f68fe536b80 1 Processor -- start 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.320+0000 7f68fe536b80 1 -- start start 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b3600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c067e060 con 0x5621c0474e40 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c05f51f0 con 0x5621c06b8cb0 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c060a440 con 0x5621c0701ea0 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56616/0 (socket says 172.21.15.73:56616) 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 -- 172.21.15.73:0/1952298868 learned_addr learned my addr 172.21.15.73:0/1952298868 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 -- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 msgr2=0x5621c06b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 -- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 msgr2=0x5621c06b3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.321+0000 7f68e659c700 1 -- 172.21.15.73:0/1952298868 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c0603620 con 0x5621c0474e40 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68e659c700 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 rx=0x7f68d807b050 tx=0x7f68d800f000).ready entity=mon.0 client_cookie=3f30a35527a4bb55 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68c77fe700 1 -- 172.21.15.73:0/1952298868 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d804fab0 con 0x5621c0474e40 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68c77fe700 1 -- 172.21.15.73:0/1952298868 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d804f8d0 con 0x5621c0474e40 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68c77fe700 1 -- 172.21.15.73:0/1952298868 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004e40 con 0x5621c0474e40 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 -- 172.21.15.73:0/1952298868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 msgr2=0x5621c06e0d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 rx=0x7f68d807b050 tx=0x7f68d800f000).stop 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 -- 172.21.15.73:0/1952298868 shutdown_connections 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06e0d90 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1952298868 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 -- 172.21.15.73:0/1952298868 >> 172.21.15.73:0/1952298868 conn(0x5621c05224a0 msgr2=0x5621c0701790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 -- 172.21.15.73:0/1952298868 shutdown_connections 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.322+0000 7f68fe536b80 1 -- 172.21.15.73:0/1952298868 wait complete. 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.323+0000 7f68fe536b80 1 -- start start 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.323+0000 7f68fe536b80 1 Processor -- start 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c070d080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c070bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c060a580 con 0x5621c0474e40 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c060a800 con 0x5621c0701ea0 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c05d5e10 con 0x5621c06b8cb0 2023-11-28T04:26:24.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68fe536b80 10 client.0 ll_register_callbacks cb 0x5621c0678460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56628/0 (socket says 172.21.15.73:56628) 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 -- 172.21.15.73:0/1856456881 learned_addr learned my addr 172.21.15.73:0/1856456881 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 -- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 msgr2=0x5621c070d080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c070d080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 -- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 msgr2=0x5621c070bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c070bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 -- 172.21.15.73:0/1856456881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68e80b0ad0 con 0x5621c0474e40 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68e659c700 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 rx=0x7f68d804e2c0 tx=0x7f68d800f000).ready entity=mon.0 client_cookie=3e465a030066510b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.324+0000 7f68c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:26:24.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d80556b0 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68d804f460 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5621c06ac720 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68d804bd60 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 10 client.10128 fetch_fsmap learned FSMap version 76 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5621c05fe9c0 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5621c06ac720 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f68d8031070 con 0x5621c0474e40 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 10 client.10128 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T04:26:24.336 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 10 client.10128 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 20 client.10128 populate_metadata read hostname 'smithi073' 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c070efc0 0x5621c070f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 4 client.10128 mds_command: new command op to 9990 tid=0[{"prefix": "get_command_descriptions"}] 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 --> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5621c0470ca0 con 0x5621c070efc0 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c070efc0 0x5621c070f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68c4ff9700 10 client.10128 ms_handle_connect on v2:172.21.15.73:6834/447432356 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.325+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c070efc0 0x5621c070f3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=885ff06883c34f23 server_cookie=68d43535e493b762 in_seq=0 out_seq=0 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68c4ff9700 1 -- 172.21.15.73:0/1856456881 <== mds.0 v2:172.21.15.73:6834/447432356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17015 (crc 0 0 0) 0x5621c0470ca0 con 0x5621c070efc0 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68c4ff9700 10 client.10128 handle_command_reply: tid=0 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 client.10128 shutdown 2023-11-28T04:26:24.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 msgr2=0x5621c06fe590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 rx=0x7f68d804e2c0 tx=0x7f68d800f000).stop 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 shutdown_connections 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c070efc0 0x5621c070f3a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c070bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68c4ff9700 0 client.10128 ms_handle_reset on v2:172.21.15.73:6834/447432356 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fe590 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1856456881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c070d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 >> 172.21.15.73:0/1856456881 conn(0x5621c05224a0 msgr2=0x5621c06b9130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 shutdown_connections 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 1 -- 172.21.15.73:0/1856456881 wait complete. 2023-11-28T04:26:24.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.326+0000 7f68fe536b80 20 client.10128 trim_cache size 0 max 16384 2023-11-28T04:26:24.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.343+0000 7f68fe536b80 1 Processor -- start 2023-11-28T04:26:24.344 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.343+0000 7f68fe536b80 1 -- start start 2023-11-28T04:26:24.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c06fa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c06e0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c06b52d0 con 0x5621c0474e40 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c06b6c00 con 0x5621c0701ea0 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c06fdf80 con 0x5621c06b8cb0 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56630/0 (socket says 172.21.15.73:56630) 2023-11-28T04:26:24.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 -- 172.21.15.73:0/1717230378 learned_addr learned my addr 172.21.15.73:0/1717230378 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c06fa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c06e0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 -- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 msgr2=0x5621c06fa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c06fa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 -- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 msgr2=0x5621c06e0550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c06e0550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 -- 172.21.15.73:0/1717230378 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5621c06ac340 con 0x5621c0474e40 2023-11-28T04:26:24.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c06e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68e659c700 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f68d800ca90 tx=0x7f68d8037000).ready entity=mon.0 client_cookie=6958748b3736857b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68c7fff700 1 -- 172.21.15.73:0/1717230378 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d8079590 con 0x5621c0474e40 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68c7fff700 1 -- 172.21.15.73:0/1717230378 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d804f460 con 0x5621c0474e40 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.344+0000 7f68c7fff700 1 -- 172.21.15.73:0/1717230378 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d804bd60 con 0x5621c0474e40 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- 172.21.15.73:0/1717230378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 msgr2=0x5621c06fcc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 rx=0x7f68d800ca90 tx=0x7f68d8037000).stop 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- 172.21.15.73:0/1717230378 shutdown_connections 2023-11-28T04:26:24.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c0701ea0 0x5621c06e0550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c06fcc10 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 --2- 172.21.15.73:0/1717230378 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c06b8cb0 0x5621c06fa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- 172.21.15.73:0/1717230378 >> 172.21.15.73:0/1717230378 conn(0x5621c05224a0 msgr2=0x5621c06b5730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- 172.21.15.73:0/1717230378 shutdown_connections 2023-11-28T04:26:24.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- 172.21.15.73:0/1717230378 wait complete. 2023-11-28T04:26:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 -- start start 2023-11-28T04:26:24.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.345+0000 7f68fe536b80 1 Processor -- start 2023-11-28T04:26:24.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b4b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c0763400 con 0x5621c0474e40 2023-11-28T04:26:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x5621c07630e0 con 0x5621c06b8cb0 2023-11-28T04:26:24.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x5621c06b5060 con 0x5621c0701ea0 2023-11-28T04:26:24.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68fe536b80 10 client.0 ll_register_callbacks cb 0x5621c0678460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-11-28T04:26:24.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68e659c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56636/0 (socket says 172.21.15.73:56636) 2023-11-28T04:26:24.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68e659c700 1 -- 172.21.15.73:0/3308366194 learned_addr learned my addr 172.21.15.73:0/3308366194 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.346+0000 7f68e6d9d700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 -- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 msgr2=0x5621c06b4b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b4b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 -- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 msgr2=0x5621c06b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 -- 172.21.15.73:0/3308366194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5621c05fe9c0 con 0x5621c0474e40 2023-11-28T04:26:24.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:24.363 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e659c700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 rx=0x7f68d804fa40 tx=0x7f68d8043070).ready entity=mon.0 client_cookie=6cbd8b3c6d0becc6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.73:3300/0 2023-11-28T04:26:24.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68d804bd60 con 0x5621c0474e40 2023-11-28T04:26:24.364 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mon.0 v2:172.21.15.73:3300/0 2 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68d8079590 con 0x5621c0474e40 2023-11-28T04:26:24.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f68d804f8d0 con 0x5621c0474e40 2023-11-28T04:26:24.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_get_version_reply(handle=1 version=76) v2 ==== 24+0+0 (secure 0 0 0) 0x7f68d804ff10 con 0x5621c0474e40 2023-11-28T04:26:24.365 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 10 client.10134 fetch_fsmap learned FSMap version 76 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,fsmap=76,monmap=2+}) v3 -- 0x5621c05fe710 con 0x5621c0474e40 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mon.0 v2:172.21.15.73:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68d804f8d0 con 0x5621c0474e40 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mon.0 v2:172.21.15.73:3300/0 5 ==== fsmap(e 76) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f68d8079190 con 0x5621c0474e40 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 10 client.10134 fetch_fsmap finished waiting for FSMap version 76 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 10 client.10134 resolve_mds: resolved name 'a' to daemon mds.a 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 20 client.10134 populate_metadata read hostname 'smithi073' 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c06b6c30 0x5621c06cf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 4 client.10134 mds_command: new command op to 9990 tid=0[{"prefix": "status"}] 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 --> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5621c04a7ca0 con 0x5621c06b6c30 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.347+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c06b6c30 0x5621c06cf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68c4ff9700 10 client.10134 ms_handle_connect on v2:172.21.15.73:6834/447432356 2023-11-28T04:26:24.366 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68e5d9b700 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c06b6c30 0x5621c06cf6c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).ready entity=mds.0 client_cookie=d31b0128d769b9e2 server_cookie=7a3c27be9dbbffd7 in_seq=0 out_seq=0 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68c4ff9700 1 -- 172.21.15.73:0/3308366194 <== mds.0 v2:172.21.15.73:6834/447432356 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x5621c04a7ca0 con 0x5621c06b6c30 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68c4ff9700 10 client.10134 handle_command_reply: tid=0 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 client.10134 shutdown 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 msgr2=0x5621c075f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 rx=0x7f68d804fa40 tx=0x7f68d8043070).stop 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 shutdown_connections 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:6834/447432356,v1:172.21.15.73:6836/447432356] conn(0x5621c06b6c30 0x5621c06cf6c0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x5621c0701ea0 0x5621c06b4b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68c4ff9700 0 client.10134 ms_handle_reset on v2:172.21.15.73:6834/447432356 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x5621c0474e40 0x5621c075f6f0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 --2- 172.21.15.73:0/3308366194 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x5621c06b8cb0 0x5621c06b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 >> 172.21.15.73:0/3308366194 conn(0x5621c05224a0 msgr2=0x5621c06b56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 shutdown_connections 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 1 -- 172.21.15.73:0/3308366194 wait complete. 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.348+0000 7f68fe536b80 20 client.10134 trim_cache size 0 max 16384 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "cluster_fsid": "c55698a5-e315-44c7-976c-effa5ffffb99", 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "whoami": 0, 2023-11-28T04:26:24.368 INFO:teuthology.orchestra.run.smithi073.stdout: "id": 9990, 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "want_state": "up:active", 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "state": "up:active", 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "fs_name": "cephfs", 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "rank_uptime": 4.6486934189999998, 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "mdsmap_epoch": 76, 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch": 212, 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "osdmap_epoch_barrier": 212, 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout: "uptime": 9.1697621530000006 2023-11-28T04:26:24.369 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:24.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 msgr2=0x7f68c80615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 0x7f68c80615e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 rx=0x7f68dc007a40 tx=0x7f68dc009040).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 msgr2=0x7f68e814c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 rx=0x7f68d0005650 tx=0x7f68d8015000).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 shutdown_connections 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68c805f120 0x7f68c80615e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f68e80affa0 0x7f68e80b0420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f68d0003910 0x7f68e814c010 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 --2- 172.21.15.73:0/1450962104 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f68e80ae2c0 0x7f68e80ae740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 >> 172.21.15.73:0/1450962104 conn(0x7f68e80cb310 msgr2=0x7f68e8155460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 shutdown_connections 2023-11-28T04:26:24.371 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.349+0000 7f68ecb9e700 1 -- 172.21.15.73:0/1450962104 wait complete. 2023-11-28T04:26:24.372 DEBUG:teuthology.orchestra.run.smithi073:> ceph fs ls 2023-11-28T04:26:24.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.471+0000 7fafcb8ad700 1 Processor -- start 2023-11-28T04:26:24.472 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.471+0000 7fafcb8ad700 1 -- start start 2023-11-28T04:26:24.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafcb8ad700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafc4152190 0x7fafc40bf1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafcb8ad700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc40bed20 con 0x7fafc40671a0 2023-11-28T04:26:24.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafcb8ad700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafc40be430 con 0x7fafc40c3670 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafcb8ad700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafc40be6d0 con 0x7fafc4152190 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafc9649700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fafc4152190 0x7fafc40bf1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafc9e4a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fafc40671a0 0x7fafc40a8c20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47144/0 (socket says 172.21.15.73:47144) 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.472+0000 7fafc9e4a700 1 -- 172.21.15.73:0/1460064138 learned_addr learned my addr 172.21.15.73:0/1460064138 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafc9649700 1 -- 172.21.15.73:0/1460064138 >> v1:172.21.15.150:6789/0 conn(0x7fafc40c3670 legacy=0x7fafc4066a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafc9649700 1 -- 172.21.15.73:0/1460064138 >> v1:172.21.15.73:6789/0 conn(0x7fafc40671a0 legacy=0x7fafc40a8c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafbbfff700 1 -- 172.21.15.73:0/1460064138 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 325334503 0 0) 0x7fafc40bed20 con 0x7fafc40671a0 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafc9649700 1 -- 172.21.15.73:0/1460064138 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc40be9b0 con 0x7fafc4152190 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafc9649700 1 --2- 172.21.15.73:0/1460064138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc4152190 0x7fafc40bf1b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 rx=0x7fafc0009760 tx=0x7fafc0009060).ready entity=mon.0 client_cookie=9908bd16354b9486 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafbbfff700 1 -- 172.21.15.73:0/1460064138 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000b520 con 0x7fafc4152190 2023-11-28T04:26:24.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafbbfff700 1 -- 172.21.15.73:0/1460064138 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000b680 con 0x7fafc4152190 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafbbfff700 1 -- 172.21.15.73:0/1460064138 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000ba00 con 0x7fafc4152190 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1460064138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc4152190 msgr2=0x7fafc40bf1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1460064138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc4152190 0x7fafc40bf1b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 rx=0x7fafc0009760 tx=0x7fafc0009060).stop 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1460064138 shutdown_connections 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1460064138 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc4152190 0x7fafc40bf1b0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1460064138 >> 172.21.15.73:0/1460064138 conn(0x7fafc40c9070 msgr2=0x7fafc40c9480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1460064138 shutdown_connections 2023-11-28T04:26:24.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.473+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1460064138 wait complete. 2023-11-28T04:26:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.474+0000 7fafcb8ad700 1 Processor -- start 2023-11-28T04:26:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 -- start start 2023-11-28T04:26:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc40c3670 0x7fafc41436d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc4152190 0x7fafc41502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40c0290 con 0x7fafc40671a0 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40c0570 con 0x7fafc4152190 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.475+0000 7fafcb8ad700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafc40c0850 con 0x7fafc40c3670 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56648/0 (socket says 172.21.15.73:56648) 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 -- 172.21.15.73:0/1949255478 learned_addr learned my addr 172.21.15.73:0/1949255478 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9e4a700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc4152190 0x7fafc41502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc8e48700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc40c3670 0x7fafc41436d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 -- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc40c3670 msgr2=0x7fafc41436d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc40c3670 0x7fafc41436d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.479 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 -- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc4152190 msgr2=0x7fafc41502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc4152190 0x7fafc41502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 -- 172.21.15.73:0/1949255478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafc00092a0 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafc9649700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 rx=0x7fafc0009eb0 tx=0x7fafc0017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc000b7f0 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafc000b950 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafc40c0b30 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafc001b620 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.476+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafc4143bc0 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.477+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafc40475e0 con 0x7fafc40671a0 2023-11-28T04:26:24.480 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.477+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fafc001ba10 con 0x7fafc40671a0 2023-11-28T04:26:24.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.478+0000 7fafb97fa700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:24.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.478+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7fafc007bf00 con 0x7fafc40671a0 2023-11-28T04:26:24.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.478+0000 7fafc8e48700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:24.481 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.478+0000 7fafc8e48700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 0x7fafa8061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fafc40c3110 tx=0x7fafb4008040).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:24.483 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.482+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fafc004d1d0 con 0x7fafc40671a0 2023-11-28T04:26:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.687+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fafc406c380 con 0x7fafc40671a0 2023-11-28T04:26:24.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.687+0000 7fafb97fa700 1 -- 172.21.15.73:0/1949255478 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v76) v1 ==== 53+0+73 (secure 0 0 0) 0x7fafc00507f0 con 0x7fafc40671a0 2023-11-28T04:26:24.690 INFO:teuthology.orchestra.run.smithi073.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-11-28T04:26:24.691 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 msgr2=0x7fafa8061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 0x7fafa8061610 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 rx=0x7fafc40c3110 tx=0x7fafb4008040).stop 2023-11-28T04:26:24.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 msgr2=0x7fafc4064950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:24.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 rx=0x7fafc0009eb0 tx=0x7fafc0017000).stop 2023-11-28T04:26:24.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 shutdown_connections 2023-11-28T04:26:24.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fafa805f150 0x7fafa8061610 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.693 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fafc4152190 0x7fafc41502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fafc40671a0 0x7fafc4064950 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 --2- 172.21.15.73:0/1949255478 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fafc40c3670 0x7fafc41436d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:24.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 >> 172.21.15.73:0/1949255478 conn(0x7fafc40c9070 msgr2=0x7fafc414f660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 shutdown_connections 2023-11-28T04:26:24.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:24.689+0000 7fafcb8ad700 1 -- 172.21.15.73:0/1949255478 wait complete. 2023-11-28T04:26:24.701 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-11-28T04:26:24.701 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-11-28T04:26:24.701 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi073.front.sepia.ceph.com 2023-11-28T04:26:24.701 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-11-28T04:26:24.702 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-11-28T04:26:24.702 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-11-28T04:26:24.702 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-11-28T04:26:24.702 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-11-28T04:26:24.702 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:26:24.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.702 DEBUG:teuthology.orchestra.run.smithi073:> ip addr 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout: inet 127.0.0.1/8 scope host lo 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 ::1/128 scope host 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.717 INFO:teuthology.orchestra.run.smithi073.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:ea brd ff:ff:ff:ff:ff:ff 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f0 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:d8:d9:eb brd ff:ff:ff:ff:ff:ff 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: altname enp7s0f1 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7c brd ff:ff:ff:ff:ff:ff 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 0c:c4:7a:8f:ce:7d brd ff:ff:ff:ff:ff:ff 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: inet 172.21.15.73/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft 36895sec preferred_lft 36895sec 2023-11-28T04:26:24.718 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::ec4:7aff:fe8f:ce7d/64 scope link 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::8488:feff:fe8e:a7d6/64 scope link 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 1a:75:a1:74:31:95 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::1875:a1ff:fe74:3195/64 scope link 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.719 INFO:teuthology.orchestra.run.smithi073.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-11-28T04:26:24.720 INFO:teuthology.orchestra.run.smithi073.stdout: link/ether 72:2c:17:d0:d9:06 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T04:26:24.720 INFO:teuthology.orchestra.run.smithi073.stdout: inet6 fe80::702c:17ff:fed0:d906/64 scope link 2023-11-28T04:26:24.720 INFO:teuthology.orchestra.run.smithi073.stdout: valid_lft forever preferred_lft forever 2023-11-28T04:26:24.720 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.720 DEBUG:teuthology.orchestra.run.smithi073:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-11-28T04:26:24.780 INFO:teuthology.orchestra.run.smithi073.stdout:1 2023-11-28T04:26:24.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.782 DEBUG:teuthology.orchestra.run.smithi073:> route 2023-11-28T04:26:24.837 INFO:teuthology.orchestra.run.smithi073.stdout:Kernel IP routing table 2023-11-28T04:26:24.838 INFO:teuthology.orchestra.run.smithi073.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-11-28T04:26:24.838 INFO:teuthology.orchestra.run.smithi073.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-11-28T04:26:24.838 INFO:teuthology.orchestra.run.smithi073.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-11-28T04:26:24.838 INFO:teuthology.orchestra.run.smithi073.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-11-28T04:26:24.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.838 DEBUG:teuthology.orchestra.run.smithi073:> (cd / && exec bash -c ' 2023-11-28T04:26:24.838 DEBUG:teuthology.orchestra.run.smithi073:> set -e 2023-11-28T04:26:24.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-11-28T04:26:24.838 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-11-28T04:26:24.839 DEBUG:teuthology.orchestra.run.smithi073:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-11-28T04:26:24.839 DEBUG:teuthology.orchestra.run.smithi073:> ') 2023-11-28T04:26:24.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.938 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:26:24.952 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T04:26:24.953 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T04:26:24.953 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-11-28T04:26:24.953 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:24.953 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link set brx.0 up 2023-11-28T04:26:25.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:25.017 DEBUG:teuthology.orchestra.run.smithi073:> mkdir -p /home/ubuntu/cephtest/mnt.0 2023-11-28T04:26:25.071 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-11-28T04:26:25.072 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:26:25.152 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:26:25.152 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:26:25.177 INFO:teuthology.orchestra.run.smithi073.stdout:parsing options: rw,norequire_active_mds,name=0,conf=/etc/ceph/ceph.conf,mds_namespace=cephfs,norbytes 2023-11-28T04:26:25.177 INFO:teuthology.orchestra.run.smithi073.stdout:mount.ceph: options "norequire_active_mds,name=0,mds_namespace=cephfs,norbytes" will pass to kernel. 2023-11-28T04:26:25.179 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-11-28T04:26:25.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:25.179 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-11-28T04:26:25.244 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-11-28T04:26:25.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.396+0000 7f3683b6d700 1 Processor -- start 2023-11-28T04:26:25.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.396+0000 7f3683b6d700 1 -- start start 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.397+0000 7f3683b6d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c063ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.397+0000 7f3683b6d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f367c155c60 0x7f367c154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.397+0000 7f3683b6d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f367c0675f0 con 0x7f367c1554c0 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.397+0000 7f3683b6d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f367c067a80 con 0x7f367c155c60 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.397+0000 7f3683b6d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c066760 con 0x7f367c0bdbd0 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f367c155c60 0x7f367c154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681909700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c063ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f368210a700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f367c1554c0 0x7f367c147bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:34380/0 (socket says 172.21.15.73:34380) 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f368210a700 1 -- 172.21.15.73:0/4130831479 learned_addr learned my addr 172.21.15.73:0/4130831479 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 -- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 msgr2=0x7f367c063ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 --2- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c063ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 -- 172.21.15.73:0/4130831479 >> v1:172.21.15.150:6789/0 conn(0x7f367c1554c0 legacy=0x7f367c147bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3680907700 1 -- 172.21.15.73:0/4130831479 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2148106245 0 0) 0x7f367c0675f0 con 0x7f367c1554c0 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 -- 172.21.15.73:0/4130831479 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c063fa0 con 0x7f367c155c60 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3681108700 1 --2- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c154fd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f366c009760 tx=0x7f366c009060).ready entity=mon.1 client_cookie=1edf9bcfd95edb72 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3680907700 1 -- 172.21.15.73:0/4130831479 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00f510 con 0x7f367c155c60 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3680907700 1 -- 172.21.15.73:0/4130831479 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c00f670 con 0x7f367c155c60 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.398+0000 7f3680907700 1 -- 172.21.15.73:0/4130831479 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00f9f0 con 0x7f367c155c60 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 -- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 msgr2=0x7f367c154fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 --2- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c154fd0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 rx=0x7f366c009760 tx=0x7f366c009060).stop 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 -- 172.21.15.73:0/4130831479 shutdown_connections 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 --2- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c063ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 --2- 172.21.15.73:0/4130831479 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c154fd0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 -- 172.21.15.73:0/4130831479 >> 172.21.15.73:0/4130831479 conn(0x7f367c0c4200 msgr2=0x7f367c0c4610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 -- 172.21.15.73:0/4130831479 shutdown_connections 2023-11-28T04:26:25.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.399+0000 7f3683b6d700 1 -- 172.21.15.73:0/4130831479 wait complete. 2023-11-28T04:26:25.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.400+0000 7f3683b6d700 1 Processor -- start 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.400+0000 7f3683b6d700 1 -- start start 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c0ab360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c0a90d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c161700 con 0x7f367c1554c0 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0ab970 con 0x7f367c155c60 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f367c0abc50 con 0x7f367c0bdbd0 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56656/0 (socket says 172.21.15.73:56656) 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 -- 172.21.15.73:0/1290120881 learned_addr learned my addr 172.21.15.73:0/1290120881 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681909700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c0ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f368210a700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c0a90d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 -- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 msgr2=0x7f367c0ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c0ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 -- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 msgr2=0x7f367c0a90d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c0a90d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 -- 172.21.15.73:0/1290120881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366c0092a0 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3681108700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 rx=0x7f367c06a510 tx=0x7f366c017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c00f7e0 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f366c00f940 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f366c01c670 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.401+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f367c0abf30 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.402+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f367c0a84c0 con 0x7f367c1554c0 2023-11-28T04:26:25.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.403+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f367c06c860 con 0x7f367c1554c0 2023-11-28T04:26:25.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.403+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f366c01ca60 con 0x7f367c1554c0 2023-11-28T04:26:25.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.403+0000 7f3671ffb700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 0x7f3660061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.403+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f366c07bdc0 con 0x7f367c1554c0 2023-11-28T04:26:25.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.403+0000 7f3681909700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 0x7f3660061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.404+0000 7f3681909700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 0x7f3660061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3678006fd0 tx=0x7f3678006f40).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.408+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f366c04d090 con 0x7f367c1554c0 2023-11-28T04:26:25.597 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.596+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f367c0c4f00 con 0x7f366005f150 2023-11-28T04:26:25.598 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.597+0000 7f3671ffb700 1 -- 172.21.15.73:0/1290120881 <== mgr.9807 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f367c0c4f00 con 0x7f366005f150 2023-11-28T04:26:25.599 INFO:teuthology.orchestra.run.smithi073.stdout:[ 2023-11-28T04:26:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: { 2023-11-28T04:26:25.599 INFO:teuthology.orchestra.run.smithi073.stdout: "name": "cephfs" 2023-11-28T04:26:25.600 INFO:teuthology.orchestra.run.smithi073.stdout: } 2023-11-28T04:26:25.600 INFO:teuthology.orchestra.run.smithi073.stdout:] 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 msgr2=0x7f3660061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 0x7f3660061610 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 rx=0x7f3678006fd0 tx=0x7f3678006f40).stop 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 msgr2=0x7f367c0a7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 rx=0x7f367c06a510 tx=0x7f366c017000).stop 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 shutdown_connections 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f366005f150 0x7f3660061610 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f367c155c60 0x7f367c0a90d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f367c0bdbd0 0x7f367c0ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 --2- 172.21.15.73:0/1290120881 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f367c1554c0 0x7f367c0a7fd0 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 >> 172.21.15.73:0/1290120881 conn(0x7f367c0c4200 msgr2=0x7f367c152b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.599+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 shutdown_connections 2023-11-28T04:26:25.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.600+0000 7f3683b6d700 1 -- 172.21.15.73:0/1290120881 wait complete. 2023-11-28T04:26:25.613 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-11-28T04:26:25.624 INFO:tasks.ceph.mds.a.smithi073.stderr:2023-11-28T04:26:25.624+0000 7fd923066700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:26:25.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.726+0000 7f6907b7e700 1 Processor -- start 2023-11-28T04:26:25.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.726+0000 7f6907b7e700 1 -- start start 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f6907b7e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000a8280 0x7f69000a8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f6907b7e700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 0x7f6900155500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f6907b7e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6900067770 con 0x7f690014e600 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f6907b7e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6900159180 con 0x7f69000a8280 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f6907b7e700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69000be8d0 con 0x7f6900158160 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f690591a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000a8280 0x7f69000a8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f690591a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f69000a8280 0x7f69000a8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56670/0 (socket says 172.21.15.73:56670) 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.727+0000 7f690591a700 1 -- 172.21.15.73:0/1594142710 learned_addr learned my addr 172.21.15.73:0/1594142710 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6905119700 1 --2- 172.21.15.73:0/1594142710 >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 0x7f6900155500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f690591a700 1 -- 172.21.15.73:0/1594142710 >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 msgr2=0x7f6900155500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f690591a700 1 --2- 172.21.15.73:0/1594142710 >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 0x7f6900155500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f690591a700 1 -- 172.21.15.73:0/1594142710 >> v1:172.21.15.150:6789/0 conn(0x7f690014e600 legacy=0x7f690014ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f690591a700 1 -- 172.21.15.73:0/1594142710 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6900158600 con 0x7f69000a8280 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6905119700 1 --2- 172.21.15.73:0/1594142710 >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 0x7f6900155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f690591a700 1 --2- 172.21.15.73:0/1594142710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000a8280 0x7f69000a8660 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7f69000be860 tx=0x7f68f00090f0).ready entity=mon.0 client_cookie=fc0430037dcb8027 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6904918700 1 -- 172.21.15.73:0/1594142710 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000b600 con 0x7f69000a8280 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6904918700 1 -- 172.21.15.73:0/1594142710 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f000b760 con 0x7f69000a8280 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6904918700 1 -- 172.21.15.73:0/1594142710 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f000bae0 con 0x7f69000a8280 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6907b7e700 1 -- 172.21.15.73:0/1594142710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000a8280 msgr2=0x7f69000a8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.728+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1594142710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000a8280 0x7f69000a8660 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 rx=0x7f69000be860 tx=0x7f68f00090f0).stop 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 -- 172.21.15.73:0/1594142710 shutdown_connections 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1594142710 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f69000a8280 0x7f69000a8660 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1594142710 >> v2:172.21.15.150:3300/0 conn(0x7f6900158160 0x7f6900155500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 -- 172.21.15.73:0/1594142710 >> 172.21.15.73:0/1594142710 conn(0x7f6900062700 msgr2=0x7f6900062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:25.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 -- 172.21.15.73:0/1594142710 shutdown_connections 2023-11-28T04:26:25.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.729+0000 7f6907b7e700 1 -- 172.21.15.73:0/1594142710 wait complete. 2023-11-28T04:26:25.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 Processor -- start 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 -- start start 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69000a8280 0x7f69000a9bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6900158160 0x7f69000c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69000aada0 con 0x7f690014e600 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69000c2b70 con 0x7f6900158160 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.730+0000 7f6907b7e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69000aa190 con 0x7f69000a8280 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56680/0 (socket says 172.21.15.73:56680) 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 -- 172.21.15.73:0/1080619736 learned_addr learned my addr 172.21.15.73:0/1080619736 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:25.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f690611b700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6900158160 0x7f69000c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f690591a700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69000a8280 0x7f69000a9bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 -- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69000a8280 msgr2=0x7f69000a9bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69000a8280 0x7f69000a9bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 -- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6900158160 msgr2=0x7f69000c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6900158160 0x7f69000c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 -- 172.21.15.73:0/1080619736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f0009370 con 0x7f690014e600 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6905119700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7f68fc00d890 tx=0x7f68fc008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc011590 con 0x7f690014e600 2023-11-28T04:26:25.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68fc0116f0 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69000aa470 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68fc011a70 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.731+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69000aaa30 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.732+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69000aaf10 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.732+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 97) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f68fc012420 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.733+0000 7f68f5ffb700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 0x7f68e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.733+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f68fc079b50 con 0x7f690014e600 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.733+0000 7f690591a700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 0x7f68e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:25.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.733+0000 7f690591a700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 0x7f68e4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f68f0009fd0 tx=0x7f68f0017000).ready entity=mgr.9807 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:25.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.737+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f68fc04adc0 con 0x7f690014e600 2023-11-28T04:26:25.944 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:25.943+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f690006c860 con 0x7f690014e600 2023-11-28T04:26:26.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.737+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 98) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f68fc077920 con 0x7f690014e600 2023-11-28T04:26:26.739 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.738+0000 7f68f5ffb700 1 -- 172.21.15.73:0/1080619736 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v98) v1 ==== 92+0+0 (secure 0 0 0) 0x7f68fc0731a0 con 0x7f690014e600 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.740+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 msgr2=0x7f68e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 0x7f68e4061610 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 rx=0x7f68f0009fd0 tx=0x7f68f0017000).stop 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 msgr2=0x7f690014d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 rx=0x7f68fc00d890 tx=0x7f68fc008040).stop 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 shutdown_connections 2023-11-28T04:26:26.742 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f68e405f150 0x7f68e4061610 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6900158160 0x7f69000c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f69000a8280 0x7f69000a9bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 --2- 172.21.15.73:0/1080619736 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f690014e600 0x7f690014d1e0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 >> 172.21.15.73:0/1080619736 conn(0x7f6900062700 msgr2=0x7f69000a7760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 shutdown_connections 2023-11-28T04:26:26.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.741+0000 7f6907b7e700 1 -- 172.21.15.73:0/1080619736 wait complete. 2023-11-28T04:26:26.755 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-11-28T04:26:26.881 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:26.880+0000 7f2dcb844680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:26.883 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:26.882+0000 7fad6b6a7680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:26.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.914+0000 7f298e975700 1 Processor -- start 2023-11-28T04:26:26.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.914+0000 7f298e975700 1 -- start start 2023-11-28T04:26:26.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298e975700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29880c86f0 0x7f29880a72e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298e975700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2988062700 0x7f2988062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298e975700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29880630a0 con 0x7f29881547f0 2023-11-28T04:26:26.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298e975700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880c7e30 con 0x7f29880c86f0 2023-11-28T04:26:26.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298e975700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29880bcf40 con 0x7f2988062700 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29880c86f0 0x7f29880a72e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f29880c86f0 0x7f29880a72e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56708/0 (socket says 172.21.15.73:56708) 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 -- 172.21.15.73:0/1452057020 learned_addr learned my addr 172.21.15.73:0/1452057020 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f298cf12700 1 --2- 172.21.15.73:0/1452057020 >> v2:172.21.15.150:3300/0 conn(0x7f2988062700 0x7f2988062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 -- 172.21.15.73:0/1452057020 >> v2:172.21.15.150:3300/0 conn(0x7f2988062700 msgr2=0x7f2988062bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:26.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 --2- 172.21.15.73:0/1452057020 >> v2:172.21.15.150:3300/0 conn(0x7f2988062700 0x7f2988062bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 -- 172.21.15.73:0/1452057020 >> v1:172.21.15.150:6789/0 conn(0x7f29881547f0 legacy=0x7f29880c6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 -- 172.21.15.73:0/1452057020 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29880c6b20 con 0x7f29880c86f0 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.915+0000 7f29877fe700 1 --2- 172.21.15.73:0/1452057020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29880c86f0 0x7f29880a72e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 rx=0x7f298806aac0 tx=0x7f29780090f0).ready entity=mon.0 client_cookie=131dee8057b307e8 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f2986ffd700 1 -- 172.21.15.73:0/1452057020 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800b600 con 0x7f29880c86f0 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f2986ffd700 1 -- 172.21.15.73:0/1452057020 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297800b760 con 0x7f29880c86f0 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f2986ffd700 1 -- 172.21.15.73:0/1452057020 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297800bb00 con 0x7f29880c86f0 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 -- 172.21.15.73:0/1452057020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29880c86f0 msgr2=0x7f29880a72e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 --2- 172.21.15.73:0/1452057020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29880c86f0 0x7f29880a72e0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 rx=0x7f298806aac0 tx=0x7f29780090f0).stop 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 -- 172.21.15.73:0/1452057020 shutdown_connections 2023-11-28T04:26:26.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 --2- 172.21.15.73:0/1452057020 >> v2:172.21.15.150:3300/0 conn(0x7f2988062700 0x7f2988062bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 --2- 172.21.15.73:0/1452057020 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f29880c86f0 0x7f29880a72e0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 -- 172.21.15.73:0/1452057020 >> 172.21.15.73:0/1452057020 conn(0x7f29880642d0 msgr2=0x7f29880c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:26.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 -- 172.21.15.73:0/1452057020 shutdown_connections 2023-11-28T04:26:26.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.916+0000 7f298e975700 1 -- 172.21.15.73:0/1452057020 wait complete. 2023-11-28T04:26:26.921 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.917+0000 7f298e975700 1 Processor -- start 2023-11-28T04:26:26.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.917+0000 7f298e975700 1 -- start start 2023-11-28T04:26:26.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29880c86f0 0x7f298813a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29881547f0 0x7f29880654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2988150070 con 0x7f2988062700 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f298814f460 con 0x7f29881547f0 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298e975700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f298814f740 con 0x7f29880c86f0 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f2987fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f2987fff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56712/0 (socket says 172.21.15.73:56712) 2023-11-28T04:26:26.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f2987fff700 1 -- 172.21.15.73:0/4003307442 learned_addr learned my addr 172.21.15.73:0/4003307442 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f29877fe700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29880c86f0 0x7f298813a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.918+0000 7f298cf12700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29881547f0 0x7f29880654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 -- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29880c86f0 msgr2=0x7f298813a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29880c86f0 0x7f298813a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 -- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29881547f0 msgr2=0x7f29880654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:26.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29881547f0 0x7f29880654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 -- 172.21.15.73:0/4003307442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2978009370 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f2987fff700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f297c00c440 tx=0x7f297c007dc0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c012400 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f297c012560 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f298814fa20 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f297c012900 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.919+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f298813b420 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.920+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f298806c860 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.920+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 98) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f297c01b3b0 con 0x7f2988062700 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.921+0000 7f296ffff700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 0x7f2974061660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.921+0000 7f29877fe700 1 -- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 msgr2=0x7f2974061660 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:26.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.921+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f297c079480 con 0x7f2988062700 2023-11-28T04:26:26.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.921+0000 7f29877fe700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 0x7f2974061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:26.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:26.925+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f297c048af0 con 0x7f2988062700 2023-11-28T04:26:26.944 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:26.943+0000 7f2dcb844680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:26.950 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:26.949+0000 7fad6b6a7680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:27.098 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.096+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f29880630a0 con 0x7f2988062700 2023-11-28T04:26:27.101 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.100+0000 7f296ffff700 1 -- 172.21.15.73:0/4003307442 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v43)=0 v43) v1 ==== 138+0+0 (secure 0 0 0) 0x7f29880630a0 con 0x7f2988062700 2023-11-28T04:26:27.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 msgr2=0x7f2974061660 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:27.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 0x7f2974061660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:27.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 msgr2=0x7f298813af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 rx=0x7f297c00c440 tx=0x7f297c007dc0).stop 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 shutdown_connections 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f297405f1a0 0x7f2974061660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f29881547f0 0x7f29880654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2988062700 0x7f298813af30 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 --2- 172.21.15.73:0/4003307442 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f29880c86f0 0x7f298813a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 >> 172.21.15.73:0/4003307442 conn(0x7f29880642d0 msgr2=0x7f2988151300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 shutdown_connections 2023-11-28T04:26:27.104 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.102+0000 7f298e975700 1 -- 172.21.15.73:0/4003307442 wait complete. 2023-11-28T04:26:27.116 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:27.116 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-11-28T04:26:27.140 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add --fs cephfs snap_test_dir1 1M 2023-11-28T04:26:27.219 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:27.219+0000 7f2dcb844680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:27.237 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:27.236+0000 7fad6b6a7680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:27.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.306+0000 7f5b073fe700 1 Processor -- start 2023-11-28T04:26:27.307 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.306+0000 7f5b073fe700 1 -- start start 2023-11-28T04:26:27.309 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b073fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b00154520 0x7f5b0006a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b073fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f5b00154900 0x7f5b0006a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b073fe700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b00067840 con 0x7f5b000bcef0 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b073fe700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b000c50e0 con 0x7f5b00154520 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b073fe700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b00066c30 con 0x7f5b00154900 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b00154520 0x7f5b0006a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f5b00154520 0x7f5b0006a3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56724/0 (socket says 172.21.15.73:56724) 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 -- 172.21.15.73:0/3095894665 learned_addr learned my addr 172.21.15.73:0/3095894665 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b04999700 1 --2- 172.21.15.73:0/3095894665 >> v2:172.21.15.150:3300/0 conn(0x7f5b00154900 0x7f5b0006a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 -- 172.21.15.73:0/3095894665 >> v2:172.21.15.150:3300/0 conn(0x7f5b00154900 msgr2=0x7f5b0006a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 --2- 172.21.15.73:0/3095894665 >> v2:172.21.15.150:3300/0 conn(0x7f5b00154900 0x7f5b0006a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.310 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 -- 172.21.15.73:0/3095894665 >> v1:172.21.15.150:6789/0 conn(0x7f5b000bcef0 legacy=0x7f5b00062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 -- 172.21.15.73:0/3095894665 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b00066f10 con 0x7f5b00154520 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5b0519a700 1 --2- 172.21.15.73:0/3095894665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b0006a3b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 rx=0x7f5b000c2290 tx=0x7f5afc0090f0).ready entity=mon.0 client_cookie=b08fa08208a9ade0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.307+0000 7f5af7fff700 1 -- 172.21.15.73:0/3095894665 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00b600 con 0x7f5b00154520 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5af7fff700 1 -- 172.21.15.73:0/3095894665 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5afc00b760 con 0x7f5b00154520 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5af7fff700 1 -- 172.21.15.73:0/3095894665 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5afc00bb00 con 0x7f5b00154520 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 -- 172.21.15.73:0/3095894665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 msgr2=0x7f5b0006a3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 --2- 172.21.15.73:0/3095894665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b0006a3b0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 rx=0x7f5b000c2290 tx=0x7f5afc0090f0).stop 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 -- 172.21.15.73:0/3095894665 shutdown_connections 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 --2- 172.21.15.73:0/3095894665 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b0006a3b0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 --2- 172.21.15.73:0/3095894665 >> v2:172.21.15.150:3300/0 conn(0x7f5b00154900 0x7f5b0006a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.311 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 -- 172.21.15.73:0/3095894665 >> 172.21.15.73:0/3095894665 conn(0x7f5b00063810 msgr2=0x7f5b00063c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 -- 172.21.15.73:0/3095894665 shutdown_connections 2023-11-28T04:26:27.312 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.308+0000 7f5b073fe700 1 -- 172.21.15.73:0/3095894665 wait complete. 2023-11-28T04:26:27.314 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.309+0000 7f5b073fe700 1 Processor -- start 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.309+0000 7f5b073fe700 1 -- start start 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b000bcef0 0x7f5b000a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b00154900 0x7f5b000c0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000a6780 con 0x7f5b00154520 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000a5b70 con 0x7f5b00154900 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.310+0000 7f5b073fe700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b000a5e50 con 0x7f5b000bcef0 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:27.315 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56740/0 (socket says 172.21.15.73:56740) 2023-11-28T04:26:27.316 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 -- 172.21.15.73:0/682897265 learned_addr learned my addr 172.21.15.73:0/682897265 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b000bcef0 0x7f5b000a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b0599b700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b00154900 0x7f5b000c0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b000bcef0 msgr2=0x7f5b000a4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b000bcef0 0x7f5b000a4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b00154900 msgr2=0x7f5b000c0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b00154900 0x7f5b000c0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5afc009370 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b04999700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f5af000d890 tx=0x7f5af0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af00125f0 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5af0012750 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b000a6130 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5af0012ad0 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.311+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b00153c20 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.312+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b0006c860 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.313+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 98) v1 ==== 75463+0+0 (secure 0 0 0) 0x7f5af0013480 con 0x7f5b00154520 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.313+0000 7f5af57fa700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:27.319 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.313+0000 7f5b0519a700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.313+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6121+0+0 (secure 0 0 0) 0x7f5af0079fc0 con 0x7f5b00154520 2023-11-28T04:26:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.313+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:27.320 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.317+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f5af00a8af0 con 0x7f5b00154520 2023-11-28T04:26:27.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.514+0000 7f5b0519a700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:27.515 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.514+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:27.518 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.517+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b00069f60 con 0x7f5ae405f1d0 2023-11-28T04:26:27.860 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:27.859+0000 7f2dcb844680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:27.868 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:27.867+0000 7fad6b6a7680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:27.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.915+0000 7f5b0519a700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:27.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:27.915+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.800000 2023-11-28T04:26:27.932 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:27.931+0000 7f2dcb844680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:27.940 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:27.939+0000 7fad6b6a7680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:28.119 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.118+0000 7f2dcb844680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:28.125 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.124+0000 7fad6b6a7680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:28.542 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.540+0000 7f2dcb844680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:28.543 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.542+0000 7fad6b6a7680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:28.645 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.644+0000 7f2dcb844680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:28.646 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.645+0000 7fad6b6a7680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:28.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:28.716+0000 7f5b0519a700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:28.718 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:28.716+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 1.600000 2023-11-28T04:26:28.798 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.797+0000 7f2dcb844680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:28.799 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.798+0000 7fad6b6a7680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:28.874 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.873+0000 7fad6b6a7680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:28.874 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.873+0000 7f2dcb844680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:28.941 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:28.940+0000 7fad6b6a7680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:28.941 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:28.941+0000 7f2dcb844680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:29.202 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:29.198+0000 7f2dcb844680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:29.202 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:29.198+0000 7fad6b6a7680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:29.381 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:29.380+0000 7fad6b6a7680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:29.387 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:29.383+0000 7f2dcb844680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:30.028 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.027+0000 7fad6b6a7680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:30.037 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.036+0000 7f2dcb844680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:30.096 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.095+0000 7fad6b6a7680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:30.105 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.104+0000 7f2dcb844680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:30.171 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.170+0000 7fad6b6a7680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:30.181 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.180+0000 7f2dcb844680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:30.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:30.317+0000 7f5b0519a700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:30.318 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:30.317+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 3.200000 2023-11-28T04:26:30.319 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.318+0000 7fad6b6a7680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:30.329 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.328+0000 7f2dcb844680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:30.391 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.390+0000 7fad6b6a7680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:30.400 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.399+0000 7f2dcb844680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:30.506 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.505+0000 7fad6b6a7680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:30.515 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.516+0000 7f2dcb844680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:30.615 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.614+0000 7fad6b6a7680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:30.623 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.623+0000 7f2dcb844680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:30.904 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.903+0000 7fad6b6a7680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:30.914 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.914+0000 7f2dcb844680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:30.976 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:30.974+0000 7fad6b6a7680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:30.985 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:30.985+0000 7f2dcb844680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:30.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:30.993+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 99) v1 ==== 75287+0+0 (secure 0 0 0) 0x7f5af0077d90 con 0x7f5b00154520 2023-11-28T04:26:30.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:30.993+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:30.994 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:30.993+0000 7f5af57fa700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:31.061 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7face2e33700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7face2e33700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7face2e33700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7face2e33700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7face2e33700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7facdfe2d700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7facdfe2d700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.062 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7facdfe2d700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.063 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7facdfe2d700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.063 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:31.058+0000 7facdfe2d700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:31.996 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:31.995+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f5af0078110 con 0x7f5b00154520 2023-11-28T04:26:31.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:31.996+0000 7f5af57fa700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:31.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:31.996+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f5b00069f60 con 0x7f5ae405f1d0 2023-11-28T04:26:31.997 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:31.996+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:31.998 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:31.998+0000 7f5b0519a700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061610 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5afc009fd0 tx=0x7f5afc017000).ready entity=mgr.10167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.016 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.015+0000 7f5af57fa700 1 -- 172.21.15.73:0/682897265 <== mgr.10167 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f5b00069f60 con 0x7f5ae405f1d0 2023-11-28T04:26:32.017 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule set for path snap_test_dir1 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 msgr2=0x7f5ae4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061610 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 rx=0x7f5afc009fd0 tx=0x7f5afc017000).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 msgr2=0x7f5b000a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 rx=0x7f5af000d890 tx=0x7f5af0008040).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 shutdown_connections 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f5ae405f1d0 0x7f5ae4061610 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f5b00154900 0x7f5b000c0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f5b000bcef0 0x7f5b000a4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 --2- 172.21.15.73:0/682897265 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f5b00154520 0x7f5b000a5330 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.019 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 >> 172.21.15.73:0/682897265 conn(0x7f5b00063810 msgr2=0x7f5b0014c6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.017+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 shutdown_connections 2023-11-28T04:26:32.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.018+0000 7f5b073fe700 1 -- 172.21.15.73:0/682897265 wait complete. 2023-11-28T04:26:32.031 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-11-28T04:26:32.031 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add --fs cephfs snap_test_dir1 1M 2023-11-28T04:26:32.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.149+0000 7f09f1a5b700 1 Processor -- start 2023-11-28T04:26:32.150 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.149+0000 7f09f1a5b700 1 -- start start 2023-11-28T04:26:32.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09f1a5b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f09ec158a30 0x7f09ec0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09f1a5b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec0c72d0 con 0x7f09ec0c6190 2023-11-28T04:26:32.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09f1a5b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09ec064570 con 0x7f09ec098d10 2023-11-28T04:26:32.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09f1a5b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09ec154bc0 con 0x7f09ec158a30 2023-11-28T04:26:32.154 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09e3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f09ec0c6190 0x7f09ec158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47166/0 (socket says 172.21.15.73:47166) 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09e3fff700 1 -- 172.21.15.73:0/2809214337 learned_addr learned my addr 172.21.15.73:0/2809214337 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/2809214337 >> v2:172.21.15.150:3300/0 conn(0x7f09ec158a30 0x7f09ec0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205449183 0 0) 0x7f09ec0c72d0 con 0x7f09ec0c6190 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09d80033f0 con 0x7f09ec0c6190 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3309808218 0 0) 0x7f09d80033f0 con 0x7f09ec0c6190 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 >> v2:172.21.15.150:3300/0 conn(0x7f09ec158a30 msgr2=0x7f09ec0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 --2- 172.21.15.73:0/2809214337 >> v2:172.21.15.150:3300/0 conn(0x7f09ec158a30 0x7f09ec0c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.150+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 >> v1:172.21.15.150:6789/0 conn(0x7f09ec098d10 legacy=0x7f09ec063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0c66c0 con 0x7f09ec0c6190 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f09d4001470 con 0x7f09ec0c6190 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 >> v1:172.21.15.73:6789/0 conn(0x7f09ec0c6190 legacy=0x7f09ec158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.155 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09d80038e0 0x7f09d8003cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09d8004690 0x7f09d8004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09d800b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec154bc0 con 0x7f09d8005000 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09d800be00 con 0x7f09d8004690 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09d800bfc0 con 0x7f09d80038e0 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09d800b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eaffd700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09d80038e0 0x7f09d8003cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09e3fff700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09d8004690 0x7f09d8004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 -- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09d80038e0 msgr2=0x7f09d8003cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09d80038e0 0x7f09d8003cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 -- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09d8004690 msgr2=0x7f09d8004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09d8004690 0x7f09d8004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 -- 172.21.15.73:0/2809214337 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09d80033f0 con 0x7f09d8005000 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09e3fff700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09d8004690 0x7f09d8004b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 -- 172.21.15.73:0/2809214337 reap_dead start 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09d800b8e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7f09ec062cb0 tx=0x7f09e400b040).ready entity=mon.0 client_cookie=d3edb52fe67159c0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4009ac0 con 0x7f09d8005000 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e400f040 con 0x7f09d8005000 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.151+0000 7f09ea7fc700 1 -- 172.21.15.73:0/2809214337 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e400ec50 con 0x7f09d8005000 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 -- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 msgr2=0x7f09d800b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09d800b8e0 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 rx=0x7f09ec062cb0 tx=0x7f09e400b040).stop 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 -- 172.21.15.73:0/2809214337 shutdown_connections 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/2809214337 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09d800b8e0 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 -- 172.21.15.73:0/2809214337 >> 172.21.15.73:0/2809214337 conn(0x7f09ec0cb310 msgr2=0x7f09ec0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.157 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 -- 172.21.15.73:0/2809214337 shutdown_connections 2023-11-28T04:26:32.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.152+0000 7f09f1a5b700 1 -- 172.21.15.73:0/2809214337 wait complete. 2023-11-28T04:26:32.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.153+0000 7f09f1a5b700 1 Processor -- start 2023-11-28T04:26:32.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.153+0000 7f09f1a5b700 1 -- start start 2023-11-28T04:26:32.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 0x7f09ec0a80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 0x7f09ec0aa060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0aed80 con 0x7f09d8005000 2023-11-28T04:26:32.160 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0aa550 con 0x7f09ec0acd80 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09ec0a8900 con 0x7f09ec0a9be0 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56868/0 (socket says 172.21.15.73:56868) 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 -- 172.21.15.73:0/1354289072 learned_addr learned my addr 172.21.15.73:0/1354289072 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 0x7f09ec0aa060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09e3fff700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 0x7f09ec0a80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 -- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 msgr2=0x7f09ec0aa060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 0x7f09ec0aa060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 -- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 msgr2=0x7f09ec0a80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 0x7f09ec0a80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 -- 172.21.15.73:0/1354289072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec0a8be0 con 0x7f09d8005000 2023-11-28T04:26:32.161 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eb7fe700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 0x7f09ec0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09e3fff700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 0x7f09ec0a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09eaffd700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7f09dc002a00 tx=0x7f09dc007b90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc003340 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09dc00f8d0 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09ec0a8ec0 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09dc00fc50 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.154+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09ec0ae930 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09ec0aeec0 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f09dc014450 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09e2ffd700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 0x7f09cc061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f09dc079f30 con 0x7f09d8005000 2023-11-28T04:26:32.162 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09e3fff700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 0x7f09cc061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.156+0000 7f09e3fff700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 0x7f09cc061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f09d4000ff0 tx=0x7f09d4009040).ready entity=mgr.10167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.163 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.160+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f09dc04aaf0 con 0x7f09d8005000 2023-11-28T04:26:32.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.350+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f09ec06c860 con 0x7f09cc05f080 2023-11-28T04:26:32.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.354+0000 7f09e2ffd700 1 -- 172.21.15.73:0/1354289072 <== mgr.10167 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f09ec06c860 con 0x7f09cc05f080 2023-11-28T04:26:32.355 INFO:teuthology.orchestra.run.smithi073.stdout:Retention added to path snap_test_dir1 2023-11-28T04:26:32.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 msgr2=0x7f09cc061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 0x7f09cc061540 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7f09d4000ff0 tx=0x7f09d4009040).stop 2023-11-28T04:26:32.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 msgr2=0x7f09ec0b0e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 rx=0x7f09dc002a00 tx=0x7f09dc007b90).stop 2023-11-28T04:26:32.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 shutdown_connections 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f09cc05f080 0x7f09cc061540 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f09ec0a9be0 0x7f09ec0aa060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f09d8005000 0x7f09ec0b0e90 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 --2- 172.21.15.73:0/1354289072 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09ec0acd80 0x7f09ec0a80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 >> 172.21.15.73:0/1354289072 conn(0x7f09ec0cb310 msgr2=0x7f09ec154f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 shutdown_connections 2023-11-28T04:26:32.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.356+0000 7f09f1a5b700 1 -- 172.21.15.73:0/1354289072 wait complete. 2023-11-28T04:26:32.370 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-11-28T04:26:32.374 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-11-28-04_27_00 in ~28s... 2023-11-28T04:26:32.374 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-11-28T04:26:32.374 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --fs cephfs --format json snap_test_dir1 2023-11-28T04:26:32.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.490+0000 7f2afbe7d700 1 Processor -- start 2023-11-28T04:26:32.499 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.490+0000 7f2afbe7d700 1 -- start start 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2afbe7d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2af4154530 0x7f2af4154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2afbe7d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40c6630 con 0x7f2af40a7ff0 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2afbe7d700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2af40a8470 con 0x7f2af40aa220 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2afbe7d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2af40c6390 con 0x7f2af4154530 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af9418700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2af40a7ff0 0x7f2af40c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47180/0 (socket says 172.21.15.73:47180) 2023-11-28T04:26:32.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af9418700 1 -- 172.21.15.73:0/2560935608 learned_addr learned my addr 172.21.15.73:0/2560935608 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af9c19700 1 --2- 172.21.15.73:0/2560935608 >> v2:172.21.15.150:3300/0 conn(0x7f2af4154530 0x7f2af4154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594051068 0 0) 0x7f2af40c6630 con 0x7f2af40a7ff0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2ae00033f0 con 0x7f2af40a7ff0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.491+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2619184810 0 0) 0x7f2ae00033f0 con 0x7f2af40a7ff0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 >> v2:172.21.15.150:3300/0 conn(0x7f2af4154530 msgr2=0x7f2af4154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 --2- 172.21.15.73:0/2560935608 >> v2:172.21.15.150:3300/0 conn(0x7f2af4154530 0x7f2af4154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 >> v1:172.21.15.150:6789/0 conn(0x7f2af40aa220 legacy=0x7f2af40c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4064230 con 0x7f2af40a7ff0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2ae4001470 con 0x7f2af40a7ff0 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 >> v1:172.21.15.73:6789/0 conn(0x7f2af40a7ff0 legacy=0x7f2af40c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae00038e0 0x7f2ae0003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae0004690 0x7f2ae0004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 0x7f2ae000b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af40c6390 con 0x7f2ae0005000 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae000bf60 con 0x7f2ae00038e0 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ae000c0f0 con 0x7f2ae0004690 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 0x7f2ae000b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af9c19700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae00038e0 0x7f2ae0003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af9418700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae0004690 0x7f2ae0004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 -- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae0004690 msgr2=0x7f2ae0004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ae0004690 0x7f2ae0004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 -- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae00038e0 msgr2=0x7f2ae0003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae00038e0 0x7f2ae0003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 -- 172.21.15.73:0/2560935608 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ae00033f0 con 0x7f2ae0005000 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 -- 172.21.15.73:0/2560935608 reap_dead start 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2afa41a700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 0x7f2ae000b8e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 rx=0x7f2aec001e30 tx=0x7f2aec007f30).ready entity=mon.0 client_cookie=9d6f85d893bd28c3 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec006a20 con 0x7f2ae0005000 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec006b80 con 0x7f2ae0005000 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.492+0000 7f2af8c17700 1 -- 172.21.15.73:0/2560935608 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00c5d0 con 0x7f2ae0005000 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 -- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 msgr2=0x7f2ae000b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 0x7f2ae000b8e0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 rx=0x7f2aec001e30 tx=0x7f2aec007f30).stop 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 -- 172.21.15.73:0/2560935608 shutdown_connections 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/2560935608 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ae0005000 0x7f2ae000b8e0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 -- 172.21.15.73:0/2560935608 >> 172.21.15.73:0/2560935608 conn(0x7f2af4094930 msgr2=0x7f2af406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.503 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 -- 172.21.15.73:0/2560935608 shutdown_connections 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.493+0000 7f2afbe7d700 1 -- 172.21.15.73:0/2560935608 wait complete. 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.494+0000 7f2afbe7d700 1 Processor -- start 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.494+0000 7f2afbe7d700 1 -- start start 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 0x7f2af4148860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 0x7f2af4164590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4148d50 con 0x7f2af4150930 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af4167da0 con 0x7f2ae0005000 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2af41537c0 con 0x7f2af4153340 2023-11-28T04:26:32.504 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56880/0 (socket says 172.21.15.73:56880) 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 -- 172.21.15.73:0/1001397875 learned_addr learned my addr 172.21.15.73:0/1001397875 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2af9418700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 0x7f2af4164590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2af9c19700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 0x7f2af4148860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 -- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 msgr2=0x7f2af4164590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 0x7f2af4164590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 -- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 msgr2=0x7f2af4148860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 0x7f2af4148860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 -- 172.21.15.73:0/1001397875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af4164d80 con 0x7f2af4150930 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2af9c19700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 0x7f2af4148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2af9418700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 0x7f2af4164590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:32.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afa41a700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7f2af4154e00 tx=0x7f2aec016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec00c5d0 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aec013c90 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af4165060 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.495+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aec006a20 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.496+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af4164a80 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.497+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af4165580 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.497+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f2aec01d950 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.497+0000 7f2ae9ffb700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.497+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f2aec07ba80 con 0x7f2af4150930 2023-11-28T04:26:32.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.497+0000 7f2af9c19700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.498+0000 7f2af9c19700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 0x7f2ad8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2af0006f10 tx=0x7f2af0008040).ready entity=mgr.10167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.502+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2af4165580 con 0x7f2af4150930 2023-11-28T04:26:32.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.689+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2af4167b70 con 0x7f2ad805f080 2023-11-28T04:26:32.692 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.691+0000 7f2ae9ffb700 1 -- 172.21.15.73:0/1001397875 <== mgr.10167 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f2af4167b70 con 0x7f2ad805f080 2023-11-28T04:26:32.692 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:32.692 INFO:teuthology.orchestra.run.smithi073.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-28T04:26:32.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 msgr2=0x7f2ad8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.694 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 0x7f2ad8061540 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7f2af0006f10 tx=0x7f2af0008040).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 msgr2=0x7f2af4150d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 rx=0x7f2af4154e00 tx=0x7f2aec016000).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 shutdown_connections 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f2ad805f080 0x7f2ad8061540 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2af4150930 0x7f2af4150d90 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ae0005000 0x7f2af4148860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 --2- 172.21.15.73:0/1001397875 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2af4153340 0x7f2af4164590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 >> 172.21.15.73:0/1001397875 conn(0x7f2af4094930 msgr2=0x7f2af40a8ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 shutdown_connections 2023-11-28T04:26:32.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.693+0000 7f2afbe7d700 1 -- 172.21.15.73:0/1001397875 wait complete. 2023-11-28T04:26:32.706 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-11-28T04:26:32.707 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-11-28T04:26:32.707 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove --fs cephfs snap_test_dir1 2023-11-28T04:26:32.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.823+0000 7f03f76e5700 1 Processor -- start 2023-11-28T04:26:32.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.823+0000 7f03f76e5700 1 -- start start 2023-11-28T04:26:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f76e5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 0x7f03f01582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f76e5700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f0159210 con 0x7f03f00aa1c0 2023-11-28T04:26:32.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f76e5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03f00c0c70 con 0x7f03f0154610 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f76e5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f0158970 con 0x7f03f00ab3b0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f5c82700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f03f00aa1c0 0x7f03f00c1100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:47194/0 (socket says 172.21.15.73:47194) 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f5c82700 1 -- 172.21.15.73:0/1415162120 learned_addr learned my addr 172.21.15.73:0/1415162120 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f4c80700 1 --2- 172.21.15.73:0/1415162120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 0x7f03f01582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1146877125 0 0) 0x7f03f0159210 con 0x7f03f00aa1c0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03dc0033f0 con 0x7f03f00aa1c0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345928185 0 0) 0x7f03f00c0c70 con 0x7f03f0154610 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03f0159210 con 0x7f03f0154610 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f4c80700 1 -- 172.21.15.73:0/1415162120 >> v1:172.21.15.150:6789/0 conn(0x7f03f0154610 legacy=0x7f03f00c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f4c80700 1 -- 172.21.15.73:0/1415162120 >> v1:172.21.15.73:6789/0 conn(0x7f03f00aa1c0 legacy=0x7f03f00c1100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f4c80700 1 -- 172.21.15.73:0/1415162120 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03f0158c50 con 0x7f03f00ab3b0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.824+0000 7f03f4c80700 1 --2- 172.21.15.73:0/1415162120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 0x7f03f01582e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f03e0009790 tx=0x7f03e0009060).ready entity=mon.2 client_cookie=693a5544c480677b server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.827 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e000f510 con 0x7f03f00ab3b0 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03e000f670 con 0x7f03f00ab3b0 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03e7fff700 1 -- 172.21.15.73:0/1415162120 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03e000f9f0 con 0x7f03f00ab3b0 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 -- 172.21.15.73:0/1415162120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 msgr2=0x7f03f01582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 --2- 172.21.15.73:0/1415162120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 0x7f03f01582e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0x7f03e0009790 tx=0x7f03e0009060).stop 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 -- 172.21.15.73:0/1415162120 shutdown_connections 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 --2- 172.21.15.73:0/1415162120 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f00ab3b0 0x7f03f01582e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 -- 172.21.15.73:0/1415162120 >> 172.21.15.73:0/1415162120 conn(0x7f03f00655a0 msgr2=0x7f03f0069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 -- 172.21.15.73:0/1415162120 shutdown_connections 2023-11-28T04:26:32.828 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.825+0000 7f03f76e5700 1 -- 172.21.15.73:0/1415162120 wait complete. 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.826+0000 7f03f76e5700 1 Processor -- start 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.826+0000 7f03f76e5700 1 -- start start 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f0154610 0x7f03f00b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03f00c2300 0x7f03f00c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00c3760 con 0x7f03f00aa1c0 2023-11-28T04:26:32.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00c3bf0 con 0x7f03f00c2300 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f76e5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03f00c2b50 con 0x7f03f0154610 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.827+0000 7f03f5481700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56890/0 (socket says 172.21.15.73:56890) 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 -- 172.21.15.73:0/754988728 learned_addr learned my addr 172.21.15.73:0/754988728 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5c82700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03f00c2300 0x7f03f00c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f4c80700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f0154610 0x7f03f00b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 -- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f0154610 msgr2=0x7f03f00b0720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f0154610 0x7f03f00b0720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 -- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03f00c2300 msgr2=0x7f03f00c47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03f00c2300 0x7f03f00c47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 -- 172.21.15.73:0/754988728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e00092a0 con 0x7f03f00aa1c0 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f5481700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7f03ec00c810 tx=0x7f03ec0026f0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.832 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00a5c0 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ec00a720 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03f00c2e30 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ec00aac0 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.828+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03f00c33f0 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.829+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03f0158050 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.829+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f03ec01b400 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.830+0000 7f03e57fa700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 0x7f03d4061570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.830+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f03ec07a010 con 0x7f03f00aa1c0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.830+0000 7f03f4c80700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 0x7f03d4061570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:32.833 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.830+0000 7f03f4c80700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 0x7f03d4061570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f03e0009790 tx=0x7f03e0017000).ready entity=mgr.10167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:32.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:32.834+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f03ec04b3a0 con 0x7f03f00aa1c0 2023-11-28T04:26:33.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.023+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 --> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "fs": "cephfs", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f03f00c38a0 con 0x7f03d405f0b0 2023-11-28T04:26:33.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.026+0000 7f03e57fa700 1 -- 172.21.15.73:0/754988728 <== mgr.10167 v2:172.21.15.73:6832/136129 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f03e001f070 con 0x7f03d405f0b0 2023-11-28T04:26:33.027 INFO:teuthology.orchestra.run.smithi073.stdout:Schedule removed for path snap_test_dir1 2023-11-28T04:26:33.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 msgr2=0x7f03d4061570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 0x7f03d4061570 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f03e0009790 tx=0x7f03e0017000).stop 2023-11-28T04:26:33.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 msgr2=0x7f03f01452b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 rx=0x7f03ec00c810 tx=0x7f03ec0026f0).stop 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 shutdown_connections 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f03d405f0b0 0x7f03d4061570 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f03f00c2300 0x7f03f00c47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f03f00aa1c0 0x7f03f01452b0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 --2- 172.21.15.73:0/754988728 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f03f0154610 0x7f03f00b0720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 >> 172.21.15.73:0/754988728 conn(0x7f03f00655a0 msgr2=0x7f03f00ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 shutdown_connections 2023-11-28T04:26:33.030 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.028+0000 7f03f76e5700 1 -- 172.21.15.73:0/754988728 wait complete. 2023-11-28T04:26:33.041 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-11-28T04:26:33.041 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:33.041 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-11-28T04:26:33.057 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:33.057 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-11-28T04:26:33.115 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-11-28T04:26:33.269 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.268+0000 7f7011e4a700 1 Processor -- start 2023-11-28T04:26:33.270 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.268+0000 7f7011e4a700 1 -- start start 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f7011e4a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f700c098d10 0x7f700c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f7011e4a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f700c0c6190 0x7f700c063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f7011e4a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f700c0c72d0 con 0x7f700c064350 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f7011e4a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c154bc0 con 0x7f700c098d10 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f7011e4a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f700c0c66c0 con 0x7f700c0c6190 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f700b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f700c098d10 0x7f700c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f700b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f700c098d10 0x7f700c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56892/0 (socket says 172.21.15.73:56892) 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.269+0000 7f700b7fe700 1 -- 172.21.15.73:0/3743211795 learned_addr learned my addr 172.21.15.73:0/3743211795 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700affd700 1 --2- 172.21.15.73:0/3743211795 >> v2:172.21.15.150:3300/0 conn(0x7f700c0c6190 0x7f700c063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700b7fe700 1 -- 172.21.15.73:0/3743211795 >> v2:172.21.15.150:3300/0 conn(0x7f700c0c6190 msgr2=0x7f700c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:33.272 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700b7fe700 1 --2- 172.21.15.73:0/3743211795 >> v2:172.21.15.150:3300/0 conn(0x7f700c0c6190 0x7f700c063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700b7fe700 1 -- 172.21.15.73:0/3743211795 >> v1:172.21.15.73:6789/0 conn(0x7f700c064350 legacy=0x7f700c158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700b7fe700 1 -- 172.21.15.73:0/3743211795 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f700c0c69a0 con 0x7f700c098d10 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700b7fe700 1 --2- 172.21.15.73:0/3743211795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c098d10 0x7f700c0c7760 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 rx=0x7f700c0c5b40 tx=0x7f6ffc0090f0).ready entity=mon.0 client_cookie=4af915f6a23ed041 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700a7fc700 1 -- 172.21.15.73:0/3743211795 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00b600 con 0x7f700c098d10 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700a7fc700 1 -- 172.21.15.73:0/3743211795 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ffc00b760 con 0x7f700c098d10 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f700a7fc700 1 -- 172.21.15.73:0/3743211795 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ffc00bae0 con 0x7f700c098d10 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 -- 172.21.15.73:0/3743211795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c098d10 msgr2=0x7f700c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 --2- 172.21.15.73:0/3743211795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c098d10 0x7f700c0c7760 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 rx=0x7f700c0c5b40 tx=0x7f6ffc0090f0).stop 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 -- 172.21.15.73:0/3743211795 shutdown_connections 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 --2- 172.21.15.73:0/3743211795 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c098d10 0x7f700c0c7760 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 --2- 172.21.15.73:0/3743211795 >> v2:172.21.15.150:3300/0 conn(0x7f700c0c6190 0x7f700c063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.270+0000 7f7011e4a700 1 -- 172.21.15.73:0/3743211795 >> 172.21.15.73:0/3743211795 conn(0x7f700c0cb310 msgr2=0x7f700c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:33.273 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.271+0000 7f7011e4a700 1 -- 172.21.15.73:0/3743211795 shutdown_connections 2023-11-28T04:26:33.274 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.271+0000 7f7011e4a700 1 -- 172.21.15.73:0/3743211795 wait complete. 2023-11-28T04:26:33.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.272+0000 7f7011e4a700 1 Processor -- start 2023-11-28T04:26:33.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.272+0000 7f7011e4a700 1 -- start start 2023-11-28T04:26:33.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c064350 0x7f700c1561e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f700c098d10 0x7f700c0ad8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c150950 con 0x7f700c0c6190 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0ac090 con 0x7f700c064350 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f700c0abd40 con 0x7f700c098d10 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:56906/0 (socket says 172.21.15.73:56906) 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 -- 172.21.15.73:0/2444711921 learned_addr learned my addr 172.21.15.73:0/2444711921 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700affd700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f700c098d10 0x7f700c0ad8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700b7fe700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c064350 0x7f700c1561e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 -- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f700c098d10 msgr2=0x7f700c0ad8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f700c098d10 0x7f700c0ad8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 -- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c064350 msgr2=0x7f700c1561e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:33.277 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c064350 0x7f700c1561e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 -- 172.21.15.73:0/2444711921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ffc009370 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f700bfff700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 rx=0x7f7004002230 tx=0x7f70040015e0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400e4a0 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f700400e600 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f700c165330 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f700400e980 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.273+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f700c1658f0 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.274+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f700c0ac200 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.275+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 100) v1 ==== 75357+0+0 (secure 0 0 0) 0x7f700400ed30 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.275+0000 7f6ff37fe700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 0x7f6ff8061540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.275+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f7004079710 con 0x7f700c0c6190 2023-11-28T04:26:33.278 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.275+0000 7f700b7fe700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 0x7f6ff8061540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:33.279 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.275+0000 7f700b7fe700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 0x7f6ff8061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6ffc009130 tx=0x7f6ffc017000).ready entity=mgr.10167 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:33.280 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.279+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7004048af0 con 0x7f700c0c6190 2023-11-28T04:26:33.488 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:33.487+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f700c0abeb0 con 0x7f700c0c6190 2023-11-28T04:26:34.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.003+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f7004077600 con 0x7f700c0c6190 2023-11-28T04:26:34.004 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.004+0000 7f6ff37fe700 1 -- 172.21.15.73:0/2444711921 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v101) v1 ==== 93+0+0 (secure 0 0 0) 0x7f7004074a10 con 0x7f700c0c6190 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 msgr2=0x7f6ff8061540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 0x7f6ff8061540 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f6ffc009130 tx=0x7f6ffc017000).stop 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 msgr2=0x7f700c164b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 rx=0x7f7004002230 tx=0x7f70040015e0).stop 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 shutdown_connections 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f6ff805f080 0x7f6ff8061540 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f700c0c6190 0x7f700c164b10 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f700c064350 0x7f700c1561e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 --2- 172.21.15.73:0/2444711921 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f700c098d10 0x7f700c0ad8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 >> 172.21.15.73:0/2444711921 conn(0x7f700c0cb310 msgr2=0x7f700c0ca300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:34.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 shutdown_connections 2023-11-28T04:26:34.009 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.007+0000 7f7011e4a700 1 -- 172.21.15.73:0/2444711921 wait complete. 2023-11-28T04:26:34.024 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-11-28T04:26:34.024 DEBUG:teuthology.orchestra.run.smithi073:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:34.060 DEBUG:teuthology.orchestra.run.smithi150:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-11-28T04:26:34.105 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-11-28T04:26:34.105 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-11-28T04:26:34.106 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-11-28T04:26:34.106 DEBUG:teuthology.orchestra.run.smithi073:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-11-28T04:26:34.143 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:26:34.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:34.143 DEBUG:teuthology.orchestra.run.smithi073:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-11-28T04:26:34.147 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:34.147+0000 7fddf3802680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:34.152 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:34.151+0000 7fd377991680 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-11-28T04:26:34.166 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:34.209 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:34.209+0000 7fddf3802680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:34.217 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:34.216+0000 7fd377991680 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-11-28T04:26:34.323 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.322+0000 7fb37bf65700 1 Processor -- start 2023-11-28T04:26:34.324 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.323+0000 7fb37bf65700 1 -- start start 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.323+0000 7fb37bf65700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb374069900 0x7fb374069d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.323+0000 7fb37bf65700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb37406a270 con 0x7fb3741544f0 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.323+0000 7fb37bf65700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3740c28c0 con 0x7fb374150890 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.323+0000 7fb37bf65700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3741552e0 con 0x7fb374069900 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb379500700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb374069900 0x7fb374069d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb37a502700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb3741544f0 0x7fb3740c2d50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39008/0 (socket says 172.21.15.73:39008) 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb37a502700 1 -- 172.21.15.73:0/3727442489 learned_addr learned my addr 172.21.15.73:0/3727442489 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:34.326 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3618090299 0 0) 0x7fb36c001490 con 0x7fb3741544f0 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb3600033f0 con 0x7fb3741544f0 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb379500700 1 -- 172.21.15.73:0/3727442489 >> v1:172.21.15.150:6789/0 conn(0x7fb374150890 legacy=0x7fb3740bbe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414378022 0 0) 0x7fb3740c28c0 con 0x7fb374150890 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb379500700 1 -- 172.21.15.73:0/3727442489 >> v1:172.21.15.73:6789/0 conn(0x7fb3741544f0 legacy=0x7fb3740c2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb379500700 1 -- 172.21.15.73:0/3727442489 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3740bc540 con 0x7fb374069900 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb379500700 1 --2- 172.21.15.73:0/3727442489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374069900 0x7fb374069d80 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7fb370009760 tx=0x7fb370009060).ready entity=mon.0 client_cookie=d2796060736aa5c7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f510 con 0x7fb374069900 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000f670 con 0x7fb374069900 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.324+0000 7fb378cff700 1 -- 172.21.15.73:0/3727442489 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f9f0 con 0x7fb374069900 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 -- 172.21.15.73:0/3727442489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374069900 msgr2=0x7fb374069d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 --2- 172.21.15.73:0/3727442489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374069900 0x7fb374069d80 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 rx=0x7fb370009760 tx=0x7fb370009060).stop 2023-11-28T04:26:34.327 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 -- 172.21.15.73:0/3727442489 shutdown_connections 2023-11-28T04:26:34.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 --2- 172.21.15.73:0/3727442489 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374069900 0x7fb374069d80 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 -- 172.21.15.73:0/3727442489 >> 172.21.15.73:0/3727442489 conn(0x7fb3740631d0 msgr2=0x7fb3740c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:34.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 -- 172.21.15.73:0/3727442489 shutdown_connections 2023-11-28T04:26:34.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.325+0000 7fb37bf65700 1 -- 172.21.15.73:0/3727442489 wait complete. 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.326+0000 7fb37bf65700 1 Processor -- start 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.326+0000 7fb37bf65700 1 -- start start 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb374069900 0x7fb3740c66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3741544f0 0x7fb3740a7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3740c70a0 con 0x7fb374150890 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3740a8dc0 con 0x7fb374069900 2023-11-28T04:26:34.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37bf65700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3740a9e80 con 0x7fb3741544f0 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35554/0 (socket says 172.21.15.73:35554) 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 -- 172.21.15.73:0/2761292644 learned_addr learned my addr 172.21.15.73:0/2761292644 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379d01700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb374069900 0x7fb3740c66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb37a502700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3741544f0 0x7fb3740a7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3741544f0 msgr2=0x7fb3740a7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3741544f0 0x7fb3740a7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb374069900 msgr2=0x7fb3740c66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb374069900 0x7fb3740c66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 -- 172.21.15.73:0/2761292644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3700092a0 con 0x7fb374150890 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb379500700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 rx=0x7fb370009e00 tx=0x7fb370017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:34.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.327+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37000f7e0 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.328+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb37000f940 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.328+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3740c72a0 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.328+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb37001b660 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.328+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3740aa0d0 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.329+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb374066130 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.329+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fb37001b9d0 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.330+0000 7fb365ffb700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 0x7fb3580617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.330+0000 7fb379d01700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 msgr2=0x7fb3580617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.330+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fb37007bf10 con 0x7fb374150890 2023-11-28T04:26:34.334 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.330+0000 7fb379d01700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 0x7fb3580617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:34.335 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.334+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb37004cf30 con 0x7fb374150890 2023-11-28T04:26:34.485 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:34.485+0000 7fddf3802680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:34.504 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:34.503+0000 7fd377991680 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-11-28T04:26:34.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.531+0000 7fb379d01700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 msgr2=0x7fb3580617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:34.532 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.531+0000 7fb379d01700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 0x7fb3580617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:34.548 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.547+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb37406c860 con 0x7fb374150890 2023-11-28T04:26:34.549 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.547+0000 7fb365ffb700 1 -- 172.21.15.73:0/2761292644 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7fb370050550 con 0x7fb374150890 2023-11-28T04:26:34.549 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:34.549 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 msgr2=0x7fb3580617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 0x7fb3580617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 msgr2=0x7fb3740c6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 rx=0x7fb370009e00 tx=0x7fb370017000).stop 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 shutdown_connections 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fb35805f300 0x7fb3580617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb3741544f0 0x7fb3740a7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb374069900 0x7fb3740c66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 --2- 172.21.15.73:0/2761292644 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb374150890 0x7fb3740c6bb0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 >> 172.21.15.73:0/2761292644 conn(0x7fb3740631d0 msgr2=0x7fb3740c0330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:34.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 shutdown_connections 2023-11-28T04:26:34.552 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.549+0000 7fb37bf65700 1 -- 172.21.15.73:0/2761292644 wait complete. 2023-11-28T04:26:34.552 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:34.563 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:34.726 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.725+0000 7f8b8569c700 1 Processor -- start 2023-11-28T04:26:34.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.725+0000 7f8b8569c700 1 -- start start 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b8569c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f8b80157a40 0x7f8b800648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b8569c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800c12c0 con 0x7f8b80157660 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b8569c700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b800c06b0 con 0x7f8b800c0180 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b8569c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b800c0990 con 0x7f8b80157a40 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b7ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f8b80157660 0x7f8b801589c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39016/0 (socket says 172.21.15.73:39016) 2023-11-28T04:26:34.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b7ffff700 1 -- 172.21.15.73:0/528133664 learned_addr learned my addr 172.21.15.73:0/528133664 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.726+0000 7f8b84e9b700 1 --2- 172.21.15.73:0/528133664 >> v2:172.21.15.150:3300/0 conn(0x7f8b80157a40 0x7f8b800648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 232180746 0 0) 0x7f8b800c12c0 con 0x7f8b80157660 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b680033f0 con 0x7f8b80157660 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3818431812 0 0) 0x7f8b800c06b0 con 0x7f8b800c0180 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8b800c12c0 con 0x7f8b800c0180 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b84e9b700 1 -- 172.21.15.73:0/528133664 >> v1:172.21.15.150:6789/0 conn(0x7f8b800c0180 legacy=0x7f8b800c1750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b84e9b700 1 -- 172.21.15.73:0/528133664 >> v1:172.21.15.73:6789/0 conn(0x7f8b80157660 legacy=0x7f8b801589c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b84e9b700 1 -- 172.21.15.73:0/528133664 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800c0c70 con 0x7f8b80157a40 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b84e9b700 1 --2- 172.21.15.73:0/528133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80157a40 0x7f8b800648d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8b78009e60 tx=0x7f8b78009780).ready entity=mon.1 client_cookie=53438a5621515add server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800e070 con 0x7f8b80157a40 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b7800bab0 con 0x7f8b80157a40 2023-11-28T04:26:34.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.727+0000 7f8b7effd700 1 -- 172.21.15.73:0/528133664 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b7800be30 con 0x7f8b80157a40 2023-11-28T04:26:34.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 -- 172.21.15.73:0/528133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80157a40 msgr2=0x7f8b800648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 --2- 172.21.15.73:0/528133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80157a40 0x7f8b800648d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 rx=0x7f8b78009e60 tx=0x7f8b78009780).stop 2023-11-28T04:26:34.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 -- 172.21.15.73:0/528133664 shutdown_connections 2023-11-28T04:26:34.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 --2- 172.21.15.73:0/528133664 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b80157a40 0x7f8b800648d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 -- 172.21.15.73:0/528133664 >> 172.21.15.73:0/528133664 conn(0x7f8b800cb000 msgr2=0x7f8b800cb410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:34.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 -- 172.21.15.73:0/528133664 shutdown_connections 2023-11-28T04:26:34.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.728+0000 7f8b8569c700 1 -- 172.21.15.73:0/528133664 wait complete. 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.729+0000 7f8b8569c700 1 Processor -- start 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.729+0000 7f8b8569c700 1 -- start start 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b800c0180 0x7f8b800adfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b80157a40 0x7f8b80150ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b801548a0 con 0x7f8b80157660 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80154b80 con 0x7f8b800c0180 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b8569c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b80154e60 con 0x7f8b80157a40 2023-11-28T04:26:34.735 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35570/0 (socket says 172.21.15.73:35570) 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 -- 172.21.15.73:0/2786590443 learned_addr learned my addr 172.21.15.73:0/2786590443 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7ffff700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b800c0180 0x7f8b800adfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b84e9b700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b80157a40 0x7f8b80150ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 -- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b80157a40 msgr2=0x7f8b80150ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b80157a40 0x7f8b80150ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 -- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b800c0180 msgr2=0x7f8b800adfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b800c0180 0x7f8b800adfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 -- 172.21.15.73:0/2786590443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b78009a20 con 0x7f8b80157660 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b7f7fe700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 rx=0x7f8b7000c440 tx=0x7f8b70007c90).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.730+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70012400 con 0x7f8b80157660 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.731+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b70012560 con 0x7f8b80157660 2023-11-28T04:26:34.736 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.731+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b80155140 con 0x7f8b80157660 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.731+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b70012920 con 0x7f8b80157660 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.731+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b800ae4a0 con 0x7f8b80157660 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f8b7001b3b0 con 0x7f8b80157660 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b8006c7c0 con 0x7f8b80157660 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b67fff700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 0x7f8b6c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b7ffff700 1 -- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 msgr2=0x7f8b6c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:34.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f8b70079cf0 con 0x7f8b80157660 2023-11-28T04:26:34.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.732+0000 7f8b7ffff700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 0x7f8b6c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:34.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.736+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8b7004acb0 con 0x7f8b80157660 2023-11-28T04:26:34.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.903+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b800bff60 con 0x7f8b80157660 2023-11-28T04:26:34.905 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.904+0000 7f8b67fff700 1 -- 172.21.15.73:0/2786590443 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+17188 (secure 0 0 0) 0x7f8b7004e2d0 con 0x7f8b80157660 2023-11-28T04:26:34.906 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:34.906 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":213,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:30.988911+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6832/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/3047075693":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/1637465872":"2023-11-29T04:26:30.988881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 msgr2=0x7f8b6c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 0x7f8b6c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 msgr2=0x7f8b80153ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 rx=0x7f8b7000c440 tx=0x7f8b70007c90).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 shutdown_connections 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f8b6c05f300 0x7f8b6c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8b80157a40 0x7f8b80150ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8b800c0180 0x7f8b800adfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 --2- 172.21.15.73:0/2786590443 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8b80157660 0x7f8b80153ea0 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 >> 172.21.15.73:0/2786590443 conn(0x7f8b800cb000 msgr2=0x7f8b800ab330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 shutdown_connections 2023-11-28T04:26:34.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:34.906+0000 7f8b8569c700 1 -- 172.21.15.73:0/2786590443 wait complete. 2023-11-28T04:26:34.924 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': 'device_health_metrics', 'create_time': '2023-11-28T02:52:35.444074+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '203', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr_devicehealth': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-11-28T02:52:37.446721+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 25, 'pool_name': 'cephfs_metadata', 'create_time': '2023-11-28T04:26:18.231141+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '211', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 26, 'pool_name': 'cephfs_data', 'create_time': '2023-11-28T04:26:18.668882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-11-28T04:26:34.925 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get device_health_metrics pg_num 2023-11-28T04:26:35.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.044+0000 7f4368acb700 1 Processor -- start 2023-11-28T04:26:35.045 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.044+0000 7f4368acb700 1 -- start start 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4368acb700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f4364151a20 0x7f436406a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4368acb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43640bd1f0 0x7f43640bd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4368acb700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43640c3470 con 0x7f4364069d10 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4368acb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43640c48f0 con 0x7f43640bd1f0 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4368acb700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43640bdb70 con 0x7f4364151a20 2023-11-28T04:26:35.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43640bd1f0 0x7f43640bd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43640bd1f0 0x7f43640bd680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35574/0 (socket says 172.21.15.73:35574) 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 -- 172.21.15.73:0/3239092515 learned_addr learned my addr 172.21.15.73:0/3239092515 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4361d9b700 1 --2- 172.21.15.73:0/3239092515 >> v2:172.21.15.150:3300/0 conn(0x7f4364151a20 0x7f436406a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 -- 172.21.15.73:0/3239092515 >> v2:172.21.15.150:3300/0 conn(0x7f4364151a20 msgr2=0x7f436406a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 --2- 172.21.15.73:0/3239092515 >> v2:172.21.15.150:3300/0 conn(0x7f4364151a20 0x7f436406a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 -- 172.21.15.73:0/3239092515 >> v1:172.21.15.73:6789/0 conn(0x7f4364069d10 legacy=0x7f43640be1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 -- 172.21.15.73:0/3239092515 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4364151ea0 con 0x7f43640bd1f0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f436159a700 1 -- 172.21.15.73:0/3239092515 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029953653 0 0) 0x7f43640c3470 con 0x7f4364069d10 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f4362d9d700 1 --2- 172.21.15.73:0/3239092515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f43640bd680 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 rx=0x7f4364069370 tx=0x7f435c009840).ready entity=mon.0 client_cookie=af81a0691090da91 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f436159a700 1 -- 172.21.15.73:0/3239092515 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00e070 con 0x7f43640bd1f0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f436159a700 1 -- 172.21.15.73:0/3239092515 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f435c00ba20 con 0x7f43640bd1f0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.045+0000 7f436159a700 1 -- 172.21.15.73:0/3239092515 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f435c00bda0 con 0x7f43640bd1f0 2023-11-28T04:26:35.048 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 -- 172.21.15.73:0/3239092515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 msgr2=0x7f43640bd680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 --2- 172.21.15.73:0/3239092515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f43640bd680 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 rx=0x7f4364069370 tx=0x7f435c009840).stop 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 -- 172.21.15.73:0/3239092515 shutdown_connections 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 --2- 172.21.15.73:0/3239092515 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f43640bd680 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 --2- 172.21.15.73:0/3239092515 >> v2:172.21.15.150:3300/0 conn(0x7f4364151a20 0x7f436406a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 -- 172.21.15.73:0/3239092515 >> 172.21.15.73:0/3239092515 conn(0x7f43640642d0 msgr2=0x7f43640c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 -- 172.21.15.73:0/3239092515 shutdown_connections 2023-11-28T04:26:35.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.046+0000 7f4368acb700 1 -- 172.21.15.73:0/3239092515 wait complete. 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.047+0000 7f4368acb700 1 Processor -- start 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.047+0000 7f4368acb700 1 -- start start 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4364069d10 0x7f43640a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4364151a20 0x7f43641438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43640a6ec0 con 0x7f43640bd1f0 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43641475f0 con 0x7f4364069d10 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4368acb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43640a6be0 con 0x7f4364151a20 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35588/0 (socket says 172.21.15.73:35588) 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 -- 172.21.15.73:0/749110226 learned_addr learned my addr 172.21.15.73:0/749110226 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4362d9d700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4364151a20 0x7f43641438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f436259c700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4364069d10 0x7f43640a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4364151a20 msgr2=0x7f43641438a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4364151a20 0x7f43641438a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4364069d10 msgr2=0x7f43640a6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4364069d10 0x7f43640a6300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.048+0000 7f4361d9b700 1 -- 172.21.15.73:0/749110226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f435c009ae0 con 0x7f43640bd1f0 2023-11-28T04:26:35.052 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f4361d9b700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 rx=0x7f435800e8b0 tx=0x7f4358009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358012600 con 0x7f43640bd1f0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4358012760 con 0x7f43640bd1f0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4364146fa0 con 0x7f43640bd1f0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4358012b00 con 0x7f43640bd1f0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.049+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43640a67f0 con 0x7f43640bd1f0 2023-11-28T04:26:35.053 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.050+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4364146e00 con 0x7f43640bd1f0 2023-11-28T04:26:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.051+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f435801a3b0 con 0x7f43640bd1f0 2023-11-28T04:26:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.051+0000 7f43527fc700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 0x7f43440617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.051+0000 7f436259c700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 msgr2=0x7f43440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.051+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f435807a500 con 0x7f43640bd1f0 2023-11-28T04:26:35.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.051+0000 7f436259c700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 0x7f43440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:35.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.055+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f4364146e00 con 0x7f43640bd1f0 2023-11-28T04:26:35.099 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:35.099+0000 7fddf3802680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:35.155 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:35.146+0000 7fd377991680 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-11-28T04:26:35.171 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:35.171+0000 7fddf3802680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:35.221 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:35.220+0000 7fd377991680 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-11-28T04:26:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.252+0000 7f436259c700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 msgr2=0x7f43440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:35.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.252+0000 7f436259c700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 0x7f43440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:35.261 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.260+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"} v 0) v1 -- 0x7f4364146e00 con 0x7f43640bd1f0 2023-11-28T04:26:35.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.260+0000 7f43527fc700 1 -- 172.21.15.73:0/749110226 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "device_health_metrics", "var": "pg_num"}]=0 v213) v1 ==== 110+0+10 (secure 0 0 0) 0x7f4364146e00 con 0x7f43640bd1f0 2023-11-28T04:26:35.262 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 1 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 msgr2=0x7f43440617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 0x7f43440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 msgr2=0x7f4364142f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 rx=0x7f435800e8b0 tx=0x7f4358009060).stop 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 shutdown_connections 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f434405f300 0x7f43440617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f4364151a20 0x7f43641438a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f4364069d10 0x7f43640a6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.262+0000 7f4368acb700 1 --2- 172.21.15.73:0/749110226 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43640bd1f0 0x7f4364142f90 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.263+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 >> 172.21.15.73:0/749110226 conn(0x7f43640642d0 msgr2=0x7f43640c4a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.263+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 shutdown_connections 2023-11-28T04:26:35.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.263+0000 7f4368acb700 1 -- 172.21.15.73:0/749110226 wait complete. 2023-11-28T04:26:35.277 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-11-28T04:26:35.362 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:35.362+0000 7fddf3802680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:35.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.399+0000 7f43b0f80700 1 Processor -- start 2023-11-28T04:26:35.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.399+0000 7f43b0f80700 1 -- start start 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43ac13bce0 0x7f43ac0c3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f43ac0c5380 0x7f43ac0c5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5e30 0x7f43ac066870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0c62e0 con 0x7f43ac13bce0 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0c7480 con 0x7f43ac0c5380 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43b0f80700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac13c0c0 con 0x7f43ac0c5e30 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43ab7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43ac13bce0 0x7f43ac0c3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43ab7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f43ac13bce0 0x7f43ac0c3510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35602/0 (socket says 172.21.15.73:35602) 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43ab7fe700 1 -- 172.21.15.73:0/2436478210 learned_addr learned my addr 172.21.15.73:0/2436478210 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43abfff700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5e30 0x7f43ac066870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.400+0000 7f43aaffd700 1 --2- 172.21.15.73:0/2436478210 >> v2:172.21.15.150:3300/0 conn(0x7f43ac0c5380 0x7f43ac0c5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 -- 172.21.15.73:0/2436478210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5e30 msgr2=0x7f43ac066870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5e30 0x7f43ac066870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 -- 172.21.15.73:0/2436478210 >> v2:172.21.15.150:3300/0 conn(0x7f43ac0c5380 msgr2=0x7f43ac0c5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 --2- 172.21.15.73:0/2436478210 >> v2:172.21.15.150:3300/0 conn(0x7f43ac0c5380 0x7f43ac0c5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 -- 172.21.15.73:0/2436478210 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0c76d0 con 0x7f43ac13bce0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43ab7fe700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac0c3510 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7f43ac0c9f20 tx=0x7f43a00090f0).ready entity=mon.0 client_cookie=58b8c690b306fa85 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43aa7fc700 1 -- 172.21.15.73:0/2436478210 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000f510 con 0x7f43ac13bce0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43aa7fc700 1 -- 172.21.15.73:0/2436478210 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a000f670 con 0x7f43ac13bce0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.401+0000 7f43aa7fc700 1 -- 172.21.15.73:0/2436478210 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a000fa10 con 0x7f43ac13bce0 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 -- 172.21.15.73:0/2436478210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 msgr2=0x7f43ac0c3510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac0c3510 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 rx=0x7f43ac0c9f20 tx=0x7f43a00090f0).stop 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 -- 172.21.15.73:0/2436478210 shutdown_connections 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac0c3510 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2436478210 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5e30 0x7f43ac066870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2436478210 >> v2:172.21.15.150:3300/0 conn(0x7f43ac0c5380 0x7f43ac0c5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 -- 172.21.15.73:0/2436478210 >> 172.21.15.73:0/2436478210 conn(0x7f43ac062700 msgr2=0x7f43ac062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 -- 172.21.15.73:0/2436478210 shutdown_connections 2023-11-28T04:26:35.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.402+0000 7f43b0f80700 1 -- 172.21.15.73:0/2436478210 wait complete. 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.403+0000 7f43b0f80700 1 Processor -- start 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.403+0000 7f43b0f80700 1 -- start start 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5380 0x7f43ac165140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac0c5e30 0x7f43ac13a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac064cc0 con 0x7f43ac13bce0 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac0c2d40 con 0x7f43ac0c5e30 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43b0f80700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac064040 con 0x7f43ac0c5380 2023-11-28T04:26:35.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35608/0 (socket says 172.21.15.73:35608) 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 -- 172.21.15.73:0/2860823188 learned_addr learned my addr 172.21.15.73:0/2860823188 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43ab7fe700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5380 0x7f43ac165140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43aaffd700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac0c5e30 0x7f43ac13a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 -- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5380 msgr2=0x7f43ac165140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5380 0x7f43ac165140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 -- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac0c5e30 msgr2=0x7f43ac13a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac0c5e30 0x7f43ac13a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 -- 172.21.15.73:0/2860823188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43a0009370 con 0x7f43ac13bce0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.404+0000 7f43abfff700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7f43a4009df0 tx=0x7f43a400b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.405+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400caa0 con 0x7f43ac13bce0 2023-11-28T04:26:35.409 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.405+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400f040 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.405+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac064320 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.405+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a400ce10 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.405+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac0648e0 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac0c4df0 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f43a4013610 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43937fe700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 0x7f43980617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43ab7fe700 1 -- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 msgr2=0x7f43980617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f43a407ad80 con 0x7f43ac13bce0 2023-11-28T04:26:35.410 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.406+0000 7f43ab7fe700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 0x7f43980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:35.415 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.414+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f43a404bd10 con 0x7f43ac13bce0 2023-11-28T04:26:35.423 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:35.422+0000 7fd377991680 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-11-28T04:26:35.583 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.582+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f43ac06c7c0 con 0x7f43ac13bce0 2023-11-28T04:26:35.584 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.582+0000 7f43937fe700 1 -- 172.21.15.73:0/2860823188 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v213) v1 ==== 92+0+10 (secure 0 0 0) 0x7f43a407c020 con 0x7f43ac13bce0 2023-11-28T04:26:35.584 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 8 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 msgr2=0x7f43980617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 0x7f43980617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 msgr2=0x7f43ac063810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 rx=0x7f43a4009df0 tx=0x7f43a400b040).stop 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 shutdown_connections 2023-11-28T04:26:35.586 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f439805f300 0x7f43980617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f43ac13bce0 0x7f43ac063810 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f43ac0c5380 0x7f43ac165140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 --2- 172.21.15.73:0/2860823188 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f43ac0c5e30 0x7f43ac13a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 >> 172.21.15.73:0/2860823188 conn(0x7f43ac062700 msgr2=0x7f43ac0c3620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 shutdown_connections 2023-11-28T04:26:35.587 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.584+0000 7f43b0f80700 1 -- 172.21.15.73:0/2860823188 wait complete. 2023-11-28T04:26:35.598 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-11-28T04:26:35.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.719+0000 7f0d9460a700 1 Processor -- start 2023-11-28T04:26:35.720 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.719+0000 7f0d9460a700 1 -- start start 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d9460a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d8c098d10 0x7f0d8c0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d9460a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0d8c158320 0x7f0d8c1587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d9460a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8c158cc0 con 0x7f0d8c0c6190 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d9460a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c0c72d0 con 0x7f0d8c098d10 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d9460a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8c064570 con 0x7f0d8c158320 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d8c098d10 0x7f0d8c0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0d8c098d10 0x7f0d8c0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35614/0 (socket says 172.21.15.73:35614) 2023-11-28T04:26:35.722 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 -- 172.21.15.73:0/1296376547 learned_addr learned my addr 172.21.15.73:0/1296376547 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d92ba7700 1 --2- 172.21.15.73:0/1296376547 >> v2:172.21.15.150:3300/0 conn(0x7f0d8c158320 0x7f0d8c1587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 -- 172.21.15.73:0/1296376547 >> v2:172.21.15.150:3300/0 conn(0x7f0d8c158320 msgr2=0x7f0d8c1587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 --2- 172.21.15.73:0/1296376547 >> v2:172.21.15.150:3300/0 conn(0x7f0d8c158320 0x7f0d8c1587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 -- 172.21.15.73:0/1296376547 >> v1:172.21.15.150:6789/0 conn(0x7f0d8c0c6190 legacy=0x7f0d8c063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.720+0000 7f0d923a6700 1 -- 172.21.15.73:0/1296376547 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8c154bc0 con 0x7f0d8c098d10 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d923a6700 1 --2- 172.21.15.73:0/1296376547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c098d10 0x7f0d8c0c7760 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f0d8c0657a0 tx=0x7f0d7c0090f0).ready entity=mon.0 client_cookie=76f58f550b55afd6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d913a4700 1 -- 172.21.15.73:0/1296376547 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00b600 con 0x7f0d8c098d10 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d913a4700 1 -- 172.21.15.73:0/1296376547 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00b760 con 0x7f0d8c098d10 2023-11-28T04:26:35.723 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d913a4700 1 -- 172.21.15.73:0/1296376547 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00bae0 con 0x7f0d8c098d10 2023-11-28T04:26:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d9460a700 1 -- 172.21.15.73:0/1296376547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c098d10 msgr2=0x7f0d8c0c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.721+0000 7f0d9460a700 1 --2- 172.21.15.73:0/1296376547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c098d10 0x7f0d8c0c7760 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 rx=0x7f0d8c0657a0 tx=0x7f0d7c0090f0).stop 2023-11-28T04:26:35.724 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 -- 172.21.15.73:0/1296376547 shutdown_connections 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 --2- 172.21.15.73:0/1296376547 >> v2:172.21.15.150:3300/0 conn(0x7f0d8c158320 0x7f0d8c1587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 --2- 172.21.15.73:0/1296376547 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c098d10 0x7f0d8c0c7760 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 -- 172.21.15.73:0/1296376547 >> 172.21.15.73:0/1296376547 conn(0x7f0d8c0cb310 msgr2=0x7f0d8c0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 -- 172.21.15.73:0/1296376547 shutdown_connections 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.722+0000 7f0d9460a700 1 -- 172.21.15.73:0/1296376547 wait complete. 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.723+0000 7f0d9460a700 1 Processor -- start 2023-11-28T04:26:35.725 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.723+0000 7f0d9460a700 1 -- start start 2023-11-28T04:26:35.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c098d10 0x7f0d8c14b560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.727 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d8c0c6190 0x7f0d8c1556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c1560a0 con 0x7f0d8c158320 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0c3470 con 0x7f0d8c098d10 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.724+0000 7f0d9460a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8c0c3d10 con 0x7f0d8c0c6190 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35626/0 (socket says 172.21.15.73:35626) 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 -- 172.21.15.73:0/2084796448 learned_addr learned my addr 172.21.15.73:0/2084796448 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d923a6700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c098d10 0x7f0d8c14b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d91ba5700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d8c0c6190 0x7f0d8c1556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 -- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d8c0c6190 msgr2=0x7f0d8c1556c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d8c0c6190 0x7f0d8c1556c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 -- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c098d10 msgr2=0x7f0d8c14b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.728 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c098d10 0x7f0d8c14b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 -- 172.21.15.73:0/2084796448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009370 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d92ba7700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 rx=0x7f0d84007f20 tx=0x7f0d84009860).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d8400eb90 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d8400ecf0 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d84014610 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d8c169fa0 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.725+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d8c0c39a0 con 0x7f0d8c158320 2023-11-28T04:26:35.729 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8c0c3ec0 con 0x7f0d8c158320 2023-11-28T04:26:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f0d84014980 con 0x7f0d8c158320 2023-11-28T04:26:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d827fc700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 0x7f0d700617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d923a6700 1 -- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 msgr2=0x7f0d700617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f0d8407a8f0 con 0x7f0d8c158320 2023-11-28T04:26:35.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.727+0000 7f0d923a6700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 0x7f0d700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:35.732 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.731+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0d8c0c3ec0 con 0x7f0d8c158320 2023-11-28T04:26:35.786 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:35.786+0000 7fddf3802680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:35.861 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:35.860+0000 7fd377991680 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-11-28T04:26:35.891 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:35.889+0000 7fddf3802680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:35.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.898+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0d8c0c3e80 con 0x7f0d8c158320 2023-11-28T04:26:35.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.898+0000 7f0d827fc700 1 -- 172.21.15.73:0/2084796448 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v213) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0d8404b910 con 0x7f0d8c158320 2023-11-28T04:26:35.900 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 msgr2=0x7f0d700617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 0x7f0d700617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 msgr2=0x7f0d8c155bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 rx=0x7f0d84007f20 tx=0x7f0d84009860).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 shutdown_connections 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f0d7005f300 0x7f0d700617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0d8c158320 0x7f0d8c155bb0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0d8c098d10 0x7f0d8c14b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 --2- 172.21.15.73:0/2084796448 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0d8c0c6190 0x7f0d8c1556c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 >> 172.21.15.73:0/2084796448 conn(0x7f0d8c0cb310 msgr2=0x7f0d8c14c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:35.902 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 shutdown_connections 2023-11-28T04:26:35.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:35.900+0000 7f0d9460a700 1 -- 172.21.15.73:0/2084796448 wait complete. 2023-11-28T04:26:35.914 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-11-28T04:26:35.970 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:35.969+0000 7fd377991680 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:36.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.030+0000 7faf09937700 1 Processor -- start 2023-11-28T04:26:36.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.030+0000 7faf09937700 1 -- start start 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf09937700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7faf04066a30 0x7faf04066e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf09937700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf041586a0 con 0x7faf040c6a30 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf09937700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf04157de0 con 0x7faf040c6eb0 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf09937700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf04158050 con 0x7faf04066a30 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf03fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faf040c6a30 0x7faf0413d450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39036/0 (socket says 172.21.15.73:39036) 2023-11-28T04:26:36.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf03fff700 1 -- 172.21.15.73:0/42017635 learned_addr learned my addr 172.21.15.73:0/42017635 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf08935700 1 --2- 172.21.15.73:0/42017635 >> v2:172.21.15.150:3300/0 conn(0x7faf04066a30 0x7faf04066e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082571752 0 0) 0x7faf041586a0 con 0x7faf040c6a30 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faef40033f0 con 0x7faf040c6a30 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344600938 0 0) 0x7faf04157de0 con 0x7faf040c6eb0 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.031+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf041586a0 con 0x7faf040c6eb0 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4294523762 0 0) 0x7faef40033f0 con 0x7faf040c6a30 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf08935700 1 -- 172.21.15.73:0/42017635 >> v1:172.21.15.150:6789/0 conn(0x7faf040c6eb0 legacy=0x7faf04063810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf08935700 1 -- 172.21.15.73:0/42017635 >> v1:172.21.15.73:6789/0 conn(0x7faf040c6a30 legacy=0x7faf0413d450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf08935700 1 -- 172.21.15.73:0/42017635 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf04158330 con 0x7faf04066a30 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf08935700 1 --2- 172.21.15.73:0/42017635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf04066a30 0x7faf04066e10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7faef8009760 tx=0x7faef8009060).ready entity=mon.1 client_cookie=69af927ae4b6fd35 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.035 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef0001470 con 0x7faf04066a30 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8011400 con 0x7faf04066a30 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf037fe700 1 -- 172.21.15.73:0/42017635 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8011780 con 0x7faf04066a30 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf09937700 1 -- 172.21.15.73:0/42017635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf04066a30 msgr2=0x7faf04066e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.032+0000 7faf09937700 1 --2- 172.21.15.73:0/42017635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf04066a30 0x7faf04066e10 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 rx=0x7faef8009760 tx=0x7faef8009060).stop 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.033+0000 7faf09937700 1 -- 172.21.15.73:0/42017635 shutdown_connections 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.033+0000 7faf09937700 1 --2- 172.21.15.73:0/42017635 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf04066a30 0x7faf04066e10 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.033+0000 7faf09937700 1 -- 172.21.15.73:0/42017635 >> 172.21.15.73:0/42017635 conn(0x7faf040c8ed0 msgr2=0x7faf040c92e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.033+0000 7faf09937700 1 -- 172.21.15.73:0/42017635 shutdown_connections 2023-11-28T04:26:36.036 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.033+0000 7faf09937700 1 -- 172.21.15.73:0/42017635 wait complete. 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.034+0000 7faf09937700 1 Processor -- start 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.034+0000 7faf09937700 1 -- start start 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf040c6a30 0x7faf040ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf040c6eb0 0x7faf04139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf04167d70 con 0x7faf04066a30 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf04168050 con 0x7faf040c6eb0 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf040aeee0 con 0x7faf040c6a30 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.040 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35638/0 (socket says 172.21.15.73:35638) 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 -- 172.21.15.73:0/849477505 learned_addr learned my addr 172.21.15.73:0/849477505 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09136700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf040c6eb0 0x7faf04139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf03fff700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf040c6a30 0x7faf040ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 -- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf040c6a30 msgr2=0x7faf040ae9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf040c6a30 0x7faf040ae9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 -- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf040c6eb0 msgr2=0x7faf04139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf040c6eb0 0x7faf04139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 -- 172.21.15.73:0/849477505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef80092a0 con 0x7faf04066a30 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf08935700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 rx=0x7faef80118a0 tx=0x7faef8016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.041 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef80106c0 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef8010820 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef8010ba0 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.035+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf040c4670 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.036+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf040c4c30 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.036+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf0406c7c0 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.037+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7faef80263b0 con 0x7faf04066a30 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.037+0000 7faf00ff9700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 0x7faee80617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.037+0000 7faf03fff700 1 -- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 msgr2=0x7faee80617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:36.042 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.037+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7faef807afc0 con 0x7faf04066a30 2023-11-28T04:26:36.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.037+0000 7faf03fff700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 0x7faee80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:36.043 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:36.043+0000 7fddf3802680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:36.043 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.041+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7faef804bf30 con 0x7faf04066a30 2023-11-28T04:26:36.118 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:36.118+0000 7fddf3802680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:36.127 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:36.126+0000 7fd377991680 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:36.186 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:36.186+0000 7fddf3802680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:36.203 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:36.202+0000 7fd377991680 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-11-28T04:26:36.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.209+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7faf040bfc20 con 0x7faf04066a30 2023-11-28T04:26:36.210 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.209+0000 7faf00ff9700 1 -- 172.21.15.73:0/849477505 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v213) v1 ==== 100+0+11 (secure 0 0 0) 0x7faef8022070 con 0x7faf04066a30 2023-11-28T04:26:36.210 INFO:teuthology.orchestra.run.smithi073.stdout:pg_num: 64 2023-11-28T04:26:36.212 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 msgr2=0x7faee80617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 0x7faee80617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 msgr2=0x7faf040c3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 rx=0x7faef80118a0 tx=0x7faef8016000).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 shutdown_connections 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7faee805f300 0x7faee80617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faf040c6eb0 0x7faf04139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faf04066a30 0x7faf040c3fb0 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 --2- 172.21.15.73:0/849477505 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faf040c6a30 0x7faf040ae9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 >> 172.21.15.73:0/849477505 conn(0x7faf040c8ed0 msgr2=0x7faf0413c500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 shutdown_connections 2023-11-28T04:26:36.213 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.211+0000 7faf09937700 1 -- 172.21.15.73:0/849477505 wait complete. 2023-11-28T04:26:36.225 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:36.275 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:36.274+0000 7fd377991680 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-11-28T04:26:36.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.342+0000 7f6465ae3700 1 Processor -- start 2023-11-28T04:26:36.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.342+0000 7f6465ae3700 1 -- start start 2023-11-28T04:26:36.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.342+0000 7f6465ae3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460066880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.345 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6465ae3700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6460066e60 con 0x7f6460155510 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6465ae3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6460067140 con 0x7f6460069bf0 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6465ae3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460067420 con 0x7f646013a8d0 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645ffff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f6460155510 0x7f64600694e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39050/0 (socket says 172.21.15.73:39050) 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645ffff700 1 -- 172.21.15.73:0/3771525631 learned_addr learned my addr 172.21.15.73:0/3771525631 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6464ae1700 1 --2- 172.21.15.73:0/3771525631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460066880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151220412 0 0) 0x7f6460066e60 con 0x7f6460155510 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64500033f0 con 0x7f6460155510 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 465602300 0 0) 0x7f6460067140 con 0x7f6460069bf0 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6460066e60 con 0x7f6460069bf0 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6464ae1700 1 -- 172.21.15.73:0/3771525631 >> v1:172.21.15.150:6789/0 conn(0x7f6460069bf0 legacy=0x7f64600c22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6464ae1700 1 -- 172.21.15.73:0/3771525631 >> v1:172.21.15.73:6789/0 conn(0x7f6460155510 legacy=0x7f64600694e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.346 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4074301842 0 0) 0x7f64500033f0 con 0x7f6460155510 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6464ae1700 1 -- 172.21.15.73:0/3771525631 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6460067700 con 0x7f646013a8d0 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.343+0000 7f6464ae1700 1 --2- 172.21.15.73:0/3771525631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460066880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f6454009790 tx=0x7f6454009060).ready entity=mon.2 client_cookie=a2a34f1094aecbe server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c001470 con 0x7f646013a8d0 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6454011400 con 0x7f646013a8d0 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f645f7fe700 1 -- 172.21.15.73:0/3771525631 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454011780 con 0x7f646013a8d0 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 -- 172.21.15.73:0/3771525631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 msgr2=0x7f6460066880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 --2- 172.21.15.73:0/3771525631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460066880 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 rx=0x7f6454009790 tx=0x7f6454009060).stop 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 -- 172.21.15.73:0/3771525631 shutdown_connections 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 --2- 172.21.15.73:0/3771525631 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460066880 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 -- 172.21.15.73:0/3771525631 >> 172.21.15.73:0/3771525631 conn(0x7f64600c7ca0 msgr2=0x7f64600c80b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 -- 172.21.15.73:0/3771525631 shutdown_connections 2023-11-28T04:26:36.347 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.344+0000 7f6465ae3700 1 -- 172.21.15.73:0/3771525631 wait complete. 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.345+0000 7f6465ae3700 1 Processor -- start 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.345+0000 7f6465ae3700 1 -- start start 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460134b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6460155510 0x7f6460161ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460133f20 con 0x7f6460069bf0 2023-11-28T04:26:36.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460133060 con 0x7f6460155510 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6465ae3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460135030 con 0x7f646013a8d0 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6464ae1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6464ae1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35648/0 (socket says 172.21.15.73:35648) 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f6464ae1700 1 -- 172.21.15.73:0/2720695262 learned_addr learned my addr 172.21.15.73:0/2720695262 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f645ffff700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460134b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.346+0000 7f64652e2700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6460155510 0x7f6460161ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 msgr2=0x7f6460134b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460134b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6460155510 msgr2=0x7f6460161ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6460155510 0x7f6460161ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 -- 172.21.15.73:0/2720695262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64540092a0 con 0x7f6460069bf0 2023-11-28T04:26:36.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6464ae1700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f64600becd0 tx=0x7f6454016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454010840 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64540109a0 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6454010d40 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64601332a0 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.347+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460133860 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.348+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64600c6db0 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.348+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f645401d640 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.349+0000 7f645cff9700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 0x7f64440617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.349+0000 7f645ffff700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 msgr2=0x7f64440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.349+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f645407b970 con 0x7f6460069bf0 2023-11-28T04:26:36.352 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.349+0000 7f645ffff700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 0x7f64440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:36.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.353+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f645404c990 con 0x7f6460069bf0 2023-11-28T04:26:36.447 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:36.446+0000 7fddf3802680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:36.545 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:36.544+0000 7fd377991680 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-11-28T04:26:36.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.550+0000 7f645ffff700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 msgr2=0x7f64440617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:36.551 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.550+0000 7f645ffff700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 0x7f64440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:36.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.566+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f646006c7c0 con 0x7f6460069bf0 2023-11-28T04:26:36.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.567+0000 7f645cff9700 1 -- 172.21.15.73:0/2720695262 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f645404ffb0 con 0x7f6460069bf0 2023-11-28T04:26:36.570 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:36.570 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 msgr2=0x7f64440617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 0x7f64440617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 msgr2=0x7f6460132b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 rx=0x7f64600becd0 tx=0x7f6454016000).stop 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 shutdown_connections 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f644405f300 0x7f64440617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.571 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f6460155510 0x7f6460161ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f6460069bf0 0x7f6460132b70 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 --2- 172.21.15.73:0/2720695262 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f646013a8d0 0x7f6460134b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 >> 172.21.15.73:0/2720695262 conn(0x7f64600c7ca0 msgr2=0x7f64600bde20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 shutdown_connections 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.569+0000 7f6465ae3700 1 -- 172.21.15.73:0/2720695262 wait complete. 2023-11-28T04:26:36.572 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:36.582 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:36.630 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:36.628+0000 7fddf3802680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:36.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.702+0000 7fc8e76a4700 1 Processor -- start 2023-11-28T04:26:36.703 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.702+0000 7fc8e76a4700 1 -- start start 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e76a4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e76a4700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8e01546f0 0x7fc8e0158b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e76a4700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8e0063380 con 0x7fc8e006b510 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e76a4700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8e00c19e0 con 0x7fc8e01546f0 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e76a4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0159060 con 0x7fc8e00c0d10 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8e01546f0 0x7fc8e0158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5440700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fc8e01546f0 0x7fc8e0158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3300/0 says I am v2:172.21.15.73:54194/0 (socket says 172.21.15.73:54194) 2023-11-28T04:26:36.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 -- 172.21.15.73:0/3024757922 learned_addr learned my addr 172.21.15.73:0/3024757922 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e4c3f700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fc8e006b510 0x7fc8e0153fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45842/0 (socket says 172.21.15.73:45842) 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5440700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:39410/0 (socket says 172.21.15.73:39410) 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 -- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 msgr2=0x7fc8e00c10f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.703+0000 7fc8e5c41700 1 -- 172.21.15.73:0/3024757922 >> v1:172.21.15.150:6789/0 conn(0x7fc8e006b510 legacy=0x7fc8e0153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e5c41700 1 -- 172.21.15.73:0/3024757922 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0062770 con 0x7fc8e01546f0 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8d7fff700 1 -- 172.21.15.73:0/3024757922 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 821581056 0 0) 0x7fc8e0063380 con 0x7fc8e006b510 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e5440700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e5c41700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e01546f0 0x7fc8e0158b70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc8d8009f20 tx=0x7fc8d8009840).ready entity=mon.1 client_cookie=c577ce45e3328ba1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8d7fff700 1 -- 172.21.15.73:0/3024757922 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800e070 con 0x7fc8e01546f0 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8d7fff700 1 -- 172.21.15.73:0/3024757922 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d800b9d0 con 0x7fc8e01546f0 2023-11-28T04:26:36.707 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8d7fff700 1 -- 172.21.15.73:0/3024757922 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d800bd50 con 0x7fc8e01546f0 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 -- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e01546f0 msgr2=0x7fc8e0158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e01546f0 0x7fc8e0158b70 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 rx=0x7fc8d8009f20 tx=0x7fc8d8009840).stop 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 -- 172.21.15.73:0/3024757922 shutdown_connections 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e01546f0 0x7fc8e0158b70 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/3024757922 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e00c0d10 0x7fc8e00c10f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.704+0000 7fc8e76a4700 1 -- 172.21.15.73:0/3024757922 >> 172.21.15.73:0/3024757922 conn(0x7fc8e00c7760 msgr2=0x7fc8e00c7b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.705+0000 7fc8e76a4700 1 -- 172.21.15.73:0/3024757922 shutdown_connections 2023-11-28T04:26:36.708 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.705+0000 7fc8e76a4700 1 -- 172.21.15.73:0/3024757922 wait complete. 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.706+0000 7fc8e76a4700 1 Processor -- start 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.706+0000 7fc8e76a4700 1 -- start start 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 0x7fc8e00aaea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0156df0 con 0x7fc8e01546f0 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0156fe0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.711 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01572c0 con 0x7fc8e006b510 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5c41700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5440700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5c41700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35664/0 (socket says 172.21.15.73:35664) 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5c41700 1 -- 172.21.15.73:0/1510923421 learned_addr learned my addr 172.21.15.73:0/1510923421 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5440700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:39416/0 (socket says 172.21.15.73:39416) 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 0x7fc8e00aaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 -- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 msgr2=0x7fc8e0139b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 -- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 msgr2=0x7fc8e0156900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 -- 172.21.15.73:0/1510923421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d00092a0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5c41700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:36.712 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e5440700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e4c3f700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 0x7fc8e00aaea0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc8e00c5560 tx=0x7fc8dc007bd0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0070d0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8dc007230 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d8009ae0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc0056e0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.707+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e0157b60 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.708+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e006c860 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.709+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fc8dc005ad0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.709+0000 7fc8d57fa700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 0x7fc8c40617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.709+0000 7fc8e5440700 1 -- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 msgr2=0x7fc8c40617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.709+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fc8dc079b30 con 0x7fc8e00c0d10 2023-11-28T04:26:36.713 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.709+0000 7fc8e5440700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 0x7fc8c40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:36.714 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.713+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc8dc04ab50 con 0x7fc8e00c0d10 2023-11-28T04:26:36.739 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:36.738+0000 7fd377991680 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-11-28T04:26:36.885 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.884+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8e0062ae0 con 0x7fc8e00c0d10 2023-11-28T04:26:36.886 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.885+0000 7fc8d57fa700 1 -- 172.21.15.73:0/1510923421 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+17188 (secure 0 0 0) 0x7fc8dc04e170 con 0x7fc8e00c0d10 2023-11-28T04:26:36.886 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:36.887 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":213,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:30.988911+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6832/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/3047075693":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/1637465872":"2023-11-29T04:26:30.988881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 msgr2=0x7fc8c40617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 0x7fc8c40617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 msgr2=0x7fc8e00aaea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 0x7fc8e00aaea0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 rx=0x7fc8e00c5560 tx=0x7fc8dc007bd0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 shutdown_connections 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fc8c405f300 0x7fc8c40617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc8e01546f0 0x7fc8e0156900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc8e006b510 0x7fc8e0139b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 --2- 172.21.15.73:0/1510923421 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc8e00c0d10 0x7fc8e00aaea0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 >> 172.21.15.73:0/1510923421 conn(0x7fc8e00c7760 msgr2=0x7fc8e00c4640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:36.889 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 shutdown_connections 2023-11-28T04:26:36.890 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:36.887+0000 7fc8e76a4700 1 -- 172.21.15.73:0/1510923421 wait complete. 2023-11-28T04:26:36.901 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:26:36.901 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:37.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.016+0000 7f1b28900700 1 Processor -- start 2023-11-28T04:26:37.017 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.016+0000 7f1b28900700 1 -- start start 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b28900700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1b24065f30 0x7f1b24066310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b28900700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b24152450 0x7f1b240bcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b28900700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b24151740 con 0x7f1b240bd430 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b28900700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240c2470 con 0x7f1b24152450 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b28900700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b240bd8a0 con 0x7f1b24065f30 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b21d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b24152450 0x7f1b240bcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b22d9d700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1b240bd430 0x7f1b240c14d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39064/0 (socket says 172.21.15.73:39064) 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b22d9d700 1 -- 172.21.15.73:0/1259268077 learned_addr learned my addr 172.21.15.73:0/1259268077 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:37.020 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b21d9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f1b24152450 0x7f1b240bcf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35670/0 (socket says 172.21.15.73:35670) 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b2259c700 1 --2- 172.21.15.73:0/1259268077 >> v2:172.21.15.150:3300/0 conn(0x7f1b24065f30 0x7f1b24066310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b21d9b700 1 -- 172.21.15.73:0/1259268077 >> v2:172.21.15.150:3300/0 conn(0x7f1b24065f30 msgr2=0x7f1b24066310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b21d9b700 1 --2- 172.21.15.73:0/1259268077 >> v2:172.21.15.150:3300/0 conn(0x7f1b24065f30 0x7f1b24066310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b21d9b700 1 -- 172.21.15.73:0/1259268077 >> v1:172.21.15.73:6789/0 conn(0x7f1b240bd430 legacy=0x7f1b240c14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b21d9b700 1 -- 172.21.15.73:0/1259268077 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24066800 con 0x7f1b24152450 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.017+0000 7f1b2159a700 1 -- 172.21.15.73:0/1259268077 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1186297321 0 0) 0x7f1b24151740 con 0x7f1b240bd430 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b21d9b700 1 --2- 172.21.15.73:0/1259268077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b240bcf40 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 rx=0x7f1b240c6e30 tx=0x7f1b14009060).ready entity=mon.0 client_cookie=12e3dcd10fdd1c71 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b2159a700 1 -- 172.21.15.73:0/1259268077 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400b580 con 0x7f1b24152450 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b2159a700 1 -- 172.21.15.73:0/1259268077 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1400b6e0 con 0x7f1b24152450 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b2159a700 1 -- 172.21.15.73:0/1259268077 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1400ba60 con 0x7f1b24152450 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 -- 172.21.15.73:0/1259268077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 msgr2=0x7f1b240bcf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 --2- 172.21.15.73:0/1259268077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b240bcf40 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 rx=0x7f1b240c6e30 tx=0x7f1b14009060).stop 2023-11-28T04:26:37.021 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 -- 172.21.15.73:0/1259268077 shutdown_connections 2023-11-28T04:26:37.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 --2- 172.21.15.73:0/1259268077 >> v2:172.21.15.150:3300/0 conn(0x7f1b24065f30 0x7f1b24066310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 --2- 172.21.15.73:0/1259268077 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b240bcf40 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 -- 172.21.15.73:0/1259268077 >> 172.21.15.73:0/1259268077 conn(0x7f1b240631d0 msgr2=0x7f1b240c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.018+0000 7f1b28900700 1 -- 172.21.15.73:0/1259268077 shutdown_connections 2023-11-28T04:26:37.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.019+0000 7f1b28900700 1 -- 172.21.15.73:0/1259268077 wait complete. 2023-11-28T04:26:37.024 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 Processor -- start 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 -- start start 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b24065f30 0x7f1b24092e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 0x7f1b24150730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24166d50 con 0x7f1b24152450 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241660f0 con 0x7f1b24065f30 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b28900700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b241663b0 con 0x7f1b240bd430 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b22d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.020+0000 7f1b22d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35686/0 (socket says 172.21.15.73:35686) 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 -- 172.21.15.73:0/1588807976 learned_addr learned my addr 172.21.15.73:0/1588807976 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b2259c700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b24065f30 0x7f1b24092e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b21d9b700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 0x7f1b24150730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 msgr2=0x7f1b24150730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 0x7f1b24150730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b24065f30 msgr2=0x7f1b24092e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b24065f30 0x7f1b24092e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 -- 172.21.15.73:0/1588807976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b140092a0 con 0x7f1b24152450 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b21d9b700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 0x7f1b24150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b22d9d700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 rx=0x7f1b1c00c790 tx=0x7f1b1c007f30).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c005d30 con 0x7f1b24152450 2023-11-28T04:26:37.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b1c007110 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b24166690 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b1c0074d0 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.021+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b24165de0 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.022+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b24162880 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.022+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f1b1c007840 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.023+0000 7f1b127fc700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 0x7f1b040617f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.023+0000 7f1b2259c700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 msgr2=0x7f1b040617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.023+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f1b1c07a480 con 0x7f1b24152450 2023-11-28T04:26:37.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.023+0000 7f1b2259c700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 0x7f1b040617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:37.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.027+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1b24162880 con 0x7f1b24152450 2023-11-28T04:26:37.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.224+0000 7f1b2259c700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 msgr2=0x7f1b040617f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:37.225 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.224+0000 7f1b2259c700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 0x7f1b040617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:37.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.247+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1b24162880 con 0x7f1b24152450 2023-11-28T04:26:37.248 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.247+0000 7f1b127fc700 1 -- 172.21.15.73:0/1588807976 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f1b24162880 con 0x7f1b24152450 2023-11-28T04:26:37.249 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:37.249 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 msgr2=0x7f1b040617f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 0x7f1b040617f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 msgr2=0x7f1b2414e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 rx=0x7f1b1c00c790 tx=0x7f1b1c007f30).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 shutdown_connections 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f1b0405f330 0x7f1b040617f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1b24152450 0x7f1b2414e4e0 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1b24065f30 0x7f1b24092e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 --2- 172.21.15.73:0/1588807976 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1b240bd430 0x7f1b24150730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.251 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 >> 172.21.15.73:0/1588807976 conn(0x7f1b240631d0 msgr2=0x7f1b24153090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 shutdown_connections 2023-11-28T04:26:37.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.249+0000 7f1b28900700 1 -- 172.21.15.73:0/1588807976 wait complete. 2023-11-28T04:26:37.252 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:37.263 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:26:37.282 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.281+0000 7fddf3802680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:37.352 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.351+0000 7fddf3802680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:37.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.384+0000 7f53dae78700 1 Processor -- start 2023-11-28T04:26:37.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.384+0000 7f53dae78700 1 -- start start 2023-11-28T04:26:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53dae78700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f53d4157680 0x7f53d406b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53dae78700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d4069330 0x7f53d40c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.396 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53dae78700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53d4062f60 con 0x7f53d4157a60 2023-11-28T04:26:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53dae78700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d4062700 con 0x7f53d4069330 2023-11-28T04:26:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53dae78700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53d40c1f30 con 0x7f53d4157680 2023-11-28T04:26:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d4069330 0x7f53d40c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.397 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f53d4069330 0x7f53d40c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35694/0 (socket says 172.21.15.73:35694) 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 -- 172.21.15.73:0/1703581791 learned_addr learned my addr 172.21.15.73:0/1703581791 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53d9e76700 1 --2- 172.21.15.73:0/1703581791 >> v2:172.21.15.150:3300/0 conn(0x7f53d4157680 0x7f53d406b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 -- 172.21.15.73:0/1703581791 >> v2:172.21.15.150:3300/0 conn(0x7f53d4157680 msgr2=0x7f53d406b870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 --2- 172.21.15.73:0/1703581791 >> v2:172.21.15.150:3300/0 conn(0x7f53d4157680 0x7f53d406b870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 -- 172.21.15.73:0/1703581791 >> v1:172.21.15.150:6789/0 conn(0x7f53d4157a60 legacy=0x7f53d4068c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.398 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 -- 172.21.15.73:0/1703581791 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53d4069f80 con 0x7f53d4069330 2023-11-28T04:26:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53da677700 1 --2- 172.21.15.73:0/1703581791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c1a40 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 rx=0x7f53d406c470 tx=0x7f53cc009840).ready entity=mon.0 client_cookie=cee4e4406efa60db server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.399 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53d8e74700 1 -- 172.21.15.73:0/1703581791 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00e070 con 0x7f53d4069330 2023-11-28T04:26:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.385+0000 7f53d8e74700 1 -- 172.21.15.73:0/1703581791 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53cc00ba20 con 0x7f53d4069330 2023-11-28T04:26:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53d8e74700 1 -- 172.21.15.73:0/1703581791 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53cc00bda0 con 0x7f53d4069330 2023-11-28T04:26:37.400 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 -- 172.21.15.73:0/1703581791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 msgr2=0x7f53d40c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 --2- 172.21.15.73:0/1703581791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c1a40 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 rx=0x7f53d406c470 tx=0x7f53cc009840).stop 2023-11-28T04:26:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 -- 172.21.15.73:0/1703581791 shutdown_connections 2023-11-28T04:26:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 --2- 172.21.15.73:0/1703581791 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c1a40 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.401 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 --2- 172.21.15.73:0/1703581791 >> v2:172.21.15.150:3300/0 conn(0x7f53d4157680 0x7f53d406b870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 -- 172.21.15.73:0/1703581791 >> 172.21.15.73:0/1703581791 conn(0x7f53d40c7eb0 msgr2=0x7f53d40c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 -- 172.21.15.73:0/1703581791 shutdown_connections 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.386+0000 7f53dae78700 1 -- 172.21.15.73:0/1703581791 wait complete. 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.387+0000 7f53dae78700 1 Processor -- start 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.387+0000 7f53dae78700 1 -- start start 2023-11-28T04:26:37.402 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 0x7f53d415a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d4157a60 0x7f53d41649b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40ab450 con 0x7f53d4069330 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40aba10 con 0x7f53d4157a60 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53dae78700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53d40c1000 con 0x7f53d4157680 2023-11-28T04:26:37.403 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53d9e76700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53d9e76700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35700/0 (socket says 172.21.15.73:35700) 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53d9e76700 1 -- 172.21.15.73:0/3795408462 learned_addr learned my addr 172.21.15.73:0/3795408462 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53da677700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d4157a60 0x7f53d41649b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.388+0000 7f53d9675700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 0x7f53d415a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 msgr2=0x7f53d415a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.404 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 0x7f53d415a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d4157a60 msgr2=0x7f53d41649b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d4157a60 0x7f53d41649b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 -- 172.21.15.73:0/3795408462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53cc009ae0 con 0x7f53d4069330 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9675700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 0x7f53d415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53d9e76700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 rx=0x7f53d000e960 tx=0x7f53d0009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.405 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000c6a0 con 0x7f53d4069330 2023-11-28T04:26:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53d000c800 con 0x7f53d4069330 2023-11-28T04:26:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53d4158dc0 con 0x7f53d4069330 2023-11-28T04:26:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53d000cb80 con 0x7f53d4069330 2023-11-28T04:26:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.389+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53d40ab6a0 con 0x7f53d4069330 2023-11-28T04:26:37.406 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.390+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53d40abcf0 con 0x7f53d4069330 2023-11-28T04:26:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.390+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f53d001b400 con 0x7f53d4069330 2023-11-28T04:26:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.391+0000 7f53c9ffb700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 0x7f53b8061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.391+0000 7f53d9675700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 msgr2=0x7f53b8061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.391+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f53d007aaa0 con 0x7f53d4069330 2023-11-28T04:26:37.407 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.391+0000 7f53d9675700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 0x7f53b8061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:37.408 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.399+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f53d004ba60 con 0x7f53d4069330 2023-11-28T04:26:37.428 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.427+0000 7fd377991680 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-11-28T04:26:37.429 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.428+0000 7fddf3802680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:37.500 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.499+0000 7fd377991680 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-11-28T04:26:37.576 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.575+0000 7fddf3802680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:37.578 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.577+0000 7fd377991680 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-11-28T04:26:37.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.592+0000 7f53d9675700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 msgr2=0x7f53b8061790 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:37.593 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.592+0000 7f53d9675700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 0x7f53b8061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:37.610 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.609+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f53d41592e0 con 0x7f53d4069330 2023-11-28T04:26:37.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.609+0000 7f53c9ffb700 1 -- 172.21.15.73:0/3795408462 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 76 v76) v1 ==== 94+0+4493 (secure 0 0 0) 0x7f53d004f080 con 0x7f53d4069330 2023-11-28T04:26:37.611 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:37.611 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":76,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[{"mdsmap":{"epoch":76,"flags":18,"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-11-28T04:26:19.699059+0000","modified":"2023-11-28T04:26:21.704836+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9990},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9990":{"gid":9990,"name":"a","rank":0,"incarnation":74,"state":"up:active","state_seq":3,"addr":"172.21.15.73:6836/447432356","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6834","nonce":447432356},{"type":"v1","addr":"172.21.15.73:6836","nonce":447432356}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[26],"metadata_pool":25,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":12}]} 2023-11-28T04:26:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 msgr2=0x7f53b8061790 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:37.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 0x7f53b8061790 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 msgr2=0x7f53d40c0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 rx=0x7f53d000e960 tx=0x7f53d0009060).stop 2023-11-28T04:26:37.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 shutdown_connections 2023-11-28T04:26:37.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f53b805f2d0 0x7f53b8061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f53d4157a60 0x7f53d41649b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f53d4069330 0x7f53d40c0b10 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 --2- 172.21.15.73:0/3795408462 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f53d4157680 0x7f53d415a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 >> 172.21.15.73:0/3795408462 conn(0x7f53d40c7eb0 msgr2=0x7f53d4068c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 shutdown_connections 2023-11-28T04:26:37.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.611+0000 7f53dae78700 1 -- 172.21.15.73:0/3795408462 wait complete. 2023-11-28T04:26:37.616 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 76 2023-11-28T04:26:37.625 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:37.647 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.646+0000 7fddf3802680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:37.734 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.733+0000 7fd377991680 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-11-28T04:26:37.763 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.741+0000 7f9d31c12700 1 Processor -- start 2023-11-28T04:26:37.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.741+0000 7f9d31c12700 1 -- start start 2023-11-28T04:26:37.764 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d31c12700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d2c151ca0 0x7f9d2c152080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.765 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d31c12700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d31c12700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d2c0c8170 con 0x7f9d2c095e80 2023-11-28T04:26:37.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d31c12700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c0060 con 0x7f9d2c151ca0 2023-11-28T04:26:37.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d31c12700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c0c8c40 con 0x7f9d2c0c7820 2023-11-28T04:26:37.769 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d2b7fe700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f9d2c151ca0 0x7f9d2c152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.770 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d22ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.771 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d2bfff700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f9d2c095e80 0x7f9d2c157f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:45860/0 (socket says 172.21.15.73:45860) 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.742+0000 7f9d2bfff700 1 -- 172.21.15.73:0/2533240160 learned_addr learned my addr 172.21.15.73:0/2533240160 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 msgr2=0x7f9d2c0c7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2b7fe700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/2533240160 >> v1:172.21.15.150:6789/0 conn(0x7f9d2c095e80 legacy=0x7f9d2c157f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2affd700 1 -- 172.21.15.73:0/2533240160 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1200580852 0 0) 0x7f9d2c0c8170 con 0x7f9d2c095e80 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/2533240160 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c0bebc0 con 0x7f9d2c151ca0 2023-11-28T04:26:37.772 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d22ffd700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2b7fe700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c152080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9d1c006fd0 tx=0x7f9d1c006f40).ready entity=mon.1 client_cookie=683d922ea39bc01c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2affd700 1 -- 172.21.15.73:0/2533240160 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005880 con 0x7f9d2c151ca0 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2affd700 1 -- 172.21.15.73:0/2533240160 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c0059e0 con 0x7f9d2c151ca0 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d2affd700 1 -- 172.21.15.73:0/2533240160 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c005d60 con 0x7f9d2c151ca0 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 -- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c151ca0 msgr2=0x7f9d2c152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c152080 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 rx=0x7f9d1c006fd0 tx=0x7f9d1c006f40).stop 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 -- 172.21.15.73:0/2533240160 shutdown_connections 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c152080 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 --2- 172.21.15.73:0/2533240160 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 -- 172.21.15.73:0/2533240160 >> 172.21.15.73:0/2533240160 conn(0x7f9d2c062700 msgr2=0x7f9d2c062b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.743+0000 7f9d31c12700 1 -- 172.21.15.73:0/2533240160 shutdown_connections 2023-11-28T04:26:37.773 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.744+0000 7f9d31c12700 1 -- 172.21.15.73:0/2533240160 wait complete. 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 Processor -- start 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 -- start start 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 0x7f9d2c0c5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c159870 con 0x7f9d2c151ca0 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c146d30 con 0x7f9d2c095e80 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.745+0000 7f9d31c12700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d2c146840 con 0x7f9d2c0c7820 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 0x7f9d2c0c5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35704/0 (socket says 172.21.15.73:35704) 2023-11-28T04:26:37.774 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2bfff700 1 -- 172.21.15.73:0/1537003476 learned_addr learned my addr 172.21.15.73:0/1537003476 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d22ffd700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 msgr2=0x7f9d2c0c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 msgr2=0x7f9d2c0c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 -- 172.21.15.73:0/1537003476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d140092a0 con 0x7f9d2c095e80 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2bfff700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d2b7fe700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 0x7f9d2c0c5eb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f9d1c013000 tx=0x7f9d1c016000).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c006770 con 0x7f9d2c095e80 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d1c00f720 con 0x7f9d2c095e80 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d1c00fae0 con 0x7f9d2c095e80 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d1c009250 con 0x7f9d2c095e80 2023-11-28T04:26:37.775 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.746+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d2c158bf0 con 0x7f9d2c095e80 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.747+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d2c06c860 con 0x7f9d2c095e80 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.748+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7f9d1c026460 con 0x7f9d2c095e80 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.748+0000 7f9d23fff700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 0x7f9d0c0617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.748+0000 7f9d22ffd700 1 -- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 msgr2=0x7f9d0c0617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.748+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7f9d1c07ba00 con 0x7f9d2c095e80 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.748+0000 7f9d22ffd700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 0x7f9d0c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:37.776 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.752+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f9d1c04c9c0 con 0x7f9d2c095e80 2023-11-28T04:26:37.776 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.762+0000 7fddf3802680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:37.810 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.809+0000 7fd377991680 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-11-28T04:26:37.871 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:37.870+0000 7fddf3802680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:37.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.923+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d2c0c83a0 con 0x7f9d2c095e80 2023-11-28T04:26:37.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.924+0000 7f9d23fff700 1 -- 172.21.15.73:0/1537003476 <== mon.1 v2:172.21.15.150:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+17188 (secure 0 0 0) 0x7f9d1c04ffe0 con 0x7f9d2c095e80 2023-11-28T04:26:37.925 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:37.925 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":213,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:30.988911+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"203","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":25,"pool_name":"cephfs_metadata","create_time":"2023-11-28T04:26:18.231141+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"211","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":26,"pool_name":"cephfs_data","create_time":"2023-11-28T04:26:18.668882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:6833/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6832/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/3047075693":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/1637465872":"2023-11-29T04:26:30.988881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:37.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 msgr2=0x7f9d0c0617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:37.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 0x7f9d0c0617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.927 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 msgr2=0x7f9d2c0c5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 0x7f9d2c0c5eb0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 rx=0x7f9d1c013000 tx=0x7f9d1c016000).stop 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 shutdown_connections 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7f9d0c05f300 0x7f9d0c0617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9d2c151ca0 0x7f9d2c0c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9d2c095e80 0x7f9d2c0c5eb0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 --2- 172.21.15.73:0/1537003476 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9d2c0c7820 0x7f9d2c0c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 >> 172.21.15.73:0/1537003476 conn(0x7f9d2c062700 msgr2=0x7f9d2c15a210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 shutdown_connections 2023-11-28T04:26:37.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:37.926+0000 7f9d31c12700 1 -- 172.21.15.73:0/1537003476 wait complete. 2023-11-28T04:26:37.930 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:37.929+0000 7fd377991680 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-11-28T04:26:37.940 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-11-28T04:26:38.046 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.045+0000 7fd377991680 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-11-28T04:26:38.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.056+0000 7fdfd40fe700 1 Processor -- start 2023-11-28T04:26:38.058 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.056+0000 7fdfd40fe700 1 -- start start 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd40fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc0c8780 0x7fdfcc0c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd40fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfcc062700 0x7fdfcc062bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd40fe700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfcc0630a0 con 0x7fdfcc13be20 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd40fe700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c7500 con 0x7fdfcc062700 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd40fe700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0bd650 con 0x7fdfcc0c8780 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd269b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdfcc062700 0x7fdfcc062bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd1e9a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdfcc13be20 0x7fdfcc0bcf40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39078/0 (socket says 172.21.15.73:39078) 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd1e9a700 1 -- 172.21.15.73:0/1657311099 learned_addr learned my addr 172.21.15.73:0/1657311099 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.057+0000 7fdfd1699700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc0c8780 0x7fdfcc0c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd269b700 1 -- 172.21.15.73:0/1657311099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc0c8780 msgr2=0x7fdfcc0c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd269b700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc0c8780 0x7fdfcc0c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd269b700 1 -- 172.21.15.73:0/1657311099 >> v1:172.21.15.73:6789/0 conn(0x7fdfcc13be20 legacy=0x7fdfcc0bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd0e98700 1 -- 172.21.15.73:0/1657311099 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4163871319 0 0) 0x7fdfbc001110 con 0x7fdfcc13be20 2023-11-28T04:26:38.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd269b700 1 -- 172.21.15.73:0/1657311099 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfcc13c200 con 0x7fdfcc062700 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd269b700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc062700 0x7fdfcc062bb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7fdfc4009e60 tx=0x7fdfc4009780).ready entity=mon.0 client_cookie=b5a0319b79b3b0f1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd0e98700 1 -- 172.21.15.73:0/1657311099 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400e070 con 0x7fdfcc062700 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd0e98700 1 -- 172.21.15.73:0/1657311099 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc400bb60 con 0x7fdfcc062700 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.058+0000 7fdfd0e98700 1 -- 172.21.15.73:0/1657311099 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc4012410 con 0x7fdfcc062700 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 -- 172.21.15.73:0/1657311099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc062700 msgr2=0x7fdfcc062bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc062700 0x7fdfcc062bb0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 rx=0x7fdfc4009e60 tx=0x7fdfc4009780).stop 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 -- 172.21.15.73:0/1657311099 shutdown_connections 2023-11-28T04:26:38.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc062700 0x7fdfcc062bb0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/1657311099 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc0c8780 0x7fdfcc0c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 -- 172.21.15.73:0/1657311099 >> 172.21.15.73:0/1657311099 conn(0x7fdfcc0642d0 msgr2=0x7fdfcc0c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 -- 172.21.15.73:0/1657311099 shutdown_connections 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.059+0000 7fdfd40fe700 1 -- 172.21.15.73:0/1657311099 wait complete. 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.060+0000 7fdfd40fe700 1 Processor -- start 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.060+0000 7fdfd40fe700 1 -- start start 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc062700 0x7fdfcc0c2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.071 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfcc13be20 0x7fdfcc1383e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c46a0 con 0x7fdfcc0c8780 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c2c60 con 0x7fdfcc13be20 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfcc0c2f40 con 0x7fdfcc062700 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35726/0 (socket says 172.21.15.73:35726) 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 -- 172.21.15.73:0/3062502740 learned_addr learned my addr 172.21.15.73:0/3062502740 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1e9a700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc062700 0x7fdfcc0c2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd269b700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfcc13be20 0x7fdfcc1383e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.072 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc062700 msgr2=0x7fdfcc0c2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc062700 0x7fdfcc0c2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfcc13be20 msgr2=0x7fdfcc1383e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfcc13be20 0x7fdfcc1383e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 -- 172.21.15.73:0/3062502740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfc4009a20 con 0x7fdfcc0c8780 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd1699700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7fdfc800e900 tx=0x7fdfc8009060).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8012600 con 0x7fdfcc0c8780 2023-11-28T04:26:38.073 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfc8012760 con 0x7fdfcc0c8780 2023-11-28T04:26:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfcc0c3220 con 0x7fdfcc0c8780 2023-11-28T04:26:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.061+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfc8012ae0 con 0x7fdfcc0c8780 2023-11-28T04:26:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.062+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfcc0c0400 con 0x7fdfcc0c8780 2023-11-28T04:26:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdfcc06c860 con 0x7fdfcc0c8780 2023-11-28T04:26:38.074 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 101) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fdfc801b400 con 0x7fdfcc0c8780 2023-11-28T04:26:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfc1ffb700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 0x7fdfb00617c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6293+0+0 (secure 0 0 0) 0x7fdfc807a980 con 0x7fdfcc0c8780 2023-11-28T04:26:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfd1e9a700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 msgr2=0x7fdfb00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:38.075 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.063+0000 7fdfd1e9a700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 0x7fdfb00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.200000 2023-11-28T04:26:38.076 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.067+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdfc804b940 con 0x7fdfcc0c8780 2023-11-28T04:26:38.162 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:38.161+0000 7fddf3802680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:38.233 INFO:tasks.ceph.mgr.x.smithi150.stderr:2023-11-28T04:26:38.233+0000 7fddf3802680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:38.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.264+0000 7fdfd1e9a700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 msgr2=0x7fdfb00617c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.73:6832/136129 2023-11-28T04:26:38.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.264+0000 7fdfd1e9a700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 0x7fdfb00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._fault waiting 0.400000 2023-11-28T04:26:38.276 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.275+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fdfcc0c68f0 con 0x7fdfcc0c8780 2023-11-28T04:26:38.287 INFO:tasks.ceph.mon.a.smithi073.stderr:2023-11-28T04:26:38.286+0000 7f2d54436700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-11-28T04:26:38.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.291+0000 7fdfc1ffb700 1 -- 172.21.15.73:0/3062502740 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v77) v1 ==== 161+0+0 (secure 0 0 0) 0x7fdfc8016020 con 0x7fdfcc0c8780 2023-11-28T04:26:38.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 msgr2=0x7fdfb00617c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-11-28T04:26:38.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 0x7fdfb00617c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 msgr2=0x7fdfcc0c41b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 rx=0x7fdfc800e900 tx=0x7fdfc8009060).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 shutdown_connections 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdfb005f300 0x7fdfb00617c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdfcc13be20 0x7fdfcc1383e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdfcc062700 0x7fdfcc0c2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 --2- 172.21.15.73:0/3062502740 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdfcc0c8780 0x7fdfcc0c41b0 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.295 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 >> 172.21.15.73:0/3062502740 conn(0x7fdfcc0642d0 msgr2=0x7fdfcc0a4db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:38.296 INFO:tasks.ceph.mds.a.smithi073.stderr: -2203> 2023-11-28T04:26:25.624+0000 7fd923066700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-11-28T04:26:38.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 shutdown_connections 2023-11-28T04:26:38.296 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.293+0000 7fdfd40fe700 1 -- 172.21.15.73:0/3062502740 wait complete. 2023-11-28T04:26:38.296 INFO:teuthology.orchestra.run.smithi073.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-11-28T04:26:38.309 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-11-28T04:26:38.317 INFO:tasks.ceph.mds.a.smithi073.stdout:starting mds.a at 2023-11-28T04:26:38.351 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.350+0000 7fd377991680 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-11-28T04:26:38.427 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.426+0000 7fd377991680 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-11-28T04:26:38.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.428+0000 7fdf08e9d700 1 Processor -- start 2023-11-28T04:26:38.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.428+0000 7fdf08e9d700 1 -- start start 2023-11-28T04:26:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf08e9d700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fdf040c6950 0x7fdf040bc6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf08e9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdf0415c170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf08e9d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf04064090 con 0x7fdf04152510 2023-11-28T04:26:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf08e9d700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf04063480 con 0x7fdf040c6950 2023-11-28T04:26:38.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf08e9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04063760 con 0x7fdf040bcbb0 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdf04152510 0x7fdf040a67e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39080/0 (socket says 172.21.15.73:39080) 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0259c700 1 -- 172.21.15.73:0/3476728873 learned_addr learned my addr 172.21.15.73:0/3476728873 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf02d9d700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdf0415c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf01d9b700 1 --2- 172.21.15.73:0/3476728873 >> v2:172.21.15.150:3300/0 conn(0x7fdf040c6950 0x7fdf040bc6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3112118863 0 0) 0x7fdf04064090 con 0x7fdf04152510 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdeec0033f0 con 0x7fdf04152510 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1075589541 0 0) 0x7fdeec0033f0 con 0x7fdf04152510 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 msgr2=0x7fdf0415c170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdf0415c170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 >> v2:172.21.15.150:3300/0 conn(0x7fdf040c6950 msgr2=0x7fdf040bc6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:38.434 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 --2- 172.21.15.73:0/3476728873 >> v2:172.21.15.150:3300/0 conn(0x7fdf040c6950 0x7fdf040bc6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf04063a40 con 0x7fdf04152510 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.429+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fdef40015f0 con 0x7fdf04152510 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 >> v1:172.21.15.73:6789/0 conn(0x7fdf04152510 legacy=0x7fdf040a67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeec0038e0 0x7fdeec003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdeec004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec004b80 0x7fdeec005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04063480 con 0x7fdeec0038e0 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04063760 con 0x7fdeec004b80 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdeec00b8e0 con 0x7fdf040bcbb0 2023-11-28T04:26:38.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeec0038e0 0x7fdeec003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf01d9b700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdeec004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf02d9d700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec004b80 0x7fdeec005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 -- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 msgr2=0x7fdeec004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040bcbb0 0x7fdeec004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 -- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec004b80 msgr2=0x7fdeec005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec004b80 0x7fdeec005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 -- 172.21.15.73:0/3476728873 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeec0033f0 con 0x7fdeec0038e0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0259c700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeec0038e0 0x7fdeec003d00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 rx=0x7fdef400c210 tx=0x7fdef40077a0).ready entity=mon.0 client_cookie=c3b190946e52e000 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef40079d0 con 0x7fdeec0038e0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdef400cae0 con 0x7fdeec0038e0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.430+0000 7fdf0159a700 1 -- 172.21.15.73:0/3476728873 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdef400ce60 con 0x7fdeec0038e0 2023-11-28T04:26:38.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 -- 172.21.15.73:0/3476728873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeec0038e0 msgr2=0x7fdeec003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/3476728873 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdeec0038e0 0x7fdeec003d00 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 rx=0x7fdef400c210 tx=0x7fdef40077a0).stop 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf02d9d700 1 -- 172.21.15.73:0/3476728873 reap_dead start 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 -- 172.21.15.73:0/3476728873 shutdown_connections 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 -- 172.21.15.73:0/3476728873 >> 172.21.15.73:0/3476728873 conn(0x7fdf040c8230 msgr2=0x7fdf040c4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 -- 172.21.15.73:0/3476728873 shutdown_connections 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.431+0000 7fdf08e9d700 1 -- 172.21.15.73:0/3476728873 wait complete. 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.432+0000 7fdf08e9d700 1 Processor -- start 2023-11-28T04:26:38.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.432+0000 7fdf08e9d700 1 -- start start 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 0x7fdf0414e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040c35c0 0x7fdf040c2730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf041528f0 con 0x7fdf040a8ad0 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf04153190 con 0x7fdeec0038e0 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf040a9440 con 0x7fdf040c35c0 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35750/0 (socket says 172.21.15.73:35750) 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 -- 172.21.15.73:0/1576914055 learned_addr learned my addr 172.21.15.73:0/1576914055 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf01d9b700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040c35c0 0x7fdf040c2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.440 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf0259c700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 0x7fdf0414e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 -- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040c35c0 msgr2=0x7fdf040c2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040c35c0 0x7fdf040c2730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 -- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 msgr2=0x7fdf0414e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:38.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 0x7fdf0414e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.441 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 -- 172.21.15.73:0/1576914055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf0414ed30 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf0259c700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 0x7fdf0414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf02d9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7fdefc0094d0 tx=0x7fdefc00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00e040 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdefc00f040 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf040c2fc0 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdefc00fe50 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.433+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf04152e20 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.434+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0406c860 con 0x7fdf040a8ad0 2023-11-28T04:26:38.442 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.435+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 102) v1 ==== 75605+0+0 (secure 0 0 0) 0x7fdefc015660 con 0x7fdf040a8ad0 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.436+0000 7fdef27fc700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.436+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6379+0+0 (secure 0 0 0) 0x7fdefc07b0c0 con 0x7fdf040a8ad0 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.437+0000 7fdf0259c700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.437+0000 7fdf0259c700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello connection peer type does not match what peer advertises 16 != 2 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.437+0000 7fdf0259c700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.437+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 msgr2=0x7fdee4061790 unknown :-1 s=STATE_CLOSED l=1).mark_down 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.437+0000 7fdef27fc700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:38.443 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.440+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdefc04bf80 con 0x7fdf040a8ad0 2023-11-28T04:26:38.450 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.446+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 103) v1 ==== 75270+0+0 (secure 0 0 0) 0x7fdefc078a80 con 0x7fdf040a8ad0 2023-11-28T04:26:38.491 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f2123700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f2123700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f2123700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f2123700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f2123700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f5129700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f5129700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f5129700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f5129700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.492 INFO:tasks.ceph.mgr.y.smithi073.stderr:2023-11-28T04:26:38.490+0000 7fd2f5129700 -1 client.0 error registering admin socket command: (17) File exists 2023-11-28T04:26:38.659 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:38.658+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fdf04063d20 con 0x7fdf040a8ad0 2023-11-28T04:26:39.301 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.300+0000 7fdef27fc700 1 -- 172.21.15.73:0/1576914055 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v78) v1 ==== 104+0+0 (secure 0 0 0) 0x7fdefc06fac0 con 0x7fdf040a8ad0 2023-11-28T04:26:39.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 msgr2=0x7fdf040a8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:39.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 rx=0x7fdefc0094d0 tx=0x7fdefc00b040).stop 2023-11-28T04:26:39.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 shutdown_connections 2023-11-28T04:26:39.303 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:6832/136129,v1:172.21.15.73:6833/136129] conn(0x7fdee405f2d0 0x7fdee4061790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdf040a8ad0 0x7fdf040a8f50 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdeec0038e0 0x7fdf0414e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 --2- 172.21.15.73:0/1576914055 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdf040c35c0 0x7fdf040c2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 >> 172.21.15.73:0/1576914055 conn(0x7fdf040c8230 msgr2=0x7fdf04150540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 shutdown_connections 2023-11-28T04:26:39.304 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.302+0000 7fdf08e9d700 1 -- 172.21.15.73:0/1576914055 wait complete. 2023-11-28T04:26:39.317 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-11-28T04:26:39.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.481+0000 7f58efa2a700 1 Processor -- start 2023-11-28T04:26:39.482 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.481+0000 7f58efa2a700 1 -- start start 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58efa2a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e8154530 0x7f58e8154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58efa2a700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58e80a7ff0 0x7f58e80c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58efa2a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58e80c6630 con 0x7f58e80aa220 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58efa2a700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58e80c6390 con 0x7f58e80a7ff0 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58efa2a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80a8470 con 0x7f58e8154530 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58ecfc5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58e80a7ff0 0x7f58e80c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58edfc7700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f58e80aa220 0x7f58e80c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39088/0 (socket says 172.21.15.73:39088) 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58ecfc5700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f58e80a7ff0 0x7f58e80c8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35810/0 (socket says 172.21.15.73:35810) 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58edfc7700 1 -- 172.21.15.73:0/1609417416 learned_addr learned my addr 172.21.15.73:0/1609417416 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.482+0000 7f58ed7c6700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e8154530 0x7f58e8154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58ecfc5700 1 -- 172.21.15.73:0/1609417416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e8154530 msgr2=0x7f58e8154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:39.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58ecfc5700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e8154530 0x7f58e8154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58ecfc5700 1 -- 172.21.15.73:0/1609417416 >> v1:172.21.15.73:6789/0 conn(0x7f58e80aa220 legacy=0x7f58e80c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58ecfc5700 1 -- 172.21.15.73:0/1609417416 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e8064230 con 0x7f58e80a7ff0 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58dffff700 1 -- 172.21.15.73:0/1609417416 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 624066563 0 0) 0x7f58e80c6630 con 0x7f58e80aa220 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58ecfc5700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80a7ff0 0x7f58e80c8580 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 rx=0x7f58d8009760 tx=0x7f58d8009060).ready entity=mon.0 client_cookie=3cf6b8fc5ffb7052 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58dffff700 1 -- 172.21.15.73:0/1609417416 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800b580 con 0x7f58e80a7ff0 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58dffff700 1 -- 172.21.15.73:0/1609417416 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d800b6e0 con 0x7f58e80a7ff0 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58dffff700 1 -- 172.21.15.73:0/1609417416 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800ba60 con 0x7f58e80a7ff0 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 -- 172.21.15.73:0/1609417416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80a7ff0 msgr2=0x7f58e80c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80a7ff0 0x7f58e80c8580 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 rx=0x7f58d8009760 tx=0x7f58d8009060).stop 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 -- 172.21.15.73:0/1609417416 shutdown_connections 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e8154530 0x7f58e8154910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 --2- 172.21.15.73:0/1609417416 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80a7ff0 0x7f58e80c8580 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 -- 172.21.15.73:0/1609417416 >> 172.21.15.73:0/1609417416 conn(0x7f58e8094930 msgr2=0x7f58e806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:39.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 -- 172.21.15.73:0/1609417416 shutdown_connections 2023-11-28T04:26:39.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.483+0000 7f58efa2a700 1 -- 172.21.15.73:0/1609417416 wait complete. 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 Processor -- start 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 -- start start 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 0x7f58e80a8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58e8154530 0x7f58e80c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80a8ff0 con 0x7f58e80aa220 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58e81651d0 con 0x7f58e8154530 2023-11-28T04:26:39.489 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.485+0000 7f58efa2a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58e80aaba0 con 0x7f58e80a7ff0 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35816/0 (socket says 172.21.15.73:35816) 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 -- 172.21.15.73:0/3547143022 learned_addr learned my addr 172.21.15.73:0/3547143022 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ed7c6700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 0x7f58e80a8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58edfc7700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58e8154530 0x7f58e80c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 -- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 msgr2=0x7f58e80a8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 0x7f58e80a8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 -- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58e8154530 msgr2=0x7f58e80c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58e8154530 0x7f58e80c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 -- 172.21.15.73:0/3547143022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58d80092a0 con 0x7f58e80aa220 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ed7c6700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 0x7f58e80a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:39.490 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58ecfc5700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7f58e806ae10 tx=0x7f58d8017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d800b850 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58d800b9b0 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58e8164b80 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58d801b620 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.486+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58e8164890 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.487+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58e806c860 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.487+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f58d801ba10 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.488+0000 7f58dd7fa700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 0x7f58cc0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.488+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f58d807bbb0 con 0x7f58e80aa220 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.488+0000 7f58ed7c6700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 0x7f58cc0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:39.491 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.488+0000 7f58ed7c6700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 0x7f58cc0614f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f58e400baa0 tx=0x7f58e4008040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:39.493 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.492+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f58d8082080 con 0x7f58e80aa220 2023-11-28T04:26:39.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:39.657+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f58e8064510 con 0x7f58e80aa220 2023-11-28T04:26:40.455 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.454+0000 7f58dd7fa700 1 -- 172.21.15.73:0/3547143022 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v216) v1 ==== 181+0+0 (secure 0 0 0) 0x7f58d8029020 con 0x7f58e80aa220 2023-11-28T04:26:40.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 msgr2=0x7f58cc0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 0x7f58cc0614f0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 rx=0x7f58e400baa0 tx=0x7f58e4008040).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 msgr2=0x7f58e80a8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 rx=0x7f58e806ae10 tx=0x7f58d8017000).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 shutdown_connections 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f58cc05f030 0x7f58cc0614f0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f58e8154530 0x7f58e80c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f58e80a7ff0 0x7f58e80a8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 --2- 172.21.15.73:0/3547143022 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f58e80aa220 0x7f58e80a8b00 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 >> 172.21.15.73:0/3547143022 conn(0x7f58e8094930 msgr2=0x7f58e80c90b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:40.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 shutdown_connections 2023-11-28T04:26:40.461 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.458+0000 7f58efa2a700 1 -- 172.21.15.73:0/3547143022 wait complete. 2023-11-28T04:26:40.461 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_metadata' removed 2023-11-28T04:26:40.473 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-11-28T04:26:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.591+0000 7f2d82ffd700 1 Processor -- start 2023-11-28T04:26:40.592 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.591+0000 7f2d82ffd700 1 -- start start 2023-11-28T04:26:40.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d82ffd700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f2d7c154530 0x7f2d7c154910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d7c0c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d82ffd700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d7c0c6630 con 0x7f2d7c0aa220 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d82ffd700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d7c0c6390 con 0x7f2d7c154530 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d82ffd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0a8470 con 0x7f2d7c0a7ff0 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d8159a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2d7c0aa220 0x7f2d7c0c6ac0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39098/0 (socket says 172.21.15.73:39098) 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d8159a700 1 -- 172.21.15.73:0/1668044013 learned_addr learned my addr 172.21.15.73:0/1668044013 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d80d99700 1 --2- 172.21.15.73:0/1668044013 >> v2:172.21.15.150:3300/0 conn(0x7f2d7c154530 0x7f2d7c154910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d7bfff700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d7c0c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 534752615 0 0) 0x7f2d7c0c6630 con 0x7f2d7c0aa220 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d680033f0 con 0x7f2d7c0aa220 2023-11-28T04:26:40.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.592+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1308388865 0 0) 0x7f2d680033f0 con 0x7f2d7c0aa220 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 msgr2=0x7f2d7c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d7c0c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 >> v2:172.21.15.150:3300/0 conn(0x7f2d7c154530 msgr2=0x7f2d7c154910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 --2- 172.21.15.73:0/1668044013 >> v2:172.21.15.150:3300/0 conn(0x7f2d7c154530 0x7f2d7c154910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c064230 con 0x7f2d7c0aa220 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f2d64001470 con 0x7f2d7c0aa220 2023-11-28T04:26:40.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 >> v1:172.21.15.73:6789/0 conn(0x7f2d7c0aa220 legacy=0x7f2d7c0c6ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d680038e0 0x7f2d68003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d68004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d68004b80 0x7f2d68005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0c6390 con 0x7f2d68004b80 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0a8470 con 0x7f2d680038e0 2023-11-28T04:26:40.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d6800b8e0 con 0x7f2d7c0a7ff0 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d68004b80 0x7f2d68005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d80d99700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d680038e0 0x7f2d68003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7bfff700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d68004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 -- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 msgr2=0x7f2d68004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0a7ff0 0x7f2d68004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 -- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d680038e0 msgr2=0x7f2d68003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d680038e0 0x7f2d68003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 -- 172.21.15.73:0/1668044013 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d680033f0 con 0x7f2d68004b80 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d8159a700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d68004b80 0x7f2d68005000 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f2d6400c210 tx=0x7f2d64001e90).ready entity=mon.0 client_cookie=dbf41d157f78ad0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d64007a60 con 0x7f2d68004b80 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d6400cae0 con 0x7f2d68004b80 2023-11-28T04:26:40.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.593+0000 7f2d7b7fe700 1 -- 172.21.15.73:0/1668044013 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d6400ce60 con 0x7f2d68004b80 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 -- 172.21.15.73:0/1668044013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d68004b80 msgr2=0x7f2d68005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/1668044013 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d68004b80 0x7f2d68005000 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 rx=0x7f2d6400c210 tx=0x7f2d64001e90).stop 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d8159a700 1 -- 172.21.15.73:0/1668044013 reap_dead start 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 -- 172.21.15.73:0/1668044013 shutdown_connections 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 -- 172.21.15.73:0/1668044013 >> 172.21.15.73:0/1668044013 conn(0x7f2d7c094930 msgr2=0x7f2d7c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 -- 172.21.15.73:0/1668044013 shutdown_connections 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.594+0000 7f2d82ffd700 1 -- 172.21.15.73:0/1668044013 wait complete. 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.595+0000 7f2d82ffd700 1 Processor -- start 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.595+0000 7f2d82ffd700 1 -- start start 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0c9c80 0x7f2d7c151720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 0x7f2d7c164600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c0b09d0 con 0x7f2d7c150580 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c151200 con 0x7f2d7c151c10 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7c152070 con 0x7f2d7c0c9c80 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35838/0 (socket says 172.21.15.73:35838) 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 -- 172.21.15.73:0/692151110 learned_addr learned my addr 172.21.15.73:0/692151110 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d8159a700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 0x7f2d7c164600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d7bfff700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0c9c80 0x7f2d7c151720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 -- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0c9c80 msgr2=0x7f2d7c151720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0c9c80 0x7f2d7c151720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 -- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 msgr2=0x7f2d7c164600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 0x7f2d7c164600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 -- 172.21.15.73:0/692151110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d7c0ae510 con 0x7f2d7c150580 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d8159a700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 0x7f2d7c164600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:40.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d80d99700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 rx=0x7f2d740119c0 tx=0x7f2d74006f40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400f940 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d7400faa0 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7c0b0c20 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.596+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7400fe20 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.597+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7c150e90 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d7c0b1140 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 104) v1 ==== 75340+0+0 (secure 0 0 0) 0x7f2d74014650 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d78ff9700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 0x7f2d5c0614f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 6156+0+0 (secure 0 0 0) 0x7f2d7407a8d0 con 0x7f2d7c150580 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d7bfff700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 0x7f2d5c0614f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:40.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.598+0000 7f2d7bfff700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 0x7f2d5c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2d6c009fd0 tx=0x7f2d6c00b040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:40.607 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.602+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2d7c0b1140 con 0x7f2d7c150580 2023-11-28T04:26:40.766 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:40.765+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f2d7c0b1140 con 0x7f2d7c150580 2023-11-28T04:26:41.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.455+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2d74078850 con 0x7f2d7c150580 2023-11-28T04:26:41.471 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.470+0000 7f2d78ff9700 1 -- 172.21.15.73:0/692151110 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v217) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2d74070a80 con 0x7f2d7c150580 2023-11-28T04:26:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 msgr2=0x7f2d5c0614f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.474 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 0x7f2d5c0614f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 rx=0x7f2d6c009fd0 tx=0x7f2d6c00b040).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 msgr2=0x7f2d7c0c9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 rx=0x7f2d740119c0 tx=0x7f2d74006f40).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 shutdown_connections 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2d5c05f030 0x7f2d5c0614f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2d7c151c10 0x7f2d7c164600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2d7c150580 0x7f2d7c0c9790 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 --2- 172.21.15.73:0/692151110 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2d7c0c9c80 0x7f2d7c151720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 >> 172.21.15.73:0/692151110 conn(0x7f2d7c094930 msgr2=0x7f2d7c0c1400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 shutdown_connections 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.473+0000 7f2d82ffd700 1 -- 172.21.15.73:0/692151110 wait complete. 2023-11-28T04:26:41.475 INFO:teuthology.orchestra.run.smithi073.stderr:pool 'cephfs_data' removed 2023-11-28T04:26:41.487 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-11-28T04:26:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.601+0000 7fe1f73ed700 1 Processor -- start 2023-11-28T04:26:41.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.601+0000 7fe1f73ed700 1 -- start start 2023-11-28T04:26:41.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f73ed700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fe1f0098d10 0x7fe1f00c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f73ed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1f0063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f73ed700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1f00c72d0 con 0x7fe1f0064350 2023-11-28T04:26:41.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f73ed700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1f0154bc0 con 0x7fe1f0098d10 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f73ed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c66c0 con 0x7fe1f00c6190 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f598a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fe1f0064350 0x7fe1f0158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39104/0 (socket says 172.21.15.73:39104) 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f598a700 1 -- 172.21.15.73:0/1215840699 learned_addr learned my addr 172.21.15.73:0/1215840699 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 148210456 0 0) 0x7fe1f00c72d0 con 0x7fe1f0064350 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f4988700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1f0063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.615 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1f5189700 1 --2- 172.21.15.73:0/1215840699 >> v2:172.21.15.150:3300/0 conn(0x7fe1f0098d10 0x7fe1f00c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe1dc0033f0 con 0x7fe1f0064350 2023-11-28T04:26:41.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 304290819 0 0) 0x7fe1dc0033f0 con 0x7fe1f0064350 2023-11-28T04:26:41.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 msgr2=0x7fe1f0063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1f0063e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.616 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 >> v2:172.21.15.150:3300/0 conn(0x7fe1f0098d10 msgr2=0x7fe1f00c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 --2- 172.21.15.73:0/1215840699 >> v2:172.21.15.150:3300/0 conn(0x7fe1f0098d10 0x7fe1f00c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f00c69a0 con 0x7fe1f0064350 2023-11-28T04:26:41.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7fe1ec001470 con 0x7fe1f0064350 2023-11-28T04:26:41.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.602+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 >> v1:172.21.15.73:6789/0 conn(0x7fe1f0064350 legacy=0x7fe1f0158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.617 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1dc0038e0 0x7fe1dc003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1dc004690 0x7fe1dc004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1dc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0154bc0 con 0x7fe1dc004690 2023-11-28T04:26:41.618 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f00c66c0 con 0x7fe1dc0038e0 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1dc00b8e0 con 0x7fe1f00c6190 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1dc004690 0x7fe1dc004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f5189700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1dc0038e0 0x7fe1dc003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f598a700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1dc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 -- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 msgr2=0x7fe1dc005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.619 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1f00c6190 0x7fe1dc005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 -- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1dc0038e0 msgr2=0x7fe1dc003d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1dc0038e0 0x7fe1dc003d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 -- 172.21.15.73:0/1215840699 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1dc0033f0 con 0x7fe1dc004690 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1f4988700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1dc004690 0x7fe1dc004b10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 rx=0x7fe1f00c65f0 tx=0x7fe1e0009060).ready entity=mon.0 client_cookie=3ae2dea89d85978e server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e0009e90 con 0x7fe1dc004690 2023-11-28T04:26:41.620 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e0009430 con 0x7fe1dc004690 2023-11-28T04:26:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.603+0000 7fe1e7fff700 1 -- 172.21.15.73:0/1215840699 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e000e850 con 0x7fe1dc004690 2023-11-28T04:26:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 -- 172.21.15.73:0/1215840699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1dc004690 msgr2=0x7fe1dc004b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/1215840699 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1dc004690 0x7fe1dc004b10 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 rx=0x7fe1f00c65f0 tx=0x7fe1e0009060).stop 2023-11-28T04:26:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f598a700 1 -- 172.21.15.73:0/1215840699 reap_dead start 2023-11-28T04:26:41.621 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 -- 172.21.15.73:0/1215840699 shutdown_connections 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 -- 172.21.15.73:0/1215840699 >> 172.21.15.73:0/1215840699 conn(0x7fe1f00cb310 msgr2=0x7fe1f00cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 -- 172.21.15.73:0/1215840699 shutdown_connections 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.604+0000 7fe1f73ed700 1 -- 172.21.15.73:0/1215840699 wait complete. 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.605+0000 7fe1f73ed700 1 Processor -- start 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.605+0000 7fe1f73ed700 1 -- start start 2023-11-28T04:26:41.622 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1dc004690 0x7fe1f00ae2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1f00ae7b0 0x7fe1f00b03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0150260 con 0x7fe1f0155cd0 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0156d80 con 0x7fe1f00ae7b0 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1f0156170 con 0x7fe1dc004690 2023-11-28T04:26:41.623 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35870/0 (socket says 172.21.15.73:35870) 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 -- 172.21.15.73:0/607437712 learned_addr learned my addr 172.21.15.73:0/607437712 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f5189700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1dc004690 0x7fe1f00ae2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f598a700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1f00ae7b0 0x7fe1f00b03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 -- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1dc004690 msgr2=0x7fe1f00ae2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.624 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1dc004690 0x7fe1f00ae2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 -- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1f00ae7b0 msgr2=0x7fe1f00b03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1f00ae7b0 0x7fe1f00b03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 -- 172.21.15.73:0/607437712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1f0156450 con 0x7fe1f0155cd0 2023-11-28T04:26:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f4988700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 rx=0x7fe1e000e4b0 tx=0x7fe1e001a000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.625 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e001d8d0 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1e001da50 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1f0156730 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1e001de10 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.606+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1f00b08a0 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.607+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1f006c860 con 0x7fe1f0155cd0 2023-11-28T04:26:41.626 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.608+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fe1e001b660 con 0x7fe1f0155cd0 2023-11-28T04:26:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.608+0000 7fe1e57fa700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 0x7fe1d4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.608+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fe1e007ba70 con 0x7fe1f0155cd0 2023-11-28T04:26:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.608+0000 7fe1f5189700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 0x7fe1d4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.608+0000 7fe1f5189700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 0x7fe1d4061610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe1e80094d0 tx=0x7fe1e800b040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.627 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.612+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fe1e004cec0 con 0x7fe1f0155cd0 2023-11-28T04:26:41.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.782+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fe1f00c6c80 con 0x7fe1f0155cd0 2023-11-28T04:26:41.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.785+0000 7fe1e57fa700 1 -- 172.21.15.73:0/607437712 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v44) v1 ==== 120+0+0 (secure 0 0 0) 0x7fe1e0016070 con 0x7fe1f0155cd0 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.787+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 msgr2=0x7fe1d4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.787+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 0x7fe1d4061610 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 rx=0x7fe1e80094d0 tx=0x7fe1e800b040).stop 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.787+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 msgr2=0x7fe1f00afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.787+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 rx=0x7fe1e000e4b0 tx=0x7fe1e001a000).stop 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 shutdown_connections 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fe1d405f150 0x7fe1d4061610 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fe1f00ae7b0 0x7fe1f00b03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fe1dc004690 0x7fe1f00ae2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 --2- 172.21.15.73:0/607437712 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fe1f0155cd0 0x7fe1f00afec0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 >> 172.21.15.73:0/607437712 conn(0x7fe1f00cb310 msgr2=0x7fe1f0151a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:41.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 shutdown_connections 2023-11-28T04:26:41.790 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.788+0000 7fe1f73ed700 1 -- 172.21.15.73:0/607437712 wait complete. 2023-11-28T04:26:41.801 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-11-28T04:26:41.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.914+0000 7fa0effd0700 1 Processor -- start 2023-11-28T04:26:41.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.914+0000 7fa0effd0700 1 -- start start 2023-11-28T04:26:41.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0effd0700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa0e8098d10 0x7fa0e80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0effd0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e8158320 0x7fa0e81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.918 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0effd0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa0e8158cc0 con 0x7fa0e80c6190 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0effd0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e80c72d0 con 0x7fa0e8158320 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0effd0700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa0e8064570 con 0x7fa0e8098d10 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e8158320 0x7fa0e81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fa0e8158320 0x7fa0e81587d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35884/0 (socket says 172.21.15.73:35884) 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 -- 172.21.15.73:0/2521336632 learned_addr learned my addr 172.21.15.73:0/2521336632 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0edd6c700 1 --2- 172.21.15.73:0/2521336632 >> v2:172.21.15.150:3300/0 conn(0x7fa0e8098d10 0x7fa0e80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 -- 172.21.15.73:0/2521336632 >> v2:172.21.15.150:3300/0 conn(0x7fa0e8098d10 msgr2=0x7fa0e80c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 --2- 172.21.15.73:0/2521336632 >> v2:172.21.15.150:3300/0 conn(0x7fa0e8098d10 0x7fa0e80c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 -- 172.21.15.73:0/2521336632 >> v1:172.21.15.150:6789/0 conn(0x7fa0e80c6190 legacy=0x7fa0e8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 -- 172.21.15.73:0/2521336632 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e8154bc0 con 0x7fa0e8158320 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ee56d700 1 --2- 172.21.15.73:0/2521336632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e81587d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 rx=0x7fa0e8083f80 tx=0x7fa0e40090f0).ready entity=mon.0 client_cookie=5e9e1444be129299 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ecd6a700 1 -- 172.21.15.73:0/2521336632 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400b5a0 con 0x7fa0e8158320 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ecd6a700 1 -- 172.21.15.73:0/2521336632 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400b700 con 0x7fa0e8158320 2023-11-28T04:26:41.919 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.915+0000 7fa0ecd6a700 1 -- 172.21.15.73:0/2521336632 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400ba80 con 0x7fa0e8158320 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 -- 172.21.15.73:0/2521336632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 msgr2=0x7fa0e81587d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 --2- 172.21.15.73:0/2521336632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e81587d0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 rx=0x7fa0e8083f80 tx=0x7fa0e40090f0).stop 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 -- 172.21.15.73:0/2521336632 shutdown_connections 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 --2- 172.21.15.73:0/2521336632 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e81587d0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 --2- 172.21.15.73:0/2521336632 >> v2:172.21.15.150:3300/0 conn(0x7fa0e8098d10 0x7fa0e80c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 -- 172.21.15.73:0/2521336632 >> 172.21.15.73:0/2521336632 conn(0x7fa0e80cb310 msgr2=0x7fa0e80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 -- 172.21.15.73:0/2521336632 shutdown_connections 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.916+0000 7fa0effd0700 1 -- 172.21.15.73:0/2521336632 wait complete. 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.917+0000 7fa0effd0700 1 Processor -- start 2023-11-28T04:26:41.920 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.917+0000 7fa0effd0700 1 -- start start 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0e8098d10 0x7fa0e80c42a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0e80c6190 0x7fa0e8156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c5550 con 0x7fa0e8158320 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e81566f0 con 0x7fa0e80c6190 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0effd0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e80c4940 con 0x7fa0e8098d10 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35888/0 (socket says 172.21.15.73:35888) 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 -- 172.21.15.73:0/299911870 learned_addr learned my addr 172.21.15.73:0/299911870 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0edd6c700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0e8098d10 0x7fa0e80c42a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ed56b700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0e80c6190 0x7fa0e8156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.923 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 -- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0e8098d10 msgr2=0x7fa0e80c42a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0e8098d10 0x7fa0e80c42a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 -- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0e80c6190 msgr2=0x7fa0e8156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0e80c6190 0x7fa0e8156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 -- 172.21.15.73:0/299911870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0e4009370 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.918+0000 7fa0ee56d700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 rx=0x7fa0e4006df0 tx=0x7fa0e4015000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.919+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e400ba80 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.919+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0e400bec0 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.919+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e80c4c20 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.919+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0e401c650 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.919+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e80c51e0 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.920+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e806c860 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.920+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa0e401ca00 con 0x7fa0e8158320 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.920+0000 7fa0ddffb700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 0x7fa0cc0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:41.924 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.920+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fa0e407bdb0 con 0x7fa0e8158320 2023-11-28T04:26:41.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.920+0000 7fa0edd6c700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 0x7fa0cc0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:41.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.921+0000 7fa0edd6c700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 0x7fa0cc0615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa0e0007540 tx=0x7fa0e000b040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:41.925 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:41.924+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fa0e404d140 con 0x7fa0e8158320 2023-11-28T04:26:42.089 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.088+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fa0e8158cc0 con 0x7fa0e8158320 2023-11-28T04:26:42.092 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.091+0000 7fa0ddffb700 1 -- 172.21.15.73:0/299911870 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa0e8158cc0 con 0x7fa0e8158320 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 msgr2=0x7fa0cc0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 0x7fa0cc0615e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 rx=0x7fa0e0007540 tx=0x7fa0e000b040).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 msgr2=0x7fa0e8150280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 rx=0x7fa0e4006df0 tx=0x7fa0e4015000).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 shutdown_connections 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa0cc05f120 0x7fa0cc0615e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa0e8158320 0x7fa0e8150280 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa0e8098d10 0x7fa0e80c42a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 --2- 172.21.15.73:0/299911870 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa0e80c6190 0x7fa0e8156200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.095 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.093+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 >> 172.21.15.73:0/299911870 conn(0x7fa0e80cb310 msgr2=0x7fa0e81530f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:42.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.094+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 shutdown_connections 2023-11-28T04:26:42.096 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.094+0000 7fa0effd0700 1 -- 172.21.15.73:0/299911870 wait complete. 2023-11-28T04:26:42.107 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-11-28T04:26:42.219 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.218+0000 7f2ff744d700 1 Processor -- start 2023-11-28T04:26:42.220 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.218+0000 7f2ff744d700 1 -- start start 2023-11-28T04:26:42.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff744d700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ff01542d0 0x7f2ff00697f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff744d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0069ce0 0x7f2ff0062700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.228 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff744d700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff0065760 con 0x7f2ff0062bf0 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff744d700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff006a160 con 0x7f2ff01542d0 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff744d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0063080 con 0x7f2ff0069ce0 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff51e9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f2ff01542d0 0x7f2ff00697f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff59ea700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f2ff0062bf0 0x7f2ff0157fe0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:39106/0 (socket says 172.21.15.73:39106) 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.219+0000 7f2ff59ea700 1 -- 172.21.15.73:0/97554383 learned_addr learned my addr 172.21.15.73:0/97554383 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:42.229 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff49e8700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0069ce0 0x7f2ff0062700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff51e9700 1 -- 172.21.15.73:0/97554383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0069ce0 msgr2=0x7f2ff0062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff51e9700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0069ce0 0x7f2ff0062700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff51e9700 1 -- 172.21.15.73:0/97554383 >> v1:172.21.15.73:6789/0 conn(0x7f2ff0062bf0 legacy=0x7f2ff0157fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2feffff700 1 -- 172.21.15.73:0/97554383 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677641553 0 0) 0x7f2ff0065760 con 0x7f2ff0062bf0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff51e9700 1 -- 172.21.15.73:0/97554383 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff00c3250 con 0x7f2ff01542d0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff51e9700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00697f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7f2fe0009e60 tx=0x7f2fe0009780).ready entity=mon.0 client_cookie=d40a96e336497241 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2feffff700 1 -- 172.21.15.73:0/97554383 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe000e070 con 0x7f2ff01542d0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2feffff700 1 -- 172.21.15.73:0/97554383 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe000bc30 con 0x7f2ff01542d0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2feffff700 1 -- 172.21.15.73:0/97554383 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe0012410 con 0x7f2ff01542d0 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 -- 172.21.15.73:0/97554383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 msgr2=0x7f2ff00697f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00697f0 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 rx=0x7f2fe0009e60 tx=0x7f2fe0009780).stop 2023-11-28T04:26:42.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 -- 172.21.15.73:0/97554383 shutdown_connections 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00697f0 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 --2- 172.21.15.73:0/97554383 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0069ce0 0x7f2ff0062700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.220+0000 7f2ff744d700 1 -- 172.21.15.73:0/97554383 >> 172.21.15.73:0/97554383 conn(0x7f2ff00c7fd0 msgr2=0x7f2ff00c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.221+0000 7f2ff744d700 1 -- 172.21.15.73:0/97554383 shutdown_connections 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.221+0000 7f2ff744d700 1 -- 172.21.15.73:0/97554383 wait complete. 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.222+0000 7f2ff744d700 1 Processor -- start 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.222+0000 7f2ff744d700 1 -- start start 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.222+0000 7f2ff744d700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0062bf0 0x7f2ff00ac230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.222+0000 7f2ff744d700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0069ce0 0x7f2ff00acab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff014f440 con 0x7f2ff01542d0 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0151970 con 0x7f2ff0069ce0 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0167760 con 0x7f2ff0062bf0 2023-11-28T04:26:42.231 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:35904/0 (socket says 172.21.15.73:35904) 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 -- 172.21.15.73:0/3048950391 learned_addr learned my addr 172.21.15.73:0/3048950391 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff51e9700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0062bf0 0x7f2ff00ac230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff49e8700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0069ce0 0x7f2ff00acab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 -- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0062bf0 msgr2=0x7f2ff00ac230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0062bf0 0x7f2ff00ac230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 -- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0069ce0 msgr2=0x7f2ff00acab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0069ce0 0x7f2ff00acab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 -- 172.21.15.73:0/3048950391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe0009a20 con 0x7f2ff01542d0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff59ea700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 rx=0x7f2fe800c490 tx=0x7f2fe8007c10).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800ce00 con 0x7f2ff01542d0 2023-11-28T04:26:42.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe80072e0 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8007660 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff0150df0 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.223+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff0151180 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.224+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff01678d0 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.225+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2fe80059b0 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.225+0000 7f2fed7fa700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 0x7f2fdc061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.225+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f2fe80796c0 con 0x7f2ff01542d0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.225+0000 7f2ff51e9700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 0x7f2fdc061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:42.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.225+0000 7f2ff51e9700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 0x7f2fdc061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2fe0009e60 tx=0x7f2fe0009780).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:42.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.229+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f2ff01678d0 con 0x7f2ff01542d0 2023-11-28T04:26:42.462 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:42.460+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f2ff01678d0 con 0x7f2ff01542d0 2023-11-28T04:26:43.465 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.464+0000 7f2fed7fa700 1 -- 172.21.15.73:0/3048950391 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3022) v1 ==== 155+0+0 (secure 0 0 0) 0x7f2ff01678d0 con 0x7f2ff01542d0 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.466+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 msgr2=0x7f2fdc061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.466+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 0x7f2fdc061610 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 rx=0x7f2fe0009e60 tx=0x7f2fe0009780).stop 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.466+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 msgr2=0x7f2ff00ad990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.466+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 rx=0x7f2fe800c490 tx=0x7f2fe8007c10).stop 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 shutdown_connections 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f2fdc05f150 0x7f2fdc061610 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f2ff01542d0 0x7f2ff00ad990 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:43.468 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f2ff0062bf0 0x7f2ff00ac230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:43.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 --2- 172.21.15.73:0/3048950391 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f2ff0069ce0 0x7f2ff00acab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:43.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 >> 172.21.15.73:0/3048950391 conn(0x7f2ff00c7fd0 msgr2=0x7f2ff00c6420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:43.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 shutdown_connections 2023-11-28T04:26:43.469 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:43.467+0000 7f2ff744d700 1 -- 172.21.15.73:0/3048950391 wait complete. 2023-11-28T04:26:43.481 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-11-28T04:26:43.482 INFO:tasks.cephfs_test_runner: 2023-11-28T04:26:43.482 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-11-28T04:26:43.482 INFO:tasks.cephfs_test_runner:Ran 11 tests in 5611.174s 2023-11-28T04:26:43.482 INFO:tasks.cephfs_test_runner: 2023-11-28T04:26:43.482 INFO:tasks.cephfs_test_runner:OK 2023-11-28T04:26:43.482 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-11-28T04:26:43.493 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-11-28T04:26:43.512 INFO:tasks.kclient:Unmounting kernel clients... 2023-11-28T04:26:43.512 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.512 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:26:43.528 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-11-28T04:26:43.529 INFO:teuthology.orchestra.run.smithi073.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-11-28T04:26:43.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.529 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-11-28T04:26:43.604 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.605 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-11-28T04:26:43.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.679 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2023-11-28T04:26:43.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.764 DEBUG:teuthology.orchestra.run.smithi073:> ip netns list 2023-11-28T04:26:43.779 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.781 DEBUG:teuthology.orchestra.run.smithi073:> sudo ip link delete ceph-brx 2023-11-28T04:26:43.847 INFO:teuthology.orchestra.run.smithi073.stderr:Cannot find device "ceph-brx" 2023-11-28T04:26:43.848 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:26:43.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.849 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T04:26:43.867 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-11-28T04:26:43.867 INFO:teuthology.orchestra.run.smithi150.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-11-28T04:26:43.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.867 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-11-28T04:26:43.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:43.945 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-11-28T04:26:44.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:44.023 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-28T04:26:44.129 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:44.129 DEBUG:teuthology.orchestra.run.smithi150:> ip netns list 2023-11-28T04:26:44.146 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-11-28T04:26:44.146 DEBUG:teuthology.orchestra.run.smithi150:> sudo ip link delete ceph-brx 2023-11-28T04:26:44.214 INFO:teuthology.orchestra.run.smithi150.stderr:Cannot find device "ceph-brx" 2023-11-28T04:26:44.215 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:26:44.216 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-11-28T04:26:44.227 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-11-28T04:26:44.227 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:44.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.341+0000 7f9004998700 1 Processor -- start 2023-11-28T04:26:44.343 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.341+0000 7f9004998700 1 -- start start 2023-11-28T04:26:44.353 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.342+0000 7f9004998700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9000156a80 0x7f90000a3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.342+0000 7f9004998700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f9000068b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.342+0000 7f9004998700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9000069060 con 0x7f9000152230 2023-11-28T04:26:44.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.342+0000 7f9004998700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f90000a4290 con 0x7f9000156a80 2023-11-28T04:26:44.354 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.342+0000 7f9004998700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90000bbed0 con 0x7f90000686e0 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9000156a80 0x7f90000a3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f9000156a80 0x7f90000a3da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40024/0 (socket says 172.21.15.73:40024) 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 -- 172.21.15.73:0/643810067 learned_addr learned my addr 172.21.15.73:0/643810067 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffed9d700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f9000068b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 -- 172.21.15.73:0/643810067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 msgr2=0x7f9000068b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:44.355 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f9000068b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 -- 172.21.15.73:0/643810067 >> v1:172.21.15.150:6789/0 conn(0x7f9000152230 legacy=0x7f9000066250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 -- 172.21.15.73:0/643810067 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90000bc1b0 con 0x7f9000156a80 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffed9d700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f9000068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffdd9b700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f90000a3da0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 rx=0x7f8ff4006fd0 tx=0x7f8ff40090f0).ready entity=mon.0 client_cookie=1a455832142d710d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffd59a700 1 -- 172.21.15.73:0/643810067 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400b600 con 0x7f9000156a80 2023-11-28T04:26:44.356 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffd59a700 1 -- 172.21.15.73:0/643810067 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff400b760 con 0x7f9000156a80 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.343+0000 7f8ffd59a700 1 -- 172.21.15.73:0/643810067 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff400bae0 con 0x7f9000156a80 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 -- 172.21.15.73:0/643810067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 msgr2=0x7f90000a3da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f90000a3da0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 rx=0x7f8ff4006fd0 tx=0x7f8ff40090f0).stop 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 -- 172.21.15.73:0/643810067 shutdown_connections 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f9000068b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.357 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 --2- 172.21.15.73:0/643810067 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f90000a3da0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 -- 172.21.15.73:0/643810067 >> 172.21.15.73:0/643810067 conn(0x7f90000631d0 msgr2=0x7f90000c6ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 -- 172.21.15.73:0/643810067 shutdown_connections 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.344+0000 7f9004998700 1 -- 172.21.15.73:0/643810067 wait complete. 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.345+0000 7f9004998700 1 Processor -- start 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.345+0000 7f9004998700 1 -- start start 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f900014d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.358 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9000152230 0x7f9000163b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000152a20 con 0x7f9000156a80 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9000092430 con 0x7f9000152230 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9000092710 con 0x7f90000686e0 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40030/0 (socket says 172.21.15.73:40030) 2023-11-28T04:26:44.359 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 -- 172.21.15.73:0/2904118616 learned_addr learned my addr 172.21.15.73:0/2904118616 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffdd9b700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9000152230 0x7f9000163b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffe59c700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f900014d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 -- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 msgr2=0x7f900014d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f900014d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 -- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9000152230 msgr2=0x7f9000163b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9000152230 0x7f9000163b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.360 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 -- 172.21.15.73:0/2904118616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff4009370 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8ffed9d700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 rx=0x7f8ff800d860 tx=0x7f8ff8008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8011590 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ff80116f0 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90000929f0 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ff8011a90 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.346+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9000093c10 con 0x7f9000156a80 2023-11-28T04:26:44.361 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.347+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f900006c860 con 0x7f9000156a80 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.348+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8ff8012420 con 0x7f9000156a80 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.348+0000 7f8fee7fc700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 0x7f8fe0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.348+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f8ff8079a30 con 0x7f9000156a80 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.348+0000 7f8ffe59c700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 0x7f8fe0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.348+0000 7f8ffe59c700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 0x7f8fe0061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8ff0004290 tx=0x7f8ff0001560).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.362 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.353+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8ff804ae20 con 0x7f9000156a80 2023-11-28T04:26:44.520 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.519+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f90000bc490 con 0x7f9000156a80 2023-11-28T04:26:44.521 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.520+0000 7f8fee7fc700 1 -- 172.21.15.73:0/2904118616 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+14406 (secure 0 0 0) 0x7f8ff804e440 con 0x7f9000156a80 2023-11-28T04:26:44.521 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:44.521 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":217,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:41.458487+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2970944951":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/3498822562":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/2665669998":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/1318635231":"2023-11-29T04:26:38.441936+0000","172.21.15.73:6836/447432356":"2023-11-29T04:26:38.276635+0000","172.21.15.73:6833/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6832/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6834/447432356":"2023-11-29T04:26:38.276635+0000","172.21.15.73:0/3047075693":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/2536116355":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/1637465872":"2023-11-29T04:26:30.988881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 msgr2=0x7f8fe0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 0x7f8fe0061610 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 rx=0x7f8ff0004290 tx=0x7f8ff0001560).stop 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 msgr2=0x7f9000091a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 rx=0x7f8ff800d860 tx=0x7f8ff8008040).stop 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 shutdown_connections 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8fe005f150 0x7f8fe0061610 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.524 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9000156a80 0x7f9000091a70 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f90000686e0 0x7f900014d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 --2- 172.21.15.73:0/2904118616 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9000152230 0x7f9000163b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 >> 172.21.15.73:0/2904118616 conn(0x7f90000631d0 msgr2=0x7f90000a4ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 shutdown_connections 2023-11-28T04:26:44.525 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.522+0000 7f9004998700 1 -- 172.21.15.73:0/2904118616 wait complete. 2023-11-28T04:26:44.535 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-11-28T04:26:44.535 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:44.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.647+0000 7f2007d89700 1 Processor -- start 2023-11-28T04:26:44.648 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.647+0000 7f2007d89700 1 -- start start 2023-11-28T04:26:44.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2007d89700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2007d89700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f20000ca180 0x7f200015e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2007d89700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000a9a90 con 0x7f20000cae90 2023-11-28T04:26:44.650 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2007d89700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2000154ce0 con 0x7f20000ca180 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2007d89700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000a8e80 con 0x7f20000caab0 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f20000ca180 0x7f200015e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2005324700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f20000cae90 0x7f20000c0ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35804/0 (socket says 172.21.15.73:35804) 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f20000ca180 0x7f200015e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40046/0 (socket says 172.21.15.73:40046) 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2005324700 1 -- 172.21.15.73:0/2303083222 learned_addr learned my addr 172.21.15.73:0/2303083222 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2005b25700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 -- 172.21.15.73:0/2303083222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 msgr2=0x7f20000aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 -- 172.21.15.73:0/2303083222 >> v1:172.21.15.73:6789/0 conn(0x7f20000cae90 legacy=0x7f20000c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2004b23700 1 -- 172.21.15.73:0/2303083222 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3385471821 0 0) 0x7f20000a9a90 con 0x7f20000cae90 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 -- 172.21.15.73:0/2303083222 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000a9160 con 0x7f20000ca180 2023-11-28T04:26:44.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2006326700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f200015e3c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7f1ff8009760 tx=0x7f1ff8009060).ready entity=mon.0 client_cookie=e5b97ad1517a3130 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2004b23700 1 -- 172.21.15.73:0/2303083222 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800f510 con 0x7f20000ca180 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2004b23700 1 -- 172.21.15.73:0/2303083222 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff800f670 con 0x7f20000ca180 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.648+0000 7f2004b23700 1 -- 172.21.15.73:0/2303083222 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff800f9f0 con 0x7f20000ca180 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 -- 172.21.15.73:0/2303083222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 msgr2=0x7f200015e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f200015e3c0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 rx=0x7f1ff8009760 tx=0x7f1ff8009060).stop 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 -- 172.21.15.73:0/2303083222 shutdown_connections 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 --2- 172.21.15.73:0/2303083222 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f200015e3c0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 -- 172.21.15.73:0/2303083222 >> 172.21.15.73:0/2303083222 conn(0x7f2000094920 msgr2=0x7f200006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 -- 172.21.15.73:0/2303083222 shutdown_connections 2023-11-28T04:26:44.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.649+0000 7f2007d89700 1 -- 172.21.15.73:0/2303083222 wait complete. 2023-11-28T04:26:44.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.650+0000 7f2007d89700 1 Processor -- start 2023-11-28T04:26:44.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.650+0000 7f2007d89700 1 -- start start 2023-11-28T04:26:44.655 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000c3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 0x7f20000c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000ae220 con 0x7f20000ca180 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000ae470 con 0x7f20000cae90 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000ae750 con 0x7f20000caab0 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40054/0 (socket says 172.21.15.73:40054) 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 -- 172.21.15.73:0/1529111408 learned_addr learned my addr 172.21.15.73:0/1529111408 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005324700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000c3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2006326700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 0x7f20000c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 -- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 msgr2=0x7f20000c3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.656 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000c3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 -- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 msgr2=0x7f20000c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 0x7f20000c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 -- 172.21.15.73:0/1529111408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff80092a0 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2006326700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 0x7f20000c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005324700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2005b25700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7f1ff0009f60 tx=0x7f1ff000b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0009c40 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff000f040 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20000aea30 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.651+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff000cee0 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.652+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2000152c90 con 0x7f20000ca180 2023-11-28T04:26:44.657 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.653+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200006c860 con 0x7f20000ca180 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.653+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1ff0013610 con 0x7f20000ca180 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.653+0000 7f1ffdffb700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 0x7f1fec061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.654+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f1ff007a970 con 0x7f20000ca180 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.654+0000 7f2005324700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 0x7f1fec061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.654+0000 7f2005324700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 0x7f1fec061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f20000c4bc0 tx=0x7f1ff4009040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.658 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.658+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1ff004bd00 con 0x7f20000ca180 2023-11-28T04:26:44.817 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.816+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f20000a9440 con 0x7f1fec05f150 2023-11-28T04:26:44.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.818+0000 7f1ffdffb700 1 -- 172.21.15.73:0/1529111408 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71637 (secure 0 0 0) 0x7f20000a9440 con 0x7f1fec05f150 2023-11-28T04:26:44.820 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:44.821 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 msgr2=0x7f1fec061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 0x7f1fec061610 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 rx=0x7f20000c4bc0 tx=0x7f1ff4009040).stop 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 msgr2=0x7f2000163a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 rx=0x7f1ff0009f60 tx=0x7f1ff000b040).stop 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 shutdown_connections 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1fec05f150 0x7f1fec061610 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f20000cae90 0x7f20000c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f20000ca180 0x7f2000163a10 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 --2- 172.21.15.73:0/1529111408 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f20000caab0 0x7f20000c3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.820+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 >> 172.21.15.73:0/1529111408 conn(0x7f2000094920 msgr2=0x7f20000c62e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:44.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.821+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 shutdown_connections 2023-11-28T04:26:44.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.821+0000 7f2007d89700 1 -- 172.21.15.73:0/1529111408 wait complete. 2023-11-28T04:26:44.824 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:44.835 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-11-28T04:26:44.454849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459304,"kb_used_data":21096,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752515416,"statfs":{"total":773094113280,"available":770575785984,"internally_reserved":0,"allocated":21602304,"data_stored":10911456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11959,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994292"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439422+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:26:38.439422+0000","last_peered":"2023-11-28T04:26:38.439422+0000","last_clean":"2023-11-28T04:26:38.439422+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:38.439422+0000","last_undegraded":"2023-11-28T04:26:38.439422+0000","last_fullsized":"2023-11-28T04:26:38.439422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437184+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:38.437184+0000","last_peered":"2023-11-28T04:26:38.437184+0000","last_clean":"2023-11-28T04:26:38.437184+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:38.437184+0000","last_undegraded":"2023-11-28T04:26:38.437184+0000","last_fullsized":"2023-11-28T04:26:38.437184+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437640+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:26:38.437640+0000","last_peered":"2023-11-28T04:26:38.437640+0000","last_clean":"2023-11-28T04:26:38.437640+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:38.437640+0000","last_undegraded":"2023-11-28T04:26:38.437640+0000","last_fullsized":"2023-11-28T04:26:38.437640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437490+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:38.437490+0000","last_peered":"2023-11-28T04:26:38.437490+0000","last_clean":"2023-11-28T04:26:38.437490+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:38.437490+0000","last_undegraded":"2023-11-28T04:26:38.437490+0000","last_fullsized":"2023-11-28T04:26:38.437490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":169,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064579+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:31.064579+0000","last_peered":"2023-11-28T04:26:31.064579+0000","last_clean":"2023-11-28T04:26:31.064579+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:31.064579+0000","last_undegraded":"2023-11-28T04:26:31.064579+0000","last_fullsized":"2023-11-28T04:26:31.064579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064648+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:31.064648+0000","last_peered":"2023-11-28T04:26:31.064648+0000","last_clean":"2023-11-28T04:26:31.064648+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:31.064648+0000","last_undegraded":"2023-11-28T04:26:31.064648+0000","last_fullsized":"2023-11-28T04:26:31.064648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064975+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:31.064975+0000","last_peered":"2023-11-28T04:26:31.064975+0000","last_clean":"2023-11-28T04:26:31.064975+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:31.064975+0000","last_undegraded":"2023-11-28T04:26:31.064975+0000","last_fullsized":"2023-11-28T04:26:31.064975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439785+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:38.439785+0000","last_peered":"2023-11-28T04:26:38.439785+0000","last_clean":"2023-11-28T04:26:38.439785+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:38.439785+0000","last_undegraded":"2023-11-28T04:26:38.439785+0000","last_fullsized":"2023-11-28T04:26:38.439785+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.281159+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:38.281159+0000","last_peered":"2023-11-28T04:26:38.281159+0000","last_clean":"2023-11-28T04:26:38.281159+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:38.281159+0000","last_undegraded":"2023-11-28T04:26:38.281159+0000","last_fullsized":"2023-11-28T04:26:38.281159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2695168,"data_stored":1365902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.61699999999999999},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.872},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.78400000000000003,"15min":0.92800000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.82599999999999996},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.89000000000000001},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66000000000000003,"5min":0.80600000000000005,"15min":0.84099999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65000000000000002,"5min":0.73299999999999998,"15min":0.83599999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.90300000000000002},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.498},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1372536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.754,"15min":0.77600000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.67200000000000004,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.84999999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.438},"min":{"1min":0.28100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.78800000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.496,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.434},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.82199999999999995},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.80100000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.82999999999999996},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575967,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303824,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068016,"statfs":{"total":96636764160,"available":96325648384,"internally_reserved":0,"allocated":2695168,"data_stored":1362815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.247},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.433},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.497}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1357120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.5}]},{"osd":2,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.29499999999999998,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.37},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40300000000000002},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307916,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063924,"statfs":{"total":96636764160,"available":96321458176,"internally_reserved":0,"allocated":2691072,"data_stored":1362714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.377},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.496,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.443},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.622},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.754,"15min":0.754},"last":0.34300000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65300000000000002},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.84599999999999997,"15min":0.94399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73899999999999999,"5min":0.876,"15min":0.96199999999999997},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":1.016},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.99099999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575968,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307924,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063916,"statfs":{"total":96636764160,"available":96321449984,"internally_reserved":0,"allocated":2699264,"data_stored":1363527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22500000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.66500000000000004},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.746},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.68000000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.92500000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0409999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.028,"15min":1.028},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1363392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.378},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.49399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1363450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.75900000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.55600000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.80800000000000005},"last":0.312},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":0.76700000000000002,"15min":0.79900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40400000000000003},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.86399999999999999},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.59199999999999997},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.83599999999999997},"last":0.372}]},{"osd":4,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.88200000000000001},"last":0.38},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.318},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.86699999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.92100000000000004},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.90300000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85799999999999998},"last":0.39200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:44.841 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:44.954 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 Processor -- start 2023-11-28T04:26:44.955 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 -- start start 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f36440caab0 0x7f36440aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36440cae90 0x7f36440c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36440a9a90 con 0x7f36440aa960 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3644154ce0 con 0x7f36440cae90 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f364ad2b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36440a8e80 con 0x7f36440caab0 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.954+0000 7f3643fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36440cae90 0x7f36440c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f36440cae90 0x7f36440c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40068/0 (socket says 172.21.15.73:40068) 2023-11-28T04:26:44.957 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 -- 172.21.15.73:0/2707541512 learned_addr learned my addr 172.21.15.73:0/2707541512 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3648ac7700 1 --2- 172.21.15.73:0/2707541512 >> v2:172.21.15.150:3300/0 conn(0x7f36440caab0 0x7f36440aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 -- 172.21.15.73:0/2707541512 >> v2:172.21.15.150:3300/0 conn(0x7f36440caab0 msgr2=0x7f36440aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 --2- 172.21.15.73:0/2707541512 >> v2:172.21.15.150:3300/0 conn(0x7f36440caab0 0x7f36440aa470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 -- 172.21.15.73:0/2707541512 >> v1:172.21.15.150:6789/0 conn(0x7f36440aa960 legacy=0x7f36440c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 -- 172.21.15.73:0/2707541512 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36440a9160 con 0x7f36440cae90 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f3643fff700 1 --2- 172.21.15.73:0/2707541512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36440cae90 0x7f36440c0ab0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f36440848c0 tx=0x7f36380090f0).ready entity=mon.0 client_cookie=632b8c6a7372c441 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f36437fe700 1 -- 172.21.15.73:0/2707541512 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800b5a0 con 0x7f36440cae90 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f36437fe700 1 -- 172.21.15.73:0/2707541512 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363800b700 con 0x7f36440cae90 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f36437fe700 1 -- 172.21.15.73:0/2707541512 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363800ba80 con 0x7f36440cae90 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f364ad2b700 1 -- 172.21.15.73:0/2707541512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36440cae90 msgr2=0x7f36440c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.955+0000 7f364ad2b700 1 --2- 172.21.15.73:0/2707541512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36440cae90 0x7f36440c0ab0 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 rx=0x7f36440848c0 tx=0x7f36380090f0).stop 2023-11-28T04:26:44.958 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 -- 172.21.15.73:0/2707541512 shutdown_connections 2023-11-28T04:26:44.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 --2- 172.21.15.73:0/2707541512 >> v2:172.21.15.150:3300/0 conn(0x7f36440caab0 0x7f36440aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 --2- 172.21.15.73:0/2707541512 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f36440cae90 0x7f36440c0ab0 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 -- 172.21.15.73:0/2707541512 >> 172.21.15.73:0/2707541512 conn(0x7f3644094920 msgr2=0x7f364406a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:44.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 -- 172.21.15.73:0/2707541512 shutdown_connections 2023-11-28T04:26:44.959 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.956+0000 7f364ad2b700 1 -- 172.21.15.73:0/2707541512 wait complete. 2023-11-28T04:26:44.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.957+0000 7f364ad2b700 1 Processor -- start 2023-11-28T04:26:44.961 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.957+0000 7f364ad2b700 1 -- start start 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36440caab0 0x7f3644157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36440cae90 0x7f3644157390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440c2720 con 0x7f364415ae30 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36440c2d00 con 0x7f36440cae90 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3644157880 con 0x7f36440caab0 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40070/0 (socket says 172.21.15.73:40070) 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 -- 172.21.15.73:0/567733381 learned_addr learned my addr 172.21.15.73:0/567733381 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:44.962 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f3648ac7700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36440caab0 0x7f3644157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f3643fff700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36440cae90 0x7f3644157390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 -- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36440caab0 msgr2=0x7f3644157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36440caab0 0x7f3644157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 -- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36440cae90 msgr2=0x7f3644157390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36440cae90 0x7f3644157390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 -- 172.21.15.73:0/567733381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3638009370 con 0x7f364415ae30 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f36492c8700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 rx=0x7f363c00c490 tx=0x7f363c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c002ea0 con 0x7f364415ae30 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f363c0123f0 con 0x7f364415ae30 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3644159460 con 0x7f364415ae30 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f363c012790 con 0x7f364415ae30 2023-11-28T04:26:44.963 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.958+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36440c3280 con 0x7f364415ae30 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.959+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f363c0113a0 con 0x7f364415ae30 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.959+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f364406c860 con 0x7f364415ae30 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.960+0000 7f3640ff9700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 0x7f3630061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.960+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f363c079550 con 0x7f364415ae30 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.960+0000 7f3648ac7700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 0x7f3630061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:44.964 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.960+0000 7f3648ac7700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 0x7f3630061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f3634007680 tx=0x7f36340098d0).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:44.965 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:44.965+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f363c047af0 con 0x7f364415ae30 2023-11-28T04:26:45.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.122+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f36440a9440 con 0x7f363005f150 2023-11-28T04:26:45.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.124+0000 7f3640ff9700 1 -- 172.21.15.73:0/567733381 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71637 (secure 0 0 0) 0x7f36440a9440 con 0x7f363005f150 2023-11-28T04:26:45.127 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 msgr2=0x7f3630061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 0x7f3630061610 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 rx=0x7f3634007680 tx=0x7f36340098d0).stop 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 msgr2=0x7f3644158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 rx=0x7f363c00c490 tx=0x7f363c007de0).stop 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 shutdown_connections 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f363005f150 0x7f3630061610 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f364415ae30 0x7f3644158f70 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.129 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f36440caab0 0x7f3644157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 --2- 172.21.15.73:0/567733381 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f36440cae90 0x7f3644157390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 >> 172.21.15.73:0/567733381 conn(0x7f3644094920 msgr2=0x7f36440acfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:45.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 shutdown_connections 2023-11-28T04:26:45.130 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.126+0000 7f364ad2b700 1 -- 172.21.15.73:0/567733381 wait complete. 2023-11-28T04:26:45.130 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:45.139 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-11-28T04:26:44.454849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459304,"kb_used_data":21096,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752515416,"statfs":{"total":773094113280,"available":770575785984,"internally_reserved":0,"allocated":21602304,"data_stored":10911456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11959,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994292"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439422+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:26:38.439422+0000","last_peered":"2023-11-28T04:26:38.439422+0000","last_clean":"2023-11-28T04:26:38.439422+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:38.439422+0000","last_undegraded":"2023-11-28T04:26:38.439422+0000","last_fullsized":"2023-11-28T04:26:38.439422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437184+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:38.437184+0000","last_peered":"2023-11-28T04:26:38.437184+0000","last_clean":"2023-11-28T04:26:38.437184+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:38.437184+0000","last_undegraded":"2023-11-28T04:26:38.437184+0000","last_fullsized":"2023-11-28T04:26:38.437184+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437640+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:26:38.437640+0000","last_peered":"2023-11-28T04:26:38.437640+0000","last_clean":"2023-11-28T04:26:38.437640+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:38.437640+0000","last_undegraded":"2023-11-28T04:26:38.437640+0000","last_fullsized":"2023-11-28T04:26:38.437640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437490+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:38.437490+0000","last_peered":"2023-11-28T04:26:38.437490+0000","last_clean":"2023-11-28T04:26:38.437490+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:38.437490+0000","last_undegraded":"2023-11-28T04:26:38.437490+0000","last_fullsized":"2023-11-28T04:26:38.437490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":169,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064579+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:31.064579+0000","last_peered":"2023-11-28T04:26:31.064579+0000","last_clean":"2023-11-28T04:26:31.064579+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:31.064579+0000","last_undegraded":"2023-11-28T04:26:31.064579+0000","last_fullsized":"2023-11-28T04:26:31.064579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064648+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:31.064648+0000","last_peered":"2023-11-28T04:26:31.064648+0000","last_clean":"2023-11-28T04:26:31.064648+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:31.064648+0000","last_undegraded":"2023-11-28T04:26:31.064648+0000","last_fullsized":"2023-11-28T04:26:31.064648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064975+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:31.064975+0000","last_peered":"2023-11-28T04:26:31.064975+0000","last_clean":"2023-11-28T04:26:31.064975+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:31.064975+0000","last_undegraded":"2023-11-28T04:26:31.064975+0000","last_fullsized":"2023-11-28T04:26:31.064975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439785+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:38.439785+0000","last_peered":"2023-11-28T04:26:38.439785+0000","last_clean":"2023-11-28T04:26:38.439785+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:38.439785+0000","last_undegraded":"2023-11-28T04:26:38.439785+0000","last_fullsized":"2023-11-28T04:26:38.439785+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.281159+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:38.281159+0000","last_peered":"2023-11-28T04:26:38.281159+0000","last_clean":"2023-11-28T04:26:38.281159+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:38.281159+0000","last_undegraded":"2023-11-28T04:26:38.281159+0000","last_fullsized":"2023-11-28T04:26:38.281159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2695168,"data_stored":1365902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.61699999999999999},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.872},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.78400000000000003,"15min":0.92800000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.82599999999999996},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.89000000000000001},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66000000000000003,"5min":0.80600000000000005,"15min":0.84099999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65000000000000002,"5min":0.73299999999999998,"15min":0.83599999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.90300000000000002},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.498},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1372536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.754,"15min":0.77600000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.67200000000000004,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.84999999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.438},"min":{"1min":0.28100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.78800000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.496,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.434},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.82199999999999995},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.80100000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.82999999999999996},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575967,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303824,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068016,"statfs":{"total":96636764160,"available":96325648384,"internally_reserved":0,"allocated":2695168,"data_stored":1362815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.247},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.433},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.497}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1357120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.5}]},{"osd":2,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.29499999999999998,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.37},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40300000000000002},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307916,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063924,"statfs":{"total":96636764160,"available":96321458176,"internally_reserved":0,"allocated":2691072,"data_stored":1362714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.377},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.496,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.443},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.622},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.754,"15min":0.754},"last":0.34300000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65300000000000002},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.84599999999999997,"15min":0.94399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73899999999999999,"5min":0.876,"15min":0.96199999999999997},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":1.016},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.99099999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575968,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307924,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063916,"statfs":{"total":96636764160,"available":96321449984,"internally_reserved":0,"allocated":2699264,"data_stored":1363527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22500000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.66500000000000004},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.746},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.68000000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.92500000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0409999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.028,"15min":1.028},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1363392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.378},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.49399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1363450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.75900000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.55600000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.80800000000000005},"last":0.312},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":0.76700000000000002,"15min":0.79900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40400000000000003},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.86399999999999999},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.59199999999999997},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.83599999999999997},"last":0.372}]},{"osd":4,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.88200000000000001},"last":0.38},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.318},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.86699999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.92100000000000004},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.90300000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85799999999999998},"last":0.39200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:45.141 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-11-28T04:26:45.142 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:45.254 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.253+0000 7fb96077e700 1 Processor -- start 2023-11-28T04:26:45.255 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.253+0000 7fb96077e700 1 -- start start 2023-11-28T04:26:45.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9580ca180 0x7fb95815f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9580a9a90 con 0x7fb9580caab0 2023-11-28T04:26:45.257 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb958154ce0 con 0x7fb9580cae90 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9580aab80 con 0x7fb9580ca180 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95ed1b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb9580ca180 0x7fb95815f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95e51a700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb9580caab0 0x7fb9580aa470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35808/0 (socket says 172.21.15.73:35808) 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95e51a700 1 -- 172.21.15.73:0/467488040 learned_addr learned my addr 172.21.15.73:0/467488040 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95ed1b700 1 -- 172.21.15.73:0/467488040 >> v1:172.21.15.150:6789/0 conn(0x7fb9580cae90 legacy=0x7fb9580c0ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95ed1b700 1 -- 172.21.15.73:0/467488040 >> v1:172.21.15.73:6789/0 conn(0x7fb9580caab0 legacy=0x7fb9580aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95d518700 1 -- 172.21.15.73:0/467488040 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1213079067 0 0) 0x7fb9580a9a90 con 0x7fb9580caab0 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95ed1b700 1 -- 172.21.15.73:0/467488040 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9580a8e80 con 0x7fb9580ca180 2023-11-28T04:26:45.258 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95ed1b700 1 --2- 172.21.15.73:0/467488040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580ca180 0x7fb95815f3d0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 rx=0x7fb950009760 tx=0x7fb950009060).ready entity=mon.0 client_cookie=317b67c273c53bc7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95d518700 1 -- 172.21.15.73:0/467488040 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000b520 con 0x7fb9580ca180 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95d518700 1 -- 172.21.15.73:0/467488040 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95000b680 con 0x7fb9580ca180 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb95d518700 1 -- 172.21.15.73:0/467488040 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000ba00 con 0x7fb9580ca180 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- 172.21.15.73:0/467488040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580ca180 msgr2=0x7fb95815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 --2- 172.21.15.73:0/467488040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580ca180 0x7fb95815f3d0 secure :-1 s=READY pgs=2162 cs=0 l=1 rev1=1 rx=0x7fb950009760 tx=0x7fb950009060).stop 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- 172.21.15.73:0/467488040 shutdown_connections 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 --2- 172.21.15.73:0/467488040 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580ca180 0x7fb95815f3d0 unknown :-1 s=CLOSED pgs=2162 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- 172.21.15.73:0/467488040 >> 172.21.15.73:0/467488040 conn(0x7fb958094920 msgr2=0x7fb95806a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:45.259 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- 172.21.15.73:0/467488040 shutdown_connections 2023-11-28T04:26:45.260 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.255+0000 7fb96077e700 1 -- 172.21.15.73:0/467488040 wait complete. 2023-11-28T04:26:45.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.256+0000 7fb96077e700 1 Processor -- start 2023-11-28T04:26:45.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 -- start start 2023-11-28T04:26:45.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 0x7fb9580c3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.262 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 0x7fb9580c6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580c5160 con 0x7fb9580cae90 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580c4e80 con 0x7fb9580ca180 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb96077e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9580c6410 con 0x7fb9580caab0 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb95ed1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb95ed1b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40088/0 (socket says 172.21.15.73:40088) 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.257+0000 7fb95ed1b700 1 -- 172.21.15.73:0/3238512869 learned_addr learned my addr 172.21.15.73:0/3238512869 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95dd19700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 0x7fb9580c6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95e51a700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 0x7fb9580c3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.263 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 -- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 msgr2=0x7fb9580c6ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 0x7fb9580c6ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 -- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 msgr2=0x7fb9580c3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 0x7fb9580c3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 -- 172.21.15.73:0/3238512869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9500092a0 con 0x7fb9580cae90 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95dd19700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 0x7fb9580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95e51a700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 0x7fb9580c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb95ed1b700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7fb950009eb0 tx=0x7fb950017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95000ba00 con 0x7fb9580cae90 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb95001b410 con 0x7fb9580cae90 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9580c66f0 con 0x7fb9580cae90 2023-11-28T04:26:45.264 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb95001b7b0 con 0x7fb9580cae90 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.258+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb958149540 con 0x7fb9580cae90 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.259+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95806c860 con 0x7fb9580cae90 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.259+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb95001bba0 con 0x7fb9580cae90 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.260+0000 7fb94e7fc700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 0x7fb93c0615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.260+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fb95007bee0 con 0x7fb9580cae90 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.260+0000 7fb95e51a700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 0x7fb93c0615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.260+0000 7fb95e51a700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 0x7fb93c0615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb954001de0 tx=0x7fb954009040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.265 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.265+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fb95004d330 con 0x7fb9580cae90 2023-11-28T04:26:45.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.423+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb9580a9160 con 0x7fb93c05f120 2023-11-28T04:26:45.426 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.426+0000 7fb94e7fc700 1 -- 172.21.15.73:0/3238512869 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71637 (secure 0 0 0) 0x7fb9580a9160 con 0x7fb93c05f120 2023-11-28T04:26:45.427 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 msgr2=0x7fb93c0615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 0x7fb93c0615e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 rx=0x7fb954001de0 tx=0x7fb954009040).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 msgr2=0x7fb958149050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 rx=0x7fb950009eb0 tx=0x7fb950017000).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 shutdown_connections 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb93c05f120 0x7fb93c0615e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb9580cae90 0x7fb958149050 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb9580ca180 0x7fb9580c3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 --2- 172.21.15.73:0/3238512869 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb9580caab0 0x7fb9580c6ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 >> 172.21.15.73:0/3238512869 conn(0x7fb958094920 msgr2=0x7fb9580abde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:45.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 shutdown_connections 2023-11-28T04:26:45.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.428+0000 7fb96077e700 1 -- 172.21.15.73:0/3238512869 wait complete. 2023-11-28T04:26:45.430 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:45.440 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2023-11-28T04:26:44.454849+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2459304,"kb_used_data":21096,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752515416,"statfs":{"total":773094113280,"available":770575785984,"internally_reserved":0,"allocated":21602304,"data_stored":10911456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11959,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994292"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439422+0000","last_change":"2023-11-28T02:52:40.038729+0000","last_active":"2023-11-28T04:26:38.439422+0000","last_peered":"2023-11-28T04:26:38.439422+0000","last_clean":"2023-11-28T04:26:38.439422+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:38.439422+0000","last_undegraded":"2023-11-28T04:26:38.439422+0000","last_fullsized":"2023-11-28T04:26:38.439422+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437184+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:38.437184+0000","last_peered":"2023-11-28T04:26:38.437184+0000","last_clean":"2023-11-28T04:26:38.437184+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:38.437184+0000","last_undegraded":"2023-11-28T04:26:38.437184+0000","last_fullsized":"2023-11-28T04:26:38.437184+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":181,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437640+0000","last_change":"2023-11-28T02:52:39.445058+0000","last_active":"2023-11-28T04:26:38.437640+0000","last_peered":"2023-11-28T04:26:38.437640+0000","last_clean":"2023-11-28T04:26:38.437640+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:38.437640+0000","last_undegraded":"2023-11-28T04:26:38.437640+0000","last_fullsized":"2023-11-28T04:26:38.437640+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":171,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.437490+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:38.437490+0000","last_peered":"2023-11-28T04:26:38.437490+0000","last_clean":"2023-11-28T04:26:38.437490+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:38.437490+0000","last_undegraded":"2023-11-28T04:26:38.437490+0000","last_fullsized":"2023-11-28T04:26:38.437490+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":169,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064579+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:31.064579+0000","last_peered":"2023-11-28T04:26:31.064579+0000","last_clean":"2023-11-28T04:26:31.064579+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:31.064579+0000","last_undegraded":"2023-11-28T04:26:31.064579+0000","last_fullsized":"2023-11-28T04:26:31.064579+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":167,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064648+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:31.064648+0000","last_peered":"2023-11-28T04:26:31.064648+0000","last_clean":"2023-11-28T04:26:31.064648+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:31.064648+0000","last_undegraded":"2023-11-28T04:26:31.064648+0000","last_fullsized":"2023-11-28T04:26:31.064648+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":168,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-11-28T04:26:31.064975+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:31.064975+0000","last_peered":"2023-11-28T04:26:31.064975+0000","last_clean":"2023-11-28T04:26:31.064975+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:31.064975+0000","last_undegraded":"2023-11-28T04:26:31.064975+0000","last_fullsized":"2023-11-28T04:26:31.064975+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":174,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.439785+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:38.439785+0000","last_peered":"2023-11-28T04:26:38.439785+0000","last_clean":"2023-11-28T04:26:38.439785+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:38.439785+0000","last_undegraded":"2023-11-28T04:26:38.439785+0000","last_fullsized":"2023-11-28T04:26:38.439785+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":191,"reported_epoch":214,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.281159+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:38.281159+0000","last_peered":"2023-11-28T04:26:38.281159+0000","last_clean":"2023-11-28T04:26:38.281159+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:38.281159+0000","last_undegraded":"2023-11-28T04:26:38.281159+0000","last_fullsized":"2023-11-28T04:26:38.281159+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575968,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307920,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063920,"statfs":{"total":96636764160,"available":96321454080,"internally_reserved":0,"allocated":2695168,"data_stored":1365902,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.61699999999999999},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.872},"last":0.48099999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.78400000000000003,"15min":0.92800000000000005},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.82599999999999996},"last":0.44900000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.89000000000000001},"last":0.372},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66000000000000003,"5min":0.80600000000000005,"15min":0.84099999999999997},"last":0.46000000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65000000000000002,"5min":0.73299999999999998,"15min":0.83599999999999997},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.90300000000000002},"last":0.438}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.498},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.46500000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.39800000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575968,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1372536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.53900000000000003,"15min":0.55400000000000005},"min":{"1min":0.379,"5min":0.33000000000000002,"15min":0.28699999999999998},"max":{"1min":0.66400000000000003,"5min":0.754,"15min":0.77600000000000002},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56999999999999995,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.76800000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.36099999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.55900000000000005,"15min":0.57199999999999995},"min":{"1min":0.41099999999999998,"5min":0.35599999999999998,"15min":0.318},"max":{"1min":0.67400000000000004,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.55100000000000005,"15min":0.56399999999999995},"min":{"1min":0.39600000000000002,"5min":0.38500000000000001,"15min":0.34899999999999998},"max":{"1min":0.69999999999999996,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.503}]},{"osd":2,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.56699999999999995,"15min":0.57799999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.34399999999999997},"max":{"1min":0.79700000000000004,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.57099999999999995,"15min":0.58899999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.34899999999999998},"max":{"1min":0.67200000000000004,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56999999999999995,"15min":0.58099999999999996},"min":{"1min":0.42199999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.75800000000000001,"5min":0.79800000000000004,"15min":0.82599999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.58299999999999996,"15min":0.59599999999999997},"min":{"1min":0.44400000000000001,"5min":0.43099999999999999,"15min":0.36799999999999999},"max":{"1min":0.73299999999999998,"5min":0.81599999999999995,"15min":0.84999999999999998},"last":0.59599999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.435,"15min":0.438},"min":{"1min":0.28100000000000003,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.70399999999999996,"5min":0.70499999999999996,"15min":0.78800000000000003},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.45900000000000002,"5min":0.46700000000000003,"15min":0.47899999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66000000000000003,"5min":0.68600000000000005,"15min":0.80700000000000005},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.496,"15min":0.5},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.65400000000000003,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.434},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.54400000000000004},"min":{"1min":0.34999999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.752,"5min":0.77700000000000002,"15min":0.82199999999999995},"last":0.40799999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.51500000000000001,"15min":0.52400000000000002},"min":{"1min":0.35799999999999998,"5min":0.34100000000000003,"15min":0.30099999999999999},"max":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.80100000000000005},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53200000000000003,"15min":0.54000000000000004},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.72099999999999997,"5min":0.749,"15min":0.82999999999999996},"last":0.52400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834575967,"num_pgs":39,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303824,"kb_used_data":2632,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94068016,"statfs":{"total":96636764160,"available":96325648384,"internally_reserved":0,"allocated":2695168,"data_stored":1362815,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.23499999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.247},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.33700000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.313},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.32300000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.54100000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.55000000000000004}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.433},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.497}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.53200000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575968,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1357120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.33400000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.5}]},{"osd":2,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.52800000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.29499999999999998,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.37},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.42399999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.76400000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.71699999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.59199999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40300000000000002},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.67100000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575969,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307916,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063924,"statfs":{"total":96636764160,"available":96321458176,"internally_reserved":0,"allocated":2691072,"data_stored":1362714,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.377},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.496,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.443},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.31}]},{"osd":1,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.622},"last":0.27000000000000002},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.754,"15min":0.754},"last":0.34300000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65300000000000002},"last":0.36299999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.84599999999999997,"15min":0.94399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73899999999999999,"5min":0.876,"15min":0.96199999999999997},"last":0.47899999999999998}]},{"osd":5,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":1.016},"last":0.49199999999999999}]},{"osd":6,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.99099999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.52500000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575968,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307924,"kb_used_data":2636,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063916,"statfs":{"total":96636764160,"available":96321449984,"internally_reserved":0,"allocated":2699264,"data_stored":1363527,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22500000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.66500000000000004},"last":0.29499999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.36099999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.746},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.68000000000000005},"last":0.33800000000000002}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.92500000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.50700000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0409999999999999},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.625,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.56999999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.627,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.028,"15min":1.028},"last":0.70499999999999996}]},{"osd":7,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.45200000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575967,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307928,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063912,"statfs":{"total":96636764160,"available":96321445888,"internally_reserved":0,"allocated":2703360,"data_stored":1363392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.434},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.42199999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.52400000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.378},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.41199999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.51500000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.48099999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.45100000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.49399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575969,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307932,"kb_used_data":2644,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063908,"statfs":{"total":96636764160,"available":96321441792,"internally_reserved":0,"allocated":2707456,"data_stored":1363450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.75900000000000001},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.55600000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.28899999999999998}]},{"osd":1,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.80800000000000005},"last":0.312},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.498}]},{"osd":2,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":0.76700000000000002,"15min":0.79900000000000004},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40400000000000003},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.86399999999999999},"last":0.434}]},{"osd":3,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.59199999999999997},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.83599999999999997},"last":0.372}]},{"osd":4,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.88200000000000001},"last":0.38},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.318},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.51600000000000001}]},{"osd":6,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.86699999999999999},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.92100000000000004},"last":0.45200000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.90300000000000002},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85799999999999998},"last":0.39200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:45.442 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-11-28T04:26:45.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.556+0000 7f27a24eb700 1 Processor -- start 2023-11-28T04:26:45.556 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.556+0000 7f27a24eb700 1 -- start start 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f27a24eb700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f279c0cae90 0x7f279c0c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f27a24eb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0ca180 0x7f279c0c0fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f27a24eb700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f279c0a9a90 con 0x7f279c0caab0 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f27a24eb700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279c0c1490 con 0x7f279c0cae90 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f27a24eb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c154ce0 con 0x7f279c0ca180 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f279c0cae90 0x7f279c0c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f279c0cae90 0x7f279c0c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40096/0 (socket says 172.21.15.73:40096) 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 -- 172.21.15.73:0/1903665051 learned_addr learned my addr 172.21.15.73:0/1903665051 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 -- 172.21.15.73:0/1903665051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0ca180 msgr2=0x7f279c0c0fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 --2- 172.21.15.73:0/1903665051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0ca180 0x7f279c0c0fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 -- 172.21.15.73:0/1903665051 >> v1:172.21.15.150:6789/0 conn(0x7f279c0caab0 legacy=0x7f279c0aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:45.560 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.557+0000 7f279b7fe700 1 -- 172.21.15.73:0/1903665051 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f279c0aab80 con 0x7f279c0cae90 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f279b7fe700 1 --2- 172.21.15.73:0/1903665051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c0c0ab0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 rx=0x7f2790006fd0 tx=0x7f27900090f0).ready entity=mon.0 client_cookie=c9b369509cc40227 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f279affd700 1 -- 172.21.15.73:0/1903665051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000b600 con 0x7f279c0cae90 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f279affd700 1 -- 172.21.15.73:0/1903665051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279000b760 con 0x7f279c0cae90 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f279affd700 1 -- 172.21.15.73:0/1903665051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279000bae0 con 0x7f279c0cae90 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 -- 172.21.15.73:0/1903665051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 msgr2=0x7f279c0c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 --2- 172.21.15.73:0/1903665051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c0c0ab0 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 rx=0x7f2790006fd0 tx=0x7f27900090f0).stop 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 -- 172.21.15.73:0/1903665051 shutdown_connections 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 --2- 172.21.15.73:0/1903665051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0ca180 0x7f279c0c0fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:45.561 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 --2- 172.21.15.73:0/1903665051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c0c0ab0 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 -- 172.21.15.73:0/1903665051 >> 172.21.15.73:0/1903665051 conn(0x7f279c094920 msgr2=0x7f279c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 -- 172.21.15.73:0/1903665051 shutdown_connections 2023-11-28T04:26:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.558+0000 7f27a24eb700 1 -- 172.21.15.73:0/1903665051 wait complete. 2023-11-28T04:26:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.559+0000 7f27a24eb700 1 Processor -- start 2023-11-28T04:26:45.562 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.559+0000 7f27a24eb700 1 -- start start 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f279c0ca180 0x7f279c1532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0caab0 0x7f279c1578e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c1659f0 con 0x7f279c0cae90 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f279c164de0 con 0x7f279c0ca180 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a24eb700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f279c1650c0 con 0x7f279c0caab0 2023-11-28T04:26:45.565 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a0a88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a0a88700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40106/0 (socket says 172.21.15.73:40106) 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f27a0a88700 1 -- 172.21.15.73:0/4177850836 learned_addr learned my addr 172.21.15.73:0/4177850836 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f279b7fe700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0caab0 0x7f279c1578e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.560+0000 7f279bfff700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f279c0ca180 0x7f279c1532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 -- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0caab0 msgr2=0x7f279c1578e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0caab0 0x7f279c1578e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 -- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f279c0ca180 msgr2=0x7f279c1532c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.566 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f279c0ca180 0x7f279c1532c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 -- 172.21.15.73:0/4177850836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009370 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a0a88700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 rx=0x7f279400bb60 tx=0x7f2794009810).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400c7e0 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f279400c960 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f279c1653a0 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f279400cb10 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.561+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f279c0c4d10 con 0x7f279c0cae90 2023-11-28T04:26:45.567 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.562+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279c165ba0 con 0x7f279c0cae90 2023-11-28T04:26:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.562+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f2794012020 con 0x7f279c0cae90 2023-11-28T04:26:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.562+0000 7f2783fff700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 0x7f2788061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.563+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f2794079f90 con 0x7f279c0cae90 2023-11-28T04:26:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.563+0000 7f279bfff700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 0x7f2788061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:45.568 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.563+0000 7f279bfff700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 0x7f2788061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f278c007fd0 tx=0x7f278c007d70).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:45.569 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.567+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f279c165ba0 con 0x7f279c0cae90 2023-11-28T04:26:45.730 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.730+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f279c165ba0 con 0x7f279c0cae90 2023-11-28T04:26:45.731 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.731+0000 7f2783fff700 1 -- 172.21.15.73:0/4177850836 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v217) v1 ==== 74+0+14406 (secure 0 0 0) 0x7f279c165ba0 con 0x7f279c0cae90 2023-11-28T04:26:45.731 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:45.731 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":217,"fsid":"c55698a5-e315-44c7-976c-effa5ffffb99","created":"2023-11-28T02:52:26.300375+0000","modified":"2023-11-28T04:26:41.458487+0000","last_up_change":"2023-11-28T02:52:35.408052+0000","last_in_change":"2023-11-28T02:52:28.743519+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":26,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"pacific","pools":[{"pool":1,"pool_name":"device_health_metrics","create_time":"2023-11-28T02:52:35.444074+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"217","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr_devicehealth":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-11-28T02:52:37.446721+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"9c698d30-5b01-4161-8d18-49a843ed4bf5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6800","nonce":90247},{"type":"v1","addr":"172.21.15.73:6801","nonce":90247}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6802","nonce":90247},{"type":"v1","addr":"172.21.15.73:6803","nonce":90247}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6806","nonce":90247},{"type":"v1","addr":"172.21.15.73:6807","nonce":90247}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6804","nonce":90247},{"type":"v1","addr":"172.21.15.73:6805","nonce":90247}]},"public_addr":"172.21.15.73:6801/90247","cluster_addr":"172.21.15.73:6803/90247","heartbeat_back_addr":"172.21.15.73:6807/90247","heartbeat_front_addr":"172.21.15.73:6805/90247","state":["exists","up"]},{"osd":1,"uuid":"84f00a60-8e08-4ed3-b366-c77822b49826","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6824","nonce":90262},{"type":"v1","addr":"172.21.15.73:6825","nonce":90262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6826","nonce":90262},{"type":"v1","addr":"172.21.15.73:6827","nonce":90262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6830","nonce":90262},{"type":"v1","addr":"172.21.15.73:6831","nonce":90262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6828","nonce":90262},{"type":"v1","addr":"172.21.15.73:6829","nonce":90262}]},"public_addr":"172.21.15.73:6825/90262","cluster_addr":"172.21.15.73:6827/90262","heartbeat_back_addr":"172.21.15.73:6831/90262","heartbeat_front_addr":"172.21.15.73:6829/90262","state":["exists","up"]},{"osd":2,"uuid":"45f934f5-ca99-4a3e-a482-b6adc132b3d3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6808","nonce":90248},{"type":"v1","addr":"172.21.15.73:6809","nonce":90248}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6810","nonce":90248},{"type":"v1","addr":"172.21.15.73:6811","nonce":90248}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6814","nonce":90248},{"type":"v1","addr":"172.21.15.73:6815","nonce":90248}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6812","nonce":90248},{"type":"v1","addr":"172.21.15.73:6813","nonce":90248}]},"public_addr":"172.21.15.73:6809/90248","cluster_addr":"172.21.15.73:6811/90248","heartbeat_back_addr":"172.21.15.73:6815/90248","heartbeat_front_addr":"172.21.15.73:6813/90248","state":["exists","up"]},{"osd":3,"uuid":"e0055829-e29c-4f7a-8a17-e21c21919c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6816","nonce":90251},{"type":"v1","addr":"172.21.15.73:6817","nonce":90251}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6818","nonce":90251},{"type":"v1","addr":"172.21.15.73:6819","nonce":90251}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6822","nonce":90251},{"type":"v1","addr":"172.21.15.73:6823","nonce":90251}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6820","nonce":90251},{"type":"v1","addr":"172.21.15.73:6821","nonce":90251}]},"public_addr":"172.21.15.73:6817/90251","cluster_addr":"172.21.15.73:6819/90251","heartbeat_back_addr":"172.21.15.73:6823/90251","heartbeat_front_addr":"172.21.15.73:6821/90251","state":["exists","up"]},{"osd":4,"uuid":"1309ffa4-9d68-4414-8506-32d09d7b1818","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6808","nonce":90482},{"type":"v1","addr":"172.21.15.150:6810","nonce":90482}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6812","nonce":90482},{"type":"v1","addr":"172.21.15.150:6814","nonce":90482}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6820","nonce":90482},{"type":"v1","addr":"172.21.15.150:6821","nonce":90482}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6816","nonce":90482},{"type":"v1","addr":"172.21.15.150:6818","nonce":90482}]},"public_addr":"172.21.15.150:6810/90482","cluster_addr":"172.21.15.150:6814/90482","heartbeat_back_addr":"172.21.15.150:6821/90482","heartbeat_front_addr":"172.21.15.150:6818/90482","state":["exists","up"]},{"osd":5,"uuid":"7b292980-e2bd-4fec-afa8-b0eb42ca4039","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6800","nonce":90480},{"type":"v1","addr":"172.21.15.150:6801","nonce":90480}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6802","nonce":90480},{"type":"v1","addr":"172.21.15.150:6803","nonce":90480}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6806","nonce":90480},{"type":"v1","addr":"172.21.15.150:6807","nonce":90480}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6804","nonce":90480},{"type":"v1","addr":"172.21.15.150:6805","nonce":90480}]},"public_addr":"172.21.15.150:6801/90480","cluster_addr":"172.21.15.150:6803/90480","heartbeat_back_addr":"172.21.15.150:6807/90480","heartbeat_front_addr":"172.21.15.150:6805/90480","state":["exists","up"]},{"osd":6,"uuid":"f96f63b5-0a1b-48b7-8f51-19e5684c1d6b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6824","nonce":90525},{"type":"v1","addr":"172.21.15.150:6825","nonce":90525}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6826","nonce":90525},{"type":"v1","addr":"172.21.15.150:6827","nonce":90525}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6830","nonce":90525},{"type":"v1","addr":"172.21.15.150:6831","nonce":90525}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6828","nonce":90525},{"type":"v1","addr":"172.21.15.150:6829","nonce":90525}]},"public_addr":"172.21.15.150:6825/90525","cluster_addr":"172.21.15.150:6827/90525","heartbeat_back_addr":"172.21.15.150:6831/90525","heartbeat_front_addr":"172.21.15.150:6829/90525","state":["exists","up"]},{"osd":7,"uuid":"415767ad-b315-4c70-ad94-d94695868e80","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":210,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6809","nonce":90481},{"type":"v1","addr":"172.21.15.150:6811","nonce":90481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6813","nonce":90481},{"type":"v1","addr":"172.21.15.150:6815","nonce":90481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6822","nonce":90481},{"type":"v1","addr":"172.21.15.150:6823","nonce":90481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6817","nonce":90481},{"type":"v1","addr":"172.21.15.150:6819","nonce":90481}]},"public_addr":"172.21.15.150:6811/90481","cluster_addr":"172.21.15.150:6815/90481","heartbeat_back_addr":"172.21.15.150:6823/90481","heartbeat_front_addr":"172.21.15.150:6819/90481","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.923062+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.924739+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.807336+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.885489+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.954847+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.900558+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.887843+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138314316775423,"old_weight":0,"last_purged_snaps_scrub":"2023-11-28T02:52:33.943825+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.73:0/2970944951":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/3498822562":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/2665669998":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/1318635231":"2023-11-29T04:26:38.441936+0000","172.21.15.73:6836/447432356":"2023-11-29T04:26:38.276635+0000","172.21.15.73:6833/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6832/136129":"2023-11-29T04:26:30.988881+0000","172.21.15.73:6834/447432356":"2023-11-29T04:26:38.276635+0000","172.21.15.73:0/3047075693":"2023-11-29T04:26:30.988881+0000","172.21.15.73:0/2536116355":"2023-11-29T04:26:38.441936+0000","172.21.15.73:0/1637465872":"2023-11-29T04:26:30.988881+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-11-28T04:26:45.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 msgr2=0x7f2788061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 0x7f2788061610 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 rx=0x7f278c007fd0 tx=0x7f278c007d70).stop 2023-11-28T04:26:45.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 msgr2=0x7f279c157dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:45.733 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 rx=0x7f279400bb60 tx=0x7f2794009810).stop 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 shutdown_connections 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f278805f150 0x7f2788061610 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f279c0cae90 0x7f279c157dd0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f279c0ca180 0x7f279c1532c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 --2- 172.21.15.73:0/4177850836 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f279c0caab0 0x7f279c1578e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 >> 172.21.15.73:0/4177850836 conn(0x7f279c094920 msgr2=0x7f279c0acd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 shutdown_connections 2023-11-28T04:26:45.734 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:45.733+0000 7f27a24eb700 1 -- 172.21.15.73:0/4177850836 wait complete. 2023-11-28T04:26:46.745 INFO:tasks.ceph:Scrubbing osd.0 2023-11-28T04:26:46.745 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:46.872 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.872+0000 7ff332b73700 1 Processor -- start 2023-11-28T04:26:46.873 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.872+0000 7ff332b73700 1 -- start start 2023-11-28T04:26:46.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.873+0000 7ff332b73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.875 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.873+0000 7ff332b73700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 0x7ff32c0aa6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.873+0000 7ff332b73700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff32c0c6630 con 0x7ff32c154530 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.873+0000 7ff332b73700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff32c0c6390 con 0x7ff32c0aa220 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.873+0000 7ff332b73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0a84e0 con 0x7ff32c154910 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff331110700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 0x7ff32c0aa6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:54840/0 (socket says 172.21.15.73:54840) 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 -- 172.21.15.73:0/3274743604 learned_addr learned my addr 172.21.15.73:0/3274743604 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff33090f700 1 --1- 172.21.15.73:0/3274743604 >> v1:172.21.15.150:6789/0 conn(0x7ff32c154530 0x7ff32c0c8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43486/0 (socket says 172.21.15.73:43486) 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 -- 172.21.15.73:0/3274743604 >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 msgr2=0x7ff32c0aa6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 --2- 172.21.15.73:0/3274743604 >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 0x7ff32c0aa6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 -- 172.21.15.73:0/3274743604 >> v1:172.21.15.150:6789/0 conn(0x7ff32c154530 legacy=0x7ff32c0c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.876 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 -- 172.21.15.73:0/3274743604 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff32c0aaba0 con 0x7ff32c154910 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff331110700 1 --2- 172.21.15.73:0/3274743604 >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 0x7ff32c0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32bfff700 1 --2- 172.21.15.73:0/3274743604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7ff314009f20 tx=0x7ff314009840).ready entity=mon.2 client_cookie=bfc938c5683a4b81 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32b7fe700 1 -- 172.21.15.73:0/3274743604 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400e070 con 0x7ff32c154910 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32b7fe700 1 -- 172.21.15.73:0/3274743604 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400b9d0 con 0x7ff32c154910 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.874+0000 7ff32b7fe700 1 -- 172.21.15.73:0/3274743604 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31400bd50 con 0x7ff32c154910 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 -- 172.21.15.73:0/3274743604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 msgr2=0x7ff32c0a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 --2- 172.21.15.73:0/3274743604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 rx=0x7ff314009f20 tx=0x7ff314009840).stop 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 -- 172.21.15.73:0/3274743604 shutdown_connections 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 --2- 172.21.15.73:0/3274743604 >> v2:172.21.15.150:3300/0 conn(0x7ff32c0aa220 0x7ff32c0aa6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.877 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 --2- 172.21.15.73:0/3274743604 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c154910 0x7ff32c0a7ff0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 -- 172.21.15.73:0/3274743604 >> 172.21.15.73:0/3274743604 conn(0x7ff32c094930 msgr2=0x7ff32c06a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:46.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 -- 172.21.15.73:0/3274743604 shutdown_connections 2023-11-28T04:26:46.878 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.875+0000 7ff332b73700 1 -- 172.21.15.73:0/3274743604 wait complete. 2023-11-28T04:26:46.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.876+0000 7ff332b73700 1 Processor -- start 2023-11-28T04:26:46.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.876+0000 7ff332b73700 1 -- start start 2023-11-28T04:26:46.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0aa220 0x7ff32c14e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 0x7ff32c0c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.881 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c154910 0x7ff32c0c2810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0b09c0 con 0x7ff32c154530 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0aaf00 con 0x7ff32c154910 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff332b73700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff32c0ab1e0 con 0x7ff32c0aa220 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 0x7ff32c0c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 0x7ff32c0c2320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40108/0 (socket says 172.21.15.73:40108) 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 -- 172.21.15.73:0/1588554509 learned_addr learned my addr 172.21.15.73:0/1588554509 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff33090f700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0aa220 0x7ff32c14e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff331110700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c154910 0x7ff32c0c2810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 -- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0aa220 msgr2=0x7ff32c14e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff32c0aa220 0x7ff32c14e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 -- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c154910 msgr2=0x7ff32c0c2810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff32c154910 0x7ff32c0c2810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.882 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.877+0000 7ff32bfff700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3240092a0 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff32bfff700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 0x7ff32c0c2320 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 rx=0x7ff31400b610 tx=0x7ff314009840).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3140092e0 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff31400bb40 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff314009ae0 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff31401d470 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.878+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff32c0ab7a0 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.879+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff32c06c860 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.879+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff31401d7e0 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.879+0000 7ff328ff9700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff30c05f150 0x7ff30c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7ff31407be00 con 0x7ff32c154530 2023-11-28T04:26:46.883 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff33090f700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff30c05f150 0x7ff30c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff328ff9700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7ff30c065a30 0x7ff30c067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff30c068510 con 0x7ff30c065a30 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff31407e420 con 0x7ff32c154530 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff331110700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7ff30c065a30 0x7ff30c067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff33090f700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff30c05f150 0x7ff30c061610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff32c1561c0 tx=0x7ff31c007d20).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.880+0000 7ff331110700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7ff30c065a30 0x7ff30c067e90 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:46.884 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.881+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== osd.0 v2:172.21.15.73:6800/90247 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff30c068510 con 0x7ff30c065a30 2023-11-28T04:26:46.900 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 --> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff32c0ab9c0 con 0x7ff30c065a30 2023-11-28T04:26:46.903 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff328ff9700 1 -- 172.21.15.73:0/1588554509 <== osd.0 v2:172.21.15.73:6800/90247 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff32c0ab9c0 con 0x7ff30c065a30 2023-11-28T04:26:46.906 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:46.906 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:46.907 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:46.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7ff30c065a30 msgr2=0x7ff30c067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6800/90247,v1:172.21.15.73:6801/90247] conn(0x7ff30c065a30 0x7ff30c067e90 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:46.908 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff30c05f150 msgr2=0x7ff30c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff30c05f150 0x7ff30c061610 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 rx=0x7ff32c1561c0 tx=0x7ff31c007d20).stop 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 msgr2=0x7ff32c0c2320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 --2- 172.21.15.73:0/1588554509 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff32c154530 0x7ff32c0c2320 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 rx=0x7ff31400b610 tx=0x7ff314009840).stop 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff331110700 1 -- 172.21.15.73:0/1588554509 reap_dead start 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 shutdown_connections 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.894+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 >> 172.21.15.73:0/1588554509 conn(0x7ff32c094930 msgr2=0x7ff32c14fce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.895+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 shutdown_connections 2023-11-28T04:26:46.909 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:46.895+0000 7ff332b73700 1 -- 172.21.15.73:0/1588554509 wait complete. 2023-11-28T04:26:46.909 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-11-28T04:26:47.022 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.022+0000 7f0e1b88b700 1 Processor -- start 2023-11-28T04:26:47.023 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 -- start start 2023-11-28T04:26:47.025 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e14098d10 0x7f0e140c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0e140c6190 0x7f0e14063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e140c72d0 con 0x7f0e14064350 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e14154bc0 con 0x7f0e14098d10 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.023+0000 7f0e1b88b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e140c66c0 con 0x7f0e140c6190 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e14098d10 0x7f0e140c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f0e14098d10 0x7f0e140c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40122/0 (socket says 172.21.15.73:40122) 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 -- 172.21.15.73:0/2985832269 learned_addr learned my addr 172.21.15.73:0/2985832269 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 -- 172.21.15.73:0/2985832269 >> v2:172.21.15.150:3300/0 conn(0x7f0e140c6190 msgr2=0x7f0e14063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.026 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e18e26700 1 --2- 172.21.15.73:0/2985832269 >> v2:172.21.15.150:3300/0 conn(0x7f0e140c6190 0x7f0e14063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 --2- 172.21.15.73:0/2985832269 >> v2:172.21.15.150:3300/0 conn(0x7f0e140c6190 0x7f0e14063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 -- 172.21.15.73:0/2985832269 >> v1:172.21.15.150:6789/0 conn(0x7f0e14064350 legacy=0x7f0e14158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 -- 172.21.15.73:0/2985832269 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140c69a0 con 0x7f0e14098d10 2023-11-28T04:26:47.027 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e19627700 1 --2- 172.21.15.73:0/2985832269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e14098d10 0x7f0e140c7760 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7f0e140c5b40 tx=0x7f0e04009840).ready entity=mon.0 client_cookie=d1a89f8e6467723d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e13fff700 1 -- 172.21.15.73:0/2985832269 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400e070 con 0x7f0e14098d10 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e13fff700 1 -- 172.21.15.73:0/2985832269 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0400ba10 con 0x7f0e14098d10 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.024+0000 7f0e13fff700 1 -- 172.21.15.73:0/2985832269 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0400bd90 con 0x7f0e14098d10 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 -- 172.21.15.73:0/2985832269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e14098d10 msgr2=0x7f0e140c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/2985832269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e14098d10 0x7f0e140c7760 secure :-1 s=READY pgs=2167 cs=0 l=1 rev1=1 rx=0x7f0e140c5b40 tx=0x7f0e04009840).stop 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 -- 172.21.15.73:0/2985832269 shutdown_connections 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/2985832269 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e14098d10 0x7f0e140c7760 unknown :-1 s=CLOSED pgs=2167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/2985832269 >> v2:172.21.15.150:3300/0 conn(0x7f0e140c6190 0x7f0e14063e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 -- 172.21.15.73:0/2985832269 >> 172.21.15.73:0/2985832269 conn(0x7f0e140cb310 msgr2=0x7f0e140cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.028 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 -- 172.21.15.73:0/2985832269 shutdown_connections 2023-11-28T04:26:47.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.025+0000 7f0e1b88b700 1 -- 172.21.15.73:0/2985832269 wait complete. 2023-11-28T04:26:47.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.026+0000 7f0e1b88b700 1 Processor -- start 2023-11-28T04:26:47.029 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.026+0000 7f0e1b88b700 1 -- start start 2023-11-28T04:26:47.031 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e14064350 0x7f0e140ac4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e14098d10 0x7f0e141594b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e140ac9a0 con 0x7f0e140c6190 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1415a890 con 0x7f0e14098d10 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e14159c80 con 0x7f0e14064350 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40136/0 (socket says 172.21.15.73:40136) 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 -- 172.21.15.73:0/3421434406 learned_addr learned my addr 172.21.15.73:0/3421434406 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19627700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e14064350 0x7f0e140ac4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 -- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e14064350 msgr2=0x7f0e140ac4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.032 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e18e26700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e14098d10 0x7f0e141594b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e14064350 0x7f0e140ac4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 -- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e14098d10 msgr2=0x7f0e141594b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e14098d10 0x7f0e141594b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 -- 172.21.15.73:0/3421434406 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e04009ae0 con 0x7f0e140c6190 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e19e28700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 rx=0x7f0e0c00c490 tx=0x7f0e0c007de0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c002ea0 con 0x7f0e140c6190 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e0c00f470 con 0x7f0e140c6190 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e14159f60 con 0x7f0e140c6190 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.027+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e0c00f810 con 0x7f0e140c6190 2023-11-28T04:26:47.033 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.028+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1415a520 con 0x7f0e140c6190 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.028+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1406c860 con 0x7f0e140c6190 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.029+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f0e0c00fc00 con 0x7f0e140c6190 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.029+0000 7f0e117fa700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 0x7f0e00061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.029+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f0e0c079490 con 0x7f0e140c6190 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.029+0000 7f0e19627700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 0x7f0e00061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.029+0000 7f0e19627700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 0x7f0e00061610 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0e04009540 tx=0x7f0e140a8040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.034 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.033+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f0e0c047af0 con 0x7f0e140c6190 2023-11-28T04:26:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.197+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e140c6c80 con 0x7f0e0005f150 2023-11-28T04:26:47.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.197+0000 7f0e117fa700 1 -- 172.21.15.73:0/3421434406 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0e140c6c80 con 0x7f0e0005f150 2023-11-28T04:26:47.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 msgr2=0x7f0e00061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 0x7f0e00061610 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 rx=0x7f0e04009540 tx=0x7f0e140a8040).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 msgr2=0x7f0e1414b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 rx=0x7f0e0c00c490 tx=0x7f0e0c007de0).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 shutdown_connections 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f0e0005f150 0x7f0e00061610 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0e140c6190 0x7f0e1414b370 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0e14064350 0x7f0e140ac4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 --2- 172.21.15.73:0/3421434406 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f0e14098d10 0x7f0e141594b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 >> 172.21.15.73:0/3421434406 conn(0x7f0e140cb310 msgr2=0x7f0e14158000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 shutdown_connections 2023-11-28T04:26:47.200 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.199+0000 7f0e1b88b700 1 -- 172.21.15.73:0/3421434406 wait complete. 2023-11-28T04:26:47.201 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 0 to deep-scrub 2023-11-28T04:26:47.211 INFO:tasks.ceph:Scrubbing osd.1 2023-11-28T04:26:47.211 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:47.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.328+0000 7faabffff700 1 Processor -- start 2023-11-28T04:26:47.328 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.328+0000 7faabffff700 1 -- start start 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabffff700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac00aa680 0x7faac00c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabffff700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac0158d50 con 0x7faac01546b0 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabffff700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faac0158140 con 0x7faac01542d0 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabffff700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faac0158420 con 0x7faac00aa680 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faab67fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7faac00aa680 0x7faac00c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabf7fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7faac01546b0 0x7faac00c64b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35818/0 (socket says 172.21.15.73:35818) 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabf7fe700 1 -- 172.21.15.73:0/374849210 learned_addr learned my addr 172.21.15.73:0/374849210 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faab67fc700 1 -- 172.21.15.73:0/374849210 >> v1:172.21.15.150:6789/0 conn(0x7faac01542d0 legacy=0x7faac00a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.331 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faab67fc700 1 -- 172.21.15.73:0/374849210 >> v1:172.21.15.73:6789/0 conn(0x7faac01546b0 legacy=0x7faac00c64b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faab67fc700 1 -- 172.21.15.73:0/374849210 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faac0158700 con 0x7faac00aa680 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabe7fc700 1 -- 172.21.15.73:0/374849210 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 923712666 0 0) 0x7faac0158d50 con 0x7faac01546b0 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faab67fc700 1 --2- 172.21.15.73:0/374849210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00aa680 0x7faac00c5fc0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7faaa8009760 tx=0x7faaa8009060).ready entity=mon.0 client_cookie=5418d00701a113e2 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabe7fc700 1 -- 172.21.15.73:0/374849210 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800b580 con 0x7faac00aa680 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabe7fc700 1 -- 172.21.15.73:0/374849210 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa800b6e0 con 0x7faac00aa680 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.329+0000 7faabe7fc700 1 -- 172.21.15.73:0/374849210 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa800ba60 con 0x7faac00aa680 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 -- 172.21.15.73:0/374849210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00aa680 msgr2=0x7faac00c5fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.332 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 --2- 172.21.15.73:0/374849210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00aa680 0x7faac00c5fc0 secure :-1 s=READY pgs=2170 cs=0 l=1 rev1=1 rx=0x7faaa8009760 tx=0x7faaa8009060).stop 2023-11-28T04:26:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 -- 172.21.15.73:0/374849210 shutdown_connections 2023-11-28T04:26:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 --2- 172.21.15.73:0/374849210 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac00aa680 0x7faac00c5fc0 unknown :-1 s=CLOSED pgs=2170 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 -- 172.21.15.73:0/374849210 >> 172.21.15.73:0/374849210 conn(0x7faac00cb010 msgr2=0x7faac00cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 -- 172.21.15.73:0/374849210 shutdown_connections 2023-11-28T04:26:47.333 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.330+0000 7faabffff700 1 -- 172.21.15.73:0/374849210 wait complete. 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.331+0000 7faabffff700 1 Processor -- start 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.331+0000 7faabffff700 1 -- start start 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac00aa680 0x7faac0168580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac01542d0 0x7faac00b0c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 0x7faac014cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac014dd30 con 0x7faac01546b0 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faac00af7c0 con 0x7faac00aa680 2023-11-28T04:26:47.337 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabffff700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faac00ae720 con 0x7faac01542d0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 0x7faac014cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 0x7faac014cd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40160/0 (socket says 172.21.15.73:40160) 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 -- 172.21.15.73:0/377331570 learned_addr learned my addr 172.21.15.73:0/377331570 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabeffd700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac00aa680 0x7faac0168580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faab67fc700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac01542d0 0x7faac00b0c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 -- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac01542d0 msgr2=0x7faac00b0c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7faac01542d0 0x7faac00b0c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 -- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac00aa680 msgr2=0x7faac0168580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7faac00aa680 0x7faac0168580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.332+0000 7faabf7fe700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80092a0 con 0x7faac01546b0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faabf7fe700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 0x7faac014cd30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 rx=0x7faab800c730 tx=0x7faab8001520).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.338 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8005d30 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faab8005e90 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faac00aea00 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faab8007300 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faac00aefc0 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.333+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7faac006c7c0 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7faab80076b0 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7faaa005f150 0x7faaa0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7faab8079ad0 con 0x7faac01546b0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faabeffd700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7faaa005f150 0x7faaa0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7faaa0065a30 0x7faaa0067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7faaa0068510 con 0x7faaa0065a30 2023-11-28T04:26:47.339 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.334+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7faab807e220 con 0x7faac01546b0 2023-11-28T04:26:47.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.335+0000 7faab67fc700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7faaa0065a30 0x7faaa0067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.335+0000 7faabeffd700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7faaa005f150 0x7faaa0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7faab0003900 tx=0x7faab0007d40).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.335+0000 7faab67fc700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7faaa0065a30 0x7faaa0067e90 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.340 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.335+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== osd.1 v2:172.21.15.73:6824/90262 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7faaa0068510 con 0x7faaa0065a30 2023-11-28T04:26:47.348 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.348+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 --> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7faac00c0120 con 0x7faaa0065a30 2023-11-28T04:26:47.349 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faab77fe700 1 -- 172.21.15.73:0/377331570 <== osd.1 v2:172.21.15.73:6824/90262 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7faac00c0120 con 0x7faaa0065a30 2023-11-28T04:26:47.349 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:47.349 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:47.349 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:47.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7faaa0065a30 msgr2=0x7faaa0067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6824/90262,v1:172.21.15.73:6825/90262] conn(0x7faaa0065a30 0x7faaa0067e90 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.350 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7faaa005f150 msgr2=0x7faaa0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7faaa005f150 0x7faaa0061610 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 rx=0x7faab0003900 tx=0x7faab0007d40).stop 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 msgr2=0x7faac014cd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 --2- 172.21.15.73:0/377331570 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7faac01546b0 0x7faac014cd30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 rx=0x7faab800c730 tx=0x7faab8001520).stop 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabf7fe700 1 -- 172.21.15.73:0/377331570 reap_dead start 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 shutdown_connections 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 >> 172.21.15.73:0/377331570 conn(0x7faac00cb010 msgr2=0x7faac00c5db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 shutdown_connections 2023-11-28T04:26:47.351 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.349+0000 7faabffff700 1 -- 172.21.15.73:0/377331570 wait complete. 2023-11-28T04:26:47.360 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-11-28T04:26:47.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.473+0000 7f3f5f844700 1 Processor -- start 2023-11-28T04:26:47.473 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.473+0000 7f3f5f844700 1 -- start start 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5f844700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58098d10 0x7f3f580c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5f844700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f3f580c6190 0x7f3f58063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5f844700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f580c72d0 con 0x7f3f58064350 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5f844700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f58154bc0 con 0x7f3f580c6190 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5f844700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580c66c0 con 0x7f3f58098d10 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5dde1700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f3f58064350 0x7f3f58158320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35828/0 (socket says 172.21.15.73:35828) 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5dde1700 1 -- 172.21.15.73:0/3908744316 learned_addr learned my addr 172.21.15.73:0/3908744316 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:47.476 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/3908744316 >> v2:172.21.15.150:3300/0 conn(0x7f3f580c6190 0x7f3f58063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f5d5e0700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58098d10 0x7f3f580c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195716254 0 0) 0x7f3f580c72d0 con 0x7f3f58064350 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.474+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3f400033f0 con 0x7f3f58064350 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5cddf700 1 -- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58098d10 msgr2=0x7f3f580c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58098d10 0x7f3f580c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5cddf700 1 -- 172.21.15.73:0/3908744316 >> v1:172.21.15.73:6789/0 conn(0x7f3f58064350 legacy=0x7f3f58158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2754711495 0 0) 0x7f3f400033f0 con 0x7f3f58064350 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5cddf700 1 -- 172.21.15.73:0/3908744316 --> v2:172.21.15.150:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f580c69a0 con 0x7f3f580c6190 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58063e60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f3f4c009760 tx=0x7f3f4c009060).ready entity=mon.1 client_cookie=668502e932f1c3c1 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00d4a0 con 0x7f3f580c6190 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00d600 con 0x7f3f580c6190 2023-11-28T04:26:47.477 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f57fff700 1 -- 172.21.15.73:0/3908744316 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00d980 con 0x7f3f580c6190 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 -- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 msgr2=0x7f3f58063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58063e60 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 rx=0x7f3f4c009760 tx=0x7f3f4c009060).stop 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 -- 172.21.15.73:0/3908744316 shutdown_connections 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58098d10 0x7f3f580c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 --2- 172.21.15.73:0/3908744316 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58063e60 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.475+0000 7f3f5f844700 1 -- 172.21.15.73:0/3908744316 >> 172.21.15.73:0/3908744316 conn(0x7f3f580cb310 msgr2=0x7f3f580cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.476+0000 7f3f5f844700 1 -- 172.21.15.73:0/3908744316 shutdown_connections 2023-11-28T04:26:47.478 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.476+0000 7f3f5f844700 1 -- 172.21.15.73:0/3908744316 wait complete. 2023-11-28T04:26:47.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.477+0000 7f3f5f844700 1 Processor -- start 2023-11-28T04:26:47.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.477+0000 7f3f5f844700 1 -- start start 2023-11-28T04:26:47.484 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58064350 0x7f3f580c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f58165350 con 0x7f3f58098d10 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580a8950 con 0x7f3f580c6190 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5f844700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f580c4d80 con 0x7f3f58064350 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40170/0 (socket says 172.21.15.73:40170) 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 -- 172.21.15.73:0/2008122572 learned_addr learned my addr 172.21.15.73:0/2008122572 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5dde1700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5d5e0700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58064350 0x7f3f580c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 -- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58064350 msgr2=0x7f3f580c23b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58064350 0x7f3f580c23b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.485 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 -- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 msgr2=0x7f3f58156650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58156650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 -- 172.21.15.73:0/2008122572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c0092a0 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.478+0000 7f3f5cddf700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7f3f580c2360 tx=0x7f3f4c016000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.479+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c00dcf0 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.479+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c00de50 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.479+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f580c5060 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.479+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c01d430 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.479+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f580c4a80 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.480+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f5806c860 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.480+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f3f4c01d7e0 con 0x7f3f58098d10 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.481+0000 7f3f557fa700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 0x7f3f44061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.486 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.481+0000 7f3f5d5e0700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 0x7f3f44061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.481+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f3f4c07b690 con 0x7f3f58098d10 2023-11-28T04:26:47.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.481+0000 7f3f5d5e0700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 0x7f3f44061610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3f4800f570 tx=0x7f3f4800b040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.487 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.485+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f3f4c04ca80 con 0x7f3f58098d10 2023-11-28T04:26:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.649+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f580a8460 con 0x7f3f4405f150 2023-11-28T04:26:47.649 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.649+0000 7f3f557fa700 1 -- 172.21.15.73:0/2008122572 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f3f4800f040 con 0x7f3f4405f150 2023-11-28T04:26:47.651 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 msgr2=0x7f3f44061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 0x7f3f44061610 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 rx=0x7f3f4800f570 tx=0x7f3f4800b040).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 msgr2=0x7f3f58164680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 rx=0x7f3f580c2360 tx=0x7f3f4c016000).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 shutdown_connections 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f3f4405f150 0x7f3f44061610 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3f580c6190 0x7f3f58156650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3f58064350 0x7f3f580c23b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 --2- 172.21.15.73:0/2008122572 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f3f58098d10 0x7f3f58164680 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 >> 172.21.15.73:0/2008122572 conn(0x7f3f580cb310 msgr2=0x7f3f58063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 shutdown_connections 2023-11-28T04:26:47.652 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.651+0000 7f3f5f844700 1 -- 172.21.15.73:0/2008122572 wait complete. 2023-11-28T04:26:47.653 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 1 to deep-scrub 2023-11-28T04:26:47.663 INFO:tasks.ceph:Scrubbing osd.2 2023-11-28T04:26:47.663 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:47.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.780+0000 7ff2fcf3b700 1 Processor -- start 2023-11-28T04:26:47.780 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.780+0000 7ff2fcf3b700 1 -- start start 2023-11-28T04:26:47.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.780+0000 7ff2fcf3b700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 0x7ff2f80a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2fcf3b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff2f81546b0 0x7ff2f80aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.783 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2fcf3b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 0x7ff2f80c6450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2fcf3b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f80aa950 con 0x7ff2f81546b0 2023-11-28T04:26:47.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2fcf3b700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2f8158d50 con 0x7ff2f81542d0 2023-11-28T04:26:47.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2fcf3b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8158140 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.784 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff2f81546b0 0x7ff2f80aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7ff2f81546b0 0x7ff2f80aa460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40174/0 (socket says 172.21.15.73:40174) 2023-11-28T04:26:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 -- 172.21.15.73:0/3547862914 learned_addr learned my addr 172.21.15.73:0/3547862914 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f77fe700 1 --2- 172.21.15.73:0/3547862914 >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 0x7ff2f80a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 0x7ff2f80c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.785 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 -- 172.21.15.73:0/3547862914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 msgr2=0x7ff2f80c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 0x7ff2f80c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 -- 172.21.15.73:0/3547862914 >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 msgr2=0x7ff2f80a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/3547862914 >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 0x7ff2f80a9f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 -- 172.21.15.73:0/3547862914 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2f8158420 con 0x7ff2f81546b0 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f77fe700 1 --2- 172.21.15.73:0/3547862914 >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 0x7ff2f80a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:47.786 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 0x7ff2f80c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:47.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f81546b0 0x7ff2f80aa460 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 rx=0x7ff2f80c07e0 tx=0x7ff2ec009840).ready entity=mon.0 client_cookie=37ea16687322b05c server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f67fc700 1 -- 172.21.15.73:0/3547862914 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00e070 con 0x7ff2f81546b0 2023-11-28T04:26:47.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f67fc700 1 -- 172.21.15.73:0/3547862914 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2ec00ba50 con 0x7ff2f81546b0 2023-11-28T04:26:47.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.781+0000 7ff2f67fc700 1 -- 172.21.15.73:0/3547862914 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2ec00bdd0 con 0x7ff2f81546b0 2023-11-28T04:26:47.787 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/3547862914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f81546b0 msgr2=0x7ff2f80aa460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f81546b0 0x7ff2f80aa460 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 rx=0x7ff2f80c07e0 tx=0x7ff2ec009840).stop 2023-11-28T04:26:47.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/3547862914 shutdown_connections 2023-11-28T04:26:47.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f80c5fc0 0x7ff2f80c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/3547862914 >> v2:172.21.15.150:3300/0 conn(0x7ff2f81542d0 0x7ff2f80a9f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.788 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/3547862914 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f81546b0 0x7ff2f80aa460 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/3547862914 >> 172.21.15.73:0/3547862914 conn(0x7ff2f80cb010 msgr2=0x7ff2f80cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/3547862914 shutdown_connections 2023-11-28T04:26:47.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.782+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/3547862914 wait complete. 2023-11-28T04:26:47.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.783+0000 7ff2fcf3b700 1 Processor -- start 2023-11-28T04:26:47.789 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.783+0000 7ff2fcf3b700 1 -- start start 2023-11-28T04:26:47.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 0x7ff2f80adfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.792 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f81542d0 0x7ff2f80af410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f81546b0 0x7ff2f814cd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f814d220 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f8154fc0 con 0x7ff2f81542d0 2023-11-28T04:26:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2fcf3b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2f80ae580 con 0x7ff2f81546b0 2023-11-28T04:26:47.793 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 0x7ff2f80adfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 0x7ff2f80adfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40182/0 (socket says 172.21.15.73:40182) 2023-11-28T04:26:47.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 -- 172.21.15.73:0/364031970 learned_addr learned my addr 172.21.15.73:0/364031970 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f81542d0 0x7ff2f80af410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f81546b0 0x7ff2f814cd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.794 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 -- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f81546b0 msgr2=0x7ff2f814cd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f81546b0 0x7ff2f814cd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 -- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f81542d0 msgr2=0x7ff2f80af410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7ff2f81542d0 0x7ff2f80af410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2ec009ae0 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7ff2f81546b0 0x7ff2f814cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:47.795 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2f77fe700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 0x7ff2f80adfa0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7ff2f000d890 tx=0x7ff2f0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.784+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f00125f0 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.785+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2f0012750 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.785+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2f80ae860 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.785+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2f0012ad0 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.796 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.785+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2f80aee20 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff2f806c7c0 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7ff2f0013480 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff2e405f150 0x7ff2e4061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7ff2f007a110 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff2e405f150 0x7ff2e4061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.797 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7ff2e4065a30 0x7ff2e4067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff2e4068510 con 0x7ff2e4065a30 2023-11-28T04:26:47.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff2f007c730 con 0x7ff2f80c5fc0 2023-11-28T04:26:47.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7ff2e4065a30 0x7ff2e4067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.798 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.786+0000 7ff2f6ffd700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff2e405f150 0x7ff2e4061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff2f80af2d0 tx=0x7ff2ec009840).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.787+0000 7ff2f7fff700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7ff2e4065a30 0x7ff2e4067e90 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.799 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.787+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== osd.2 v2:172.21.15.73:6808/90248 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7ff2e4068510 con 0x7ff2e4065a30 2023-11-28T04:26:47.801 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.800+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 --> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff2f81552a0 con 0x7ff2e4065a30 2023-11-28T04:26:47.802 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.800+0000 7ff2df7fe700 1 -- 172.21.15.73:0/364031970 <== osd.2 v2:172.21.15.73:6808/90248 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff2f81552a0 con 0x7ff2e4065a30 2023-11-28T04:26:47.804 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:47.804 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:47.804 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:47.805 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7ff2e4065a30 msgr2=0x7ff2e4067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6808/90248,v1:172.21.15.73:6809/90248] conn(0x7ff2e4065a30 0x7ff2e4067e90 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff2e405f150 msgr2=0x7ff2e4061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7ff2e405f150 0x7ff2e4061610 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 rx=0x7ff2f80af2d0 tx=0x7ff2ec009840).stop 2023-11-28T04:26:47.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 msgr2=0x7ff2f80adfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.807 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 --2- 172.21.15.73:0/364031970 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7ff2f80c5fc0 0x7ff2f80adfa0 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 rx=0x7ff2f000d890 tx=0x7ff2f0008040).stop 2023-11-28T04:26:47.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2f7fff700 1 -- 172.21.15.73:0/364031970 reap_dead start 2023-11-28T04:26:47.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 shutdown_connections 2023-11-28T04:26:47.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 >> 172.21.15.73:0/364031970 conn(0x7ff2f80cb010 msgr2=0x7ff2f814daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 shutdown_connections 2023-11-28T04:26:47.808 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.801+0000 7ff2fcf3b700 1 -- 172.21.15.73:0/364031970 wait complete. 2023-11-28T04:26:47.812 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-11-28T04:26:47.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.926+0000 7fc32e4f2700 1 Processor -- start 2023-11-28T04:26:47.926 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.926+0000 7fc32e4f2700 1 -- start start 2023-11-28T04:26:47.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc32e4f2700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3280c6190 0x7fc3280c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.928 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc32e4f2700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3280c72d0 con 0x7fc328098d10 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc32e4f2700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc328064570 con 0x7fc328158320 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc32e4f2700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc328154bc0 con 0x7fc3280c6190 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc3277fe700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fc3280c6190 0x7fc3280c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc327fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fc328098d10 0x7fc328063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35842/0 (socket says 172.21.15.73:35842) 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc327fff700 1 -- 172.21.15.73:0/3805599365 learned_addr learned my addr 172.21.15.73:0/3805599365 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc3277fe700 1 -- 172.21.15.73:0/3805599365 >> v1:172.21.15.150:6789/0 conn(0x7fc328158320 legacy=0x7fc3280a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc3277fe700 1 -- 172.21.15.73:0/3805599365 >> v1:172.21.15.73:6789/0 conn(0x7fc328098d10 legacy=0x7fc328063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.927+0000 7fc326ffd700 1 -- 172.21.15.73:0/3805599365 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720489161 0 0) 0x7fc3280c72d0 con 0x7fc328098d10 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc3277fe700 1 -- 172.21.15.73:0/3805599365 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3280c66c0 con 0x7fc3280c6190 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc3277fe700 1 --2- 172.21.15.73:0/3805599365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280c6190 0x7fc3280c7760 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7fc31c009760 tx=0x7fc31c009060).ready entity=mon.0 client_cookie=b39cea3237364f87 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc326ffd700 1 -- 172.21.15.73:0/3805599365 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00b580 con 0x7fc3280c6190 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc326ffd700 1 -- 172.21.15.73:0/3805599365 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc31c00b6e0 con 0x7fc3280c6190 2023-11-28T04:26:47.929 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc326ffd700 1 -- 172.21.15.73:0/3805599365 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc31c00ba60 con 0x7fc3280c6190 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 -- 172.21.15.73:0/3805599365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280c6190 msgr2=0x7fc3280c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/3805599365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280c6190 0x7fc3280c7760 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 rx=0x7fc31c009760 tx=0x7fc31c009060).stop 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 -- 172.21.15.73:0/3805599365 shutdown_connections 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/3805599365 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc3280c6190 0x7fc3280c7760 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 -- 172.21.15.73:0/3805599365 >> 172.21.15.73:0/3805599365 conn(0x7fc3280cb310 msgr2=0x7fc3280cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 -- 172.21.15.73:0/3805599365 shutdown_connections 2023-11-28T04:26:47.930 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.928+0000 7fc32e4f2700 1 -- 172.21.15.73:0/3805599365 wait complete. 2023-11-28T04:26:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.929+0000 7fc32e4f2700 1 Processor -- start 2023-11-28T04:26:47.932 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 -- start start 2023-11-28T04:26:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc328098d10 0x7fc32814e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.933 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3280c6190 0x7fc328155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280afe00 con 0x7fc328158320 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280af280 con 0x7fc3280c6190 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32e4f2700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3280af560 con 0x7fc328098d10 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.930+0000 7fc32ca8f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40188/0 (socket says 172.21.15.73:40188) 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 -- 172.21.15.73:0/577189899 learned_addr learned my addr 172.21.15.73:0/577189899 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc3277fe700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3280c6190 0x7fc328155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc327fff700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc328098d10 0x7fc32814e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 -- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc328098d10 msgr2=0x7fc32814e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc328098d10 0x7fc32814e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 -- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3280c6190 msgr2=0x7fc328155950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:47.934 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3280c6190 0x7fc328155950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 -- 172.21.15.73:0/577189899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc31c0092a0 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32ca8f700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 rx=0x7fc32000c7c0 tx=0x7fc320001510).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000e4a0 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc32000e600 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3280af840 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc32000e980 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.931+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3280aef90 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.932+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc32806c860 con 0x7fc328158320 2023-11-28T04:26:47.935 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.932+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fc3200123f0 con 0x7fc328158320 2023-11-28T04:26:47.936 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.933+0000 7fc30ffff700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 0x7fc3140615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.933+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fc3200797b0 con 0x7fc328158320 2023-11-28T04:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.933+0000 7fc327fff700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 0x7fc3140615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.933+0000 7fc327fff700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 0x7fc3140615e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc3180011e0 tx=0x7fc318007b90).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:47.937 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:47.937+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fc320047af0 con 0x7fc328158320 2023-11-28T04:26:48.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.100+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3280c69a0 con 0x7fc31405f120 2023-11-28T04:26:48.100 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.100+0000 7fc30ffff700 1 -- 172.21.15.73:0/577189899 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fc3280c69a0 con 0x7fc31405f120 2023-11-28T04:26:48.102 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 msgr2=0x7fc3140615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 0x7fc3140615e0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 rx=0x7fc3180011e0 tx=0x7fc318007b90).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 msgr2=0x7fc3280aeaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 rx=0x7fc32000c7c0 tx=0x7fc320001510).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 shutdown_connections 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fc31405f120 0x7fc3140615e0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fc328158320 0x7fc3280aeaa0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fc328098d10 0x7fc32814e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 --2- 172.21.15.73:0/577189899 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fc3280c6190 0x7fc328155950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 >> 172.21.15.73:0/577189899 conn(0x7fc3280cb310 msgr2=0x7fc328063990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 shutdown_connections 2023-11-28T04:26:48.103 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.102+0000 7fc32e4f2700 1 -- 172.21.15.73:0/577189899 wait complete. 2023-11-28T04:26:48.104 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 2 to deep-scrub 2023-11-28T04:26:48.114 INFO:tasks.ceph:Scrubbing osd.3 2023-11-28T04:26:48.114 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:48.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.229+0000 7f74d9187700 1 Processor -- start 2023-11-28T04:26:48.230 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.229+0000 7f74d9187700 1 -- start start 2023-11-28T04:26:48.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d9187700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 0x7f74d41582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d9187700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74d40aa1c0 0x7f74d40aa600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d9187700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74d4159210 con 0x7f74d40ab3b0 2023-11-28T04:26:48.232 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d9187700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d40c0c70 con 0x7f74d40aa1c0 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d9187700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74d40c2110 con 0x7f74d4154610 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d359e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74d40aa1c0 0x7f74d40aa600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d359e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f74d40aa1c0 0x7f74d40aa600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40200/0 (socket says 172.21.15.73:40200) 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d359e700 1 -- 172.21.15.73:0/2398629154 learned_addr learned my addr 172.21.15.73:0/2398629154 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.230+0000 7f74d259c700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f74d40ab3b0 0x7f74d40c80c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35858/0 (socket says 172.21.15.73:35858) 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/2398629154 >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 0x7f74d41582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d359e700 1 -- 172.21.15.73:0/2398629154 >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 msgr2=0x7f74d41582e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d359e700 1 --2- 172.21.15.73:0/2398629154 >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 0x7f74d41582e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d359e700 1 -- 172.21.15.73:0/2398629154 >> v1:172.21.15.73:6789/0 conn(0x7f74d40ab3b0 legacy=0x7f74d40c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d1d9b700 1 -- 172.21.15.73:0/2398629154 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3475743023 0 0) 0x7f74d4159210 con 0x7f74d40ab3b0 2023-11-28T04:26:48.233 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d359e700 1 -- 172.21.15.73:0/2398629154 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74d40aaaf0 con 0x7f74d40aa1c0 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/2398629154 >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 0x7f74d41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d359e700 1 --2- 172.21.15.73:0/2398629154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d40aa1c0 0x7f74d40aa600 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 rx=0x7f74d4062e70 tx=0x7f74cc009840).ready entity=mon.0 client_cookie=b29ed9c7eb5f1039 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d1d9b700 1 -- 172.21.15.73:0/2398629154 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00e070 con 0x7f74d40aa1c0 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d1d9b700 1 -- 172.21.15.73:0/2398629154 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc00ba50 con 0x7f74d40aa1c0 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d1d9b700 1 -- 172.21.15.73:0/2398629154 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc00bdd0 con 0x7f74d40aa1c0 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d9187700 1 -- 172.21.15.73:0/2398629154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d40aa1c0 msgr2=0x7f74d40aa600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.231+0000 7f74d9187700 1 --2- 172.21.15.73:0/2398629154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d40aa1c0 0x7f74d40aa600 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 rx=0x7f74d4062e70 tx=0x7f74cc009840).stop 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 -- 172.21.15.73:0/2398629154 shutdown_connections 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 --2- 172.21.15.73:0/2398629154 >> v2:172.21.15.150:3300/0 conn(0x7f74d4154610 0x7f74d41582e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 --2- 172.21.15.73:0/2398629154 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d40aa1c0 0x7f74d40aa600 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 -- 172.21.15.73:0/2398629154 >> 172.21.15.73:0/2398629154 conn(0x7f74d40655a0 msgr2=0x7f74d4069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.234 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 -- 172.21.15.73:0/2398629154 shutdown_connections 2023-11-28T04:26:48.235 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.232+0000 7f74d9187700 1 -- 172.21.15.73:0/2398629154 wait complete. 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.233+0000 7f74d9187700 1 Processor -- start 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.233+0000 7f74d9187700 1 -- start start 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74d40aa1c0 0x7f74d40b0e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 0x7f74d4167f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74d4165620 0x7f74d4168440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4164940 con 0x7f74d4154610 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4168930 con 0x7f74d4165620 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74d4165190 con 0x7f74d40aa1c0 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 0x7f74d4167f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 0x7f74d4167f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40214/0 (socket says 172.21.15.73:40214) 2023-11-28T04:26:48.238 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 -- 172.21.15.73:0/1342598689 learned_addr learned my addr 172.21.15.73:0/1342598689 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d359e700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74d4165620 0x7f74d4168440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74d40aa1c0 0x7f74d40b0e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 -- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74d40aa1c0 msgr2=0x7f74d40b0e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74d40aa1c0 0x7f74d40b0e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 -- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74d4165620 msgr2=0x7f74d4168440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f74d4165620 0x7f74d4168440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc009ae0 con 0x7f74d4154610 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f74d40aa1c0 0x7f74d40b0e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:48.239 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d259c700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 0x7f74d4167f50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 rx=0x7f74c800c4e0 tx=0x7f74c8007c60).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80072e0 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c8007440 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c80058f0 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74d4168b30 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.234+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74d4164e20 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.235+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f74d4153eb0 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f74c8005ca0 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f74c005f150 0x7f74c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f74c80797a0 con 0x7f74d4154610 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f74c005f150 0x7f74c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f74c0065a30 0x7f74c0067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.240 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f74c0068510 con 0x7f74c0065a30 2023-11-28T04:26:48.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7f74c807bdc0 con 0x7f74d4154610 2023-11-28T04:26:48.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74d359e700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f74c0065a30 0x7f74c0067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.236+0000 7f74d2d9d700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f74c005f150 0x7f74c0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f74c4006d80 tx=0x7f74c4008040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.237+0000 7f74d359e700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f74c0065a30 0x7f74c0067e90 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.241 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.237+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== osd.3 v2:172.21.15.73:6816/90251 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f74c0068510 con 0x7f74c0065a30 2023-11-28T04:26:48.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.250+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 --> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f74d406c860 con 0x7f74c0065a30 2023-11-28T04:26:48.250 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.250+0000 7f74baffd700 1 -- 172.21.15.73:0/1342598689 <== osd.3 v2:172.21.15.73:6816/90251 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f74d406c860 con 0x7f74c0065a30 2023-11-28T04:26:48.251 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:48.251 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:48.251 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f74c0065a30 msgr2=0x7f74c0067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6816/90251,v1:172.21.15.73:6817/90251] conn(0x7f74c0065a30 0x7f74c0067e90 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f74c005f150 msgr2=0x7f74c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f74c005f150 0x7f74c0061610 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 rx=0x7f74c4006d80 tx=0x7f74c4008040).stop 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 msgr2=0x7f74d4167f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 --2- 172.21.15.73:0/1342598689 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f74d4154610 0x7f74d4167f50 secure :-1 s=READY pgs=2181 cs=0 l=1 rev1=1 rx=0x7f74c800c4e0 tx=0x7f74c8007c60).stop 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d359e700 1 -- 172.21.15.73:0/1342598689 reap_dead start 2023-11-28T04:26:48.252 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 shutdown_connections 2023-11-28T04:26:48.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 >> 172.21.15.73:0/1342598689 conn(0x7f74d40655a0 msgr2=0x7f74d40c1100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 shutdown_connections 2023-11-28T04:26:48.253 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.251+0000 7f74d9187700 1 -- 172.21.15.73:0/1342598689 wait complete. 2023-11-28T04:26:48.262 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-11-28T04:26:48.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.376+0000 7f927be1e700 1 Processor -- start 2023-11-28T04:26:48.376 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.376+0000 7f927be1e700 1 -- start start 2023-11-28T04:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f927be1e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f92740c7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f927be1e700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92740c7500 con 0x7f92740c8780 2023-11-28T04:26:48.383 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f927be1e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92740bd650 con 0x7f927413be20 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f927be1e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f927413c200 con 0x7f9274062e10 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f92793b9700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f92740c8780 0x7f9274062700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35866/0 (socket says 172.21.15.73:35866) 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f92793b9700 1 -- 172.21.15.73:0/2293821529 learned_addr learned my addr 172.21.15.73:0/2293821529 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f927a3bb700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f92740c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3992152240 0 0) 0x7f92740c7500 con 0x7f92740c8780 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.377+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92600033f0 con 0x7f92740c8780 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197184893 0 0) 0x7f92740bd650 con 0x7f927413be20 2023-11-28T04:26:48.384 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92740c7500 con 0x7f927413be20 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 355965563 0 0) 0x7f92600033f0 con 0x7f92740c8780 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 msgr2=0x7f92740c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f92740c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 >> v1:172.21.15.150:6789/0 conn(0x7f927413be20 legacy=0x7f92740bcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740c68f0 con 0x7f92740c8780 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f9264001470 con 0x7f92740c8780 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f927a3bb700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f92740c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:48.385 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 >> v1:172.21.15.73:6789/0 conn(0x7f92740c8780 legacy=0x7f9274062700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260003aa0 0x7f9260003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9260004690 0x7f9260004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f9260005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f927413c200 con 0x7f9260003aa0 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f926000b8e0 con 0x7f9260004690 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f926000ba70 con 0x7f9274062e10 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260003aa0 0x7f9260003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.386 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f927a3bb700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f9260005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f92793b9700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9260004690 0x7f9260004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 -- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 msgr2=0x7f9260005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9274062e10 0x7f9260005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 -- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9260004690 msgr2=0x7f9260004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f9260004690 0x7f9260004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 -- 172.21.15.73:0/2293821529 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f926c009a20 con 0x7f9260003aa0 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9279bba700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260003aa0 0x7f9260003ea0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 rx=0x7f92700018d0 tx=0x7f9270007bb0).ready entity=mon.0 client_cookie=a12cfc251c0a8436 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270007020 con 0x7f9260003aa0 2023-11-28T04:26:48.387 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9270007180 con 0x7f9260003aa0 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.378+0000 7f9278bb8700 1 -- 172.21.15.73:0/2293821529 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9270007500 con 0x7f9260003aa0 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 -- 172.21.15.73:0/2293821529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260003aa0 msgr2=0x7f9260003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 --2- 172.21.15.73:0/2293821529 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f9260003aa0 0x7f9260003ea0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 rx=0x7f92700018d0 tx=0x7f9270007bb0).stop 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927a3bb700 1 -- 172.21.15.73:0/2293821529 reap_dead start 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 -- 172.21.15.73:0/2293821529 shutdown_connections 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 -- 172.21.15.73:0/2293821529 >> 172.21.15.73:0/2293821529 conn(0x7f92740642d0 msgr2=0x7f92740c7fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 -- 172.21.15.73:0/2293821529 shutdown_connections 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.379+0000 7f927be1e700 1 -- 172.21.15.73:0/2293821529 wait complete. 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.380+0000 7f927be1e700 1 Processor -- start 2023-11-28T04:26:48.388 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.380+0000 7f927be1e700 1 -- start start 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9260003aa0 0x7f9274138030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92740c0bc0 0x7f92740ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9274138520 con 0x7f92740aaf90 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92740c0730 con 0x7f92740c0bc0 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927be1e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92740abdd0 con 0x7f9260003aa0 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40224/0 (socket says 172.21.15.73:40224) 2023-11-28T04:26:48.389 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 -- 172.21.15.73:0/3077572085 learned_addr learned my addr 172.21.15.73:0/3077572085 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f9279bba700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9260003aa0 0x7f9274138030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f92793b9700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92740c0bc0 0x7f92740ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 -- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9260003aa0 msgr2=0x7f9274138030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9260003aa0 0x7f9274138030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 -- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92740c0bc0 msgr2=0x7f92740ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92740c0bc0 0x7f92740ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 -- 172.21.15.73:0/3077572085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92740ab4e0 con 0x7f92740aaf90 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f927a3bb700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7f92740a7890 tx=0x7f926c00b040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00e040 con 0x7f92740aaf90 2023-11-28T04:26:48.390 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.381+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f926c00f040 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.382+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92740bfe00 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.382+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f926c00fd40 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.382+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92740c03c0 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.383+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f927406c860 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.383+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f926c015660 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.383+0000 7f9269ffb700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 0x7f9258061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.383+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f926c07ac50 con 0x7f92740aaf90 2023-11-28T04:26:48.391 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.383+0000 7f9279bba700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 0x7f9258061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.384+0000 7f9279bba700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 0x7f9258061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f92700018d0 tx=0x7f9270016000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.392 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.387+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f926c04c0a0 con 0x7f92740aaf90 2023-11-28T04:26:48.554 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.554+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f92740c6bd0 con 0x7f925805f150 2023-11-28T04:26:48.555 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.554+0000 7f9269ffb700 1 -- 172.21.15.73:0/3077572085 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f92740c6bd0 con 0x7f925805f150 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 msgr2=0x7f9258061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 0x7f9258061610 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 rx=0x7f92700018d0 tx=0x7f9270016000).stop 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 msgr2=0x7f9274158250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 rx=0x7f92740a7890 tx=0x7f926c00b040).stop 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 shutdown_connections 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f925805f150 0x7f9258061610 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.557 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f92740aaf90 0x7f9274158250 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f9260003aa0 0x7f9274138030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 --2- 172.21.15.73:0/3077572085 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f92740c0bc0 0x7f92740ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.556+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 >> 172.21.15.73:0/3077572085 conn(0x7f92740642d0 msgr2=0x7f92740a5520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.557+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 shutdown_connections 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.557+0000 7f927be1e700 1 -- 172.21.15.73:0/3077572085 wait complete. 2023-11-28T04:26:48.558 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 3 to deep-scrub 2023-11-28T04:26:48.568 INFO:tasks.ceph:Scrubbing osd.4 2023-11-28T04:26:48.568 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:48.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.685+0000 7f382c149700 1 Processor -- start 2023-11-28T04:26:48.685 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.685+0000 7f382c149700 1 -- start start 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382c149700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 0x7f38241582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382c149700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3824159210 con 0x7f38240aa1c0 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382c149700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38240c0c70 con 0x7f38240ab3b0 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382c149700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38240c2110 con 0x7f3824154610 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382a6e6700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f38240aa1c0 0x7f38240aa5f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35882/0 (socket says 172.21.15.73:35882) 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f382a6e6700 1 -- 172.21.15.73:0/4063512300 learned_addr learned my addr 172.21.15.73:0/4063512300 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f3829ee5700 1 --2- 172.21.15.73:0/4063512300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 0x7f38241582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.688 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1676758 0 0) 0x7f3824159210 con 0x7f38240aa1c0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.686+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f380c0033f0 con 0x7f38240aa1c0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111205709 0 0) 0x7f38240c0c70 con 0x7f38240ab3b0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 --> v1:172.21.15.150:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3824159210 con 0x7f38240ab3b0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3829ee5700 1 -- 172.21.15.73:0/4063512300 >> v1:172.21.15.150:6789/0 conn(0x7f38240ab3b0 legacy=0x7f38240c80c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3829ee5700 1 -- 172.21.15.73:0/4063512300 >> v1:172.21.15.73:6789/0 conn(0x7f38240aa1c0 legacy=0x7f38240aa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2965963278 0 0) 0x7f380c0033f0 con 0x7f38240aa1c0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3829ee5700 1 -- 172.21.15.73:0/4063512300 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3824158970 con 0x7f3824154610 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3829ee5700 1 --2- 172.21.15.73:0/4063512300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 0x7f38241582e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3814009e90 tx=0x7f3814009780).ready entity=mon.2 client_cookie=7fb1c206dfad7640 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381400e070 con 0x7f3824154610 2023-11-28T04:26:48.689 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381400b7a0 con 0x7f3824154610 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f3828ee3700 1 -- 172.21.15.73:0/4063512300 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381c001470 con 0x7f3824154610 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f382c149700 1 -- 172.21.15.73:0/4063512300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 msgr2=0x7f38241582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f382c149700 1 --2- 172.21.15.73:0/4063512300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 0x7f38241582e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 rx=0x7f3814009e90 tx=0x7f3814009780).stop 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f382c149700 1 -- 172.21.15.73:0/4063512300 shutdown_connections 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f382c149700 1 --2- 172.21.15.73:0/4063512300 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f3824154610 0x7f38241582e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.687+0000 7f382c149700 1 -- 172.21.15.73:0/4063512300 >> 172.21.15.73:0/4063512300 conn(0x7f38240655a0 msgr2=0x7f3824069e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.688+0000 7f382c149700 1 -- 172.21.15.73:0/4063512300 shutdown_connections 2023-11-28T04:26:48.690 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.688+0000 7f382c149700 1 -- 172.21.15.73:0/4063512300 wait complete. 2023-11-28T04:26:48.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 Processor -- start 2023-11-28T04:26:48.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 -- start start 2023-11-28T04:26:48.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 0x7f382414c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3824154610 0x7f38240caee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.695 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38240b07c0 0x7f38240b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824154fb0 con 0x7f38240aa1c0 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3824155140 con 0x7f3824154610 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382c149700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382414cbf0 con 0x7f38240b07c0 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f3829ee5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 0x7f382414c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f3829ee5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 0x7f382414c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40240/0 (socket says 172.21.15.73:40240) 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f3829ee5700 1 -- 172.21.15.73:0/2997335846 learned_addr learned my addr 172.21.15.73:0/2997335846 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f382a6e6700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38240b07c0 0x7f38240b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.689+0000 7f38296e4700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3824154610 0x7f38240caee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.696 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 -- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38240b07c0 msgr2=0x7f38240b0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f38240b07c0 0x7f38240b0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 -- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3824154610 msgr2=0x7f38240caee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f3824154610 0x7f38240caee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3814009a20 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3829ee5700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 0x7f382414c700 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 rx=0x7f381400b610 tx=0x7f3814009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3814009440 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f381400be50 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38240cb3d0 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f381401d670 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.690+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38240aeea0 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.691+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f382406c860 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f38140263b0 con 0x7f38240aa1c0 2023-11-28T04:26:48.697 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f381005f150 0x7f3810061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f381407c050 con 0x7f38240aa1c0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f38296e4700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f381005f150 0x7f3810061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7f3810065a30 0x7f3810067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3810068510 con 0x7f3810065a30 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7f381407c400 con 0x7f38240aa1c0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f382a6e6700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7f3810065a30 0x7f3810067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.692+0000 7f38296e4700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f381005f150 0x7f3810061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f38240c02a0 tx=0x7f3818007bd0).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.693+0000 7f382a6e6700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7f3810065a30 0x7f3810067e90 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.698 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.693+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== osd.4 v2:172.21.15.150:6808/90482 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7f3810068510 con 0x7f3810065a30 2023-11-28T04:26:48.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.706+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 --> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f3824158c50 con 0x7f3810065a30 2023-11-28T04:26:48.706 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f3821ffb700 1 -- 172.21.15.73:0/2997335846 <== osd.4 v2:172.21.15.150:6808/90482 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f3824158c50 con 0x7f3810065a30 2023-11-28T04:26:48.707 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:48.707 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:48.707 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:48.709 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7f3810065a30 msgr2=0x7f3810067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.150:6808/90482,v1:172.21.15.150:6810/90482] conn(0x7f3810065a30 0x7f3810067e90 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f381005f150 msgr2=0x7f3810061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f381005f150 0x7f3810061610 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 rx=0x7f38240c02a0 tx=0x7f3818007bd0).stop 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 msgr2=0x7f382414c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 --2- 172.21.15.73:0/2997335846 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f38240aa1c0 0x7f382414c700 secure :-1 s=READY pgs=2186 cs=0 l=1 rev1=1 rx=0x7f381400b610 tx=0x7f3814009780).stop 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382a6e6700 1 -- 172.21.15.73:0/2997335846 reap_dead start 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 shutdown_connections 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 >> 172.21.15.73:0/2997335846 conn(0x7f38240655a0 msgr2=0x7f38240ad790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 shutdown_connections 2023-11-28T04:26:48.710 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.707+0000 7f382c149700 1 -- 172.21.15.73:0/2997335846 wait complete. 2023-11-28T04:26:48.719 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-11-28T04:26:48.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.831+0000 7f095ffe5700 1 Processor -- start 2023-11-28T04:26:48.831 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.831+0000 7f095ffe5700 1 -- start start 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095ffe5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095ffe5700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 0x7f09581587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095ffe5700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0958158cc0 con 0x7f09580c6190 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095ffe5700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09580c72d0 con 0x7f0958158320 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095ffe5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0958064570 con 0x7f0958098d10 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.832+0000 7f095dd81700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095e582700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 0x7f09581587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:54974/0 (socket says 172.21.15.73:54974) 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095d580700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7f09580c6190 0x7f0958063e60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43528/0 (socket says 172.21.15.73:43528) 2023-11-28T04:26:48.834 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 -- 172.21.15.73:0/1741858065 learned_addr learned my addr 172.21.15.73:0/1741858065 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 -- 172.21.15.73:0/1741858065 >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 msgr2=0x7f09581587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 --2- 172.21.15.73:0/1741858065 >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 0x7f09581587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 -- 172.21.15.73:0/1741858065 >> v1:172.21.15.150:6789/0 conn(0x7f09580c6190 legacy=0x7f0958063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 -- 172.21.15.73:0/1741858065 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0958154bc0 con 0x7f0958098d10 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095e582700 1 --2- 172.21.15.73:0/1741858065 >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 0x7f09581587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095dd81700 1 --2- 172.21.15.73:0/1741858065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f0948009f20 tx=0x7f0948009840).ready entity=mon.2 client_cookie=59b33c59a5b2e2a0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095cd7f700 1 -- 172.21.15.73:0/1741858065 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800e070 con 0x7f0958098d10 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095cd7f700 1 -- 172.21.15.73:0/1741858065 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094800b9d0 con 0x7f0958098d10 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.833+0000 7f095cd7f700 1 -- 172.21.15.73:0/1741858065 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094800bd50 con 0x7f0958098d10 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 -- 172.21.15.73:0/1741858065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 msgr2=0x7f09580c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 --2- 172.21.15.73:0/1741858065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 rx=0x7f0948009f20 tx=0x7f0948009840).stop 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 -- 172.21.15.73:0/1741858065 shutdown_connections 2023-11-28T04:26:48.835 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 --2- 172.21.15.73:0/1741858065 >> v2:172.21.15.150:3300/0 conn(0x7f0958158320 0x7f09581587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 --2- 172.21.15.73:0/1741858065 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f09580c7760 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 -- 172.21.15.73:0/1741858065 >> 172.21.15.73:0/1741858065 conn(0x7f09580cb310 msgr2=0x7f09580cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:48.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 -- 172.21.15.73:0/1741858065 shutdown_connections 2023-11-28T04:26:48.836 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.834+0000 7f095ffe5700 1 -- 172.21.15.73:0/1741858065 wait complete. 2023-11-28T04:26:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 Processor -- start 2023-11-28T04:26:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 -- start start 2023-11-28T04:26:48.838 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 0x7f09580c1240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0958158320 0x7f09580c1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580c1c20 con 0x7f0958158320 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09580b1250 con 0x7f09580c6190 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.835+0000 7f095ffe5700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09580b03f0 con 0x7f0958098d10 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095dd81700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095e582700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0958158320 0x7f09580c1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 0x7f09580c1240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095dd81700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:54976/0 (socket says 172.21.15.73:54976) 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095dd81700 1 -- 172.21.15.73:0/333989897 learned_addr learned my addr 172.21.15.73:0/333989897 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:48.839 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 -- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 msgr2=0x7f0958152210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 -- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0958158320 msgr2=0x7f09580c1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0958158320 0x7f09580c1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 -- 172.21.15.73:0/333989897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09500092a0 con 0x7f09580c6190 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095dd81700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095d580700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 0x7f09580c1240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f094c00c410 tx=0x7f094c007d30).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c00ce30 con 0x7f09580c6190 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f094c0072e0 con 0x7f09580c6190 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0948009ae0 con 0x7f09580c6190 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.836+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f094c0056e0 con 0x7f09580c6190 2023-11-28T04:26:48.840 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.837+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09580b06c0 con 0x7f09580c6190 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.837+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f095806c860 con 0x7f09580c6190 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.838+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f094c005ad0 con 0x7f09580c6190 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.838+0000 7f0955ffb700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 0x7f09440615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.838+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f094c0795f0 con 0x7f09580c6190 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.838+0000 7f095dd81700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 0x7f09440615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:48.841 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.838+0000 7f095dd81700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 0x7f09440615e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0948009880 tx=0x7f09580a8040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:48.842 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:48.842+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mon.1 v2:172.21.15.150:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f094c04aa40 con 0x7f09580c6190 2023-11-28T04:26:49.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.005+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0958158cc0 con 0x7f094405f120 2023-11-28T04:26:49.005 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.005+0000 7f0955ffb700 1 -- 172.21.15.73:0/333989897 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0958158cc0 con 0x7f094405f120 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 msgr2=0x7f09440615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 0x7f09440615e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 rx=0x7f0948009880 tx=0x7f09580a8040).stop 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 msgr2=0x7f09580c1240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 0x7f09580c1240 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 rx=0x7f094c00c410 tx=0x7f094c007d30).stop 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 shutdown_connections 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f094405f120 0x7f09440615e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.007 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f0958158320 0x7f09580c1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f0958098d10 0x7f0958152210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 --2- 172.21.15.73:0/333989897 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f09580c6190 0x7f09580c1240 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 >> 172.21.15.73:0/333989897 conn(0x7f09580cb310 msgr2=0x7f0958152f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 shutdown_connections 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.007+0000 7f095ffe5700 1 -- 172.21.15.73:0/333989897 wait complete. 2023-11-28T04:26:49.008 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 4 to deep-scrub 2023-11-28T04:26:49.018 INFO:tasks.ceph:Scrubbing osd.5 2023-11-28T04:26:49.018 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:49.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.135+0000 7fec6a376700 1 Processor -- start 2023-11-28T04:26:49.135 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.135+0000 7fec6a376700 1 -- start start 2023-11-28T04:26:49.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec6a376700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fec640aa680 0x7fec640c5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.137 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec6a376700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec641546b0 0x7fec640c64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec6a376700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec64158d50 con 0x7fec641542d0 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec6a376700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64158140 con 0x7fec641546b0 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec6a376700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec64158420 con 0x7fec640aa680 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fec641546b0 0x7fec640c64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69374700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fec641542d0 0x7fec640a9f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35892/0 (socket says 172.21.15.73:35892) 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69374700 1 -- 172.21.15.73:0/1233054487 learned_addr learned my addr 172.21.15.73:0/1233054487 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec68b73700 1 --2- 172.21.15.73:0/1233054487 >> v2:172.21.15.150:3300/0 conn(0x7fec640aa680 0x7fec640c5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 -- 172.21.15.73:0/1233054487 >> v2:172.21.15.150:3300/0 conn(0x7fec640aa680 msgr2=0x7fec640c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 --2- 172.21.15.73:0/1233054487 >> v2:172.21.15.150:3300/0 conn(0x7fec640aa680 0x7fec640c5fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 -- 172.21.15.73:0/1233054487 >> v1:172.21.15.73:6789/0 conn(0x7fec641542d0 legacy=0x7fec640a9f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec5bfff700 1 -- 172.21.15.73:0/1233054487 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 916092173 0 0) 0x7fec64158d50 con 0x7fec641542d0 2023-11-28T04:26:49.138 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 -- 172.21.15.73:0/1233054487 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec64158700 con 0x7fec641546b0 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec69b75700 1 --2- 172.21.15.73:0/1233054487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c64b0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 rx=0x7fec640add40 tx=0x7fec5c009780).ready entity=mon.0 client_cookie=3c31d441e605e21f server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec5bfff700 1 -- 172.21.15.73:0/1233054487 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00e070 con 0x7fec641546b0 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec5bfff700 1 -- 172.21.15.73:0/1233054487 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c00bb60 con 0x7fec641546b0 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.136+0000 7fec5bfff700 1 -- 172.21.15.73:0/1233054487 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c012410 con 0x7fec641546b0 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 -- 172.21.15.73:0/1233054487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 msgr2=0x7fec640c64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 --2- 172.21.15.73:0/1233054487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c64b0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 rx=0x7fec640add40 tx=0x7fec5c009780).stop 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 -- 172.21.15.73:0/1233054487 shutdown_connections 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 --2- 172.21.15.73:0/1233054487 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c64b0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 --2- 172.21.15.73:0/1233054487 >> v2:172.21.15.150:3300/0 conn(0x7fec640aa680 0x7fec640c5fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 -- 172.21.15.73:0/1233054487 >> 172.21.15.73:0/1233054487 conn(0x7fec640cb010 msgr2=0x7fec640cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.139 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 -- 172.21.15.73:0/1233054487 shutdown_connections 2023-11-28T04:26:49.140 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.137+0000 7fec6a376700 1 -- 172.21.15.73:0/1233054487 wait complete. 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.138+0000 7fec6a376700 1 Processor -- start 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.138+0000 7fec6a376700 1 -- start start 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec640aa680 0x7fec641571a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec641542d0 0x7fec640c3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.143 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64154ac0 con 0x7fec641546b0 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec64155640 con 0x7fec641542d0 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec6a376700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec64154d10 con 0x7fec640aa680 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec69b75700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec69b75700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c4060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40262/0 (socket says 172.21.15.73:40262) 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec69b75700 1 -- 172.21.15.73:0/3054700499 learned_addr learned my addr 172.21.15.73:0/3054700499 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec68b73700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec641542d0 0x7fec640c3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.139+0000 7fec69374700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec640aa680 0x7fec641571a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 -- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec640aa680 msgr2=0x7fec641571a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.144 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fec640aa680 0x7fec641571a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 -- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec641542d0 msgr2=0x7fec640c3b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fec641542d0 0x7fec640c3b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec5c009a20 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec69b75700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c4060 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7fec5c00c000 tx=0x7fec5c009780).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c00b6f0 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec5c01d630 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec64154ff0 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec5c01d9b0 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.140+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec64147e90 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.141+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fec6406c7c0 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.141+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fec5c026410 con 0x7fec641546b0 2023-11-28T04:26:49.145 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.141+0000 7fec597fa700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fec4805f150 0x7fec48061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fec5c07bef0 con 0x7fec641546b0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec69374700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fec4805f150 0x7fec48061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec597fa700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7fec48065a30 0x7fec48067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fec48068510 con 0x7fec48065a30 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7fec5c07e510 con 0x7fec641546b0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec68b73700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7fec48065a30 0x7fec48067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec69374700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fec4805f150 0x7fec48061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fec600011e0 tx=0x7fec60007bb0).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.142+0000 7fec68b73700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7fec48065a30 0x7fec48067e90 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.146 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.143+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== osd.5 v2:172.21.15.150:6800/90480 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fec48068510 con 0x7fec48065a30 2023-11-28T04:26:49.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.156+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 --> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fec641557d0 con 0x7fec48065a30 2023-11-28T04:26:49.156 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.156+0000 7fec597fa700 1 -- 172.21.15.73:0/3054700499 <== osd.5 v2:172.21.15.150:6800/90480 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fec641557d0 con 0x7fec48065a30 2023-11-28T04:26:49.156 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:49.156 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:49.157 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.156+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7fec48065a30 msgr2=0x7fec48067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.156+0000 7fec6a376700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.150:6800/90480,v1:172.21.15.150:6801/90480] conn(0x7fec48065a30 0x7fec48067e90 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fec4805f150 msgr2=0x7fec48061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fec4805f150 0x7fec48061610 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 rx=0x7fec600011e0 tx=0x7fec60007bb0).stop 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 msgr2=0x7fec640c4060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 --2- 172.21.15.73:0/3054700499 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fec641546b0 0x7fec640c4060 secure :-1 s=READY pgs=2189 cs=0 l=1 rev1=1 rx=0x7fec5c00c000 tx=0x7fec5c009780).stop 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec69b75700 1 -- 172.21.15.73:0/3054700499 reap_dead start 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 shutdown_connections 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 >> 172.21.15.73:0/3054700499 conn(0x7fec640cb010 msgr2=0x7fec6414fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 shutdown_connections 2023-11-28T04:26:49.158 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.157+0000 7fec6a376700 1 -- 172.21.15.73:0/3054700499 wait complete. 2023-11-28T04:26:49.168 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-11-28T04:26:49.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.281+0000 7f7d5505a700 1 Processor -- start 2023-11-28T04:26:49.281 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.281+0000 7f7d5505a700 1 -- start start 2023-11-28T04:26:49.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d5505a700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f7d50098d10 0x7f7d500c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d5505a700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d500c72d0 con 0x7f7d50158320 2023-11-28T04:26:49.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d5505a700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d50064570 con 0x7f7d500c6190 2023-11-28T04:26:49.287 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d5505a700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d50154bc0 con 0x7f7d50098d10 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d4f59e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f7d50158320 0x7f7d500a80f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35898/0 (socket says 172.21.15.73:35898) 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d4f59e700 1 -- 172.21.15.73:0/3391365759 learned_addr learned my addr 172.21.15.73:0/3391365759 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/3391365759 >> v2:172.21.15.150:3300/0 conn(0x7f7d50098d10 0x7f7d500c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1486765231 0 0) 0x7f7d500c72d0 con 0x7f7d50158320 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.282+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7d3c0033f0 con 0x7f7d50158320 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 540403427 0 0) 0x7f7d3c0033f0 con 0x7f7d50158320 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 >> v2:172.21.15.150:3300/0 conn(0x7f7d50098d10 msgr2=0x7f7d500c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 --2- 172.21.15.73:0/3391365759 >> v2:172.21.15.150:3300/0 conn(0x7f7d50098d10 0x7f7d500c7760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 >> v1:172.21.15.150:6789/0 conn(0x7f7d500c6190 legacy=0x7f7d50063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d500c66c0 con 0x7f7d50158320 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f7d480017f0 con 0x7f7d50158320 2023-11-28T04:26:49.288 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 >> v1:172.21.15.73:6789/0 conn(0x7f7d50158320 legacy=0x7f7d500a80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 0x7f7d3c003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3c004690 0x7f7d3c004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c005000 0x7f7d3c00b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d50154bc0 con 0x7f7d3c0038e0 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c00beb0 con 0x7f7d3c004690 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d3c00c070 con 0x7f7d3c005000 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 0x7f7d3c003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d47fff700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3c004690 0x7f7d3c004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4f59e700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c005000 0x7f7d3c00b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 -- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c005000 msgr2=0x7f7d3c00b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c005000 0x7f7d3c00b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 -- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3c004690 msgr2=0x7f7d3c004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.289 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d3c004690 0x7f7d3c004b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 -- 172.21.15.73:0/3391365759 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d3c0033f0 con 0x7f7d3c0038e0 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4f59e700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c005000 0x7f7d3c00b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4f59e700 1 -- 172.21.15.73:0/3391365759 reap_dead start 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 0x7f7d3c003d00 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f7d40009a60 tx=0x7f7d40009060).ready entity=mon.0 client_cookie=4aedcb6557d3510 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d40009ea0 con 0x7f7d3c0038e0 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d40009430 con 0x7f7d3c0038e0 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.283+0000 7f7d4e59c700 1 -- 172.21.15.73:0/3391365759 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4000e870 con 0x7f7d3c0038e0 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 -- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 msgr2=0x7f7d3c003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 0x7f7d3c003d00 secure :-1 s=READY pgs=2191 cs=0 l=1 rev1=1 rx=0x7f7d40009a60 tx=0x7f7d40009060).stop 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 -- 172.21.15.73:0/3391365759 shutdown_connections 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 --2- 172.21.15.73:0/3391365759 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d3c0038e0 0x7f7d3c003d00 unknown :-1 s=CLOSED pgs=2191 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.290 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 -- 172.21.15.73:0/3391365759 >> 172.21.15.73:0/3391365759 conn(0x7f7d500cb310 msgr2=0x7f7d500cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 -- 172.21.15.73:0/3391365759 shutdown_connections 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.284+0000 7f7d5505a700 1 -- 172.21.15.73:0/3391365759 wait complete. 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.285+0000 7f7d5505a700 1 Processor -- start 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.285+0000 7f7d5505a700 1 -- start start 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c0038e0 0x7f7d50150620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 0x7f7d500aad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d500ab550 con 0x7f7d500aca70 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d500aed20 con 0x7f7d501576f0 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d50157b70 con 0x7f7d3c0038e0 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40272/0 (socket says 172.21.15.73:40272) 2023-11-28T04:26:49.291 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 -- 172.21.15.73:0/1323311323 learned_addr learned my addr 172.21.15.73:0/1323311323 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d47fff700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 0x7f7d500aad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c0038e0 0x7f7d50150620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 -- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c0038e0 msgr2=0x7f7d50150620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c0038e0 0x7f7d50150620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 -- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 msgr2=0x7f7d500aad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 0x7f7d500aad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 -- 172.21.15.73:0/1323311323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d500ae3f0 con 0x7f7d500aca70 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d47fff700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 0x7f7d500aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d4f59e700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f7d48001350 tx=0x7f7d48007ec0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800e7d0 con 0x7f7d500aca70 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d4800e930 con 0x7f7d500aca70 2023-11-28T04:26:49.292 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d500ae6d0 con 0x7f7d500aca70 2023-11-28T04:26:49.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.286+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d4800ecf0 con 0x7f7d500aca70 2023-11-28T04:26:49.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.287+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d500ab260 con 0x7f7d500aca70 2023-11-28T04:26:49.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.287+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d5006c860 con 0x7f7d500aca70 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.288+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f7d48014640 con 0x7f7d500aca70 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.288+0000 7f7d46ffd700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 0x7f7d30061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.288+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 0x7f7d30061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.288+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f7d48079640 con 0x7f7d500aca70 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.289+0000 7f7d4ed9d700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 0x7f7d30061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7d40006d50 tx=0x7f7d4001a000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.294 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.292+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f7d4804aa30 con 0x7f7d500aca70 2023-11-28T04:26:49.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.457+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f7d500c69a0 con 0x7f7d3005f150 2023-11-28T04:26:49.457 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.457+0000 7f7d46ffd700 1 -- 172.21.15.73:0/1323311323 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f7d500c69a0 con 0x7f7d3005f150 2023-11-28T04:26:49.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 msgr2=0x7f7d30061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.459 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 0x7f7d30061610 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 rx=0x7f7d40006d50 tx=0x7f7d4001a000).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 msgr2=0x7f7d500acef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 secure :-1 s=READY pgs=2192 cs=0 l=1 rev1=1 rx=0x7f7d48001350 tx=0x7f7d48007ec0).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 shutdown_connections 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f7d3005f150 0x7f7d30061610 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f7d500aca70 0x7f7d500acef0 unknown :-1 s=CLOSED pgs=2192 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f7d3c0038e0 0x7f7d50150620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 --2- 172.21.15.73:0/1323311323 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f7d501576f0 0x7f7d500aad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 >> 172.21.15.73:0/1323311323 conn(0x7f7d500cb310 msgr2=0x7f7d500ca2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 shutdown_connections 2023-11-28T04:26:49.460 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.459+0000 7f7d5505a700 1 -- 172.21.15.73:0/1323311323 wait complete. 2023-11-28T04:26:49.461 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 5 to deep-scrub 2023-11-28T04:26:49.471 INFO:tasks.ceph:Scrubbing osd.6 2023-11-28T04:26:49.472 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:49.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.589+0000 7fb93184b700 1 Processor -- start 2023-11-28T04:26:49.590 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.589+0000 7fb93184b700 1 -- start start 2023-11-28T04:26:49.599 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb93184b700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb92c0c4d80 0x7fb92c156120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb93184b700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c063ae0 con 0x7fb92c0c71b0 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb93184b700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb92c0c6610 con 0x7fb92c156610 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb93184b700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb92c0c7590 con 0x7fb92c0c4d80 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb92affd700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fb92c0c71b0 0x7fb92c0bc660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35900/0 (socket says 172.21.15.73:35900) 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb9227fc700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fb92c0c4d80 0x7fb92c156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.590+0000 7fb92affd700 1 -- 172.21.15.73:0/3564328923 learned_addr learned my addr 172.21.15.73:0/3564328923 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:49.600 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb9227fc700 1 -- 172.21.15.73:0/3564328923 >> v1:172.21.15.150:6789/0 conn(0x7fb92c156610 legacy=0x7fb92c14fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb9227fc700 1 -- 172.21.15.73:0/3564328923 >> v1:172.21.15.73:6789/0 conn(0x7fb92c0c71b0 legacy=0x7fb92c0bc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb92a7fc700 1 -- 172.21.15.73:0/3564328923 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3205293499 0 0) 0x7fb92c0c6610 con 0x7fb92c156610 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb9227fc700 1 -- 172.21.15.73:0/3564328923 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb92c156a40 con 0x7fb92c0c4d80 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb9227fc700 1 --2- 172.21.15.73:0/3564328923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c0c4d80 0x7fb92c156120 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7fb914009760 tx=0x7fb914009060).ready entity=mon.0 client_cookie=c91567b814480be server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb92a7fc700 1 -- 172.21.15.73:0/3564328923 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91400b580 con 0x7fb92c0c4d80 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb92a7fc700 1 -- 172.21.15.73:0/3564328923 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91400b6e0 con 0x7fb92c0c4d80 2023-11-28T04:26:49.601 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.591+0000 7fb92a7fc700 1 -- 172.21.15.73:0/3564328923 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91400ba60 con 0x7fb92c0c4d80 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 -- 172.21.15.73:0/3564328923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c0c4d80 msgr2=0x7fb92c156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 --2- 172.21.15.73:0/3564328923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c0c4d80 0x7fb92c156120 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 rx=0x7fb914009760 tx=0x7fb914009060).stop 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 -- 172.21.15.73:0/3564328923 shutdown_connections 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 --2- 172.21.15.73:0/3564328923 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c0c4d80 0x7fb92c156120 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 -- 172.21.15.73:0/3564328923 >> 172.21.15.73:0/3564328923 conn(0x7fb92c0c8170 msgr2=0x7fb92c0c41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 -- 172.21.15.73:0/3564328923 shutdown_connections 2023-11-28T04:26:49.602 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.592+0000 7fb93184b700 1 -- 172.21.15.73:0/3564328923 wait complete. 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.593+0000 7fb93184b700 1 Processor -- start 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.593+0000 7fb93184b700 1 -- start start 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0c4d80 0x7fb92c146270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb92c0c71b0 0x7fb92c0c0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 0x7fb92c065f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0ab660 con 0x7fb92c156610 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0aaa50 con 0x7fb92c0c4d80 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92c0aad30 con 0x7fb92c0c71b0 2023-11-28T04:26:49.603 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 0x7fb92c065f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 0x7fb92c065f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40278/0 (socket says 172.21.15.73:40278) 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 -- 172.21.15.73:0/4162705814 learned_addr learned my addr 172.21.15.73:0/4162705814 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92affd700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0c4d80 0x7fb92c146270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb9227fc700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb92c0c71b0 0x7fb92c0c0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 -- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb92c0c71b0 msgr2=0x7fb92c0c0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fb92c0c71b0 0x7fb92c0c0e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 -- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0c4d80 msgr2=0x7fb92c146270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fb92c0c4d80 0x7fb92c146270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140092a0 con 0x7fb92c156610 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb92b7fe700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 0x7fb92c065f20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 rx=0x7fb924002230 tx=0x7fb924007f70).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924010510 con 0x7fb92c156610 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb924010670 con 0x7fb92c156610 2023-11-28T04:26:49.604 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb924010a10 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb92c0ab010 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.594+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb92c0aa690 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.595+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb92c06c860 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fb92401b400 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb90c05f150 0x7fb90c061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fb924079e50 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb92affd700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb90c05f150 0x7fb90c061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7fb90c065a30 0x7fb90c067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb90c068510 con 0x7fb90c065a30 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb92407c470 con 0x7fb92c156610 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb9227fc700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7fb90c065a30 0x7fb90c067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.596+0000 7fb92affd700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb90c05f150 0x7fb90c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb91c007fd0 tx=0x7fb91c000bc0).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.605 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.597+0000 7fb9227fc700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7fb90c065a30 0x7fb90c067e90 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.606 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.597+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== osd.6 v2:172.21.15.150:6824/90525 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fb90c068510 con 0x7fb90c065a30 2023-11-28T04:26:49.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.610+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 --> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb92c0631b0 con 0x7fb90c065a30 2023-11-28T04:26:49.611 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb9237fe700 1 -- 172.21.15.73:0/4162705814 <== osd.6 v2:172.21.15.150:6824/90525 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb92c0631b0 con 0x7fb90c065a30 2023-11-28T04:26:49.611 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:49.612 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:49.612 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:49.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7fb90c065a30 msgr2=0x7fb90c067e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.150:6824/90525,v1:172.21.15.150:6825/90525] conn(0x7fb90c065a30 0x7fb90c067e90 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb90c05f150 msgr2=0x7fb90c061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.613 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fb90c05f150 0x7fb90c061610 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 rx=0x7fb91c007fd0 tx=0x7fb91c000bc0).stop 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 msgr2=0x7fb92c065f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 --2- 172.21.15.73:0/4162705814 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fb92c156610 0x7fb92c065f20 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 rx=0x7fb924002230 tx=0x7fb924007f70).stop 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb92b7fe700 1 -- 172.21.15.73:0/4162705814 reap_dead start 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 shutdown_connections 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 >> 172.21.15.73:0/4162705814 conn(0x7fb92c0c8170 msgr2=0x7fb92c151ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 shutdown_connections 2023-11-28T04:26:49.614 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.611+0000 7fb93184b700 1 -- 172.21.15.73:0/4162705814 wait complete. 2023-11-28T04:26:49.623 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-11-28T04:26:49.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.736+0000 7f1e9ea5c700 1 Processor -- start 2023-11-28T04:26:49.737 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.736+0000 7f1e9ea5c700 1 -- start start 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e9ea5c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e98063b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e9ea5c700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7f1e98064000 0x7f1e980c6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e9ea5c700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e980644b0 con 0x7f1e980c1d40 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e9ea5c700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e980c18b0 con 0x7f1e98064000 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e9ea5c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c0ca0 con 0x7f1e980c7170 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e977fe700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f1e980c1d40 0x7f1e980c5c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35914/0 (socket says 172.21.15.73:35914) 2023-11-28T04:26:49.738 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.737+0000 7f1e977fe700 1 -- 172.21.15.73:0/3236786086 learned_addr learned my addr 172.21.15.73:0/3236786086 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:26:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e9cff9700 1 --2- 172.21.15.73:0/3236786086 >> v2:172.21.15.150:3300/0 conn(0x7f1e98064000 0x7f1e980c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540804034 0 0) 0x7f1e8c001110 con 0x7f1e980c1d40 2023-11-28T04:26:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e98063b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.743 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e800033f0 con 0x7f1e980c1d40 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622497297 0 0) 0x7f1e800033f0 con 0x7f1e980c1d40 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 msgr2=0x7f1e98063b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e98063b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 >> v2:172.21.15.150:3300/0 conn(0x7f1e98064000 msgr2=0x7f1e980c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 --2- 172.21.15.73:0/3236786086 >> v2:172.21.15.150:3300/0 conn(0x7f1e98064000 0x7f1e980c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980c0f80 con 0x7f1e980c1d40 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f1e8c001600 con 0x7f1e980c1d40 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 >> v1:172.21.15.73:6789/0 conn(0x7f1e980c1d40 legacy=0x7f1e980c5c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e800038e0 0x7f1e80003d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e80004690 0x7f1e80004b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.744 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e80005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c18b0 con 0x7f1e800038e0 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c0ca0 con 0x7f1e80004690 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e8000b8e0 con 0x7f1e980c7170 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e800038e0 0x7f1e80003d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e9cff9700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e80005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 -- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 msgr2=0x7f1e80005000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c7170 0x7f1e80005000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 -- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e80004690 msgr2=0x7f1e80004b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e80004690 0x7f1e80004b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).stop 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.738+0000 7f1e97fff700 1 -- 172.21.15.73:0/3236786086 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e800033f0 con 0x7f1e800038e0 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e97fff700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e800038e0 0x7f1e80003d00 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7f1e88009a60 tx=0x7f1e88009060).ready entity=mon.0 client_cookie=a808a6d0ffdfe4f7 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.745 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e88009ea0 con 0x7f1e800038e0 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e88009430 con 0x7f1e800038e0 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e96ffd700 1 -- 172.21.15.73:0/3236786086 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800e870 con 0x7f1e800038e0 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/3236786086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e800038e0 msgr2=0x7f1e80003d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/3236786086 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e800038e0 0x7f1e80003d00 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 rx=0x7f1e88009a60 tx=0x7f1e88009060).stop 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9cff9700 1 -- 172.21.15.73:0/3236786086 reap_dead start 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/3236786086 shutdown_connections 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/3236786086 >> 172.21.15.73:0/3236786086 conn(0x7f1e980cb260 msgr2=0x7f1e980cb670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.746 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/3236786086 shutdown_connections 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.739+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/3236786086 wait complete. 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.740+0000 7f1e9ea5c700 1 Processor -- start 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 -- start start 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e800038e0 0x7f1e980ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c4720 0x7f1e980b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980a8900 con 0x7f1e980afe70 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c4c80 con 0x7f1e800038e0 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e9ea5c700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e980c4f60 con 0x7f1e980c4720 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.741+0000 7f1e977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40300/0 (socket says 172.21.15.73:40300) 2023-11-28T04:26:49.747 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 -- 172.21.15.73:0/2264200879 learned_addr learned my addr 172.21.15.73:0/2264200879 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e97fff700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e800038e0 0x7f1e980ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e9cff9700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c4720 0x7f1e980b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 -- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c4720 msgr2=0x7f1e980b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c4720 0x7f1e980b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 -- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e800038e0 msgr2=0x7f1e980ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e800038e0 0x7f1e980ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 -- 172.21.15.73:0/2264200879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e980a82b0 con 0x7f1e980afe70 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e977fe700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 rx=0x7f1e8c0017b0 tx=0x7f1e8c001250).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c007170 con 0x7f1e980afe70 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8c0072d0 con 0x7f1e980afe70 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8c0048c0 con 0x7f1e980afe70 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e980a8590 con 0x7f1e980afe70 2023-11-28T04:26:49.748 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.742+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e980b02f0 con 0x7f1e980afe70 2023-11-28T04:26:49.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.743+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e980a8720 con 0x7f1e980afe70 2023-11-28T04:26:49.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.743+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f1e8c004c70 con 0x7f1e980afe70 2023-11-28T04:26:49.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.744+0000 7f1e7ffff700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 0x7f1e84061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:49.749 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.744+0000 7f1e97fff700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 0x7f1e84061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:49.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.744+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f1e8c079810 con 0x7f1e980afe70 2023-11-28T04:26:49.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.744+0000 7f1e97fff700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 0x7f1e84061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1e88009a60 tx=0x7f1e8801a000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:49.750 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.748+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f1e8c04ac60 con 0x7f1e980afe70 2023-11-28T04:26:49.912 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.911+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e980a8a40 con 0x7f1e8405f150 2023-11-28T04:26:49.913 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.912+0000 7f1e7ffff700 1 -- 172.21.15.73:0/2264200879 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1e980a8a40 con 0x7f1e8405f150 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 msgr2=0x7f1e84061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 0x7f1e84061610 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 rx=0x7f1e88009a60 tx=0x7f1e8801a000).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 msgr2=0x7f1e980c4230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 rx=0x7f1e8c0017b0 tx=0x7f1e8c001250).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 shutdown_connections 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f1e8405f150 0x7f1e84061610 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f1e980c4720 0x7f1e980b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f1e800038e0 0x7f1e980ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 --2- 172.21.15.73:0/2264200879 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f1e980afe70 0x7f1e980c4230 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.913+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 >> 172.21.15.73:0/2264200879 conn(0x7f1e980cb260 msgr2=0x7f1e980c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:49.915 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.914+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 shutdown_connections 2023-11-28T04:26:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:49.914+0000 7f1e9ea5c700 1 -- 172.21.15.73:0/2264200879 wait complete. 2023-11-28T04:26:49.916 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 6 to deep-scrub 2023-11-28T04:26:49.926 INFO:tasks.ceph:Scrubbing osd.7 2023-11-28T04:26:49.927 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-11-28T04:26:49.972 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:26:49.975 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:26:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.045+0000 7fa5a9dd3700 1 Processor -- start 2023-11-28T04:26:50.046 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.045+0000 7fa5a9dd3700 1 -- start start 2023-11-28T04:26:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a9dd3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a9dd3700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 0x7fa5a40aa460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a9dd3700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a4158d50 con 0x7fa5a41546b0 2023-11-28T04:26:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a9dd3700 1 -- --> v2:172.21.15.150:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40aa950 con 0x7fa5a40c66d0 2023-11-28T04:26:50.047 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a9dd3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4158140 con 0x7fa5a41542d0 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a8dd1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a95d2700 1 --2- >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 0x7fa5a40aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a8dd1700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.150:3301/0 says I am v2:172.21.15.73:55052/0 (socket says 172.21.15.73:55052) 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a3fff700 1 --1- >> v1:172.21.15.150:6789/0 conn(0x7fa5a41546b0 0x7fa5a40c5fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.150:6789/0 says I am v1:172.21.15.73:43544/0 (socket says 172.21.15.73:43544) 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.046+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/3611938335 learned_addr learned my addr 172.21.15.73:0/3611938335 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.049 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/3611938335 >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 msgr2=0x7fa5a40aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a8dd1700 1 --2- 172.21.15.73:0/3611938335 >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 0x7fa5a40aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/3611938335 >> v1:172.21.15.150:6789/0 conn(0x7fa5a41546b0 legacy=0x7fa5a40c5fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a37fe700 1 -- 172.21.15.73:0/3611938335 <== mon.1 v1:172.21.15.150:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 136931747 0 0) 0x7fa5a4158d50 con 0x7fa5a41546b0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/3611938335 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a4158420 con 0x7fa5a41542d0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a95d2700 1 --2- 172.21.15.73:0/3611938335 >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 0x7fa5a40aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_done state changed! 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a8dd1700 1 --2- 172.21.15.73:0/3611938335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fa598006fd0 tx=0x7fa5980090f0).ready entity=mon.2 client_cookie=d864c956202ce4f6 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a37fe700 1 -- 172.21.15.73:0/3611938335 <== mon.2 v2:172.21.15.150:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59800b580 con 0x7fa5a41542d0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a37fe700 1 -- 172.21.15.73:0/3611938335 <== mon.2 v2:172.21.15.150:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800b6e0 con 0x7fa5a41542d0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.047+0000 7fa5a37fe700 1 -- 172.21.15.73:0/3611938335 <== mon.2 v2:172.21.15.150:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa59800ba60 con 0x7fa5a41542d0 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/3611938335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 msgr2=0x7fa5a40a9f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.050 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/3611938335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 rx=0x7fa598006fd0 tx=0x7fa5980090f0).stop 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/3611938335 shutdown_connections 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/3611938335 >> v2:172.21.15.150:3300/0 conn(0x7fa5a40c66d0 0x7fa5a40aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/3611938335 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41542d0 0x7fa5a40a9f70 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/3611938335 >> 172.21.15.73:0/3611938335 conn(0x7fa5a40cb010 msgr2=0x7fa5a40cb420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/3611938335 shutdown_connections 2023-11-28T04:26:50.051 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.048+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/3611938335 wait complete. 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.049+0000 7fa5a9dd3700 1 Processor -- start 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.049+0000 7fa5a9dd3700 1 -- start start 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 0x7fa5a40ace00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a41542d0 0x7fa5a4144930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41546b0 0x7fa5a4144e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4145310 con 0x7fa5a40c66d0 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4168160 con 0x7fa5a41542d0 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a9dd3700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4168440 con 0x7fa5a41546b0 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 0x7fa5a40ace00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.054 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 0x7fa5a40ace00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40302/0 (socket says 172.21.15.73:40302) 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/2741438343 learned_addr learned my addr 172.21.15.73:0/2741438343 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a95d2700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41546b0 0x7fa5a4144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a3fff700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a41542d0 0x7fa5a4144930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41546b0 msgr2=0x7fa5a4144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fa5a41546b0 0x7fa5a4144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a41542d0 msgr2=0x7fa5a4144930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fa5a41542d0 0x7fa5a4144930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c009a20 con 0x7fa5a40c66d0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a8dd1700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 0x7fa5a40ace00 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 rx=0x7fa598009130 tx=0x7fa598017000).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.050+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598000bc0 con 0x7fa5a40c66d0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.051+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa59800b850 con 0x7fa5a40c66d0 2023-11-28T04:26:50.055 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.051+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa598009370 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.051+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa598000d20 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.051+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a4168a00 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.051+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa5a406c7c0 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fa59801c8e0 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa58805f120 0x7fa5880615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fa59807bd60 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a3fff700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa58805f120 0x7fa5880615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7fa588065a00 0x7fa588067e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa5880684e0 con 0x7fa588065a00 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.052+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_get_version_reply(handle=1 version=217) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa59807e590 con 0x7fa5a40c66d0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.053+0000 7fa5a95d2700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7fa588065a00 0x7fa588067e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.053+0000 7fa5a3fff700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa58805f120 0x7fa5880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fa5a40b1100 tx=0x7fa590007d50).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.056 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.053+0000 7fa5a95d2700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7fa588065a00 0x7fa588067e60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.057 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.054+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== osd.7 v2:172.21.15.150:6809/90481 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23213 (crc 0 0 0) 0x7fa5880684e0 con 0x7fa588065a00 2023-11-28T04:26:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.067+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 --> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fa5a40c86d0 con 0x7fa588065a00 2023-11-28T04:26:50.068 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.067+0000 7fa5a0ff9700 1 -- 172.21.15.73:0/2741438343 <== osd.7 v2:172.21.15.150:6809/90481 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fa5a40c86d0 con 0x7fa588065a00 2023-11-28T04:26:50.068 INFO:teuthology.orchestra.run.smithi073.stdout:{ 2023-11-28T04:26:50.068 INFO:teuthology.orchestra.run.smithi073.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-11-28T04:26:50.069 INFO:teuthology.orchestra.run.smithi073.stdout:} 2023-11-28T04:26:50.069 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7fa588065a00 msgr2=0x7fa588067e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.150:6809/90481,v1:172.21.15.150:6811/90481] conn(0x7fa588065a00 0x7fa588067e60 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa58805f120 msgr2=0x7fa5880615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fa58805f120 0x7fa5880615e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 rx=0x7fa5a40b1100 tx=0x7fa590007d50).stop 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 msgr2=0x7fa5a40ace00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 --2- 172.21.15.73:0/2741438343 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fa5a40c66d0 0x7fa5a40ace00 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 rx=0x7fa598009130 tx=0x7fa598017000).stop 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a95d2700 1 -- 172.21.15.73:0/2741438343 reap_dead start 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 shutdown_connections 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 >> 172.21.15.73:0/2741438343 conn(0x7fa5a40cb010 msgr2=0x7fa5a40c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 shutdown_connections 2023-11-28T04:26:50.070 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.068+0000 7fa5a9dd3700 1 -- 172.21.15.73:0/2741438343 wait complete. 2023-11-28T04:26:50.080 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-11-28T04:26:50.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.194+0000 7fcdb445e700 1 Processor -- start 2023-11-28T04:26:50.195 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.194+0000 7fcdb445e700 1 -- start start 2023-11-28T04:26:50.197 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb445e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb445e700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdac0c6190 0x7fcdac063e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb445e700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac0c72d0 con 0x7fcdac064350 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb445e700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac154bc0 con 0x7fcdac0c6190 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb445e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0c66c0 con 0x7fcdac098d10 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdac0c6190 0x7fcdac063e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fcdac0c6190 0x7fcdac063e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40318/0 (socket says 172.21.15.73:40318) 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 -- 172.21.15.73:0/3468364147 learned_addr learned my addr 172.21.15.73:0/3468364147 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb21fa700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 -- 172.21.15.73:0/3468364147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 msgr2=0x7fcdac0c7760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 -- 172.21.15.73:0/3468364147 >> v1:172.21.15.150:6789/0 conn(0x7fcdac064350 legacy=0x7fcdac158320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:50.198 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.195+0000 7fcdb19f9700 1 -- 172.21.15.73:0/3468364147 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac0c69a0 con 0x7fcdac0c6190 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb19f9700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac0c6190 0x7fcdac063e60 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fcda0006fd0 tx=0x7fcda00090f0).ready entity=mon.0 client_cookie=5f939d8d2a4c5c0a server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb11f8700 1 -- 172.21.15.73:0/3468364147 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000b600 con 0x7fcdac0c6190 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb11f8700 1 -- 172.21.15.73:0/3468364147 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda000b760 con 0x7fcdac0c6190 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb11f8700 1 -- 172.21.15.73:0/3468364147 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda000bae0 con 0x7fcdac0c6190 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 -- 172.21.15.73:0/3468364147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac0c6190 msgr2=0x7fcdac063e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac0c6190 0x7fcdac063e60 secure :-1 s=READY pgs=2200 cs=0 l=1 rev1=1 rx=0x7fcda0006fd0 tx=0x7fcda00090f0).stop 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 -- 172.21.15.73:0/3468364147 shutdown_connections 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0c7760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 --2- 172.21.15.73:0/3468364147 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac0c6190 0x7fcdac063e60 unknown :-1 s=CLOSED pgs=2200 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 -- 172.21.15.73:0/3468364147 >> 172.21.15.73:0/3468364147 conn(0x7fcdac0cb310 msgr2=0x7fcdac0cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 -- 172.21.15.73:0/3468364147 shutdown_connections 2023-11-28T04:26:50.199 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.196+0000 7fcdb445e700 1 -- 172.21.15.73:0/3468364147 wait complete. 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.197+0000 7fcdb445e700 1 Processor -- start 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.197+0000 7fcdb445e700 1 -- start start 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0a99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 0x7fcdac0ad430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0a9530 con 0x7fcdac064350 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac168d40 con 0x7fcdac0c6190 2023-11-28T04:26:50.202 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb445e700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0a9eb0 con 0x7fcdac098d10 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb21fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb21fa700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40326/0 (socket says 172.21.15.73:40326) 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.198+0000 7fcdb21fa700 1 -- 172.21.15.73:0/1742342849 learned_addr learned my addr 172.21.15.73:0/1742342849 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 -- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 msgr2=0x7fcdac0a99c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb19f9700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0a99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb29fb700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 0x7fcdac0ad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0a99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 -- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 msgr2=0x7fcdac0ad430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 0x7fcdac0ad430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 -- 172.21.15.73:0/1742342849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda0009370 con 0x7fcdac064350 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb29fb700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 0x7fcdac0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).send_auth_request state changed! 2023-11-28T04:26:50.203 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb21fa700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 rx=0x7fcd9c00cfc0 tx=0x7fcd9c007890).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c016400 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c016560 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac0a8920 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0168e0 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.199+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac0a8ee0 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.200+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac0a9810 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.200+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fcd9c009030 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.201+0000 7fcdaa7fc700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 0x7fcd98061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.201+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fcd9c07aa00 con 0x7fcdac064350 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.201+0000 7fcdb19f9700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 0x7fcd98061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.204 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.201+0000 7fcdb19f9700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 0x7fcd98061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fcda0006fd0 tx=0x7fcda0017000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.206 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.205+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fcd9c04bdf0 con 0x7fcdac064350 2023-11-28T04:26:50.367 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.366+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdac06c860 con 0x7fcd9805f150 2023-11-28T04:26:50.368 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.367+0000 7fcdaa7fc700 1 -- 172.21.15.73:0/1742342849 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcdac06c860 con 0x7fcd9805f150 2023-11-28T04:26:50.369 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 msgr2=0x7fcd98061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 0x7fcd98061610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 rx=0x7fcda0006fd0 tx=0x7fcda0017000).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 msgr2=0x7fcdac168850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 rx=0x7fcd9c00cfc0 tx=0x7fcd9c007890).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 shutdown_connections 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fcd9805f150 0x7fcd98061610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fcdac0c6190 0x7fcdac0ad430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fcdac064350 0x7fcdac168850 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 --2- 172.21.15.73:0/1742342849 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fcdac098d10 0x7fcdac0a99c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 >> 172.21.15.73:0/1742342849 conn(0x7fcdac0cb310 msgr2=0x7fcdac152510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 shutdown_connections 2023-11-28T04:26:50.370 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.368+0000 7fcdb445e700 1 -- 172.21.15.73:0/1742342849 wait complete. 2023-11-28T04:26:50.371 INFO:teuthology.orchestra.run.smithi073.stderr:instructed osd(s) 7 to deep-scrub 2023-11-28T04:26:50.381 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:26:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.496+0000 7f8fb7a16700 1 Processor -- start 2023-11-28T04:26:50.497 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.496+0000 7f8fb7a16700 1 -- start start 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb7a16700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb00caab0 0x7f8fb00aa470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb7a16700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8fb00cae90 0x7f8fb00c0ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb7a16700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fb00a9a90 con 0x7f8fb00aa960 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb7a16700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fb0154ce0 con 0x7f8fb00cae90 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb7a16700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb00a8e80 con 0x7f8fb00caab0 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8fb00cae90 0x7f8fb00c0ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7f8fb00cae90 0x7f8fb00c0ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40332/0 (socket says 172.21.15.73:40332) 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 -- 172.21.15.73:0/3263858051 learned_addr learned my addr 172.21.15.73:0/3263858051 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.500 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb5fb3700 1 --1- 172.21.15.73:0/3263858051 >> v1:172.21.15.73:6789/0 conn(0x7f8fb00aa960 0x7f8fb00c9170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:35922/0 (socket says 172.21.15.73:35922) 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb57b2700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb00caab0 0x7f8fb00aa470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 -- 172.21.15.73:0/3263858051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb00caab0 msgr2=0x7f8fb00aa470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb00caab0 0x7f8fb00aa470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 -- 172.21.15.73:0/3263858051 >> v1:172.21.15.73:6789/0 conn(0x7f8fb00aa960 legacy=0x7f8fb00c9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 -- 172.21.15.73:0/3263858051 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fb00a9160 con 0x7f8fb00cae90 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fa7fff700 1 -- 172.21.15.73:0/3263858051 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1657427270 0 0) 0x7f8fb00a9a90 con 0x7f8fb00aa960 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.497+0000 7f8fb4fb1700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00cae90 0x7f8fb00c0ab0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 rx=0x7f8fa0006fd0 tx=0x7f8fa00090f0).ready entity=mon.0 client_cookie=9e3c7a9ab03b2fa9 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fa7fff700 1 -- 172.21.15.73:0/3263858051 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000b600 con 0x7f8fb00cae90 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fa7fff700 1 -- 172.21.15.73:0/3263858051 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fa000b760 con 0x7f8fb00cae90 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fa7fff700 1 -- 172.21.15.73:0/3263858051 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fa000bae0 con 0x7f8fb00cae90 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 -- 172.21.15.73:0/3263858051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00cae90 msgr2=0x7f8fb00c0ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.501 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00cae90 0x7f8fb00c0ab0 secure :-1 s=READY pgs=2203 cs=0 l=1 rev1=1 rx=0x7f8fa0006fd0 tx=0x7f8fa00090f0).stop 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 -- 172.21.15.73:0/3263858051 shutdown_connections 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb00caab0 0x7f8fb00aa470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/3263858051 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00cae90 0x7f8fb00c0ab0 unknown :-1 s=CLOSED pgs=2203 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 -- 172.21.15.73:0/3263858051 >> 172.21.15.73:0/3263858051 conn(0x7f8fb0094920 msgr2=0x7f8fb006a610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 -- 172.21.15.73:0/3263858051 shutdown_connections 2023-11-28T04:26:50.502 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.498+0000 7f8fb7a16700 1 -- 172.21.15.73:0/3263858051 wait complete. 2023-11-28T04:26:50.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.499+0000 7f8fb7a16700 1 Processor -- start 2023-11-28T04:26:50.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.499+0000 7f8fb7a16700 1 -- start start 2023-11-28T04:26:50.505 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb00cae90 0x7f8fb0156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 0x7f8fb0154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0157840 con 0x7f8fb00caab0 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0155c60 con 0x7f8fb00cae90 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb7a16700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fb0155f40 con 0x7f8fb01573c0 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb57b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb57b2700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:40348/0 (socket says 172.21.15.73:40348) 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.500+0000 7f8fb57b2700 1 -- 172.21.15.73:0/311875518 learned_addr learned my addr 172.21.15.73:0/311875518 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb5fb3700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 0x7f8fb0154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb4fb1700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb00cae90 0x7f8fb0156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 -- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 msgr2=0x7f8fb0154ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.506 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 0x7f8fb0154ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 -- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb00cae90 msgr2=0x7f8fb0156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb00cae90 0x7f8fb0156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 -- 172.21.15.73:0/311875518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fa0009370 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb5fb3700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 0x7f8fb0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb57b2700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 rx=0x7f8fac00c880 tx=0x7f8fac008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac010580 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fac0106e0 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fb0156220 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fac010a60 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.501+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fb01527c0 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.502+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fb006c860 con 0x7f8fb00caab0 2023-11-28T04:26:50.507 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.502+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f8fac01b460 con 0x7f8fb00caab0 2023-11-28T04:26:50.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.503+0000 7f8fa57fa700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 0x7f8f940615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:26:50.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.503+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f8fac079f10 con 0x7f8fb00caab0 2023-11-28T04:26:50.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.503+0000 7f8fb4fb1700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 0x7f8f940615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:26:50.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.503+0000 7f8fb4fb1700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 0x7f8f940615e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8fb00c6040 tx=0x7f8fa0017000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:26:50.508 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.507+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f8fac04b300 con 0x7f8fb00caab0 2023-11-28T04:26:50.668 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.667+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8fb00ae220 con 0x7f8f9405f120 2023-11-28T04:26:50.670 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.669+0000 7f8fa57fa700 1 -- 172.21.15.73:0/311875518 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71514 (secure 0 0 0) 0x7f8fb00ae220 con 0x7f8f9405f120 2023-11-28T04:26:50.671 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 msgr2=0x7f8f940615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 0x7f8f940615e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 rx=0x7f8fb00c6040 tx=0x7f8fa0017000).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 msgr2=0x7f8fb01522d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 rx=0x7f8fac00c880 tx=0x7f8fac008040).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 shutdown_connections 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.671+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f8f9405f120 0x7f8f940615e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f8fb01573c0 0x7f8fb0154ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f8fb00caab0 0x7f8fb01522d0 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 --2- 172.21.15.73:0/311875518 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f8fb00cae90 0x7f8fb0156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 >> 172.21.15.73:0/311875518 conn(0x7f8fb0094920 msgr2=0x7f8fb00aae70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 shutdown_connections 2023-11-28T04:26:50.673 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:26:50.672+0000 7f8fb7a16700 1 -- 172.21.15.73:0/311875518 wait complete. 2023-11-28T04:26:50.674 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:26:50.684 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2023-11-28T04:26:50.455847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2459488,"kb_used_data":21280,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752515232,"statfs":{"total":773094113280,"available":770575597568,"internally_reserved":0,"allocated":21790720,"data_stored":11009976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11959,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994292"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:49.449118+0000","last_change":"2023-11-28T04:26:49.449118+0000","last_active":"2023-11-28T04:26:49.449118+0000","last_peered":"2023-11-28T04:26:49.449118+0000","last_clean":"2023-11-28T04:26:49.449118+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:49.449118+0000","last_undegraded":"2023-11-28T04:26:49.449118+0000","last_fullsized":"2023-11-28T04:26:49.449118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:49.449013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:49.449013+0000","last_clean_scrub_stamp":"2023-11-28T04:26:49.449013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":173,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-11-28T04:26:40.459497+0000","last_change":"2023-11-28T02:52:39.446566+0000","last_active":"2023-11-28T04:26:40.459497+0000","last_peered":"2023-11-28T04:26:40.459497+0000","last_clean":"2023-11-28T04:26:40.459497+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:40.459497+0000","last_undegraded":"2023-11-28T04:26:40.459497+0000","last_fullsized":"2023-11-28T04:26:40.459497+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":188,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:48.903929+0000","last_change":"2023-11-28T04:26:48.903929+0000","last_active":"2023-11-28T04:26:48.903929+0000","last_peered":"2023-11-28T04:26:48.903929+0000","last_clean":"2023-11-28T04:26:48.903929+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:48.903929+0000","last_undegraded":"2023-11-28T04:26:48.903929+0000","last_fullsized":"2023-11-28T04:26:48.903929+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:48.903841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:48.903841+0000","last_clean_scrub_stamp":"2023-11-28T04:26:48.903841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":173,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-11-28T04:26:40.458287+0000","last_change":"2023-11-28T02:52:39.446881+0000","last_active":"2023-11-28T04:26:40.458287+0000","last_peered":"2023-11-28T04:26:40.458287+0000","last_clean":"2023-11-28T04:26:40.458287+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:40.458287+0000","last_undegraded":"2023-11-28T04:26:40.458287+0000","last_fullsized":"2023-11-28T04:26:40.458287+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":171,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-11-28T04:26:39.638586+0000","last_change":"2023-11-28T02:52:38.027627+0000","last_active":"2023-11-28T04:26:39.638586+0000","last_peered":"2023-11-28T04:26:39.638586+0000","last_clean":"2023-11-28T04:26:39.638586+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:39.638586+0000","last_undegraded":"2023-11-28T04:26:39.638586+0000","last_fullsized":"2023-11-28T04:26:39.638586+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:36.415279+0000","last_clean_scrub_stamp":"2023-11-28T02:52:36.415279+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":169,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-11-28T04:26:39.638688+0000","last_change":"2023-11-28T02:52:39.445356+0000","last_active":"2023-11-28T04:26:39.638688+0000","last_peered":"2023-11-28T04:26:39.638688+0000","last_clean":"2023-11-28T04:26:39.638688+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:39.638688+0000","last_undegraded":"2023-11-28T04:26:39.638688+0000","last_fullsized":"2023-11-28T04:26:39.638688+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":170,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-11-28T04:26:39.638994+0000","last_change":"2023-11-28T02:52:40.038773+0000","last_active":"2023-11-28T04:26:39.638994+0000","last_peered":"2023-11-28T04:26:39.638994+0000","last_clean":"2023-11-28T04:26:39.638994+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:39.638994+0000","last_undegraded":"2023-11-28T04:26:39.638994+0000","last_fullsized":"2023-11-28T04:26:39.638994+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":175,"reported_epoch":215,"state":"active+clean","last_fresh":"2023-11-28T04:26:38.488500+0000","last_change":"2023-11-28T02:52:39.447010+0000","last_active":"2023-11-28T04:26:38.488500+0000","last_peered":"2023-11-28T04:26:38.488500+0000","last_clean":"2023-11-28T04:26:38.488500+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:38.488500+0000","last_undegraded":"2023-11-28T04:26:38.488500+0000","last_fullsized":"2023-11-28T04:26:38.488500+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":193,"reported_epoch":216,"state":"active+clean","last_fresh":"2023-11-28T04:26:40.458392+0000","last_change":"2023-11-28T02:52:39.448365+0000","last_active":"2023-11-28T04:26:40.458392+0000","last_peered":"2023-11-28T04:26:40.458392+0000","last_clean":"2023-11-28T04:26:40.458392+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:40.458392+0000","last_undegraded":"2023-11-28T04:26:40.458392+0000","last_fullsized":"2023-11-28T04:26:40.458392+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T02:52:38.423909+0000","last_clean_scrub_stamp":"2023-11-28T02:52:38.423909+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.61699999999999999},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.872},"last":0.52100000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.78400000000000003,"15min":0.92800000000000005},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.82599999999999996},"last":0.46200000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.89000000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66000000000000003,"5min":0.80600000000000005,"15min":0.84099999999999997},"last":0.57799999999999996}]},{"osd":3,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65000000000000002,"5min":0.73299999999999998,"15min":0.83599999999999997},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.90300000000000002},"last":0.53100000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.498},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.54400000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.60099999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.58899999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.77600000000000002},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.54500000000000004}]},{"osd":1,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68000000000000005,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.34899999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.45700000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.58399999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34899999999999998},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.47899999999999998}]},{"osd":3,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.82599999999999996},"last":0.625},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.58999999999999997}]},{"osd":4,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.33200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.5}]},{"osd":5,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.504},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.56000000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.53000000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575969,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.376,"15min":0.372},"min":{"1min":0.32800000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.52000000000000002,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.39600000000000002,"15min":0.39300000000000002},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.23499999999999999},"max":{"1min":0.501,"5min":0.626,"15min":0.67000000000000004},"last":0.379}]},{"osd":2,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.41299999999999998,"15min":0.41099999999999998},"min":{"1min":0.28799999999999998,"5min":0.27300000000000002,"15min":0.251},"max":{"1min":0.53200000000000003,"5min":0.623,"15min":0.81200000000000006},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.45000000000000001,"5min":0.44800000000000001,"15min":0.44},"min":{"1min":0.36299999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.57799999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439}]},{"osd":3,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.48299999999999998,"15min":0.48299999999999998},"min":{"1min":0.36599999999999999,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.626,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.48099999999999998,"5min":0.45900000000000002,"15min":0.45300000000000001},"min":{"1min":0.35999999999999999,"5min":0.29099999999999998,"15min":0.28999999999999998},"max":{"1min":0.71299999999999997,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.44800000000000001}]},{"osd":4,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.62,"15min":0.61499999999999999},"min":{"1min":0.435,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.81299999999999994,"5min":0.93100000000000005,"15min":0.93100000000000005},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.63800000000000001,"15min":0.63},"min":{"1min":0.50800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.86399999999999999,"5min":0.86499999999999999,"15min":0.98599999999999999},"last":0.54800000000000004}]},{"osd":5,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.56200000000000006,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":1.0029999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.68600000000000005,"15min":0.68300000000000005},"min":{"1min":0.55500000000000005,"5min":0.48199999999999998,"15min":0.39100000000000001},"max":{"1min":0.876,"5min":0.94599999999999995,"15min":1.0349999999999999},"last":0.64800000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.49199999999999999,"5min":0.44400000000000001,"15min":0.38200000000000001},"max":{"1min":0.82299999999999995,"5min":0.92800000000000005,"15min":0.95299999999999996},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.66800000000000004,"15min":0.66600000000000004},"min":{"1min":0.51800000000000002,"5min":0.48499999999999999,"15min":0.46500000000000002},"max":{"1min":0.86799999999999999,"5min":0.91300000000000003,"15min":0.999},"last":0.51300000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:25:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.66400000000000003,"15min":0.65500000000000003},"min":{"1min":0.53600000000000003,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.877,"5min":0.91600000000000004,"15min":1.02},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.66500000000000004,"15min":0.65800000000000003},"min":{"1min":0.53700000000000003,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.82599999999999996,"5min":0.86699999999999999,"15min":0.92000000000000004},"last":0.66000000000000003}]}]},{"osd":0,"up_from":13,"seq":55834575970,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.27500000000000002},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.38100000000000001}]},{"osd":2,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.318}]},{"osd":3,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.29499999999999998,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.35499999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.58999999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.54400000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40300000000000002},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.63500000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.377},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.496,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.443},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.307}]},{"osd":1,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.622},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.754,"15min":0.754},"last":0.34100000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65300000000000002},"last":0.29099999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.84599999999999997,"15min":0.94399999999999995},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73899999999999999,"5min":0.876,"15min":0.96199999999999997},"last":0.39500000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":1.016},"last":0.47699999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.99099999999999999},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.44600000000000001}]},{"osd":7,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.48199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575970,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22500000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.379,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.66500000000000004},"last":0.309}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.33300000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.746},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.68000000000000005},"last":0.35799999999999998}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.92500000000000004},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.52400000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0409999999999999},"last":0.622},{"interface":"front","average":{"1min":0.625,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.57299999999999995}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.627,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.028,"15min":1.028},"last":0.56100000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.55600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.56599999999999995,"15min":0.56899999999999995},"min":{"1min":0.36499999999999999,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.72999999999999998,"5min":0.749,"15min":0.80600000000000005},"last":0.437},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38500000000000001,"5min":0.35899999999999999,"15min":0.316},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.45600000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.57099999999999995,"15min":0.57599999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.74099999999999999,"5min":0.75800000000000001,"15min":0.78800000000000003},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.56499999999999995,"15min":0.57799999999999996},"min":{"1min":0.498,"5min":0.38700000000000001,"15min":0.37},"max":{"1min":0.78800000000000003,"5min":0.81399999999999995,"15min":0.83699999999999997},"last":0.496}]},{"osd":2,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.55700000000000005,"15min":0.57099999999999995},"min":{"1min":0.46000000000000002,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.71899999999999997,"5min":0.747,"15min":0.83299999999999996},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.57099999999999995,"15min":0.57999999999999996},"min":{"1min":0.496,"5min":0.40300000000000002,"15min":0.377},"max":{"1min":0.76700000000000002,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.51200000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.53000000000000003,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.83999999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.629,"15min":0.64100000000000001},"min":{"1min":0.52200000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.872},"last":0.52700000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46999999999999997,"15min":0.48299999999999998},"min":{"1min":0.35899999999999999,"5min":0.31,"15min":0.308},"max":{"1min":0.65300000000000002,"5min":0.65500000000000003,"15min":0.78300000000000003},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.46999999999999997,"5min":0.502,"15min":0.51400000000000001},"min":{"1min":0.39100000000000001,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.68799999999999994,"5min":0.78500000000000003,"15min":0.83499999999999996},"last":0.47199999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.48899999999999999,"15min":0.50700000000000001},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.64600000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.52200000000000002,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.72999999999999998,"5min":0.75800000000000001,"15min":0.85099999999999998},"last":0.54500000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:26:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54900000000000004,"15min":0.56399999999999995},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.74399999999999999,"5min":0.82499999999999996,"15min":0.85999999999999999},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.56599999999999995,"15min":0.58199999999999996},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.34200000000000003},"max":{"1min":0.753,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.56100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.75900000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.55600000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.501}]},{"osd":1,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.80800000000000005},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":0.76700000000000002,"15min":0.79900000000000004},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40400000000000003},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.86399999999999999},"last":0.65600000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.59199999999999997},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.83599999999999997},"last":0.66900000000000004}]},{"osd":4,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.88200000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.318},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.47899999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.86699999999999999},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.92100000000000004},"last":0.46300000000000002}]},{"osd":7,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.90300000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85799999999999998},"last":0.56799999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:26:50.688 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.688 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.688 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-11-28T02:52:36.415279+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=36, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.689 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.689 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.689 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.689 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-11-28T02:52:38.423909+0000 time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=2, tm_min=52, tm_sec=38, tm_wday=1, tm_yday=332, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=11, tm_mday=28, tm_hour=4, tm_min=26, tm_sec=45, tm_wday=1, tm_yday=332, tm_isdst=0) 2023-11-28T04:26:50.689 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-11-28T04:27:10.690 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-11-28T04:27:10.815 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.814+0000 7fdbb9043700 1 Processor -- start 2023-11-28T04:27:10.816 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.814+0000 7fdbb9043700 1 -- start start 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb9043700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbb40c45d0 0x7fdbb40c2640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb9043700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb40a7490 con 0x7fdbb4068090 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb9043700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdbb4154d20 con 0x7fdbb4067cb0 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb9043700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbb4151670 con 0x7fdbb40c45d0 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb259c700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fdbb40c45d0 0x7fdbb40c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb359e700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7fdbb4068090 0x7fdbb40c2b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34802/0 (socket says 172.21.15.73:34802) 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.815+0000 7fdbb359e700 1 -- 172.21.15.73:0/1636651563 learned_addr learned my addr 172.21.15.73:0/1636651563 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb259c700 1 -- 172.21.15.73:0/1636651563 >> v1:172.21.15.150:6789/0 conn(0x7fdbb4067cb0 legacy=0x7fdbb40c3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb259c700 1 -- 172.21.15.73:0/1636651563 >> v1:172.21.15.73:6789/0 conn(0x7fdbb4068090 legacy=0x7fdbb40c2b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.818 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb1d9b700 1 -- 172.21.15.73:0/1636651563 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3698770208 0 0) 0x7fdbac001490 con 0x7fdbb4068090 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb259c700 1 -- 172.21.15.73:0/1636651563 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbb41555c0 con 0x7fdbb40c45d0 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb259c700 1 --2- 172.21.15.73:0/1636651563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb40c45d0 0x7fdbb40c2640 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 rx=0x7fdba80096e0 tx=0x7fdba8009060).ready entity=mon.0 client_cookie=14e9bd3727642e00 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb1d9b700 1 -- 172.21.15.73:0/1636651563 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800f510 con 0x7fdbb40c45d0 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb1d9b700 1 -- 172.21.15.73:0/1636651563 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba800f670 con 0x7fdbb40c45d0 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.816+0000 7fdbb1d9b700 1 -- 172.21.15.73:0/1636651563 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba800f9f0 con 0x7fdbb40c45d0 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 -- 172.21.15.73:0/1636651563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb40c45d0 msgr2=0x7fdbb40c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 --2- 172.21.15.73:0/1636651563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb40c45d0 0x7fdbb40c2640 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 rx=0x7fdba80096e0 tx=0x7fdba8009060).stop 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 -- 172.21.15.73:0/1636651563 shutdown_connections 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 --2- 172.21.15.73:0/1636651563 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb40c45d0 0x7fdbb40c2640 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 -- 172.21.15.73:0/1636651563 >> 172.21.15.73:0/1636651563 conn(0x7fdbb40c7fd0 msgr2=0x7fdbb40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 -- 172.21.15.73:0/1636651563 shutdown_connections 2023-11-28T04:27:10.819 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.817+0000 7fdbb9043700 1 -- 172.21.15.73:0/1636651563 wait complete. 2023-11-28T04:27:10.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.818+0000 7fdbb9043700 1 Processor -- start 2023-11-28T04:27:10.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.818+0000 7fdbb9043700 1 -- start start 2023-11-28T04:27:10.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:10.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 0x7fdbb40abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:10.822 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbb40c45d0 0x7fdbb40c5da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb40ac580 con 0x7fdbb4067cb0 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4165290 con 0x7fdbb4068090 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbb4165570 con 0x7fdbb40c45d0 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36238/0 (socket says 172.21.15.73:36238) 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 -- 172.21.15.73:0/3358846175 learned_addr learned my addr 172.21.15.73:0/3358846175 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb259c700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 0x7fdbb40abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb359e700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbb40c45d0 0x7fdbb40c5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 -- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbb40c45d0 msgr2=0x7fdbb40c5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbb40c45d0 0x7fdbb40c5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 -- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 msgr2=0x7fdbb40abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.823 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 0x7fdbb40abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 -- 172.21.15.73:0/3358846175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdba80092a0 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb259c700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 0x7fdbb40abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb2d9d700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x7fdba400ea20 tx=0x7fdba4007e40).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400c9b0 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdba400cb10 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbb4165850 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.819+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdba400ce90 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.820+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbb4154480 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.821+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbb41650f0 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.821+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fdba4011680 con 0x7fdbb4067cb0 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.821+0000 7fdb9affd700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 0x7fdba0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:10.824 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.821+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fdba407c6f0 con 0x7fdbb4067cb0 2023-11-28T04:27:10.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.821+0000 7fdbb259c700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 0x7fdba0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:10.825 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.822+0000 7fdbb259c700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 0x7fdba0061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fdbb40c49d0 tx=0x7fdba8017000).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:10.826 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.825+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fdbb41650f0 con 0x7fdbb4067cb0 2023-11-28T04:27:10.984 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.983+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 --> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdbb41650f0 con 0x7fdba005f150 2023-11-28T04:27:10.987 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.986+0000 7fdb9affd700 1 -- 172.21.15.73:0/3358846175 <== mgr.10248 v2:172.21.15.73:6834/136129 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+71283 (secure 0 0 0) 0x7fdbb41650f0 con 0x7fdba005f150 2023-11-28T04:27:10.988 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:27:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 msgr2=0x7fdba0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 0x7fdba0061610 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 rx=0x7fdbb40c49d0 tx=0x7fdba8017000).stop 2023-11-28T04:27:10.989 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 msgr2=0x7fdbb4162730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 rx=0x7fdba400ea20 tx=0x7fdba4007e40).stop 2023-11-28T04:27:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 shutdown_connections 2023-11-28T04:27:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fdba005f150 0x7fdba0061610 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.990 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fdbb40c45d0 0x7fdbb40c5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fdbb4067cb0 0x7fdbb4162730 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 --2- 172.21.15.73:0/3358846175 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fdbb4068090 0x7fdbb40abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:10.991 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 >> 172.21.15.73:0/3358846175 conn(0x7fdbb40c7fd0 msgr2=0x7fdbb40c7620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:10.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 shutdown_connections 2023-11-28T04:27:10.992 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:10.988+0000 7fdbb9043700 1 -- 172.21.15.73:0/3358846175 wait complete. 2023-11-28T04:27:10.992 INFO:teuthology.orchestra.run.smithi073.stderr:dumped all 2023-11-28T04:27:11.001 INFO:teuthology.orchestra.run.smithi073.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-11-28T04:27:10.459434+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2459488,"kb_used_data":21280,"kb_used_omap":0,"kb_used_meta":2438144,"kb_avail":752515232,"statfs":{"total":773094113280,"available":770575597568,"internally_reserved":0,"allocated":21790720,"data_stored":11009976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2496659456},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-11959,"num_objects":-24,"num_object_clones":0,"num_object_copies":-48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-24,"num_whiteouts":0,"num_read":-1,"num_read_kb":-1,"num_write":-63,"num_write_kb":-33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.994292"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":180,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:49.449118+0000","last_change":"2023-11-28T04:26:49.449118+0000","last_active":"2023-11-28T04:26:49.449118+0000","last_peered":"2023-11-28T04:26:49.449118+0000","last_clean":"2023-11-28T04:26:49.449118+0000","last_became_active":"2023-11-28T02:52:40.038400+0000","last_became_peered":"2023-11-28T02:52:40.038400+0000","last_unstale":"2023-11-28T04:26:49.449118+0000","last_undegraded":"2023-11-28T04:26:49.449118+0000","last_fullsized":"2023-11-28T04:26:49.449118+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:49.449013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:49.449013+0000","last_clean_scrub_stamp":"2023-11-28T04:26:49.449013+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":178,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:50.429928+0000","last_change":"2023-11-28T04:26:50.429928+0000","last_active":"2023-11-28T04:26:50.429928+0000","last_peered":"2023-11-28T04:26:50.429928+0000","last_clean":"2023-11-28T04:26:50.429928+0000","last_became_active":"2023-11-28T02:52:39.446150+0000","last_became_peered":"2023-11-28T02:52:39.446150+0000","last_unstale":"2023-11-28T04:26:50.429928+0000","last_undegraded":"2023-11-28T04:26:50.429928+0000","last_fullsized":"2023-11-28T04:26:50.429928+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:50.429832+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:50.429832+0000","last_clean_scrub_stamp":"2023-11-28T04:26:50.429832+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":188,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:48.903929+0000","last_change":"2023-11-28T04:26:48.903929+0000","last_active":"2023-11-28T04:26:48.903929+0000","last_peered":"2023-11-28T04:26:48.903929+0000","last_clean":"2023-11-28T04:26:48.903929+0000","last_became_active":"2023-11-28T02:52:39.444786+0000","last_became_peered":"2023-11-28T02:52:39.444786+0000","last_unstale":"2023-11-28T04:26:48.903929+0000","last_undegraded":"2023-11-28T04:26:48.903929+0000","last_fullsized":"2023-11-28T04:26:48.903929+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:48.903841+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:48.903841+0000","last_clean_scrub_stamp":"2023-11-28T04:26:48.903841+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":178,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:51.416236+0000","last_change":"2023-11-28T04:26:51.416236+0000","last_active":"2023-11-28T04:26:51.416236+0000","last_peered":"2023-11-28T04:26:51.416236+0000","last_clean":"2023-11-28T04:26:51.416236+0000","last_became_active":"2023-11-28T02:52:39.446606+0000","last_became_peered":"2023-11-28T02:52:39.446606+0000","last_unstale":"2023-11-28T04:26:51.416236+0000","last_undegraded":"2023-11-28T04:26:51.416236+0000","last_fullsized":"2023-11-28T04:26:51.416236+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:51.416117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:51.416117+0000","last_clean_scrub_stamp":"2023-11-28T04:26:51.416117+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":176,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:51.977449+0000","last_change":"2023-11-28T04:26:51.977449+0000","last_active":"2023-11-28T04:26:51.977449+0000","last_peered":"2023-11-28T04:26:51.977449+0000","last_clean":"2023-11-28T04:26:51.977449+0000","last_became_active":"2023-11-28T02:52:38.027326+0000","last_became_peered":"2023-11-28T02:52:38.027326+0000","last_unstale":"2023-11-28T04:26:51.977449+0000","last_undegraded":"2023-11-28T04:26:51.977449+0000","last_fullsized":"2023-11-28T04:26:51.977449+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:51.977340+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:51.977340+0000","last_clean_scrub_stamp":"2023-11-28T04:26:51.977340+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":174,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:52.954791+0000","last_change":"2023-11-28T04:26:52.954791+0000","last_active":"2023-11-28T04:26:52.954791+0000","last_peered":"2023-11-28T04:26:52.954791+0000","last_clean":"2023-11-28T04:26:52.954791+0000","last_became_active":"2023-11-28T02:52:39.445050+0000","last_became_peered":"2023-11-28T02:52:39.445050+0000","last_unstale":"2023-11-28T04:26:52.954791+0000","last_undegraded":"2023-11-28T04:26:52.954791+0000","last_fullsized":"2023-11-28T04:26:52.954791+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:52.954684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:52.954684+0000","last_clean_scrub_stamp":"2023-11-28T04:26:52.954684+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":175,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:53.912680+0000","last_change":"2023-11-28T04:26:53.912680+0000","last_active":"2023-11-28T04:26:53.912680+0000","last_peered":"2023-11-28T04:26:53.912680+0000","last_clean":"2023-11-28T04:26:53.912680+0000","last_became_active":"2023-11-28T02:52:40.038524+0000","last_became_peered":"2023-11-28T02:52:40.038524+0000","last_unstale":"2023-11-28T04:26:53.912680+0000","last_undegraded":"2023-11-28T04:26:53.912680+0000","last_fullsized":"2023-11-28T04:26:53.912680+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:53.912545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:53.912545+0000","last_clean_scrub_stamp":"2023-11-28T04:26:53.912545+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":180,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:54.433359+0000","last_change":"2023-11-28T04:26:54.433359+0000","last_active":"2023-11-28T04:26:54.433359+0000","last_peered":"2023-11-28T04:26:54.433359+0000","last_clean":"2023-11-28T04:26:54.433359+0000","last_became_active":"2023-11-28T02:52:39.446666+0000","last_became_peered":"2023-11-28T02:52:39.446666+0000","last_unstale":"2023-11-28T04:26:54.433359+0000","last_undegraded":"2023-11-28T04:26:54.433359+0000","last_fullsized":"2023-11-28T04:26:54.433359+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:54.433245+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:54.433245+0000","last_clean_scrub_stamp":"2023-11-28T04:26:54.433245+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":198,"reported_epoch":217,"state":"active+clean","last_fresh":"2023-11-28T04:26:50.041966+0000","last_change":"2023-11-28T04:26:50.041966+0000","last_active":"2023-11-28T04:26:50.041966+0000","last_peered":"2023-11-28T04:26:50.041966+0000","last_clean":"2023-11-28T04:26:50.041966+0000","last_became_active":"2023-11-28T02:52:39.448063+0000","last_became_peered":"2023-11-28T02:52:39.448063+0000","last_unstale":"2023-11-28T04:26:50.041966+0000","last_undegraded":"2023-11-28T04:26:50.041966+0000","last_fullsized":"2023-11-28T04:26:50.041966+0000","mapping_epoch":16,"log_start":"0'0","ondisk_log_start":"0'0","created":16,"last_epoch_clean":17,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-11-28T04:26:50.041873+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-11-28T04:26:50.041873+0000","last_clean_scrub_stamp":"2023-11-28T04:26:50.041873+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575974,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.57999999999999996,"15min":0.60799999999999998},"min":{"1min":0.376,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.70199999999999996,"5min":0.81899999999999995,"15min":0.83699999999999997},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.58599999999999997,"15min":0.61699999999999999},"min":{"1min":0.34799999999999998,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.872},"last":0.52500000000000002}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.501,"5min":0.55200000000000005,"15min":0.56899999999999995},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.309},"max":{"1min":0.72499999999999998,"5min":0.78400000000000003,"15min":0.92800000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.55000000000000004,"15min":0.56599999999999995},"min":{"1min":0.39500000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.70099999999999996,"5min":0.78100000000000003,"15min":0.82599999999999996},"last":0.47899999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.89000000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.55600000000000005,"15min":0.57299999999999995},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66000000000000003,"5min":0.80600000000000005,"15min":0.84099999999999997},"last":0.53700000000000003}]},{"osd":3,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.54900000000000004,"15min":0.57899999999999996},"min":{"1min":0.36299999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.65000000000000002,"5min":0.73299999999999998,"15min":0.83599999999999997},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58399999999999996,"15min":0.59999999999999998},"min":{"1min":0.39600000000000002,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.76900000000000002,"5min":0.81299999999999994,"15min":0.90300000000000002},"last":0.66200000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47399999999999998,"15min":0.46800000000000003},"min":{"1min":0.34300000000000003,"5min":0.30299999999999999,"15min":0.29699999999999999},"max":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.50900000000000001,"15min":0.498},"min":{"1min":0.318,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65300000000000002,"5min":0.79500000000000004,"15min":0.79500000000000004},"last":0.44600000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.50600000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.70199999999999996,"5min":0.77600000000000002,"15min":0.85499999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.54900000000000004,"15min":0.54100000000000004},"min":{"1min":0.39800000000000002,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.69499999999999995,"5min":0.78400000000000003,"15min":0.81399999999999995},"last":0.59599999999999997}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.57099999999999995,"15min":0.56299999999999994},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.67600000000000005,"5min":0.82799999999999996,"15min":0.82799999999999996},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.56599999999999995,"15min":0.56499999999999995},"min":{"1min":0.36499999999999999,"5min":0.36199999999999999,"15min":0.34200000000000003},"max":{"1min":0.68300000000000005,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.60499999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575974,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.53800000000000003,"15min":0.55200000000000005},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71899999999999997,"5min":0.754,"15min":0.77600000000000002},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.56899999999999995,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.2789999999999999,"5min":1.2789999999999999,"15min":1.2789999999999999},"last":0.76600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.54800000000000004,"15min":0.56799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.68000000000000005,"5min":0.80500000000000005,"15min":0.86199999999999999},"last":0.746},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.39200000000000002,"5min":0.38600000000000001,"15min":0.34899999999999998},"max":{"1min":1.349,"5min":1.349,"15min":1.349},"last":0.64700000000000002}]},{"osd":2,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56499999999999995,"15min":0.57499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.56699999999999995,"15min":0.58399999999999996},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.34899999999999998},"max":{"1min":0.69099999999999995,"5min":0.81299999999999994,"15min":0.82199999999999995},"last":0.67400000000000004}]},{"osd":3,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.56699999999999995,"15min":0.57899999999999996},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.34399999999999997},"max":{"1min":0.72799999999999998,"5min":0.79500000000000004,"15min":0.82599999999999996},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.59199999999999997,"15min":0.59499999999999997},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.21,"5min":1.21,"15min":1.21},"last":0.69999999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.45100000000000001,"15min":0.441},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":1.3069999999999999,"5min":1.3069999999999999,"15min":1.3069999999999999},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.48299999999999998,"15min":0.48199999999999998},"min":{"1min":0.33200000000000002,"5min":0.311,"15min":0.311},"max":{"1min":1.2370000000000001,"5min":1.2370000000000001,"15min":1.2370000000000001},"last":0.51000000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.50900000000000001,"15min":0.504},"min":{"1min":0.31900000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.54800000000000004,"15min":0.54600000000000004},"min":{"1min":0.378,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.68300000000000005}]},{"osd":7,"last update":"Tue Nov 28 04:26:43 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.53300000000000003,"15min":0.52700000000000002},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.54700000000000004,"15min":0.54300000000000004},"min":{"1min":0.32100000000000001,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.63400000000000001}]}]},{"osd":1,"up_from":13,"seq":55834575973,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303852,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":301184,"kb_avail":94067988,"statfs":{"total":96636764160,"available":96325619712,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":308412416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.32800000000000001,"5min":0.372,"15min":0.36699999999999999},"min":{"1min":0.20499999999999999,"5min":0.20499999999999999,"15min":0.20499999999999999},"max":{"1min":0.495,"5min":0.55200000000000005,"15min":0.70799999999999996},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.35699999999999998,"5min":0.38300000000000001,"15min":0.39000000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.53100000000000003,"5min":0.60699999999999998,"15min":0.67000000000000004},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.40799999999999997,"5min":0.41299999999999998,"15min":0.40899999999999997},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":1.143,"5min":1.143,"15min":1.143},"last":0.372},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.44,"15min":0.435},"min":{"1min":0.30499999999999999,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.51200000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.38100000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.47199999999999998,"15min":0.47799999999999998},"min":{"1min":0.313,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.58599999999999997,"5min":0.67500000000000004,"15min":0.72999999999999998},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.40300000000000002,"5min":0.45100000000000001,"15min":0.44700000000000001},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.56899999999999995,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.34200000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":1.1930000000000001,"5min":1.1930000000000001,"15min":1.1930000000000001},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.629,"15min":0.626},"min":{"1min":0.44800000000000001,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.96899999999999997},"last":0.57099999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.65500000000000003,"15min":0.64600000000000002},"min":{"1min":0.40600000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":1.0029999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.68500000000000005,"15min":0.68000000000000005},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.1679999999999999},"last":0.57899999999999996}]},{"osd":6,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.628,"15min":0.621},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.503},{"interface":"front","average":{"1min":0.63,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.2170000000000001,"5min":1.2170000000000001,"15min":1.2170000000000001},"last":0.59999999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:26:55 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.64800000000000002,"15min":0.64900000000000002},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":0.71099999999999997,"5min":0.88900000000000001,"15min":1.0009999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.65700000000000003,"15min":0.65300000000000002},"min":{"1min":0.438,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.80500000000000005,"5min":0.86699999999999999,"15min":0.91800000000000004},"last":0.55700000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575974,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.38900000000000001,"15min":0.371},"min":{"1min":0.29199999999999998,"5min":0.24099999999999999,"15min":0.221},"max":{"1min":2.25,"5min":2.25,"15min":2.25},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.51800000000000002,"5min":0.40999999999999998,"15min":0.40000000000000002},"min":{"1min":0.24299999999999999,"5min":0.223,"15min":0.223},"max":{"1min":2.1669999999999998,"5min":2.1669999999999998,"15min":2.1669999999999998},"last":0.47299999999999998}]},{"osd":2,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.41099999999999998,"15min":0.39200000000000002},"min":{"1min":0.26500000000000001,"5min":0.23799999999999999,"15min":0.214},"max":{"1min":2.0129999999999999,"5min":2.0129999999999999,"15min":2.0129999999999999},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.48199999999999998,"15min":0.47099999999999997},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":1.929,"5min":1.929,"15min":1.929},"last":0.63100000000000001}]},{"osd":3,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.432,"15min":0.41999999999999998},"min":{"1min":0.29499999999999998,"5min":0.26900000000000002,"15min":0.26700000000000002},"max":{"1min":2.379,"5min":2.379,"15min":2.379},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.46600000000000003,"15min":0.45100000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":2.1520000000000001,"5min":2.1520000000000001,"15min":2.1520000000000001},"last":0.58099999999999996}]},{"osd":4,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.63900000000000001,"15min":0.64100000000000001},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":2.2709999999999999,"5min":2.2709999999999999,"15min":2.2709999999999999},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.66100000000000003,"15min":0.65900000000000003},"min":{"1min":0.39200000000000002,"5min":0.38400000000000001,"15min":0.38100000000000001},"max":{"1min":2.0800000000000001,"5min":2.0800000000000001,"15min":2.0800000000000001},"last":0.81899999999999995}]},{"osd":5,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.67300000000000004,"15min":0.66700000000000004},"min":{"1min":0.442,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":2.2890000000000001,"5min":2.2890000000000001,"15min":2.2890000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.754,"5min":0.65800000000000003,"15min":0.66100000000000003},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":2.0510000000000002,"5min":2.0510000000000002,"15min":2.0510000000000002},"last":0.77100000000000002}]},{"osd":6,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.436,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":2.1360000000000001,"5min":2.1360000000000001,"15min":2.1360000000000001},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.747,"5min":0.66000000000000003,"15min":0.65200000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":2.4460000000000002,"5min":2.4460000000000002,"15min":2.4460000000000002},"last":0.79900000000000004}]},{"osd":7,"last update":"Tue Nov 28 04:26:39 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.67500000000000004,"15min":0.67200000000000004},"min":{"1min":0.43099999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":2.1120000000000001,"5min":2.1120000000000001,"15min":2.1120000000000001},"last":0.85899999999999999},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.70599999999999996,"15min":0.70099999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.40300000000000002},"max":{"1min":2.4100000000000001,"5min":2.4100000000000001,"15min":2.4100000000000001},"last":0.871}]}]},{"osd":2,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.38400000000000001,"5min":0.373,"15min":0.377},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.22600000000000001},"max":{"1min":0.496,"5min":0.56000000000000005,"15min":0.61899999999999999},"last":0.379},{"interface":"front","average":{"1min":0.42199999999999999,"5min":0.438,"15min":0.443},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.53700000000000003,"5min":0.63400000000000001,"15min":0.67000000000000004},"last":0.38800000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.374,"15min":0.38300000000000001},"min":{"1min":0.23300000000000001,"5min":0.23300000000000001,"15min":0.23300000000000001},"max":{"1min":0.54300000000000004,"5min":0.57599999999999996,"15min":0.622},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.40500000000000003,"15min":0.41099999999999998},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.754,"15min":0.754},"last":0.34599999999999997}]},{"osd":3,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.433,"15min":0.44},"min":{"1min":0.29899999999999999,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.60199999999999998,"5min":0.65900000000000003,"15min":0.69199999999999995},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.41499999999999998,"5min":0.41299999999999998,"15min":0.42099999999999999},"min":{"1min":0.28499999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.65300000000000002},"last":0.30199999999999999}]},{"osd":4,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.59799999999999998,"15min":0.60899999999999999},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.40100000000000002},"max":{"1min":0.82499999999999996,"5min":0.84599999999999997,"15min":0.94399999999999995},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.622,"15min":0.63100000000000001},"min":{"1min":0.45900000000000002,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.73899999999999999,"5min":0.876,"15min":0.96199999999999997},"last":0.64600000000000002}]},{"osd":5,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.65200000000000002,"15min":0.66400000000000003},"min":{"1min":0.46899999999999997,"5min":0.46100000000000002,"15min":0.44500000000000001},"max":{"1min":0.76800000000000002,"5min":0.98399999999999999,"15min":1.0760000000000001},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.66200000000000003,"15min":0.67400000000000004},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.89400000000000002,"5min":0.96299999999999997,"15min":1.016},"last":0.80600000000000005}]},{"osd":6,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.64500000000000002,"15min":0.65600000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.99099999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67600000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.82699999999999996,"5min":0.94899999999999995,"15min":1.0529999999999999},"last":0.61499999999999999}]},{"osd":7,"last update":"Tue Nov 28 04:26:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.501,"5min":0.435,"15min":0.41099999999999998},"max":{"1min":0.90400000000000003,"5min":0.93500000000000005,"15min":0.95599999999999996},"last":0.754},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.64500000000000002,"15min":0.66200000000000003},"min":{"1min":0.48999999999999999,"5min":0.433,"15min":0.42799999999999999},"max":{"1min":0.78900000000000003,"5min":0.97799999999999998,"15min":1.0129999999999999},"last":0.63700000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.36799999999999999,"15min":0.37},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22500000000000001},"max":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"last":0.36699999999999999},{"interface":"front","average":{"1min":0.379,"5min":0.38500000000000001,"15min":0.39200000000000002},"min":{"1min":0.27500000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.66500000000000004},"last":0.29999999999999999}]},{"osd":1,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.39700000000000002,"5min":0.38200000000000001,"15min":0.38800000000000001},"min":{"1min":0.28000000000000003,"5min":0.26000000000000001,"15min":0.245},"max":{"1min":0.54400000000000004,"5min":0.61499999999999999,"15min":0.65500000000000003},"last":0.246},{"interface":"front","average":{"1min":0.41299999999999998,"5min":0.42199999999999999,"15min":0.42799999999999999},"min":{"1min":0.26500000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"last":0.309}]},{"osd":2,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.42399999999999999,"15min":0.434},"min":{"1min":0.28699999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.57999999999999996,"5min":0.60099999999999998,"15min":0.746},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45300000000000001,"15min":0.46000000000000002},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.68000000000000005},"last":0.40400000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.60599999999999998,"15min":0.61399999999999999},"min":{"1min":0.39700000000000002,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":0.90800000000000003,"5min":0.91200000000000003,"15min":0.92500000000000004},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.625},"min":{"1min":0.42999999999999999,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.88600000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.45000000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.629,"15min":0.63100000000000001},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":1.0409999999999999},"last":0.497},{"interface":"front","average":{"1min":0.625,"5min":0.69599999999999995,"15min":0.66900000000000004},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85099999999999998,"5min":4.3049999999999997,"15min":4.3049999999999997},"last":0.56299999999999994}]},{"osd":6,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64700000000000002,"15min":0.64500000000000002},"min":{"1min":0.433,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.94999999999999996,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.505},{"interface":"front","average":{"1min":0.627,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.47799999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.90300000000000002,"5min":1.028,"15min":1.028},"last":0.47699999999999998}]},{"osd":7,"last update":"Tue Nov 28 04:26:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.66200000000000003,"15min":0.66500000000000004},"min":{"1min":0.47399999999999998,"5min":0.45200000000000001,"15min":0.44},"max":{"1min":0.92200000000000004,"5min":1.008,"15min":1.008},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.65800000000000003,"15min":0.66400000000000003},"min":{"1min":0.45000000000000001,"5min":0.44500000000000001,"15min":0.40799999999999997},"max":{"1min":0.96299999999999997,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.441}]}]},{"osd":4,"up_from":13,"seq":55834575973,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.54400000000000004,"15min":0.56100000000000005},"min":{"1min":0.376,"5min":0.34399999999999997,"15min":0.28299999999999997},"max":{"1min":0.65600000000000003,"5min":0.749,"15min":0.80600000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.47399999999999998,"5min":0.54500000000000004,"15min":0.56100000000000005},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.316},"max":{"1min":0.57699999999999996,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.50600000000000001}]},{"osd":1,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.55100000000000005,"15min":0.56899999999999995},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.63,"5min":0.75800000000000001,"15min":0.76900000000000002},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.503,"5min":0.55100000000000005,"15min":0.57199999999999995},"min":{"1min":0.373,"5min":0.373,"15min":0.37},"max":{"1min":0.66400000000000003,"5min":0.78800000000000003,"15min":0.82699999999999996},"last":0.57499999999999996}]},{"osd":2,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.54000000000000004,"15min":0.56000000000000005},"min":{"1min":0.378,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.58599999999999997,"5min":0.73599999999999999,"15min":0.83299999999999996},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.55200000000000005,"15min":0.57599999999999996},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.67000000000000004,"5min":0.76700000000000002,"15min":0.83499999999999996},"last":0.52300000000000002}]},{"osd":3,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.59599999999999997,"15min":0.60899999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.69199999999999995,"5min":0.83999999999999997,"15min":0.84399999999999997},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60999999999999999,"15min":0.63400000000000001},"min":{"1min":0.45500000000000002,"5min":0.44600000000000001,"15min":0.42499999999999999},"max":{"1min":0.67700000000000005,"5min":0.83199999999999996,"15min":0.872},"last":0.63600000000000001}]},{"osd":5,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.47899999999999998,"15min":0.48299999999999998},"min":{"1min":0.33700000000000002,"5min":0.31,"15min":0.308},"max":{"1min":0.72799999999999998,"5min":0.72799999999999998,"15min":0.78300000000000003},"last":0.625},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.501,"15min":0.51400000000000001},"min":{"1min":0.36699999999999999,"5min":0.31900000000000001,"15min":0.29599999999999999},"max":{"1min":0.71899999999999997,"5min":0.76300000000000001,"15min":0.83499999999999996},"last":0.41999999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.49199999999999999,"15min":0.505},"min":{"1min":0.39200000000000002,"5min":0.34000000000000002,"15min":0.30599999999999999},"max":{"1min":0.65000000000000002,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.52600000000000002,"15min":0.54200000000000004},"min":{"1min":0.38800000000000001,"5min":0.36399999999999999,"15min":0.35499999999999998},"max":{"1min":0.747,"5min":0.747,"15min":0.85099999999999998},"last":0.46899999999999997}]},{"osd":7,"last update":"Tue Nov 28 04:27:01 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.55700000000000005,"15min":0.56399999999999995},"min":{"1min":0.36299999999999999,"5min":0.35399999999999998,"15min":0.33000000000000002},"max":{"1min":0.77500000000000002,"5min":0.78800000000000003,"15min":0.85999999999999999},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56499999999999995,"15min":0.57899999999999996},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.76200000000000001,"5min":0.78300000000000003,"15min":0.82399999999999995},"last":0.66400000000000003}]}]},{"osd":5,"up_from":13,"seq":55834575975,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307948,"kb_used_data":2660,"kb_used_omap":0,"kb_used_meta":305280,"kb_avail":94063892,"statfs":{"total":96636764160,"available":96321425408,"internally_reserved":0,"allocated":2723840,"data_stored":1376247,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":312606720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.52000000000000002,"15min":0.54000000000000004},"min":{"1min":0.35599999999999998,"5min":0.35399999999999998,"15min":0.32500000000000001},"max":{"1min":0.63600000000000001,"5min":0.71099999999999997,"15min":0.75900000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.53800000000000003,"15min":0.55600000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.76900000000000002,"15min":0.80000000000000004},"last":0.67700000000000005}]},{"osd":1,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.56799999999999995,"15min":0.58599999999999997},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.68500000000000005,"5min":0.77100000000000002,"15min":0.80800000000000005},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.60199999999999998,"15min":0.61099999999999999},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.71099999999999997,"5min":0.81899999999999995,"15min":0.84799999999999998},"last":0.748}]},{"osd":2,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.59499999999999997,"15min":0.59899999999999998},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.70699999999999996,"5min":0.76700000000000002,"15min":0.79900000000000004},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.61199999999999999,"15min":0.63},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40400000000000003},"max":{"1min":0.69499999999999995,"5min":0.79300000000000004,"15min":0.86399999999999999},"last":0.73599999999999999}]},{"osd":3,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.56100000000000005,"15min":0.56999999999999995},"min":{"1min":0.40999999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"max":{"1min":0.71699999999999997,"5min":0.78200000000000003,"15min":0.95299999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57199999999999995,"15min":0.59199999999999997},"min":{"1min":0.372,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.73099999999999998,"5min":0.76500000000000001,"15min":0.83599999999999997},"last":0.65700000000000003}]},{"osd":4,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.432,"5min":0.44900000000000001,"15min":0.45700000000000002},"min":{"1min":0.32400000000000001,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.66100000000000003,"5min":0.69599999999999995,"15min":0.88200000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.54900000000000004,"15min":0.56000000000000005},"min":{"1min":0.38,"5min":0.38,"15min":0.318},"max":{"1min":0.754,"5min":0.76400000000000001,"15min":0.98599999999999999},"last":0.60199999999999998}]},{"osd":6,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.497,"15min":0.504},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.86699999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.56399999999999995,"15min":0.57799999999999996},"min":{"1min":0.42599999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.74199999999999999,"5min":0.75900000000000001,"15min":0.92100000000000004},"last":0.68999999999999995}]},{"osd":7,"last update":"Tue Nov 28 04:26:35 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.52200000000000002,"15min":0.52500000000000002},"min":{"1min":0.35799999999999998,"5min":0.33700000000000002,"15min":0.32000000000000001},"max":{"1min":0.68999999999999995,"5min":0.76300000000000001,"15min":0.90300000000000002},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.55400000000000005,"15min":0.56399999999999995},"min":{"1min":0.34300000000000003,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.85799999999999998},"last":0.625}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-11-28T04:27:11.004 DEBUG:teuthology.orchestra.run.smithi073:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-11-28T04:27:11.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.115+0000 7feced5d0700 1 Processor -- start 2023-11-28T04:27:11.116 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.115+0000 7feced5d0700 1 -- start start 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7feced5d0700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fece8098d10 0x7fece80c7760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7feced5d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 0x7fece81587d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7feced5d0700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fece8158cc0 con 0x7fece80c6190 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7feced5d0700 1 -- --> v2:172.21.15.73:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fece80c72d0 con 0x7fece8098d10 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7feced5d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8064570 con 0x7fece8158320 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fece8098d10 0x7fece80c7760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 --2- >> v2:172.21.15.73:3300/0 conn(0x7fece8098d10 0x7fece80c7760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36248/0 (socket says 172.21.15.73:36248) 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 -- 172.21.15.73:0/2474463824 learned_addr learned my addr 172.21.15.73:0/2474463824 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece77fe700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 0x7fece81587d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 -- 172.21.15.73:0/2474463824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 msgr2=0x7fece81587d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:27:11.119 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 0x7fece81587d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 -- 172.21.15.73:0/2474463824 >> v1:172.21.15.150:6789/0 conn(0x7fece80c6190 legacy=0x7fece8063e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 -- 172.21.15.73:0/2474463824 --> v2:172.21.15.73:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8154bc0 con 0x7fece8098d10 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece77fe700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 0x7fece81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).handle_auth_reply_more state changed! 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.116+0000 7fece6ffd700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8098d10 0x7fece80c7760 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x7fecd8009f20 tx=0x7fecd8009840).ready entity=mon.0 client_cookie=ea36a301a938a550 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7fece5ffb700 1 -- 172.21.15.73:0/2474463824 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd800e070 con 0x7fece8098d10 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7fece5ffb700 1 -- 172.21.15.73:0/2474463824 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecd800ba80 con 0x7fece8098d10 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7fece5ffb700 1 -- 172.21.15.73:0/2474463824 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecd800be00 con 0x7fece8098d10 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 -- 172.21.15.73:0/2474463824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8098d10 msgr2=0x7fece80c7760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8098d10 0x7fece80c7760 secure :-1 s=READY pgs=2208 cs=0 l=1 rev1=1 rx=0x7fecd8009f20 tx=0x7fecd8009840).stop 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 -- 172.21.15.73:0/2474463824 shutdown_connections 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8158320 0x7fece81587d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.120 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 --2- 172.21.15.73:0/2474463824 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8098d10 0x7fece80c7760 unknown :-1 s=CLOSED pgs=2208 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 -- 172.21.15.73:0/2474463824 >> 172.21.15.73:0/2474463824 conn(0x7fece80cb310 msgr2=0x7fece80cb720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:11.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 -- 172.21.15.73:0/2474463824 shutdown_connections 2023-11-28T04:27:11.121 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.117+0000 7feced5d0700 1 -- 172.21.15.73:0/2474463824 wait complete. 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.118+0000 7feced5d0700 1 Processor -- start 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.118+0000 7feced5d0700 1 -- start start 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8098d10 0x7fece80b0fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fece80c6190 0x7fece8156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.123 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8157670 con 0x7fece8158320 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8156a60 con 0x7fece80c6190 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7feced5d0700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8156d40 con 0x7fece8098d10 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7fece77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7fece77fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36262/0 (socket says 172.21.15.73:36262) 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7fece77fe700 1 -- 172.21.15.73:0/3611671176 learned_addr learned my addr 172.21.15.73:0/3611671176 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7fece67fc700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fece80c6190 0x7fece8156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.119+0000 7fece6ffd700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8098d10 0x7fece80b0fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 -- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8098d10 msgr2=0x7fece80b0fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8098d10 0x7fece80b0fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 -- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fece80c6190 msgr2=0x7fece8156230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fece80c6190 0x7fece8156230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.124 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 -- 172.21.15.73:0/3611671176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecd8009ae0 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fece77fe700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x7fece000d860 tx=0x7fece0008040).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0011590 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fece00116f0 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fece8157020 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fece0011a90 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.120+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fece8157b00 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.121+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fece8157820 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.121+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7fece001c460 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.122+0000 7fecceffd700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 0x7fecd40615e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.122+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7fece0079d80 con 0x7fece8158320 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.122+0000 7fece6ffd700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 0x7fecd40615e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.125 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.122+0000 7fece6ffd700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 0x7fecd40615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fecd8009880 tx=0x7fece80a8040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.127 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.126+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7fece004b170 con 0x7fece8158320 2023-11-28T04:27:11.293 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.292+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7fece806c860 con 0x7fece8158320 2023-11-28T04:27:11.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.296+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fece004b170 con 0x7fece8158320 2023-11-28T04:27:11.297 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.296+0000 7fecceffd700 1 -- 172.21.15.73:0/3611671176 <== mon.0 v2:172.21.15.73:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v46)=0 v46) v1 ==== 123+0+0 (secure 0 0 0) 0x7fece004e790 con 0x7fece8158320 2023-11-28T04:27:11.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 msgr2=0x7fecd40615e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.299 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 0x7fecd40615e0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 rx=0x7fecd8009880 tx=0x7fece80a8040).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 msgr2=0x7fece80ad0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 secure :-1 s=READY pgs=2209 cs=0 l=1 rev1=1 rx=0x7fece000d860 tx=0x7fece0008040).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 shutdown_connections 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7fecd405f120 0x7fecd40615e0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7fece8158320 0x7fece80ad0b0 unknown :-1 s=CLOSED pgs=2209 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7fece8098d10 0x7fece80b0fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 --2- 172.21.15.73:0/3611671176 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7fece80c6190 0x7fece8156230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 >> 172.21.15.73:0/3611671176 conn(0x7fece80cb310 msgr2=0x7fece8153b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 shutdown_connections 2023-11-28T04:27:11.300 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.298+0000 7feced5d0700 1 -- 172.21.15.73:0/3611671176 wait complete. 2023-11-28T04:27:11.311 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-11-28T04:27:11.311 DEBUG:teuthology.orchestra.run.smithi073:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-11-28T04:27:11.424 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.423+0000 7f61da7f6700 1 Processor -- start 2023-11-28T04:27:11.425 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.423+0000 7f61da7f6700 1 -- start start 2023-11-28T04:27:11.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61da7f6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61d40668c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61da7f6700 1 -- --> v1:172.21.15.73:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d4065760 con 0x7f61d41542d0 2023-11-28T04:27:11.428 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61da7f6700 1 -- --> v1:172.21.15.150:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d4063040 con 0x7f61d4062700 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61da7f6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c3250 con 0x7f61d4062bb0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61d3fff700 1 --1- >> v1:172.21.15.73:6789/0 conn(0x7f61d41542d0 0x7f61d40697f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.73:6789/0 says I am v1:172.21.15.73:34806/0 (socket says 172.21.15.73:34806) 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61d3fff700 1 -- 172.21.15.73:0/3968209044 learned_addr learned my addr 172.21.15.73:0/3968209044 (peer_addr_for_me v1:172.21.15.73:0/0) 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v1:172.21.15.73:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3080199656 0 0) 0x7f61d4065760 con 0x7f61d41542d0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61d8d93700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61d40668c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.424+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 --> v1:172.21.15.73:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61bc0033f0 con 0x7f61d41542d0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v1:172.21.15.73:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 578065188 0 0) 0x7f61bc0033f0 con 0x7f61d41542d0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 msgr2=0x7f61d40668c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61d40668c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 >> v1:172.21.15.150:6789/0 conn(0x7f61d4062700 legacy=0x7f61d40661b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 --> v1:172.21.15.73:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d406a340 con 0x7f61d41542d0 2023-11-28T04:27:11.429 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v1:172.21.15.73:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3407970787 0 0) 0x7f61c4001470 con 0x7f61d41542d0 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 >> v1:172.21.15.73:6789/0 conn(0x7f61d41542d0 legacy=0x7f61d40697f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc003910 0x7f61bc003d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61bc004690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61bc004b80 0x7f61bc005000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40c3250 con 0x7f61bc004b80 2023-11-28T04:27:11.430 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00b940 con 0x7f61bc003910 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc00bb00 con 0x7f61d4062bb0 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61bc004b80 0x7f61bc005000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d3fff700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc003910 0x7f61bc003d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d37fe700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61bc004690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 -- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 msgr2=0x7f61bc004690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d4062bb0 0x7f61bc004690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.431 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 -- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc003910 msgr2=0x7f61bc003d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61bc003910 0x7f61bc003d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 -- 172.21.15.73:0/3968209044 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61bc0033f0 con 0x7f61bc004b80 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d8d93700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61bc004b80 0x7f61bc005000 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7f61cc00e210 tx=0x7f61cc00b040).ready entity=mon.0 client_cookie=e40ca111e2163f9d server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.425+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc009ac0 con 0x7f61bc004b80 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f61cc00f040 con 0x7f61bc004b80 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61d2ffd700 1 -- 172.21.15.73:0/3968209044 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc00ecf0 con 0x7f61bc004b80 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 -- 172.21.15.73:0/3968209044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61bc004b80 msgr2=0x7f61bc005000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 --2- 172.21.15.73:0/3968209044 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61bc004b80 0x7f61bc005000 secure :-1 s=READY pgs=2211 cs=0 l=1 rev1=1 rx=0x7f61cc00e210 tx=0x7f61cc00b040).stop 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61d8d93700 1 -- 172.21.15.73:0/3968209044 reap_dead start 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 -- 172.21.15.73:0/3968209044 shutdown_connections 2023-11-28T04:27:11.432 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 -- 172.21.15.73:0/3968209044 >> 172.21.15.73:0/3968209044 conn(0x7f61d40c7fd0 msgr2=0x7f61d40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:11.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 -- 172.21.15.73:0/3968209044 shutdown_connections 2023-11-28T04:27:11.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.426+0000 7f61da7f6700 1 -- 172.21.15.73:0/3968209044 wait complete. 2023-11-28T04:27:11.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.427+0000 7f61da7f6700 1 Processor -- start 2023-11-28T04:27:11.433 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.427+0000 7f61da7f6700 1 -- start start 2023-11-28T04:27:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 --2- >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d414c9a0 0x7f61d40aa490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 --2- >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d40a7ab0 0x7f61d40a7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 -- --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40ab8d0 con 0x7f61d40aaf60 2023-11-28T04:27:11.435 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 -- --> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40ada20 con 0x7f61d414c9a0 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61da7f6700 1 -- --> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d40ac980 con 0x7f61d40a7ab0 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 --2- >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).handle_hello peer v2:172.21.15.73:3300/0 says I am v2:172.21.15.73:36290/0 (socket says 172.21.15.73:36290) 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 -- 172.21.15.73:0/4165318410 learned_addr learned my addr 172.21.15.73:0/4165318410 (peer_addr_for_me v2:172.21.15.73:0/0) 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d8d93700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d40a7ab0 0x7f61d40a7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d3fff700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d414c9a0 0x7f61d40aa490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 -- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d40a7ab0 msgr2=0x7f61d40a7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d40a7ab0 0x7f61d40a7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 -- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d414c9a0 msgr2=0x7f61d40aa490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d414c9a0 0x7f61d40aa490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.428+0000 7f61d37fe700 1 -- 172.21.15.73:0/4165318410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d40acc60 con 0x7f61d40aaf60 2023-11-28T04:27:11.436 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61d37fe700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 rx=0x7f61c80028c0 tx=0x7f61c8007bb0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800f6c0 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f61c800f820 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c800fbe0 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d40acf40 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.429+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d40ad500 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.430+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d40ad720 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.430+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 4 ==== mgrmap(e 105) v1 ==== 75446+0+0 (secure 0 0 0) 0x7f61c8014450 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.430+0000 7f61bbfff700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 0x7f61c0061610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0).connect 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.431+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 5742+0+0 (secure 0 0 0) 0x7f61c8079cf0 con 0x7f61d40aaf60 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.431+0000 7f61d3fff700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 0x7f61c0061610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 rx=0 tx=0)._handle_peer_banner_payload supported=1 required=0 2023-11-28T04:27:11.437 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.431+0000 7f61d3fff700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 0x7f61c0061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f61c4001ce0 tx=0x7f61c4009040).ready entity=mgr.10248 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-11-28T04:27:11.438 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.435+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+162528 (secure 0 0 0) 0x7f61c804b230 con 0x7f61d40aaf60 2023-11-28T04:27:11.641 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.640+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 --> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f61d406c860 con 0x7f61d40aaf60 2023-11-28T04:27:11.642 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.641+0000 7f61bbfff700 1 -- 172.21.15.73:0/4165318410 <== mon.0 v2:172.21.15.73:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 78 v78) v1 ==== 94+0+3533 (secure 0 0 0) 0x7f61c804e850 con 0x7f61d40aaf60 2023-11-28T04:27:11.642 INFO:teuthology.orchestra.run.smithi073.stdout: 2023-11-28T04:27:11.642 INFO:teuthology.orchestra.run.smithi073.stdout:{"epoch":78,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7892,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6834/1502721650","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6832","nonce":1502721650},{"type":"v1","addr":"172.21.15.150:6834","nonce":1502721650}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":7895,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.150:6835/227821377","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.150:6833","nonce":227821377},{"type":"v1","addr":"172.21.15.150:6835","nonce":227821377}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":9987,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6837/2782485709","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6835","nonce":2782485709},{"type":"v1","addr":"172.21.15.73:6837","nonce":2782485709}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73},{"gid":10332,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.73:6833/3275083137","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.73:6832","nonce":3275083137},{"type":"v1","addr":"172.21.15.73:6833","nonce":3275083137}]},"join_fscid":-1,"export_targets":[],"features":4540138314316775423,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":78}],"filesystems":[]} 2023-11-28T04:27:11.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 msgr2=0x7f61c0061610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 0x7f61c0061610 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 rx=0x7f61c4001ce0 tx=0x7f61c4009040).stop 2023-11-28T04:27:11.644 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 msgr2=0x7f61d40ab3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 secure :-1 s=READY pgs=2212 cs=0 l=1 rev1=1 rx=0x7f61c80028c0 tx=0x7f61c8007bb0).stop 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 shutdown_connections 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:6834/136129,v1:172.21.15.73:6836/136129] conn(0x7f61c005f150 0x7f61c0061610 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3301/0,v1:172.21.15.150:6790/0] conn(0x7f61d40a7ab0 0x7f61d40a7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.150:3300/0,v1:172.21.15.150:6789/0] conn(0x7f61d414c9a0 0x7f61d40aa490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 --2- 172.21.15.73:0/4165318410 >> [v2:172.21.15.73:3300/0,v1:172.21.15.73:6789/0] conn(0x7f61d40aaf60 0x7f61d40ab3e0 unknown :-1 s=CLOSED pgs=2212 cs=0 l=1 rev1=1 rx=0 tx=0).stop 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 >> 172.21.15.73:0/4165318410 conn(0x7f61d40c7fd0 msgr2=0x7f61d40c83e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 shutdown_connections 2023-11-28T04:27:11.645 INFO:teuthology.orchestra.run.smithi073.stderr:2023-11-28T04:27:11.643+0000 7f61da7f6700 1 -- 172.21.15.73:0/4165318410 wait complete. 2023-11-28T04:27:11.646 INFO:teuthology.orchestra.run.smithi073.stderr:dumped fsmap epoch 78 2023-11-28T04:27:11.656 DEBUG:tasks.cephfs.filesystem:already dead... 2023-11-28T04:27:11.656 INFO:teuthology.misc:Shutting down mds daemons... 2023-11-28T04:27:11.656 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-11-28T04:27:11.657 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.665 INFO:tasks.ceph.mds.a:Stopped 2023-11-28T04:27:11.665 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-11-28T04:27:11.665 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.713 INFO:tasks.ceph.mds.c:Stopped 2023-11-28T04:27:11.714 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-11-28T04:27:11.714 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.724 INFO:tasks.ceph.mds.b:Stopped 2023-11-28T04:27:11.724 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-11-28T04:27:11.724 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.775 INFO:tasks.ceph.mds.d:Stopped 2023-11-28T04:27:11.775 INFO:teuthology.misc:Shutting down osd daemons... 2023-11-28T04:27:11.776 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-11-28T04:27:11.776 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.814 INFO:tasks.ceph.osd.0:Stopped 2023-11-28T04:27:11.814 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-11-28T04:27:11.814 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.880 INFO:tasks.ceph.osd.1:Stopped 2023-11-28T04:27:11.881 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-11-28T04:27:11.881 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:11.952 INFO:tasks.ceph.osd.2:Stopped 2023-11-28T04:27:11.952 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-11-28T04:27:11.952 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.023 INFO:tasks.ceph.osd.3:Stopped 2023-11-28T04:27:12.023 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-11-28T04:27:12.023 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.067 INFO:tasks.ceph.osd.4:Stopped 2023-11-28T04:27:12.067 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-11-28T04:27:12.067 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.100 INFO:tasks.ceph.osd.5:Stopped 2023-11-28T04:27:12.100 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-11-28T04:27:12.101 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.175 INFO:tasks.ceph.osd.6:Stopped 2023-11-28T04:27:12.175 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-11-28T04:27:12.175 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.242 INFO:tasks.ceph.osd.7:Stopped 2023-11-28T04:27:12.243 INFO:teuthology.misc:Shutting down mgr daemons... 2023-11-28T04:27:12.243 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-11-28T04:27:12.243 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.269 INFO:tasks.ceph.mgr.y:Stopped 2023-11-28T04:27:12.269 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-11-28T04:27:12.269 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.310 INFO:tasks.ceph.mgr.x:Stopped 2023-11-28T04:27:12.311 INFO:teuthology.misc:Shutting down mon daemons... 2023-11-28T04:27:12.311 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-11-28T04:27:12.311 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.330 INFO:tasks.ceph.mon.a:Stopped 2023-11-28T04:27:12.330 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-11-28T04:27:12.330 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.367 INFO:tasks.ceph.mon.b:Stopped 2023-11-28T04:27:12.368 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-11-28T04:27:12.368 INFO:teuthology.orchestra.run:waiting for 300 2023-11-28T04:27:12.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2023-11-28T04:27:12.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-11-28T04:27:12.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c is failed for ~0s 2023-11-28T04:27:12.425 INFO:tasks.ceph.mon.c:Stopped 2023-11-28T04:27:12.425 INFO:tasks.ceph:Checking cluster log for badness... 2023-11-28T04:27:12.426 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-28T04:27:12.455 INFO:teuthology.orchestra.run.smithi073.stdout:2023-11-28T03:06:19.269245+0000 mon.a (mon.0) 594 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-28T04:27:12.455 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-11-28T04:27:12.455 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-28T04:27:12.524 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-28T04:27:12.593 DEBUG:teuthology.orchestra.run.smithi073:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-11-28T04:27:12.660 INFO:teuthology.orchestra.run.smithi073.stdout:2023-11-28T03:06:19.269245+0000 mon.a (mon.0) 594 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2023-11-28T04:27:12.660 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:27:12.661 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-11-28T04:27:12.918 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:27:12.918 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-11-28T04:27:13.014 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:27:13.014 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-11-28T04:27:13.088 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:27:13.088 DEBUG:teuthology.orchestra.run.smithi073:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-11-28T04:27:13.161 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T04:27:13.161 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-11-28T04:27:13.410 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T04:27:13.410 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-11-28T04:27:13.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T04:27:13.488 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-11-28T04:27:13.557 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T04:27:13.558 DEBUG:teuthology.orchestra.run.smithi150:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-11-28T04:27:13.640 INFO:tasks.ceph:Archiving mon data... 2023-11-28T04:27:13.642 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/data/mon.a.tgz 2023-11-28T04:27:13.642 DEBUG:teuthology.orchestra.run.smithi073:> mktemp 2023-11-28T04:27:13.658 INFO:teuthology.orchestra.run.smithi073.stdout:/tmp/tmp.mGytszakyT 2023-11-28T04:27:13.658 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.mGytszakyT 2023-11-28T04:27:14.154 DEBUG:teuthology.orchestra.run.smithi073:> sudo chmod 0666 /tmp/tmp.mGytszakyT 2023-11-28T04:27:14.236 DEBUG:teuthology.orchestra.remote:smithi073:/tmp/tmp.mGytszakyT is 2MB 2023-11-28T04:27:14.385 DEBUG:teuthology.orchestra.run.smithi073:> rm -fr /tmp/tmp.mGytszakyT 2023-11-28T04:27:14.401 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/data/mon.b.tgz 2023-11-28T04:27:14.401 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-28T04:27:14.416 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.KBBAtNSDml 2023-11-28T04:27:14.417 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KBBAtNSDml 2023-11-28T04:27:14.931 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.KBBAtNSDml 2023-11-28T04:27:15.013 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.KBBAtNSDml is 3MB 2023-11-28T04:27:15.135 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.KBBAtNSDml 2023-11-28T04:27:15.150 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/data/mon.c.tgz 2023-11-28T04:27:15.150 DEBUG:teuthology.orchestra.run.smithi150:> mktemp 2023-11-28T04:27:15.204 INFO:teuthology.orchestra.run.smithi150.stdout:/tmp/tmp.kbnwyjMaVv 2023-11-28T04:27:15.204 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.kbnwyjMaVv 2023-11-28T04:27:15.730 DEBUG:teuthology.orchestra.run.smithi150:> sudo chmod 0666 /tmp/tmp.kbnwyjMaVv 2023-11-28T04:27:15.813 DEBUG:teuthology.orchestra.remote:smithi150:/tmp/tmp.kbnwyjMaVv is 3MB 2023-11-28T04:27:15.922 DEBUG:teuthology.orchestra.run.smithi150:> rm -fr /tmp/tmp.kbnwyjMaVv 2023-11-28T04:27:15.937 INFO:tasks.ceph:Cleaning ceph cluster... 2023-11-28T04:27:15.937 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T04:27:15.940 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-11-28T04:27:16.001 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-11-28T04:27:16.001 DEBUG:teuthology.orchestra.run.smithi073:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T04:27:16.008 DEBUG:teuthology.orchestra.run.smithi150:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-11-28T04:27:16.034 INFO:teuthology.orchestra.run.smithi073.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T04:27:16.066 INFO:teuthology.orchestra.run.smithi150.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-11-28T04:27:16.072 INFO:tasks.ceph:Archiving crash dumps... 2023-11-28T04:27:16.074 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi073/crash 2023-11-28T04:27:16.074 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T04:27:16.107 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi150/crash 2023-11-28T04:27:16.108 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-11-28T04:27:16.135 INFO:tasks.ceph:Shutting down logrotate 2023-11-28T04:27:16.135 DEBUG:teuthology.orchestra.run.smithi073:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:27:16.151 DEBUG:teuthology.orchestra.run.smithi150:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-11-28T04:27:16.229 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T04:27:16.255 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-11-28T04:27:16.284 INFO:tasks.ceph:Compressing logs... 2023-11-28T04:27:16.284 DEBUG:teuthology.orchestra.run.smithi073:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T04:27:16.306 DEBUG:teuthology.orchestra.run.smithi150:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-11-28T04:27:16.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90208.log 2023-11-28T04:27:16.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90291.log 2023-11-28T04:27:16.369 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90085.log 2023-11-28T04:27:16.369 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90208.log: /var/log/ceph/ceph-client.admin.90291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90382.log 2023-11-28T04:27:16.369 INFO:teuthology.orchestra.run.smithi150.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.90208.log.gz 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90291.log.gz 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90085.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90126.log 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90382.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90085.log.gz 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90382.log.gz 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.94776.log 2023-11-28T04:27:16.370 INFO:teuthology.orchestra.run.smithi150.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.90126.log.gz 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.3.94776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94418.log 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.3.94776.log.gz 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93986.log 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.94418.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.93986.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94418.log.gz 2023-11-28T04:27:16.371 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph.audit.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93986.log.gz 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.94390.log 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.1.94390.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.94390.log.gz 2023-11-28T04:27:16.372 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mds.b.log: 93.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90250.log 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90250.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.90250.log.gz 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.6.log: 94.4% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-28T04:27:16.373 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-11-28T04:27:16.387 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90166.log 2023-11-28T04:27:16.393 INFO:teuthology.orchestra.run.smithi150.stderr: 92.7% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-11-28T04:27:16.393 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90166.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.90166.log.gz 2023-11-28T04:27:16.393 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-11-28T04:27:16.399 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-mon.b.log: 92.6% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-11-28T04:27:16.399 INFO:teuthology.orchestra.run.smithi150.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90340.log 2023-11-28T04:27:16.399 INFO:teuthology.orchestra.run.smithi150.stderr:/var/log/ceph/ceph-client.admin.90340.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.90340.log.gz 2023-11-28T04:27:16.442 INFO:teuthology.orchestra.run.smithi150.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94731.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115318.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94731.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136658.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95798.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.94731.log.gz 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115318.log: /var/log/ceph/ceph-client.admin.136658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132236.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115318.log.gz 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114462.log 2023-11-28T04:27:16.588 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137872.log 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114336.log 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97587.log 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95798.log.gz 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132236.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.136658.log.gz 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114462.log: /var/log/ceph/ceph-client.admin.137872.log: /var/log/ceph/ceph-client.admin.114336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99290.log 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.132236.log.gz 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137872.log.gz 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114462.log.gz 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133650.log 2023-11-28T04:27:16.589 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114336.log.gz 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99290.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97587.log.gz 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117667.log 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133650.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112586.log 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99290.log.gz 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117667.log: 88.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.133650.log.gz--verbose 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.141907.log 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112586.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117667.log.gz 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106206.log 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141907.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121737.log 2023-11-28T04:27:16.590 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112586.log.gz 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9%/var/log/ceph/ceph-client.admin.106206.log: -- replaced with /var/log/ceph/ceph-client.admin.141907.log.gz 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108112.log 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96261.log 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106206.log.gz 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99248.log 2023-11-28T04:27:16.591 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.121737.log.gz 2023-11-28T04:27:16.592 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136988.log 2023-11-28T04:27:16.592 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96261.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.108112.log.gz 2023-11-28T04:27:16.592 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142754.log 2023-11-28T04:27:16.592 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.96261.log.gz 2023-11-28T04:27:16.592 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.136988.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132758.log 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99248.log.gz 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124079.log 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.136988.log.gz 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132758.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117583.log 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142754.log.gz 2023-11-28T04:27:16.593 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.132758.log.gz 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124079.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125258.log 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94308.log 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124079.log.gz 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125258.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.117583.log.gz 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111294.log 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94308.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125258.log.gz 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121629.log 2023-11-28T04:27:16.594 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111294.log: gzip -5 --verbose -- 88.5%/var/log/ceph/ceph-client.admin.124554.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94308.log.gz 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138515.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111294.log.gz 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124554.log: gzip -5 88.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.121629.log.gz/var/log/ceph/ceph-client.admin.125342.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138515.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124554.log.gz 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132534.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129746.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.138515.log.gz 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132534.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134474.log 2023-11-28T04:27:16.595 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129746.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96219.log 87.7% 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.132534.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.125342.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134474.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131148.log 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.129746.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134474.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119917.log 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98441.log 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96219.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119917.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117709.log 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131148.log.gz 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98441.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.119917.log.gz--verbose 2023-11-28T04:27:16.596 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.100384.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117709.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98441.log.gz 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142389.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117709.log.gz 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144382.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142389.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101422.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.100384.log.gz 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98101.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.142389.log.gz 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101422.log: gzip -5 --verbose -- 88.2%/var/log/ceph/ceph-client.admin.132299.log 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.144382.log.gz 2023-11-28T04:27:16.597 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137128.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.101422.log.gz 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132299.log: gzip 87.8%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.98101.log.gz/var/log/ceph/ceph-client.admin.103395.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137128.log: gzip 89.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.132299.log.gz 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.admin.144560.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103395.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137128.log.gz 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99850.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132945.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.103395.log.gz 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111024.log 2023-11-28T04:27:16.598 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.144560.log.gz 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95588.log 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.99850.log.gz 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.132945.log.gz 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118610.log 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95588.log: gzip 87.9%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.111024.log.gz-- 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120043.log 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118610.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95588.log.gz 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118269.log 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120043.log: 87.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.118610.log.gz--verbose 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.114420.log 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3%/var/log/ceph/ceph-client.admin.118269.log: -- replaced with /var/log/ceph/ceph-client.admin.120043.log.gz 2023-11-28T04:27:16.599 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111159.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.140949.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111159.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114420.log.gz 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134726.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.118269.log.gz 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.111159.log.gz 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.140949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115444.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.140949.log.gz 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132114.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144200.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.134726.log.gz 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99892.log 2023-11-28T04:27:16.600 INFO:teuthology.orchestra.run.smithi073.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.115444.log.gz 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130613.log 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.132114.log.gz 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99892.log: gzip -5 --verbose 88.8%-- /var/log/ceph/ceph-client.admin.106809.log -- replaced with /var/log/ceph/ceph-client.admin.144200.log.gz 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130613.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117975.log 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.99892.log.gz 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114504.log 2023-11-28T04:27:16.601 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.130613.log.gz 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4%/var/log/ceph/ceph-client.admin.117975.log: -- replaced with /var/log/ceph/ceph-client.admin.106809.log.gz 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118101.log 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141350.log 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117975.log.gz 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118101.log: 87.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.114504.log.gz-5 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.admin.130403.log 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118101.log.gz 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97975.log 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130403.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.141350.log.gz 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145714.log 2023-11-28T04:27:16.602 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97975.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.130403.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.97975.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97933.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123612.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145714.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95714.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123612.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145210.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97933.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95714.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123612.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94647.log 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145210.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95714.log.gz 2023-11-28T04:27:16.603 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145084.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.145210.log.gz 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94647.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116745.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145084.log: gzip 87.6%-5 -- replaced with /var/log/ceph/ceph-client.admin.94647.log.gz--verbose 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.144335.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5%/var/log/ceph/ceph-client.admin.116745.log: -- replaced with /var/log/ceph/ceph-client.admin.145084.log.gz 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112366.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.116745.log.gz 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113866.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112366.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106476.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.144335.log.gz 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113866.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.112366.log.gz 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97797.log 2023-11-28T04:27:16.604 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113866.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120952.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97797.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106476.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138082.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120952.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.97797.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119959.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.120952.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141529.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142083.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138082.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141529.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.119959.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138695.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142083.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.141529.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137998.log 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.142083.log.gz 2023-11-28T04:27:16.605 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138695.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124470.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115486.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138695.log.gz 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124470.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137998.log.gz 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107384.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124470.log.gz 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99080.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123945.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115486.log.gz 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99080.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107384.log.gz 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98143.log 2023-11-28T04:27:16.606 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.99080.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100912.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121347.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.123945.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100912.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.98143.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110761.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100912.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145882.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.110761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138737.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145882.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.110761.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 91.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.121347.log.gz/var/log/ceph/ceph-client.admin.134516.log 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.145882.log.gz 2023-11-28T04:27:16.607 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112855.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9%/var/log/ceph/ceph-client.admin.134516.log: -- replaced with /var/log/ceph/ceph-client.admin.138737.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93888.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143134.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134516.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.93888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135243.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.admin.112855.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143134.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.93888.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131412.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.135243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119626.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131412.log: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.143134.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.135243.log.gz 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124905.log 2023-11-28T04:27:16.608 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131412.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119626.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129888.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124905.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.119626.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125732.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129888.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114294.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124905.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125732.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129888.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135512.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.125732.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113245.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.135512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101250.log 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.114294.log.gz 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113245.log: 88.5%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.135512.log.gz--verbose 2023-11-28T04:27:16.609 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.124947.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101250.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.113245.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120310.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.101250.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117143.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120310.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124947.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107648.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.120310.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139123.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106342.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117143.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139123.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.107648.log.gz 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117017.log 2023-11-28T04:27:16.610 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106342.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.139123.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137269.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106342.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111384.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142515.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117017.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137269.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114090.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5%/var/log/ceph/ceph-client.admin.142515.log: -- replaced with /var/log/ceph/ceph-client.admin.111384.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131454.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114090.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142515.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.139315.log 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114090.log.gz 2023-11-28T04:27:16.611 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131454.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144291.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131454.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.139315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111719.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.139315.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144291.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116566.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.144291.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111719.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94689.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94353.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116566.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.111719.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94689.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.116566.log.gz-5 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.admin.113306.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.94689.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94353.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133583.log 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94353.log.gz 2023-11-28T04:27:16.612 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104941.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93810.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.113306.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.104941.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.133583.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99934.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.93810.log: gzip -5 --verbose -- 88.4%/var/log/ceph/ceph-client.admin.94857.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.104941.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99934.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93810.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107468.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94857.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.99934.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135321.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107468.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.94857.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125031.log 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107468.log.gz 2023-11-28T04:27:16.613 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 /var/log/ceph/ceph-client.admin.135321.log: --verbose -- /var/log/ceph/ceph-client.admin.119492.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119671.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135321.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119492.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.125031.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143008.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119492.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98483.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143008.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.119671.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144246.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98483.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101654.log 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.143008.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98483.log.gz 2023-11-28T04:27:16.614 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.144246.log: -- /var/log/ceph/ceph-client.admin.116835.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101654.log: gzip -5 --verbose -- 88.1%/var/log/ceph/ceph-client.admin.123201.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.144246.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116835.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101654.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142347.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123201.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116835.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113723.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.123201.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142347.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145546.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113723.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142347.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134810.log 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145546.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.113723.log.gz 2023-11-28T04:27:16.615 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117889.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145546.log.gz/var/log/ceph/ceph-client.admin.134810.log: 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121221.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117889.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100450.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134810.log.gz 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121221.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117889.log.gz 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105192.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121221.log.gz 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100450.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103163.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.105192.log: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.100450.log.gz--verbose 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.131003.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5%/var/log/ceph/ceph-client.admin.103163.log: -- replaced with /var/log/ceph/ceph-client.admin.105192.log.gz 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119093.log 2023-11-28T04:27:16.616 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131003.log: gzip 88.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.103163.log.gz--verbose 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.106432.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135135.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.131003.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106432.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.119093.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117101.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106432.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.135135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97839.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117101.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116975.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.135135.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.117101.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97839.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98693.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97839.log.gz 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123342.log 2023-11-28T04:27:16.617 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98693.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99976.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.116975.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.98693.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144104.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131104.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1%/var/log/ceph/ceph-client.admin.99976.log: -- replaced with /var/log/ceph/ceph-client.admin.123342.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138040.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99976.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131104.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.144104.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108528.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138040.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131104.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.83950.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108528.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.138040.log.gz 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121078.log 2023-11-28T04:27:16.618 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.108528.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.tmp-client.admin.83950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94180.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.83950.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121078.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138653.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.121078.log.gz/var/log/ceph/ceph-client.admin.94180.log: 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123477.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138653.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115635.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94180.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145504.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138653.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115635.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123477.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101089.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115635.log.gz 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115360.log 2023-11-28T04:27:16.619 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101089.log: gzip -5 --verbose -- 86.9%/var/log/ceph/ceph-client.admin.130157.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.145504.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115360.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101089.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98777.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.115360.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.130157.log: -- /var/log/ceph/ceph-client.admin.96135.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142645.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130157.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96135.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.98777.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98819.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142645.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.96135.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95235.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124735.log 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.142645.log.gz 2023-11-28T04:27:16.620 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95235.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98819.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100060.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.95235.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139075.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139399.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.124735.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119402.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139399.log: 91.2% -- replaced with /var/log/ceph/ceph-client.admin.100060.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110655.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.139075.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.139399.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119402.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134168.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96051.log 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.110655.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.119402.log.gz 2023-11-28T04:27:16.621 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112416.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.110655.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.134168.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125115.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112416.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96051.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104438.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9%/var/log/ceph/ceph-client.admin.125115.log: -- replaced with /var/log/ceph/ceph-client.admin.112416.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102588.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.125115.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.104438.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94899.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102588.log: gzip 87.7%-5 -- replaced with /var/log/ceph/ceph-client.admin.104438.log.gz--verbose 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.94941.log 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94899.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.102588.log.gz 2023-11-28T04:27:16.622 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108028.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94899.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94941.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138603.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116022.log 87.9% 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.94941.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138603.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.108028.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99122.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.138603.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116022.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99808.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107860.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.116022.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99808.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99122.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118185.log 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.99808.log.gz 2023-11-28T04:27:16.623 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107860.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119447.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138949.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.107860.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119447.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.118185.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117931.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119447.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113512.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117931.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.138949.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113512.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.117931.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95420.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.113512.log.gz 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137494.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130487.log 2023-11-28T04:27:16.624 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137494.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95420.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124640.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137494.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130487.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146110.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124640.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130487.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121263.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.124640.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.146110.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97251.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96419.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121263.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.146110.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.126028.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.121263.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129575.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96419.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97251.log.gz 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.126028.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120824.log 2023-11-28T04:27:16.625 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.96419.log.gz 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137629.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129575.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.126028.log.gz 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120824.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129575.log.gzgzip 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.107300.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141619.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137629.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.120824.log.gz 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114000.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137629.log.gz 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121120.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141619.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.107300.log.gz 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114000.log: 87.3%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.141619.log.gz--verbose 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.139718.log 2023-11-28T04:27:16.626 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144845.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4%/var/log/ceph/ceph-client.admin.121120.log: -- replaced with /var/log/ceph/ceph-client.admin.114000.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139718.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112322.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.121120.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141865.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144845.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.139718.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121162.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144845.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138781.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141865.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.112322.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121162.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106154.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141865.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138347.log 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138781.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.121162.log.gz 2023-11-28T04:27:16.627 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106154.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102420.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138781.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr: 89.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94222.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138347.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.106154.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131832.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138347.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94222.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.102420.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144761.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131832.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94222.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119717.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111627.log 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.131832.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119717.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144761.log.gz 2023-11-28T04:27:16.628 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94521.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.119717.log.gz 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133754.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94521.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.111627.log.gz 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123903.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.94521.log.gz 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133754.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95882.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123903.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133754.log.gzgzip 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.98315.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95882.log: gzip -5 88.5%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.123903.log.gz/var/log/ceph/ceph-client.admin.130919.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95882.log.gz 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143458.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123702.log 2023-11-28T04:27:16.629 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.98315.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143458.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.130919.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124989.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143458.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144887.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130529.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.123702.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144887.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.124989.log.gz-5 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.admin.141957.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144887.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120169.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131370.log 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130529.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120169.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.141957.log.gz 2023-11-28T04:27:16.630 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141251.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120169.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131370.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103279.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141251.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.131370.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134040.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103279.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141251.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137450.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103279.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114135.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137450.log: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.134040.log.gz-5 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.admin.130877.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114135.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.137450.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99588.log 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.114135.log.gz 2023-11-28T04:27:16.631 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145252.log 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142966.log 88.0% 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130877.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145252.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.99588.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124226.log 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145252.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142966.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117235.log 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124226.log: gzip -5 --verbose 87.9%-- /var/log/ceph/ceph-client.admin.115192.log -- replaced with /var/log/ceph/ceph-client.admin.142966.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117235.log: 88.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.124226.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.142125.log 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.117235.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115528.log 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142125.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.115192.log.gz 2023-11-28T04:27:16.632 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114894.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115528.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.142125.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107081.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115528.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99038.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130067.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.114894.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99038.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.107081.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145756.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99038.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144425.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143862.log 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.130067.log.gz 2023-11-28T04:27:16.633 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144425.log: gzip 88.5%-5 -- replaced with /var/log/ceph/ceph-client.admin.145756.log.gz--verbose 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.95756.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.144425.log.gz 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143862.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115587.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95756.log: 86.9%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.143862.log.gz--verbose 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.115911.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115587.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.95756.log.gz 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130961.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115911.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94479.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.115587.log.gz 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130961.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.115911.log.gz 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124819.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134124.log 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94479.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.130961.log.gz 2023-11-28T04:27:16.634 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116423.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.94479.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134124.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124819.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145840.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116423.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134124.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142695.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145840.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.116423.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141039.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.145840.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142695.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141039.log: gzip -5 --verbose -- 87.9%/var/log/ceph/ceph-client.admin.98525.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.142695.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph.audit.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.141039.log.gz 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97419.log 2023-11-28T04:27:16.635 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97335.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97419.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.98525.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141574.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:gzip /var/log/ceph/ceph-client.admin.97335.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.136045.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.97419.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141574.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97335.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144059.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141574.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114378.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.136045.log: /var/log/ceph/ceph-client.admin.144059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118676.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.136045.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114378.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.144059.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146028.log 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.114378.log.gz 2023-11-28T04:27:16.636 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119538.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112194.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.146028.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.118676.log.gz 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119538.log: gzip 87.1%-5 -- replaced with /var/log/ceph/ceph-client.admin.146028.log.gz--verbose 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.95067.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119538.log.gz 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124777.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112194.log: /var/log/ceph/ceph-client.admin.95067.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99164.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.112194.log.gz 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124777.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95067.log.gz 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96851.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.124777.log.gz 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114546.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90375.log 2023-11-28T04:27:16.637 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96851.log: 90.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.99164.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125073.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.96851.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.90375.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.114546.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131538.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.90375.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133832.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125073.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124037.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133832.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.131538.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132989.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124037.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.133832.log.gz 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94011.log 2023-11-28T04:27:16.638 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.124037.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142840.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94011.log: 88.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.132989.log.gz-5 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:--verbose -- /var/log/ceph/ceph-client.0.113095.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.94011.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129842.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.142840.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.113095.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120560.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.113095.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134432.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120560.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.129842.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94395.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134432.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120560.log.gz 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117339.log 2023-11-28T04:27:16.639 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134432.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.94395.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98861.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131045.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97377.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.117339.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131045.log: gzip 88.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.98861.log.gz--verbose 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.137173.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97377.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.131045.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145126.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137173.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97377.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142265.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137173.log.gz 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102798.log 2023-11-28T04:27:16.640 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142265.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145126.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130717.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102798.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.142265.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123290.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102798.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130717.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105443.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123290.log: 88.3%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99374.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.admin.130717.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.105443.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123290.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121395.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.105443.log.gz 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93969.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121395.log: gzip 87.3%-5 -- replaced with /var/log/ceph/ceph-client.admin.99374.log.gz--verbose 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.124386.log 2023-11-28T04:27:16.641 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.93969.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.121395.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125216.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.93969.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124386.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107342.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125216.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124386.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115977.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107342.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.125216.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102856.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.107342.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117185.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102856.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.115977.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112796.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117185.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.102856.log.gz 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-11-28T04:27:16.642 INFO:teuthology.orchestra.run.smithi073.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.117185.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107606.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mds.c.log: gzip 88.0%-5 --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.112796.log.gz/var/log/ceph/ceph-client.admin.107734.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98273.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.107606.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116657.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107734.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131748.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98273.log: /var/log/ceph/ceph-client.admin.116657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123388.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.98273.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.116657.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131748.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107902.log 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.131748.log.gz 2023-11-28T04:27:16.643 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123388.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141664.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.123388.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117625.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107902.log: /var/log/ceph/ceph-client.admin.141664.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130795.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.107902.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117625.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.141664.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133541.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100735.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130795.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.117625.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98609.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130795.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.133541.log.gz 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145588.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100735.log: /var/log/ceph/ceph-client.admin.98609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116700.log 2023-11-28T04:27:16.644 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.100735.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94264.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.98609.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116700.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.145588.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137788.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116700.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114728.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94264.log: /var/log/ceph/ceph-client.admin.137788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116610.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.94264.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114728.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.137788.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120602.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133914.log 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116610.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.114728.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120602.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116610.log.gz 2023-11-28T04:27:16.645 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113679.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120602.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108154.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133914.log: /var/log/ceph/ceph-client.admin.113679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134346.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.133914.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108154.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.113679.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98059.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.108154.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134346.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107776.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120085.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98059.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.134346.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118227.log 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.98059.log.gz 2023-11-28T04:27:16.646 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131706.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107776.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120085.log: /var/log/ceph/ceph-client.admin.118227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93713.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.120085.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131706.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.118227.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106065.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0%/var/log/ceph/ceph-client.admin.93713.log: -- replaced with /var/log/ceph/ceph-client.admin.131706.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106893.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.93713.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96177.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106065.log: /var/log/ceph/ceph-client.admin.106893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142431.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106065.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106893.log.gz 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106725.log 2023-11-28T04:27:16.647 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134900.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142431.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.96177.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106725.log: gzip -5 88.5%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.142431.log.gz/var/log/ceph/ceph-client.admin.98567.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.106725.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120740.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134900.log: /var/log/ceph/ceph-client.admin.98567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99544.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.134900.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120740.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.98567.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100342.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120740.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95109.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119009.log 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100342.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.99544.log.gz 2023-11-28T04:27:16.648 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95109.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120782.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.100342.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95109.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95151.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119009.log: /var/log/ceph/ceph-client.admin.120782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107196.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.119009.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95151.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120782.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108847.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.95151.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103047.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132900.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108847.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.107196.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103047.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131790.log 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.108847.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103047.log.gz 2023-11-28T04:27:16.649 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137539.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132900.log: /var/log/ceph/ceph-client.admin.131790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102931.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.132900.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137539.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131790.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106388.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137539.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97629.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.102931.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108655.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106388.log: /var/log/ceph/ceph-client.admin.97629.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133415.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106388.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108655.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97629.log.gz 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98651.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125793.log 2023-11-28T04:27:16.650 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133415.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.108655.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101293.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133415.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.98651.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101611.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125793.log: /var/log/ceph/ceph-client.admin.101293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112278.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125793.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101611.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101293.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105943.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101611.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133956.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102756.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.105943.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112278.log.gz 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142924.log 2023-11-28T04:27:16.651 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.105943.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.133956.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130023.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102756.log: /var/log/ceph/ceph-client.admin.142924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114852.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.102756.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130023.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142924.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99638.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114852.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.130023.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130361.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114852.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119051.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99638.log: /var/log/ceph/ceph-client.admin.130361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125390.log 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99638.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.130361.log.gz 2023-11-28T04:27:16.652 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119051.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107258.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.119051.log.gz 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137315.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125390.log: /var/log/ceph/ceph-client.admin.107258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111986.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.125390.log.gz 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137315.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.107258.log.gz 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94096.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117059.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111986.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137315.log.gz 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94096.log: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.111986.log.gz--verbose 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.142882.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94096.log.gz 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97755.log 2023-11-28T04:27:16.653 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117059.log: /var/log/ceph/ceph-client.admin.142882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123246.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.117059.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97755.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.142882.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102630.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.97755.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118317.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133169.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102630.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.123246.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118317.log: 86.5%gzip -- replaced with /var/log/ceph/ceph-client.admin.102630.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.125666.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.118317.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130445.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133169.log: /var/log/ceph/ceph-client.admin.125666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141207.log 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133169.log.gz 2023-11-28T04:27:16.654 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130445.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.125666.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114810.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141207.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130445.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137956.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134642.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114810.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141207.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137956.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.114810.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95193.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.137956.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141439.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134642.log: /var/log/ceph/ceph-client.admin.95193.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124863.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.134642.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141439.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.95193.log.gz 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142041.log 2023-11-28T04:27:16.655 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141439.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115108.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.135177.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142041.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.124863.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99332.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.142041.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.115108.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99458.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.135177.log: /var/log/ceph/ceph-client.admin.99332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134768.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.135177.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99458.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99332.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123567.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.99458.log.gz 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111070.log 2023-11-28T04:27:16.656 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131496.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123567.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.134768.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118143.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.123567.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.111070.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111845.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131496.log: /var/log/ceph/ceph-client.admin.118143.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94138.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131496.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111845.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.118143.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132591.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.111845.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94138.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107692.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136558.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132591.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.94138.log.gz 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101379.log 2023-11-28T04:27:16.657 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132591.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.107692.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134304.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.136558.log: /var/log/ceph/ceph-client.admin.101379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107944.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.136558.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134304.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.101379.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118721.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.134304.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107944.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138823.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101336.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118721.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.107944.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130202.log 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr: 89.8% -- replaced with /var/log/ceph/ceph-client.admin.118721.log.gz 2023-11-28T04:27:16.658 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138823.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose /var/log/ceph/ceph-client.admin.101336.log: -- /var/log/ceph/ceph-client.2.97209.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130202.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119356.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.101336.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.2.97209.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.130202.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112500.log 0.0% 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr: -- replaced with /var/log/ceph/ceph-client.2.97209.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119356.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141304.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112500.log: gzip 87.1%-5 --verbose -- replaced with /var/log/ceph/ceph-client.admin.119356.log.gz-- 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.3.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.112500.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143416.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.141304.log.gz 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118059.log 2023-11-28T04:27:16.659 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.143416.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99680.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118059.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.143416.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117541.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.122987.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99680.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.118059.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117541.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121671.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.99680.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117541.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.122987.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117847.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113137.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.122987.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132713.log 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.121671.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117847.log.gz 2023-11-28T04:27:16.660 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142796.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113137.log.gz 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112754.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132713.log: /var/log/ceph/ceph-client.admin.142796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107426.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132713.log.gz 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112754.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.142796.log.gz 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106977.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.125582.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107426.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.112754.log.gz 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106977.log: 87.7%gzip -5 -- replaced with /var/log/ceph/ceph-client.admin.107426.log.gz--verbose 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-osd.1.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.106977.log.gz 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116521.log 2023-11-28T04:27:16.661 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.125582.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.125582.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113821.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138305.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113821.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116521.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.113821.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138305.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138305.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124512.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111115.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124512.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95462.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111115.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.124512.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95462.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.111115.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.95462.log.gz 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145983.log 2023-11-28T04:27:16.662 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138223.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108244.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145983.log.gz 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138223.log: /var/log/ceph/ceph-client.admin.108244.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.138223.log.gz 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.108244.log.gz 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112903.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134684.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97293.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112903.log.gz 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114936.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104187.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97293.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.134684.log.gz 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96009.log 2023-11-28T04:27:16.663 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.97293.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.114936.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.104187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116379.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.104187.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102546.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116379.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.96009.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139761.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.116379.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107019.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113775.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139761.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.102546.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102504.log 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.139761.log.gz 2023-11-28T04:27:16.664 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113775.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107019.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98399.log 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113775.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102504.log: /var/log/ceph/ceph-client.admin.98399.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102504.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.98399.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116475.log 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133036.log 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108890.log 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133036.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.116475.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133036.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108890.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.108890.log.gz 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121798.log 2023-11-28T04:27:16.665 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98017.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137584.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98017.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.121798.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137584.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.98017.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.137584.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112152.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120868.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112152.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.90246.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120868.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112152.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139357.log 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.90246.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.120868.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139357.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.90246.log.gz 2023-11-28T04:27:16.666 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.139357.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.108444.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112068.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.108444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120646.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.108444.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97713.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120646.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112068.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97713.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.120646.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.97713.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106767.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137404.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106767.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119875.log 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.106767.log.gz 2023-11-28T04:27:16.667 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106297.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119875.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.137404.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.119875.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111677.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.106297.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.111677.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.111677.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144148.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124304.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144148.log: /var/log/ceph/ceph-client.admin.124304.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.144148.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.124304.log.gz 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113179.log 2023-11-28T04:27:16.668 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137217.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120127.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0%/var/log/ceph/ceph-client.admin.137217.log: -- replaced with /var/log/ceph/ceph-client.admin.113179.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120127.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.137217.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.120127.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133692.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94437.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133692.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115020.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.133692.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145294.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115020.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94437.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108196.log 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.115020.log.gz 2023-11-28T04:27:16.669 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145294.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.145294.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108196.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.108196.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103687.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95378.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103687.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106851.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95378.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.103687.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106851.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.95378.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106567.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.106851.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117753.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133457.log 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117753.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.106567.log.gz 2023-11-28T04:27:16.670 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133457.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.117753.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.133457.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111205.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111543.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111205.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.100300.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.111205.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111543.log: /var/log/ceph/ceph-client.0.100300.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.100300.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111543.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145336.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.118526.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134082.log 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145336.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.118526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.118526.log.gz 2023-11-28T04:27:16.671 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134082.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.134082.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98956.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141486.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124596.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.98956.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141486.log: /var/log/ceph/ceph-client.admin.124596.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.141486.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130112.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.124596.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94053.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138991.log 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94053.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130112.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.94053.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138991.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.138991.log.gz 2023-11-28T04:27:16.672 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103337.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99416.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103337.log: /var/log/ceph/ceph-client.admin.99416.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.103337.log.gz 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.99416.log.gz 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108594.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143585.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108594.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139526.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr: 86.9%/var/log/ceph/ceph-client.admin.143585.log: -- replaced with /var/log/ceph/ceph-client.admin.108594.log.gz 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139526.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.143585.log.gz 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113634.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139526.log.gz 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119304.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116212.log 2023-11-28T04:27:16.673 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.113634.log.gz 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119304.log: /var/log/ceph/ceph-client.admin.116212.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.119304.log.gz 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.116212.log.gz 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94983.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.146068.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131880.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.94983.log.gz/var/log/ceph/ceph-client.admin.146068.log: 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131880.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.146068.log.gz 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.131880.log.gz 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101465.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124163.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101465.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101148.log 2023-11-28T04:27:16.674 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124163.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.101465.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101148.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124163.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.132072.log 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101148.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102672.log 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.132072.log: 0.0%gzip -5 -- replaced with /var/log/ceph/ceph-client.0.132072.log.gz--verbose 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:-- /var/log/ceph/ceph-client.admin.112236.log 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119260.log 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102672.log: /var/log/ceph/ceph-client.admin.112236.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.102672.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119260.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.112236.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.119260.log.gz 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108070.log 2023-11-28T04:27:16.675 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111908.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108070.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120001.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.108070.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111908.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129933.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.111908.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120001.log: /var/log/ceph/ceph-client.admin.129933.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.120001.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.129933.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106935.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103437.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106935.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95294.log 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103437.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.106935.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95294.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.103437.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95294.log.gz 2023-11-28T04:27:16.676 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110883.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102462.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.110883.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118913.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.110883.log.gz/var/log/ceph/ceph-client.admin.102462.log: 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118913.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131284.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.102462.log.gz 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.118913.log.gz 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131284.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.131284.log.gz 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124121.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145672.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124121.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145420.log 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.124121.log.gz 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145672.log: /var/log/ceph/ceph-client.admin.145420.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.145672.log.gz 2023-11-28T04:27:16.677 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.145420.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121305.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117797.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121305.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144515.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.121305.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117797.log: /var/log/ceph/ceph-client.admin.144515.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.117797.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.144515.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100971.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110972.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100971.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133998.log 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.100971.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.110972.log: /var/log/ceph/ceph-client.admin.133998.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.110972.log.gz 2023-11-28T04:27:16.678 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133998.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114978.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143182.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114978.log: /var/log/ceph/ceph-client.admin.143182.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.114978.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.143182.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103105.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141162.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103105.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.103105.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141162.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.141162.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144803.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.108486.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.96823.log 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.108486.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.144803.log.gz 2023-11-28T04:27:16.679 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.108486.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.96823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.96823.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114045.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120392.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114045.log: /var/log/ceph/ceph-client.admin.120392.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114045.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.120392.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.143374.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103937.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.143374.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.143374.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114588.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96303.log 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103937.log: /var/log/ceph/ceph-client.admin.114588.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.103937.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.114588.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96303.log: 90.8% -- replaced with /var/log/ceph/ceph-client.admin.96303.log.gz 2023-11-28T04:27:16.680 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100108.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134558.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100108.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139033.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.100108.log.gz 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134558.log: /var/log/ceph/ceph-client.admin.139033.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.134558.log.gz 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.89976.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.139033.log.gz 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113910.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.89976.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129790.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113910.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.89976.log.gz 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129698.log 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129790.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.113910.log.gz 2023-11-28T04:27:16.681 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143820.log 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.129790.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.129698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133214.log 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143820.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129698.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133214.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143820.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.133214.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145798.log 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97461.log 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130571.log 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1%/var/log/ceph/ceph-client.admin.97461.log: -- replaced with /var/log/ceph/ceph-client.admin.145798.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130571.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.97461.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.130571.log.gz 2023-11-28T04:27:16.682 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102989.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97503.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138145.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.102989.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97503.log: /var/log/ceph/ceph-client.admin.138145.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.97503.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.138145.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99206.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111803.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107556.log 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.99206.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111803.log: /var/log/ceph/ceph-client.admin.107556.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.111803.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.107556.log.gz 2023-11-28T04:27:16.683 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118568.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143777.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118568.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125624.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8%/var/log/ceph/ceph-client.admin.143777.log: -- replaced with /var/log/ceph/ceph-client.admin.118568.log.gz 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125624.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.143777.log.gz 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120476.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.125624.log.gz 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138473.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112628.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2%/var/log/ceph/ceph-client.admin.138473.log: -- replaced with /var/log/ceph/ceph-client.admin.120476.log.gz 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124685.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112628.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138473.log.gz 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-11-28T04:27:16.684 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112628.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124685.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.124685.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145378.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132158.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145378.log: /var/log/ceph/ceph-client.admin.132158.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.145378.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132158.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107986.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106110.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107986.log: /var/log/ceph/ceph-client.admin.106110.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.107986.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.106110.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141823.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111339.log 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141823.log: /var/log/ceph/ceph-client.admin.111339.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.141823.log.gz 2023-11-28T04:27:16.685 INFO:teuthology.orchestra.run.smithi073.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.111339.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111249.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107818.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111249.log: /var/log/ceph/ceph-client.admin.107818.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.111249.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.107818.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144622.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97545.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144622.log: /var/log/ceph/ceph-client.admin.97545.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.144622.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.97545.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95504.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97671.log 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95504.log: /var/log/ceph/ceph-client.admin.97671.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.95504.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.97671.log.gz 2023-11-28T04:27:16.686 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120232.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143524.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120232.log: /var/log/ceph/ceph-client.admin.143524.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.120232.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.143524.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131580.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134852.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131580.log: /var/log/ceph/ceph-client.admin.134852.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131580.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.134852.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95546.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131192.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95546.log: /var/log/ceph/ceph-client.admin.131192.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.95546.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.131192.log.gz 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120434.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95672.log 2023-11-28T04:27:16.687 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120434.log: /var/log/ceph/ceph-client.admin.95672.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.120434.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.95672.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142187.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120994.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142187.log: /var/log/ceph/ceph-client.admin.120994.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142187.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.120994.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94773.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.129977.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94773.log: /var/log/ceph/ceph-client.admin.129977.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.94773.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.129977.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.121587.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.105693.log 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.121587.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.121587.log.gz 2023-11-28T04:27:16.688 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.105693.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.105693.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101030.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123995.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101030.log: /var/log/ceph/ceph-client.admin.123995.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.101030.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.123995.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100528.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125300.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100528.log: /var/log/ceph/ceph-client.admin.125300.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.100528.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.125300.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131664.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112458.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131664.log: /var/log/ceph/ceph-client.admin.112458.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.131664.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.112458.log.gz 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143092.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.110928.log 2023-11-28T04:27:16.689 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143092.log: /var/log/ceph/ceph-client.admin.110928.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.143092.log.gz 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.110928.log.gz 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119216.log 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138865.log 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119216.log: /var/log/ceph/ceph-client.admin.138865.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119216.log.gz 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138865.log.gz 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117499.log 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137830.log 2023-11-28T04:27:16.690 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117499.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.117499.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137830.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.137830.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143050.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145168.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143050.log: /var/log/ceph/ceph-client.admin.145168.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.143050.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.145168.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94563.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133373.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94563.log: /var/log/ceph/ceph-client.admin.133373.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94563.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133373.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115276.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132854.log 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115276.log: /var/log/ceph/ceph-client.admin.132854.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.115276.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.132854.log.gz 2023-11-28T04:27:16.691 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133079.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138389.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133079.log: /var/log/ceph/ceph-client.admin.138389.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.133079.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.138389.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145462.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112670.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145462.log: /var/log/ceph/ceph-client.admin.112670.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.145462.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.112670.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95336.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-mgr.y.log: /var/log/ceph/ceph-client.admin.95336.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.95336.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123433.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123433.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.123433.log.gz 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131622.log 2023-11-28T04:27:16.692 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131622.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.131622.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94815.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94815.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.94815.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125157.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125157.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.125157.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115234.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115234.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.115234.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121036.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121036.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.121036.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.114650.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.114650.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.114650.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.143936.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.143936.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.143936.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123077.log 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123077.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.123077.log.gz 2023-11-28T04:27:16.693 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115150.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115150.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.115150.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95840.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95840.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95840.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132802.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132802.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.132802.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133124.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133124.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133124.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134390.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134390.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134390.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136087.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.136087.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.136087.log.gz 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115402.log 2023-11-28T04:27:16.694 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115402.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.115402.log.gz 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115869.log 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115869.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.115869.log.gz 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112110.log 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112110.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.112110.log.gz 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95967.log 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95967.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.95967.log.gz 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.97883.log 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.97883.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.97883.log.gz 2023-11-28T04:27:16.695 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.132492.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.132492.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.132492.log.gz 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101207.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101207.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.101207.log.gz 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131242.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131242.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.131242.log.gz 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100018.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100018.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.100018.log.gz 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100794.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100794.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.100794.log.gz 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145941.log 2023-11-28T04:27:16.697 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145941.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.145941.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.113957.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.113957.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.113957.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99722.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99722.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.99722.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.103221.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.103221.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.103221.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142557.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142557.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.142557.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137360.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137360.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.137360.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112544.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112544.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112544.log.gz 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106252.log 2023-11-28T04:27:16.698 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106252.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.106252.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144929.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144929.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.144929.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.100853.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.100853.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.100853.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95630.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95630.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.95630.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128577.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.128577.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.128577.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138431.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138431.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.138431.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142601.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142601.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.142601.log.gz 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99500.log 2023-11-28T04:27:16.699 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99500.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.99500.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120690.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120690.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.120690.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123657.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123657.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.123657.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.135092.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.135092.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.135092.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101508.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101508.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.101508.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.142473.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.142473.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.142473.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.136616.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.136616.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.136616.log.gz 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138907.log 2023-11-28T04:27:16.700 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138907.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.138907.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.94605.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.94605.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.94605.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.115064.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.115064.log: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.115064.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.115827.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.0.115827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.115827.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.119581.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.119581.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.119581.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134262.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134262.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.134262.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111585.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111585.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.111585.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134212.log 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134212.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.134212.log.gz 2023-11-28T04:27:16.701 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141999.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141999.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.141999.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.121993.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.121993.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.121993.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.99766.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.99766.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.99766.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.139465.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.139465.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.139465.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.118017.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.118017.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.118017.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.128619.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.128619.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.128619.log.gz 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.102714.log 2023-11-28T04:27:16.702 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.102714.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.102714.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.107512.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.107512.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.107512.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123524.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123524.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.123524.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117417.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117417.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.117417.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.141394.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.141394.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.141394.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.101551.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.101551.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.101551.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.117277.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.117277.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.117277.log.gz 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.124428.log 2023-11-28T04:27:16.703 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.124428.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.124428.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98735.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98735.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.98735.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.104690.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.104690.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.104690.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.93768.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.93768.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.93768.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.125985.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.125985.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.125985.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.137914.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.137914.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.137914.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116790.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116790.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.116790.log.gz 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120910.log 2023-11-28T04:27:16.704 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120910.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.120910.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.145630.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.145630.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.145630.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.144470.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.144470.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.144470.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.131326.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.131326.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.131326.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95924.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95924.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.95924.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.112712.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.112712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.95025.log 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.112712.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.95025.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.95025.log.gz 2023-11-28T04:27:16.705 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.134600.log 2023-11-28T04:27:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.134600.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.134600.log.gz 2023-11-28T04:27:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.123861.log 2023-11-28T04:27:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.123861.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.123861.log.gz 2023-11-28T04:27:16.706 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.120518.log 2023-11-28T04:27:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.120518.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.120518.log.gz 2023-11-28T04:27:16.717 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.98357.log 2023-11-28T04:27:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.98357.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.98357.log.gz 2023-11-28T04:27:16.730 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.106522.log 2023-11-28T04:27:16.742 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.106522.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.106522.log.gz 2023-11-28T04:27:16.742 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.96093.log 2023-11-28T04:27:16.754 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.96093.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.96093.log.gz 2023-11-28T04:27:16.754 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.130655.log 2023-11-28T04:27:16.768 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.130655.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.130655.log.gz 2023-11-28T04:27:16.768 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.116334.log 2023-11-28T04:27:16.780 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.116334.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.116334.log.gz 2023-11-28T04:27:16.780 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.138559.log 2023-11-28T04:27:16.794 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.138559.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.138559.log.gz 2023-11-28T04:27:16.794 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.111761.log 2023-11-28T04:27:16.806 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.111761.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.111761.log.gz 2023-11-28T04:27:16.816 INFO:teuthology.orchestra.run.smithi073.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.133499.log 2023-11-28T04:27:16.827 INFO:teuthology.orchestra.run.smithi073.stderr:/var/log/ceph/ceph-client.admin.133499.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.133499.log.gz 2023-11-28T04:27:17.175 INFO:teuthology.orchestra.run.smithi073.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.4 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.6 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.7 has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.b has been restored 2023-11-28T04:27:17.430 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.c has been restored 2023-11-28T04:27:17.541 INFO:teuthology.orchestra.run.smithi073.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2023-11-28T04:27:17.983 INFO:teuthology.orchestra.run.smithi150.stderr: 92.5% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-11-28T04:27:18.004 INFO:teuthology.orchestra.run.smithi150.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-11-28T04:27:18.166 INFO:teuthology.orchestra.run.smithi073.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-11-28T04:27:18.316 INFO:teuthology.orchestra.run.smithi150.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-11-28T04:27:18.377 INFO:teuthology.orchestra.run.smithi150.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-11-28T04:27:18.779 INFO:teuthology.orchestra.run.smithi073.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-11-28T04:27:18.953 INFO:teuthology.orchestra.run.smithi073.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-11-28T04:27:19.040 INFO:teuthology.orchestra.run.smithi150.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-11-28T04:27:19.213 INFO:teuthology.orchestra.run.smithi150.stderr: 94.2% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-11-28T04:27:19.215 INFO:teuthology.orchestra.run.smithi150.stderr: 2023-11-28T04:27:19.215 INFO:teuthology.orchestra.run.smithi150.stderr:real 0m2.876s 2023-11-28T04:27:19.215 INFO:teuthology.orchestra.run.smithi150.stderr:user 0m12.462s 2023-11-28T04:27:19.215 INFO:teuthology.orchestra.run.smithi150.stderr:sys 0m0.483s 2023-11-28T04:27:19.253 INFO:teuthology.orchestra.run.smithi073.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-11-28T04:27:19.296 INFO:teuthology.orchestra.run.smithi073.stderr: 94.0% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-11-28T04:27:20.061 INFO:teuthology.orchestra.run.smithi073.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-11-28T04:27:20.063 INFO:teuthology.orchestra.run.smithi073.stderr: 2023-11-28T04:27:20.063 INFO:teuthology.orchestra.run.smithi073.stderr:real 0m3.744s 2023-11-28T04:27:20.063 INFO:teuthology.orchestra.run.smithi073.stderr:user 0m17.123s 2023-11-28T04:27:20.063 INFO:teuthology.orchestra.run.smithi073.stderr:sys 0m0.911s 2023-11-28T04:27:20.063 INFO:tasks.ceph:Archiving logs... 2023-11-28T04:27:20.064 DEBUG:teuthology.misc:Transferring archived files from smithi073:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi073/log 2023-11-28T04:27:20.065 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T04:27:22.915 DEBUG:teuthology.misc:Transferring archived files from smithi150:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi150/log 2023-11-28T04:27:22.916 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /var/log/ceph -- . 2023-11-28T04:27:24.544 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-11-28T04:27:24.555 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-11-28T04:27:24.555 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T04:27:24.558 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-11-28T04:27:24.655 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-11-28T04:27:24.667 INFO:teuthology.task.clock:Checking final clock skew... 2023-11-28T04:27:24.668 DEBUG:teuthology.orchestra.run.smithi073:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T04:27:24.670 DEBUG:teuthology.orchestra.run.smithi150:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-11-28T04:27:24.683 INFO:teuthology.orchestra.run.smithi073.stderr:bash: ntpq: command not found 2023-11-28T04:27:24.686 INFO:teuthology.orchestra.run.smithi150.stderr:bash: ntpq: command not found 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:210 Number of sources = 4 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:=============================================================================== 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 31 +24us[ +24us] +/- 78ms 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:^* hv02.front.sepia.ceph.com 3 7 377 92 -114us[ -108us] +/- 71ms 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:^+ hv03.front.sepia.ceph.com 3 7 377 89 +228us[ +228us] +/- 69ms 2023-11-28T04:27:24.687 INFO:teuthology.orchestra.run.smithi073.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T04:27:24.689 INFO:teuthology.orchestra.run.smithi150.stdout:210 Number of sources = 4 2023-11-28T04:27:24.689 INFO:teuthology.orchestra.run.smithi150.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-11-28T04:27:24.689 INFO:teuthology.orchestra.run.smithi150.stdout:=============================================================================== 2023-11-28T04:27:24.690 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 29 +53us[ +53us] +/- 78ms 2023-11-28T04:27:24.690 INFO:teuthology.orchestra.run.smithi150.stdout:^* hv02.front.sepia.ceph.com 3 7 377 88 -117us[ -99us] +/- 71ms 2023-11-28T04:27:24.690 INFO:teuthology.orchestra.run.smithi150.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 24 +224us[ +224us] +/- 70ms 2023-11-28T04:27:24.690 INFO:teuthology.orchestra.run.smithi150.stdout:^? hv04.front.sepia.ceph.com 0 10 0 - +0ns[ +0ns] +/- 0ns 2023-11-28T04:27:24.691 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-11-28T04:27:24.703 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-11-28T04:27:24.703 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-11-28T04:27:24.717 DEBUG:teuthology.orchestra.run.smithi073:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T04:27:24.777 DEBUG:teuthology.orchestra.run.smithi150:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-11-28T04:27:24.877 DEBUG:teuthology.orchestra.run.smithi073:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T04:27:24.904 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:27:24.905 DEBUG:teuthology.orchestra.run.smithi150:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-11-28T04:27:24.936 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:27:24.937 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-11-28T04:27:24.949 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-11-28T04:27:24.963 INFO:teuthology.task.internal:Duration was 6354.218858 seconds 2023-11-28T04:27:24.964 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-11-28T04:27:24.978 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-11-28T04:27:24.978 DEBUG:teuthology.orchestra.run.smithi073:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T04:27:24.981 DEBUG:teuthology.orchestra.run.smithi150:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-11-28T04:27:25.023 INFO:teuthology.orchestra.run.smithi073.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T04:27:25.025 INFO:teuthology.orchestra.run.smithi150.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-11-28T04:27:25.387 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-11-28T04:27:25.388 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi073.front.sepia.ceph.com 2023-11-28T04:27:25.388 DEBUG:teuthology.orchestra.run.smithi073:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T04:27:25.406 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi150.front.sepia.ceph.com 2023-11-28T04:27:25.407 DEBUG:teuthology.orchestra.run.smithi150:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-11-28T04:27:25.425 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-11-28T04:27:25.426 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T04:27:25.449 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-11-28T04:27:25.506 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-11-28T04:27:25.507 DEBUG:teuthology.orchestra.run.smithi073:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T04:27:25.509 DEBUG:teuthology.orchestra.run.smithi150:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-11-28T04:27:25.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-11-28T04:27:25.637 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-11-28T04:27:25.637 DEBUG:teuthology.orchestra.run.smithi073:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T04:27:25.661 DEBUG:teuthology.orchestra.run.smithi150:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-11-28T04:27:25.685 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-11-28T04:27:25.695 DEBUG:teuthology.orchestra.run.smithi073:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T04:27:25.703 DEBUG:teuthology.orchestra.run.smithi150:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-11-28T04:27:25.724 INFO:teuthology.orchestra.run.smithi073.stdout:kernel.core_pattern = core 2023-11-28T04:27:25.748 INFO:teuthology.orchestra.run.smithi150.stdout:kernel.core_pattern = core 2023-11-28T04:27:25.762 DEBUG:teuthology.orchestra.run.smithi073:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T04:27:25.791 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:27:25.792 DEBUG:teuthology.orchestra.run.smithi150:> test -e /home/ubuntu/cephtest/archive/coredump 2023-11-28T04:27:25.815 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-11-28T04:27:25.816 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-11-28T04:27:25.829 INFO:teuthology.task.internal:Transferring archived files... 2023-11-28T04:27:25.830 DEBUG:teuthology.misc:Transferring archived files from smithi073:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi073 2023-11-28T04:27:25.830 DEBUG:teuthology.orchestra.run.smithi073:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T04:27:25.866 DEBUG:teuthology.misc:Transferring archived files from smithi150:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703/remote/smithi150 2023-11-28T04:27:25.866 DEBUG:teuthology.orchestra.run.smithi150:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-11-28T04:27:25.902 INFO:teuthology.task.internal:Removing archive directory... 2023-11-28T04:27:25.902 DEBUG:teuthology.orchestra.run.smithi073:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T04:27:25.904 DEBUG:teuthology.orchestra.run.smithi150:> rm -rf -- /home/ubuntu/cephtest/archive 2023-11-28T04:27:25.951 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-11-28T04:27:25.962 INFO:teuthology.task.internal:Not uploading archives. 2023-11-28T04:27:25.963 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-11-28T04:27:25.978 INFO:teuthology.task.internal:Tidying up after the test... 2023-11-28T04:27:25.978 DEBUG:teuthology.orchestra.run.smithi073:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T04:27:25.980 DEBUG:teuthology.orchestra.run.smithi150:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-11-28T04:27:25.992 INFO:teuthology.orchestra.run.smithi073.stdout: 263007 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 04:27 /home/ubuntu/cephtest 2023-11-28T04:27:26.005 INFO:teuthology.orchestra.run.smithi150.stdout: 262183 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Nov 28 04:27 /home/ubuntu/cephtest 2023-11-28T04:27:26.006 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-11-28T04:27:26.019 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-11-28T04:27:26.052 INFO:teuthology.nuke:Checking targets against current locks 2023-11-28T04:27:26.079 DEBUG:teuthology.nuke:shortname: smithi073 2023-11-28T04:27:26.079 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T04:27:26.101 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi073.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 02:34:04.405562', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLrYwp82qOkNzBtVWZdiiw7/4vzuuaQgmcQg21uwlcmpDvvwZwnShDLiOf7e7CBPwpQcWBLI4+9CevKpbl/G7vQ='} 2023-11-28T04:27:26.105 DEBUG:teuthology.nuke:shortname: smithi150 2023-11-28T04:27:26.105 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-11-28T04:27:26.129 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi150.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-11-27_22:36:25-fs-wip-yuri-testing-2023-11-27-1028-pacific-distro-default-smithi/7468703', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.4', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-11-28 02:34:04.406976', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOcsXlOEX6iheXxSe0SiQYyxcdk15GcmPxRXO6f5afn5qNlnTo44BVJqyXGhrJt5x77Pw+PUYTWTkbOBwmE5x6A='} 2023-11-28T04:27:26.153 INFO:teuthology.orchestra.console.smithi073:Power off 2023-11-28T04:27:26.153 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T04:27:26.175 DEBUG:teuthology.orchestra.console.smithi073:power off output: Chassis Power Control: Down/Off 2023-11-28T04:27:26.176 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:26.199 INFO:teuthology.orchestra.console.smithi150:Power off 2023-11-28T04:27:26.199 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-11-28T04:27:26.211 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2023-11-28T04:27:26.223 DEBUG:teuthology.orchestra.console.smithi150:power off output: Chassis Power Control: Down/Off 2023-11-28T04:27:26.223 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:26.249 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-28T04:27:30.213 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:30.250 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:30.323 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is on 2023-11-28T04:27:30.363 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is on 2023-11-28T04:27:34.326 DEBUG:teuthology.orchestra.console.smithi073:pexpect command: ipmitool -H smithi073.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:34.364 DEBUG:teuthology.orchestra.console.smithi150:pexpect command: ipmitool -H smithi150.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-11-28T04:27:34.437 DEBUG:teuthology.orchestra.console.smithi073:check power output: Chassis Power is off 2023-11-28T04:27:34.475 DEBUG:teuthology.orchestra.console.smithi150:check power output: Chassis Power is off 2023-11-28T04:27:34.538 INFO:teuthology.orchestra.console.smithi073:Power off completed 2023-11-28T04:27:34.576 INFO:teuthology.orchestra.console.smithi150:Power off completed 2023-11-28T04:27:34.683 INFO:teuthology.run:Summary data: description: fs/functional/{begin clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{rhel_8} mount/kclient/{mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6354.218857526779 failure_reason: '"2023-11-28T03:06:19.269245+0000 mon.a (mon.0) 594 : cluster [WRN] Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-11-28T04:27:34.683 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-11-28T04:27:34.769 INFO:teuthology.run:FAIL